Ravello Nested ESXi Home Lab Environment


The past week I found myself in need of additional capacity for a Cloud Native App deployment that my current home lab lacked and it was a perfect opportunity to activate my vExpert Ravello Systems account. I had read multiple posts on how to run a nested VMware vSphere environment within Ravello and so I jumped straight into the deep end of the pool. After logging into the system, building an initial ESXi VM image, I started building out the lab environment. The application blueprinting functionality in the Ravello interface is pretty intuitive and left me wishing some other tools I’ve used in the past were as simplistic.

The blueprint includes a small external management cluster, an internal management cluster and the workgroup cluster. Here is a screenshot of the lab environment.

Ravello CNA Lab

External Management Cluster

The external management cluster runs the virtual machines services that are necessary to support the internal lab.

  • VMware vCenter Server virtual appliance (VCSA).
  • Linux VM running iSCSI for the ESXi hosts.
  • Linux VM running DHCPD for the 2nd-level nested VMs.
  • Windows 7 IE11 VM to act as a bastion host for accessing the environment.

Management Cluster

The internal management cluster run the components such as VMware Big Data Extensions and vRealize Operations.

  • 4 ESXi hosts running 5.5U2

Workgroup Cluster

The workgroup cluster runs the nested VMs deployed by Big Data Extensions, such as Hadoop, Mesos and Kubernetes clusters.

Configuration Items

There were a couple of issues I encountered when I first started using the lab environment. I was able to find a resolution for each one in various places on the internet, but I want to share them all here for future reference.

vCenter Server

The vCenter server will throw an error that will require user-intervention to boot due to the fact that when it is deployed as a 1st-level VM, it is running inside a KVM container. To solve the issue, follow these instructions from Ravello found on their site.


2nd-level Nested VMs

When I went to deploy 2nd-level VMs within the Ravello lab, I found they would not power on due to an incompatible hypervisor. To solve the issue, I had to add the Advanced Setting on each of the virtual machines deployed as a 2nd-level VM.

vmx.allowNested = “TRUE”


The next problem I encountered with 2nd-level VMs was the lack of DHCP functionality. The Ravello provided DHCP server for the networks within the application blueprint does not work beyond the 1st-level VMs. To solve the issue, I created a CentOS Linux VM (shown in the External Management Cluster above) and added a reserved IP address for it in the configuration.


Deploying OVF/OVA to the vCenter VM failed

After getting the vCenter configured with the ESXi hosts and HA/DRS clusters, the next step was to deploy the VMware Big Data Extensions vApp. During the configuration of the vApp (storage, network, etc), it failed to allow me to complete the deployment. After looking into the issue, it became apparent it was an issue of the public IP address being a NAT into the private network running inside Ravello.

To solve the issue, I deployed the Windows 7 IE11 VM and then opened an RDC session to the environment. From there I was able to access the vCenter server on the private IP network and perform the OVA deployment of BDE.

Note: If you need a free Windows VM that you can deploy any vSphere environment, log onto the Microsoft Edge site.

Overall Experience

My overall experience with Ravello this past week and consuming around 1500 CPU hours was pretty good. I liked the interface for creating the environment, being able to set a shutdown/startup schedule and overall deployment of the 1st-level VMs. The negatives I encountered were solely around the performance of the 2nd-level nested VMs. Deploying a Mesos cluster through BDE took a long time — I started a small 9 node cluster, went out to dinner with my wife, and it was finishing when we got home 1 1/2 hours later.

With it being VMworld this week, I went down to the Solutions Exchange area and talked to a few people at Ravello about the performance issues I encountered and they offered me some suggestions. I will be checking the environment to see if the settings they recommended make a noticeable difference. For now I am giving them the benefit of the doubt.

Overall, I think the system is a really great opportunity for VMware evangelists to spin up a low-cost lab environment to test new functionality as 1st-level VMs. Which makes it a great place to work on certification objectives for VCP, VCAP/VCIX exams. Trying to test out Cloud Native App frameworks (Kubernetes, Mesos, Docker Swarm) is probably not the best environment.

I will continue to use the environment — especially because of the vExpert program they offer — and I am hopeful the performance will increase over time for the 2nd-level workloads.