VMware released the latest version of Big Data Extensions during Hadoop Summit on June 4, 2015. Included in the release notes, there are two features that have me really excited for this version.
Resize Hadoop Clusters on Demand. You can reduce the number of virtual machines in a running Hadoop cluster, allowing you to manage resources in your VMware ESXi and vCenter Server environments. The virtual machines are deleted, releasing all resources such as memory, CPU, and I/O.
Increase Cloning Performance and Resource Usage of Virtual Machines. You can rapidly clone and deploy virtual machines using Instant Clone, a feature of vSphere 6.0. Using Instant Clone, a parent virtual machine is forked, and then a child virtual machine (or instant clone) is created. The child virtual machine leverages the storage and memory of the parent, reducing resource usage.
These are really outstanding features, especially the ability to use the Instant Clone (aka VMFork) functionality introduced in vSphere 6.0. The Instant Clone technology has very interesting implications when considered with the work in the Cloud Native Application space. Deploying a very small Photon VM to immediately launch a Docker container workload in a matter of seconds will add a huge benefit to running virtualized Apache Mesos clusters (with Marathon) that have been deployed using the BDE framework.
I had hoped to see the functionality from the BDE Fling that included recipes for Mesos and Kubernetes to be incorporated into the official VMware release. On a positive note, the cookbooks for Mesos, Marathon and Kubernetes are present on the management server. It will just take a little effort to unlock those features.
06.27.2015 UPDATE: I have confirmed all of the cookbooks for Mesos, Docker and Kubernetes are present on the BDE v2.2 management server. I will have a post shortly describing how to unlock them for use.
Overall, a big release from the VMware team and it appears they are on the right track to increase the functionality of the BDE framework!
Having recently returned from Hadoop Summit 2014 in San Jose, I wanted to take some time to jot down my thoughts on the sessions. I primarily focused on the sessions that revolved around operational management of Hadoop to see how other companies are tackling the same problems I am facing. It is comforting to know that I am not alone in my quest to deliver a reliable Hadoop platform across the development lifecycle for my internal customers to consume. However, one of the frustrating things to witness was the inherent lack of large-scale organizations operating within their own private cloud environments. Many of the demonstrations involved utilizing resources from AWS or made the assumption you would never run out of bare-metal hardware to deploy on. My experience is wholly different.
The challenging part of offering a true Hadoop-as-a-Service platform is the expectation that additional resources will always be available for an Engineering team or Operations team to consume at a moments notice. For that to be the case, in my experience, AWS becomes too expensive too quickly and bare-metal hardware is difficult to procure at a moments notice within a large, publicly traded organization. For that, a private cloud environment is perfect — but no one wants to openly talk about running Hadoop on a virtual platform. Which, when you start thinking about it is quite humorous because most demonstrations showed Hadoop running in AWS — what do they think an EC2 instance is exactly?
My talk with Andrew Nelson on running a production Hadoop-as-a-Service platform using VMware vCenter Big Data Extensions went well. The audience was well-educated and we received some rather good questions at the end. Virtualizing Hadoop for my organization has been a great way to solve many of the lifecycle management issues faced in today’s rapidly changing environment.
All that being said, here are the key takeaways/questions I gained from Hadoop Summit 2014:
- Failure handling (Docker) — What happens when a container is lost and you are waiting for a new container to be created?
- Docker can easily be virtualized within existing private cloud environments.
- Data locality and container affinity can be accomplished with existing private cloud environments.
- Writing an Application Master is hard and error prone.
- Performance evaluation != Workload
- Hbase regionserver splits are expensive & it is suggested that you pre-split the region — elasticity is really lacking here.
- Best session by far was from Alex Moundalexis @Cloudera: http://tiny.cloudera.com/7steps
- Performance tuning the Linux OS is key and oftentimes overlooked by DevOps. There are several low lift, high yield changes that can be made.
- Ambari Apache Project is another manager to evaluate.
- Many projects trying to solve the same problems that VMware vCloud Automation Center already offers at a larger-scale and more feature rich solution.
- Read and re-read Hadoop Operations.