VMworld 2015 Monday Recap

VMworld 2015

Monday was my first day at VMworld 2015 in San Francisco, CA and it was outstanding! I followed the Cloud Native Apps track throughout the day and it kicked off with a bang with the announcements around Photon Platform and vSphere Integrated Containers.

I attended the following sessions today:

  • STO5443 Case Study of Virtualized Hadoop on VMware Software Designed Storage Technologies
  • CNA6261 Internals: How We Integrated Kubernetes with the SDDC
  • CNA6649-S Build and run Cloud Native Apps in your Software-Defined Data Center

All great sessions each in their own right, but I am most excited around the things coming out of the Cloud Native Apps team and the Office of the CTO at VMware. Here are a few of the key takeaways from the sessions today I had.

Cells as the new layer of abstraction

One of the new pieces of technology coming out of VMware in the coming months is the ability to deploy clusters through a self-service cloud portal to the IaaS layer of your choosing (vCloud Air, vSphere, EC2 and GCE). The PaaS offering plans to include support for Kubernetes and Mesos clusters, including many other applications such as MongoDB, Cassandra, etc. The motivating idea behind the technology is to abstract the notion of VMs away from the developers and instead deliver the cell as the new abstraction layer. This should allow developers to standardize management of all the apps across the enterprise and take the workload between the different public and private cloud offerings.

I understand they wrote the framework from scratch and did not use the framework Big Data Extensions currently uses. I am excited to get a look at the framework to see how they are accomplishing this and what the key differentiators from BDE are.

vSphere Integrated Containers

This is really outstanding.  From the VMware website, it describes the project key points as follows.

With VMware vSphere at its foundation, the new offering will help IT operations team meet the following enterprise requirements for containers:

  • Security and Isolation – Assuring the integrity and authenticity of containers and their underlying infrastructure, Project Bonneville, a technology preview, isolates and starts up each container in a virtual machine with minimal overhead using the Instant Clone feature of VMware vSphere 6.
  • Storage and Data Persistence – While many container services are stateless today, customers have the desire to enable stateful services to support cloud-native databases. VMware vSphere Integrated Containers will enable provisioning of persistent data volumes for containers in VMware vSphere environments. This will enable IT operations and development teams to take advantage of the speed and portability of containerized applications in conjunction with highly resilient VMware vSphere storage, including VMware Virtual SAN™ and VMware vSphere Virtual Volumes™-enabled external storage.
  • Networking – VMware NSX™ supports production container deployments today. With VMware NSX, IT can apply fine-grained network micro-segmentation and policy-based security to cloud-native applications. Additionally, VMware NSX provides IT with greater visibility into the behavior of containers. Finally, with VMware NSX, containers can be integrated with the rest of the data center, and can be connected to quarantine, forensics and/or monitoring networks for additional monitoring and troubleshooting.
  • Service-Level Agreements (SLAs) – IT teams will be able to assure service-level agreements for container workloads with VMware vSphere Distributed Resource Scheduler as well as reduce planned and unplanned downtime with VMware vSphere High Availability and VMware vSphere vMotion®.
  • Management – Administrators will be able to use VMware vCenter Server™ to view and manage their containers without the need for new tools or additional training through Project Bonneville, which will enable the seamless integration of containers into VMware vSphere. Customers can further achieve consistent management and configuration compliance across private and public clouds using the VMware vRealize™ Suite.

Pretty outstanding stuff!

Photon Platform

Kit talked about the Photon Platform today and some of the features it will be bringing to the Cloud Native App environments are really going to be game changers. I had gotten a look at this technology almost a year ago when it was known by another codename and it looks even better now! The announcement around the Photon Controller being released as open source continues the example Project Photon and Project Lightwave went down a few months ago with their announcements.

The other announcement was the vSphere driver for Flocker is a welcome addition to the Cloud Native App storyline. Persistent data in containers is one of the bigger challenges the industry is still working to solve in a manner that will work for enterprise environments. Having the container itself own a VMDK that is persistent and then is available when the container migrates through the environment is huge. The code is available to on the VMware GitHub account and I am anxious to get my hands on it ASAP!

Conclusion

Overall an outstanding way to start the conference. I am excited for what tomorrow is going to bring and look forward to working with many of these technologies coming from VMware. To be completely honest, the story VMware is telling in the Cloud Native App space and the internal projects surrounding it is one of the primary reasons I joined VMware this summer. It is really great to work for a company that is passionate about technology and pushing the envelope with what is possible today!

I love my job!

Exposing Apache Mesos on VMware Big Data Extensions v2.2

The VMware Big Data Extensions v2.2 release included the cookbooks for Apache Mesos and Kubernetes from the Fling released this past spring. However, those cookbooks are not exposed when you deploy the new version. Fortunately, unlocking them only takes a few minutes before they can be made available! I will cover exactly what is needed in order to begin using these Cloud Native App cluster deployments below.

If you jump onto your v2.2 management server and look in the /opt/serengeti/chef/cookbooks directory, you will see all of the Cloud Native App additions.

BDE-Cookbooks-v2.2

A quick look to be sure the Chef roles are still defined tells us that they are.

Chef-BDE-v2.2-Roles

 

They even did us the favor of including the JSON spec files in the /opt/serengeti/www/specs/Ironfan directory.

BDE-v2.2-Mesos-JSON

The missing pieces are the entries in the /opt/serengeti/www/specs/map and /opt/serengeti/www/distros/manifest files. Those are rather easy to copy out of the VMware Fling itself or re-create manually. If you want to edit the files yourself, here is what needs to be added to the files.

/opt/serengeti/www/specs/map

{
  "vendor" : "Kubernetes",
  "version" : "^(\\d)+(\\.\\w+)*",
  "type" : "Basic Kubernetes Cluster",
  "appManager" : "Default",
  "path" : "Ironfan/kubernetes/basic/spec.json"
},
{
  "vendor" : "Mesos",
  "version" : "^(\\d)+(\\.\\w+)*",
  "type" : "Basic Mesos Cluster",
  "appManager" : "Default",
  "path" : "Ironfan/mesos/basic/spec.json"
},

/opt/serengeti/www/distros/manifest

{
  "name" : "kubernetes",
  "vendor" : "KUBERNETES",
  "version" : "0.5.4",
  "packages" : [
    {
      "tarball": "kubernetes/kubernetes-0.5.4.tar.gz",
      "roles": [
        "kubernetes_workstation",
        "kubernetes_master",
        "kubernetes_minion"
      ]
    }
  ]
},
{
  "name" : "mesos",
  "vendor" : "MESOS",
  "version" : "0.21.0",
  "packages" : [
    {
      "package_repos": [ "https://0.0.0.0/yum/mesos.repo"],
      "roles" : [
        "zookeeper",
        "mesos_master",
        "mesos_slave",
        "mesos_docker",
        "mesos_chronos",
        "mesos_marathon"
      ]
    }
  ]
}

The repos built into the Fling are not present (unfortunately) on the management server. This was the only tedious portion of the entire process. The easiest method is to grab the files out of an existing BDE Fling management server and copy them into the new one. The other option is find the latest RPMs on the Internet and add them to the management server manually. In either case, you’ll need to run the CentOS syntax for creating the repository.

Create local repo for Apache Mesos

# su - serengeti
$ cd /opt/serengeti/www/yum
$ vim mesos.repo
[a-mesos]
name=Apache Mesos
baseurl=https://0.0.0.0/yum/repos/mesos/current/
enabled=1
gpgcheck=0
sslverify=1
sslcacert=/etc/chef/trusted_certs/serengeti-base.pem

$ mkdir -p repos/mesos/current/RPMS
$ cd repos/mesos/current

The Fling included the following files:
- bigtop-utils-0.8.0.4-1.el6.noarch.rpm
- chronos-2.3.0-0.1.20141121000021.x86_64.rpm
- docker-io-1.3.1-2.el6.x86_64.rpm
- marathon-0.7.5-1.0.x86_64.rpm
- mesos-0.21.0-1.0.centos65.x86_64.rpm
- subversion-1.6.11-10.el6_5.x86_64.rpm
- zookeeper-3.4.5.4-1.el6.noarch.rpm
- zookeeper-server-3.4.5.4-1.el6.noarch.rpm

$ createrepo .

A restart of Tomcat is all that is needed and then you will be able to start deploying Apache Mesos and Kubernetes clusters through BDE v2.2.

If you want to take advantage of the Instant Clone functionality, you will need to be running vSphere 6.0 and BDE v2.2. There are also a couple adjustments to the /opt/serengeti/conf/serengeti.properties files that will be need to be made. I will be going over those in a future post discussing how to use the Photon OS as the template for BDE to deploy.

Installing Apache Mesos on Photon Linux OS

Ever since the VMware Photon Technical Preview was made available, I have wanted to install Apache Mesos on a node. The Photon OS is a very minimal Linux installation and so my initial attempts to work through the process led me down the old-fashioned rabbit hole of manually compiling packages. It was very reminiscent of the later 1990’s and using Debian installed via floppy disk. I finally found the time to work my way down and back out of the rabbit hole and have been able to successfully get Mesos to install and run on Photon! This is a good first step towards building a Photon template to be used inside Big Data Extensions for deploying Cloud Native Apps with Mesos|Marathon|Chronos or Kubernetes.

The lab environment I used is running vSphere 5.5 and consists of a small set of nested ESXi hypervisors. I am not going to cover install Photon on a VM, but just be sure you have one that was installed with the full OS — not the minimal installation. After the Photon VM is configured to communicate with the Internet, you can follow these instructions to get Apache Mesos installed. The end result of the guide will be a working Mesos node running on Photon that can launch Docker containers.

Apache Mesos on Photon

Be sure to follow it in the order listed as the ordering of the packages is important. Also have a Photon VM with at least 3GB of Memory allocated to it for the compile processes.

HOWTO Guide

Missing prerequisites
APR Library
# wget http://apache.claz.org//apr/apr-1.5.2.tar.gz
# tar zxvf apr-1.5.2.tar.gz
# cd apr-1.5.2
# ./configure —prefix=/usr/local/lib/apr
# make
# make test
# make install

APR-UTIL Library
# wget http://apache.claz.org//apr/apr-util-1.5.4.tar.gz
# tar zxvf apr-util-1.5.4.tar.gz
# cd apr-util-1.5.4
# ./configure —prefix=/usr/local/lib/apr —with-apr=/usr/local/lib/apr
# make
# make install

Subversion
# wget http://apache.osuosl.org/subversion/subversion-1.8.13.tar.gz
# tar zxvf subversion-1.8.13.tar.gz
# cd subversion-1.8.13
# ./configure —prefix=/usr/local/lib/subversion —with-apr=/usr/local/lib/apr —with-apr-util=/usr/local/lib/apr
# make
# make install

OpenJDK Java
Download the Java JDK source tarball from the Oracle website (http://download.oracle.com/otn-pub/java/jdk/7u79-b15/jdk-7u79-linux-x64.tar.gz)
# tar zxvf jdk-7u79-linux-x64.tar.gz
# mv jdk1.7.0_79 /usr/local/java
# echo JAVA_HOME=/usr/local/java >> /etc/environment
# source /etc/environment

Apache Maven Library
# wget http://apache.mirrors.ionfish.org//ant/binaries/apache-ant-1.9.5-bin.tar.gz
# tar zxvf apache-ant-1.9.5-bin.tar.gz
# mv apache-ant-1.9.5 /usr/local
# ln -s /usr/local/apache-ant-1.9.5 /usr/local/apache-ant
# wget http://apache.cs.utah.edu/maven/maven-3/3.3.3/source/apache-maven-3.3.3-src.tar.gz
# tar zxvf apache-maven-3.3.3-src.tar.gz
# cd apache-maven-3.3.3
# /usr/local/apache-ant/bin/ant -Dmaven.home=“/usr/local/maven-3.3.3"
# echo MAVEN_HOME=/usr/local/maven-3.3.3 >> /etc/environment
# export /etc/environment

Install Apache Mesos
# wget http://www.apache.org/dist/mesos/0.22.1/mesos-0.22.1.tar.gz
# tar zxvf mesos-0.22.1.tar.gz
# cd mesos-0.22.1
# mkdir build
# cd build
# ../configure --prefix=/usr/local/mesos —with-apr=/usr/local/lib/apr —with-svn=/usr/local/lib/subversion
# make
# make check
# make install
After Apache Mesos is installed, you can start both the master and slave processes on the node to run a quick test.
# /usr/local/mesos/bin/mesos-master.sh —ip=127.0.0.1 —work_dir=/var/lib/mesos
# /usr/local/mesos/bin/mesos-slave.sh —master=127.0.0.1:5050
Afterwards, open a web browser and point to to http://127.0.0.1:5050 and you will see the Apache Mesos interface. The next step will be to deploy multiple Photon nodes and configure them to be a part of a single cluster. I will likely orchestrate all of this through Chef next and incorporate it into the Big Data Extensions framework.
There is always more to do, but I am glad to have gotten this working and sharing it with everyone. Reach out to me on Twitter if you have questions.

Building on Project Photon & Project Lightwave

The opportunities for VMware with Project Photon and Project Lightwave are significant. The press release stated:

Designed to help enterprise developers securely build, deploy and manage cloud-native applications, these new open source projects will integrate into VMware’s unified platform for the hybrid cloud — creating a consistent environment across the private and public cloud to support cloud-native and traditional applications. By open sourcing these projects, VMware will work with a broad ecosystem of partners and the developer community to drive common standards, security and interoperability within the cloud-native application market — leading to improved technology and greater customer choice.

What I always find interesting is the lack of discussion around the orchestration and automation of the supporting applications. The orchestration layer does not miraculously appear within a private cloud environment for the developers to consume. The pieces have to be in place in order for developers to consume the services a Mesos cluster offers them. For me, the choice is pretty obvious — expand what the Big Data Extensions framework is capable of providing. I alluded to this thought on Monday when the announcement was made.

Building on that thought and after seeing a diagram of VMware’s vision for how all the pieces tie together, I worked on a logical diagram of how the entire architecture could look like. I believe it looks something like this:

CNA

 

In this environment, Project Photon and Project Lightwave are able to be leveraged beyond just ESXi. By enhancing the deployment options for BDE to include ESXi on vCloud Air (not shown above), KVM and physical (through Ironic), the story is slightly changed. The story now sounds something like this:

For a developer, you choose what Cloud Native application orchestration layer (Mesos, Marathon, Chronos, CloudFoundry, etc.) you would like and communicate with it over the provided API. For operations, the deployment of the tenants within the private cloud environment can be deployed using the OpenStack API (with Heat templates). For both sides, SDLC consistency is maintained through the development process to production.

Simplicity is achieved by only interacting with two APIs — one for operations and one for development. There is large amount of work to do here. First, I need to continue to improve the OpenStack resource plugin to be production-ready. Second, testing of Project Photon inside BDE needs to take place — I imagine there will be some work to have it integrated correctly with the Chef server. Third, the deployment mechanism inside BDE needs to be enhanced to support other options. If the first two were a heavy lift, the last one is going to take a small army — but it is a challenge I am ready to take on!

Ultimately, I feel the gaps in OpenStack around Platform-as-a-Service orchestration can be solved though integrating Big Data Extensions. The framework is more robust and mature when compared to the Sahara offering. The potential is there, it just needs to be executed on.

Thoughts on the VMware Project Photon Announcement

project photon

VMware announced a new open source project call Project Photon today. The full announcement call be seen here. Essentially Project Photon is a lightweight Linux operating system built to support Docket and rkt (formerly Rocket) containers. The footprint is less than 400MB and can run containers immediately upon instantiation. I had heard rumors the announcement today was going to include some sort of OS, but I was not very excited about it until I started reading the material being released prior to the launch event in a few hours.

Having seen the demo and read the material, my mind went into overdrive for the possibilities both open source projects offer organizations who are venturing down the Cloud Native Apps (or Platform 3) road. I believe VMware has a huge opportunity here to cement themselves as the foundation for running robust, secure and enterprise-ready Cloud Native Applications. If you think about the performance gains vSphere 6.0 has provided, and then look at how they are playing in the OpenStack space with both VIO and NSX, the choice becomes obvious.

The area of focus now needs to be on tying all of the pieces together to offer organizations an enterprise-class end-to-end Platform-as-a-Service solution. This is where, I believe, the VMware Big Data Extensions framework should play an enormous part. The framework already allows deployment of Hadoop, Mesos and Kubernetes clusters. Partner the framework with Project Photon and you now have a minimal installation VM that can be launched within seconds with VMFork. From there, the resource plugin Virtual Elephant launched today could be mainstreamed (and improved) to allow for the entire deployment of a Mesos stack, backed by Project Photon, through the open source API OpenStack offers with Heat.

Epic win!

There is still work VMware could do with the Big Data Extensions framework to improve its capabilities, especially with newcomers SequenceIQ and their Cloudbreak offering stiff competition. Expanding BDE to be able to deploy clusters beyond an internal vSphere environment, but also to the major public cloud environments — including their own vCloud Air — will be key going forward. The code for BDE is already an open source project — by launching these two new open source projects they are showing the open source community they are serious.

This is a really exciting time in virtualization and I just got even more excited today!