Apache Mesos and Marathon Deployment Demo at VMworld

VMworld 2015

Andrew Nelson and Tom Twyman  spoke on Wednesday morning about Apache Mesos and Marathon at VMworld. During their session they showed a demo of a cluster deployment — although they experienced a couple technical difficulties. The session covered the overall basics of how to operationalize Cloud Native Applications using Apache Mesos, Mesosphere Marathon and Docker on a VMware private cloud.

Here is an alternate cut of the demo they showed yesterday.

 

The video walks a user through a deployment of a Apache Mesos cluster using VMware Big Data Extensions, shows the running UI for Apache Mesos, Mesosphere Marathon and Chronos. Behind the scenes, HAProxy has been installed to automatically add any workloads launched and Docker support on each node. After the deployment is complete, a NGiNX Docker workload is launched into Marathon using the API. The workload is scaled from 1 to 6 instances and shows the HAProxy ruleset being updated to include each instance that is running. Finally, the video shows the Apache Mesos cluster itself being scaled while the same NGiNX workload is still running.

A quick 3-minute video showing how versatile Cloud Native Apps on top of VMware infrastructure can be to enable developers to take advantages of the newest technologies for running containers.

Exposing Apache Mesos on VMware Big Data Extensions v2.2

The VMware Big Data Extensions v2.2 release included the cookbooks for Apache Mesos and Kubernetes from the Fling released this past spring. However, those cookbooks are not exposed when you deploy the new version. Fortunately, unlocking them only takes a few minutes before they can be made available! I will cover exactly what is needed in order to begin using these Cloud Native App cluster deployments below.

If you jump onto your v2.2 management server and look in the /opt/serengeti/chef/cookbooks directory, you will see all of the Cloud Native App additions.

BDE-Cookbooks-v2.2

A quick look to be sure the Chef roles are still defined tells us that they are.

Chef-BDE-v2.2-Roles

 

They even did us the favor of including the JSON spec files in the /opt/serengeti/www/specs/Ironfan directory.

BDE-v2.2-Mesos-JSON

The missing pieces are the entries in the /opt/serengeti/www/specs/map and /opt/serengeti/www/distros/manifest files. Those are rather easy to copy out of the VMware Fling itself or re-create manually. If you want to edit the files yourself, here is what needs to be added to the files.

/opt/serengeti/www/specs/map

{
  "vendor" : "Kubernetes",
  "version" : "^(\\d)+(\\.\\w+)*",
  "type" : "Basic Kubernetes Cluster",
  "appManager" : "Default",
  "path" : "Ironfan/kubernetes/basic/spec.json"
},
{
  "vendor" : "Mesos",
  "version" : "^(\\d)+(\\.\\w+)*",
  "type" : "Basic Mesos Cluster",
  "appManager" : "Default",
  "path" : "Ironfan/mesos/basic/spec.json"
},

/opt/serengeti/www/distros/manifest

{
  "name" : "kubernetes",
  "vendor" : "KUBERNETES",
  "version" : "0.5.4",
  "packages" : [
    {
      "tarball": "kubernetes/kubernetes-0.5.4.tar.gz",
      "roles": [
        "kubernetes_workstation",
        "kubernetes_master",
        "kubernetes_minion"
      ]
    }
  ]
},
{
  "name" : "mesos",
  "vendor" : "MESOS",
  "version" : "0.21.0",
  "packages" : [
    {
      "package_repos": [ "https://0.0.0.0/yum/mesos.repo"],
      "roles" : [
        "zookeeper",
        "mesos_master",
        "mesos_slave",
        "mesos_docker",
        "mesos_chronos",
        "mesos_marathon"
      ]
    }
  ]
}

The repos built into the Fling are not present (unfortunately) on the management server. This was the only tedious portion of the entire process. The easiest method is to grab the files out of an existing BDE Fling management server and copy them into the new one. The other option is find the latest RPMs on the Internet and add them to the management server manually. In either case, you’ll need to run the CentOS syntax for creating the repository.

Create local repo for Apache Mesos

# su - serengeti
$ cd /opt/serengeti/www/yum
$ vim mesos.repo
[a-mesos]
name=Apache Mesos
baseurl=https://0.0.0.0/yum/repos/mesos/current/
enabled=1
gpgcheck=0
sslverify=1
sslcacert=/etc/chef/trusted_certs/serengeti-base.pem

$ mkdir -p repos/mesos/current/RPMS
$ cd repos/mesos/current

The Fling included the following files:
- bigtop-utils-0.8.0.4-1.el6.noarch.rpm
- chronos-2.3.0-0.1.20141121000021.x86_64.rpm
- docker-io-1.3.1-2.el6.x86_64.rpm
- marathon-0.7.5-1.0.x86_64.rpm
- mesos-0.21.0-1.0.centos65.x86_64.rpm
- subversion-1.6.11-10.el6_5.x86_64.rpm
- zookeeper-3.4.5.4-1.el6.noarch.rpm
- zookeeper-server-3.4.5.4-1.el6.noarch.rpm

$ createrepo .

A restart of Tomcat is all that is needed and then you will be able to start deploying Apache Mesos and Kubernetes clusters through BDE v2.2.

If you want to take advantage of the Instant Clone functionality, you will need to be running vSphere 6.0 and BDE v2.2. There are also a couple adjustments to the /opt/serengeti/conf/serengeti.properties files that will be need to be made. I will be going over those in a future post discussing how to use the Photon OS as the template for BDE to deploy.

VM or Docker? What shall I do?

docker-vm

The topic of how to determine if a workload should exist within a virtual machine or a container — specifically Docker — has been pretty hot around the water cooler lately. As I have given this a good deal of thought, a quote from the movie “Can’t Hardly Wait” kept popping into my head from Seth Green rather early on in the movie.

“Class or sex? What shall I do?”

There is additional context to that quote that I have omitted — which you can research for yourself — but the question posed can be changed to “VM or Docker? What shall I do?” Determining which resource should be utilized for which workloads is going to continue to be debated over the coming months and years as applications begin to take advantage of Docker, Mesos, Marathon, Kubernetes and other emerging tools. That being said, there are a few guidelines which I have been focused on when I am asked the question.

Virtual Machine

  • The entity cannot be disposed of or thrown away.
  • The entity needs to be managed via a configuration management system (Chef, Puppet, Salt, etc).
  • The entity needs to run multiple daemons that are not reliant upon one another.
  • The entity needs a SSH daemon to be running on it or provide some sort of shell access.
  • The entity needs internalized persistent data.

Container (Docker)

  • The entity can be thrown away or disposed of with no impact to the application/service offering.
  • The entity exists for seconds or minutes.
  • The entity runs a single daemon/process that is exposed as a service.
  • No persistent data.
  • The entity never requires maintenance or manual intervention. When a newer version of the service is released, the old entities are simply thrown away and new ones are deployed.

These are by no means hard and fast rules, merely a set of guidelines I think about when trying to determine if a workload should be built into a container or a virtual machine. Ultimately the service owner is going to have to make the decision for themselves. It is the responsibility of the cloud provider to be capable of supporting both virtual machines and containers within their environments.

The next six months to a year will be pretty exciting as some of these guidelines are refined and a clearer answer to the question becomes available. I enjoy the debate because it means smart people are thinking about the problems we are facing today in the virtualization space.