Exposing Apache Mesos on VMware Big Data Extensions v2.2

The VMware Big Data Extensions v2.2 release included the cookbooks for Apache Mesos and Kubernetes from the Fling released this past spring. However, those cookbooks are not exposed when you deploy the new version. Fortunately, unlocking them only takes a few minutes before they can be made available! I will cover exactly what is needed in order to begin using these Cloud Native App cluster deployments below.

If you jump onto your v2.2 management server and look in the /opt/serengeti/chef/cookbooks directory, you will see all of the Cloud Native App additions.

BDE-Cookbooks-v2.2

A quick look to be sure the Chef roles are still defined tells us that they are.

Chef-BDE-v2.2-Roles

 

They even did us the favor of including the JSON spec files in the /opt/serengeti/www/specs/Ironfan directory.

BDE-v2.2-Mesos-JSON

The missing pieces are the entries in the /opt/serengeti/www/specs/map and /opt/serengeti/www/distros/manifest files. Those are rather easy to copy out of the VMware Fling itself or re-create manually. If you want to edit the files yourself, here is what needs to be added to the files.

/opt/serengeti/www/specs/map

{
  "vendor" : "Kubernetes",
  "version" : "^(\\d)+(\\.\\w+)*",
  "type" : "Basic Kubernetes Cluster",
  "appManager" : "Default",
  "path" : "Ironfan/kubernetes/basic/spec.json"
},
{
  "vendor" : "Mesos",
  "version" : "^(\\d)+(\\.\\w+)*",
  "type" : "Basic Mesos Cluster",
  "appManager" : "Default",
  "path" : "Ironfan/mesos/basic/spec.json"
},

/opt/serengeti/www/distros/manifest

{
  "name" : "kubernetes",
  "vendor" : "KUBERNETES",
  "version" : "0.5.4",
  "packages" : [
    {
      "tarball": "kubernetes/kubernetes-0.5.4.tar.gz",
      "roles": [
        "kubernetes_workstation",
        "kubernetes_master",
        "kubernetes_minion"
      ]
    }
  ]
},
{
  "name" : "mesos",
  "vendor" : "MESOS",
  "version" : "0.21.0",
  "packages" : [
    {
      "package_repos": [ "https://0.0.0.0/yum/mesos.repo"],
      "roles" : [
        "zookeeper",
        "mesos_master",
        "mesos_slave",
        "mesos_docker",
        "mesos_chronos",
        "mesos_marathon"
      ]
    }
  ]
}

The repos built into the Fling are not present (unfortunately) on the management server. This was the only tedious portion of the entire process. The easiest method is to grab the files out of an existing BDE Fling management server and copy them into the new one. The other option is find the latest RPMs on the Internet and add them to the management server manually. In either case, you’ll need to run the CentOS syntax for creating the repository.

Create local repo for Apache Mesos

# su - serengeti
$ cd /opt/serengeti/www/yum
$ vim mesos.repo
[a-mesos]
name=Apache Mesos
baseurl=https://0.0.0.0/yum/repos/mesos/current/
enabled=1
gpgcheck=0
sslverify=1
sslcacert=/etc/chef/trusted_certs/serengeti-base.pem

$ mkdir -p repos/mesos/current/RPMS
$ cd repos/mesos/current

The Fling included the following files:
- bigtop-utils-0.8.0.4-1.el6.noarch.rpm
- chronos-2.3.0-0.1.20141121000021.x86_64.rpm
- docker-io-1.3.1-2.el6.x86_64.rpm
- marathon-0.7.5-1.0.x86_64.rpm
- mesos-0.21.0-1.0.centos65.x86_64.rpm
- subversion-1.6.11-10.el6_5.x86_64.rpm
- zookeeper-3.4.5.4-1.el6.noarch.rpm
- zookeeper-server-3.4.5.4-1.el6.noarch.rpm

$ createrepo .

A restart of Tomcat is all that is needed and then you will be able to start deploying Apache Mesos and Kubernetes clusters through BDE v2.2.

If you want to take advantage of the Instant Clone functionality, you will need to be running vSphere 6.0 and BDE v2.2. There are also a couple adjustments to the /opt/serengeti/conf/serengeti.properties files that will be need to be made. I will be going over those in a future post discussing how to use the Photon OS as the template for BDE to deploy.