Building Mesosphere & Apache Mesos into BDE:
After playing with Mesosphere in AWS for the week, getting familiar with the packages and the deployment process, the real work has begun — getting the Mesosphere stack (Apache Mesos, Apache Zookeeper, Mesosphere Marathon, Chronos and HAProxy) deployed through VMware Big Data Extensions. Fortunately, BDE v2.1 has some example JSON cluster definition files that can be used for deploying different types of clusters and these are perfect for modification in this use-case.
The example files are located in the directory /opt/serengeti/samples. I used the basic_cluster.json file in the directory as the template. From there, I modified the file based on what the Mesosphere stack deployed in AWS, with some slight modifications. I chose to have a base Mesos cluster include 3 master nodes and 6 worker nodes. The master nodes are allocated with 2vCPU, 8GB RAM and 50GB of disk space. The worker nodes are allocated with 2vCPU, 8GB RAM and 100GB of disk space.
The remainder of the post will go through all the various pieces that are necessary to utilize the Big Data Extensions framework to offer the Mesosphere stack within a VMware virtual environment.
Big Data Extensions has multiple parts that make up the framework, and it is the framework that I believe offers VMware architects to extend the offerings beyond just Hadoop. As I said, along with Andrew Nelson (@vmwnelson), during our VMworld 2014 sessions, this very framework is the true strength of BDE.
The cluster definition file includes the following syntax.
1 // Based off of the sample basic_cluster.json file provided by Serengeti.
2 // Modified to handle deployment of the Mesosphere stack.
3 {
4 "nodeGroups":[
5 {
6 "name": "master",
7 "roles": [
8 "zookeeper",
9 "mesos"
10 ],
11 "instanceNum": 3,
12 "cpuNum": 2,
13 "memCapacityMB": 8192
14 "storage": {
15 "type": "SHARED",
16 "sizeGB": 50
17 },
18 "haFlag": "on"
19 },
20 {
21 "name": "worker",
22 "roles": [
23 "mesos"
24 ],
25 "instanceNum": 6,
26 "cpuNum": 2,
27 "memCapacityMB": 8192,
28 "storage": {
29 "type": "LOCAL",
30 "sizeGB": 100
31 },
32 "haFlag": "on"
33 }
34 ],
35 "configuration": {
36 "zookeeper": {
37 "java.env": {
38 "JVMFLAGS": "-Xmx2g"
39 },
40 "log4j.properties": {
41 "zookeeper.root.logger": "DEBUG,DRFA"
42 }
43 }
44 }
45 }
There is a section for additional configuration where additional options can be added to specific files for the roles. This allows a great amount of customization, especially if there are multiple roles sharing Chef cookbook recipes.
A definition for the manifest file is required so that BDE will offer up the Mesos cluster as an option for use. The /opt/serengeti/www/distros/manifest file was edited and the following was added.
1 [
2 {
3 "name" : "Mesos",
4 "vendor" : "Mesosphere",
5 "version" : "0.20.1",
6 "packages" : [
7 {
8 "roles" : ["master", "worker"],
9 "package_repos" : ["https://[IP.ADD.RE.SS]/yum/mesos.repo"]
10 }
11 ]
12 }
13 ]
Mesos will also need to be defined in the /opt/serengeti/www/specs/map file to tie all the pieces within BDE together.
1 [
2 {
3 "vendor" : "Mesosphere",
4 "version" : "^0(\\.\\w+)*",
5 "type" : "Mesos Cluster"
6 "appManager" : "Default",
7 "path" : "Mesos/spec.json"
8 }
9 ]
One additional file needs to be modified — /opt/serengeti/conf/serengeti.properties — edit line #50:
50 serengeti.distro_vendor = GENERIC,APACHE,BIGTOP,GPHD,PHP,HDP,CDH,MAPR,MESOSPHERE
At this point, the framework pieces are in place for a Mesos cluster to be deployed using BDE within the VMware environment. The next post will cover configuring the management server to use the Mesosphere RPM repository and the Chef cookbook recipes required for the individual roles.
Apache Mesos Clusters – Part 3
