Ansible RabbitMQ Playbooks

I am working on an AMQP Message Broker service architecture, using RabbitMQ, at work right now. As part of the design work, I have spent a bit of time in my vSphere lab standing up the cluster to work out all the configuration, RBAC, policies and other various settings that will be required by the solution. If you haven’t been able to tell lately, my automation tool of choice is Ansible for all the things — I just cannot get enough of it!

Once again, Ansible did not let me down and provides a set of built-in modules for managing RabbitMQ. I found several examples of using the modules to configure a RabbitMQ node and based the work I’ve done off of those. The reason I wrote my own, rather than just git cloning someone else’s work was so that I can write the playbooks (and eventually roles) based on the service architecture specifications I am documenting for the work project.

I have created a new project space on GitHub to host the RabbitMQ playbooks and you are welcome to clone or fork the code based on your needs.

There are currently two playbooks — one for deploying an Ubuntu template into a vSphere environment, one for installing and configuring RabbitMQ on the deployed nodes. I kept the two playbooks separate so that if you want to use install RabbitMQ on a bare-metal or AWS environment, the second playbook can be used as a standalone. If you are choosing to install RabbitMQ in a vSphere environment, the create_vms.yml playbook can be used.

The rabbitmq.yml Ansible playbook will read in a set of environment variables from rabbitmq-vars.yml and then go through the installation steps. I use official repositories for all of the RabbitMQ and Erlang packages.

Note: If you are not using the 16.04 Xenial release, you can change the playbook to use the distribution of Ubuntu you are using inside your environment. I have been sticking with Ubuntu 16.04 LTS mostly because the open-vm-tools package fully support dynamic configuration of the network interfaces through Ansible. If/when 17.10 or 18.04 fully support this configuration through Ansible, I will upgrade my template.

The first part of the playbook adds the official repositories for RabbitMQ and Erlang, then performs the installation of the RabbitMQ package on the hosts.

The next part is a good example of how to use the built-in RabbitMQ modules Ansible includes as part of the core distribution. The playbook starts the plugins needed for RabbitMQ, adds a new administrator user and removes the default RabbitMQ user.

As I finalize the AMQP Message Broker service architecture, the Ansible playbooks will more fully represent the specifications within the documentation. I hope to publicize the service architecture when it is complete in the coming week.


Docker ‘ubuntu-ansible’ update

I have been working with Ansible and all of the vSphere modules an enormous amount recently. As part of that work, I’ve extended the functionality of the Docker container I use for all of my development work. The container can be downloaded from Docker Hub and consumed by anyone — there is no proprietary information within the container.

The updated version includes two vSAN Python modules required for an updated vSAN Ansible module I am working on. In addition, the container now pulls the upstream NSX-v Ansible module from VMware, instead of my cloned repo on The reason being, all of the code I’ve written for NSX-v is now in the upstream module.

The full docker file can be obtained on GitHub.

  1 # Dockerfile for creating an Ansible Control Server with
  2 # the VMware modules necessary to build a complete Kubernetes
  3 # stack.
  4 # Blog details available:
  6 FROM ubuntu:artful
  7 MAINTAINER Chris Mutchler <>
  9 RUN \
 10   apt-get -y update && \
 11   apt-get -y dist-upgrade && \
 12   apt-get -y install software-properties-common python-software-properties vim && \
 13   apt-add-repository ppa:ansible/ansible
 15 # Install packages needed for NSX modules in Ansible
 16 RUN \
 17   apt-get -y update && \
 18   apt-get -y install ansible python-pip python-dev libxml2 libxml2-dev libxslt1-dev zlib1g-dev npm git && \
 19   pip install --upgrade pyvmomi && \
 20   pip install pysphere && \
 21   pip install nsxramlclient && \
 22   npm install -g && \
 23   npm  install -g raml-fleece
 25 # Get NSXRAML
 27 # Add additional Ansible modules for NSX and VM folders
 28 RUN \
 29   git clone -b 6.4 /opt/nsxraml && \
 30   git clone && \
 31   git clone && \
 32   rm -rf nsxansible/library/ && \
 33   cp nsxansible/library/*.py /usr/lib/python2.7/dist-packages/ansible/modules/cloud/vmware/ && \
 34   git clone && \
 35   /bin/cp openshift-ansible-contrib/reference-architecture/vmware-ansible/playbooks/library/vmware*.py /usr/lib/python2.7/dist-packages/ansible/modules/cloud/vmw    are/
 37 # Add vSAN Python API modules - must be done after pyVmomi installation
 38 COPY /usr/lib/python2.7/
 39 COPY /usr/lib/python2.7/
 41 # Setup container to properly use SSH bastion host for Ansible
 42 RUN mkdir /root/.ssh
 43 RUN chmod 740 /root/.ssh
 44 COPY config /root/.ssh/config
 45 COPY ansible.cfg /etc/ansible/
 47 # Edit MOTD to give container consumer info
 48 COPY motd /etc/motd
 49 RUN echo '[ ! -z "$TERM" -a -r /etc/motd ] && cat /etc/issue && cat /etc/motd' >> /etc/bash.bashrc

I am still mounting a local volume that contains the Ansible playbooks within it. For reference, I run the container with the following command:

$ docker run -it --rm --name ansible-sddc -v /PATH/TO/ANSIBLE:/opt/ansible virtualelephant/ubuntu-ansible

If you run into an issues with the Docker container, please let me know on Twitter. Enjoy!

Deploying an SDDC with Ansible

The small effort I started at the end of last year using Ansible to deploy NSX components has snowballed a bit and found its way into a project at work. As we are working to deploy a new HCI architecture internally, one of the efforts we are embarking on is a fully automated, infrastructure-as-code architecture design. There are several components that are working in conjunction with one another to be able to accomplish that task, but the part I am going to talk about today is automation through Ansible.

As many of you have seen, I’ve recently been automating NSX component delivery and configuration using the open source VMware NSX Ansible modules. I’ve been fortunate enough to put my meager coding skills to work and enhance those models this year — adding new capabilities exposed through the API for NSX Edge configuration. In addition to the NSX Ansible modules, there are a multitude of upstream Ansible modules for VMware components. The first step was evaluating what the current upstream modules were capable of performing and putting together a small demo for my colleagues to observe both the power of Ansible and the ease of use.

My initial impressions of Ansible is that it is probably the most user-friendly of the configuration management/automation tools currently available. And for the VMware SDDC components, it appears to be rather robust. I have identified a few holes, but nothing insurmountable — the great thing is if something is exposed via an API, creating an Ansible module to leverage said API is rather simplistic.

The Ansible playbooks are a first step, I really want to convert most of them into Ansible roles. I’ve started committing the code in my Github space. You can download the playbooks and start using them if you’d like.

I currently have playbooks for creating a datacenter, cluster, adding hosts, configuring several advanced settings on each ESXi host, creating a DVS with port groups and performing a few other configuration tasks. The bit that I want to work out next is deployment of the vCenter server through Ansible. It’s currently a work in progress, but it has been a fun effort thus far.


NSX Ansible Updates

It has been a hectic few months for me. I relocated my family to Colorado last month, and as a result all of my side-projects were put on the back-burner. During the time I was away, I was selected for the fourth year in a row as a vExpert! I am grateful to be a part of this awesome community! I strive to make the work I do and submit here worthwhile and informative for others.

Now that I am back into the swing of things, I was able to jump back into improving the NSX-v Ansible module. Recently a member of the community opened an issue regarding the implementation method I had used for Edge NAT rules. Sure enough, they were correct in that the method I was using was really an append and not a creation.

Note: I wrote and tested the code against a pre-release version of NSX-v 6.4.1. There are no documented differences between the two API calls used in the Ansible module in NSX 6.4.x or 6.3.x

When I looked at the NSX API, I realized there were two methods for adding NAT rules to an NSX Edge:

PUT /api/4.0/edges/{edgeId}/nat/config

URI Parameters:
edgeId (required) Specify the ID of the edge in edgeId.
Configure NAT rules for an Edge.

If you use this method to add new NAT rules, you must include all existing rules in the request body. Any rules that are omitted will be deleted.

And also:

POST /api/4.0/edges/{edgeId}/nat/config/rules

URI Parameters:
edgeId (required)
Specify the ID of the edge in edgeId.

Query Parameters:
aboveRuleId (optional)
Specified rule ID. If no NAT rules exist, you can specify rule ID 0.

Add a NAT rule above a specific rule in the NAT rules table (using aboveRuleId query parameter) or append NAT rules to the bottom.

The original code was using the second method, which meant each time an Ansible playbook was run, it would return an OK status because it was adding the rules — even if they already existed.

I decided to dive into the issue and spent a few more hours than I anticipated improving the code. It is now possible to use either method — one to create a full set of rules (deleting any existing rules) or appending new rules to the existing ruleset.

In order to create multiple rules, I modified how the Ansible playbook is interpreted. The example is in the file, but I want to highlight it here:

  1 ---
  2 - hosts: localhost
  3   connection: local
  4   gather_facts: False
  5   vars_files:
  6     - nsxanswer.yml
  7     - envanswer.yml
  9   tasks:
 10   - name: Create NAT rules
 11     nsx_edge_nat:
 12       nsxmanager_spec: '{{ nsxmanager_spec }}'
 13       mode: 'create'
 14       name: '{{ edge_name }}'
 15       rules:
 16         dnat0: { description: 'Ansible created HTTP NAT rule',
 17             loggingEnabled: 'true',
 18             rule_type: 'dnat',
 19             nat_enabled: 'true',
 20             dnatMatchSourceAddress: 'any',
 21             dnatMatchSourcePort: 'any',
 22             vnic: '0',
 23             protocol: 'tcp',
 24             originalAddress: '',
 25             originalPort: '80',
 26             translatedAddress: '',
 27             translatedPort: '80'
 28           }
 29         dnat1: { description: 'Ansible created HTTPS NAT rule',
 30             loggingEnabled: 'true',
 31             rule_type: 'dnat',
 32             vnic: '0',
 33             nat_enabled: 'true',
 34             dnatMatchSourceAddress: 'any',
 35             dnatMatchSourcePort: 'any',
 36             protocol: 'tcp',
 37             originalAddress: '',
 38             originalPort: '443',
 39             translatedAddress: '',
 40             translatedPort: '443'
 41           }

Please note the identifiers, dnat0 and dnat1, are merely that — identifiers for your playbook. They do not influence the API call made to the NSX Manager.

A new function was required in the Ansible module to allow for multiple rules to be appended to one another to make a single API call that would add each rule. The data structure used to create this dictionary of lists was rather convoluted since Python struggles to convert these sort of thing to XML properly. With some help from a few people in the NSBU, I was able to get it working.

def create_init_nat_rules:

 55 def create_init_nat_rules(client_session, module):
 56     """
 57     Create single dictionary with all of the NAT rules, both SNAT and DNAT, to be used
 58     in a single API call. Should be used when wiping out ALL existing rules or when
 59     a new NSX Edge is created.
 60     :return: return dictionary with the full NAT rules list
 61     """
 62     nat_rules = module.params['rules']
 63     params_check_nat_rules(module)
 65     nat_rules_info = {}
 66     nat_rules_info['natRule'] = []
 68     for rule_key, nat_rule in nat_rules.items():
 69         rules_index = rule_key[-1:]
 70         rule_type = nat_rule['rule_type']
 71         if rule_type == 'snat':
 72             nat_rules_info['natRule'].append(
 73                                     {'action': rule_type, 'vnic': nat_rule['vnic'], 'originalAddress': nat_rule['originalAddress'],
 74                                      'translatedAddress': nat_rule['translatedAddress'], 'loggingEnabled': nat_rule['loggingEnabled'],
 75                                      'enabled': nat_rule['nat_enabled'], 'protocol': nat_rule['protocol'], 'originalPort': nat_rule['originalPort'],
 76                                      'translatedPort': nat_rule['translatedPort'], 'snatMatchDestinationAddress': nat_rule['snatMatchDestinationAddress'],
 77                                      'snatMatchDestinationPort': nat_rule['snatMatchDestinationPort'], 'description': nat_rule['description']
 78                                     }
 79                                   )
 80         elif rule_type == 'dnat':
 81             nat_rules_info['natRule'].append(
 82                                     {'action': rule_type, 'vnic': nat_rule['vnic'], 'originalAddress': nat_rule['originalAddress'],
 83                                      'translatedAddress': nat_rule['translatedAddress'], 'loggingEnabled': nat_rule['loggingEnabled'],
 84                                      'enabled': nat_rule['nat_enabled'], 'protocol': nat_rule['protocol'], 'originalPort': nat_rule['originalPort'],
 85                                      'translatedPort': nat_rule['translatedPort'], 'dnatMatchSourceAddress': nat_rule['dnatMatchSourceAddress'],
 86                                      'dnatMatchSourcePort': nat_rule['dnatMatchSourcePort'], 'description': nat_rule['description']
 87                                     }
 88                                   )
 90         if nat_rule['protocol'] == 'icmp':
 91             nat_rules_info['natRule']['icmpType'] = nat_rule['icmpType']
 93     return nat_rules_info

I also took the opportunity to clean up some of the excessively long lines of code to make it more clearly readable. The result is a new working playbook for initial Edge NAT rule creation and the ability to add new rules later. There are a few items that remain where I would like to see some improvements — mainly I would like to add logic in to the code that, when performing an append, it will check to see if the rule already exists and skip it.

In the meantime, the code has been checked into GitHub and the Docker image I use for running Ansible has been updated.


Ansible NSX module for creating NAT rules

After working on the code last weekend and testing the functionality this past week, I am proud to announce the ability to create NAT rules on an NSX Edge is possible through Ansible! The ability to create SNAT and DNAT rules on the NSX Edge was a necessity for the Infrastructure-as-Code project, as each environment deployed uses it’s own micro-segmented network. I am doing that so that each environment can be stood up multiple times within the same vSphere environment and be solely dependent upon itself.

The current Ansible module allows for the creation of both SNAT and DNAT rules. I used the NSX API Guide to be able to determine which variables are acceptable to be passed to either types of NAT rules and included each one in the function. As such, there are no features missing from the module today.

The module can be downloaded from the GitHub virtualelephant/nsxansible repo.

To use the module, I have created an example Ansible playbook (also available on GitHub):


  1 ---
  2 - hosts: localhost
  3   connection: local
  4   gather_facts: False
  5   vars_files:
  6     - nsxanswer.yml
  8   tasks:
  9   - name: Create SSH DNAT rule
 10     nsx_edge_nat:
 11       nsxmanager_spec: '{{ nsxmanager_spec }}'
 12       mode: 'create'
 13       name: '{{ edge_name }}'
 14       rule_type: 'dnat'
 15       vnic: '0'
 16       protocol: 'tcp'
 17       originalAddress: ''
 18       originalPort: '22'
 19       translatedAddress: ''
 20       translatedPort: '22'
 22   - name: Create default outbound SNAT rule
 23     nsx_edge_nat:
 24       nsxmanager_spec: '{{ nsxmanager_spec }}'
 25       mode: 'create'
 26       name: '{{ edge_name }}'
 27       rule_type: 'snat'
 28       vnic: '0'
 29       protocol: 'any'
 30       originalAddress: ''
 31       originalPort: 'any'
 32       translatedAddress: ''
 33       translatedPort: 'any'

Update Jan 30th:

The module was updated a few days ago to include the ability to delete a NAT rule from an Edge. The functionality allows the consumer to write a playbook with the following information to delete an individual rule.

  1 ---
  2 - hosts: localhost
  3   connection: local
  4   gather_facts: False
  5   vars_files:
  6     - nsxanswer.yml
  7     - envanswer.yml
  9   tasks:
 10   - name: Delete HTTP NAT rule
 11     nsx_edge_nat:
 12       nsxmanager_spec: '{{ nsxmanager_spec }}'
 13       mode: 'delete'
 14       name: '{{ edge_name }}'
 15       ruleId: '196622'

Let me know if you are using the NSX Ansible modules and what other functionality you would like to see added.