VCDX Quick Hit: Convey Credibility

A few months ago I participated in a ‘Trusted Adviser’ training internally at VMware. The focus of the seminar was to help us learn how to become a trusted adviser with a customer with little or no previous contact prior to the engagement. During the seminar a few things stood out as relating to the VCDX certification and defense. Similar to how we strive to become trusted adviser’s to our customers when we have an engagement, the role of a VCDX candidate (in part) is to convey credibility to the member of the panel.

be in command of the content

Whether you are defending an architecture you designed entirely on your own or worked on as part of a team, being in command of the content is a key factor in successfully defending it in front of the panel. During the defense, you should demonstrate to the panelists that no one understands the design better than you. This is especially important if you did work on the design as part of a team — you are expected to know every facet of the design, the reasoning behind all of the design decisions made, the risk and implications introduced as if you had been the sole architect working on the project.

Demonstrate to the panelists you are the authority on the design.

Being able to effectively communicate this authority, clearly and concisely, will come as you practice your initial 10-15 minute presentation over and over again leading up to the defense itself. Critically assess your design prior to the defense by anticipating areas where the panelists may have questions or where you believe the design could have been improved if the requirements or constraints of the design had been different.

be honest and believable

This might seem trite, but being honest during your defense is critical. It is okay to not have an answer for every question the panel asks you. We talk about this during the VCDX workshop, responding to a question with “I don’t know” or “I would need to investigate that further” is an adequate occasional response. Explain what you do know or understand about the question and then move on. Generally speaking, people (and the panelists are people) can tell when a person is being deceptive within a few minutes time.

Just be mindful of how often “I don’t know” is your answer — see above about commanding the content.


The VCDX defense day is relatively short compared to the number of hours invested in the process of becoming a VCDX candidate invited to defend. However, in that limited time you need to be consistent in your messaging and reasoning behind the design decisions you made in your design and during the design scenario. Our previous experiences have a direct impact on how we address requirements, constraints and risks inside our designs.

These experiences are different for each architect, however they are what make up our core beliefs. Be consciously aware of what your core beliefs are and let them shine through to the panel by being consistent during your defense.

be concise

Similarly to the first point, adequate preparation prior to the defense should include anticipating areas of the design where the panelists may pose questions. When answering a question posed by a panelist, be concise in your answers, avoid tangents or repeating the question back to the panel. Time is limited and having concise answers prepared for the potential questions, will help you manage the time.

Remember (generally) the more questions you can be asked and answer, will increase the probability of receiving a passing score and earning the VCDX certification.

For any potential VCDX candidates, I hope these personal insights will be helpful to you on your journey towards earning the VCDX certification.

VMworld US 2018 – Session and Schedule

With the VMworld US 2018 conference set to start in just over a month, my schedule is beginning to take shape. 2018 will be my 5th trip to the US show and I am extremely excited to be presenting again on several different stages in Las Vegas!

Sunday – TAM Day table from 4:15PM -5PM

Come find me to talk about the full vSphere SDDC, including NSX-v, NSX-T, vSphere, vCloud Director and/or the VCDX certification.

Monday – VMTN Lounge from 1PM – 1:30PM

Session ID: CODE5542U – Enhancing the SDDC with Ansible

New engineers are often unsure how to contribute to an Open Source project and timid when it is time to make their first merge request. Drawing insight from the recent Ansible automation efforts within the VMware Private Cloud team, learn how to identify areas within an Open Source project where you can contribute using a real-world use-case. In this session, we will walk through the open source Ansible VMware modules leveraged to fully automate the deployment of the software-defined datacenter, including vSphere, vSAN and NSX components. We’ll dive into how gaps in the current Ansible modules were identified and how our engineers were able to begin writing additional functionality and contributing it upstream. The presentation will also highlight how improvements to our own VMware Open Source projects on GitHub can benefit and be leveraged by our customers.

Tuesday – experts bar from 11:3oAM – 1:30PM

Come find me to talk about the full vSphere SDDC, including NSX-v, NSX-T, vSphere, PKS and/or the VCDX certification.

Tuesday – solutions exchange theatre 3:30pm – 4PM

Come learn about how VMware utilizes multiple public cloud providers, in addition to our on-premise clouds, to deliver a memorable VMworld Hands-on-Lab experience each year! I will go over the VCDX methodologies and how they were leveraged to design the Hands-on-Lab architecture.

Tuesday – meet the experts @ solutions exchange theatre from 3:30pm – 4PM

Come find me to talk about the full vSphere stack, including NSX-v, NSX-T, vSphere, PKS and/or the VCDX certification.

thursday – Transformers: How vmware IT transitioned to a services-based organization from 10:30AM – 11:30AM

Session ID: LDT1515PU

Hot on the heels of the digital transformation process sweeping across IT organizations worldwide, the VMware private cloud architects will discuss how they are leveraging their operational experience to propel VMware IT into a services-based organization. As VMware IT has learned, the digital transformation process is less about the technologies adopted and more about the people involved. In order to drive true adoption, the digital transformation starts with strong leadership, but also requires engagement from those closest to the consumers. Come discover how to drive your organization’s digital transformation initiatives and gain true adoption from a technical practitioner’s perspective. Challenges and lessons learned will also be covered.

Docker ‘ubuntu-ansible’ update

I have been working with Ansible and all of the vSphere modules an enormous amount recently. As part of that work, I’ve extended the functionality of the Docker container I use for all of my development work. The container can be downloaded from Docker Hub and consumed by anyone — there is no proprietary information within the container.

The updated version includes two vSAN Python modules required for an updated vSAN Ansible module I am working on. In addition, the container now pulls the upstream NSX-v Ansible module from VMware, instead of my cloned repo on The reason being, all of the code I’ve written for NSX-v is now in the upstream module.

The full docker file can be obtained on GitHub.

  1 # Dockerfile for creating an Ansible Control Server with
  2 # the VMware modules necessary to build a complete Kubernetes
  3 # stack.
  4 # Blog details available:
  6 FROM ubuntu:artful
  7 MAINTAINER Chris Mutchler <>
  9 RUN \
 10   apt-get -y update && \
 11   apt-get -y dist-upgrade && \
 12   apt-get -y install software-properties-common python-software-properties vim && \
 13   apt-add-repository ppa:ansible/ansible
 15 # Install packages needed for NSX modules in Ansible
 16 RUN \
 17   apt-get -y update && \
 18   apt-get -y install ansible python-pip python-dev libxml2 libxml2-dev libxslt1-dev zlib1g-dev npm git && \
 19   pip install --upgrade pyvmomi && \
 20   pip install pysphere && \
 21   pip install nsxramlclient && \
 22   npm install -g && \
 23   npm  install -g raml-fleece
 25 # Get NSXRAML
 27 # Add additional Ansible modules for NSX and VM folders
 28 RUN \
 29   git clone -b 6.4 /opt/nsxraml && \
 30   git clone && \
 31   git clone && \
 32   rm -rf nsxansible/library/ && \
 33   cp nsxansible/library/*.py /usr/lib/python2.7/dist-packages/ansible/modules/cloud/vmware/ && \
 34   git clone && \
 35   /bin/cp openshift-ansible-contrib/reference-architecture/vmware-ansible/playbooks/library/vmware*.py /usr/lib/python2.7/dist-packages/ansible/modules/cloud/vmw    are/
 37 # Add vSAN Python API modules - must be done after pyVmomi installation
 38 COPY /usr/lib/python2.7/
 39 COPY /usr/lib/python2.7/
 41 # Setup container to properly use SSH bastion host for Ansible
 42 RUN mkdir /root/.ssh
 43 RUN chmod 740 /root/.ssh
 44 COPY config /root/.ssh/config
 45 COPY ansible.cfg /etc/ansible/
 47 # Edit MOTD to give container consumer info
 48 COPY motd /etc/motd
 49 RUN echo '[ ! -z "$TERM" -a -r /etc/motd ] && cat /etc/issue && cat /etc/motd' >> /etc/bash.bashrc

I am still mounting a local volume that contains the Ansible playbooks within it. For reference, I run the container with the following command:

$ docker run -it --rm --name ansible-sddc -v /PATH/TO/ANSIBLE:/opt/ansible virtualelephant/ubuntu-ansible

If you run into an issues with the Docker container, please let me know on Twitter. Enjoy!

NSX Ansible Updates

It has been a hectic few months for me. I relocated my family to Colorado last month, and as a result all of my side-projects were put on the back-burner. During the time I was away, I was selected for the fourth year in a row as a vExpert! I am grateful to be a part of this awesome community! I strive to make the work I do and submit here worthwhile and informative for others.

Now that I am back into the swing of things, I was able to jump back into improving the NSX-v Ansible module. Recently a member of the community opened an issue regarding the implementation method I had used for Edge NAT rules. Sure enough, they were correct in that the method I was using was really an append and not a creation.

Note: I wrote and tested the code against a pre-release version of NSX-v 6.4.1. There are no documented differences between the two API calls used in the Ansible module in NSX 6.4.x or 6.3.x

When I looked at the NSX API, I realized there were two methods for adding NAT rules to an NSX Edge:

PUT /api/4.0/edges/{edgeId}/nat/config

URI Parameters:
edgeId (required) Specify the ID of the edge in edgeId.
Configure NAT rules for an Edge.

If you use this method to add new NAT rules, you must include all existing rules in the request body. Any rules that are omitted will be deleted.

And also:

POST /api/4.0/edges/{edgeId}/nat/config/rules

URI Parameters:
edgeId (required)
Specify the ID of the edge in edgeId.

Query Parameters:
aboveRuleId (optional)
Specified rule ID. If no NAT rules exist, you can specify rule ID 0.

Add a NAT rule above a specific rule in the NAT rules table (using aboveRuleId query parameter) or append NAT rules to the bottom.

The original code was using the second method, which meant each time an Ansible playbook was run, it would return an OK status because it was adding the rules — even if they already existed.

I decided to dive into the issue and spent a few more hours than I anticipated improving the code. It is now possible to use either method — one to create a full set of rules (deleting any existing rules) or appending new rules to the existing ruleset.

In order to create multiple rules, I modified how the Ansible playbook is interpreted. The example is in the file, but I want to highlight it here:

  1 ---
  2 - hosts: localhost
  3   connection: local
  4   gather_facts: False
  5   vars_files:
  6     - nsxanswer.yml
  7     - envanswer.yml
  9   tasks:
 10   - name: Create NAT rules
 11     nsx_edge_nat:
 12       nsxmanager_spec: '{{ nsxmanager_spec }}'
 13       mode: 'create'
 14       name: '{{ edge_name }}'
 15       rules:
 16         dnat0: { description: 'Ansible created HTTP NAT rule',
 17             loggingEnabled: 'true',
 18             rule_type: 'dnat',
 19             nat_enabled: 'true',
 20             dnatMatchSourceAddress: 'any',
 21             dnatMatchSourcePort: 'any',
 22             vnic: '0',
 23             protocol: 'tcp',
 24             originalAddress: '',
 25             originalPort: '80',
 26             translatedAddress: '',
 27             translatedPort: '80'
 28           }
 29         dnat1: { description: 'Ansible created HTTPS NAT rule',
 30             loggingEnabled: 'true',
 31             rule_type: 'dnat',
 32             vnic: '0',
 33             nat_enabled: 'true',
 34             dnatMatchSourceAddress: 'any',
 35             dnatMatchSourcePort: 'any',
 36             protocol: 'tcp',
 37             originalAddress: '',
 38             originalPort: '443',
 39             translatedAddress: '',
 40             translatedPort: '443'
 41           }

Please note the identifiers, dnat0 and dnat1, are merely that — identifiers for your playbook. They do not influence the API call made to the NSX Manager.

A new function was required in the Ansible module to allow for multiple rules to be appended to one another to make a single API call that would add each rule. The data structure used to create this dictionary of lists was rather convoluted since Python struggles to convert these sort of thing to XML properly. With some help from a few people in the NSBU, I was able to get it working.

def create_init_nat_rules:

 55 def create_init_nat_rules(client_session, module):
 56     """
 57     Create single dictionary with all of the NAT rules, both SNAT and DNAT, to be used
 58     in a single API call. Should be used when wiping out ALL existing rules or when
 59     a new NSX Edge is created.
 60     :return: return dictionary with the full NAT rules list
 61     """
 62     nat_rules = module.params['rules']
 63     params_check_nat_rules(module)
 65     nat_rules_info = {}
 66     nat_rules_info['natRule'] = []
 68     for rule_key, nat_rule in nat_rules.items():
 69         rules_index = rule_key[-1:]
 70         rule_type = nat_rule['rule_type']
 71         if rule_type == 'snat':
 72             nat_rules_info['natRule'].append(
 73                                     {'action': rule_type, 'vnic': nat_rule['vnic'], 'originalAddress': nat_rule['originalAddress'],
 74                                      'translatedAddress': nat_rule['translatedAddress'], 'loggingEnabled': nat_rule['loggingEnabled'],
 75                                      'enabled': nat_rule['nat_enabled'], 'protocol': nat_rule['protocol'], 'originalPort': nat_rule['originalPort'],
 76                                      'translatedPort': nat_rule['translatedPort'], 'snatMatchDestinationAddress': nat_rule['snatMatchDestinationAddress'],
 77                                      'snatMatchDestinationPort': nat_rule['snatMatchDestinationPort'], 'description': nat_rule['description']
 78                                     }
 79                                   )
 80         elif rule_type == 'dnat':
 81             nat_rules_info['natRule'].append(
 82                                     {'action': rule_type, 'vnic': nat_rule['vnic'], 'originalAddress': nat_rule['originalAddress'],
 83                                      'translatedAddress': nat_rule['translatedAddress'], 'loggingEnabled': nat_rule['loggingEnabled'],
 84                                      'enabled': nat_rule['nat_enabled'], 'protocol': nat_rule['protocol'], 'originalPort': nat_rule['originalPort'],
 85                                      'translatedPort': nat_rule['translatedPort'], 'dnatMatchSourceAddress': nat_rule['dnatMatchSourceAddress'],
 86                                      'dnatMatchSourcePort': nat_rule['dnatMatchSourcePort'], 'description': nat_rule['description']
 87                                     }
 88                                   )
 90         if nat_rule['protocol'] == 'icmp':
 91             nat_rules_info['natRule']['icmpType'] = nat_rule['icmpType']
 93     return nat_rules_info

I also took the opportunity to clean up some of the excessively long lines of code to make it more clearly readable. The result is a new working playbook for initial Edge NAT rule creation and the ability to add new rules later. There are a few items that remain where I would like to see some improvements — mainly I would like to add logic in to the code that, when performing an append, it will check to see if the rule already exists and skip it.

In the meantime, the code has been checked into GitHub and the Docker image I use for running Ansible has been updated.


Ansible NSX module for creating NAT rules

After working on the code last weekend and testing the functionality this past week, I am proud to announce the ability to create NAT rules on an NSX Edge is possible through Ansible! The ability to create SNAT and DNAT rules on the NSX Edge was a necessity for the Infrastructure-as-Code project, as each environment deployed uses it’s own micro-segmented network. I am doing that so that each environment can be stood up multiple times within the same vSphere environment and be solely dependent upon itself.

The current Ansible module allows for the creation of both SNAT and DNAT rules. I used the NSX API Guide to be able to determine which variables are acceptable to be passed to either types of NAT rules and included each one in the function. As such, there are no features missing from the module today.

The module can be downloaded from the GitHub virtualelephant/nsxansible repo.

To use the module, I have created an example Ansible playbook (also available on GitHub):


  1 ---
  2 - hosts: localhost
  3   connection: local
  4   gather_facts: False
  5   vars_files:
  6     - nsxanswer.yml
  8   tasks:
  9   - name: Create SSH DNAT rule
 10     nsx_edge_nat:
 11       nsxmanager_spec: '{{ nsxmanager_spec }}'
 12       mode: 'create'
 13       name: '{{ edge_name }}'
 14       rule_type: 'dnat'
 15       vnic: '0'
 16       protocol: 'tcp'
 17       originalAddress: ''
 18       originalPort: '22'
 19       translatedAddress: ''
 20       translatedPort: '22'
 22   - name: Create default outbound SNAT rule
 23     nsx_edge_nat:
 24       nsxmanager_spec: '{{ nsxmanager_spec }}'
 25       mode: 'create'
 26       name: '{{ edge_name }}'
 27       rule_type: 'snat'
 28       vnic: '0'
 29       protocol: 'any'
 30       originalAddress: ''
 31       originalPort: 'any'
 32       translatedAddress: ''
 33       translatedPort: 'any'

Update Jan 30th:

The module was updated a few days ago to include the ability to delete a NAT rule from an Edge. The functionality allows the consumer to write a playbook with the following information to delete an individual rule.

  1 ---
  2 - hosts: localhost
  3   connection: local
  4   gather_facts: False
  5   vars_files:
  6     - nsxanswer.yml
  7     - envanswer.yml
  9   tasks:
 10   - name: Delete HTTP NAT rule
 11     nsx_edge_nat:
 12       nsxmanager_spec: '{{ nsxmanager_spec }}'
 13       mode: 'delete'
 14       name: '{{ edge_name }}'
 15       ruleId: '196622'

Let me know if you are using the NSX Ansible modules and what other functionality you would like to see added.