When I posted on Twitter that I had begun building out a new home lab SDDC, I had several people reach out and ask me to document what I was building.
After a few years of regularly leveraging a physical SDDC environment inside a VMware data center (perks of my previous role), I realized that I needed to build a new home lab to improve my skillset with NSX-T and the multiple Kubernetes offerings from VMware. As I set out to build the new lab, I knew that it had certain requirements it needed to fulfill. I also knew that I had a limited budget and would have to make sacrifices here and there.
The lab had to meet the following requirements:
- Servers had to be compatible with and able to run vSphere 7.0.
- Servers had to have 4 NICs — 2 dedicated for VDS and 2 dedicated for NSX-T.
- All-Flash vSAN HCL compatible and capable servers.
- Environment needed to be able to run NSX-T, PKS, Project Pacific, Tanzu Grid / Mission Control, vRealize Log Insight, vRealize Operations, and vRealize Network Insight.
- Minimum of 3 servers.
- Initial spend should be around $2,500 USD.
On a recent trip to Canada, a co-worker and I were talking about home labs and where to find decent equipment. My previous home lab (circa 2014) was based on several Intel NUCs and those are grossly inadequate to my current needs. During our discussion, he mentioned a site called Orange Computers as a place to checkout for used servers. This site was the answer to my budget requirement.
I was able to pick up 3 Dell PowerEdge R630 servers with the following configuration:
- Single Intel Xeon E5-2660 v3 10-Core 2.6GHz 25MB 9.6GT/s 105W
- 64GB (2 x 32GB) DDR4-2133
- PERC H730 1GB NV -12G SAS,6G SATA- (RAID 0,1,5,6,10,50,60)
- Dell 4-Port Gigabit Ethernet NDC
- Single 495W Platinum PSU
Not what I’d consider a production-grade server, but for the home lab it should be more than adequate.
The memory is currently at 64GB, but I have plans to expand each node to 128GB shortly. Memory is still expensive so I’ll likely wait a bit to increase the memory footprint.
The E5-2660 v3 processor is on the HCL for vSphere 7.0 and the PERC H730 is on the vSAN HCL. Both should allow for some longevity in the hardware for the next few years.
The Cisco SG-300 10-port Gigabit switch from the previous lab iteration was a bit long in the tooth and didn’t have support for BGP, which I knew I wanted for NSX-T stuff. As such, I went hunting on Amazon and found a circa 2005 replacement switch. The “new to me” Cisco Catalyst 3560G 24-port Gigabit switch will fulfill all my networking needs. It has both L2 and L3 functionality and will pair nicely with the NSX-T components.
You might ask why not a 10 Gigabit network? Mostly cost and my lab requirements couldn’t justify that expense. This will literally be just a lab — no plans to run a bunch of long-term VMs or applications. It’s supposed to be a playground for testing new VMware products and technologies.
You’ll notice there are no SSDs mentioned in the BOM. That is because I am still considering what to buy from an SSD perspective and because the initial build used up my $2,500 budget. I plan on adding drives to servers in the next month or so, as additional funds become available. As soon as I’ve made a determination and purchase, I’ll update the post.
In the meantime, I am leveraging my “legacy” Synology 1515+ array from my original home lab. It currently is configured with 5 Western Digital 2TB Red drives. I have a single 4TB iSCSI LUN exposed to the SDDC with a bonded 4GB interface being fed into the Catalyst switch.
Purpose Built Lab
The entire purpose of the new lab build is to be able to support, learn and develop blog and workshop content on the myriad of VMware Kubernetes offerings. In addition, NSX-T is becoming a more critical part of my role within the Advanced Customer Engagement team and I’ve needed to dedicate more time to becoming a SME with it.
That being said, all of the hardware arrived this week and with a week between business trips, I set out to configure the environment. Thus far I have been able to install vSphere 6.7U3, vRealize Log Insight 8.0, and vRealize Operations 7.5.
As the lab build-out continues, I will provide updates on my blog. I’m really looking forward to diving into the depths of NSX-T and PKS initially!