Tanzu BYOH Ubuntu 22.04
Custom OS Support
I have forked the official VMware BYOH repo on GitHub in an effort to support Ubuntu 22.04 for deploying BYOH workload clusters with Kubernetes. I hope to get the changes merged into the upstream repository at some point in the near future.
Start by cloning my forked repository to the TKG Bootstrap VM:
git clone https://github.com/virtualelephant/cluster-api-provider-bringyourownhost.git
The next step is to install Go on the TKG Bootstrap VM, as it is needed to create the Docker container.
bootstrap$ curl https://go.dev/dl/go1.19.linux-amd64.tar.gz
bootstrap$ sudo mkdir -p /usr/local/go
bootstrap$ sudo tar -C /usr/local -zxvf go1.19.linux-amd64.tar.gz
Edit the /etc/environment file and put /usr/local/go/bin in the PATH variable.
The last step is to create the new container image. You will want to have a local Harbor server and a repository where you can upload the container to after creating it. The URL for the Harbor repository will be specified in the CLI when generating the container image.
bootstrap$ IMG=harbor.home.virtualelephant.com/tanzu/cluster-api-byoh-controller:v0.3.1 make docker-build
Now simply upload the new container to the Harbor repository:
bootstrap$ docker push -a harbor.home.virtualelephant.com/tanzu/cluster-api-byoh-controller
Leveraging clusterctl, we can generate the YAML file used to implement the BYOH pod inside the Tanzu management cluster and then modify it to include our updated container image.
bootstrap$ clusterctl generate provider --infrastructure byoh > infrastructure-byoh-ubuntu22-support.yaml
bootstrap$ vim infrastructure-byoh-ubuntu22-support.yaml
1816 #image: projects.registry.vmware.com/cluster_api_provider_bringyourownhost/cluster-api-byoh-controller:v0.3.0
1817 image: harbor.home.virtualelephant.com/tanzu/cluster_api_provider_bringyourownhost/cluster-api-byoh-controller:v0.3.0
bootstrap$ kubectl apply -f infrastructure-byoh-ubuntu22-support.yaml
The pod should be running after it downloads the new container from the Harbor repository and restarts.
bootstrap$ kubectl get pods -n byoh-system
NAME READY STATUS RESTARTS AGE
byoh-controller-manager-5c4b847b56-hd8ws 2/2 Running 7 (4d2h ago) 15d
The next step is to create a new OS packages bundle on your Harbor repository to allow for both Ubuntu 20.04 and Ubuntu 22.04 operating systems to be supported.
Execute the following commands:
bootstrap$ imgpkg copy -i projects.registry.vmware.com/cluster_api_provider_bringyourownhost/byoh-bundle-ubuntu_20.04.1_x86-64_k8s:v1.23.5 --to-tar byoh-bundle-ubuntu_20.04.01_x86-64_k8s.tar
bootstrap$ imgpkg copy --registry-verify-certs=false byoh-bundle-ubuntu_20.04.01_x86-64_k8s.tar --to-repo harbor.home.virtualelephant.com/tanzu/byoh-bundle-ubuntu_20.04.1_x86-64_k8s
bootstrap$ cp byoh-bundle-ubuntu_20.04.01_x86-64_k8s.tar byoh-bundle-ubuntu_22.04.01_x86-64_k8s.tar
bootstrap$ imgpkg copy --registry-verify-certs=false --tar byoh-bundle-ubuntu_22.04.01_x86-64_k8s.tar --to-repo harbor.home.virtualelephant.com/tanzu/byoh-bundle-ubuntu_22.04.1_x86-64_k8s
This will put both OS bundles on your local Harbor repository. Be sure to change the URL to reflect your Harbor FQDN and project name.
In order to leverage the new bundles you’ve created and uploaded onto your local Harbor repository, you will need to update the YAML file used to create the TKG BYOH workload cluster. The YAML file includes changes on lines 165, 212, and 223 to point the Harbor repository and you can download it here.
At this point, you can create the new TKG BYOH workload cluster by using the YAML file (after modifying it for your environment) with the following command:
bootstrap$ kubectl apply -f cluster-os22.yaml