VMware Tanzu Kubernetes Grid-TKG 1.6 Deployment on vSphere 8.0

Basic Requirement for deploying the TKG clusters on vSphere

1. Bootstrap Machine   – Linux or Windows
2. TKG OVA
3. TKG CLI and kubectl packages
4. DHCP Network with Internet access
5. AVi  — optional

My Deployment Environment: –

1. VSphere version – 8.0
2. Bootstrap machine – Ubuntu 22.0
3. TKG OVA Version – photon-3-kube-v1.23.8+vmware.2
4. AVI – 22.1.2
5. DHCP Network Pool – 172.100.0.20 – 172.100.0.60, VIP – 172.100.0.70 -80.

Deployment Steps

1. Download the required files from VMware Portal.
2. Using TKG OVA, Create a template.
3. Prepare Bootstrap node – installing docker, Tkg CLI, and kubectl.
4. Deploying TKg Management/Supervisor Cluster.
5. Deploying TKg Workload Cluster.
6. Deploy sample pod.

1. Download the Required below files from VMware Portal

• Login to vSphere and deploy the downloaded TKG OVA File

Right Click on Cluster > Deploy OVF Template

Upload Files > Select OVA File > Next

Keep OVA default name (Don’t change OVA VM Name)

Select Compute Destination > Next

 Next

 Accept License > Next

              Select Storage > Next

              Next

                Finish

OVA Deployment will take a few minutes. Check the deployment status on Recent tasks. Deployment completed.

Right Click on Deployed OVA VM > Template > Convert to Template

Check on Folder & Template section

• Copy Tanzu Cli and Kubectl files to the Bootstrap machine.

Install Docker on Bastion Machine

Commands to install Docker:-

$ sudo apt-get update

$ sudo apt-get install ca-certificates curl gnupg lsb-release

$ sudo mkdir -p /etc/apt/keyrings

$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg –dearmor -o /etc/apt/keyrings/docker.gpg
echo \
“deb [arch=$(dpkg –print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable” | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

Extract the Tanzu Cli package

$ tar –xvf filename

Install tanzu cli

$ Cd cli

$ Install core/v0.25.0/tanzu-core-linux_amd64    /usr/local/bin/tanzu

Unzip Kubectl package

$ chmod ugo+x kubectl-linux-v1.23.8+vmware.2

$ sudo install kubectl-linux-v1.23.8+vmware.2 /usr/local/bin/kubectl

Initiate Tanzu Cli and install all the plug-ins

Create SSH Key on Bastion machine

$ ssh-keygen -t rsa -b 4096 -C "email@example.com"

Add the private key to the SSH agent running on your machine

$ ssh-add ~/.ssh/id_rsa

Next, run the tanzu mc create command with the --ui option

Access the TKG deployment portal using the above Address – http://127.0.0.1:8080

Choose VMware vSphere > Deploy.

Specify required vSphere details (vCenter Server, Username, and password) >Connect.

Select Deploy TKG Management Cluster

Copy previously generated ssh copy files from the bastion machine

Paste SSH key on SSH Public key box > Next

Choose Deployment based on your requirements (Development or Production) > My requirement dev only selected dev cluster small.

Specify the Management Cluster Name, Choose Control plane endpoint provides (Kube-vip or AVI ) (In My env, I select kube-vip), Control plane Endpoint (Each management and workload cluster required one VIP).

Ignore the NSX-T LB Section; we are not using NSX AVI LB

Ignore Metadata section

Select the VM Folder and Datastore from the drop-down list 

Select Network and check any conflict by default Service and Pod CIDRS conflicting with your existing, any conflict with your existing network you can update to a different CIDRS.

Ignore IM

Select Image > Next

Unselect CEIP

Review Configuration

Click on Deploy Management cluster

Depending upon your infra deployment will take a few minutes, you can check the deployment status

You can check the status of the management cluster deployment status –Due to a bug in the 1.6 Version, cluster deployment failed .1.6 older versions’ Deployment will work without any errors/issues.

If you’re deploying the 1.6 below version no need to redeploy the management cluster using CLI  

For 1.6, fix for above Bug:

Using Bastion Cli Delete Failed management cluster

$ Tanzu mc delete

By default cluster config path is- ~/.config/tanzu/tkg/clusterconfigs

 You will find a yaml file with a random name. Just open yaml file and update yaml file – comment CLUSTER_LABELS and save the file

And run cmd to create a management cluster

$ Tanzu mc create –file path

Check management cluster deployment status, now, Deployment is successful.

You can check on vSphere management clusters VMs and cluster status through CLI

• You copy the existing master cluster yaml file to a new file (ex: workload.yaml) – Update Cluster name, You can update the yaml file based on your Requirement.

Run the below command to create a workload cluster

The Deployment will take a few minutes. Check the status workload cluster created successfully

Check Workload cluster VMs on vSphere

Switch to workload cluster using the following command and check cluster node status

• Deploying sample pod on workload cluster

Already I created a sample Nginx yaml file – dinesh.yaml

Run kubectl apply –f dinesh.yaml file to deploy the pod

In my env, Already I deployed AVI LB ,So I exposed nginx pod to AVI LB .(Soon I will post Avi deployment and avi tkg cluster integration doc )

You can check Nginx exposed to LB IP – 172.100.0.150

Checking nginx on Browser