Tanzu Kubernetes Grid (TKG) is opensource VMware Kubernetes product.

Officially in VMware vRealize Automation, we can deploy vSphere Tanzu Kubernetes (TKGS). But currently don’t have the option to deploy TKG open-source Kubernetes clusters using VRA.

Below doc is how We can deploy open-source VMware TKG Clusters using VRA Code stream services.

Using VRA, TKG cluster deployment Flow

You can see in the above diagram how we can provide cluster deployment as a service to end users using VRA.

First, create pipelines in the VRA code stream for deploying, scaling, and deleting the TKG clusters—release pipeline to Catalog. Based on the end-user request pipeline will execute the backend.

EX: – The user requested TKG mgmt Cluster deployment using the VRA service broker catalog.

TKG Mgmt Cluster Deployment pipeline will execute.

S.No

Catalog Item

Comments

1

TKG Mgmt Cluster Deployment

Deploying TKG Clusters required one Bootstrap VM. The TKG Mgmt Cluster pipeline will deploy Bootstrap VM, and the Mgmt cluster will be provisioned on vSphere. We can manage and access TKG Cluster using a bootstrap VM.

2

TKG Workload Cluster Deployment

TKG workload cluster will be provisioned on vSphere

3

TKG Workload Cluster Scale

Workload cluster master or worker nodes will scale based on user request.

4

TKG workload Cluster Deletion

The workload Cluster will be deleted.

5

TKG Mgmt Cluster Deletion

The Mgmt cluster will be deleted.

Requirements for Deploying TKG cluster using VRA: –

  1. DHCP Network with Internet (You can also deploy tkg clusters without the internet, but the following doc was prepared using a network with internet))
  2. TKG OVA File – You can download OVA from VMware Portal.
  3. One Linux template CentsOS/RHEL – My env used RedHat (Pipeline script prepared for Centos/RHEL).
  4. VRA Blueprint

Deployment Steps: –

  1. Preparation Environment
  2. Create Blueprint.
  3. Configuring the VRA Code stream pipeline and adding pipelines to VRA Service Broker.
  4. Deploying TKG Management Cluster.
  5. Deploying TKG Workload Cluster.
  6. Deploying Sample Application on TKG workload cluster using VRA pipeline
  7. Scaling Existing TKG workload cluster
  8. Deleting Workload cluster
  9. Deleting Management Cluster
  10. Preparation Environment: –

vCenter DRS Enabled

DHCP Server configured.

Download TKG OVA from VMware Portal. Deploy OVA as a VM on vCenter, and convert it into the template. Also, Deploy the VM with Centos/RHEL and convert it into the template.

Creating a VRA Blueprint: –

You can download the blueprint code from the following link – https://github.com/Dineshk1205/vratkg/blob/main/Redhat8-1.yaml

Update Blueprint configuration according to your infra.

Configuring the VRA Code stream pipeline and adding pipelines to VRA Service Broker: –

Login to VRA.

Enter the VRA username and password. Sign in.

Open Code Stream Service

Import Pipelines – You can download the pipeline code using the below link and import it (During Import, Change the project name according to your env).

https://github.com/Dineshk1205/vratkg

In Configuration, Select the Variable option and create variables.

vCenter_Password – Your vCenter password

vCenter_Server – Your vCenter Server IP/URL

vCenter_User – vCenter username

RedHatUsername – VM Template username

RedHatpassword – VM Template Password

VSPHER_TLS-THUMBPRINT – vCenter SHA1 Key

Note: – Don’t change variable names.

VSPHERE TLS_THUMBPRINT you can get from vCenter Certificate

Right Click on vCenter Certificate. Click on the View Certificate option.

Enable and Release all pipelines.

Next, Switch to Service Broker services. Add Pipeline Item to Catalog.

Deploying TKG Management Cluster: –

Select TKG Mgmt cluster deployment. Click on request.

Enter all details. Submit.

Once the request is submitted, the bootstrap VM and TKG Management cluster by pipeline will take a few minutes.

You can also check deployment progress in In Code stream execution section.

Successfully mgmt. Cluster deployment is completed.

Log in to Bootstrap VM and check the Tanzu Management Cluster status.

Deploying TKG Workload Cluster: –

In Catalog. Request TKG workload Cluster Deployment. Enter All workload spec details and submit.

It will take a few minutes to provision the workload cluster.

Using Bootstrap VM, access workload cluster.

Deploying Sample Application on TKG workload cluster using VRA pipeline: –

Add Tanzu Workload Cluster as a VRA Endpoint.

Import sample app code using the VRA pipeline

The pipeline contains three stages – creating the metallb namespace, deploying metallb pods, configuring metallb(update configuration based on your infra), and deploying the nginx pod.

You can download a sample application code from the following link – https://github.com/Dineshk1205/vratkg/blob/main/Sample%20Web%20Application.yaml

Run Pipeline

Once Pipeline execution is completed successfully. You can check the nginx service Load Balancer Ip and access the nginx web page using the Load Balancer IP.

Scaling Existing TKG workload cluster: –

In Catalog. Select TKG Workload cluster Scale. Enter Bootstrap IP and Workload cluster Name and machine count to scale up nodes. Submit.

It will take a few minutes to scale nodes.

Deleting Workload cluster: –

In catalog. Request TKG workload Cluster Deletion. Enter Cluster name and Bootstrap Ip. Submit.

You can check in VCenter recent tasks workload cluster was deleted successfully.

Deleting Management Cluster: –

Similarly, if you want to delete the management cluster. In catalog Item. Request TKG Management Cluster Deletion request. Enter Bootstrap Ip and Management cluster name. Submit.