Using VMware Tanzu Mission Control to control AKS RBAC



I have recently had the opportunity to play with VMware Tanzu Mission Control (TMC). This is a platform that allows your manage your Kubernetes clusters both cloud native flavours (AKS, EKS or GKE) or onprem via VMware’s Tanzu Kubernates Grid (TKG). TMC enables platform administrators to centrally manage Access/RBAC Policies, Network Polices, Pod Security Policies and deploy Open Policy Agent (OPA) Policies, eg Image registry control.  For more of a read please check out VMware’s site.

One thing it does lack however and from what I have been told is work in progress is manage RBAC for attached Cloud native Clusters (ie AKS, EKS or GKE). The RBAC only works if you are building TKG Clusters via TMC.  However while testing this recently I found that you can use TMC to manage AKS RBAC by the use of AAD Identities. If you want details on how AAD Integrated RBAC works with AKS check out my last few posts on AAD and AKS here

But today we are going to focus on how to assign RoleBindings on attached AKS Clusters via TMC and how to also do this via TMC CLI. 

TMC allows the option bind access policies on clusters or cluster group levels, it comes by default with a few roles you can use, eg, cluster.edit or cluster.view. Via the portal it is relativity easy. By selecting a cluster you can set a direct access policy and go add new binding define the role and identity. 

Now in the identity field provided your AKS cluster has AAD Integrated RBAC enabled you can assign an AAD object here, if it is a group you need to get the Group ID, which you can do via the az cli command  

az ad group show --group $ADGROUP |jq .objectId -r 

and then putting in your objected as the identity in the TMC Console. 

Once you create the binding you will see in the AKS Cluster it creates a rolebinding as per below: 

$ kubectl describe rolebinding cs-rid-c-5db14e03-f6dd-49bd-bc47-c7b9447f6a02-attached-attached-monty-private-rbac.authorization.k8s.io-ClusterRole-edit 

Name:         cs-rid-c-5db14e03-f6dd-49bd-bc47-c7b9447f6a02-attached-attached-monty-private-rbac.authorization.k8s.io-ClusterRole-edit 

Labels:       tmc.cloud.vmware.com/managed=true 

Annotations:  tmc.cloud.vmware.com/iam-policy-rid: rid:c:5db14e03-f6dd-49bd-bc47-c7b9447f6a02:attached:attached:monty-private 

              tmc.cloud.vmware.com/iam-policy-version: 34911 

Role: 

  Kind:  ClusterRole 

  Name:  edit 

Subjects: 

  Kind   Name                                  Namespace 

  ----   ----                                  --------- 

  Group  be6c4c53-20ec-40de-beb0-15a453038d01 

This role binding is the same type of role binding we did in our AKS RBAC blog we did here. And now we can login via kubectl using the az credentials you get the same process of authenticating via AZ CLI. 

$ az aks get-credentials -n $CLUSTERNAME -g P-montytest-DEV-aue-rg  

The behavior of this command has been altered by the following extension: aks-preview 

A different object named clusterUser_P-montytest-DEV-aue-rg_monty-private already exists in your kubeconfig file. 

Overwrite? (y/n): y 

Merged "monty-private" as current context in /home/admin/.kube/config 
$ kubectl get pods 

To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code HJY3JEHXT to authenticate. 

NAME                    READY   STATUS    RESTARTS   AGE 
nginx-5dddd7654-675bc   1/1     Running   0          17d 

nginx-5dddd7654-bv6r8   1/1     Running   0          17d 

nginx-5dddd7654-q95wx   1/1     Running   0          17d 

Automating RBAC 

Now lets look at how we can use TMC CLI,  AZ CLI and kubectl  to automate the process end-to-end. 

Attaching AKS Cluster to TMC 

Lets first attach our AKS cluster into TMC, in this we will login to TMC via the API token, attach the cluster which will generate a yaml file for us to run in the cluster.  

$ tmc login --name tmc-full-admin -c 
i $TMC_API_TOKEN is set 
√ Successfully created context tmc-full-admin, to manage your contexts run `tmc system context -h` 
$ tmc cluster attach --name $CLUSTERNAME -g $TMCCLUSTERGROUP 
√ cluster "monty-private" created successfully 
i Run `kubectl apply -f k8s-attach-manifest.yaml` to attach the cluster 
$ az aks get-credentials -n $CLUSTERNAME -g P-montytest-DEV-aue-rg --subscription $SUBSCRIPTION --admin 
The behavior of this command has been altered by the following extension: aks-preview 
Merged "monty-private-admin" as current context in /home/admin/.kube/config 
$ kubectl apply -f k8s-attach-manifest.yaml 
namespace/vmware-system-tmc created 
configmap/stack-config created 
secret/tmc-access-secret created 
customresourcedefinition.apiextensions.k8s.io/agents.clusters.tmc.cloud.vmware.com created 
customresourcedefinition.apiextensions.k8s.io/extensionconfigs.intents.tmc.cloud.vmware.com created 
customresourcedefinition.apiextensions.k8s.io/extensionintegrations.clusters.tmc.cloud.vmware.com created 
customresourcedefinition.apiextensions.k8s.io/extensionresourceowners.clusters.tmc.cloud.vmware.com created 
customresourcedefinition.apiextensions.k8s.io/extensions.clusters.tmc.cloud 
service/extension-updater created 
$ tmc cluster namespace attach -c $CLUSTERNAME -n default -k $TMCWORKSPACE 
i using template "attached" 
√ namespace "default" is being created 

Adding the Access Policy in TMC for the Cluster 

We will now get the Azure AD Group Object ID and assign it as a direct Access Policy for the cluster and binding it to the role cluster.edit. 

$ AADGROUPID=$(az ad group show --group $ADGROUP |jq .objectId) 

$ tmc cluster iam add-binding $CLUSTERNAME -g $AADGROUPID -r cluster.edit 

√ role-binding added to cluster IAM policy 

Last thing is to assign the same AAD Group to be added to the Azure IAM role 

$ AKSCLUSTER=$(az aks show -n $CLUSTERNAME -g P-montytest-DEV-aue-rg --subscription $SUBSCRIPTION|jq .id -r) 

The behavior of this command has been altered by the following extension: aks-preview 

$ az role assignment create --role "Azure Kubernetes Service Cluster User Role" --assignee-object-id $AADGROUPID --scope $AKSCLUSTER --assignee-principal-type Group 

The Full Code 

Here is a copy of the full sh script from enabling aad RBAC, attaching cluster to TMC and setting Role Bindings. 

CLUS_NAME=monty-private 
TMCCLUSTERGROUP=development-aks 
TMCWORKSPACE=development-aks-workspace 
ADGROUP=Kube_Developers 
RESOURCEGROUP=P-montytest-DEV-aue-rg 
SUBSCRIPTIONID=*** 

az aks get-credentials -n $CLUS_NAME -g $RESOUREGROUP --SUBSCRIPTIONID $SUBSCRIPTIONID --admin 

echo "LOGINING ON TO TMC" 

export TMC_API_TOKEN=*** 

tmc login --name tmc-full-admin -c 


echo "Attach Cluster to TMC" 

tmc cluster attach --name $CLUS_NAME -g $TMCCLUSTERGROUP 

kubectl apply -f k8s-attach-manifest.yaml 


echo "SETUP RBAC using AAD via TMC" 

AADGROUPID=$(az ad group show --group $ADGROUP |jq .objectId -r) 
tmc cluster iam add-binding $CLUS_NAME -g $AADGROUPID -r cluster.edit 
tmc cluster namespace attach -c $CLUS_NAME -n default -k $TMCWORKSPACE 

echo "ENABLE RBAC FOR GROUP IN AZURE IAM" 

AKSCLUSTER=$(az aks show -n $CLUS_NAME -g $RESOURCEGROUP --SUBSCRIPTIONID $SUBSCRIPTIONID|jq .id -r) 

az role assignment create --role "Azure Kubernetes Service Cluster User Role" --assignee-object-id $AADGROUPID --scope $AKSCLUSTER --assignee-principal-type Group 

rm ./k8s-attach-manifest.yaml