I have recently had the opportunity to play with VMware Tanzu Mission Control (TMC). This is a platform that allows your manage your Kubernetes clusters both cloud native flavours (AKS, EKS or GKE) or onprem via VMware’s Tanzu Kubernates Grid (TKG). TMC enables platform administrators to centrally manage Access/RBAC Policies, Network Polices, Pod Security Policies and deploy Open Policy Agent (OPA) Policies, eg Image registry control. For more of a read please check out VMware’s site.
One thing it does lack however and from what I have been told is work in progress is manage RBAC for attached Cloud native Clusters (ie AKS, EKS or GKE). The RBAC only works if you are building TKG Clusters via TMC. However while testing this recently I found that you can use TMC to manage AKS RBAC by the use of AAD Identities. If you want details on how AAD Integrated RBAC works with AKS check out my last few posts on AAD and AKS here.
But today we are going to focus on how to assign RoleBindings on attached AKS Clusters via TMC and how to also do this via TMC CLI.
TMC allows the option bind access policies on clusters or cluster group levels, it comes by default with a few roles you can use, eg, cluster.edit or cluster.view. Via the portal it is relativity easy. By selecting a cluster you can set a direct access policy and go add new binding define the role and identity.
Now in the identity field provided your AKS cluster has AAD Integrated RBAC enabled you can assign an AAD object here, if it is a group you need to get the Group ID, which you can do via the az cli command
az ad group show --group $ADGROUP |jq .objectId -r
and then putting in your objected as the identity in the TMC Console.
Once you create the binding you will see in the AKS Cluster it creates a rolebinding as per below:
$ kubectl describe rolebinding cs-rid-c-5db14e03-f6dd-49bd-bc47-c7b9447f6a02-attached-attached-monty-private-rbac.authorization.k8s.io-ClusterRole-edit
Name: cs-rid-c-5db14e03-f6dd-49bd-bc47-c7b9447f6a02-attached-attached-monty-private-rbac.authorization.k8s.io-ClusterRole-edit
Labels: tmc.cloud.vmware.com/managed=true
Annotations: tmc.cloud.vmware.com/iam-policy-rid: rid:c:5db14e03-f6dd-49bd-bc47-c7b9447f6a02:attached:attached:monty-private
tmc.cloud.vmware.com/iam-policy-version: 34911
Role:
Kind: ClusterRole
Name: edit
Subjects:
Kind Name Namespace
---- ---- ---------
Group be6c4c53-20ec-40de-beb0-15a453038d01 This role binding is the same type of role binding we did in our AKS RBAC blog we did here. And now we can login via kubectl using the az credentials you get the same process of authenticating via AZ CLI.
$ az aks get-credentials -n $CLUSTERNAME -g P-montytest-DEV-aue-rg
The behavior of this command has been altered by the following extension: aks-preview
A different object named clusterUser_P-montytest-DEV-aue-rg_monty-private already exists in your kubeconfig file.
Overwrite? (y/n): y
Merged "monty-private" as current context in /home/admin/.kube/config
$ kubectl get pods
To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code HJY3JEHXT to authenticate.
NAME READY STATUS RESTARTS AGE
nginx-5dddd7654-675bc 1/1 Running 0 17d
nginx-5dddd7654-bv6r8 1/1 Running 0 17d
nginx-5dddd7654-q95wx 1/1 Running 0 17d Automating RBAC
Now lets look at how we can use TMC CLI, AZ CLI and kubectl to automate the process end-to-end.
Attaching AKS Cluster to TMC
Lets first attach our AKS cluster into TMC, in this we will login to TMC via the API token, attach the cluster which will generate a yaml file for us to run in the cluster.
$ tmc login --name tmc-full-admin -c
i $TMC_API_TOKEN is set
√ Successfully created context tmc-full-admin, to manage your contexts run `tmc system context -h`
$ tmc cluster attach --name $CLUSTERNAME -g $TMCCLUSTERGROUP
√ cluster "monty-private" created successfully
i Run `kubectl apply -f k8s-attach-manifest.yaml` to attach the cluster
$ az aks get-credentials -n $CLUSTERNAME -g P-montytest-DEV-aue-rg --subscription $SUBSCRIPTION --admin
The behavior of this command has been altered by the following extension: aks-preview
Merged "monty-private-admin" as current context in /home/admin/.kube/config
$ kubectl apply -f k8s-attach-manifest.yaml
namespace/vmware-system-tmc created
configmap/stack-config created
secret/tmc-access-secret created
customresourcedefinition.apiextensions.k8s.io/agents.clusters.tmc.cloud.vmware.com created
customresourcedefinition.apiextensions.k8s.io/extensionconfigs.intents.tmc.cloud.vmware.com created
customresourcedefinition.apiextensions.k8s.io/extensionintegrations.clusters.tmc.cloud.vmware.com created
customresourcedefinition.apiextensions.k8s.io/extensionresourceowners.clusters.tmc.cloud.vmware.com created
customresourcedefinition.apiextensions.k8s.io/extensions.clusters.tmc.cloud
service/extension-updater created
$ tmc cluster namespace attach -c $CLUSTERNAME -n default -k $TMCWORKSPACE
i using template "attached"
√ namespace "default" is being created Adding the Access Policy in TMC for the Cluster
We will now get the Azure AD Group Object ID and assign it as a direct Access Policy for the cluster and binding it to the role cluster.edit.
$ AADGROUPID=$(az ad group show --group $ADGROUP |jq .objectId)
$ tmc cluster iam add-binding $CLUSTERNAME -g $AADGROUPID -r cluster.edit
√ role-binding added to cluster IAM policy
Last thing is to assign the same AAD Group to be added to the Azure IAM role
$ AKSCLUSTER=$(az aks show -n $CLUSTERNAME -g P-montytest-DEV-aue-rg --subscription $SUBSCRIPTION|jq .id -r)
The behavior of this command has been altered by the following extension: aks-preview
$ az role assignment create --role "Azure Kubernetes Service Cluster User Role" --assignee-object-id $AADGROUPID --scope $AKSCLUSTER --assignee-principal-type Group The Full Code
Here is a copy of the full sh script from enabling aad RBAC, attaching cluster to TMC and setting Role Bindings.
CLUS_NAME=monty-private
TMCCLUSTERGROUP=development-aks
TMCWORKSPACE=development-aks-workspace
ADGROUP=Kube_Developers
RESOURCEGROUP=P-montytest-DEV-aue-rg
SUBSCRIPTIONID=***
az aks get-credentials -n $CLUS_NAME -g $RESOUREGROUP --SUBSCRIPTIONID $SUBSCRIPTIONID --admin
echo "LOGINING ON TO TMC"
export TMC_API_TOKEN=***
tmc login --name tmc-full-admin -c
echo "Attach Cluster to TMC"
tmc cluster attach --name $CLUS_NAME -g $TMCCLUSTERGROUP
kubectl apply -f k8s-attach-manifest.yaml
echo "SETUP RBAC using AAD via TMC"
AADGROUPID=$(az ad group show --group $ADGROUP |jq .objectId -r)
tmc cluster iam add-binding $CLUS_NAME -g $AADGROUPID -r cluster.edit
tmc cluster namespace attach -c $CLUS_NAME -n default -k $TMCWORKSPACE
echo "ENABLE RBAC FOR GROUP IN AZURE IAM"
AKSCLUSTER=$(az aks show -n $CLUS_NAME -g $RESOURCEGROUP --SUBSCRIPTIONID $SUBSCRIPTIONID|jq .id -r)
az role assignment create --role "Azure Kubernetes Service Cluster User Role" --assignee-object-id $AADGROUPID --scope $AKSCLUSTER --assignee-principal-type Group
rm ./k8s-attach-manifest.yaml 
