# Setting Up Access Controls for Kubernetes The Teleport Kubernetes Service is a proxy that sits between Kubernetes users and one or more Kubernetes clusters. In this guide, we will use a local Kubernetes cluster to show you how to configure Teleport's role-based access control (RBAC) system to manage access to Kubernetes clusters, groups, users, and resources. ## How it works When a user authenticates to Teleport, they receive a kubeconfig that lets them send requests to their authorized Kubernetes clusters via the Teleport Kubernetes Service. The Kubernetes Service can then inspect, modify, or disallow these requests depending on the privileges you have assigned to the Teleport user via their roles. ## Prerequisites - A running Teleport cluster. If you want to get started with Teleport, [sign up](https://goteleport.com/signup) for a free trial or [set up a demo environment](https://goteleport.com/docs/get-started/deploy-community.md). - The `tctl` and `tsh` clients. Installing `tctl` and `tsh` clients 1. Determine the version of your Teleport cluster. The `tctl` and `tsh` clients must be at most one major version behind your Teleport cluster version. Send a GET request to the Proxy Service at `/v1/webapi/find` and use a JSON query tool to obtain your cluster version. Replace teleport.example.com:443 with the web address of your Teleport Proxy Service: ``` $ TELEPORT_DOMAIN=teleport.example.com:443 $ TELEPORT_VERSION="$(curl -s https://$TELEPORT_DOMAIN/v1/webapi/find | jq -r '.server_version')" ``` 2. Follow the instructions for your platform to install `tctl` and `tsh` clients: **Mac** Download the signed macOS .pkg installer for Teleport, which includes the `tctl` and `tsh` clients: ``` $ curl -O https://cdn.teleport.dev/teleport-${TELEPORT_VERSION?}.pkg ``` In Finder double-click the `pkg` file to begin installation. --- DANGER Using Homebrew to install Teleport is not supported. The Teleport package in Homebrew is not maintained by Teleport and we can't guarantee its reliability or security. --- **Windows - Powershell** ``` $ curl.exe -O https://cdn.teleport.dev/teleport-v${TELEPORT_VERSION?}-windows-amd64-bin.zip Unzip the archive and move the `tctl` and `tsh` clients to your %PATH% NOTE: Do not place the `tctl` and `tsh` clients in the System32 directory, as this can cause issues when using WinSCP. Use %SystemRoot% (C:\Windows) or %USERPROFILE% (C:\Users\) instead. ``` **Linux** All of the Teleport binaries in Linux installations include the `tctl` and `tsh` clients. For more options (including RPM/DEB packages and downloads for i386/ARM/ARM64) see our [installation page](https://goteleport.com/docs/installation.md). ``` $ curl -O https://cdn.teleport.dev/teleport-v${TELEPORT_VERSION?}-linux-amd64-bin.tar.gz $ tar -xzf teleport-v${TELEPORT_VERSION?}-linux-amd64-bin.tar.gz $ cd teleport $ sudo ./install Teleport binaries have been copied to /usr/local/bin ``` * To check that you can connect to your Teleport cluster, sign in with `tsh login`, then verify that you can run `tctl` commands using your current credentials. For example, run the following command, assigning teleport.example.com to the domain name of the Teleport Proxy Service in your cluster and email\@example.com to your Teleport username: ``` $ tsh login --proxy=teleport.example.com --user=email@example.com $ tctl status Cluster teleport.example.com Version 19.0.0-dev CA pin sha256:abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678 ``` If you can connect to the cluster and run the `tctl status` command, you can use your current credentials to run subsequent `tctl` commands from your workstation. If you host your own Teleport cluster, you can also run `tctl` commands on the computer that hosts the Teleport Auth Service for full permissions. To run the local demo environment, ensure that you have the following tools installed on your workstation: | Tool | Purpose | Installation link | | -------- | -------------------------------- | ------------------------------------------------------------- | | minikube | Local Kubernetes deployment tool | [Install minikube](https://minikube.sigs.k8s.io/docs/start/) | | Helm | Kubernetes package manager | [Install Helm](https://helm.sh/docs/intro/install/) | | kubectl | Kubernetes admin CLI | [Install kubectl](https://kubernetes.io/docs/tasks/tools/) | | Docker | Required minikube driver | [Get Started With Docker](https://www.docker.com/get-started) | ## Step 1/3. Prepare Kubernetes resources ### Start minikube Start minikube with the Docker driver: ``` $ minikube start --driver=docker ``` This command should start a local Kubernetes cluster and set your context to `minikube`. To verify this, run the following command: ``` $ kubectl config current-context minikube ``` ### Deploy demo pods On your workstation, create a manifest file called `pods.yaml` with the following content: ``` apiVersion: v1 kind: Namespace metadata: name: development labels: name: development --- apiVersion: v1 kind: Namespace metadata: name: production labels: name: production --- apiVersion: apps/v1 kind: Deployment metadata: name: webapp namespace: development spec: selector: matchLabels: app: nginx-webapp template: metadata: labels: app: nginx-webapp spec: containers: - name: nginx image: nginx:1.23 --- apiVersion: apps/v1 kind: Deployment metadata: name: webapp namespace: production spec: selector: matchLabels: app: nginx-webapp template: metadata: labels: app: nginx-webapp spec: containers: - name: nginx image: nginx:1.23 --- apiVersion: apps/v1 kind: Deployment metadata: name: loadbalancer namespace: development spec: selector: matchLabels: app: nginx-loadbalancer template: metadata: labels: app: nginx-loadbalancer spec: containers: - name: nginx image: nginx:1.23 --- apiVersion: apps/v1 kind: Deployment metadata: name: loadbalancer namespace: production spec: selector: matchLabels: app: nginx-loadbalancer template: metadata: labels: app: nginx-loadbalancer spec: containers: - name: nginx image: nginx:1.23 ``` This manifest creates two namespaces, `development` and `production`, and deploys two `nginx` pods into each one: `webapp` and `loadbalancer`. Apply the new resources: ``` $ kubectl apply -f pods.yaml ``` Ensure that the resources are deployed: ``` $ kubectl -n development get pods $ kubectl -n production get pods ``` You should see both the `loadbalancer` and `webapp` pods in each namespace. ### Install Kubernetes RBAC resources Now that we have deployed our `webapp` and `loadbalancer` pods in our `development` and `production` namespaces, we will create a Kubernetes role that can view all pods in all namespaces. Later in this guide, we will define a Teleport role that further restricts the access Teleport users can have to resources in your cluster. Create a manifest file called `k8s-rbac.yaml` with the following content: ``` apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: pod-viewer rules: - apiGroups: [""] resources: ["pods"] verbs: ["get", "watch", "list"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: pod-viewer subjects: - kind: Group name: developers apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole name: pod-viewer apiGroup: rbac.authorization.k8s.io ``` Apply your changes: ``` $ kubectl apply -f k8s-rbac.yaml ``` ### Install the Teleport Kubernetes Service Now that you have some workloads running on Kubernetes and RBAC resources to manage access to them, install the Teleport Kubernetes Service in your demo cluster so you can get more control into the resources that Kubernetes users can access. Configure Helm to fetch Teleport charts from the Teleport Helm repository: ``` $ helm repo add teleport https://charts.releases.teleport.dev ``` Refresh the local Helm cache by fetching the latest charts: ``` $ helm repo update ``` Request a token that the Kubernetes Service will use to join your Teleport cluster: ``` $ tctl tokens add --type=kube,app,discovery --ttl=1h --format=text ``` Copy this token so you can use it when running the Teleport Kubernetes Service. Install the Teleport Kubernetes Service in your cluster, assigning proxy-address to the host **and port** of your Teleport Proxy Service (e.g., `mytenant.teleport.sh:443`) and token to the token you requested earlier: ``` $ helm install teleport-agent teleport/teleport-kube-agent \ --set kubeClusterName=minikube \ --set roles="kube\,app\,discovery" \ --set proxyAddr=proxy-address \ --set authToken=token \ --set labels.region=local --set labels.platform=minikube \ --create-namespace \ --namespace=teleport-agent \ --version 19.0.0-dev ``` This `helm install` command supplies the soon-to-be-added Kubernetes Service instance with two labels: `region:local` and `platform:minikube`. We will use these to configure access controls for the cluster later in this guide. Verify that the `teleport` pod is running in your cluster: ``` $ kubectl -n teleport-agent get pods ``` You can check that the Teleport Kubernetes Service registered itself with your Teleport cluster by executing the following command: ``` $ tctl get kube_servers ``` The output should resemble the following: ``` kind: kube_server metadata: expires: "2023-01-24T16:20:00.571214635Z" id: 0000000000000000000 name: minikube spec: cluster: kind: kube_cluster metadata: labels: platform: minikube region: local name: minikube spec: aws: {} azure: {} gcp: {} version: v3 host_id: 00000000-0000-0000-0000-000000000000 hostname: remote.kube.proxy.teleport.cluster.local rotation: current_id: "" last_rotated: "0001-01-01T00:00:00Z" schedule: standby: "0001-01-01T00:00:00Z" update_clients: "0001-01-01T00:00:00Z" update_servers: "0001-01-01T00:00:00Z" started: "0001-01-01T00:00:00Z" version: 19.0.0-dev version: v3 ``` ## Step 2/3. Define a Teleport role The Teleport Kubernetes Service determines how to proxy a Teleport user's requests to a Kubernetes API server by looking up the user's roles. Based on this information, the Kubernetes Service accepts or denies the request. For valid requests, the Kubernetes Service rewrites the request headers to impersonate the Teleport user's desired Kubernetes user and groups, and forwards the request to the API server. In this section, we will define a Teleport role that: - Authenticates the user to a Kubernetes cluster as a member of the `developers` group. In the previous section, we authorized members of this group to view pods in all namespaces. - Enables the user to access `webapp` pods in the `production` namespace and all pods in the `development` namespace. - Denies the user access to all other pods. ### Define a role Create a file called `kube-access.yaml` with the following content: ``` kind: role metadata: name: kube-access version: v7 spec: allow: kubernetes_labels: 'region': '*' 'platform': 'minikube' kubernetes_resources: - kind: pod namespace: "production" name: "^webapp-[a-z0-9-]+$" - kind: pod namespace: "development" name: "*" kubernetes_groups: - developers kubernetes_users: - minikube deny: {} ``` In this role, we have defined the following `allow` rules: - `kubernetes_labels`: Allows access to Kubernetes clusters in all regions, but only with the `platform:minikube` label. - `kubernetes_resources`: Allows access to pods in the `webapp` deployment in the `production` namespace and all pods in the `development` namespace. Note the use of a regular expression (beginning `^` and ending in `$`) to match pod names that Kubernetes generates automatically. - `kubernetes_groups`: Authenticates the user to your Kubernetes cluster as a member of the Kubernetes group `developers`, which we associated with the `pod-viewer` Kubernetes `Role` earlier in this guide. - `kubernetes_users`: Authenticates the user to your Kubernetes cluster as the default `minikube` user. ### Create the role Once you have finished configuring the `kube-access` role, create it using the following command: ``` $ tctl create kube-access.yaml ``` --- TIP You can also create and edit roles using the Web UI. Go to **Access -> Roles** and click **Create New Role** or pick an existing role to edit. --- Assign the `kube-access` role to your Teleport user by running the appropriate commands for your authentication provider: **Local User** 1. Retrieve your local user's roles as a comma-separated list: ``` $ ROLES=$(tsh status -f json | jq -r '.active.roles | join(",")') ``` 2. Edit your local user to add the new role: ``` $ tctl users update $(tsh status -f json | jq -r '.active.username') \ --set-roles "${ROLES?},kube-access" ``` 3. Sign out of the Teleport cluster and sign in again to assume the new role. **GitHub** 1. Open your `github` authentication connector in a text editor: ``` $ tctl edit github/github ``` 2. Edit the `github` connector, adding `kube-access` to the `teams_to_roles` section. The team you should map to this role depends on how you have designed your organization's role-based access controls (RBAC). However, the team must include your user account and should be the smallest team possible within your organization. Here is an example: ``` teams_to_roles: - organization: octocats team: admins roles: - access + - kube-access ``` 3. Apply your changes by saving and closing the file in your editor. 4. Sign out of the Teleport cluster and sign in again to assume the new role. **SAML** 1. Retrieve your `saml` configuration resource: ``` $ tctl get --with-secrets saml/mysaml > saml.yaml ``` Note that the `--with-secrets` flag adds the value of `spec.signing_key_pair.private_key` to the `saml.yaml` file. Because this key contains a sensitive value, you should remove the saml.yaml file immediately after updating the resource. 2. Edit `saml.yaml`, adding `kube-access` to the `attributes_to_roles` section. The attribute you should map to this role depends on how you have designed your organization's role-based access controls (RBAC). However, the group must include your user account and should be the smallest group possible within your organization. Here is an example: ``` attributes_to_roles: - name: "groups" value: "my-group" roles: - access + - kube-access ``` 3. Apply your changes: ``` $ tctl create -f saml.yaml ``` 4. Sign out of the Teleport cluster and sign in again to assume the new role. **OIDC** 1. Retrieve your `oidc` configuration resource: ``` $ tctl get oidc/myoidc --with-secrets > oidc.yaml ``` Note that the `--with-secrets` flag adds the value of `spec.signing_key_pair.private_key` to the `oidc.yaml` file. Because this key contains a sensitive value, you should remove the oidc.yaml file immediately after updating the resource. 2. Edit `oidc.yaml`, adding `kube-access` to the `claims_to_roles` section. The claim you should map to this role depends on how you have designed your organization's role-based access controls (RBAC). However, the group must include your user account and should be the smallest group possible within your organization. Here is an example: ``` claims_to_roles: - name: "groups" value: "my-group" roles: - access + - kube-access ``` 3. Apply your changes: ``` $ tctl create -f oidc.yaml ``` 4. Sign out of the Teleport cluster and sign in again to assume the new role. ## Step 3/3. Access resources At this point, you have configured the Teleport Kubernetes Service to give your Teleport user access to the `webapp` pod in the `production` namespace. In this step, we will authenticate to your Kubernetes cluster via Teleport and test our new access controls. List the Kubernetes clusters you can access via Teleport: ``` $ tsh kube ls ``` You should see the `minikube` cluster you registered earlier: ``` Kube Cluster Name Labels Selected ----------------- ------------------------------ -------- minikube platform=minikube region=local ``` To access your Kubernetes cluster via Teleport, authenticate to it and update your kubeconfig: ``` $ tsh kube login minikube ``` When listing pods in all namespaces, the Teleport Kubernetes Service will filter the pods it retrieves to show only those that your Teleport user can access. Run the following command: ``` $ kubectl get pods --all-namespaces ``` The output will show the `webapp` pod in the `production` namespace and both the `webapp` and `loadbalancer` pods in the `development` namespace: ``` NAMESPACE NAME READY STATUS RESTARTS AGE development loadbalancer-000000000-00000 1/1 Running 0 36m development webapp-0000000000-00000 1/1 Running 0 36m production webapp-0000000000-00000 1/1 Running 0 36m ``` You can access information about the `webapp` pod in the `production` namespace: ``` $ kubectl -n production get pods/webapp-0000000000-00000 -o json ``` Also note that the `kube-access` role we created earlier mapped your Teleport user to the `developers` Kubernetes group, which has permissions only to view pods: ``` $ kubectl auth can-i create pods no ``` By configuring Teleport roles and Kubernetes RBAC resources, you can fine-tune the access that users in your organization have to your Kubernetes-based infrastructure. When you authenticated to your `minikube` cluster via `tsh kube login`, Teleport generated a kubeconfig that connects to your cluster via Teleport: ``` $ kubectl config current-context teleport.example.com-minikube ``` If you want to regain full control of your `minikube` cluster, you can use the default `minikube` context instead: ``` $ kubectl config use-context minikube ``` ## Next steps For more detailed information on how Teleport RBAC for Kubernetes works, consult the Kubernetes [Access Controls Guide](https://goteleport.com/docs/enroll-resources/kubernetes-access/controls.md). You can leave your `minikube` cluster running so you can try out different Teleport and Kubernetes RBAC configurations. Now that you know how to configure Teleport's RBAC system to control access to Kubernetes clusters, learn how to set up [Resource Access Requests](https://goteleport.com/docs/identity-governance/access-requests/resource-requests.md) for just-in-time access and [Access Request plugins](https://goteleport.com/docs/identity-governance/access-requests/plugins.md) so you can manage access with your communication workflow of choice.