Skip to main content

Teleport Kubernetes Access Controls

Report an issue with this page

This guide explains the way the Teleport Kubernetes Service applies role-based access controls when a Teleport user interacts with a Kubernetes cluster. The Kubernetes Service intercepts requests to a Kubernetes API server and modifies each request depending on the user's Teleport roles.

In this guide, we will show you how to configure the fields available in a Teleport role to manage access to Kubernetes clusters you have connected to Teleport.

For an example of how to use Teleport roles to manage access to Kubernetes with a local minikube cluster, see our RBAC how-to guide.

Role fields for managing Kubernetes access​

In this section, we will explain the fields within a Teleport role that configure access to Kubernetes clusters.

To manage access to Kubernetes clusters, a Teleport role must include the following fields in the spec.allow section:

Here is an example of a Teleport role that restricts access to Kubernetes clusters:

kind: role
metadata:
  name: kube-access
version: v8
spec:
  allow:
    kubernetes_labels:
      region: '*'
      platform: minikube
    kubernetes_resources:
      - kind: pods
        namespace: production
        name: '^webapp-[a-z0-9-]+$'
        api_group: ''
      - kind: pods
        namespace: development
        name: '*'
        api_group: ''
      - kind: deployments
        namespace: development
        name: '*'
        api_group: apps
    kubernetes_groups:
    - developers
    kubernetes_users:
    - minikube
  deny: {}

kubernetes_labels​

You can add labels to a Kubernetes cluster when you register it with Teleport. You can restrict a user's access to Kubernetes clusters with different labels using a role's kubernetes_labels field.

kind: role
metadata:
  name: kube-access
version: v8
spec:
  allow:
    kubernetes_labels:
      region: '*'
      environment: development
      # ...
  deny: {}

The value of the kubernetes_labels field is a mapping from label keys to one or more label values (i.e., either a string or a list).

How the Kubernetes Service evaluates kubernetes_labels​

If both the key and the value of a label are wildcards, *, the Teleport Kubernetes Service allows the user to access Kubernetes clusters with all tags:

spec:
  allow:
    kubernetes_labels:
      '*': '*'
    # ...

Otherwise, the Kubernetes Service checks whether all of the keys in kubernetes_labels match the keys corresponding to a registered Kubernetes cluster. If they do not, there is no matching Kubernetes cluster, and the Kubernetes Service denies the request.

For example, a cluster with labels that include the environment key but not the region key would not match the kubernetes_labels field in the kube-access role above.

The Kubernetes Service then retrieves the values of the Kubernetes cluster labels with the keys in kubernetes_labels. The value of each key in kubernetes_labels must match the value of a Kubernetes cluster's label before the Kubernetes Service lets a user access the cluster.

For example, the kube-access role above allows a user to access Kubernetes clusters with the region key and any value. It restricts the user to Kubernetes clusters with the environment key and the development value. We will explain valid values of keys within kubernetes_labels in the next section.

Label values​

For the key of a label in kubernetes_labels to match the key of a Kubernetes cluster, the match must be exact. For values, however, you can configure regular expressions, wildcards, and multiple values to provide flexibility.

Regular expressions and wildcards​

You can use regular expressions or wildcard characters to match subsets or variations of a string. If a value begins with ^ and ends in $, the Kubernetes Service will treat it as a regular expression using Go's re2 syntax (see the re2 README).

Otherwise, the Kubernetes Service evaluates wildcards within the value, matching them to any sequence of characters in a label.

Here is an example:

spec:
  allow:
    kubernetes_labels:
      region: 'us-east-*'
      team: '^data-eng-[a-z-]+$'
    # ...

This allow rule matches clusters with the labels region:us-east-1 and region:us-east-2b. It also matches clusters with the labels team:data-eng-analytics and team:data-eng-ml-training.

Multiple values​

If a key in kubernetes_labels has multiple values, the Kubernetes Service will consider the label values a match if any of these values match a Kubernetes cluster's labels. For example, this kubernetes_labels configuration matches clusters with the region:us-east-2 label and either the development or staging environments:

spec:
  allow:
    kubernetes_labels:
      region: 'us-east-*'
      environment: ['development', 'staging']
    # ...

Applying labels​

You can apply labels to an instance of the Teleport Kubernetes Service. The way to do this depends on how you have launched the service:

Set labels when installing or upgrading the teleport-kube-agent Helm chart, e.g.:

helm upgrade teleport-agent teleport-kube-agent --set kubeClusterName={CLUSTER?}\  --set proxyAddr=${PROXY?} --set authToken=${TOKEN?} --create-namespace --namespace=teleport-agent\  --set labels.env=prod --set labels.region=us-west-1

kubernetes_groups and kubernetes_users​

The Teleport Kubernetes Service receives requests from end users, e.g., via kubectl, and forwards them to a Kubernetes API server. The Kubernetes Service uses impersonation headers to send requests to the API server with one Kubernetes user and zero or more Kubernetes groups.

Impersonation

The kubernetes_users and kubernetes_groups fields indicate which users and groups to allow a user to assume when they send requests to a Kubernetes API server:

kind: role
metadata:
  name: kube-access
version: v7
spec:
  allow:
    kubernetes_groups:
    - developers
    - viewers
    kubernetes_users:
    - myuser
    - system:serviceaccount:someNamespace:saName # Service account name
    # ...
  deny: {}

The value of kubernetes_groups and kubernetes_users is a list of names of groups, users, or service accounts to enable impersonation for.

Kubernetes Users and Groups​

Kubernetes Users and Groups are entities that exist in the Kubernetes Cluster and for which permissions are controlled through ClusterRoleBinding or RoleBinding resources. If they do not exist in the cluster, Kubernetes RBAC will ignore them.

Here's an example of a ClusterRoleBinding resource that assigns the built-in view ClusterRole to a Group cluster-viewer-group and to a User cluster-viewer-user:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: cluster-viewer
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: view
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: cluster-viewer-group
- apiGroup: rbac.authorization.k8s.io
  kind: User
  name: cluster-viewer-user

At this point, the User and the Group have the same permissions to view resources in Kubernetes Namespaces. It's not mandatory to assign the same permissions to Kubernetes Users and Groups as Kubernetes merges the permissions associated with the impersonation principals used in the request.

Kubernetes Service Accounts​

Teleport supports Service Account impersonation by using the fully-qualified name of the Service Account in the kubernetes_users field.

The Service Account's fully-qualified name consists of the following pattern:

system:serviceaccount:<namespace>:<service_account_name>

The FQN must be prefixed with system:serviceaccount:, otherwise Kubernetes will evaluate it as a normal User.

An example of a role that impersonates a Service Account can be found below.

kind: role
metadata:
  name: kube-access-impersonate-sa
version: v7
spec:
  allow:
    kubernetes_users:
    - system:serviceaccount:someNamespace:saName
    # ...
  deny: {}

How Teleport users impersonate Kubernetes Users, Groups and Service Accounts​

There are two ways for an end user to specify which user or service account and groups to impersonate:

Manually​

When a user runs tsh kube login to authenticate to a Kubernetes cluster, they can use the --as and --as-groups flags to manually specify the user and groups to authenticate as. The Teleport Kubernetes Service determines whether the user and groups belong to a user's kubernetes_users and kubernetes_groups configuration and, if not, denies the user access.

Automatically​

If the user has not explicitly determined a Kubernetes user and Kubernetes groups when authenticating to a cluster, the Teleport Kubernetes Service determines this from the kubernetes_users and kubernetes_groups fields in a user's roles.

If a user has exactly one value in kubernetes_users, the Teleport Kubernetes Service impersonates that user. If there are no values or a wildcard (*) in kubernetes_users, the Kubernetes Service uses the user's Teleport username.

The Kubernetes Service will deny a request if a user has multiple kubernetes_users and has not specified one when authenticating to a cluster (i.e., using the --as flag described in the previous section).

If the user has not specified a Kubernetes group to impersonate, the Kubernetes Service uses all values within kubernetes_groups.

warning

When impersonating a less privileged user, remember that unless you're also manually impersonating specific groups (e.g. using --as-groups flag), the Kubernetes Service will automatically impersonate any groups within kubernetes_groups.

This can be confusing because you will have the combined permissions of both the user and any automatically-impersonated groups.

With the kube-access role above, after you authenticate to Teleport, the Kubernetes Service uses impersonation headers to forward requests to the API server with the developers group and the myuser Kubernetes user.

Enabling impersonation​

To enable the Kubernetes Service to forward user requests with impersonation headers, you must ensure that its service account has permissions to impersonate Kubernetes RBAC principals within your cluster. The Kubernetes Service denies requests to impersonate any user or group that a user does not have access to.

Below is a Kubernetes ClusterRole that grants the minimum set of permissions to enable impersonation, and a ClusterRoleBinding that grants these permissions to a service account.

tip

There is usually no need to define these resources manually. The manual methods and automatic methods for registering Kubernetes clusters with Teleport include steps for setting up the Kubernetes RBAC resources that Teleport needs to allow access to clusters.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: teleport-impersonation
rules:
- apiGroups:
  - ""
  resources:
  - users
  - groups
  - serviceaccounts
  verbs:
  - impersonate
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - get
- apiGroups:
  - "authorization.k8s.io"
  resources:
  - selfsubjectaccessreviews
  verbs:
  - create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: teleport
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: teleport-impersonation
subjects:
- kind: ServiceAccount
  name: teleport-serviceaccount
  namespace: default

Specifying groups and users based on user traits​

You can specify Kubernetes groups and users for each Teleport user individually, rather than hardcoding this information into your Teleport roles. To do so, you can add template variables to Teleport your roles, and the Teleport Auth Service will populate them with information from each authenticating user.

For more information on how template variable expansion works in Teleport roles, see the Access Controls Reference.

Single Sign-On provider traits​

Teleport's roles map OIDC claims or SAML attributes using template variables. The Teleport Auth Service will substitute any template variable in the format {{external.*}} with the corresponding SAML attribute or OIDC claim:

kind: role
version: v7
metadata:
  name: group-member
spec:
  allow:
    kubernetes_groups: ["{{external.groups}}"]
    kubernetes_users: ["{{external.kube_username}}"]
    # ...

If a user authenticates to Teleport via a SAML connector, for example, and the user has a kube_username attribute with the value myuser and a groups attribute with values developers and viewers, the group-member role above will evaluate to the following:

kind: role
version: v7
metadata:
  name: group-member
spec:
  allow:
    kubernetes_groups: ["developers", "viewers"]
    kubernetes_users: ["myuser"]
    # ...
Local user traits​

For local users, you can specify arbitrary key-value data in the spec.traits field of a user resource, then use the {{external.*}} template variable in a role to refer to those traits.

For example, this role fills in kubernetes_users and kubernetes_groups with internal traits:

kind: role
version: v7
metadata:
  name: group-member
spec:
  allow:
    kubernetes_groups: ["{{external.groups}}"]
    kubernetes_users: ["{{external.kube_username}}"]
    # ...

You can then supply the values for these template variables when creating or modifying a local user. For example, this user definition includes traits that the Auth Service will use to populate the role definition above:

kind: user
version: v2
metadata:
  name: alice
spec:
  roles:
    - group-member
  traits:
    groups:
      - developers
      - viewers
    kube_username:
      - myuser

Configuring just-in-time access​

If you are setting up a Teleport role to enable just-in-time access to a specific Kubernetes resources, you should set the role's kubernetes_groups and kubernetes_users to a principal with a role that has no access to Kubernetes resource beside the Kubernetes resources that Teleport is able to restrict access for.

This is because, if a user requests access to a Kubernetes pod, and the request is approved, the Teleport Kubernetes Service will use the kubernetes_groups and kubernetes_users fields in the role to add impersonation headers to the user's requests to a Kubernetes API server. Under these conditions, Teleport will be able to restrict access to all supported Kubernetes resources kinds except for the desired pod.

Teleport is also able to restrict access to namespaced-scoped custom resources but not cluster-scoped custom resources. CRDs resources that are cluster scoped will be accessible to the user if the principals in the kubernetes_users and kubernetes_groups fields have access to them.

Requesting access to a Kubernetes Namespace allows you to access all resources in that namespace but you won't be able to access any other supported resources in the cluster.

kubernetes_resources​

The kubernetes_resources field enables a Teleport role to configure access to specific resources in a Kubernetes cluster.

The value of this field is a list of mappings, where each mapping is described as follows:

Role V8​

warning

Role V8 added support for managing access to any kubernetes resource kind, including custom resource definitions (CRDs). To do so it has multiple changes to the way the kubernetes_resources section is handled compared to prior role versions:

  • the kind field must always specify the plural form of the resource kind
  • the api_group field must be set for resources not found in the core API group
  • kind: namespaces now matches namespace resources, it no longer matches resources within the namespace
  • the namespace field when set and with a value of anything but * will not match cluster-wide resources.

As this behavior differs from earlier role versions, if you migrate a role to V8 from a previous version you will likely need to adjust the kubernetes_resources section.

kind: role
metadata:
  name: kube-access
version: v8
spec:
  allow:
    kubernetes_labels:
      "*": "*"
    kubernetes_resources:
      - kind: pods
        api_group: ""
        namespace: production
        name: webapp
        verbs: ["*"]
      - kind: deployments
        api_group: apps
        namespace: development
        name: "*"
    # ...

  • kind: The kind of resource to enable access to. It can be * or the plural form of the kind (e.g. pods, deployments, cronjobs, mycustomresources). If the resource has a group, it must be specified in the api_group field, for example, while pods doesn't need a group, deployments require the api_group field be set to apps.

    tip

    The full list of available resources, along with their api group can be found by running the following command:

    kubectl api-resources --namespaced=true -o=name
kubectl api-resources --namespaced=false -o=name
    • If the line has a ., then the kind is the first element before the . and the api group is everything after the first ..
    • If the line doesn't have a ., then the kind is the full element and there is no api group.
    warning
    • The namespaces kind doesn't include all the namespaced resources, it only covers the namespace itself. This behavior is different from earlier role versions where it covered the resource itself and everything in the namespace, which means that if you migrate an existing role to V8 from a prior version you will likely need to adjust the kubernetes_resources section.
    • The kind, including * now enforces the namespace field. To match a cluster-wide resource, the namespace field must be empty or *. This behavior is different from earlier role versions where it included cluster-wide resources regardless of the namespace field.

  • api_group: The API group of the resource. In kube-access, in the example above, the pods resource being in a core resource has "" for api_group, the value is set to apps for the deployments resource kind. A wildcard can be used to match any API groups.

  • namespace: The Kubernetes namespace in which to allow access to a resource. Must be empty or * for cluster-wide resources. Any other value, including other wildcards, will only match namespaced resources. In the kube-access role, we are allowing access to a pod in the production namespace.

    tip

    Note that when using the wildcard *, it will match any resources, including cluster-wide ones (based on kind/api_group). Using empty string "" will match only cluster-wide resources. Any other value, like ^.+$ will only match namespaced resources and exclude cluster-wide resources.

  • name: The name of the pod to allow access to. In kube-access, this is the webapp pod.

  • verbs: The operations to allow on the resource. Currently, Teleport supports:

    VerbGrants access to
    *All operations
    getRead a resource
    listList resources
    createCreate a resource
    updateUpdate a resource
    patchPatch a resource
    deleteDelete a resource
    deletecollectionDelete a collection of resources
    watchWatch resources
    portforwardCreate portforward requests for Pods
    execExecute commands in Pods

For the namespace, name and api_group fields, you can add a wildcard character (*) to replace any sequence of characters. For example, name: "pod-*-*" matches pods named pod-1-a and pod-2-c. As with kubernetes_labels, if a value begins with ^ and ends in $, the Kubernetes Service will treat it as a regular expression using Go's re2 syntax (see the re2 README).

tip

For a user to access a pod named in a role's kubernetes_resources field, the user must be assigned a Teleport role that contains at least one value within kubernetes_groups or kubernetes_users. Teleport does not alter Kubernetes roles to allow or deny access. Read the next section for an explanation of how the Kubernetes Service evaluates Teleport roles in order to allow or deny access to pods in a cluster.

Role V7​

tip

Role V7 added support for more kind values. It uses singular names while later role versions use the plural form.

warning

The wildcard (*) and namespace kinds have special meaning, when using those, pay extra attention to the intent and the difference of behavior with later versions.

kind: role
metadata:
  name: kube-access
version: v7
spec:
  allow:
    kubernetes_labels:
      "*": "*"
    kubernetes_resources:
      - kind: pod
        namespace: production
        name: webapp
        verbs: ["*"]
    # ...

  • kind: The kind of resource to enable access to. Currently, Teleport supports the following kinds:

    warning

    The * kind behavior slightly differs from later versions. In V7, it allows access to all resources including the cluster-wide ones regardless of the namespace field. In later role versions, the namespace field is enforced, which means that when upgrading you may need to adjust your resources, especially on the deny side.

    warning

    The namespace kind covers the namespace resource as well as all resources within it. This behavior is different from later role versions where it only covers the resource itself which means that when upgrading you may need to adjust your resources, especially on the deny side.

    KindGrants access to
    *All resources, including cluster-wide ones regardless of the namespace field
    podPods
    secretSecrets
    configmapConfigMaps
    namespaceNamespaces and all resources within.
    serviceServices
    serviceaccountServiceAccounts
    kube_nodeNodes
    persistentvolumePersistentVolumes
    persistentvolumeclaimPersistentVolumeClaims
    deploymentDeployments
    replicasetReplicaSets
    statefulsetStatefulSets
    daemonsetDaemonSets
    clusterroleClusterRoles
    kube_roleRoles
    clusterrolebindingClusterRoleBindings
    rolebindingRoleBindings
    cronjobCronJobs
    jobJobs
    certificatesigningrequestCertificateSigningRequests
    ingressIngresses

  • namespace: The Kubernetes namespace in which to allow access to a resource. In the kube-access role, we are allowing access to a pod in the production namespace. Note that when the kind field is set to *, the namespace field is ignored, which is a different behavior from later role versions where the namespace field is enforced.

  • name: The name of the pod to allow access to. In kube-access, this is the webapp pod.

  • verbs: The operations to allow on the resource. Currently, Teleport supports:

    VerbGrants access to
    *All operations
    getRead a resource
    listList resources
    createCreate a resource
    updateUpdate a resource
    patchPatch a resource
    deleteDelete a resource
    deletecollectionDelete a collection of resources
    watchWatch resources
    portforwardCreate portforward requests for Pods
    execExecute commands in Pods

For both the namespace and name fields, you can add a wildcard character (*) to replace any sequence of characters. For example, name: "pod-*-*" matches pods named pod-1-a and pod-2-c. As with kubernetes_labels, if a value begins with ^ and ends in $, the Kubernetes Service will treat it as a regular expression using Go's re2 syntax (see the re2 README).

tip

For a user to access a pod named in a role's kubernetes_resources field, the user must be assigned a Teleport role that contains at least one value within kubernetes_groups or kubernetes_users. Teleport does not alter Kubernetes roles to allow or deny access. Read the next section for an explanation of how the Kubernetes Service evaluates Teleport roles in order to allow or deny access to pods in a cluster.

Role V6​

tip

Role V6 introduced the support for the kubernetes_resources field, but was restricted to only allow access to pods with kind 'pod'. Later role versions extended support to additional resources.

kind: role
metadata:
  name: kube-access
version: v6
spec:
  allow:
    kubernetes_labels:
      "*": "*"
    kubernetes_resources:
      - kind: pod
        namespace: production
        name: webapp
    # ...
  • kind: The kind of resource to enable access to. The only supported value is pod.
  • namespace: The Kubernetes namespace in which to allow access to a resource. In the kube-access role, we are allowing access to a pod in the production namespace.
  • name: The name of the pod to allow access to. In kube-access, this is the webapp pod.

For both the namespace and name fields, you can add a wildcard character (*) to replace any sequence of characters. For example, name: "pod-*-*" matches pods named pod-1-a and pod-2-c. As with kubernetes_labels, if a value begins with ^ and ends in $, the Kubernetes Service will treat it as a regular expression using Go's re2 syntax (see the re2 README).

tip

For a user to access a pod named in a role's kubernetes_resources field, the user must be assigned a Teleport role that contains at least one value within kubernetes_groups or kubernetes_users. Teleport does not alter Kubernetes roles to allow or deny access. Read the next section for an explanation of how the Kubernetes Service evaluates Teleport roles in order to allow or deny access to pods in a cluster.

Examples​

Full access to namespaced resources except to production​

The following roles will grant full access to all namespaced resources in all namespaces except the production one, where no resources will be accessible.

kind: role
metadata:
  name: kube-access
version: v7
spec:
  allow:
    kubernetes_labels:
      "*": "*"
    kubernetes_resources:
      # In v7, namespace means everything in the namespace.
      - kind: namespace  # v7 uses singular.
        name: "*"
        verbs: ["*"]
  deny:
    kubernetes_resources:
      - kind: namespace  # v7 uses singular.
        name: production
        verbs: ["*"]
    # ...

kind: role
metadata:
  name: kube-access
version: v8
spec:
  allow:
    kubernetes_labels:
      "*": "*"
    kubernetes_resources:
      # Grant access to namespaced resources in all namespaces.
      - kind: "*"
        api_group: "*"
        name: "*"
        namespace: "^.+$"  # Match any namespaced resource, will not match cluster-wide resources.
        verbs: ["*"]
      # Grant access to the namespace resource itself.
      - kind: namespaces  # In v8, namespaces mean the namespace itself. As it is considered a cluster-wide resource, it must be added separately.
        name: "*"
        verbs: ["*"]
  deny:
    kubernetes_resources:
      # Deny access to namespaced resources in the production namespace.
      - kind: "*"
        api_group: "*"
        name: "*"
        namespace: production
        verbs: ["*"]
      # Deny access to the namespace resource itself.
      - kind: namespaces  # v8 uses plural.
        name: production
        verbs: ["*"]
    # ...

Alternatively, this could also be used:

kind: role
metadata:
  name: kube-access
version: v8
spec:
  allow:
    kubernetes_labels:
      "*": "*"
    kubernetes_resources:
      # Grant full access on everything.
      - kind: "*"
        api_group: "*"
        name: "*"
        namespace: "*"  # In v8, '*' will match both namespaced and cluster-wide resources.
        verbs: ["*"]
  deny:
    kubernetes_resources:
      # Deny access to cluster-wide resources.
      - kind: "*"
        api_group: "*"
        name: "*"
        namespace: ""  # Empty namespace means only cluster-wide resources will match.
        verbs: ["*"]
      # Deny access to namespaced resources in the production namespace.
      - kind: "*"
        api_group: "*"
        name: "*"
        namespace: production
        verbs: ["*"]
      # Deny access to the namespace resource itself.
      - kind: namespaces  # v8 uses plural.
        name: production
        verbs: ["*"]
    # ...
Full access to dev namespace and all cluster-wide resources except clusterroles​

The following roles will grant access to all resources in the dev namespace and all cluster-wide resources except clusterroles.

kind: role
metadata:
  name: kube-access
version: v7
spec:
  allow:
    kubernetes_labels:
      "*": "*"
    kubernetes_resources:
      # In v7, "*" includes cluster-wide resources regardless of the namespace.
      - kind: "*"
        name: "*"
        namespace: dev
        verbs: ["*"]
  deny:
    kubernetes_resources:
      - kind: clusterrole  # v7 uses singular.
        name: "*"
        verbs: ["*"]
    # ...

kind: role
metadata:
  name: kube-access
version: v8
spec:
  allow:
    kubernetes_labels:
      "*": "*"
    kubernetes_resources:
      # Grant access to all resources in the dev namespace.
      # In v8, "*" doesn't include cluster-wide resources if the namespace is set.
      - kind: "*"
        api_group: "*"
        name: "*"
        namespace: dev
        verbs: ["*"]
      # Grant access to all cluster-wide resources.
      - kind: "*"
        api_group: "*"
        name: "*"
        namespace: ""  # Empty namespace means only cluster-wide resources will match.
        verbs: ["*"]
  deny:
    kubernetes_resources:
      - kind: clusterroles  # v8 uses plural.
        api_group: "*"
        name: "*"
    # ...
Full access to everything​
kind: role
metadata:
  name: kube-access
version: v7
spec:
  allow:
    kubernetes_labels:
      "*": "*"
    kubernetes_resources:
      - kind: "*"
        name: "*"
        namespace: "*"
        verbs: ["*"]
    # ...

kind: role
metadata:
  name: kube-access
version: v8
spec:
  allow:
    kubernetes_labels:
      "*": "*"
    kubernetes_resources:
      - kind: "*"
        api_group: "*"
        name: "*"
        namespace: "*"  # Wildcard '*' will match both namespaced and cluster-wide resources.
        verbs: ["*"]
    # ...

How the Kubernetes Service evaluates Teleport roles​

When a Teleport user makes a request to a Kubernetes cluster's API server, the Teleport Kubernetes Service intercepts the request and inspects the user's authorization. The Kubernetes Service denies the request if the user is not authorized to view a particular resource. If the user is authorized to perform their request, the Kubernetes Service modifies the request and forwards it to the appropriate API server.

Authorizing user requests​

When the Teleport Kubernetes Service receives a request, it evaluates two fields within the user's roles. If either of these fields does not allow the user to perform the request, the Kubernetes Service returns an error to the user:

kubernetes_labels​

The Teleport Kubernetes Service will only allow a user access to a pod if the cluster where the pod is running has a label that matches a user's kubernetes_labels configuration.

kubernetes_resources​

Some resource URIs within the Kubernetes API server include the names of specific resources.

For example, if a user runs kubectl exec to execute a command against the webapp pod in the development namespace, kubectl sends a request to the target cluster's API server at the following path:

"/api/v1/namespaces/development/pods/webapp/exec"

If a Kubernetes pod is available within the URL path of the request to a Kubernetes API server, the Teleport Kubernetes Service will check whether the user is authorized to access that pod.

In the example above, the Kubernetes Service checks if the user is authorized to access the webapp pod in the development namespace and, if not, denies the request.

Forwarding user requests​

Once the Teleport Kubernetes Service has authorized the user to perform a request against a Kubernetes cluster and (if applicable) a particular resource, it assembles a request to the upstream API server. It adds impersonation headers to the request based on the kubernetes_groups and kubernetes_users fields in the user's role (see the discussion of these fields earlier).

Because the Teleport Kubernetes Service accesses the upstream API server via the RBAC principals listed in the Teleport user's kubernetes_groups and kubernetes_users fields, the principals you specify in these fields must have access to the resources listed in the user's kubernetes_resources field. Otherwise, the Kubernetes Service will forward a request to the upstream API server with improper authorization, and the API server will deny the request.

Multiple roles​

How the Kubernetes Service evaluates multiple roles​

Before evaluating a user's request to a Kubernetes API server, the Teleport Kubernetes Service checks each of the user's roles. If one role's spec.allow.kubernetes_labels or spec.allow.kubernetes_resources conditions do not match the user's request, the Kubernetes Service checks the next role, and so on.

If the Kubernetes Service finds a role with a spec.allow condition that matches all of the cluster's labels and the request's resources, it looks up the role's allow.kubernetes_groups and allow.kubernetes_users fields. It adds these values to a list of RBAC principals it will use to write impersonation headers later on.

Next, the Kubernetes Service checks each of the user's roles for spec.deny conditions. If one role's spec.deny.kubernetes_labels or spec.deny.kubernetes_resources fields match the user's request, the Kubernetes Service looks up the role's spec.deny.kubernetes_groups and spec.deny.kubernetes_users fields. It removes each of these values from the list of users and groups it created earlier, denying the user access to these RBAC principals.

Example​

Let's say you have assigned the following three roles to a user:

kind: role
metadata:
  name: allow-dev-us-east-2
version: v7
spec:
  allow:
    kubernetes_labels:
      "region": "us-east-2"
    kubernetes_resources:
      - kind: pod
        namespace: "development"
        name: "redis-*"
      - kind: pod
        namespace: "development"
        name: "nginx-*"
    kubernetes_groups:
      - dev-viewers # Allows the user to view pods in the development namespace
---
kind: role
metadata:
  name: allow-exec
  version: v7
spec:
  allow:
    kubernetes_labels:
      - "*": "*"
    kubernetes_resources:
      - kind: pod
        namespace: "*"
        name: "*"
    kubernetes_groups:
      - executors # Allows the user to execute commands against any pod
---
kind: role
metadata:
  name: deny-redis-exec
  version: v7
spec:
  deny:
    kubernetes_resources:
      - kind: pod
        namespace: "*"
        name: "redis-*"
    kubernetes_groups:
      - executors

The dev-viewers Kubernetes group allows the user to view pods in the development namespace. The executors Kubernetes group allows the user to execute commands against any pod in any namespace.

If a user with these roles runs kubectl get pods/redis-1 in the development namespace, and the cluster has the label region:us-east-2, the Kubernetes Service will accept the request. Since the deny-redis-exec role denies the executors group for redis-* pods, the Kubernetes Service will forward the request while impersonating the dev-viewers group but not the executors group,

However, if the same user runs kubectl exec -it nginx /bin/bash in the development namespace, against the same cluster, the Kubernetes Service will forward the request with an impersonation header for both the dev-viewers and the executors groups, since the deny-redis-exec role's deny condition does not match the request.

Progressively enabling access to resources​

You can design your Teleport and Kubernetes RBAC to progressively allow access to limited subsets of Kubernetes resources. In other words, your users would have limited access to a wide range of resources in your Kubernetes cluster. Subsets of these users would have greater access to more limited sets of resources. Of these subsets of users, you can assign an even smaller group greater access to another set of resources, and so on.

To do this, define multiple Teleport roles where:

  • Some roles enable a lower degree of access to more Kubernetes resources
  • Other roles allow a higher degree of access to fewer Kubernetes resources

You can then assign combinations of roles to different users.

For example, this combination of roles allows a user to view all pods in all registered Kubernetes clusters, but only run kubectl exec or kubectl logs on nginx-* pods:

kind: role
metadata:
  name: kube-viewer
version: v7
spec:
  allow:
    kubernetes_labels:
      '*': '*'
    kubernetes_resources:
      - kind: pod
        namespace: "*"
        name: "*"
    kubernetes_groups:
    - viewer # can get and list pods but not execute commands or retrieve logs
---
kind: role
metadata:
  name: nginx-exec
version: v7
spec:
  allow:
    kubernetes_labels:
      '*': '*'
    kubernetes_resources:
      - kind: pod
        namespace: "*"
        name: "nginx-*"
    kubernetes_groups:
    - execAndLogs

In this case, the kube-viewer role maps a user to the Kubernetes viewer group, which allows a user to get and list pods but not execute commands and retrieve logs. With the nginx-exec role, the user can access the execAndLogs group, which can execute commands and retrieve logs, but only on nginx pods.

In this setup, you could also combine the kube-viewer role with other roles that grant elevated access to subsets of pods, depending on the requirements of your Teleport users.

Security consideration: Resource namespace restrictions​

When a Teleport user sends a request to list pods, e.g., with kubectl get pods, the Teleport Kubernetes Service does the following:

  • Retrieves available pods from the upstream Kubernetes API server, adding impersonation headers to the request based on the user's Teleport roles. These include the Kubernetes user and groups the user sends the request as.
  • Filters the list of available pods based on the pods that the user is authorized to access via kubernetes_resources.
  • Returns the list of pods to the user.

To avoid leaking resources unintentionally, you should ensure that namespace restrictions in your Kubernetes RBAC line up with those you have set up in Teleport.

For example, let's say a user has a Teleport role that grants access to any pod in any namespace, and maps that user to the default-pod-viewer Kubernetes group. This group can only view pods in the default namespace:

kind: role
metadata:
  name: kube-access-1
version: v7
spec:
  allow:
    kubernetes_groups:
      - default-pod-viewer
    kubernetes_resources:
      - kind: pod
        namespace: "*"
        name: "*"
    # ...

The user has a second Teleport role that maps the user to the system:masters Kubernetes group (which can access any pod in any namespace), and grants access only to the webapp pod in the default namespace:

metadata:
  name: kube-access-2
version: v7
spec:
  allow:
    kubernetes_groups:
      - system:masters
    kubernetes_resources:
      - kind: pod
        namespace: "default"
        name: "webapp"
    # ...

Since the kube-access-2 role maps the user to system:masters, when the Kubernetes Service forwards a request from this user, it will fetch all pods from the Kubernetes cluster by adding the system:masters group to the request's impersonation headers.

However, since the user also has a role (kube-access-1) that allows access to all pods in all namespaces, the Kubernetes Service will not filter the pods it retrieved via its first request to the API server.

In other words, the Kubernetes Service has no way to know that Teleport had mapped the user to the system:masters group in order to grant access to only the webapp pod in the default namespace.

If you have namespace restrictions in your Teleport RBAC, you should make sure that the same namespace restrictions exist in the Kubernetes RBAC resources you map to your Teleport users.

For example, you should rewrite the kube-access-1 role to have the following permissions, restricting the user to pods in the default namespace:

kind: role
metadata:
  name: kube-access-1
version: v7
spec:
  allow:
    kubernetes_groups:
      - default-pod-viewer
    kubernetes_resources:
      - kind: pod
        namespace: "default"
        name: "*"
```    # ...