Skip to main content

Welcome to a Grasp lesson

Create a free account to keep studying and create your own study path.

Lesson 38

ServiceAccounts and API Server Interaction

Recommended resources

Hello! Welcome back to your Kubernetes crash course.

In our last lesson, we established the fundamental difference between the two identity types in Kubernetes: User Accounts for humans operating outside the cluster and ServiceAccounts for processes running inside Pods.

Today, we'll dive into the mechanics of that second category. Your learning outcome is to explain the role of ServiceAccounts in providing an identity for pods to interact with the API server. We'll uncover how a Pod gets its identity and uses it to communicate securely within the cluster.

This concept is directly analogous to the cloud identity mechanisms you're familiar with. Just as you use IAM Roles in AWS or Managed Identities in Azure to allow a service (like a VM or a function) to securely access other cloud APIs without hardcoding credentials, a Kubernetes ServiceAccount gives a Pod a native identity to interact with the Kubernetes API server.

1. How a Pod Gets Its Identity

Every Pod running in a Kubernetes cluster needs an identity. Kubernetes ensures this by automatically assigning a ServiceAccount to every Pod. If you don't explicitly specify one, Kubernetes assigns the default ServiceAccount from the Pod's namespace.

Let's see a quick demonstration of this.

What are Kubernetes Service Accounts?

The video 'What are Kubernetes Service Accounts?' from Peter Jausovec shows how a Pod automatically receives the 'default' ServiceAccount if one isn't specified in its configuration.

Watch the section from 00:57 to 02:07. The presenter creates a simple Pod and inspects its YAML definition, revealing that Kubernetes has automatically added serviceAccountName: default.

This automatic assignment is a key part of the "batteries-included" nature of Kubernetes. It guarantees that every workload has the potential to interact with the API, even if it doesn't have any permissions by default.


2. The Mechanism: The Auto-Mounted Token

So, a Pod is assigned a ServiceAccount. But how does that translate into a usable identity for the application code running inside the container?

When a Pod is scheduled onto a node, the Kubernetes control plane (specifically, the Kubelet on that node) performs a crucial step: it automatically mounts a token associated with the Pod's ServiceAccount into the container's filesystem.

This token is a JSON Web Token (JWT) that the application can use to prove its identity to the API server. The token and other useful information are mounted at a well-known location: /var/run/secrets/kubernetes.io/serviceaccount/.

This directory contains three important files:

  • token: The actual JWT bearer token that the application uses for authentication.
  • ca.crt: The cluster's root certificate. This is used to verify the identity of the API server, protecting against man-in-the-middle attacks.
  • namespace: A plain text file containing the name of the namespace the Pod is running in, which is a useful convenience for applications.

The following video provides a great walkthrough of this process.

What are Kubernetes Service Accounts?

Let's continue with the same video, which now explores what's inside the ServiceAccount and where the token gets mounted.

Watch from 02:07 to 04:35. The video explains the components of a ServiceAccount and then uses kubectl exec to show the token, ca.crt, and namespace files inside the running Pod.

This entire flow, from Pod creation to the token being available inside the container, is a core Kubernetes automation feature.

The diagram below provides a high-level visual summary of this entire authentication process.

Kubernetes Workload Authentication and Authorization with ServiceAccount Flow
This diagram illustrates the full lifecycle of a ServiceAccount token. When a Pod is created, the ServiceAccount admission controller ensures it has a ServiceAccount. The Kubelet on the node then requests a token and mounts it as a volume into the Pod's container, making it available for the application to use for authenticating with the API Server.

3. Using the Identity to Call the API Server

Now that the application inside the Pod has a token, it can use it to make authenticated calls to the Kubernetes API server. It does this by including the token in the Authorization header of its HTTP requests, using the "Bearer" scheme.

Let's see what happens when a Pod tries to use its default identity to list other Pods.

Kubernetes Service Account: What It Is and How to Use It

The article 'Kubernetes Service Account: What It Is and How to Use It' by vcluster provides a clear, text-based example of this process. This is excellent for seeing the exact commands and the API server's response.

Read the section titled 'Default Service Account' and follow the command-line example. Pay close attention to the curl command and the JSON response from the API server. Note the 403 status code and the Forbidden message.

The result of the curl command in that article is the most important takeaway of this lesson:

{
  "kind": "Status",
  "apiVersion": "v1",
  "status": "Failure",
  "message": "pods is forbidden: User \"system:serviceaccount:default:default\" cannot list resource \"pods\" in API group \"\" in the namespace \"default\"",
  "reason": "Forbidden",
  "code": 403
}

Let's break down this response:

  1. Authentication Succeeded: The API server successfully validated the token and identified the caller. The message explicitly states the caller's identity: User "system:serviceaccount:default:default". This is the standard format: system:serviceaccount:<namespace>:<serviceaccount-name>.
  2. Authorization Failed: After identifying the user, the API server checked if this user had permission to list the pods resource. It did not. Therefore, the server returned a 403 Forbidden error.

This clearly demonstrates the separation of authentication (who you are) from authorization (what you can do). The ServiceAccount successfully provided an identity, but that identity had no permissions.

Test your understanding!

An application is running in a Pod in the app-prod namespace. The Pod is configured to use a ServiceAccount named config-reader. The application tries to read a Secret from the API server and receives a 403 Forbidden error.

Based on what you've learned, what identity would the API server's error message say was forbidden from performing the action?

Show answer

The API server's error message would identify the user as system:serviceaccount:app-prod:config-reader. This tells you that authentication was successful—the API server knew exactly which ServiceAccount was making the request—but the authorization check failed because that ServiceAccount did not have the required permissions to read Secrets.


4. The Principle of Least Privilege and the default ServiceAccount

The fact that the default ServiceAccount has no permissions is a deliberate security feature. It enforces the principle of least privilege.

It would be a significant security risk if any Pod could automatically list Secrets or modify Deployments just by being created. By granting the default account no permissions, Kubernetes forces you to be explicit about what your workloads are allowed to do.

A common anti-pattern is to grant permissions to the default ServiceAccount to "fix" an access issue. This is dangerous because it gives those permissions to any Pod that doesn't have a specific ServiceAccount assigned. The correct approach, which we will cover next, is to create dedicated ServiceAccounts for specific tasks.

Kubernetes IAM & RBAC for DevOps & SRE

To reinforce this point, let's look at a list of best practices from the article 'Kubernetes IAM & RBAC for DevOps & SRE'.

Read 'SECTION 11 — Best Practices'. Note the recommendations to 'use ServiceAccounts for workloads' and the corresponding anti-pattern in 'SECTION 10' to avoid using 'default ServiceAccounts'.


Conclusion

In this lesson, we've unpacked the mechanism that gives in-cluster workloads a secure identity. This is a fundamental building block for creating secure, automated, and cloud-native applications on Kubernetes.

Key Takeaways:

  • A ServiceAccount provides an identity for a Pod. Every Pod has one, receiving the default SA if not otherwise specified.
  • Kubernetes automatically mounts the ServiceAccount's JWT token into the Pod's filesystem at /var/run/secrets/kubernetes.io/serviceaccount/.
  • Applications use this token as a Bearer Token in HTTP headers to authenticate to the Kubernetes API server.
  • The identity presented to the API server is system:serviceaccount:<namespace>:<serviceaccount-name>.
  • Having an identity (authentication) does not automatically grant permissions (authorization). By design, the default ServiceAccount has no permissions.

Next Up:
Now that you understand how a Pod gets and uses its identity, the next logical step is to create our own. In the next lesson, we will create custom ServiceAccounts and learn how to use them in our Pods, allowing us to move beyond the limitations of the default identity and prepare for assigning granular permissions.