Kubernetes

What is Kubernetes?

Kubernetes allows you to orchestrate the configuration and deployment of containers across multiple hosts. It's also an integral part of any microservice setup.

Containers are useful for a single application component, but deploying containers individually can get tedious and error-prone, especially at scale. If you want to deploy an entire application stack to a cluster of servers, Kubernetes can manage the entire thing for you.

For instance, let's say your application has a webserver running NGINX, a DB server running MariaDB, and a cache server running redis. Deploying with just containers is certainly doable, but we should work harder, not smarter! With Kubernetes you can run a single command to spin up the entire stack, upgrade each component, provide load balancing, scale the number of hosts dynamically, and more.

What Goes Into a Kubernetes Cluster?

First things first, we need machines for our containers. Each individual node in our cluster can obviously host one or more containers - this set of containers is called a pod.

When initializing the pod, we need to consider the startup costs of the pod backend and the network namespace that will connect our containers to everything else. The naive approach would be that if all of the containers were to die, we'd just set up the pod and all of the networking again from zero. However, that's less than ideal - we're wasting cycles unnecessarily. Instead, Kubernetes automatically creates the pause container, which is basically just an infinite loop to prevent the pod itself from dying if a container goes down.

TIP

Also see:

• K8s — pause container
• What is the role of 'pause' container?

The rest of the core pod mechanics are pretty straightforward. Each container will share the same network namespace and have the same IP address (beware port numbers!). When it comes to storage, there are volumes just like we'd use for standalone containers.

In order to create a pod (along with all the following concepts), we need to write some YAML. For a very basic example, this is how we would create an NGINX container (credit to the Kubernetes docs):

apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
  - name: nginx
    image: nginx:1.14.2
    ports:
    - containerPort: 80

To perform the actual creation, we'd save the YAML to a file and then apply this pod declaration with the kubectl binary:

kubectl apply -f ./nginx.yaml

How does this pod even get created? Magic has been impossible for a while now, so we need a controller (the control plane) to create that illusion. Behind the scenes, kubectl is making HTTP requests to the Kubernetes API server, which reaches out to each node's "agent" (kubelet) and decides which node can handle the new container. Once a node is allocated, the API server then tells that node to create the container within a pod, with the kubelet agent being responsible for applying the operation.

TIP

Kubernetes is a declarative system, not an imperative one. If we want three NGINX containers, then Kubernetes will ensure that there are three containers, with the controller automatically handling pod/node failures when they appear.

Now, if we create just individual pods, we're missing out on a lot of the benefits of Kubernetes such as automatic failover or scaling. To that end, we can instead create pods via a deployment. This is generally what you'll see for an actual application.

Nodes also need a way to handle network traffic. This piece is is managed by kube-proxy, which proxies container traffic, configures iptables and IPVS, etc.

Of course, none of this will be a static configuration. We may need expand or reduce the number of pods/nodes, they'll be created and destroyed, and they may move from one host to the next. Maintaining this configuration by hand would be a nightmare for anything that has to interact with the pods as a collective. We need some kind of abstraction layer that knows where our containers currently live and can route traffic to them even if the internal configuration changes. To that end, you can expose one or more pods as a Kubernetes service, which acts as a load balancer. It will provide a single DNS name, virtual IP, and incoming/outgoing port pair for your service.

When a service is created, a DNS record is created for it. For example, this could be app.default.svc.cluster.local. The service also creates an SVC record for the named port. If we wanted to expose port 80, then this would be _80-80._tcp.app.default.svc.cluster.local. And finally, with the right configuration, the service will also create a CNAME record for humans to navigate to.

Not every cluster will just contain a single deployment and service, though. You may be sharing a cluster with other teams, there may be tools monitoring for malicious containers, so on so forth. To provide a level of isolation, services, deployments, secrets, etc. are split into a number of namespaces. In other words, namespaces are logical "clusters" of sorts.

There are two primary universal namespaces to know about. The first one is default, which is where resources are deployed when a different namespace isn't specified. Then there's kube-system, which is where Kubernetes' default control plane components reside.

RBAC

Kubernetes provides RBAC because giving everybody admin privileges is less-than-ideal.

There are two main objects to be aware of:

1. A Role specifies a list of actions for specific resources, for example listing pods.
2. A RoleBinding maps a Role to a principal (i.e. user).

Additionally, roles are scoped to a specific namespace. To grant cluster-level permissions, you'll need to use ClusterRole and ClusterRoleBinding.

For an example of what a Role looks like (credit to the Kubernetes docs):

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: default
  name: pod-reader
rules:
- apiGroups: [""] # "" indicates the core API group
  resources: ["pods"]
  verbs: ["get", "watch", "list"]

And a RoleBinding that gives jane the pod-reader role in the default namespace:

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: read-pods
  namespace: default
subjects:
- kind: User
  name: jane
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

Reference: Control Plane Components

The control plane has several core components:

1. The API server is the frontend of the control plane. You as the developer will give it deployments to run using the kubectl command, and the API server will kick off the operations required to fulfill your request.
2. The etcd server is the data store for all storage-related concerns and retains the state of every resource in the cluster.
3. The scheduler is responsible for selecting nodes to host new pods.
4. The controller manager runs various controller processes to handle node failure, run one-off tasks, and more. Despite being multiple processes, this is just a single binary.
5. Cluster DNS is responsible for, well, managing DNS records. This component isn't required, but highly recommended for reasons mentioned earlier.