The Power of DevOps: Streamlining Software Development and Delivery - Part 2 ( Kubernetes -Orchestration & Automation)

Orchestration & Automation

 Kubernetes is a rapidly evolving platform that manages container-based applications and their associated networking and storage components. You can build and run modern, portable, microservices-based applications, using Kubernetes to orchestrate and manage the availability of the application components. Kubernetes supports both stateless and stateful applications as teams progress through the adoption of microservices-based applications. As an open platform, Kubernetes allows you to build your applications with your preferred programming language, OS, libraries, or messaging bus. Existing continuous integration and continuous delivery (CI/CD) tools can integrate with Kubernetes to schedule and deploy releases.

When it comes to managing a multi-containerized environment, Kubernetes has become the solution for a lot of challenges such as:

•            Automated Scaling

•            Continuous Monitoring of services

•            Self-Healing

•            Automated Scheduling

and much more!

At its core, Kubernetes also known as K8s is an open-source system for automating deployment, scaling, and management of multi-containerized applications with a touch of "automation" to it.

Kubernetes is particularly useful for DevOps teams since it offers service discovery, load balancing within the cluster, automated rollouts and rollbacks,  resource allocation, isolation, and scaling your pods vertically or horizontally. self-healing of containers that fail, and configuration management. Plus, Kubernetes is a critical tool for building robust DevOps CI/CD pipelines. It has been embraced by the open source community and is now part of the Cloud Native Computing Foundation. Amazon, Microsoft, and Google all offer managed Kubernetes services on their cloud computing platforms, which significantly reduces the operational burden of running and maintaining Kubernetes clusters and their containerized workloads.

Lets see its architecture from different perspectives.

Kubernetes cluster architecture

A Kubernetes cluster in On-Prem

In the context of Kubernetes, everything runs "on top of" a cluster. A Cluster is a group of multiple nodes running together. A node is the actual machine or a server ( Physical or Virtual) that runs a containerized application.

Generally, in the context of Kubernetes, we interact with nodes that are Virtual Machines either running locally on your system (using virtualization software ) or on cloud platforms like AWS, Azure, GCP.

 So to understand better we should know the components more in detail.

A Kubernetes Cluster consists of mainly two kinds of nodes:

●       Master Node or Control Plane

●       Worker Nodes

Master Node or The Control Plane

This is the MOST IMPORTANT node or virtual machine in the entire cluster as it is responsible for making global decisions about the cluster. The tasks such as:

●       receiving and further processing an external request,

●       scheduling an application on a node,

●       detecting and responding to cluster events

and much more, are performed by the components that make up the Control Plane.

API Server - Brain of the Cluster

It's important to understand that under the hood, Kubernetes has a RESTful API (written in Go programming language) that supports the standard HTTP operations - POST, PUT, PATCH, DELETE, GET.

The API server component exposes this API to enable communication between:

●       the end users,

●       different parts of the cluster, and

●       external components

Apart from this, the Kube API server acts as the BRAIN of the entire cluster as "every external request to the cluster is first received by the API server" and then is directed to other components.

ETCD - The Cluster Database

A Kubernetes Cluster has an internal database called etcd, which by default stores all the cluster data. Essentially, ETCD is a distributed key-value store that provides a reliable way to store data that needs to be accessed by a distributed system or cluster of machines and that's EXACTLY our use case in a Kubernetes cluster.

The keyword here is "key-value store" which means that the data is stored in the form of key-value pairs rather than the traditional way of using tables.

It's interesting to note that, ETCD is itself an open source CNCF Graduated project (just like Kubernetes).

As all the cluster-related information is stored in ETCD, it can also be used to periodically take a backup of the cluster data which is certainly recommended to recover Kubernetes clusters under disaster scenarios, such as losing a control plane node itself. This is one of the MOST IMPORTANT use cases of etcd in Kubernetes.

It's important to note that, "any operation in the cluster is considered as COMPLETE only after it has been written into ETCD".

Scheduler - The Node Allocator

As we discussed in the previous section, Nodes are the machines (physical or virtual) that run a containerized application. Right now, we are discussing the Master Node or the Control plane which is like an ADMIN for the cluster operations. The other kind of node in a Kubernetes cluster is called a "worker node" where the actual containerized application runs.

The main task of the Scheduler component of the control plane is "to find the best-fit worker node to run our application and then schedule the app on that".

There are a few parameters/factors involved here according to which the scheduler finds this "best-fit worker node":

●       resources requirements to run the app,

●       number of internal/external requests made by the app,

●       hardware/software constraints, policy constraints, and more.

A real-life use case for this can be - Node Failure. In case a node, running your containerized application fails or goes down, the scheduler is responsible for quickly ASSIGNING A NEW NODE to your application so that there is NO or LESS DOWNTIME. We'll discuss how this is done when we understand how an external request is processed throughout the cluster in a later section!

Multiple methods/concepts are involved in assigning a new node to an application. Some of them are listed below:

●       nodeSelector

●       nodeName

●       Node Affinity - Node affinity is a property of Pods that attracts them to a set of nodes (either as a preference or a hard requirement). Taints are the opposite -- they allow a node to repel a set of pods.

●       Taints and Tolerations (Tolerations are applied to pods. Tolerations allow the scheduler to schedule pods with matching taints. Tolerations allow scheduling but don't guarantee scheduling: the scheduler also evaluates other parameters as part of its function. Taints and tolerations work together to ensure that pods are not scheduled onto inappropriate nodes. One or more taints are applied to a node; this marks that the node should not accept any pods that do not tolerate the taints)

Controller Manager - Cluster State Supervisor

This is again, an IMPORTANT component to understand in the control plane because of the responsibility it has. If we look at an online definition for this:

Kubernetes controller manager is responsible for managing various controllers that regulate the state of the cluster.

To break this down a bit, a Controller is a program or a control loop that continuously watches the state of the cluster through the API server and makes necessary changes with the sole aim of "moving the current cluster state towards the desired state".

There are different types of controllers responsible for different Kubernetes objects available.

Kubernetes Objects are like building blocks or configurations that describe what should run or how resources should be managed within a Kubernetes cluster.

There are different kinds of Kubernetes objects, which enable to create and manage various aspects of a containerized application, such as:

●       Pods (also an important component of the architecture, discussed later!)

●       Deployments

●       Services

●       ReplicaSets

●       Persistent Volumes, etc.

Each of these controllers are separate process, but to reduce complexity, they are all compiled into a single binary and run in a single process through the Controller Manager component of the control plane.

Let's understand the purpose of the Controller Manager with a simple example:

Imagine that you have your containerized app running on a worker node. You want to ensure that a specific number of identical instances are always running to maintain the desired level of availability for your application. We call the "number of identical instances" replicas and even if one of the application instances fails or goes down, Kubernetes will always bring up another one to match the number of replicas we defined when creating our app.

How is this done? That's where the work of the Controller Manager comes in. A "controller" will continuously monitor the app instance's current state and will try to keep the desired number of app instances (that are defined by us) always up and running.

In summary, If anytime the current state ≠ desired state, the controller is triggered and makes the necessary changes to bring the state to what we desire or what we define!

There are mainly two kinds of controllers:

Built-in Controllers: (for each Kubernetes object)

●       Deployment Controller

●       ReplicaSet Controller

●       Service Controller

and more!

DIY Controllers:

You can even build YOUR controller that fits your unique use case, by extending the Kubernetes API.

This is a really interesting concept to learn and this is one of the features that makes Kubernetes "highly configurable and extensible".

Cloud Controller Manager - The Cloud State Supervisor

This component of the control plane enables a Kubernetes cluster to interact and communicate with a cloud provider's API. It provides cloud-specific controllers that automate the management and provisioning of cloud resources, simplifying the deployment and operation of Kubernetes clusters on various cloud platforms like AWS, GCP, Azure etc.

You would have heard of the services such as:

●       AWS EKS (Elastic Kubernetes Service)

●       GKE (Google Cloud Engine)

●       AKS (Azure Kubernetes Service)

These are some of the cloud provider-specific services that enable you to deploy a full-fledged Kubernetes cluster on the cloud.

In this case, the Cloud Controller Manager plays a critical role in seamlessly integrating Kubernetes with the cloud provider's services and resources, making it easier to automate processes and manage the cluster resources efficiently.

An interesting point to note here is that if you are running Kubernetes in a learning environment inside your PC (local cluster), the cluster does not have a cloud controller manager component.

Worker Node

We have mentioned this a few times in the previous section and from the name itself it's clear that this is the actual WORKING MACHINE that runs a containerized application in the cluster.

Let us discuss and understand each of its components in detail!

Pod

Let us first start with the Pod because this is important!

As we discussed previously, the Controller Manager component in the control plane manages various kinds of controllers that constantly monitor the state of the Kubernetes objects. Also, we learned that there are different kinds of Kubernetes objects, which enables us to create and manage various aspects of a containerized application.

Now, a Pod is the smallest Kubernetes object that we can run and manage in a cluster. A simple definition is given in the documentation:

A Pod (as in a pod of whales or pea pod) is a group of one or more containers, with shared storage and network resources, and a specification for how to run the containers.

Throughout the previous sections, we have talked about how Kubernetes efficiently runs and manages multi-containerized applications. The truth is, there are various challenges involved in running multiple containers especially if one or more containers are dependent on each other.

Some of the challenges include:

●       Network Isolation: Container network isolation is essential for managing the network rules for each container individually. Managing network isolation between containers manually can be a complex and error-prone process.

●       Resource Allocation: Managing the resource allocation for individual containers can be less efficient, especially if we talk about a production environment where 1000s of containers may be running at the same time.

●       Monitoring and Logging: In a multi-container environment, it's crucial to collect and analyze logs and metrics from all containers efficiently. Gathering, aggregating, and correlating logs and metrics from individual containers can lead to increased complexity in monitoring and troubleshooting, as we'll need to track the interactions and dependencies between each container.

To solve the above-mentioned challenges and more, containers are encapsulated within a Pod and all the containers within that pod have shared storage and network resources.

To solve the above-mentioned challenges and more, containers are encapsulated within a Pod and all the containers within that pod have shared storage and network resources.

Generally, the rule of thumb is to have "one or more application containers which are relatively tightly coupled, in the same pod".

For example, imagine you have a web application that requires two containers to run properly:

Main application container - which runs your web application code

Database container - which hosts your database management system such as MySQL, PostgreSQL etc.

Let us understand how running these containers within a Pod solves the above-discussed challenges:

As these containers are within the same pod, they share the same network namespace, allowing them to communicate with each other using localhost.

The application container and the database container can also share the same storage volumes within the Pod. This enables the application to read and write data to the database using a shared storage volume.

In case you have more users and you need to scale your application, you can scale the Pod itself, which will ensure that both the application and the database containers are scaled together. This maintains the tight coupling between them.

Additionally, the controllers like Deployments or StatefulSets can manage the scaling, rolling updates, and self-healing of the entire Pod, thus, simplifying the management of both containers.

I believe this answered the question: "Why do we need a Pod in Kubernetes?" quite thoroughly and I hope you understand its need!

Now that we have understood the concept of Pods, you may replace all the instances in which we used "application containers" with "Pods" in the above sections and you'll see things have already started to fall in place

Kubelet - Brain of the Worker Node

Yes, you heard it right! If the API Server is the brain of the entire cluster, the Kubelet component is the brain of a worker node.

Kubelet is essentially an agent which runs on each worker node and its main responsibility is to ensure that the containers described in the PodSpecs are running and healthy.

PodSpec is a file written in YAML or JSON, wherein we define our Pod - what our application should look like.

Some of the things that we define in a pod specification file are:

• container images to use for the containers 

• ports at which the container runs 

• resource allocation for the pod itself 

• and more!

Kubernetes components and concepts

#Daemon Set: Ensures that a specific pod runs on all or some nodes in a cluster, ideal for tasks like logging or monitoring.

#Replica Set: Maintains a defined number of replica pods, ensuring high availability and fault tolerance.

#Replication Controller: Ensures a specified number of pod replicas are running at all times, ensuring high availability.

#Stateful Set: Perfect for managing stateful applications, providing stable network identities and ordered, graceful scaling.

#Job Scheduler: Manages batch jobs, ensuring they run to completion, which is crucial for tasks like data processing.

 #Job: Represents a single task in a workflow, running to completion, often used for one-time or periodic tasks.

#Pod: The smallest deployable unit, containing one or more containers, sharing the same network namespace.

#Pod Disruption Budget: Specifies the number of disruptions allowed to application pods during voluntary disruptions, crucial for stability.

#Service: Exposes a set of pods as a network service, enabling load balancing and service discovery.

#Container: A lightweight, standalone, executable package that includes everything needed to run a piece of software, including code, runtime, system tools, and libraries.

#Horizontal Pod Scaler: Automatically scales the number of pod replicas in a deployment or replica set based on CPU utilization or other selected metrics.

 #Ingress: Manages external access to services, typically HTTP, and provides features like SSL termination and virtual host support.

#Volume: Provides a way to store data in containers, even after the container is deleted.

 #Persistent Volume Claim: Requests storage resources from a persistent volume to be used by a pod.

#Config Map: Stores configuration data separately from the application's container image, making it easier to manage and update.

#Secret: Similar to Config Maps but used for sensitive data like passwords and tokens.

Elements Map for Kubernetes

A Kubernetes cluster in public cloud

AKS provides a managed Kubernetes service that reduces the complexity of deployment and core management tasks, like upgrade coordination. The Azure platform manages the

AKS control plane, and you only pay for the AKS nodes that run your applications.

The bottom line is, although Kubernetes originally started as a way to orchestrate and manage multiple containers, due to continuous development throughout these years, it has become way more than that.

The importance of streamlined development processes and continuous integrations/deployment cannot be overstated. Devops tools like Docker, Kubernetes and Jenkins are fundamental for achieving efficiency and scalability.

Manual Deployment of Monolithic or Microservice is not sustainable when you have an increasing number of services that need to be deployed multiple times a day. So, as part of your transition to microservices it is critical that you embrace continuous delivery to reduce the risks of release failure, as well as ensure your team is focused on building and running the application, rather than being stuck deploying it. Practicing continuous delivery also means that your service passed acceptance tests before going to production.