DEVOPS

BOOTCAMP Container Orchestration

with Kubernetes
 Official Definition

Open source container orchestration tool

Developed by Google
Automates many processes involved in deploying, managing and scaling
containerized applications

Most used container orchestration platform

Also known as "K8s" or "Kube"
 The need for Kubernetes
Trend from Monolith to
Microservices
Containers are the perfect host for
microservice applications
Microservices
Resulted in an increased usage of monolith microservices
in containers
containers

Manually managing hundreds or hundreds of 1000s containers is a lot of effort

Kubernetes automates many of those manual tasks and provides

High Availability or no downtime Disaster Recovery - Backup and Restore
Automatic Scaling Self-Healing
 Core Kubernetes Components - 1
Kubernetes has many components, but these are the main ones you need to know:

SERVICE CONFIGMAP
POD
INGRESS SECRET

DEPLOYMENT
VOLUMES
STATEFULSET
Core Kubernetes Components - 2

POD

Group of 1 or more containers

Smallest unit of K8s

An abstraction over container

Usually 1 application/container per Pod

Pods are ephemeral

New IP address assigned on re-creation

 Core Kubernetes Components - 3

Internal vs External Service

SERVICE
When creating a service you can specify its type:
Basically a static or
Internal Service: By default, for example a
permanent IP address that
database, which should not be accessible
can be attached to each Pod

Also serves as a loadbalancer from outside

Lifecycles of Service and Pod External Service: Application accessible

are not connected through browser

If Pod crashes, the Service and

its IP address will be the same

 Core Kubernetes Components - 4

URL of external Service: http://124.89.101.2:8080

URL of Ingress Service: https://my-app.com

INGRESS

Ingress is the entrypoint to your K8s cluster

Request goes to Ingress first, which does the

forwarding to the Service

 Core Kubernetes Components - 5
For external configuration, Kubernetes has these 2 components:

CONFIGMAP
Pods can consume
To store non-confidential
ConfigMaps and Secrets as
data in key-value pairs
environment variables, CLI
SECRET arguments or as config files in a

Similar to ConfigMap, but to Volume

store sensitive data such as

passwords or tokens
Storing the data in a Secret component doesn't automatically make it secure. There are built-in
mechanisms (like encryption, defining authorization policies) for basic security, which are not enabled
by default! Recommended to use third-party secret management tools, because the provided
capabilities by K8s are not enough for most companies
 Core Kubernetes Components - 6
When a container crashes, K8s restarts the container but with a clean state. Meaning your data is lost!

VOLUME

Volume component basically

attaches a physical storage on a

hard drive to your Pod

Think of storage as an external hard
Storage could be either on a local drive plugged in to your K8s cluster
server or outside the K8s cluster

K8s doesn't manage any data persistence,

meaning you are responsible for
backing up, replicating the data etc.
 Core Kubernetes Components - 7
Deployment and StatefulSet are an abstraction of Pods

DEPLOYMENT STATEFULSET

Blueprint for Pods Blueprint for stateful applications

You work with Deployments Like databases etc

and by defining the number of In addition to replicating features,

replicas, K8s creates Pods StatefulSet makes sure database

reads and writes are synchronized to

avoid data inconsistencies

Having load balanced replicas our

setup is much more robust
 2 Types of Nodes

A Kubernetes cluster consists of a set of worker machines, called "Nodes"

Worker Node Control Plane

The containerized applications Manages the Worker Nodes

run on the Worker Nodes and the Pods in the cluster

Each Node runs multiple Pods on Much more important and

Control Plane
it needs to be replicated

Much more compute resources So in production, replicas run

needed, because the actual across multiple machines

workload runs on them

 Worker Node Components

Each worker node needs to have 3 processes installed:

1) Container Runtime 2) Kubelet

Software responsible for Agent that makes sure

running containers containers are running in a Pod

For example containerd, CRI-O Talks to underlying server (to

or Docker get resources for Pod) and

container runtime (to start

3) Kube-proxy
containers in Pod)
A network proxy with

intelligent forwarding of

requests to the Pods

 Control Plane Components - 1

Each control plane needs to have 4 processes installed:

1) API Server 2) Scheduler

3) Controller Manager 4) etcd

Control Plane makes global decisions about the cluster

Detects and responds to cluster events Control Plane

 Control Plane Components - 2

1) API Server 2) Scheduler

The cluster gateway - single Decides on which Node new Pod

entrypoint to the cluster should be scheduled

Acts as a gatekeeper for Factors taken into account for

authentication, validating the request scheduling decisions: resource

Clients to interact with the API server: requirements, hardware/software/

UI, API or CLI policy constraints, data locality, ...

 Control Plane Components - 3

3) Controller Manager 4) etcd

Detects state changes, like crashing K8s' backing store for all cluster data. A

of Pods and tries to recover the consistent, high-available key-value store

cluster state as soon as possible Think of it as a cluster brain, every change in the cluster gets saved or
For that it makes request to the updated into it
Scheduler to reschedule those Pods All other processes like Scheduler, Controller Manager etc work based
and the same cycle happens on the data in etcd as well as communicate with each other through etcd

store

The actual application data is NOT

stored in the etcd store
 Increase Kubernetes Cluster Capacity
As your application grows and its demand for resources increases, you may actually add more Nodes t​ o your

cluster, thus forming a more powerful and robust cluster to meet your application resource requirements

Add a Control Plane Node Add a Worker Node

1. Get a fresh new server

1. Get a fresh new server
2. Install all the Worker Node processes, like
2. Install all control plane processes on it
container runtime, Kubelet and KubeProxy on it
3. Join it to the K8s cluster using a K8s command
3. Join it to the K8s cluster using a K8s command

Control Plane Control Plane

 Minikube - Local Cluster Setup
Minikube implements a local K8s cluster

Useful for local K8s application development,

because running a test cluster would be complex

Control Plane and Worker

processes run on ONE machine

Run Minikube either as a container or virtual machine on your laptop

So you need a container runtime or virtual

machine manager on your laptop

 Kubectl - Command Line Tool

CLI Tool to interact with your K8s cluster

Basic kubectl commands:

In order for kubectl to access a K8s cluster, it needs a

kubeconfig file, which is created automatically when

deploying your minikube cluster Get status of different

components
By default, config file is located at ~/.kube/config

CRUD

kubectl API server Debugging

 K8s YAML Configuration File - 1
Also called "Kubernetes manifest" Config files are in YAML format, which is user-friendly,

Declarative: A manifest specifies the but strict indentation!

desired state of a K8s component Config files should be stored in version control

Each configuration file has 3 parts:

1) metadata 2) specification

Attributes of "spec" are specific to the kind

 K8s YAML Configuration File - 2
3) status

Automatically generated and added by Kubernetes

Current status: 1 replica
K8s gets this information from etcd, which holds the

current status of any K8s component

Desired status: 2 replica

 Deployment Configuration File
Deployment Configuration is a bit special

Since it's an abstraction over Pod, we have

the Pod configuration inside Deployment

configuration

Own "metadata" and "spec" section

Blueprint for Pod

 Labels & Selectors
Labels
Label Selectors
Labels are key/value pairs that are attached to Labels do not provide uniqueness

resources, such as Pods. Via selector the user can identify a set of

Used to specify identifying attributes that are resources

meaningful and relevant to users

Connecting Services to Deployments

 Ports in Service and Pod
In Service component you need to specify: In Deployment component:

port = the port where the service itself is accessible containerPort = port, the container

targetPort = port, the container accepts traffic on accepts traffic on

 Browser Request Flow through the K8s components
URL: IP address of Node + Port of external Service
K8s Components needed in this setup
Example Setup: 2 Deployment / Pod
2 Services
Mongo Express as UI 1 ConfigMap
1 Secret
MongoDB as database

User updates entries in database via browser

ConfigMap and Secret holds the MongoDB's

endpoint (Service name of MongoDB) and

credentials (user, pwd), which gets injected to

MongoExpress Pod, so MongoExpress can connect

to the DB
 Namespaces - 1
Namespaces provide a mechanism for isolating groups of resources within a single cluster

Names of resources need to be unique within a namespace, but not across namespaces

Like a virtual cluster inside a cluster

"Default" namespace

Namespaces per default Start deploying your application in the default

Namespaces per namespace called "default"

default, when you Create a new namespace

install K8s
Via kubectl command: kubectl create namespace my-ns
"kube-system" has
Via configuration file:
control plane processes

running

DON'T modify kube-system!!

 Namespaces - 2
Use Cases for when to use namespaces:

1 - Group resources logically 2 - Isolate team resources

Instead of having all in the "default" namespace To avoid conflicts

 Namespaces - 3
Use Cases for when to use namespaces:

3 - Share resources between different 4 - Limit permissions and compute

environments resources for teams
 Namespaces - 4
There are resources, which can't be created within a namespace, called cluster-wide resources:

Namespaced resources Cluster-wide resources

Most K8s resources (e.g. pods, services, etc.) are Live globally in a cluster, you can't isolate them

in some namespaces Low-level resources, like Volumes, Nodes

Access service in another namespace:

 Kubernetes Services - 1
Why Service?
Abstract way to expose an application running
Stable IP address
on a set of Pods
Loadbalancing
Different types of services:
Loose coupling

3 Service type attributes Within & Outside Cluster

ClusterIP is the default type, when you don't specify a type

 Kubernetes Services - 2

Service defines a logical set of Pods

The set of Pods targeted by a

Service is determined by a

selector

This creates a new Service named

"miroservice-one-service", which
targets port 3000 on any Pod with
the app=microservice-one label.

PORTS:
The service port is arbitrary

targetPort must match the port the container is listening at

 ClusterIP Service & its subtypes
ClusterIP is an internal service, not accessible from outside the cluster

All Pods in the cluster can talk to this internal service

Multi-Port Internal Service
Headless Internal Service
When you need to expose more than 1 port
When client needs to communicate with 1 specific
K8s lets you configure multiple port definitions on a
Pod directly, instead of randomly selected
Service

Use Case: When Pod In that case, you must

replicas are not give all of your ports

identical. For example names so that these

stateful apps, like when are unambiguous

only master is allowed

to write to database
 NodePort Service
Unlike internal Service, is accessible directly from outside cluster

Exposes the Service on each Node's IP at a static port

Node
URL = IP address of Worker Node : nodePort

A ClusterIP Service, to which the NodePort

Service routes, is automatically created

Not Secure: External traffic has access to fixed port on each Worker Node
 Loadbalancer Service
Exposes the Service externally using a cloud provider's load balancer

NodePort and ClusterIP Services, to which the external load balancer routes, are automatically created

More secure, more

efficient with 1 entry point

Node Loadbalancer
 Ingress - 1
External Services are a way to access applications in K8s from outside

In production a better alternative is Ingress!

Not a Service type, but acts as the entry point for your cluster

More intelligent and flexible: Let's you consolidate your routing rules into a single resource as it

can expose multiple services under the same IP address

 Configure multiple paths for same host:
Ingress - 2

Configure multiple sub-domains or domains:

Configure TLS

Certificate - https://
 Ingress - 3

How to configure Ingress in your cluster Ingress Controller

You need an implementation for Ingress Evaluates all the rules

Manages redirections
Which is Ingress Controller
Entry point to the cluster

Many third-party implementations

K8s Nginx Ingress Controller

 Ingress - 4

Before Ingress Controller you still need 1 load balancer

Option 1 - Cloud service provider: have out-of-the-box K8s solutions

Option 2 - Bare Metal: You need to configure some kind of entry point. Either

inside the cluster or outside as separate server

Source: https://kubernetes.io/docs/concepts/services-networking/ingress/
 Volumes - 1

K8s offers no data persistence out of the box

At its core, a volume is a directory (with some data in it), which is accessible to the containers in a Pod

K8s supports many types of volumes

Ephemeral volume types have a lifetime of a Pod, persistent volumes exist beyond the lifetime of a Pod

Storage Requirements

In this lecture we talk about persistent Storage that doesn't depend on pod lifecycle

Storage must be available on all Nodes

volumes, with these storage requirements:
Storage needs to survive even if cluster

crashes
 Volumes - 2

The way to persist data in K8s using Volumes is with these 3 resources:

Persistent Volume (PV) Storage Class (SC)

Storage in the cluster that has been provisioned by SC provisions PV dynamically when PVC claims it

an administrator or dynamically provisioned using

Storage Classes

Persistent Volume Claim (PVC)

A request for storage by a user

Similar to Pods. While Pods consume node

resources, PVCs consume PV resources

 Persistent Volume - 1

Persistent Volumes are NOT

namespaced, so PV resource is
Configuration Example
accessible to the whole cluster:

Depending on storage type, spec

attributes differ

In official documentation you can

find a complete list of storage

backends supported by K8s:

 Persistent Volume - 2

Local vs Remote Volume Types

For DB persistence use remote storage!

Each volume type has its own use case!

Local volume types violate 2. and 3. requirement for data persistence:

Being tied to 1 specific Node

Not surviving cluster crashes

 Persistent Volume Claim
Request for storage by a user

Claims can request specific size and access modes (e.g. they can be mounted

ReadWriteOnce, ReadOnlyMany or ReadWriteMany).

Storage Class
Storage Class Usage

1. Pod claims storage via PVC

Provisions PV dynamically
2. PVC requests storage from SC

3. SC creates PV that meets the

needs of the Claim

StorageBackend is defined in the SC resource

via "provisioner" attribute
each storage backend has own provisioner
internal provisioner - "kubernetes.io"
external provisioner
configure parameters for storage we want to
request for PV
 StatefulSet
Used to manage stateful applications, like databases

Manages the deployment and scaling of a set of Pods and provides guarantees about the

ordering and uniqueness of these Pods

Stateless Applications Stateful Applications

Doesn't depend on previous data Update data based on previous data

Deployed using Deployment Query data

Depends on most up-to-date data/state

Both manage Pods based on container specification

 Deployment vs StatefulSet
Unlike a Deployment, a StatefulSet maintains a sticky identity for

each of their Pods

These pods are created from the same spec, but are not

interchangeable: each has a persistent identifier that it maintains $(statefulset name)-$(ordinal)

across any rescheduling

Pods created from
Pods created from
Deployment
StatefulSet
Identical and interchangeable
More difficult
Created in random order with
Can't be created/deleted at
random hashes
same time
1 Service that load balances to
Can't be randomly addressed
any Pod
 Scaling database applications

Only 1 replica, can make changes

So replicas are not identical

Each replica has its own storage

These storages are constantly

synchronized
 Kubernetes on Cloud platform
2 options to create a Kubernetes cluster on a cloud platform

Create own cluster from scratch Use Managed K8s Service

You need to manage everything You only care about Worker Nodes

yourself Everything pre-installed

Not practical, when you want to Control Plane Nodes created and managed by cloud

setup things fast and easy provider

You only pay for the Worker Nodes

Use cloud native load balancer for Ingress controller

Use cloud storage

Less effort and time

 Example Managed Kubernetes Services

AWS: Elastic Kubernetes Service (EKS)

Azure: Azure Kubernetes Service (AKS)

Google: Google Kubernetes Engine (GKE)

Linode: Linode Kubernetes Engine (LKE)

 Helm - Package Manager
Helm is the package manager for Kubernetes. Think of it like apt/yum for Kubernetes

Packages YAML files and distributes them in public and private repositories

Helm Helm Chart

Tool that installs and manages K8s Helm package that contain

applications description of the package Chart.yaml

This is done via Charts, so Helm manages 1 or more templates, which contain K8s

these Charts manifest files

You can create your own Helm Charts with

Helm, push to Helm Repository

Download and use existing ones

 Helm Charts
Why Helm Charts?

Instead of everyone creating their own

K8s config files, 1 bundle of all needed

K8s manifests

Use existing official Charts: Sharing Helm Charts:

Many of these created by the official sources. Charts are hosted on their own repositories

Mysql Chart from Mysql or ElasticSearch Chart There are public repos, but you can have own

from ElasticSearch private repos in your company. E.g. Nexus

helm search <keyword> or Helm Hub website

 Helm as Templating Engine - 1
How to:
Helm renders the templates and communicates
1. Define a common blueprint
with the Kubernetes API 2. Dynamic values are replaced by

placeholders

Helm Chart Structure

Top level mychart folder name of chart

Chart.yaml meta info about chart

values.yaml values for the template files

charts folder chart dependencies

templates folder the actual template files

 Helm as Templating Engine - 2

many YAML files just 1 YAML file

Many

values are

the same!

Another use case: Deploy the same

bundle of K8s YAML files across

multiple clusters:
Pull Docker Images into K8s cluster
PRIVATE REPO: For K8s to fetch

the Docker Image into K8s

cluster, it needs explicit access

PUBLIC REPO: For public

images, like mongodb etc.

pulled from public repositories

no credentials needed
 Steps to pull image from Private Registry

1 - Create Secret Component 2 - Configure Deployment

Contains credentials for Docker registry Use Secret using imagePullSecrets

docker login token is stored in dockerconfig file

 Kubernetes Operators - 1

Stateful applications need constant management and syncing after deployment. So

stateful applications, like database need to be operated

Instead of a human operator, you have an automated scripted operator

Operators are created

by official maintainers
Stateless applications:

Is managed by Kubernetes

Stateful applications:
Operators
K8s can't automate the process natively
 Kubernetes Operators - 2
How it works:

Control loop mechanism

Custom Resource Definitions

Makes use of CRD's Custom K8s component (extends the K8s API)

Include domain/App-specific-knowledge, e.g. mysql:

How to create mysql cluster

How to run it

How to synchronize the data

How to update
 Layers of Security
In K8s, you must be authenticated

(logged in) before your request can Request to K8s cluster

be authorized (granted permission

to access)

Authentication Authorization
K8s doesn't manage Users natively K8s supports multiple authorization modules, such
No K8s resources exist for representing normal as ABAC mode, RBAC Mode and Webhook mode
user accounts On cluster creation, admins configure the
Admins can choose from different authentication authorization modules that should be used
strategies K8s checks each module, and if any module
authorizes the request, then the request can
proceed
 Authentication
API server handles authentication of all the requests

Available authentication strategies:

Client Certificates Static Token File 3rd Party Identitiy Service, like LDAP

A csv file with a minimum of 3

columns: token, user name, user

uid, followed by optional group

names
 Authorization
As a security best practice, we only want to give people or services just enough permission to do

their tasks: Least Privilege Rule

Developers need only limited

Admins need cluster-wide
access for example to deploy
access to do tasks like configure
applications to 1 specific
namespaces etc.
namespace
 Authorization with RBAC - 1
Role-based access control (RBAC)

A method of regulating access to resources based on the roles of users within the organization

Enabling RBAC: kube-apiserver --authorization-mode=RBAC --other-options

4 kinds of K8s resources: ClusterRole, Role, ClusterRoleBinding, RoleBinding

Role and ClusterRole RoleBinding and ClusterRoleBinding

Contains rules that represent a set Link ("Bind") a Role or ClusterRole to a
of permissions User or Group
Permissions are additive
 Authorization with RBAC - 2

ClusterRole
Role

Define permissions cluster wide

Define namespaced permissions
For example: configure cluster-
through Role
wide volumes, namespaces
Bound to a specific namespace

What resources in that namespace

you can access

What action you can do with this

resource
 Authorization with RBAC - 3

ClusterRoleBinding RoleBinding

Link ("Bind") ClusterRole to a User or Group Link ("Bind") Role to a User or Group
 Authorization for Applications

ServiceAccounts provide an identity for processes

that run in a Pod, for example Jenkins or Prometheus

A RoleBinding or

ClusterRoleBinding can also

bind a role to a

ServiceAccount
 Introduction to Microservices

Microservices are an architectural approach to software development

Software is composed of small independent services (instead of having a huge monolith)

Each business functionality is encapsulated into own Microservice (MS)

Benefits

Each MS can be developed, packaged and released

independently

Changes in 1 MS doesn't affect other MS

Less interconnected logic, loosely coupled

Each MS can be developed by separate developer teams

 Communication between Microservices

Communication between those services can happen in different ways:

Service-to-Service API Calls Message-Based Communication Service Mesh Architecture

Direct communication Communication through a Platform layer on top of the

message broker, like Rabbitmq infrastructure layer, like Istio,

or Redis Linkerd, HashiCorp Consul

Enables managed, observable

and secure communication

 DevOps Task: Deploy Microservices App - 1

Information you need from developer:

1. Which services you need to deploy?

2. Which service is talking to which

service?
Developers develop the As a DevOps engineer your
3. How are they communicating?
microservice applications task would be to deploy the
4. Which database are they using? 3rd
existing microservices
party services
application in a K8s cluster
5. On which port does each service

run?
 DevOps Task: Deploy Microservices App - 2

HOW TO 1. Prepare K8s environment

a. Deploy any 3rd party apps

b. Create Secrets and ConfigMaps for microservices

2. Create Deployment and Service for each microservices

1) 2) When deploying multiple

similar services to K8s, you
can use helm chart with 1
common template and
replace specific values for
each service on the fly
during deployment
 Best Practices - 1

Specify a pinned version on each container image

Why? Otherwise, latest version is fetched, which makes it unpredictable and intransparent as to which
versions are deployed in the cluster

Configure a liveness probe on each container

Why? K8s knows the Pod state, not the application state. Sometimes
pod is running, but container inside crashed. With liveness probe we
can let K8s know when it needs to restart the container

Configure a readiness probe on each container

Why? Let's K8s know if application is ready to receive traffic

 Best Practices - 2

Configure resource limits & requests for each container

Why? To make sure 1 buggy container doesn't eat up all

resources, breaking the cluster

Don't use NodePort in production

Why? NodePort exposes Worker Nodes directly, multiple points of entry to

secure. Better alternative: Loadbalancer or Ingresss

Always deploy more than 1 replica for for each application

Why? To make sure your application is always available, no downtime for users!
 Best Practices - 3

Always have more than 1 Worker Node

Why? Avoid single point of failure with just 1 Node

Label all your K8s resources

Why? Have an identifier for your components to group pods and reference in Service e.g.

Use namespaces to group your resources

Why? To organize resources and to define

access rights based on namespaces e.g.

 Security Best Practices

Ensure Images are free of vulnerabilities

Why? Third-party libraries or base images can have known vulnerabilities. You

can do manual vulnerability scans or better automated scans in CI/CD pipeline

No root access for containers

Why? With root access they have access to host-level resources. Much more

damage possible, if container gets hacked!

Keep K8s version up to date

Why & How? Latest versions include patches to previous security issues etc.

Upgrade with zero downtime by having multiple nodes and pod replicas on different nodes