Container Orchestration with Kubernetes

Overview

Kubernetes (K8s) is the leading container orchestration platform that automates deployment, scaling, and management of containerized applications. This article explores Kubernetes architecture, core concepts, and best practices for orchestrating containers at scale.

Kubernetes Architecture

Control Plane Components

The Kubernetes control plane manages the cluster and makes global decisions about the cluster.

kube-apiserver

Exposes the Kubernetes API
Frontend for the Kubernetes control plane
Validates and configures API objects

etcd

Consistent and highly-available key-value store
Stores all cluster data
Requires backup and maintenance

kube-scheduler

Watches for newly created pods without assigned nodes
Selects a node for the pod to run on
Considers resource requirements and constraints

kube-controller-manager

Runs controller processes
Includes node controller, replication controller, endpoints controller
Manages lifecycle events

Node Components

Node components run on every node and maintain running pods.

kubelet

Ensures containers are running in a pod
Communicates with the control plane
Reports node status

kube-proxy

Maintains network rules on nodes
Enables network communication to pods
Implements service abstraction

Container Runtime

Software responsible for running containers
Examples: Docker, containerd, CRI-O

Cluster Architecture Diagram

TEXT

┌─────────────────────────────────────────────────────────┐
│                   Control Plane                         │
│  ┌─────────────┐  ┌────────────────┐  ┌─────────────┐  │
│  │ kube-apisvr │  │ kube-controller│  │ kube-sched  │  │
│  └─────────────┘  │ -manager       │  └─────────────┘  │
│                   └────────────────┘                   │
│                           │                            │
│                    ┌─────────────┐                     │
│                    │    etcd     │                     │
│                    └─────────────┘                     │
└─────────────────────────────────────────────────────────┘
                              │
          ┌───────────────────┼───────────────────┐
          │                   │                   │
    ┌─────────────┐    ┌─────────────┐    ┌─────────────┐
    │   Node 1    │    │   Node 2    │    │   Node 3    │
    │             │    │             │    │             │
    │ kubelet     │    │ kubelet     │    │ kubelet     │
    │ kube-proxy  │    │ kube-proxy  │    │ kube-proxy  │
    │ Container   │    │ Container   │    │ Container   │
    │ Runtime     │    │ Runtime     │    │ Runtime     │
    └─────────────┘    └─────────────┘    └─────────────┘

Core Kubernetes Concepts

Pods

The smallest deployable units in Kubernetes that can be created and managed.

Pod Characteristics:

One or more containers that share storage/network
Colocated and scheduled together
Have a unique IP address
Share the same namespace

Pod Specification Example:

YAML

apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  containers:
  - name: my-container
    image: nginx:1.20
    ports:
    - containerPort: 80
    env:
    - name: ENV_VAR
      value: "production"

Deployments

Deployments manage the lifecycle of pods and replica sets.

Deployment Features:

Declarative updates
Rollbacks
Scaling
Self-healing

Deployment Example:

YAML

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: my-container
        image: nginx:1.20
        ports:
        - containerPort: 80
        resources:
          requests:
            memory: "64Mi"
            cpu: "250m"
          limits:
            memory: "128Mi"
            cpu: "500m"

Services

Services provide stable networking to pods that can change.

Service Types:

ClusterIP: Internal cluster communication
NodePort: Expose service on each node's IP at a static port
LoadBalancer: Expose service externally using cloud provider's LB
ExternalName: Map service to external DNS name

Service Example:

YAML

apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  selector:
    app: my-app
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  type: LoadBalancer

Namespaces

Namespaces provide virtual clusters within a physical cluster.

Common Namespaces:

default: Default namespace for user objects
kube-system: Objects created by the Kubernetes system
kube-public: Publically readable resources
kube-node-lease: Node lease objects

Kubernetes Configuration

YAML Manifest Structure

Kubernetes resources are defined using YAML manifests with four required fields:

YAML

apiVersion: # Kubernetes API version
kind:       # Type of resource
metadata:   # Object identification
spec:       # Desired state specification

ConfigMaps and Secrets

ConfigMaps

Store non-confidential data in key-value pairs.

YAML

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
data:
  database_url: "mysql://localhost:3306/mydb"
  log_level: "info"

Secrets

Store sensitive information like passwords and keys.

YAML

apiVersion: v1
kind: Secret
metadata:
  name: app-secret
type: Opaque
data:
  username: dXNlcg==  # base64 encoded
  password: cGFzc3dvcmQ=  # base64 encoded

Kubernetes Networking

Service Discovery

Kubernetes provides several mechanisms for service discovery:

DNS-based discovery
Environment variables
API-based discovery

Ingress Controllers

Ingress controllers manage external access to services, typically HTTP/HTTPS.

Ingress Example:

YAML

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: app-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: myapp.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: my-service
            port:
              number: 80

Kubernetes Storage

Persistent Volumes

Persistent Volumes (PVs) provide storage that persists beyond pod lifecycles.

PV Example:

YAML

apiVersion: v1
kind: PersistentVolume
metadata:
  name: my-pv
spec:
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: standard
  hostPath:
    path: "/mnt/data"

Persistent Volume Claims

Persistent Volume Claims (PVCs) request storage from PVs.

YAML

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
  storageClassName: standard

Kubernetes Workloads

ReplicaSet

Ensures a specified number of pod replicas are running at any given time.

StatefulSets

Manages stateful applications with stable, unique identities.

DaemonSets

Ensures that all (or some) nodes run a copy of a pod.

Jobs and CronJobs

Jobs: Run pods to completion
CronJobs: Create jobs on a repeating schedule

Kubernetes Commands

Common kubectl Commands

Cluster Information:

BASH

# Get cluster information
kubectl cluster-info

# Get cluster nodes
kubectl get nodes

# Describe a node
kubectl describe node <node-name>

Resource Management:

BASH

# Get resources
kubectl get pods
kubectl get deployments
kubectl get services
kubectl get all

# Describe resources
kubectl describe pod <pod-name>
kubectl describe deployment <deployment-name>

# Create resources
kubectl create -f resource.yaml
kubectl apply -f resource.yaml

# Update resources
kubectl set image deployment/my-app my-container=nginx:1.21
kubectl scale deployment my-app --replicas=5

# Delete resources
kubectl delete pod <pod-name>
kubectl delete -f resource.yaml

Pod Management:

BASH

# View pod logs
kubectl logs <pod-name>
kubectl logs -f <pod-name>  # Follow logs

# Execute commands in pods
kubectl exec -it <pod-name> -- bash

# Port forwarding
kubectl port-forward <pod-name> 8080:80

Kubernetes Scaling

Horizontal Pod Autoscaler (HPA)

Automatically scales the number of pods based on CPU utilization or custom metrics.

YAML

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: my-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-app
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70

Vertical Pod Autoscaler (VPA)

Automatically adjusts CPU and memory requests/limits for pods.

Cluster Autoscaler

Automatically adjusts the size of the Kubernetes cluster based on resource demands.

Kubernetes Security

RBAC (Role-Based Access Control)

Controls access to Kubernetes API resources.

Role Example:

YAML

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: default
  name: pod-reader
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "watch", "list"]

Network Policies

Control traffic flow between pods.

YAML

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-all
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress

Security Contexts

Define privilege and access control settings for pods and containers.

Helm Package Manager

Helm is the package manager for Kubernetes that uses charts to define applications.

Helm Concepts:

Chart: Package of pre-configured Kubernetes resources
Release: Instance of a chart running in Kubernetes
Repository: Place to store and share charts

Common Helm Commands:

BASH

# Search for charts
helm search repo stable

# Install a chart
helm install my-release stable/mysql

# Upgrade a release
helm upgrade my-release stable/mysql

# Uninstall a release
helm uninstall my-release

Monitoring and Observability

Essential Monitoring Components

Prometheus

Metrics collection and storage
Powerful query language (PromQL)
Built-in alerting

Grafana

Visualization and dashboarding
Multiple data source support
Alerting capabilities

Fluentd/Elasticsearch/Kibana (EFK)

Log aggregation and analysis
Centralized logging
Search and visualization

Best Practices

Deployment Best Practices

Use namespaces for resource organization
Implement health checks (liveness/readiness probes)
Set resource requests and limits
Use labels for organization and selection
Implement proper security contexts

Configuration Best Practices

Use ConfigMaps for configuration data
Use Secrets for sensitive data
Implement proper RBAC rules
Use network policies for security
Enable audit logging

Security Best Practices

Run containers as non-root users
Use read-only root filesystems when possible
Implement pod security policies
Scan images for vulnerabilities
Use least-privilege principles

Performance Best Practices

Right-size resource requests and limits
Use node selectors and affinities appropriately
Implement proper monitoring and alerting
Use cluster autoscaling
Optimize pod density

Troubleshooting

Common Issues and Solutions

Pod Issues

ImagePullBackOff: Check image name and credentials
CrashLoopBackOff: Check logs and resource limits
Pending: Check resource availability and node selectors

Service Issues

Connection refused: Check service port mappings
DNS resolution: Verify service names and namespaces

Diagnostic Commands:

BASH

# Get detailed pod information
kubectl describe pod <pod-name>

# View cluster events
kubectl get events --sort-by='.lastTimestamp'

# Check resource usage
kubectl top nodes
kubectl top pods

Conclusion

Kubernetes provides a powerful platform for container orchestration, enabling organizations to manage containerized applications at scale. Understanding Kubernetes architecture, core concepts, and best practices is essential for modern application deployment and management.

In the next article, we'll explore container security best practices, covering how to secure your containerized applications and infrastructure.

Series

Container Series

Introduction to Containers

Docker Fundamentals

Container Orchestration with Kubernetes

Container Security Best Practices

Container Networking

Container Storage

Container Monitoring and Observability

Container Deployment Strategies

Container Scaling and Resource Management

Container Ecosystem Tools and Technologies

Share this article

You might also like