Container Security Best Practices

Overview

Container security is critical for protecting applications and infrastructure in modern cloud-native environments. This article covers comprehensive security best practices for securing containers throughout their lifecycle, from image creation to runtime operations.

Container Security Fundamentals

The Shared Responsibility Model

Container security follows a shared responsibility model where security is distributed across multiple layers:

Infrastructure Layer

• Host OS security
• Kernel updates and patches
• Network security
• Physical security

Orchestrator Layer

• Kubernetes security
• RBAC configuration
• Network policies
• Admission controllers

Container Layer

• Image security
• Runtime security
• Process isolation
• Resource limits

Security Domains

Image Security

• Base image selection
• Vulnerability scanning
• Dependency management
• Supply chain security

Runtime Security

• Process monitoring
• Network security
• File integrity
• Behavior analysis

Orchestration Security

• Cluster hardening
• RBAC policies
• Network segmentation
• Secrets management

Image Security

Secure Base Images

Choose Trusted Base Images

• Use official images from verified publishers
• Select minimal base images (alpine, distroless)
• Verify image signatures
• Check for security advisories

Example of Secure Base Image Selection:

# Good: Specific version of official image
FROM nginx:1.21.6-alpine

# Better: Use digest for immutability
FROM nginx@sha256:abc123...

# Avoid: Latest tag (can change unexpectedly)
FROM nginx:latest

Image Scanning

Automated Scanning Tools

• Trivy: Open-source vulnerability scanner
• Clair: CoreOS vulnerability analysis tool
• Anchore: Full lifecycle container analysis
• Commercial tools: Twistlock, Aqua Security, Sysdig

CI/CD Integration:

# Scan image in CI pipeline
trivy image --exit-code 1 --severity HIGH,CRITICAL myapp:latest

# Scan Dockerfile
trivy config --exit-code 1 .

Image Building Security

Non-Root User:

# Create application user
RUN addgroup -g 1001 -S appgroup && \
    adduser -u 1001 -S appuser -G appgroup

# Switch to non-root user
USER appuser

Clean Up During Build:

# Clean package manager cache in same layer
RUN apt-get update && apt-get install -y \
    package1 \
    package2 \
    && rm -rf /var/lib/apt/lists/*

Supply Chain Security

Image Signing

• Sign images with tools like Notary or Cosign
• Verify signatures before deployment
• Implement policy enforcement

SBOM (Software Bill of Materials)

• Generate SBOMs for all images
• Track all components and dependencies
• Enable vulnerability tracking

Runtime Security

Container Runtime Configuration

Security Context:

apiVersion: v1
kind: Pod
metadata:
  name: secure-pod
spec:
  securityContext:
    runAsNonRoot: true
    runAsUser: 1000
    fsGroup: 2000
  containers:
  - name: app-container
    image: myapp:latest
    securityContext:
      readOnlyRootFilesystem: true
      allowPrivilegeEscalation: false
      capabilities:
        drop:
        - ALL
        add:
        - NET_BIND_SERVICE

Privileged Containers

• Avoid privileged containers unless absolutely necessary
• Use specific capabilities instead of privileged mode
• Implement admission controls to prevent privileged containers

Runtime Monitoring

Behavioral Analysis

• Monitor system calls
• Track file access patterns
• Detect anomalous network connections
• Log suspicious activities

Example Falco Rule:

- rule: Terminal shell in container
  desc: A shell was used as the entrypoint/executed in a container
  condition: spawned_process and container and shell_procs and proc.entrypoint=true
  output: A shell was spawned in a container with command (user=%user.name command=%proc.cmdline %container.info image=%k8s.pod.labels)
  priority: WARNING

Network Security

Network Policies:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-app-to-db
spec:
  podSelector:
    matchLabels:
      app: frontend
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: backend
    ports:
    - protocol: TCP
      port: 5432

Kubernetes Security

RBAC Best Practices

Principle of Least Privilege:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: production
  name: deployment-manager
rules:
- apiGroups: ["apps"]
  resources: ["deployments"]
  verbs: ["get", "list", "create", "update", "patch", "delete"]
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "list"]

Service Account Security:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: restricted-sa
  namespace: production
automountServiceAccountToken: false  # Disable token mounting

Admission Controllers

Pod Security Standards:

apiVersion: v1
kind: Namespace
metadata:
  name: development
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/audit: restricted
    pod-security.kubernetes.io/warn: restricted

Custom Admission Controllers:

• Implement policy enforcement
• Validate resource configurations
• Inject security controls
• Prevent misconfigurations

Secrets Management

Kubernetes Secrets Security:

apiVersion: v1
kind: Secret
metadata:
  name: secure-secret
type: Opaque
stringData:
  password: "secure-password"
---
apiVersion: v1
kind: Pod
metadata:
  name: secret-pod
spec:
  containers:
  - name: app
    image: myapp:latest
    env:
    - name: DB_PASSWORD
      valueFrom:
        secretKeyRef:
          name: secure-secret
          key: password

External Secrets Management:

• HashiCorp Vault: Enterprise secrets management
• AWS Secrets Manager: Cloud-native secrets
• Azure Key Vault: Microsoft cloud secrets
• GCP Secret Manager: Google Cloud secrets

Container Runtime Security

Container Isolation

User and Group IDs:

• Use non-root users
• Implement user namespaces
• Isolate UID/GID spaces

Namespace Security:

• Enable user namespaces
• Isolate PID, network, and IPC namespaces
• Prevent privilege escalation

Capability Management

Dropping Capabilities:

# Remove all capabilities and add only needed ones
# In Kubernetes:
securityContext:
  capabilities:
    drop:
    - ALL
    add:
    - NET_BIND_SERVICE  # Only bind to low ports if needed

Available Capabilities:

• CAP_NET_BIND_SERVICE: Bind to privileged ports
• CAP_SYS_TIME: Set system clock
• CAP_CHOWN: Change ownership of files
• CAP_FSETID: Don't override file SUID/SGID bits

Monitoring and Detection

Security Monitoring Tools

Runtime Security Platforms:

• Falco: Cloud-native runtime security
• Sysdig: Container forensics and monitoring
• Aqua Security: Container security platform
• Twistlock: Prisma Cloud security

Key Metrics to Monitor:

• Unauthorized network connections
• Suspicious process execution
• File system changes
• Privilege escalation attempts
• Container breakout attempts

Logging and Auditing

Kubernetes Audit Logs:

apiVersion: audit.k8s.io/v1
kind: Policy
rules:
- level: Metadata  # Log basic request metadata
- level: Request  # Log request body
- level: RequestResponse  # Log request and response
omitStages:
- RequestReceived

Container Runtime Logs:

• Monitor container logs
• Aggregate logs centrally
• Implement log analysis
• Set up alerting for anomalies

Compliance and Governance

Security Standards

Industry Standards:

• CIS Benchmarks: Center for Internet Security
• PCI DSS: Payment Card Industry standards
• SOC 2: Service Organization Control
• ISO 27001: Information security management

Kubernetes Security Standards:

• CIS Kubernetes Benchmark
• NSA/CISA Kubernetes Hardening Guide
• NIST SP 800-190 Application Container Security Guide

Policy Enforcement

Open Policy Agent (OPA):

package kubernetes.admission

deny[msg] {
    input.request.kind.kind == "Pod"
    input.request.object.spec.containers[_].securityContext.privileged == true
    msg := "privileged containers are not allowed"
}

Gatekeeper:

• Policy enforcement for Kubernetes
• Constraint templates
• Audit capabilities
• Admission control

Security Tools and Technologies

Image Analysis Tools

Static Analysis:

• Docker Scout: Docker's security analysis tool
• Snyk: Developer-first security platform
• WhiteSource: Continuous open source security

Runtime Security:

• Aqua Trivy: Open source security scanner
• Clair: Vulnerability static analysis
• Grype: Vulnerability scanner

Infrastructure Security

Infrastructure as Code Security:

• Terrascan: Security scanning for Terraform
• Checkov: Infrastructure security scanning
• KICS: Infrastructure code security analyzer

Common Security Mistakes

Image-Related Issues

• Using latest tags without verification
• Including unnecessary packages/tools
• Running as root user
• Not scanning for vulnerabilities

Runtime Issues

• Excessive privileges
• Insecure network configurations
• Improper resource limits
• Missing security context

Configuration Issues

• Weak RBAC policies
• Insecure default configurations
• Missing network policies
• Poor secrets management

Incident Response

Security Event Handling

Detection and Response:

1. Detection: Identify security events
2. Analysis: Investigate the incident
3. Containment: Isolate affected resources
4. Eradication: Remove threats
5. Recovery: Restore normal operations
6. Lessons Learned: Improve security

Container-Specific Response:

• Terminate compromised containers
• Quarantine affected images
• Update security policies
• Review access controls

Forensics

Container Forensics:

• Collect container logs
• Analyze image layers
• Examine network traffic
• Review configuration changes

Best Practices Summary

Build Time Security

• Use minimal base images
• Scan images for vulnerabilities
• Run as non-root user
• Remove unnecessary packages
• Sign images for integrity

Runtime Security

• Implement least privilege
• Use read-only filesystems
• Drop unnecessary capabilities
• Monitor runtime behavior
• Implement network policies

Orchestration Security

• Enable RBAC
• Use network policies
• Secure secrets management
• Implement admission controls
• Monitor cluster activity

Operational Security

• Regular security updates
• Continuous monitoring
• Security training
• Incident response planning
• Compliance auditing

Conclusion

Container security requires a comprehensive approach that addresses security at every stage of the container lifecycle. By implementing these best practices, organizations can significantly reduce their security risks while maintaining the agility and benefits of containerized applications.

In the next article, we'll explore container networking, covering how containers communicate with each other and the outside world.