Devops

Docker has become the standard for containerization, but running containers in production requires following best practices for security, performance, and reliability. This guide covers essential practices for production Docker deployments. Image optimization Use multi-stage builds Reduce final image size by using multi-stage builds: # Stage 1: Build FROM node:18-alpine AS builder WORKDIR /app COPY package*.json ./ RUN npm ci COPY . . RUN npm run build # Stage 2: Runtime FROM node:18-alpine WORKDIR /app RUN addgroup -g 1001 -S nodejs && \ adduser -S nodejs -u 1001 COPY --from=builder --chown=nodejs:nodejs /app/dist ./dist COPY --from=builder --chown=nodejs:nodejs /app/node_modules ./node_modules COPY --chown=nodejs:nodejs package*.json ./ USER nodejs EXPOSE 3000 CMD ["node", "dist/index.js"] Use minimal base images Prefer Alpine or distroless images: ...

Kubernetes is powerful, but it’s also complex. This is my journey of trying to build a “simple” Kubernetes stack and the lessons learned along the way. The Goal I wanted to create a simple, maintainable Kubernetes setup for a small to medium-sized application. The requirements were: Easy to understand and maintain Cost-effective Scalable when needed Developer-friendly What I Started With Initial Stack Kubernetes: EKS (AWS) Ingress: NGINX Ingress Controller Database: Managed PostgreSQL (RDS) Monitoring: Prometheus + Grafana Logging: ELK Stack CI/CD: GitLab CI The Reality Check Complexity Crept In What started as “simple” quickly became complex: ...

Deploying FastAPI applications to production on a VPS requires careful configuration. This step-by-step guide will walk you through the entire process. Prerequisites A VPS with Ubuntu 20.04 or later Domain name (optional but recommended) Basic knowledge of Linux commands Step 1: Server Setup Update System sudo apt update sudo apt upgrade -y Install Python and Dependencies sudo apt install python3.9 python3-pip python3-venv nginx supervisor -y Step 2: Create Application Directory mkdir -p /var/www/myapp cd /var/www/myapp Create Virtual Environment python3 -m venv venv source venv/bin/activate Step 3: Deploy Your Application Install Dependencies pip install fastapi uvicorn[standard] gunicorn Create Application File # main.py from fastapi import FastAPI app = FastAPI() @app.get("/") def read_root(): return {"Hello": "World"} @app.get("/health") def health_check(): return {"status": "healthy"} Step 4: Configure Gunicorn Create gunicorn_config.py: ...

Metrics Lead time, MTTR, change failure rate, deploy frequency. Stage timing (queue, build, test, deploy); flake rate; retry counts. Tracing & logs Trace pipeline executions with build SHA, branch, trigger source; annotate stage spans. Structured logs with status, duration, infra node; keep artifacts linked. Guardrails Quality gates (tests, lint, security scans) per PR; fail fast on criticals. Retry budget per job to avoid infinite flake loops. Rollback hooks + auto-stop on repeated failures. Ops Parallelize where safe; cache dependencies; pin tool versions. Alert on SLA breaches (queue time, total duration) and rising flake rates. Keep dashboards per repo/team; trend regressions release to release.

During incident Roles: incident commander, comms lead, ops/feature SMEs, scribe. Declare severity quickly; open shared channel/bridge; timestamp actions. Stabilize first: roll back, feature-flag off, scale up, or shed load. Runbooks & tooling Prebuilt runbooks per service: restart/rollback steps, dashboards, logs, feature flags. One-click access to dashboards (metrics, traces, logs), recent deploys, and toggles. Paging rules with escalation; avoid noisy alerts. Comms Single source of truth: incident doc; external status page if needed. Regular updates with impact, scope, mitigation, ETA. After incident Blameless postmortem; timeline, root causes, contributing factors. Action items with owners/deadlines; track to completion. Add tests/alerts/runbook updates; reduce time-to-detect and time-to-recover.

Elasticsearch is a powerful search and analytics engine, but optimizing it for production requires understanding indexing strategies, query patterns, and cluster configuration. This guide covers essential optimization techniques. Cluster architecture Node roles Configure nodes with specific roles: # Master node node.roles: [master] # Data node node.roles: [data] # Ingest node node.roles: [ingest] # Coordinating node (default) node.roles: [] # No specific role Shard strategy Primary shards: Set at index creation (cannot be changed) ...

Kubernetes provides several deployment strategies to ensure zero-downtime updates and safe rollouts of new application versions. Understanding these strategies is crucial for maintaining reliable production systems. Deployment strategy overview Kubernetes deployment strategies determine how new versions of your application replace old ones. The choice depends on: Risk tolerance: How critical is zero downtime? Traffic patterns: Can you route traffic to multiple versions? Rollback speed: How quickly can you revert if issues occur? Resource constraints: Can you run multiple versions simultaneously? Rolling update (default) The default Kubernetes deployment strategy gradually replaces old pods with new ones. ...

Containerized systems have unique failure modes. Here’s how to identify and prevent common issues. 1. Resource Exhaustion Memory Limits # docker-compose.yml services: app: deploy: resources: limits: memory: 512M reservations: memory: 256M CPU Throttling services: app: deploy: resources: limits: cpus: '1.0' 2. Container Restart Loops Health Checks # Dockerfile HEALTHCHECK --interval=30s --timeout=3s --start-period=40s \ CMD curl -f http://localhost:8080/health || exit 1 Restart Policies services: app: restart: unless-stopped # Options: no, always, on-failure, unless-stopped 3. Network Issues Port Conflicts services: app: ports: - "8080:8080" # host:container DNS Resolution services: app: dns: - 8.8.8.8 - 8.8.4.4 4. Volume Mount Problems Permission Issues # Fix permissions RUN chown -R appuser:appuser /app USER appuser Volume Mounts services: app: volumes: - ./data:/app/data:ro # Read-only - cache:/app/cache 5. Image Layer Caching Optimize Dockerfile # Bad: Changes invalidate cache COPY . . RUN npm install # Good: Layer caching COPY package*.json ./ RUN npm install COPY . . 6. Log Management Log Rotation services: app: logging: driver: "json-file" options: max-size: "10m" max-file: "3" 7. Security Issues Non-Root User RUN useradd -m appuser USER appuser Secrets Management services: app: secrets: - db_password environment: DB_PASSWORD_FILE: /run/secrets/db_password Prevention Strategies Set resource limits Implement health checks Use proper restart policies Monitor container metrics Test failure scenarios Use orchestration tools (Kubernetes, Docker Swarm) Conclusion Prevent container failures by: ...

Effective testing in CI/CD pipelines requires balancing speed, coverage, and reliability. This guide covers strategies to optimize your testing approach for continuous integration and deployment. Testing pyramid for CI/CD The testing pyramid applies to CI/CD with some modifications: /\ / \ E2E Tests (few, slow, expensive) /____\ / \ Integration Tests (some, moderate) /________\ / \ Unit Tests (many, fast, cheap) /____________\ CI/CD testing layers Unit tests: Fast, run on every commit Integration tests: Moderate speed, run on PRs E2E tests: Slow, run on main branch or scheduled Performance tests: Run periodically or on release candidates Pipeline testing strategy Stage 1: Pre-commit (local) Run fast checks before committing: ...