Kubernetes is powerful, but it’s also complex. This is my journey of trying to build a “simple” Kubernetes stack and the lessons learned along the way.

The Goal

I wanted to create a simple, maintainable Kubernetes setup for a small to medium-sized application. The requirements were:

  • Easy to understand and maintain
  • Cost-effective
  • Scalable when needed
  • Developer-friendly

What I Started With

Initial Stack

  • Kubernetes: EKS (AWS)
  • Ingress: NGINX Ingress Controller
  • Database: Managed PostgreSQL (RDS)
  • Monitoring: Prometheus + Grafana
  • Logging: ELK Stack
  • CI/CD: GitLab CI

The Reality Check

Complexity Crept In

What started as “simple” quickly became complex:

  1. Networking: VPCs, subnets, security groups, load balancers
  2. Storage: Persistent volumes, storage classes, backups
  3. Secrets Management: Kubernetes secrets, external secret operators
  4. Service Mesh: Considered Istio, then Linkerd, then nothing
  5. Observability: Multiple tools, dashboards, alerts

Cost Analysis

After 3 months:

  • EKS cluster: $73/month
  • Worker nodes: $150/month
  • Load balancer: $20/month
  • RDS: $100/month
  • Monitoring tools: $50/month
  • Total: ~$400/month

For a small application, this was overkill.

The Simplification Journey

Phase 1: Move to Managed Kubernetes

Switched to DigitalOcean Kubernetes:

  • Simpler setup
  • Lower costs (~$50/month for basic cluster)
  • Better developer experience
  • Still Kubernetes, but less complexity

Phase 2: Simplify Observability

Replaced Prometheus + Grafana + ELK with:

  • Datadog: All-in-one monitoring (free tier available)
  • Loki: Lightweight logging (replaces ELK)

Phase 3: Use Helm Charts

Standardized deployments with Helm:

# values.yaml
replicaCount: 2
image:
  repository: myapp
  tag: latest
service:
  type: ClusterIP
  port: 80
ingress:
  enabled: true
  hosts:
    - host: myapp.com

Phase 4: Infrastructure as Code

Everything in Terraform:

resource "digitalocean_kubernetes_cluster" "main" {
  name    = "myapp-cluster"
  region  = "nyc1"
  version = "1.29.0-do.0"
  
  node_pool {
    name       = "worker-pool"
    size       = "s-2vcpu-4gb"
    node_count = 2
  }
}

Lessons Learned

1. Start Simple, Scale When Needed

Don’t over-engineer from the start:

  • Start with basic Kubernetes
  • Add complexity only when you need it
  • Use managed services when possible

2. Cost Matters

For small applications:

  • Consider simpler alternatives (Docker Compose, Railway, Render)
  • Use managed Kubernetes (DigitalOcean, Linode) over cloud providers
  • Monitor costs from day one

3. Developer Experience

Make it easy for your team:

  • Use Helm charts for consistency
  • Document everything
  • Provide local development setup
  • Use tools like Skaffold or Tilt

4. Observability is Important, But…

You don’t need everything:

  • Start with basic logging
  • Add metrics when needed
  • Use managed solutions when possible

The Final Stack

Simplified Architecture

┌─────────────────┐
│   Load Balancer │
└────────┬────────┘
         │
┌────────▼────────┐
│  Ingress (NGINX)│
└────────┬────────┘
         │
┌────────▼────────┐
│  Kubernetes     │
│  - App Pods     │
│  - Redis        │
└────────┬────────┘
         │
┌────────▼────────┐
│  Managed DB     │
└─────────────────┘

Tools

  • Kubernetes: DigitalOcean Kubernetes
  • Ingress: NGINX Ingress Controller
  • Database: Managed PostgreSQL
  • Monitoring: Datadog (free tier)
  • CI/CD: GitHub Actions
  • IaC: Terraform

Monthly Cost: ~$150

When to Use Kubernetes

Use Kubernetes when:

  • You have multiple services
  • You need auto-scaling
  • You have a dedicated DevOps team
  • You’re running at scale

Consider alternatives when:

  • Single application
  • Small team
  • Limited budget
  • Simple requirements

Alternatives to Consider

For Small Applications

  1. Docker Compose: Simple, local development
  2. Railway: Deploy with git push
  3. Render: Managed platform
  4. Fly.io: Global deployment

For Medium Applications

  1. DigitalOcean App Platform: Managed PaaS
  2. Heroku: Classic PaaS
  3. AWS App Runner: Serverless containers

Conclusion

Kubernetes is powerful, but it’s not always the right choice. My “simple” stack taught me:

  1. Start simple: Don’t over-engineer
  2. Monitor costs: They add up quickly
  3. Use managed services: Save time and complexity
  4. Document everything: Future you will thank you
  5. Know when to simplify: Sometimes less is more

The best infrastructure is the one that:

  • Meets your needs
  • Stays within budget
  • Your team can maintain
  • Scales when needed

Sometimes, that’s not Kubernetes. And that’s okay.