Backend

Client defaults Set Timeout on http.Client; set Transport with DialContext timeout (e.g., 3s), TLSHandshakeTimeout (3s), ResponseHeaderTimeout (5s), IdleConnTimeout (90s), MaxIdleConns/MaxIdleConnsPerHost. Retry only idempotent methods with backoff + jitter; cap attempts. Use context.WithTimeout per request; cancel on exit. Server defaults ReadHeaderTimeout (e.g., 5s) to mitigate slowloris. ReadTimeout/WriteTimeout to bound handler time (align with business SLAs). IdleTimeout to recycle idle connections; prefer HTTP/2 when available. Patterns Wrap handlers with middleware for deadline + logging when timeouts hit. For upstreams, expose metrics: connect latency, TLS handshake, TTFB, retries. Prefer connection re-use; avoid per-request clients. Checklist Timeouts set on both client and server. Retries limited to idempotent verbs with jitter. Connection pooling tuned; idle conns reused. Metrics for latency stages and timeouts.

Test design Define goals: latency budgets (p95/p99), error ceilings, throughput targets. Scenarios: ramping arrival rate, soak tests, spike tests; match production payloads. Include auth headers and realistic think time. k6 script skeleton import http from "k6/http"; import { check, sleep } from "k6"; export const options = { thresholds: { http_req_duration: ["p(95)<300"] }, scenarios: { api: { executor: "ramping-arrival-rate", startRate: 10, timeUnit: "1s", preAllocatedVUs: 50, maxVUs: 200, stages: [ { target: 100, duration: "3m" }, { target: 100, duration: "10m" }, { target: 0, duration: "2m" }, ]}, }, }; export default function () { const res = http.get("https://api.example.com/resource"); check(res, { "status 200": (r) => r.status === 200 }); sleep(1); } Run & observe Capture k6 summary + JSON output; feed to Grafana for trends. Correlate with Go metrics (pprof, Prometheus) to find CPU/alloc hot paths. Record build SHA; compare runs release over release. Checklist Scenarios match prod traffic shape. Thresholds tied to SLOs; tests fail on regressions. Service metrics/pprof captured during runs.

When to use CPU-bound tasks (crypto, compression, image/video processing) that would block event loop. Avoid for short, tiny tasks—overhead may outweigh benefit. Patterns Use a pool (e.g., piscina/workerpool) with bounded size; queue tasks. Pass data via Transferable when large buffers; avoid heavy serialization. Propagate cancellation/timeouts; surface errors to main thread. Observability Track queue length, task duration, worker utilization, crashes. Monitor event loop lag to confirm offload benefits. Safety Validate inputs in main thread; avoid untrusted code in workers. Cleanly shut down pool on SIGTERM; drain and close workers. Checklist CPU tasks isolated to workers; pool sized to cores. Transferables used for big buffers; timeouts set. Metrics for queue/utilization/errors in place.

Performance notes Fastify: schema-driven, AJV validation, low-overhead routing; typically better RPS and lower p99s. Express: mature ecosystem; middleware can add overhead; great for quick prototypes. Bench considerations Compare with same validation/parsing; disable unnecessary middleware in Express. Test under keep-alive and HTTP/1.1 & HTTP/2 where applicable. Measure event loop lag and heap; not just RPS. Migration tips Start new services with Fastify; for existing Express apps, migrate edge routes first. Replace middleware with hooks/plugins; map Express middlewares to Fastify equivalents. Validate payloads with JSON schema for speed and safety. Operational tips Use pino (Fastify default) for structured logs; avoid console log overhead. Keep plugin count minimal; watch async hooks cost. Load test with realistic payload sizes; profile hotspots before/after migration. Takeaway: Fastify wins on raw throughput and structured DX; Express still fine for smaller apps, but performance-critical services benefit from Fastify’s lower overhead.

Approaches Token bucket for burst+steady control; sliding window for fairness. Enforce at edge (gateway/ingress) plus app-level for per-tenant safety. Spring implementation Use filters/interceptors with Redis/Lua for atomic buckets. Key by tenant/user/IP; return 429 with Retry-After. Expose metrics per key and rule; alert on near-capacity. Considerations Separate auth failures from rate limits; avoid blocking login endpoints too aggressively. Keep rule configs dynamic; hot-reload from config store. Combine with circuit breakers/timeouts for upstream dependencies. Checklist Edge and app-level limits defined. Redis-based atomic counters/buckets with TTL. Metrics + logs for limit decisions; alerts in place.

Principles Keep goroutine count bounded; size pools to CPU/core and downstream QPS. Apply backpressure: bounded channels + select with default to shed load early. Context everywhere: cancel on timeouts/parent cancellation; close resources. Prefer immutability; minimize shared state. Use channels for coordination. Patterns Worker pool with buffered jobs; fan-out/fan-in via contexts. Rate limit with time.Ticker or golang.org/x/time/rate. Semaphore via buffered channel for limited resources (DB, disk, external API). Sync cheatsheet sync.Mutex for critical sections; avoid long hold times. sync.WaitGroup for bounded concurrent tasks; errgroup for cancellation on first error. sync.Map only for high-concurrency, write-light cases; prefer map+mutex otherwise. Instrument & guard pprof: net/http/pprof + CPU/mem profiles in staging under load. Trace blocking: GODEBUG=schedtrace=1000,scavtrace=1 when diagnosing. Metrics: goroutines, GC pause, allocations, queue depth, worker utilization. Checklist Context per request; timeouts set at ingress. Bounded goroutines; pools sized and observable. Backpressure on queues; drop/timeout strategy defined. pprof/metrics enabled in non-prod and behind auth in prod. Load tests for saturation behavior and graceful degradation.

Workers & balancing Define queue priorities; dedicate workers per queue (emails, webhooks, default). Use balance strategies (simple, auto) and cap max processes per supervisor. Reliability Set retry/backoff per job; push non-idempotent tasks carefully. Configure timeout and retry_after (keep retry_after > max job time). Use Redis with persistence; enable horizon:supervisors monitors. Observability Horizon dashboard: throughput, runtime, failures, retries. Alert on rising failures and long runtimes; log payload/context for failed jobs. Prune failed jobs with retention policy; send to DLQ when needed. Deployment Restart Horizon on deploy to pick up code; use horizon:terminate. Ensure supervisor/systemd restarts Horizon if it dies. Checklist Queues prioritized; supervisors sized. Retries/backoff and timeouts set; DLQ plan. Monitoring/alerts configured; failed job retention in place.

Metrics Use Micrometer + Prometheus: management.endpoints.web.exposure.include=prometheus,health,info. Add JVM+Tomcat/db pool meters; set percentiles for latencies. Create SLIs: request latency, error rate, saturation (threads/connections), GC pauses. Traces Spring Boot 3 ships with OTel starter: add spring-boot-starter-actuator + micrometer-tracing-bridge-otel + exporter (OTLP/Zipkin/Jaeger). Propagate headers (traceparent); ensure async executors use ContextPropagatingExecutor. Sample smartly: lower rates on noisy paths; raise for errors. Logs Use JSON layout; include traceId/spanId for correlation. Avoid verbose INFO in hot paths; keep payload size bounded. Dashboards & alerts Latency/error SLO dashboards per endpoint. DB pool saturation, thread pool queue depth, GC pause, heap used %, 5xx rate. Alerts on SLO burn rates; include exemplars linking metrics → traces → logs. Checklist Actuator endpoints secured and exposed only where needed. OTLP exporter configured; sampling tuned. Trace/log correlation verified in staging. Dashboards + alerts reviewed with oncall.

Core settings max.poll.interval.ms sized to processing time; max.poll.records to batch size. fetch.min.bytes/fetch.max.wait.ms to trade latency vs throughput. enable.auto.commit=false; commit sync/async after processing batch. Concurrency Prefer multiple consumer instances over massive max.poll.records. For CPU-bound steps, hand off to bounded executor; avoid blocking poll thread. Ordering & retries Keep partition affinity when ordering matters; use DLT for poison messages. Backoff with jitter on retries; limit attempts per message. Observability Metrics: lag per partition, commit latency, rebalances, processing time, error rates. Log offsets and partition for errors; trace batch sizes. Checklist Poll loop never blocks; work delegated to bounded pool. Commits after successful processing; DLT in place. Lag and rebalance metrics monitored.