Aug 9, 2024
node
Keys & TTLs Namespaced keys: app:domain:entity:id. Set TTLs per data volatility; use jitter to avoid thundering expirations. Version keys on schema changes to prevent stale reads. Stampede protection Use SETNX/lock around rebuild; short lock TTL with fallback. Serve stale-while-revalidate: return cached value, refresh asynchronously. Serialization & size Prefer JSON with bounded fields; compress only large blobs. Avoid massive lists/hashes; paginate or split keys. Operations Monitor hit rate, command latency, memory, evictions. Use connection pooling; set timeouts and retries with backoff. Cluster/replica for HA; read from replicas if consistency allows. Checklist Keys versioned; TTLs with jitter. Stampede controls in place. Metrics for hit/miss/latency/evictions; alerts configured.
ReadJul 28, 2024
php
Query hygiene Add composite indexes matching filters/order; avoid leading wildcards. Use EXPLAIN to verify index usage; watch for filesort/temp tables. Prefer keyset pagination over OFFSET for large tables. Config basics Set innodb_buffer_pool_size (50-70% RAM), innodb_log_file_size, innodb_flush_log_at_trx_commit=1 (durable) or 2 (faster). max_connections aligned with app pool size; avoid connection storms. Enable slow query log with sane threshold; sample for tuning. App considerations Reuse connections (pooling); avoid long transactions. Limit selected columns; cap payload sizes; avoid large unbounded IN lists. For read-heavy workloads, add replicas; route reads carefully. Checklist Indexes audited; EXPLAIN reviewed. Buffer pool sized; slow log enabled. Pagination and payloads bounded; connections pooled.
ReadJul 20, 2024
testing
Comprehensive testing is essential for reliable backend systems. Here’s a guide to effective testing strategies.
Testing Pyramid /\ / \ E2E Tests (Few) /____\ / \ Integration Tests (Some) /________\ / \ Unit Tests (Many) /____________\ 1. Unit Tests What to Test Individual functions Business logic Data transformations Edge cases Example describe('UserService', () => { it('should create user with valid data', () => { const userData = { email: '[email protected]', name: 'Test User' }; const user = userService.createUser(userData); expect(user.email).toBe('[email protected]'); expect(user.id).toBeDefined(); }); it('should throw error for duplicate email', () => { userService.createUser({ email: '[email protected]' }); expect(() => { userService.createUser({ email: '[email protected]' }); }).toThrow('Email already exists'); }); }); 2. Integration Tests What to Test API endpoints Database operations External service interactions Service integration Example describe('User API', () => { beforeEach(async () => { await db.migrate.latest(); }); afterEach(async () => { await db.migrate.rollback(); }); it('POST /api/users should create user', async () => { const response = await request(app) .post('/api/users') .send({ email: '[email protected]', name: 'Test User', password: 'password123' }) .expect(201); expect(response.body.email).toBe('[email protected]'); const user = await db('users').where({ email: '[email protected]' }).first(); expect(user).toBeDefined(); }); }); 3. End-to-End Tests What to Test Complete user workflows System integration Critical paths Example describe('User Registration Flow', () => { it('should complete full registration process', async () => { // Register user const registerResponse = await request(app) .post('/api/auth/register') .send({ email: '[email protected]', password: 'password123' }); expect(registerResponse.status).toBe(201); // Login const loginResponse = await request(app) .post('/api/auth/login') .send({ email: '[email protected]', password: 'password123' }); expect(loginResponse.status).toBe(200); expect(loginResponse.body.token).toBeDefined(); // Access protected route const profileResponse = await request(app) .get('/api/users/me') .set('Authorization', `Bearer ${loginResponse.body.token}`); expect(profileResponse.status).toBe(200); }); }); 4. Test Data Management Fixtures const userFixtures = { validUser: { email: '[email protected]', name: 'Test User', password: 'password123' }, adminUser: { email: '[email protected]', name: 'Admin', password: 'admin123', role: 'admin' } }; Factories function createUser(overrides = {}) { return { email: '[email protected]', name: 'Test User', password: 'password123', ...overrides }; } 5. Mocking External Services // Mock external API jest.mock('../services/paymentService', () => ({ processPayment: jest.fn().mockResolvedValue({ transactionId: 'tx_123', status: 'success' }) })); 6. Database Testing // Use test database const testDb = { host: process.env.TEST_DB_HOST, database: process.env.TEST_DB_NAME }; // Clean database between tests beforeEach(async () => { await db.raw('TRUNCATE TABLE users CASCADE'); }); 7. Test Coverage // Aim for high coverage // Focus on critical paths // Don't obsess over 100% // Use coverage tools // jest --coverage // nyc Best Practices Write tests first (TDD) Test behavior, not implementation Keep tests independent Use descriptive test names Test edge cases Mock external dependencies Clean up test data Run tests in CI/CD Maintain test code quality Review test failures Conclusion Effective testing requires:
...
ReadJul 7, 2024
node
Pooling Use a pool (pg/pgbouncer); size = (CPU * 2–4) per app instance; avoid per-request connections. For PgBouncer in transaction mode, avoid session features (temp tables, session prep statements). Query hygiene Parameterize queries; prevent plan cache thrash; set statement timeout. Add indexes; avoid wild % patterns; paginate with keyset when possible. Monitor slow queries; cap max rows returned; avoid huge JSON blobs. App settings Set statement_timeout, idle_in_transaction_session_timeout. Use prepared statements judiciously; for PgBouncer, prefer server-prepared off or use pgbouncer session mode. Pool instrumentation: queue wait time, checkout latency, timeouts. OS/DB basics Keep Postgres on same AZ/region; latency kills. Tune work_mem, shared_buffers, effective_cache_size appropriately (DB side). Use EXPLAIN (ANALYZE, BUFFERS) in staging for heavy queries. Checklist Pool sized and monitored; PgBouncer for many short connections. Query timeouts set; slow logs monitored. Key indexes present; pagination optimized. App-level metrics for pool wait, query latency, error rates.
ReadJun 30, 2024
frontend
This note condenses the DEV article “Mastering Frontend Performance: Harnessing the Power of Caching” into actionable steps for modern apps.
Why cache Reduce network and CPU cost for repeated data/computation. Improve perceived speed and resilience to flaky networks. Keep UIs responsive under load. Layers to combine HTTP caching: set Cache-Control, ETag, Last-Modified, stale-while-revalidate for API/static responses; prefer immutable, versioned assets. Client memoization: cache expensive computations/render data (useMemo, useCallback, memoized selectors). Data caching: use React Query/SWR/Apollo to dedupe fetches, retry, refetch on focus. Service worker (when appropriate): offline/near-edge caching for shell + static assets. React hook hygiene Memoize derived data: useMemo(() => heavyCompute(input), [input]). Memoize callbacks passed to children to avoid re-renders: useCallback(fn, deps). Keep props stable; avoid recreating objects/functions each render. HTTP cache playbook Static assets: long max-age + immutable on versioned filenames. APIs: choose strategy per route: idempotent reads: max-age/stale-while-revalidate with ETag. personalized or sensitive data: no-store. list endpoints: shorter max-age + revalidation. Prefer CDN edge caching; compress (Brotli) and serve modern formats (AVIF/WebP). UI checks No spinner longer than a couple of seconds; use skeletons and optimistic updates where safe. Avoid layout shift when cached data arrives—reserve space. Track Core Web Vitals (LCP/INP/CLS) and hit-rate for key caches. Quick checklist Versioned static assets + long-lived caching headers. API cache policy per route with ETag/stale-while-revalidate. React memoization for heavy work and stable callbacks. Data-layer cache (React Query/SWR) with sensible stale times + retries. RUM/CI dashboards watching Web Vitals and cache hit rates. Takeaway: Combine HTTP caching, client memoization, and data-layer caches to ship faster pages and keep them fast under real traffic.
ReadJun 22, 2024
go
Deadlines & retries Require client deadlines; enforce server-side context with grpc.DeadlineExceeded handling. Configure retry/backoff on idempotent calls; avoid retry storms with jitter + max attempts. Interceptors Unary/stream interceptors for auth, metrics (Prometheus), logging, and panic recovery. Use per-RPC circuit breakers and rate limits for critical dependencies. TLS & auth Enable TLS everywhere; prefer mTLS for internal services. Rotate certs automatically; watch expiry metrics. Add authz checks in interceptors; propagate identity via metadata. Resource protection Limit concurrent streams and max message sizes. Bounded worker pools for handlers performing heavy work. Tune keepalive to detect dead peers without flapping. Observability Metrics: latency, error codes, message sizes, active streams, retries. Traces: annotate methods, peer info, attempt counts; sample smartly. Logs: structured fields for method, code, duration, peer. Checklist Deadlines required; retries only for idempotent calls with backoff. Interceptors for auth/metrics/logging/recovery. TLS/mTLS enabled; cert rotation automated. Concurrency and message limits set; keepalive tuned.
ReadJun 15, 2024
java
Understanding Java class initialization order is crucial for avoiding subtle bugs. Here’s the real initialization sequence and common pitfalls.
Initialization Order Static Fields and Blocks public class Example { // 1. Static fields initialized first private static int staticField = initializeStatic(); // 2. Static blocks executed in order static { System.out.println("Static block 1"); } static { System.out.println("Static block 2"); } private static int initializeStatic() { System.out.println("Static field initialization"); return 1; } } Instance Fields and Constructors public class Example { // 3. Instance fields initialized private int instanceField = initializeInstance(); // 4. Instance initialization blocks { System.out.println("Instance block"); } // 5. Constructor executed last public Example() { System.out.println("Constructor"); } private int initializeInstance() { System.out.println("Instance field initialization"); return 1; } } Complete Initialization Sequence public class InitializationDemo { // Step 1: Static fields private static String staticField = "Static field"; // Step 2: Static blocks (in order) static { System.out.println("Static block 1"); } static { System.out.println("Static block 2"); } // Step 3: Instance fields private String instanceField = "Instance field"; // Step 4: Instance blocks { System.out.println("Instance block"); } // Step 5: Constructor public InitializationDemo() { System.out.println("Constructor"); } } Common Pitfalls 1. Forward References // Bad: Forward reference public class BadExample { private int value = getValue(); // Error: forward reference private int multiplier = 10; private int getValue() { return multiplier * 2; // multiplier might not be initialized } } // Good: Initialize in correct order public class GoodExample { private int multiplier = 10; private int value = getValue(); private int getValue() { return multiplier * 2; } } 2. Static vs Instance Initialization public class ConfusingExample { private static int staticCounter = 0; private int instanceCounter = 0; static { staticCounter++; // Cannot access instanceCounter here! } { instanceCounter++; staticCounter++; // Can access static } } 3. Inheritance Initialization class Parent { static { System.out.println("Parent static block"); } { System.out.println("Parent instance block"); } public Parent() { System.out.println("Parent constructor"); } } class Child extends Parent { static { System.out.println("Child static block"); } { System.out.println("Child instance block"); } public Child() { System.out.println("Child constructor"); } } // Output order: // Parent static block // Child static block // Parent instance block // Parent constructor // Child instance block // Child constructor Best Practices Initialize fields in logical order Avoid forward references Keep initialization blocks simple Use constructors for complex initialization Document initialization dependencies Conclusion Understanding initialization order helps you:
...
ReadMay 20, 2024
docker
Containerized systems have unique failure modes. Here’s how to identify and prevent common issues.
1. Resource Exhaustion Memory Limits # docker-compose.yml services: app: deploy: resources: limits: memory: 512M reservations: memory: 256M CPU Throttling services: app: deploy: resources: limits: cpus: '1.0' 2. Container Restart Loops Health Checks # Dockerfile HEALTHCHECK --interval=30s --timeout=3s --start-period=40s \ CMD curl -f http://localhost:8080/health || exit 1 Restart Policies services: app: restart: unless-stopped # Options: no, always, on-failure, unless-stopped 3. Network Issues Port Conflicts services: app: ports: - "8080:8080" # host:container DNS Resolution services: app: dns: - 8.8.8.8 - 8.8.4.4 4. Volume Mount Problems Permission Issues # Fix permissions RUN chown -R appuser:appuser /app USER appuser Volume Mounts services: app: volumes: - ./data:/app/data:ro # Read-only - cache:/app/cache 5. Image Layer Caching Optimize Dockerfile # Bad: Changes invalidate cache COPY . . RUN npm install # Good: Layer caching COPY package*.json ./ RUN npm install COPY . . 6. Log Management Log Rotation services: app: logging: driver: "json-file" options: max-size: "10m" max-file: "3" 7. Security Issues Non-Root User RUN useradd -m appuser USER appuser Secrets Management services: app: secrets: - db_password environment: DB_PASSWORD_FILE: /run/secrets/db_password Prevention Strategies Set resource limits Implement health checks Use proper restart policies Monitor container metrics Test failure scenarios Use orchestration tools (Kubernetes, Docker Swarm) Conclusion Prevent container failures by:
...
ReadApr 18, 2024
java
When it shines IO-heavy workloads with many concurrent requests. Simplifies thread-per-request code without callback hell. Great for blocking JDBC (with drivers that release threads), HTTP clients, and file IO. Caveats Avoid blocking operations that pin VTs (synchronized blocks, some native calls). Watch libraries that block on locks; prefer async-friendly drivers when possible. Pinning shows as carrier thread exhaustion; monitor. Usage Executors: Executors.newVirtualThreadPerTaskExecutor(). For servers (e.g., Spring): set spring.threads.virtual.enabled=true (Spring Boot 3.2+). Keep per-request timeouts; use structured concurrency where possible. Observability Metrics: carrier thread pool usage, VT creation rate, blocked/pinned threads. Profiling: use JDK Flight Recorder; check for pinning events. Checklist Dependencies vetted for blocking/pinning. Timeouts on all IO; circuit breakers still apply. Dashboards for carrier thread utilization and pinning. Load test before prod; compare throughput/latency vs platform threads.
Read