Category Go
Scaling Go Websocket Servers to Millions of Concurrent Connections
The memory math nobody does up front
Serialization Benchmarks: Go JSON vs. Protobuf vs. FlatBuffers
Why serialization keeps topping your pprof
Multi-tenant Database Routing and Connection Pooling in Go microservices
The simplest multi-tenant design in Go is also the most dangerous: one *sql.DB per tenant, looked up from a map at request time. It works beautifully for ten tenants and...
Go Scheduler Deep Dive: Work-Stealing, Preemption, and OS Thread Matching
Go’s concurrency story rests on one runtime component most engineers never look at: the scheduler. If you have ever seen a service with 300,000 goroutines and 12 cores stay responsive,...
Go Generics under the Hood: Performance Costs and Monomorphization
When Go 1.18 shipped generics, the design had to thread a needle: be fast like C++ templates without their binary bloat, and be ergonomic without Java’s boxing tax. The answer...
Detecting and Debugging Goroutine Leaks in Production Go Applications
A goroutine leak is the sneakiest way to lose memory in Go, because the memory is not garbage: it is a live stack, referenced by a blocked goroutine, unreachable to...
Go Concurrency Patterns: Channels vs. Mutexes in High-Performance Applications
The advice “share memory by communicating” has done more damage than good. It was written for clarity, not throughput, and teams have shipped channel-based counters and caches that run 5-10x...
Building a High-Performance JSON Parser using Go Code Generation
Every few months someone on the team proposes “we should just use a faster JSON library” and benchmarks encoding/json against a dropped-in replacement, sees a 2x, and ships it. That’s...
Go Compiler Internals: Escape Analysis, Function Inlining, and devirtualization
The difference between a handler that does 40 allocations per request and one that does four is usually not your data structures. It is whether the compiler managed to keep...
Designing a Low-Latency API Gateway Routing Tree in Go using Radix Trees
An API gateway sits in front of every upstream call, so its router is the hottest code path in the fleet. A linear scan over registered patterns is O(N) in...
Go Garbage Collector Tuning: GOGC and GOMEMLIMIT Deep Dive
A 2GB Kubernetes pod running a Go service with default GOGC=100 has a GC that will happily let the heap double before it starts collecting. That is by design: the...
Structured Logging Performance: Custom zap and slog Configurations in Go
Logging is the last thing anyone profiles and the first thing that sinks a latency budget. A service doing 10k req/s with two or three log lines per request is...
Writing eBPF Programs in C and Loading Them with Go: System Telemetry at Scale
Our Redis client was pushing a sustained 8% CPU on every API node, and pprof could not tell us why. The CPU profile blamed runtime.futex, which is not an answer;...
Context Propagation in Go: Building Distributed Tracing Middleware
We spent a week hunting a checkout outage the tracing dashboards could not explain. The gateway started a span, then the trace went dark: payment logged an empty trace ID,...
Building a Custom Key-Value Store from Scratch in Go using LSM-Trees
Why the write path hates B-trees
Custom Network Programming with Epoll and Kqueue in Go
Go already has a netpoll – know why you would reimplement it
Understanding Go Interface Devirtualization and its Performance Impact
The real cost of an interface call
Structured Concurrency in Go: Managing Subtask Lifecycles with golang.org/x/sync/errgroup
sync.WaitGroup answers one question: “is everyone done?” It cannot tell you someone failed, and it certainly cannot stop the others when one does. For fan-out over shards, partitions, or upstreams...
gRPC Client-side Load Balancing in Go: Implementing Custom Resolvers
gRPC is built on HTTP/2, and HTTP/2 wants a few long-lived connections, not many short ones. Put a classic L4 or L7 load balancer in front and you have re-introduced...
Writing a High-Throughput Custom DNS Resolver in Go
A crawler or an ingress resolving thousands of distinct hostnames per second exposes a gap in the standard library. The folklore is that net.LookupHost blocks OS threads via cgo —...
Profiling Go Memory and CPU Bottlenecks with `go tool pprof` and `trace`
Guessing where CPU goes is how services die slowly. Go ships a world-class profiler; the real skill is using it correctly, because each profile type answers a different question and...
Designing a High-Throughput Worker Pool in Go for API Ingestion
Goroutines are cheap — a couple of KiB of stack, sub-microsecond spawn — so teams over-rotated from “thread per request” to “goroutine per request” and hit a wall. Unbounded concurrency...
High-Performance I/O in Go: Custom io.Reader and io.Writer Optimizations
A read syscall costs a few hundred nanoseconds to a few microseconds depending on kernel and storage. That sounds cheap until you multiply by bytes. A 1 GiB log file...
Implementing Graceful Shutdown in Distributed Go Microservices
Kubernetes terminates pods the same way every time: it sends SIGTERM, waits terminationGracePeriodSeconds (30 by default), then sends SIGKILL. If your service treats SIGTERM as “start dying right now”, every...
Understanding the Go Memory Allocator: Arenas, Spans, and MCaches
Every object you allocate passes through the runtime allocator, and in a latency-sensitive Go service it is quietly responsible for more of your p99 than almost anything else. It decides...
Implementing Lock-Free Concurrent Data Structures in Go using `sync/atomic`
A shared index-update queue on one of our workers was the wall in every profile: sixteen goroutines contending on a single mutex, and the flame graph showing sync.(*Mutex).Lock as a...
Zero-Allocation Parsing and Serialization in High-Throughput Go Services
Our ingest path parses text records off a network buffer and indexes them. At 1.2M events/s, the parser was the hottest function in the service – not because parsing is...
Building a Replicated State Machine in Go using the Raft Consensus Algorithm
Every gateway node in our fleet needs the same piece of truth at boot: the shard-to-node mapping, the config revision, the list of revoked keys. Before Raft, that truth lived...
Go `sync.Pool` Mechanics: Minimizing Allocation Pressure and GC Interactions
Our request marshaling path allocated a handful of small buffers per request – fine at 1k req/s, a garbage problem at 20k. sync.Pool looked like the obvious fix, and it...
Building an API Gateway Rate Limiter in Go using the Token Bucket Algorithm
Our gateway terminates traffic for public APIs, and the first thing every unauthenticated request hits is a per-API-key limiter. With 50k active keys and a sustained 100k req/s, the limiting...
Optimizing Struct Layouts in Go to Reduce Memory Padding and Cache Misses
Eight bytes per struct. That is what a careless field order cost us in a 1M-record in-memory index we cache in a service that answers every lookup request. Eight bytes...
Implementing Distributed Circuit Breakers in Go without External Dependencies
We ran a checkout pipeline that fronted a legacy payments gateway. The gateway had a 3-second timeout and, once every few weeks, it would wedge for ten minutes. The client...
Using unsafe.Pointer in Go for Zero-Copy Struct Conversion and System Calls
Every packet that crosses our gateway gets decoded into a struct. The naive path – encoding/binary reading seven fields one at a time, on every packet – is slow in...
Go Slice Growth Mechanics: Benchmarking Pre-allocation Strategies
The append loop that grows a slice one element at a time is the most common allocation smell we find in Go production code. It is not wrong, exactly –...
Category Performance
Speculative Decoding: Accelerating LLM Inference in Production
Decode is memory-bandwidth-bound, and that single fact is why speculative decoding is the cheapest throughput win an inference team can spend a week on. Every autoregressive step loads the entire...
Scaling Go Websocket Servers to Millions of Concurrent Connections
The memory math nobody does up front
Serialization Benchmarks: Go JSON vs. Protobuf vs. FlatBuffers
Why serialization keeps topping your pprof
High-Performance JSON Serialization: Benchmarking Oj, Blueprinter, and Fast JSON API
The last place I look when an API endpoint is slow is the serializer, and that’s a mistake. On a typical list endpoint the serializer is where most of the...
Go Generics under the Hood: Performance Costs and Monomorphization
When Go 1.18 shipped generics, the design had to thread a needle: be fast like C++ templates without their binary bloat, and be ergonomic without Java’s boxing tax. The answer...
Detecting and Debugging Goroutine Leaks in Production Go Applications
A goroutine leak is the sneakiest way to lose memory in Go, because the memory is not garbage: it is a live stack, referenced by a blocked goroutine, unreachable to...
Go Concurrency Patterns: Channels vs. Mutexes in High-Performance Applications
The advice “share memory by communicating” has done more damage than good. It was written for clarity, not throughput, and teams have shipped channel-based counters and caches that run 5-10x...
Building a High-Performance JSON Parser using Go Code Generation
Every few months someone on the team proposes “we should just use a faster JSON library” and benchmarks encoding/json against a dropped-in replacement, sees a 2x, and ships it. That’s...
eBPF-Based Telemetry for LLM Inference Endpoints: Profiling CPU/GPU Latencies
If you serve LLM inference at any real scale, your standard monitoring stack has a blind spot exactly where your tokens actually spend time: the kernel. Prometheus exporters and Datadog...
Rails Solid Cache: Dropping Redis for Database-Backed Caching
The Redis cache bill is a weird tax. You’re paying premium RAM prices for data that is, by definition, disposable — regenerate it and nothing breaks. When our cache footprint...
Scaling Sidekiq: Advanced Job Deduplication and Multi-tenant Rate Limiting
Sidekiq’s contract is “push work, forget it,” but the moment you have an event bus that emits per-entity changes, you discover how literally Sidekiq honors that contract. Ten indexing jobs...
Go Garbage Collector Tuning: GOGC and GOMEMLIMIT Deep Dive
A 2GB Kubernetes pod running a Go service with default GOGC=100 has a GC that will happily let the heap double before it starts collecting. That is by design: the...
Structured Logging Performance: Custom zap and slog Configurations in Go
Logging is the last thing anyone profiles and the first thing that sinks a latency budget. A service doing 10k req/s with two or three log lines per request is...
KV-Cache Eviction and PagedAttention in High-Throughput LLM Serving
The first time we load-tested vLLM for a multi-tenant embedding-and-completion service, we watched nvidia-smi and assumed something was leaking. Four Mistral-7B instances on two A100s, steady 512-token requests, and the...
Debugging Solid Queue Deadlocks: Transaction Isolation Levels and Row Locks
Rails 8 shipped on November 7, 2024 with Solid Queue as the default background-job adapter — a Postgres-backed queue that fits the “one less database” story. We migrated a roughly...
Ruby YJIT Memory Footprint: Optimization Strategies for Kubernetes Deployments
Since Rails 7.2 made YJIT the default JIT in August 2024, most Rails apps picked up a 15-25% throughput bump for free. The bill shows up on Kubernetes in a...
Unlocking Speed: Rails Performance Tuning with Ruby 3 YJIT
In Ruby 3.3, YJIT is production-ready. Shopify’s numbers, the Optcarrot benchmark, and the Rails core team all point the same direction: roughly 15–25% latency improvement on real Rails workloads, for...
ActiveRecord Eager Loading Pitfalls: Subqueries vs. IN Clauses
includes is the most convenient lie in ActiveRecord. You write one method call, and ActiveRecord decides for you whether to emit two queries joined by an IN clause (preload) or...
ActiveJob Scaling: Thread Pools, DB Pool Sizes, and PostgreSQL Contention
The incident is boring and predictable: someone bumps Sidekiq concurrency from 10 to 25, and within an hour pager duty is looking at ActiveRecord::ConnectionTimeoutError across every web and worker process....
Decoupled Elasticsearch Indexing: Scaling ActiveRecord Callbacks via Sidekiq
The pattern starts innocently:
Custom Network Programming with Epoll and Kqueue in Go
Go already has a netpoll – know why you would reimplement it
Scaling ActionCable to 10k Active WebSockets with Redis Pub/Sub
WebSockets look like the same stack as HTTP and are, in every way that matters, the opposite. An HTTP request occupies a thread for a hundred milliseconds and leaves. A...
Understanding Go Interface Devirtualization and its Performance Impact
The real cost of an interface call
Profiling Go Memory and CPU Bottlenecks with `go tool pprof` and `trace`
Guessing where CPU goes is how services die slowly. Go ships a world-class profiler; the real skill is using it correctly, because each profile type answers a different question and...
High-Performance I/O in Go: Custom io.Reader and io.Writer Optimizations
A read syscall costs a few hundred nanoseconds to a few microseconds depending on kernel and storage. That sounds cheap until you multiply by bytes. A 1 GiB log file...
Safe Multi-threading inside Rails Controllers with Concurrent Ruby
The classic slow controller is a dashboard that aggregates three independent services. Balance from billing, usage from the analytics service, unread count from notifications. Called sequentially, a request that should...
Understanding the Go Memory Allocator: Arenas, Spans, and MCaches
Every object you allocate passes through the runtime allocator, and in a latency-sensitive Go service it is quietly responsible for more of your p99 than almost anything else. It decides...
Writing Custom Rack Middleware for Prometheus and OpenTelemetry instrumentation
Rails logs measure the controller. They don’t measure the queue, the auth middleware, the rate limiter, or the request as the client actually experienced it. That gap is exactly where...
Zero-Allocation Parsing and Serialization in High-Throughput Go Services
Our ingest path parses text records off a network buffer and indexes them. At 1.2M events/s, the parser was the hottest function in the service – not because parsing is...
Go `sync.Pool` Mechanics: Minimizing Allocation Pressure and GC Interactions
Our request marshaling path allocated a handful of small buffers per request – fine at 1k req/s, a garbage problem at 20k. sync.Pool looked like the obvious fix, and it...
Optimizing Struct Layouts in Go to Reduce Memory Padding and Cache Misses
Eight bytes per struct. That is what a careless field order cost us in a 1M-record in-memory index we cache in a service that answers every lookup request. Eight bytes...
Using ActiveSupport::Notifications for Advanced App Tracing and Telemetry
Rails instruments almost every meaningful point in the request lifecycle already: SQL execution, view and partial rendering, cache reads, and full controller actions. It publishes all of it on a...
Go Slice Growth Mechanics: Benchmarking Pre-allocation Strategies
The append loop that grows a slice one element at a time is the most common allocation smell we find in Go production code. It is not wrong, exactly –...
Category Optimization
Serialization Benchmarks: Go JSON vs. Protobuf vs. FlatBuffers
Why serialization keeps topping your pprof
Prompt Caching Strategies to Reduce LLM Latency and API Costs
Every RAG or agent application I’ve reviewed re-sends the same few thousand tokens on every request: the system prompt, tool schemas, a hundred pages of policy docs, the conversation so...
Building a High-Performance JSON Parser using Go Code Generation
Every few months someone on the team proposes “we should just use a faster JSON library” and benchmarks encoding/json against a dropped-in replacement, sees a 2x, and ships it. That’s...
Practical Guide to Model Distillation: From DeepSeek-R1 to edge-ready SLMs
When DeepSeek-R1 landed in January 2025, it proved that chain-of-thought plus reinforcement learning could push open models to near-frontier reasoning on math and code. It also proved something less convenient:...
Structured Logging Performance: Custom zap and slog Configurations in Go
Logging is the last thing anyone profiles and the first thing that sinks a latency budget. A service doing 10k req/s with two or three log lines per request is...
Quantization Deep Dive: Math of GGUF, AWQ, and BitsAndBytes Double Quantization
Quantization is how a 7B model becomes a weekend desktop project instead of a dedicated server: fp16 weights are 14 GB, int8 is 7 GB, 4-bit is about 3.5 GB....
High-Performance I/O in Go: Custom io.Reader and io.Writer Optimizations
A read syscall costs a few hundred nanoseconds to a few microseconds depending on kernel and storage. That sounds cheap until you multiply by bytes. A 1 GiB log file...
Zero-Allocation Parsing and Serialization in High-Throughput Go Services
Our ingest path parses text records off a network buffer and indexes them. At 1.2M events/s, the parser was the hottest function in the service – not because parsing is...
Go `sync.Pool` Mechanics: Minimizing Allocation Pressure and GC Interactions
Our request marshaling path allocated a handful of small buffers per request – fine at 1k req/s, a garbage problem at 20k. sync.Pool looked like the obvious fix, and it...
Optimizing Struct Layouts in Go to Reduce Memory Padding and Cache Misses
Eight bytes per struct. That is what a careless field order cost us in a 1M-record in-memory index we cache in a service that answers every lookup request. Eight bytes...
Go Slice Growth Mechanics: Benchmarking Pre-allocation Strategies
The append loop that grows a slice one element at a time is the most common allocation smell we find in Go production code. It is not wrong, exactly –...
Category Ruby on Rails
Real-Time Event Streaming in Rails using ActionController::Live and Server-Sent Events (SSE)
“Real-time dashboard” is the feature that silently becomes an architecture decision. The naive version polls a JSON endpoint every second: one client is nothing, but 500 dashboards at 1 req/s...
High-Performance JSON Serialization: Benchmarking Oj, Blueprinter, and Fast JSON API
The last place I look when an API endpoint is slow is the serializer, and that’s a mistake. On a typical list endpoint the serializer is where most of the...
The Rails 8 Solid Suite: Moving to a Zero-Dependency Stack
Rails 8 (November 2024) did something bold: it made the database the default home for everything that used to need Redis. Solid Queue for background jobs, Solid Cache for the...
Rails Solid Cable: Redis-free WebSockets at Scale
ActionCable’s Redis dependency always annoyed me, and not for the reasons people usually cite. The latency argument against database pub/sub is mostly wrong at small scale — the real cost...
Rails Solid Cache: Dropping Redis for Database-Backed Caching
The Redis cache bill is a weird tax. You’re paying premium RAM prices for data that is, by definition, disposable — regenerate it and nothing breaks. When our cache footprint...
Scaling Sidekiq: Advanced Job Deduplication and Multi-tenant Rate Limiting
Sidekiq’s contract is “push work, forget it,” but the moment you have an event bus that emits per-entity changes, you discover how literally Sidekiq honors that contract. Ten indexing jobs...
Propshaft Asset Pipeline: A Modern, Light Replacement for Sprockets
Sprockets was designed for the Rails era where the framework compiled CoffeeScript, SCSS, and ERB-templated JavaScript on the fly. That era is over. Modern browsers ship ES modules and CSS...
Custom Database Adapters and Connection Pools in Rails 7.2
We run a 12-server Puma fleet against a Postgres primary and two read replicas. For a long time our read/write splitting was manual: a sidecar script rewrote database.yml on deploy...
Debugging Solid Queue Deadlocks: Transaction Isolation Levels and Row Locks
Rails 8 shipped on November 7, 2024 with Solid Queue as the default background-job adapter — a Postgres-backed queue that fits the “one less database” story. We migrated a roughly...
Ruby YJIT Memory Footprint: Optimization Strategies for Kubernetes Deployments
Since Rails 7.2 made YJIT the default JIT in August 2024, most Rails apps picked up a 15-25% throughput bump for free. The bill shows up on Kubernetes in a...
Deploying Rails at Scale with Kamal, Docker, and Tailscale
The platform bill grows linearly with a problem you did not choose: a few years of real traffic later, the Heroku/Render invoices looked like a third engineering salary, and Kubernetes...
Unlocking Speed: Rails Performance Tuning with Ruby 3 YJIT
In Ruby 3.3, YJIT is production-ready. Shopify’s numbers, the Optcarrot benchmark, and the Rails core team all point the same direction: roughly 15–25% latency improvement on real Rails workloads, for...
ActiveRecord Eager Loading Pitfalls: Subqueries vs. IN Clauses
includes is the most convenient lie in ActiveRecord. You write one method call, and ActiveRecord decides for you whether to emit two queries joined by an IN clause (preload) or...
ActiveJob Scaling: Thread Pools, DB Pool Sizes, and PostgreSQL Contention
The incident is boring and predictable: someone bumps Sidekiq concurrency from 10 to 25, and within an hour pager duty is looking at ActiveRecord::ConnectionTimeoutError across every web and worker process....
Implementing Idempotent API Endpoints in Rails using Redis Lock and Request Signatures
A client POSTs a payment, the connection drops mid-response, and the client retries. If your endpoint is not idempotent, the second request charges the card twice. At roughly 3M mutating...
Decoupled Elasticsearch Indexing: Scaling ActiveRecord Callbacks via Sidekiq
The pattern starts innocently:
Building Real-Time Collaboration Canvas using Hotwire and Turbo Streams
A canvas is a bad reason to go SPA
Testing Complex Race Conditions and Concurrency in RSpec
The trap: a “concurrent” test that never races
Hardening Rails APIs: Multi-tiered Rate Limiting with Rack::Attack and Redis
Rate limiting is a liveness problem
Multi-Tenant SaaS Architectures: PostgreSQL Schemas vs. Row-Level Security in Rails
Two ways to isolate tenants in one database
Scaling ActionCable to 10k Active WebSockets with Redis Pub/Sub
WebSockets look like the same stack as HTTP and are, in every way that matters, the opposite. An HTTP request occupies a thread for a hundred milliseconds and leaves. A...
Hardening Rails Security: Implementing Strict Content Security Policy (CSP) and SRI Headers
Browser XSS filters were always a poor backstop, and they’re not even that anymore. The real defense is telling the browser exactly what it’s allowed to execute, and Content Security...
Safe Multi-threading inside Rails Controllers with Concurrent Ruby
The classic slow controller is a dashboard that aggregates three independent services. Balance from billing, usage from the analytics service, unread count from notifications. Called sequentially, a request that should...
Database Sharding in Rails: Implementing Multi-Database Architectures
There’s a point in every growing service where the primary database becomes the bottleneck, and it’s not about reads. Read replicas solved reads. The pain is writes: one node, one...
Writing Custom Rack Middleware for Prometheus and OpenTelemetry instrumentation
Rails logs measure the controller. They don’t measure the queue, the auth middleware, the rate limiter, or the request as the client actually experienced it. That gap is exactly where...
Zero-Downtime Database Migrations in Rails: Guarding Production with Strong Migrations
The migration that nearly took us down wasn’t a clever one. It was add_index :orders, :customer_id, run at 2pm against a table with 40 million rows. Postgres holds a SHARE...
ActiveRecord Validations vs. Database Constraints: Finding the Optimal Balance
validates_uniqueness_of is not a data-integrity mechanism. It is a user-experience mechanism, and treating it as anything more is how you end up with duplicate invoices. The failure mode is embarrassingly...
Using ActiveSupport::Notifications for Advanced App Tracing and Telemetry
Rails instruments almost every meaningful point in the request lifecycle already: SQL execution, view and partial rendering, cache reads, and full controller actions. It publishes all of it on a...
Category Telemetry
Writing eBPF Programs in C and Loading Them with Go: System Telemetry at Scale
Our Redis client was pushing a sustained 8% CPU on every API node, and pprof could not tell us why. The CPU profile blamed runtime.futex, which is not an answer;...
Context Propagation in Go: Building Distributed Tracing Middleware
We spent a week hunting a checkout outage the tracing dashboards could not explain. The gateway started a span, then the trace went dark: payment logged an empty trace ID,...
Profiling Go Memory and CPU Bottlenecks with `go tool pprof` and `trace`
Guessing where CPU goes is how services die slowly. Go ships a world-class profiler; the real skill is using it correctly, because each profile type answers a different question and...
Writing Custom Rack Middleware for Prometheus and OpenTelemetry instrumentation
Rails logs measure the controller. They don’t measure the queue, the auth middleware, the rate limiter, or the request as the client actually experienced it. That gap is exactly where...
Using ActiveSupport::Notifications for Advanced App Tracing and Telemetry
Rails instruments almost every meaningful point in the request lifecycle already: SQL execution, view and partial rendering, cache reads, and full controller actions. It publishes all of it on a...
Category Language
Go Generics under the Hood: Performance Costs and Monomorphization
When Go 1.18 shipped generics, the design had to thread a needle: be fast like C++ templates without their binary bloat, and be ergonomic without Java’s boxing tax. The answer...
Go Compiler Internals: Escape Analysis, Function Inlining, and devirtualization
The difference between a handler that does 40 allocations per request and one that does four is usually not your data structures. It is whether the compiler managed to keep...
Understanding Go Interface Devirtualization and its Performance Impact
The real cost of an interface call
Using unsafe.Pointer in Go for Zero-Copy Struct Conversion and System Calls
Every packet that crosses our gateway gets decoded into a struct. The naive path – encoding/binary reading seven fields one at a time, on every packet – is slow in...
Category Systems
Multi-tenant Database Routing and Connection Pooling in Go microservices
The simplest multi-tenant design in Go is also the most dangerous: one *sql.DB per tenant, looked up from a map at request time. It works beautifully for ten tenants and...
Go Scheduler Deep Dive: Work-Stealing, Preemption, and OS Thread Matching
Go’s concurrency story rests on one runtime component most engineers never look at: the scheduler. If you have ever seen a service with 300,000 goroutines and 12 cores stay responsive,...
Go Compiler Internals: Escape Analysis, Function Inlining, and devirtualization
The difference between a handler that does 40 allocations per request and one that does four is usually not your data structures. It is whether the compiler managed to keep...
Designing a Low-Latency API Gateway Routing Tree in Go using Radix Trees
An API gateway sits in front of every upstream call, so its router is the hottest code path in the fleet. A linear scan over registered patterns is O(N) in...
Go Garbage Collector Tuning: GOGC and GOMEMLIMIT Deep Dive
A 2GB Kubernetes pod running a Go service with default GOGC=100 has a GC that will happily let the heap double before it starts collecting. That is by design: the...
Writing eBPF Programs in C and Loading Them with Go: System Telemetry at Scale
Our Redis client was pushing a sustained 8% CPU on every API node, and pprof could not tell us why. The CPU profile blamed runtime.futex, which is not an answer;...
Building a Custom Key-Value Store from Scratch in Go using LSM-Trees
Why the write path hates B-trees
Custom Network Programming with Epoll and Kqueue in Go
Go already has a netpoll – know why you would reimplement it
Implementing Graceful Shutdown in Distributed Go Microservices
Kubernetes terminates pods the same way every time: it sends SIGTERM, waits terminationGracePeriodSeconds (30 by default), then sends SIGKILL. If your service treats SIGTERM as “start dying right now”, every...
Understanding the Go Memory Allocator: Arenas, Spans, and MCaches
Every object you allocate passes through the runtime allocator, and in a latency-sensitive Go service it is quietly responsible for more of your p99 than almost anything else. It decides...
Implementing Lock-Free Concurrent Data Structures in Go using `sync/atomic`
A shared index-update queue on one of our workers was the wall in every profile: sixteen goroutines contending on a single mutex, and the flame graph showing sync.(*Mutex).Lock as a...
Building a Replicated State Machine in Go using the Raft Consensus Algorithm
Every gateway node in our fleet needs the same piece of truth at boot: the shard-to-node mapping, the config revision, the list of revoked keys. Before Raft, that truth lived...
Implementing Distributed Circuit Breakers in Go without External Dependencies
We ran a checkout pipeline that fronted a legacy payments gateway. The gateway had a 3-second timeout and, once every few weeks, it would wedge for ten minutes. The client...
Using unsafe.Pointer in Go for Zero-Copy Struct Conversion and System Calls
Every packet that crosses our gateway gets decoded into a struct. The naive path – encoding/binary reading seven fields one at a time, on every packet – is slow in...
Category Concurrency
Real-Time Event Streaming in Rails using ActionController::Live and Server-Sent Events (SSE)
“Real-time dashboard” is the feature that silently becomes an architecture decision. The naive version polls a JSON endpoint every second: one client is nothing, but 500 dashboards at 1 req/s...
Go Scheduler Deep Dive: Work-Stealing, Preemption, and OS Thread Matching
Go’s concurrency story rests on one runtime component most engineers never look at: the scheduler. If you have ever seen a service with 300,000 goroutines and 12 cores stay responsive,...
Detecting and Debugging Goroutine Leaks in Production Go Applications
A goroutine leak is the sneakiest way to lose memory in Go, because the memory is not garbage: it is a live stack, referenced by a blocked goroutine, unreachable to...
Go Concurrency Patterns: Channels vs. Mutexes in High-Performance Applications
The advice “share memory by communicating” has done more damage than good. It was written for clarity, not throughput, and teams have shipped channel-based counters and caches that run 5-10x...
Context Propagation in Go: Building Distributed Tracing Middleware
We spent a week hunting a checkout outage the tracing dashboards could not explain. The gateway started a span, then the trace went dark: payment logged an empty trace ID,...
Testing Complex Race Conditions and Concurrency in RSpec
The trap: a “concurrent” test that never races
Scaling ActionCable to 10k Active WebSockets with Redis Pub/Sub
WebSockets look like the same stack as HTTP and are, in every way that matters, the opposite. An HTTP request occupies a thread for a hundred milliseconds and leaves. A...
Structured Concurrency in Go: Managing Subtask Lifecycles with golang.org/x/sync/errgroup
sync.WaitGroup answers one question: “is everyone done?” It cannot tell you someone failed, and it certainly cannot stop the others when one does. For fan-out over shards, partitions, or upstreams...
gRPC Client-side Load Balancing in Go: Implementing Custom Resolvers
gRPC is built on HTTP/2, and HTTP/2 wants a few long-lived connections, not many short ones. Put a classic L4 or L7 load balancer in front and you have re-introduced...
Writing a High-Throughput Custom DNS Resolver in Go
A crawler or an ingress resolving thousands of distinct hostnames per second exposes a gap in the standard library. The folklore is that net.LookupHost blocks OS threads via cgo —...
Designing a High-Throughput Worker Pool in Go for API Ingestion
Goroutines are cheap — a couple of KiB of stack, sub-microsecond spawn — so teams over-rotated from “thread per request” to “goroutine per request” and hit a wall. Unbounded concurrency...
Implementing Graceful Shutdown in Distributed Go Microservices
Kubernetes terminates pods the same way every time: it sends SIGTERM, waits terminationGracePeriodSeconds (30 by default), then sends SIGKILL. If your service treats SIGTERM as “start dying right now”, every...
Safe Multi-threading inside Rails Controllers with Concurrent Ruby
The classic slow controller is a dashboard that aggregates three independent services. Balance from billing, usage from the analytics service, unread count from notifications. Called sequentially, a request that should...
Implementing Lock-Free Concurrent Data Structures in Go using `sync/atomic`
A shared index-update queue on one of our workers was the wall in every profile: sixteen goroutines contending on a single mutex, and the flame graph showing sync.(*Mutex).Lock as a...
Building an API Gateway Rate Limiter in Go using the Token Bucket Algorithm
Our gateway terminates traffic for public APIs, and the first thing every unauthenticated request hits is a per-API-key limiter. With 50k active keys and a sustained 100k req/s, the limiting...
Implementing Distributed Circuit Breakers in Go without External Dependencies
We ran a checkout pipeline that fronted a legacy payments gateway. The gateway had a 3-second timeout and, once every few weeks, it would wedge for ten minutes. The client...
Category Networking
Scaling Go Websocket Servers to Millions of Concurrent Connections
The memory math nobody does up front
Designing a Low-Latency API Gateway Routing Tree in Go using Radix Trees
An API gateway sits in front of every upstream call, so its router is the hottest code path in the fleet. A linear scan over registered patterns is O(N) in...
gRPC Client-side Load Balancing in Go: Implementing Custom Resolvers
gRPC is built on HTTP/2, and HTTP/2 wants a few long-lived connections, not many short ones. Put a classic L4 or L7 load balancer in front and you have re-introduced...
Writing a High-Throughput Custom DNS Resolver in Go
A crawler or an ingress resolving thousands of distinct hostnames per second exposes a gap in the standard library. The folklore is that net.LookupHost blocks OS threads via cgo —...
Building an API Gateway Rate Limiter in Go using the Token Bucket Algorithm
Our gateway terminates traffic for public APIs, and the first thing every unauthenticated request hits is a per-API-key limiter. With 50k active keys and a sustained 100k req/s, the limiting...
Category Distributed
Building a Replicated State Machine in Go using the Raft Consensus Algorithm
Every gateway node in our fleet needs the same piece of truth at boot: the shard-to-node mapping, the config revision, the list of revoked keys. Before Raft, that truth lived...
Category Database
Multi-tenant Database Routing and Connection Pooling in Go microservices
The simplest multi-tenant design in Go is also the most dangerous: one *sql.DB per tenant, looked up from a map at request time. It works beautifully for ten tenants and...
Rails Solid Cache: Dropping Redis for Database-Backed Caching
The Redis cache bill is a weird tax. You’re paying premium RAM prices for data that is, by definition, disposable — regenerate it and nothing breaks. When our cache footprint...
Custom Database Adapters and Connection Pools in Rails 7.2
We run a 12-server Puma fleet against a Postgres primary and two read replicas. For a long time our read/write splitting was manual: a sidecar script rewrote database.yml on deploy...
Debugging Solid Queue Deadlocks: Transaction Isolation Levels and Row Locks
Rails 8 shipped on November 7, 2024 with Solid Queue as the default background-job adapter — a Postgres-backed queue that fits the “one less database” story. We migrated a roughly...
ActiveRecord Eager Loading Pitfalls: Subqueries vs. IN Clauses
includes is the most convenient lie in ActiveRecord. You write one method call, and ActiveRecord decides for you whether to emit two queries joined by an IN clause (preload) or...
ActiveJob Scaling: Thread Pools, DB Pool Sizes, and PostgreSQL Contention
The incident is boring and predictable: someone bumps Sidekiq concurrency from 10 to 25, and within an hour pager duty is looking at ActiveRecord::ConnectionTimeoutError across every web and worker process....
Building a Custom Key-Value Store from Scratch in Go using LSM-Trees
Why the write path hates B-trees
Multi-Tenant SaaS Architectures: PostgreSQL Schemas vs. Row-Level Security in Rails
Two ways to isolate tenants in one database
Database Sharding in Rails: Implementing Multi-Database Architectures
There’s a point in every growing service where the primary database becomes the bottleneck, and it’s not about reads. Read replicas solved reads. The pain is writes: one node, one...
Zero-Downtime Database Migrations in Rails: Guarding Production with Strong Migrations
The migration that nearly took us down wasn’t a clever one. It was add_index :orders, :customer_id, run at 2pm against a table with 40 million rows. Postgres holds a SHARE...
ActiveRecord Validations vs. Database Constraints: Finding the Optimal Balance
validates_uniqueness_of is not a data-integrity mechanism. It is a user-experience mechanism, and treating it as anything more is how you end up with duplicate invoices. The failure mode is embarrassingly...
Category Engineering
Agent Orchestration: Comparing ReAct, Plan-and-Solve, and Reflection Patterns
Almost every agent starts as a naive while True: call_model, run_tool, call_model loop, and almost every production agent ends up as something else. The reason is that the three canonical...
Enterprise Text-to-SQL: Building Self-Correcting Execution Loops
Text-to-SQL looks solved in demos and falls apart in production, always in the same four ways: the model invents column names the schema never had, it emits syntax that parses...
Guaranteed JSON Formats: Grammars and Regex-Constrained LLM Decoding
Prompting a model to “return valid JSON” fails at a rate that is invisible until it isn’t: at low volume a 2% malformed-output rate is a retry. At 5M calls/day,...
DeepSeek-R1 reasoning models: Reinforcement Learning without Supervised Fine-Tuning
The standard recipe for a “reasoning” model is crushing amounts of supervised data: millions of human-curated chain-of-thought traces, distilled from a bigger model, cleaned by contractors, and fine-tuned into the...
Building Multi-modal RAG Pipelines with Llama 3.1 and Qdrant
Let me correct the framing up front: Llama 3.1 is text-only. The 8B, 70B, and 405B models released in July 2024 consume tokens, not pixels. Any pipeline that claims to...
Propshaft Asset Pipeline: A Modern, Light Replacement for Sprockets
Sprockets was designed for the Rails era where the framework compiled CoffeeScript, SCSS, and ERB-templated JavaScript on the fly. That era is over. Modern browsers ship ES modules and CSS...
Fine-tuning Embedding Models for Domain-Specific Vector Space Alignment
Off-the-shelf embedders understand generic language and nothing else. BGE-small and OpenAI’s Ada-002 encode “what is the refund policy” beautifully; ask them about an internal part number like OCT-4472-REV3 or a...
Custom Database Adapters and Connection Pools in Rails 7.2
We run a 12-server Puma fleet against a Postgres primary and two read replicas. For a long time our read/write splitting was manual: a sidecar script rewrote database.yml on deploy...
Self-hosting Llama 3 70B: Inference Optimization with vLLM and Triton
A client in EU pharma needs inference that never leaves the data center: weights, prompts, and outputs all stay on-prem, and the model has to be legally auditable. Llama 3...
State Machine Validation for GPT-4o Function Calling Applications
GPT-4o function calling still managed to call charge_payment before create_transaction produced an ID. The model emitted both calls in one turn — valid tool calls, invalid order — and the...
Agentic RAG: Knowledge Graph Indexing and Traversal with Neo4j
Vector RAG answered “what does the payment service do?” beautifully and went blind on “what did the author build for the payment service while reporting to project manager X?” The...
Designing an LLM Evaluation Harness: Judge-LLM Alignment and Benchmarks
BLEU and ROUGE did not catch a single regression for us. The rewrite that scored +3 ROUGE against the old transcripts was, by any human reading, worse — it hedged,...
Multi-modal Retrieval: CLIP vs. SigLIP for Visual Semantic Search
Catalog search lives and dies on the visual queries text search never sees: “floral dress, but not the one with the collar”, “refrigerator that fits under a 32-inch counter”, “product...
Deep Dive into LLM Context Window Mechanics: RoPE, YaRN, and FlashAttention
“128K context” sounds like a feature but it is two separate engineering problems stapled together, and they fail independently. The first is arithmetic: attention is O(n²) in time and memory,...
Parameter-Efficient Fine-Tuning: Math and Mechanics of LoRA and QLoRA
Full-parameter fine-tuning of a 7B model needs about 112 GB of VRAM: fp16 weights (14 GB) plus gradients, fp32 Adam moments, and fp32 master weights add up to roughly 16...
Mixture-of-Experts (MoE) Architectures: Routing Math and Load Balancing
Dense scaling hits a hard wall: every parameter you add is a parameter you pay for on every token, in both VRAM and bandwidth. Mixture-of-Experts breaks that coupling by making...
Implementing Idempotent API Endpoints in Rails using Redis Lock and Request Signatures
A client POSTs a payment, the connection drops mid-response, and the client retries. If your endpoint is not idempotent, the second request charges the card twice. At roughly 3M mutating...
Direct Preference Optimization (DPO) vs. PPO: A Pragmatic Comparison
Classic RLHF is a four-model circus: a frozen reference policy, a trainable actor, a reward model, and a critic network, plus a KL term to keep the actor honest. It...
Memory-Augmented Agents: MemGPT and Semantic DB Storage Layer design
The naive fix for agents that forget is to buy a bigger context window. We ran that playbook and it is a trap: the failure mode of a long conversation...
Building Real-Time Collaboration Canvas using Hotwire and Turbo Streams
A canvas is a bad reason to go SPA
Vector Search Indexing: HNSW vs. IVF-PQ in pgvector and Qdrant
Every vector index is a bet on a triangle: recall, latency, and memory. Flat scan sits at one corner — perfect recall, but at 1M 1536-d vectors (about 6 GB...
Optimizing RAG: BM25 + Dense Retriever Hybrid Search and Cross-Encoder Reranking
We shipped our first RAG pipeline with a single dense retriever and got paged within a week: users searching for a firmware error code like ERR_0x84F2 or a part number...
Hardening Rails APIs: Multi-tiered Rate Limiting with Rack::Attack and Redis
Rate limiting is a liveness problem
Structured Concurrency in Go: Managing Subtask Lifecycles with golang.org/x/sync/errgroup
sync.WaitGroup answers one question: “is everyone done?” It cannot tell you someone failed, and it certainly cannot stop the others when one does. For fan-out over shards, partitions, or upstreams...
Designing a High-Throughput Worker Pool in Go for API Ingestion
Goroutines are cheap — a couple of KiB of stack, sub-microsecond spawn — so teams over-rotated from “thread per request” to “goroutine per request” and hit a wall. Unbounded concurrency...
Hardening Rails Security: Implementing Strict Content Security Policy (CSP) and SRI Headers
Browser XSS filters were always a poor backstop, and they’re not even that anymore. The real defense is telling the browser exactly what it’s allowed to execute, and Content Security...
Database Sharding in Rails: Implementing Multi-Database Architectures
There’s a point in every growing service where the primary database becomes the bottleneck, and it’s not about reads. Read replicas solved reads. The pain is writes: one node, one...
ActiveRecord Validations vs. Database Constraints: Finding the Optimal Balance
validates_uniqueness_of is not a data-integrity mechanism. It is a user-experience mechanism, and treating it as anything more is how you end up with duplicate invoices. The failure mode is embarrassingly...
Category DevOps
The Rails 8 Solid Suite: Moving to a Zero-Dependency Stack
Rails 8 (November 2024) did something bold: it made the database the default home for everything that used to need Redis. Solid Queue for background jobs, Solid Cache for the...
Rails Solid Cable: Redis-free WebSockets at Scale
ActionCable’s Redis dependency always annoyed me, and not for the reasons people usually cite. The latency argument against database pub/sub is mostly wrong at small scale — the real cost...
Deploying Rails at Scale with Kamal, Docker, and Tailscale
The platform bill grows linearly with a problem you did not choose: a few years of real traffic later, the Heroku/Render invoices looked like a third engineering salary, and Kubernetes...
Zero-Downtime Database Migrations in Rails: Guarding Production with Strong Migrations
The migration that nearly took us down wasn’t a clever one. It was add_index :orders, :customer_id, run at 2pm against a table with 40 million rows. Postgres holds a SHARE...
Category Security
Implementing Llama Guard: Multi-stage Safety Filtering for LLM Gateways
We run a customer-facing LLM gateway in front of a hosted model, and the threat model is unglamorous but real: prompt injection trying to exfiltrate another tenant’s data, jailbreaks attempting...
Implementing Idempotent API Endpoints in Rails using Redis Lock and Request Signatures
A client POSTs a payment, the connection drops mid-response, and the client retries. If your endpoint is not idempotent, the second request charges the card twice. At roughly 3M mutating...
Hardening Rails APIs: Multi-tiered Rate Limiting with Rack::Attack and Redis
Rate limiting is a liveness problem
Multi-Tenant SaaS Architectures: PostgreSQL Schemas vs. Row-Level Security in Rails
Two ways to isolate tenants in one database
Hardening Rails Security: Implementing Strict Content Security Policy (CSP) and SRI Headers
Browser XSS filters were always a poor backstop, and they’re not even that anymore. The real defense is telling the browser exactly what it’s allowed to execute, and Content Security...
Category Testing
Testing Complex Race Conditions and Concurrency in RSpec
The trap: a “concurrent” test that never races
Category AI
Speculative Decoding: Accelerating LLM Inference in Production
Decode is memory-bandwidth-bound, and that single fact is why speculative decoding is the cheapest throughput win an inference team can spend a week on. Every autoregressive step loads the entire...
Agent Orchestration: Comparing ReAct, Plan-and-Solve, and Reflection Patterns
Almost every agent starts as a naive while True: call_model, run_tool, call_model loop, and almost every production agent ends up as something else. The reason is that the three canonical...
Enterprise Text-to-SQL: Building Self-Correcting Execution Loops
Text-to-SQL looks solved in demos and falls apart in production, always in the same four ways: the model invents column names the schema never had, it emits syntax that parses...
Guaranteed JSON Formats: Grammars and Regex-Constrained LLM Decoding
Prompting a model to “return valid JSON” fails at a rate that is invisible until it isn’t: at low volume a 2% malformed-output rate is a retry. At 5M calls/day,...
Prompt Caching Strategies to Reduce LLM Latency and API Costs
Every RAG or agent application I’ve reviewed re-sends the same few thousand tokens on every request: the system prompt, tool schemas, a hundred pages of policy docs, the conversation so...
DeepSeek-R1 reasoning models: Reinforcement Learning without Supervised Fine-Tuning
The standard recipe for a “reasoning” model is crushing amounts of supervised data: millions of human-curated chain-of-thought traces, distilled from a bigger model, cleaned by contractors, and fine-tuned into the...
Fine-tuning vs Prompt-engineering – what to choose in 2025
The 2023 version of this question was about knowledge: “should we fine-tune so the model knows our domain?” That framing is dead in 2025. Current models — Claude Sonnet 4,...
eBPF-Based Telemetry for LLM Inference Endpoints: Profiling CPU/GPU Latencies
If you serve LLM inference at any real scale, your standard monitoring stack has a blind spot exactly where your tokens actually spend time: the kernel. Prometheus exporters and Datadog...
LLM Agents in Production – What Changed in 2025
The model got smarter, but the reason agents work in production in 2025 is the harness around them. We run them in customer-support triage, internal ops, and incident summarization, and...
Building Multi-modal RAG Pipelines with Llama 3.1 and Qdrant
Let me correct the framing up front: Llama 3.1 is text-only. The 8B, 70B, and 405B models released in July 2024 consume tokens, not pixels. Any pipeline that claims to...
Fine-tuning Embedding Models for Domain-Specific Vector Space Alignment
Off-the-shelf embedders understand generic language and nothing else. BGE-small and OpenAI’s Ada-002 encode “what is the refund policy” beautifully; ask them about an internal part number like OCT-4472-REV3 or a...
Practical Guide to Model Distillation: From DeepSeek-R1 to edge-ready SLMs
When DeepSeek-R1 landed in January 2025, it proved that chain-of-thought plus reinforcement learning could push open models to near-frontier reasoning on math and code. It also proved something less convenient:...
Implementing Llama Guard: Multi-stage Safety Filtering for LLM Gateways
We run a customer-facing LLM gateway in front of a hosted model, and the threat model is unglamorous but real: prompt injection trying to exfiltrate another tenant’s data, jailbreaks attempting...
KV-Cache Eviction and PagedAttention in High-Throughput LLM Serving
The first time we load-tested vLLM for a multi-tenant embedding-and-completion service, we watched nvidia-smi and assumed something was leaking. Four Mistral-7B instances on two A100s, steady 512-token requests, and the...
Self-hosting Llama 3 70B: Inference Optimization with vLLM and Triton
A client in EU pharma needs inference that never leaves the data center: weights, prompts, and outputs all stay on-prem, and the model has to be legally auditable. Llama 3...
State Machine Validation for GPT-4o Function Calling Applications
GPT-4o function calling still managed to call charge_payment before create_transaction produced an ID. The model emitted both calls in one turn — valid tool calls, invalid order — and the...
Agentic RAG: Knowledge Graph Indexing and Traversal with Neo4j
Vector RAG answered “what does the payment service do?” beautifully and went blind on “what did the author build for the payment service while reporting to project manager X?” The...
Designing an LLM Evaluation Harness: Judge-LLM Alignment and Benchmarks
BLEU and ROUGE did not catch a single regression for us. The rewrite that scored +3 ROUGE against the old transcripts was, by any human reading, worse — it hedged,...
Moving Beyond LangChain: Why DSPy is the Future of Declarative AI Pipelines
We rewrote a production pipeline from LangChain to DSPy last spring, and the reason was not a benchmark — it was a two-day incident. A security team bumped our model...
Synthetic Data Generation pipelines for Enterprise Domain Adaptation
Fine-tuning a model for internal tooling usually means one thing is missing: the data. Our first attempt to adapt a model for internal ticket routing needed 10,000 labeled examples, and...
Multi-modal Retrieval: CLIP vs. SigLIP for Visual Semantic Search
Catalog search lives and dies on the visual queries text search never sees: “floral dress, but not the one with the collar”, “refrigerator that fits under a 32-inch counter”, “product...
Deep Dive into LLM Context Window Mechanics: RoPE, YaRN, and FlashAttention
“128K context” sounds like a feature but it is two separate engineering problems stapled together, and they fail independently. The first is arithmetic: attention is O(n²) in time and memory,...
Parameter-Efficient Fine-Tuning: Math and Mechanics of LoRA and QLoRA
Full-parameter fine-tuning of a 7B model needs about 112 GB of VRAM: fp16 weights (14 GB) plus gradients, fp32 Adam moments, and fp32 master weights add up to roughly 16...
Mixture-of-Experts (MoE) Architectures: Routing Math and Load Balancing
Dense scaling hits a hard wall: every parameter you add is a parameter you pay for on every token, in both VRAM and bandwidth. Mixture-of-Experts breaks that coupling by making...
Direct Preference Optimization (DPO) vs. PPO: A Pragmatic Comparison
Classic RLHF is a four-model circus: a frozen reference policy, a trainable actor, a reward model, and a critic network, plus a KL term to keep the actor honest. It...
Memory-Augmented Agents: MemGPT and Semantic DB Storage Layer design
The naive fix for agents that forget is to buy a bigger context window. We ran that playbook and it is a trap: the failure mode of a long conversation...
Vector Search Indexing: HNSW vs. IVF-PQ in pgvector and Qdrant
Every vector index is a bet on a triangle: recall, latency, and memory. Flat scan sits at one corner — perfect recall, but at 1M 1536-d vectors (about 6 GB...
Quantization Deep Dive: Math of GGUF, AWQ, and BitsAndBytes Double Quantization
Quantization is how a 7B model becomes a weekend desktop project instead of a dedicated server: fp16 weights are 14 GB, int8 is 7 GB, 4-bit is about 3.5 GB....
Optimizing RAG: BM25 + Dense Retriever Hybrid Search and Cross-Encoder Reranking
We shipped our first RAG pipeline with a single dense retriever and got paged within a week: users searching for a firmware error code like ERR_0x84F2 or a part number...
Category RAG
Building Multi-modal RAG Pipelines with Llama 3.1 and Qdrant
Let me correct the framing up front: Llama 3.1 is text-only. The 8B, 70B, and 405B models released in July 2024 consume tokens, not pixels. Any pipeline that claims to...
Agentic RAG: Knowledge Graph Indexing and Traversal with Neo4j
Vector RAG answered “what does the payment service do?” beautifully and went blind on “what did the author build for the payment service while reporting to project manager X?” The...
Optimizing RAG: BM25 + Dense Retriever Hybrid Search and Cross-Encoder Reranking
We shipped our first RAG pipeline with a single dense retriever and got paged within a week: users searching for a firmware error code like ERR_0x84F2 or a part number...
Category LLM
Speculative Decoding: Accelerating LLM Inference in Production
Decode is memory-bandwidth-bound, and that single fact is why speculative decoding is the cheapest throughput win an inference team can spend a week on. Every autoregressive step loads the entire...
Enterprise Text-to-SQL: Building Self-Correcting Execution Loops
Text-to-SQL looks solved in demos and falls apart in production, always in the same four ways: the model invents column names the schema never had, it emits syntax that parses...
Guaranteed JSON Formats: Grammars and Regex-Constrained LLM Decoding
Prompting a model to “return valid JSON” fails at a rate that is invisible until it isn’t: at low volume a 2% malformed-output rate is a retry. At 5M calls/day,...
Prompt Caching Strategies to Reduce LLM Latency and API Costs
Every RAG or agent application I’ve reviewed re-sends the same few thousand tokens on every request: the system prompt, tool schemas, a hundred pages of policy docs, the conversation so...
DeepSeek-R1 reasoning models: Reinforcement Learning without Supervised Fine-Tuning
The standard recipe for a “reasoning” model is crushing amounts of supervised data: millions of human-curated chain-of-thought traces, distilled from a bigger model, cleaned by contractors, and fine-tuned into the...
Fine-tuning vs Prompt-engineering – what to choose in 2025
The 2023 version of this question was about knowledge: “should we fine-tune so the model knows our domain?” That framing is dead in 2025. Current models — Claude Sonnet 4,...
eBPF-Based Telemetry for LLM Inference Endpoints: Profiling CPU/GPU Latencies
If you serve LLM inference at any real scale, your standard monitoring stack has a blind spot exactly where your tokens actually spend time: the kernel. Prometheus exporters and Datadog...
Practical Guide to Model Distillation: From DeepSeek-R1 to edge-ready SLMs
When DeepSeek-R1 landed in January 2025, it proved that chain-of-thought plus reinforcement learning could push open models to near-frontier reasoning on math and code. It also proved something less convenient:...
Implementing Llama Guard: Multi-stage Safety Filtering for LLM Gateways
We run a customer-facing LLM gateway in front of a hosted model, and the threat model is unglamorous but real: prompt injection trying to exfiltrate another tenant’s data, jailbreaks attempting...
KV-Cache Eviction and PagedAttention in High-Throughput LLM Serving
The first time we load-tested vLLM for a multi-tenant embedding-and-completion service, we watched nvidia-smi and assumed something was leaking. Four Mistral-7B instances on two A100s, steady 512-token requests, and the...
Self-hosting Llama 3 70B: Inference Optimization with vLLM and Triton
A client in EU pharma needs inference that never leaves the data center: weights, prompts, and outputs all stay on-prem, and the model has to be legally auditable. Llama 3...
State Machine Validation for GPT-4o Function Calling Applications
GPT-4o function calling still managed to call charge_payment before create_transaction produced an ID. The model emitted both calls in one turn — valid tool calls, invalid order — and the...
Designing an LLM Evaluation Harness: Judge-LLM Alignment and Benchmarks
BLEU and ROUGE did not catch a single regression for us. The rewrite that scored +3 ROUGE against the old transcripts was, by any human reading, worse — it hedged,...
Moving Beyond LangChain: Why DSPy is the Future of Declarative AI Pipelines
We rewrote a production pipeline from LangChain to DSPy last spring, and the reason was not a benchmark — it was a two-day incident. A security team bumped our model...
Synthetic Data Generation pipelines for Enterprise Domain Adaptation
Fine-tuning a model for internal tooling usually means one thing is missing: the data. Our first attempt to adapt a model for internal ticket routing needed 10,000 labeled examples, and...
Deep Dive into LLM Context Window Mechanics: RoPE, YaRN, and FlashAttention
“128K context” sounds like a feature but it is two separate engineering problems stapled together, and they fail independently. The first is arithmetic: attention is O(n²) in time and memory,...
Parameter-Efficient Fine-Tuning: Math and Mechanics of LoRA and QLoRA
Full-parameter fine-tuning of a 7B model needs about 112 GB of VRAM: fp16 weights (14 GB) plus gradients, fp32 Adam moments, and fp32 master weights add up to roughly 16...
Mixture-of-Experts (MoE) Architectures: Routing Math and Load Balancing
Dense scaling hits a hard wall: every parameter you add is a parameter you pay for on every token, in both VRAM and bandwidth. Mixture-of-Experts breaks that coupling by making...
Direct Preference Optimization (DPO) vs. PPO: A Pragmatic Comparison
Classic RLHF is a four-model circus: a frozen reference policy, a trainable actor, a reward model, and a critic network, plus a KL term to keep the actor honest. It...
Quantization Deep Dive: Math of GGUF, AWQ, and BitsAndBytes Double Quantization
Quantization is how a 7B model becomes a weekend desktop project instead of a dedicated server: fp16 weights are 14 GB, int8 is 7 GB, 4-bit is about 3.5 GB....
Category VectorSearch
Fine-tuning Embedding Models for Domain-Specific Vector Space Alignment
Off-the-shelf embedders understand generic language and nothing else. BGE-small and OpenAI’s Ada-002 encode “what is the refund policy” beautifully; ask them about an internal part number like OCT-4472-REV3 or a...
Multi-modal Retrieval: CLIP vs. SigLIP for Visual Semantic Search
Catalog search lives and dies on the visual queries text search never sees: “floral dress, but not the one with the collar”, “refrigerator that fits under a 32-inch counter”, “product...
Vector Search Indexing: HNSW vs. IVF-PQ in pgvector and Qdrant
Every vector index is a bet on a triangle: recall, latency, and memory. Flat scan sits at one corner — perfect recall, but at 1M 1536-d vectors (about 6 GB...
Category Frontend
Real-Time Event Streaming in Rails using ActionController::Live and Server-Sent Events (SSE)
“Real-time dashboard” is the feature that silently becomes an architecture decision. The naive version polls a JSON endpoint every second: one client is nothing, but 500 dashboards at 1 req/s...
Building Real-Time Collaboration Canvas using Hotwire and Turbo Streams
A canvas is a bad reason to go SPA
Category Agentic
Agent Orchestration: Comparing ReAct, Plan-and-Solve, and Reflection Patterns
Almost every agent starts as a naive while True: call_model, run_tool, call_model loop, and almost every production agent ends up as something else. The reason is that the three canonical...
Memory-Augmented Agents: MemGPT and Semantic DB Storage Layer design
The naive fix for agents that forget is to buy a bigger context window. We ran that playbook and it is a trap: the failure mode of a long conversation...
Category Ruby
Rails 7.2 — Better eager loading with Zeitwerk
Rails 7.2 (August 2024) did not ship a magic filter that “skips patterns” during eager loading — that idea circulates in a lot of upgrade blog posts, but it is...
Unlocking Speed: Rails Performance Tuning with Ruby 3 YJIT
In Ruby 3.3, YJIT is production-ready. Shopify’s numbers, the Optcarrot benchmark, and the Rails core team all point the same direction: roughly 15–25% latency improvement on real Rails workloads, for...
Pattern-matching refinements in Ruby 3.3
If you read the Ruby 3.3 changelog looking for “right-hand patterns”, stop. Pattern matching arrived in Ruby 2.7, and the => rightward assignment shipped in the same release; the syntax...
Category Programming
Fine-tuning vs Prompt-engineering – what to choose in 2025
The 2023 version of this question was about knowledge: “should we fine-tune so the model knows our domain?” That framing is dead in 2025. Current models — Claude Sonnet 4,...
LLM Agents in Production – What Changed in 2025
The model got smarter, but the reason agents work in production in 2025 is the harness around them. We run them in customer-support triage, internal ops, and incident summarization, and...
Moving Beyond LangChain: Why DSPy is the Future of Declarative AI Pipelines
We rewrote a production pipeline from LangChain to DSPy last spring, and the reason was not a benchmark — it was a two-day incident. A security team bumped our model...
Pattern-matching refinements in Ruby 3.3
If you read the Ruby 3.3 changelog looking for “right-hand patterns”, stop. Pattern matching arrived in Ruby 2.7, and the => rightward assignment shipped in the same release; the syntax...
Category Search
Decoupled Elasticsearch Indexing: Scaling ActiveRecord Callbacks via Sidekiq
The pattern starts innocently:
Category Infrastructure
The Rails 8 Solid Suite: Moving to a Zero-Dependency Stack
Rails 8 (November 2024) did something bold: it made the database the default home for everything that used to need Redis. Solid Queue for background jobs, Solid Cache for the...
Rails Solid Cable: Redis-free WebSockets at Scale
ActionCable’s Redis dependency always annoyed me, and not for the reasons people usually cite. The latency argument against database pub/sub is mostly wrong at small scale — the real cost...
Scaling Sidekiq: Advanced Job Deduplication and Multi-tenant Rate Limiting
Sidekiq’s contract is “push work, forget it,” but the moment you have an event bus that emits per-entity changes, you discover how literally Sidekiq honors that contract. Ten indexing jobs...
Deploying Rails at Scale with Kamal, Docker, and Tailscale
The platform bill grows linearly with a problem you did not choose: a few years of real traffic later, the Heroku/Render invoices looked like a third engineering salary, and Kubernetes...
Category Data
Synthetic Data Generation pipelines for Enterprise Domain Adaptation
Fine-tuning a model for internal tooling usually means one thing is missing: the data. Our first attempt to adapt a model for internal ticket routing needed 10,000 labeled examples, and...
Category Rails
Rails 7.2 — Better eager loading with Zeitwerk
Rails 7.2 (August 2024) did not ship a magic filter that “skips patterns” during eager loading — that idea circulates in a lot of upgrade blog posts, but it is...
Category Kubernetes
Ruby YJIT Memory Footprint: Optimization Strategies for Kubernetes Deployments
Since Rails 7.2 made YJIT the default JIT in August 2024, most Rails apps picked up a 15-25% throughput bump for free. The bill shows up on Kubernetes in a...
Category AssetPipeline
Propshaft Asset Pipeline: A Modern, Light Replacement for Sprockets
Sprockets was designed for the Rails era where the framework compiled CoffeeScript, SCSS, and ERB-templated JavaScript on the fly. That era is over. Modern browsers ship ES modules and CSS...
Category Agents
LLM Agents in Production – What Changed in 2025
The model got smarter, but the reason agents work in production in 2025 is the harness around them. We run them in customer-support triage, internal ops, and incident summarization, and...
Category API
High-Performance JSON Serialization: Benchmarking Oj, Blueprinter, and Fast JSON API
The last place I look when an API endpoint is slow is the serializer, and that’s a mistake. On a typical list endpoint the serializer is where most of the...