Go Middleware Chains for Payment API Pipelines — Why Ordering Matters More Than You Think

The Middleware Pattern in Go

If you've spent any time with Go's net/http package, you've probably seen this signature floating around:

type Middleware func(http.Handler) http.Handler

That's it. A middleware takes a handler, wraps it with some behavior, and returns a new handler. It's functions all the way down. No frameworks, no magic — just composition. This is one of the things I genuinely love about Go's standard library. The http.Handler interface is so small that building on top of it feels natural rather than forced.

Here's what a basic middleware looks like in practice:

func RequestID(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        id := r.Header.Get("X-Request-ID")
        if id == "" {
            id = uuid.NewString()
        }
        ctx := context.WithValue(r.Context(), requestIDKey, id)
        w.Header().Set("X-Request-ID", id)
        next.ServeHTTP(w, r.WithContext(ctx))
    })
}

The pattern is always the same: do something before, call next.ServeHTTP, optionally do something after. When you stack these together, you get a pipeline where each layer has a clear, single responsibility.

Why Ordering Matters in Payment Contexts

In a typical CRUD app, middleware ordering is mostly about convenience. Log before auth? After auth? Doesn't really matter — nobody loses money either way. Payment APIs are different. The order of your middleware directly affects whether you charge someone twice, whether you rate-limit the wrong traffic, and whether your audit trail is actually useful in a dispute.

Here's the chain we settled on after a few painful lessons:

Each position in that chain exists for a reason. Let me walk through the non-obvious ones.

Rate limiting before auth

We rate-limit before authentication. This sounds backwards — why not rate-limit per authenticated user? Because auth is expensive. It involves database lookups, token verification, sometimes calls to an external identity provider. If someone is hammering your endpoint with garbage tokens, you want to drop those requests before you burn compute on verifying them. We do a coarse IP-based rate limit first, then a finer per-merchant limit after auth.

Idempotency after auth

Idempotency keys must be scoped to an authenticated merchant. If you check idempotency before auth, an attacker can replay someone else's idempotency key with a different payload and either get a cached response they shouldn't see, or worse, block the legitimate request from going through. The idempotency key is always (merchant_id, idempotency_key) — never just the key alone.

Recovery at the outermost layer

The panic recovery middleware wraps everything. If any middleware or handler panics, we catch it, log the stack trace with the request ID (which is why request ID is the second layer), and return a clean 500. In payment systems, an unhandled panic that kills the process mid-transaction is how you end up with money debited but not credited.

Scenario	Wrong Order	Right Order	Consequence
Auth before rate limit	Auth → Rate Limit	Rate Limit → Auth	DDoS with invalid tokens burns auth resources
Idempotency before auth	Idempotency → Auth	Auth → Idempotency	Attackers replay keys across merchants
Logging after auth	Auth → Logging	Logging → Auth	Failed auth attempts go unlogged
Recovery inside the chain	Auth → Recovery	Recovery → Auth	Panics in auth crash the process

Building the Chain

We use a simple Chain function that composes middleware right-to-left, so the first middleware in the list is the outermost wrapper. This reads top-to-bottom in the same order the request flows through:

// Chain applies middleware in order. The first middleware
// in the slice is the outermost (runs first on request).
func Chain(handler http.Handler, mw ...Middleware) http.Handler {
    // Apply in reverse so the first middleware wraps outermost
    for i := len(mw) - 1; i >= 0; i-- {
        handler = mw[i](handler)
    }
    return handler
}

// Usage
mux := http.NewServeMux()
mux.HandleFunc("POST /v1/charges", handleCreateCharge)

server := Chain(
    mux,
    Recovery,       // 1st: catch panics
    RequestID,      // 2nd: tag every request
    AuditLog,       // 3rd: log everything
    RateLimit,      // 4th: drop excessive traffic
    Authenticate,   // 5th: verify merchant identity
    Idempotency,    // 6th: deduplicate by (merchant, key)
    ValidateBody,   // 7th: schema validation
)

This is intentionally not a framework. It's a function. You can read the ordering in one glance, and there's no hidden registration or priority system. When someone new joins the team, they can look at this and understand the full request lifecycle in thirty seconds.

The Idempotency Middleware in Detail

Since idempotency is the one that bites hardest when it's wrong, here's a simplified version of ours:

func Idempotency(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        key := r.Header.Get("Idempotency-Key")
        if key == "" {
            next.ServeHTTP(w, r)
            return
        }

        merchantID := MerchantFromContext(r.Context())
        cacheKey := merchantID + ":" + key

        // Check if we've seen this request before
        if cached, ok := store.Get(cacheKey); ok {
            w.Header().Set("Idempotent-Replayed", "true")
            w.WriteHeader(cached.StatusCode)
            w.Write(cached.Body)
            return
        }

        // Capture the response
        rec := &responseRecorder{ResponseWriter: w}
        next.ServeHTTP(rec, r)

        // Only cache successful responses
        if rec.statusCode >= 200 && rec.statusCode < 300 {
            store.Set(cacheKey, rec.Result(), 24*time.Hour)
        }
    })
}

The War Story: How Wrong Ordering Cost Us Real Money

In early 2024, we were processing card payments for a marketplace platform. Our middleware chain had idempotency checking before authentication. It worked fine for months — until it didn't.

A merchant's integration had a retry bug. Their system would fire the same charge request three times in quick succession with the same idempotency key. Normally, our idempotency layer would catch the duplicates and return the cached response. That's exactly what it's for.

The problem: during a deploy, our auth service had about 400ms of elevated latency. The first request hit idempotency (no cache entry yet), passed through to auth (slow), and started processing. The second request hit idempotency, also found no cache entry (the first hadn't completed yet), passed through to auth, and also started processing. Both requests made it to the payment processor. Both charged the card.

The root cause wasn't the retry bug or the slow auth. It was that our idempotency layer wasn't using the merchant ID as part of the cache key — because it ran before auth, so it didn't have the merchant ID. It was keying on the idempotency header alone. When we moved idempotency after auth and added a distributed lock on (merchant_id, idempotency_key) with a short TTL, the race condition disappeared.

We refunded the duplicate charges, filed the incident report, and rewrote the middleware ordering that same week. Total cost: about $12,000 in duplicate charges across multiple merchants, plus the engineering time to untangle the reconciliation.

Testing Middleware in Isolation

One of the best things about the func(http.Handler) http.Handler pattern is that each middleware is independently testable. You don't need to boot the whole server. Just wrap a test handler and assert on the behavior:

func TestRateLimit_BlocksExcessiveRequests(t *testing.T) {
    // Inner handler that always returns 200
    inner := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        w.WriteHeader(http.StatusOK)
    })

    handler := RateLimit(inner)

    // Fire 110 requests from the same IP
    for i := 0; i < 110; i++ {
        req := httptest.NewRequest("POST", "/v1/charges", nil)
        req.RemoteAddr = "192.0.2.1:12345"
        rec := httptest.NewRecorder()
        handler.ServeHTTP(rec, req)

        if i < 100 && rec.Code != 200 {
            t.Fatalf("request %d: expected 200, got %d", i, rec.Code)
        }
        if i >= 100 && rec.Code != 429 {
            t.Fatalf("request %d: expected 429, got %d", i, rec.Code)
        }
    }
}

We also test the full chain as an integration test, but the unit tests on individual middleware catch most regressions. A few tips that have saved us time:

• Use httptest.NewRecorder() liberally. It's the standard library's gift to middleware authors.
• Test both the "pass-through" case (middleware lets the request continue) and the "short-circuit" case (middleware rejects the request).
• For idempotency tests, use a real Redis instance via testcontainers rather than mocking the store. The race conditions you're trying to catch won't show up with an in-memory mock.
• Test middleware ordering by writing a chain integration test that sends a request missing auth headers and verifies it gets rejected after rate limiting but before idempotency lookup.

Closing Thoughts

Middleware chains in Go are deceptively simple. The pattern itself takes five minutes to learn. But in payment systems, the ordering of that chain encodes critical business logic — logic that doesn't show up in any database schema or API spec. Document it. Test it. And when someone proposes adding a new middleware, the first question should always be: where in the chain does it go, and why?