Any spend limit that depends on process memory has a blind spot on infrastructure that doesn't persist state between runs — and CI is exactly that infrastructure. I found the clearest version of this reviewing our own content pipeline's spend ceiling: a per-run cap, a per-day cap, a per-month cap, all checked in code, before any paid API call could go out, rather than audited after the fact. Three real layers. They also did essentially nothing, because the counter each one depended on reset to zero every single time the scheduled job ran.
Why a limit enforced "in code" isn't automatically enforced at all
The bug wasn't in the spend-checking logic itself — the arithmetic was correct, the thresholds were reasonable, the check ran before every call exactly as designed. The problem was what the check was comparing against: a running total held in memory, populated from whatever the current process had spent so far. On a long-running server that's a fine place to keep a counter. On a scheduled job, it's not, because each GitHub-hosted runner starts as a new virtual machine provisioned fresh for that run and torn down afterward. Nothing about the environment persists from one scheduled firing to the next unless you deliberately make it persist — GitHub's own caching mechanism exists specifically because a clean checkout otherwise remembers nothing about any previous run.
Our daily cap, in other words, was being checked correctly against a total that started at zero on every single invocation. A job that ran ten times in one day would have its per-day limit "enforced" ten separate times against ten separate zeroes. The limit wasn't broken; it was checking the wrong thing, faithfully, every time.
The part that made this hard to catch
This is the kind of gap that hides in plain sight because every individual run looks completely correct. Trigger the job once, watch it respect the cap, and everything checks out — the code does exactly what it says it does, on that one execution. The failure only exists in the aggregate, across runs, which is precisely the scope a single test or a single manual check never covers. Nobody sets out to verify "does my daily cap survive being invoked ten separate times from ten separate clean environments," because the natural instinct when a guard passes its test is to stop looking, not to ask what state it's implicitly assuming will carry over.
We found it while reasoning through a much narrower question: whether MAX_ARTICLES_PER_DAY was actually doing anything in production, since production runs happen in CI rather than on any one long-lived machine. Walking through what CI actually persists between invocations was what surfaced it — not a monitoring alert, not a cost spike large enough to notice, just the process of asking what a stateless environment does and doesn't remember.
What actually closes the gap
The fix has to give the limit somewhere durable to read from and write to, and it has to fail safe if that record is ever missing or corrupted.
Persist the spend record outside the ephemeral runner. A running counter in memory dies with the process. The fix writes every call's actual cost to a small ledger file, and — this is the part that's easy to skip — commits that ledger back to the repository at the end of every run, so the very next scheduled invocation starts from an accurate history instead of a blank slate. Without that commit step, you've built exactly the same bug one layer further down: a ledger that's correct within a single run and forgotten the moment the runner is torn down.
Fail closed if the ledger can't be trusted. A ledger file that's missing, malformed, or fails to parse needs to stop the run rather than quietly proceed as if no spend has ever happened — a parse failure that gets swallowed and defaults to zero recreates the identical blind spot the fix was built to close, just with an extra step in front of it. We made a corrupt or unreadable ledger throw immediately, on the reasoning that a spend guard which fails open is not meaningfully different from having no spend guard at all.
Add a second cap for legitimate bulk work. Once the per-day and per-month ceilings were actually being enforced against real history, a batch job that intentionally needed to generate something like 10 pieces in one sitting started hitting the daily cap partway through — correctly, but not usefully. The fix there wasn't to raise the daily ceiling; it was a separate, explicit per-batch allowance for the specific case of a deliberate bulk run, so the common case stays tightly capped without the legitimate exception needing to loosen the rule for everyone else.
The broader pattern
Any rate limit, spend cap, or counter that depends on process memory is really a statement about that one process's lifetime, not about the workload it's meant to govern. The moment the thing you're limiting can run across more than one process — a new CI job, a new container, a new serverless invocation — an in-memory counter stops being a limit and becomes a per-invocation illusion of one. The question worth asking about any guard like this isn't "does the check run" — it's "where does the number it's checking actually live, and does that location survive exactly the kind of restart the workload will actually go through in production."
FAQ
Wouldn't a shorter-lived, more frequent check have caught this sooner?
No — the check itself was running correctly every time; the problem was entirely in what it was comparing against. A more frequent check against the same in-memory zero would have "passed" just as confidently, just more often.
Is committing a ledger file back to the repo the only way to persist this kind of state?
No, it's simply the option that fit a workflow already built around Git as the source of truth — a small external key-value store or a database row would solve the same problem. The requirement isn't "commit to Git" specifically; it's "the record has to live somewhere that outlives the process enforcing the limit."
Does this only affect spend caps, or other kinds of limits too?
It affects any limit whose state lives only in process memory or a local variable — daily rate limits, retry counters, deduplication caches, anything meant to hold across more than one invocation of a job that actually runs in a fresh environment each time. Spend caps are simply the version where the consequence of getting it wrong is a dollar amount instead of a duplicate action.
How do you verify a fix like this actually works, rather than just looking correct?
Simulate the exact scenario that broke it: run the job multiple times in a row from genuinely fresh environments and confirm the cap trips at the right cumulative point, not the same non-trip result every single time. A guard that's never been watched to actually trigger is a guard you're trusting on faith.
Where to go from here
This closes out the cost half of the Build with Claude course for now — see the cheapest way to run scheduled Claude jobs and the env var that tripled our AI coding bill for the earlier lessons in this module. The next module covers the broader pattern this incident is one instance of — a check that looks like it's working while silently checking nothing. Work with me or subscribe to the newsletter for it as it ships.