Skip to main content

Command Palette

Search for a command to run...

#devops

Articles tagged with #devops

  1. You are buying the same 2,000 tokens on every LLM call. Prompt caching is the refund.

    A giant system prompt, a block of few-shot examples, a document pasted in front of every request - if it never changes and you have not turned on prompt caching, you re-buy those exact tokens on every call. Here is why that is a silent 5-10x overspend, how the OpenAI and Anthropic caches actually differ, and primecache, a static gate that fails the build when a cacheable prefix ships uncached.

    Oct 14, 20269 min read1
    You are buying the same 2,000 tokens on every LLM call. Prompt caching is the refund.