hard20 pts · first answer countsprompt-cachingprompt-debugging

Prompt cache hit rate near zero

Your long system prompt should be cached, but the cache hit rate is almost zero. Look at how the prompt is built. Why?

system = f"""Current time: {datetime.now().isoformat()}
You are the support assistant for Acme...
(6,000 tokens of fixed policy and tool docs)"""

A.Caching only works for prompts under 1,000 tokensB.The policy text is too long to cacheC.Python f-strings cannot be cachedD.The timestamp changes on every request and sits at the very start, so the prefix is never identical; move dynamic values to the end

Discussion 0

Join free to post in the discussion.

Prompt cache hit rate near zero

Discussion 0

Related questions