medium10 pts · first answer countsprompt-cachingtoken-cost
Savings from a cached prefix
Each request has an 8,000-token shared prefix and 2,000 new tokens. Input costs $2 per million tokens, and cached prefix reads cost 10% of that. Ignoring cache write costs, how much cheaper is the input of a cache-hit request than an uncached one?
Discussion 0
Join free to post in the discussion.