hard20 pts · first answer countscost-optimizationllm-serving
Daily cost of an LLM feature
A feature serves 1 million requests per day. Each uses 1,200 input tokens and 300 output tokens. Input costs $0.50 per million tokens and output costs $1.50 per million. What is the daily cost?
Discussion 0
Join free to post in the discussion.
Related questions
- Memory for 4-bit weightsMachine Learning Engineer
- KV cache size for one sequenceMachine Learning Engineer
- Speculative decoding and output qualityMachine Learning Engineer
- Why serving engines page the KV cacheMachine Learning Engineer
- Logging bill tripledDevOps Engineer
- Huge NAT gateway chargesDevOps Engineer
- Running workloads on spotDevOps Engineer
- Choosing an S3 storage classCloud Engineer