easy5 pts · first answer countsmodel-servingcostcold-start
Serverless GPU for low traffic
A demo gets about 20 requests a day. Serverless GPU hosting scales to zero when idle. What is the main trade-off?
Discussion 0
Join free to post in the discussion.
Related questions
- When batch inference fitsMLOps Engineer
- Traffic before the model has loadedMLOps Engineer
- Why export a model to ONNXMLOps Engineer
- Protocol for internal inference callsMLOps Engineer
- GPU bill keeps risingMLOps Engineer
- Cost of an 8-GPU node for a dayMLOps Engineer
- Matching the GPU to the jobMLOps Engineer
- Securing GPUs for a 3-month runMLOps Engineer