medium10 pts · first answer countsmodel-selectioncost-optimization

Big model for every request

Your app sends every request to a large, slow, expensive model, though most requests are simple FAQ lookups. What is a sound approach?

A.Keep the large model; cost does not matterB.Switch everything to the smallest model without testingC.Remove the FAQ featureD.Route simple requests to a smaller, faster model and send only hard ones to the large model, checking both paths with evals

Discussion 0

Join free to post in the discussion.

Big model for every request

Discussion 0

Related questions