hard20 pts · first answer countsfine-tuningcost
Fine-tuned small model versus API
A large API model costs ₹0.40 per request. A fine-tuned 8B model on a rented GPU costs ₹150 per hour and handles 2,000 requests an hour at full load. At full load, what is the cost per request on the GPU?
Discussion 0
Join free to post in the discussion.
Related questions
- What full fine-tuning changesMachine Learning Engineer
- Why begin from a pretrained modelMachine Learning Engineer
- Training only a new headMachine Learning Engineer
- Learning rate for fine-tuningMachine Learning Engineer
- Shape of SFT training dataMachine Learning Engineer
- Reading train and validation lossMachine Learning Engineer
- Meaning of PEFTMachine Learning Engineer
- What the LoRA rank r setsMachine Learning Engineer