Skip to content
RED SAGSkills
medium10 pts · first answer countsdeep-learninginferencemodel-serving

Raising GPU throughput when serving

A model server runs each request alone on the GPU, and utilisation is low under heavy traffic. What technique usually raises throughput most?

Choose one answer

Discussion 0

Join free to post in the discussion.

Related questions

More Deep learning and PyTorch questionsMore Inference questionsMore Model serving and inference questions