hard20 pts · first answer countsgpu-computingcapacity-planning

Estimating total training time

You pretrain a 1B-parameter model on 20B tokens (about 6ND FLOPs) using 8 GPUs at 312 TFLOPS peak each and 40% MFU. Roughly how long will it take?

A.About 3.3 hoursB.About 13 daysC.About 33 hoursD.About 330 hours

Discussion 0

Join free to post in the discussion.