hard20 pts · first answer countsgpu-computingscaling-laws

Splitting a fixed compute budget

You have 1.2 × 10^22 FLOPs. Using C ≈ 6ND and the Chinchilla rule D ≈ 20N, which model size and token count are compute-optimal?

A.About 100B parameters on 20B tokensB.About 1B parameters on 2T tokensC.About 30B parameters on 600B tokensD.About 10B parameters on 200B tokens

Discussion 0

Join free to post in the discussion.