AI Engineer (LLM & Generative AI)
Deep learning and PyTorch questions
28 questions on Deep learning and PyTorch for ai engineer (llm & generative ai)s, easiest first.
Easy 9
Medium 12
- Cosine similarity by handmedium
- Pre-norm versus post-norm blocksmedium
- Turning a loss into perplexitymedium
- Loss of an untrained modelmedium
- Compute-optimal training tokensmedium
- Estimating training FLOPsmedium
- Memory for full fine-tuning with Adammedium
- Why MoE training needs load balancingmedium
- What gradient checkpointing tradesmedium
- Why training starts with warmupmedium
- Packing short examples into one sequencemedium
- What tensor parallelism splitsmedium