easy5 pts · first answer countsllm-basicstransformers
What LLM pretraining learns
What is the main training objective of a decoder-only LLM during pretraining?
Discussion 0
Join free to post in the discussion.
Related questions
- What is a tokenAI Engineer (LLM & Generative AI)
- Effect of temperatureAI Engineer (LLM & Generative AI)
- What the context window limitsAI Engineer (LLM & Generative AI)
- What a hallucination isAI Engineer (LLM & Generative AI)
- Temperature close to zeroAI Engineer (LLM & Generative AI)
- Why long contexts eat GPU memoryAI Engineer (LLM & Generative AI)
- Sizing the KV cacheAI Engineer (LLM & Generative AI)
- Why attention divides by sqrt(d_k)AI Engineer (LLM & Generative AI)