medium10 pts · first answer countsfine-tuningtraining-data
Quality versus quantity of SFT data
A team has 200,000 auto-scraped Q&A pairs of mixed quality, or could hand-check 2,000 high-quality ones. For teaching a chat style, what does experience (e.g. the LIMA study) suggest?
Discussion 0
Join free to post in the discussion.
Related questions
- What full fine-tuning changesMachine Learning Engineer
- Why begin from a pretrained modelMachine Learning Engineer
- Training only a new headMachine Learning Engineer
- Learning rate for fine-tuningMachine Learning Engineer
- Shape of SFT training dataMachine Learning Engineer
- Reading train and validation lossMachine Learning Engineer
- Meaning of PEFTMachine Learning Engineer
- What the LoRA rank r setsMachine Learning Engineer