medium10 pts · first answer countsfine-tuningsft
Label value -100 in Hugging Face
In this Hugging Face SFT preprocessing, what happens to positions whose label is -100?
labels = input_ids.clone() labels[:prompt_len] = -100 loss = model(input_ids=input_ids, labels=labels).loss
Discussion 0
Join free to post in the discussion.
Related questions
- What full fine-tuning changesMachine Learning Engineer
- Why begin from a pretrained modelMachine Learning Engineer
- Training only a new headMachine Learning Engineer
- Learning rate for fine-tuningMachine Learning Engineer
- Shape of SFT training dataMachine Learning Engineer
- Reading train and validation lossMachine Learning Engineer
- Meaning of PEFTMachine Learning Engineer
- What the LoRA rank r setsMachine Learning Engineer