hard20 pts · first answer countsragembeddingsfine-tuning

Duplicates during contrastive training

Your training set for embedding fine-tuning has many near-duplicate FAQ questions. Fine-tuning with in-batch negatives makes results worse. Why?

A.Duplicates make the batch too small to train onB.Duplicates in a batch are wrongly treated as negatives of each otherC.In-batch negatives only work for images, not textD.The model learns to output the same vector for everything

Discussion 0

Join free to post in the discussion.

Duplicates during contrastive training

Discussion 0

Related questions