easy5 pts · first answer countsragembeddingschunking
Long chunks and embedding limits
Your embedding model accepts at most 512 tokens. You embed 2,000-token chunks without checking. What usually happens?
Discussion 0
Join free to post in the discussion.
Related questions
- What text embeddings areAI Engineer (LLM & Generative AI)
- What RAG doesAI Engineer (LLM & Generative AI)
- Comparing two embeddingsAI Engineer (LLM & Generative AI)
- Dot product of two vectorsAI Engineer (LLM & Generative AI)
- What a vector database doesAI Engineer (LLM & Generative AI)
- What semantic search adds over keywordsAI Engineer (LLM & Generative AI)
- What an embedding dimension isAI Engineer (LLM & Generative AI)
- What top-k means in retrievalAI Engineer (LLM & Generative AI)