medium10 pts · first answer countsragvector-databasesquantization
Memory after int8 quantisation
You apply int8 scalar quantisation to those 1 million 768-dimension float32 vectors. How much memory do the vectors take now?
Discussion 0
Join free to post in the discussion.
Related questions
- What RAG doesAI Engineer (LLM & Generative AI)
- What a vector database doesAI Engineer (LLM & Generative AI)
- What semantic search adds over keywordsAI Engineer (LLM & Generative AI)
- What an embedding dimension isAI Engineer (LLM & Generative AI)
- What top-k means in retrievalAI Engineer (LLM & Generative AI)
- Why vector search is approximateAI Engineer (LLM & Generative AI)
- When a flat index is fineAI Engineer (LLM & Generative AI)
- Why store metadata with each chunkAI Engineer (LLM & Generative AI)