Skip to content

Problems QuestionsSkills Interview Contest Leaderboard

Problems QuestionsSkills Interview Contest Leaderboard

Home
AI Engineer (LLM & Generative AI)
Questions
LLM evaluation

AI Engineer (LLM & Generative AI)

LLM evaluation questions

12 questions on LLM evaluation for ai engineer (llm & generative ai)s, easiest first.

Easy 3

Why keep a separate test seteasy
Computing precisioneasy
What a golden set is foreasy

Medium 4

Pairwise LLM judge biasmedium
Computing recall at 5medium
Computing MRRmedium
Judge prefers longer answersmedium

Hard 5

Shipping a prompt change safelyhard
A judge with 90% agreementhard
Computing nDCG at 3hard
Is a 4-point gain realhard
Fine-tuned model scores too wellhard

More AI Engineer (LLM & Generative AI) skills

RAG and vector search 26 LLM fundamentals 18 AI agents and tool calling 14 Transformers and LLM internals 11 Prompt engineering 9 Resilience and fault tolerance 7 Model serving and inference 7 LLM safety and guardrails 6 Deep learning and PyTorch 5 Performance and debugging 5 Web application security 5 CI/CD 5 Statistics and probability 4 Machine learning 4 Fine-tuning 4

LLM evaluation across all roles →

🏆 LLM evaluation leaderboard →

RED SAG Skills

Free practice for every job role: coding problems, role challenges and mock interviews. Built and managed by RED SAG Fintech Solutions.

Software Engineering

Frontend Developer interview questions
Backend Developer interview questions
Full Stack Developer interview questions
Mobile App Developer interview questions
Game Developer interview questions
Blockchain Developer interview questions

Data, AI & Machine Learning

Machine Learning Engineer interview questions
AI Engineer (LLM & Generative AI) interview questions
Data Scientist interview questions
MLOps Engineer interview questions
Prompt Engineer interview questions
Data Analyst interview questions

Cloud, DevOps & Security

DevOps Engineer interview questions
Cloud Engineer interview questions
Site Reliability Engineer interview questions
Database Administrator interview questions
Cybersecurity Analyst interview questions

© 2026 RED SAG Fintech SolutionsAll roles Coding problems Leaderboard About Careers at RED SAG