Learn a skill
How to learn LLM Evaluation, for the jobs that ask for it.
5% of tech job ads in the United States and Canada name LLM Evaluation (3,434 in 180 days), and 80% of those require it. Start with 3 free resources from Mettlespark's learning plan for it.
From 73,506 tech job ads in the United States and Canada, May 19, 2026 to November 15, 2026. Updated November 15, 2026. How we count. Free to reuse with credit: CC BY 4.0
- Job ads naming it
- 3,4345% of tech job ads
- Required when named
- 80%of the job ads that name it
- Time to learn
- about 20 hoursafter LLMs; about 150 from zero
Start here
Evaluating AI Agents
A free DeepLearning.AI short course with Arize AI on tracing and evaluating an agent, for beginners with basic Python.
Evals, error analysis, and better prompts: A systematic approach to improving your AI products
Hamel Husain shows product teams how to move from vibe checks to error analysis on real conversations, suited to beginners and product managers.
Your AI Product Needs Evals
Hamel Husain's case for evals in LLM products and his three level approach, for anyone starting to evaluate an AI product.
The whole LLM Evaluation plan, in Mettlespark
This page shows 3 of its resources. The plan sets out every topic in order, sized to the hours you have each week, and tracks what you finish.
- Topics
- 9from first steps to job-ready, in stages
- Resources
- 29hand-picked: courses, videos, docs, books and papers
- Projects
- 3to build, and show employers
Projects in the plan
- Golden set and graders for a support assistant
- RAG eval harness with a CI regression gate
- Agent evaluation with trajectory grading and pass^k
Where you stand
What AI Engineer job ads ask for
2,804 job ads from 1,149 employers, last 180 days. Tick what you have.
of what AI Engineer job ads ask for. Tick what you have.20% remote. Full AI Engineer report
See where you standFree in early access. Your ticks and the role come with you.
Who asks for it
| Role | Share of its job ads naming LLM Evaluation |
|---|---|
| AI Engineer | |
| Machine Learning Engineer | |
| Applied Scientist | |
| Computer Vision Engineer | |
| MLOps Engineer | |
| Research Scientist (ML) | |
| Data Scientist | |
| Data Science Manager |
On this page
- How many job ads ask for LLM Evaluation, and which roles
- 3 free places to start
Only in Mettlespark
- The whole LLM Evaluation plan: 9 topics, 29 resources, 3 projects, in order
- Sized to the hours you have each week, and to what your target role needs first
- Your gaps from your CV, across every skill your target role asks for
- Plans for 178 skills, each with topics, resources and projects
- Progress you can see as you finish each step
Quick answers
How long does it take to learn LLM Evaluation?
About 20 hours once you know LLMs, by Mettlespark's estimate; about 150 hours from zero, counting what comes before it.
Which jobs ask for LLM Evaluation?
Most often AI Engineer (28% of its job ads), Machine Learning Engineer (26% of its job ads) and Applied Scientist (17% of its job ads).
Is LLM Evaluation required in job ads?
80% of the job ads that name LLM Evaluation mark it required; the rest list it as nice to have.