What it means
Also called: evals.
Testing what a model can do by having it answer a set of questions and scoring the answers, often by people working from a rubric. “Evals” are those test sets and the scoring runs.
Used in: AI evaluator and rater roles, and expert roles in most fields.
In open roles right now
4,035 of the open roles we list mention “evaluation” or a word for it in the title or description, on 57 platforms (as of 23 October 2026). Of those that state an hourly rate (840 distinct postings), the median is $70/hr, and the middle half pay $35 to $105.
Most of them on: Invisible (553), Lilt (371), Alignerr (344), Mercor (225), SuperAnnotate (224), Handshake AI (222), and 51 more.
Generalist (posted today)JT AI Labs·15 countries
$30–49/hrToday
Business Professionals: Presentation Slide Deck Evaluation for AI Training (posted today)Terac·United States
$70/hrToday
Software Engineer – Retail Investing Platforms (Paid Expert Interview) (posted today)Mercor·United States
$100–120/hrToday
Data Quality Specialist - Swedish, AGI Data Services - G T Quality (posted today)Amazon AGI Data Services·United Kingdom·Full-time
Pay set per projectToday
Generalist (posted today)micro1·United States
$30–49/hrToday
Senior Video EditorSuperAnnotate·22 countries
Up to $50/hr1 day ago
Related terms
Fact-checkingChecking each claim in a model's answer against reliable sources and marking what is wrong, unsupported or out of date.Preference rankingComparing two or more model answers to the same prompt and saying which is better, often with a reason and a rating of how much better.RLHFA way of training a model on people's judgements. ← EQPrevious term, A to Z. Fact-checking →Next term, A to Z. The whole glossaryEvery term on one page, A to Z.Open →