A prompt written to make a model fail: to give a wrong, unsafe or rule-breaking answer. Adversarial prompts are used to find weaknesses and to build hard test sets.
- Adversarial prompt #
- Agentic task (trajectory) #
-
A task in which a model works in several steps, such as searching, running code or using a website, rather than giving a single answer. The trajectory is the full record of those steps. Workers write these tasks, carry them out to show the model how, or grade a model's trajectory step by step.
- AI interview #
-
A screening interview run by software: an AI interviewer asks questions by video or voice, records your answers, and the platform reviews them. Some platforms use one as an early step instead of, or before, a call with a person.
- Annotation #
-
Adding information to raw data so a model can learn from it: tags on text, boxes on images, timestamps on audio. Postings mostly use it to mean the same as labelling.
- Assessment (screening test) #
-
A test you take after applying and before you get project work: a skills quiz, a writing sample, a coding exercise or a trial of real tasks. A few platforms pay for it; most don't. Some let you retake it after a waiting period, others don't.
- Benchmark #
-
A fixed set of test questions used to compare models, with a published score for each. Experts are hired to write questions hard enough that current models get them wrong.
- Bounding box #
-
A rectangle drawn around an object in an image or video frame, with a label such as """car""" or """pedestrian""". One of the most common image-labelling tasks.
- Chain of thought (reasoning trace) #
-
The step-by-step working a model writes before its final answer. Experts are asked to write such traces, or to check each step of a model's trace and mark where it first goes wrong.
- Contractor vs employee #
-
Most AI training platforms take people on as independent contractors rather than employees. Contractors are usually paid only for the time or tasks they log, without holiday pay, sick pay or pension contributions, and usually handle their own tax. Some roles, mostly full-time ones, are employment. Which applies depends on your contract and the law where you live; this is not legal advice.
- EQ (empty queue) #
-
Worker shorthand for a project with no tasks available to you at the moment. You stay on the project, but there is nothing to do, and on most platforms nothing to be paid for, until tasks come back.
- Evaluation (evals) #
-
Testing what a model can do by having it answer a set of questions and scoring the answers, often by people working from a rubric. """Evals""" are those test sets and the scoring runs.
- Fact-checking #
-
Checking each claim in a model's answer against reliable sources and marking what is wrong, unsupported or out of date. Often part of rating and rewriting tasks.
- Golden set (gold standard) #
-
Tasks whose correct answers are already agreed, often mixed into your work without being marked, to check your accuracy. Also used to train new workers. Missing too many can lower your quality score or take you off a project.
- Grounding #
-
Tying a model's answer to a given source, such as a document or search results, so that every claim can be traced to it. Grounding tasks ask you to check whether each statement is supported by the source provided.
- Hallucination #
-
A confident statement from a model that is false or made up, such as an invented citation, quote or figure. Spotting hallucinations is a large part of rating and fact-checking work.
- ID verification #
-
A check that you are who you say you are, usually a photo of a government ID and a selfie or short video, handled by a verification service. Most real platforms ask for it before paying you. Upload ID only on the platform's own site, never by chat or email to a """recruiter""".
- Jailbreak #
-
A prompt that gets a model to ignore its safety rules, for example through role-play, a hypothetical, or a harmful request split into harmless-looking parts. Red-teamers write jailbreaks so the model can be trained to resist them.
- Labelling #
-
Putting a category or tag on a piece of data: an image, a sentence, a search result, a clip of audio. Usually short, repeated tasks. Postings mostly use it to mean the same as annotation.
- LiDAR #
-
A sensor that measures distance with laser pulses and produces a 3D """point cloud""". LiDAR annotation means drawing 3D boxes (cuboids) around cars, people and other objects in those point clouds, mostly for self-driving cars and robots.
- Onboarding #
-
The steps between being accepted and doing paid work: reading project guidelines, watching training material, passing a qualification quiz and setting up payment. Some platforms pay for onboarding time and some don't; the project terms say which.
- Per approved hour #
-
Pay counted only for the hours the platform accepts. Time on tasks that fail review, or time beyond a task's time limit, may not be paid, so pay per approved hour can come to less than the hours you actually worked.
- Preference ranking (pairwise comparison, side-by-side) #
-
Comparing two or more model answers to the same prompt and saying which is better, often with a reason and a rating of how much better. These judgements are the main input to RLHF.
- Project pause #
-
When a platform stops work on a project for a while, often while the client reviews the data or changes the guidelines. Contractors are usually not paid during a pause, and some paused projects don't restart.
- Project-based work #
-
Work organised by project, each with its own guidelines, pay rate and length, starting and ending when the client needs it. You may work on one project for weeks and then wait for the next, so hours are rarely steady.
- Prompt writing #
-
Writing the questions or instructions a model is trained or tested on. Good prompts are realistic, specific and hard enough to show where the model fails; many projects want prompts from your own field.
- Quality score #
-
A platform's running measure of how good your work is, built from reviewer ratings, golden-set accuracy and audits. It decides which projects and tasks you're offered, and on some platforms your pay rate. A low score can take you off a project.
- Red-teaming #
-
Deliberately trying to make a model produce harmful, unsafe or wrong output, so the developer can find and fix weaknesses before release. Roles range from general prompt writing to specialists in security, chemistry, biology or law.
- Response rewriting #
-
Editing a model's answer into what a good answer should be: fixing errors, filling gaps, and correcting tone and format. The rewritten answers are used as training examples for SFT.
- Reviewer #
-
A worker who checks other contributors' tasks, scores them and sends back corrections. Reviewer work is usually offered to people with a good quality score on that project.
- Reward model #
-
A model trained on people's preference judgements to predict which answer people would prefer. In RLHF it scores the main model's answers, so a limited number of human judgements can guide training on far more examples.
- RL environment #
-
A setting built so a model can practise a task and be scored automatically, such as a code repository with tests, a simulated website or a set of tools. Workers build environments and write the tasks and the checks that decide whether the model succeeded.
- RLHF (reinforcement learning from human feedback) #
-
A way of training a model on people's judgements. People compare or rate the model's answers; those judgements train a reward model, which is then used to train the model towards answers people prefer. Most rating and ranking work feeds into it.
- Rubric #
-
A written list of what a good answer must contain or avoid, used to grade answers consistently. Expert projects often ask you to write a rubric for your own prompt, with points that can each be marked as met or not met.
- Segmentation #
-
Marking the exact outline of objects in an image, pixel by pixel, rather than drawing a box round them. Semantic segmentation labels every pixel by class (road, sky, person); instance segmentation also separates each object from others of the same class.
- SFT (supervised fine-tuning) #
-
Training a model on examples of prompts paired with good answers written or approved by people. Writing those prompt-and-answer pairs is a common task for writers and subject experts.
- Talent pool #
-
An open sign-up rather than a specific opening: you apply, get assessed and wait to be matched to a project. Being accepted doesn't guarantee work. Roles on this site marked """Talent pool""" are of this kind.
- Task-based pay vs hourly pay #
-
Hourly pay is for time you log, often with a time limit per task. Task-based pay is a fixed amount per completed task, so what you earn in an hour depends on how fast you work. When a posting gives an hourly figure for task-based work, it is an estimate.
- Timesheet (time tracking) #
-
How hours are recorded for hourly work: a timer in the task tool, a separate time-tracking app, or a timesheet you fill in. Some platforms check logged time against your activity and pay only for time they count as active.
- Transcription #
-
Writing down exactly what is said in an audio or video recording, following a style guide on punctuation, filler words, background noise and speaker changes. Speech projects need native speakers of each language.
- Utterance #
-
A single short piece of speech, such as a sentence or a voice command. Speech data projects ask you to record utterances from a script, or to transcribe or label recorded ones.
More guides
- AI training jobs: a plain guide: what the work is, how hiring and pay work.
- How to spot a fake AI training job.
- Pay calculator: what an hourly rate comes to over a week, month and year.