Annotation question types
Each page covers one kind of question. It shows a small input file, the screen reviewers get, the answer they submit and the result you export. Start with the type closest to your data, then add up to four questions to one screen.
Single choice
Each sample gets exactly one label from a short list: sentiment, topic, spam or not, the right translation.
Multi-label
Each sample can carry several labels at once: every topic in an article, every defect in a photo.
Yes or no
A two-option question with keyboard shortcuts. The fastest way to review, filter or verify at volume.
Rating scales
Reviewers rate each sample on a numeric scale, such as 1 to 5 for fluency, relevance or helpfulness.
Speech quality (MOS)
Listeners rate an audio clip, usually for naturalness on a 1 to 5 scale. The result is a Mean Opinion Score.
Pairwise preference
Reviewers see two options for the same input and pick the better one. The standard format for preference data.
Free text
Reviewers write an answer: a transcript, a translation, a short explanation. Every reviewer's text comes back.
Text spans (NER)
Reviewers select parts of a text and give each a label, such as person, place or organisation. The basis of named entity recognition data.
Relations
After marking spans, reviewers link two of them with a named relation, such as works for or capital of.
Bounding boxes
Reviewers draw rectangles around objects in an image and label each one. The base format for object detection data.
Polygons
Reviewers click around the outline of an object to make a polygon. Tighter than a box, and the base for segmentation masks.
Keypoints
Reviewers click to place labelled points on an image: eyes, joints, corners, landmarks.
Audio and video segments
Reviewers play a clip and mark where something starts and ends, with a label for each stretch.