Text span labelling

When to use it

• Named entity recognition
• Marking claims, quotes or sources
• Marking personal data to remove
• Tagging terms in a domain text

What your file looks like

Give each row the full text to mark. Reviewers select words inside it, so keep one passage per row. A file like this is enough:

id,text
1,Zarghona visited Kandahar in March
2,The meeting in Kabul was chaired by Ahmad Shah

What reviewers see

The text, a row of label buttons, and the ability to select words with the mouse or a finger. Each mark is shown in the label's colour and can be removed.

What a reviewer submits

A span is a character range with a label. start is inclusive and end is exclusive, counted in string positions (UTF-16 units, the same as JavaScript slice) of the text as imported.

{
  "answer": {
    "spans": [
      {
        "start": 0,
        "end": 8,
        "label": "PERSON"
      },
      {
        "start": 17,
        "end": 25,
        "label": "PLACE"
      }
    ]
  }
}

What you get back

value lists the spans most reviewers marked with the same start, end and label, with the covered text added. The export also converts to CoNLL (BIO tags) and Label Studio JSON.

{
  "n": 3,
  "value": [
    {
      "start": 0,
      "end": 8,
      "label": "PERSON",
      "text": "Zarghona"
    }
  ],
  "agreement": null,
  "tie": false
}

Quality checks

• Only spans a majority marked survive into the consensus, so noisy marks are filtered by design.
• Write the boundary rules into the question: include titles or not, include the article or not.
• Pre-fill spans from a model as suggestions and ask reviewers to correct them.

Tips

• Exact boundaries are the main source of disagreement. Show two or three worked examples in your instructions.
• Keep the label list short, up to 20 labels at most. Rare labels deserve their own campaign.
• Right-to-left text is supported. Check that your file keeps text in logical order, not visual order.

Limits

At least one label and up to 20. Up to 200 spans per sample.