When humans interact with machines, their experience is what decides about the success of the interaction. Prodigy helps you collect annotations faster and more efficiently by focusing on one task at a time, and reducing each annotation to simple and intuitive decisions. For more background on the Prodigy philosophy and thoughts that went into developing the tool, see our blog posts on UX-driven data collection and front-end development for AI.

The web app includes a range of built-in annotation interfaces, key bindings and mobile swipe gestures and fully customizable UI themes. When you annotate a task, the decision – accept, reject or ignore – is added to the task as the "answer" key. The ten most recent decisions are shown in the sidebar and remain editable, either by clicking “undo”, or by selecting them from the history. Annotated tasks are sent back to the REST API in batches. To save the current progress manually, you can always hit the save button, or press command+s (on Mac) or ctrl+s.

Try the live demo View annotation interfaces

---

Key bindings and swipe gestures

For more convenient annotation, you can use the following key bindings:

‌‌	Action	Description	Key options
	accept	Accept an annotation.	a
	reject	Reject an annotation.	x
	ignore	Ignore an annotation.	space
	undo	Undo the last annotation decision.	backspace, del
	save	Save the current buffer of annotations.	command+s, ctrl+s
	flag	Flag an example to bookmark it for later, if enabled.	f
	help	Show annotation instructions, if available.	h
	choice	Select a label in a manual interface or an option in a choice interface.	0 - 9

The numbers 1 to 9 map to the options 1 through 9, and 0 maps to option 10. For interfaces with more than 10 options, e.g. manual annotation with larger label sets, the options 11 though 20 are available via shift+num and 21 through 30 via shift+alt+num. For example, shift+6 will select option 16, and shift+alt+0 option 30.

Image shortcuts

The image_manual interface also supports the following keyboard shortcuts:

‌‌	Action	Description	Key options
	rect	Draw a rectangular shape.	r
	polygon	Draw a polygon shape.	t, p
	freehand	Draw a freehand shape.	f
	delete	Delete the selected shape.	d
	reset	Reset the image to its original or previous state.	q
	exit	Exit the current shape.	escape
	labels	Toggle the “All labels” UI setting.	l
	select	Hold down to select a shape by clicking on it (instead of its label).	shift
	ratio	Draw rects and resize shapes with a fixed ratio.	shift

In addition to the shortcuts listed above, you can also use the arrow keys ↑ ↓ ← → to move the selected shape by one pixel in either direction.

Audio shortcuts New: 1.10

The audio_manual UI provides the following additional shortcuts:

‌‌	Action	Description	Key options
	playpause	Start the playback if it’s currently paused.	enter
	playpause	Pause the playback if it’s currently playing.	enter
	zoom	Focus the zoom slider (adjusted using left ← and right → arrow keys).	z
	loop	Toogle looping.	l
	reset	Reset the regions to their original or previous state.	q

Dependency relation shortcuts New: 1.10

The relations UI provides the following additional shortcuts:

‌‌	Action	Description	Key options
	rels	Switch to dependency annotation mode.	r
	spans	Switch to span highlighting mode.	s
	delete	Delete the selected span in span highlighting mode.	d
	reset	Reset the relations and spans to their original or previous state.	q
	exit	Exit the current selection.	escape
	select	Hold down to select multi-token span (click on first token, then last token).	shift

Custom keyboard shortcuts New: 1.9

As of v1.9, Prodigy lets you customize the keyboard shortcuts for the main actions (currently excluding the number shortcuts for options and labels). For each action name, you can override the key bindings with a list of one or more keys or key combinations.

prodigy.json{
  "keymap": {
      "accept": ["o", "a"],
      "reject": ["k", "x"],
      "save": ["command+space"]
    }
}

Custom keyboard shortcuts for labels New: 1.9.4

In addition to the "keymap", v1.9.4 also lets you customize the shortcuts for individual labels in interfaces like ner_manual or image_manual using the "keymap_by_label" setting. It maps label names or numbers ("1" for the first choice option, and so on), to single keys. If no custom key is provided for a given label or option, Prodigy will fall back to its default number key.

prodigy.json{
  "keymap_by_label": {"PERSON": "p", "ORG": "o", "1": "q", "2": "w", "3": "e"}
}

Swipe gestures

You can also enable swipe gestures on touch devices by setting "swipe": true in your prodigy.json. This will let you swipe left for accept and swipe right for reject.

Customizing action buttons New: 1.10

As of v1.10, you can customize the buttons displayed in the app via the "buttons" config setting, either in your prodigy.json or the "config" returned by a custom recipe. This can be useful if you want to limit the annotation answer choices.s If any of the answers accept, reject or ignore are disabled, the user won’t be able to assign them anymore and the keyboard shortcuts will be disabled, too. The undo action will always be available via its shortcut and by clicking on a history entry in the sidebar, even if you hide the action button.

prodigy.json or recipe config{
  "buttons": ["accept", "ignore"]
}

Prodigy with custom buttons

This live demo requires JavaScript to be enabled.

Statistics and progress

The sidebar helps you keep an overview of your annotation project and includes a progress widget showing the project’s annotation count, progress and optional detailed statistics for the accept, reject and ignore decisions. To re-add an already annotated task to the front of the queue and change your decision, simply click on it in the history, or use the undo button or the backspace key. See the config settings for customization options.

prodigy.json{
  "project_info": ["dataset", "session", "lang", "recipe_name", "view_id", "label"],
  "show_stats": true,
  "history_length": 10
}

Flagging tasks

Flagging a task lets you to bookmark it for later, independent of the annotation decisions accept, reject and ignore. This can be useful to keep a record of seemingly strange predictions of your model, to investigate later on, or to save interesting examples you come across. Since flagging is mostly developer functionality, it’s disabled by default and requires the "show_flag" setting. Flagged tasks will receive a "flagged": true entry, and can be exported separately by setting --flagged-only on db-out.

prodigy.json{
    "show_flag": true
}

Annotation instructions

If you need to provide more detailed annotation instructions, you can use the "instructions" config setting to load in a text or HTML file. The instructions will then be displayed in a modal, accessible via the help icon at the top, or by pressing h.

prodigy.json{
    "instructions": "/path/to/instructions.html"
}

Annotation UI language New: 1.10

The "ui_lang" setting lets you change the language of descriptions, messages and tooltips in the annotation UI. This can be useful if your annotators feel more comfortable if the app speaks their language. Note that the UI language has nothing to do with the language of the data you’re annotating. You can annotate text in any language, and set the UI language to any of the available options.

prodigy.json{
    "ui_lang": "en"
}

en	English (default)	View live demo →
de	German	View live demo →
nl	Dutch	View live demo →
es	Spanish	View live demo →
zh	Chinese	View live demo →

---

Themes

Prodigy comes with three built-in theme options. To change the theme, simply edit the theme setting in your configuration file and restart the web app. The theme options are "basic" (default), "eighties", "spacy" and "brutalist".

You can also overwrite a range of colors and other layout properties under the "custom_theme" setting to customize the look and feel of the application.

Setting	Description	Default
accept	Color to visualize “accept” action.	#4fd364
reject	Color to visualize “reject” action.	#f74c4a
ignore	Color to visualize “ignore” action.	#b9b9b9
undo	Color to visualize “undo” action.	#b9b9b9
bgCard	Background of annotation card.	#ffffff
bgCardSecondary	Second background color option, used for accents.	#f6f6f6
bgCardTertiary	Third background color option, used for accents.	#dddddd
bgCardQuaternary	Fourth background color option, used for accents.	#aaaaaa
bgPage	Page background.	#f6f6f6
bgSidebar	General sidebar background.	#54606e
bgSidebarDark	Darker shade of sidebar color for header and accents.	#384451
bgHighlight	Background of highlighted text, e.g. entities.	#ffe184
bgLowlight	Background of subtly highlighted text, e.g. spans outside entities.	#d9d9d9
bgCardTitle	Background color of card title, e.g. text category.	#583fcf
bgProgress	Color of progress bar.	#ffffff
bgButton	Default annotation button background (disabled).	#b9b9b9
bgMeta	Background of card meta text.	#f6f6f6
colorText	General text color.	#444444
colorMeta	Text color of card meta text.	#939393
colorMessage	Text color of loading or error message if no card is displayed.	#6e6e6e
colorSidebar	Text color in sidebar.	#ffffff
colorSidebarHeadline	Text color of sidebar headlines.	#ffffff
colorSidebarLabel	Text color of of sidebar sub-headlines, e.g. “Dataset”.	#b9b9b9
colorButton	Icon color of annotation buttons.	#ffffff
colorHighlightLabel	Text color of label on highlighted text, e.g. entity label.	#583fcf
colorCardTitle	Text color of card title, e.g. category.	#ffffff
cardBorder	Border around card.	1px solid #ddd
cardShadow	Box shadow around card.	1px 1px 5px #ddd
cardBorderRadius	Border radius of card.	0
sidebarBorder	Right border on sidebar.	none
labels	Custom label colors, keyed by label name.	see below

You can also customize a theme’s fonts. The primary font is used for most text, while the secondary font is used for headlines and labels.

Setting	Default
fontPrimary	"Lato", "Trebuchet MS", Roboto, Helvetica, Arial, sans-serif
fontSecondary	"Roboto Condensed", "Arial Narrow", sans-serif

Theme sizes

Themes can also overwrite some of the default text and container sizes (in pixels):

Setting	Description	Default
largeText	Large annotation card text (for short texts).	37.5
mediumText	Medium annotation card text (for medium texts).	24
smallText	Small annotation card text (for long texts).	20
buttonSize	Width and height of annotation buttons.	100
progressHeight	Height of progress bar.	10
cardMaxWidth	Maximum width of card.	675
cardMinWidth	Minimal width of card.	300
cardMinHeight	Minimal height of card.	100
relationHeight	Maximum height of arc space in relations.	325
relationHeightWrap	Maximum height of arc space in relations if lines wrap.	80

Color palettes

The following color palettes are used across the different interface types, unless other colors are specified in the data or via custom label colors.

Images (boxes)	#ffff00 #00ffff #ff00ff #00ff7f #ff6347 #00bfff #ffa500 #ff69b4 #7fffd4 #ffd700 #ffdab9 #adff2f #d2b48c #dcdcdc
Audio (regions)	#ffd700 #00ffff #ff00ff #00ff00 #9932cc #00bfff #ff7f50 #66cdaa
Relations (spans and arcs)	#c5bdf4 #ffd882 #d9fbad #c2f2f6 #ffdaf9 #b5c6c9 #96e8ce #ffd1b2
Spans (part-of-speech tags)	#ffe184 #ffa9b8 #d1caff #b4e8fc #d3fe7d #9effd6 #ffa973 #dddddd

Custom label colors

Using the labels setting, a theme can specify custom label colors for entity labels, part-of-speech tags, bounding boxes, audio regions and so on. The colors will be used in both the manual and non-manual interfaces. If no label color is set, a default color is used.

prodigy.json (excerpt){
  "custom_theme": {
    "labels": {
      "VERB": "#ffa9b8",
      "PERSON": "#d1caff",
      "COREF": "#00ffff"
    }
  }
}

---

Rest API

Prodigy communicates with its web application via a REST API. You can also use this API to send requests to Prodigy directly, to the same port that’s running the web application. By default, cross-origin resource sharing (CORS) is enabled, allowing you to make a request from a different origin, like a different port or domain. To disable CORS, set "cors": false in your configuration file.

Interactive API documentation New: 1.8

As of Prodigy v1.9, Prodigy’s REST API is powered by FastAPI (in v1.8 you can optionally enable it with the PRODIGY_FASTAPI variable). This doesn’t only make the API faster – it also gives you interactive API docs out-of-the-box if you navigate to localhost:8080/docs.

Most relevant endpoints

/project get

Get a project’s meta data and the current configuration. The web app makes one request to this endpoint on load to fetch the project information (displayed in the sidebar) and the front-end settings.

Example response{
  "id": "dataset",
  "description": "Project description",
  "author": "You",
  "theme": "basic",
  "view_id": "ner",
  "version": "1.0.0"
}

/get_questions get

Get a new queue of tasks. The number of new tasks sent with each request can be specified via the batch_size setting of a recipe or in your Prodigy config. On each request, the endpoint also returns the number of total annotations and the progress calculated by the annotation recipe.

Example response{
  "tasks": [
    {
      "_task_hash": 10354113762182654000,
      "_input_hash": 503541133681926542210,
      "spans": [{"start": 16, "end": 22, "label": "ORG"}],
      "score": 0.340494123,
      "text": "Check it out in action."
    }
  ],
  "total": 268,
  "progress": 0.19027213177874192
}

/get_session_questions postNew: 1.7

Get the next batch of tasks to annotate for a given session_id. Used when Prodigy is accessed from a named multi-user session.

Example request{
  "session_id": "user1"
}

Example response{
  "tasks": [
    {
      "_task_hash": 10354113762182654000,
      "_input_hash": 503541133681926542210,
      "spans": [{"start": 16, "end": 22, "label": "ORG"}],
      "score": 0.340494123,
      "text": "Check it out in action."
    }
  ],
  "total": 268,
  "progress": 0.19027213177874192
}

/give_answers post

Send a list of annotated tasks containing an answer key and receive a progress value. The web app makes a request to this endpoint every time a batch is annotated, or if the user chooses to manually save the current progress. The lower the batch_size, the more frequent the updates.

Example request{
  "answers": [
    {
      "_task_hash": 10354113762182654000,
      "_input_hash": 503541133681926542210,
      "spans": [{"start": 16, "end": 22, "label": "ORG"}],
      "score": 0.340494123,
      "text": "Check it out in action.",
      "answer": "reject"
    }
  ]
}

Example response{
  "progress": 0.19027213177874192
}

/validate_answer postNew: 1.10

Send an answer submitted by the user in the web app and validate it using a function provided by the recipe. Only called if a validate_answer callback is provided.

Example request{
  "answer": {
      "_task_hash": 10354113762182654000,
      "_input_hash": 503541133681926542210,
      "spans": [],
      "text": "Check it out in action.",
      "answer": "accept"
    }
}

Example response{
  "ok": false,
  "message": "Annotation should include at least one span."
}

---

Named multi-user sessions New: 1.7

As of v1.7.0, Prodigy supports multiple named sessions within the same instance. This makes it easier to implement custom multi-user workflows and controlling the data that’s sent out to individual annotators. The update was shipped in preparation of the upcoming Prodigy Teams, a full-featured, standalone application for large-scale multi-user annotation projects powered by Prodigy.

To create a custom named session, add ?session=xxx to the annotation app URL. For example, annotator Alex may access a running Prodigy project via http://localhost:8080/?session=alex. Internally, this will request and send back annotations with a session identifier consisting of the current dataset name and the session ID – for example, ner_person-alex. Every time annotator Alex labels examples for this dataset, their annotations will be associated with this session identifier.

The "feed_overlap" setting in your prodigy.json or recipe config lets you configure how examples should be sent out across multiple sessions. By default (true), each example in the dataset will be sent out once for each session, so you’ll end up with overlapping annotations (e.g. one per example per annotator). Setting "feed_overlap" to false will send out each example in the data once to whoever is available. As a result, your data will have each example labelled only once in total.

As of v1.8.0, the PRODIGY_ALLOWED_SESSIONS environment variable lets you define comma-separated string names of sessions that are allowed to be set via the app. For instance, PRODIGY_ALLOWED_SESSIONS=alex,jo would only allow ?session=alex and ?session=jo, and other parameters would raise an error.