Skip to content

Data annotation and model feedback

Human-labelled training data across text, image, video, audio and 3D, and the feedback layer for language models: preference data, written demonstrations, rubric scoring, red-teaming.

Two kinds of work sit under one name here. The older kind is supervised labelling: a person reads a document, looks at an image or a frame or a point cloud, and records what is in it against a label set you define. The newer kind is model feedback: a person reads what a model produced and ranks it, scores it, rewrites it or tries to break it.

They are one service because they need the same three things from us. Written guidelines precise enough that two people reach the same answer. People who understand the subject well enough to notice when the guideline is wrong. And a record of who decided what, so a disputed label can be traced rather than argued about.

Supervised labelling

Classic annotation across five modalities. If your task is not on this list, describe it and we will tell you honestly whether it fits.

label set · 9 classes
text

Text classification

Documents sorted into your own label set.

text

Named entity recognition

Names, places and quantities marked in place.

text

Sentiment analysis

Polarity and intensity, per document or per span.

image

Image classification

Whole-image labels against a fixed taxonomy.

image

Object detection

Bounding boxes around every instance.

image

Semantic segmentation

Pixel-level masks, one class per pixel.

video

Video annotation

Objects tracked frame by frame.

audio

Audio transcription

Speech to timed text, speakers separated.

3d

3D annotation

Cuboids and points in lidar or depth data.

Model feedback and evaluation

The work language models need after pre-training, where the label is a judgement rather than a fact about the input.

label set · 7 classes
llm

Preference ranking

Two or more answers, ordered against a written rubric.

llm

Response writing

Gold answers written by someone who knows the domain.

llm

Rubric scoring

One answer scored per criterion: instruction-following, accuracy, tone.

llm

Factuality checks

Each claim traced to its cited source, or marked unsupported.

llm

Safety labelling

Prompts and answers classified against your policy, with severity.

llm

Red-teaming

Adversarial prompts written by hand, with what broke recorded.

llm

Agent trace review

Tool calls and multi-step runs judged step by step.

Annotation geometries

The shapes the work is actually drawn in, ported from manthano.ai/annotation-services/. The catalogue above says what a task is; this says what an annotator draws, which is the level most briefs and most searches are written at.

2D box annotation
Rectangles around each instance. The cheapest geometry per object and the right one when position and rough extent are enough.
Polygon annotation
An outline that follows the object's actual edge. More expensive per object than a box and worth it when the box would be mostly background, as it is for anything long, thin or irregular.
Lines annotation
Polylines along continuous features: lane markings, cables, seams, pipes. Cheaper than a mask and more useful than a box for anything whose length matters and whose width does not.
Point annotation
Single coordinates, for counting and for keypoints such as joints or landmarks. Fast, and the only sensible geometry when objects are too small or too dense to outline.
Lidar 3D box annotation
Oriented cuboids in a point cloud, with heading. The standard geometry for anything that has to know how a vehicle or object is turned, not just where it is.
Lidar pixelwise annotation
Every point in the cloud classified individually. The most expensive geometry we offer and the one that supports drivable-surface and terrain work that boxes cannot express.
3D box annotation
Cuboids in depth or stereo data where a full lidar sweep is not available.
Video annotation
Any of the above tracked across frames with consistent instance identity, which is the part that makes video cost more than the equivalent number of stills.

How a project runs

The same shape whether the batch is a thousand images or an ongoing feedback stream. The pilot is not a formality; it is where most guideline problems surface.

  1. Guidelines

    We write the label definitions with you, including the edge cases you already know about. A guideline that only covers the easy cases produces a dataset that only covers the easy cases.

  2. Pilot batch

    A small batch, labelled by more than one person, before anything scales. You see real output on your own data early enough for it to change the guideline.

  3. Calibration

    Where two annotators disagreed, we adjudicate and ask why. Usually the guideline was ambiguous rather than the person wrong, so the guideline gets amended and the affected items are relabelled.

  4. Production

    The batch runs against the settled guideline. Questions that cannot be answered from it come back to you rather than being guessed at.

  5. Quality pass

    A sample is re-labelled blind and compared, and the disagreement rate is reported to you rather than kept internally. You get the number even when it is not flattering.

  6. Handover

    The dataset, the final guideline, and the record of what changed during the project. The guideline matters: it is what lets you extend the dataset later without drifting away from it.

Who does the work

Labelling is done by people who applied to a programme and were assessed on the kind of work they will actually be given, not by an anonymous pool. German, French, Italian and English are covered; other languages depend on who is available and we will say so plainly rather than quietly assigning a non-native speaker.

For domain work such as clinical text or legal documents, the constraint is usually finding annotators who genuinely know the field. That is worth naming during scoping, because it affects both the timeline and what the labels are worth.

Where the data lives

Processing inside Swiss borders is available on request rather than by default, because it depends on which tools a project needs and where the people doing the work are. Ask when you scope the project and we will confirm in writing what is possible for your data before anything moves.

If a project has no residency constraint, saying so early usually widens what we can offer and shortens the setup.

Questions we get asked

What is the smallest project you take?

There is no fixed floor. A pilot batch is usually the right way to start regardless of eventual size, because it tests the guideline rather than the throughput. What decides feasibility is whether the task can be written down clearly, not how many items there are.

Do you work in our annotation tool or your own?

Either. If you have a platform your team already trusts, annotators work inside it. If you do not, we bring tooling appropriate to the modality. Which way round it goes is a scoping question, and it changes the setup time rather than the price of the labelling itself.

How do you handle disagreement between annotators?

Disagreement is treated as information about the guideline rather than a fault in the annotator. Contested items are adjudicated, the guideline is amended where it was ambiguous, and affected items are relabelled against the corrected version. The disagreement rate is reported to you.

Can our data stay in Switzerland?

Processing inside Swiss borders is available on request rather than by default, because it depends on the tooling a project needs and where the people doing the work are. Ask during scoping and we will confirm in writing what is possible for your data before anything moves.

Do you label data for language model training and evaluation?

Yes. Preference ranking, written gold responses, rubric scoring, factuality checks against cited sources, safety labelling against your policy, adversarial red-teaming, and step-by-step review of agent traces. These are judgement tasks, so the rubric matters more than it does for supervised labelling and we will push to get it written down properly first.

The others

Most projects touch more than one of these. One partner across all of them means the labelling, the preparation and the application are not three vendors blaming each other.

AI application development

Custom AI-driven applications, taken from concept through to deployment by the team that labelled the data.

Synthetic data generation

Rendered training data built in 3D instead of collected and labelled: for classes that are rare, expensive to photograph, or cannot leave the building.

Tell us what needs labelling.