Mechanical Turk workshop

Shower overview. Click a slide to present.

Northville teacher workshop

The Mechanical
Turk

Where is the work
happening?

A hidden human player
made the moves.

Jason Kridner

Pencil-style illustration of a chess automaton, with a human operator visible inside the cabinet.
Artistic illustration, not a record of a historical encounter

Amazon Mechanical Turk

Human work becomes
training data

  1. 01
    A requester defines a task.
  2. 02
    Workers label, sort
    or judge examples.
  3. 03
    The data can support
    model training.

Who sets the task?

Who gets paid?

Paper activity

One word at a time

PROMPT

Tell a short story about a surprising discovery at school

RESPONSE

At recess, we found a…

  1. Read the prompt and
    the whole response.
  2. Add one word,
    then pass the paper.
  3. When it feels complete,
    write STOP.

After the paper activity

What shaped the story?

Same prompt.
Different stories.

Which words felt likely?
Where did someone surprise you?

You brought experience and judgment.
Language models generate tokens,
which can be parts of words.

This activity explores context and choice. It does not simulate model training.

A 2022 training example

Human feedback
in InstructGPT

  1. 01

    Demonstrate

    People write example answers.
    The model learns from them.

  2. 02

    Compare

    People rank model responses.
    A reward model learns
    their preferences.

  3. 03

    Fine-tune

    Further training uses
    the reward model’s scores.

This is a separate example of human labor in AI, not a claim that OpenAI used MTurk.

Questions for our classrooms

Whose judgment
counts?

Whose examples define “good”?

Who has access to the tools?

Who does the work,
and who receives the value?

What would you want to know before using an AI tool with students?