Fleet Schemas ↑↓ move within a pane · ←→ move between panes · Enter opens · Esc backs out
GenTracesExperimentJudgments 13 fields

autoresearch-experiments/GenTracesExperimentJudgments

protocolHash string matching /^sha256:[a-f0-9]{64}$/
SHA-256 of the recipe that produced these rows: for a reply record the assembled messages, model, temperature, and Git pins; for a judge record the judge templates, predicate, and judge model.
required
runs list of object
One row per recorded gen run, in execution order - here, judge rows.
required
runId non-empty string
Canonical gen run id resolving this row's exact input and output.
required
judged exactly "RED" OR exactly "GREEN" OR exactly "BLACK"
Whether this judge run graded its reply correctly: GREEN when its verdict matches the reference color for the reply it judged (the human hand grade in a trial, otherwise the reply's settled color), RED when it does not. Correctness is only informative once a trial has supplied hand grades; rows recorded before any trial read GREEN because they produced the color they were recorded against. When the reply settled no color at all, because its samples split or too few survived, there is no reference to agree with, so the row carries that sample's own verdict instead.
required
judgedInput non-empty string
Gen run id of the reply this judge run graded; absent on reply rows.
lawId non-empty string
Judge rows only: the law this one judge sample named as violated, exactly as it appeared in the judge block's `###` heading. Present only when the sample's own verdict was RED; a GREEN or BLACK sample names no violated law. Independent of the row's `judged`, which records the sample's correctness rather than its verdict.
lawIds non-empty list of non-empty string
Reply rows only: the distinct law ids named by the samples that settled this reply's color, in code-point order so the row is a function of the samples rather than of the order the judge pool returned them. A reply settles on the COLOR alone, so three agreeing RED samples that name three different laws still settle RED and keep all three ids here - which is what lets a reader see the sibling-clause split the one-prompt judge design accepts. Absent when no settling sample named a law.
cutFollowup one of: matched-historical | black-unmatched | none
Reply rows only: what happened when this reply's first emission at a mid-turn cut was tool calls and nothing else. `matched-historical` = every call matched one the agent made later in the same turn, so each was answered with its recorded output and the model generated exactly once more; the recorded reply text carries both emissions. `black-unmatched` = at least one call had no historical match, so the harness stopped rather than invent a result, and the reply stands as its first emission alone. `none` = the reply carried prose, or made no tool call, so no follow-up applied. Absent when the fire declared no historical follow-up answers at all.
followupRunId non-empty string
Reply rows only: gen run id of the SECOND generation of a `matched-historical` reply, recorded beside the row's own first-generation runId so the whole judged unit is replayable. Absent on every other row.
errors non-empty list of object
Failed fire or judge rows excluded from the verdict denominator.
86
stage exactly "fire" OR exactly "judge"
Boundary that failed before a binary verdict could be recorded.
required
runId non-empty string
Canonical gen run id when the model reply exists and only judging failed.
message non-empty string
Failure reported outside the color denominator; distinct from BLACK, which is a usable reply that proves nothing.
required

Checks

  • _checkRunIdentityUniqueness
  • _checkRecordRows
  • _checkJudgeRowsCarryJudgedInput

Defects

Checked 200 of 945 records (the per-store cap is 200), so this is a sample, not a whole-store verdict.

No defects in the 200 records checked.

Where it lives

Bound byautoresearch-experiments
Declared in~/mnt/mdr/skills/autoresearch-experiments/assets/schemas/gen-traces-experiment-results.ts
BindinggenTracesExperimentJudgeDataStore
Directory~/mnt/runs/autoresearch-experiments
Files*/*/*/artifacts/trace-judge.yml
Referenced asgen-traces-experiment-judgments