Fleet Schemas ↑↓ move within a pane · ←→ move between panes · Enter opens · Esc backs out
EvalLabel 40 fields

corpus/EvalLabel

schemaVersion exactly 3
Eval record-grammar version; `3` is the only live grammar.
required
id non-empty string
Content-addressed label id; doubles as the filename.
required
policyId non-empty string
Owning policy id within the project partition.
required
candidateIds list of non-empty string
Candidates this verdict applies to.
required
dimension one of: flag | fix
Whether the verdict judges the detector flag or the proposed fix.
required
kind one of: accept | reject | pairwise-pick | judge-pick | edit-delta | skip
Rating action taken.
required
outputIds list of non-empty string
Outputs compared or edited; required for every fix-dimension verdict except accept/reject/skip on a bare flag.
reason string
Why the winner beat the loser; presence derives the thin-vs-thick margin.
24
humanSession string
Harness session of the rating human.
0
judgeId string
Judge identity; present exactly on judge picks.
37
judgeModel string
Model that produced a judge pick.
37
orderFlip boolean
Whether A/B order was flipped when the verdict was collected.
37
sourceLabelId string
Prior label this verdict re-serves or supersedes.
57
falseFlag boolean
Flag-dimension only: the detector flag itself was wrong.
0
contentHash string
Flag-dimension only: hash of the flagged content at rating time.
0
rejectedDimension one of: rule | editing
Which half of a rejected fix was at fault.
0
retry boolean
Whether the rejection queued a retry.
0
winnerId non-empty string OR exactly null
Winning output id, or null for a tie or both-bad verdict.
ranking list of non-empty string
N-way preference order over outputIds.
83
critique object
Per-criterion judge critique parsed from the reply.
0
rubricAdherence string
no description declared
evidenceGrounding string
no description declared
voiceAndStyle string
no description declared
weeklySignal string
no description declared
tie boolean
Both outputs were equally good.
2
bothBad boolean
Both outputs were unacceptable.
8
gold boolean
The winner is exemplar-grade.
0
goldReason string
Why the winner is exemplar-grade.
0
purpose one of: calibration | improvement
Whether the verdict calibrates the judge or drives generator improvement.
9
editArtifact object
Pointer to an artifact used by an evaluation record.
0
uri string
Canonical artifact location.
required
localPath string
On-disk path when the artifact is materialized locally.
mimeType string
MIME type of the referenced bytes.
required
sha256 string matching /^[a-f0-9]{64}$/
Content hash of the referenced bytes.
bytes integer >= 0
Size of the referenced bytes.
runId string
query-llm run id that produced the artifact.
thumbnailUri string
Generated thumbnail location.
previewUri string
Generated preview location.
metadata map of any shape (not yet constrained by the schema)
Producer-owned annotations; keys and value shapes belong to the writing producer.
61
createdAt non-empty string
ISO 8601 creation timestamp; the canonical sort key for every store.
required

Checks

  • _flagVerdictFieldsMatchDimension - Keeps flag-only fields on flag verdicts.
  • _comparisonFieldsRequireComparisonKind - Reserves comparison fields for comparison verdicts.
  • _fixDimensionLabelsCarryOutputs - Requires outputs for every fix verdict.
  • _editDeltaCarriesItsArtifact - Pairs edit-delta verdicts with their artifact.
  • _nullWinnerRequiresTieOrBothBad - Explains null human winners and validates winner membership.
  • _rejectionFieldsRequireRejection - Reserves rejection fields for rejection verdicts.
  • _goldReasonRequiresGold - Requires an exemplar-grade winner for goldReason.

Defects

No defects in the 98 records checked.

Where it lives

Bound bylib-corpus
Declared in~/mnt/mdr/skills/lib-corpus/assets/schemas/eval-label.ts
BindingevalLabelDataStore
Directory~/corpus/training
Files*/*/labels/*.json
Referenced aseval-label