Fleet Schemas ↑↓ move within a pane · ←→ move between panes · Enter opens · Esc backs out
EvalLabel 40 fields

corpus/EvalLabel

schemaVersion exactly 3
Eval record-grammar version; `3` is the only live grammar.
required
id non-empty string
Content-addressed label id; doubles as the filename.
required
policyId non-empty string
Owning policy id within the project partition.
required
candidateIds list of non-empty string
Candidates this verdict applies to.
required
dimension one of: flag | fix
Whether the verdict judges the detector flag or the proposed fix.
required
kind one of: accept | reject | pairwise-pick | judge-pick | edit-delta | skip
Rating action taken.
required
outputIds list of non-empty string
Outputs compared or edited; required for every fix-dimension verdict except accept/reject/skip on a bare flag.
reason string
Why the winner beat the loser; presence derives the thin-vs-thick margin.
humanSession string
Harness session of the rating human.
judgeId string
Judge identity; present exactly on judge picks.
judgeModel string
Model that produced a judge pick.
orderFlip boolean
Whether A/B order was flipped when the verdict was collected.
sourceLabelId string
Prior label this verdict re-serves or supersedes.
falseFlag boolean
Flag-dimension only: the detector flag itself was wrong.
contentHash string
Flag-dimension only: hash of the flagged content at rating time.
rejectedDimension one of: rule | editing
Which half of a rejected fix was at fault.
retry boolean
Whether the rejection queued a retry.
winnerId non-empty string OR exactly null
Winning output id, or null for a tie or both-bad verdict.
ranking list of non-empty string
N-way preference order over outputIds.
critique object (additional keys allowed) 4 sub
Per-criterion judge critique parsed from the reply.
rubricAdherence string
no description declared
evidenceGrounding string
no description declared
voiceAndStyle string
no description declared
weeklySignal string
no description declared
tie boolean
Both outputs were equally good.
bothBad boolean
Both outputs were unacceptable.
gold boolean
The winner is exemplar-grade.
goldReason string
Why the winner is exemplar-grade.
purpose one of: calibration | improvement
Whether the verdict calibrates the judge or drives generator improvement.
editArtifact object (additional keys allowed) 8 sub
Pointer to an artifact used by an evaluation record.
uri string
Canonical artifact location.
required
localPath string
On-disk path when the artifact is materialized locally.
mimeType string
MIME type of the referenced bytes.
required
sha256 string matching /^[a-f0-9]{64}$/
Content hash of the referenced bytes.
bytes integer >= 0
Size of the referenced bytes.
runId string
query-llm run id that produced the artifact.
thumbnailUri string
Generated thumbnail location.
previewUri string
Generated preview location.
metadata object (additional keys allowed)
Producer-owned annotations; keys and value shapes belong to the writing producer.
createdAt non-empty string
ISO 8601 creation timestamp; the canonical sort key for every store.
required

Rules

  • _flagVerdictFieldsMatchDimension
  • _comparisonFieldsRequireComparisonKind
  • _fixDimensionLabelsCarryOutputs
  • _editDeltaCarriesItsArtifact
  • _nullWinnerRequiresTieOrBothBad
  • _rejectionFieldsRequireRejection
  • _goldReasonRequiresGold

Defects

No defects in the 98 records checked.

Where it lives

Bound bylib-corpus
Declared in~/mnt/mdr/skills/lib-corpus/assets/schemas/eval-label.ts
BindingevalLabelDataStore
Directory~/mnt/corpus/training
Files*/*/labels/*.json
Referenced aseval-label