Mike-1

MikeBench

Mike-1 is frontier model orchestration plus our legal tooling and memory. A hill-climbing loop of evals, judges and fixes trains it run after run. This page logs every run.

data.public.jsonrows.public.csv
Loading counts
Loading counts
Loading counts
Loading counts
Loading counts
Loading counts

Training log

Model rematch

1 Nov 2026

What the rematch decided

On 1 Nov 2026 this run set the model Mike-1 orchestrates by default: scored level with gpt-6-astra, for less per answer.

Panel mean
4.045 vs 4.085intervals overlap
Cost per answer
$0.041 vs $0.2486.0x lower
Time to first token
46.2 s vs 75.7 smedian
Hard fails
1 vs 1of 20 answers

Mike-1 on prompt A against gpt-6-astra on prompt A.

Mike-1†
Production
4.045of 5

Panel mean · 95% CI 3.65 to 4.285 · 20 answers

Three judges, prompt A, 1 Nov 2026 rematch.

Panel mean4.045gpt-6-astra 4.085
Hard fails1 of 20gpt-6-astra 1
Cost per answer$0.041gpt-6-astra $0.248
TTFT, median46.2 sgpt-6-astra 75.7 s

Panel mean by model and prompt

0 to 5
gpt-6-astra
Prompt A
4.085
1 hard fails
Mike-1
Prompt A
4.045
1 hard fails
Mike-1 fallback
Prompt A
3.895
6 hard fails
gpt-6-astra
Prompt B
3.825
2 hard fails
Mike-1 model
Prompt B
3.78
2 hard fails
Mike-1 fallback
Prompt B
3.73
7 hard fails
gemini-4-argon
Prompt A
3.595
11 hard fails
gemini-4-argon
Prompt B
3.445
14 hard fails
012345
Whiskers are 95% bootstrap intervals. Outlined bars are prompt B.

Results

160 answers
ModelPromptMean95% CIHard fails$/answerTTFT p50Head to head
gpt-6-astraA · v2.6 pack4.0853.73 to 4.3451$0.24875.7 s0.33 (n 30)
Mike-1ProductionA · v2.6 pack4.0453.65 to 4.2851$0.04146.2 s0.40 (n 30)
Mike-1 fallbackA · v2.6 pack3.8953.555 to 4.156$0.161538.5 s0.71 (n 24)
gpt-6-astraB · Light arm3.8253.365 to 4.182$0.19847.2 s0.75 (n 12)
Mike-1 modelB · Light arm3.783.33 to 4.1352$0.031635.3 s0.50 (n 12)
Mike-1 fallbackB · Light arm3.733.415 to 3.9857$0.126228.8 s0.50 (n 12)
gemini-4-argonA · v2.6 pack3.5953.36 to 3.78511$0.066232.2 s0.58 (n 12)
gemini-4-argonB · Light arm3.4453.18 to 3.67514$0.060833.4 s0.42 (n 12)

Mean is the panel median per answer, averaged. Head to head is the share of blind votes won.

Each judge, prompt A

ModelMike-1 modelgemini-4-argonMike-1 fallback
gpt-6-astra3.564.613.44
Mike-13.564.553.40
Mike-1 fallback2.954.713.68
gemini-4-argon2.444.823.60

Mean rubric score each judge gave. The Mike-1 fallback judge scored every 4th answer.

Head to head

PairFirst won95% CIVotes
Mike-1 fallback A vs Mike-1 fallback B0.500.25 to 0.7512
gemini-4-argon A vs gemini-4-argon B0.580.32 to 0.8112
gpt-6-astra A vs Mike-1 fallback A0.170.03 to 0.566
gpt-6-astra A vs gpt-6-astra B0.250.09 to 0.5312
gpt-6-astra A vs Mike-1 A0.500.25 to 0.7512
Mike-1 A vs Mike-1 fallback A0.000.00 to 0.396
Mike-1 A vs Mike-1 model B0.500.25 to 0.7512

Share of votes won by the first entry. Ties count half. Judges never vote on their own family.

Method

Contestants
4 models, 2 prompts, 20 answers each
Tasks
15 legal tasks from the held-out set and the bake-off
Prompt A
v2.6 pack, the production prompt
Prompt B
Same facts and safety rules, no fixed answer templates
Score
Five rubric dimensions, 0 to 5, capped at 2 on a hard fail
Judge
Mike-1 model, every answer
Judge
gemini-4-argon, every answer
Judge
Mike-1 fallback, every 4th answer
Intervals
Percentile bootstrap over answers; Wilson for head to head
TTFT
Seconds to the first visible answer token
Spend
$27.17 of a $40 cap
Run
rematch-2026-11-01 · 1 Nov 2026, 3:02 PM PT
Labels
"Mike-1 model" is the model Mike-1 orchestrates, shown on another prompt or as a judge; "Mike-1 fallback" is its automatic backup.

Raw data

Credentials scrubbed
  • Per-answer results (CSV)
    results.csv
    Download
  • Model x arm summary (CSV)
    summary.csv
    Download
  • All answers (JSONL)
    answers.jsonl
    Download
  • All answers (CSV)
    answers.csv
    Download
  • Rubric judge scores (JSONL)
    rubric_scores.jsonl
    Download
  • Rubric judge scores (CSV)
    rubric_scores.csv
    Download
  • Pairwise judgments (JSONL)
    pairwise.jsonl
    Download
  • Pairwise judgments (CSV)
    pairwise.csv
    Download
  • Methods
    METHODS.md
    Download
  • Limitations
    LIMITATIONS.md
    Download
  • Everything (zip)
    rematch-2026-11-01-public.zip
    Download

Methods