Skip to content

WO20: tiny-student distillation (Arabic r7) + static-int8 sweep on released artifacts #441

Open

Description

@ronaldtse

Goal

Sharpen the ML claims with zero new data: two executable campaigns on the EXISTING canonical teachers and released artifacts.

A. Distill the canonical Arabic teacher (r7) to tiny students

  • The durable distillation pipeline (microkimi recipe: scan-first ridge R2, closed-form stitch/bridge across widths, paired bootstrap eval) applied to r7's outputs → small byte-level students.
  • Judge: paired bootstrap against r7 on the held-out eval; report WER + latency/size deltas honestly. Publish-worthy only if the student holds within the pipeline's established tolerance; negative results are recorded as negatives (the program's convention).

B. Static-int8 quantization sweep on released models

  • The DeepSeek round's one positive lever (static int8) applied to the other released artifacts (urd-diac-1.0, the ByT5 Thai line, Hebrew if artifacts exist).
  • Judge: conformance vectors per model MUST stay within the established tolerance (the existing conformance kits); size/latency deltas reported. Any model failing tolerance keeps its fp32 artifact — record and move on.

Rules: training/distillation runs on the established infra (HF jobs, --timeout 4h + step checkpoints, mounted data — never local corpora); license gates untouched (urd-g2p stays owner-gated; r7 outputs are in-program). Report under 300 words per campaign with commands + outputs. Branch: wo20-ml-tiny (and any quant branch names it needs).

Activity

  1. ronaldtse commented on Nov 16, 2026

    @ronaldtse
    ContributorAuthor

    WO20 opened — milestone plan (branch wo20-ml-tiny, both repos).

    Campaign A — r7 → tiny students (microkimi recipe):

    • Teacher: run-007-news (r7, 2.2864 full-set DER-CE) + teacher_labels_r7.jsonl from teachers-arabic-v1 (sha e70ce991… verified in-job).
    • Stage 1 (pre-registered kill gate): the microkimi scan — closed-form directional ridge R² of r7 decoder-layer activations from ByT5-small decoder-layer streams on a calibration slice (minutes, no gradients). Kill rule: if the best directional R² does not clear the scan's alignment bar, no distillation GPU spend is armed; the scan result is recorded as the negative.
    • Stage 2 (scan-gated): ByT5-small student, base 2.0 recipe (Muon, r7 labels, sequence KD) + frozen scan-selected low-rank bridges as FitNets-style hidden-MSE projectors — the one mechanism in the bridge/stitch family not yet in the lever ledger.
    • Judge: paired bootstrap against r7 on the full 1,200-paragraph SadeedDiac-25 windowed zero-skip protocol (teacher decoded in-job on the same paragraphs; documents = resampling unit). Tolerance: the pipeline's established student gate, teacher + 0.5pp = 2.7864. Verdict recorded honestly either way.

    Campaign B — static-int8 sweep: fp32 released artifacts urd-vocalized-1.0 (urd-diac-1.0), tha-phonemic-base-1.0 + tha-phonemic-1.0 (ByT5 Thai line), heb-vocalized-1.0, heb-vocalized-small-1.0, heb-vocalized-1.1 (Hebrew). Per model: build via the shipped build_int8_static_zip path (dynamic-int8 encoder + static-calibrated decoder), judge against the conformance-kit vectors where pinned and the WO03 precision gate (CER-delta ≤ 2.0pp int8, confident flips ≤ 1%) on banked fp32-reference decode; size + latency deltas reported. Any model failing tolerance keeps its fp32 artifact. urd-g2p stays owner-gated; nothing is published without owner sign-off.

    Runs post here as they land.

  2. ronaldtse commented on Nov 16, 2026

    @ronaldtse
    ContributorAuthor

    Milestone: both campaigns are in flight (branches wo20-ml-tiny pushed on interscript-models + interscript-train).

    • Campaign A scan (kill-gate): job wo20-scan rhtse/6afb07bef7633f04f3556de5, t4-small, --timeout 4h. Verifies the r7 teacher + teacher_labels_r7.jsonl (sha e70ce991…) from teachers-arabic-v1, captures decoder streams, closed-form directional ridge-R² scan with scrambled controls, applies the pre-registered kill rule (R² ≥ 0.30 and ≥ 2× control). ARMED → bridges solved and the distill job launches; KILLED → recorded as the negative, no distillation spend.
    • Campaign B sweep: job wo20-quant rhtse/6afb0851f7633f04f3556e3d, a10g-small, --timeout 4h, resume-safe per-model markers. Six released fp32 artifacts → build_int8_static_zip → WO03 int8 gate (CER-Δ ≤ 2.0pp) + confident-flip gate (≤ 1%) vs the fp32 zip under one framing, kit/fixture conformance rows, size + CPU/CUDA latency.

    Two earlier scan launches are recorded as failed starts, both infra: the teacher zip's checkpoint lives at run-007-news/best/ (first launch pointed at the parent dir) and the labels file's target column is teacher, not tgt — both fixed in train_wo20_tiny_hf.py; no GPU numbers were produced by either.

  3. ronaldtse commented on Nov 16, 2026

    @ronaldtse
    ContributorAuthor

    Milestone: Campaign A scan verdict — ARMED; distillation launched.

    Scan job wo20-scan (rhtse/6afb07bef7633f04f3556de5) completed. The closed-form directional ridge-R² scan over all 5×7 (student layer, teacher layer) pairings cleared the pre-registered kill rule — best R² 1.0000 (student L0 → teacher L0) vs scrambled control 0.2539 (rule: ≥ 0.30 and ≥ 2× control). All five per-student-layer best pairings armed at rank-256 bridges: L0→L0 (1.0000), L1→L0 (0.9985), L2→L2 (0.9939), L3→L2 (0.9928), L4→L2 (0.9906). Honest caveat, recorded now: the near-1.0 R² across nearly all early-layer pairs is the microkimi scan's documented "early states align with everything" inflation — teacher-forced decoder streams share token-identity structure, so the kill rule passes easily; the informative test is the downstream gate, not the scan.

    Distill job wo20-distill rhtse/6afb09f9f7633f04f3556f31 (a100-large, --timeout 4h, step checkpoints every 500 + resume): the 2.0 rung verbatim (Muon 0.01 + AdamW 1e-4, sequence KD α 0.5 T 2.0, BS 8, 3 epochs, canonical r7 labels sha e70ce991…) + the single pre-registered delta β·MSE through the frozen bridges. Epilogue: full 1,200-paragraph SadeedDiac-25 windowed zero-skip decode for student AND in-job teacher, paired bootstrap vs r7 (documents = resampling unit), gate = teacher + 0.5pp = 2.7864, secondary read = delta vs the 4.8218 base rung.

    Campaign B sweep continues on wo20-quant (urd reference decode done; int8static build in progress).

  4. ronaldtse commented on Nov 16, 2026

    @ronaldtse
    ContributorAuthor

    Milestone: distillation training started — recipe now bit-verified against the published 2.0 rung.

    Job wo20-distill rhtse/6afb0ba1f7633f04f3556fc0 (a100-large, --timeout 4h, step ckpts): 29,322 usable label pairs × 3 epochs = 10,995 steps — exactly the published run-006 step count — Muon 0.01 (128 matrix params) + AdamW 1e-4 (42 embedding-like), sequence KD α 0.5 T 2.0, gradient checkpointing, plus the single pre-registered delta β 0.1·MSE through the five frozen scan-selected bridges. Two earlier distill launches are recorded as failed starts, both my wiring bugs (armed_pairs format; a 384-B data filter copied from the wrong lineage call site — the ara spec's real caps are units ≤ 1450 B, labels ≤ 2×max_len); no GPU numbers were produced by either.

    Campaign B sweep wo20-quant (rhtse/6afb0ad5f7633f04f3556f93): urd-vocalized-1.0 past reference decode, int8static build in progress; the first sweep launch is recorded as a failed start (per-row session rebuild in the margin pass — fixed to build once per zip).

  5. ronaldtse commented on Nov 16, 2026

    @ronaldtse
    ContributorAuthor

    Milestone: first campaign-B verdict + measured training cadence.

    • urd-vocalized-1.0 (urd-diac-1.0): PASS — int8static viable. Parity CER-Δ 0.0902pp (gate ≤ 2.0pp), zero confident flips, row-exact int8-vs-fp32 disclosed in the report; artifact ×0.359 size (1.42 GiB → 510 MiB), ×1.91 CPU decode speed. Report + the built variant sit on the bucket at wo20-quant/.
    • Training cadence measured on wo20-distill: ~5 steps/min (teacher-forced fwd + checkpointed bwd at BS 8 × 1450-B windows, teacher fp16 in the loop). The 10,995-step arm therefore spans ~36h of GPU time — it will run as a sequence of --timeout 4h jobs resuming from the 500-step bucket checkpoints (labels.sha-pinned), the established resume pattern. Training is healthy: step 200, loss 12.92 → 2.98 (ce 1.57, kd 1.45, bridge-MSE 27.7 → 14.7 as the frozen bridges pull student activations toward teacher geometry).
    • One more sweep fix recorded: the tha slice rows key as word/phonemic (tha-phonemic corpora convention), not src/tgt — loader normalizes; urd's PASS marker made the relaunch skip it.
  6. ronaldtse commented on Nov 16, 2026

    @ronaldtse
    ContributorAuthor

    Milestone: second campaign-B verdict — the confident-flip gate bites.

    • tha-phonemic-base-1.0: FAIL — keeps its fp32 artifact. Parity CER-Δ 0.6583pp (inside the ≤ 2.0pp int8 gate) but confident flips 11.01% vs the ≤ 1% margin gate — exactly the failure mode the teacher-forced margin analysis exists to catch: quantization noise flipping confident argmaxes where CER on the surface stays quiet. Size ×0.368, CPU speed ×1.87 were healthy; the fragility gate fails the artifact. Recorded, moving on.
    • Sweep now on tha-phonemic-1.0 (ByT5-small line). Distill at step 900, loss 0.615 (ce 0.136, kd 0.128, bridge-MSE 4.83, still descending).
    • Framing note, corrected on the banked reports: the job image's onnxruntime-gpu cannot load the CUDA EP (needs CUDA 13 libs), so every measurement is on the CPU contract framing, both precisions, one framing — the urd report on the bucket now says so explicitly.
  7. ronaldtse commented on Nov 16, 2026

    @ronaldtse
    ContributorAuthor

    Milestone: third campaign-B verdict.

    • heb-vocalized-1.0 (heb-diac-1.0): FAIL — keeps its fp32 artifact. Parity CER-Δ 3.2769pp breaches the ≤ 2.0pp int8 gate outright, and confident flips 1.46% also breach the ≤ 1% margin gate — both judges reject the static-int8 variant.
    • tha-phonemic-1.0: the build crashed with StopIteration — that artifact is a plain-decoder zip (no decoder-kv.onnx), and the static recipe re-quantizes the KV decode path. The sweep now short-circuits such artifacts as N/A — keeps fp32 (committed); it re-records on the next sweep relaunch via the marker chain.
    • heb-vocalized-small-1.0 in flight; heb-vocalized-1.1 + the tha-1.0 N/A rerun queued.
    • Distill: step 5350/10995, loss 0.351 (ce 0.075, kd 0.042, bridge-MSE 2.92) — roughly the halfway mark, resume chain continues.
  8. ronaldtse commented on Nov 16, 2026

    @ronaldtse
    ContributorAuthor

    Milestone: fourth campaign-B verdict.

    • heb-vocalized-small-1.0 (heb-diac-small-1.0): FAIL — keeps its fp32 artifact. Parity CER-Δ 7.9907pp — nearly 4x the ≤ 2.0pp int8 gate (confident flips 0.16% pass, but the CER gate is decisive). Notable asymmetry vs heb-vocalized-1.0 (big model: flips-dominated fail at 1.46%, CER-Δ 3.28pp; small model: CER-dominated fail with clean margins) — static-int8 degrades the two Hebrew artifacts through different mechanisms.
    • Sweep verdicts so far: urd PASS / tha-base FAIL / tha-1.0 N/A (pending rerun; plain-decoder) / heb-1.0 FAIL / heb-small FAIL. heb-vocalized-1.1 downloading; the sweep job's 4h window closes soon and the marker chain resumes it for the remainder.
    • Distill: step 7050/10995 (~64%), loss 0.343 (ce 0.075, kd 0.038, bridge-MSE 2.87).
  9. ronaldtse commented on Nov 16, 2026

    @ronaldtse
    ContributorAuthor

    Milestone: training complete; eval epilogue in progress.

    All 10,9995/10,995 steps done — final train loss 0.35 (ce 0.075, kd 0.037, bridge-MSE 2.79), checkpoints at step-10000/10500 on the bucket. The job resumed cleanly from step-8500 across the 4h window boundary ("resume from step-8500" in the log — the labels.sha-pinned checkpoint chain works exactly as designed).

    The eval epilogue is running: full 1,200-paragraph SadeedDiac-25 windowed zero-skip decode for the student (in flight; early windows show real well-voweled Arabic) then the in-job teacher on the same paragraphs, then the paired bootstrap. Measured decode cadence ~5 windows/min (batch-8, the protocol's framing) → the dual decode spans roughly a day across several --timeout 4h windows; every window is checkpointed to the bucket and resumes losslessly.

    Campaign B: 5 of 6 verdicts banked (urd PASS; tha-base, heb-1.0, heb-small FAIL; tha-1.0 N/A plain-decoder). heb-vocalized-1.1 fp32 reference decode in flight — the last target.

  10. ronaldtse commented on Nov 16, 2026

    @ronaldtse
    ContributorAuthor

    WO20 COMPLETE — both campaigns landed. Verdicts below; full entries in RESULTS.md (PR #442, branch wo20-ml-tiny).

    Campaign A — r7 → tiny student (microkimi recipe)

    • Scan: ARMED (best directional R² 1.0000 vs control 0.2539; the near-1.0 inflation across pairs is recorded as the scan's documented early-layer effect).
    • Bridge arm (2.0 rung verbatim + β 0.1 frozen-bridge MSE, canonical r7 labels, 10,995 steps): full-set DER-CE 4.0545 (teacher in-run 2.2093 ✓ reproduction of published 2.2864).
    • Strict tolerance (teacher + 0.5pp = 2.7093): FAIL — honestly recorded.
    • But the frontier MOVED: paired Δ vs G2a = −0.4385pp, CI95 [−0.65, −0.28], p = 0 (G2a's released predictions re-scored under the identical evaluator), at HALF G2a's epochs (3 vs 6), and ~−0.77pp vs the matched 3-epoch 2.0 rung. The first separated-positive student-side lever on record — the ledger's standing rule was satisfied by the pre-registered novel mechanism (scan-first directional ridge pairing + frozen bridges).
    • Run: wo20-distill (final job + predecessors), scan wo20-scan. Provenance: isx-training:/wo20-tiny-g2w/ (preds, final_eval.json), /wo20-tiny-scan/ (scan.json, bridges.npz). Single-seed caveat attached.
    • PRs: interscript-models#442, interscript-train#143.

    Campaign B — static-int8 sweep on released artifacts

    model verdict CER-Δ (≤2.0) confident flips (≤1%) size CPU
    urd-vocalized-1.0 PASS 0.0902 0.00% ×0.359 ×1.91
    tha-phonemic-base-1.0 FAIL 0.66 ✓ 11.01% ✗ ×0.368 ×1.87
    tha-phonemic-1.0 N/A plain-decoder artifact — recipe is KV-only
    heb-vocalized-1.0 FAIL 3.28 ✗ 1.46% ✗ ×0.368 ×1.35
    heb-vocalized-small-1.0 FAIL 7.99 ✗ 0.16% ✓ ×0.359 ×1.35
    heb-vocalized-1.1 PASS 0.2907 0.12% ×0.368 ×1.59

    Key finding: the CER gate alone would have shipped two bad artifacts (tha-base and heb-1.0 pass CER while failing margins — two different failure mechanisms). Confident-flip gating stays mandatory. Every failing model keeps its fp32 artifact; urd-g2p untouched (owner-gated); nothing published, no versions picked — publication of the two PASS variants is the owner's call. Reports + built variants on isx-training:/wo20-quant/. Run: wo20-quant (final + predecessors).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions