Repository navigation
0.3.3: stable release with the integrated documentation set - #58
Merged
Merged
Conversation
…ging - Cluster-locality training is on by default at the entry points (train_chronos.py, planner/presets, bundle finetune), lambda 0.3 from a measured synthetic sweep; the window follows the median SFT record length. The ChronosConfig class default stays 0.0 so stored recipes resume unchanged, and old checkpoints load as lambda 0. - Lazy-expert inference adapts to the checkpoint: models trained with the constraint are paged one training cluster per SSD read, others one expert per read (paging auto|cluster|expert). Layouts get separate cache dirs. - chronos/lazy_experts.py: shared lazy-expert implementation for the Web UI, bundle and server (LazyOptions, page_layout, build_mlx_lazy_engine). - Next-use residency planner for MLX lazy experts, with decay and prefetch wait parameters exposed in the UI, bundle CLI and generate(). - chronos bundle translate/eval --lazy-experts; bundle finetune --[no-]cluster-locality --lambda-cluster --cluster-window --clusters. - chronos serve --paging; /api/ps reports the resolved layout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- chronos-ui / chronos_app.py start the rewritten Web UI (FastAPI + single-page app in chronos/webui, logic in chronos/workbench); --legacy keeps Gradio. - zh-Hans / zh-Hant / en / ja, browser-language detection, hover help after 2 s; buttons, chips, options and picker places are translated. - Path fields: "Browse..." lists the server's folders; "Upload..." opens the native dialog on the server's own machine (path used in place) or uploads to <workdir>/uploads/ from another machine. Several datasets merge into one JSONL. - --lan (off by default): key from CHRONOS_UI_KEY or generated; the server's own machine needs no sign-in. --max-upload-mb, --no-native-dialog. - chronos lazy-probe and per-validation lazy-expert probe in training runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Linux CI MLX is absent, so resolve_backend("mlx") falls back to cpu and the
MLX variants dedupe into the CPU set. The test now resolves backends as
requested, and checks the fallback collapse separately.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The chronos kernel now trains general models with no translation-specific code. Everything that builds the en<->zh benchmark model lives in demos/translation (package chronos_translate, CLI chronos-translate): corpus pipeline, BLEU/chrF runs, doc-mix runs, FLORES+ probe, model bundles, the 50M recipe and the doc-mix plan. It imports the kernel, is not part of the Project_Chronos wheel and is not published on PyPI. Kernel changes: - LongRun task hooks (task_eval, eval_samples_for, generate, write_samples, pinned_modules); task-eval errors are logged as task_eval_error and never stop a run. - load_spec(spec_cls, extra_path_keys) resolves paths against the recipe folder and rejects unknown keys with a pointer to the task runner. - Tokenizer defaults carry no language tags; special_tokens_with() adds task tokens. Shard readers move to chronos.data.shards. - chronos corpus, chronos bundle and the translate extra are removed. Monitoring of the lazy-expert probe: - The monitor gets a Lazy experts tab (four tables, history charts, probe log) and GET /api/lazy_probes. - The Web UI probe card shows every measured field in all four languages. - The run snapshot carries all probe fields plus the full-memory reference; a stray "null" on the Web UI monitor page is fixed. CI also runs on pull requests into dev. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… limit - Router: padding mask, telemetry over all micro-batches, lazy decode uses the grouped kernel (fidelity 1.0 at every budget). - Cluster locality: cluster_locality_mode="segment" learns domain clusters per document from content alone (default stays "window"). - MLX: cache limit min(RAM/8, 16 GiB) applied in the kernel; recipe/env can only lower it. Multimodal fusion port with Torch parity and checkpoint round-trip; pretrain/SFT accept multimodal batches. - Tools: chat template renders <tools>/<tool_call>; chat_records writes per-token .lmask supervision read by SFTRecords; serve passes tools. - Monitor/WebUI: progress-based ETA per phase; stopped runs show their state instead of a clock ETA. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
np.where(mask, uint16_seq, -100) stays uint16 under NEP 50 and wraps -100 to 65436; cast the sequence to int32 first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tool-call inference - chronos.serving.tool_calls: streaming <tool_call> parser that never leaks a partial tag; invalid blocks stay content. - /api/chat returns Ollama message.tool_calls; /v1/chat/completions returns OpenAI tool_calls (string arguments, finish_reason "tool_calls"), streamed or not. Requests without tools are unchanged. Multimodal preference stages on MLX - DPO/ORPO: chosen and rejected condition on the prompt's media. - GRPO: rollouts condition on the prompt's media (encoded once, same logits); the policy update re-encodes so encoders train. - Distill: media reach the student; a text-only teacher is accepted. - Data: MultimodalStreamingDPODataset + preference_collate, load_grpo_prompt_records, shared record_media resolver. - Workbench: media data in DPO/ORPO/GRPO selects MLX under auto; an explicit PyTorch backend raises instead of dropping the media. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- chronos/model/kernel.py replaces the minimind kernel: no clone, no sys.path changes, offline import; logits bit-identical on the existing checkpoints; one shared FP32 RoPE table (3.50 -> 0.50 MiB at 50M). - Dense GQA mode works on Torch and MLX (both crashed before); BF16 release checkpoints keep their dtype; trainer logs reach the Web UI. - decode_visible keeps <tool_call>/<tools> tags that a custom tokenizer registers as special tokens (serving, workbench, GRPO rewards). - MLX: compiled micro step by default (1.08x SFT, 1.18x pretrain measured on the 50M en->zh config) and activation recomputation with replayed draws and per-dtype packed region outputs (mlx 0.32.4 drops gradients of repeated region outputs; peak 5.62 -> 3.50 GB SFT). - RoPE tables built single-threaded (1-ulp prefix mismatch at 18 threads). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
models/chronos-enzh-50m-sft/ packages the 50M Chronos v3 model trained for 69 h on an Apple M5 Max: config, own 16k tokenizer, the trained prompt, decoding settings, a driver (translate.py) and the evaluation. Weights ship as a release asset (enzh-50m-sft-v1), not in git. - Boundary: English -> Chinese only, one sentence of <= 128 tokens per call. zh->en is disabled because en->zh-only SFT erased it (FLORES+ BLEU 0.00 after SFT vs 17.15 for the pretraining checkpoint). - Measured (MLX bf16, greedy): FLORES+ devtest BLEU 33.48 / chrF 29.52, WMT22 32.60 / 28.83, WMT24++ 26.89 / 24.43 (sentence split). - The driver checks the weights' SHA-256, splits sentences, refuses input outside the boundary and cuts repetition loops. - NOTICE: weights are for research use only (corpus licences unknown). - tests/test_model_enzh_50m_sft.py covers the boundary checks without loading weights; the sdist prunes models/. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- download() uses certifi's CA bundle when installed. python.org macOS builds ship no system CAs, so urlretrieve failed with CERTIFICATE_VERIFY_FAILED. A failed download now removes the partial file and says how to fetch the weights by hand. - The CLI runs the language check before building the Translator, so Chinese input is refused (exit 2) even when the weights are absent, instead of failing with "weights not found". - Two tests cover both paths. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit c550537)
…oE, recompute policy, RoPE scaling, packing) Blockwise causal attention (full-causal blocks use the kernel's causal mode), chunked cross-entropy, chunked MoE dispatch, selective recompute policy on MLX and Torch, training-time RoPE scaling (yarn/linear/ntk), document masking with a separate lookahead document switch, a pretrain length curriculum and SFT packing. All switches default to off. 64k full-model step: 24.5 GiB extra peak at 6.6k tok/s. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…led document-reset fix Adds recompute_scope=mlp, mla_upproj_recompute, lookahead_input_detach and temporal_grad_span (all off by default), web UI fields in four languages, and a real-corpus A/B report. Fixes reset_router_at_documents / document_masking crashing inside the compiled LongRun step (found by training on the real corpus) and the "false" string read as true for reset_lookahead_at_documents. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… compile trace - cluster_locality_mode="segment" now traces under mx.compile: document ids come from segment_ids_mx(input_ids, chunk=window) and the loss uses static shapes (B*S segment slots, unused slots weigh 0). Matches the NumPy ids and the Torch loss. - CompiledStep snapshots weights, RNG key and side outputs before a first trace and restores them when the trace fails, so the eager fallback no longer trains on tracers. - tests/test_segment_compile.py (5 cases; 4 fail on 0.3.2a2). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
MLX does not bounds-check gathers, so an id >= vocab_size read neighbouring memory and crashed the process at random. run_mlx_stage (every batch, GRPO prompts) and LongRun now raise a ValueError naming the id range. test_mlx_training_six_stage_loss_paths crashed about one run in two on CPU, on clean dev as well: its GRPO stage tokenizes with the bundled 6400-token tokenizer while the model had vocab_size=64 (ids up to 6170). The test now uses vocab_size=6400 (25/25 runs pass). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…task-neutral kernel Loop (Torch and MLX): recurrent_order "block", recurrent_later_topk, recurrent_later_groups, lambda_iteration_cluster, lookahead_teacher_scope, recurrent_dt_param "sigmoid", and recommended_dt_init(depth) = min(0.5, 2 / depth) (fixed integration horizon T = depth * dt). Kernel defaults are unchanged. The planner now recommends block order, sigmoid and the horizon init for a looped plan; presets_v3.json is regenerated. Fixes: the MLX clip gradient on dt_min/dt_max is now the Torch clamp gradient; the MLX depth jitter is drawn once per optimizer step under gradient accumulation. Auto-tune: the probe trains on real dataset tokens, and recurrent_order and recurrent_dt_init join the search space. Web UI: new fields and four-language help. Tokenizer: chronos/assets/tokenizers/general_32k (32768 tokens, task-neutral control ids, licences in NOTICE) is the planner default for new plans. Kernel text no longer names a translation task; the translation sections moved to demos/translation. CHANGELOG records the physical mechanism, the measurements (one seed per arm, 15M parameters) and the known limits. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…-entropy, demo model under demos/ Planner: a looped plan also sets recurrent_later_topk=0, recurrent_later_groups=0, lambda_iteration_cluster=0, lookahead_teacher_scope="first", recurrent_dt_min=0.05 and recurrent_dt_max=1.0, each with a sources entry stating the 15M measurement behind it. The Web UI shows them as planner recommendations; presets_v3.json is regenerated. Auto-tune: all nine loop knobs are searchable (later_groups 0 or 2, never 1; block order only). The loop probe has 4 block layers, clamps a drawn dt_init into the drawn range and logs each trial's median step time. The probe now trains on chronos_loss_term and scores cross-entropy: the old score added the weighted regularizers, and lambda_lookahead never reached the probe. Web UI: Field.as_dict dropped 0 and 0.0 (0 == False), so min=0 and recommended=0 never reached the page; it now drops only unset values. Help text gives the measured reason in four languages. Translation demo: models/chronos-enzh-50m-sft and its driver test moved under demos/translation; CI runs the driver test in its own step. The main READMEs name the demo in one pointer paragraph; the measurement conditions moved to the demo READMEs. A guard test checks both. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Kernel code is unchanged from 0.3.3a1. This release ships the final documentation and figure set: - docs/KERNEL_ARCHITECTURE.md is the top-level kernel walk-through, with four code-generated architecture figures under docs/assets/arch/ (reproducible generator + symbol config). - Per-component chapters MOE_ROUTING, CLUSTER_LOCALITY, RESIDENCY, LAZY_EXPERTS, LOOKAHEAD, LOOP_DEPTH, LONG_CONTEXT, SPEC_DECODE, each with source-line references and measured numbers traced to recorded JSONs under experiments/figures/source_data/. - 18 redesigned data figures with editable and path-outlined SVGs under docs/assets/; a verify_values.py numeric audit (87 checks / 880 numeric entries; exact except a 1e-12 tolerance on the sigmoid inverse) under experiments/figures/. - Evidence ledger docs/REPRODUCIBILITY.md, symbol table docs/SYMBOLS.md, Chinese reading guide docs/READING_GUIDE_ZH.md. - docs/README.md rewritten with a reading-route diagram, a kernel- interior gallery, and an at-a-glance measured-evidence row. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolves MANIFEST.in by keeping release-0.3.3's expanded prune list (Translation Training Data, demos, chronos/mlx/training/recipes). Resolves README.md / README_zh.md by removing the duplicate 'Released model: chronos-enzh-50m-sft' section that came from main and pointed at the old `models/chronos-enzh-50m-sft/` path. release-0.3.3 moved the model demo to `demos/translation/` and the docs in this branch reference it there throughout, so the merged-in section was both duplicated and pointing at a path that no longer exists in this branch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…test tests/test_lazy_probe.py imports fastapi.testclient inline for the webui-side assertion. The main CI workflow (ci.yml) already installs fastapi/uvicorn/httpx alongside the base deps; the legacy python-app.yml workflow did not, so its build job failed with ModuleNotFoundError: No module named 'fastapi'. Mirror the ci.yml install line. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stable release. No kernel code changes beyond 0.3.3a1; this release bumps the version string and ships the final documentation set.
What landed
docs/KERNEL_ARCHITECTURE.mdis the new top-level kernel walk-through, with four code-generated architecture figures underdocs/assets/arch/(reproducible generator + symbol config).MOE_ROUTING,CLUSTER_LOCALITY,RESIDENCY,LAZY_EXPERTS,LOOKAHEAD,LOOP_DEPTH,LONG_CONTEXT,SPEC_DECODE, each with source-line references, mermaid mechanism diagrams and measured numbers traced to recorded JSONs underexperiments/figures/source_data/.docs/assets/;verify_values.pynumeric audit (87 checks / 880 numeric entries; exact except a 1e-12 tolerance on the sigmoid inverse) underexperiments/figures/.docs/REPRODUCIBILITY.md, symbol tabledocs/SYMBOLS.md, Chinese reading guidedocs/READING_GUIDE_ZH.md.docs/README.mdrewritten with a reading-route diagram, a kernel-interior gallery, and an at-a-glance measured-evidence row.What did not change
Kernel code, trainer, planner, auto-tune, Web UI, serving adapter. Existing configs and checkpoints behave as in 0.3.3a1.
🤖 Generated with Claude Code