From 062f35f5a4fc07b362dabe2b00c9368849c36998 Mon Sep 17 00:00:00 2001 From: Fona <62028332+FonaTech@users.noreply.github.com> Date: Sat, 14 Nov 2026 02:30:55 +0800 Subject: [PATCH] docs: fix GitHub MathJax rendering across docs (0.3.3 docs-only) GitHub's MathJax configuration disallows \operatorname, leaves \! (negative thin space) as a literal '!', and trips on a bare 'h^*' superscript inside an aligned block. Together these cascaded into equations K.4-K.9 of docs/KERNEL_ARCHITECTURE.md showing as raw text and eq (3) of docs/LAZY_EXPERTS.md losing its multiplication sign. Fixes (docs/, 10 files): - \operatorname{X} -> \mathrm{X}; \operatorname*{X}_y -> \mathop{\mathrm{X}}\limits_y (preserves limit position). - Bare h^* -> h^{\ast}. - Drop \! everywhere in math (ATTENTION, CLUSTER_LOCALITY, LAZY_EXPERTS, LONG_CONTEXT, SPEC_DECODE, THEORY). - LAZY_EXPERTS eq (3) rewritten as Q = max(1, 2d), d = \texttt{prefetch\_depth}, and the RAM-budget expression as B * 2^20 / p_page, B = \texttt{ram\_budget\_mb}, to avoid \mathrm{...\_...} MathJax ambiguity. Version string unchanged at 0.3.3: docs are not shipped in the wheel (MANIFEST.in and setuptools packages.find exclude them), so the PyPI 0.3.3 release remains valid; only the GitHub v0.3.3 tag needs to point at this fix. Matplotlib-mathtext strings in docs/assets/arch/make_arch.py and the generated SVG XML comments are unaffected (different renderer). Co-Authored-By: Claude Opus 5.5 --- docs/ATTENTION.md | 8 ++++---- docs/CLUSTER_LOCALITY.md | 4 ++-- docs/KERNEL_ARCHITECTURE.md | 12 ++++++------ docs/LAZY_EXPERTS.md | 8 ++++++-- docs/LONG_CONTEXT.md | 4 ++-- docs/LOOKAHEAD.md | 18 ++++++++++-------- docs/MOE_ROUTING.md | 4 ++-- docs/OVERVIEW.md | 4 ++-- docs/SPEC_DECODE.md | 4 ++-- docs/THEORY.md | 28 ++++++++++++++-------------- docs/vllm_integration.md | 4 ++-- 11 files changed, 52 insertions(+), 46 deletions(-) diff --git a/docs/ATTENTION.md b/docs/ATTENTION.md index 124f29d..e6a70a0 100644 --- a/docs/ATTENTION.md +++ b/docs/ATTENTION.md @@ -46,7 +46,7 @@ Valid grouped-head shapes require the query heads to map consistently onto KV gr For an attention input $z$, the normalized shared KV latent is $$ -c_t=\operatorname{RMSNorm}_{KV}(W_{DKV}z_t)\in\mathbb R^{d_c}. +c_t=\mathrm{RMSNorm}_{KV}(W_{DKV}z_t)\in\mathbb R^{d_c}. \tag{A1} $$ @@ -55,9 +55,9 @@ RMSNorm denotes root mean square normalization. The projections for a query head $$ \begin{aligned} q^C_{t,h}&=N_Q^C(W_{QC,h}z_t)\in\mathbb R^{d_n},\\ -q^R_{t,h}&=\operatorname{RoPE}_t\!\left(N_Q^R(W_{QR,h}z_t)\right)\in\mathbb R^{d_r},\\ +q^R_{t,h}&=\mathrm{RoPE}_t\left(N_Q^R(W_{QR,h}z_t)\right)\in\mathbb R^{d_r},\\ k^C_{s,m}&=W_{KC,m}c_s\in\mathbb R^{d_n},\\ -k^R_{s,m}&=\operatorname{RoPE}_s(W_{KR,m}c_s)\in\mathbb R^{d_r},\\ +k^R_{s,m}&=\mathrm{RoPE}_s(W_{KR,m}c_s)\in\mathbb R^{d_r},\\ v_{s,m}&=W_{V,m}c_s\in\mathbb R^{d_h}. \end{aligned} \tag{A2} @@ -96,7 +96,7 @@ $$ Causal, padding and document masks define the allowed positions. At evaluation, or with attention dropout disabled, softmax and value aggregation are $$ -a_{t,s,h}=\operatorname{softmax}_{s}(s_{t,s,h}+M_{t,s}),\qquad +a_{t,s,h}=\mathrm{softmax}_{s}(s_{t,s,h}+M_{t,s}),\qquad u_{t,h}=\sum_sa_{t,s,h}v_{s,m(h)}. \tag{A4} $$ diff --git a/docs/CLUSTER_LOCALITY.md b/docs/CLUSTER_LOCALITY.md index 13ba623..6ff5b5b 100644 --- a/docs/CLUSTER_LOCALITY.md +++ b/docs/CLUSTER_LOCALITY.md @@ -26,7 +26,7 @@ $$ The current training path obtains $M$ from `training_clusters(config)`. It splits expert IDs into contiguous, near-equal groups. The cluster count is the explicit positive `cluster_locality_clusters`, or otherwise $$ - N_c=\operatorname{clip}_{[1,E]}\!\left(\left\lceil\frac{E}{2k}\right\rceil\right). + N_c=\mathrm{clip}_{[1,E]}\left(\left\lceil\frac{E}{2k}\right\rceil\right). $$ There is no implemented `clusters.partition_json` training field in this snapshot. A cluster manifest used by storage is a separate object. [Partition implementation](../chronos/expert_clusters.py#L28). @@ -174,7 +174,7 @@ The implementation uses mean total variation from the iteration mean: $$ L_{\mathrm{iter}} - =\operatorname{mean}_{b,t,g,r} + =\mathrm{mean}_{b,t,g,r} \left[\frac12\sum_c|q_{b,t,g,r,c}-\bar q_{b,t,g,c}|\right], $$ diff --git a/docs/KERNEL_ARCHITECTURE.md b/docs/KERNEL_ARCHITECTURE.md index 5050ac2..d4a04a3 100644 --- a/docs/KERNEL_ARCHITECTURE.md +++ b/docs/KERNEL_ARCHITECTURE.md @@ -34,7 +34,7 @@ The input operation is a lookup. The output operation is a linear projection: $$ h_{b,t}^{(0)}=W_{\mathrm{in}}[x_{b,t}],\qquad -z_{b,t}=W_{\mathrm{out}}\operatorname{RMSNorm}(h_{b,t}^{\mathrm{coda}}), +z_{b,t}=W_{\mathrm{out}}\mathrm{RMSNorm}(h_{b,t}^{\mathrm{coda}}), \quad W_{\mathrm{in}},W_{\mathrm{out}}\in\mathbb R^{V\times d}. \tag{K.1} $$ @@ -86,13 +86,13 @@ Physical parity includes the prelude offset. Recurrent group $g$ occupies physic For the default SiLU activation, a bias-free SwiGLU expert with intermediate width $m$ is $$ -E(h)=W_d\bigl[\operatorname{SiLU}(W_g h)\odot(W_u h)\bigr]. +E(h)=W_d\bigl[\mathrm{SiLU}(W_g h)\odot(W_u h)\bigr]. \tag{K.3} $$ Its three matrices contain $3dm$ weights. Dense and expert intermediate widths are separate configuration fields. There is no general requirement that either equals $4d$. -For routed expert probabilities $p=\operatorname{softmax}_{\mathrm{FP32}}(W_r h)$, let $I$ be the selected top-$k$ set. The normal fully available MoE path is +For routed expert probabilities $p=\mathrm{softmax}_{\mathrm{FP32}}(W_r h)$, let $I$ be the selected top-$k$ set. The normal fully available MoE path is $$ M(h)=\sum_{i\in I}a_iE_i(h)+\bar S(h),\qquad @@ -110,8 +110,8 @@ The implemented contracted step is indexed by recurrent group and iteration. It $$ \begin{aligned} -h^*&=h+e_r+u_g,\\ -\Phi_{r,g}(h)&=h+\alpha_g\left[F_{P+g}(h^*;\mathcal K_g^{(0)})-h\right]. +h^{\ast}&=h+e_r+u_g,\\ +\Phi_{r,g}(h)&=h+\alpha_g\left[F_{P+g}(h^{\ast};\mathcal K_g^{(0)})-h\right]. \end{aligned} \tag{K.5} $$ @@ -202,7 +202,7 @@ in bounded decode. A shared prefill view can be longer than the retained cache. Define expert-to-cluster membership $A_{i,c}\in\{0,1\}$. Aggregate routing probabilities over tokens and dispatches within a stream unit $w$: $$ -p(c\mid w)=\sum_i A_{i,c}\,\operatorname{mean}_{(t,v)\in w}p_{t,v,i}. +p(c\mid w)=\sum_i A_{i,c}\,\mathrm{mean}_{(t,v)\in w}p_{t,v,i}. \tag{K.9} $$ diff --git a/docs/LAZY_EXPERTS.md b/docs/LAZY_EXPERTS.md index 52350f7..e47b447 100644 --- a/docs/LAZY_EXPERTS.md +++ b/docs/LAZY_EXPERTS.md @@ -60,10 +60,12 @@ Torch replaces evicted experts with `LazyExpertPlaceholder` modules. Before disc `AsyncPrefetcher` owns a `queue.Queue`, not a deadline-priority queue. Each entry is a batch of expert IDs. `CacheManager` sets its capacity to $$ -Q_{\mathrm{batches}}=\max\!\left(1,\,2\,\mathrm{prefetch\_depth}\right). +Q_{\mathrm{batches}}=\max\left(1,\;2d\right). \tag{3} $$ +Here $d$ denotes the `prefetch_depth` configuration value. + The default `prefetch_depth=2` therefore gives four request batches through this manager. The standalone prefetcher constructor has its own default of eight batches. Neither value means a fixed number of outstanding SSD system calls. [Manager construction](../chronos/runtime/cache_manager.py#L35), [queue implementation](../chronos/io/async_prefetcher.py#L47) ```mermaid @@ -194,11 +196,13 @@ The code neither zeros missing terms nor renormalizes the surviving experts afte $$ K_{\mathrm{RAM}}=\max\left(1,\left\lfloor -\frac{\mathrm{ram\_budget\_mb}\,2^{20}}{p_{\mathrm{page}}} +\frac{B\cdot 2^{20}}{p_{\mathrm{page}}} \right\rfloor\right). \tag{8} $$ +Here $B$ denotes the `ram_budget_mb` value in megabytes and $p_{\mathrm{page}}$ is the on-disk page size in bytes. + A requested budget smaller than one page cannot create a sub-page cache. These hard counts coexist with backend safety heuristics and dynamic pressure handling. They are not a bound on the entire process, KV cache, activations or transient cluster materialization. [Budget conversion](../chronos/io/expert_store.py#L31) The MLX allocator cache limit, `min(physical RAM / 8, 16 GiB)`, bounds reclaimable Metal buffer cache. It is **not** the resident-expert budget. Live arrays remain active memory. [Allocator limit](../chronos/mlx/memory.py#L1) diff --git a/docs/LONG_CONTEXT.md b/docs/LONG_CONTEXT.md index 0e842ef..a621013 100644 --- a/docs/LONG_CONTEXT.md +++ b/docs/LONG_CONTEXT.md @@ -123,8 +123,8 @@ For batch $B$, query heads $H_q$ and head width $d_h$, the leading full-causal a For the implemented SWA block span, a corresponding work estimate is $$ -O\!\left(\frac{T}{c}\,BH_q\,c(W+c)d_h\right) -=O\!\left(BH_qT(W+c)d_h\right). +O\left(\frac{T}{c}\,BH_q\,c(W+c)d_h\right) +=O\left(BH_qT(W+c)d_h\right). \tag{7} $$ diff --git a/docs/LOOKAHEAD.md b/docs/LOOKAHEAD.md index ff8dcbf..9cea999 100644 --- a/docs/LOOKAHEAD.md +++ b/docs/LOOKAHEAD.md @@ -13,9 +13,11 @@ That architectural separation does not make prediction errors harmless. A wrong Use $t$ for token position, $\delta$ for future token offset, $g$ for a recurrent physical group and $r$ for recurrent iteration. The future horizon is $$ - \delta\in\{1,\ldots,K\},\qquad K=\texttt{lookahead\_steps}. +\delta\in\{1,\ldots,K\}. $$ +Here $K$ denotes the `lookahead_steps` configuration value. + The head additionally emits slot zero. Its output is $$ @@ -58,7 +60,7 @@ flowchart TB Let $d$ be backbone width and $m=\max(1,\lfloor d/4\rfloor)$. The current head is $$ - u=\operatorname{SiLU}(W_1h),\qquad z=W_2u,\qquad Q=\operatorname{softmax}_E(\operatorname{reshape}(z)), + u=\mathrm{SiLU}(W_1h),\qquad z=W_2u,\qquad Q=\mathrm{softmax}_E(\mathrm{reshape}(z)), $$ with bias-free matrices @@ -111,12 +113,12 @@ The teacher is a **soft future routing distribution**, not just hard expert IDs. The accepted string is `"iterations"`, not `"all"`. Averaging probabilities across recurrent calls is not taking a union of their hard selected IDs. The separate future-union objective below concerns **token offsets**. [Scope validation](../chronos/model/config.py#L301), [teacher sources](../chronos/trainer/loss_mixin.py#L101). -When predictions lack a group axis, teacher probabilities are averaged across physical layers. With a group axis, the trainer selects physical recurrent layers $P$ through $P+G-1$. In either case, the teacher is stop-gradient. Denote that aligned distribution by $\operatorname{sg}(P_{b,t,g,e})$. +When predictions lack a group axis, teacher probabilities are averaged across physical layers. With a group axis, the trainer selects physical recurrent layers $P$ through $P+G-1$. In either case, the teacher is stop-gradient. Denote that aligned distribution by $\mathrm{sg}(P_{b,t,g,e})$. Input detachment is a different decision: $$ - h_{\mathrm{LA}}=\begin{cases}\operatorname{sg}(h),&\text{input detach enabled},\\h,&\text{otherwise}.\end{cases} + h_{\mathrm{LA}}=\begin{cases}\mathrm{sg}(h),&\text{input detach enabled},\\h,&\text{otherwise}.\end{cases} $$ `lookahead_input_detach` defaults to false. MLX applies the switch at the head call. In the reviewed Torch call sites, the flag is not consulted: changing it alone does not create a stopped-gradient input there. This is an implementation-parity boundary, not a feature that documentation can assume works identically. [Config](../chronos/model/config.py#L166), [MLX switch](../chronos/mlx/model.py#L269), [Torch call](../chronos/model/model_chronos.py#L492). @@ -145,14 +147,14 @@ For valid future offset $\delta$, align prediction at $t$ with routing at $t+\de $$ \ell_{b,t,g}^{(\delta)} - =-\sum_e\operatorname{sg}(P_{b,t+\delta,g,e})\log Q_{b,t,\delta,g,e}. + =-\sum_e\mathrm{sg}(P_{b,t+\delta,g,e})\log Q_{b,t,\delta,g,e}. $$ The loss averages valid token/group examples within each offset, then averages available offsets: $$ L_{\mathrm{LA}}=\frac1{|\mathcal D|}\sum_{\delta\in\mathcal D} - \operatorname{mean}_{b,t,g:\,t+\delta0$; omit the shared branch when no shared experts are Let $A_{ic}$ be the fixed expert-to-cluster membership matrix. Aggregate routing mass across tokens and all dispatches in a window $w$: $$ -p_w(c)=\operatorname{mean}_{(b,t,v)\in w}\sum_i p_{b,t,v,i}A_{ic}. +p_w(c)=\mathrm{mean}_{(b,t,v)\in w}\sum_i p_{b,t,v,i}A_{ic}. \tag{T15} $$ @@ -210,8 +210,8 @@ For more than one cluster, the implemented window objective is $$ \mathcal L_{\mathrm{cluster}} -=1-\frac{H(\overline p)-\operatorname{mean}_wH(p_w)}{\log N_c}, -\qquad \overline p=\operatorname{mean}_wp_w. +=1-\frac{H(\overline p)-\mathrm{mean}_wH(p_w)}{\log N_c}, +\qquad \overline p=\mathrm{mean}_wp_w. \tag{T16} $$ @@ -247,13 +247,13 @@ For a detached teacher distribution $P$, the basic supervision is soft-target cr $$ \mathcal L_{\mathrm{LA}} -=\operatorname{mean}_{j\in\mathcal J} -\left[-\operatorname{mean}_{b,t,g} +=\mathrm{mean}_{j\in\mathcal J} +\left[-\mathrm{mean}_{b,t,g} \sum_iP_{b,t+j,g,i}\log Q_{b,t,j,g,i}\right]. \tag{T18} $$ -Only horizons $\mathcal J$ with available token pairs contribute. The implementation clamps predicted probabilities below at $10^{-9}$ before taking logs. For fixed detached $P$, soft-target cross-entropy equals $\operatorname{KL}(P\|Q)+H(P)$. The teacher entropy has no gradient through this loss, so its optimization direction matches forward KL. +Only horizons $\mathcal J$ with available token pairs contribute. The implementation clamps predicted probabilities below at $10^{-9}$ before taking logs. For fixed detached $P$, soft-target cross-entropy equals $\mathrm{KL}(P\|Q)+H(P)$. The teacher entropy has no gradient through this loss, so its optimization direction matches forward KL. Padding and document-boundary handling depend on the active training path. This loss differs from classifying one cluster ID. Expert-to-page mapping occurs later in the runtime. Code: [lookahead_supervision_loss](../chronos/model/temporal_loss.py#L259). @@ -282,7 +282,7 @@ $$ D_{\mathrm{used}}=\sum_jjw_j, \qquad \mathcal L_{\mathrm{ponder}}= -\operatorname{mean}\max(D_{\mathrm{used}}-1,0). +\mathrm{mean}\max(D_{\mathrm{used}}-1,0). \tag{T20} $$ diff --git a/docs/vllm_integration.md b/docs/vllm_integration.md index ff08485..068a426 100644 --- a/docs/vllm_integration.md +++ b/docs/vllm_integration.md @@ -111,8 +111,8 @@ flowchart LR For MLA, the adapter must define whether it stores latents and rotary keys or expanded K/V, and how the attention kernel reconstructs the same scores. For SWA, it must preserve the moving window and absolute end index. For speculation, append and rollback must commute with $\Phi$: $$ -\Phi(\operatorname{truncate}(C,n)) -=\operatorname{truncate}_{\mathrm{engine}}(\Phi(C),n). +\Phi(\mathrm{truncate}(C,n)) +=\mathrm{truncate}_{\mathrm{engine}}(\Phi(C),n). \tag{3} $$