Repository navigation
fix(ml): reject float16 graph-I/O feeds with the artifact-contract error (#111 root cause) - #134
Merged
Merged
Conversation
Issue #111's "not enough space: expected N, got 0" from onnxruntime-node's session.run is float16 graph I/O meeting the binding, not a windowing or size bug: - The issue-era fp16 exports shipped float16 graph boundaries (last_hidden_state / encoder_hidden_states / logits). Python ORT consumes fp16 I/O natively, so the reference decoder and the frozen vectors never saw it (the models-repo WO79 finding). - On runtimes with native Float16Array (Node >= 24), ORT's JS Tensor converts every float16 feed into a Float16Array view whose bytes the N-API binding reads as zero, and run() fails with the opaque error. The issue's numbers decode exactly as seq x d_model x 2 bytes of feed with a 0-byte buffer: fas 288512 = 98 x 1472 (byt5-small) x 2, urd 73728 = 24 x 1536 (byt5-base) x 2. Size-independent because "got" is always the buffer length, not the artifact. - Node 20/22 (no Float16Array) consume the same feed as Uint16Array and decode fine: verified 20.20.2/22.20.0 ok, 24.14.0/26.5.0 broken. The shipped artifacts have been IO-normalized since 2026-10-21 (models-repo WO79 #327 replaced both release zips), so both conformance legs pass today; the runtime had no defense against the next fp16-IO export and CI cannot see it (the tiny fixtures are fp32-I/O). Fix: reject float16 feeds at the onnx-node binding boundary with the artifact-contract error (input name + normalize_fp16_io.py pointer) instead of the opaque binding failure, and type Tensor honestly for what already flows (float16 / Uint16Array). fp32-I/O artifacts never construct a float16 feed, so the guard is a no-op for them; the browser (WASM) backend handles fp16 feeds itself and is untouched. Regression: test/fixtures/tiny-fp16io.zip (~1KB, the pre-WO79 export shape Cast-wrapped from tiny-imf.zip with recomputed member digests) plus test/imf-fp16io.test.ts, which reproduced the opaque binding error before the fix and locks both runtime branches (decode on Node 20/22, explicit rejection on Node 24+).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root cause (issue #111)
The
not enough space: expected N, got 0failure from onnxruntime-node'ssession.runis float16 graph I/O meeting the native binding — not awindowing bug and not a memory/size problem.
The issue-era artifacts (fas-diac-base-1.0, urd-diac-base-1.0, released
2026-10-18) were fp16 exports whose graph boundaries stayed float16:
last_hidden_state,encoder_hidden_states,logits. Python ORTconsumes fp16 I/O natively, so the reference decoder — and the frozen
conformance vectors generated with it — never saw the condition; every
other runtime breaks (the models-repo WO79 finding,
tools/normalize_fp16_io.py's own docstring).Mechanism, reproduced from a 1KB fixture on current HEAD
last_hidden_statecomes back astype=float16 dims=[1,4] data=Float16Array(4).encoder_hidden_states. ORT's JSTensorconstructor converts every float16 feed into aFloat16Arrayview when the runtime has native Float16Array(Node >= 24) —
onnxruntime-commontensor-impl.js(Uint16Array→ Float16Array wrap).
it sees byte length 0 — and
run()fails:not enough space: expected 8, got 0from the binding(
lib/backend.ts:130, the exact stack location and signature fromthe issue).
The issue's own numbers confirm it
"expected" = dims product × the float16 element size of the
encoder_hidden_statesfeed, "got" = the feed's byte length (0):288512 = 98 × 1472 × 2— vector row 1 is 97 UTF-8 bytes →seq 98 (bytes + EOS); d_model 1472 (byt5-small); fp16 element size.
73728 = 24 × 1536 × 2— vector row 1 is 23 bytes → seq 24;d_model 1536 (byt5-base); fp16 element size.
"got" is always the buffer length, never the artifact size — which is
why the failure is size-independent (711MB and 1.35GB artifacts fail
identically).
Node-version dependence (verified empirically)
Float16ArrayWhy it stopped reproducing — and why it is still live
WO79 (#327) shipped
tools/normalize_fp16_io.pyand replaced bothrelease zips in place with fp32-graph-I/O repacks. Both frozen
conformance legs pass byte-exactly today (fas 12/12, urd 12/12,
re-verified on this branch).
artifact exported with fp16 graph I/O crashes on Node >= 24 with the
opaque binding error, and CI cannot see it — the tiny fixture zips
are fp32-I/O (
test/fixtures/tiny-imf.zip). The issue's originalwindowing suspicion (
windows.ts) is exonerated: the failurereproduces on a 3-char input through the single-window
translateDirectpath, with no window split involved.The fix
Minimal, at the binding boundary where the condition is known
(
src/ml/session/onnx-node.ts):runtime where the binding cannot consume it, reject it with the
artifact-contract error — naming the offending input and the
re-export path (
tools/normalize_fp16_io.py) — instead of theopaque binding failure.
Tensor.type/TensorDatanow includefloat16/Uint16Array:fp16 outputs have been flowing through these paths via an unsound
cast; the types now say what actually flows.
No decode-path behavior change: fp32-I/O artifacts (all shipped ones)
never construct a float16 feed, so the guard is a no-op for them; the
browser (WASM) backend handles fp16 feeds itself and is untouched.
Regression coverage
test/fixtures/tiny-fp16io.zip(~1KB):tiny-imf.zipwith float16Cast-wrapped across both graph boundaries (encoder output, decoder
input, decoder output) and recomputed member digests — the pre-WO79
export shape, CI-loadable.
test/imf-fp16io.test.tslocks both behaviors. Red first: againstthe unguarded base it failed with exactly
got 'not enough space: expected 8, got 0'. Green after the fix onNode 24 (rejection branch) and on Node 22.20.0 (decode branch).
Verification
test/imf-fp16io.test.ts: green on Node 24.14.0 and Node 22.20.0post-fix (226.9s); urd leg 12/12 (106.5s, base commit)
npx vitest run: 352 passed; the only failing file istest/cli.test.ts(13 tests) — verified failing identically at thebase commit (2186ebe) in a clean base worktree, pre-existing and
unrelated
tsc --noEmit, eslint, prettier cleanCloses #111 (the shipped artifacts already conform; this closes the
runtime-side hole and locks the contract with a CI-loadable fixture).
Fixes #111.