Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 40 additions & 2 deletions .claude/knowledge/add-pm-probe.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,13 +61,45 @@ D2 normalization not gated on presence (3);
D3 saturated sum treated as exact, carry forced 0 (4);
D4 opposite polarity takes the smaller magnitude's sign (4).

## Subtraction (`neg_pm`, `sub_pm`)

`sub_pm(a, b) = add_pm(a, neg_pm(b))`, with `neg_pm` flipping the polarity
on lanes with a magnitude: `P_¬b = P_b ⊕ (M_b ≠ 0)`. XOR governs polarity;
magnitudes stay unsigned. A zero magnitude keeps its polarity, so the
never-touched code and `+0` are fixed points and `−0` is never produced.
There is no separate subtractor.

| test | what |
|---|---|
| `negation_of_every_codeword_matches_the_oracle` | all 512 codewords: value negated, magnitude and touched kept |
| `every_codeword_pair_difference_matches_the_i32_oracle` | all 512 × 512 pairs; carry count = the touched opposite-polarity pairs with \|a\|+\|b\| > 255 (65,280) |
| `sub_pm_equals_negate_then_add_on_every_pair` | bit-identical to `add_pm(a, neg_pm(b))` |
| `the_four_mandatory_cases` | −5 + +5 = +0; −0.499999 encodes to +0 (scale 1, ties even, batch encoder) and stays touched; +0 − +0 = +0; never touched − never touched = never touched |
| `a_difference_fold_with_the_carry_escape_is_exact` | 2,000-block subtraction fold, carry lanes escaped to a wide tally, equals the i64 result |

Disable controls, each red: polarity flipped on every lane (3 tests);
never flipped (3); flipped only on zero magnitudes (4).

**Backends.** The 11 tests of `tests/add_pm.rs` pass on all seven tiers:
AVX-512 (native), AVX2 (`config-v3`), AVX (`config-avx`, qemu SandyBridge),
x86-64-v2 / scalar backend (`config-v2`, qemu Nehalem), NEON (aarch64 cross
build, qemu), wasm32 +simd128 and wasm32 scalar (wasip1 under node's WASI).
CI runs the first five rows through `scripts/add-pm-backends.sh`, which fails
on zero executed tests. The wasm rows ran once by hand: the crate's
`criterion` dev-dependency does not build for `wasm32-wasip1`, so they were
built from a copy of the test file in a crate without dev-dependencies.

## Cost (ns per 64 lanes, best of 50, 4,096 blocks, L2-resident)

| tier | `add_pm` | decode → i32 → encode |
|---|---|---|
| AVX-512 (default `target-cpu=native`, built `avx512bw=true`) | 11.8 – 11.9 | 89.7 |
| AVX2 (`--config .cargo/config-v3.toml`, built `avx512bw=false`) | 284.7 – 285.3 | 133.5 – 134.7 |

`sub_pm` adds the negation's one compare and XOR: 11.7 ns against `add_pm`'s
10.2 ns on AVX-512, 244.8 against 240.3 on AVX2 (re-measured with this
change, same host; the AVX2 gap is still the array-backed `U8x64`).

On AVX-512 the in-place sum is about 7.5x the decode path. On AVX2 it is
about 2x SLOWER, and not because of the algebra: the AVX2 `U8x64` is
array-backed, and its `mask_blend`, `saturating_add` and `saturating_sub` are
Expand All @@ -93,5 +125,11 @@ the carry is the escape. Not adopted into `codec9` or the facade.
lanes exactly; carries exceed them by the deltas that came back (churn,
batch 65,536: 994 carries − 266 demotions = 728 overflows). Degrees stay
exact, with an exact wide escape and demotion back to a lane.
- Only addition is covered. Subtraction is addition with the polarity bit
flipped on touched lanes (an untouched lane must stay untouched); not built.
- ~~Only addition is covered. Subtraction is addition with the polarity bit
flipped on touched lanes (an untouched lane must stay untouched); not built.~~
Done: § Subtraction (the flip is on lanes with a magnitude, which keeps +0 too).
- Promote `add_pm` / `neg_pm` / `sub_pm` from this example to an experimental
`ndarray::hpc::abakus::pm` module (Codec9 keeps the representation), then
drop the lab copy in lance-graph `xlog-relcomp-probe`.
- wasm rows of `scripts/add-pm-backends.sh`: blocked on `criterion` not
building for `wasm32-wasip1`.
8 changes: 8 additions & 0 deletions .github/workflows/simd-matrix.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,8 @@ jobs:
run: CARGO_ARGS='--config .cargo/config-v3.toml' bash scripts/masking-parity.sh native
- name: codegen witness (avx2, pinned to v3)
run: CARGO_ARGS='--config .cargo/config-v3.toml' bash scripts/codegen-witness.sh avx2
- name: add_pm / sub_pm probe tests (x86-64-v3, pinned)
run: bash scripts/add-pm-backends.sh avx2
- name: AMX realization report (runtime-gated; prints this runner's gates)
run: cargo run --example amx_realization_report
- name: AMX encoding + detection tests (no tile op executes)
Expand Down Expand Up @@ -302,6 +304,8 @@ jobs:
# One thread: qemu-user itself has crashed (internal SIGSEGV) under
# the multithreaded harness while a test was panicking.
qemu-x86_64-static -cpu SandyBridge "$bin" simd --test-threads=1
- name: add_pm / sub_pm probe tests under qemu -cpu SandyBridge
run: bash scripts/add-pm-backends.sh avx-qemu

v2:
# x86-64-v2 realization (SSE4.2, no AVX): `simd.rs` routes it to the
Expand Down Expand Up @@ -349,6 +353,8 @@ jobs:
| jq -r 'select(.executable != null and .target.kind == ["lib"] and .target.name == "ndarray") | .executable')
# One thread: see the `avx` job.
qemu-x86_64-static -cpu Nehalem "$bin" simd --test-threads=1
- name: add_pm / sub_pm probe tests under qemu -cpu Nehalem
run: bash scripts/add-pm-backends.sh v2-qemu

neon:
# aarch64 realization: cross-build on the x86 runner, run under qemu-user.
Expand Down Expand Up @@ -376,6 +382,8 @@ jobs:
run: CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc bash scripts/codegen-witness.sh neon aarch64-unknown-linux-gnu
- name: NEON asm rung 3
run: CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc bash scripts/neon-asm-rung3.sh
- name: add_pm / sub_pm probe tests (aarch64, qemu)
run: bash scripts/add-pm-backends.sh neon-qemu

wasm:
# wasm32 with +simd128 = the `simd_wasm` realization; wasm32 WITHOUT it is
Expand Down
3 changes: 2 additions & 1 deletion examples/add_pm/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ use std::hint::black_box;
use std::time::Instant;

use ndarray::simd::U8x64;
use pm::{add_decoded, add_pm, Pm};
use pm::{add_decoded, add_pm, sub_pm, Pm};

fn blocks(n: usize, seed: u64) -> Vec<Pm> {
let mut s = seed;
Expand Down Expand Up @@ -70,5 +70,6 @@ fn main() {
for _ in 0..2 {
time("add_pm (in place)", &a, &b, add_pm);
time("add_decoded (decode -> i32 -> encode)", &a, &b, add_decoded);
time("sub_pm (negate, then add_pm)", &a, &b, sub_pm);
}
}
30 changes: 30 additions & 0 deletions examples/add_pm/pm.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@
//! The carry mask is the exact-escape signal Codec9's own encoder lacks (it
//! saturates silently): a caller applies the exact sum of those lanes
//! through a wider path. Off the carry lanes the result is exact.
//!
//! Subtraction is the same addition of the polarity-negated operand
//! ([`neg_pm`], [`sub_pm`]). XOR governs polarity; magnitudes stay unsigned.
use ndarray::simd::U8x64;

/// 64 `P × M` lanes.
Expand Down Expand Up @@ -69,6 +72,27 @@ pub fn add_pm(a: Pm, b: Pm) -> (Pm, u64) {
(Pm { mag, pos }, carry)
}

/// `-b` lane-wise: the polarity bit flips on lanes with a magnitude.
///
/// `P_¬b = P_b ⊕ (M_b ≠ 0)`. A zero magnitude keeps its polarity, so the
/// never-touched code (`P=0, M=0`) stays never touched and `+0` stays `+0`:
/// `-0` is never produced, and remains only the never-touched sentinel.
#[inline(always)]
pub fn neg_pm(b: Pm) -> Pm {
let nonzero = !b.mag.cmpeq_mask(U8x64::splat(0));
Pm {
mag: b.mag,
pos: b.pos ^ nonzero,
}
}

/// `a ⊖ b` lane-wise: `a ⊕ (-b)`, with `add_pm`'s carry contract. There is
/// no separate subtractor.
#[inline(always)]
pub fn sub_pm(a: Pm, b: Pm) -> (Pm, u64) {
add_pm(a, neg_pm(b))
}

// ------------------------------------------------------------------ scalar reference
// Decode to i32 and back: the path `add_pm` avoids, and the oracle it is checked by.

Expand All @@ -95,6 +119,12 @@ pub fn add_ref(a: Option<i32>, b: Option<i32>) -> (Option<i32>, bool) {
}
}

/// The reference difference of one lane: `a - b`, never touched only when
/// both are, and overflow outside ±255.
pub fn sub_ref(a: Option<i32>, b: Option<i32>) -> (Option<i32>, bool) {
add_ref(a, b.map(|y| -y))
}

/// `add_pm` by decode → i32 → encode, 64 lanes. Carry lanes are saturated.
pub fn add_decoded(a: Pm, b: Pm) -> (Pm, u64) {
let (am, bm) = (a.mag.to_array(), b.mag.to_array());
Expand Down
34 changes: 34 additions & 0 deletions scripts/add-pm-backends.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
#!/usr/bin/env bash
# Runs tests/add_pm.rs (the Codec9 P x M add/sub probe) on one SIMD tier, so
# the exhaustive codeword oracle and the mandatory sign cases are checked on
# that backend, not only on the host tier.
#
# scripts/add-pm-backends.sh avx2 x86-64-v3 build, native run
# scripts/add-pm-backends.sh avx-qemu sandybridge build, qemu -cpu SandyBridge
# scripts/add-pm-backends.sh v2-qemu x86-64-v2 build (scalar backend), qemu -cpu Nehalem
# scripts/add-pm-backends.sh neon-qemu aarch64 cross build, qemu-aarch64
#
# A run that executes zero tests is a failure: the file is gated on `std`.
set -euo pipefail
ARM="${1:?avx2|avx-qemu|v2-qemu|neon-qemu}"
export CARGO_PROFILE_DEV_DEBUG=0 CARGO_PROFILE_TEST_DEBUG=0
build() { # $1 = cargo config file or "", rest = test args; prints the test binary
local cfg=(); [ -n "$1" ] && cfg=(--config "$1"); shift
env -u RUSTFLAGS cargo "${cfg[@]}" test --release --features std --test add_pm --no-run --message-format=json "$@" \
| jq -r 'select(.executable != null and .target.name == "add_pm") | .executable'
}
case "$ARM" in
avx2) bin=$(build .cargo/config-v3.toml); run=("$bin") ;;
avx-qemu) bin=$(build .cargo/config-avx.toml); run=(qemu-x86_64-static -cpu SandyBridge "$bin") ;;
v2-qemu) bin=$(build .cargo/config-v2.toml); run=(qemu-x86_64-static -cpu Nehalem "$bin") ;;
neon-qemu)
export CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc
bin=$(build "" --target aarch64-unknown-linux-gnu)
run=(qemu-aarch64-static -L /usr/aarch64-linux-gnu "$bin") ;;
*) echo "unknown arm: $ARM" >&2; exit 2 ;;
esac
[ -n "$bin" ] || { echo "::error::no add_pm test binary built" >&2; exit 1; }
out=$("${run[@]}" --test-threads=1)
echo "$out"
n=$(echo "$out" | sed -n 's/^test result: ok\. \([0-9]*\) passed.*/\1/p')
[ "${n:-0}" -gt 0 ] || { echo "::error::add_pm ran zero tests on $ARM" >&2; exit 1; }
142 changes: 141 additions & 1 deletion tests/add_pm.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,10 @@
#[path = "../examples/add_pm/pm.rs"]
mod pm;

use ndarray::hpc::codec8::TiesEven;
use ndarray::hpc::codec9::Codec9Runtime;
use ndarray::simd::U8x64;
use pm::{add_decoded, add_pm, add_ref, decode_lane, touched, Pm};
use pm::{add_decoded, add_pm, add_ref, decode_lane, neg_pm, sub_pm, sub_ref, touched, Pm};

fn lane(x: Pm, i: usize) -> Option<i32> {
decode_lane(x.mag.to_array()[i], x.pos >> i & 1 == 1)
Expand Down Expand Up @@ -196,3 +198,141 @@ fn a_fold_with_the_carry_escape_is_exact() {
}
assert!(escapes > 0, "anti-vacuity: the stream must overflow");
}

// ------------------------------------------------------------------ subtraction

#[test]
fn negation_of_every_codeword_matches_the_oracle() {
let codes: Vec<u16> = (0..512).collect();
for c in codes.chunks(64) {
let b = block_of(c);
let n = neg_pm(b);
assert_eq!(n.mag, b.mag, "negation keeps the magnitude");
for i in 0..c.len() {
assert_eq!(lane(n, i), lane(b, i).map(|y| -y), "code {:#x}", c[i]);
}
assert_eq!(touched(n), touched(b), "touched is unchanged");
}
// the never-touched code and +0 are fixed points, -0 is never produced
let z = block_of(&[0, 0x100]);
let n = neg_pm(z);
assert_eq!((n.mag.to_array()[0], n.pos & 1), (0, 0), "never touched stays never touched");
assert_eq!((n.mag.to_array()[1], n.pos >> 1 & 1), (0, 1), "+0 stays +0");
}

#[test]
fn every_codeword_pair_difference_matches_the_i32_oracle() {
let (mut over, mut checked) = (0usize, 0usize);
for (a, b, codes) in all_pairs() {
let (s, carry) = sub_pm(a, b);
for i in 0..codes.len() {
let (want, overflow) = sub_ref(lane(a, i), lane(b, i));
assert_eq!(carry >> i & 1 == 1, overflow, "carry, codes {:?}", codes[i]);
checked += 1;
if overflow {
over += 1;
assert_eq!(s.mag.to_array()[i], 255, "a carry lane saturates");
assert_eq!(lane(s, i).map(i32::signum), want.map(i32::signum), "{:?}", codes[i]);
} else {
assert_eq!(lane(s, i), want, "codes {:?}", codes[i]);
}
}
assert_eq!(touched(s), touched(a) | touched(b), "touched(a - b)");
}
assert_eq!(checked, 512 * 512);
// opposite-polarity pairs with |a| + |b| > 255 carry under subtraction
let expect: usize = (1..=255i32)
.flat_map(|x| (1..=255i32).map(move |y| usize::from(x + y > 255)))
.sum::<usize>()
* 2;
assert_eq!(over, expect, "carry count");
}

#[test]
fn sub_pm_equals_negate_then_add_on_every_pair() {
for (a, b, _) in all_pairs() {
assert_eq!(sub_pm(a, b), add_pm(a, neg_pm(b)));
}
}

/// The four cases the contract names, on one block.
#[test]
fn the_four_mandatory_cases() {
// -5 + +5 = +0
let (s, c) = add_pm(block_of(&[5]), block_of(&[0x105]));
assert_eq!((c & 1, s.mag.to_array()[0], s.pos & 1), (0, 0, 1), "-5 + +5 = +0");

// -0.499999 quantizes to +0 (scale 1, ties to even), through the batch
// encoder into a block, then survives an addition with never touched
let codec = Codec9Runtime::<TiesEven>::new(1.0).unwrap();
let mut mag = [0u8; 64];
let mut pos = [0u64; 1];
codec
.encode_touched_batch(&[-0.499_999f32; 64], &mut mag, &mut pos)
.unwrap();
let q = Pm {
mag: U8x64::from_array(mag),
pos: pos[0],
};
assert_eq!((q.mag.to_array()[0], q.pos & 1), (0, 1), "-0.499999 is +0, not -0");
let (s, _) = add_pm(q, Pm::untouched());
assert_eq!(touched(s), u64::MAX, "still touched");
assert!(s.mag.to_array().iter().all(|&m| m == 0) && s.pos == u64::MAX);

// +0 - +0 = +0
let (s, c) = sub_pm(block_of(&[0x100]), block_of(&[0x100]));
assert_eq!((c & 1, s.mag.to_array()[0], s.pos & 1), (0, 0, 1), "+0 - +0 = +0");

// never_touched - never_touched = never_touched
let (s, c) = sub_pm(Pm::untouched(), Pm::untouched());
assert_eq!((c, touched(s)), (0, 0), "never touched - never touched");
}

#[test]
fn a_difference_fold_with_the_carry_escape_is_exact() {
let mut r = Rng(0xD1FF_5EED);
let n = 2000;
let mut acc = Pm::untouched();
let mut wide = [0i64; 64];
let mut escaped = 0u64;
let mut exact = [None::<i64>; 64];
let mut carries = 0;
for _ in 0..n {
let b = r.block(160);
let bm = b.mag.to_array();
for i in 0..64 {
if let Some(v) = decode_lane(bm[i], b.pos >> i & 1 == 1) {
exact[i] = Some(exact[i].unwrap_or(0) - i64::from(v));
}
}
// escaped lanes take the delta exactly; the rest go through sub_pm
let off = escaped;
let b_in = Pm {
mag: U8x64::mask_blend(off, b.mag, U8x64::splat(0)),
pos: b.pos & !off,
};
let before = acc;
let (s, carry) = sub_pm(acc, b_in);
acc = s;
for i in 0..64 {
let d = decode_lane(bm[i], b.pos >> i & 1 == 1).map(i64::from);
if escaped >> i & 1 == 1 {
wide[i] -= d.unwrap_or(0);
} else if carry >> i & 1 == 1 {
carries += 1;
let a = lane(before, i).map(i64::from).unwrap_or(0);
wide[i] = a - d.unwrap_or(0);
escaped |= 1 << i;
}
}
}
for i in 0..64 {
let got = if escaped >> i & 1 == 1 {
Some(wide[i])
} else {
lane(acc, i).map(i64::from)
};
assert_eq!(got, exact[i], "lane {i}");
}
assert!(carries > 0, "the fold must carry");
}
Loading