Skip to content

test(probe): stop the soak from measuring itself (report 331, ADR-389) - #355

Merged
wangzhizhou merged 1 commit into
mainfrom
dev
Nov 20, 2026
Merged

wangzhizhou merged 1 commit into
mainfrom
dev

Conversation

@wangzhizhou

Copy link
Copy Markdown
Contributor

报告 331:1.4.0 候选验收 + R-1 可靠性体检的结论,以及体检抓到的两处探针问题(ADR-389)。

体检四项(候选 ff14a6b / 执行 1162855f…)

项 结果
30 min 连续键入 178 轮 / 2,173 键;p95 16.2 ms、最大 30.5 ms、慢轮 0;footprint 冷 8.3 → 稳态 43.4(窗口内 +0.5);1,685 采样点 0 外联;词库 2,523 行键入前后与宿主重启后一致
睡眠唤醒 真实入睡(唤醒间隔 235 s),同一宿主进程继续工作,键入正常、词库不变、0 连接、无崩溃记录
多屏落点 内建 1440×900 + LG HDR 4K,窗口内/全屏 4/4 PASS
扩展词库档 存活堆 48.3–48.4 MiB(门禁 64),对比 1.3.0 同档 48.0–48.2 → +0.2;复原后配置逐字节一致

抓到的问题:都在探针侧

第一次久用报出「每 6–9 轮一轮 p95 400–700 ms」,看着像产品卡顿。排除过程:

  • ABC 输入源控制组(65 轮):0 次慢键,最大 17 ms → 不是系统/窗口;
  • 同一批文本单独重复(各 8–12 轮,含 soak 的 250 ms 节奏):12–31 ms → 不是内容;
  • 在宿主旁边每 2 s 跑 vmmap -summary 再打字:10/10 轮 277–605 ms → 就是它。

vmmap 会先 task_suspend 目标再遍历地址空间(~1 s),被暂停期间的按键被记成"卡顿"——ADR-385 明令不许把探针伪影当门禁量,这次伪影伪装成了性能问题。

修法(只改探针,产品代码零改动):

  1. 新增 tools/macos-probe/footprint:proc_pid_rusage 读 ri_phys_footprint,15 ms、不挂起,与 vmmap 差 ≤0.5 MiB;
  2. footprint 只在冷启动窗口(900 s)判定,窗口外标 informational;跑前先把宿主重启成冷态;
  3. 久用硬门 40 → 44(ADR-385/379 的值,不抬高项目门禁;40 是旧值,实测会 41.8→43.5 假 FAIL);
  4. 延迟进门禁:latency_p95_median_ms ≤ 40 ms(P2-3 触发线),另记慢轮数。

修后同机同候选同 30 min:每轮 p95 中位数 16.18 → 9.29 ms,慢轮 24/174 → 0/178,单键 >200 ms 66 → 0(最大 694 → 30.5 ms)。

测试

新增 platforms/macos/tests/test-soak-harness.py(3 条):久用不许出现 vmmap、硬门必须 44、延迟必须判定。
make verify 1,197 产物 0 网络符号;离线回归 74 文件 / 371 例。

合并方式

merge commit。

…y honestly

The first 30-minute soak of the R-1 sweep reported a p95 of 400-700 ms
every 6-9 rounds, which looked like the IME stalling on commit. It was the
probe: `vmmap -summary` task-suspends the target before walking its address
space, so keys that arrive while the host is frozen get charged the full
walk. Running vmmap next to the host reproduced it 10/10 times (277-605 ms);
an ABC control group (65 rounds), the same text at the soak's 250 ms rhythm
(12 rounds each), and the untouched host all stayed at 12-31 ms.

Reporting that as a product regression would have been wrong in both
directions, so fix the harness (ADR-389):

- New tools/macos-probe/footprint reads ri_phys_footprint via
  proc_pid_rusage: 15 ms, no suspension, within 0.5 MiB of vmmap. The soak
  samples with it every 5 s.
- Footprint is only comparable inside the cold-start window (ADR-385), so
  the soak restarts the host first and reports footprint_gate as
  informational outside the 900 s window instead of gating on stale
  readings.
- The soak's own hard limit was still 40 MiB, the pre-ADR-385 value, which
  false-failed at 41.8 -> 43.5 on a machine that had just run other probes.
  It now uses the project's 44 MiB, unchanged.
- Key latency becomes a gate (median per-round p95 <= 40 ms, the P2-3
  trigger line). The soak never looked at latency before, which is why a
  whole class of stall could have shipped unnoticed.

test-soak-harness.py pins all three: no vmmap in the soak, 44 MiB, latency
judged.

Re-measured on the same candidate, same machine, same 30 minutes: median
per-round p95 9.29 ms, slow rounds 0/178 (was 24/174), worst single key
30.5 ms (was 694 ms).

Signed-off-by: wangzhizhou <824219521@qq.com>
@wangzhizhou
wangzhizhou merged commit ea4f150 into main Nov 20, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant