Skip to content

refactor(testing): unify shared model tool behavior harness primitives - #8889

Open
loopx-agent wants to merge 1 commit into
mainfrom
codex/testing-behavior-harness-slim-20261117
Open

loopx-agent wants to merge 1 commit into
mainfrom
codex/testing-behavior-harness-slim-20261117

Conversation

@loopx-agent

@loopx-agent loopx-agent commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

Goal And Delivered Outcome

  • Goal & Gap: Several loopx/control_plane/testing/*_behavior.py qualification modules independently duplicated identical actor lifecycle boilerplate (__init__, from_environment), hermetic clock parsers, fixture quota-guard matchers, bounded read-plan subprocess loops, quota action signature summaries, tool exchange appenders, and redacted qualification receipt builders.
  • Delivered Slice (-234 net LOC, rebased cleanly onto origin/main@4a0188c1ca at 854e951eebbcaa6192f8b4907e4291c211bccbfc): Consolidated the shared harness primitives into loopx/control_plane/testing/model_tool_behavior.py and migrated all five exec-tool actor harnesses to reuse them (6 files changed, 431 insertions(+), 665 deletions(-)), preserving every existing receipt field, boundary flag, failure code, fixture timestamp, and origin/main runtime-owned quota_spend (quota_settlement_plan_v2 / SETTLEMENT_RECEIPT_SCHEMA_VERSION) qualification contract.

Summary Of Changes

Verification (Rebased Exact Head 854e951eebbcaa6192f8b4907e4291c211bccbfc on Base origin/main@4a0188c1ca)

  • uv run --no-sync python -m ruff check on all 6 changed files: All checks passed!
  • uv run --no-sync python -m mypy (17 strict files): Success: no issues found
  • collect_oversized_decision_functions on touched files: [] (no new maintainability debt introduced; pre-existing interaction_contract debt on origin/main@4a0188c1ca is tracked separately in #8950)
  • uv run --no-sync loopx --format json canary premerge --tier quick --from-git-diff --git-diff-base origin/main: ok: true, merge_gate_passed: true, direct_failure_count: 0, run_failure_count: 0, rfc_bilingual_diff: passed, module_ceiling_colocation: passed, 3/3 catalog canaries passed (bounded-context-namespace-smoke.py, control-plane-risk-characterization-smoke.py, hot-path-interface-budget-smoke.py)
  • loopx check --scan-path ... on all 6 changed files: errors: 0, warnings: 0
  • 13-file behavior harness pytest suite (355 tests): 341 passed (test_model_tool_context.py, test_selected_todo_tool_behavior.py, test_scoped_gate_successor_tool_behavior.py, test_terminal_settlement_tool_behavior.py, test_actual_default_model_behavior_portfolio.py, test_progress_scope_owner.py, test_research_composition_gate.py, test_agent_scope_frontier_contract.py, test_acceptance_restoration.py all passed cleanly); the remaining 14 failures in test_replan_semantic_action_behavior.py (3), test_capability_monitor_repair_tool_behavior.py (1), test_required_vision_closeout_behavior.py (6), and test_quota_acceptance_references.py (4) were all reproduced identically (14/14 failed with matching signatures) on immutable base origin/main@4a0188c1ca.

@loopx-agent
loopx-agent force-pushed the codex/testing-behavior-harness-slim-20261117 branch from 8f2d77c to 1a4637c Compare November 17, 2026 05:18
@mergify

mergify Bot commented Nov 18, 2026

Copy link
Copy Markdown

This pull request has merge conflicts with main and cannot be merged
until they are resolved. Please rebase or merge the base branch, @loopx-agent.

Choose the remote for the base repository, not an out-of-date fork.
For a fork clone, first inspect git remote -v; upstream must point
to https://github.com/loopx-project/loopx.git. If it is absent, add it
with git remote add upstream https://github.com/loopx-project/loopx.git.
Then run:

git fetch upstream
git rebase upstream/main
# Resolve each conflict, git add the resolved files, then git rebase --continue.
git push --force-with-lease origin HEAD

For a same-repository clone whose origin points to
https://github.com/loopx-project/loopx.git, use origin instead of
upstream for fetch/rebase. If you prefer merging the base, use
git merge <base-remote>/main and push normally.

Keep the DCO Signed-off-by trailer on every commit when you rebase.
https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify Bot added the needs-rebase Mergify: the pull request has merge conflicts with its base branch label Nov 18, 2026
Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>
@loopx-agent
loopx-agent force-pushed the codex/testing-behavior-harness-slim-20261117 branch from 1a4637c to 854e951 Compare November 18, 2026 15:20
@mergify mergify Bot removed the needs-rebase Mergify: the pull request has merge conflicts with its base branch label Nov 18, 2026

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent (Gemini, Google DeepMind, self_reported)

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

exact head 854e951eebbcaa6192f8b4907e4291c211bccbfc unifies the duplicated exec-tool actor harness boilerplate into loopx/control_plane/testing/model_tool_behavior.py across all five behavioral qualification modules (-234 net LOC across 6 files), resolves the single rebase conflict in terminal_settlement_tool_behavior.py on origin/main@4a0188c1ca while preserving runtime-owned quota_spend checks (quota_settlement_plan_v2), passes ruff, strict mypy, AST ratchet checks, and canary premerge --tier quick cleanly, and confirms all 341 passing behavior tests while isolating the 14 failing tests as pre-existing on origin/main@4a0188c1ca.

动机 / Motivation

维护控制面模型工具行为验证套件(loopx/control_plane/testing/*_behavior.py)的开发者、评审者与预合并检查均受该重复样板代码问题影响。

在本 PR 之前,replan_semantic_action_behavior.py、terminal_settlement_tool_behavior.py、selected_todo_tool_behavior.py、capability_monitor_repair_tool_behavior.py 与 scoped_gate_successor_tool_behavior.py 五个模块各自重复实现 DoubaoExecToolActor 的 __init__/from_environment、时钟输出解析、quota should-run 守卫匹配、有界工作区读命令执行、动作契约摘要、消息交换追加及脱敏回执拼装,导致一处修复要在多份测试工具模块间手工同步,且合入最近 main 时在 terminal_settlement_tool_behavior.py 发生合并冲突;在本 PR 提交 854e951eebbcaa6192f8b4907e4291c211bccbfc 之后,上述共享原语集中收敛到 model_tool_behavior.py,5 个行为模块消解冲突并净减少 234 行代码。

共 6 个文件 +431 / -665(净减 234 行),修改文件在 AST 棘轮下无任何超限决策函数(oversized_touched = []),ruff check 全部通过,mypy 17 个严格文件零报错,loopx canary premerge --tier quick 全绿(failures = 0),13 份测试文件 355 项用例中 341 项通过,其余 14 项经 origin/main@4a0188c1ca 不可变基线同命令复现为既有失败(pre_existing_unrelated)。

本 PR 不改动 quota、interaction_contract 或 effect_program 等生产控制面决策逻辑,不放宽任何行为验证断言、边界字段或失败码,也不顺带修复与本次去重无关的 14 项 origin/main 既有行为测试失败。

放任五个行为验证模块长期拷贝相同脚手架会在每次演进配额或收口契约时放大冲突面和漏改风险,而本 PR 仅收口测试工具原语并在 _quota_contract 中妥善整合 origin/main 的运行时扣额契约,属于高内聚、可独立回滚的无多余扩张收敛改动。

改动思路 / Approach

入口为各资格验证类(如 DoubaoTerminalSettlementToolBehaviorActor、DoubaoReplanSemanticActionBehaviorActor 等)的 qualify 方法及其继承的共享基类 DoubaoExecToolBehaviorActorBase,权威输入仍为真实 CLI quota should-run 输出的包与受限临时夹具目录,决策归属仍在各独立行为模块自身的违约判定与回执语义校验(如 _quota_contract 对 quota_settlement_plan_v2 的判定),通过共享原语完成纯数据抽取、子进程有界执行与统一脱敏回执拼装,避免引入第二套语义规则或弱化任何行为测试。

之所以把公共脚手架抽入既有的 loopx/control_plane/testing/model_tool_behavior.py 而保持五个测试行为模块各自的夹具构建与领域断言独立,是因为这组测试共同使用 Ark/Doubao 函数调用工具循环和脱敏回执结构,但分别校验选中待办、能力修复、作用域门禁、终态结算和自治重规划契约;reconcile 后的 _quota_contract 兼顾共享基座提取与 origin/main 运行时扣额不变量。

当前 PR 边界严格限定于 loopx/control_plane/testing/ 目录下 6 份测试工具模块的公共原语提取、调用点替换以及对 origin/main@4a0188c1ca 冲突的单点对齐;不包含生产代码逻辑变更、远端 GitHub CI 运行等待,也不捆绑修复基线上已存在的 14 项无关回归失败。

具体改动 / Changes

全部改动局限在 6 个测试支持模块(+431 / -665,净减 234 行),无文档、生成产物或无关生产模块改动;terminal_settlement_tool_behavior.py 的单处 rebase 冲突已在 origin/main@4a0188c1ca 上消解并完整保留 SETTLEMENT_RECEIPT_SCHEMA_VERSION 与运行时自扣额检查。

规范对齐说明:本 PR 属于既有测试辅助工具去重与基线冲突消解,无对应新建 RFC 特性契约(no_spec),以 problem_context 与不改变现有行为测试断言、回执边界及失败码为验收标准。

关键代码讲解

  1. DoubaoExecToolBehaviorActorBase、qualification_turn_instance_id、hermetic_clock_output 及 L897-L935
    统一封装 DoubaoExecToolClient 初始化、ARK_API_KEY 环境变量注入检查(from_environment)、确定性 turn ID 生成,以及允许按测试夹具参数化 utc_output/local_output 的受控时钟命令解析,保留原异常信息与失败路径。
  2. extract_quota_action_contract_base、execute_bounded_read_steps、build_qualification_receipt、append_exec_tool_exchange
    提取通用的 quota_action_signature_document 关键字段抽取、遵循 ;/&&/|| 状态与行截断策略的子进程有界读取循环、统一隐私脱敏边界字典拼装(支持 boundary 覆盖合并),以及对话消息列表追加逻辑,不隐藏任何子进程错误码或状态迁移。
  3. _quota_contract 与 DoubaoTerminalSettlementToolBehaviorActor 及 L1000
    冲突消解后先通过 extract_quota_action_contract_base(packet) 提取基础契约,再校验 origin/main 新增的 quota_settlement_plan_v2、spend.get("owner") == "runtime" 及无 command_template 约束,并在后续 _settlement_receipt_steps 与 _execute_terminal_cli 完整保留 SETTLEMENT_RECEIPT_SCHEMA_VERSION 和双扣额拒绝逻辑。
  4. _capability_repair_contract、_bounded_workspace_read_plan 及各行为 Actor 类
    五个模块统一继承 DoubaoExecToolBehaviorActorBase 并切换至共享原语(见 capability_monitor_repair_tool_behavior.py L429、scoped_gate_successor_tool_behavior.py L296、selected_todo_tool_behavior.py L956、replan_semantic_action_behavior.py L1736),各自特有的状态校验(如 bounded_workspace_read_plan 权限校验)原样保留。

对主干的风险 / Risks

  • 阻断项(Blocking findings):无阻塞问题(no blocking finding)。
  • 负向与失效路径核验:若 ARK_API_KEY 缺失或模型/超时超限,DoubaoExecToolBehaviorActorBase.from_environment 与 DoubaoExecToolClient.__init__ 仍抛出完全相同的 RuntimeError/ValueError;若 execute_bounded_read_steps 遇到失败退出码的 && 前驱步骤,按原逻辑短路后续步骤并保留状态码;terminal_settlement_tool_behavior.py 对 quota_spend 发出重复扣额仍触发 _TerminalToolViolation("quota_spend_contract_mismatch")。
  • 本地验证与基线对照归因:
    • PASS:ruff check(6 份改动文件全通过)、mypy 严格集(17 文件 0 错)、AST 决策函数上限检查(oversized_touched = [])、loopx check --scan-path(6 份文件 0 错误/0 警告)。
    • PASS:loopx canary premerge --tier quick --from-git-diff --git-diff-base origin/main(ok: true、merge_gate_passed: true、direct_failure_count: 0、run_failure_count: 0,含 3 个目录烟雾脚本全部通过)。
    • PASS(主要回归套件):13 个相关测试文件中 341 / 355 通过,其中 test_model_tool_context.py、test_selected_todo_tool_behavior.py、test_scoped_gate_successor_tool_behavior.py、test_terminal_settlement_tool_behavior.py、test_actual_default_model_behavior_portfolio.py、test_progress_scope_owner.py、test_research_composition_gate.py、test_agent_scope_frontier_contract.py、test_acceptance_restoration.py 全数通过,直接证明重构与冲突消解行为等价。
    • PRE-EXISTING UNRELATED FAILURE(14 项既有失败,非本 PR 引入):test_replan_semantic_action_behavior.py 3 项(assert 10 == 7 与 DoubaoActorTransportError)、test_capability_monitor_repair_tool_behavior.py 1 项(KeyError: 'monitor_schedule_gap_items')、test_required_vision_closeout_behavior.py 6 项、test_quota_acceptance_references.py 4 项(sources[0] vs sources[1]),已直接在不可变 origin/main@4a0188c1ca 独立工作区用相同 pytest 命令全部 14 项同签名复现(4 failed + 10 failed)。
    • PENDING(诊断行,required=false):GitHub 远端队列中的 12 项检查尚在排队中,合并前仍需远端 CI 转绿。
  • 残留风险:mypy --follow-imports=silent 检查 6 份改动文件时仍有 8 条预存在静态类型告警(含主干尚未合入 #8950 的 build_heartbeat_prompt 无注解调用 5 处,以及 replan_semantic_action_behavior.py 原有的 3 处导入/夹具类型标注差异),未扩展为本 PR 的严格 mypy 改造范围以免混入无关变更;改动全在 testing/ 目录,可单提交整体回滚且不影响生产路径。

我的整体评价 / Verdict

综合判定为 APPROVE(由于作者与当前认证账号均为 loopx-agent,按契约以 COMMENTED 形式提交批准结论):

  • 长程与用户体验影响:long_horizon 为 preserved(仅整合测试支撑原语,不改变任何长程调度、配额扣减或工作流持久化路径,同时清除了 #8889 长期挂起的冲突);user_experience 为 preserved(终端用户界面与 CLI 输出无变化,测试失败信息与回执结构保持一致)。
  • 视觉检查说明:对全部 6 份改动文件完成检查后确认均属于控制面测试工具代码且无任何前端页面、HTML、CSS 或交互视觉呈现改动,视觉检查不适用,未验证视觉范围(unverified_scope):none。
  • 代码体量与兼容边界:净减 234 行,删去五份测试模块间的大量重复脚手架,model_tool_behavior.py(959 行)及所有被迁移模块均保留独立公开行为类名与契约;未新建、扩展或更改共享协议与状态词汇。
  • 后续建议:当前精确提交 854e951eebbcaa6192f8b4907e4291c211bccbfc 已解除合并冲突并完成独立验证,可等待远端 CI 跑完后通过正常维护者路径合入。

English verdict: APPROVE

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant