Repository navigation
Make Auto Reviewer closing lines deterministic and retry invalid drafts once - #12
Conversation
Drop any model-written signature and Next step lines and append the canonical ones in Add review duration, so a missing or malformed trailer no longer fails Publish. Fixes #11.
There was a problem hiding this comment.
Readiness: Ready with fixes
| Intent | System | Craft | Critical | Important | Minor |
|---|---|---|---|---|---|
| PASS | FINDINGS | PASS | 0 | 0 | 1 |
Details
Findings
-
$\color{green}◎$ Low R1-F1.github/workflows/auto-reviewer-shared.yaml:1251Fallback Next step ignores readiness and can point to/verify -> /finalize squashon a Not ready review
R1-F1 - Minor, confidence medium, .github/workflows/auto-reviewer-shared.yaml:1251.
Impact: when the model's Next step is missing or invalid, the fallback is chosen only from the three finding counts in the Summary table. With all counts at zero it always writes Next step: \/verify -> /finalize squash`. Zero findings does not always mean a clean result. A fresh review pass can leave a previous finding not addressed, and the workflow prompt then requires /cr-received PR #. A review can also be Not ready, or UNASSESSED, with zero findings. In those cases the draft gets a merge-forward recommendation that contradicts the report. The same path drops the plan path the prompt asks for in the clean case. Evidence: lines 1248-1254 read only countsfrom the table. TheReadiness:line is in the body and is not consulted. The tests inDeterministicTrailerTestuse PASS verdicts only, so they do not cover this case. Suggestion: in the fallback, parse theReadiness:line and use/verify -> /finalize squashonly for a plainReady. Otherwise use /cr-received PR #, or none` with a reason for UNASSESSED. Add a test with a Not ready body and zero counts.
Target
Target: PR #12 (261112T1021-dev-session -> main)
Boundary: committed range. Base 57aeb53, head d123f4d, Tree 074603c8c0326f95532d94571a9c25b6754bda72. Uncommitted state: none. Excluded paths: none.
Range: 57aeb53...d123f4d - 5 files changed, 104 insertions(+), 22 deletions(-)
Commits:
-
d123f4dMake the Auto Reviewer closing lines deterministic
Coverage: a single self-review from the diff, not independent: the harness had no isolated reviewer, so I read the whole diff and the Add review duration step and its env in the workflow file. I did not read the Publish validator (Validate review body) in full. I checked the grammar match only by comparing the regex at line 1544 with the one at line 1240. Skipped: the rest of the workflow and the tests beyond the diff. Intent sources: the PR title and description (Fixes #11), AGENTS.md, and CHANGELOG ## v0.0.7. There is no plan file or spec.
Axis verdicts
The change does what the PR says, and the only issue is a weak fallback in an edge case.
- Intent: PASS. The step drops the model's trailing signature and Next step lines and appends the canonical ones. A missing or malformed trailer no longer fails Publish. The CHANGELOG, README and transport doc are updated. The own caller stays on the released tag, and no workflow other than the shared one changes (AGENTS.md rule 3).
- System: FINDINGS. R1-F1: the fallback Next step is chosen from counts only. Otherwise the inline
python3 -Iis consistent with the repo rule. Its heredoc indentation is dedented by YAML.NUMBERis provided by the job-level env (line 800), and the tests supply it. The kept-Next-step regex matches the Publish validator's grammar at line 1544. - Craft: PASS. The comment is updated. The new
DeterministicTrailerTestcovers missing, malformed, reversed and CRLF trailers, validator acceptance, the fallback, and a kept agent Next step.
No previous yngwald review exists on this PR (pr.json has no reviews), so this is the first pass and there are no previous findings to report.
Set aside, open questions, and review gaps
Set aside:
- The code strips all
\rcharacters in the body, not only line endings. This is harmless for Markdown. - Only trailing lines are dropped, so a signature quoted higher up in the body stays as written. This is intended.
- Usage and cost parsing is unchanged by this PR.
Open questions: none.
Review gaps:
- Tests were not run in this job. CI (validate) is green on the head SHA, as reported by the caller.
- This was a self-review, not independent.
- The follow-up model retry is out of scope per the PR description.
Evidence attribution: all of the above was read by me from the diff and the workflow file. The CI result is taken from the caller. No strengths beyond those listed under Craft.
Used: cr-request 0.0.17 · auto-reviewer-shared 0.0.4 · claude-sonnet-5-5 medium · 0 min 57 sec · 210,252 tokens · $0.1903
Pre-validate the draft in the review job with the publish validators and run the review once more with the errors when it fails. Fixes #11.
Review cycle summary
Review usage
|
Fixes #11
Add review durationdrops any trailingReviewed with:andNext step:lines the model wrote and appends the canonical signature plus a Next step (the model's when valid, else derived from the Summary table counts). It also closes a Details block the model left open. The published format is unchanged.Pre-validate review draftruns the publish job's own review-actions validator (one definition, shared by YAML anchor) and a Details/readiness/Summary table check. On any error it writes.auto-reviewer/retry-error.txtand one moreclaude-code-actionrun (same prompt, anchored) fixes only the listed errors; closing lines, usage (summed over both runs) and inline comments are produced again. Publish validation is unchanged and still decides.Covers the failures in the issue: missing trailer (storypoker #81, this repo #10), "inline finding mismatch" and missing
</details>(agilecharts #859).Tests: 164 OK (new
DeterministicTrailerTest,RetryTest), actionlint and contract checker clean. CHANGELOG## v0.0.7; the own caller stays on v0.0.6.