Skip to content

verdicts: only a trusted author's role-prefixed comment counts, in pr-poll and pr-watch (#148) - #150

Merged
deanbot merged 4 commits into
mainfrom
148-trusted-verdicts
Nov 2, 2026
Merged

deanbot merged 4 commits into
mainfrom
148-trusted-verdicts

Conversation

@deanbot

@deanbot deanbot commented Nov 1, 2026 •

Copy link
Copy Markdown
Owner

Follows #147 (#134, trusted claims, merged): it uses that PR's trustTest.

What changed and why

The signal scripts classified agent comments by prefix alone, so anyone who can comment could post [reviewer] LGTM and give the executor MERGE_READY, revoke a real LGTM with a fake finding, silence a reviewer's pending operator comment with a fake [reviewer] line, or park every reviewer off a PR with a fake [reviewer] Reviewing (<clone>). #134 closed this for [executor] Claimed by; this applies the same rule to every role comment.

  • lib/block.mjs: ROLE_PREFIX ([executor], [reviewer], [qa]).
  • pr-poll.mjs: a role-prefixed issue or inline comment from an author trustTest doesn't trust is dropped before the thread is read. It is no verdict, finding, question, revocation or answer: an outsider's [executor] line no longer makes a live finding look answered, and an outsider's inline [executor] reply no longer answers an inline comment. With the block unreadable nobody is trusted, so every role comment is dropped for that run (MERGE_READY was already withheld then).
  • pr-watch.mjs: the same filter on every comment read, so an outsider's [reviewer] line is no reviewer post (it marks nothing read, silences nothing) and no review claim. It now needs the block every run. A one-shot run that can't read it prints NONE and keeps its state. A waiter's pass throws, so the failure is logged and retried. --check-claim exits 2 with a pr-watch: line and no answer.
  • pr-qa.mjs: no code change. It has no own post that moves a cursor or silences a signal (any bracketed comment is just not operator direction), so who wrote a [qa] line doesn't matter. A test pins this, and the header says so.
  • --repo is checked against GitHub's name characters in pr-poll.mjs and pr-watch.mjs, as claims.mjs does: the trust read puts it on a gh command line.
  • Reviewer texts (Claude Code, generic, Kiro): a failed --check-claim run drops the PR for the cycle and never claims on it.
  • SPEC: the comment prefix section says a prefix counts only from a trusted author. The trusted-authors and allow-author-associations rows say every agent's login must be trusted. The claim section has --check-claim's exit 2. AGENTS.md: the pr-poll, pr-watch and block.mjs entries.

Done items

  • Same trust rule as Claims: only a trusted author's claim comment counts #134 for every verdict and own-post reader → trustTest in pr-poll and pr-watch. pr-qa reads no own post (above). Tests: trusted verdicts (#148): ….
  • An untrusted [reviewer] LGTM → no MERGE_READY → … an untrusted [reviewer] LGTM → no MERGE_READY, and it is not shown.
  • An untrusted finding never revokes a trusted LGTM → … an untrusted finding after a trusted LGTM → still MERGE_READY.
  • [qa] the same way with qa-gate: on → … an untrusted [qa] PASS → no MERGE_READY, … an untrusted [qa] BLOCKED after a trusted PASS → still MERGE_READY.
  • A trusted second account → … a trusted second account's LGTM (on trusted-authors, association NONE) → MERGE_READY.
  • The association fallback with no list → … a MEMBER's LGTM counts, a CONTRIBUTOR's does not, … allow-author-associations narrows it.
  • An untrusted own-prefix comment neither silences a signal nor marks earlier comments read → … an untrusted [reviewer] line after an operator comment → REVIEW_COMMENTS:7, … after an [executor] reply marks nothing read, … watch, only an untrusted [reviewer] LGTM → REVIEW:7. pr-qa: … qa, an operator comment followed by an untrusted [qa] PASS → QA_READY:7.
  • An untrusted Reviewing (<clone>) makes no reviewer skip → … rev-b still gets REVIEW:7, and --check-claim says OPEN.
  • The trust list is read from the default branch's block, and an unreadable block never means trust-by-default → poll (#148): … with the block unreadable nobody is trusted, … --check-claim with the block unreadable → exit 2, … watch with the block unreadable → NONE, wait: a transient API error is retried, not fatal.
  • An untrusted role comment isn't operator direction either → it is dropped entirely. The tests check that the dropped text never appears in the output.
  • SPEC lines → above.

Calls a reviewer might push back on

  • How an untrusted role comment is surfaced: it isn't. It is dropped silently. A stderr line every poll would repeat on every waiter pass, and the operator sees the comment on GitHub anyway.
  • pr-watch withholds every signal on an unreadable block, CLOSED and a standing MERGE_CONFLICT included, not just CI_DONE as before. Trusting nobody would make every reviewed PR look unreviewed (REVIEW: floods). Trusting everyone reopens the hole. So the run reports nothing and keeps its state for the next run.
  • Adapter prompts unchanged on reading the thread. The scripts' signals are what decide. A model reading gh pr view --comments doesn't see the author's association, so a prompt rule to "skip untrusted role comments" couldn't be followed reliably from that output. Left as a possible follow-up if it matters.
  • Unprefixed comments from untrusted logins are still operator direction, as before: this issue covers role prefixes only.

Known limits

  • In pr-poll, a role comment dropped while the block was unreadable is lost as a one-off NEW_COMMENT if a later comment moves the cursor past it. Verdicts, live findings and unanswered inline comments are read fresh each run, so they count again. A [reviewer] Question would not be re-shown.
  • Not in scope, and unchanged: an outsider's unprefixed comment, and an outsider's formal review submission (NEW_REVIEW_SUBMISSION, CHANGES_REQUESTED included), still reach the executor as work, as before. Neither carries a role prefix, and neither can give or withhold MERGE_READY.
  • A one-shot pr-watch run that can't read the block prints NONE and saves no state, so the next waiter's baseline is NONE. It wakes at once on a standing signal (a held REVIEW:<n>), and the idle count resets: one extra cycle per failed read.
  • Single-account mode: every role posts as one login, which must be trusted (OWNER by default). Nothing else changes.

Restart note

Reviewer and executor roles need a fresh session (pr-watch and pr-poll change, and the reviewer texts' --check-claim step changed). QA needs none. Setup need not be re-run.

Closes #148

@deanbot
deanbot marked this pull request as ready for review November 1, 2026 23:40
@deanbot
deanbot merged commit 1dd26de into main Nov 2, 2026
12 checks passed
@deanbot
deanbot deleted the 148-trusted-verdicts branch November 2, 2026 01:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Verdicts: only a trusted author's [reviewer] / [qa] comment counts

1 participant