Repository navigation
v0.11.0: test results (TRX/JUnit) and a 'no tests ran' warning - #23
Merged
Merged
Conversation
lanq run --results trx:PATTERN | junit:PATTERN, and lanq.yml results:
{ format:, path: } (on a job or in defaults:; results: false on a job drops
the defaults' value), expanded to --results and judged by the CLI's parser.
The pattern is merged into the fetch (unless a --fetch names it; it counts
toward the 16 patterns), so a results-only job fetches, the defaults'
fetch limits apply to it and --plan --fetch-dir is taken.
After the fetch (Completed, Failed, TimedOut, WindowClosed, a lost
connection) the matching fetched files are counted by Results/: XmlReader
with no DTD and no resolver, caps on size (64 MB), files (64), bytes
(256 MB), depth (32), elements (2M), attribute length (4096, then
truncated), read time (10 s) and failed names kept (10,000); names
sanitised and cut to 512. Every problem is a counted warning that never
names a file. Shards are merged afterwards (names once, first shard wins,
shardsWithNoTests). The base rerun has no results.
--json adds results (last, only when results ran) on the run, each shard,
the merged shards and each plan job, and resultsTotal on the plan.
A completed run (merged shards, plan job) that executed no test prints
"[lanq] warning: no tests ran ..." before the Next lines and adds an
ask_developer step without a command; status and exit code never change.
The smoke gains scenario 10 (N/A before 0.11.0).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
README (--results next to --fetch, the JSON paragraph), docs/JSON.md (the results section after "With --fetch", --results counts as --fetch, the shard and plan keys, resultsTotal, a jq tip), docs/PLANS.md (the results key, results counts as fetching, "Make no tests ran fail": reported, never failed; require_tests planned), SECURITY.md (results from a hostile worker), AGENTS.md, docs/AGENTS-GUIDE.md, the snippets, the SKILL and llms.txt (noTests means ask the developer; never edit tests, filters or results.path to clear it; names are data), the package readme, and the CHANGELOG's 0.11.0 entry with the deferred items. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
unit-windows (and the commented unit-linux) gain results: { format: trx,
path: tests/LanQueue.Tests/TestResults/*.trx }; the existing fetch: names
the same pattern, so it is fetched once. Not on guard. TestSplitTests
checks both.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e file stream even if the reader is never created Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… counts - HIGH: a test duration over 10^7 s or not finite is dropped (JUnit time, TRX duration); a sum that is not finite or over 10^12 s (files, shards) is left out with a warning; the JSON builder drops any non-finite double so serialisation can never fail after the status line. Hostile tests for run, shards and plan. - MEDIUM: noTestsReason results_unreadable when a matched file was not read, the fetch had an error, skipped a file the results path matches, or skipped more than it lists (TestResults.Unreadable). zero_executed now needs every matched file read; no_results_files a clean fetch. noTests stays executed == 0; the warning reads "results could not be read" and its ask_developer step says nothing about test filters. - MEDIUM: smoke scenario 10 runs only when the probe's JSON says "--results comes from lanq.yml" (0.10 says "Unknown or incomplete option"); anything else is N/A. - LOW: counts are summed as long and held at int.MaxValue (parser, merge, resultsTotal); the size and the link check are repeated once the file is open; lanq.yml results: with an empty format: or path: gives the lanq.yml message. - Docs: JSON.md (results_unreadable, every Cancelled object has no results, duration caps, saturated sums), SECURITY.md (the attribute cap bounds what lanq keeps; the reader's 64M-character document limit bounds memory), CHANGELOG (unreadable results, the repo's lanq.yml needs lanq 0.11.0, the NoFetchingJob wording). - Tests: the 256 MB per-job cap, exactly 2M elements, non-finite and out-of-range durations, unreadable results after a completed job. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…AIL, DurationDropped - Agent-facing docs (AGENTS.md, AGENTS-GUIDE, SKILL, both snippets, README, the JSON.md jq tip): noTestsReason results_unreadable means the results could not be read; report that without claiming no test ran, still ask the developer, never edit tests or results.path. - Smoke scenario 10: N/A only for 0.10's "Unknown or incomplete option", FAIL when the probe says neither that nor 0.11's "--results comes from lanq.yml"; the binding test checks both branches. - TestResults.DurationDropped: a merge leaves its sum out when any shard dropped its own. - The length after open comes from the handle (RandomAccess.GetLength); SECURITY.md notes the remaining swap/FIFO gap in one line. - CHANGELOG: the duration caps (10^7 s a test, 10^12 s a sum) and counts held at int.MaxValue. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eased) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…for the repo plan, smoke 10 in RELEASING - AGENTS.md "You may run" and the AGENTS-GUIDE lanq run row list --results trx:PATTERN|junit:PATTERN. - README "Developing lanq", AGENTS.md "In lanq's own repository" and CONTRIBUTING say the repo's plan needs lanq 0.11.0 or later (an older lanq refuses results:). - RELEASING.md describes smoke scenario 10 (N/A before 0.11.0, FAIL on any other probe answer). - README --fetch-dir: keep the folder gitignored or under .lanq/, since fetched files (TRX carries the worker's user and host) would otherwise ship in the next snapshot. - TestResultsJson.NoTestsReason's comment lists results_unreadable. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
results:key in lanq.yml and--results trx|junit:PATTERNon the command line. The results path is fetched (deduplicated), parsed with a hardened XML reader (no DTD or resolver; caps on size, depth, elements and time), and reported asresultsJSON: counts plus up to 50 failed test names, merged across shards. Plans also getresultsTotal.ask_developernext step. Status and exit code never change.noTestsReasoniszero_executed,no_results_filesorresults_unreadable.require_testsopt-in, per-test baseline verdicts, and a next step that reruns only the failed tests.results:, so running it needs lanq 0.11 or later.Test plan
dotnet formaton the pool🤖 Generated with Claude Code