Repository navigation
feat(agent): count post-compaction fog in an opt-in JSONL ledger - #301
Open
raymondginger2018-sudo wants to merge 1 commit into
Open
raymondginger2018-sudo wants to merge 1 commit into
raymondginger2018-sudo wants to merge 1 commit into
Conversation
LangWatch measured a ~28-step post-compaction correction window where the model's error rate is 41.9% against a 17.7% baseline (2.37x): a summary restores the context budget but not the working state, so the agent keeps re-deriving what it dropped. This records the cheap half of that finding. Each successful compaction appends one six-count FogObservation row to the opt-in `AgentRunSpec.fog_ledger_path`; `None` (the default) does zero I/O. Only counts, never decides: the seam is the existing `compaction.reset` checkpoint key, read by key only, and the message list is never mutated. A failed ledger write is logged and swallowed so a measurement can never abort the run it documents. Rates are recomputed by readers; a zero denominator yields `None`, never `inf`/`NaN`. Split from HKUDS#300: this PR carries the fog mechanism, its runner call site and its tests. The paired capability/efficiency gate framework is a separate change with no runtime consumer yet.
This was referenced Nov 8, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
LangWatch measured a ~28-step correction window after a context compaction where the model's error rate is 41.9% vs a 17.7% baseline (2.37x): the summary restores the context budget but not the working state, so the agent keeps re-deriving what it dropped. This records the cheap half of that finding.
Each successful compaction appends one six-count
FogObservationrow to the opt-inAgentRunSpec.fog_ledger_path.None(the default) does zero I/O.Only counts; never decides. The seam is the existing
compaction.resetcheckpoint key (read by key, no import of the compactor), the message list is never mutated, and a failed ledger write is logged and swallowed so a measurement can never abort the run it documents.Split out of #300, which bundled this mechanism with an unrelated paired-gate framework; #300 is closed in favour of the two smaller PRs.
Changes
core/agent_runtime/fog_ledger.py(new, stdlib only):FogObservation(six counts,Nonerates with no samples,fog_ratioisNone— neverinf/NaN— when the baseline is zero),measure_post_compaction_fog(window after each checkpoint, retained tail skipped, orphan tool results not counted),FogLedger(append-only JSONL,O_APPEND+ fsync).core/agent_runtime/runner.py(+13):AgentRunSpec.fog_ledger_path: str | Path | None = None; after a successful auto-compaction, measure the pre-compaction history and append one row. A failing write islogger.exception-ed and swallowed.tests/test_fog_ledger.py(new): counting, window cap, retained tail, checkpoint detection, error-signal, non-string tool content, rates/ratio edge cases, ledger rows.tests/test_session_compaction.py(+52): wiring — one row per successful compaction, nothing written when compaction is skipped, ledger failure swallowed.Notes
fog_ratioisNonerather than infinite when the denominator has no samples, so readers never parseinf/NaN.Tests
pytest tests/test_fog_ledger.py tests/test_session_compaction.py→ 36 passed;ruff check+ruff formatclean; pre-commit hooks pass.test_agent_runner_kernel,test_compaction_memory,test_compaction_tool_call_budget) pass except one pre-existing Windowsenvironment_contextfailure that reproduces identically on unmodifiedmain.