Skip to content

fix(sleep): compare the harvest cutoff with transcript times as instants - #389

Merged
Yifan Yang (Yif-Yang) merged 2 commits into
microsoft:mainfrom
1AbdullahButt:fix/harvest-cutoff-timezone
Nov 10, 2026
Merged

Yifan Yang (Yif-Yang) merged 2 commits into
microsoft:mainfrom
1AbdullahButt:fix/harvest-cutoff-timezone

Conversation

@1AbdullahButt

Copy link
Copy Markdown
Contributor

Fixes #387

The nightly cycle stores its harvest cutoff with state._now_iso, which writes local time with no offset. Claude Code, Codex and Pi transcripts record UTC (...Z), and their harvesters compared the two as strings. That shifts the cutoff by the host's UTC offset:

  • East of UTC: a session that ended in the hours after a harvest compared as older than the cutoff and was never harvested.
  • West of UTC: sessions were harvested again.

The fix adds _iso_epoch / _ended_before to harvest.py and uses them in the Claude Code, Codex and Pi harvesters. Both values are parsed to epoch seconds: a naive value is read in the host timezone, and Z as UTC. This is what the Copilot, Cursor and OpenCode harvesters already do. Values that cannot be parsed fall back to the old string comparison.

test_a_source_still_skips_what_it_already_harvested wrote datetime.now() (local time) with a Z suffix, which only passed because of this bug. Its fixture now writes UTC.

Tests

  • New tests/test_harvest_cutoff_timezone.py pins the process timezone to UTC+5 with time.tzset() (or runs on a host east of UTC where tzset is missing, otherwise skips). For each of the three harvesters, it checks that a session ending one minute after the cutoff is kept and one ending one minute before is dropped. All 3 fail on main and pass with this change, both on Windows at UTC+5 and on Linux (python:3.11-slim, UTC host via tzset).
  • python -m pytest -q: All 3 new tests fail on main and pass here. Full suite on Linux (python:3.11-slim): 2077 passed; the same 11 tests fail on unmodified main in that container (Devin installer, run-sleep fallback, reproducibility CLI: environment-dependent, unrelated). On Windows the harvest and sleep-engine tests pass apart from failures that also occur on main.
  • No docs changed.

This change was written with Claude Code.

🤖 Generated with Claude Code

The nightly cycle stores its harvest cutoff with state._now_iso, which
writes local time with no offset. Claude Code, Codex and Pi transcripts
record UTC ("...Z"), and their harvesters compared the two as strings,
which shifts the cutoff by the host's UTC offset. East of UTC, a session
that ended in the hours after a harvest compared as older than the
cutoff and was never harvested; west of UTC, sessions were harvested
again.

Parse both sides to epoch seconds (a naive value in the host timezone,
"Z" as UTC), as the Copilot, Cursor and OpenCode harvesters already do,
and keep the string comparison only for values that cannot be parsed.

test_a_source_still_skips_what_it_already_harvested wrote local time
with a "Z" suffix, which only passed because of this bug; it now writes
UTC.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keep both CHANGELOG entries; no other change.
@Yif-Yang
Yifan Yang (Yif-Yang) merged commit f2f721a into microsoft:main Nov 10, 2026
@Yif-Yang

Copy link
Copy Markdown
Contributor

Merged, thank you. The harvest cutoff is now compared with transcript timestamps as instants rather than as strings, so a host in a non-UTC timezone no longer skips or re-reads sessions near the checkpoint, with a string-comparison fallback for unparseable values. On main, the new tests fail (3 failures) and pass here. Closes #387. Your branch and an earlier PR in this batch each added an entry at the same spot in CHANGELOG.md, so I pushed one maintainer merge commit to your branch that keeps both entries and changes nothing else, then merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

harvest: the local-time cutoff is string-compared with UTC transcript timestamps, so sessions are skipped east of UTC

2 participants