Skip to content

Fix --incremental-by-files skipping updated items east of UTC - #542

Merged
josegonzalez merged 1 commit into
josegonzalez:masterfrom
Iamrodos:fix/incremental-by-files-timezone
Nov 4, 2026
Merged

josegonzalez merged 1 commit into
josegonzalez:masterfrom
Iamrodos:fix/incremental-by-files-timezone

Conversation

@Iamrodos

@Iamrodos Iamrodos commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

--incremental-by-files skips an item when its saved file is newer than GitHub's updated_at. The file time came from datetime.fromtimestamp(), which gives local time, but was labelled Z and compared as UTC. East of UTC, an item updated shortly after its file was saved (within 11 hours in AEDT) looked unchanged, and the update was missed. West of UTC, unchanged items were fetched again. This PR compares in UTC.

This PR makes the following changes:

  • The modification time is converted in UTC for issues, pull requests and discussions.
  • get_repository_checkpoint_time had the same mistake in its fallback for repositories with no updated_at or pushed_at. It now uses time.gmtime() instead of time.localtime().

Compatibility

No breaking changes. Options, file formats, file names and the backup layout are unchanged, and existing backups work as they are. The fix changes only which items are skipped:

  • East of UTC: the first run after upgrading fetches any items whose updates were missed, so existing backups catch up without any action.
  • West of UTC: fewer items are fetched again unnecessarily.
  • The checkpoint fallback is the only change to a written value. GitHub's repository API always returns updated_at and pushed_at, so in practice the fallback isn't used, and existing last_update files aren't touched.

Testing

  • tests/test_incremental_by_files.py runs with the local time zone at UTC+10. For issues, pull requests and discussions, a file saved before the update is fetched again and one saved after is skipped. The checkpoint fallback is checked against UTC. The fetch-again cases and the checkpoint test fail without the fix.
  • pytest: 186 passed under UTC, America/Los_Angeles and Australia/Sydney. flake8 with CI's flags passes.
  • Real runs in AEDT, master vs this branch, across 25 repositories with --incremental-by-files. Fresh runs and immediate reruns matched, and both skipped all 866 issues and 422 pull requests on the rerun. With one issue file and one pull request file set an hour before their updated_at, master skipped both and this branch fetched exactly those two again.

The modification time of a saved issue, pull request or discussion was
formatted in local time but labelled UTC, then compared with GitHub's UTC
updated_at. East of UTC, items updated within the offset after they were saved
were skipped and their changes missed; west of UTC, unchanged items were
fetched again. The checkpoint time used when a repository has no updated_at or
pushed_at had the same local-time mistake.
@Iamrodos
Iamrodos force-pushed the fix/incremental-by-files-timezone branch from b4e3c99 to 7f204b8 Compare November 3, 2026 22:44
@Iamrodos Iamrodos changed the title Compare --incremental-by-files timestamps in UTC Fix --incremental-by-files skipping updated items east of UTC Nov 3, 2026
@josegonzalez
josegonzalez merged commit cb1fcd1 into josegonzalez:master Nov 4, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants