Skip to content

Shell tasks: remove a finished task's launcher image - #1365

Merged
ualtinok merged 3 commits into
mainfrom
land/task-launcher-image-cleanup
Nov 3, 2026
Merged

ualtinok merged 3 commits into
mainfrom
land/task-launcher-image-cleanup

Conversation

@aft-alfonso

@aft-alfonso aft-alfonso Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Each shell task keeps a startup-image/ folder in its io directory: a hard link to the running AFT binary plus the gh shim, so the task's launcher and shim stay executable even if AFT is replaced while it runs. Nothing removed it when the task finished. On one Mac, bash-tasks held 8,175 such links from about 9,000 tasks in a day: 0.5 GB on disk, but 923 GB to disk tools and backup software that count each link.

When an image is removed: only once the task is terminal and no process from it can still need the launcher (task_image_cleanup.rs::quiescent):

  • the saved shell is dead, and the process group listing is clean;
  • every saved descendant is gone, judged by PID plus start time, so a reused PID doesn't count as a survivor;
  • a survivor whose start time can't be read keeps the image;
  • an in-flight kill, a pending descendant sample, or a running post-exit probe keeps the image.

Windows: AFT can't list a task's descendants there, so a dead shell isn't proof the task ended, and images stay. On NTFS they're normally hard links. Universal job tracking would let Windows clean them; that changes how tasks spawn, so it's a separate change.

Where it runs:

  • when a task's terminal state is recorded;
  • in the finished-task cleanup pass;
  • in the persisted cleanup pass, which also cleans images left by tasks that finished before this change, within its existing per-pass entry budget.

Removal uses the same pinned, no-follow directory operations as bundle cleanup. A failed removal logs at debug. Task status, output files and completion delivery don't change.

Tests: a finished image is removed on replay without changing output; old images are cleaned across bounded passes; a running record keeps its image; a descendant that outlives its shell keeps the image until it exits; a failed process listing keeps it; a survivor without a readable start time keeps it.

Gate on current main: 953 bash_background and pinned_executable tests pass, plus formatting and the Windows check. The 12 secrets::audit_followups tests fail on the build server the same way they fail on main there: it has no native sandbox. CI runs them.


Summary by cubic

Removes a finished shell task's startup-image folder so backup tools no longer count thousands of stale launcher hard links (0.5 GB on disk, but 923 GB for tools that count each link on one Mac). The image is only removed once the task is terminal and no process from it can still need the launcher.

When the image is removed

  • Requires a dead shell, a clean process group listing, and no surviving saved descendants (matched by PID plus start time); an unreadable start time or a pending kill, sample, or probe keeps the image.
  • Windows can't enumerate task descendants, so images there stay until job tracking is added.
  • Runs when terminal state is recorded, in the finished-task cleanup pass, and in the persisted cleanup pass, which also removes images from tasks that finished before this change.
  • Removal uses the same pinned, no-follow directory operations as bundle cleanup; failed removals only log at debug, and task status, output, and completion delivery are unchanged.

Written for commit d8beaea. Summary will update on new commits.

View guided diff Turn on auto-fix

ualtinok and others added 3 commits November 3, 2026 16:45
Co-authored-by: alfonso-aft <289616620+alfonso-aft@users.noreply.github.com>
Co-authored-by: alfonso-aft <289616620+alfonso-aft@users.noreply.github.com>
Co-authored-by: alfonso-aft <289616620+alfonso-aft@users.noreply.github.com>
@ualtinok
ualtinok added this pull request to the merge queue Nov 3, 2026
Merged via the queue into main with commit 7ca30b5 Nov 3, 2026
84 of 85 checks passed
@ualtinok
ualtinok deleted the land/task-launcher-image-cleanup branch November 3, 2026 18:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant