Repository navigation
Shell tasks: remove a finished task's launcher image - #1365
Merged
Merged
Conversation
Co-authored-by: alfonso-aft <289616620+alfonso-aft@users.noreply.github.com>
Co-authored-by: alfonso-aft <289616620+alfonso-aft@users.noreply.github.com>
Co-authored-by: alfonso-aft <289616620+alfonso-aft@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Each shell task keeps a
startup-image/folder in its io directory: a hard link to the running AFT binary plus theghshim, so the task's launcher and shim stay executable even if AFT is replaced while it runs. Nothing removed it when the task finished. On one Mac,bash-tasksheld 8,175 such links from about 9,000 tasks in a day: 0.5 GB on disk, but 923 GB to disk tools and backup software that count each link.When an image is removed: only once the task is terminal and no process from it can still need the launcher (
task_image_cleanup.rs::quiescent):Windows: AFT can't list a task's descendants there, so a dead shell isn't proof the task ended, and images stay. On NTFS they're normally hard links. Universal job tracking would let Windows clean them; that changes how tasks spawn, so it's a separate change.
Where it runs:
Removal uses the same pinned, no-follow directory operations as bundle cleanup. A failed removal logs at debug. Task status, output files and completion delivery don't change.
Tests: a finished image is removed on replay without changing output; old images are cleaned across bounded passes; a running record keeps its image; a descendant that outlives its shell keeps the image until it exits; a failed process listing keeps it; a survivor without a readable start time keeps it.
Gate on current main: 953
bash_backgroundandpinned_executabletests pass, plus formatting and the Windows check. The 12secrets::audit_followupstests fail on the build server the same way they fail on main there: it has no native sandbox. CI runs them.Summary by cubic
Removes a finished shell task's
startup-imagefolder so backup tools no longer count thousands of stale launcher hard links (0.5 GB on disk, but 923 GB for tools that count each link on one Mac). The image is only removed once the task is terminal and no process from it can still need the launcher.When the image is removed
Written for commit d8beaea. Summary will update on new commits.