Repository navigation
ADFA-6854: Lazy Kotlin VFS over JarListing - #2231
Draft
itsaky-adfa wants to merge 11 commits into
Draft
itsaky-adfa wants to merge 11 commits into
itsaky-adfa wants to merge 11 commits into
Conversation
…ace a newer handler
…by another cache consumer
|
To enable Code Reviewer, link your GitHub account to your Atlassian account. This is a one-time task that takes less than a minute. Once your account is linked, resubmit the pull request to trigger a code review. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft, part 4 of 4 of the ADFA-6854 stack.
Fixes ADFA-6854. Do not merge. CI fails until the kt-android release, and the open items below block the stack.
Stack (bottom to top):
JarListing: one packed index per JAR for zipfs2 and javacJarListingWhat
ListingJarFileSystemis the Kotlin LSP'sjar://VirtualFileSystem. It is backed byJarListingCache.SHAREDand replaces IntelliJ'sCoreJarFileSystem, which built aCoreJarVirtualFilefor every entry of every JAR.VirtualFileis created only for entries something asks for. It stays the same object while referenced (weak-value cache per JAR).statcheck. Old files then reportisValid() == false, and handler generations keep a slow lookup from installing a stale handler.LibraryRootsScopefalls back to matching a file's JAR root by URL, so a replaced JAR's entries stay in their module.KotlinApplicationEnvironmentPininstalls the file system through a new kt-android hook,KotlinCoreApplicationEnvironment.jarFileSystemFactory(patch 0011). It does this before any application environment exists, and checks the result after acquiring it.Measured, whole stack
OnePlus Pad 2, cold open of CoGo at
stageffaa75be84, heap dump after open. "stack" is this branch (3254dcc1db) with the locally built kt-android jar.stageJarListing(PR 3)CoreJarHandler-> lazy VFS)stageThe timings are from the
a71e6da99cstack run, on battery (52%), while the baseline ran on the charger, so treat them as indicative. Heap figures are not affected.A moto g34 milestone run (the device the earlier memory milestones were measured on) is in progress.
Open items (block merging)
Scope completion regressed. In
JavaModuleAccessibilityChecker.kt(warm open, scope completion with prefixjav) it takes 12.5 / 14.2 / 13.6 s, against 2.3 / 2.1 / 2.2 s onstage. Member completion there is unchanged (0.49 / 0.08 / 0.07 s against 0.58 / 0.04 / 0.05 s). Measured so far:HeapTaskDaemon);stage;stage. Most of the growth is Kotlin known-class-name sets (18,974LinkedHashSets held inConcurrentHashMapcaches that are only softly reachable).Inferred, not yet confirmed: with the JAR copies gone, softly cached LL FIR library sessions survive longer, refill the freed heap up to the limit, and completion then spends its time in GC. Next: find the holder of those soft references and cap or release it.
javac residual, 26.6 MB. The shared package map now works (PR 2). What remains is javac's per-manager
pathsAndContainersByLocationAndRelativeDirectoryin each of 98 file managers. Reducing it needs a change inside javac or fewer file managers; 98 managers for one project may also mean managers pile up across syncs, still to be checked.kt-android release.
subprojects/kotlin-analysis-api/build.gradle.ktsstill pinsv2.3.255-dc58d97, which lacks the hook. One kt-android release with patch 0011 follows once the stack is verified, then the tag and SHA-256 bump lands here.Kotlin source indexing reported
indexed=2025of 2,027 scanned files on both stack runs (stage: 2,027). No index errors were logged; still to be explained.Tests
ListingJarFileSystemTest(13): shape againstCoreJarFileSystem, identity, laziness, STORED/DEFLATED/stub-prefix contents, rename and same-size replacement, handler races.LibraryRootsScopeTest, including a test that the environment's JAR file system isListingJarFileSystem.JarListingCacheTest(11).:lsp:kotlin:testV7DebugUnitTest649/649 on the new file system (with the local kt-android jar).