Repository navigation
fix(jni): keep Java owners alive during native calls - #111
Merged
Merged
Conversation
Signed-off-by: Christopher Wirt <cwirt@quantcast.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Christopher Wirt <cwirt@quantcast.com>
Fence only the native call in VortexWriter.bytesWritten/bufferedBytes, hoist DataSource.scan argument extraction out of the fenced block, and drop no-op JVM flag and daemon setting from ExpressionReachabilityTest. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Christopher Wirt <cwirt@quantcast.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #110.
Problem
Expression,Session,VortexWriter) own a native pointer. The Cleaner frees it once the wrapper is unreachable.x.nativePointer(), a plainlong, into JNI. Once thatlongis read, the JIT can treatxas dead.INfilters build thousands of temporary literals and pass them straight toliteralList, which makes this likely. The result is an invalid expression or a SIGSEGV.Fix
Wrap each JNI call that borrows a pointer in
try { ... } finally { Reference.reachabilityFence(owner); }:Expressionfactories: the operands, operand arrays, and dtype prototypes.DataSource.scan: the projection and filter expressions. Before this change onlythiswas fenced.DataSource.open*,NativeFiles.*,Session.enableEdition/knownEditions, andVortexWritercreate,writeBatch, and the metrics calls.IndependentBuffersTestpassedSession.create().nativePointer()from a temporary. It now keeps the session alive.Regression test
ExpressionReachabilityTeststarts a child JVM with a 32 MiB heap, a compiledliteralList, and a backgroundSystem.gc()loop. It builds 2,000 lists of 2,000 literals and checks the results of a realIN-filtered scan.Verification
All runs used a freshly built debug
libvortex_jnifrom this branch. The branch changes no Rust code.must be a literal expression, got @…(a freed expression):vortex-jni:testsuitespotlessJavaCheck,git diff --checkNot run: the production Spark workload, the
vortex-sparktests, and CI-wide checks. The test is probabilistic, so on the original code it can still pass in some runs.🤖 Generated with Claude Code