Repository navigation
Expose the facets of a result set in both packages (closes #290) - #299
Merged
Merged
Conversation
`ResultSet.facets` returns the engine's `FacetResults` of a `SELECT ... FACETS` query as a plain dict (`meta` with `exact`, `patched`, `lag_ms`; one entry per alias with `type`, `source`, optional `reason`, `buckets`, `other_count`, `missing`), or None for a statement without the clause. An unavailable facet is kept with its reason. The engine computes the facets when the rows are exhausted; asking first computes them at once and ends the rows (the result set then reads as empty), as the engine documents. Why neither package names the facet classes at compile time: both are also built against engines before 26.12.1 (the native pull request gate links 26.11.1, release-jvm builds the bridge against the image of the release version), and `ResultSet.getFacets()` and `com.arcadedb.query.facet` do not exist there. - arcadedb-jvm: pure Python over JPype (results.py). `getFacets()` is read through the engine's own accessors, so no bridge class changes; an engine without the method gives None. NaN and the infinities come back through `convert_java_to_python` as floats. The engine keeps the computed facets after `close()` (FacetCollectStep.facets() returns the memoized value, SelectExecutionPlan.close() only closes the steps), so reading them after the rows drained and the bridge closed the Java set works. - arcadedb (native): `FacetsReply.java` reads `ResultSet.getFacets()` by reflection (the classes are listed in reachability-metadata.json with the parser nodes of the clause) and writes the dict with `NativeCodec.append`, so non-finite floats, dates and decimals carry the tags rows use. `batchOf` appends it to the reply that drains the cursor, before the cursor is closed (`ArcadeNative.batchOf` closes it with that reply); a failure is the member `ferr`, raised when the facets are read. `arcadedb_n_facets` answers for a cursor with rows left: facets non-null closes the cursor (the engine ended the rows), null leaves it open. The lazy `Prefetched` cursors are read through their tail (`tail()`), since `Prefetched` cannot override a method of an interface that older engines lack. - A result set closed before its end raises ArcadeDBError for `facets`, like its rows. tests/test_result_set_facets.py runs on both packages with the public API only and is version-gated like the smoke test: an older engine must refuse the clause with 'SQL syntax error' and a plain query has no facets. Covered: TERMS, RANGES (and DEFAULT), calendar HISTOGRAM, disjunctive FILTER, STATS and DISTINCT, an unavailable HIERARCHY facet, NaN and the infinities, named and positional parameters in RANGES and INCLUDE through query and command, rows first then facets, facets first, a lazy command cursor, a cursor with rows left, and the counts against the same query's GROUP BY. Nothing was run locally (the laptop guard blocks workloads); CI is the first run. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…l index in two schema tests (refs #290) - test_rows_first_then_facets indexed a NativeResult with []; it reads the id through get(). - Asking for facets before the rows: on the native build the first batch is already buffered, so the engine's 'facets end the rows' rule is not observable the same way as on the JVM. The docstrings and the test now promise only that facets read first equal those read after the rows, and that whether rows not fetched yet stay readable is unspecified (read the rows first if both are needed). Neither package raises or returns a different dict when facets come first. - test_drop_index and test_drop_existing_index_force took the first index whose name matched, which on the 26.12.1 snapshot can be a per-bucket index (`User_0_<id>`) that the engine refuses to drop alone. They take the type-level `User[temp]`, valid on 26.11.1 too. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
tae898
added a commit
that referenced
this pull request
Nov 7, 2026
…lumnar settings (closes #292) (#300) * Test the COLUMNAR index on both builds and document its settings (refs #292) Why no change to reachability-metadata.json: I read every class of the engine's index/columnar and query/facet packages (26.12.1 snapshot, ArcadeData/arcadedb main 4a91847) for reflection, ServiceLoader, resources and Class.forName. There is none: the projection is plain JDK (FileChannel, VarHandle byte-array views, java.util.concurrent pools and a daemon ticker thread), the index handler is registered by instance in LocalSchema, facet.searchValues by instance in DefaultSQLFunctionFactory, and the only parser nodes the COLUMNAR and FACETS statements add are FacetsClause, FacetItem and FacetRangeItem, which PR #299 already listed. Upstream's own reachability-metadata.json lists no columnar class either. The entries upstream has and ours lacks (SQLFunctionEnsureLabelTypes, antlr StatementExpression and SubqueryExpression, three Ratis gRPC classes) are not columnar and are left for a change of their own. Why a child process for the unclean close: DatabaseInternal.kill(), which the engine's ColumnarCrashRecoveryTest uses, is not reachable from the native package, and a JVM-only hook would not test both. A writer that ends with os._exit after its commits leaves database.lck behind and the commits in the write-ahead log, which is the state the engine's recovery reads (a crash of the process, not a power loss). Two variants: commits unflushed, and commits flushed to published segments before the rest is written. Why no new setter for arcadedb.columnar.*: ALTER DATABASE already takes the DATABASE-scope keys (immediately, stored with the database) and refuses the JVM and SERVER ones; those are system properties through ARCADEDB_JVM_ARGS or ARCADEDB_NATIVE_ISOLATE_ARGS, as for arcadedb.fileSync. The test pins that, and that GROUP BY returns the same rows with the acceleration on and off (PROFILE names the path). tests/test_columnar_index.py is version-gated like the smoke test (an older engine must refuse the index and the keys with its own error), runs on both packages with the public API, and polls states with deadlines. Nothing was run locally (the laptop guard blocks workloads); CI is the first run. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Return a Python str from Database.get_database_path(), get_name() and list_vector_indexes() on arcadedb-jvm (refs #292) The first CI run of tests/test_columnar_index.py failed on arcadedb-jvm only: Database.get_database_path() returned the raw java.lang.String of getDatabasePath(), which os.path.join refuses (TypeError: expected str, bytes or os.PathLike object, not String). The native build returns a str, so the lanes disagreed. Same family as #286 (get_type_name, fixed in #287): the binding wraps the Java getter in str(). Sweep of src/arcadedb/*.py for a method that documents a str and returns a Java getter result unconverted (an AST pass over every def annotated with str or Path, looking at what each return expression is): Database.get_name(), Database.get_database_path() and Schema.list_vector_indexes() (a list of java.lang.String). Fixed with str(). Left alone because they are documented as returning Java objects: Document.get_identity(), Schema.get_type(), get_types(), get_index_by_name(), Document.get_raw(), Result.get_raw(). Result.get_rid(), Document.get_rid(), get_type_name() and the vector index names already convert. tests/test_str_returns.py asserts type(x) is str on both packages for the four str returns that can be read through the public API (name, path, vector index names, Result.get_rid), plus that the path joins. Nothing was run locally; CI is the first run. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
tae898
added a commit
that referenced
this pull request
Nov 7, 2026
…searchValues (closes #291) (#301) * Facet kinds from Python: HIERARCHY, DISTINCT, STATS, AFTER and facet.searchValues (closes #291) Reading the engine (ArcadeData/arcadedb PR #11344) against the decoders of #299 found no gap in either package: STATS min and max, DISTINCT value and exact, the hierarchy attributes (parent, depth, orphan) and the AFTER cursor all go through the same value conversion a row uses (convert_java_to_python on the JVM, NativeCodec.append in FacetsReply on native), and facet.searchValues is an ordinary SQL function whose rows decode like any other. So no code changes, only what the issue asks for: proof and documentation. - tests/test_result_set_facets_kinds.py runs on both packages through the public API and is version-gated like test_result_set_facets.py (an older engine must refuse the clause with 'SQL syntax error'). STATS: int, FLOAT and DOUBLE columns with exact sums, DATE and DATETIME extremes equal to what a row of the column holds (sum and avg None), no match, records without a value, NaN and the infinities against SQL's own min, max, sum and avg, signed zeros. DISTINCT: exact counts against GROUP BY, a list indexed BY ITEM, PRECISION n with exact False and meta exact False. HIERARCHY: the flat bucket list with parent, depth and orphan, TOP and other_count, a FILTER parameter. AFTER: paging a STRING, LONG and DATETIME facet until the cursor is None and comparing with GROUP BY, MISSING once on the last page, a positional cursor, a cursor past the end, other_count on the last page, the refusal of ORDER BY COUNT. facet.searchValues: prefix, limit, counts against GROUP BY, named, positional and literal arguments, a deleted record's value, the errors. - Parameters: named and positional in every FILTER (STATS, DISTINCT, HIERARCHY's neighbour) and in AFTER, through query and command. The grammar takes TOP, MIN_COUNT and PRECISION as integer literals only; that is documented. - facet.searchValues reads dictionaries the projection has applied, so its tests retry until the projection caught up (a hang detector, not a latency bound); facet queries need no wait, the engine patches lagging records. - The facets docstring of both packages now states the shape of each kind and which clauses take parameters. Nothing was run locally (the laptop guard blocks workloads); CI is the first run. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Facet kinds tests: make the approximate DISTINCT case cross the engine's threshold (refs #291) The native snapshot run showed `DISTINCT PRECISION 10` on a column of ten values reports exact True. The engine (DistinctFacetCollector) keeps the exact set while it holds at most n values and switches to a HyperLogLog only when exactSize() > precisionThreshold, so ten values never cross PRECISION 10; my test guessed the flag. The approximate case now uses PRECISION 5 on the ten-valued DOUBLE column price (the numeric path the engine documents as approximable), with the error tolerance widened to 2, and a new test pins the other side: PRECISION 10 on the same ten values stays exact. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #290. Part of #289.
I added
ResultSet.facetsto both packages. It returns the facets of aSELECT ... FACETSquery as a plain dict:meta(exact,patched,lag_ms) and one entry per alias withtype,source, an optionalreason,buckets(value, orkeywithfromandto, pluscountand themissinganddefaultflags),other_countfor terms and hierarchy, andmissing. A statement without the clause givesNone. A facet that isunavailablekeeps itsreasonand is not dropped. NaN and the infinities in facet values are Python floats, as in rows. The facets cover every match whateverSKIPandLIMITkept. As the engine documents, they are computed when the rows are exhausted, and asking for them first computes them at once and ends the rows (the result set then reads as empty). A result set closed before its end raisesArcadeDBErrorforfacets, like its rows.Neither package names the facet classes at compile time, because both are also built against engines before 26.12.1 (the native pull request gate links 26.11.1, and the JVM release builds the bridge against the image of the release version), and
ResultSet.getFacets()does not exist there.arcadedb-jvmreadsgetFacets()through JPype inresults.pyand converts it with the engine's own accessors, so no bridge class changes. The engine keeps computed facets afterclose()(FacetCollectStep.facets()returns the memoized value andSelectExecutionPlan.close()only closes the steps), which is why reading them after the rows drained works. I read that in the engine source, I did not run it.arcadedb(native) readsgetFacets()by reflection in the newFacetsReply.javaand writes the dict withNativeCodec.append, so non-finite floats, dates and decimals carry the tags rows use.batchOfappends it to the reply that drains the cursor, before the cursor is closed.arcadedb_n_facetsanswers for a cursor with rows left. The lazyPrefetchedcursors are read through their tail. The facet classes and the three parser nodes of the clause are added toreachability-metadata.json; entries for classes an older engine lacks are ignored by the image build, as the existingandroid.app.Applicationentry shows.tests/test_result_set_facets.pyruns on both packages with the public API only and is version-gated like the smoke test, so on an engine older than 26.12.1 it asserts the clause is refused withSQL syntax errorand that a plain query has no facets. It coversTERMS,RANGES, a calendarHISTOGRAM, a disjunctiveFILTER,STATSandDISTINCT, an unavailableHIERARCHYfacet, NaN and the infinities, named and positional parameters inRANGESandINCLUDEthroughqueryandcommand, rows first then facets, facets first, a lazycommandcursor, a cursor with rows left, and the counts against the same query'sGROUP BY.Verified: black, isort, pyflakes,
compile()of every Python file and Bandit are clean. Not verified: nothing was run locally (the laptop blocks workloads), so the Java compile, the reflection on the native image, the engine's behaviour for the unavailable facet and the disjunctiveFILTER, and every test are first exercised by CI. I dispatch a native run against26.12.1-SNAPSHOTas the proof of the native side, because the pull request gate still links 26.11.1. The user guide is #293 and is not part of this change.🤖 Generated with Claude Code