Repository navigation
refactor: define every stage's interfaces from #1207 and delete the ones they replace (#1207 PR 0) - #1226
refactor: define every stage's interfaces from #1207 and delete the ones they replace (#1207 PR 0)#1226zzylol wants to merge 19 commits into
Conversation
4772efb to
f57ae08
Compare
|
Per-file test deletions in #1226 (origin/main → PR head).
Restored in the 2026-11-06 revision through the test-only |
…nes they replace (#1207 PR 0)
f57ae08 to
2966355
Compare
…iming, lifetime }
Deletes Schedule and the σ-only InvalidPhysicalProperties variants; the only
invalid combination left is (IngestionTime, OneEvaluation). satisfies_physical
checks timing x lifetime only; delivers_on is the per-context alignment check
(w | I and w | φ). A OneEvaluation result runs in one evaluation context, so
x_{κ,v} covers it as for ℓ = None.
…physical drops memo InvalidLogicalProperties gains SketchBuildWithoutParameters and renames FloorNotInParameterList to ParametersNotInList, returned by LogicalProperties::new. LogicalRequirement::new no longer checks list membership (a floor is any value of the family's parameter type, e.g. heap_size >= k), so it returns Self. satisfies_physical takes only the producer and consumer.
Context::QueryEvaluation(QueryId) becomes Context::Evaluation(EvaluationSchedule):
queries with the same (interval, phase) are evaluated together and share
their OneEvaluation results. evaluation_schedule returns that schedule.
QueryId stays for z_{q,v}. A non-periodic query has no context yet.
InvalidLogicalProperties -> LogicalPropertiesError, InvalidPhysicalProperties -> PhysicalPropertiesError, LogicalRequirement -> LogicalRequirements (parameter_floor -> parameter_floor_for_required_accuracy), satisfies_logical / satisfies_physical -> producer_satisfies_logical_requirements / producer_satisfies_physical_requirements, Medium -> StorageMedium, Context / ContextId -> ExecutionContext / ExecutionContextId (x_in_context / y_in_context -> x_in_execution_context / y_in_execution_context). PhysicalCandidate and PhysicalCandidateKey fields -> logical_candidate_id, physical_properties; edge fields -> producer_candidate_id, consumer_candidate_id; PhysicalASAPMemoCandidates fields -> logical_candidates, physical_candidates.
Deletes EvaluationSchedule and ExecutionContext::evaluation_schedule. An
evaluation context is Evaluation { recurrence, range_evaluation }: queries
with equal ones share it and its OneEvaluation results, one-off queries
included. ExecutionContext, ExecutionContextId and RefreshWindowId move to
asap_types::stages so PhysicalCandidate::delivers_on can take the context.
Contexts are deduplicated by equality (QueryRecurrence holds an f64).
PhysicalASAPDAGValidationError::InvalidPhysicalProperties becomes
PhysicalProperties { node, reason }. In the files this unit edits, "kept"
describing a materialized result or retained data becomes "materialized"
or "retained" (cost, materialized views, realizer, installable plan,
deployment capabilities; locals kept/raw_kept_rows -> retained/
raw_retained_rows), and "reader" becomes "query" where it is a query
(execution demand, ScheduleGroup::queries, cost usage, labels, tests) or
"consumer" where it is a consuming candidate.
find_or_add_group returns the node rules' SchemaDerivationError and add_candidate a CandidateError (the node rules, or an output that differs from the group's). Bodies stay todo!() until PR 2.
…mo (#1207 PR 0) Memo::groups is private, read through groups(); groups are added only through find_or_add_group and queries through add_query (todo!() until PR 2). memo_of_queries returns Result and its body is todo!() until PR 2, which builds Stage 0 on find_or_add_group; Stage 0's node-level CSE (cse.rs), its test_support export and the dead share_identical helper are deleted.
200ec5c to
341c98b
Compare
…OperatorInput; coverage only in the memo (#1207 PR 0) MemoGroup { expression, group_properties: GroupProperties, parents, queries, candidates } replaces logical_node and child_groups; GroupProperties { schema, result_kind, summary_coverage, shareable_when_equal } is derived once by derive_group_properties, which applies the existing rules to the expression over MemoGroupAsInputs. The rules (output_schema, output_kind, validate_inputs, is_shareable_when_equal, summary builds, window chains, scalar subqueries, top-k, populations) are generic over OperatorInput, implemented for Rc<OperatorNode> and MemoGroupAsInput; only how they read an input changed. BuiltOrInput carries the builds a rule makes while deriving (one build over a chain's window, the build a rollup or a selection stands for). SummaryCoverage { definition: SummaryStateDefinition { input_group, unlifted_operators }, selection } is derived only from memo groups (derive_from_inputs) and stored in GroupProperties; node-tree coverage, NodeToSummaryCoverageCache, derive_cached and OperatorNode::coverage are deleted, so node trees have no merge check. Memo::logical_node_trees is deleted: Pass 1 target discovery (PR 3), devtools group labels (PR 9) and the export's view coverage (PR 6) are todo!(). find_or_add_group and add_candidate derive the properties, then todo!() until PR 2.
341c98b to
3a75e1e
Compare
…elpers (#1207 PR 0) One state's θ is no longer sized for several consumers: θ comes from parameter_list per candidate. size_for_all_readers had no non-test caller; min_recorded, kll::shared_k and univmon::shared_heap served only it.
|
Key code interfaces, continued (moved here from the PR body, which GitHub limits to 65,536 characters). Links point at the head.
|
| File | Tests | Restored by |
|---|---|---|
types/tests/summary_coverage.rs (file deleted) |
32: coverage derived on node trees (lifting, merges, time windows, rollups, definition ids) | PR 2's types/tests/memo_summary_coverage.rs (8) over memo groups; the rest as later PRs need them |
integration-tests/tests/summary_coverage_e2e.rs (file deleted) |
8: query string → planned state → coverage() |
no replacement planned: it read node-tree coverage of front-end trees |
types/tests/summary_merge_structure.rs |
idempotent_families_merge_overlapping_states (node merge check) |
PR 2's states_with_overlapping_selections_do_not_merge |
types/tests/tumbling_window_chain.rs |
a_rollup_of_the_newest_tumbling_windows_covers_their_window; the coverage asserts of 5 more |
PR 2's a_tumbling_window_chain_covers_its_whole_window |
types/tests/tumbling_window_chain.rs |
a_merge_of_the_newest_tumbling_windows_covers_their_window (read the build through the node-tree merged_window_build) |
PR 4, with the merged-windows guarantee over groups |
devtools/src/bin/asap_planner.rs |
memo_node_table_stores_a_shared_node_once (with MemoNodes::add) |
PR 9 |
accuracy/src/estimators/mod.rs, mod shared_size_tests |
4: size_for_all_readers |
no replacement: deleted with the function |
The coverage asserts in integration-tests/tests/sketch_rollup.rs (2 tests) and logical-asap-optimization's per_computation_candidates_tests.rs (1 test) are removed; the tests stay.
Kept by building the DAG directly (decided 2026-11-06). Tests that hand-built operator roots and ran them now export them with the test-only physical_asap_dag_at_query_time(_with_node_ids) instead of the deleted roots-based export. Restored: executor time_series_window, node_ids, current_series_heap, planspace_series_identity_heap, 3 of deployment_computation's tests; integration filtered_aggregates, kll_tumbling_window_execution, per_series_exact_execution, sql_cardinality, sql_frequency, exact_composition, and parts of sketch_rollup, stage1_logical_asap_optimization, operator_design_examples, operator_sharing, promql_numeric_regressions, sql_to_post_asap, promql_to_post_asap; types physical_export; frontend count_planning, pearson_corr. types/tests/physical_export.rs::untimed_plan_is_rejected (the +1 above, deleted after the comment was posted) is deleted: the builder always assigns query time, and an untimed export is no longer possible. Restored sketch tests fail at the executor's todo!() until PR 6, because the builder leaves parameters: None. Tests whose DAG came from the planner (runtime_install's planning tests, sql_per_group_topk, sql_table_scans, per_sample_updater, the planner-built deployment_computation tests, and others) stay deleted; PR 6 restores them.
Moved from the PR body (2026-11-08, body size limit)
2026-11-07 round, 10 more:
| File | Tests | Restored by |
|---|---|---|
types/tests/tumbling_window_chain.rs |
6: merged_tumbling_windows_cover_the_whole_window, each_tumbling_window_is_one_shifted_minute, expansion_merges_the_tumbling_windows, a_merge_of_the_newest_tumbling_windows_reads_one_chain_and_fewer_than_all_tumbling_windows, newest_tumbling_windows_serializes_only_when_set, expansion_keeps_a_rollup_of_the_tumbling_windows (window count, newest windows, expansion) |
PR 1 (TimeRange over a chain), PR 6 (executor by window index) |
types/src/ir/properties/summary_coverage.rs |
3: large_reverse_ordered_tumbling_window_union_is_one_exact_interval, interval_fast_path_matches_general_union_for_permutations, sorted_union_preserves_gaps_boundaries_and_original_group_order (relative_time union) |
PR 2 (timestamp-column intervals) |
planner/tests/runtime_install.rs |
tumbling_windows_of_an_hour_share_one_definition_with_distinct_times (expanded windows' relative_time) and its now-unused helpers plan_json, plan_json_for |
PR 6 |
…perators; delete merged_window_build_node (#1207 PR 0) - BuiltOrInput / BuiltInput renamed InputOrTemporaryBuild / TemporaryBuild: a build a rule constructs only while it derives, never stored in the memo. - SummaryStateDefinition::unlifted_operators is Vec<Operator<Option<MemoGroupId>>>: None marks an operator's main input, Some(g) another input group it reads (a scalar subquery's). States whose definitions read subqueries compare and merge; CoverageError::SubqueryInDefinition deleted. - merged_window_build_node deleted with its test; the accuracy guarantee of a read of merged tumbling windows is todo!() until PR 4 rewrites it over groups.
…, TimeBound, Pass 2 pruned list, Stage 2 entry (#1207 PR 0) - Delete WindowLayout, LogicalProperties::window_layout and its two error variants, SummaryMerge::newest_tumbling_windows and SummaryTumblingWindowChain::tumbling_window_count, with the count-based chain builds and expansion; behaviour that read the count is todo!() for PR 1, 2, 6, 7 or 9. - SelectionBox { columns } only, and TimeBound { At, FromEvaluationTime }; lifting TimeRange/TimeShift is todo!() for PR 2. - PrunedLogicalCandidate, PruneCause, RootPruneReason and LogicalASAPMemoCandidates::pruned. - error_propagation(consumer, metric) -> Option<ErrorPropagation>; Memo::set_input_requirements. - PhysicalCandidateKey deleted (PrunedCandidate<PhysicalCandidate>), PrunedCandidate::group and PhysicalASAPMemoCandidates::query_roots dropped; new(logical, physical, edges, pruned). - Stage 2 entry takes &DeploymentCapabilities (moved into asap-types) and cannot fail; PruneReason follows the doc's Stage 2 table. - DataArrival::{Ingesting, Mixed} gain max_out_of_order_ms. - GroupProperties::summary_coverage -> aggregate_coverage.
633460c to
22caa51
Compare
…l; ScalarValue::Timestamp (#1207 PR 0) Replace TimeBound by IntervalConstraintLimit { FixedValue(ScalarValue), EvaluationTimePlusMs(i64) }: Constraint::Interval holds Bound<IntervalConstraintLimit> on every column, so the timestamp column and value columns share one interval type. Doc comments carry the examples (m[5m], offset 1h, @ T, SQL ts >= literal, price > 10). ScalarValue::Timestamp(ms since the epoch) is the fixed value of an @ T or a SQL timestamp literal.
…s; delete ParametersOfAnotherFamily (#1207 PR 0) A range m[r] evaluated at t reads (t - r, t]: IntervalConstraintLimit and Constraint::Interval examples, the chain window doc and the primitive-schema table now use Excluded lower and Included upper ends. A parameter list holds only its own family's parameters, so ParametersNotInList also rejects another family's parameters.
…acy, the plan is one PhysicalASAPDAG (#1207 PR 0) - Terminology: ops' `family` is `summary_type`; FieldDataType::Sketch(SketchAlgorithm) (SketchKind, GroupingStrategy, HydraKind and the ops' `grouping` deleted); FamilyCapability is SummaryTypeCapability; parameter_list(algorithm). - θ is structure only: SketchParams derives equality; KLL failure_probability, UnivMon distinct_failure_probability and HLL/KMV relative_error deleted. - ValueSign and GroupProperties::column_sign; SourceFacts::column_signs replaces values_non_negative. - LogicalRequirements { required_accuracy: Option<RequiredAccuracy> }; RootPruneReason::BelowRequiredAccuracy; Memo::add_query takes the target and n_q; AggIntent measures carry no accuracy and front ends take none. - asap-accuracy: AccuracyEvidence::measured_guarantee, theoretical_guarantee (was local_guarantee), producer_satisfies_logical_requirements(memo, producer, consumer, input_index, evidence); the AccuracyModel trait and evidence provider, Pass 1's accuracy pruning and AccuracyCheckInputs deleted. - Physical: ExecutionContext::Evaluation { grid: EvaluationGrid }, IngestionPipeline(RefreshWindow); RefreshWindowId deleted; producer_satisfies_physical_requirements(producer, consumer_timing); PruneReason::CountMinWeightsNotProvenNonNegative. - Plan: PlanOutput { plan: PhysicalASAPDAG }; PhysicalASAPDAGNode:: selected_physical_candidate; select_plan returns (PhysicalASAPDAG, Selection); Selection { accepted_accuracy_per_query, pruned_candidates, milp_* }; RealizedPlan, PhysicalWorkloadPlan, DagOfSelectedCandidates, SelectedEdge, SelectedNodeId, AssignmentRealizer, ScanWindow and the scan/view window functions, SearchReport, SearchStatus, InexactObjective, ObjectiveDiffersFromPlanCost, SelectedPlanFailedCheck and PlanCost::cost_unit deleted. - Plan file wire version 14 (summary_type, Sketch(algorithm), no grouping). - Cost names: NodeCostSource, cost_source, PlanRanking::Cost; Serialize on PruneReason, PhysicalProperties, OriginOfCandidateForMemoGroup and RootPruneReason.
|
PR 0 body as of 7e83db9 (before the 2026-11-08 interface round), kept here so the PR body fits. Items this round changed are described in the body; where they differ, the body wins.
WhyThe memo, the properties, the edges and the Stage 3 variables of the optimization formulation have no named home in today's code:
#1207's new order (rewritten 2026-11-06) is interfaces first. This PR defines every data structure and interface of the concepts doc What
Decisions: approved and pendingEach new item, with its status. Approved items were decided by the user; pending items await approval.
Key code interfacesAll items below are new, except where marked:
Function bodies are shown as
|
| Doc section | Interface | Code |
|---|---|---|
| 3, Stage 0 |
MemoGroupId, MemoGroup { expression, group_properties, parents, queries, candidates }, GroupProperties, MemoGroupAsInput
|
crates/types/src/stages/memo.rs:16 |
| 3, Stage 0/1 |
Memo { groups (private), query_roots }, groups, find_or_add_group, add_candidate, add_query, candidate, CandidateError, derive_group_properties
|
crates/types/src/stages/candidates.rs:25 |
| 3, Pass 1 |
CandidateForMemoGroupId, CandidateForMemoGroup, CandidateExpression, LogicalProperties, parameter_list, Memo::set_input_requirements, LogicalRequirements, producer_satisfies_logical_requirements
|
crates/types/src/stages/candidates.rs |
| 3, group properties |
OperatorInput, InputOrTemporaryBuild, TemporaryBuild; SummaryStateDefinition, SummaryCoverage::{derive_from_inputs, definition_id}
|
crates/types/src/ir/operator/input.rs, crates/types/src/ir/properties/summary_coverage.rs
|
| 4 |
ErrorPropagation, LogicalRequirements::error_propagation
|
crates/types/src/stages/candidates.rs |
| 3, Pass 2 | rule signature fn(&mut Memo) -> Result<(), Pass2RuleError> (rules 1–7), LogicalEdge, LogicalASAPMemoCandidates { memo, edges, pruned }, PrunedLogicalCandidate, PruneCause, RootPruneReason
|
crates/logical-asap-optimization/src/pass2/, crates/types/src/stages/logical_memo_candidates.rs
|
| 3, Stage 2 |
ExecutionTiming (existing, re-exported), StorageMedium, Lifetime, PhysicalProperties
|
crates/types/src/physical/properties.rs |
| 3, Stage 2 |
PhysicalCandidateId, PhysicalEdgeId, PhysicalCandidate (+ delivers_on), producer_satisfies_physical_requirements, PhysicalEdge, PhysicalASAPMemoCandidates (+ edges, admissible_edges_into, admissible_edges_out_of), PrunedCandidate<PhysicalCandidate>; Stage 2 entry generate_physical_asap_memo_candidates(logical, data, capabilities)
|
crates/types/src/stages/physical/memo_candidates.rs |
| 3, Stage 3 |
SelectionVariables { x, y, z, t }, constraints (A)–(F), SelectedEdge { producer, consumer, input_index_of_consumer }, DagOfSelectedCandidates
|
crates/plan-selection/src/milp/model.rs, crates/types/src/stages/physical/graph.rs
|
| 5 |
ExecutionContextId, ExecutionContext, RefreshWindowId
|
crates/types/src/stages/physical/execution_context.rs |
| 5 |
|
crates/plan-selection/src/milp/model.rs |
| 4 |
|
crates/plan-selection/src/milp/model.rs |
Deleted, and what replaces each
| Deleted | Replaced by |
|---|---|
Materialization (types/src/physical/materialization.rs) |
PhysicalProperties { timing, lifetime } |
OperatorNode::{timing, materialization}, with_timing, with_materialization
|
each PhysicalCandidate::properties (doc section 3, realizer) |
PhysicalPropertiesAssignment/MaterializationAssignment side map, the timing pass timing.rs (apply_*, data_state, timing_under_consumer, validate_at_ingestion, planned_data_state, NodeToTimedNodeCache) |
realizer and export from PhysicalCandidate (PR 6) |
PhysicalWorkloadPlan::physical_properties, TumblingWindowChainExpansion::physical_properties, PlanOutput::physical_properties
|
same, PR 6 |
DagOfLogicalMemoGroups (+ group_for_node, roots_over_groups) |
one Memo
|
MemoGroup::{logical_node, child_groups} |
expression and group_properties
|
accuracy::estimators::size_for_all_readers (one state's θ shared by several consumers, sized alone), its helpers min_recorded, kll::shared_k, univmon::shared_heap, and its 4 tests (shared_size_tests); the develop doc's paragraph on kll::shared_k (decided 2026-11-06) |
deleted, no replacement: θ is chosen per candidate from parameter_list, and consumers of one group read its candidates through edges, so nothing sizes one state for several consumers. It had no non-test caller after Pass 2's shared builds were deleted. |
node-tree summary coverage: SummaryCoverage::{derive, derive_cached}, NodeToSummaryCoverageCache, OperatorNode::coverage(), the merge check in OperatorNode::new / validate_structure, definition_hash/same_definition over nodes, materialized_view::with_replaced
|
coverage derived once per memo group (derive_from_inputs, stored in GroupProperties); the export's view coverage reads it in PR 6 |
Memo::candidates (Vec<Vec<_>>) |
MemoGroup::candidates |
CandidateForMemoGroup::{root_node, input_groups, accuracy_budget_parts} |
expression: CandidateExpression (children = input groups), logical_properties, input_requirements
|
OriginOfCandidateForMemoGroup::{SharedBuild, SummaryResult} |
hash-consing plus one state group per family; readouts are groups (PR 3) |
Pass 1 memo_population.rs (try_populate_pass1, NodeToStructuralIdCache), size_for_failure_probability, budget-share sizing (accuracy_budget_parts_per_measure, compose_*_with_accuracy_budget_parts, Sizing::shares), PerComputationCandidates::walked
|
Pass 1 candidates via find_or_add_group/add_candidate with parameter_list (PR 3) |
pass2/asap_aware_cse.rs (node-level window, rollup, top-k and scan-widening rewrites) and pass2/finer_aggregate.rs
|
Stage 0 hash-consing and find_or_add_group, with rules 1–7 over groups (PR 4) |
rule 8 summary_results.rs
|
Pass 1 makes the readout a group (PR 3); the doc deletes the rule |
rule 9 shared_configuration_readers.rs, shared_producer.rs, shared_configurations.rs
|
the state group holds every θ; |
step 10 state_groups.rs (add_shared_summary_state_groups, per-size groups, check_unexpanded_memo) and reuse.rs
|
one state group per family (PR 3/4) |
pass2/memo/enumeration.rs (ChosenCandidatePerMemoGroup, enumerate, build_logical_asap_dags_from_chosen_candidates), asap_planner::selected_chosen_candidates
|
doc Appendix A enumerator (PR 6 check) |
OperatorNodeOfCandidateForMemoGroup(Id), PhysicalCandidate::{operator_node, timing, materialization, producer_candidates_per_input, may_produce_query_output}, PhysicalASAPMemoCandidates::{operator_nodes_of_candidates_for_memo_groups, implementation_roots, query_output_candidates, output_reference_count, physical_node, memo_group_of}
|
PhysicalCandidate { logical_candidate_id, physical_properties }, edges(), admissible_edges_*
|
ProducerCandidates, ProducerSetInterner, producer_storage_metrics
|
PhysicalEdge list (PR 5) |
ReaderConstraint (all four variants) and reader_constraints
|
the per-context alignment check delivers_on; the cost model (doc section 5, "No consumer-count policies") |
may_keep_across_evaluations_per_sole_reader_query |
Lifetime in PhysicalProperties, chosen by Stage 3 |
Stage 2 catalog (configurations.rs incl. compatible_timing, factoring.rs, inventory.rs, pruning.rs, reconstruction.rs), split_by_timing.rs, materialization and ingestion-eligibility rules in lib.rs
|
generate_physical_asap_memo_candidates and producer_satisfies_physical_requirements (PR 1, PR 5); realizer (PR 6) |
reference pricer optimize_physical_asap(_chosen), features reference-pricer/test-support of the Stage 2 crate |
doc Appendix A enumerator (PR 6 check) |
plan-selection reference_pricer, selector.rs, WholePlanSelection, feature reference-pricer
|
same |
groups.rs (ResultSlot, slot_of, result_slots, run_kind, execution contexts, dominance pruning), groups/accuracy_positions.rs
|
ExecutionContext and |
milp/model.rs (#1144: p[ctx,c], e binaries, schedule groups z[m,G], N[q], A[q, parts], constraint 8, constraint 9) |
SelectionVariables and (A)–(F) (PR 6), contexts (PR 7), (K) (PR 8) |
milp/cost_terms.rs, warm_start.rs, completion.rs, decode.rs
|
PR 6 (baseline objective, decode |
milp/lp_diagnosis.rs, feature lp-diagnosis
|
deleted, no replacement |
checks.rs (post-selection accuracy and latency checks), accuracy_binding.rs
|
(K) in PR 8; the latency check: deleted, no replacement (latency is out of scope) |
SelectionModels |
deleted with its only users |
devtools memo_pricing (reference_select, reference_enumeration, DEFAULT_REFERENCE_CHOICES, PlanningStagesRun::reference, the max_combinations/reference_choices parameters), fuzz SearchKind::Exhaustive and EXHAUSTIVE_MAX_COMBINATIONS
|
doc Appendix A (PR 6) |
AnalyticalCostError::PhysicalPropertiesError, PhysicalASAPDAGNode::physical_properties (both from the #1214 port) |
the wire validator checks (IngestionTime, OneEvaluation) directly |
PerComputationLogicalMemoCandidates, Pass 1's PrunedWayToCompute / pruned_ways_to_compute / pruned_of_target, and PruneReason's Pass 1 variants (RequiresExact, NullableAvgInput, NoRelationalCountSketchRealization, UnsupportedByDeployment, QuantileOutsideUnitInterval, TopKHeapOverOneRowPerItem, NoErrorBoundForQueryTarget) |
Pass 1 returns Memo; showing what Pass 1 leaves out moves to #1205 |
SketchKind inside FieldDataType::Sketch; accuracy::sizing::sized (budget-share sizing); propagation::with_resized_summaries, sizing::{with_summaries_sized_for_failure_probability, with_summaries_sized_for_targets, per_input_failure_probability, sized_for_failure_probability}
|
θ in LogicalProperties::parameters from parameter_list (PR 1/3) |
roots-based export: compile_physical_asap_dag, compile_physical_asap_workload(_with_node_ids), PhysicalASAPDAGCompilation, PhysicalASAPNodeIdentityMap, PhysicalASAPDAG::{with_every_node_at_query_time, query_roots}, ReferencePlanError, PlanOutput::physical_asap_dag, ExecutionDataStateError, InstallablePlanError::Timing
|
AssignmentRealizer::realize (PR 6) |
asap_planner::test_support::query_time_reference and the planner's test-support feature |
deleted, no replacement yet: a query-time reference needs a realization at query time; PR 6 decides |
2026-11-07: WindowLayout, LogicalProperties::window_layout, LogicalPropertiesError::{WindowOnOperatorThatCannotBeWindowed, ZeroWindowWidth}
|
the chain operator's width argument (chain groups, PR 4 rule 1) |
2026-11-07: SummaryTumblingWindowChain::tumbling_window_count, SummaryMerge::newest_tumbling_windows (and the pane_count/newest_panes aliases); the count-based builds whole_window_build, merged_window_build, window_build, tumbling_window, shifted, expand_tumbling_window_chains, node_of_build, InputOrTemporaryBuild::to_node, test_support::{whole_window_build, tumbling_window}; CoverageError::TumblingWindowChainExpansion, InstallablePlanError::TumblingWindowChainExpansion
|
TimeRange/TimeShift over a range-free chain select windows (PR 1, PR 2); the executor runs them by window index (PR 6); retention is |
2026-11-07: SelectionBox::relative_time with its wire bounds, windows_ordered_apart, disjoint_in_time, the time-box union fast path union_ordered_time_boxes
|
intervals on the timestamp column with IntervalConstraintLimit ends (PR 2) |
2026-11-07: TimeBound { At, FromEvaluationTime } (from the first 11-07 round) |
IntervalConstraintLimit in Constraint::Interval
|
2026-11-08: LogicalPropertiesError::ParametersOfAnotherFamily
|
ParametersNotInList: each family's list holds only its own parameters (PR 1 deletes the check) |
2026-11-07: cost's chain_input_offset_ms and the count-based chain cost and retention (now todo!()); fuzz helpers find_first, collect; integration helper planner_layering_common::retention_ms (unused) |
|
2026-11-07: PhysicalCandidateKey, PrunedCandidate::group, PhysicalASAPMemoCandidates::query_roots, ErrorGeneratingPhysicalASAPMemoCandidates, SelectionError::PhysicalASAPCandidates
|
PrunedCandidate<PhysicalCandidate>; logical_candidates.memo.query_roots; Stage 2 cannot fail |
2026-11-07: PruneReason::{PerEntityAggregateNotExecutable, TooFewQueryReaders}
|
deleted, no replacement: the planner does no executor-support check (the executor is extended, PR 6) and has no consumer-count policy (the cost model) |
Deleted tests
653 #[test] functions are deleted (643 before 2026-11-07, 10 in the 2026-11-07 round) and 2 are added (PhysicalProperties::new). The per-file table, with what each file covered and which PR restores it, is in this comment. By group:
| Group | Deleted | Restored by |
|---|---|---|
#509 acceptance examples (planner_layering_example{1,2,3,4}) over the reference pricer |
105 | PR 6 (baseline, selection), PR 7 (materialization), PR 8 (accuracy) |
| plan-selection: whole-plan selector, #1144 MILP, result slots, checks | 141 | PR 6 (A)–(F) and decode, PR 7 contexts and (R), PR 8 (K); latency checks: no replacement planned |
| Pass 2 node-level rewrites, rule 9, step 10 per-size groups | 61 | PR 4 (rules 1–7); rule 9 and step 10: no replacement planned |
| Pass 1 private-node population, budget-share sizing, pruned reasons | 34 | PR 3; pruned reasons: #1205; budget split: no replacement planned |
| Stage 2 catalog, timing split, realization; the timing pass | 52 | PR 5, PR 6 |
| Other planner-dependent tests: DAGs or plans from the planner, reuse rules, materialized views, reference pricing (executor, integration, planner, cost, types, frontend, devtools) | 238 + 1 | PR 6; materialized variants PR 7; tools PR 9 |
| θ per node (state bytes, kernels, KLL equality across δ) | 11 | PR 1, PR 6 |
The 2026-11-07 round's 10 deleted tests, per file, are in the interfaces comment.
a_chain_reads_an_instant_vector now checks a zero width (was a zero count); names_before_the_tumbling_window_rename_still_read reads only the chain. Earlier detail (node-tree coverage, tests kept by building the DAG directly) is in the interfaces comment.
Test status
cargo test -p <crate> --all-features --no-fail-fast at the head (7e83db9). Except for the stale fixture, every failure panics at a todo!(); the column names it and the PR that fills it in.
| Crate | Passed | Failed | Ignored | Fails at |
|---|---|---|---|---|
| asap-types | 160 | 2 | 0 | reject the invalid combination of section 2 (PR 1): 2 |
| asap-accuracy | 73 | 0 | 0 | — |
| asap-cost | 31 | 1 | 0 | check θ, which the exported node carries (PR 6): 1 |
| asap-frontend-common | 49 | 0 | 0 | — |
| asap-sql-function-catalog | 10 | 0 | 0 | — |
| metricsql_common | 3 | 0 | 0 | — |
| asap-frontend-promql | 219 | 8 | 1 | Stage 0 over Memo::find_or_add_group (PR 2): 8 |
| asap-frontend-metricsql | 18 | 0 | 0 | — |
| asap-frontend-sql | 174 | 1 | 0 | Stage 0 over Memo::find_or_add_group (PR 2): 1 |
| asap-executor | 210 | 63 | 1 | check θ, which the exported node carries (PR 6): 32; the Hydra grid's sizes are θ, which the exported node carries (PR 6): 13; the view's coverage from its memo group's GroupProperties::aggregate_coverage (PR 6): 8; the weighted-frequency θ, which the exported node carries (PR 6): 7; check the weighted-frequency θ, which the exported node carries (PR 6): 2; Stage 0 over Memo::find_or_add_group (PR 2): 1 |
| asap-planner | 9 | 18 | 1 | Stage 0 over Memo::find_or_add_group (PR 2): 18 |
| asap-devtools | 38 | 58 | 3 | Stage 0 over Memo::find_or_add_group (PR 2): 56; an asap_planner subprocess that stops at a todo!(): 2 |
| asap-integration-tests | 83 | 87 | 4 | Stage 0 over Memo::find_or_add_group (PR 2): 83; the view's coverage from its memo group's GroupProperties::aggregate_coverage (PR 6): 3; the stale fixture (old wire format; PR 9): 1 |
| asap-logical-asap-optimization | 27 | 53 | 0 | Stage 0 over Memo::find_or_add_group (PR 2): 51; the guarantee of θ from the candidate's LogicalProperties::parameters (PR 4): 1; the distinct-count guarantee of θ from LogicalProperties::parameters (PR 4): 1 |
| asap-physical-asap-optimization | 0 | 0 | 0 | — |
| asap-plan-selection | 3 | 0 | 0 | — |
| Total | 1107 | 291 | 10 |
- Stale fixture.
materialized_views_equal_the_stored_outputs_main_listedreads a checked-in plan file in the old wire format (Sketchwith θ). PR 9 re-baselines it with the snapshots. - 2026-11-08 commit (7e83db9). Interval examples and tests use left-open, right-closed ends;
ParametersOfAnotherFamilydeleted. Compared by test name withfacaa92f8(backupbackup/1207-interfaces-facaa92f8), no test changed result.
Verification
cargo check --workspace --all-targets --all-features: clean, no warnings.cargo clippy --workspace --all-targets --all-features -- -D warnings: clean. The only allowances are the commented#[allow(dead_code)]markers listed under "New abstractions".cargo fmt --check: clean.- Tests per crate: see "Test status".
- No snapshot or plan comparison applies: planning panics at Stage 0 until PR 2.
…e trait, MemoQuery, evaluation grids of groups (#1207 PR 0) 2026-11-09 decisions: - theoretical_guarantee and AccuracyEvidence::measured_guarantee take the failure probability and return the error bound that holds at it (None when θ cannot fail that rarely); bodies stay todo!() for PR 4. - AccuracyEvidence is a trait; NoMeasuredGuarantees has no measurement. PlanningModels::evidence and Pass 2 take &dyn AccuracyEvidence. - The memo holds its queries in one Vec<MemoQuery> { root, accuracy_target, values_per_evaluation }; add_query(query). - SupportedSummary::family is summary_type. - satisfies is deleted with metric_meets, scope_meets, native_target and AccuracyScope::covers, which only it used; target_limit is the comparison limit with the tolerance. - PhysicalCandidate::delivers_on(&PhysicalASAPMemoCandidates, context). - GroupProperties::depends_on_evaluation_time (derived in PR 2) and LogicalASAPMemoCandidates::evaluation_grid_of_group (filled in PR 4).
fb0a46a to
e9d9f5a
Compare
… the tests that use it (#1207 PR 0) pass1/composition.rs composed one whole-workload candidate DAG from one per-computation candidate per target, for tests only; planning adds every way to compute to the memo instead. It and the 74 tests and fixture helpers that compose through it are deleted (no interim code for PR 3).
e9d9f5a to
2ed6ec0
Compare
|
#1207 PR 0, 2026-11-09 (P11): the 74 tests deleted with
|
Why
#1207's order is interfaces first: this PR defines every data structure and interface of the concepts doc
memo-groups-properties-requirements.md(branchdocs/memo-group-model, 3e4fb10), withtodo!()bodies where the behavior lands in a later PR, and deletes the interfaces they replace. Each later PR fills in one stage behind these interfaces, so no PR needs an adapter between an old interface and a new one.The 2026-11-08 decisions changed four things the earlier rounds had built differently:
FieldDataTypevariant, an algorithm is one structure within it (SketchAlgorithm, Hydra included), a summary type is the two together (oneFieldDataTypevalue), and θ is the structure's sizes. The code called the summary type "family" and carried Hydra as a separate grouping axis.AccuracyModel, the post-solve check and the "accept regardless" mode go.PhysicalASAPDAGdirectly; the selected-candidate DAG, the realizer and the plan wrappers go.What
All decisions below are dated 2026-11-08 or 2026-11-09 and approved; nothing is pending.
2026-11-09, second round (P11)
Approved 2026-11-09 and applied in 2ed6ec0 (a new commit on be6ea1d):
pass1/composition.rs(test support only: one whole-workload candidate DAG from one per-computation candidate per target) is deleted with every test fixture that composes through it:test_support::{compose_logical_candidate, cms_summary, weighted, estimate_over}, integration-tests'planning::{explicit_local_candidates, explicit_summary_candidate}, the PromQL tests'logical_candidates/logical_plan/local_computation_candidates, the executor tests'computation_candidates, and the 74 tests that use them (see "Deleted tests"). No interim code is left for PR 3, which re-creates Pass 1 coverage over the memo. The module doc's "Whole-workload candidates" section goes with it.2026-11-09 round
Approved 2026-11-09 and applied in be6ea1d:
theoretical_guarantee(summary_type, θ, statistic, failure_probability)andAccuracyEvidence::measured_guarantee(…, failure_probability)return the error bound that holds at that δ, orNonewhen θ cannot fail that rarely. The bound is anAccuracyGuaranteewhose failure probability is the δ asked for (P18). Bodies staytodo!()(PR 4).AccuracyEvidenceis a trait with one method;NoMeasuredGuaranteesreturnsNonealways and isPlanningModels::builtin()'s evidence.PlanningModels::evidence,with_evidence, Pass 2's entries and α^log take&dyn AccuracyEvidence.Vec<MemoQuery>.Memo::queries: Vec<MemoQuery>,MemoQuery { root, accuracy_target, values_per_evaluation },Memo::add_query(query); replacesquery_rootsand the two parallel vectors.root_groupsandqueries_rooted_attake&[MemoQuery].memo_of_queries(roots, accuracy_targets, data)keeps its signature: it builds theMemoQuerys (PR 2).SupportedSummary::summary_type(wasfamily; JSON fieldsummary_type);summary_supported's argument too; the viewer reads it.satisfiesdeleted, withmetric_meets,scope_meets,native_targetandAccuracyScope::covers, which only it used.pub fn target_limit(limit) -> f64is the one comparison limit with the tolerance (limit · (1 + 1e-9) + f64::EPSILON).PhysicalCandidate::delivers_on(&self, candidates: &PhysicalASAPMemoCandidates, context)(was(memo, context)), so aMaterialized(v)context can read v (its branch istodo!()until PR 7).GroupProperties::depends_on_evaluation_time: bool(derived in PR 2,todo!()besidecolumn_sign) andLogicalASAPMemoCandidates::evaluation_grid_of_group: Vec<Option<EvaluationGrid>>(Pass 2 fills it, PR 4): X =sum_over_time(m[5m])read every 60 s and every 120 s has grid 60 s @ 0, so a stored X delivers on 60 s and 120 s grids, not on 30 s. PR 1 implementsdelivers_onover them.library-api,end-to-end-accuracy-guarantees,accuracy-models,evidence-dependent-candidates) and the concepts doc (3e4fb10) show the new signatures.2026-11-08 round (as updated)
FieldDataType::Sketch(SketchAlgorithm)(wasSketch(SketchAlgorithm, GroupingStrategy)): a Hydra is its own algorithm (HydraKll,HydraCms,HydraCountSketch, …).SketchKind,GroupingStrategy,HydraKindand thegroupingfield ofSummaryAggandSummaryTumblingWindowChainare deleted. The operators'familyfield issummary_type(alsoSummaryJoin's).FamilyCapabilityisSummaryTypeCapability(modulesummary_type_capability); itsof_per_instance_sketchis gone (of(&Sketch(a))covers it);shares_one_state_across_groupsnames the Hydra test thatmerges_groupsand the executor share.parameter_list(algorithm).SketchParamsderivesPartialEq; KLLfailure_probability, UnivMondistinct_failure_probability, HLL and KMVrelative_errorand the hand-written equality are deleted, with the δ-resizing (with_failure_probability, three copies) that wrote them.ValueSign { NonNegative, NonPositive, Unknown }andGroupProperties::column_sign(derivationtodo!(), PR 2). A source declares its columns' signs:SourceFacts::column_signs: BTreeMap<String, ValueSign>replacesvalues_non_negative: bool(JSON"column_signs": {"value": "non_negative"}).LogicalRequirements { required_accuracy: Option<RequiredAccuracy> },RequiredAccuracy { metric, max_error, failure_probability }; the parameter floor andLogicalRequirements::neware deleted.RootPruneReason::BelowRequiredAccuracy { consumer, input, required }(wasBelowAccuracyFloorRequirement { floor }).Memo::queries: Vec<MemoQuery>,MemoQuery { root, accuracy_target: Option<AccuracyTarget>, values_per_evaluation: Option<u64> }, added byMemo::add_query(query)(2026-11-09). Stage 0 ismemo_of_queries(roots, accuracy_targets, data): it computesignore_accuracy_guaranteesthe facade passes no target.AggIntent::{Count, Quantile, TopK, Cardinality, FrequencyL2, FrequencyEntropy}loseaccuracy(Countis a unit variant). The front ends take none:lower_sql(query, catalog),lower_sql_dialect(query, catalog, dialect),lower_metricsql_query(query, sample_interval), the PromQL lowerer.trait AccuracyEvidence { measured_guarantee(summary_type, θ, statistic, failure_probability) }withNoMeasuredGuarantees;theoretical_guarantee(summary_type, θ, statistic, failure_probability)(waslocal_guarantee(family, query); bodytodo!(), PR 4); both return the error bound at that δ (2026-11-09).producer_satisfies_logical_requirements(memo, producer, consumer, input_index_of_consumer, evidence: &dyn AccuracyEvidence)moves here fromasap-types(it needs the guarantee; bodytodo!(), PR 4).satisfiesis deleted;target_limitis the comparison limit (2026-11-09). Deleted: theAccuracyModeltrait (satisfies,accepts_output_regardless_of_guarantee,may_accept_regardless_of_guarantee,metric_can_meet),BuiltinAccuracyModel,IgnoreAccuracyGuarantees,AccuracyEvidenceProvider(operand_domain,output_guarantee,may_provide_evidence),NoAccuracyEvidence,OperandDomain,sizing_target,accuracy_budget,DEFAULT_DELTA.pass1_memo(memo, supported_summaries)(loses targets, model, evidence, data).without_summaries_no_query_can_useandoutput_can_meet_targetare deleted (Pass 2's dead-candidate elimination replaces them), and with themPerComputationCandidateError::InvalidAccuracy. The whole-workload composition they used (compose_logical_candidateand its helpers) is now only test support: it moves topass1/composition.rsbehindtest-support; PR 3 deletes it.generate_logical_asap_memo_candidates(pass1, entries, evidence: &dyn AccuracyEvidence).ExecutionContext::{Evaluation { grid: EvaluationGrid }, IngestionPipeline(RefreshWindow), Materialized(PhysicalCandidateId)},EvaluationGrid { period_ms, phase_ms };RefreshWindowIddeleted.producer_satisfies_physical_requirements(producer, consumer_timing: ExecutionTiming)(was(producer, consumer: &PhysicalCandidate)).PruneReason::CountMinWeightsNotProvenNonNegative(L8a; Stage 2 produces it in PR 5), andRejectionCause::CountMinWeightsNotProvenNonNegative, which nothing built, is deleted.PlanOutput { plan: PhysicalASAPDAG };PhysicalASAPDAGNode::selected_physical_candidate: Option<PhysicalCandidateId>(#[serde(skip)], not in the plan file);select_plan(physical, entries, data, cost_inputs) -> Result<(PhysicalASAPDAG, Selection), ErrorSelectingPlan>(bodytodo!(), PR 6);Selection { accepted_accuracy_per_query, pruned_candidates: Vec<PrunedCandidate<PhysicalCandidate>>, milp_variables, milp_constraints, milp_construction_ms, milp_solve_ms };AcceptedAccuracy { accepted_metric, sketches }(losestarget_metricand the "accepted regardless" fields).PlanningRun::selectionholds(PhysicalASAPDAG, Selection). The pass contract checks the DAG (OptimizeError::InvalidPlan) and that its roots and scalar roots answer every entry.RealizedPlan,PhysicalWorkloadPlan,DagOfSelectedCandidates,SelectedEdge,SelectedNodeId,AssignmentRealizer,Stage2AssignmentRealizer,PhysicalCandidateError,AccuracyCheckInputs,SearchReport,SearchStatus,InexactObjective,relative_gap,ErrorSelectingPlan::{RealizerRejected, ObjectiveDiffersFromPlanCost, SelectedPlanFailedCheck},SelectionError::Realization,PlanningRun::{selected_physical_plan, selected_plan},PlanningReport::search_report,QueryPlan,PlanOutput::{plans, scalar_roots, scalar_plans, selection, tumbling_window_retention_ms},PlanCost::cost_unitandCOST_UNIT,Selection::{selected_dag, nodes_with_proven_non_negative_weights, cost, search_report}and with the last one the weight-proof application (with_confirmed_weights(_at),confirmed_weights,WeightDomain::confirmed).ScanWindow(+union),PhysicalASAPDAG::{scan_window, scan_windows, view_window, view_windows},time_range_windowandoffset_belowleaveasap-types. AScanis an ordinary node; its retention is itsretention_ms(asap-cost, a query-timeScan's read and retention aretodo!()(PR 7), andplan_costlosesmaterialized_chains(materialized_tumbling_window_chains,trailing_windowandwidest_window_read_msdeleted). The installer'sset_retention,raw_data_retentionandrefresh_windowsaretodo!()(PR 6). The executor still needs a window per data-store read: the same code is now executor-private (executor/src/sources/window.rs) until PR 6 reads the bounds offTimeRange/TimeShiftlocally.INSTALLABLE_PLAN_WIRE_VERSIONis 14 (was 13): ops carrysummary_type, a sketch state is{"Sketch": "Kll"}, nogrouping;selected_physical_candidateis not written. The checked-in version-10 fixture's DAG is migrated; the workload fixtures and the viewer readcolumn_signs.installable_plan_of(plan: &PhysicalASAPDAG, cost: Option<&PlanCost>, entries);SelectedPlanReport { cost_per_second, predicted_usage }(losessearch_report,queries_accepted_without_guarantee);InstallablePlanError::TumblingWindowRetentionOfUnknownChaindeleted.NodeCostSourceandNodeCost::cost_source(wereNodePriceSource,price_source);PlanRanking::Cost(wasPrice);summary_type_name(wasfamily_name).PruneReason,PhysicalProperties,OriginOfCandidateForMemoGroup,RootPruneReason(and the ids andRequiredAccuracythey hold).SelectionVariables::t: Vec<Option<Variable>>(only on strongly connected components);AccuracyVariables::{error_bound, error_bound_of_input}per metric (asap_plannerdocument has noL1/P1candidate lists any more (they were the realized plan);stage3_plan_selectionholdsplanand the newSelectionfields.ShownLogicalCandidate,logical_candidates(),physical_candidates(),candidate_labelsandphysical_labelare deleted (decided R10).sketch_coverageloses--max-error. The fuzz planner's plan read and the peak-memory warning aretodo!()(PR 9, PR 6).asap-primitive-schema§1 and §6, the architecture, library-API, mapping-contracts, extension, input-output and accuracy docs use the new names; no deleted identifier remains outside historical proposals.Decided: abstractions and choices not in the doc (were pending)
Nothing is pending. All rows below were decided on 2026-11-09. Approved 2026-11-09 and removed from this table: P1, P2 (trait + bound at δ), P4 (
MemoQuery), P7 (column_signs; the workload JSON in other repositories changes in one PR), P10 (ScanWindowexecutor-private until PR 6), P14 (summary_type), P15 (WeightDomaindeleted in PR 5, Count-Min readscolumn_sign), P16 (satisfiesdeleted).producer_satisfies_logical_requirementsinasap-accuracy(wasasap-types)asap-typescannot depend onasap-accuracy.asap-accuracy)memo_of_queries(roots, accuracy_targets, data)series_read,values_per_result, kept for it) and reads the declared signs fromdata.find_or_add_groupreads the sources' declared signs for aScan'scolumn_signScan), so the signs must be reachable from the memo.Memostores the declared signs (Memo::new(declared_column_signs), private field; PR 2)SummaryTypeCapability::shares_one_state_across_groupsmerges_groupsand the executor's kernel check (it was duplicated). The doc's Hydra capability (subpopulation readout) comes when Hydra is in scope.PhysicalASAPDAG::query_roots()(test support) and integration-tests'plan_cost_per_secondselected_planroots orSelection::costread the plan instead.pass1/composition.rsbehindtest-supportOptimizeError::InvalidPlan { pass, error }installable_plan_of(plan, cost: Option<&PlanCost>, entries);SelectedPlanReport { cost_per_second, predicted_usage }MemoQuery::accuracy_target: Option<AccuracyTarget>accuracy_target: AccuracyTarget;Noneis howignore_accuracy_guaranteesand the doc's "no accuracy target reaches it" are carried today, and no target value means "none".OptionAccuracyGuarantee(itsfailure_probabilityis the δ asked for)BoundExpr, which may hold an unknown statistic) and provenance. No new type.Not in PR 0, so nothing to do here (later PRs must not add them):
family_of_operator/summary_type_of_operatorandsketch_algorithm_of(PR 1 adds them; withSketch(SketchAlgorithm)the latter is a match);floor_for_cap,strongest,weakest,dominates,combine(PR 1/PR 4);ExportedNodeAssignment,physical_asap_dag_with_assignments(PR 6);AccuracyCheckInputs::query_roots(the struct is deleted).Key code interfaces
Files at 7708f3668. Only the interfaces this round changes; the rest are in the old body.
types/src/ir/schema/mod.rs,sketch.rs,summary_type_capability.rstypes/src/ir/operator/asap.rs,agg_intent.rstypes/src/stages/memo.rs,candidates.rs,logical_memo_candidates.rs,pruning.rstypes/src/stages/physical/,types/src/ir/physical_export.rs,types/src/workload/mod.rsaccuracy/src/model.rs,estimators/mod.rs,lib.rsStages:
stage0.rs,pass1/logical_memo_candidates.rs,pass2/logical_memo_candidates.rs,plan-selectionFacade and tools:
planner/src/pass.rs,run.rs,installable.rs,cost, front endsUsage (end to end; the bodies are
todo!()until the PRs named, so this is the call flow, not a run):Major algorithm changes
None: the bodies of the new interfaces are
todo!(). One behavior is removed: Pass 1 no longer prunes summaries no query could use for accuracy (without_summaries_no_query_can_use); Pass 2's dead-candidate elimination (PR 4) replaces it. The executor's window derivation is unchanged code in a new place (P10).Before this PR
A sketch state's type was
Sketch(SketchAlgorithm, GroupingStrategy), with Hydra as a grouping axis; an operator's summary type was calledfamily; θ recorded the error and δ it was sized for, so two KLLs of onekwere equal only through a hand-written equality. A consumer input carried a θ floor; measures carried the query's accuracy; Pass 1 pruned with a pluggableAccuracyModel; Stage 3 decoded aDagOfSelectedCandidates, had Stage 2 realize it into aRealizedPlan, checked accuracy after the solve, and the pass returned per-query operator trees (PlanOutput::plans) beside the selection report.After this PR
A summary type is one$n_q$ ; measures carry no accuracy. Stage 3 returns the plan, a
FieldDataTypevalue (Sketch(Kll),Sketch(HydraCms)); θ is structure only. A readout's input carries aRequiredAccuracy, and α^log compares the producer's measured-or-theoretical guarantee with it (PR 4). Queries enter the memo with their target andPhysicalASAPDAGwhose nodes name their physical candidate, beside itsSelectionreport;PlanOutputholds only the plan.Deleted, and what replaces each (this round)
SketchKind,GroupingStrategy,HydraKind, ops'grouping,PhysicalASAPDAGValidationError::SummaryGroupingMismatchFieldDataType::Sketch(SketchAlgorithm)with theHydra*algorithmsFamilyCapability,of_per_instance_sketch,family_capability.rsSummaryTypeCapability::of(summary_type_capability.rs)PartialEq,with_failure_probability(×3), KMV/HLL sized εLogicalRequirements { parameter_floor_for_required_accuracy },LogicalRequirements::new,RootPruneReason::BelowAccuracyFloorRequirementrequired_accuracy: Option<RequiredAccuracy>,BelowRequiredAccuracyasap_types::stages::producer_satisfies_logical_requirements(producer, requirement)asap_accuracy::producer_satisfies_logical_requirements(memo, producer, consumer, input, evidence)AccuracyModel,BuiltinAccuracyModel,IgnoreAccuracyGuarantees,AccuracyEvidenceProvider,NoAccuracyEvidence,OperandDomain,AccuracyCheckInputs,PlanningModels::{accuracy, accuracy_check_inputs, with_accuracy}AccuracyEvidence(Pass 2); ignore mode = no targets before Pass 2local_guaranteetheoretical_guaranteepub struct AccuracyEvidence {}(2026-11-09)trait AccuracyEvidence+NoMeasuredGuaranteesMemo::{query_roots, query_accuracy_targets, query_values_per_evaluation}(2026-11-09)Memo::queries: Vec<MemoQuery>satisfies,metric_meets,scope_meets,native_target,AccuracyScope::covers(2026-11-09)target_limit(the comparison with the tolerance); the metric and scope rules: deleted, no replacement (α^log compares one metric at δ_r, PR 4)SupportedSummary::family(2026-11-09)SupportedSummary::summary_typewithout_summaries_no_query_can_use,output_can_meet_target,sizing_target,accuracy_budget,DEFAULT_DELTA,PerComputationCandidateError::InvalidAccuracy)AggIntentmeasures'accuracy; front ends' accuracy parameterMemo::query_accuracy_targets)SourceFacts::values_non_negativeSourceFacts::column_signs(P7)ExecutionContext::Evaluation { recurrence, range_evaluation },RefreshWindowIdEvaluation { grid: EvaluationGrid },IngestionPipeline(RefreshWindow)RealizedPlan,PhysicalWorkloadPlan,DagOfSelectedCandidates,SelectedEdge,SelectedNodeId,AssignmentRealizer,Stage2AssignmentRealizer,PhysicalCandidateError,SelectionError::Realization,ErrorSelectingPlan::RealizerRejectedPhysicalASAPDAG;selected_physical_candidateSearchReport,SearchStatus,InexactObjective,relative_gap,ObjectiveDiffersFromPlanCost,SelectedPlanFailedCheck,Selection::{selected_dag, nodes_with_proven_non_negative_weights, cost, search_report},PlanCost::cost_unit,COST_UNIT,AcceptedAccuracy::{target_metric, accepted_regardless_of_guarantee, guarantee_check_rejection},RejectionCause::CountMinWeightsNotProvenNonNegativeSelection::{milp_*};plan_cost(&plan); (K) has every bound;PruneReason::CountMinWeightsNotProvenNonNegativewith_confirmed_weights(_at),confirmed_weights,WeightDomain::confirmedcolumn_sign, PR 4–5)PlanOutput::{plans, scalar_roots, scalar_plans, selection, tumbling_window_retention_ms},QueryPlan,PlanningRun::{selected_plan, selected_physical_plan},PlanningReport::search_report,InstallablePlanError::TumblingWindowRetentionOfUnknownChainPlanOutput { plan },PlanningRun::selection; retention is on the nodesScanWindow,union,scan_window(s),view_window(s),time_range_window,offset_below(types);materialized_tumbling_window_chains,trailing_window,widest_window_read_ms,node_usage'sscan_window(cost)retention_ms(NodePriceSource,price_source,PlanRanking::Price,family_nameNodeCostSource,cost_source,PlanRanking::Cost,summary_type_namepass1/composition.rs(compose_logical_candidateand its helpers),test_support::{cms_summary, weighted, estimate_over},planning::{explicit_local_candidates, explicit_summary_candidate}, the PromQL tests'logical_candidates/logical_plan, the executor tests'computation_candidates(2026-11-09, P11)ShownLogicalCandidate,logical_candidates(),physical_candidates(),candidate_labels,physical_labeland its helpers,usage_json,sketch_coverage --max-errorplanDeleted tests (this round)
24 test functions deleted, 1 added (
contract_rejects_a_scalar_root_keyed_beyond_the_entries), 3 renamed:accuracy/src/estimators/mod.rsenlarging_meets_the_failure_probability_at_no_larger_error,a_distinct_count_univmon_grows_its_heap,local_guarantee_needs_a_statistic_the_family_answerslocal_guaranteedeletedaccuracy/src/estimators/kll.rssizing_meets_a_failure_probability_below_one_percent,sizing_below_one_percent_names_a_distinct_contract,an_invalid_failure_probability_gives_no_guaranteefailure_probabilitydeleted; PR 4 evaluates the guarantee at δ_raccuracy/src/estimators/cardinality.rskmv_sized_for_a_failure_probability_meets_it,kmv_k_follows_the_failure_probabilitytypes/src/ir/schema/sketch.rssketch_kind_keeps_a_valid_algorithm_and_params_pair,sketch_kind_rejects_params_from_another_algorithm,grouping_strategy_default_is_per_subpopulation_instanceSketchKind,GroupingStrategydeletedtypes/tests/physical_export.rsa_scan_window_is_the_union_of_its_readers_windows,a_view_window_is_the_union_of_the_windows_below_it,a_scan_read_under_an_at_anchor_has_an_unbounded_windowlogical-asap-optimization/src/pass1/per_computation_candidates.rscount_min_estimate_over_raw_data_samples_is_gated_on_the_metriclocal_guaranteewithout θ; the gating is replaced bycolumn_sign(PR 4)planner/src/pass.rscontract_rejects_a_scalar_reference_without_its_plan,contract_checks_scalar_plan_order_entry_and_countscalar_plansdeleteddevtools/src/candidate_labels.rsconnected_labels_describe_selected_states,connected_labels_name_actual_shared_state,connected_labels_name_exact_resultsfrontend-promql/tests/promql_lowering.rs,frontend-sql/tests/sql_lowering.rsaccuracy_target_flows_into_quantile_intent,median_threads_the_accuracy_targetintegration-tests/tests/promql_numeric_regressions.rslogical_candidates_fails_on_a_candidate_that_does_not_composeShownLogicalCandidatedeletedintegration-tests/tests/joint_search_equivalence.rscustom_accuracy_policy_is_respectedAccuracyModeldeletedRenamed, with the accuracy assertion dropped:
count_maps_to_count(wascount_maps_to_count_and_inherits_accuracy),distinct_over_time_lowers_to_cardinality_over_nested_windows,asap_input_is_rejected(its invalid-accuracy half deleted withInvalidAccuracy). Tests that readPlanOutput::plans,selected_planorSelection::costnow readplan.query_roots()andplan_cost_per_second(P9).2026-11-09, second round (P11)
74 tests deleted, all composing whole-workload candidates through
pass1/composition.rs(names: comment). Files left without a test are deleted with their registrations:frontend-promql/tests/univmon_candidates.rs,frontend-promql/tests/observability/metrics_observability.rs(test target),integration-tests/tests/{exact_composition, promql_numeric_regressions, sketch_rollup, stage1_logical_asap_optimization}.rs. On be6ea1d they were 70 failed (most at Stage 0'stodo!()), 3 passed, 1 ignored.frontend-metricsql/tests/external_corpus.rsfrontend-promql/tests/count_planning.rsfrontend-promql/tests/observability/metrics_observability.rsfrontend-promql/tests/observability/promql_corpus.rsfrontend-promql/tests/univmon_candidates.rsintegration-tests/tests/exact_composition.rsintegration-tests/tests/per_computation_candidates_sql.rsintegration-tests/tests/promql_numeric_regressions.rsintegration-tests/tests/sketch_rollup.rsintegration-tests/tests/sql_to_post_asap.rsintegration-tests/tests/stage1_logical_asap_optimization.rslogical-asap-optimization/src/pass1/per_computation_candidates.rslogical-asap-optimization/src/pass1/per_computation_candidates_tests.rslogical-asap-optimization/src/pass1/realization.rs2026-11-09 round
8 tests deleted, 1 added (
accuracytests::target_limit_admits_only_floating_point_noise):accuracy/src/lib.rssatisfies_is_fail_closed_on_unknowns_and_exact,satisfies_requires_the_targets_metric,a_target_without_delta_ignores_the_failure_probability,per_value_guarantees_meet_whole_result_targets_over_g_values,a_record_share_target_is_met_over_one_over_share_groups,scope_matters_only_with_a_failure_probability,a_target_without_a_metric_accepts_any_metricsatisfiesdeletedtypes/src/ir/properties/accuracy.rsa_wider_scope_covers_a_narrower_oneAccuracyScope::coversdeleted (onlysatisfiesused it)The estimator tests that called
satisfies(CMS, Count Sketch, HLL, KLL, UnivMon, composition) now check the evaluated error and δ againsttarget_limitthrough a test-onlymeets; their metric is asserted beside it, as before.Self-review: over-abstraction, duplication, dead code
2026-11-09, second round: the empty
mod stream_norms {}(Pass 1 tests) and two emptymod tests {}(devtoolsplanning_stages_text.rs,candidate_labels.rs) are deleted; no other code lost its last user.2026-11-09 round:
metric_meets,scope_meets,native_target(test-only),AccuracyScope::covers, the staticNO_ACCURACY_EVIDENCE(a unit struct needs none).target_limit; the test-onlymeetscalls it.target_limithas no production caller until PR 4 (α^log) and PR 8 ((K));root_groupshas no caller in PR 0 (Stage 2 and 3 use it, PR 5–6);AccuracyGuarantee::over_scopehas no production caller (PR 4/8 state a whole-result δ);NoMeasuredGuaranteesis the doc's default evidence;depends_on_evaluation_timeis a second unreachabletodo!()insidederive_group_properties(PR 2 derives both).AccuracyGuarantee(P18) instead of a new type.2026-11-08 round:
Found and fixed in this round:
PhysicalASAPDAG::validate'sScalarRootPosition→ the contract counts the roots, then validates. The Hydra algorithm list was inSummaryTypeCapability::merges_groupsand in the executor's kernel check → oneshares_one_state_across_groups(P8).of_per_instance_sketchduplicatedofonce Hydra is an algorithm → deleted.InvalidAccuracyand its mapping;computation_candidates_for_intentreturned aResultthat never failed (now infallible);sources_read;with_confirmed_weights(_at),confirmed_weights,WeightDomain::confirmed;materialized_tumbling_window_chains,trailing_window,widest_window_read_ms;payloads_by_id,offset_below;TumblingWindowRetentionOfUnknownChain;OperandDomain;DEFAULT_DELTA; the label helpers (for_queries,query_names,duration_name,build_label,state_label,dag_readers) andusage_json;parse_max_error; test helperswindow_over,exported_with_scan,series_scan,approximate,error_and_delta,delta_and_error.plan_cost's andeach_node_usage'smaterialized_chains,each_node_usage'sroot_scan_lookback_ms,node_usage'sscan_window, devtools'lower_promql_with_sample_interval's accuracy, and test helpers'accuracy.PerComputationCandidates::{roots, aggregate}behindtest-support; the whole-workload composition is deleted (P11, 2026-11-09).Left, and why:
AccuracyEvidence {}with aNonemethod is a placeholder the doc requires (P1).series_read,values_per_resulthave no caller until Stage 0 computessatisfieshas no production caller (P16).RejectionCause,RejectionandErrorSelectingPlan::NoValidCandidateare never constructed, as before this round; PR 6 decides whether the MILP's infeasibility uses them.#[allow(dead_code)]markers from earlier rounds (θ-based estimators and sizing,milp/model.rs,valid_matrix,GroupStates::Keyed) stay until PRs 4–8.lower_promql(query, accuracy)still takes a target: it fills the workload entry, which no longer reaches the measures.Test status
cargo test -p <crate> --all-features --no-fail-fastat the head (2ed6ec0). Except for the stale fixture, every failure panics at atodo!(); the column names it and the PR that fills it in.Result::unwrap()on anErrvalue: Error("invalid type: sequence, expected string or map", line: 0, column: 0): 1pass1/composition.rs(70 failed, 3 passed, 1 ignored there); no other test changed result.satisfies's 7 inasap-accuracy,a_wider_scope_covers_a_narrower_oneinasap-types) and the addedtarget_limit_admits_only_floating_point_noise(passes). Passed 1088 → 1081, failed 287 → 287.crate_graph: passes.materialized_views_equal_the_stored_outputs_main_listedreads a checked-in plan file in an old wire format; PR 9 re-baselines it with the snapshots.Verification
cargo clippy --workspace --all-targets --all-features -- -D warningsandcargo clippy --workspace -- -D warnings: clean. No new#[allow];derive_group_propertiescarries#[allow(unreachable_code)]until PR 2 derivescolumn_sign.cargo fmt --check: clean.crate_graph(integration tests): passes; no stage crate depends on another.