Skip to content

refactor: define every stage's interfaces from #1207 and delete the ones they replace (#1207 PR 0) - #1226

Open
zzylol wants to merge 19 commits into
mainfrom
refactor/1207-interfaces
Open

zzylol wants to merge 19 commits into
mainfrom
refactor/1207-interfaces

Conversation

@zzylol

@zzylol zzylol commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Merge unit. This is PR 0 of #1207's merge unit (PRs 0–9). The PRs merge into main together, after PR 9. Each one is stacked on the one before it for review. This PR compiles, but planning panics at the first todo!(). Tests that reach a todo!() fail until the PR that fills it in (see "Test status").

2026-11-09 round. This body describes the head be6ea1d: one commit on 7708f36 applying the 2026-11-09 decisions (listed under "2026-11-09 round" below; earlier rounds' text is unchanged). The body as of 7e83db9 (decisions of 2026-11-06 and 2026-11-07, its interface list and its deletion tables) is kept in this comment; the 2026-11-07 detail is in the interfaces comment.

Why

#1207's order is interfaces first: this PR defines every data structure and interface of the concepts doc memo-groups-properties-requirements.md (branch docs/memo-group-model, 3e4fb10), with todo!() bodies where the behavior lands in a later PR, and deletes the interfaces they replace. Each later PR fills in one stage behind these interfaces, so no PR needs an adapter between an old interface and a new one.

The 2026-11-08 decisions changed four things the earlier rounds had built differently:

  • Terminology. A family is a FieldDataType variant, an algorithm is one structure within it (SketchAlgorithm, Hydra included), a summary type is the two together (one FieldDataType value), and θ is the structure's sizes. The code called the summary type "family" and carried Hydra as a separate grouping axis.
  • Accuracy is a requirement on a readout's input, not a θ floor. A build meets a readout when its guarantee, evaluated at the required failure probability, is within the required error; one rule for every algorithm. θ records no error it was sized for.
  • The accuracy source is decided up front (measured guarantee, else the algorithm's formula), so nothing is checked after Stage 3: the pluggable AccuracyModel, the post-solve check and the "accept regardless" mode go.
  • One plan structure. Stage 3 builds the PhysicalASAPDAG directly; the selected-candidate DAG, the realizer and the plan wrappers go.

What

All decisions below are dated 2026-11-08 or 2026-11-09 and approved; nothing is pending.

2026-11-09, second round (P11)

Approved 2026-11-09 and applied in 2ed6ec0 (a new commit on be6ea1d):

  • Pass 1's whole-workload composition is deleted (P11). pass1/composition.rs (test support only: one whole-workload candidate DAG from one per-computation candidate per target) is deleted with every test fixture that composes through it: test_support::{compose_logical_candidate, cms_summary, weighted, estimate_over}, integration-tests' planning::{explicit_local_candidates, explicit_summary_candidate}, the PromQL tests' logical_candidates / logical_plan / local_computation_candidates, the executor tests' computation_candidates, and the 74 tests that use them (see "Deleted tests"). No interim code is left for PR 3, which re-creates Pass 1 coverage over the memo. The module doc's "Whole-workload candidates" section goes with it.
  • All items of the former "Pending" table are decided (below).

2026-11-09 round

Approved 2026-11-09 and applied in be6ea1d:

  • Guarantee at a failure probability. theoretical_guarantee(summary_type, θ, statistic, failure_probability) and AccuracyEvidence::measured_guarantee(…, failure_probability) return the error bound that holds at that δ, or None when θ cannot fail that rarely. The bound is an AccuracyGuarantee whose failure probability is the δ asked for (P18). Bodies stay todo!() (PR 4).
  • AccuracyEvidence is a trait with one method; NoMeasuredGuarantees returns None always and is PlanningModels::builtin()'s evidence. PlanningModels::evidence, with_evidence, Pass 2's entries and α^log take &dyn AccuracyEvidence.
  • One Vec<MemoQuery>. Memo::queries: Vec<MemoQuery>, MemoQuery { root, accuracy_target, values_per_evaluation }, Memo::add_query(query); replaces query_roots and the two parallel vectors. root_groups and queries_rooted_at take &[MemoQuery]. memo_of_queries(roots, accuracy_targets, data) keeps its signature: it builds the MemoQuerys (PR 2).
  • SupportedSummary::summary_type (was family; JSON field summary_type); summary_supported's argument too; the viewer reads it.
  • satisfies deleted, with metric_meets, scope_meets, native_target and AccuracyScope::covers, which only it used. pub fn target_limit(limit) -> f64 is the one comparison limit with the tolerance (limit · (1 + 1e-9) + f64::EPSILON).
  • Alignment of stored results (PR 1's P2, P3). PhysicalCandidate::delivers_on(&self, candidates: &PhysicalASAPMemoCandidates, context) (was (memo, context)), so a Materialized(v) context can read v (its branch is todo!() until PR 7). GroupProperties::depends_on_evaluation_time: bool (derived in PR 2, todo!() beside column_sign) and LogicalASAPMemoCandidates::evaluation_grid_of_group: Vec<Option<EvaluationGrid>> (Pass 2 fills it, PR 4): X = sum_over_time(m[5m]) read every 60 s and every 120 s has grid 60 s @ 0, so a stored X delivers on 60 s and 120 s grids, not on 30 s. PR 1 implements delivers_on over them.
  • Repo docs (library-api, end-to-end-accuracy-guarantees, accuracy-models, evidence-dependent-candidates) and the concepts doc (3e4fb10) show the new signatures.

2026-11-08 round (as updated)

  • Summary types. FieldDataType::Sketch(SketchAlgorithm) (was Sketch(SketchAlgorithm, GroupingStrategy)): a Hydra is its own algorithm (HydraKll, HydraCms, HydraCountSketch, …). SketchKind, GroupingStrategy, HydraKind and the grouping field of SummaryAgg and SummaryTumblingWindowChain are deleted. The operators' family field is summary_type (also SummaryJoin's). FamilyCapability is SummaryTypeCapability (module summary_type_capability); its of_per_instance_sketch is gone (of(&Sketch(a)) covers it); shares_one_state_across_groups names the Hydra test that merges_groups and the executor share. parameter_list(algorithm).
  • θ is structure only. SketchParams derives PartialEq; KLL failure_probability, UnivMon distinct_failure_probability, HLL and KMV relative_error and the hand-written equality are deleted, with the δ-resizing (with_failure_probability, three copies) that wrote them.
  • Value sign. ValueSign { NonNegative, NonPositive, Unknown } and GroupProperties::column_sign (derivation todo!(), PR 2). A source declares its columns' signs: SourceFacts::column_signs: BTreeMap<String, ValueSign> replaces values_non_negative: bool (JSON "column_signs": {"value": "non_negative"}).
  • Required accuracy. LogicalRequirements { required_accuracy: Option<RequiredAccuracy> }, RequiredAccuracy { metric, max_error, failure_probability }; the parameter floor and LogicalRequirements::new are deleted. RootPruneReason::BelowRequiredAccuracy { consumer, input, required } (was BelowAccuracyFloorRequirement { floor }).
  • Queries enter the memo with their target and $n_q$. Memo::queries: Vec<MemoQuery>, MemoQuery { root, accuracy_target: Option<AccuracyTarget>, values_per_evaluation: Option<u64> }, added by Memo::add_query(query) (2026-11-09). Stage 0 is memo_of_queries(roots, accuracy_targets, data): it computes $n_q$ from the data workload, which also declares the sources' signs (PR 2). With ignore_accuracy_guarantees the facade passes no target.
  • Measures carry no accuracy (N1). AggIntent::{Count, Quantile, TopK, Cardinality, FrequencyL2, FrequencyEntropy} lose accuracy (Count is a unit variant). The front ends take none: lower_sql(query, catalog), lower_sql_dialect(query, catalog, dialect), lower_metricsql_query(query, sample_interval), the PromQL lowerer.
  • Accuracy crate. trait AccuracyEvidence { measured_guarantee(summary_type, θ, statistic, failure_probability) } with NoMeasuredGuarantees; theoretical_guarantee(summary_type, θ, statistic, failure_probability) (was local_guarantee(family, query); body todo!(), PR 4); both return the error bound at that δ (2026-11-09). producer_satisfies_logical_requirements(memo, producer, consumer, input_index_of_consumer, evidence: &dyn AccuracyEvidence) moves here from asap-types (it needs the guarantee; body todo!(), PR 4). satisfies is deleted; target_limit is the comparison limit (2026-11-09). Deleted: the AccuracyModel trait (satisfies, accepts_output_regardless_of_guarantee, may_accept_regardless_of_guarantee, metric_can_meet), BuiltinAccuracyModel, IgnoreAccuracyGuarantees, AccuracyEvidenceProvider (operand_domain, output_guarantee, may_provide_evidence), NoAccuracyEvidence, OperandDomain, sizing_target, accuracy_budget, DEFAULT_DELTA.
  • Pass 1 prunes nothing for accuracy. pass1_memo(memo, supported_summaries) (loses targets, model, evidence, data). without_summaries_no_query_can_use and output_can_meet_target are deleted (Pass 2's dead-candidate elimination replaces them), and with them PerComputationCandidateError::InvalidAccuracy. The whole-workload composition they used (compose_logical_candidate and its helpers) is now only test support: it moves to pass1/composition.rs behind test-support; PR 3 deletes it.
  • Pass 2 reads the evidence. generate_logical_asap_memo_candidates(pass1, entries, evidence: &dyn AccuracyEvidence).
  • Execution contexts. ExecutionContext::{Evaluation { grid: EvaluationGrid }, IngestionPipeline(RefreshWindow), Materialized(PhysicalCandidateId)}, EvaluationGrid { period_ms, phase_ms }; RefreshWindowId deleted.
  • Physical compatibility. producer_satisfies_physical_requirements(producer, consumer_timing: ExecutionTiming) (was (producer, consumer: &PhysicalCandidate)). PruneReason::CountMinWeightsNotProvenNonNegative (L8a; Stage 2 produces it in PR 5), and RejectionCause::CountMinWeightsNotProvenNonNegative, which nothing built, is deleted.
  • One plan structure. PlanOutput { plan: PhysicalASAPDAG }; PhysicalASAPDAGNode::selected_physical_candidate: Option<PhysicalCandidateId> (#[serde(skip)], not in the plan file); select_plan(physical, entries, data, cost_inputs) -> Result<(PhysicalASAPDAG, Selection), ErrorSelectingPlan> (body todo!(), PR 6); Selection { accepted_accuracy_per_query, pruned_candidates: Vec<PrunedCandidate<PhysicalCandidate>>, milp_variables, milp_constraints, milp_construction_ms, milp_solve_ms }; AcceptedAccuracy { accepted_metric, sketches } (loses target_metric and the "accepted regardless" fields). PlanningRun::selection holds (PhysicalASAPDAG, Selection). The pass contract checks the DAG (OptimizeError::InvalidPlan) and that its roots and scalar roots answer every entry.
  • Deleted with it: RealizedPlan, PhysicalWorkloadPlan, DagOfSelectedCandidates, SelectedEdge, SelectedNodeId, AssignmentRealizer, Stage2AssignmentRealizer, PhysicalCandidateError, AccuracyCheckInputs, SearchReport, SearchStatus, InexactObjective, relative_gap, ErrorSelectingPlan::{RealizerRejected, ObjectiveDiffersFromPlanCost, SelectedPlanFailedCheck}, SelectionError::Realization, PlanningRun::{selected_physical_plan, selected_plan}, PlanningReport::search_report, QueryPlan, PlanOutput::{plans, scalar_roots, scalar_plans, selection, tumbling_window_retention_ms}, PlanCost::cost_unit and COST_UNIT, Selection::{selected_dag, nodes_with_proven_non_negative_weights, cost, search_report} and with the last one the weight-proof application (with_confirmed_weights(_at), confirmed_weights, WeightDomain::confirmed).
  • Scan windows (ScanWindow deletion). ScanWindow (+ union), PhysicalASAPDAG::{scan_window, scan_windows, view_window, view_windows}, time_range_window and offset_below leave asap-types. A Scan is an ordinary node; its retention is its retention_ms ($h_v$). In asap-cost, a query-time Scan's read and retention are todo!() (PR 7), and plan_cost loses materialized_chains (materialized_tumbling_window_chains, trailing_window and widest_window_read_ms deleted). The installer's set_retention, raw_data_retention and refresh_windows are todo!() (PR 6). The executor still needs a window per data-store read: the same code is now executor-private (executor/src/sources/window.rs) until PR 6 reads the bounds off TimeRange/TimeShift locally.
  • Plan file. INSTALLABLE_PLAN_WIRE_VERSION is 14 (was 13): ops carry summary_type, a sketch state is {"Sketch": "Kll"}, no grouping; selected_physical_candidate is not written. The checked-in version-10 fixture's DAG is migrated; the workload fixtures and the viewer read column_signs.
  • Installable plan. installable_plan_of(plan: &PhysicalASAPDAG, cost: Option<&PlanCost>, entries); SelectedPlanReport { cost_per_second, predicted_usage } (loses search_report, queries_accepted_without_guarantee); InstallablePlanError::TumblingWindowRetentionOfUnknownChain deleted.
  • Cost names (M8). NodeCostSource and NodeCost::cost_source (were NodePriceSource, price_source); PlanRanking::Cost (was Price); summary_type_name (was family_name).
  • Serialize (R4) on PruneReason, PhysicalProperties, OriginOfCandidateForMemoGroup, RootPruneReason (and the ids and RequiredAccuracy they hold).
  • Stage 3 MILP variables (doc §3–4). SelectionVariables::t: Vec<Option<Variable>> (only on strongly connected components); AccuracyVariables::{error_bound, error_bound_of_input} per metric ($E_{c,m}$, $F_{c,i,m}$).
  • Tools. The asap_planner document has no L1/P1 candidate lists any more (they were the realized plan); stage3_plan_selection holds plan and the new Selection fields. ShownLogicalCandidate, logical_candidates(), physical_candidates(), candidate_labels and physical_label are deleted (decided R10). sketch_coverage loses --max-error. The fuzz planner's plan read and the peak-memory warning are todo!() (PR 9, PR 6).
  • Docs. asap-primitive-schema §1 and §6, the architecture, library-API, mapping-contracts, extension, input-output and accuracy docs use the new names; no deleted identifier remains outside historical proposals.

Decided: abstractions and choices not in the doc (were pending)

Nothing is pending. All rows below were decided on 2026-11-09. Approved 2026-11-09 and removed from this table: P1, P2 (trait + bound at δ), P4 (MemoQuery), P7 (column_signs; the workload JSON in other repositories changes in one PR), P10 (ScanWindow executor-private until PR 6), P14 (summary_type), P15 (WeightDomain deleted in PR 5, Count-Min reads column_sign), P16 (satisfies deleted).

# Item Why Decision
P3 producer_satisfies_logical_requirements in asap-accuracy (was asap-types) It needs the guarantee, and asap-types cannot depend on asap-accuracy. approved 2026-11-09 (asap-accuracy)
P5 memo_of_queries(roots, accuracy_targets, data) Stage 0 computes $n_q$ (series_read, values_per_result, kept for it) and reads the declared signs from data. approved 2026-11-09
P6 How find_or_add_group reads the sources' declared signs for a Scan's column_sign Rules add groups too (a rollup's finest common Scan), so the signs must be reachable from the memo. approved 2026-11-09: the Memo stores the declared signs (Memo::new(declared_column_signs), private field; PR 2)
P8 SummaryTypeCapability::shares_one_state_across_groups One Hydra list, read by merges_groups and the executor's kernel check (it was duplicated). The doc's Hydra capability (subpopulation readout) comes when Hydra is in scope. approved 2026-11-09
P9 PhysicalASAPDAG::query_roots() (test support) and integration-tests' plan_cost_per_second Tests that inspected selected_plan roots or Selection::cost read the plan instead. approved 2026-11-09
P11 pass1/composition.rs behind test-support Only tests compose whole-workload candidates now; PR 3 deletes it. 2026-11-09: deleted in this PR instead (no interim code)
P12 OptimizeError::InvalidPlan { pass, error } The pass contract validates the returned DAG. approved 2026-11-09
P13 installable_plan_of(plan, cost: Option<&PlanCost>, entries); SelectedPlanReport { cost_per_second, predicted_usage } The report is the plan's cost, not the selection's. approved 2026-11-09
P17 MemoQuery::accuracy_target: Option<AccuracyTarget> The approval's sketch reads accuracy_target: AccuracyTarget; None is how ignore_accuracy_guarantees and the doc's "no accuracy target reaches it" are carried today, and no target value means "none". approved 2026-11-09: keep Option
P18 The bound at δ is an AccuracyGuarantee (its failure_probability is the δ asked for) An existing type: metric, error (BoundExpr, which may hold an unknown statistic) and provenance. No new type. approved 2026-11-09

Not in PR 0, so nothing to do here (later PRs must not add them): family_of_operator/summary_type_of_operator and sketch_algorithm_of (PR 1 adds them; with Sketch(SketchAlgorithm) the latter is a match); floor_for_cap, strongest, weakest, dominates, combine (PR 1/PR 4); ExportedNodeAssignment, physical_asap_dag_with_assignments (PR 6); AccuracyCheckInputs::query_roots (the struct is deleted).

Key code interfaces

Files at 7708f3668. Only the interfaces this round changes; the rest are in the old body.

types/src/ir/schema/mod.rs, sketch.rs, summary_type_capability.rs

/// A column's type: a plain value or a summary type (family + algorithm). Stays.
pub enum FieldDataType {
    Plain(DataType),                                  // unchanged
    ExactAggregate(ExactKind, ExactParams),           // unchanged
    /// Sketch state of one algorithm, Hydra included; θ is not in the type.
    Sketch(SketchAlgorithm),                          // was Sketch(SketchAlgorithm, GroupingStrategy)
    SummarySample(SamplingKind, SamplingParams),      // unchanged
    Wavelet(WaveletKind, WaveletParams),              // unchanged
    StatModel(StatModelKind, StatModelParams),        // unchanged
}

/// θ: structure only; equality derived over every field. Stays.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]   // PartialEq derived (was hand-written)
pub enum SketchParams {
    UnivMon { heap_size: u32, sketch_rows: u32, sketch_cols: u32, layers: u8 }, // distinct_failure_probability removed
    Kll { k: u32 },                                   // failure_probability removed
    Cms { width: u32, depth: u32 },                   // unchanged
    Hll { precision: u8 },                            // relative_error removed
    DDSketch { alpha: f64 },                          // unchanged
    CmsWithHeap { width: u32, depth: u32, heap_size: u32 }, // unchanged
    Kmv { k: u32 },                                   // relative_error removed
    CountSketch { width: u32, depth: u32 },           // unchanged
    CountSketchWithHeap { width: u32, depth: u32, heap_size: u32 }, // unchanged
    /// θ of the Hydra* algorithms.
    Hydra(HydraParams),                               // unchanged
}
pub struct SketchKind { .. }                          // removed
pub enum GroupingStrategy { .. }                      // removed
pub enum HydraKind { .. }                             // removed

/// One summary type's row of the capability table. Was FamilyCapability.
pub struct SummaryTypeCapability { pub merge_inputs: Option<MergeInputRule>, /* readouts */ }
impl SummaryTypeCapability {
    /// The row of `summary_type`: sketchlib's for a sketch, the planner's for an exact accumulator.
    pub fn of(summary_type: &FieldDataType) -> Self;                         // was of(family)
    pub fn of_per_instance_sketch(algorithm: &SketchAlgorithm) -> Self;      // removed (use of)
    /// Whether a rollup may merge different groups' states.
    pub fn merges_groups(summary_type: &FieldDataType) -> bool;              // unchanged
    /// Whether `summary_type` is a Hydra algorithm: one structure for every group.
    pub fn shares_one_state_across_groups(summary_type: &FieldDataType) -> bool; // new
}

types/src/ir/operator/asap.rs, agg_intent.rs

pub enum ASAPOp<C> {
    SummaryAgg { child: C, summary_type: FieldDataType /* was family */, input: SummaryUpdate,
                 reduction: Reduction, /* grouping removed */ filter: Option<Predicate<C>> },
    SummaryTumblingWindowChain { child: C, summary_type: FieldDataType /* was family */,
                 input: SummaryUpdate, reduction: Reduction, /* grouping removed */
                 filter: Option<Predicate<C>>, tumbling_window_width_ms: u64 },
    SummaryJoin { outer: C, inner: C, key: ColumnId, summary_type: FieldDataType /* was family */ },
    // every other variant unchanged
}
pub enum AggIntent<C> {
    Count,                                  // was Count { accuracy }
    Quantile { col: Option<C>, q: f64 },    // accuracy removed
    TopK { k: usize },                      // accuracy removed
    Cardinality { cols: Vec<C> },           // accuracy removed
    FrequencyL2 { col: Option<C> },         // accuracy removed
    FrequencyEntropy { col: Option<C> },    // accuracy removed
    // every other variant unchanged
}

types/src/stages/memo.rs, candidates.rs, logical_memo_candidates.rs, pruning.rs

/// The sign of one column's values; a group property, derived like lineage. New.
pub enum ValueSign { NonNegative, NonPositive, Unknown }
pub struct GroupProperties {
    pub schema: Schema, pub result_kind: OperatorResultKind,           // unchanged
    /// Per output column: its sign (derived in PR 2).
    pub column_sign: Vec<ValueSign>,                                      // new
    /// Whether the result changes with the evaluation time (derived in PR 2).
    pub depends_on_evaluation_time: bool,                                 // new (2026-11-09)
    pub aggregate_coverage: Option<SummaryCoverage>, pub shareable_when_equal: bool, // unchanged
}
pub struct Memo {
    /// The queries, by index in the workload.
    pub queries: Vec<MemoQuery>,                                         // was query_roots: Vec<QueryRoot<MemoGroupId>>
    // private groups, unchanged
}
/// One query: its root, A_q (None: accuracy not enforced, P17) and n_q (None: not bounded). New.
pub struct MemoQuery { pub root: QueryRoot<MemoGroupId>, pub accuracy_target: Option<AccuracyTarget>, pub values_per_evaluation: Option<u64> }
impl Memo {
    pub fn add_query(&mut self, query: MemoQuery) -> usize;             // was add_query(root)
}
pub fn root_groups(queries: &[MemoQuery]) -> BTreeSet<MemoGroupId>;            // was &[QueryRoot<MemoGroupId>]
pub fn queries_rooted_at(queries: &[MemoQuery], group: MemoGroupId) -> Vec<usize>; // was &[QueryRoot<MemoGroupId>]
/// r^log of one input: only a readout's input carries a required accuracy.
pub struct LogicalRequirements { pub required_accuracy: Option<RequiredAccuracy> } // was parameter_floor_for_required_accuracy
/// ε_r, δ_r (per value) in a metric; also the readout's own output error. New.
pub struct RequiredAccuracy { pub metric: AccuracyMetric, pub max_error: f64, pub failure_probability: f64 }
pub fn parameter_list(algorithm: SketchAlgorithm) -> &'static [SketchParams]; // arg was family
pub struct LogicalASAPMemoCandidates {
    // memo, edges, pruned: unchanged
    /// Per group: the coarsest grid holding every evaluation time of the queries reaching it (Pass 2, PR 4).
    pub evaluation_grid_of_group: Vec<Option<EvaluationGrid>>,            // new (2026-11-09)
}
pub fn producer_satisfies_logical_requirements(producer: &LogicalProperties, requirement: &LogicalRequirements) -> bool; // removed (moved, P3)
#[derive(Serialize)] pub enum OriginOfCandidateForMemoGroup { /* unchanged */ }   // Serialize new
#[derive(Serialize)] pub enum RootPruneReason {
    /// The producer's guarantee at the required δ exceeds the required error.
    BelowRequiredAccuracy { consumer: CandidateForMemoGroupId, input: usize, required: RequiredAccuracy }, // was BelowAccuracyFloorRequirement { .., floor }
    MetricRule { .. }, NoConsumer,                                        // unchanged
}
#[derive(Serialize)] pub enum PruneReason {
    // NoIngestionPipeline, RawDataNotRetained, QueryTimeResultsNotRetained,
    // InputHasNoBuildableProducer, ReachesNoQueryOutput, DominatedBy, UsageNotCostable: unchanged
    /// Stage 2: a Count-Min build whose weight column is not proven NonNegative.
    CountMinWeightsNotProvenNonNegative,                                  // new
}

types/src/stages/physical/, types/src/ir/physical_export.rs, types/src/workload/mod.rs

/// κ: executions at the same times share one run. Now Copy + Eq + Hash.
pub enum ExecutionContext {
    Evaluation { grid: EvaluationGrid },     // was { recurrence, range_evaluation }
    IngestionPipeline(RefreshWindow),        // was (RefreshWindowId)
    Materialized(PhysicalCandidateId),       // unchanged
}
/// The evaluation times phase_ms + k·period_ms. New.
pub struct EvaluationGrid { pub period_ms: u64, pub phase_ms: u64 }
pub struct RefreshWindowId(pub usize);       // removed
/// Alignment: a chain by its width; any other stored result by its group's grid (PR 1).
pub fn delivers_on(&self, candidates: &PhysicalASAPMemoCandidates, context: &ExecutionContext) -> bool; // was (memo, context) (2026-11-09)
/// α^phys: timing and lifetime only.
pub fn producer_satisfies_physical_requirements(producer: &PhysicalCandidate, consumer_timing: ExecutionTiming) -> bool; // was consumer: &PhysicalCandidate
pub struct PhysicalCandidateId(pub usize);   // moved from graph.rs; Serialize new
pub struct DagOfSelectedCandidates { .. }    // removed
pub struct SelectedEdge { .. }               // removed
pub struct SelectedNodeId(pub usize);        // removed
pub trait AssignmentRealizer { .. }          // removed
pub struct RealizedPlan { .. }               // removed
pub enum PhysicalCandidateError { .. }       // removed
pub struct PhysicalWorkloadPlan { .. }       // removed
#[derive(Serialize)] pub struct PhysicalProperties { /* unchanged */ }   // Serialize new
pub struct PhysicalASAPDAGNode {
    // id, payload, output_state, lifetime, parameters, output_schema, guarantee, retention_ms: unchanged
    /// The physical candidate the node was built from; not in the plan file; None for a custom pass.
    #[serde(skip)] pub selected_physical_candidate: Option<PhysicalCandidateId>,   // new
}
impl PhysicalASAPDAG {
    /// Test support: the plan's query roots as IR, in entry order.
    #[cfg(any(test, feature = "test-support"))] pub fn query_roots(&self) -> Result<Vec<QueryRoot>, PhysicalASAPDAGValidationError>; // new (P9)
    pub fn scan_window(..); pub fn scan_windows(..); pub fn view_window(..); pub fn view_windows(..); // removed
}
pub enum ScanWindow { .. }                   // removed (P10)
pub fn time_range_window(..);                // removed (P10)
pub enum PhysicalASAPDAGValidationError {
    SummaryTypeSchemaMismatch { node },      // was SummaryFamilySchemaMismatch
    SummaryGroupingMismatch { node },        // removed
    // others unchanged
}
pub struct SourceFacts {
    /// Each column's declared sign, e.g. {"value": "non_negative"}; default none.
    pub column_signs: BTreeMap<String, ValueSign>,   // was values_non_negative: bool (P7)
    // others unchanged
}

accuracy/src/model.rs, estimators/mod.rs, lib.rs

/// The deployment's measured accuracy, per summary configuration. New.
pub trait AccuracyEvidence {
    /// The measured error bound at failure_probability; None without a measurement or when θ cannot fail that rarely.
    fn measured_guarantee(&self, summary_type: &FieldDataType, parameters: &SketchParams,
        statistic: &SketchStatistic, failure_probability: f64) -> Option<AccuracyGuarantee>;
}
/// No measurement: every guarantee is the theoretical one. New.
pub struct NoMeasuredGuarantees;
impl AccuracyEvidence for NoMeasuredGuarantees { .. }   // always None
/// The algorithm's error bound at failure_probability (todo, PR 4). Was local_guarantee(family, query).
pub fn theoretical_guarantee(summary_type: &FieldDataType, parameters: &SketchParams,
    statistic: &SketchStatistic, failure_probability: f64) -> Option<AccuracyGuarantee>;
/// α^log of one edge (todo, PR 4). Moved here from asap-types (P3).
pub fn producer_satisfies_logical_requirements(memo: &Memo, producer: CandidateForMemoGroupId,
    consumer: CandidateForMemoGroupId, input_index_of_consumer: usize, evidence: &dyn AccuracyEvidence) -> bool;
/// The largest evaluated error (or δ) that meets `limit`: limit·(1 + 1e-9) + f64::EPSILON. New.
pub fn target_limit(limit: f64) -> f64;
pub fn satisfies(guarantee: &AccuracyGuarantee, target: &AccuracyTarget) -> bool;  // removed (2026-11-09)
pub trait AccuracyModel { .. }               // removed (satisfies, accepts_*, may_*, metric_can_meet)
pub struct BuiltinAccuracyModel;             // removed
pub struct IgnoreAccuracyGuarantees;         // removed
pub trait AccuracyEvidenceProvider { .. }    // removed (operand_domain, output_guarantee, may_provide_evidence)
pub struct NoAccuracyEvidence;               // removed
pub struct OperandDomain { .. }              // removed
pub fn with_failure_probability(..);         // removed
pub fn sizing_target(..); pub fn accuracy_budget(..);   // removed

Stages: stage0.rs, pass1/logical_memo_candidates.rs, pass2/logical_memo_candidates.rs, plan-selection

/// Stage 0 (todo, PR 2): each query with its target and n_q; signs from `data` (P5).
pub fn memo_of_queries(roots: &[QueryRoot], accuracy_targets: &[Option<AccuracyTarget>], data: &DataWorkload) -> Result<Memo, SchemaDerivationError>; // gains two args
/// Pass 1: no accuracy pruning.
pub fn pass1_memo(memo: Memo, supported_summaries: Option<&[SupportedSummary]>) -> Result<Memo, Pass1Error>; // loses accuracy_targets, accuracy, evidence, data
/// Pass 2: α^log reads the evidence.
pub fn generate_logical_asap_memo_candidates(pass1: Memo, entries: &[QueryWorkloadEntry], evidence: &dyn AccuracyEvidence) -> Result<LogicalASAPMemoCandidates, Pass2RuleError>; // gains evidence
/// Stage 3 (todo, PR 6): selects and builds the plan.
pub fn select_plan(physical: &PhysicalASAPMemoCandidates, entries: &[QueryWorkloadEntry], data: &DataWorkload,
    cost_inputs: &CostInputs) -> Result<(PhysicalASAPDAG, Selection), ErrorSelectingPlan>; // loses accuracy_checks, realizer
/// Stage 3's report about its own run.
pub struct Selection {
    pub accepted_accuracy_per_query: Vec<AcceptedAccuracy>,                // unchanged
    pub pruned_candidates: Vec<PrunedCandidate<PhysicalCandidate>>,        // was PrunedCandidate<PhysicalCandidateId>
    pub milp_variables: usize, pub milp_constraints: usize,                // new (were in SearchReport)
    pub milp_construction_ms: u64, pub milp_solve_ms: u64,                 // new (were model_construction_ms, solver_ms)
    // selected_dag, nodes_with_proven_non_negative_weights, cost, search_report: removed
}
/// The accepted error's metric and the sketches a query reads.
pub struct AcceptedAccuracy { pub accepted_metric: Option<AccuracyMetric>, pub sketches: Vec<String> } // target_metric, accepted_regardless_of_guarantee, guarantee_check_rejection removed
pub struct AccuracyCheckInputs { .. }  pub struct SearchReport { .. }  pub enum SearchStatus { .. }  pub struct InexactObjective { .. } // removed
pub enum ErrorSelectingPlan { /* RealizerRejected, ObjectiveDiffersFromPlanCost, SelectedPlanFailedCheck removed; others unchanged */ }
pub enum RejectionCause { /* CountMinWeightsNotProvenNonNegative removed; others unchanged */ }
struct SelectionVariables { x, y, z, /** only on cycles */ t: Vec<Option<Variable>> }          // t was Vec<Variable>
struct AccuracyVariables { error_bound: HashMap<(PhysicalCandidateId, AccuracyMetric), Variable>, // per metric (E_{c,m})
    error_bound_of_input: HashMap<(PhysicalCandidateId, usize, AccuracyMetric), Variable>, .. }      // per metric (F_{c,i,m})

Facade and tools: planner/src/pass.rs, run.rs, installable.rs, cost, front ends

pub struct PlanningModels<'a> {
    /// The deployment's measured accuracy; none by default.
    pub evidence: &'a dyn AccuracyEvidence,       // was &dyn AccuracyEvidenceProvider
    pub cost_inputs: &'a CostInputs, pub deployment_capabilities: &'a DeploymentCapabilities, // unchanged
    /// Give every query no accuracy target before Pass 2.
    pub ignore_accuracy_guarantees: bool,         // unchanged field, new meaning
    // accuracy: &dyn AccuracyModel removed; accuracy_check_inputs(), with_accuracy() removed
}
impl PlanningModels<'_> { pub fn new(evidence: &dyn AccuracyEvidence) -> Self; } // was new(accuracy, evidence)
/// The selected plan: the only plan structure.
pub struct PlanOutput { pub plan: PhysicalASAPDAG }   // was { plans, scalar_roots, scalar_plans, selection, tumbling_window_retention_ms }
pub struct QueryPlan { .. }                           // removed
pub enum OptimizeError { /* .. */ InvalidPlan { pass: &'static str, error: PhysicalASAPDAGValidationError } } // InvalidPlan new (P12)
pub struct PlanningRun {
    pub pass1_memo, pub logical_asap_candidates, pub physical_asap_candidates, pub report, // unchanged
    pub selection: Result<(PhysicalASAPDAG, Selection), SelectionError>,                  // was Result<Selection, _>
    // selected_physical_plan, selected_plan removed
}
pub fn installable_plan_of(plan: &PhysicalASAPDAG, cost: Option<&PlanCost>, entries: &[QueryWorkloadEntry])
    -> Result<(InstallablePlan, Option<SelectedPlanReport>), InstallablePlanError>; // was (realized: &RealizedPlan, selection, entries) (P13)
pub enum NodeCostSource { Analytical, Measured }      // was NodePriceSource; NodeCost::cost_source was price_source
pub enum PlanRanking { Cost, Latency, Memory }        // Cost was Price
pub fn plan_cost(dag, nodes, recurrence, range_evaluations, data, settings, formula_scale); // loses materialized_chains
pub fn lower_sql(query: &str, catalog: &SqlCatalog) -> Result<Rc<OperatorNode>, SqlError>; // loses accuracy
pub fn lower_metricsql_query(query: &str, sample_interval: Duration) -> Result<QueryRoot, MetricsqlError>; // loses accuracy

Usage (end to end; the bodies are todo!() until the PRs named, so this is the call flow, not a run):

// The user's workload: p99 / p50 of `latency` (declared non-negative), relative error 0.03.
let workload: PlanningWorkload = serde_json::from_str(r#"{
  "query_workload": {"language": "promql", "repeating_queries": [{
    "query": "quantile_over_time(0.99, latency[1m]) / quantile_over_time(0.5, latency[1m])",
    "demand": {"fixed_interval_at": {"interval": 10000, "evaluation_phase": 0}},
    "requirements": {"accuracy": {"accuracy_metric": "relative", "max_error": 0.03}},
    "predictability": {"predictable": {"known_at": null}}}]},
  "data_workload": {"arrival": {"ingesting": {"rows_per_second": 1000.0}},
    "sources": {"latency": {"series_count": 1, "column_signs": {"value": "non_negative"}}}}}"#)?;
// The front end lowers the query with no accuracy on its measures (Quantile { col, q }).
// run_planning: Stage 0 adds the query with Some(target) and n_q = 1 (memo_of_queries, PR 2);
// Pass 1 adds KLL and DDSketch builds, one per listed θ (pass1_memo, PR 3);
// Pass 2 gives each readout a RequiredAccuracy level and computes α^log with
// producer_satisfies_logical_requirements(memo, build, readout, 0, &AccuracyEvidence {}) (PR 4):
// a KLL readout is cut by MetricRule, a DDSketch build with alpha 0.05 by
// BelowRequiredAccuracy { required: relative 0.02 }.
// Stage 3 returns (plan, selection) (select_plan, PR 6): plan.nodes[i].selected_physical_candidate
// names each node's candidate; the user gets PlanOutput { plan }, and its cost is plan_cost(&plan).
let output = e2e_plan(UserInput::new(&workload, FrontendInput::Promql, PlanningModels::builtin())).await?;

Major algorithm changes

None: the bodies of the new interfaces are todo!(). One behavior is removed: Pass 1 no longer prunes summaries no query could use for accuracy (without_summaries_no_query_can_use); Pass 2's dead-candidate elimination (PR 4) replaces it. The executor's window derivation is unchanged code in a new place (P10).

Before this PR

A sketch state's type was Sketch(SketchAlgorithm, GroupingStrategy), with Hydra as a grouping axis; an operator's summary type was called family; θ recorded the error and δ it was sized for, so two KLLs of one k were equal only through a hand-written equality. A consumer input carried a θ floor; measures carried the query's accuracy; Pass 1 pruned with a pluggable AccuracyModel; Stage 3 decoded a DagOfSelectedCandidates, had Stage 2 realize it into a RealizedPlan, checked accuracy after the solve, and the pass returned per-query operator trees (PlanOutput::plans) beside the selection report.

After this PR

A summary type is one FieldDataType value (Sketch(Kll), Sketch(HydraCms)); θ is structure only. A readout's input carries a RequiredAccuracy, and α^log compares the producer's measured-or-theoretical guarantee with it (PR 4). Queries enter the memo with their target and $n_q$; measures carry no accuracy. Stage 3 returns the plan, a PhysicalASAPDAG whose nodes name their physical candidate, beside its Selection report; PlanOutput holds only the plan.

Deleted, and what replaces each (this round)

Deleted Replaced by
SketchKind, GroupingStrategy, HydraKind, ops' grouping, PhysicalASAPDAGValidationError::SummaryGroupingMismatch FieldDataType::Sketch(SketchAlgorithm) with the Hydra* algorithms
FamilyCapability, of_per_instance_sketch, family_capability.rs SummaryTypeCapability::of (summary_type_capability.rs)
sized-for θ fields, hand PartialEq, with_failure_probability (×3), KMV/HLL sized ε structure-only θ; the δ levels are list entries (PR 1) and the guarantee is evaluated at δ_r (PR 4)
LogicalRequirements { parameter_floor_for_required_accuracy }, LogicalRequirements::new, RootPruneReason::BelowAccuracyFloorRequirement required_accuracy: Option<RequiredAccuracy>, BelowRequiredAccuracy
asap_types::stages::producer_satisfies_logical_requirements(producer, requirement) asap_accuracy::producer_satisfies_logical_requirements(memo, producer, consumer, input, evidence)
AccuracyModel, BuiltinAccuracyModel, IgnoreAccuracyGuarantees, AccuracyEvidenceProvider, NoAccuracyEvidence, OperandDomain, AccuracyCheckInputs, PlanningModels::{accuracy, accuracy_check_inputs, with_accuracy} AccuracyEvidence (Pass 2); ignore mode = no targets before Pass 2
local_guarantee theoretical_guarantee
pub struct AccuracyEvidence {} (2026-11-09) trait AccuracyEvidence + NoMeasuredGuarantees
Memo::{query_roots, query_accuracy_targets, query_values_per_evaluation} (2026-11-09) Memo::queries: Vec<MemoQuery>
satisfies, metric_meets, scope_meets, native_target, AccuracyScope::covers (2026-11-09) target_limit (the comparison with the tolerance); the metric and scope rules: deleted, no replacement (α^log compares one metric at δ_r, PR 4)
SupportedSummary::family (2026-11-09) SupportedSummary::summary_type
Pass 1 accuracy pruning (without_summaries_no_query_can_use, output_can_meet_target, sizing_target, accuracy_budget, DEFAULT_DELTA, PerComputationCandidateError::InvalidAccuracy) Pass 2 dead-candidate elimination (PR 4)
AggIntent measures' accuracy; front ends' accuracy parameter the query's target in the memo (Memo::query_accuracy_targets)
SourceFacts::values_non_negative SourceFacts::column_signs (P7)
ExecutionContext::Evaluation { recurrence, range_evaluation }, RefreshWindowId Evaluation { grid: EvaluationGrid }, IngestionPipeline(RefreshWindow)
RealizedPlan, PhysicalWorkloadPlan, DagOfSelectedCandidates, SelectedEdge, SelectedNodeId, AssignmentRealizer, Stage2AssignmentRealizer, PhysicalCandidateError, SelectionError::Realization, ErrorSelectingPlan::RealizerRejected Stage 3 builds PhysicalASAPDAG; selected_physical_candidate
SearchReport, SearchStatus, InexactObjective, relative_gap, ObjectiveDiffersFromPlanCost, SelectedPlanFailedCheck, Selection::{selected_dag, nodes_with_proven_non_negative_weights, cost, search_report}, PlanCost::cost_unit, COST_UNIT, AcceptedAccuracy::{target_metric, accepted_regardless_of_guarantee, guarantee_check_rejection}, RejectionCause::CountMinWeightsNotProvenNonNegative Selection::{milp_*}; plan_cost(&plan); (K) has every bound; PruneReason::CountMinWeightsNotProvenNonNegative
with_confirmed_weights(_at), confirmed_weights, WeightDomain::confirmed deleted, no replacement: their only consumer applied Stage 3's weight proofs (column_sign, PR 4–5)
PlanOutput::{plans, scalar_roots, scalar_plans, selection, tumbling_window_retention_ms}, QueryPlan, PlanningRun::{selected_plan, selected_physical_plan}, PlanningReport::search_report, InstallablePlanError::TumblingWindowRetentionOfUnknownChain PlanOutput { plan }, PlanningRun::selection; retention is on the nodes
ScanWindow, union, scan_window(s), view_window(s), time_range_window, offset_below (types); materialized_tumbling_window_chains, trailing_window, widest_window_read_ms, node_usage's scan_window (cost) Scan reads per edge and retention_ms ($h_v$) (PR 6–7); executor-private copy until PR 6 (P10)
NodePriceSource, price_source, PlanRanking::Price, family_name NodeCostSource, cost_source, PlanRanking::Cost, summary_type_name
pass1/composition.rs (compose_logical_candidate and its helpers), test_support::{cms_summary, weighted, estimate_over}, planning::{explicit_local_candidates, explicit_summary_candidate}, the PromQL tests' logical_candidates/logical_plan, the executor tests' computation_candidates (2026-11-09, P11) deleted, no replacement: planning adds every way to compute to the memo (Pass 1, PR 3), and PR 3 re-creates Pass 1 coverage over the memo
ShownLogicalCandidate, logical_candidates(), physical_candidates(), candidate_labels, physical_label and its helpers, usage_json, sketch_coverage --max-error deleted (R10); the document's stage 3 holds plan

Deleted tests (this round)

24 test functions deleted, 1 added (contract_rejects_a_scalar_root_keyed_beyond_the_entries), 3 renamed:

File Deleted Why
accuracy/src/estimators/mod.rs enlarging_meets_the_failure_probability_at_no_larger_error, a_distinct_count_univmon_grows_its_heap, local_guarantee_needs_a_statistic_the_family_answers δ resizing and the exact local_guarantee deleted
accuracy/src/estimators/kll.rs sizing_meets_a_failure_probability_below_one_percent, sizing_below_one_percent_names_a_distinct_contract, an_invalid_failure_probability_gives_no_guarantee KLL's sized-for failure_probability deleted; PR 4 evaluates the guarantee at δ_r
accuracy/src/estimators/cardinality.rs kmv_sized_for_a_failure_probability_meets_it, kmv_k_follows_the_failure_probability KMV's sized-for ε deleted
types/src/ir/schema/sketch.rs sketch_kind_keeps_a_valid_algorithm_and_params_pair, sketch_kind_rejects_params_from_another_algorithm, grouping_strategy_default_is_per_subpopulation_instance SketchKind, GroupingStrategy deleted
types/tests/physical_export.rs a_scan_window_is_the_union_of_its_readers_windows, a_view_window_is_the_union_of_the_windows_below_it, a_scan_read_under_an_at_anchor_has_an_unbounded_window the DAG's window functions deleted (P10)
logical-asap-optimization/src/pass1/per_computation_candidates.rs count_min_estimate_over_raw_data_samples_is_gated_on_the_metric compared with local_guarantee without θ; the gating is replaced by column_sign (PR 4)
planner/src/pass.rs contract_rejects_a_scalar_reference_without_its_plan, contract_checks_scalar_plan_order_entry_and_count scalar_plans deleted
devtools/src/candidate_labels.rs connected_labels_describe_selected_states, connected_labels_name_actual_shared_state, connected_labels_name_exact_results the L1 labels deleted
frontend-promql/tests/promql_lowering.rs, frontend-sql/tests/sql_lowering.rs accuracy_target_flows_into_quantile_intent, median_threads_the_accuracy_target measures carry no accuracy
integration-tests/tests/promql_numeric_regressions.rs logical_candidates_fails_on_a_candidate_that_does_not_compose ShownLogicalCandidate deleted
integration-tests/tests/joint_search_equivalence.rs custom_accuracy_policy_is_respected AccuracyModel deleted

Renamed, with the accuracy assertion dropped: count_maps_to_count (was count_maps_to_count_and_inherits_accuracy), distinct_over_time_lowers_to_cardinality_over_nested_windows, asap_input_is_rejected (its invalid-accuracy half deleted with InvalidAccuracy). Tests that read PlanOutput::plans, selected_plan or Selection::cost now read plan.query_roots() and plan_cost_per_second (P9).

2026-11-09, second round (P11)

74 tests deleted, all composing whole-workload candidates through pass1/composition.rs (names: comment). Files left without a test are deleted with their registrations: frontend-promql/tests/univmon_candidates.rs, frontend-promql/tests/observability/metrics_observability.rs (test target), integration-tests/tests/{exact_composition, promql_numeric_regressions, sketch_rollup, stage1_logical_asap_optimization}.rs. On be6ea1d they were 70 failed (most at Stage 0's todo!()), 3 passed, 1 ignored.

File Tests deleted
frontend-metricsql/tests/external_corpus.rs 1
frontend-promql/tests/count_planning.rs 4
frontend-promql/tests/observability/metrics_observability.rs 2
frontend-promql/tests/observability/promql_corpus.rs 1
frontend-promql/tests/univmon_candidates.rs 1
integration-tests/tests/exact_composition.rs 1
integration-tests/tests/per_computation_candidates_sql.rs 4
integration-tests/tests/promql_numeric_regressions.rs 7
integration-tests/tests/sketch_rollup.rs 3
integration-tests/tests/sql_to_post_asap.rs 7
integration-tests/tests/stage1_logical_asap_optimization.rs 13
logical-asap-optimization/src/pass1/per_computation_candidates.rs 19
logical-asap-optimization/src/pass1/per_computation_candidates_tests.rs 10
logical-asap-optimization/src/pass1/realization.rs 1

2026-11-09 round

8 tests deleted, 1 added (accuracy tests::target_limit_admits_only_floating_point_noise):

File Deleted Why
accuracy/src/lib.rs satisfies_is_fail_closed_on_unknowns_and_exact, satisfies_requires_the_targets_metric, a_target_without_delta_ignores_the_failure_probability, per_value_guarantees_meet_whole_result_targets_over_g_values, a_record_share_target_is_met_over_one_over_share_groups, scope_matters_only_with_a_failure_probability, a_target_without_a_metric_accepts_any_metric satisfies deleted
types/src/ir/properties/accuracy.rs a_wider_scope_covers_a_narrower_one AccuracyScope::covers deleted (only satisfies used it)

The estimator tests that called satisfies (CMS, Count Sketch, HLL, KLL, UnivMon, composition) now check the evaluated error and δ against target_limit through a test-only meets; their metric is asserted beside it, as before.

Self-review: over-abstraction, duplication, dead code

2026-11-09, second round: the empty mod stream_norms {} (Pass 1 tests) and two empty mod tests {} (devtools planning_stages_text.rs, candidate_labels.rs) are deleted; no other code lost its last user.

2026-11-09 round:

  • Dead after the change, deleted: metric_meets, scope_meets, native_target (test-only), AccuracyScope::covers, the static NO_ACCURACY_EVIDENCE (a unit struct needs none).
  • Duplication: the satisfaction tolerance formula lives once, in target_limit; the test-only meets calls it.
  • Left, and why: target_limit has no production caller until PR 4 (α^log) and PR 8 ((K)); root_groups has no caller in PR 0 (Stage 2 and 3 use it, PR 5–6); AccuracyGuarantee::over_scope has no production caller (PR 4/8 state a whole-result δ); NoMeasuredGuarantees is the doc's default evidence; depends_on_evaluation_time is a second unreachable todo!() inside derive_group_properties (PR 2 derives both).
  • Over-abstraction: none added; the bound at δ reuses AccuracyGuarantee (P18) instead of a new type.

2026-11-08 round:

Found and fixed in this round:

  • Duplication. The pass contract's scalar-key check repeated PhysicalASAPDAG::validate's ScalarRootPosition → the contract counts the roots, then validates. The Hydra algorithm list was in SummaryTypeCapability::merges_groups and in the executor's kernel check → one shares_one_state_across_groups (P8). of_per_instance_sketch duplicated of once Hydra is an algorithm → deleted.
  • Dead after the deletions, deleted: InvalidAccuracy and its mapping; computation_candidates_for_intent returned a Result that never failed (now infallible); sources_read; with_confirmed_weights(_at), confirmed_weights, WeightDomain::confirmed; materialized_tumbling_window_chains, trailing_window, widest_window_read_ms; payloads_by_id, offset_below; TumblingWindowRetentionOfUnknownChain; OperandDomain; DEFAULT_DELTA; the label helpers (for_queries, query_names, duration_name, build_label, state_label, dag_readers) and usage_json; parse_max_error; test helpers window_over, exported_with_scan, series_scan, approximate, error_and_delta, delta_and_error.
  • Unused parameters removed: plan_cost's and each_node_usage's materialized_chains, each_node_usage's root_scan_lookback_ms, node_usage's scan_window, devtools' lower_promql_with_sample_interval's accuracy, and test helpers' accuracy.
  • Test-only code marked as such: PerComputationCandidates::{roots, aggregate} behind test-support; the whole-workload composition is deleted (P11, 2026-11-09).

Left, and why:

  • AccuracyEvidence {} with a None method is a placeholder the doc requires (P1).
  • series_read, values_per_result have no caller until Stage 0 computes $n_q$ (PR 2) (P5).
  • satisfies has no production caller (P16).
  • RejectionCause, Rejection and ErrorSelectingPlan::NoValidCandidate are never constructed, as before this round; PR 6 decides whether the MILP's infeasibility uses them.
  • The #[allow(dead_code)] markers from earlier rounds (θ-based estimators and sizing, milp/model.rs, valid_matrix, GroupStates::Keyed) stay until PRs 4–8.
  • The front-end test helper lower_promql(query, accuracy) still takes a target: it fills the workload entry, which no longer reaches the measures.

Test status

cargo test -p <crate> --all-features --no-fail-fast at the head (2ed6ec0). Except for the stale fixture, every failure panics at a todo!(); the column names it and the PR that fills it in.

Crate Passed Failed Ignored Fails at
asap-types 153 2 0 reject the invalid combination of section 2 (PR 1): 2
asap-accuracy 59 0 0 —
asap-cost 31 1 0 check θ, which the exported node carries (PR 6): 1
asap-frontend-common 49 0 0 —
asap-sql-function-catalog 10 0 0 —
metricsql_common 3 0 0 —
asap-frontend-promql 218 1 0 Stage 0 over Memo::find_or_add_group, each query with its target and n_q (PR 2): 1
asap-frontend-metricsql 17 0 0 —
asap-frontend-sql 173 1 0 Stage 0 over Memo::find_or_add_group, each query with its target and n_q (PR 2): 1
asap-logical-asap-optimization 25 24 0 Stage 0 over Memo::find_or_add_group, each query with its target and n_q (PR 2): 23; the distinct-count guarantee of θ from LogicalProperties::parameters (PR 4): 1
asap-physical-asap-optimization 0 0 0 —
asap-plan-selection 3 0 0 —
asap-executor 210 63 1 check θ, which the exported node carries (PR 6): 32; the Hydra grid's sizes are θ, which the exported node carries (PR 6): 13; each materialized node's retention_ms is its h_v, which Stage 3 sets (PR 6): 8; the weighted-frequency θ, which the exported node carries (PR 6): 7; check the weighted-frequency θ, which the exported node carries (PR 6): 2; Stage 0 over Memo::find_or_add_group, each query with its target and n_q (PR 2): 1
asap-planner 8 18 1 Stage 0 over Memo::find_or_add_group, each query with its target and n_q (PR 2): 18
asap-devtools 37 56 3 Stage 0 over Memo::find_or_add_group, each query with its target and n_q (PR 2): 54; OTHER: assertion failed: output.status.success(): 2
asap-integration-tests 82 51 4 Stage 0 over Memo::find_or_add_group, each query with its target and n_q (PR 2): 47; each materialized node's retention_ms is its h_v, which Stage 3 sets (PR 6): 3; OTHER: called Result::unwrap() on an Err value: Error("invalid type: sequence, expected string or map", line: 0, column: 0): 1
Total 1078 217 9
  • Compared by test name with be6ea1d (2026-11-09, before P11): the only differences are the 74 tests deleted with pass1/composition.rs (70 failed, 3 passed, 1 ignored there); no other test changed result.
  • Compared by test name with 7708f36 (r1108b logs): no test that passed fails now. The only differences are the 8 deleted tests (satisfies's 7 in asap-accuracy, a_wider_scope_covers_a_narrower_one in asap-types) and the added target_limit_admits_only_floating_point_noise (passes). Passed 1088 → 1081, failed 287 → 287. crate_graph: passes.
  • Stale fixture. materialized_views_equal_the_stored_outputs_main_listed reads a checked-in plan file in an old wire format; PR 9 re-baselines it with the snapshots.

Verification

  • cargo clippy --workspace --all-targets --all-features -- -D warnings and cargo clippy --workspace -- -D warnings: clean. No new #[allow]; derive_group_properties carries #[allow(unreachable_code)] until PR 2 derives column_sign.
  • cargo fmt --check: clean.
  • crate_graph (integration tests): passes; no stage crate depends on another.
  • Tests per crate: see "Test status".
  • No snapshot or plan comparison applies: planning panics at Stage 0 until PR 2.

@zzylol

zzylol commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

Per-file test deletions in #1226 (origin/main → PR head). D = file deleted, M = modified; counts are #[test] functions.

File main now deleted What it covered Restored by
cost/src/state_bytes.rs M 8 4 4 held memory / refresh runs over hand-assigned materialized DAGs; state bytes per θ PR 6–7
cost/tests/held_memory.rs D 1 0 1 held memory / refresh runs over hand-assigned materialized DAGs; state bytes per θ PR 6–7
cost/tests/ingestion_refresh_runs.rs D 7 0 7 held memory / refresh runs over hand-assigned materialized DAGs; state bytes per θ PR 6–7
devtools/src/candidate_labels.rs M 4 3 1 reference pricing (memo_pricing), stage text, labels PR 6 (Appendix A check), PR 9 (tools)
devtools/src/planning_stages_text.rs M 1 0 1 reference pricing (memo_pricing), stage text, labels PR 6 (Appendix A check), PR 9 (tools)
devtools/tests/asap_planner.rs M 46 43 3 reference pricing (memo_pricing), stage text, labels PR 6 (Appendix A check), PR 9 (tools)
devtools/tests/memo_pricing.rs D 6 0 6 reference pricing (memo_pricing), stage text, labels PR 6 (Appendix A check), PR 9 (tools)
executor/src/physical_planner/mod.rs M 3 1 2 executor runs of planner-built DAGs; kernels per θ PR 6
executor/src/summary_kernels/factory.rs M 7 5 2 executor runs of planner-built DAGs; kernels per θ PR 6
executor/tests/deployment_computation.rs M 16 3 13 executor runs of planner-built DAGs; kernels per θ PR 6
executor/tests/plan_properties.rs M 4 2 2 executor runs of planner-built DAGs; kernels per θ PR 6
executor/tests/precompute_candidates.rs D 2 0 2 executor runs of planner-built DAGs; kernels per θ PR 6
executor/tests/state_column_updater.rs M 7 5 2 executor runs of planner-built DAGs; kernels per θ PR 6
executor/tests/weighted_topk_binding.rs M 2 1 1 executor runs of planner-built DAGs; kernels per θ PR 6
frontend-promql/tests/univmon_candidates.rs M 2 1 1 planner-built plans exported and run PR 6
frontend-sql/tests/pearson_corr.rs M 7 6 1 planner-built plans exported and run PR 6
integration-tests/tests/declared_group_counts.rs D 3 0 3 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/deployment_capabilities.rs D 2 0 2 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/exact_composition.rs M 5 1 4 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/finer_aggregate_reuse.rs M 10 2 8 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/frequency_execution.rs D 1 0 1 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/having_shared_aggregate.rs D 1 0 1 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/materialization_e2e.rs D 4 0 4 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/materialized_views.rs M 4 2 2 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/memo_enumeration.rs D 3 0 3 per-group enumeration counts PR 6 (Appendix A)
integration-tests/tests/operator_design_examples.rs M 6 5 1 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/planner_layering_example1.rs D 31 0 31 #509 example acceptance suite over the reference pricer (WholePlanSelection) PR 6 (baseline), PR 7 (materialization), PR 8 (accuracy)
integration-tests/tests/planner_layering_example2.rs D 35 0 35 #509 example acceptance suite over the reference pricer (WholePlanSelection) PR 6 (baseline), PR 7 (materialization), PR 8 (accuracy)
integration-tests/tests/planner_layering_example3.rs D 22 0 22 #509 example acceptance suite over the reference pricer (WholePlanSelection) PR 6 (baseline), PR 7 (materialization), PR 8 (accuracy)
integration-tests/tests/planner_layering_example4.rs D 17 0 17 #509 example acceptance suite over the reference pricer (WholePlanSelection) PR 6 (baseline), PR 7 (materialization), PR 8 (accuracy)
integration-tests/tests/precompute_raw_samples.rs D 3 0 3 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/production_rollup_workloads.rs M 4 3 1 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/promql_to_post_asap.rs M 13 9 4 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/range_evaluation.rs D 6 0 6 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/reuse_rules.rs D 7 0 7 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/rollup_controlled.rs M 3 1 2 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/runtime_handoff.rs D 8 0 8 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/sharing_issue_regressions.rs M 12 6 6 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/sketch_rollup_planning.rs D 3 0 3 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/sql_group_bound.rs D 7 0 7 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/sql_to_physical.rs D 1 0 1 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/sql_trailing_window.rs D 3 0 3 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/sql_two_percentiles.rs D 4 0 4 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/stage1_logical_asap_optimization.rs M 14 13 1 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/stage2_physical_asap_optimization_sql.rs D 4 0 4 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/tumbling_window_merge.rs D 14 0 14 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
logical-asap-optimization/src/pass1/logical_memo_candidates.rs M 3 0 3 candidate population with private nodes, budget-share sizing, Pass 1 pruned reasons, θ in nodes PR 3; pruned reasons: #1205; budget split: no replacement planned
logical-asap-optimization/src/pass1/memo_population.rs D 13 0 13 candidate population with private nodes, budget-share sizing, Pass 1 pruned reasons, θ in nodes PR 3; pruned reasons: #1205; budget split: no replacement planned
logical-asap-optimization/src/pass1/per_computation_candidates.rs M 56 42 14 candidate population with private nodes, budget-share sizing, Pass 1 pruned reasons, θ in nodes PR 3; pruned reasons: #1205; budget split: no replacement planned
logical-asap-optimization/src/pass1/per_computation_candidates_tests.rs M 35 31 4 candidate population with private nodes, budget-share sizing, Pass 1 pruned reasons, θ in nodes PR 3; pruned reasons: #1205; budget split: no replacement planned
logical-asap-optimization/src/pass2/asap_aware_cse.rs D 27 0 27 node-level Pass 2 rewrites (windows, rollups, top-k, scan widening) PR 4
logical-asap-optimization/src/pass2/memo/rules/rollup.rs M 1 0 1 node-level Pass 2 rewrites (windows, rollups, top-k, scan widening) PR 4
logical-asap-optimization/src/pass2/memo/rules/shared_configuration_readers.rs D 2 0 2 rule 9 shared configurations no replacement planned (deleted by design)
logical-asap-optimization/src/pass2/memo/shared_configurations.rs D 6 0 6 rule 9 configuration enumeration no replacement planned
logical-asap-optimization/src/pass2/memo/state_groups.rs D 23 0 23 step 10 per-size state groups no replacement planned (deleted by design)
logical-asap-optimization/src/pass2/memo/window_candidates.rs M 2 0 2 node-level Pass 2 rewrites (windows, rollups, top-k, scan widening) PR 4
physical-asap-optimization/src/catalog/realize.rs M 1 0 1 Stage 2 catalog, materialization options, timing split, realization PR 5 (expansion), PR 6 (realization)
physical-asap-optimization/src/lib.rs M 24 0 24 Stage 2 catalog, materialization options, timing split, realization PR 5 (expansion), PR 6 (realization)
physical-asap-optimization/src/split_by_timing.rs D 4 0 4 Stage 2 catalog, materialization options, timing split, realization PR 5 (expansion), PR 6 (realization)
plan-selection/src/groups/accuracy_positions.rs D 1 0 1 #1144 MILP, result slots, dominance, whole-plan selector and its checks PR 6 (A)–(F)/decode, PR 7 contexts/(R), PR 8 (K); latency checks: no replacement planned
plan-selection/src/groups/tests.rs D 11 0 11 #1144 MILP, result slots, dominance, whole-plan selector and its checks PR 6 (A)–(F)/decode, PR 7 contexts/(R), PR 8 (K); latency checks: no replacement planned
plan-selection/src/lib.rs M 104 0 104 #1144 MILP, result slots, dominance, whole-plan selector and its checks PR 6 (A)–(F)/decode, PR 7 contexts/(R), PR 8 (K); latency checks: no replacement planned
plan-selection/src/milp/model/tests.rs D 20 0 20 #1144 MILP, result slots, dominance, whole-plan selector and its checks PR 6 (A)–(F)/decode, PR 7 contexts/(R), PR 8 (K); latency checks: no replacement planned
plan-selection/src/select/tests.rs D 4 0 4 #1144 MILP, result slots, dominance, whole-plan selector and its checks PR 6 (A)–(F)/decode, PR 7 contexts/(R), PR 8 (K); latency checks: no replacement planned
plan-selection/src/selector.rs D 1 0 1 #1144 MILP, result slots, dominance, whole-plan selector and its checks PR 6 (A)–(F)/decode, PR 7 contexts/(R), PR 8 (K); latency checks: no replacement planned
planner/src/installable.rs M 4 2 2 installed plans vs query_time_reference, plan_installable, reference pricer, Pass 1 pruned reasons PR 6 (install), PR 8 (accuracy model)
planner/src/pass.rs M 7 6 1 installed plans vs query_time_reference, plan_installable, reference pricer, Pass 1 pruned reasons PR 6 (install), PR 8 (accuracy model)
planner/tests/deployment_accuracy_model.rs D 3 0 3 installed plans vs query_time_reference, plan_installable, reference pricer, Pass 1 pruned reasons PR 6 (install), PR 8 (accuracy model)
planner/tests/per_sample_updater.rs D 2 0 2 installed plans vs query_time_reference, plan_installable, reference pricer, Pass 1 pruned reasons PR 6 (install), PR 8 (accuracy model)
planner/tests/runtime_install.rs M 42 5 37 installed plans vs query_time_reference, plan_installable, reference pricer, Pass 1 pruned reasons PR 6 (install), PR 8 (accuracy model)
planner/tests/sql_per_group_topk.rs D 6 0 6 installed plans vs query_time_reference, plan_installable, reference pricer, Pass 1 pruned reasons PR 6 (install), PR 8 (accuracy model)
planner/tests/sql_table_scans.rs M 11 3 8 installed plans vs query_time_reference, plan_installable, reference pricer, Pass 1 pruned reasons PR 6 (install), PR 8 (accuracy model)
planner/tests/summary_sharing.rs M 13 4 9 installed plans vs query_time_reference, plan_installable, reference pricer, Pass 1 pruned reasons PR 6 (install), PR 8 (accuracy model)
types/src/ir/properties/timing.rs D 23 0 23 timing pass over the side map PR 6
types/src/ir/schema/sketch.rs M 6 5 1 KLL equality across δ (θ in the type) PR 1
types/src/stages/physical/producers.rs D 2 0 2 producer-set interning no replacement planned
types/tests/materialized_view.rs D 10 0 10 export and materialized views with non-default physical properties; node timing PR 6
types/tests/physical_export.rs M 19 12 7 export and materialized views with non-default physical properties; node timing PR 6
types/tests/schema_rebuilding.rs M 9 7 2 export and materialized views with non-default physical properties; node timing PR 6
types/tests/structure_contract.rs M 7 6 1 export and materialized views with non-default physical properties; node timing PR 6
types/tests/summary_coverage.rs M 33 32 1 export and materialized views with non-default physical properties; node timing PR 6
types/tests/summary_merge_structure.rs M 9 8 1 export and materialized views with non-default physical properties; node timing PR 6
types/tests/tumbling_window_chain.rs M 14 10 4 export and materialized views with non-default physical properties; node timing PR 6
total 642

Restored in the 2026-11-06 revision through the test-only asap_types::test_support::physical_asap_dag_at_query_time(_with_node_ids) (every node at query time, unmaterialized, no θ): executor time_series_window, node_ids, current_series_heap, planspace_series_identity_heap, deployment_computation (3 of 15); integration filtered_aggregates, kll_tumbling_window_execution, per_series_exact_execution, sql_cardinality, sql_frequency, exact_composition, and parts of sketch_rollup, stage1_logical_asap_optimization, operator_design_examples, operator_sharing, promql_numeric_regressions, sql_to_post_asap, promql_to_post_asap; types physical_export; frontend count_planning, pearson_corr. Tests whose DAG came from the planner stay deleted (PR 6 restores them). Restored sketch tests fail at the executor's todo!() (θ from the node's parameters) until PR 6.

…iming, lifetime }

Deletes Schedule and the σ-only InvalidPhysicalProperties variants; the only
invalid combination left is (IngestionTime, OneEvaluation). satisfies_physical
checks timing x lifetime only; delivers_on is the per-context alignment check
(w | I and w | φ). A OneEvaluation result runs in one evaluation context, so
x_{κ,v} covers it as for ℓ = None.
…physical drops memo

InvalidLogicalProperties gains SketchBuildWithoutParameters and renames
FloorNotInParameterList to ParametersNotInList, returned by
LogicalProperties::new. LogicalRequirement::new no longer checks list
membership (a floor is any value of the family's parameter type, e.g.
heap_size >= k), so it returns Self. satisfies_physical takes only the
producer and consumer.
Context::QueryEvaluation(QueryId) becomes Context::Evaluation(EvaluationSchedule):
queries with the same (interval, phase) are evaluated together and share
their OneEvaluation results. evaluation_schedule returns that schedule.
QueryId stays for z_{q,v}. A non-periodic query has no context yet.
InvalidLogicalProperties -> LogicalPropertiesError, InvalidPhysicalProperties
-> PhysicalPropertiesError, LogicalRequirement -> LogicalRequirements
(parameter_floor -> parameter_floor_for_required_accuracy), satisfies_logical
/ satisfies_physical -> producer_satisfies_logical_requirements /
producer_satisfies_physical_requirements, Medium -> StorageMedium,
Context / ContextId -> ExecutionContext / ExecutionContextId
(x_in_context / y_in_context -> x_in_execution_context /
y_in_execution_context). PhysicalCandidate and PhysicalCandidateKey fields
-> logical_candidate_id, physical_properties; edge fields ->
producer_candidate_id, consumer_candidate_id; PhysicalASAPMemoCandidates
fields -> logical_candidates, physical_candidates.
@zzylol zzylol self-assigned this Nov 6, 2026
Deletes EvaluationSchedule and ExecutionContext::evaluation_schedule. An
evaluation context is Evaluation { recurrence, range_evaluation }: queries
with equal ones share it and its OneEvaluation results, one-off queries
included. ExecutionContext, ExecutionContextId and RefreshWindowId move to
asap_types::stages so PhysicalCandidate::delivers_on can take the context.
Contexts are deduplicated by equality (QueryRecurrence holds an f64).
PhysicalASAPDAGValidationError::InvalidPhysicalProperties becomes
PhysicalProperties { node, reason }. In the files this unit edits, "kept"
describing a materialized result or retained data becomes "materialized"
or "retained" (cost, materialized views, realizer, installable plan,
deployment capabilities; locals kept/raw_kept_rows -> retained/
raw_retained_rows), and "reader" becomes "query" where it is a query
(execution demand, ScheduleGroup::queries, cost usage, labels, tests) or
"consumer" where it is a consuming candidate.
find_or_add_group returns the node rules' SchemaDerivationError and
add_candidate a CandidateError (the node rules, or an output that differs
from the group's). Bodies stay todo!() until PR 2.
…mo (#1207 PR 0)

Memo::groups is private, read through groups(); groups are added only
through find_or_add_group and queries through add_query (todo!() until
PR 2). memo_of_queries returns Result and its body is todo!() until PR 2,
which builds Stage 0 on find_or_add_group; Stage 0's node-level CSE (cse.rs),
its test_support export and the dead share_identical helper are deleted.
@zzylol
zzylol force-pushed the refactor/1207-interfaces branch from 200ec5c to 341c98b Compare November 6, 2026 21:28
…OperatorInput; coverage only in the memo (#1207 PR 0)

MemoGroup { expression, group_properties: GroupProperties, parents, queries,
candidates } replaces logical_node and child_groups; GroupProperties {
schema, result_kind, summary_coverage,
shareable_when_equal } is derived once by derive_group_properties, which
applies the existing rules to the expression over MemoGroupAsInputs.

The rules (output_schema, output_kind, validate_inputs,
is_shareable_when_equal, summary builds, window chains, scalar subqueries,
top-k, populations) are generic over OperatorInput, implemented for
Rc<OperatorNode> and MemoGroupAsInput; only how they read an input changed.
BuiltOrInput carries the builds a rule makes while deriving (one build over
a chain's window, the build a rollup or a selection stands for).

SummaryCoverage { definition: SummaryStateDefinition { input_group,
unlifted_operators }, selection } is derived only from memo groups
(derive_from_inputs) and stored in GroupProperties; node-tree coverage,
NodeToSummaryCoverageCache, derive_cached and OperatorNode::coverage are
deleted, so node trees have no merge check. Memo::logical_node_trees is
deleted: Pass 1 target discovery (PR 3), devtools group labels (PR 9) and
the export's view coverage (PR 6) are todo!(). find_or_add_group and
add_candidate derive the properties, then todo!() until PR 2.
@zzylol
zzylol force-pushed the refactor/1207-interfaces branch from 341c98b to 3a75e1e Compare November 6, 2026 21:37
…elpers (#1207 PR 0)

One state's θ is no longer sized for several consumers: θ comes from
parameter_list per candidate. size_for_all_readers had no non-test caller;
min_recorded, kll::shared_k and univmon::shared_heap served only it.
@zzylol

zzylol commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor Author

Key code interfaces, continued (moved here from the PR body, which GitHub limits to 65,536 characters). Links point at the head.

crates/types/src/stages/logical_memo_candidates.rs

/// $(u, c, i)$ over logical candidates: `producer_candidate_id` is a
/// candidate of $G_{c,i}$, the group input `input_index_of_consumer` of
/// `consumer_candidate_id` reads.
pub struct LogicalEdge {
    pub producer_candidate_id: CandidateForMemoGroupId,
    pub consumer_candidate_id: CandidateForMemoGroupId,
    pub input_index_of_consumer: usize,
    /// $\alpha^{\log}_{u,c,i}$.
    pub satisfies_logical_requirement: bool,
}

/// Stage 1, Pass 2's output.
pub struct LogicalASAPMemoCandidates {
    /// The memo after the last rule.
    pub memo: Memo,
    /// Every $(u, c, i)$ with $u \in G_{c,i}$ between candidates that
    /// survive dead-candidate elimination, grouped by $(c, i)$.
    pub edges: Vec<LogicalEdge>,
    /// The candidates dead-candidate elimination removed. // new 2026-11-07
    pub pruned: Vec<PrunedLogicalCandidate>,
}

/// A candidate eliminated by Pass 2. // new 2026-11-07
pub struct PrunedLogicalCandidate {
    pub candidate: CandidateForMemoGroupId,
    /// Every cause: one per failing edge it is cut off by.
    pub causes: Vec<PruneCause>,
}

/// One path from the candidate, through consumers that are themselves
/// eliminated, to the edge that fails. One shortest path per failing edge. // new 2026-11-07
pub struct PruneCause {
    /// The eliminated consumers on the path, nearest first.
    pub through_pruned_consumers: Vec<CandidateForMemoGroupId>,
    pub root: RootPruneReason,
}

// new 2026-11-07
pub enum RootPruneReason {
    /// The producer's parameters are weaker than the consumer input's floor.
    BelowAccuracyFloorRequirement { consumer: CandidateForMemoGroupId, input: usize, floor: SketchParams },
    /// The consumer's error contract carries none of the producer's metrics.
    MetricRule { consumer: CandidateForMemoGroupId, input: usize, producer_metric: AccuracyMetric, consumer_carries: Vec<AccuracyMetric> },
    /// The candidate never had a consumer and is not a query output.
    NoConsumer,
}

crates/physical-asap-optimization/src/catalog/mod.rs (Stage 2 entry, 2026-11-07)

/// Stage 2. Costs nothing, and cannot fail. // was (logical, entries, data, query_time_results_retained, raw_data_retained) -> Result<_, ErrorGeneratingPhysicalASAPMemoCandidates>
pub fn generate_physical_asap_memo_candidates(
    logical: &LogicalASAPMemoCandidates,
    data: &DataWorkload,
    capabilities: &DeploymentCapabilities, // moved from asap-logical-asap-optimization into asap_types::deployment_capabilities
) -> PhysicalASAPMemoCandidates;

crates/types/src/stages/pruning.rs (2026-11-07)

/// One candidate a stage removed, and why: Stage 2's `PhysicalCandidate`,
/// Stage 3's `PhysicalCandidateId`. // loses `group`
pub struct PrunedCandidate<C> {
    pub candidate: C,
    pub reason: PruneReason,
}

pub enum PruneReason {
    NoIngestionPipeline,          // new
    RawDataNotRetained,
    QueryTimeResultsNotRetained,  // new
    InputHasNoBuildableProducer,
    ReachesNoQueryOutput,
    DominatedBy(PhysicalCandidateId),
    UsageNotCostable,
    // deleted: PerEntityAggregateNotExecutable (no executor-support check), TooFewQueryReaders (no consumer-count policies)
}

crates/types/src/workload/mod.rs (excerpt, 2026-11-07)

pub enum DataArrival {
    AtRest,
    Ingesting { rows_per_second: f64, max_out_of_order_ms: Option<u64> }, // max_out_of_order_ms new
    Mixed { rows_per_second: f64, max_out_of_order_ms: Option<u64> },     // max_out_of_order_ms new
    Unknown,
}

crates/types/src/physical/properties.rs

/// Where a materialized output is stored.
pub enum StorageMedium {
    Memory,
    Disk,
}

/// $\ell_v$: how long a candidate's output lives. Only a materialized output
/// has a medium.
pub enum Lifetime {
    /// Discarded after its consumer's execution completes.
    #[default]
    None,
    /// Materialized until every consumer within one evaluation finishes.
    OneEvaluation(StorageMedium),
    /// Materialized across evaluations, so later evaluations reuse it.
    AcrossEvaluations(StorageMedium),
}

/// A timing and lifetime that cannot go together (the invalid combination
/// of the concepts doc, section 2).
pub enum PhysicalPropertiesError {
    #[error("an ingestion-time output cannot be materialized for one evaluation: there is no evaluation at ingestion time")]
    OneEvaluationAtIngestion,
}

/// $p^{\mathrm{phys}}_v = (\tau, \ell)$. [`Self::new`] rejects the invalid
/// combination (concepts doc, section 2).
pub struct PhysicalProperties {
    timing: ExecutionTiming,
    lifetime: Lifetime,
}

impl PhysicalProperties {
    /// The properties `(timing, lifetime)`, unless they cannot go together.
    pub fn new(
        timing: ExecutionTiming,
        lifetime: Lifetime,
    ) -> Result<Self, PhysicalPropertiesError>;

    /// $\tau_v$.
    pub fn timing(&self) -> ExecutionTiming;

    /// $\ell_v$.
    pub fn lifetime(&self) -> Lifetime;

    /// Whether $\ell_v \ne$ None. Derived, not stored.
    pub fn is_materialized(&self) -> bool;
}

crates/types/src/stages/physical/memo_candidates.rs

/// Index into [`PhysicalASAPMemoCandidates::edges`].
pub struct PhysicalEdgeId(pub usize);

/// One physical alternative of a logical candidate: what Stage 3 selects.
pub struct PhysicalCandidate {
    /// $e_v$ and $p^{\log}_v$.
    pub logical_candidate_id: CandidateForMemoGroupId,
    /// $p^{\mathrm{phys}}_v$.
    pub physical_properties: PhysicalProperties,
}

impl PhysicalCandidate {
    /// Whether a consumer running in `context` can read this producer (the
    /// concepts doc's alignment table, section 2). True for $\ell$ = None,
    /// and for `OneEvaluation` in the same context. A chain producer with
    /// $\ell$ = `AcrossEvaluations`, with $w$ the chain operator's width in
    /// its expression in `memo`, needs every evaluation time of the context on its
    /// grid: $w$ divides the context's evaluation grid (interval, range step
    /// and phase), each scheduled time, or a one-off query's time; false
    /// when the times are not known (an interval without a phase, an
    /// estimated rate, an unknown recurrence).
    pub fn delivers_on(&self, memo: &Memo, context: &ExecutionContext) -> bool;
}

/// $\alpha^{\mathrm{phys}}_{u,c,i}$: timing and lifetime only (the concepts
/// doc's table, section 2). Alignment is checked per context in Stage 3 with
/// [`PhysicalCandidate::delivers_on`].
pub fn producer_satisfies_physical_requirements(producer: &PhysicalCandidate, consumer: &PhysicalCandidate) -> bool;

/// $(u, c, i)$ over physical candidates.
pub struct PhysicalEdge {
    pub producer_candidate_id: PhysicalCandidateId,
    pub consumer_candidate_id: PhysicalCandidateId,
    pub input_index_of_consumer: usize,
    /// $\alpha^{\log}_{u,c,i}$.
    pub satisfies_logical_requirement: bool,
    /// $\alpha^{\mathrm{phys}}_{u,c,i}$.
    pub satisfies_physical_requirement: bool,
}

/// Stage 2's output, which Stage 3 selects from.
pub struct PhysicalASAPMemoCandidates {
    /// Pass 2's output: the memo, its candidates' $e_v$, $p^{\log}_v$,
    /// $r^{\log}_{c,i}$, and the logical edges.
    pub logical_candidates: LogicalASAPMemoCandidates,
    /// $\mathcal C$, indexed by [`PhysicalCandidateId`].
    pub physical_candidates: Vec<PhysicalCandidate>,
    /// The alternatives Stage 2 pruned, each as the `PhysicalCandidate` it
    /// would have been. // was PrunedCandidate<PhysicalCandidateKey>; query_roots deleted (read logical_candidates.memo.query_roots)
    pub pruned_candidates: Vec<PrunedCandidate<PhysicalCandidate>>,
    /// $\mathcal E$, grouped by (consumer, input); indexed by
    /// [`PhysicalEdgeId`].
    edges: Vec<PhysicalEdge>,
    /// $P_{c,i}$ after $\alpha$: the admissible edges into each (consumer, input).
    admissible_edges_into_input: HashMap<(PhysicalCandidateId, usize), Vec<PhysicalEdgeId>>,
    /// The admissible edges out of each candidate, by [`PhysicalCandidateId`].
    admissible_edges_out_of_candidate: Vec<Vec<PhysicalEdgeId>>,
}

impl PhysicalASAPMemoCandidates {
    /// Stage 2's output over `physical_candidates` and `edges` (grouped by
    /// (consumer, input)), with the admissible-edge indexes built once.
    pub fn new(
        logical_candidates: LogicalASAPMemoCandidates,
        physical_candidates: Vec<PhysicalCandidate>,
        edges: Vec<PhysicalEdge>,
        pruned_candidates: Vec<PrunedCandidate<PhysicalCandidate>>,
    ) -> Self; // 2026-11-07: query_roots dropped, pruned_candidates added

    /// $\mathcal E$: every edge, in (consumer, input) order.
    pub fn edges(&self) -> impl Iterator<Item = (PhysicalEdgeId, PhysicalEdge)> + '_;

    /// $P^{\mathrm{adm}}_{c,i}$: the admissible edges into input
    /// `input_index_of_consumer` of `consumer`
    /// ($\alpha^{\log} = \alpha^{\mathrm{phys}} = 1$).
    pub fn admissible_edges_into(
        &self,
        consumer: PhysicalCandidateId,
        input_index_of_consumer: usize,
    ) -> &[PhysicalEdgeId];

    /// The admissible edges out of `producer`: every consumer that may share it.
    pub fn admissible_edges_out_of(&self, producer: PhysicalCandidateId) -> &[PhysicalEdgeId];
}

/// The output references of `query_roots`: each operator reference, in
/// order (a scalar root's in `map_operator_refs` order), with the index of
/// the query whose root it is.
pub fn output_references<C: Clone>(query_roots: &[QueryRoot<C>]) -> Vec<(usize, C)>;

crates/types/src/stages/realizer.rs

/// Why a selected assignment could not be realized. Only rejected
/// assignments are safe to exclude from the finite search; adapter or
/// identity failures must stop with diagnostics, not become no-good cuts.
pub enum PhysicalCandidateError {
    #[error("selected physical assignment is infeasible: {0}")]
    RejectedAssignment(String),
    #[error("invalid or unsupported physical reconstruction: {0}")]
    InvalidOrUnsupported(String),
}

impl From<String> for PhysicalCandidateError {
    fn from(message: String) -> Self;
}
impl From<&str> for PhysicalCandidateError {
    fn from(message: &str) -> Self;
}

/// A realized selection (was `SelectedPhysicalCandidate`): the physical
/// plan Stage 2 built for it, how long each materialized tumbling-window chain of that plan
/// keeps its tumbling windows, and the operator node each selected node became.
pub struct RealizedPlan {
    /// The selection rebuilt as one physical plan, one operator node per
    /// selected node, each with its candidate's timing and lifetime.
    pub plan: PhysicalWorkloadPlan,
    /// For each tumbling-window chain of `plan` materialized beyond one evaluation
    /// (maintained at ingestion time, or built at query time and materialized
    /// across evaluations), by node id in `plan.dag`: how far back from the
    /// newest evaluation the runtime must keep its completed tumbling windows, in
    /// milliseconds. It is how far its tumbling windows reach back: the offset its
    /// newest tumbling window ends before the evaluation (`[60m] offset 10m`: 10 min),
    /// plus the chain's window, plus the longest range-evaluation span of
    /// its queries, so the earliest step of the next request still finds
    /// its tumbling windows; without an offset or range evaluations, just the window.
    /// Empty when the plan keeps no tumbling windows.
    pub tumbling_window_retention_ms: BTreeMap<PhysicalASAPNodeId, u64>,
    /// The operator node of `plan` each selected node became, indexed by
    /// [`SelectedNodeId`](super::SelectedNodeId).
    pub operator_node_of_selected_node: Vec<Rc<OperatorNode>>,
}

/// Realizes a [`DagOfSelectedCandidates`] through Stage 2, one plan node
/// per selected node. Implemented by the
/// Stage 2 crate, which holds the planning context (the workload's
/// original query roots, its entries and data, and whether query-time
/// results are retained); the facade builds it, passes it to Stage 3, and
/// calls the same method to build the selected plan.
///
/// Guarantees: a returned result has one operator node per selected node, each
/// rebuilt over its selected producers with its identity kept, guarantees
/// recomposed, and every output contract (result kind and schema) met; a
/// realized assignment also passes Stage 2's timing and dependency rules
/// and the physical DAG validator, with every node keeping its
/// candidate's timing and lifetime.
///
/// It checks structure and execution legality only. It does not check cost,
/// and it does not check accuracy targets: Stage 3 does both.
pub trait AssignmentRealizer {
    fn realize(
        &self,
        physical: &PhysicalASAPMemoCandidates,
        selected: &DagOfSelectedCandidates,
    ) -> Result<RealizedPlan, PhysicalCandidateError>;
}

crates/types/src/stages/physical/execution_context.rs (new)

/// A refresh window of the workload's ingestion pipelines, by index.
pub struct RefreshWindowId(pub usize);

/// Index of an [`ExecutionContext`] in the workload's list of distinct
/// contexts. Contexts are deduplicated by equality: [`QueryRecurrence`]
/// holds an `f64` (`EstimatedRate`), so a context cannot be hashed.
pub struct ExecutionContextId(pub usize);

/// $\kappa$: a class of executions with a common owner and schedule, where
/// a candidate with $\ell$ = None or `OneEvaluation` runs.
pub enum ExecutionContext {
    /// The evaluation of every query with this recurrence and range
    /// evaluation, the workload's own description of when it runs: queries
    /// with equal ones are evaluated together and share its `OneEvaluation`
    /// results. A and B both every 10 s at phase 0 give one context; a
    /// one-off query's context is its `OneTime` recurrence.
    Evaluation {
        recurrence: QueryRecurrence,
        range_evaluation: Option<RangeEvaluation>,
    },
    /// $I_r$: the ingestion pipeline of a refresh window.
    IngestionPipeline(RefreshWindowId),
    /// $\kappa_m$: inside an `AcrossEvaluations` candidate's own computation.
    Materialized(PhysicalCandidateId),
}

crates/plan-selection/src/milp/model.rs

/// A query, by index in the workload.
pub struct QueryId(pub usize);

/// The baseline variables (section 3).
struct SelectionVariables {
    /// $x_v$, by [`PhysicalCandidateId`].
    x: Vec<Variable>,
    /// $y_{u,c,i}$, on admissible edges only.
    y: HashMap<PhysicalEdgeId, Variable>,
    /// $z_{q,v}$, for $v \in G_q$.
    z: HashMap<(QueryId, PhysicalCandidateId), Variable>,
    /// $t_v$, by [`PhysicalCandidateId`].
    t: Vec<Variable>,
}

/// The execution-context variables (section 5), which replace $x$ and the
/// unindexed $y$ of the baseline.
struct ExecutionContextVariables {
    /// $x^{\mathrm{mat}}_v$, for an `AcrossEvaluations` candidate.
    x_materialized: HashMap<PhysicalCandidateId, Variable>,
    /// $x_{\kappa,v}$, for a candidate with $\ell$ = None or `OneEvaluation`.
    x_in_execution_context: HashMap<(ExecutionContextId, PhysicalCandidateId), Variable>,
    /// $y_{\kappa,u,c,i}$, indexed by the consumer's context.
    y_in_execution_context: HashMap<(ExecutionContextId, PhysicalEdgeId), Variable>,
    /// $h_v$, for an `AcrossEvaluations` producer.
    retention_amount: HashMap<PhysicalCandidateId, Variable>,
}

/// The accuracy variables of constraint (K) (section 4), all continuous.
struct AccuracyVariables {
    /// $E_c$, by [`PhysicalCandidateId`].
    error_bound: Vec<Variable>,
    /// $F_{c,i}$: the chosen producer's error bound on input $i$ of $c$.
    error_bound_of_input: HashMap<(PhysicalCandidateId, usize), Variable>,
    /// $D_c$, by [`PhysicalCandidateId`].
    failure_probability: Vec<Variable>,
    /// $H_{c,i}$: the chosen producer's failure probability on input $i$ of $c$.
    failure_probability_of_input: HashMap<(PhysicalCandidateId, usize), Variable>,
}

/// (A) $\sum_{u \in P_{c,i}} y_{u,c,i} = x_c$.
fn input_satisfaction_constraints(
    physical: &PhysicalASAPMemoCandidates,
    variables: &SelectionVariables,
) -> Vec<Constraint>;

/// (B) $y_{u,c,i} \le x_u$.
fn producer_selection_constraints(
    physical: &PhysicalASAPMemoCandidates,
    variables: &SelectionVariables,
) -> Vec<Constraint>;

/// (C) $\sum_{v \in G_q} z_{q,v} = 1$; (D) $z_{q,v} \le x_v$.
fn query_coverage_constraints(
    physical: &PhysicalASAPMemoCandidates,
    variables: &SelectionVariables,
) -> Vec<Constraint>;

/// (E) $t_u + 1 \le t_c + |\mathcal C|(1 - y_{u,c,i})$.
fn acyclicity_constraints(
    physical: &PhysicalASAPMemoCandidates,
    variables: &SelectionVariables,
) -> Vec<Constraint>;

/// (F) $x_v \le \sum_{(v,c,i)} y_{v,c,i} + \sum_q z_{q,v}$.
fn utilization_constraints(
    physical: &PhysicalASAPMemoCandidates,
    variables: &SelectionVariables,
) -> Vec<Constraint>;

/// (R) $h_v \ge \operatorname{need}_{\kappa,v,c,i}\, y_{\kappa,v,c,i}$.
///
/// Assumptions (decided 2026-11-06): an edge's need is the time horizon
/// the queries it serves read, a constant per edge (a 5m range over 10 s
/// windows needs 30 windows), so (R) is linear; query latency is ignored
/// until the latency design: a result counts as read when its evaluation
/// starts.
fn retention_constraints(
    physical: &PhysicalASAPMemoCandidates,
    variables: &ExecutionContextVariables,
) -> Vec<Constraint>;

/// (K): $E_c$ and $D_c$ per candidate, big-M on selected edges, and the
/// root checks $E_v \le \varepsilon_q + M_q(1 - z_{q,v})$,
/// $D_v \le \delta_q + (1 - z_{q,v})$.
fn accuracy_constraints(
    physical: &PhysicalASAPMemoCandidates,
    selection: &SelectionVariables,
    accuracy: &AccuracyVariables,
) -> Vec<Constraint>;

crates/logical-asap-optimization/src/pass2/memo/rules.rs

pub(crate) mod finer_aggregate;
pub(crate) mod rollup;
pub(crate) mod scan_labels;
pub(crate) mod summary_inputs;
pub(crate) mod topk;

/// Why a Pass 2 rule failed on a memo.
pub enum Pass2RuleError {
    #[error("memo contains an out-of-range group reference")]
    InvalidGroupReference,
    #[error("memo group capacity exceeds the group identifier range")]
    TooManyGroups,
    #[error("memo dependencies contain a cycle")]
    CyclicDependencies,
}

crates/types/src/ir/schema/mod.rs (excerpt)

/// What a schema field carries (the type stays).
pub enum FieldDataType {
    /// An ordinary, readable value.
    Plain(DataType), // unchanged
    /// Exact, mergeable accumulator state.
    ExactAggregate(ExactKind, ExactParams), // unchanged
    /// Approximate sketch state: its family and grouping layout. θ is a
    /// candidate's `LogicalProperties::parameters`.
    Sketch(SketchAlgorithm, GroupingStrategy), // was Sketch(SketchKind, GroupingStrategy)
    /// A summary sample (e.g. a reservoir).
    SummarySample(SamplingKind, SamplingParams), // unchanged
    /// Wavelet-transform summary state.
    Wavelet(WaveletKind, WaveletParams), // unchanged
    /// Fitted statistical-model summary state.
    StatModel(StatModelKind, StatModelParams), // unchanged
}

crates/types/src/ir/physical_export.rs (excerpt)

pub struct PhysicalASAPDAGNode {
    // id, payload, output_state, lifetime, output_schema, guarantee, retention_ms: unchanged
    /// θ of a summary build: its candidate's `LogicalProperties::parameters`.
    /// `None` on every other node and for an exact build. Filled in PR 6.
    #[serde(default, skip_serializing_if = "Option::is_none")]
    pub parameters: Option<SketchParams>, // new
}
pub const INSTALLABLE_PLAN_WIRE_VERSION: u32 = 13; // was 11 (in crates/planner/src/installable.rs)

crates/types/src/ir/schema/sketch.rs (excerpt)

pub enum SketchParams {
    // Kll, Cms, CountSketch, …: unchanged
    Hydra(HydraParams), // new: Hydra's sizes are θ
}
pub enum GroupingStrategy {
    PerSubpopulationInstance,
    SharedMultiSubpopulation { kind: HydraKind }, // was { kind, params: HydraParams }
}
impl HydraParams {
    pub fn per_subpopulation(&self) -> SketchParams; // new
}

crates/executor/src/summary_kernels/factory.rs and crates/types/src/test_support.rs (excerpts)

pub fn create_planner_accumulator(
    family: &SummaryFamilyType,
    parameters: Option<&SketchParams>, // new: the exported node's θ
    input: &SummaryUpdate,
    grouping: &GroupingStrategy,
) -> Result<Box<dyn AccumulatorUpdater>, String>;

// feature `test-support`: every node at query time, Lifetime::None, parameters None
pub fn physical_asap_dag_at_query_time(root: &Rc<OperatorNode>)
    -> Result<PhysicalASAPDAG, PhysicalASAPDAGValidationError>;
pub fn physical_asap_dag_at_query_time_with_node_ids(roots: &[QueryRoot])
    -> Result<(PhysicalASAPDAG, Vec<Rc<OperatorNode>>), PhysicalASAPDAGValidationError>;

Ambiguities and questions (answered)

The 2026-11-06 answers are applied above: θ out of the operator, Pass 1 returns Memo, export via realize, memo_of_queries, x_materialized, Hydra sizes as θ, θ on the wire, the Pass 1 renames, deleting expand_tumbling_window_chains_in_workload, and tests that build PhysicalASAPDAG directly. The schedule σ is dropped (decided 2026-11-06): Schedule and its error variants are deleted, producer_satisfies_physical_requirements checks timing x lifetime only, and delivers_on is Stage 3's per-context alignment check. Also decided 2026-11-06: the renames agreed in review (#1224): LogicalPropertiesError, PhysicalPropertiesError, LogicalRequirements (parameter_floor_for_required_accuracy), producer_satisfies_logical_requirements, producer_satisfies_physical_requirements, StorageMedium, ExecutionContext/ExecutionContextId (x_in_execution_context, y_in_execution_context), and the fields logical_candidate_id, physical_properties, producer_candidate_id, consumer_candidate_id, logical_candidates, physical_candidates; a sketch build needs θ from its family's list; a requirement's floor may be outside the list; producer_satisfies_physical_requirements takes only the producer and consumer; an evaluation context is the workload's recurrence and range evaluation (ExecutionContext::Evaluation { recurrence, range_evaluation }), one-off queries included. No open questions; the items marked "pending" in the table above await approval.

Moved from the PR body (2026-11-07, body size limit)

crates/types/src/ir/operator/input.rs (new)

/// An input of an operator, as the rules read it.
pub trait OperatorInput: Sized + Clone {
    fn schema(&self) -> &Schema;
    fn result_kind(&self) -> OperatorResultKind;
    /// The input's own operator, its children as inputs again.
    fn operator(&self) -> Operator<Self>;
    /// What the input covers when it is an aggregate group in the memo;
    /// `None` for any other input. // renamed 2026-11-07
    fn aggregate_coverage(&self) -> Option<&SummaryCoverage>;
}

/// A front-end node tree. Node trees carry no summary coverage.
impl OperatorInput for Rc<OperatorNode> { … }

/// An input of a rule: a real input `I`, or a build the rule constructs
/// only while it derives.
pub(crate) enum InputOrTemporaryBuild<I> {
    Input(I),
    TemporaryBuild(Rc<TemporaryBuild<I>>),
}

/// A build a rule constructs only while it derives (the build a rollup stands for, the build a selection of
/// states stands for): an operator over `I`s with its derived schema and
/// kind. Never stored in the memo. A rule never builds one over another
/// rule's temporary build, so the type does not nest.
pub(crate) struct TemporaryBuild<I> {
    operator: Operator<InputOrTemporaryBuild<I>>,
    schema: Schema,
    result_kind: OperatorResultKind,
}
impl<I: OperatorInput> OperatorInput for InputOrTemporaryBuild<I> { … }

crates/types/src/ir/operator/node.rs (excerpt)

/// The derivation rules, one set for every kind of input. // was impl Operator (node trees only)
impl<I: OperatorInput> Operator<I> {
    pub(crate) fn output_schema(&self) -> Result<Schema, SchemaDerivationError>;
    pub(crate) fn output_kind(&self) -> OperatorResultKind;
    pub(crate) fn validate_inputs(&self) -> Result<(), SchemaDerivationError>;
    pub(crate) fn is_shareable_when_equal(&self) -> bool; // new: an ASAP operator, or NonASAPOp::is_shareable_when_equal
}
// The same for NonASAPOp<I>, ASAPOp<I> and ScalarExpr<C>::scalar_type.

crates/types/src/ir/properties/summary_coverage.rs (excerpt)

/// What a summary state computes, with every lifted selection removed. // new
pub struct SummaryStateDefinition {
    /// The group the unlifted operators read: the first input under the
    /// SummaryAgg that is not Filter / range TimeRange / TimeShift without @
    /// / direct-column Project. `None` when that input is a `Scan` with a
    /// liftable predicate (a PromQL label matcher): it is lifted too, and the
    /// `Scan` with the rest of its predicates is the last unlifted operator,
    /// so states over one source with different matchers may merge.
    pub input_group: Option<MemoGroupId>,
    /// The operators not lifted into the selection, listed top down from the
    /// SummaryAgg, each with only its residual arguments. `None` marks an
    /// operator's main input (the next unlifted operator, or input_group);
    /// `Some(g)` is another input group it reads, such as the group a scalar
    /// subquery in a Filter reads. Two states may merge only if input_group
    /// and these operators are equal.
    pub unlifted_operators: Vec<Operator<Option<MemoGroupId>>>,
}

pub struct SummaryCoverage {
    pub(crate) definition: SummaryStateDefinition, // was Rc<OperatorNode>
    /// Union of boxes over the output rows of the unlifted operators' input.
    pub(crate) selection: Vec<SelectionBox>,
}

impl SummaryCoverage {
    /// The coverage of a summary state a memo group computes with `operator`
    /// over its input groups. // was derive(node), derive_cached(node, cache)
    pub(crate) fn derive_from_inputs<'m>(
        operator: &Operator<MemoGroupAsInput<'m>>,
    ) -> Result<Self, CoverageError>;
    /// 128-bit FNV-1a of the definition; equal definitions of one memo get
    /// equal ids.
    pub(crate) fn definition_id(&self) -> String;
}

/// A conjunction of per-column constraints. The timestamp column is one of
/// the columns; there is no separate time field. // 2026-11-07: relative_time deleted
pub struct SelectionBox {
    pub(crate) columns: BTreeMap<ColumnIdentity, Constraint>,
}

pub enum Constraint {
    In(Vec<ScalarValue>),
    NotIn(Vec<ScalarValue>),
    /// `m[5m]`: (Excluded(EvaluationTimePlusMs(-300_000)), Included(EvaluationTimePlusMs(0))]; // 2026-11-08: (t − r, t]
    /// `price > 10`: lower Excluded(FixedValue(Float64(10.0))), upper Unbounded. // 2026-11-07
    Interval { lower: Bound<IntervalConstraintLimit>, upper: Bound<IntervalConstraintLimit> },
}

/// An end of an interval; limits compare only within one variant.
/// // new 2026-11-07, replaces TimeBound
pub enum IntervalConstraintLimit {
    FixedValue(ScalarValue),      // `@ T`, SQL timestamp literals: Timestamp(ms since the epoch)
    EvaluationTimePlusMs(i64),    // relative to the evaluation time t
}

pub enum CoverageError {
    // … unchanged variants; TumblingWindowChainExpansion deleted 2026-11-07
}

Major algorithm change: summary coverage over memo groups (2026-11-06)

Summary coverage over memo groups.

  • Problem. Coverage was derived from a node tree, so a memo group needed a rebuilt tree to know what its state covers, and a merge check walked the whole input tree.
  • Before. SummaryCoverage::derive(node) walked down from the SummaryAgg node, rebuilt the residual nodes into a definition: Rc<OperatorNode>, and cached coverage per node address (NodeToSummaryCoverageCache); OperatorNode::new checked every SummaryMerge.
  • After. derive_from_inputs(operator over MemoGroupAsInputs) runs the same walk over the input groups' expressions (OperatorInput::operator()), the same lifting rules (residual, constraint_of, column_at_top, time windows), and records the definition as input_group plus the unlifted operators (Operator<Option<MemoGroupId>>: None for each operator's main input, Some(g) for a group a scalar subquery reads). A merge reads its inputs' stored coverage and compares definitions by value. A chain, a merge of newest windows, a rollup and a selection of states use the same builds as before, as InputOrTemporaryBuild::TemporaryBuilds.
  • Correctness. Equal definitions read one group through equal operators, so they compute the same thing; the selection rules are unchanged. Kept behaviors: Scan predicates (PromQL matchers) are lifted, so matcher states still merge. PR 2's tests check the doc's examples.
  • Complexity. The walk visits only the liftable operators below the SummaryAgg; a merge compares n definitions in O(n · |unlifted_operators|) instead of walking node trees.

Other algorithms are deleted (see "Deleted"); later PRs re-implement them behind these interfaces.

Deleted tests, 2026-11-06 detail

Deleted with node-tree coverage, the node-tree merged_window_build and size_for_all_readers (decided 2026-11-06), 48 more:

File Tests Restored by
types/tests/summary_coverage.rs (file deleted) 32: coverage derived on node trees (lifting, merges, time windows, rollups, definition ids) PR 2's types/tests/memo_summary_coverage.rs (8) over memo groups; the rest as later PRs need them
integration-tests/tests/summary_coverage_e2e.rs (file deleted) 8: query string → planned state → coverage() no replacement planned: it read node-tree coverage of front-end trees
types/tests/summary_merge_structure.rs idempotent_families_merge_overlapping_states (node merge check) PR 2's states_with_overlapping_selections_do_not_merge
types/tests/tumbling_window_chain.rs a_rollup_of_the_newest_tumbling_windows_covers_their_window; the coverage asserts of 5 more PR 2's a_tumbling_window_chain_covers_its_whole_window
types/tests/tumbling_window_chain.rs a_merge_of_the_newest_tumbling_windows_covers_their_window (read the build through the node-tree merged_window_build) PR 4, with the merged-windows guarantee over groups
devtools/src/bin/asap_planner.rs memo_node_table_stores_a_shared_node_once (with MemoNodes::add) PR 9
accuracy/src/estimators/mod.rs, mod shared_size_tests 4: size_for_all_readers no replacement: deleted with the function

The coverage asserts in integration-tests/tests/sketch_rollup.rs (2 tests) and logical-asap-optimization's per_computation_candidates_tests.rs (1 test) are removed; the tests stay.

Kept by building the DAG directly (decided 2026-11-06). Tests that hand-built operator roots and ran them now export them with the test-only physical_asap_dag_at_query_time(_with_node_ids) instead of the deleted roots-based export. Restored: executor time_series_window, node_ids, current_series_heap, planspace_series_identity_heap, 3 of deployment_computation's tests; integration filtered_aggregates, kll_tumbling_window_execution, per_series_exact_execution, sql_cardinality, sql_frequency, exact_composition, and parts of sketch_rollup, stage1_logical_asap_optimization, operator_design_examples, operator_sharing, promql_numeric_regressions, sql_to_post_asap, promql_to_post_asap; types physical_export; frontend count_planning, pearson_corr. types/tests/physical_export.rs::untimed_plan_is_rejected (the +1 above, deleted after the comment was posted) is deleted: the builder always assigns query time, and an untimed export is no longer possible. Restored sketch tests fail at the executor's todo!() until PR 6, because the builder leaves parameters: None. Tests whose DAG came from the planner (runtime_install's planning tests, sql_per_group_topk, sql_table_scans, per_sample_updater, the planner-built deployment_computation tests, and others) stay deleted; PR 6 restores them.

Moved from the PR body (2026-11-08, body size limit)

2026-11-07 round, 10 more:

File Tests Restored by
types/tests/tumbling_window_chain.rs 6: merged_tumbling_windows_cover_the_whole_window, each_tumbling_window_is_one_shifted_minute, expansion_merges_the_tumbling_windows, a_merge_of_the_newest_tumbling_windows_reads_one_chain_and_fewer_than_all_tumbling_windows, newest_tumbling_windows_serializes_only_when_set, expansion_keeps_a_rollup_of_the_tumbling_windows (window count, newest windows, expansion) PR 1 (TimeRange over a chain), PR 6 (executor by window index)
types/src/ir/properties/summary_coverage.rs 3: large_reverse_ordered_tumbling_window_union_is_one_exact_interval, interval_fast_path_matches_general_union_for_permutations, sorted_union_preserves_gaps_boundaries_and_original_group_order (relative_time union) PR 2 (timestamp-column intervals)
planner/tests/runtime_install.rs tumbling_windows_of_an_hour_share_one_definition_with_distinct_times (expanded windows' relative_time) and its now-unused helpers plan_json, plan_json_for PR 6

…perators; delete merged_window_build_node (#1207 PR 0)

- BuiltOrInput / BuiltInput renamed InputOrTemporaryBuild / TemporaryBuild:
  a build a rule constructs only while it derives, never stored in the memo.
- SummaryStateDefinition::unlifted_operators is Vec<Operator<Option<MemoGroupId>>>:
  None marks an operator's main input, Some(g) another input group it reads
  (a scalar subquery's). States whose definitions read subqueries compare and
  merge; CoverageError::SubqueryInDefinition deleted.
- merged_window_build_node deleted with its test; the accuracy guarantee of a
  read of merged tumbling windows is todo!() until PR 4 rewrites it over groups.
…, TimeBound, Pass 2 pruned list, Stage 2 entry (#1207 PR 0)

- Delete WindowLayout, LogicalProperties::window_layout and its two error
  variants, SummaryMerge::newest_tumbling_windows and
  SummaryTumblingWindowChain::tumbling_window_count, with the count-based
  chain builds and expansion; behaviour that read the count is todo!() for
  PR 1, 2, 6, 7 or 9.
- SelectionBox { columns } only, and TimeBound { At, FromEvaluationTime };
  lifting TimeRange/TimeShift is todo!() for PR 2.
- PrunedLogicalCandidate, PruneCause, RootPruneReason and
  LogicalASAPMemoCandidates::pruned.
- error_propagation(consumer, metric) -> Option<ErrorPropagation>;
  Memo::set_input_requirements.
- PhysicalCandidateKey deleted (PrunedCandidate<PhysicalCandidate>),
  PrunedCandidate::group and PhysicalASAPMemoCandidates::query_roots
  dropped; new(logical, physical, edges, pruned).
- Stage 2 entry takes &DeploymentCapabilities (moved into asap-types) and
  cannot fail; PruneReason follows the doc's Stage 2 table.
- DataArrival::{Ingesting, Mixed} gain max_out_of_order_ms.
- GroupProperties::summary_coverage -> aggregate_coverage.
@zzylol
zzylol force-pushed the refactor/1207-interfaces branch from 633460c to 22caa51 Compare November 7, 2026 17:16
…l; ScalarValue::Timestamp (#1207 PR 0)

Replace TimeBound by IntervalConstraintLimit { FixedValue(ScalarValue),
EvaluationTimePlusMs(i64) }: Constraint::Interval holds
Bound<IntervalConstraintLimit> on every column, so the timestamp column
and value columns share one interval type. Doc comments carry the
examples (m[5m], offset 1h, @ T, SQL ts >= literal, price > 10).
ScalarValue::Timestamp(ms since the epoch) is the fixed value of an
@ T or a SQL timestamp literal.
…s; delete ParametersOfAnotherFamily (#1207 PR 0)

A range m[r] evaluated at t reads (t - r, t]: IntervalConstraintLimit and
Constraint::Interval examples, the chain window doc and the primitive-schema
table now use Excluded lower and Included upper ends. A parameter list holds
only its own family's parameters, so ParametersNotInList also rejects another
family's parameters.
…acy, the plan is one PhysicalASAPDAG (#1207 PR 0)

- Terminology: ops' `family` is `summary_type`; FieldDataType::Sketch(SketchAlgorithm)
  (SketchKind, GroupingStrategy, HydraKind and the ops' `grouping` deleted);
  FamilyCapability is SummaryTypeCapability; parameter_list(algorithm).
- θ is structure only: SketchParams derives equality; KLL failure_probability,
  UnivMon distinct_failure_probability and HLL/KMV relative_error deleted.
- ValueSign and GroupProperties::column_sign; SourceFacts::column_signs
  replaces values_non_negative.
- LogicalRequirements { required_accuracy: Option<RequiredAccuracy> };
  RootPruneReason::BelowRequiredAccuracy; Memo::add_query takes the target
  and n_q; AggIntent measures carry no accuracy and front ends take none.
- asap-accuracy: AccuracyEvidence::measured_guarantee, theoretical_guarantee
  (was local_guarantee), producer_satisfies_logical_requirements(memo,
  producer, consumer, input_index, evidence); the AccuracyModel trait and
  evidence provider, Pass 1's accuracy pruning and AccuracyCheckInputs deleted.
- Physical: ExecutionContext::Evaluation { grid: EvaluationGrid },
  IngestionPipeline(RefreshWindow); RefreshWindowId deleted;
  producer_satisfies_physical_requirements(producer, consumer_timing);
  PruneReason::CountMinWeightsNotProvenNonNegative.
- Plan: PlanOutput { plan: PhysicalASAPDAG }; PhysicalASAPDAGNode::
  selected_physical_candidate; select_plan returns (PhysicalASAPDAG,
  Selection); Selection { accepted_accuracy_per_query, pruned_candidates,
  milp_* }; RealizedPlan, PhysicalWorkloadPlan, DagOfSelectedCandidates,
  SelectedEdge, SelectedNodeId, AssignmentRealizer, ScanWindow and the
  scan/view window functions, SearchReport, SearchStatus, InexactObjective,
  ObjectiveDiffersFromPlanCost, SelectedPlanFailedCheck and
  PlanCost::cost_unit deleted.
- Plan file wire version 14 (summary_type, Sketch(algorithm), no grouping).
- Cost names: NodeCostSource, cost_source, PlanRanking::Cost; Serialize on
  PruneReason, PhysicalProperties, OriginOfCandidateForMemoGroup and
  RootPruneReason.
@zzylol

zzylol commented Nov 8, 2026

Copy link
Copy Markdown
Contributor Author

PR 0 body as of 7e83db9 (before the 2026-11-08 interface round), kept here so the PR body fits. Items this round changed are described in the body; where they differ, the body wins.

Merge unit. This is PR 0 of #1207's merge unit (PRs 0–9). The PRs merge into main together, after PR 9. Each one is stacked on the one before it for review. This PR compiles, but planning panics at the first todo!(). Tests that reach a todo!() fail until the PR that fills it in (see "Test status").

Why

The memo, the properties, the edges and the Stage 3 variables of the optimization formulation have no named home in today's code:

  • a candidate's size lives inside its private root_node chain, sized for a budget share (accuracy_budget_parts);
  • timing and lifetime are folded into one Materialization enum;
  • edges are implicit in ProducerCandidates sets;
  • Stage 3 has no $y$.

#1207's new order (rewritten 2026-11-06) is interfaces first. This PR defines every data structure and interface of the concepts doc memo-groups-properties-requirements.md (sections 3–5, #1224 at 206d84b), with todo!() bodies. It also deletes the old interfaces and the code they replace. Each later PR fills in one stage behind these interfaces, so no PR needs an adapter between an old interface and a new one.

What

  • Types (asap-types). It adds every type and signature of the doc's section 3, and the section 4–5 types that Stage 3 uses:

    • one Memo, with candidates in MemoGroup::candidates;
    • CandidateExpression, LogicalProperties { parameters }, LogicalRequirements, ErrorPropagation, parameter_list, producer_satisfies_logical_requirements, Memo::set_input_requirements;
    • LogicalEdge, LogicalASAPMemoCandidates { memo, edges, pruned }, PrunedLogicalCandidate, PruneCause, RootPruneReason;
    • SelectionBox { columns }, Constraint::Interval { lower, upper: Bound<IntervalConstraintLimit> } and IntervalConstraintLimit { FixedValue(ScalarValue), EvaluationTimePlusMs(i64) }, with ScalarValue::Timestamp(ms);
    • PhysicalProperties { timing, lifetime }, Lifetime, StorageMedium. There is no schedule property σ (decided 2026-11-06): a None or OneEvaluation result runs when the context that runs it runs, and every tumbling grid is anchored at phase 0;
    • PhysicalCandidate { logical_candidate_id, physical_properties }, PhysicalCandidate::delivers_on(memo, context), producer_satisfies_physical_requirements;
    • ExecutionContext, ExecutionContextId, RefreshWindowId (in asap_types::stages, so delivers_on can take the context);
    • PhysicalEdge, PhysicalASAPMemoCandidates::{new, edges, admissible_edges_into, admissible_edges_out_of}, PrunedCandidate<PhysicalCandidate>;
    • SelectedEdge::input_index_of_consumer.

    Accessors are real code. Every function the doc assigns to PRs 1–8 is todo!().

  • Stage 0 is memo_of_queries (was generate_dag_of_logical_memo_groups). It returns Result<Memo, SchemaDerivationError>, a Memo whose candidate lists are empty; DagOfLogicalMemoGroups is deleted. Its body is todo!() until PR 2, which builds the groups through Memo::find_or_add_group; the old node-level hash-consing (stage0/cse.rs), its test_support::share_common_sub_dags export and the unused share_identical test helper are deleted. The facade propagates the error (run_planning with ?; devtools' run_stages_until as PlanningStagesError::Validation; sketch_coverage counts the root as rejected).

  • The memo (decided 2026-11-06). A MemoGroup is its expression and its properties: MemoGroup { expression: Operator<MemoGroupId>, group_properties: GroupProperties, parents, queries, candidates }, where each id in expression is an input group; logical_node and child_groups are deleted. GroupProperties { schema, result_kind, aggregate_coverage, shareable_when_equal } is derived once, when the group is created, by derive_group_properties(memo, expression). Memo::groups is private, read through groups(), so groups are added only through find_or_add_group. find_or_add_group returns Result<MemoGroupId, SchemaDerivationError> and add_candidate returns Result<CandidateForMemoGroupId, CandidateError>; both derive the properties, then reach todo!() (PR 2: hash-consing and storage). add_query adds a query's root (todo!() until PR 2). Memo::logical_node_trees is deleted.

  • One set of rules, read through OperatorInput (decided 2026-11-06). The existing derivation rules (output_schema, output_kind, validate_inputs, is_shareable_when_equal, summary_build, scalar subqueries, top-k, populations) are generic over trait OperatorInput { schema(), result_kind(), operator(), aggregate_coverage() }, implemented for Rc<OperatorNode> (front-end node trees) and MemoGroupAsInput<'m> { memo, group_id } (reads the input group's group_properties and expression). Only how they read an input changed: child.schema became child.schema(), and so on. The builds a rule makes while deriving (the build a rollup or a selection of states stands for) are InputOrTemporaryBuild::TemporaryBuilds (approved 2026-11-06): a build a rule constructs only while it derives, never stored in the memo.

  • Summary coverage is memo-only (decided 2026-11-06). SummaryCoverage { definition: SummaryStateDefinition { input_group, unlifted_operators }, selection } has no type parameter. SummaryCoverage::derive_from_inputs walks down through the input groups' expressions; find_or_add_group stores the result in GroupProperties::aggregate_coverage (renamed from summary_coverage on 2026-11-07; PR 2 derives it for exact Aggregate groups too), and a SummaryMerge group reads its inputs' stored coverage. Node-tree coverage is deleted: NodeToSummaryCoverageCache, derive_cached, the Rc<OperatorNode> definition, OperatorNode::coverage() and the node-tree merge check (check_merge in OperatorNode::new and validate_structure). PromQL label matchers (Scan predicates) are still lifted, so latency{job="api"} and latency{job="web"} states still merge, as today. unlifted_operators is Vec<Operator<Option<MemoGroupId>>> (decided 2026-11-06): None marks an operator's main input, Some(g) another group it reads, such as a scalar subquery's. So states whose definitions read subqueries compare and merge.

  • todo!() sites that read node trees or node coverage. Pass 1's target discovery (logical_node_trees(memo) in per_computation_candidates.rs, PR 3); devtools' group labels (group_label in planning_stages_text.rs, group_node in asap_planner, PR 9); the export's view coverage (coverage_of in materialized_view.rs, PR 6, which reads the plan's coverage and definition_id); the accuracy guarantee of a read of merged tumbling windows (merged_tumbling_windows_guarantee in accuracy/src/propagation.rs, PR 4, which rewrites local propagation over groups: Tracking: implement the optimization formulation (memo candidates, properties, edges y_{u,c,i}, MILP) as named code per stage #1207's Order gives PR 4 the doc's section 4 rules for Pass 2's edges, with Pass 2: tumbling-window builds as candidates of the state group, and merge-then-aggregate over raw tumbling windows (window-specific; memo representation is #1203) #1160's tumbling layouts; PR 8 only composes the bounds to the roots). No executor code recomputes coverage.

  • θ is out of the operator (decided 2026-11-06). FieldDataType::Sketch is now Sketch(SketchAlgorithm, GroupingStrategy) (was Sketch(SketchKind, GroupingStrategy)).

    • ASAPOp::SummaryAgg's and SummaryTumblingWindowChain's family carries only the family, so k=200 and k=345 hash-cons to one group. θ lives only in LogicalProperties::parameters.
    • Code that read θ off a node reaches todo!() for the PR that brings θ from the candidate: the cost's state shape (PR 6), the executor's sketch factory and checks (PR 6), and the accuracy model's per-node sketch guarantee (PR 4).
    • Hydra's sizes are θ too (decided 2026-11-06). GroupingStrategy::SharedMultiSubpopulation is now { kind: HydraKind } (was { kind, params: HydraParams }), and the sizes are SketchParams::Hydra(HydraParams). Two Hydra sizes hash-cons to one group. HydraParams::per_subpopulation gives the per-subpopulation sketch's θ, which the error model and the state shape read.
    • SketchKind (family + θ) stays as a value type: the executor's sketch factory pairs the family with the node's parameters to configure weighted frequency.
  • Stage 1.

  • Stage 2. generate_physical_asap_memo_candidates(logical, data, capabilities: &DeploymentCapabilities) -> PhysicalASAPMemoCandidates (2026-11-07: cannot fail; DeploymentCapabilities moved into asap_types::deployment_capabilities so the Stage 2 crate can take it) is todo!() (PR 5) and Stage2AssignmentRealizer::realize is todo!() (PR 6). Everything else in the crate (catalog, timing split, reference pricer) is deleted.

  • Stage 3. milp/model.rs holds:

    • the baseline SelectionVariables { x, y, z, t } and the constraint functions (A)–(F);
    • the section 5 ExecutionContextVariables ($x^{\mathrm{mat}}$, $x_\kappa$, $y_\kappa$, $h$) and retention_constraints (R);
    • the section 4 AccuracyVariables ($E$, $F$, $D$, $H$) and accuracy_constraints (K).

    select_plan validates the cost inputs, then reaches todo!() (PR 6). The feat(plan-selection): group selection MILP over execution contexts as Stage 3's default search (#1095) #1144 MILP is deleted, with its groups, cost terms, warm start, completion, decode, LP diagnosis, checks and accuracy binding.

    • Assumptions (decided 2026-11-06), on h / need / (R). An edge's retention need is the time horizon the queries it serves read: a 5m range over 10 s windows needs 30 windows. It is a constant per edge, so (R) is linear. Query latency is ignored until the latency design: a result counts as read when its evaluation starts. PR 7, which implements (R), repeats this.
  • Realizer and export (decided 2026-11-06). The only way to export a plan is AssignmentRealizer::realize(&self, physical: &PhysicalASAPMemoCandidates, selected: &DagOfSelectedCandidates) -> Result<RealizedPlan, PhysicalCandidateError>.

    • RealizedPlan::plan.dag is the PhysicalASAPDAG, built directly, with each node's properties read from its PhysicalCandidate.
    • I used the existing trait method rather than a new free function because it already has this signature. Stage 2's Stage2AssignmentRealizer::realize is todo!() (PR 6).
    • Export from bare roots is deleted: compile_physical_asap_dag, compile_physical_asap_workload(_with_node_ids), PhysicalASAPDAGCompilation, PhysicalASAPNodeIdentityMap, PhysicalASAPDAG::with_every_node_at_query_time, ReferencePlanError, PlanOutput::physical_asap_dag, and asap_planner::test_support::query_time_reference (with the test-support feature).
    • installable_plan_of now takes the &RealizedPlan. Its tumbling-window expansion and plan_installable are todo!() (PR 6).
    • No side map holds per-node physical properties, and ExecutionDataStateError (the timing pass's error) is deleted.
    • The wire node carries lifetime in place of materialization, and θ as PhysicalASAPDAGNode::parameters: Option<SketchParams> (decided 2026-11-06): the build's candidate's LogicalProperties::parameters, None on every other node. PR 0 defines the field; PR 6's realizer fills it. INSTALLABLE_PLAN_WIRE_VERSION is 13. The backend changes with PR 6.
    • The executor's sketch factory reads θ from it: create_planner_accumulator(family, parameters: Option<&SketchParams>, input, grouping), which refuses a sketch build with None. The θ checks and kernels not wired to it yet stay todo!() (PR 6).
    • expand_tumbling_window_chains_in_workload is deleted (decided 2026-11-06), and with the window count (2026-11-07) every count-based chain build and expansion: the executor runs a chain, and a TimeRange/TimeShift over it, by window index (PR 6).
    • Tests build PhysicalASAPDAG directly (decided 2026-11-06). The test-only asap_types::test_support::physical_asap_dag_at_query_time(root) and physical_asap_dag_at_query_time_with_node_ids(roots) export hand-built operator roots with every node at query time, not materialized and with parameters: None. Executor, runtime-install and frontend end-to-end tests use them instead of the deleted roots-based export.
  • 2026-11-07 decisions (chain groups, Pass 2 pruned list, Stage 2 entry).

    • A tumbling-window chain is a range-free group: SummaryTumblingWindowChain loses tumbling_window_count, SummaryMerge loses newest_tumbling_windows, and LogicalProperties loses window_layout (WindowLayout and LogicalPropertiesError::{WindowOnOperatorThatCannotBeWindowed, ZeroWindowWidth} deleted). Code that read the count reaches todo!(): the schema of a finalize or rollup over merged chain windows (PR 1), a chain's coverage (PR 2), the export's scan and view windows and the executor's chain run (PR 6), the chain's cost and retention (PR 7), two fuzz witnesses (PR 9).
    • SelectionBox { columns } only: the timestamp column is one of the columns. An interval's ends are IntervalConstraintLimits (decided 2026-11-07, replacing TimeBound): FixedValue(ScalarValue) (SQL price > 10 → lower Excluded(FixedValue(Float64(10))); m[5m] @ 1700000000 → (FixedValue(Timestamp(1699999700000)), FixedValue(Timestamp(1700000000000))]) or EvaluationTimePlusMs(i64) (m[5m] → (EvaluationTimePlusMs(-300000), EvaluationTimePlusMs(0)]; offset 1h → (-3900000, -3600000]). A PromQL range m[r] at t reads (t − r, t]: left-open, right-closed (corrected 2026-11-08; the first version had the ends the other way round). The doc comments carry these examples. ScalarValue::Timestamp(ms since the epoch) is new, for @ T and SQL timestamp literals (the executor's Value::Timestamp already exists). Lifting TimeRange/TimeShift, and the counter families' disjoint-in-time merge check, are todo!() (PR 2).
    • Pass 2 records dead candidates in LogicalASAPMemoCandidates::pruned (PrunedLogicalCandidate, PruneCause, RootPruneReason; PR 4 fills it). error_propagation(consumer, metric) -> Option<ErrorPropagation>; Memo::set_input_requirements is real code (asserts one requirement per input group).
    • Stage 2: PhysicalCandidateKey, PrunedCandidate::group and PhysicalASAPMemoCandidates::query_roots are deleted; PruneReason is the doc's Stage 2 table (NoIngestionPipeline, RawDataNotRetained, QueryTimeResultsNotRetained, InputHasNoBuildableProducer, ReachesNoQueryOutput) plus Stage 3's two. DataArrival::{Ingesting, Mixed} gain max_out_of_order_ms: Option<u64>.
  • Tools.

Decisions: approved and pending

Each new item, with its status. Approved items were decided by the user; pending items await approval.

parameter_list and PhysicalCandidate::delivers_on are in the doc and approved, so they are not listed here.

Item Where Why it is needed Proposed status
LogicalPropertiesError::ParametersOnExactOperator types/src/stages/candidates.rs The doc names the error type without variants. One per rejection that section 3 lists for LogicalProperties::new. pending (the two window variants deleted 2026-11-07; ParametersOfAnotherFamily deleted 2026-11-08: ParametersNotInList covers it)
LogicalPropertiesError::{SketchBuildWithoutParameters, ParametersNotInList} (ParametersNotInList was FloorNotInParameterList) same A sketch build needs θ, and a candidate's θ must be in parameter_list. approved 2026-11-06
LogicalRequirements::new(parameter_floor_for_required_accuracy) -> Self (was -> Result<Self, LogicalPropertiesError>) same A floor may be any value of the family's parameter type (e.g. heap_size ≥ k), so new has nothing to reject. approved 2026-11-06 (floor outside the list); infallible signature follows from it
PhysicalPropertiesError::OneEvaluationAtIngestion types/src/physical/properties.rs The doc names the error type without variants. This is the one row of section 2's "Invalid combinations" table. pending
QueryId(pub usize) (plan-selection/src/milp/model.rs), RefreshWindowId(pub usize) (types/src/stages/physical/execution_context.rs) as listed SelectionVariables::z ($z_{q,v}$) names QueryId and ExecutionContext names RefreshWindowId, but the doc does not define them. They index the workload's queries and its ingestion refresh windows. pending
ExecutionContext::Evaluation { recurrence: QueryRecurrence, range_evaluation: Option<RangeEvaluation> } (was QueryEvaluation(QueryId)); EvaluationSchedule and evaluation_schedule() deleted types/src/stages/physical/execution_context.rs (moved from plan-selection/src/milp/model.rs) Contexts reuse the workload's own types: queries with equal recurrence and range evaluation are evaluated together and share its OneEvaluation results; one-off queries are supported (ρ_κ = invocations). QueryRecurrence holds an f64, so contexts are deduplicated by equality and ExecutionContextId indexes the distinct contexts; no Hash. Moved to asap_types because PhysicalCandidate::delivers_on takes it. approved 2026-11-06
PhysicalASAPDAGValidationError::PhysicalProperties { node, reason } (was InvalidPhysicalProperties) types/src/ir/physical_export.rs Reads with its PhysicalPropertiesError reason. approved 2026-11-06
ExecutionContextVariables { x_materialized, x_in_execution_context, y_in_execution_context, retention_amount } plan-selection/src/milp/model.rs The doc gives the section 5 variables ($x^{\mathrm{mat}}$, $x_\kappa$, $y_\kappa$, $h$) as a table, not a struct. They are kept apart from the baseline SelectionVariables so that $y$ never has two meanings (section 5). pending
AccuracyVariables { error_bound, error_bound_of_input, failure_probability, failure_probability_of_input } same The doc gives $E_c, F_{c,i}, D_c, H_{c,i}$ (section 4) as formulas, not a struct. pending
retention_constraints, accuracy_constraints and the constraint function signatures (&PhysicalASAPMemoCandidates, &…Variables) -> Vec<good_lp::Constraint> same The doc says "signatures omitted". pending
#![allow(dead_code)] on milp/model.rs same select_plan does not build the MILP until PR 6, so nothing calls these yet. PR 6 removes the allowance. interim, deleted in PR 6–8
PhysicalASAPMemoCandidates::new(logical_candidates, physical_candidates, edges, pruned_candidates) same Query roots are read from logical_candidates. approved 2026-11-07
its private fields edges, admissible_edges_into_input, admissible_edges_out_of_candidate types/src/stages/physical/memo_candidates.rs The doc says "edges stored per (consumer, input); see edges()". The accessors return slices, so the indexes must be stored, and private fields need a constructor. pending
PhysicalCandidateKey, PrunedCandidate::group, PhysicalASAPMemoCandidates::query_roots deleted; PrunedCandidate<PhysicalCandidate> types/src/stages/ A pruned alternative is its PhysicalCandidate value; its group is its logical candidate's. approved 2026-11-07
generate_physical_asap_memo_candidates(logical, data, capabilities: &DeploymentCapabilities) -> PhysicalASAPMemoCandidates; ErrorGeneratingPhysicalASAPMemoCandidates and SelectionError::PhysicalASAPCandidates deleted physical-asap-optimization/src/catalog/mod.rs, planner/src/run.rs Stage 2 cannot fail. approved 2026-11-07
DeploymentCapabilities, DeploymentCapabilitiesError, SupportedSummary, summary_supported moved to asap_types::deployment_capabilities (same move as PR 0b) types/src/deployment_capabilities.rs The Stage 2 crate depends only on asap-types. A move, no new type. follows from the approved entry
PruneReason: NoIngestionPipeline, QueryTimeResultsNotRetained added; PerEntityAggregateNotExecutable, TooFewQueryReaders deleted types/src/stages/pruning.rs The doc's Stage 2 prune table; no executor-support check and no consumer-count policy in the planner. approved 2026-11-07
PrunedLogicalCandidate { candidate, causes }, PruneCause { through_pruned_consumers, root }, RootPruneReason { BelowAccuracyFloorRequirement, MetricRule, NoConsumer }, LogicalASAPMemoCandidates::pruned types/src/stages/logical_memo_candidates.rs Pass 2's dead-candidate elimination with traces. approved 2026-11-07
LogicalRequirements::error_propagation(consumer, metric: AccuracyMetric) -> Option<ErrorPropagation>; Memo::set_input_requirements(CandidateForMemoGroupId, Vec<LogicalRequirements>) types/src/stages/candidates.rs The metric rule; the only writer of $r^{\log}$. approved 2026-11-07
SelectionBox { columns } (loses relative_time); Constraint::Interval { lower: Bound<IntervalConstraintLimit>, upper: Bound<IntervalConstraintLimit> }, IntervalConstraintLimit { FixedValue(ScalarValue), EvaluationTimePlusMs(i64) } (replaces TimeBound) types/src/ir/properties/summary_coverage.rs Absolute and relative time are intervals on the timestamp column; value columns use the same interval type. approved 2026-11-07
ScalarValue::Timestamp(i64) (ms since the epoch) types/src/ir/scalar/expr_ir.rs (+ literal typing, executor literal arms) The FixedValue(Timestamp(..)) of the approved examples (@ T, SQL ts >= '2026-11-01'). pending (implied by the 2026-11-07 examples)
DataArrival::{Ingesting, Mixed} { rows_per_second, max_out_of_order_ms: Option<u64> } types/src/workload/mod.rs Counter states at ingestion time need the out-of-order bound. approved 2026-11-07
GroupProperties::aggregate_coverage, OperatorInput::aggregate_coverage (were summary_coverage) types/src/stages/memo.rs, types/src/ir/operator/input.rs Renamed at the interface; PR 2 derives it for exact Aggregate groups. approved 2026-11-07
SummaryTumblingWindowChain loses tumbling_window_count; SummaryMerge loses newest_tumbling_windows (and its newest_panes alias); WindowLayout deleted types/src/ir/operator/asap.rs, types/src/stages/candidates.rs Chain groups are range-free; TimeRange/TimeShift over a chain select windows. approved 2026-11-07
Pass2RuleError::{InvalidGroupReference, TooManyGroups, CyclicDependencies} logical-asap-optimization/src/pass2/memo/rules.rs The doc's rule signature returns Pass2RuleError. These are the existing variants that do not name deleted rules. existing, reduced
PhysicalASAPDAGNode::lifetime replaces materialization (wire version 11 → 13, with parameters below) types/src/ir/physical_export.rs Materialization is deleted, and the doc says the export writes timing and lifetime per node. pending (PR 6 changes the backend with it)
materialized_tumbling_window_chains (was kept_tumbling_window_chains) cost/src/lib.rs Renamed: "materialized", not "kept". rename
ScheduleGroup::queries (was readers), and the queries argument of schedule_groups / evaluation_rate cost/src/execution_demand.rs Naming sweep: a query, not a "reader". In the files this unit edits, "kept" for a materialized result or retained data is now "materialized" or "retained", and "reader" is "query" or "consumer". rename
PlanningReport loses operator_nodes and physical_producer_links planner/src/run.rs Both counted deleted concepts. Nothing is added. deleted
memo_of_queries (was generate_dag_of_logical_memo_groups) logical-asap-optimization/src/stage0.rs Renamed. approved 2026-11-06
memo_of_queries(roots) -> Result<Memo, SchemaDerivationError>; Memo::find_or_add_group -> Result<MemoGroupId, SchemaDerivationError>; Memo::add_candidate -> Result<CandidateForMemoGroupId, CandidateError> stage0.rs, types/src/stages/candidates.rs A group and a candidate are checked by the node rules (schema derivation, input validation, merge check); a candidate must also produce its group's output. approved 2026-11-06
CandidateError::{Schema(SchemaDerivationError), OutputDiffersFromGroup { group }} types/src/stages/candidates.rs The two ways add_candidate rejects a candidate; both are bugs in the rule that built it. approved 2026-11-06
MemoGroup { expression, group_properties, parents, queries, candidates } (loses logical_node, child_groups) types/src/stages/memo.rs A group is what it computes over input groups; hash-consing compares the expression directly. approved 2026-11-06
GroupProperties { schema, result_kind, aggregate_coverage, shareable_when_equal } same What every candidate of the group produces, derived once from the input groups. approved 2026-11-06
derive_group_properties(memo, expression) -> Result<GroupProperties, SchemaDerivationError> types/src/stages/candidates.rs The one derivation for a group and a candidate. approved 2026-11-06
trait OperatorInput { schema, result_kind, operator, aggregate_coverage }, impls for Rc<OperatorNode> and MemoGroupAsInput<'m> { memo, group_id }; the rules generic over it, names kept types/src/ir/operator/input.rs, types/src/stages/memo.rs One set of rules for node trees and memo groups. approved 2026-11-06
OperatorInput is pub (not pub(crate)) and requires Clone types/src/ir/operator/input.rs With a pub(crate) trait, every pub generic rule (ScalarExpr::scalar_type, selected_from, ranked_column, …) trips private_bounds under -D warnings. operator() returns the input's children as inputs again, which clones them. pending
InputOrTemporaryBuild<I> { Input(I), TemporaryBuild(Rc<TemporaryBuild<I>>) }, TemporaryBuild<I> { operator, schema, result_kind } types/src/ir/operator/input.rs A build a rule constructs only while it derives (the rollup build, the selected build); never stored in the memo. Before, the rules built temporary OperatorNodes. A rule never builds over another rule's temporary build, so the type does not nest. approved 2026-11-06
SummaryStateDefinition { input_group, unlifted_operators }; SummaryCoverage { definition, selection } (no type parameter); definition_id() types/src/ir/properties/summary_coverage.rs Coverage over group ids. approved 2026-11-06
SummaryStateDefinition::input_group: Option<MemoGroupId> (was MemoGroupId in the decision) same Keeps today's behavior for PromQL label matchers, which the front end lowers into the Scan's predicates: a lifted Scan predicate leaves the Scan, with the rest of its predicates, as the last unlifted operator, and no input group. Otherwise latency{job="api"} and latency{job="web"} states would read different Scan groups and stop merging. Some(group) in every other case, including a Filter(job = …) over a bare Scan. pending
SummaryCoverage::derive_from_inputs(operator: &Operator<MemoGroupAsInput<'_>>) (not generic over I) same A definition names its input group by id, which only a memo input has; node trees carry no coverage. pending
SummaryStateDefinition::unlifted_operators: Vec<Operator<Option<MemoGroupId>>> same None marks an operator's main input (the next unlifted operator, or input_group); Some(g) is another input group it reads, such as the group a scalar subquery in a Filter reads. States whose definitions contain subqueries compare and merge; the walk passes a Filter or Project with a subquery and keeps the subquery's conjuncts as residuals. approved 2026-11-06
merged_window_build and the other count-based chain builds deleted (2026-11-07); merged_chain stays as the trigger of the PR 1 todo!()s. Its accuracy caller merged_tumbling_windows_guarantee is todo!() until PR 4 types/src/ir/operator/tumbling_window_chain.rs, accuracy/src/propagation.rs The accuracy propagation still reads node trees; PR 4 rewrites it over groups. approved 2026-11-06/07
OperatorNode::summary_build(self: &Rc<Self>) -> Option<ASAPOp> (was (&self) -> Option<&ASAPOp>); selected_from, summary_build, merged_chain return owned inputs types/src/ir/operator/ The generic rules read an input's operator by value (operator()). pending
private Memo::groups with Memo::groups() -> &[MemoGroup] types/src/stages/candidates.rs Groups are added only through find_or_add_group, so its hash-consing index covers every group. approved 2026-11-06
Memo::add_query(&mut self, root: QueryRoot<MemoGroupId>) -> usize same With groups private, Stage 0 cannot set MemoGroup::queries; this adds a query's root and lists the query on every group it reaches. approved 2026-11-06
Memo::logical_node_trees deleted; Pass 1 target discovery (PR 3) and devtools group labels (PR 9) are todo!(); devtools' MemoNodes::add and its test are deleted with it Pass 1, devtools Nothing rebuilds node trees from groups. approved 2026-11-06
stage0/cse.rs, test_support::share_common_sub_dags, integration-tests' share_identical deleted logical-asap-optimization, integration-tests Stage 0 builds groups through find_or_add_group (PR 2); one equality. Moved here from PR 2 because private groups leaves Stage 0 nothing else to build with. pending
FieldDataType::Sketch(SketchAlgorithm, GroupingStrategy) (was Sketch(SketchKind, …)) types/src/ir/schema/mod.rs θ is out of the operator. approved 2026-11-06
installable_plan_of(realized: &RealizedPlan, selection, entries) (was (roots, selection, tumbling_window_retention_ms, entries)) planner/src/installable.rs Export no longer takes bare roots. pending
#[allow(dead_code)] on the θ-based error model: estimators::sketch_guarantee, estimators::size_params, sizing::default_size_params, hll::classic_guarantee, univmon::{distinct_guarantee, incomplete_top_layer_statistic} accuracy/src/ Nothing reads θ off a node any more, so no production code calls them until PRs 1–4 compute ε per listed θ. Deleting them would delete the families' error model, which those PRs need. interim, removed in PR 1–4
pass1_memo, Pass1Error, PlanningRun::pass1_memo (were generate_per_computation_logical_memo_candidates, PerComputationLogicalMemoCandidatesError, PlanningRun::per_computation_candidates) Pass 1, planner/src/run.rs Renamed: they hold or return a Memo. approved 2026-11-06
expand_tumbling_window_chains_in_workload deleted types/src/ir/operator/tumbling_window_chain.rs No production caller after export from roots was deleted. approved 2026-11-06
PhysicalASAPDAGNode::parameters: Option<SketchParams> (wire version 12 → 13) types/src/ir/physical_export.rs θ is out of the operator, so the executor needs it per node. Filled in PR 6. approved 2026-11-06
SketchParams::Hydra(HydraParams); GroupingStrategy::SharedMultiSubpopulation { kind } (loses params) types/src/ir/schema/sketch.rs Hydra's sizes are θ. The direction is approved; the variant's shape (one Hydra variant wrapping HydraParams, rather than one variant per Hydra kind) is mine. direction approved 2026-11-06; shape pending
HydraParams::per_subpopulation(&self) -> SketchParams same The error model (sketch_guarantee) and the state shape (sketch_shape) read the emulated per-subpopulation sketch's θ, which GroupingStrategy used to carry. pending
create_planner_accumulator(family, parameters: Option<&SketchParams>, input, grouping) (gains parameters) executor/src/summary_kernels/factory.rs The factory reads θ from the exported node's parameters. pending
test-only builders asap_types::test_support::{physical_asap_dag_at_query_time, physical_asap_dag_at_query_time_with_node_ids} (feature test-support) types/src/test_support.rs, types/src/ir/physical_export.rs Keep the executor and runtime-install end-to-end tests by building PhysicalASAPDAG directly (decided 2026-11-06). Every node at query time, Lifetime::None, parameters: None. pending
#[cfg(any(test, feature = "test-support"))] on physical_export::grouping_compatibility (tumbling_window, shifted deleted 2026-11-07) types/src/ir/ Only tests and the test builder call them. pending
#[allow(dead_code)] on capability::valid_matrix and GroupStates::Keyed executor/src/ Their θ-based callers are todo!() until PR 6. interim, removed in PR 6

Key code interfaces

All items below are new, except where marked:

  • MemoGroup gains candidates.
  • Memo loses candidates.
  • CandidateForMemoGroupId moved here from stages/physical/memo_candidates.rs.
  • OriginOfCandidateForMemoGroup loses SharedBuild and SummaryResult.
  • Memo::topological_order is unchanged in signature and reads the expressions' children.
  • ExecutionTiming is the existing enum, re-exported from physical::properties.
  • AssignmentRealizer and RealizedPlan are unchanged; realize is now the only export path.

Function bodies are shown as ;. The bodies of the functions the doc assigns to PRs 1–8 are todo!() in the code.

crates/types/src/stages/memo.rs

/// A group, by index in the memo's groups.
pub struct MemoGroupId(pub u32);

/// One distinct logical sub-expression of the workload: what it computes
/// and the output every candidate of it must produce.
pub struct MemoGroup {
    /// What it computes: one operator, with its arguments. Each
    /// `MemoGroupId` in it is an input group, never this group's own id. // new (was logical_node, child_groups)
    pub expression: Operator<MemoGroupId>,
    /// Derived once when the group is created, from its input groups; never
    /// changes. // new
    pub group_properties: GroupProperties,
    /// The groups that read it through their expression or a candidate's.
    pub parents: Vec<MemoGroupId>,
    /// The queries whose roots reach it, ascending.
    pub queries: Vec<usize>,
    /// Its candidates: empty after Stage 0, filled by Stage 1.
    pub candidates: Vec<CandidateForMemoGroup>,
}

/// What every candidate of a group produces, derived from the group's
/// expression and its input groups' properties by the rules that check
/// operator nodes. // new
pub struct GroupProperties {
    pub schema: Schema,
    pub result_kind: OperatorResultKind,
    /// What an aggregate group covers: a summary state, a chain, a merge
    /// over a chain's windows, or an exact `Aggregate` result; `None` for
    /// any other group. // renamed 2026-11-07 (was summary_coverage)
    pub aggregate_coverage: Option<SummaryCoverage>,
    /// Whether two equal expressions return the same rows for the same
    /// input rows, so they may be one group (not an unordered `LIMIT`).
    pub shareable_when_equal: bool,
}

/// A memo group read as an operator's input: its `group_properties` and
/// its `expression`, whose inputs are read the same way. // new
pub(crate) struct MemoGroupAsInput<'m> {
    pub(crate) memo: &'m Memo,
    pub(crate) group_id: MemoGroupId,
}

impl OperatorInput for MemoGroupAsInput<'_> { … } // new

impl MemoGroup {
    /// The groups that read it.
    pub fn parent_groups(&self) -> &[MemoGroupId];

    /// The queries whose roots reach it, by index in the workload, ascending.
    pub fn reached_by_queries(&self) -> &[usize];
}

/// The groups some query's root reads, ascending: the groups of
/// `query_roots` that are a query's result or a scalar query's operand.
pub fn root_groups(query_roots: &[QueryRoot<MemoGroupId>]) -> BTreeSet<MemoGroupId>;

/// The queries whose root reads `group`, ascending.
pub fn queries_rooted_at(query_roots: &[QueryRoot<MemoGroupId>], group: MemoGroupId) -> Vec<usize>;

crates/types/src/stages/candidates.rs

/// One candidate: position `index_in_group` in group `group`'s candidates
/// ([`MemoGroup::candidates`]).
pub struct CandidateForMemoGroupId {
    pub group: MemoGroupId,
    pub index_in_group: usize,
}

/// The memo, Stage 0's and Stage 1's one type: the memo groups, each with
/// its candidates ([`MemoGroup::candidates`]: empty after Stage 0), and each
/// query's root over them.
pub struct Memo {
    /// The groups, by id; added only through `find_or_add_group`. // private (was pub)
    groups: Vec<MemoGroup>,
    /// Each query's root over groups, by index in the workload.
    pub query_roots: Vec<QueryRoot<MemoGroupId>>,
}

impl Memo {
    /// The groups, by id. // new
    pub fn groups(&self) -> &[MemoGroup];

    /// Adds a query whose root is `root`; returns its index. // new
    pub fn add_query(&mut self, root: QueryRoot<MemoGroupId>) -> usize;

    /// Hash-consing: the group with this expression, added if new, with the
    /// `GroupProperties` `derive_group_properties` derives. Fails when the
    /// expression breaks the node rules. // was -> MemoGroupId
    pub fn find_or_add_group(
        &mut self,
        expression: Operator<MemoGroupId>,
    ) -> Result<MemoGroupId, SchemaDerivationError>;

    /// Adds `candidate` to `group`'s candidates. Fails when its expression
    /// breaks the node rules or its output differs from the group's. // was -> CandidateForMemoGroupId
    pub fn add_candidate(
        &mut self,
        group: MemoGroupId,
        candidate: CandidateForMemoGroup,
    ) -> Result<CandidateForMemoGroupId, CandidateError>;

    /// The candidate `id`.
    pub fn candidate(&self, id: CandidateForMemoGroupId) -> &CandidateForMemoGroup;

    /// Replaces candidate `id`'s $r^{\log}_{c,i}$, one per input group: the
    /// only way to change them. Panics when the vector does not have one
    /// entry per input group. // new 2026-11-07 (real code)
    pub fn set_input_requirements(
        &mut self,
        id: CandidateForMemoGroupId,
        input_requirements: Vec<LogicalRequirements>,
    );

    /// The groups, each after every group it or one of its candidates
    /// reads, the lower index first among those ready: a dependency order for
    /// candidate traversal, whatever order rules appended groups in. `None`
    /// when those reads form a cycle, a group reading itself included.
    pub fn topological_order(&self) -> Option<Vec<MemoGroupId>>;
}

/// The properties of a group, or a candidate, computing `expression` over
/// its input groups: the node rules over `MemoGroupAsInput`s. // new
fn derive_group_properties(
    memo: &Memo,
    expression: &Operator<MemoGroupId>,
) -> Result<GroupProperties, SchemaDerivationError>;

/// A candidate `add_candidate` rejects; the memo is unchanged. // new
pub enum CandidateError {
    /// The candidate's expression fails the node rules (schema derivation,
    /// input validation, merge check).
    Schema(SchemaDerivationError),
    /// Its output schema or output kind differs from the group's.
    OutputDiffersFromGroup { group: MemoGroupId },
}

/// $v = (e_v, p^{\log}_v)$: one candidate of a memo group.
pub struct CandidateForMemoGroup {
    /// $e_v$.
    pub expression: CandidateExpression,
    /// $p^{\log}_v$.
    pub logical_properties: LogicalProperties,
    /// $r^{\log}_{c,i}$, one per input group: per child of
    /// `expression.operator`, in [`Operator::children`] order.
    pub input_requirements: Vec<LogicalRequirements>,
    /// The groups a fused computation also implements, beyond the group
    /// whose candidate it is; empty otherwise.
    pub fused_groups: Vec<MemoGroupId>,
    /// Which rule added it. Not in the formulation: kept for
    /// `stage_pipeline --text` and the viewer (approved 2026-11-05).
    pub origin: OriginOfCandidateForMemoGroup,
}

/// Which rule added a candidate. Nothing in planning reads it.
pub enum OriginOfCandidateForMemoGroup {
    /// Pass 1: exact, exact accumulator, or a summary family.
    Pass1,
    /// Canonical value preprocessing feeding a summary build.
    SummaryInput,
    /// A tumbling-window chain, or a merge reading one, of a summary state group.
    TumblingWindows,
    /// A coarse group regrouping a finer group.
    Rollup,
    /// A top-k read from another group's ranking.
    TopKReuse,
    /// Its metric's scans widened to a shared label set, read through a
    /// `Project` (scan widening).
    WidenedScans,
}

/// $e_v = \operatorname{op}_v(G_{v,1}, \dots, G_{v,n})$: one operator, with
/// its arguments, whose children are groups. The input groups $G_{v,i}$ are
/// [`Operator::children`], in that order.
pub struct CandidateExpression {
    pub operator: Operator<MemoGroupId>,
}

/// $p^{\log}_v = \boldsymbol\theta_v$. The family is the operator's
/// argument, and a chain's width is the chain operator's argument.
/// [`Self::new`] rejects parameters of another family, not in the family's
/// list, missing on a sketch build (a chain included) or present on an
/// exact operator. // 2026-11-07: window_layout and WindowLayout deleted
pub struct LogicalProperties {
    /// $\boldsymbol\theta_v$, from the family's predefined list; `None` for
    /// an exact operator.
    parameters: Option<SketchParams>,
}

impl LogicalProperties {
    pub fn new(
        expression: &CandidateExpression,
        parameters: Option<SketchParams>,
    ) -> Result<Self, LogicalPropertiesError>;

    /// $\boldsymbol\theta_v$.
    pub fn parameters(&self) -> Option<&SketchParams>;
}

/// Logical properties that do not fit their operator: bad input to
/// [`LogicalProperties::new`].
pub enum LogicalPropertiesError {
    #[error("an exact operator has no summary parameters")]
    ParametersOnExactOperator,
    #[error("a sketch build needs parameters from its family's list")]
    SketchBuildWithoutParameters,
    #[error("the parameters are not in their family's predefined list")]
    ParametersNotInList,
}

/// The family's predefined $\boldsymbol\theta$ list (#1207 step 6), each with
/// its error bound derived through `estimators::local_guarantee` for a given
/// statistic.
pub fn parameter_list(family: SketchAlgorithm) -> &'static [SketchParams];

/// $r^{\log}_{c,i}$: only what $G_{c,i}$'s identity does not fix.
pub struct LogicalRequirements {
    /// $\boldsymbol\theta^{\min}_{c,i}$: the weakest parameters whose
    /// $\varepsilon$, for the statistic the consumer reads, is at most
    /// $\text{cap}_{c,i}$ = min(validity cap of $g_c$, pruning cap), or a
    /// direct need such as heap_size ≥ k. Any value of the family's parameter
    /// type; it need not be in the family's list.
    parameter_floor_for_required_accuracy: Option<SketchParams>,
}

impl LogicalRequirements {
    pub fn new(parameter_floor_for_required_accuracy: Option<SketchParams>) -> Self;

    /// $\boldsymbol\theta^{\min}_{c,i}$.
    pub fn parameter_floor_for_required_accuracy(&self) -> Option<&SketchParams>;

    /// $g_c$'s linear bound for errors in `metric`: $\beta_{c,i}$ and
    /// $b_{c,i}$ per input. `None` when the consumer has no error contract
    /// for `metric` (the metric rule); no factor is guessed. // 2026-11-07: gains metric, returns Option
    pub fn error_propagation(
        consumer: &Operator<MemoGroupId>,
        metric: AccuracyMetric,
    ) -> Option<ErrorPropagation>;
}

/// The local error propagation of one consumer: its output error is at
/// most $\sum_i \beta_{c,i}\varepsilon_{c,i}$ while each input's error stays
/// within its validity cap (concepts doc, section 4; approved 2026-11-05).
pub struct ErrorPropagation {
    /// $\beta_{c,i}$ per input, for that input's metric.
    pub factor_per_input: Vec<f64>,
    /// $b_{c,i}$ per input: the cap under which the linear bound holds;
    /// `None` when the bound holds everywhere (e.g. a ratio caps only its
    /// denominator).
    pub validity_cap_per_input: Vec<Option<f64>>,
}

/// $\alpha^{\log}_{u,c,i}$: $\boldsymbol\theta_u \succeq
/// \boldsymbol\theta^{\min}_{c,i}$, compared per family on the parameters
/// that set $\varepsilon$. No floor, or an exact producer, always satisfies.
pub fn producer_satisfies_logical_requirements(producer: &LogicalProperties, requirement: &LogicalRequirements) -> bool;

The other files' interfaces (operator inputs, the derivation rules, summary coverage with SelectionBox/TimeBound, logical memo candidates with the Pass 2 pruned list, the Stage 2 entry, prune reasons, data arrival, physical properties, physical memo candidates, realizer, execution contexts, the MILP model, Pass 2 rules, schema, export, sketch parameters, the executor's factory) and the answered questions are in this comment.

Usage: not applicable yet. Every planning path reaches a todo!() (Stage 0 first). The first end-to-end usage example comes with PR 6.

Major algorithm changes

Summary coverage over memo groups (2026-11-06) is described in the interfaces comment. Other algorithms are deleted (see "Deleted"); later PRs re-implement them behind these interfaces.

Before this PR

The memo held private operator chains (root_node + input_groups) whose summaries carried θ in the operator, sized per budget share. Pass 2 copied consumers for shared builds and per-size groups. Stage 2 enumerated Materialization per operator node with interned producer sets. The #1144 MILP selected one candidate per result slot and context, with implicit edges. Export compiled any operator roots against a side map of physical properties.

After this PR

Every type and signature of the concepts doc's sections 3–5 exists with the doc's names:

  • θ lives only in LogicalProperties;
  • Pass 1 returns Memo;
  • the only export path is AssignmentRealizer::realize.

None of the old interfaces it replaces remains. Planning panics at the first unimplemented stage. PRs 1–8 fill in the stages in order, and PR 9 restores the tools and snapshots.

Interfaces per stage (doc section → code)

Doc section Interface Code
3, Stage 0 MemoGroupId, MemoGroup { expression, group_properties, parents, queries, candidates }, GroupProperties, MemoGroupAsInput crates/types/src/stages/memo.rs:16
3, Stage 0/1 Memo { groups (private), query_roots }, groups, find_or_add_group, add_candidate, add_query, candidate, CandidateError, derive_group_properties crates/types/src/stages/candidates.rs:25
3, Pass 1 CandidateForMemoGroupId, CandidateForMemoGroup, CandidateExpression, LogicalProperties, parameter_list, Memo::set_input_requirements, LogicalRequirements, producer_satisfies_logical_requirements crates/types/src/stages/candidates.rs
3, group properties OperatorInput, InputOrTemporaryBuild, TemporaryBuild; SummaryStateDefinition, SummaryCoverage::{derive_from_inputs, definition_id} crates/types/src/ir/operator/input.rs, crates/types/src/ir/properties/summary_coverage.rs
4 ErrorPropagation, LogicalRequirements::error_propagation crates/types/src/stages/candidates.rs
3, Pass 2 rule signature fn(&mut Memo) -> Result<(), Pass2RuleError> (rules 1–7), LogicalEdge, LogicalASAPMemoCandidates { memo, edges, pruned }, PrunedLogicalCandidate, PruneCause, RootPruneReason crates/logical-asap-optimization/src/pass2/, crates/types/src/stages/logical_memo_candidates.rs
3, Stage 2 ExecutionTiming (existing, re-exported), StorageMedium, Lifetime, PhysicalProperties crates/types/src/physical/properties.rs
3, Stage 2 PhysicalCandidateId, PhysicalEdgeId, PhysicalCandidate (+ delivers_on), producer_satisfies_physical_requirements, PhysicalEdge, PhysicalASAPMemoCandidates (+ edges, admissible_edges_into, admissible_edges_out_of), PrunedCandidate<PhysicalCandidate>; Stage 2 entry generate_physical_asap_memo_candidates(logical, data, capabilities) crates/types/src/stages/physical/memo_candidates.rs
3, Stage 3 SelectionVariables { x, y, z, t }, constraints (A)–(F), SelectedEdge { producer, consumer, input_index_of_consumer }, DagOfSelectedCandidates crates/plan-selection/src/milp/model.rs, crates/types/src/stages/physical/graph.rs
5 ExecutionContextId, ExecutionContext, RefreshWindowId crates/types/src/stages/physical/execution_context.rs
5 $x^{\mathrm{mat}}, x_\kappa, y_\kappa, h$, (R) crates/plan-selection/src/milp/model.rs
4 $E, F, D, H$, (K) crates/plan-selection/src/milp/model.rs

Deleted, and what replaces each

Deleted Replaced by
Materialization (types/src/physical/materialization.rs) PhysicalProperties { timing, lifetime }
OperatorNode::{timing, materialization}, with_timing, with_materialization each PhysicalCandidate::properties (doc section 3, realizer)
PhysicalPropertiesAssignment/MaterializationAssignment side map, the timing pass timing.rs (apply_*, data_state, timing_under_consumer, validate_at_ingestion, planned_data_state, NodeToTimedNodeCache) realizer and export from PhysicalCandidate (PR 6)
PhysicalWorkloadPlan::physical_properties, TumblingWindowChainExpansion::physical_properties, PlanOutput::physical_properties same, PR 6
DagOfLogicalMemoGroups (+ group_for_node, roots_over_groups) one Memo
MemoGroup::{logical_node, child_groups} expression and group_properties
accuracy::estimators::size_for_all_readers (one state's θ shared by several consumers, sized alone), its helpers min_recorded, kll::shared_k, univmon::shared_heap, and its 4 tests (shared_size_tests); the develop doc's paragraph on kll::shared_k (decided 2026-11-06) deleted, no replacement: θ is chosen per candidate from parameter_list, and consumers of one group read its candidates through edges, so nothing sizes one state for several consumers. It had no non-test caller after Pass 2's shared builds were deleted.
node-tree summary coverage: SummaryCoverage::{derive, derive_cached}, NodeToSummaryCoverageCache, OperatorNode::coverage(), the merge check in OperatorNode::new / validate_structure, definition_hash/same_definition over nodes, materialized_view::with_replaced coverage derived once per memo group (derive_from_inputs, stored in GroupProperties); the export's view coverage reads it in PR 6
Memo::candidates (Vec<Vec<_>>) MemoGroup::candidates
CandidateForMemoGroup::{root_node, input_groups, accuracy_budget_parts} expression: CandidateExpression (children = input groups), logical_properties, input_requirements
OriginOfCandidateForMemoGroup::{SharedBuild, SummaryResult} hash-consing plus one state group per family; readouts are groups (PR 3)
Pass 1 memo_population.rs (try_populate_pass1, NodeToStructuralIdCache), size_for_failure_probability, budget-share sizing (accuracy_budget_parts_per_measure, compose_*_with_accuracy_budget_parts, Sizing::shares), PerComputationCandidates::walked Pass 1 candidates via find_or_add_group/add_candidate with parameter_list (PR 3)
pass2/asap_aware_cse.rs (node-level window, rollup, top-k and scan-widening rewrites) and pass2/finer_aggregate.rs Stage 0 hash-consing and find_or_add_group, with rules 1–7 over groups (PR 4)
rule 8 summary_results.rs Pass 1 makes the readout a group (PR 3); the doc deletes the rule
rule 9 shared_configuration_readers.rs, shared_producer.rs, shared_configurations.rs the state group holds every θ; $y$ chooses per consumer input
step 10 state_groups.rs (add_shared_summary_state_groups, per-size groups, check_unexpanded_memo) and reuse.rs one state group per family (PR 3/4)
pass2/memo/enumeration.rs (ChosenCandidatePerMemoGroup, enumerate, build_logical_asap_dags_from_chosen_candidates), asap_planner::selected_chosen_candidates doc Appendix A enumerator (PR 6 check)
OperatorNodeOfCandidateForMemoGroup(Id), PhysicalCandidate::{operator_node, timing, materialization, producer_candidates_per_input, may_produce_query_output}, PhysicalASAPMemoCandidates::{operator_nodes_of_candidates_for_memo_groups, implementation_roots, query_output_candidates, output_reference_count, physical_node, memo_group_of} PhysicalCandidate { logical_candidate_id, physical_properties }, edges(), admissible_edges_*
ProducerCandidates, ProducerSetInterner, producer_storage_metrics PhysicalEdge list (PR 5)
ReaderConstraint (all four variants) and reader_constraints the per-context alignment check delivers_on; the cost model (doc section 5, "No consumer-count policies")
may_keep_across_evaluations_per_sole_reader_query Lifetime in PhysicalProperties, chosen by Stage 3
Stage 2 catalog (configurations.rs incl. compatible_timing, factoring.rs, inventory.rs, pruning.rs, reconstruction.rs), split_by_timing.rs, materialization and ingestion-eligibility rules in lib.rs generate_physical_asap_memo_candidates and producer_satisfies_physical_requirements (PR 1, PR 5); realizer (PR 6)
reference pricer optimize_physical_asap(_chosen), features reference-pricer/test-support of the Stage 2 crate doc Appendix A enumerator (PR 6 check)
plan-selection reference_pricer, selector.rs, WholePlanSelection, feature reference-pricer same
groups.rs (ResultSlot, slot_of, result_slots, run_kind, execution contexts, dominance pruning), groups/accuracy_positions.rs ExecutionContext and $x_\kappa$/$y_\kappa$ (PR 7)
milp/model.rs (#1144: p[ctx,c], e binaries, schedule groups z[m,G], N[q], A[q, parts], constraint 8, constraint 9) SelectionVariables and (A)–(F) (PR 6), contexts (PR 7), (K) (PR 8)
milp/cost_terms.rs, warm_start.rs, completion.rs, decode.rs PR 6 (baseline objective, decode $y$), PR 7 (context costs, $h$)
milp/lp_diagnosis.rs, feature lp-diagnosis deleted, no replacement
checks.rs (post-selection accuracy and latency checks), accuracy_binding.rs (K) in PR 8; the latency check: deleted, no replacement (latency is out of scope)
SelectionModels deleted with its only users
devtools memo_pricing (reference_select, reference_enumeration, DEFAULT_REFERENCE_CHOICES, PlanningStagesRun::reference, the max_combinations/reference_choices parameters), fuzz SearchKind::Exhaustive and EXHAUSTIVE_MAX_COMBINATIONS doc Appendix A (PR 6)
AnalyticalCostError::PhysicalPropertiesError, PhysicalASAPDAGNode::physical_properties (both from the #1214 port) the wire validator checks (IngestionTime, OneEvaluation) directly
PerComputationLogicalMemoCandidates, Pass 1's PrunedWayToCompute / pruned_ways_to_compute / pruned_of_target, and PruneReason's Pass 1 variants (RequiresExact, NullableAvgInput, NoRelationalCountSketchRealization, UnsupportedByDeployment, QuantileOutsideUnitInterval, TopKHeapOverOneRowPerItem, NoErrorBoundForQueryTarget) Pass 1 returns Memo; showing what Pass 1 leaves out moves to #1205
SketchKind inside FieldDataType::Sketch; accuracy::sizing::sized (budget-share sizing); propagation::with_resized_summaries, sizing::{with_summaries_sized_for_failure_probability, with_summaries_sized_for_targets, per_input_failure_probability, sized_for_failure_probability} θ in LogicalProperties::parameters from parameter_list (PR 1/3)
roots-based export: compile_physical_asap_dag, compile_physical_asap_workload(_with_node_ids), PhysicalASAPDAGCompilation, PhysicalASAPNodeIdentityMap, PhysicalASAPDAG::{with_every_node_at_query_time, query_roots}, ReferencePlanError, PlanOutput::physical_asap_dag, ExecutionDataStateError, InstallablePlanError::Timing AssignmentRealizer::realize (PR 6)
asap_planner::test_support::query_time_reference and the planner's test-support feature deleted, no replacement yet: a query-time reference needs a realization at query time; PR 6 decides
2026-11-07: WindowLayout, LogicalProperties::window_layout, LogicalPropertiesError::{WindowOnOperatorThatCannotBeWindowed, ZeroWindowWidth} the chain operator's width argument (chain groups, PR 4 rule 1)
2026-11-07: SummaryTumblingWindowChain::tumbling_window_count, SummaryMerge::newest_tumbling_windows (and the pane_count/newest_panes aliases); the count-based builds whole_window_build, merged_window_build, window_build, tumbling_window, shifted, expand_tumbling_window_chains, node_of_build, InputOrTemporaryBuild::to_node, test_support::{whole_window_build, tumbling_window}; CoverageError::TumblingWindowChainExpansion, InstallablePlanError::TumblingWindowChainExpansion TimeRange/TimeShift over a range-free chain select windows (PR 1, PR 2); the executor runs them by window index (PR 6); retention is $h_v$ (PR 7)
2026-11-07: SelectionBox::relative_time with its wire bounds, windows_ordered_apart, disjoint_in_time, the time-box union fast path union_ordered_time_boxes intervals on the timestamp column with IntervalConstraintLimit ends (PR 2)
2026-11-07: TimeBound { At, FromEvaluationTime } (from the first 11-07 round) IntervalConstraintLimit in Constraint::Interval
2026-11-08: LogicalPropertiesError::ParametersOfAnotherFamily ParametersNotInList: each family's list holds only its own parameters (PR 1 deletes the check)
2026-11-07: cost's chain_input_offset_ms and the count-based chain cost and retention (now todo!()); fuzz helpers find_first, collect; integration helper planner_layering_common::retention_ms (unused) $h_v$ and (R) (PR 7); fuzz witnesses (PR 9)
2026-11-07: PhysicalCandidateKey, PrunedCandidate::group, PhysicalASAPMemoCandidates::query_roots, ErrorGeneratingPhysicalASAPMemoCandidates, SelectionError::PhysicalASAPCandidates PrunedCandidate<PhysicalCandidate>; logical_candidates.memo.query_roots; Stage 2 cannot fail
2026-11-07: PruneReason::{PerEntityAggregateNotExecutable, TooFewQueryReaders} deleted, no replacement: the planner does no executor-support check (the executor is extended, PR 6) and has no consumer-count policy (the cost model)

Deleted tests

653 #[test] functions are deleted (643 before 2026-11-07, 10 in the 2026-11-07 round) and 2 are added (PhysicalProperties::new). The per-file table, with what each file covered and which PR restores it, is in this comment. By group:

Group Deleted Restored by
#509 acceptance examples (planner_layering_example{1,2,3,4}) over the reference pricer 105 PR 6 (baseline, selection), PR 7 (materialization), PR 8 (accuracy)
plan-selection: whole-plan selector, #1144 MILP, result slots, checks 141 PR 6 (A)–(F) and decode, PR 7 contexts and (R), PR 8 (K); latency checks: no replacement planned
Pass 2 node-level rewrites, rule 9, step 10 per-size groups 61 PR 4 (rules 1–7); rule 9 and step 10: no replacement planned
Pass 1 private-node population, budget-share sizing, pruned reasons 34 PR 3; pruned reasons: #1205; budget split: no replacement planned
Stage 2 catalog, timing split, realization; the timing pass 52 PR 5, PR 6
Other planner-dependent tests: DAGs or plans from the planner, reuse rules, materialized views, reference pricing (executor, integration, planner, cost, types, frontend, devtools) 238 + 1 PR 6; materialized variants PR 7; tools PR 9
θ per node (state bytes, kernels, KLL equality across δ) 11 PR 1, PR 6

The 2026-11-07 round's 10 deleted tests, per file, are in the interfaces comment.

a_chain_reads_an_instant_vector now checks a zero width (was a zero count); names_before_the_tumbling_window_rename_still_read reads only the chain. Earlier detail (node-tree coverage, tests kept by building the DAG directly) is in the interfaces comment.

Test status

cargo test -p <crate> --all-features --no-fail-fast at the head (7e83db9). Except for the stale fixture, every failure panics at a todo!(); the column names it and the PR that fills it in.

Crate Passed Failed Ignored Fails at
asap-types 160 2 0 reject the invalid combination of section 2 (PR 1): 2
asap-accuracy 73 0 0 —
asap-cost 31 1 0 check θ, which the exported node carries (PR 6): 1
asap-frontend-common 49 0 0 —
asap-sql-function-catalog 10 0 0 —
metricsql_common 3 0 0 —
asap-frontend-promql 219 8 1 Stage 0 over Memo::find_or_add_group (PR 2): 8
asap-frontend-metricsql 18 0 0 —
asap-frontend-sql 174 1 0 Stage 0 over Memo::find_or_add_group (PR 2): 1
asap-executor 210 63 1 check θ, which the exported node carries (PR 6): 32; the Hydra grid's sizes are θ, which the exported node carries (PR 6): 13; the view's coverage from its memo group's GroupProperties::aggregate_coverage (PR 6): 8; the weighted-frequency θ, which the exported node carries (PR 6): 7; check the weighted-frequency θ, which the exported node carries (PR 6): 2; Stage 0 over Memo::find_or_add_group (PR 2): 1
asap-planner 9 18 1 Stage 0 over Memo::find_or_add_group (PR 2): 18
asap-devtools 38 58 3 Stage 0 over Memo::find_or_add_group (PR 2): 56; an asap_planner subprocess that stops at a todo!(): 2
asap-integration-tests 83 87 4 Stage 0 over Memo::find_or_add_group (PR 2): 83; the view's coverage from its memo group's GroupProperties::aggregate_coverage (PR 6): 3; the stale fixture (old wire format; PR 9): 1
asap-logical-asap-optimization 27 53 0 Stage 0 over Memo::find_or_add_group (PR 2): 51; the guarantee of θ from the candidate's LogicalProperties::parameters (PR 4): 1; the distinct-count guarantee of θ from LogicalProperties::parameters (PR 4): 1
asap-physical-asap-optimization 0 0 0 —
asap-plan-selection 3 0 0 —
Total 1107 291 10
  • Stale fixture. materialized_views_equal_the_stored_outputs_main_listed reads a checked-in plan file in the old wire format (Sketch with θ). PR 9 re-baselines it with the snapshots.
  • 2026-11-08 commit (7e83db9). Interval examples and tests use left-open, right-closed ends; ParametersOfAnotherFamily deleted. Compared by test name with facaa92f8 (backup backup/1207-interfaces-facaa92f8), no test changed result.

Verification

  • cargo check --workspace --all-targets --all-features: clean, no warnings.
  • cargo clippy --workspace --all-targets --all-features -- -D warnings: clean. The only allowances are the commented #[allow(dead_code)] markers listed under "New abstractions".
  • cargo fmt --check: clean.
  • Tests per crate: see "Test status".
  • No snapshot or plan comparison applies: planning panics at Stage 0 until PR 2.

…e trait, MemoQuery, evaluation grids of groups (#1207 PR 0)

2026-11-09 decisions:
- theoretical_guarantee and AccuracyEvidence::measured_guarantee take the
  failure probability and return the error bound that holds at it (None
  when θ cannot fail that rarely); bodies stay todo!() for PR 4.
- AccuracyEvidence is a trait; NoMeasuredGuarantees has no measurement.
  PlanningModels::evidence and Pass 2 take &dyn AccuracyEvidence.
- The memo holds its queries in one Vec<MemoQuery> { root,
  accuracy_target, values_per_evaluation }; add_query(query).
- SupportedSummary::family is summary_type.
- satisfies is deleted with metric_meets, scope_meets, native_target and
  AccuracyScope::covers, which only it used; target_limit is the
  comparison limit with the tolerance.
- PhysicalCandidate::delivers_on(&PhysicalASAPMemoCandidates, context).
- GroupProperties::depends_on_evaluation_time (derived in PR 2) and
  LogicalASAPMemoCandidates::evaluation_grid_of_group (filled in PR 4).
@zzylol
zzylol force-pushed the refactor/1207-interfaces branch from fb0a46a to e9d9f5a Compare November 9, 2026 04:21
… the tests that use it (#1207 PR 0)

pass1/composition.rs composed one whole-workload candidate DAG from one
per-computation candidate per target, for tests only; planning adds every
way to compute to the memo instead. It and the 74 tests and fixture helpers
that compose through it are deleted (no interim code for PR 3).
@zzylol
zzylol force-pushed the refactor/1207-interfaces branch from e9d9f5a to 2ed6ec0 Compare November 9, 2026 04:25
@zzylol

zzylol commented Nov 9, 2026

Copy link
Copy Markdown
Contributor Author

#1207 PR 0, 2026-11-09 (P11): the 74 tests deleted with pass1/composition.rs, by file (head 2ed6ec0):

  • crates/frontend-metricsql/tests/external_corpus.rs (1): corpus_lowers_and_binds
  • crates/frontend-promql/tests/count_planning.rs (4): aggregate_over_topk_candidates_compose, cms_count_updates_total_ten_for_zero_positive_and_negative_samples, frequency_count_candidates_use_unit_weights, grouped_count_offers_hydra_but_selection_keeps_per_group_state
  • crates/frontend-promql/tests/observability/metrics_observability.rs (2): benchmark_corpora_are_total_and_keep_their_coverage, kubernetes_mixin_aggregates_over_topk_compose
  • crates/frontend-promql/tests/observability/promql_corpus.rs (1): binding_is_total_over_the_entire_corpus
  • crates/frontend-promql/tests/univmon_candidates.rs (1): uncalibrated_frequency_evaluations_do_not_bypass_accuracy_targets
  • crates/integration-tests/tests/exact_composition.rs (1): physical_export_carries_explicit_timing_for_a_composed_plan
  • crates/integration-tests/tests/per_computation_candidates_sql.rs (4): a_sql_fold_over_estimates_leaves_its_row_count_unbound, rownumber_per_group_topk_offers_heap_sketches, topk_heap_over_a_key_named_value_counts_raw_rows, topk_heap_over_an_int64_key_counts_raw_rows
  • crates/integration-tests/tests/promql_numeric_regressions.rs (7): checked_ratio_must_not_certify_cross_zero_interpolation, count_up_counts_targets_even_when_values_repeat_or_change_sign, ddsketch_interpolation_requires_a_nonnegative_domain, quantile_over_temporal_average_keeps_a_legal_candidate, sketch_counts_use_unit_weights_and_signed_sums_keep_value_weights, sum_rate_and_increase_have_real_exact_accumulator_nodes, window_counts_and_sums_distinguish_one_zero_three_and_negative_values
  • crates/integration-tests/tests/sketch_rollup.rs (3): distinct_counts_rolled_up_equal_the_direct_build, invalid_rollups_are_rejected, quantiles_rolled_up_keep_the_sketch_bound
  • crates/integration-tests/tests/sql_to_post_asap.rs (7): clickhouse_outer_sum_recursively_binds_inner_temporal_aggregate, clickhouse_temporal_sql_reuses_rate_and_increase_physical_summaries, map_projection_export_keeps_the_map_over_an_avg_of_exact_states, sql_count_distinct_with_epsilon_binds_hll_over_named_column, sql_full_query_retains_project_and_binds_inner_aggregate, sql_join_recursively_binds_both_temporal_aggregate_children, sql_quantile_binds_kll_sketch_over_named_column
  • crates/integration-tests/tests/stage1_logical_asap_optimization.rs (13): filtered_sql_avg_is_null_without_matching_rows, promql_avg_divides_sum_and_count_vectors, promql_candidates_compose, sql_avg_counts_its_own_input, sql_avg_from_sum_and_count_executes_as_avg, sql_avg_keeps_its_explicit_output_name, sql_avg_of_an_int64_column_keeps_the_exact_columns, sql_candidates_compose, sql_count_of_a_nullable_column_counts_its_values, sql_count_star_without_a_non_null_column_is_unsupported, subquery_in_a_measure_filter_is_substituted, topk_estimates_carry_no_frequency_bound, topk_over_an_aggregate_ranks_its_labels
  • crates/logical-asap-optimization/src/pass1/per_computation_candidates.rs (19): a_norm_follows_the_update_weight, a_quantile_estimate_shares_its_raw_data_samples_sign_only_inside_the_unit_interval, a_quantile_of_merged_tumbling_windows_shares_its_raw_data_samples_sign, aggregate_replacement_preserves_sql_field_identity, avg_states_out_of_order_are_refused, count_ranked_topk_fuses_to_one_global_heap_sketch, heap_weights_over_raw_data_samples_record_an_open_requirement, multidimensional_ranked_item_is_a_structured_tuple, nested_sum_merges_into_one_exact_accumulator, normalized_additive_errors_compose_as_absolute_errors, subpopulation_columns_are_not_part_of_the_ranked_item_tuple, temporal_average_divides_with_a_finite_division_guard, temporal_per_entity_topk_uses_series_identity_and_sample_value, the_absolute_bound_names_the_weighted_norm, topk_keeps_known_labels_beside_series_identity, values_estimating_raw_data_samples_name_their_sign_by_the_metric, whole_expression_topk_over_a_grouped_sum_of_range_sums_reads_the_raw_data_samples, whole_expression_topk_partitions_index_the_heap_input, whole_expression_topk_replaces_the_heap_over_the_exact_sum
  • crates/logical-asap-optimization/src/pass1/per_computation_candidates_tests.rs (10): composed_avg_divides_sum_and_count_states, composed_candidate_replaces_chosen_target_with_summary_evaluation, composed_filters_and_having, composed_guarantees, composed_mixed_exact_and_sketch_measures_keep_output_names, composed_multi_measure_aggregate_joins_its_measures, grouped_avg_composition_preserves_the_whole_schema, own_having_subquery_is_substituted, per_series_avg_with_series_identity_composes_from_sum_and_count, per_series_rate_keeps_labels_and_retypes_its_state
  • crates/logical-asap-optimization/src/pass1/realization.rs (1): summarised_input_reads_the_intents_columns

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CSE optimizer Workload optimization and plan selection refactor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants