[xiedeyantu]

Jira Link

CALCITE-7928

Changes Proposed

To be honest, I’m not very familiar with this area; I only discovered it because it was causing a blockage in the CI pipeline. The code here was implemented entirely with the help of Codex—hopefully, it might serve as a source of inspiration for others.

[rubenada]

@xiedeyantu what have you observed locally in terms of execution time of the problematic test before and after the patch?
btw, could you please check who's been working on the TopDownGeneralDecorrelator and request them a review here?

[xiedeyantu]

@xiedeyantu what have you observed locally in terms of execution time of the problematic test before and after the patch? btw, could you please check who's been working on the TopDownGeneralDecorrelator and request them a review here?

@rubenada I reran the problematic query locally with top-down decorrelation enabled (macOS ARM64, JDK 17.0.12, -Xmx1536m). In fresh JVMs, the first execution took about 29.9 s before the patch and 1.13 s after it. These timings include planning, execution, and fetching results, excluding JVM startup and connection setup. Both returned the expected two rows.

I know this part of the code was written by @silundong , with @mihaibudiu as the reviewer. I’m not sure if they have the time to fix this bug or review this PR.

[mihaibudiu]

Would be nice to understand how such a simple query can take such a long time.

[mihaibudiu]

This PR does not address the fundamental problem; it makes the decorrelator hand off a smaller plan to the planner, but the problem is in the planner.

[xiedeyantu]

Mihai has created a new PR. I agree with the issue he raised, so I am closing this PR.

[rubenada]

Thanks for your work here anyway @xiedeyantu.
Unrelated to CALCITE-7928, if this proposed patch helps the top-down decorrelator to produce simpler (and yet correct) plans, IMO we should consider it as a separate improvement ticket, it'd be a pity to discard a potential optimization on this area. If you have a bit of time, feel free to explore that possibility :)

[xiedeyantu]

@rubenada Do you mean I should treat this as an optimization for the new top-down decorrelator, rather than as a fix for a CI issue? That might be a valid approach—should I submit a new JIRA ticket? However, as @mihaibudiu pointed out, this doesn't seem to address the root cause of the problem.

[rubenada]

@xiedeyantu correct, we could open a separate ticket for this top-down decorrelator optimization, unrelated to the CI issue.