Repository navigation
Conversation
…rom an unsynced base A graft or `graphman copy` made while the source has not reached the chain head yet produced a destination marked `postponed_indexes_created = true` without the postponed attribute indexes. `copy_data` recreates on the destination only the indexes that exist on the source, so a source that has not built them yet hands over none of them, and `create_indexes` then set the flag unconditionally. `create_postponed_indexes` returns early once that flag is set, so the destination never built them, not even at the chain head, and queries filtering on those attributes ran without an index. Set the flag on the destination only when the source has already created its own postponed indexes. That keeps the reason the flag was added to the copy path -- not recreating indexes that were dropped on the source on purpose -- while leaving it unset for a source that is simply still syncing, so the destination builds the indexes when it gets near the head. The regression test grafts at block 1 before the base builds anything, then lets both deployments run `create_postponed_indexes` and checks that the destination ends up with the attribute indexes for the columns it shares with the base. Signed-off-by: 梁旭 <liangxu.5@bytedance.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
With
GRAPH_POSTPONE_ATTRIBUTE_INDEX_CREATION=true, a graft (orgraphman copy create) taken while the source has not reached the chain head yet produces a destination withpostponed_indexes_created = truebut without the postponed attribute indexes. Because that flag short-circuitscreate_postponed_indexes, the destination never builds them — not even once it reaches the head — and queries filtering on those attributes run without an index.This mostly hits grafted subgraphs: an indexer picking one up from scratch indexes the base only up to the graft block, so the base never gets near the head and never builds its postponed indexes; the graft then copies a base without them and marks them as created. Nothing in the logs says anything.
Reported in #6749 with a production case: deployment B copied from A while A was syncing ended up with 343 indexes against A's 709, the 366 missing ones exactly the BTree attribute indexes (~8.4 GB on A).
Root Cause
store/postgres/src/copy.rs,create_indexes(l. 302-344): the destination only gets the indexes inindex_list, i.e. those that exist on the source, plus indexes for new columns — and thenset_postponed_indexes_created(dst)is called unconditionally.The flag was added to the copy path in #6583 so that a copy does not recreate indexes that were intentionally dropped on the source. It cannot tell "dropped on the source" from "not built yet on the source", and the copy path treats both as the former.
Fix
Read
postponed_indexes_createdfor the source and set the flag on the destination only when the source has it set. A source that finished syncing and then had indexes dropped on purpose still has the flag, so the behaviour #6583 introduced is preserved; a source that is still syncing does not, so the destination is left pending and builds the indexes through the normalcreate_postponed_indexespath once it gets near the head.Testing
Added
graft_before_postponed_indexestostore/test-store/tests/postgres/graft.rs. It grafts at block 1 while the base has no postponed index and the flag is false, then runscreate_postponed_indexeson both deployments and checks the destination ends up with the attribute indexes for the columns it shares with the base.On master the test fails on the last assertion:
With the fix the destination gets them:
Full suite against PostgreSQL 14, with and without
GRAPH_POSTPONE_ATTRIBUTE_INDEX_CREATION=true:Fixes #6749