Repository navigation
fix(concurrency): lock multi-row writes in a fixed order (Beacon spec 239 FR-013) - #122
Merged
Merged
Conversation
… 239 FR-013) Task moves, column reorder, column delete and column retype now lock every row they write before the first write, in one fixed order: the board columns, then the tasks, each by ascending id (SELECT ... WHERE id IN (...) ORDER BY id FOR UPDATE; a no-op on SQLite). Two concurrent writes can no longer take rows in crossing order and deadlock (MySQL 1213, PostgreSQL 40P01). Positions and written values are unchanged; events are still dispatched after the write; lock conflicts still reach the caller as the driver's exception. - New internal SPI Service\OrderedRowLockInterface with the Doctrine implementation Repository\DoctrineOrmOrderedRowLock (stateless, joins a caller's transaction, rolls back and resets a closed manager otherwise). - TaskManager: move(), moveTo*() and a column-changing update() lock the entered column, then the tasks. - BoardColumnManager: reorder(), delete() and a retyping update() lock the columns, then the tasks. - TaskMover, TaskBoardRestructurer and DoctrineOrmTaskTreeStore read each tree level ordered by id. - Tests: query-log order on every platform, and a real second connection on MySQL / PostgreSQL showing the old order deadlocks and the fixed one waits.
…the lock A renumbering move now locks its column's row first (every move that renumbers the column waits there), then re-reads the column's tasks with a locking read and refreshes any copy that is out of date (OrderedRowLockInterface::lockedColumnTasks()). A move that waited for a concurrent one computes the new positions from the committed column instead of the positions it read before the lock, so two concurrent moves in one column end with unique, contiguous positions. - Test: a second process moves a task through the real TaskManager while holding the column; the test's move ends with positions 0..n-1 computed from the committed order (fails without the re-read). - Docs: known limits of the fixed order (two-phase locking inside a caller's transaction, import and provider sync) in CHANGELOG and SECURITY.md.
…lock (audit SF) - SF-M1: the Task API's locked updates (bulk, If-Match / base_revision, parent) lock the columns their status fields enter before the task rows (lockedRevision), the fixed order; they no longer deadlock against a renumbering move or a column delete / retype. - SF-L1: a move, moveTo*() or column-changing update() whose task changed board, column, archive or completion state after its checks writes nothing and throws TaskTreeRefusedException::conflict() (API 409 PARENT_004; board flash and 409 on the drag). - SF-L2: lockedColumnTasks() never refreshes an out-of-date copy that holds unsaved changes; it throws the conflict instead. - SF-L3: moveTo*() and a column-changing update() take the next free position from the column as committed, after its lock. - SF-L4: a column delete re-reads the deleted and target columns after its lock; a task that entered meanwhile moves too (policy asked for it). - SF-L5: at the outermost level a deadlock / lock wait timeout hidden behind a failed savepoint rollback (MySQL 1305) reaches the caller as the DBAL DeadlockException / LockWaitTimeoutException. - SF-I2: a move no longer locks the siblings it read before the lock; the column scan locks them in id order. - SF-I1: the column scan uses the column_id index (EXPLAIN: ref, no filesort); gap-lock note in SECURITY.md. No new index. - SF-I5: CHANGELOG scoped, SECURITY limits corrected, UPGRADING To 2.21.0. - Tests: probes P1-P4 as regression tests (real second process on MySQL and PostgreSQL; P1 through the Task API), SQLite variants, UI conflict test.
…flicts (audit SF-2) - SF-D1: TaskManager::create() with a parent and max_subtask_depth locks the column it enters (and reads its next position as committed) before lockedDepth() locks the parent's rows. Before, a renumbering move in that column (column held, waiting on the parent) deadlocked against it (MySQL 1213, PostgreSQL 40P01). Probe P5 is a two-process regression test through the real create(); it fails without the fix on both databases. - Task API: a conflict on an existing task (status change, field update, checklist item) answers 409 TASK_409 with current_revision, also as the item's entry in BULK_001; PARENT_004 stays for re-parent and subtask conflicts (TaskApiRefusal::taskWriteRefused(), withinPlan(..., $task)). - Docs: CHANGELOG, API.md, SECURITY.md; UPGRADING says a DDD host wires the lock into TaskApiTaskOperations and TaskManager together or into neither.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Beacon spec 239, block B, FR-013 (hardening): the bundle's multi-row writes lock their rows in one fixed order before the first write, so two concurrent writes can no longer hold one row each while waiting for the other's (MySQL 1213, PostgreSQL 40P01). Beacon already retries 1213/1205; this removes the cause.
Fixed order: board columns, then tasks, each by ascending id, via
SELECT … WHERE id IN (…) ORDER BY id FOR UPDATE(chunks of 500, in order; a no-op on SQLite, which serialises writers).Write paths changed:
TaskManager::move()(renumberColumn): the column a task enters, then every task of the target column (the moved one included). Before, the unit of work issued the sibling/moved-task updates in its own order.TaskManager::moveToPreviousColumn()/moveToNextColumn()/moveToDone()/moveToColumn()and anupdate()that changes the column: the entered column (its row is checked by the foreign key), then the task.BoardColumnManager::reorder(): every column, by id (was identity-map order).BoardColumnManager::delete(): every column (the deleted one included), then the tasks that move.BoardColumnManager::update()with a type change: every column, then the column's tasks.TaskMover::lockTree(),TaskBoardRestructurer::lock(),DoctrineOrmTaskTreeStore::lockedHeight(): each tree level is readORDER BY id.New internal SPI
Service\OrderedRowLockInterfacewith the Doctrine implementationRepository\DoctrineOrmOrderedRowLock(bundle's entity manager or the private one). It opens a transaction or joins the caller's (no savepoint); a failure, a lock conflict included, reaches the caller as the driver's own exception (so a host retry on 1213/1205 keeps working). Stateless (FrankenPHP worker mode, kernel reset false): rollback throughTransactionRollback, a manager closed by a failed flush is reset through the registry. ATaskManager/BoardColumnManagerbuilt without it (hand-built, DDD host doubles) writes as before.Read after the lock. A renumbering move locks its column's row first (every move that renumbers the column waits there), then re-reads the column's tasks with a locking read (
OrderedRowLockInterface::lockedColumnTasks(), refreshing any out-of-date copy) and computes the new positions from the committed column, never from what it read before the lock.Known limits (CHANGELOG and
docs/SECURITY.md, "Concurrency: fixed lock order"): two-phase locking when one of these writes runs inside a caller's transaction that already holds rows (e.g. the Task API bulk update), and the import batches and provider sync writers (file or remote order). Both can still deadlock against a concurrent move; the database rolls one back for the caller to retry.No behaviour change apart from the lock order: positions and written values are the same, events are dispatched after the write as before. One side effect: these writes now run in one transaction, so a move's renumbered siblings roll back with it if its final save fails (they used to stay saved).
Reviewed, not changed: Task API bulk update (already locks in id order),
UnmatchedColumnsCleaner(already locks), import batches and provider sync writers (one run per board, in file/remote order, inside their own transaction),BoardColumnsApplier/ folder status sets (structural writes inside the folder transaction), custom field update (one field row, then its values).Type of Change
Checklist
docs/CHANGELOG.mdif applicableTesting
tests/Functional/Move/FixedLockOrderTest.phptests/Fixtures/Deadlock/lock_two_tasks.php) plays a concurrent move of the column's lowest task L while the realTaskManager::move()moves the highest task H to the top. Old order (no lock service): the move of H writes L then H, the old move of L takes H then L, and one of the two is chosen as the deadlock victim. Fixed order: both take L then H; the move waits and both finish with the expected positions. The other move is a raw two-row lock in the order of the code it stands for, not a second kernel.tests/Fixtures/Deadlock/move_worker.php, own kernel, realTaskManager) holds the column and moves H to the top while the test's move, which read the column before it waited, inserts a task. The result is unique, contiguous positions in the committed order (H, new, L, M). Checked locally that this test fails without the re-read.tests/Unit/Service/FixedLockOrderTest.php: plain-PHP lock double (DDD path), lock calls before the first save, same positions with and without the lock.tests/Unit/Repository/DoctrineOrmOrderedRowLockTest.php: commit, join, rollback, failed rollback, rethrow as is.Locally: php-cs-fixer, PHPStan, full default suite on SQLite (2620 tests),
FixedLockOrderTeston MySQL 8.4 and PostgreSQL 16; CI runs the full real-database suites.