Skip to content

fix(concurrency): lock multi-row writes in a fixed order (Beacon spec 239 FR-013) - #122

Merged
HecFranco merged 6 commits into
mainfrom
fix/fixed-lock-order
Nov 17, 2026
Merged

HecFranco merged 6 commits into
mainfrom
fix/fixed-lock-order

Conversation

@HecFranco

@HecFranco HecFranco commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Description

Beacon spec 239, block B, FR-013 (hardening): the bundle's multi-row writes lock their rows in one fixed order before the first write, so two concurrent writes can no longer hold one row each while waiting for the other's (MySQL 1213, PostgreSQL 40P01). Beacon already retries 1213/1205; this removes the cause.

Fixed order: board columns, then tasks, each by ascending id, via SELECT … WHERE id IN (…) ORDER BY id FOR UPDATE (chunks of 500, in order; a no-op on SQLite, which serialises writers).

Write paths changed:

  • TaskManager::move() (renumberColumn): the column a task enters, then every task of the target column (the moved one included). Before, the unit of work issued the sibling/moved-task updates in its own order.
  • TaskManager::moveToPreviousColumn() / moveToNextColumn() / moveToDone() / moveToColumn() and an update() that changes the column: the entered column (its row is checked by the foreign key), then the task.
  • BoardColumnManager::reorder(): every column, by id (was identity-map order).
  • BoardColumnManager::delete(): every column (the deleted one included), then the tasks that move.
  • BoardColumnManager::update() with a type change: every column, then the column's tasks.
  • TaskMover::lockTree(), TaskBoardRestructurer::lock(), DoctrineOrmTaskTreeStore::lockedHeight(): each tree level is read ORDER BY id.

New internal SPI Service\OrderedRowLockInterface with the Doctrine implementation Repository\DoctrineOrmOrderedRowLock (bundle's entity manager or the private one). It opens a transaction or joins the caller's (no savepoint); a failure, a lock conflict included, reaches the caller as the driver's own exception (so a host retry on 1213/1205 keeps working). Stateless (FrankenPHP worker mode, kernel reset false): rollback through TransactionRollback, a manager closed by a failed flush is reset through the registry. A TaskManager / BoardColumnManager built without it (hand-built, DDD host doubles) writes as before.

Read after the lock. A renumbering move locks its column's row first (every move that renumbers the column waits there), then re-reads the column's tasks with a locking read (OrderedRowLockInterface::lockedColumnTasks(), refreshing any out-of-date copy) and computes the new positions from the committed column, never from what it read before the lock.

Known limits (CHANGELOG and docs/SECURITY.md, "Concurrency: fixed lock order"): two-phase locking when one of these writes runs inside a caller's transaction that already holds rows (e.g. the Task API bulk update), and the import batches and provider sync writers (file or remote order). Both can still deadlock against a concurrent move; the database rolls one back for the caller to retry.

No behaviour change apart from the lock order: positions and written values are the same, events are dispatched after the write as before. One side effect: these writes now run in one transaction, so a move's renumbered siblings roll back with it if its final save fails (they used to stay saved).

Reviewed, not changed: Task API bulk update (already locks in id order), UnmatchedColumnsCleaner (already locks), import batches and provider sync writers (one run per board, in file/remote order, inside their own transaction), BoardColumnsApplier / folder status sets (structural writes inside the folder transaction), custom field update (one field row, then its values).

Type of Change

  • Bug fix (non-breaking change which fixes an issue)

Checklist

  • My code follows the project's coding standards (PSR-12)
  • I have performed a self-review of my code
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit/integration tests pass locally with my changes
  • I have updated docs/CHANGELOG.md if applicable

Testing

  • tests/Functional/Move/FixedLockOrderTest.php
    • Query log (every platform): move, column reorder and column delete issue the locking reads first (columns, then tasks, ids ascending) and every write after them; positions as before.
    • Real second connection (MySQL and PostgreSQL; skipped on SQLite): a subprocess (tests/Fixtures/Deadlock/lock_two_tasks.php) plays a concurrent move of the column's lowest task L while the real TaskManager::move() moves the highest task H to the top. Old order (no lock service): the move of H writes L then H, the old move of L takes H then L, and one of the two is chosen as the deadlock victim. Fixed order: both take L then H; the move waits and both finish with the expected positions. The other move is a raw two-row lock in the order of the code it stands for, not a second kernel.
    • Two concurrent moves (MySQL and PostgreSQL): a second process (tests/Fixtures/Deadlock/move_worker.php, own kernel, real TaskManager) holds the column and moves H to the top while the test's move, which read the column before it waited, inserts a task. The result is unique, contiguous positions in the committed order (H, new, L, M). Checked locally that this test fails without the re-read.
  • tests/Unit/Service/FixedLockOrderTest.php: plain-PHP lock double (DDD path), lock calls before the first save, same positions with and without the lock.
  • tests/Unit/Repository/DoctrineOrmOrderedRowLockTest.php: commit, join, rollback, failed rollback, rethrow as is.

Locally: php-cs-fixer, PHPStan, full default suite on SQLite (2620 tests), FixedLockOrderTest on MySQL 8.4 and PostgreSQL 16; CI runs the full real-database suites.

… 239 FR-013)

Task moves, column reorder, column delete and column retype now lock every
row they write before the first write, in one fixed order: the board
columns, then the tasks, each by ascending id (SELECT ... WHERE id IN (...)
ORDER BY id FOR UPDATE; a no-op on SQLite). Two concurrent writes can no
longer take rows in crossing order and deadlock (MySQL 1213, PostgreSQL
40P01). Positions and written values are unchanged; events are still
dispatched after the write; lock conflicts still reach the caller as the
driver's exception.

- New internal SPI Service\OrderedRowLockInterface with the Doctrine
  implementation Repository\DoctrineOrmOrderedRowLock (stateless, joins a
  caller's transaction, rolls back and resets a closed manager otherwise).
- TaskManager: move(), moveTo*() and a column-changing update() lock the
  entered column, then the tasks.
- BoardColumnManager: reorder(), delete() and a retyping update() lock the
  columns, then the tasks.
- TaskMover, TaskBoardRestructurer and DoctrineOrmTaskTreeStore read each
  tree level ordered by id.
- Tests: query-log order on every platform, and a real second connection on
  MySQL / PostgreSQL showing the old order deadlocks and the fixed one waits.
…the lock

A renumbering move now locks its column's row first (every move that
renumbers the column waits there), then re-reads the column's tasks with a
locking read and refreshes any copy that is out of date
(OrderedRowLockInterface::lockedColumnTasks()). A move that waited for a
concurrent one computes the new positions from the committed column instead
of the positions it read before the lock, so two concurrent moves in one
column end with unique, contiguous positions.

- Test: a second process moves a task through the real TaskManager while
  holding the column; the test's move ends with positions 0..n-1 computed
  from the committed order (fails without the re-read).
- Docs: known limits of the fixed order (two-phase locking inside a caller's
  transaction, import and provider sync) in CHANGELOG and SECURITY.md.
…lock (audit SF)

- SF-M1: the Task API's locked updates (bulk, If-Match / base_revision,
  parent) lock the columns their status fields enter before the task rows
  (lockedRevision), the fixed order; they no longer deadlock against a
  renumbering move or a column delete / retype.
- SF-L1: a move, moveTo*() or column-changing update() whose task changed
  board, column, archive or completion state after its checks writes
  nothing and throws TaskTreeRefusedException::conflict() (API 409
  PARENT_004; board flash and 409 on the drag).
- SF-L2: lockedColumnTasks() never refreshes an out-of-date copy that holds
  unsaved changes; it throws the conflict instead.
- SF-L3: moveTo*() and a column-changing update() take the next free
  position from the column as committed, after its lock.
- SF-L4: a column delete re-reads the deleted and target columns after its
  lock; a task that entered meanwhile moves too (policy asked for it).
- SF-L5: at the outermost level a deadlock / lock wait timeout hidden behind
  a failed savepoint rollback (MySQL 1305) reaches the caller as the DBAL
  DeadlockException / LockWaitTimeoutException.
- SF-I2: a move no longer locks the siblings it read before the lock; the
  column scan locks them in id order.
- SF-I1: the column scan uses the column_id index (EXPLAIN: ref, no
  filesort); gap-lock note in SECURITY.md. No new index.
- SF-I5: CHANGELOG scoped, SECURITY limits corrected, UPGRADING To 2.21.0.
- Tests: probes P1-P4 as regression tests (real second process on MySQL and
  PostgreSQL; P1 through the Task API), SQLite variants, UI conflict test.
…flicts (audit SF-2)

- SF-D1: TaskManager::create() with a parent and max_subtask_depth locks the
  column it enters (and reads its next position as committed) before
  lockedDepth() locks the parent's rows. Before, a renumbering move in that
  column (column held, waiting on the parent) deadlocked against it
  (MySQL 1213, PostgreSQL 40P01). Probe P5 is a two-process regression test
  through the real create(); it fails without the fix on both databases.
- Task API: a conflict on an existing task (status change, field update,
  checklist item) answers 409 TASK_409 with current_revision, also as the
  item's entry in BULK_001; PARENT_004 stays for re-parent and subtask
  conflicts (TaskApiRefusal::taskWriteRefused(), withinPlan(..., $task)).
- Docs: CHANGELOG, API.md, SECURITY.md; UPGRADING says a DDD host wires the
  lock into TaskApiTaskOperations and TaskManager together or into neither.
@HecFranco
HecFranco merged commit 443773e into main Nov 17, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant