Skip to content

feat: add scheduler backend (incomplete) - #1282

Draft
kvlonge wants to merge 115 commits into
mainfrom
feat/orchestration-backend
Draft

kvlonge wants to merge 115 commits into
mainfrom
feat/orchestration-backend

Conversation

@kvlonge

@kvlonge kvlonge commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Why

Native orchestration backend; Python locations/workers; UI separate.

Changes

  • SQLite/PostgreSQL storage, fenced scheduling, immutable snapshots, API/watchdog and audited administration.
  • Pinned workers; atomic groups, frozen retries/backfills, calendars, webhooks and scoped sensors.
  • Bounded histories/reconciliation. Complete catalog scans alone authorize removal; reads enforce live View grants and exact namespaces.
  • Indexed build evidence and planner identities separate physical observations, definitions and builds. Source-data remains unknown: consumed-input provenance is not recorded.
  • Hourly metadata-only statistics with bounded history and independent cleanup.
  • Frozen task/asset/check source; attached-check observations feed sensors without expanding runnable membership. Cancellation is not failure health.
  • Optional decision_id reuses durable denial IDs; authorization, audit, CSRF and admin boundaries unchanged.
  • Restore native input capture; fix fault fixtures. Separate PostgreSQL fixture databases preserve within-contract replicas and production locking.
  • Incomplete: custom/node grants, signed approvals, SSO groups/CLI login, broader retention/rebuild, non-Linux recovery and consumed-input provenance.

Verification

  • Capture/coverage/denial: 19 passes; attached checks: 5. Generated client, exact make check-ci and hygiene pass; bounded reviews found no blockers.
  • Native default-concurrent suite: 77/77 twice; baseline 76/77 then77/77. Lock sampling confirms cross-fixture contention; budgets/assertions unchanged.
  • Harness-only isolation adds no project/lifecycle shape; existing DuckShop cases unchanged.
  • Prior exact-head native/goldens passed; CLI delta/full text comparison failed on an extra progress line, exact local case passes. New CI pending, not green.
  • Evidence retained; normal Cargo jobs, no OOM. Draft/unmerged/auto-merge off, no release/deployment/UI. Check-only retries and fuller denial explanations remain decisions.

Preserve in-progress storage, scheduling, claims, credentials and native API/CLI work. The orchestration backend remains incomplete and is not ready for delivery.

Amp-Thread-ID: https://ampcode.com/threads/T-01a18715-4e00-708e-a87d-6bd109748d66
Add claim-scoped execution inputs and native snapshot drift validation.

Incomplete orchestration backend checkpoint; worker launch and the remaining control-plane integration are not implemented yet. Targeted SQLite/PostgreSQL tests pass; full static readiness is pending.

Amp-Thread-ID: https://ampcode.com/threads/T-01a18715-4e00-708e-a87d-6bd109748d66
Verify pinned inputs before start and expose retryable run reports and cancellation heartbeats over the authenticated API. Disable redirects for credentialed requests and cover the lifecycle against SQLite and PostgreSQL. Keep the orchestration PR explicitly incomplete pending actual launchers and the remaining backend scope.

Amp-Thread-ID: https://ampcode.com/threads/T-01a18715-4e00-708e-a87d-6bd109748d66
Separate lease expiry, replica connections, terminal run fixtures and fault cases so each integration binary imports only the support it uses. Preserve SQLite and PostgreSQL transaction coverage.

Amp-Thread-ID: https://ampcode.com/threads/T-01a18715-4e00-708e-a87d-6bd109748d66
Observe heartbeat loss in fenced, bounded transactions while retaining physical write claims. Recover through existing worker heartbeats and continue full batches without an idle delay.

Amp-Thread-ID: https://ampcode.com/threads/T-01a18715-4e00-708e-a87d-6bd109748d66
Own worker heartbeats on a dedicated native runtime and connection pool. Propagate cancellation and revoked credentials without cancelling work on transient API failures; stop and join with the execution owner.

Amp-Thread-ID: https://ampcode.com/threads/T-01a18715-4e00-708e-a87d-6bd109748d66
Move bounded client construction, credential redaction, URL validation and JSON response handling to the API owner so worker and location flows can share them. Keep independent heartbeat pools and worker-facing error categories.

Amp-Thread-ID: https://ampcode.com/threads/T-01a18715-4e00-708e-a87d-6bd109748d66
Add privileged native administration with JSON results, bounded location credentials and atomic security audit. Detect lost result delivery without claiming committed mutations rolled back.

Amp-Thread-ID: https://ampcode.com/threads/T-01a18715-4e00-708e-a87d-6bd109748d66
Keep the public Python API limited to authoring records and native delegation. Resolve selectors and validate schedule data in Rust before publication. Preserve stable shorthand trigger names across cron edits.

Add the explicitly approved narrow Python facade boundary entries without increasing movable Python or native crossing budgets.

Amp-Thread-ID: https://ampcode.com/threads/T-01a18715-4e00-708e-a87d-6bd109748d66
Collect exported jobs and nested factory containers natively, deduplicating aliases and terminating cyclic containers. Explicit empty declarations skip user module execution. Remove failed imports from the module cache so publication retries cannot reuse partial definitions.

Amp-Thread-ID: https://ampcode.com/threads/T-01a18715-4e00-708e-a87d-6bd109748d66
Verify retained compiler baselines before worker startup and join native heartbeat work on interpreter-side close. Preserve claim identity and report retry keys without treating resource teardown as physical warehouse completion.

Amp-Thread-ID: https://ampcode.com/threads/T-01a18715-4e00-708e-a87d-6bd109748d66
Restrict forward and reverse cron expansion and pure tick planning to the scheduler. Preserve scheduling bodies and all existing fault, DST and simulation coverage.

Amp-Thread-ID: https://ampcode.com/threads/T-01a18715-4e00-708e-a87d-6bd109748d66
Expose owner-only cancellation with stable retry identity. Commit lifecycle facts, projections and security audit together while retaining active physical claims until termination.

Amp-Thread-ID: https://ampcode.com/threads/T-01a18715-4e00-708e-a87d-6bd109748d66
Resolve manual requests against the current published job inside one write transaction. Preserve the accepted snapshot and execution inputs across retries after redeployment, and commit queue publication with the trusted owner audit.

Amp-Thread-ID: https://ampcode.com/threads/T-01a18715-4e00-708e-a87d-6bd109748d66
Compile in the project interpreter and retain native snapshots through publication. Keep JSON results separate from progress and distinguish uncertain acknowledgements from failed result delivery.

Amp-Thread-ID: https://ampcode.com/threads/T-01a18715-4e00-708e-a87d-6bd109748d66
Preserve immutable model-move and compatibility-view facts through the existing fenced warehouse history pipeline, with nullable legacy field support and real-writer acceptance fixtures.

Amp-Thread-ID: https://ampcode.com/threads/T-01a18715-4e00-708e-a87d-6bd109748d66
Include task, asset and check source in immutable node content so workers reject edits before executing models. Retain source locations rather than additional callables or source reads during ordinary compilation.

Cover unchanged and edited attached checks through real scheduled workers, plus relocatable task, asset and check snapshots using DuckShop fixtures.

Amp-Thread-ID: https://ampcode.com/threads/T-01a18715-4e00-708e-a87d-6bd109748d66
Report claim-bound actual check starts and outcomes without expanding frozen runnable membership, write claims or node counters. Feed existing sensors from check observations and preserve decisive independent health.

Refresh cancellation before classifying a failed check result. Cover heartbeat-only driver cancellation, real outcome workers, failure-triggered inventory execution, and SQLite/PostgreSQL authority and history boundaries with synthetic fixtures.
Restore execution-input capture without relaxing coverage. Expose existing durable authorization decision IDs through the typed error body without changing permissions or error text.

Keep cleanup fault injection scoped and release its sleeping backend after reset. Move blocking SQLite fault setup off the async runtime and make driver-cancellation work independent of machine speed.

Amp-Thread-ID: https://ampcode.com/threads/T-01a18715-4e00-708e-a87d-6bd109748d66
Schemas do not isolate PostgreSQL advisory locks. Give unrelated native contracts separate disposable databases while retaining each contract’s schema, replicas and production writer fence.

Keep default test concurrency, deadline budgets and all assertions unchanged. Update harness feedback to describe database isolation.

Amp-Thread-ID: https://ampcode.com/threads/T-01a18715-4e00-708e-a87d-6bd109748d66

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant