Skip to content

Build identity state once in a core identity app - #1623

Open
dhruv8sh wants to merge 17 commits into
feat/openrtb-user-eidsfrom
feat/identity-app-in-core
Open

dhruv8sh wants to merge 17 commits into
feat/openrtb-user-eidsfrom
feat/identity-app-in-core

Conversation

@dhruv8sh

@dhruv8sh dhruv8sh commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Adds IdentityApp in core. It holds the settings, partner registry, EC record store and rate limiter, and an adapter builds it once from the platform services it supplies through IdentityPlatform. Identity routes therefore stop rebuilding the partner registry, record store and limiter on every request (SH-1).
  • Fastly's AppState now holds an IdentityApp with its EC record store and Edge Rate Limiting counter. Identify, batch sync, auction, page bids, the publisher fallback, EC finalization and post-delivery pull sync all use its registry, store and limiter. Fastly's limits are unchanged (ID-6, MVP).
  • Every adapter mounts the admin identity routes (/_ts/admin/ec, /_ts/admin/ec/{id}, /_ts/admin/eids) from core through IdentityApp::mount. A new guard test makes sure adapters reach identity only through the core app, and its list of exceptions can only shrink (SH-5).

Stacks on #1614 (feat/identify-batch-sync-local-all-adapters); review only the commits above it. #1624 (#1593) stacks on this one.

Decisions and scope

  • Only the admin routes are mounted from core. Identify and batch sync stay in the adapters and come into the core app with the EC lifecycle move, after Mirror provider EIDs to a cookie and run pull sync as an identity provider #1489 lands. Until then, the guard test's PENDING_LIFECYCLE_MOVE list names each adapter use that remains. The test fails when an entry stops matching, so the list can only shrink.
  • Startup refusal is left to T6. ID-1's "startup with identity configuration fails on an unsupported adapter" belongs with T6, which owns the record backend and decides which adapters have a supported store. Cloudflare and Spin build an IdentityApp with no record store, so admin EC lookup answers 501 from core. IdentityApp::build already refuses a record store supplied without a rate limiter, so an adapter cannot run batch or pull sync unlimited by omission.
  • GPT diagnostics preparation now runs on the admin routes on every adapter. Before, Axum ran gpt_diagnostics::prepare_request on every admin lookup and Cloudflare ran it on the EIDs echo. Fastly and Spin skipped it there. The routes mounted from core run it first on all three, so on Fastly and Spin the reserved console query and cookie are now stripped on these routes too, and a preparation error stops the handler.
  • The partner registry is still built twice at startup. validate_settings_for_runtime builds one and throws it away, and IdentityApp::build builds the one it keeps. Runtime settings validation runs inside settings loading, before any adapter builds the identity app. Sharing one registry would mean either changing the settings-loading return types in all four adapters, or caching the registry inside Settings, where it could go stale when settings change after validation (tests do this). The cost is one extra build per process start, so it is left as is.
  • CI change: .github/workflows/test.yml adds a "Guard adapter identity wiring" step that runs cargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test adapter_identity_guard, next to the parity tests.
  • Cloudflare and Spin still build app state per request. At the EdgeZero pin f84ffbbb, edgezero_adapter_cloudflare::run_app and edgezero_adapter_spin::run_app build the app on every request, so on those adapters the IdentityApp is built per request too. EdgeZero now offers dispatch_app for an app the caller keeps; adopting it is out of scope here (T8 for Cloudflare). Axum and Fastly build it once per process or sandbox. Axum's HTTP client stays per request, as the issue requires.

Rebase onto #1614's latest head

This branch was rebased onto 9552a89e0, which merged #1506. Each commit keeps its intent, adapted to the post-#1506 code:

  • On Fastly, EC finalization and pull sync moved into finalize_response and post_delivery.rs. FinalizeInputs and FinalizeState now carry the identity app instead of rebuilding a partner registry, record store and limiter.
  • Cut Trusted Server over to EdgeZero PR 275 #1506 already removed the per-request settings reload on finalize, so the commit that dropped it is no longer needed.
  • Core already shares http_error, so the error commit now only removes the identity app's own copy.

Changes

File Change
crates/trusted-server-core/src/ec/app.rs New IdentityApp and IdentityPlatform: build state once, refuse a record store without a limiter, mount the admin routes with GPT diagnostics preparation
crates/trusted-server-core/src/ec/mod.rs Export the app module
crates/trusted-server-core/src/ec/admin.rs Admin handlers are now private to the EC module
crates/trusted-server-core/src/ec/kv_backend.rs, ec/rate_limiter.rs Record stores and rate limiters must be Send + Sync; NoRateLimit for adapters without a limiter
crates/trusted-server-adapter-fastly/src/app.rs AppState holds the identity app; routes use its registry, store and limiter; admin routes mounted from core
crates/trusted-server-adapter-fastly/src/main.rs Finalize uses the retained identity app
crates/trusted-server-adapter-fastly/src/post_delivery.rs Pull sync uses the identity app's registry, store and limiter
crates/trusted-server-adapter-fastly/src/sandbox.rs Test: a retained app shares its identity state across requests
crates/trusted-server-adapter-axum/src/app.rs Build the identity app once; mount admin routes from core; no per-request partner registry
crates/trusted-server-adapter-cloudflare/src/app.rs, crates/trusted-server-adapter-spin/src/app.rs Mount admin routes from core; drop duplicate admin wrappers and the per-request registry
crates/trusted-server-integration-tests/tests/adapter_identity_guard.rs, Cargo.toml Guard test: comment-, literal- and nesting-aware scan of adapter non-test source
.github/workflows/test.yml Run the guard test in CI

Closes

Closes #1600

Acceptance criteria:

Spec

Spec: server-side identity on every adapter (SH-1, SH-5, ID-1, ID-3, ID-6); plan task T21.

Test plan

  • cargo test-fastly && cargo test-fastly-reuse && cargo test-axum && cargo test-cloudflare && cargo test-spin
  • cargo clippy-fastly && cargo clippy-axum && cargo clippy-cloudflare && cargo clippy-cloudflare-wasm && cargo clippy-spin-native && cargo clippy-spin-wasm && cargo clippy-cli && cargo clippy-codegen
  • cargo fmt --all -- --check
  • JS tests: cd crates/trusted-server-js/lib && npx vitest run
  • JS format: cd crates/trusted-server-js/lib && npm run format
  • Docs format: cd docs && npm run format
  • WASM build: cargo build --package trusted-server-adapter-fastly --release --target wasm32-wasip1
  • Manual testing via fastly compute serve
  • Other: integration tests --test parity and --test adapter_identity_guard with clippy on that crate; native core build-digest test and clippy; npm run lint; markdown prettier check; scripts/lint-wasm-clock.sh; ts dev lint domains --changed-vs origin/feat/identify-batch-sync-local-all-adapters

Checklist

  • Feature PRs: spec linked above, ending in a review log of two or more agent review rounds
  • Changes follow AGENTS.md conventions
  • No unwrap() in production code — use expect("should ...")
  • Uses log macros (not println!)
  • New code has tests
  • No secrets or credentials committed

A shared identity app built once holds both in state that EdgeZero handlers capture, which must be Send and Sync. Every existing implementation already is.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
IdentityApp holds the settings, partner registry, EC record store and rate limiter that adapters supply through IdentityPlatform. An adapter without a limiter gets NoRateLimit. No adapter uses it yet.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
IdentityApp::mount registers GET on /_ts/admin/ec, /_ts/admin/ec/{id} and /_ts/admin/eids with the registry and record store built once. Identify and batch sync stay in the adapters until the EC lifecycle moves into core.
Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
AppState now holds an IdentityApp with Fastly's EC record store and Edge Rate Limiting counter. Identify, batch sync, auction, page bids, the publisher fallback, EC finalization and pull sync use its registry, store and limiter instead of rebuilding them per request, and the admin lookups are mounted from core. Fastly's limits are unchanged.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>

# Conflicts:
#	crates/trusted-server-adapter-fastly/src/app.rs
#	crates/trusted-server-adapter-fastly/src/main.rs
The Axum dev server builds an IdentityApp once with no EC record store and no rate limit, since its deployer limits traffic at the load balancer. The per-request partner registry for the EIDs echo is gone. The HTTP client stays per request.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>

# Conflicts:
#	crates/trusted-server-adapter-axum/src/app.rs
…entity app

Both adapters build an IdentityApp with their app state and no EC record store, so admin EC lookup answers 501 from core. This drops their duplicate admin_ec_lookup_not_supported wrappers and the per-request partner registry for the EIDs echo.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>

# Conflicts:
#	crates/trusted-server-adapter-cloudflare/src/app.rs
#	crates/trusted-server-adapter-spin/src/app.rs
Scans adapter non-test source for direct identity entry points. Uses the lifecycle move has not yet taken into core are listed and must stay in use, so the list only shrinks.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
IdentityApp::build used to fall back to no limit whenever the platform left the limiter out, so an adapter that adds a record store could run batch and pull sync unlimited by omission.
Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
The old core test compared two reads of the same Arc field and could not fail. The new one seeds the store the platform supplied after mounting and reads it back on two requests, which fails if a request builds its own store. The Fastly sandbox test drops checks implied by its identity app comparison and its expect messages now follow the should convention.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
The request state cloned the identity graph per request but only checked whether it was present, since the entry point takes the graph from the identity app built once.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>

# Conflicts:
#	crates/trusted-server-adapter-fastly/src/app.rs
The identity app kept its own copy of the error-to-response mapping. It now uses trusted_server_core::error::http_error, which every adapter already shares, so admin routes mounted from core answer errors like every other route.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Only the core identity app calls the admin EC and EIDs lookups now, so the compiler enforces that adapters reach them through IdentityApp::mount and the guard test no longer lists them.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
The guard treated every cfg(test) as a whole item, so after a test-only struct field it also dropped the production lines that followed. It now strips comments and literal contents, ends a test-only field or statement at its own nesting depth, keeps scanning code under cfg(any(test, ..)), skips inner cfg(test) files and out-of-line test modules, and matches symbols on identifier boundaries.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>
Axum ran diagnostics request preparation on every admin lookup and Cloudflare on the EIDs echo, but the routes mounted from core skipped it, so the reserved console query and cookie reached the handlers and a preparation error no longer stopped them. Every adapter now runs it on all three admin routes; on Fastly and Spin this newly strips the reserved query and cookie there.

Signed-off-by: dhruv8sh <dhruv8sh@proton.me>

# Conflicts:
#	crates/trusted-server-core/src/ec/app.rs
@dhruv8sh dhruv8sh self-assigned this Nov 18, 2026
@dhruv8sh
dhruv8sh marked this pull request as ready for review November 18, 2026 10:27
@dhruv8sh dhruv8sh linked an issue Nov 18, 2026 that may be closed by this pull request
9 tasks
Core identity app. IdentityApp now carries the provider stack's identity
state as well as its own: the partner registry, the page and after-response
provider coordinators (#1487, #1489), the EC record store and the rate
limiter. IdentityPlatform takes an optional IdentityProviders (the registry
the coordinators were built against plus both coordinators); without one the
app builds the registry with PartnerRegistry::from_ec and runs no provider,
matching the base's non-Fastly adapters. The admin EC lookup awaits the async
handler from #1526.

Send and Sync. #1526 keeps EcKvStore free of Send and Sync bounds and tests a
non-Send store, so the T21 supertrait bounds on EcKvStore and RateLimiter are
dropped. The identity app instead holds its record store and rate limiter as
Send and Sync trait objects, and IdentityApp::identity_graph hands out a graph
over that one shared store through the new KvIdentityGraph::from_shared.
IdentityPlatform's identity_graph field becomes record_store accordingly.

Fastly app. AppState drops the base's partner_registry, page_identity and
after_response_identity fields for the identity app. build_identity_app builds
the kind-aware registry and both coordinators from the identity-providers
crate, then the app with Fastly's record store and rate limiter. Page lookups
and after-response preparation read their coordinators and rate limiter from
the identity app; identify, batch sync, auction, page bids and the publisher
fallback use its registry and graph. The admin lookups stay mounted by the
identity app, so the base's async admin early dispatch is removed.

Fastly entry point. finalize_response keeps the base's single block_on around
the async EC finalize, which collects the page lookups first. FinalizeInputs
keeps the base's AppState, and FinalizeState carries the identity app and the
after-response lookups. maybe_identity_graph is gone; the test-only
require_identity_graph reads the identity app's graph.

Post-delivery. The after-response providers, pull sync included, take the
partner registry, graph and rate limiter from the identity app carried in
FinalizeState. The base's pull-sync tests build their state through the
identity app.

Axum, Cloudflare and Spin. The identity app mounts the admin lookups; the
base's provider rejection at startup and #1614's local 501 and 405 partner API
answers are kept.

Tests. The core identity app tests use record stores and the async graph, and
two tests cover the carried registry and coordinators. The Fastly identity
fixture builds an identity app around its echo-provider coordinator. The
adapter identity guard replaces dispatch_pull_sync, which #1489 deleted, with
commit_after_response_lookups and also guards PartnerRegistry::from_ec.
@aram356
aram356 changed the base branch from feat/identify-batch-sync-local-all-adapters to feat/s2s-eid-collection-providers November 19, 2026 05:02
@aram356

aram356 commented Nov 19, 2026

Copy link
Copy Markdown
Collaborator

For the identity integration chain (#1608), this PR now stacks on #1489, the top of the provider stack, which carries #1506's EdgeZero cutover, #1614, #1525, #1526, #1487 and #1488. a2e54de38 merges #1489's head a7262c7db; git show --remerge-diff a2e54de38 shows the resolution. The base is now feat/s2s-eid-collection-providers, so the diff still shows only this PR's changes.

Area Resolution
Core ec/app.rs The identity app carries the provider stack's state: IdentityPlatform gains providers (IdentityProviders, the partner registry and the page and after-response coordinators); identity_graph() returns a graph over the one shared store through a new KvIdentityGraph::from_shared; the admin EC lookup awaits #1526's async handler
EcKvStore, RateLimiter Stay without a Send + Sync supertrait, because #1526 and #1487 test non-Send stores and limiters; the app holds them as Arc<dyn ... + Send + Sync>, which EdgeZero's handlers need. So IdentityPlatform.identity_graph became record_store
Fastly app.rs, main.rs, post_delivery.rs AppState holds identity: Arc<IdentityApp> in place of #1487's registry and coordinator fields. Page lookups are still collected in the async finalize under #1506's single block_on, and #1489's after-response lookups, pull sync included, take &IdentityApp. One FastlyRateLimiter serves batch sync and both lookup paths
Axum, Cloudflare, Spin app.rs This PR's admin routes from the identity app, with #1614's local 501 and 405 answers and reject_enabled_providers kept
adapter_identity_guard.rs dispatch_pull_sync, which #1489 deleted, is replaced by commit_after_response_lookups, and PartnerRegistry::from_ec joins the guarded entry points

Behavior this PR brings onto the new base: on Axum, Cloudflare and Spin the partner registry is built once at startup; on Fastly the admin lookups run GPT diagnostics preparation first, as on the other adapters.

Checked locally at a2e54de38: cargo fmt, all eight clippy aliases and the core build-digest clippy, scripts/lint-wasm-clock.sh, test-fastly (core 3,629, Fastly 244), test-fastly-reuse (244), test-axum, test-cloudflare, test-spin, parity (33), the adapter identity guard (9), build-digest (10), host CLI tests, docs and Markdown format, and the URL-host lint against #1489, all passing.

Takes #1489's new head e4ae9ca, which brings #1506's repin of
EdgeZero to rc/identity at 9d02d7b3. The merge is clean, and this PR's tree
changes only by that repin.
@aram356 aram356 mentioned this pull request Nov 19, 2026
20 of 21 tasks
The identity integration chain now runs #1489, #1612, #1623, so this PR
takes #1612's head 470f8e1, which sends EIDs in OpenRTB 2.6 user.eids. The
merge is clean, and this PR's tree changes only by #1612's own change.
aram356 added a commit that referenced this pull request Nov 19, 2026
The base is #1623's head 78e1f2f, which carries #1612 and #1506's repin of
EdgeZero to rc/identity 9d02d7b3. The tree
is the base plus this pull request's own changes against its previous base
branch. The cutover, the EdgeZero pin and the closed #1605 reconciliation
commits that this branch carried come from the base: EdgeZero stays pinned
to rc/identity 9d02d7b3 by rev, and the RC tag pin,
its Cargo.lock, the #1605 changelog lines, its S3 direct-client patch, its
Chrome version pin, its cache harness target-root change, its
browser-collector poll fixture and its boxed pre-dispatch test helper are
dropped in favor of the base.

Axum startup: main runs EdgeZero's run_production_app_with_initializer with
the native prepared-store initializer. EdgeZero installs the logger at
EDGEZERO__LOGGING__LEVEL as it does on the base, so the adapter's
SimpleLogger, its RUST_LOG parsing and its owns_logging override are not
revived, and the parse_log_level tests go with them. EdgeZero validates the
hosting settings before it installs that logger, so a fatal startup error
that finds no logger goes to standard error instead of being dropped, as
it was visible before.

Axum app: the base's shared identity app, its local identify and batch-sync
answers and its enabled-provider rejection stay as they are. The router
built by the initializer goes through routes_with_settings, so the
production path builds the same IdentityApp and rejects enabled providers
the same way. Hooks::routes returns the empty pre-initialization router,
Hooks::stores declares the config and secret stores, and every router
carries an empty native store snapshot in state.

Axum platform: the environment-backed config and secret stores are removed
and request services read the immutable prepared snapshots.

Tests and harnesses: the route startup test, the integration and EOF
harnesses and the smoke script use the prepared local config map and
EDGEZERO__SECRETS__ values. The startup, native acceptance and smoke
processes set EDGEZERO__LOGGING__LEVEL in place of RUST_LOG, and the
provider script drops its unused RUST_LOG value.

Docs: the README, the Axum guide and the native RC guide describe the
production runner, its fatal invalid host, port and log level values, and
the workspace EdgeZero pin 9d02d7b3 in place of the RC tag.

Cargo.lock: the base lock plus the 29 packages that the native AWS provider
features add, at the same versions as this pull request's own lock.
@aram356
aram356 changed the base branch from feat/s2s-eid-collection-providers to feat/openrtb-user-eids November 19, 2026 06:23
@aram356

aram356 commented Nov 19, 2026

Copy link
Copy Markdown
Collaborator

For the identity integration chain (#1608), this PR now stacks on #1612, and its base is feat/openrtb-user-eids. The chain is now one line: #1489 ← #1612 ← #1623 ← #1531 (A1) ← #1532 (A4) ← #1613 ← #1616 ← #1617 ← #1615. Every pull request in that line conflicts with at least one other, so each one takes the fix for the PR below it. #1612 is approved, so it sits lowest.

78e1f2f35 merges #1612's head 470f8e109. The merge is clean, and this PR's tree changes by exactly #1612's change.

aram356 added a commit that referenced this pull request Nov 19, 2026
Fastly entry point: maybe_identity_graph is gone, as in #1623, so the
conflicting hunk takes #1623's side. The conformance hook that #1613 put at
the top of maybe_identity_graph is dropped with it, and
require_identity_graph keeps #1623's identity-app signature.

Fastly identity app: the test-only conformance hook now sits where the
adapter picks the identity app's EC record store. While a conformance runner
assembles the application, a store named identity-conformance resolves to the
runner's recording fixture instead of a Fastly KV store, so the identity app
holds the fixture for every request. Production builds are unchanged.

Fastly conformance runner: the thread-local holds the recording store rather
than a graph, and the runner installs it around assemble_app instead of
around each navigation. The identity app captures it at assembly and every
request path (EC request state, finalization, identify, batch sync and
post-delivery) reads it through the identity app. The scenarios and their
assertions are unchanged.

Workflow: #1613's conformance steps and #1623's adapter identity guard step
merge without overlap.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

One identity app in core for every adapter

2 participants