Skip to content

Add shared identity conformance fixtures and Fastly coverage - #1613

Open
prk-Jr wants to merge 9 commits into
baseline/protected-geo-20261110from
feat/1586-identity-conformance
Open

prk-Jr wants to merge 9 commits into
baseline/protected-geo-20261110from
feat/1586-identity-conformance

Conversation

@prk-Jr

@prk-Jr prk-Jr commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Adds a shared identity conformance suite: one set of scenarios that checks an adapter never issues an Edge Cookie (ts-ec) without its stored record, and never drops an existing identity when geo is unavailable.
  • Runs all nine scenarios on Fastly through the real router, the production finalization callback and the full HTML body. Storage, origin and platform metadata are test fixtures, and native controls prove the assertions reject broken results.
  • Adds a CI launcher that fails if the selected adapter's test is missing or runs zero tests, and that refuses adapters that have no runner yet.

This PR adds tests and test infrastructure only. It changes no production identity behavior and adds identity to no other adapter.

Stacked on #1506 (spec/edgezero-pr275-migration, baseline cc811fc7aa0d234e447a868af5b8ab4ea359326a). Retarget to main after the cutover merges.

Changes

File Change
.github/workflows/test.yml Run the Fastly conformance suite, the launcher tests and the native rejection controls.
crates/trusted-server-adapter-fastly/src/identity_conformance.rs Run the shared scenarios through the actual Fastly application path with deterministic platform fixtures.
crates/trusted-server-adapter-fastly/src/main.rs Share the finalization callback between production and the runner; add a test-only identity store seam.
crates/trusted-server-adapter-fastly/src/platform.rs Add a test-only failed geo lookup fixture.
crates/trusted-server-core/src/ec/kv_backend.rs Expose the existing in-memory store behind the test/test-utils gates.
crates/trusted-server-core/src/test_support.rs Expose the conformance test-support module.
crates/trusted-server-core/src/test_support/identity_conformance.rs Shared scenarios, recording/failing store, cookie replay and expiry checks, and rejection controls.
crates/trusted-server-integration-tests/README.md How to run the suite, what it covers, and what it does not (including post-delivery work).
docs/superpowers/specs/2026-11-16-identity-conformance-design.md Approved design, feasibility limits, delivery boundaries and review log.
docs/superpowers/plans/2026-11-16-identity-conformance.md Implementation steps, progress and dependent follow-up work.
scripts/test-identity-conformance.mjs Require the selected adapter's exact conformance test to exist and pass.
scripts/test-identity-conformance.test.mjs Reject unavailable or unknown adapters and missing or zero-test runs.

Closes

Does not close an issue. This is the first slice of #1586, which cannot close until other planned work lands.

#1586 asks for three things. Where each one stands:

#1586 acceptance criterion Status Waiting on
The suite passes on each supported adapter (MVP: Fastly and Axum) Fastly: done in this PR. Axum: not yet #1574 adds identity to Axum. Axum has no identity feature today, so there is nothing to run the suite against yet. #1574's own criteria require running this suite.
Startup fails on an unsupported adapter when identity is configured Not started An "identity enabled" setting, delivered by #1600 and #1587. Using today's ec.ec_store instead would break startup for existing Axum, Cloudflare and Spin configs.
SH-5: no adapter calls EC code directly; one shared app runs everywhere Not started #1600 moves identity into one shared app in core. Until then, Fastly's own adapter code still calls EC generation and finalization directly.

Order of the remaining work: #1506 → #1600 → #1574, then the startup check and the SH-5 check. Cloudflare follows in #1588.

Spec

Spec: Identity conformance design

Plan: Identity conformance implementation plan

Both documents include two review rounds. The missing-browser-metadata scenario reflects today's browser gate and must change when #1592 permits absent JA4.

What the suite does not cover

  • Post-delivery work. The suite stops when the finalized response is delivered. Telemetry and EC pull sync are not run, so its release-ordering checks cover writes made during finalization only.
  • Real browser fingerprints. Local Viceroy provides no JA4, so the unmodified build cannot create a fresh identity over local HTTP. Fresh creation on the real ingress needs a deployed service.
  • Other adapters. Axum, Cloudflare and Spin are rejected explicitly by the launcher until their runners exist.

Test plan

Local checks use Rust 1.98.1, Node 24.20.0 and Viceroy 0.21.0.

  • cargo test-fastly && cargo test-fastly-reuse && cargo test-axum && cargo test-cloudflare && cargo test-spin
  • Target-matched adapter clippy aliases, including Cloudflare/Spin WASM, plus cargo clippy-cli && cargo clippy-codegen
  • cargo fmt --all -- --check
  • JS tests: 80 files / 3,299 tests passed, no type errors
  • JS format, full docs format, Markdown outside docs, and changed JS/YAML format
  • Fastly suite (nine scenarios), five launcher tests, and eight native rejection controls
  • Integration parity: 33 tests passed
  • Native template build-digest test and lint
  • WASM clock lint, CI Node pin tests/guard, and URL-host lint against the stacked base
  • Documentation proposal and release producer tests/shellcheck
  • Fastly absolute-deadline network fixtures: six tests passed
  • Cloudflare and Spin deadline fixture builds, lint and runtime probes (Wrangler 4.83.0, wasm-bindgen 0.2.126, Spin 4.1.0)
  • Headless Chrome lifecycle check (below)
  • Hosted CI on the latest PR revision

Negative controls. Disconnecting finalization made the suite fail the missing-cookie check. Treating missing geo as consent withdrawal made it fail at "should retain the reader's cookie". Both pass again when restored. Native controls also reject late writes, late failures, expired cookies and tombstones counted as live records.

Headless Chrome. Real Chrome through the local hostname proxy to the publisher page, with us_privacy=1YNN (Viceroy's default location is a US state that needs a consent signal):

Step Build Result
Fresh visit Unmodified release No ts-ec; consent allowed, refused at the bot gate (ja4=None)
Fresh visit Test build with a fixed JA4 One ts-ec and a live record for the same ID
Reload Test build Same cookie, record still live
GPC (Sec-GPC: 1) Test build Cookie deleted, record tombstoned
Next visit with GPC Test build Nothing created
Existing identity, no geo Test build Cookie and record kept across two visits

The test build differs from the reviewed code only by a one-line fixed JA4 fallback, so this proves the lifecycle mechanics, not the real production ingress. Server-side DataDome protection was disabled in the disposable local config, protected image assets were blocked, and only the HTML document was checked.

The supplemental native core --all-targets --features test-utils clippy run reports the pre-existing result_large_err lint in run_pre_dispatch_hook, reproduced on the unchanged cutover base. This PR does not touch it.

Checklist

@prk-Jr prk-Jr self-assigned this Nov 16, 2026
@prk-Jr prk-Jr linked an issue Nov 16, 2026 that may be closed by this pull request
3 tasks
…al before writes

The Fastly runner now uses the production native request read and prepare,
replacing only the client IP, geo result and JA4-backed device signals that
Viceroy cannot supply. Refused scenarios must stop before any write attempt,
with a native control proving the check rejects one. Docs describe the
failed-geo fixture and record current progress.
@prk-Jr
prk-Jr marked this pull request as ready for review November 17, 2026 05:28
Fastly entry point: serve keeps the base's request-local identity lookup
slot, dropped before and after the request, and sends through this pull
request's with_finalization helper, so the conformance runner and
production share one finalization callback. FinalizeInputs now carries
the app state, and the conformance runner passes the retained app's
state. The runner also drops leftover navigation lookups around each
request, as serve does.

EC store test support: the module opens to the test-utils feature as
this pull request intended. Every test-only double, including the base's
new write-failing store, stays behind cfg(test); only the in-memory store
is shared.

Conformance fixture: the recording store implements the async EcKvStore
trait and awaits the in-memory store. Scenario::verify and the native
rejection-control tests drive store reads and writes with block_on at
their synchronous test boundary, because the adapter runner already
blocks inside the Fastly lifecycle. The conditional-write test checks
the recording store's create-only insert directly, since the base
removed KvIdentityGraph::create_if_absent.

Fastly runner: a document navigation now sends Sec-Fetch-Mode: navigate
next to Sec-Fetch-Dest: document, and a subresource sends no-cors. The
base's page-load write permit requires both headers, as a browser sends
them.

The integration-tests README says post-delivery work now covers the
after-response identity lookups, pull sync among them.
@aram356
aram356 changed the base branch from spec/edgezero-pr275-migration to feat/s2s-eid-collection-providers November 19, 2026 05:37
@aram356

aram356 commented Nov 19, 2026

Copy link
Copy Markdown
Collaborator

For the identity integration chain (#1608), this PR now stacks on #1489, the top of the provider stack, which carries #1506's EdgeZero cutover and #1526's async store. 7611bc936 merges #1489's head a7262c7db, and the base is now feat/s2s-eid-collection-providers.

Area Resolution
Fastly main.rs serve keeps the base's navigation-lookup cleanup around each request, sends through this PR's with_finalization, and keeps the base's post-delivery callback
ec/kv_backend.rs The module is cfg(any(test, feature = "test-utils")) as this PR intends; every test double except InMemoryEcKv, including the base's WriteFailingEcKv, is #[cfg(test)]
Conformance suite RecordingStore implements the async EcKvStore; Scenario::verify, assert_created and the native rejection controls block at their synchronous test boundary, because the Fastly runner already blocks inside the Fastly lifecycle. The Fastly runner passes the app state to FinalizeInputs and drops leftover navigation lookups as serve does

Tests changed beyond the async conversion: identity_conformance_preserves_conditional_write_preconditions drives RecordingStore::insert with EcKvWriteMode::Add twice (Written, then PreconditionFailed), because the provider stack removed create_if_absent. The Fastly runner's document navigation now also sends Sec-Fetch-Mode: navigate, and the subresource scenario sends no-cors, because the provider stack's NavigationPermit requires both destination and mode.

Checked locally at 7611bc936: cargo fmt, all eight clippy aliases and the core build-digest clippy, scripts/lint-wasm-clock.sh, test-fastly, test-fastly-reuse, test-axum (64), test-cloudflare (66), test-spin (89), parity (33), build-digest (10), host CLI tests (972), docs and Markdown format, and the URL-host lint against #1489, all passing (test-fastly 4,066), plus scripts/test-identity-conformance.test.mjs (5), scripts/test-identity-conformance.mjs fastly (all 9 scenarios) and the core identity_conformance tests (9).

Takes #1489's new head e4ae9ca, which brings #1506's repin of
EdgeZero to rc/identity at 9d02d7b3. The merge is clean, and this PR's tree
changes only by that repin.
@aram356 aram356 mentioned this pull request Nov 19, 2026
20 of 21 tasks
Fastly entry point: maybe_identity_graph is gone, as in #1623, so the
conflicting hunk takes #1623's side. The conformance hook that #1613 put at
the top of maybe_identity_graph is dropped with it, and
require_identity_graph keeps #1623's identity-app signature.

Fastly identity app: the test-only conformance hook now sits where the
adapter picks the identity app's EC record store. While a conformance runner
assembles the application, a store named identity-conformance resolves to the
runner's recording fixture instead of a Fastly KV store, so the identity app
holds the fixture for every request. Production builds are unchanged.

Fastly conformance runner: the thread-local holds the recording store rather
than a graph, and the runner installs it around assemble_app instead of
around each navigation. The identity app captures it at assembly and every
request path (EC request state, finalization, identify, batch sync and
post-delivery) reads it through the identity app. The scenarios and their
assertions are unchanged.

Workflow: #1613's conformance steps and #1623's adapter identity guard step
merge without overlap.
Workflow: the Axum job keeps both #1613's native identity conformance
rejection controls step and #1532's protected CloudFront Geo tests step, in
that order, ahead of the finalization harness cleanup step.

Fastly conformance runner: the GDPR fixture geo takes #1532's GeoInfo shape,
converted as #1532 converted the same literal elsewhere. The empty city
becomes None, the continent stays EU, the zero coordinates become a complete
zero pair and the zero metro code stays zero. The scenario still resolves to
Germany, so the GDPR consent gate and every assertion are unchanged.

Fastly identity app: the test-only conformance hook in the identity app's
record-store selection is untouched by #1531 and #1532, so the fixture store
still reaches every request path through the identity app.
@aram356
aram356 changed the base branch from feat/s2s-eid-collection-providers to baseline/protected-geo-20261110 November 19, 2026 06:27
@aram356

aram356 commented Nov 19, 2026

Copy link
Copy Markdown
Collaborator

For the identity integration chain (#1608), this PR now stacks on #1532 (A4), and its base is baseline/protected-geo-20261110. The chain is one line: #1489 ← #1612 ← #1623 ← #1531 ← #1532 ← #1613 ← #1616 ← #1617 ← #1615. It conflicted with #1623 in Fastly main.rs and with #1532 in test.yml, so it takes both.

No test is dropped and no assertion changed. All 9 Fastly scenarios and the 9 native rejection controls run. With the hook temporarily disabled, identity_conformance_fastly fails, which shows the fixture reaches the request path. One doc line was already inaccurate before this port, and I left it: the design doc (line 96) says pull sync opens its own store through require_identity_graph.

Checked locally at c88a7e9a5: cargo fmt, all eight clippy aliases and the core build-digest clippy, scripts/lint-wasm-clock.sh, test-fastly (4,101), test-fastly-reuse (245), test-cloudflare (69), test-spin (89), parity (33), build-digest (10), docs and Markdown format, and the URL-host lint against #1532, all passing. Also passing: node --test scripts/test-identity-conformance.test.mjs (5 of 5), node scripts/test-identity-conformance.mjs fastly, the core identity_conformance controls (9 of 9) and #1623's adapter_identity_guard (9 of 9).

Two failures are macOS-only:

  • test-axum gives 95 passed and 1 failed. The failure is Extract native prepared startup on the demo baseline #1531's managed_ingress_ignores_direct_spoof_and_isolates_two_gateway_clients, which binds 127.0.0.2, an address macOS does not route.
  • The first host CLI run had two real-Chrome tests time out waiting for the browser to exit. Both passed alone, and the whole script then passed (972).

Takes #1532's new head fdfad29, which carries #1531's fix that ignores the
two-source ingress acceptance test outside Linux. The merge is clean, and this
PR's tree changes only by that attribute.
Takes #1532's new head a557a4e, which carries #1531's change that runs the
ingress acceptance test from 127.0.0.1 only. The merge is clean, and this PR's
tree changes only by that test change.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Identity conformance suite that runs against each adapter, Fastly and Axum first

2 participants