Repository navigation
Conversation
A test can now prove which scenario a fake camera is playing before it asserts on it. The four tests that need a live camera on 127.0.0.1:8443 get the integration marker.
…o.json scenario.json gains an expected block (alerts, sequences, location, timeout_s), validated offline by tests/test_scenarios.py. tests/e2e waits for the first alert, then for every fake camera to finish, and compares what the API returns with that block. Tests that need Postgres or S3 get the integration marker so the offline suite passes with no stack.
Fresh stack, in-repo images built, engine and front profiles up, then the e2e tests. The exit code is kept and the compose logs, ps and junit go to e2e-artifacts/. The three images with no arm64 manifest are pinned to linux/amd64 so the stack starts on Apple Silicon.
This was referenced Nov 12, 2026
Author
|
@MateoLostanlen @fe51 can you review these? I can't request reviews on this repo (read access only). Each one stacks on the previous, so in this order:
Before #73 merges, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
#65 gives us the fake camera and three scenarios, but nothing waits for the engine to produce an alert and checks what reached the API. No single command runs the chain either, and the expected result of a scenario is only written in prose (#51).
What this adds
make e2e SCENARIO=<name>: fresh stack,up -d --wait, pytest ontests/e2e, logs ine2e-artifacts/.expectedblock inscenario.json(alerts, sequences, location, timeout).tests/test_scenarios.pyvalidates it offline for every scenario.tests/e2e/: 5 tests reading that block (first alert, alert count, sequences per camera and preset, detection images, location).GET /scenarioreturns the scenario name -> the tests stop if the cameras play something else thanSCENARIO.make testskipstests/e2eand the tests that need a live camera (newintegrationmarker). The CI job skipstests/e2e.Example
data/scenarios/smoke-then-clear/scenario.json(7 sequences, one per pose):Checks
pytest tests --ignore=tests/e2e -m "not integration"make e2e SCENARIO=smoke-then-clearmake e2e SCENARIO=never-smokeuv run:tests/requirements.txtinstalls neither on 3.14 (psycopg2-binary 2.9.9) nor on 3.9 (pytest 9.0.3)Timings are single runs on an arm64 Mac, app images under amd64 emulation.
What it does not prove yet
never-smokepasses but cannot fail: 1 frame per camera, and the engine skips a frame identical to the previous one. Asked on feat: fake Reolink camera driven by scenarios #65.smoke-then-clearexpects 5 alerts and 7 sequences because the 7 poses play the same frames (same counts on 3 runs). The "one alert" baseline of Baseline end-to-end test #52 needs one frame folder per preset -> next PR, stacked on this one.false-positivehas noexpectedblock yet.Part of #52, stacked on #65.