Skip to content

test: first browser tests with Playwright - #74

Open
Acruve15 wants to merge 7 commits into
pyronear:feat/fake-reolink-camerafrom
Acruve15:alexis/e2e-playwright
Open

Acruve15 wants to merge 7 commits into
pyronear:feat/fake-reolink-camerafrom
Acruve15:alexis/e2e-playwright

Conversation

@Acruve15

Copy link
Copy Markdown

Problem

#72 and #73 check the alert through the API only. Nothing opens the platform -> a broken alert card, player or image URL would still pass. #53 asks for a first browser test on the same scenario, smoke-one-camera.

What this adds

  • tests/e2e/ui/: pytest-playwright, Chromium on the host (the platform and the presigned image URLs point at localhost).
  • One login per session through the real login page (storage state). Locale en-US and timezone UTC pinned.
  • 3 read-only tests: the alert card names croix-augas-01, the player image is loaded and is one the API serves for the sequence, the close-up is one of its crops and can be hidden and shown.
  • make e2e runs the API tests then the browser tests, the first failing code wins. make e2e-install installs Chromium, once per machine.
  • A failing browser test leaves a trace and a screenshot in e2e-artifacts/playwright/.

Flow

Example

$ make e2e-install
$ make e2e SCENARIO=smoke-one-camera
================== 5 passed, 2 warnings in 616.68s (0:10:16) ===================
========================= 3 passed, 1 warning in 3.42s =========================

Only the browser tests, on a stack that is already up:

$ SCENARIO=smoke-one-camera uv run --python 3.12 --with-requirements tests/requirements.txt pytest tests/e2e/ui --headed

Checks

Check Result
make e2e SCENARIO=smoke-one-camera, fresh stack exit 0, 682 s wall, 5 API tests and 3 browser tests passed
Browser tests alone on a live stack, 4 runs in a row 3 passed each time, under 4 s
Logins per session (count of POST /login/creds from the browser) 1
Forced failure (wrong camera name) 1 failed, 2 passed, trace.zip and test-failed-1.png in e2e-artifacts/playwright/
E2E_PLATFORM_URL=http://localhost:1 make e2e SCENARIO=smoke-one-camera (platform unreachable) 5 API tests passed, 3 browser tests in error at login, make exits 2, compose.log and ps.txt still written
Offline suite: pytest tests --ignore=tests/e2e -m "not integration" 35 passed, 6 deselected, nothing collected from tests/e2e
black 23.3.0 and ruff 0.0.290, run by hand clean

Single runs on an arm64 Mac, app images under amd64 emulation.

Open points

  • The platform has no data-testid -> selectors go by role and English text (Login, Detected by :, Hide close-up). The images have no alt -> they are found by URL path against url and crop_url from the API. A small PR on new-pyro-platform adding test ids would make this sturdier.
  • The player plays on its own -> the check is "an image of this sequence is loaded", not "the image of one given detection".
  • If no alert came, the browser run waits for it again before failing (up to 10 min more).
  • Shows the 6 commits of feat: add make e2e and the first end-to-end API tests #72 and feat: one frame folder per preset and the smoke-one-camera baseline #73 until they merge, the last one is new.

Closes #53.

A test can now prove which scenario a fake camera is playing before it asserts on it. The four tests that need a live camera on 127.0.0.1:8443 get the integration marker.
…o.json

scenario.json gains an expected block (alerts, sequences, location, timeout_s), validated offline by tests/test_scenarios.py. tests/e2e waits for the first alert, then for every fake camera to finish, and compares what the API returns with that block. Tests that need Postgres or S3 get the integration marker so the offline suite passes with no stack.
Fresh stack, in-repo images built, engine and front profiles up, then the e2e tests. The exit code is kept and the compose logs, ps and junit go to e2e-artifacts/. The three images with no arm64 manifest are pinned to linux/amd64 so the stack starts on Apple Silicon.
Writes the frames into <camera>/<preset>/ and takes the sequence seen on the pose whose patrol_id is that preset. Without --preset nothing changes.
A preset plays <scenario>/<camera>/<preset>/*.jpg when the folder exists and falls back to the camera-level frames, so the existing scenarios are unchanged. finished now waits for every preset declared in poses, and GET /scenario reports visits, frame and frames per preset. The e2e never-polled abort keys on those visits.
Preset 3 of croix-augas-01 plays 18 frames of prod alert 49767 then a clear frame, the six other poses hold one clear frame. Expected: one alert, one sequence on that pose. SCENARIOS_URL can now be overridden, so the images can be fetched from another release.
Three read-only tests in tests/e2e/ui on the smoke-one-camera alert: the alert
card names the camera, the player image is loaded and is one the API serves for
the sequence, the close-up is one of its crops and can be hidden and shown.

One login for the session through the real login page, reused through the
storage state. make e2e runs the API tests then the browser tests, and
make e2e-install installs Chromium.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant