From 62ef87b94dc707cea7f9522dd94aa6395053ff6f Mon Sep 17 00:00:00 2001 From: Mateusz Wiktor Date: Tue, 17 Nov 2026 09:33:18 +0100 Subject: [PATCH 01/13] feat: add live integration testing setup and configurations - Updated .gitignore to include environment files for live integration tests. - Enhanced .mocharc.json to exclude live integration tests from general test runs. - Added multiple scripts in package.json for running various live integration tests. - Introduced GitHub Actions workflow for live integration testing, ensuring secure handling of secrets and scheduled runs. - Implemented network blocking in test setup to prevent real network calls during tests. - Added new test cases for heartbeat failure reporting and HTTP status code handling. - Created example environment files for live integration tier configuration. --- .github/workflows/live-integration.yml | 159 ++ .gitignore | 5 + .mocharc.json | 3 +- package.json | 9 + .../components/subscription_manager.test.ts | 61 + .../endpoints/data_sync/fixtures.ts | 4 +- .../endpoints/fetch_messages.test.ts | 288 +-- test/integration/endpoints/history.test.ts | 76 +- .../endpoints/message_actions.test.ts | 423 +---- .../endpoints/objects/channel.test.ts | 125 +- test/integration/endpoints/time.test.ts | 48 + test/integration/live/.env.example | 16 + test/integration/live/.eslintrc.cjs | 10 + test/integration/live/.mocharc.cjs | 11 + test/integration/live/.mocharc.endpoints.cjs | 17 + test/integration/live/README.md | 146 ++ .../access-manager-legacy.test.ts | 249 +++ .../live/api-reference/access-manager.test.ts | 457 +++++ .../live/api-reference/channel-groups.test.ts | 127 ++ .../configuration-legacy.test.ts | 201 +++ .../live/api-reference/configuration.test.ts | 234 +++ .../live/api-reference/datasync.test.ts | 1397 +++++++++++++++ .../api-reference/encryption-legacy.test.ts | 62 + .../live/api-reference/encryption.test.ts | 119 ++ .../api-reference/file-sharing-legacy.test.ts | 88 + .../live/api-reference/file-sharing.test.ts | 263 +++ .../api-reference/message-actions.test.ts | 136 ++ .../message-persistence-legacy.test.ts | 585 ++++++ .../api-reference/message-persistence.test.ts | 246 +++ .../live/api-reference/misc.test.ts | 94 + .../live/api-reference/mobile-push.test.ts | 363 ++++ .../live/api-reference/objects.test.ts | 726 ++++++++ .../live/api-reference/presence.test.ts | 325 ++++ .../live/api-reference/publish-legacy.test.ts | 59 + .../live/api-reference/publish.test.ts | 255 +++ .../api-reference/subscribe-legacy.test.ts | 632 +++++++ .../live/api-reference/subscribe.test.ts | 467 +++++ test/integration/live/coverage.json | 1587 +++++++++++++++++ .../data_sync/events.channel.test.ts | 27 +- .../endpoints/data_sync/events.entity.test.ts | 27 +- .../data_sync/events.membership.test.ts | 28 +- .../data_sync/events.relationship.test.ts | 28 +- .../endpoints/data_sync/events.user.test.ts | 27 +- .../endpoints/data_sync/projections.test.ts | 129 +- .../live/endpoints/fetch_messages.test.ts | 145 ++ .../live/endpoints/history.test.ts | 65 + .../live/endpoints/message_actions.test.ts | 235 +++ .../integration/live/fixtures/cat_picture.jpg | Bin 0 -> 134 bytes .../logging-file-logger.test.ts | 103 ++ .../live/getting-started/logging.test.ts | 457 +++++ .../live/getting-started/quickstart-node.mjs | 44 + .../getting-started/quickstart-node.test.ts | 171 ++ .../set-up-your-environment.test.ts | 174 ++ .../getting-started/status-events.test.ts | 443 +++++ .../getting-started/troubleshooting.test.ts | 86 + .../live/migration-guides/v11.test.ts | 120 ++ .../live/migration-guides/v12.test.ts | 59 + .../live/migration-guides/v8.test.ts | 54 + .../live/provision/datasync-classes.json | 63 + .../live/provision/provision-datasync.ts | 359 ++++ .../live/shared/import-pubnub.test.ts | 47 + test/integration/live/snippet-exceptions.json | 132 ++ .../live/support/access-manager.ts | 3 + test/integration/live/support/assertions.ts | 45 + test/integration/live/support/clients.ts | 110 ++ .../live/support/compare-snippets.ts | 51 + test/integration/live/support/console.ts | 48 + .../live/support/datasync-fragments.ts | 48 + .../support/datasync-live-helpers.ts} | 120 +- .../live/support/datasync-state.ts | 32 + test/integration/live/support/docs-check.ts | 552 ++++++ test/integration/live/support/env.ts | 173 ++ test/integration/live/support/file-workdir.ts | 29 + test/integration/live/support/history.ts | 122 ++ test/integration/live/support/hooks.ts | 121 ++ test/integration/live/support/janitor-cli.ts | 28 + .../live/support/janitor-datasync.ts | 155 ++ test/integration/live/support/janitor-util.ts | 55 + test/integration/live/support/janitor.ts | 429 +++++ test/integration/live/support/lint-markers.ts | 479 +++++ .../integration/live/support/lint-selftest.ts | 250 +++ test/integration/live/support/manifest.ts | 199 +++ test/integration/live/support/placeholders.ts | 32 + test/integration/live/support/preflight.ts | 204 +++ test/integration/live/support/proxy.ts | 75 + test/integration/live/support/resources.ts | 218 +++ test/integration/live/support/retry.ts | 86 + test/integration/live/support/snippet-scan.ts | 541 ++++++ test/integration/live/support/snippets.ts | 126 ++ .../live/support/subscribe-groups.ts | 38 + test/integration/live/support/tsc-scratch.ts | 67 + test/integration/live/support/util.ts | 10 + test/integration/live/support/wait.ts | 284 +++ test/integration/live/tsconfig.json | 22 + test/integration/live/tsconfig.run.json | 9 + .../live/use-cases/automated-polling.test.ts | 225 +++ .../use-cases/fan-behavior-management.test.ts | 521 ++++++ .../live/use-cases/fan-re-engagement.test.ts | 291 +++ .../use-cases/game-chat-moderation.test.ts | 290 +++ .../live/use-cases/live-commentary.test.ts | 175 ++ .../live-event-rate-limiting.test.ts | 137 ++ .../live/use-cases/live-polls.test.ts | 278 +++ .../live/use-cases/match-stats.test.ts | 158 ++ .../live/use-cases/real-time-ads.test.ts | 123 ++ .../live/use-cases/real-time-chat.test.ts | 286 +++ .../live/use-cases/score-alerts.test.ts | 296 +++ test/setup-why.ts | 19 +- 107 files changed, 19630 insertions(+), 1026 deletions(-) create mode 100644 .github/workflows/live-integration.yml create mode 100644 test/integration/live/.env.example create mode 100644 test/integration/live/.eslintrc.cjs create mode 100644 test/integration/live/.mocharc.cjs create mode 100644 test/integration/live/.mocharc.endpoints.cjs create mode 100644 test/integration/live/README.md create mode 100644 test/integration/live/api-reference/access-manager-legacy.test.ts create mode 100644 test/integration/live/api-reference/access-manager.test.ts create mode 100644 test/integration/live/api-reference/channel-groups.test.ts create mode 100644 test/integration/live/api-reference/configuration-legacy.test.ts create mode 100644 test/integration/live/api-reference/configuration.test.ts create mode 100644 test/integration/live/api-reference/datasync.test.ts create mode 100644 test/integration/live/api-reference/encryption-legacy.test.ts create mode 100644 test/integration/live/api-reference/encryption.test.ts create mode 100644 test/integration/live/api-reference/file-sharing-legacy.test.ts create mode 100644 test/integration/live/api-reference/file-sharing.test.ts create mode 100644 test/integration/live/api-reference/message-actions.test.ts create mode 100644 test/integration/live/api-reference/message-persistence-legacy.test.ts create mode 100644 test/integration/live/api-reference/message-persistence.test.ts create mode 100644 test/integration/live/api-reference/misc.test.ts create mode 100644 test/integration/live/api-reference/mobile-push.test.ts create mode 100644 test/integration/live/api-reference/objects.test.ts create mode 100644 test/integration/live/api-reference/presence.test.ts create mode 100644 test/integration/live/api-reference/publish-legacy.test.ts create mode 100644 test/integration/live/api-reference/publish.test.ts create mode 100644 test/integration/live/api-reference/subscribe-legacy.test.ts create mode 100644 test/integration/live/api-reference/subscribe.test.ts create mode 100644 test/integration/live/coverage.json rename test/integration/{ => live}/endpoints/data_sync/events.channel.test.ts (81%) rename test/integration/{ => live}/endpoints/data_sync/events.entity.test.ts (82%) rename test/integration/{ => live}/endpoints/data_sync/events.membership.test.ts (86%) rename test/integration/{ => live}/endpoints/data_sync/events.relationship.test.ts (89%) rename test/integration/{ => live}/endpoints/data_sync/events.user.test.ts (81%) rename test/integration/{ => live}/endpoints/data_sync/projections.test.ts (92%) create mode 100644 test/integration/live/endpoints/fetch_messages.test.ts create mode 100644 test/integration/live/endpoints/history.test.ts create mode 100644 test/integration/live/endpoints/message_actions.test.ts create mode 100644 test/integration/live/fixtures/cat_picture.jpg create mode 100644 test/integration/live/getting-started/logging-file-logger.test.ts create mode 100644 test/integration/live/getting-started/logging.test.ts create mode 100644 test/integration/live/getting-started/quickstart-node.mjs create mode 100644 test/integration/live/getting-started/quickstart-node.test.ts create mode 100644 test/integration/live/getting-started/set-up-your-environment.test.ts create mode 100644 test/integration/live/getting-started/status-events.test.ts create mode 100644 test/integration/live/getting-started/troubleshooting.test.ts create mode 100644 test/integration/live/migration-guides/v11.test.ts create mode 100644 test/integration/live/migration-guides/v12.test.ts create mode 100644 test/integration/live/migration-guides/v8.test.ts create mode 100644 test/integration/live/provision/datasync-classes.json create mode 100644 test/integration/live/provision/provision-datasync.ts create mode 100644 test/integration/live/shared/import-pubnub.test.ts create mode 100644 test/integration/live/snippet-exceptions.json create mode 100644 test/integration/live/support/access-manager.ts create mode 100644 test/integration/live/support/assertions.ts create mode 100644 test/integration/live/support/clients.ts create mode 100644 test/integration/live/support/compare-snippets.ts create mode 100644 test/integration/live/support/console.ts create mode 100644 test/integration/live/support/datasync-fragments.ts rename test/integration/{endpoints/data_sync/helpers.ts => live/support/datasync-live-helpers.ts} (85%) create mode 100644 test/integration/live/support/datasync-state.ts create mode 100644 test/integration/live/support/docs-check.ts create mode 100644 test/integration/live/support/env.ts create mode 100644 test/integration/live/support/file-workdir.ts create mode 100644 test/integration/live/support/history.ts create mode 100644 test/integration/live/support/hooks.ts create mode 100644 test/integration/live/support/janitor-cli.ts create mode 100644 test/integration/live/support/janitor-datasync.ts create mode 100644 test/integration/live/support/janitor-util.ts create mode 100644 test/integration/live/support/janitor.ts create mode 100644 test/integration/live/support/lint-markers.ts create mode 100644 test/integration/live/support/lint-selftest.ts create mode 100644 test/integration/live/support/manifest.ts create mode 100644 test/integration/live/support/placeholders.ts create mode 100644 test/integration/live/support/preflight.ts create mode 100644 test/integration/live/support/proxy.ts create mode 100644 test/integration/live/support/resources.ts create mode 100644 test/integration/live/support/retry.ts create mode 100644 test/integration/live/support/snippet-scan.ts create mode 100644 test/integration/live/support/snippets.ts create mode 100644 test/integration/live/support/subscribe-groups.ts create mode 100644 test/integration/live/support/tsc-scratch.ts create mode 100644 test/integration/live/support/util.ts create mode 100644 test/integration/live/support/wait.ts create mode 100644 test/integration/live/tsconfig.json create mode 100644 test/integration/live/tsconfig.run.json create mode 100644 test/integration/live/use-cases/automated-polling.test.ts create mode 100644 test/integration/live/use-cases/fan-behavior-management.test.ts create mode 100644 test/integration/live/use-cases/fan-re-engagement.test.ts create mode 100644 test/integration/live/use-cases/game-chat-moderation.test.ts create mode 100644 test/integration/live/use-cases/live-commentary.test.ts create mode 100644 test/integration/live/use-cases/live-event-rate-limiting.test.ts create mode 100644 test/integration/live/use-cases/live-polls.test.ts create mode 100644 test/integration/live/use-cases/match-stats.test.ts create mode 100644 test/integration/live/use-cases/real-time-ads.test.ts create mode 100644 test/integration/live/use-cases/real-time-chat.test.ts create mode 100644 test/integration/live/use-cases/score-alerts.test.ts diff --git a/.github/workflows/live-integration.yml b/.github/workflows/live-integration.yml new file mode 100644 index 000000000..279bdeb56 --- /dev/null +++ b/.github/workflows/live-integration.yml @@ -0,0 +1,159 @@ +name: Live integration + +# Live tier (test/integration/live/README.md): the public documentation's code samples run against the real PubNub service. +# Nightly and on demand only. The live job never runs on a pull request, so fork PRs never see secrets. Pull requests get the +# offline job only (lint, manifest freshness, types), and it is not a required check. The old harness workflow +# (docs-snippets.yml) stays until the cutover. + +on: + # Pull requests: offline job only, and only when something the tier or its mocked samples depend on changed. + pull_request: + paths: + - 'test/integration/live/**' + - 'test/integration/endpoints/objects/channel.test.ts' + - 'test/integration/endpoints/time.test.ts' + - 'test/integration/components/subscription_manager.test.ts' + - 'package.json' + - '.github/workflows/live-integration.yml' + # 04:43 UTC, away from docs-snippets.yml (03:17 UTC), so the two never queue behind each other for the shared keysets. + schedule: + - cron: '43 4 * * *' + workflow_dispatch: + inputs: + grep: + description: 'Run only tests whose title matches this pattern (mocha --grep). Empty runs everything.' + required: false + default: '' + type: string + +defaults: + run: + shell: bash + +permissions: + contents: read + +jobs: + offline: + name: Offline checks (lint, types) + runs-on: + group: organization/Default + timeout-minutes: 15 + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + persist-credentials: false + - name: Setup NodeJS 22 + uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4 + with: + node-version: 22 + cache: 'npm' + - name: Install dependencies + run: npm ci + - name: Build the SDK for Node + run: npm run build:node + - name: Lint (markers, manifest, eslint) + run: npm run test:integration:live:lint + - name: Type check + run: npm run test:integration:live:types + + live: + name: Live tier + needs: [offline] + # Never on a pull request: no secrets for fork code. Scheduled and manual runs only. + if: ${{ github.event_name != 'pull_request' }} + runs-on: + group: organization/Default + timeout-minutes: 40 + # Same group name as the live job of docs-snippets.yml on purpose: both use the same keysets, and this tier uses fixed + # resource names, so the two must never overlap. A running job is never cancelled. + concurrency: + group: docs-snippets-live + cancel-in-progress: false + env: + # Not a secret. Set on the job (not the steps) because it is only a keyset id for the Admin API feature check. + DOCS_SNIPPETS_DS_KEYSET_ID: ${{ vars.SDK_DOCS_DS_KEYSET_ID }} + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + persist-credentials: false + - name: Setup NodeJS 22 + uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4 + with: + node-version: 22 + cache: 'npm' + # Secrets are set per step below, never on the job, so install scripts of dependencies and the build never see them. + - name: Install dependencies + run: npm ci + - name: Build the SDK for Node + id: build + run: npm run build:node + - name: Check the DataSync classes (read only) + if: ${{ steps.build.outcome == 'success' }} + env: + DS_SUBSCRIBE_KEY: ${{ secrets.SDK_DOCS_DS_SUBSCRIBE_KEY }} + DS_PUBLISH_KEY: ${{ secrets.SDK_DOCS_DS_PUBLISH_KEY }} + DS_SECRET_KEY: ${{ secrets.SDK_DOCS_DS_SECRET_KEY }} + PUBNUB_ADMIN_API_KEY: ${{ secrets.SDK_DOCS_ADMIN_API_KEY }} + run: npm run test:integration:live:provision:check + - name: Docs samples (janitor and preflight on) + id: samples + if: ${{ !cancelled() && steps.build.outcome == 'success' }} + env: + SUBSCRIBE_KEY: ${{ secrets.SDK_DOCS_OPEN_SUBSCRIBE_KEY }} + PUBLISH_KEY: ${{ secrets.SDK_DOCS_OPEN_PUBLISH_KEY }} + PAM_SUBSCRIBE_KEY: ${{ secrets.SDK_DOCS_PAM_SUBSCRIBE_KEY }} + PAM_PUBLISH_KEY: ${{ secrets.SDK_DOCS_PAM_PUBLISH_KEY }} + PAM_SECRET_KEY: ${{ secrets.SDK_DOCS_PAM_SECRET_KEY }} + DS_SUBSCRIBE_KEY: ${{ secrets.SDK_DOCS_DS_SUBSCRIBE_KEY }} + DS_PUBLISH_KEY: ${{ secrets.SDK_DOCS_DS_PUBLISH_KEY }} + DS_SECRET_KEY: ${{ secrets.SDK_DOCS_DS_SECRET_KEY }} + PUBNUB_ADMIN_API_KEY: ${{ secrets.SDK_DOCS_ADMIN_API_KEY }} + # Untrusted input: reaches the script only through the environment. + GREP: ${{ inputs.grep }} + run: | + if [ -n "$GREP" ]; then + npm run test:integration:live -- --grep "$GREP" + else + npm run test:integration:live + fi + # The SDK's own keyset on purpose (the one run-tests.yml uses), not the docs keysets: the real-network endpoint tests are not + # documentation samples. The six DataSync files need the DataSync classes that exist only on the SDK's DataSync keyset + # (SDK_DS_*), and fetch_messages, history and message_actions need the SDK's open keyset with Message Persistence and + # message actions (SDK_SUB_KEY / SDK_PUB_KEY). Runs after a failed samples step too, so both results show. If either step + # fails the job fails (a failed step marks the job failed even when later steps pass). + - name: Endpoint tests (SDK keysets) + id: endpoints + if: ${{ !cancelled() && steps.build.outcome == 'success' }} + env: + SUBSCRIBE_KEY: ${{ secrets.SDK_SUB_KEY }} + PUBLISH_KEY: ${{ secrets.SDK_PUB_KEY }} + DS_SUBSCRIBE_KEY: ${{ secrets.SDK_DS_SUB_KEY }} + DS_PUBLISH_KEY: ${{ secrets.SDK_DS_PUB_KEY }} + DS_SECRET_KEY: ${{ secrets.SDK_DS_SEC_KEY }} + GREP: ${{ inputs.grep }} + run: | + if [ -n "$GREP" ]; then + npm run test:integration:live:endpoints -- --grep "$GREP" + else + npm run test:integration:live:endpoints + fi + # After a failure or a cancel the tier's own after-hook may not have run (killed process): revoke application-level + # Access Manager grants and remove leftovers, so a keyset is never left open until the next run. + - name: Revoke grants and clean up + if: ${{ (cancelled() || failure()) && steps.build.outcome == 'success' }} + env: + SUBSCRIBE_KEY: ${{ secrets.SDK_DOCS_OPEN_SUBSCRIBE_KEY }} + PUBLISH_KEY: ${{ secrets.SDK_DOCS_OPEN_PUBLISH_KEY }} + PAM_SUBSCRIBE_KEY: ${{ secrets.SDK_DOCS_PAM_SUBSCRIBE_KEY }} + PAM_PUBLISH_KEY: ${{ secrets.SDK_DOCS_PAM_PUBLISH_KEY }} + PAM_SECRET_KEY: ${{ secrets.SDK_DOCS_PAM_SECRET_KEY }} + DS_SUBSCRIBE_KEY: ${{ secrets.SDK_DOCS_DS_SUBSCRIBE_KEY }} + DS_PUBLISH_KEY: ${{ secrets.SDK_DOCS_DS_PUBLISH_KEY }} + DS_SECRET_KEY: ${{ secrets.SDK_DOCS_DS_SECRET_KEY }} + run: npm run test:integration:live:janitor + # No artifact upload: the tier prints to stdout only (mocha spec reporter) and writes no report file. + + # Failure alerts (Slack or similar) are not wired yet: the run logs in GitHub are the only signal for now. diff --git a/.gitignore b/.gitignore index 5d928bc9c..ec59be7a2 100644 --- a/.gitignore +++ b/.gitignore @@ -32,3 +32,8 @@ docs-snippets/report/ .env .env.* !docs-snippets/.env.example + +# Live integration tier (test/integration/live/README.md) # +########################################################### +test/integration/live/.env.local +!test/integration/live/.env.example diff --git a/.mocharc.json b/.mocharc.json index 7a5a5a1de..2963512db 100644 --- a/.mocharc.json +++ b/.mocharc.json @@ -5,7 +5,8 @@ "exclude": [ "test/dist/*.{js,ts}", "test/feature/*.{js,ts}", - "test/integration/shared-worker/*.{js,ts}" + "test/integration/shared-worker/*.{js,ts}", + "test/integration/live/**" ], "timeout": 5000, "reporter": "spec" diff --git a/package.json b/package.json index ac462d04d..dbf1fb95d 100644 --- a/package.json +++ b/package.json @@ -14,6 +14,15 @@ "test:node": "TS_NODE_PROJECT='./tsconfig.json' mocha --project tsconfig.mocha.json", "clean": "rimraf lib dist upload", "lint": "eslint \"src/**/*\" --config .eslintrc.cjs", + "test:integration:live": "TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json mocha --config test/integration/live/.mocharc.cjs", + "test:integration:live:endpoints": "TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json mocha --config test/integration/live/.mocharc.endpoints.cjs", + "test:integration:live:types": "tsc -p test/integration/live/tsconfig.json", + "test:integration:live:janitor": "TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json tsx test/integration/live/support/janitor-cli.ts", + "test:integration:live:provision": "TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json tsx test/integration/live/provision/provision-datasync.ts", + "test:integration:live:provision:check": "TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json tsx test/integration/live/provision/provision-datasync.ts --check", + "test:integration:live:lint": "TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json tsx test/integration/live/support/lint-markers.ts && TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json tsx test/integration/live/support/manifest.ts --check && eslint \"test/integration/live/**/*.ts\"", + "test:integration:live:manifest": "TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json tsx test/integration/live/support/manifest.ts", + "test:integration:live:docs-check": "TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json tsx test/integration/live/support/docs-check.ts", "test:snippets": "tsc --project docs-snippets/tsconfig.json --noEmit", "test:docs-snippets": "TSX_TSCONFIG_PATH=docs-snippets/tsconfig.run.json tsx docs-snippets/harness/cli.ts live", "test:docs-snippets:dry": "TSX_TSCONFIG_PATH=docs-snippets/tsconfig.run.json tsx docs-snippets/harness/cli.ts dry", diff --git a/test/integration/components/subscription_manager.test.ts b/test/integration/components/subscription_manager.test.ts index 895436b51..04f4c7348 100644 --- a/test/integration/components/subscription_manager.test.ts +++ b/test/integration/components/subscription_manager.test.ts @@ -456,6 +456,67 @@ describe('#components/subscription_manager', () => { }); }); + it('reports a failed heartbeat to the status listener with category, operation and status code', async () => { + let calls = 0; + const scope = utils + .createNock() + .get('/v2/presence/sub-key/mySubKey/channel/ch1,ch2/heartbeat') + .query({ + pnsdk: `PubNub-JS-Nodejs/${pubnub.getVersion()}`, + uuid: 'myUUID', + heartbeat: 300, + state: '{}', + }) + .reply(() => { + calls++; + return [400, '{"status": 400, "message": "OK", "service": "Presence"}', { 'content-type': 'text/javascript' }]; + }); + + type FailedHeartbeat = { category: string; operation?: string; statusCode: number; error?: unknown }; + let reportFailure: (status: FailedHeartbeat) => void = () => {}; + const heartbeatFailure = new Promise((resolve) => { + reportFailure = resolve; + }); + const logged: unknown[][] = []; + const originalLog = console.log; + console.log = (...args: unknown[]) => { + logged.push(args); + }; + + try { + // snippet.heartbeatFailureStatusListener + pubnub.addListener({ + status: (statusEvent) => { + if (statusEvent.operation === PubNub.OPERATIONS.PNHeartbeatOperation && statusEvent.error) { + console.log('Heartbeat failed:', statusEvent.category, statusEvent.statusCode); + // snippet.hide + reportFailure(statusEvent as unknown as FailedHeartbeat); + // snippet.show + } + }, + }); + + pubnub.subscribe({ + channels: ['ch1', 'ch2'], + withPresence: true, + withHeartbeats: true, + }); + // snippet.end + + const failure = await heartbeatFailure; + assert.equal(failure.category, PubNub.CATEGORIES.PNBadRequestCategory); + assert.equal(failure.operation, PubNub.OPERATIONS.PNHeartbeatOperation); + assert.equal(failure.statusCode, 400); + assert.equal(failure.error, true); + } finally { + console.log = originalLog; + } + + assert.equal(scope.isDone(), true); + assert.equal(calls, 1); + assert.deepEqual(logged, [['Heartbeat failed:', PubNub.CATEGORIES.PNBadRequestCategory, 400]]); + }); + it('reports when heartbeats pass', (done) => { const scope = utils .createNock() diff --git a/test/integration/endpoints/data_sync/fixtures.ts b/test/integration/endpoints/data_sync/fixtures.ts index f318eb0c7..9545aba77 100644 --- a/test/integration/endpoints/data_sync/fixtures.ts +++ b/test/integration/endpoints/data_sync/fixtures.ts @@ -86,7 +86,7 @@ export type EntityResponseObject = DataSync.EntityObject & { entityClassLevel: 'Global' | 'SubKey'; }; -/** The payload template `helpers.ts::customerPayload` seeds, as the service echoes it back. */ +/** The payload template `customerPayload` of the live DataSync helpers seeds, as the service echoes it back. */ export const customerPayload = (id: string, overrides: Record = {}) => ({ customerId: id, firstName: 'Alice', @@ -173,7 +173,7 @@ export type UserResponseObject = DataSync.UserObject & { entityClassLevel: 'Global' | 'SubKey'; }; -/** The payload template `helpers.ts::userPayload` seeds, as the service echoes it back. */ +/** The payload template `userPayload` of the live DataSync helpers seeds, as the service echoes it back. */ export const userPayload = (overrides: Record = {}) => ({ firstName: 'Alice', lastName: 'Verma', diff --git a/test/integration/endpoints/fetch_messages.test.ts b/test/integration/endpoints/fetch_messages.test.ts index 9ede06b8b..db1f28fb0 100644 --- a/test/integration/endpoints/fetch_messages.test.ts +++ b/test/integration/endpoints/fetch_messages.test.ts @@ -5,151 +5,17 @@ import assert from 'assert'; import nock from 'nock'; -import * as MessageActions from '../../../src/core/types/api/message-action'; -import * as Publish from '../../../src/core/endpoints/publish'; -import { Payload } from '../../../src/core/types/api'; import PubNub from '../../../src/node/index'; import utils from '../../utils'; import { PubNubError } from '../../../src/errors/pubnub-error'; -/** - * Published test message shape. - */ -type TestMessage = { messageIdx: string; time: number }; - -/** - * Prepare messages history. - * - * @param client - PubNub client instance which will be used to publish messages. - * @param count - How many messages should be published. - * @param channel - Name of the channel into which messages should be published. - * @param customMessageType - User-provided message type (ignored if empty). - * @param completion - Messages set publish completion function. - */ -function publishMessagesToChannel( - client: PubNub, - count: number, - channel: string, - customMessageType: string, - completion: (published: { message: TestMessage; timetoken: string }[]) => void, -) { - let messages: { message: TestMessage; timetoken: string }[] = []; - let publishCompleted = 0; - - const publish = (messageIdx: number) => { - const payload: Publish.PublishParameters = { - message: { messageIdx: [channel, messageIdx].join(': '), time: Date.now() }, - channel, - }; - - if (customMessageType.length) payload.customMessageType = customMessageType; - if (messageIdx % 2 === 0) payload.meta = { time: (payload.message as TestMessage).time }; - - client.publish(payload, (status, response) => { - publishCompleted += 1; - - if (!status.error && response) { - messages.push({ message: payload.message as TestMessage, timetoken: response.timetoken }); - messages = messages.sort((left, right) => parseInt(left.timetoken, 10) - parseInt(right.timetoken, 10)); - } else { - console.error('Publish did fail:', status); - } - - if (publishCompleted < count) { - publish(publishCompleted); - } else if (publishCompleted === count) { - completion(messages); - } - }); - }; - - publish(publishCompleted); -} - -/** - * Attach message actions to the previously published messages. - * - * @param client - PubNub client instance which should be used to add message action to the message. - * @param count - How many message actions should be added to each referenced message. - * @param messageTimetokens - List of referenced messages' timetokens. - * @param channel - Name of the channel where referenced messages has been published. - * @param completion - Message actions addition completion function. - */ -function addActionsInChannel( - client: PubNub, - count: number, - messageTimetokens: string[], - channel: string, - completion: (added: MessageActions.MessageAction[]) => void, -) { - const types = ['reaction', 'receipt', 'custom']; - const values = [ - PubNub.generateUUID(), - PubNub.generateUUID(), - PubNub.generateUUID(), - PubNub.generateUUID(), - PubNub.generateUUID(), - PubNub.generateUUID(), - PubNub.generateUUID(), - PubNub.generateUUID(), - PubNub.generateUUID(), - PubNub.generateUUID(), - ]; - let actions: MessageActions.MessageAction[] = []; - const actionsToAdd: { - messageTimetoken: string; - action: Pick; - }[] = []; - let actionsAdded = 0; - - for (let messageIdx = 0; messageIdx < messageTimetokens.length; messageIdx += 1) { - const messageTimetoken = messageTimetokens[messageIdx]; - - for (let messageActionIdx = 0; messageActionIdx < count; messageActionIdx += 1) { - const action = { type: types[(messageActionIdx + 1) % 3], value: values[(messageActionIdx + 1) % 10] }; - - actionsToAdd.push({ messageTimetoken, action }); - } - } - - /** - * Attach set of message actions. - * - * @param actionIdx - Index of currently adding message action. - */ - const addAction = (actionIdx: number) => { - const { messageTimetoken, action } = actionsToAdd[actionIdx]; - - client.addMessageAction({ channel, messageTimetoken, action }, (status, response) => { - actionsAdded += 1; - - if (!status.error && response) { - actions.push(response.data); - actions = actions.sort( - (left, right) => parseInt(left.actionTimetoken, 10) - parseInt(right.actionTimetoken, 10), - ); - } else { - console.error('Action add did fail:', status); - } - - if (actionsAdded < actionsToAdd.length) { - addAction(actionsAdded); - } else if (actionsAdded === actionsToAdd.length) { - completion(actions); - } - }); - }; - - addAction(actionsAdded); -} - describe('fetch messages endpoints', () => { - const subscribeKey = process.env.SUBSCRIBE_KEY || 'demo'; - const publishKey = process.env.PUBLISH_KEY || 'demo'; + // Fake keys: every test of this file is nock-mocked. Real-network tests: test/integration/live/endpoints/. + const subscribeKey = 'demo'; + const publishKey = 'demo'; let pubnub: PubNub; afterEach(() => { - nock.enableNetConnect(); pubnub.removeAllListeners(); pubnub.unsubscribeAll(); pubnub.destroy(true); @@ -314,26 +180,6 @@ describe('fetch messages endpoints', () => { }); }); - it('supports metadata', (done) => { - const channel = PubNub.generateUUID(); - const expectedMessagesCount = 10; - - publishMessagesToChannel(pubnub, expectedMessagesCount, channel, '', (messages) => { - pubnub.fetchMessages({ channels: [channel], count: 25, includeMeta: true }, (_, response) => { - try { - assert(response !== null); - const channelMessages = response.channels[channel]; - - assert.deepEqual(channelMessages[0].meta, { time: messages[0].message.time }); - assert(!channelMessages[1].meta); - done(); - } catch (error) { - done(error); - } - }); - }); - }).timeout(60000); - it('throws when requested actions for multiple channels', async () => { let errorCatched = false; @@ -351,134 +197,6 @@ describe('fetch messages endpoints', () => { assert(errorCatched); }); - it('supports custom message type', (done) => { - const channel = PubNub.generateUUID(); - const expectedMessagesCount = 2; - - publishMessagesToChannel(pubnub, expectedMessagesCount, channel, 'test-message-type', (messages) => { - const messageTimetokens = messages.map((message) => message.timetoken); - - pubnub.fetchMessages({ channels: [channel], includeCustomMessageType: true }, (status, response) => { - assert.equal(status.error, false, `Fetch messages error: ${JSON.stringify(status.errorData)}`); - - try { - assert.equal(status.error, false); - assert(response !== null); - const fetchedMessages = response.channels[channel]; - fetchedMessages.forEach((message) => { - assert.equal(message.customMessageType, 'test-message-type'); - }); - - done(); - } catch (error) { - done(error); - } - }); - }); - }).timeout(60000); - - it("supports actions (stored as 'data' field)", (done) => { - const channel = PubNub.generateUUID(); - const expectedMessagesCount = 2; - const expectedActionsCount = 4; - - publishMessagesToChannel(pubnub, expectedMessagesCount, channel, '', (messages) => { - const messageTimetokens = messages.map((message) => message.timetoken); - - addActionsInChannel(pubnub, expectedActionsCount, messageTimetokens, channel, (actions) => { - setTimeout(() => { - pubnub.fetchMessages({ channels: [channel], includeMessageActions: true }, (status, response) => { - try { - assert.equal(status.error, false); - assert(response !== null); - const fetchedMessages = response.channels[channel]; - // TypeScript types system now requires to figure out type of object before using it. - assert('actions' in fetchedMessages[0]); - const actionsByType = fetchedMessages[0].data ?? {}; - let historyActionsCount = 0; - - Object.keys(actionsByType).forEach((actionType) => { - Object.keys(actionsByType[actionType]).forEach((actionValue) => { - let actionFound = false; - historyActionsCount += 1; - - actions.forEach((action) => { - if (action.value === actionValue) actionFound = true; - }); - - assert.equal(actionFound, true); - }); - }); - - assert.equal(historyActionsCount, expectedActionsCount); - assert.equal(fetchedMessages[0].timetoken, messageTimetokens[0]); - assert.equal( - fetchedMessages[fetchedMessages.length - 1].timetoken, - messageTimetokens[messageTimetokens.length - 1], - ); - - done(); - } catch (error) { - done(error); - } - }); - }, 2000); - }); - }); - }).timeout(60000); - - it("supports actions (stored as 'actions' field)", (done) => { - const channel = PubNub.generateUUID(); - const expectedMessagesCount = 2; - const expectedActionsCount = 4; - - publishMessagesToChannel(pubnub, expectedMessagesCount, channel, '', (messages) => { - const messageTimetokens = messages.map((message) => message.timetoken); - - addActionsInChannel(pubnub, expectedActionsCount, messageTimetokens, channel, (actions) => { - setTimeout(() => { - pubnub.fetchMessages({ channels: [channel], includeMessageActions: true }, (status, response) => { - try { - assert.equal(status.error, false); - assert(response !== null); - const fetchedMessages = response.channels[channel]; - // TypeScript types system now requires to figure out type of object before using it. - assert('actions' in fetchedMessages[0]); - const actionsByType = fetchedMessages[0].actions ?? {}; - let historyActionsCount = 0; - - Object.keys(actionsByType).forEach((actionType) => { - Object.keys(actionsByType[actionType]).forEach((actionValue) => { - let actionFound = false; - historyActionsCount += 1; - - actions.forEach((action) => { - if (action.value === actionValue) { - actionFound = true; - } - }); - - assert.equal(actionFound, true); - }); - }); - - assert.equal(historyActionsCount, expectedActionsCount); - assert.equal(fetchedMessages[0].timetoken, messageTimetokens[0]); - assert.equal( - fetchedMessages[fetchedMessages.length - 1].timetoken, - messageTimetokens[messageTimetokens.length - 1], - ); - - done(); - } catch (error) { - done(error); - } - }); - }, 2000); - }); - }); - }).timeout(60000); - it('should return "more" field when server sends it', (done) => { nock.disableNetConnect(); const scope = utils diff --git a/test/integration/endpoints/history.test.ts b/test/integration/endpoints/history.test.ts index 3537cbf15..ee834c590 100644 --- a/test/integration/endpoints/history.test.ts +++ b/test/integration/endpoints/history.test.ts @@ -4,68 +4,16 @@ import assert from 'assert'; import nock from 'nock'; -import { Payload } from '../../../src/core/types/api'; import PubNub from '../../../src/node/index'; import utils from '../../utils'; -/** - * Published test message shape. - */ -type TestMessage = { messageIdx: string; time: number }; - -/** - * Prepare messages history. - * - * @param client - PubNub client instance which will be used to publish messages. - * @param count - How many messages should be published. - * @param channel - Name of the channel into which messages should be published. - * @param completion - Messages set publish completion function. - */ -function publishMessagesToChannel( - client: PubNub, - count: number, - channel: string, - completion: (published: { message: TestMessage; timetoken: string }[]) => void, -) { - let messages: { message: TestMessage; timetoken: string }[] = []; - let publishCompleted = 0; - - const publish = (messageIdx: number) => { - const payload: { channel: string; message: TestMessage; meta?: Payload } = { - message: { messageIdx: [channel, messageIdx].join(': '), time: Date.now() }, - channel, - }; - - if (messageIdx % 2 === 0) payload.meta = { time: payload.message.time }; - - client.publish(payload, (status, response) => { - publishCompleted += 1; - - if (!status.error && response) { - messages.push({ message: payload.message, timetoken: response.timetoken }); - messages = messages.sort((left, right) => parseInt(left.timetoken, 10) - parseInt(right.timetoken, 10)); - } else { - console.error('Publish did fail:', status); - } - - if (publishCompleted < count) { - publish(publishCompleted); - } else if (publishCompleted === count) { - completion(messages); - } - }); - }; - - publish(publishCompleted); -} - describe('history endpoints', () => { - const subscribeKey = process.env.SUBSCRIBE_KEY || 'demo'; - const publishKey = process.env.PUBLISH_KEY || 'demo'; + // Fake keys: every test of this file is nock-mocked. Real-network tests: test/integration/live/endpoints/. + const subscribeKey = 'demo'; + const publishKey = 'demo'; let pubnub: PubNub; afterEach(() => { - nock.enableNetConnect(); pubnub.removeAllListeners(); pubnub.unsubscribeAll(); pubnub.destroy(true); @@ -156,24 +104,6 @@ describe('history endpoints', () => { }); }); - it('supports metadata', (done) => { - const channel = PubNub.generateUUID(); - const expectedMessagesCount = 10; - - publishMessagesToChannel(pubnub, expectedMessagesCount, channel, (messages) => { - pubnub.history({ channel, includeMeta: true }, (_, response) => { - try { - assert(response !== null); - assert.deepEqual(response.messages[0].meta, { time: messages[0].message.time }); - assert(!response.messages[1].meta); - done(); - } catch (error) { - done(error); - } - }); - }); - }).timeout(60000); - it('handles unencrypted payload with cryptoModule', (done) => { nock.disableNetConnect(); const scope = utils diff --git a/test/integration/endpoints/message_actions.test.ts b/test/integration/endpoints/message_actions.test.ts index af649a47c..5be911814 100644 --- a/test/integration/endpoints/message_actions.test.ts +++ b/test/integration/endpoints/message_actions.test.ts @@ -13,124 +13,13 @@ import utils from '../../utils'; */ type MessageAction = MessageActions.AddMessageActionParameters['action']; -/** - * Prepare messages history. - * - * @param client - PubNub client instance which will be used to publish messages. - * @param count - How many messages should be published. - * @param channel - Name of the channel into which messages should be published. - * @param completion - Messages set publish completion function. - */ -function publishMessages(client: PubNub, count: number, channel: string, completion: (timetokens: string[]) => void) { - let publishCompleted = 0; - const timetokens: string[] = []; - - const publish = (messageIdx: number) => { - const message = { messageIdx, time: Date.now() }; - - client.publish({ message, channel }, (status, response) => { - publishCompleted += 1; - - if (!status.error && response) { - timetokens.push(response.timetoken); - } else { - console.error('Publish did fail:', status); - } - - if (publishCompleted < count) { - publish(publishCompleted); - } else if (publishCompleted === count) { - completion(timetokens.sort((left, right) => parseInt(left, 10) - parseInt(right, 10))); - } - }); - }; - - publish(publishCompleted); -} - -/** - * Attach message actions to the previously published messages. - * - * @param client - PubNub client instance which should be used to add message action to the message. - * @param count - How many message actions should be added to each referenced message. - * @param messageTimetokens - List of referenced messages' timetokens. - * @param channel - Name of the channel where referenced messages has been published. - * @param completion - Message actions addition completion function. - */ -function addActions( - client: PubNub, - count: number, - messageTimetokens: string[], - channel: string, - completion: (timetokens: string[]) => void, -) { - const types = ['reaction', 'receipt', 'custom']; - const values = [ - PubNub.generateUUID(), - PubNub.generateUUID(), - PubNub.generateUUID(), - PubNub.generateUUID(), - PubNub.generateUUID(), - PubNub.generateUUID(), - PubNub.generateUUID(), - PubNub.generateUUID(), - PubNub.generateUUID(), - PubNub.generateUUID(), - ]; - const actionsToAdd: { messageTimetoken: string; action: MessageAction }[] = []; - const timetokens: string[] = []; - let actionsAdded = 0; - - for (let messageIdx = 0; messageIdx < messageTimetokens.length; messageIdx += 1) { - const messageTimetoken = messageTimetokens[messageIdx]; - - for (let messageActionIdx = 0; messageActionIdx < count; messageActionIdx += 1) { - const action: MessageAction = { - type: types[(messageActionIdx + 1) % 3], - value: values[(messageActionIdx + 1) % 10], - }; - - actionsToAdd.push({ messageTimetoken, action }); - } - } - - /** - * Attach set of message actions. - * - * @param actionIdx - Index of currently adding message action. - */ - const addAction = (actionIdx: number) => { - const { messageTimetoken, action } = actionsToAdd[actionIdx]; - - client.addMessageAction({ channel, messageTimetoken, action }, (status, response) => { - actionsAdded += 1; - - if (!status.error && response) { - timetokens.push(response.data.actionTimetoken); - } else { - console.error('Action add did fail:', action, '\n', status); - } - - if (actionsAdded < actionsToAdd.length) { - addAction(actionsAdded); - } else if (actionsAdded === actionsToAdd.length) { - setTimeout(() => { - completion(timetokens.sort((left, right) => parseInt(left, 10) - parseInt(right, 10))); - }, 500); - } - }); - }; - - addAction(actionsAdded); -} - describe('message actions endpoints', () => { - const subscribeKey = process.env.SUBSCRIBE_KEY || 'demo'; - const publishKey = process.env.PUBLISH_KEY || 'demo'; + // Fake keys: every test of this file is nock-mocked. Real-network tests: test/integration/live/endpoints/. + const subscribeKey = 'demo'; + const publishKey = 'demo'; let pubnub: PubNub; afterEach(() => { - nock.enableNetConnect(); pubnub.removeAllListeners(); pubnub.unsubscribeAll(); pubnub.destroy(true); @@ -302,62 +191,6 @@ describe('message actions endpoints', () => { }); }); - it('add message action', (done) => { - const messageAction: MessageAction = { type: 'custom', value: PubNub.generateUUID() }; - const channel = PubNub.generateUUID(); - - publishMessages(pubnub, 1, channel, (timetokens) => { - pubnub.addMessageAction( - { channel, messageTimetoken: timetokens[0], action: messageAction }, - (status, response) => { - try { - assert.equal(status.error, false); - assert(response !== null); - assert.equal(response.data.type, messageAction.type); - assert.equal(response.data.value, messageAction.value); - assert.equal(response.data.uuid, pubnub.getUUID()); - assert.equal(response.data.messageTimetoken, timetokens[0]); - assert(response.data.actionTimetoken); - - done(); - } catch (error) { - done(error); - } - }, - ); - }); - }).timeout(60000); - - it('add message action with encoded channel', (done) => { - const messageAction: MessageAction = { type: 'custom', value: PubNub.generateUUID() }; - const channel = `${PubNub.generateUUID()}#1`; - - publishMessages(pubnub, 1, channel, (timetokens) => { - pubnub.addMessageAction( - { channel, messageTimetoken: timetokens[0], action: messageAction }, - (status, response) => { - try { - assert.equal(status.error, false); - assert(response !== null); - if (undefined === response.data.uuid) { - console.log(`Received unexpected response:`); - console.dir(response, { depth: 20 }); - } - assert.equal(response.data.type, messageAction.type); - assert.equal(response.data.value, messageAction.value); - assert.equal(response.data.uuid, pubnub.getUUID()); - assert.equal(response.data.messageTimetoken, timetokens[0]); - assert(response.data.actionTimetoken); - - done(); - } catch (error) { - done(error); - } - }, - ); - }); - }).timeout(60000); - it('add message action and 207 status code', (done) => { nock.disableNetConnect(); const scope = utils @@ -400,43 +233,6 @@ describe('message actions endpoints', () => { }, ); }); - - it('add message action should trigger event', (done) => { - const messageAction: MessageAction = { type: 'custom', value: PubNub.generateUUID() }; - const channel = PubNub.generateUUID(); - let messageTimetoken: string | null = null; - - pubnub.addListener({ - status: (status) => { - if (status.category === 'PNConnectedCategory') { - pubnub.publish({ channel, message: { hello: 'test' }, sendByPost: true }, (publishStatus, response) => { - assert(response !== null); - messageTimetoken = response.timetoken; - - pubnub.addMessageAction({ channel, messageTimetoken, action: messageAction }); - }); - } - }, - messageAction: (messageActionEvent) => { - try { - assert(messageActionEvent.data); - assert.equal(messageActionEvent.data.type, messageAction.type); - assert.equal(messageActionEvent.data.value, messageAction.value); - assert.equal(messageActionEvent.data.uuid, pubnub.getUUID()); - assert.equal(messageActionEvent.data.messageTimetoken, messageTimetoken); - assert(messageActionEvent.data.actionTimetoken); - assert.equal(messageActionEvent.event, 'added'); - pubnub.unsubscribeAll(); - - done(); - } catch (error) { - done(error); - } - }, - }); - - pubnub.subscribe({ channels: [channel] }); - }).timeout(60000); }); describe('removeMessageAction', () => { @@ -516,113 +312,6 @@ describe('message actions endpoints', () => { }); }); }); - - it('remove message action', (done) => { - const channel = PubNub.generateUUID(); - - publishMessages(pubnub, 1, channel, (messageTimetokens) => { - addActions(pubnub, 1, messageTimetokens, channel, (actionTimetokens) => { - pubnub.getMessageActions({ channel }, (status, response) => { - assert.equal(status.error, false); - assert(response !== null); - assert.equal(response.data.length, actionTimetokens.length); - - pubnub.removeMessageAction( - { channel, actionTimetoken: actionTimetokens[0], messageTimetoken: messageTimetokens[0] }, - (removeMessageStatus) => { - assert.equal(removeMessageStatus.error, false); - - setTimeout(() => { - pubnub.getMessageActions({ channel }, (getMessagesStatus, getMessagesResponse) => { - try { - assert.equal(getMessagesStatus.error, false); - assert(getMessagesResponse !== null); - assert.equal(getMessagesResponse.data.length, 0); - - done(); - } catch (error) { - done(error); - } - }); - }, 2000); - }, - ); - }); - }); - }); - }).timeout(60000); - - it('remove message action with encoded channel', (done) => { - const channel = `${PubNub.generateUUID()}#1`; - - publishMessages(pubnub, 1, channel, (messageTimetokens) => { - addActions(pubnub, 1, messageTimetokens, channel, (actionTimetokens) => { - pubnub.getMessageActions({ channel }, (status, response) => { - assert.equal(status.error, false); - assert(response !== null); - assert.equal(response.data.length, actionTimetokens.length); - - pubnub.removeMessageAction( - { channel, actionTimetoken: actionTimetokens[0], messageTimetoken: messageTimetokens[0] }, - (removeMessageStatus) => { - assert.equal(removeMessageStatus.error, false); - - setTimeout(() => { - pubnub.getMessageActions({ channel }, (getMessagesStatus, getMessagesResponse) => { - try { - assert.equal(getMessagesStatus.error, false); - assert(getMessagesResponse !== null); - assert.equal(getMessagesResponse.data.length, 0); - - done(); - } catch (error) { - done(error); - } - }); - }, 2000); - }, - ); - }); - }); - }); - }).timeout(60000); - - it('remove message action should trigger event', (done) => { - const channel = PubNub.generateUUID(); - - publishMessages(pubnub, 1, channel, (messageTimetokens) => { - addActions(pubnub, 1, messageTimetokens, channel, (actionTimetokens) => { - pubnub.addListener({ - status: (status) => { - if (status.category === 'PNConnectedCategory') { - pubnub.removeMessageAction( - { channel, actionTimetoken: actionTimetokens[0], messageTimetoken: messageTimetokens[0] }, - (removeMessagesStatus) => { - assert.equal(removeMessagesStatus.error, false); - }, - ); - } - }, - messageAction: (messageActionEvent) => { - try { - assert(messageActionEvent.data); - assert.equal(messageActionEvent.data.uuid, pubnub.getUUID()); - assert.equal(messageActionEvent.data.messageTimetoken, messageTimetokens[0]); - assert.equal(messageActionEvent.data.actionTimetoken, actionTimetokens[0]); - assert.equal(messageActionEvent.event, 'removed'); - pubnub.unsubscribeAll(); - - done(); - } catch (error) { - done(error); - } - }, - }); - - pubnub.subscribe({ channels: [channel] }); - }); - }); - }).timeout(60000); }); describe('getMessageAction', () => { @@ -644,112 +333,6 @@ describe('message actions endpoints', () => { }); }); }); - - it('fetch message actions', (done) => { - const channel = PubNub.generateUUID(); - - publishMessages(pubnub, 2, channel, (messageTimetokens) => { - addActions(pubnub, 3, messageTimetokens, channel, (actionTimetokens) => { - const lastPublishedActionTimetoken = actionTimetokens[actionTimetokens.length - 1]; - const firstPublishedActionTimetoken = actionTimetokens[0]; - - pubnub.getMessageActions({ channel }, (status, response) => { - try { - assert.equal(status.error, false); - assert(response !== null); - const firstFetchedActionTimetoken = response.data[0].actionTimetoken; - const lastFetchedActionTimetoken = response.data[response.data.length - 1].actionTimetoken; - assert.equal(firstFetchedActionTimetoken, firstPublishedActionTimetoken); - assert.equal(lastFetchedActionTimetoken, lastPublishedActionTimetoken); - assert.equal(response.data.length, actionTimetokens.length); - assert.equal(response.start, firstPublishedActionTimetoken); - assert.equal(response.end, lastPublishedActionTimetoken); - - done(); - } catch (error) { - done(error); - } - }); - }); - }); - }).timeout(60000); - - it('fetch message actions with encoded channel', (done) => { - const channel = `${PubNub.generateUUID()}#1`; - - publishMessages(pubnub, 2, channel, (messageTimetokens) => { - addActions(pubnub, 3, messageTimetokens, channel, (actionTimetokens) => { - const lastPublishedActionTimetoken = actionTimetokens[actionTimetokens.length - 1]; - const firstPublishedActionTimetoken = actionTimetokens[0]; - - pubnub.getMessageActions({ channel }, (status, response) => { - try { - assert.equal(status.error, false); - assert(response !== null); - const firstFetchedActionTimetoken = response.data[0].actionTimetoken; - const lastFetchedActionTimetoken = response.data[response.data.length - 1].actionTimetoken; - assert.equal(firstFetchedActionTimetoken, firstPublishedActionTimetoken); - assert.equal(lastFetchedActionTimetoken, lastPublishedActionTimetoken); - assert.equal(response.data.length, actionTimetokens.length); - assert.equal(response.start, firstPublishedActionTimetoken); - assert.equal(response.end, lastPublishedActionTimetoken); - - done(); - } catch (error) { - done(error); - } - }); - }); - }); - }).timeout(60000); - - it('fetch next message actions page', (done) => { - const channel = PubNub.generateUUID(); - - publishMessages(pubnub, 2, channel, (messageTimetokens) => { - addActions(pubnub, 5, messageTimetokens, channel, (actionTimetokens) => { - const lastPublishedActionTimetoken = actionTimetokens[actionTimetokens.length - 1]; - const halfSize = Math.floor(actionTimetokens.length * 0.5); - const firstPublishedActionTimetoken = actionTimetokens[0]; - const middleMinusOnePublishedActionTimetoken = actionTimetokens[halfSize - 1]; - const middlePublishedActionTimetoken = actionTimetokens[halfSize]; - - pubnub.getMessageActions({ channel, limit: halfSize }, (status, response) => { - assert.equal(status.error, false); - assert(response !== null); - let firstFetchedActionTimetoken = response.data[0].actionTimetoken; - let lastFetchedActionTimetoken = response.data[response.data.length - 1].actionTimetoken; - assert.equal(firstFetchedActionTimetoken, middlePublishedActionTimetoken); - assert.equal(lastFetchedActionTimetoken, lastPublishedActionTimetoken); - assert.equal(response.data.length, halfSize); - assert.equal(response.start, middlePublishedActionTimetoken); - assert.equal(response.end, lastPublishedActionTimetoken); - - pubnub.getMessageActions( - { channel, start: middlePublishedActionTimetoken, limit: halfSize }, - (getMessageActionsStatus, getMessageActionsResponse) => { - try { - assert.equal(getMessageActionsStatus.error, false); - assert(getMessageActionsResponse !== null); - firstFetchedActionTimetoken = getMessageActionsResponse.data[0].actionTimetoken; - lastFetchedActionTimetoken = - getMessageActionsResponse.data[getMessageActionsResponse.data.length - 1].actionTimetoken; - assert.equal(firstFetchedActionTimetoken, firstPublishedActionTimetoken); - assert.equal(lastFetchedActionTimetoken, middleMinusOnePublishedActionTimetoken); - assert.equal(getMessageActionsResponse.data.length, halfSize); - assert.equal(getMessageActionsResponse.start, firstPublishedActionTimetoken); - assert.equal(getMessageActionsResponse.end, middleMinusOnePublishedActionTimetoken); - - done(); - } catch (error) { - done(error); - } - }, - ); - }); - }); - }); - }).timeout(60000); }); describe('edge cases and error handling', () => { diff --git a/test/integration/endpoints/objects/channel.test.ts b/test/integration/endpoints/objects/channel.test.ts index 5774d234f..35c6ecdb3 100644 --- a/test/integration/endpoints/objects/channel.test.ts +++ b/test/integration/endpoints/objects/channel.test.ts @@ -4,7 +4,6 @@ import nock from 'nock'; import { asResponse, allChannels, channel1 } from './fixtures.js'; import PubNub from '../../../../src/node/index'; import utils from '../../../utils'; -import { RetryPolicy } from '../../../../src/core/components/retry-policy.js'; describe('objects channel', () => { const PUBLISH_KEY = 'myPublishKey'; @@ -732,37 +731,58 @@ describe('objects channel', () => { it('should not retry on 404 Not Found error with linear retry policy', async () => { const channelName = 'non-existent-channel'; + // Count the requests which reach the (mocked) service. + let calls = 0; + // Set up nock mock before creating PubNub instance const scope = utils .createNock() .get(`/v2/objects/${SUBSCRIBE_KEY}/channels/${channelName}`) .times(1) .query(true) - .reply(404, { - status: 404, - error: { - message: 'Requested object was not found.', - source: 'objects', - }, + .reply(() => { + calls++; + return [ + 404, + { + status: 404, + error: { + message: 'Requested object was not found.', + source: 'objects', + }, + }, + ]; }); - // Create a new PubNub instance with linear retry policy - const pubnubWithRetry = new PubNub({ + // Test-only client options (fake keys, no request identifiers in the query). + const testConfig: Record = { subscribeKey: SUBSCRIBE_KEY, publishKey: PUBLISH_KEY, - uuid: UUID, - // @ts-expect-error Force override default value. + userId: UUID, useRequestId: false, authKey: AUTH_KEY, - retryConfiguration: RetryPolicy.LinearRetryPolicy({ + }; + + // Create a new PubNub instance with linear retry policy + // snippet.retryPolicyLinear + const pubnub = new PubNub({ + subscribeKey: 'YOUR_SUBSCRIBE_KEY', + publishKey: 'YOUR_PUBLISH_KEY', + userId: 'YOUR_USER_ID', + // snippet.hide + ...testConfig, + // snippet.show + // Retry a failed request up to 2 times, 2 seconds apart. + retryConfiguration: PubNub.LinearRetryPolicy({ delay: 2, maximumRetry: 2, }), }); + // snippet.end let caughtError: any; try { - await pubnubWithRetry.objects.getChannelMetadata({ channel: channelName }); + await pubnub.objects.getChannelMetadata({ channel: channelName }); } catch (error) { caughtError = error; } @@ -782,8 +802,83 @@ describe('objects channel', () => { // Verify the scope was called exactly once (no retries on 404 expect(scope.isDone()).to.be.true; - - pubnubWithRetry.destroy(true); + expect(calls).to.equal(1); + + pubnub.destroy(true); + }); + + it('should retry on 503 error with exponential retry policy until maximum retry count is reached', async () => { + const channelName = 'unavailable-channel'; + let calls = 0; + + // Retry delays are at least 2 seconds (policy validation), so record them and run the retries without waiting. + const retryDelays: number[] = []; + const originalSetTimeout = global.setTimeout; + global.setTimeout = ((handler: any, timeout?: number, ...args: any[]) => { + if (timeout !== undefined && timeout >= 1000) { + retryDelays.push(timeout); + return originalSetTimeout(handler, 1, ...args); + } + return originalSetTimeout(handler, timeout, ...args); + }) as typeof global.setTimeout; + + const scope = utils + .createNock() + .get(`/v2/objects/${SUBSCRIBE_KEY}/channels/${channelName}`) + .times(3) + .query(true) + .reply(() => { + calls++; + return [503, { status: 503, error: { message: 'Service unavailable.', source: 'objects' } }]; + }); + + // Test-only client options (fake keys, no request identifiers in the query). + const testConfig: Record = { + subscribeKey: SUBSCRIBE_KEY, + publishKey: PUBLISH_KEY, + userId: UUID, + useRequestId: false, + authKey: AUTH_KEY, + }; + + // snippet.retryPolicyExponential + const pubnub = new PubNub({ + subscribeKey: 'YOUR_SUBSCRIBE_KEY', + publishKey: 'YOUR_PUBLISH_KEY', + userId: 'YOUR_USER_ID', + // snippet.hide + ...testConfig, + // snippet.show + // Retry a failed request up to 2 times, doubling the wait from 2 seconds up to 10 seconds. + retryConfiguration: PubNub.ExponentialRetryPolicy({ + minimumDelay: 2, + maximumDelay: 10, + maximumRetry: 2, + }), + }); + // snippet.end + + let caughtError: any; + try { + await pubnub.objects.getChannelMetadata({ channel: channelName }); + } catch (error) { + caughtError = error; + } finally { + global.setTimeout = originalSetTimeout; + } + + // The original request plus `maximumRetry` retries, and the last failure is reported. + expect(calls).to.equal(3); + expect(scope.isDone()).to.be.true; + expect(caughtError).to.exist; + expect(caughtError.status.statusCode).to.equal(503); + + // Waits double from `minimumDelay` (2 s, then 4 s), each plus up to 1 s of jitter. + expect(retryDelays).to.have.length(2); + expect(retryDelays[0]).to.be.within(2000, 3000); + expect(retryDelays[1]).to.be.within(4000, 5000); + + pubnub.destroy(true); }); }); }); diff --git a/test/integration/endpoints/time.test.ts b/test/integration/endpoints/time.test.ts index 22619ba10..90bc72863 100644 --- a/test/integration/endpoints/time.test.ts +++ b/test/integration/endpoints/time.test.ts @@ -89,4 +89,52 @@ describe('time endpoints', () => { } }); }); + + it('reports the HTTP status code of a failed request without retrying it when retries are disabled', async () => { + let calls = 0; + const scope = utils + .createNock() + .get('/time/0') + .times(1) + .query(true) + .reply(() => { + calls++; + return [500, undefined]; + }); + + // Test-only client options (fake keys, no request identifiers in the query). + const testConfig: Record = { + userId: 'myUUID', + useRequestId: false, + }; + + // snippet.retryPolicyNone + const pubnub = new PubNub({ + subscribeKey: 'YOUR_SUBSCRIBE_KEY', + publishKey: 'YOUR_PUBLISH_KEY', + userId: 'YOUR_USER_ID', + // snippet.hide + ...testConfig, + // snippet.show + // Never retry a failed request. + retryConfiguration: PubNub.NoneRetryPolicy(), + }); + // snippet.end + + let reportedStatusCode: number | undefined; + // snippet.handleRequestErrorStatusCode + await pubnub.time().catch((error) => { + // The HTTP status code of the failed request is in the error's status. + console.log('Request failed with HTTP status', error.status.statusCode); + // snippet.hide + reportedStatusCode = error.status.statusCode; + // snippet.show + }); + // snippet.end + + assert.equal(reportedStatusCode, 500); + assert.equal(calls, 1, 'the failed request is sent once and not retried'); + assert.equal(scope.isDone(), true); + pubnub.destroy(true); + }); }); diff --git a/test/integration/live/.env.example b/test/integration/live/.env.example new file mode 100644 index 000000000..45332e187 --- /dev/null +++ b/test/integration/live/.env.example @@ -0,0 +1,16 @@ +# Keys for the live tier (test/integration/live/README.md). Copy to test/integration/live/.env.local, which git ignores. +# Real environment variables win over the file. LIVE_ENV_FILE= points at another file. + +# open: a keyset without Access Manager +SUBSCRIBE_KEY= +PUBLISH_KEY= + +# pam: a keyset with Access Manager +PAM_SUBSCRIBE_KEY= +PAM_PUBLISH_KEY= +PAM_SECRET_KEY= + +# datasync: a keyset with Access Manager and DataSync (may be the same as pam) +DS_SUBSCRIBE_KEY= +DS_PUBLISH_KEY= +DS_SECRET_KEY= diff --git a/test/integration/live/.eslintrc.cjs b/test/integration/live/.eslintrc.cjs new file mode 100644 index 000000000..cc5208b03 --- /dev/null +++ b/test/integration/live/.eslintrc.cjs @@ -0,0 +1,10 @@ +// The root `npm run lint` covers src/ only. Run the tier with `npx eslint "test/integration/live/**/*.ts"`. This file lets +// long comments and test titles pass; the sample lines between the markers are never reformatted. +module.exports = { + rules: { + 'max-len': [ + 'error', + { code: 120, tabWidth: 2, ignoreComments: true, ignoreStrings: true, ignoreTemplateLiterals: true }, + ], + }, +}; diff --git a/test/integration/live/.mocharc.cjs b/test/integration/live/.mocharc.cjs new file mode 100644 index 000000000..5d6adcc02 --- /dev/null +++ b/test/integration/live/.mocharc.cjs @@ -0,0 +1,11 @@ +// Mocha config of the live tier's docs-sample tests (everything except endpoints/). Run it through `npm run test:integration:live`, +// which also sets TSX_TSCONFIG_PATH so that `import PubNub from 'pubnub'` resolves to the built library (lib/node/index.js). +// Paths are relative to the repository root. The tests of endpoints/ have their own config, .mocharc.endpoints.cjs. +module.exports = { + require: ['tsx', 'test/integration/live/support/hooks.ts'], + spec: ['test/integration/live/**/*.test.ts'], + exclude: ['test/integration/live/endpoints/**'], + timeout: 30000, + retries: 0, + reporter: 'spec', +}; diff --git a/test/integration/live/.mocharc.endpoints.cjs b/test/integration/live/.mocharc.endpoints.cjs new file mode 100644 index 000000000..7ac547531 --- /dev/null +++ b/test/integration/live/.mocharc.endpoints.cjs @@ -0,0 +1,17 @@ +// Mocha config of the real-network endpoint tests (test/integration/live/endpoints/**). Run it through +// `npm run test:integration:live:endpoints`. No role is required up front: each test needs its own (`datasync` for the DataSync +// files, which use classes that exist only on the separate DS_* keyset; `open` for the other files) and fails with a "missing keys" +// message naming the env variables when they are absent, so a --grep run needs only the keys of the selected tests. The janitor and the keyset preflight are off by default: their resource list and feature checks +// belong to the docs keysets, and these tests clean up their own uniquely named objects. `LIVE_JANITOR=on` or `LIVE_PREFLIGHT=on` +// in the environment turns either back on. A spec argument on the command line is added to the spec below (Mocha merges them). +process.env.LIVE_TIER = 'endpoints'; +process.env.LIVE_JANITOR ||= 'off'; +process.env.LIVE_PREFLIGHT ||= 'off'; + +module.exports = { + require: ['tsx', 'test/integration/live/support/hooks.ts'], + spec: ['test/integration/live/endpoints/**/*.test.ts'], + timeout: 30000, + retries: 0, + reporter: 'spec', +}; diff --git a/test/integration/live/README.md b/test/integration/live/README.md new file mode 100644 index 000000000..176621d2c --- /dev/null +++ b/test/integration/live/README.md @@ -0,0 +1,146 @@ +# Live integration tier + +Mocha tests that run the code samples of the public docs against the real PubNub service. The nock-mocked tests in `test/integration/**` stay as they are, and `npm run test:node` never runs this folder (`.mocharc.json` excludes it). + +## Run + +```bash +npm run build:node # samples import 'pubnub', which resolves to lib/node/index.js of this checkout +cp test/integration/live/.env.example test/integration/live/.env.local # then fill in the keys (git ignores .env.local) +npm run test:integration:live # the docs-sample tier: everything except endpoints/ (needs the docs keysets) +npm run test:integration:live:endpoints # only endpoints/**: real-network endpoint tests that are not doc samples (each test needs its own role: DS_* for DataSync, SUBSCRIBE_KEY and PUBLISH_KEY for the rest) +npm run test:integration:live -- --grep publishMinimal +npm run test:integration:live -- --dry-run # lists the tests, no network, no keys needed +npm run test:integration:live:types # type check (needs the build for lib/types) +npx eslint "test/integration/live/**/*.ts" # lint (npm run lint covers src/ only). Do not add --config: it drops the max-len override of live/.eslintrc.cjs +``` + +Env names: `SUBSCRIBE_KEY`, `PUBLISH_KEY` (open keyset), `PAM_SUBSCRIBE_KEY`, `PAM_PUBLISH_KEY`, `PAM_SECRET_KEY` (Access Manager), `DS_*` (DataSync). Real environment variables win over the file. `LIVE_ENV_FILE=` points at another file. The run fails at once when the open keyset is missing. A docs-tier run without `--grep` (and without spec file arguments) needs every role: it fails at once, with the missing names, when `pam` or `datasync` is not set. A run with `--grep` or a file argument checks a role only when a test builds its client, so a page that needs only the open keyset runs with only that keyset. Run one live run per keyset at a time. + +**Two runs, two keysets.** The tier has two Mocha configs. + +| Script | Config | Runs | Keys | Janitor / preflight | +| :--- | :--- | :--- | :--- | :--- | +| `test:integration:live` | `.mocharc.cjs` | every test except `endpoints/` (`api-reference/`, `getting-started/`, `migration-guides/`, `shared/`, `use-cases/`) | open, pam and datasync keysets of the docs | on (`LIVE_JANITOR=off`, `LIVE_PREFLIGHT=off` switch them off) | +| `test:integration:live:endpoints` | `.mocharc.endpoints.cjs` | only `endpoints/**`: 6 DataSync event and projection files (`endpoints/data_sync/`) and 3 files with the real-network tests of the SDK endpoint suite (`endpoints/fetch_messages.test.ts`, `history.test.ts`, `message_actions.test.ts`; 14 tests) | no role is checked up front, each test checks its own: `datasync` (`DS_SUBSCRIBE_KEY`, `DS_PUBLISH_KEY`, `DS_SECRET_KEY`) for the DataSync files, `open` (`SUBSCRIBE_KEY`, `PUBLISH_KEY`, with Message Persistence and message actions) for the other three | off by default (`LIVE_JANITOR=on` / `LIVE_PREFLIGHT=on` turn them on) | + +The DataSync endpoint tests need the DataSync classes `JSCustomer`, `JSLoanQuote` and `JSREQUESTED_BY`, which exist on a different keyset from the one the docs samples use. The other three files (`fetch_messages`, `history`, `message_actions`) are the tests that publish to the real service and read the result back; they were moved out of `test/integration/endpoints/` of the same names, whose remaining tests are nock-mocked, so that `npm run test:node` runs fully offline (a root hook in `test/setup-why.ts` refuses every request that no nock scope matches). They are not documentation samples and carry no `// snippet.` markers. CI runs the whole folder as one separate step with the SDK's own keysets (the ones `run-tests.yml` uses: `SDK_SUB_KEY`/`SDK_PUB_KEY` as `SUBSCRIBE_KEY`/`PUBLISH_KEY`, and `SDK_DS_*` as `DS_*`), not the docs keysets. Every test titles itself `endpoints/: ...`, so `--grep '^endpoints/(fetch_messages|history|message_actions)'` runs only the open-keyset files and `--grep '^endpoints/data_sync'` only the DataSync files; a test whose role is not set fails with `missing keys: the keyset is not configured, set `. These tests use channels named after a fresh UUID per test, so they are safe on a shared, persistent keyset, and they poll eventually consistent reads with `eventually` instead of a fixed wait. The janitor's resource list and the preflight's feature checks belong to the docs keysets, and the endpoint tests remove their own uniquely named objects. `.mocharc.endpoints.cjs` sets `LIVE_TIER=endpoints`, which `support/hooks.ts` reads: no role is required up front and the full-run rule below does not apply. Mocha adds a spec file on the command line to the config's `spec` (it does not replace it), so the docs-tier config excludes `endpoints/**` with `exclude`, and a file argument of the other tier is not added to a run. Check what a script runs with `-- --dry-run`. The root `npm run test:node` lists none of this folder. The workflow (`.github/workflows/live-integration.yml`) runs nightly and on demand; it does not alert anyone on failure yet (the notify job was removed, the GitHub run logs are the only signal). + +Rebuild with `npm run build:node` after a change under `src/`, otherwise the tier tests the previous build. + +## Conventions + +- One file per docs page: `.test.ts`, named like the old page module (`docs-snippets/api-reference/publish.ts` becomes `api-reference/publish.test.ts`; keep the folder (`api-reference/`, `getting-started/`, `migration-guides/`, `shared/`) so the docs URL changes only its prefix and suffix). One `describe` per page, one `it` per old step. Start the `it` title with the snippet id(s), then what it proves. +- Plain `describe` / `it`, `node:assert/strict`. No staging, no recorder, no `page` / `step` / `check`. +- Markers sit inside the `it` body: `// snippet.` ... `// snippet.end`. The code between them is copied character for character from the old page (indentation may differ, readers get it un-indented). Everything else (setup, listeners, assertions) stays outside. Use `// snippet.hide` / `// snippet.show` only where the sample cannot run without a line readers must not see. +- Check each page with the equivalence script (below). Every snippet id must stay and its text must not change. +- **Literal names.** The code in the markers uses exactly the names the docs show (`'my_channel'`, `'foo'`, ...), and that is what runs. Isolation comes from one live run per keyset at a time and a cleanup pass before and after (the janitor). +- **Assertions must not assume a private, clean resource.** Channels, users and groups are shared and persist across runs. Assert on this run's own data: the `timetoken` the sample returned, an ID, or a unique value your test generates outside the markers. Do not assert exact counts or "the latest N". When the sample does not keep its response, match as narrowly as the sample allows (publisher, payload) and say why in a comment. +- **Reset, then assert exactly.** A test that works on a literal resource (a channel group, a user, a device token, a DataSync object) first resets it to a known state (remove, then seed what it needs), and then asserts the exact result. This is safe because the janitor cleans the listed names and one run owns a keyset at a time. Put the reset in the test or in a hook of the page, not in the sample. +- **Real keys.** A sample shows placeholder keys. Put the real ones in a `// snippet.hide` ... `// snippet.show` block inside the call: `...openKeys(),`, `...securedKeys(),` or `...subscribeKeyOnly(),` from `support/env.ts` (typed `Partial`, so no cast and no TS2783). Readers never see the block. +- **Placeholder literals.** When a sample shows a placeholder where a run needs a live value (a token, a file id `'...'`, a timetoken), the sample text stays and the test passes it a client wrapped by `withPlaceholders(client, { placeholder: liveValue })` (`support/placeholders.ts`). +- **Prettier.** Wrap a marker region in `/* eslint-disable prettier/prettier */` ... `/* eslint-enable prettier/prettier */`, placed outside the markers, when `eslint --fix` would reformat the sample. +- **Unique run ids.** Anything a test asserts on exactly and does not own (a channel it reads back, a payload marker) carries a per-run suffix: `unique('name')` from `support/util.ts`, or `randomUUID()` in the payload. Such names cannot be listed in `support/resources.ts`, so the janitor never clears them. Their messages expire with the retention of the keyset. +- **Console.** A client that logs at Debug or Trace level prints its configuration with the real keys. Wrap the test in `quietConsole()` (`support/console.ts`) and call `restore()` in a `finally`. The root hook also mutes the console while it destroys clients, so the destroy-time dumps stay out of the log. Sample `console.log` output is fine, but a sample that prints a granted token (`console.log('Granted Token:', token)`) runs under `captureConsole('log')`. The root hook restores any capture a failed test left behind (`restoreAllConsoles()`). +- **Transient service errors: retry setup, never the sample.** Setup and cleanup code outside the markers (seeding, resets) may wrap a call in `retryTransient` (`support/retry.ts`). It repeats only HTTP 5xx, 429 and network, timeout or unclassified transport failures, never a 4xx, a validation error or an assertion error, and it logs one `retryTransient:` line per retry. Code inside the markers, the assertions about a sample's effect, and calls whose failure a test asserts are never retried or wrapped. Mocha `retries` stays 0 and timeouts are not lengthened to hide flakiness. +- **Retried publishes can be stored twice.** A failed attempt may have reached the service. A seeder whose test counts messages puts `run: seedRun.id` in the payload, publishes through `seedPublish` and calls `settleSeeded` (`support/history.ts`), which deletes stored copies the caller never received a timetoken for. +- **Waiting.** Poll with `eventually`, listen with `watch` / `expectEvent`. Use a fixed `sleep` only where the test proves that something did NOT arrive. `PRESENCE_EVENT_TIMEOUT` (30 s) is for a presence event the test caused; `PRESENCE_TIMEOUT` (45 s) is for polling `hereNow` and similar reads. +- Clients come from `support/clients.ts` and are destroyed after every test by the root hook. Do not call `destroy()` for them yourself. A client built by hand (the sample's own `new PubNub(...)`) goes through `track()`. + +## Helpers (`support/`) + +| File | Exports | +| :--- | :--- | +| `env.ts` | `env()`, `keysOf(role)`, `openKeys()`, `securedKeys(role?)`, `subscribeKeyOnly()`, `describeRoles()`, `loadEnv()`, `redact()` | +| `clients.ts` | `createClient(role, userId, extra?)`, `shopper()`, `friend()`, `observer()`, `server(role?)`, `unreachable`, `track()`, `destroyClients()` | +| `wait.ts` | `eventually`, `watch` (returns a `Watch` with `events`, `next`, `stop`), `connected`, `expectEvent`, `expectStatus`, `sleep`, `PRESENCE_TIMEOUT`, `PRESENCE_EVENT_TIMEOUT` | +| `assertions.ts` | `assertTimetoken` (a string, as `publish()` returns it), `assertServerTime` (a number, as `time()` returns it), `addToTimetoken`, `rejectsValidation`, `rejectsWithCategory` | +| `placeholders.ts` | `withPlaceholders(client, values)` | +| `util.ts` | `unique(base)`, `sorted(values)` | +| `history.ts` | `storedMessage(client, channel, timetoken)`, `untilStored(channel, tokens)`, `newSeedRun`, `seedPublish`, `settleSeeded` (seeding with retry and duplicate removal) | +| `retry.ts` | `retryTransient(fn, { attempts, label })`, `isTransient(error)`; the janitor's `retry` calls it too | +| `console.ts` | `captureConsole(method)`, `quietConsole()`, `restoreAllConsoles()` | +| `resources.ts` | the janitor's resource list; `DATASYNC_PAGE_OBJECTS` is shared with `datasync-state.ts` | +| `hooks.ts` | root hooks: role check before a full run, janitor, preflight, client cleanup after each test | +| `snippets.ts`, `compare-snippets.ts` | snippet extraction with the docs rules, and the equivalence check | + +Other files in `support/` serve one page or one tool (`janitor*.ts`, `preflight.ts`, `datasync-*.ts`, `subscribe-groups.ts`, `proxy.ts`, `file-workdir.ts`, `access-manager.ts`). + +Every `observer()`, `shopper()` call builds a new client, so a test can have two clients with the same user id when it needs them. + +## Equivalence check + +```bash +npx tsx test/integration/live/support/compare-snippets.ts docs-snippets/api-reference/publish.ts test/integration/live/api-reference/publish.test.ts +``` + +It extracts every snippet of both files with the rules of the docs extractor and compares them per id. It exits 1 on any difference, missing or extra id, or marker problem. + +## Cleanup (janitor), preflight, provisioning + +The tests use the literal names the docs show (`'my_channel'`, `'family'`, ...), so the state they leave behind is shared and persistent. Three pieces keep a run honest. + +### Janitor (`support/janitor.ts`, `janitor-datasync.ts`, `resources.ts`) + +`runJanitor()` cleans **exactly the resources listed in `support/resources.ts`**, per keyset role, and nothing else. It never lists a keyset to delete what it finds, never touches a DataSync class and never changes keyset configuration. A role whose keys are missing is skipped with a note. It is best effort and idempotent, and it reports what it removed, what it could not remove and what it left on purpose. + +| Role | What it does | +| :--- | :--- | +| open | deletes the listed channel groups; removes the listed App Context users and channels and the memberships/members between listed ones; removes push registrations of the listed device tokens on fcm, apns and apns2 (production, development); deletes all files on the listed file channels (paged); deletes the message history of the listed channels (`deleteMessages` without start/end, only on channels that have messages) | +| pam | revokes the application-level Access Manager grant; the same lists as open when the role has any; history of its listed channels | +| datasync | the same revoke (once when pam and datasync are one keyset); DataSync relationships and memberships, then entities, users and channels from the list; history | + +It runs automatically from the root hooks: once before all tests (a failure fails the run with the list of what could not be removed; on a secured keyset it then waits until a client without a token is refused again, which can take up to about 60 s when the previous run just changed a grant; it gives up after 120 s), and once after all tests (a failure is printed in a loud banner and does not fail the run, except a failed grant revoke, which leaves a secured keyset open; the next run's cleanup before the run retries and fails when the keyset is still dirty). + +```bash +npm run test:integration:live:janitor # the janitor alone, all roles; exit code 1 when something could not be removed +npm run test:integration:live:janitor -- --role open +``` + +A full pass walks every listed name, so it takes a while (see the measured time in the report of this slice). For a single page or a `--grep` run, switch it off: `LIVE_JANITOR=off npm run test:integration:live -- --grep publishMinimal`. Clean up with the janitor command before and after such runs when you want a clean slate. + +**Adding a name.** Add one string to the right list of the right role in `support/resources.ts` (`channels`, `fileChannels`, `channelGroups`, `appContext.users`, `appContext.channels`, `pushDevices`, `dataSync.*`). The file is data only. A resource a test creates that is not listed there is never cleaned and leaks from run to run. One channel is left out on purpose: `history_documented_behavior` holds more than 100 messages that its test writes once and reuses (see the comment in `resources.ts`). Names with a per-run suffix are not listed either; their messages expire with the keyset's retention. Channels that appear in a channel group must be listed under `channels` as well (deleting a group does not delete its member channels). + +### Preflight (`support/preflight.ts`) + +After the janitor, the root hook fails fast, with a message that names each missing feature. With `PUBNUB_ADMIN_API_KEY` and the keyset ids (`DOCS_SNIPPETS_OPEN_KEYSET_ID`, `DOCS_SNIPPETS_PAM_KEYSET_ID`, `DOCS_SNIPPETS_DS_KEYSET_ID`) it reads each keyset's configuration through the Admin API (read only) and lists what the role needs and lacks (Presence, Message Persistence, Delete-From-History, App Context and its events, Files, Stream Controller, Access Manager, DataSync). Live probes on the open keyset always run: publish (keys and network), Message Persistence, Delete-From-History, and Presence when the Admin API did not vouch for it. The features each keyset needs are listed in `docs-snippets/README.md`. + +### Provisioning (`provision/`) + +DataSync classes are created by `provision/provision-datasync.ts` from `provision/datasync-classes.json`. It is idempotent, never deletes a class and fails on a class whose immutable fields differ. The tier does not run it. Run it when a datasync keyset is new or the class file changes. + +```bash +npm run test:integration:live:provision:check # dry diff, writes nothing, exit code 1 when anything differs +npm run test:integration:live:provision # creates missing classes, updates drifted mutable fields +``` + +Without `PUBNUB_ADMIN_API_KEY` it only checks through the SDK that each class exists. The Admin API key needs a DataSync permission row for the datasync keyset. + +### Environment variables + +| Name | Meaning | +| :--- | :--- | +| `SUBSCRIBE_KEY`, `PUBLISH_KEY`, `PAM_*`, `DS_*` | keysets, as above | +| `PUBNUB_ADMIN_API_KEY` | optional; Admin API feature check and provisioning | +| `DOCS_SNIPPETS_OPEN_KEYSET_ID`, `DOCS_SNIPPETS_PAM_KEYSET_ID`, `DOCS_SNIPPETS_DS_KEYSET_ID` | optional; keyset ids for the feature check (not secrets) | +| `LIVE_JANITOR=off` | skip the janitor before and after the run (any other value runs it) | +| `LIVE_PREFLIGHT=off` | skip the preflight | +| `LIVE_ENV_FILE` | another env file | + +Everything the janitor, the preflight and the provisioning print goes through `redact()` (`support/env.ts`), which masks every key of the environment, the admin key, key-shaped strings and long token-like strings. + +## Offline checks: marker lint, coverage manifest, docs cross-check + +Three scripts that need no keys and no network. They answer one question: does every snippet the public docs embed from this repository run against the real service in a test here, or sit on a listed exception with a reason? + +```bash +npm run test:integration:live:lint # marker lint + manifest --check + eslint of the tier +npm run test:integration:live:manifest # regenerate coverage.json +npm run test:integration:live:docs-check # docs cross-check (needs the documentation checkout) +``` + +**Lint** (`support/lint-markers.ts`; a self-test with deliberately broken files is `support/lint-selftest.ts`). It scans every `*.test.ts` and sample file under this folder and fails with `file:line` messages on: malformed, unbalanced or nested markers (the docs extractor would silently cut the outer snippet), `snippet.hide` without `snippet.show`, hidden blocks longer than 3 lines, an id outside `[a-zA-Z0-9_-]`, an id used twice anywhere in the tier, an empty snippet, test-only code in the visible text (`assert`, `expect`, `eventually`, `watch`, anything imported from `support/` or `node:assert`), and `try` / `catch` / `console.error` in a sample (allow-list `ERROR_HANDLING_SNIPPETS` and the `use-cases/` folder, `ERROR_HANDLING_FOLDERS`, both in the lint file). The `use-cases/` exemption is a decision of the docs owner: use-case samples are application code that shows its own error handling (try/catch around each call, `console.error`), so the protection against a swallowed error there is the assertion after the snippet. Instead of the old run-time proof it checks statically, with the TypeScript compiler API, that each snippet sits inside an `it()` (or a named helper an `it()` calls), that an assertion call follows the snippet's end line in that `it()` (`assert*`, `rejects*`, `expect*`, `eventually`, or an `await` of a promise armed earlier by such a call, like `const back = expectStatus(...)` ... `await back`), and that the snippet has no `return` of its own that could skip the rest. When the static check cannot see an assertion, put `// lint: asserted-via ` after the snippet in the `it()` body: the helper must be called after the snippet, and the lint prints every use. Only import-only snippets may sit at the top of a file (mode `compile-only`). One narrow escape exists for a file-level sample that starts with an `import` but also holds functions and other code (it cannot sit in an `it()`): put `// lint: asserted-by ` on the line right after its `// snippet.end`. The text must match exactly one `it()` title of the file, that `it()` must contain an assertion call, the manifest records mode `live` with that test, and the lint prints every use as a note (today: `fanBehaviorStateStore`, whose state file the test reads back). A hidden block stays at 3 lines or fewer: put setup in a helper defined before the snippet and call it from the hidden block. Known blind spots: the check proves an assertion call exists after the snippet, not that it checks the snippet's effect; a helper called through a variable or callback is not followed. + +**Manifest** (`support/manifest.ts`, output `coverage.json`, generated, do not edit). One entry per snippet id: file, the full Mocha title that runs it (describe + it), keyset role when the test names one (heuristic), and `mode`: `live` (inside an `it()`), `compile-only` (import-only samples: type-checked and loaded, not run as an API call), `mocked` (inside a nock-mocked test listed in `MOCKED_FILES` in `support/snippet-scan.ts`: `test/integration/endpoints/objects/channel.test.ts`, `test/integration/endpoints/time.test.ts`, `test/integration/components/subscription_manager.test.ts`, five samples; the same lint rules apply, except the title-names-the-id warning, which is the live tier's convention). `files` lists whole-file samples that a test runs (`getting-started/quickstart-node.mjs`). `-- --check` fails when `coverage.json` is stale. Things the docs embed that are deliberately not run (the browser, React and React Native quickstarts) are listed by hand, with a reason, in `snippet-exceptions.json`. + +**Docs cross-check** (`support/docs-check.ts`). Reads every `` that points at `pubnub/javascript` in the documentation checkout (first argument, `DOCS_REPO`, or `../documentation`) and reports each embed as live, compile-only, mocked, exception or NOT COVERED (exit code 1 unless `--report-only`). It understands both URL styles. An old-style URL (`docs-snippets//.ts`) is mapped to `test/integration/live//.test.ts`. A new-style URL (a path under `test/integration/live/`, or one of the mocked test files) is checked as it stands: the file must exist and `coverage.json` must list each requested snippet id for that very file (a whole-file embed needs a file named by a test in `coverage.json`, or a listed exception); its `action` in the repoint list is `done`. Use `--out ` when you run it on a checkout so the default repoint list is not overwritten. For an embed that points at an old duplicate copy (`docs-snippets/basic-usage/*`, `docs-snippets/.ts`) it finds the same id in the tier and says whether the old copy and the tested twin are IDENTICAL or DIFFERENT (diffs in `/tmp/pn-live/dup-diffs/`). Such embeds stay listed as NOT COVERED until the docs are repointed. It writes the list for repointing the docs URLs to `/tmp/pn-live/docs-repoint.json` (`--out ` to change); its `action` is `repoint` (the mapped path), `repoint-to-twin` (the embed points at an old duplicate copy: change it to `newPath`, the tested twin), `keep` (documented exception), `done` (the URL already points at the tested tier) or `blocked` (not covered and no twin). The pages `users`, `spaces`, `memberships` and `access-manager-v2` of the API reference are out of scope. The documentation repository is only read. diff --git a/test/integration/live/api-reference/access-manager-legacy.test.ts b/test/integration/live/api-reference/access-manager-legacy.test.ts new file mode 100644 index 000000000..820f5d504 --- /dev/null +++ b/test/integration/live/api-reference/access-manager-legacy.test.ts @@ -0,0 +1,249 @@ +import assert from 'node:assert/strict'; +import type PubNub from 'pubnub'; +import { rejectsValidation } from '../support/assertions'; +import { server } from '../support/clients'; +import { granted } from '../support/access-manager'; + +/** + * Access Manager token examples that use the deprecated `spaces`, `users` and `uuids` terms. The SDK maps them to the + * current terms, so a `spaces` grant appears as `channels` in the parsed token. + * Port of docs-snippets/api-reference/access-manager-legacy.ts. Only `grantToken` is called (no application-level grant), + * so no cleanup hook is needed. Tokens expire by themselves (ttl 15). + */ +describe('api-reference/access-manager-legacy', () => { + interface LegacyGrantClient { + grantToken(parameters: PubNub.PAM.ObjectsGrantTokenParameters): Promise; + parseToken: PubNub['parseToken']; + } + const legacy = (): LegacyGrantClient => server() as unknown as LegacyGrantClient; + + it('grantTokenSpacesBasicUsage: a grant that uses the deprecated spaces scope returns a token whose parsed form lists the space under channels', async () => { + const pubnub = legacy(); + let tokenOut = ''; + // prettier-ignore + // snippet.grantTokenSpacesBasicUsage + try { + const token = await pubnub.grantToken({ + ttl: 15, + authorizedUserId: "my-authorized-userId", + resources: { + spaces: { + "my-space": { + read: true, + }, + }, + }, + }); + // snippet.hide + tokenOut = token; + // snippet.show + } catch (status) { + console.log(status); + } + // snippet.end + // The sample swallows errors with its catch, so an empty token means the grant failed. + assert.ok(tokenOut, 'the grant returned a token'); + const parsed = pubnub.parseToken(tokenOut); + assert.equal(parsed?.authorized_uuid, 'my-authorized-userId'); + assert.deepEqual(granted(parsed?.resources?.channels?.['my-space']), { read: true }); + }); + + it('grantTokenSpacesVariousResources: a token with mixed spaces and users resources keeps one space entry and one user entry with their permissions', async () => { + const pubnub = legacy(); + let tokenOut = ''; + // prettier-ignore + // snippet.grantTokenSpacesVariousResources + try { + const token = await pubnub.grantToken({ + ttl: 15, + authorizedUserId: "my-authorized-userId", + resources: { + spaces: { + "space-a": { + read: true, + }, + "space-b": { + read: true, + write: true, + }, + "space-c": { + read: true, + write: true, + }, + "space-d": { + read: true, + write: true, + }, + }, + users: { + "userId-c": { + get: true, + }, + "userId-d": { + get: true, + update: true, + }, + }, + }, + }); + // snippet.hide + tokenOut = token; + // snippet.show + } catch (status) { + console.log(status); + } + // snippet.end + // The sample swallows errors with its catch, so an empty token means the grant failed. + assert.ok(tokenOut, 'the grant returned a token'); + const parsed = pubnub.parseToken(tokenOut); + assert.deepEqual(granted(parsed?.resources?.channels?.['space-b']), { read: true, write: true }); + assert.deepEqual(granted(parsed?.resources?.users?.['userId-d']), { get: true, update: true }); + }); + + it('grantTokenSpacesRegEx: a pattern-only grant with the deprecated spaces scope keeps the regular expression unchanged under patterns.channels', async () => { + const pubnub = legacy(); + let tokenOut = ''; + // prettier-ignore + // snippet.grantTokenSpacesRegEx + try { + const token = await pubnub.grantToken({ + ttl: 15, + authorizedUserId: "my-authorized-userId", + patterns: { + spaces: { + "^space-[A-Za-z0-9]$": { + read: true, + }, + }, + }, + }); + // snippet.hide + tokenOut = token; + // snippet.show + } catch (status) { + console.log(status); + } + // snippet.end + // The sample swallows errors with its catch, so an empty token means the grant failed. + assert.ok(tokenOut, 'the grant returned a token'); + const parsed = pubnub.parseToken(tokenOut); + assert.deepEqual(granted(parsed?.patterns?.channels?.['^space-[A-Za-z0-9]$']), { read: true }); + }); + + it('grantTokenSpacesRegExAndResources: resources and patterns in one deprecated-scope grant both survive in the parsed token', async () => { + const pubnub = legacy(); + let tokenOut = ''; + // prettier-ignore + // snippet.grantTokenSpacesRegExAndResources + try { + const token = await pubnub.grantToken({ + ttl: 15, + authorizedUserId: "my-authorized-userId", + resources: { + spaces: { + "space-a": { + read: true, + }, + "space-b": { + read: true, + write: true, + }, + "space-c": { + read: true, + write: true, + }, + "space-d": { + read: true, + write: true, + }, + }, + users: { + "userId-c": { + get: true, + }, + "userId-d": { + get: true, + update: true, + }, + }, + }, + patterns: { + spaces: { + "^space-[A-Za-z0-9]$": { + read: true + }, + }, + }, + }); + // snippet.hide + tokenOut = token; + // snippet.show + } catch (status) { + console.log(status); + } + // snippet.end + // The sample swallows errors with its catch, so an empty token means the grant failed. + assert.ok(tokenOut, 'the grant returned a token'); + const parsed = pubnub.parseToken(tokenOut); + assert.deepEqual(granted(parsed?.resources?.channels?.['space-a']), { read: true }); + assert.deepEqual(granted(parsed?.patterns?.channels?.['^space-[A-Za-z0-9]$']), { read: true }); + }); + + it('accessManagerLegacyAuthorizedUuidAlias: authorized_uuid and authorizedUserId produce the same token owner, and passing both is a validation error', async () => { + const staff = server(); + const user = 'my-authorized-userId'; + const resources = { channels: { 'legacy-alias': { read: true } } }; + const viaAlias = await staff.grantToken({ ttl: 15, authorized_uuid: user, resources }); + const viaCurrent = await staff.grantToken({ ttl: 15, authorizedUserId: user, resources }); + assert.equal(staff.parseToken(viaAlias)?.authorized_uuid, user); + assert.equal(staff.parseToken(viaCurrent)?.authorized_uuid, user); + await rejectsValidation( + staff.grantToken({ ttl: 15, authorizedUserId: user, authorized_uuid: user, resources } as never), + /Cannot mix `authorizedUserId` with `authorized_uuid`/, + ); + }); + + // The `uuids` scope is the old name of `users`. The parsed token keeps it under `uuids`, for resources and patterns. + it('accessManagerLegacyUuidsScope: the uuids scope is the old name of users and the parsed token keeps it under uuids for resources and patterns', async () => { + const staff = server(); + const token = await staff.grantToken({ + ttl: 15, + authorizedUserId: 'my-authorized-userId', + resources: { uuids: { 'uuid-1': { get: true, update: true, delete: true } } }, + patterns: { uuids: { 'uuid-.*': { get: true } } }, + }); + const parsed = staff.parseToken(token); + assert.deepEqual(granted(parsed?.resources?.uuids?.['uuid-1']), { get: true, update: true, delete: true }); + assert.deepEqual(granted(parsed?.patterns?.uuids?.['uuid-.*']), { get: true }); + }); + + // The SDK refuses to mix old and new scope names in one grant. These are client-side validation errors, so no + // request leaves the process and the secured keyset sees no traffic. + it('accessManagerLegacyMixingErrors: mixing old and new scope names in one grant is refused locally with a validation error', async () => { + const staff = server(); + const ttl = 15; + // users and uuids: the check spans resources and patterns together. + await rejectsValidation( + staff.grantToken({ + ttl, + resources: { users: { a: { get: true } } }, + patterns: { uuids: { 'a.*': { get: true } } }, + }), + /Cannot mix `users` with `uuids`/, + ); + await rejectsValidation( + staff.grantToken({ + ttl, + resources: { spaces: { a: { read: true } }, channels: { b: { read: true } } }, + } as never), + /Cannot mix `spaces` with `channels`/, + ); + await rejectsValidation( + staff.grantToken({ + ttl, + resources: { spaces: { a: { read: true } }, uuids: { b: { get: true } } }, + } as never), + /Cannot mix `spaces` with `uuids`/, + ); + }); +}); diff --git a/test/integration/live/api-reference/access-manager.test.ts b/test/integration/live/api-reference/access-manager.test.ts new file mode 100644 index 000000000..897656ff9 --- /dev/null +++ b/test/integration/live/api-reference/access-manager.test.ts @@ -0,0 +1,457 @@ +import assert from 'node:assert/strict'; +import type PubNub from 'pubnub'; +import { assertTimetoken, rejectsValidation, rejectsWithCategory } from '../support/assertions'; +import { captureConsole } from '../support/console'; +import { createClient, server, shopper } from '../support/clients'; +import { withPlaceholders } from '../support/placeholders'; +import { granted } from '../support/access-manager'; +import { connected, eventually } from '../support/wait'; + +/** + * Access Manager v3 token examples. The `grantToken` calls run on a server that holds the secret key (pam keyset). + * Port of docs-snippets/api-reference/access-manager.ts. + * + * Tokens are bound to a user id and a resource, never to an application-wide grant, so nothing here opens the keyset for + * strangers and no cleanup hook is needed. The only residue is tokens that expire by themselves (ttl 15 minutes) and the + * revoked token of the revoke test, which stays on the service's revoke list until it would have expired. + * + * Samples with a placeholder literal ('use-token-string-generated-by-grantToken()', the token to revoke) run unchanged: + * the test passes the sample a client wrapper (`withPlaceholders`) that swaps the literal for the live token. + */ +describe('api-reference/access-manager', () => { + const room = 'am_room'; + const other = 'am_other'; + const GRANTED = 'use-token-string-generated-by-grantToken()'; + let aliceToken = ''; + let bobToken = ''; + + /** Publishes to a channel and resolves with the timetoken. */ + const publishTo = async (client: PubNub, channel: string): Promise => { + const response = await client.publish({ channel, message: { text: 'am-check' } }); + return response.timetoken; + }; + + before(async function () { + this.timeout(60_000); + // Alice gets read and write on the room, Bob read only. + const staff = server(); + aliceToken = await staff.grantToken({ + ttl: 15, + authorizedUserId: 'user-alice', + resources: { channels: { [room]: { read: true, write: true } } }, + }); + bobToken = await staff.grantToken({ + ttl: 15, + authorizedUserId: 'user-bob', + resources: { channels: { [room]: { read: true } } }, + }); + }); + + it('accessManagerTokenFixture: Alice and Bob tokens give Bob read only access: publish is denied, subscribe connects', async function () { + this.timeout(60_000); + const bob = createClient('pam', 'user-bob'); + bob.setToken(bobToken); + assert.equal(bob.getToken(), bobToken); + // Bob holds a read-only token: he can subscribe to the room but not publish to it. + await rejectsWithCategory(bob.publish({ channel: room, message: 'x' }), 'PNAccessDeniedCategory'); + const subscription = bob.channel(room).subscription(); + const ready = connected(bob, { channels: [room] }); + subscription.subscribe(); + await ready; + subscription.unsubscribe(); + // Alice with write access can publish. + const alice = createClient('pam', 'user-alice'); + alice.setToken(aliceToken); + assertTimetoken(await publishTo(alice, room)); + }); + + it('accessManagerBasicUsage: grantToken returns a token whose parsed form carries the user id, channel permissions and ttl that were asked for', async () => { + const pubnub = server(); + // The sample prints the granted token. Keep it out of the test output. + const logs = captureConsole('log'); + // snippet.accessManagerBasicUsage + // Function to use grantToken method + const token = await pubnub.grantToken({ + ttl: 15, + authorizedUserId: 'my-authorized-uuid', + resources: { + channels: { + 'my-channel': { + read: true, + write: true, + }, + }, + }, + }); + console.log('Granted Token:', token); + // snippet.end + logs.restore(); + const parsed = pubnub.parseToken(token); + assert.equal(parsed?.authorized_uuid, 'my-authorized-uuid'); + assert.deepEqual(granted(parsed?.resources?.channels?.['my-channel']), { read: true, write: true }); + assert.equal(parsed?.ttl, 15); + }); + + it('revokeTokenBasicUsage: revokeToken makes a working token stop working, for the client that was allowed before and for fresh clients with the same token', async function () { + // The docs say a revoke can take up to one minute, because the non-revoked state is cached for that long. + this.timeout(300_000); + // Fixture: a user with a working token, so the effect of the revoke is visible. + const staff = server(); + const revokee = createClient('pam', 'am-revokee'); + const token = await staff.grantToken({ + ttl: 15, + authorizedUserId: 'am-revokee', + resources: { channels: { [other]: { read: true, write: true } } }, + }); + revokee.setToken(token); + assertTimetoken(await publishTo(revokee, other)); + + const pubnub = withPlaceholders(server(), { + p0AkFl043rhDdHRsple3KgQ3NwY6BDcENnctokenVzcqBDczaWdYIGOAeTyWGJI: token, + }); + // snippet.revokeTokenBasicUsage + const response = await pubnub.revokeToken('p0AkFl043rhDdHRsple3KgQ3NwY6BDcENnctokenVzcqBDczaWdYIGOAeTyWGJI'); + // snippet.end + assert.deepEqual(response, {}, 'revokeToken resolves with an empty object'); + await eventually(() => rejectsWithCategory(publishTo(revokee, other), 'PNAccessDeniedCategory'), { + timeout: 100_000, + interval: 2_000, + label: 'revoked token is rejected', + }); + for (const attempt of [1, 2]) { + const fresh = createClient('pam', 'am-revokee'); + fresh.setToken(token); + await eventually(() => rejectsWithCategory(publishTo(fresh, other), 'PNAccessDeniedCategory'), { + timeout: 60_000, + interval: 2_000, + label: `revoked token is rejected for fresh client ${attempt}`, + }); + } + }); + + it('parseTokenBasicUsage: parseToken decodes a token locally, and the fixture token shows the right owner and permissions', () => { + const pubnub = withPlaceholders(server(), { [GRANTED]: aliceToken }); + // snippet.parseTokenBasicUsage + pubnub.parseToken('use-token-string-generated-by-grantToken()'); + // snippet.end + const parsed = pubnub.parseToken(aliceToken); + assert.equal(parsed?.authorized_uuid, 'user-alice'); + assert.deepEqual(granted(parsed?.resources?.channels?.[room]), { read: true, write: true }); + }); + + it('setTokenBasicUsage: setToken lets Alice publish to her room but not to another, while Bob stays read only', async function () { + this.timeout(60_000); + const pubnub = withPlaceholders(createClient('pam', 'user-alice'), { [GRANTED]: aliceToken }); + // Without a token the secured keyset refuses Alice. + pubnub.setToken(undefined); + await rejectsWithCategory(publishTo(pubnub, room), 'PNAccessDeniedCategory'); + // snippet.setTokenBasicUsage + pubnub.setToken('use-token-string-generated-by-grantToken()'); + // snippet.end + assert.equal(pubnub.getToken(), aliceToken); + assertTimetoken(await publishTo(pubnub, room)); + // The token names one room only. + await rejectsWithCategory(publishTo(pubnub, other), 'PNAccessDeniedCategory'); + // Bob (read only) still cannot publish. + const bob = createClient('pam', 'user-bob'); + bob.setToken(bobToken); + await rejectsWithCategory(publishTo(bob, room), 'PNAccessDeniedCategory'); + }); + + // Channels, groups and uuids in one token. Property keys are not suffixed, so the parsed token is read with the plain keys. + it('grantTokenVariousResources: a token with channels, groups and uuids keeps each resource with its permissions in the parsed form', async () => { + const pubnub = server(); + // snippet.grantTokenVariousResources + const token = await pubnub.grantToken({ + ttl: 15, + authorizedUserId: 'my-authorized-uuid', + resources: { + channels: { + 'channel-a': { + read: true, + }, + 'channel-b': { + read: true, + write: true, + }, + 'channel-c': { + read: true, + write: true, + }, + 'channel-d': { + read: true, + write: true, + }, + }, + groups: { + 'channel-group-b': { + read: true, + }, + }, + uuids: { + 'uuid-c': { + get: true, + }, + 'uuid-d': { + get: true, + update: true, + }, + }, + }, + }); + // snippet.end + const parsed = pubnub.parseToken(token); + assert.deepEqual(granted(parsed?.resources?.channels?.['channel-a']), { read: true }); + assert.deepEqual(granted(parsed?.resources?.channels?.['channel-d']), { read: true, write: true }); + assert.deepEqual(granted(parsed?.resources?.groups?.['channel-group-b']), { read: true }); + assert.deepEqual(granted(parsed?.resources?.uuids?.['uuid-d']), { get: true, update: true }); + }); + + // Pattern-only token: the regular expression key must survive unchanged in `patterns.channels`. + it('grantTokenUsingRegEx: a pattern-only token keeps the regular expression unchanged under patterns.channels', async () => { + const pubnub = server(); + // snippet.grantTokenUsingRegEx + const token = await pubnub.grantToken({ + ttl: 15, + authorizedUserId: 'my-authorized-uuid', + patterns: { + channels: { + '^channel-[A-Za-z0-9]$': { + read: true, + }, + }, + }, + }); + // snippet.end + const parsed = pubnub.parseToken(token); + assert.deepEqual(granted(parsed?.patterns?.channels?.['^channel-[A-Za-z0-9]$']), { read: true }); + }); + + // Resources and patterns together, both read back from the parsed token. + it('grantTokenRegExAndResources: a token with resources and patterns returns both sections when parsed', async () => { + const pubnub = server(); + // snippet.grantTokenRegExAndResources + const token = await pubnub.grantToken({ + ttl: 15, + authorizedUserId: 'my-authorized-uuid', + resources: { + channels: { + 'channel-a': { + read: true, + }, + 'channel-b': { + read: true, + write: true, + }, + 'channel-c': { + read: true, + write: true, + }, + 'channel-d': { + read: true, + write: true, + }, + }, + groups: { + 'channel-group-b': { + read: true, + }, + }, + uuids: { + 'uuid-c': { + get: true, + }, + 'uuid-d': { + get: true, + update: true, + }, + }, + }, + patterns: { + channels: { + '^channel-[A-Za-z0-9]$': { + read: true, + }, + }, + }, + }); + // snippet.end + const parsed = pubnub.parseToken(token); + assert.deepEqual(granted(parsed?.resources?.channels?.['channel-b']), { read: true, write: true }); + assert.deepEqual(granted(parsed?.patterns?.channels?.['^channel-[A-Za-z0-9]$']), { read: true }); + }); + + // Category permissions (`get` on all channel metadata and all uuid metadata). + it('grantTokenWithCategories: a category token carries get on all channel metadata and all uuid metadata', async () => { + const pubnub = server(); + // The sample prints the granted token. Keep it out of the test output. + const logs = captureConsole('log'); + // snippet.grantTokenWithCategories + // Grants `my-authorized-uuid` permission to list all channel metadata and all uuid metadata. + // Category permissions support only the `get` operation. + const token = await pubnub.grantToken({ + ttl: 15, + authorizedUserId: 'my-authorized-uuid', + categories: { + channels: { get: true }, + uuids: { get: true }, + }, + }); + console.log('Granted token:', token); + // snippet.end + logs.restore(); + const parsed = pubnub.parseToken(token); + assert.equal(parsed?.categories?.channels?.get, true); + assert.equal(parsed?.categories?.uuids?.get, true); + }); + + it('parseTokenWithCategories: parseToken reads the category permissions of a category token', async () => { + const staff = server(); + const categoryToken = await staff.grantToken({ + ttl: 15, + authorizedUserId: 'my-authorized-uuid', + categories: { channels: { get: true }, uuids: { get: true } }, + }); + const pubnub = withPlaceholders(server(), { [GRANTED]: categoryToken }); + // The sample logs the result and does not return it, so read it back from the console output. + const logs = captureConsole('log'); + try { + // snippet.parseTokenWithCategories + // Decodes a token that was granted with category permissions. + const parsedToken = pubnub.parseToken('use-token-string-generated-by-grantToken()'); + + console.log('Can list all channel metadata:', parsedToken?.categories?.channels?.get); + console.log('Can list all uuid metadata:', parsedToken?.categories?.uuids?.get); + // snippet.end + } finally { + logs.restore(); + } + assert.deepEqual(logs.lines, ['Can list all channel metadata: true', 'Can list all uuid metadata: true']); + }); + + it('grantTokenDataSyncResources: a token can carry DataSync entity scopes and the top-level users scope, and parseToken returns them', async () => { + const pubnub = server('datasync'); + // The sample prints the granted token. Keep it out of the test output. + const logs = captureConsole('log'); + // snippet.grantTokenDataSyncResources + // Grants `user-alice`: + // - Get access to the entity `product-sneaker-42`, and to all entities matching the + // `product-.*` RegEx pattern. + // - Get access to the DataSync user `user-bob` through the top-level `users` scope, because + // DataSync users are not permissioned under `dataSync`. + const token = await pubnub.grantToken({ + ttl: 15, + authorizedUserId: 'user-alice', + resources: { + users: { + 'user-bob': { + get: true, + }, + }, + dataSync: { + entities: { + 'product-sneaker-42': { + get: true, + }, + }, + }, + }, + patterns: { + dataSync: { + entities: { + 'product-.*': { + get: true, + }, + }, + }, + }, + }); + console.log('Granted Token:', token); + // snippet.end + logs.restore(); + const parsed = pubnub.parseToken(token); + assert.equal(parsed?.authorized_uuid, 'user-alice'); + assert.ok(parsed?.resources, 'the token carries resources'); + }); + + it('grantTokenWithProjection: a token with dataSyncProjections is created and parsed, and carries the projection assignment of the entity', async () => { + const pubnub = server('datasync'); + // The sample prints the granted token. Keep it out of the test output. + const logs = captureConsole('log'); + // snippet.grantTokenWithProjection + // Grants `user-alice` the same access as above and assigns the `public` projection to the entity + // `product-sneaker-42` and to all entities matching `product-.*`. The entity class must declare + // a `public` projection. With it, Alice reads and writes only the fields that the projection exposes. + const token = await pubnub.grantToken({ + ttl: 15, + authorizedUserId: 'user-alice', + resources: { + users: { + 'user-bob': { + get: true, + }, + }, + dataSync: { + entities: { + 'product-sneaker-42': { + get: true, + }, + }, + }, + }, + patterns: { + dataSync: { + entities: { + 'product-.*': { + get: true, + }, + }, + }, + }, + dataSyncProjections: { + resources: { + entities: { + 'product-sneaker-42': 'public', + }, + }, + patterns: { + entities: { + 'product-.*': 'public', + }, + }, + }, + }); + console.log('Granted Token:', token); + // snippet.end + logs.restore(); + const parsed = pubnub.parseToken(token) as + | { meta?: Record; pat?: Record }> } + | undefined; + assert.ok(parsed, 'the token parses'); + assert.equal(parsed.meta?.['pn-projections']?.res?.['datasync:entities:product-sneaker-42'], 'public'); + assert.equal(parsed.meta?.['pn-projections']?.pat?.['datasync:entities:product-.*'], 'public'); + }); + + it('accessManagerGrantTokenErrors: the documented grantToken mistakes are refused locally with validation errors and create no token', async () => { + const staff = server(); + await rejectsValidation(staff.grantToken({ ttl: 15 } as never), /Missing values for either Resources or Patterns/); + await rejectsValidation( + staff.grantToken({ ttl: 15, resources: { channels: {} } }), + /Missing values for either Resources or Patterns/, + ); + await rejectsValidation(staff.revokeToken(''), /token can't be empty/); + await rejectsValidation(shopper().revokeToken('some-token'), /Missing Secret Key/); + // The open client has no secret key, so it cannot grant. + await rejectsValidation( + shopper().grantToken({ ttl: 15, resources: { channels: { x: { read: true } } } }), + /Missing Secret Key/, + ); + }); + + // A client that never received a token is rejected on the secured keyset. Fails while an application-level (v2) grant + // from another page or an earlier run is still open on the pam keyset: the tier-wide cleanup closes it. + it('accessManagerDeniedWithoutToken: a client that never received a token is refused with a 403 on the secured keyset', async () => { + const stranger = createClient('pam', 'am-denied'); + await rejectsWithCategory(publishTo(stranger, room), 'PNAccessDeniedCategory'); + }); +}); diff --git a/test/integration/live/api-reference/channel-groups.test.ts b/test/integration/live/api-reference/channel-groups.test.ts new file mode 100644 index 000000000..832acc83e --- /dev/null +++ b/test/integration/live/api-reference/channel-groups.test.ts @@ -0,0 +1,127 @@ +import assert from 'node:assert/strict'; +import type PubNub from 'pubnub'; +import { observer, shopper } from '../support/clients'; +import { retryTransient } from '../support/retry'; +import { eventually } from '../support/wait'; +import { sorted } from '../support/util'; + +/** + * Channel group examples. Port of docs-snippets/api-reference/channel-groups.ts. + * The groups 'myChannelGroup' and 'family' are shared literal names, so every test deletes the group it works on first + * (reset approach). After the reset the exact channel list assertions are legitimate. + */ +describe('api-reference/channel-groups', () => { + // Channel order in a group is not guaranteed, so compare sorted copies. + + const listed = async (pubnub: PubNub, channelGroup: string): Promise => + (await pubnub.channelGroups.listChannels({ channelGroup })).channels; + + /** Deletes the group and waits until another client sees it empty. */ + async function resetGroup(channelGroup: string): Promise { + const reader = observer(); + await reader.channelGroups.deleteGroup({ channelGroup }); + await eventually(async () => assert.deepEqual(await listed(reader, channelGroup), []), { + label: `group ${channelGroup} reset`, + }); + } + + /** Resets a group, fills it with `channels` and waits until another client lists exactly those. */ + async function seedGroup(channelGroup: string, channels: string[]): Promise { + await resetGroup(channelGroup); + const reader = observer(); + await retryTransient(() => reader.channelGroups.addChannels({ channelGroup, channels }), { + label: 'seed channel group', + }); + await eventually(async () => assert.deepEqual(sorted(await listed(reader, channelGroup)), sorted(channels)), { + label: `group ${channelGroup} seeded`, + }); + } + + after(async () => { + // The samples leave both groups behind. + const reader = observer(); + await reader.channelGroups.deleteGroup({ channelGroup: 'myChannelGroup' }); + await reader.channelGroups.deleteGroup({ channelGroup: 'family' }); + }); + + it('addChannelsToGroupBasicUsage: addChannels adds two channels to a group, and a second client reads them back from the group', async () => { + const pubnub = shopper(); + // Reset approach: the group starts empty, so the list below is exactly this sample's two channels. + await resetGroup('myChannelGroup'); + + // snippet.addChannelsToGroupBasicUsage + const response = await pubnub.channelGroups.addChannels({ + channels: ['ch1', 'ch2'], + channelGroup: 'myChannelGroup', + }); + console.log('addChannels to Group response:', response); + // snippet.end + + // Another client confirms the effect. + const reader = observer(); + await eventually(async () => assert.deepEqual(sorted(await listed(reader, 'myChannelGroup')), ['ch1', 'ch2']), { + label: 'group lists ch1 and ch2', + }); + }); + + it('listChannelsInGroupBasicUsage: listChannels returns the channels that were added to the group', async () => { + const pubnub = shopper(); + // Reset approach: the sample says to add channels first, so this test does that on a freshly reset group. + await seedGroup('myChannelGroup', ['ch1', 'ch2']); + + // snippet.listChannelsInGroupBasicUsage + // assuming an intialized PubNub instance already exists + // to get some data in response, first add some channels to the group using addChannels() method. + const response = await pubnub.channelGroups.listChannels({ + channelGroup: 'myChannelGroup', + }); + console.log('Channels in the group:', response); + response.channels.forEach((channel) => { + console.log(channel); + }); + // snippet.end + + assert.deepEqual(sorted(response.channels), ['ch1', 'ch2']); + }); + + it('removeChannelsFromGroupBasicUsage: removeChannels removes one channel from a group and leaves only the other one', async () => { + const pubnub = shopper(); + // The docs say the channel must already be in the group to observe the removal. Reset approach on 'family'. + await seedGroup('family', ['son', 'daughter']); + + // snippet.removeChannelsFromGroupBasicUsage + // assuming an initialized PubNub instance already exists + // and channel which is going to be removed from the group is aredaly added to the group to observe the removal + const response = await pubnub.channelGroups.removeChannels({ + channels: ['son'], + channelGroup: 'family', + }); + console.log('removeChannels from group response:', response); + // snippet.end + + const reader = observer(); + await eventually(async () => assert.deepEqual(await listed(reader, 'family'), ['daughter']), { + label: 'son removed, daughter stays', + }); + }); + + it('deleteChannelGroupBasicUsage: deleteGroup removes the group, and a deleted group lists no channels', async () => { + const pubnub = shopper(); + // The group must exist to see the deletion effect. Reset approach: fill 'family' with exactly one channel. + await seedGroup('family', ['son']); + + // snippet.deleteChannelGroupBasicUsage + // assuming an initialized PubNub instance already exists + // and channel group which is getting deleted already exist to see the deletion effect. + const response = await pubnub.channelGroups.deleteGroup({ + channelGroup: 'family', + }); + console.log('deleteChannelGroup response:', response); + // snippet.end + + const reader = observer(); + await eventually(async () => assert.deepEqual(await listed(reader, 'family'), []), { + label: 'family group is empty after deletion', + }); + }); +}); diff --git a/test/integration/live/api-reference/configuration-legacy.test.ts b/test/integration/live/api-reference/configuration-legacy.test.ts new file mode 100644 index 000000000..56b5eaa73 --- /dev/null +++ b/test/integration/live/api-reference/configuration-legacy.test.ts @@ -0,0 +1,201 @@ +import assert from 'node:assert/strict'; +import PubNub from 'pubnub'; +import { createClient, track } from '../support/clients'; +import { quietConsole } from '../support/console'; +import { keysOf } from '../support/env'; +import { unique } from '../support/util'; +import { connected, eventually } from '../support/wait'; + +/** + * Examples for the deprecated configuration parameters `uuid`, `logVerbosity`, `cipherKey` and `useRandomIVs`, and for the + * deprecated `stop()` method. Port of docs-snippets/api-reference/configuration-legacy.ts. The page has no snippet markers. + * The cipherKey and useRandomIVs tests read stored messages back, so Message Persistence must be on for the open keyset. + */ + +// These tests build clients with raw (deprecated) configuration objects. +const keys = (): { publishKey: string; subscribeKey: string } => { + const { publishKey, subscribeKey } = keysOf('open'); + return { publishKey, subscribeKey }; +}; + +// The logger records what the SDK logs so the tests can assert deprecation warnings without parsing stdout. +/** A logger that records the levels it is asked to handle. */ +const recorder = (): { logger: PubNub.Logger; seen: Array<{ level: string; text: string }> } => { + const seen: Array<{ level: string; text: string }> = []; + const handle = (level: string) => (m: PubNub.LogMessage) => { + seen.push({ level, text: m.messageType === 'text' ? m.message : 'details' in m ? (m.details ?? '') : '' }); + }; + return { + seen, + logger: { + trace: handle('trace'), + debug: handle('debug'), + info: handle('info'), + warn: handle('warn'), + error: handle('error'), + }, + }; +}; + +// Reads a stored message with a client that has no crypto, so the cipher text is visible as a string. History is +// eventually consistent after a publish, hence `eventually`. +/** Ciphertext of one message, read by a client that has no crypto at all. */ +const rawMessage = async (channel: string, timetoken: string): Promise => { + const raw = createClient('open', 'legacy-raw'); + const history = await eventually( + async () => { + const response = await raw.fetchMessages({ channels: [channel], count: 25 }); + const found = response.channels[channel]?.find((m) => String(m.timetoken) === timetoken); + assert.ok(found, 'message stored'); + return found; + }, + { timeout: 30_000, label: 'encrypted message stored' }, + ); + assert.equal( + typeof history.message, + 'string', + 'an encrypted message arrives as a string for a client without crypto', + ); + return history.message as string; +}; + +describe('api-reference/configuration-legacy', () => { + // Row `uuid`: still read when `userId` is missing, and both together are an error. + it('configurationLegacyUuid: the deprecated uuid option is still read when userId is missing, and passing both is an error', () => { + // "The SDK reads uuid when userId is not set." + const client = track(new PubNub({ ...keys(), uuid: 'legacy-uuid' } as unknown as PubNub.PubNubConfiguration)); + assert.equal(client.getUserId(), 'legacy-uuid'); + // "Setting both throws a PubNubError." + assert.throws( + () => new PubNub({ ...keys(), userId: 'a', uuid: 'b' } as unknown as PubNub.PubNubConfiguration), + (error: unknown) => (error as Error).name === 'PubNubError' && /use only 'userId'/.test((error as Error).message), + ); + }); + + // Row `logVerbosity`. The logger is injected, so no real log output and no timing is involved. + it('configurationLegacyLogVerbosity: logVerbosity true switches on debug logging and logs a deprecation warning, and logLevel Debug does it without the warning', () => { + // Debug level makes the console logger print the configuration, which holds the keys. Keep it out of the test output. + const quiet = quietConsole(); + try { + // "true sets logLevel to LogLevel.Debug", also when a different logLevel is passed, and logs a deprecation warning. + const { logger, seen } = recorder(); + const client = track( + new PubNub({ + ...keys(), + userId: 'legacy-verbosity', + logVerbosity: true, + logLevel: PubNub.LogLevel.Error, + loggers: [logger], + } as PubNub.PubNubConfiguration), + ); + assert.ok(client); + assert.ok( + seen.some((s) => s.level === 'warn' && /logVerbosity/.test(s.text) && /deprecated/.test(s.text)), + `deprecation warning logged. Saw: ${JSON.stringify(seen)}`, + ); + assert.ok( + seen.some((s) => s.level === 'debug'), + 'logVerbosity: true turns on debug logging although logLevel is Error', + ); + + // The replacement: logLevel Debug records debug entries without the deprecation warning. + const replacement = recorder(); + track( + new PubNub({ + ...keys(), + userId: 'legacy-verbosity-new', + logLevel: PubNub.LogLevel.Debug, + loggers: [replacement.logger], + }), + ); + assert.ok(replacement.seen.some((s) => s.level === 'debug')); + assert.ok(!replacement.seen.some((s) => /logVerbosity/.test(s.text))); + } finally { + quiet.restore(); + } + }); + + // Row `cipherKey`. Needs persistence because the cipher text is read back from history. The PNED header (base64 + // `UE5FRA`) tells the AES-CBC format apart from the legacy one. + it('configurationLegacyCipherKey: the deprecated cipherKey option encrypts messages in the legacy format, and the AES-CBC module writes the PNED header', async function () { + this.timeout(60_000); + const channel = unique('legacy_cipher'); + const cipherKey = 'legacy-cipher-key'; + const legacyClient = createClient('open', 'legacy-cipher', { cipherKey } as Partial); + const published = await legacyClient.publish({ channel, message: { secret: 'legacy' } }); + const cipherText = await rawMessage(channel, published.timetoken); + // "Defaults to legacy 128-bit encryption." The legacy format has no PNED header, the AES-CBC format has one. + assert.ok( + !cipherText.startsWith('UE5FRA'), + 'cipherKey without a crypto module produces the legacy format (no PNED header)', + ); + // "Pass the key to cryptoModule instead": the legacy crypto module reads it back, and so does the cipherKey client. + const viaModule = createClient('open', 'legacy-cipher-module', { + cryptoModule: PubNub.CryptoModule.legacyCryptoModule({ cipherKey }), + }); + const read = await viaModule.fetchMessages({ channels: [channel], count: 25 }); + const item = read.channels[channel]?.find((m) => String(m.timetoken) === published.timetoken); + assert.deepEqual(item?.message, { secret: 'legacy' }); + + // The AES-CBC module opt-in produces the PNED format. + const aes = createClient('open', 'legacy-aes', { + cryptoModule: PubNub.CryptoModule.aesCbcCryptoModule({ cipherKey }), + }); + const aesPublished = await aes.publish({ channel, message: { secret: 'aes' } }); + assert.ok( + (await rawMessage(channel, aesPublished.timetoken)).startsWith('UE5FRA'), + 'aesCbcCryptoModule writes the PNED header', + ); + }); + + // Row `useRandomIVs`. Equal cipher text for equal input is only possible with a fixed IV, which is what makes the + // comparison a proof and not a guess. + it('configurationLegacyUseRandomIVs: useRandomIVs false gives equal cipher text for equal input and true gives different cipher text', async function () { + this.timeout(60_000); + const channel = unique('legacy_ivs'); + const cipherKey = 'legacy-cipher-key'; + const send = async (client: PubNub): Promise => + rawMessage(channel, (await client.publish({ channel, message: { same: 'message' } })).timetoken); + + // Default (true): the initialization vector is random, so the same message encrypts differently each time. + const random = createClient('open', 'legacy-iv-random', { cipherKey } as Partial); + const [r1, r2] = [await send(random), await send(random)]; + assert.notEqual(r1, r2, 'random IVs make the cipher text differ'); + + // useRandomIVs: false: a fixed IV, so the cipher text repeats. + const fixed = createClient('open', 'legacy-iv-fixed', { + cipherKey, + useRandomIVs: false, + } as Partial); + const [f1, f2] = [await send(fixed), await send(fixed)]; + assert.equal(f1, f2, 'useRandomIVs: false repeats the cipher text'); + + // The replacement: useRandomIVs on legacyCryptoModule() behaves the same. + const replacement = createClient('open', 'legacy-iv-module', { + cryptoModule: PubNub.CryptoModule.legacyCryptoModule({ cipherKey, useRandomIVs: false }), + }); + const [m1, m2] = [await send(replacement), await send(replacement)]; + assert.equal(m1, m2); + assert.equal(m1, f1, 'the module option gives the same cipher text as the deprecated parameter'); + }); + + it('configurationLegacyStop: stop() unsubscribes the client and logs a deprecation warning', async () => { + const channel = unique('ch1'); + const { logger, seen } = recorder(); + // The same client both subscribes and is stopped, and it logs the warning. + const client = createClient('open', 'legacy-stop', { logLevel: PubNub.LogLevel.Warn, loggers: [logger] }); + const ready = connected(client, { channels: [channel] }); + const subscription = client.channel(channel).subscription(); + subscription.subscribe(); + await ready; + + // "stop() is an alias of destroy()": it invalidates subscriptions and unsubscribes from everything. + client.stop(); + await eventually(() => assert.deepEqual(client.getSubscribedChannels(), []), { label: 'stop unsubscribes' }); + + assert.ok( + seen.some((s) => s.level === 'warn' && /stop/.test(s.text) && /deprecated/.test(s.text)), + `stop() logs a deprecation warning. Saw: ${JSON.stringify(seen)}`, + ); + }); +}); diff --git a/test/integration/live/api-reference/configuration.test.ts b/test/integration/live/api-reference/configuration.test.ts new file mode 100644 index 000000000..169dbffe1 --- /dev/null +++ b/test/integration/live/api-reference/configuration.test.ts @@ -0,0 +1,234 @@ +import assert from 'node:assert/strict'; +import PubNub from 'pubnub'; +import { assertServerTime } from '../support/assertions'; +import { createClient, friend, shopper, track } from '../support/clients'; +import { quietConsole } from '../support/console'; +import { keysOf, openKeys, securedKeys, subscribeKeyOnly } from '../support/env'; +import { eventually, watch } from '../support/wait'; + +/** + * Configuration examples. Port of docs-snippets/api-reference/configuration.ts. + * The samples show placeholder keys. The hidden spread after them puts the real keys of the keyset in, so the text readers see + * stays generic. The spread is typed Partial, otherwise TypeScript reports the overwritten visible keys (TS2783). + */ + +describe('api-reference/configuration', () => { + it('configurationBasicUsage: the full constructor sample yields a working client with its user id, auth key, crypto module and Debug logging, and time() succeeds', async () => { + // Debug level makes the SDK print its configuration, which holds the keys. Keep it out of the test output. + const debugLog = quietConsole(); + // snippet.configurationBasicUsage + // Initialize PubNub with your keys + const pubnubConfig = new PubNub({ + subscribeKey: 'YOUR_SUBSCRIBE_KEY', + publishKey: 'YOUR_PUBLISH_KEY', + userId: 'YOUR_USER_ID', + cryptoModule: PubNub.CryptoModule?.aesCbcCryptoModule({ cipherKey: 'YOUR_CIPHER_KEY' }), + authKey: 'accessMangerToken', + logLevel: PubNub.LogLevel.Debug, + ssl: true, + presenceTimeout: 130, + // snippet.hide + ...openKeys(), + // snippet.show + }); + + // snippet.end + track(pubnubConfig); + try { + assert.equal(pubnubConfig.getUserId(), 'YOUR_USER_ID'); + assert.equal(pubnubConfig.getAuthKey(), 'accessMangerToken'); + // cryptoModule is active: encrypt and decrypt round trip. encrypt() with a crypto module takes a string and decrypt() + // JSON-parses the plain text, so pass stringified JSON. + assert.deepEqual(pubnubConfig.decrypt(pubnubConfig.encrypt(JSON.stringify({ greeting: 'hello' }))), { + greeting: 'hello', + }); + assertServerTime((await pubnubConfig.time()).timetoken); + } finally { + debugLog.restore(); + } + assert.ok(debugLog.lines.length > 0, 'the Debug logger wrote no output, so logLevel: Debug did not take effect'); + }); + + it('configurationServerOpertaion: the server configuration sample builds a client with the secret key and its user id, and time() succeeds', async () => { + // snippet.configurationServerOpertaion + const pubnubServer = new PubNub({ + subscribeKey: 'mySubscribeKey', + publishKey: 'myPublishKey', + userId: 'myUniqueUserId', + secretKey: 'secretKey', + heartbeatInterval: 0, + // snippet.hide + ...securedKeys(), + // snippet.show + }); + // snippet.end + track(pubnubServer); + assert.equal(pubnubServer.getUserId(), 'myUniqueUserId'); + assertServerTime((await pubnubServer.time()).timetoken); + }); + + // The id keeps the docs' spelling ("Real"). A subscribe-only client has no publishKey, so the SDK refuses to publish + // locally. The `time()` call shows the subscribe key itself is valid. + it('configurationRealOnlyClient: a client with only a subscribe key works for time() and refuses to publish locally', async () => { + // snippet.configurationRealOnlyClient + // Initialize for Read Only Client + + const pubnubReadOnly = new PubNub({ + subscribeKey: 'mySubscribeKey', + userId: 'myUniqueUserId', + // snippet.hide + ...subscribeKeyOnly(), + // snippet.show + }); + // snippet.end + track(pubnubReadOnly); + // Read-only means no publishKey: publishing is refused before any request is sent. + await assert.rejects( + pubnubReadOnly.publish({ channel: 'my_channel', message: { text: 'x' } }), + (error: unknown) => { + assert.match(JSON.stringify((error as { status?: unknown }).status), /publishKey/); + return true; + }, + ); + assertServerTime((await pubnubReadOnly.time()).timetoken); + }); + + it("configurationCryptoModule: two clients with different crypto modules decrypt each other's data as the sample comments claim", () => { + // snippet.configurationCryptoModule + // encrypts using 256-bit AES-CBC cipher (recommended) + // decrypts data encrypted with the legacy and the 256 bit AES-CBC ciphers + const pubnub = new PubNub({ + subscribeKey: 'YOUR_SUBSCRIBE_KEY', + userId: 'YOUR_USER_ID', + cryptoModule: PubNub.CryptoModule.aesCbcCryptoModule({ cipherKey: 'pubnubenigma' }), + // snippet.hide + ...subscribeKeyOnly(), + // snippet.show + }); + + // encrypts with 128-bit cipher key entropy (legacy) + // decrypts data encrypted with the legacy and the 256-bit AES-CBC ciphers + const pubnubLegacy = new PubNub({ + subscribeKey: 'YOUR_SUBSCRIBE_KEY', + userId: 'YOUR_USER_ID', + cryptoModule: PubNub.CryptoModule.legacyCryptoModule({ cipherKey: 'pubnubenigma' }), + // snippet.hide + ...subscribeKeyOnly(), + // snippet.show + }); + // snippet.end + track(pubnub); + track(pubnubLegacy); + const payload = JSON.stringify({ note: 'same input' }); + assert.deepEqual(pubnub.decrypt(pubnubLegacy.encrypt(payload)), { note: 'same input' }); + assert.deepEqual(pubnubLegacy.decrypt(pubnub.encrypt(payload)), { note: 'same input' }); + // The two modules produce different ciphertext (random IVs, different headers), so the choice is applied. + assert.notEqual(pubnub.encrypt(payload), pubnubLegacy.encrypt(payload)); + }); + + it('setUserIdBasicUsage: setUserId renames the user and the client keeps working', async () => { + const pubnub = shopper(); + // snippet.setUserIdBasicUsage + pubnub.setUserId('myUniqueUserId'); + // snippet.end + assert.equal(pubnub.getUserId(), 'myUniqueUserId'); + assertServerTime((await pubnub.time()).timetoken); + }); + + it('getUserIdBasicUsage: getUserId returns the user id given at construction', () => { + const pubnub = shopper(); + // snippet.getUserIdBasicUsage + const currentUserId = pubnub.getUserId(); + // snippet.end + assert.equal(currentUserId, 'shopper'); + assert.equal(currentUserId, pubnub.userId); + }); + + // Local state only: the auth key matters when Access Manager is on, which the access manager pages cover. + it('setAuthKeyBasicUsage: setAuthKey stores the auth key on the client', () => { + const pubnub = shopper(); + // snippet.setAuthKeyBasicUsage + pubnub.setAuthKey('my_authkey'); + // snippet.end + assert.equal(pubnub.getAuthKey(), 'my_authkey'); + }); + + it('getAuthKeyBasicUsage: getAuthKey returns the auth key that was set', () => { + const pubnub = shopper(); + pubnub.setAuthKey('my_authkey'); + // snippet.getAuthKeyBasicUsage + const currentAuthKey = pubnub.getAuthKey(); + // snippet.end + assert.equal(currentAuthKey, 'my_authkey'); + }); + + // Local check of the stored expression. The wire effect is proven by the test `filterExpressionDropsMessage` below. + // The older docs literal `such=wow` is not valid filter syntax: a live probe (2026-11-03) showed the server delivers + // no message at all for it, while `such == 'wow'` delivers only the matching one. + it('setFilterExpressionBasicUsage: setFilterExpression stores the filter expression on the client', () => { + const pubnub = shopper(); + // snippet.setFilterExpressionBasicUsage + pubnub.setFilterExpression("such == 'wow'"); + // snippet.end + assert.equal(pubnub.getFilterExpression(), "such == 'wow'"); + }); + + it('getFilterExpressionBasicUsage: getFilterExpression returns the expression that was set', () => { + const pubnub = shopper(); + pubnub.setFilterExpression("such == 'wow'"); + // Record what the sample's call returns. + const results: Array = []; + const original = pubnub.getFilterExpression.bind(pubnub); + pubnub.getFilterExpression = () => { + const result = original(); + results.push(result); + return result; + }; + // snippet.getFilterExpressionBasicUsage + pubnub.getFilterExpression(); + // snippet.end + assert.deepEqual(results, ["such == 'wow'"]); + }); + + it('destroyBasicUsage: destroy() stops a subscribed client and clears its subscriptions', async () => { + const pubnub = shopper(); + await watch(pubnub, ['my_channel']); + assert.ok(pubnub.getSubscribedChannels().includes('my_channel'), 'subscribed before destroy'); + // snippet.destroyBasicUsage + pubnub.destroy(); + // snippet.end + await eventually(() => assert.deepEqual(pubnub.getSubscribedChannels(), []), { label: 'destroy unsubscribes' }); + }); + + it('configurationInitializationErrors: each documented constructor error is thrown for the matching bad configuration', () => { + const base = { subscribeKey: keysOf('open').subscribeKey }; + assert.throws(() => new PubNub({ ...base } as unknown as PubNub.PubNubConfiguration), /'userId' not set/); + assert.throws(() => new PubNub({ ...base, userId: ' ' }), /'userId' is empty/); + assert.throws( + () => new PubNub({ ...base, userId: 'a', uuid: 'b' } as unknown as PubNub.PubNubConfiguration), + /use only 'userId'/, + ); + assert.throws(() => new PubNub({ ...base, userId: '' }), /'userId' not set/); + }); + + it('filterExpressionDropsMessage: a subscribe filter expression makes the server drop messages that do not match, observed on the wire', async function () { + this.timeout(45_000); + // A unique channel of this run: the test publishes its own messages and counts what arrives, so it must not share a channel. + const channel = `filter_channel-${Date.now().toString(36)}${Math.random().toString(36).slice(2, 6)}`; + const subscriber = createClient('open', 'filtered'); + subscriber.setFilterExpression("such == 'wow'"); + const listening = await watch(subscriber, [channel]); + const publisher = friend(); + await publisher.publish({ channel, message: { n: 1 }, meta: { such: 'nope' } }); + await publisher.publish({ channel, message: { n: 2 }, meta: { such: 'wow' } }); + // The publishes are ordered, so once message 2 has arrived message 1 would have arrived before it, had it passed. + const delivered = await listening.next('message', (m) => (m.message as { n?: number }).n === 2); + assert.equal((delivered.userMetadata as { such?: string }).such, 'wow'); + assert.equal( + listening.events.filter((e) => e.kind === 'message').length, + 1, + 'the message with meta such=nope must be dropped by the filter', + ); + listening.stop(); + }); +}); diff --git a/test/integration/live/api-reference/datasync.test.ts b/test/integration/live/api-reference/datasync.test.ts new file mode 100644 index 000000000..d285e686f --- /dev/null +++ b/test/integration/live/api-reference/datasync.test.ts @@ -0,0 +1,1397 @@ +import assert from 'node:assert/strict'; +import type PubNub from 'pubnub'; +import { server } from '../support/clients'; +import { captureConsole } from '../support/console'; +import { filterFragments, patchFragments, sortFragments } from '../support/datasync-fragments'; +import { errorText, removePageObjects } from '../support/datasync-state'; +import { connected, eventually } from '../support/wait'; + +/** + * DataSync examples, including the DataSync listener and subscription examples. Port of docs-snippets/api-reference/datasync.ts. + * + * Runs on the `datasync` keyset. The samples create DataSync objects with fixed literal ids, and the service answers a create of + * an existing id with 409 (DS-0301), it does not overwrite. So a before hook removes every literal object the page creates + * (support/datasync-state.ts), and an after hook does the same, best effort. The classes `product` (v1) and `ProductOwner` + * (v1) are provisioned separately and must exist on the keyset. The Admin Portal event rules for User, Channel and product + * must be on for the event tests. + */ +const ALICE = 'user-alice'; +const BOB = 'seller-bob'; +const SUMMER = 'channel-summer-sale'; +const WINTER = 'channel-winter-sale'; +const MEMBERSHIP = 'membership-alice-summer-sale'; +const WINTER_MEMBERSHIP = 'membership-alice-winter-sale'; +const PRODUCT = 'product-sneaker-42'; +const REL = 'rel-bob-owns-sneaker-42'; + +const ids = (response: { data: Array<{ id: string }> }): string[] => response.data.map((o) => o.id); + +/** Runs `run` with console.log captured, and hands it the captured lines. */ +async function withLogs(run: (lines: string[]) => Promise): Promise { + const logs = captureConsole('log'); + try { + await run(logs.lines); + } finally { + logs.restore(); + } +} + +/** + * The samples show an example eTag in `ifMatchesEtag`. Left as it is, the guard (correctly) rejects the write, so the request + * carries the real eTag instead and the sample text stays exactly what readers see. + */ +function useRealEtag(api: PubNub['dataSync'], method: 'setUser' | 'updateEntity', example: string, real: string): void { + const target = api as unknown as Record Promise>; + const original = target[method].bind(api); + target[method] = (parameters) => + original(parameters.ifMatchesEtag === example ? { ...parameters, ifMatchesEtag: real } : parameters); +} + +/** Creates the channel `channel-winter-sale` and Alice's membership on it with `status: 'archived'`. Removing the channel removes the membership. */ +async function seedArchivedMembership(pubnub: PubNub): Promise { + await pubnub.dataSync.createChannel({ id: WINTER, data: { classVersion: 1, payload: { name: 'Winter Sale 2026' } } }); + await pubnub.dataSync.createMembership({ + id: WINTER_MEMBERSHIP, + userId: ALICE, + channelId: WINTER, + data: { classVersion: 1, status: 'archived' }, + }); +} + +describe('api-reference/datasync', () => { + // Reset before and best effort after: the literal objects persist on the keyset across runs. + const reset = async (): Promise => { + const pubnub = server('datasync'); + try { + await removePageObjects(pubnub); + } finally { + pubnub.destroy(); + } + }; + before(async function () { + this.timeout(60_000); + await reset(); + }); + after(async function () { + this.timeout(60_000); + await reset().catch(() => undefined); + }); + + describe('preconditions', () => { + // The classes are provisioned separately (docs-snippets/provision/). This only reports a missing one. + it('the product entity class exists on the keyset', async () => { + const klass = await server('datasync').dataSync.getEntities({ class: 'product', classVersion: 1, limit: 1 }); + assert.equal(klass.status, 200); + }); + + it('the ProductOwner relationship class exists on the keyset', async () => { + const links = await server('datasync').dataSync.getRelationships({ class: 'ProductOwner', limit: 1 }); + assert.equal(links.status, 200); + }); + }); + + // The describe blocks below run in order and build on each other (create, read, update, patch of the same objects, then the + // removal block at the end). The before hook above resets every literal object, so the sequence starts from a known state. + + describe('users', () => { + it('createUserBasicUsage: createUser creates Alice, echoes the payload and returns an eTag', async () => { + const pubnub = server('datasync'); + // snippet.createUserBasicUsage + const response = await pubnub.dataSync.createUser({ + id: 'user-alice', + data: { + classVersion: 1, + payload: { name: 'Alice', type: 'shopper' }, + }, + }); + console.log('createUser response:', response.data); + // snippet.end + assert.ok(response.status >= 200 && response.status < 300, `status ${response.status}`); + assert.equal(response.data.id, ALICE); + assert.equal(response.data.entityClassVersion, 1); + assert.deepEqual(response.data.payload, { name: 'Alice', type: 'shopper' }); + assert.ok(response.data.eTag, 'response carries an eTag'); + }); + + it('getUserBasicUsage: getUser reads Alice back by id', async () => { + const pubnub = server('datasync'); + // snippet.getUserBasicUsage + const response = await pubnub.dataSync.getUser({ id: 'user-alice' }); + console.log('getUser response:', response.data); + // snippet.end + assert.equal(response.data.id, ALICE); + assert.equal(response.data.payload?.name, 'Alice'); + }); + + it('getUsersBasicUsage: getUsers lists users sorted by createdAt, verified on the returned timestamps', async () => { + const pubnub = server('datasync'); + // snippet.getUsersBasicUsage + const response = await pubnub.dataSync.getUsers({ + limit: 20, + sort: { createdAt: 'asc' }, + }); + console.log('getUsers response:', response.data); + console.log('getUsers meta:', response.meta); + // snippet.end + assert.ok(Array.isArray(response.data)); + assert.ok(response.data.length <= 20); + assert.equal(typeof response.meta?.has_next, 'boolean'); + const created = response.data.map((u) => Date.parse(u.createdAt)); + assert.deepEqual( + created, + [...created].sort((a, b) => a - b), + 'sorted by createdAt ascending', + ); + }); + + it('getUsersFilterExample: getUsers with filter finds Alice once the eventually consistent index has caught up', async () => { + const pubnub = server('datasync'); + // `filter` is eventually consistent, so wait until the service indexed the object before running the sample. + await eventually( + async () => { + assert.ok(ids(await pubnub.dataSync.getUsers({ filter: 'name LIKE "*Alice*"' })).includes(ALICE)); + }, + { timeout: 45_000, label: 'filter index caught up with the new user' }, + ); + // snippet.getUsersFilterExample + // `filter` runs over properties declared with filtering mode `full` and is evaluated against + // eventually consistent storage. + const response = await pubnub.dataSync.getUsers({ + filter: 'name LIKE "*Alice*"', + }); + console.log('getUsers (filter) response:', response.data); + // snippet.end + assert.ok(ids(response).includes(ALICE), 'the filter finds Alice'); + }).timeout(60000); + + it('getUsersFilterFastExample: getUsers with filterFast finds Alice at once (strongly consistent)', async () => { + const pubnub = server('datasync'); + // snippet.getUsersFilterFastExample + // `filterFast` is evaluated against strongly consistent storage, so it matches an object you just + // wrote. + const response = await pubnub.dataSync.getUsers({ + filterFast: 'type == "shopper"', + }); + console.log('getUsers (filterFast) response:', response.data); + // snippet.end + assert.ok(ids(response).includes(ALICE), 'filterFast sees the user right after the write'); + }); + + it('getUsersCursorPagingExample: the cursor paging loop for users runs and lists Alice', async () => { + const pubnub = server('datasync'); + await withLogs(async (lines) => { + // snippet.getUsersCursorPagingExample + // Pass no `cursor` on the first call. Take `meta.next_cursor` from the response and pass it back + // as `cursor` on the next call. Stop when `meta.has_next` is `false`. + let cursor: string | undefined; + let hasNext = true; + let page = 0; + + while (hasNext) { + const response = await pubnub.dataSync.getUsers({ + filterFast: 'type == "shopper"', + limit: 20, + cursor, + }); + console.log(`Page ${++page}:`, response.data); + cursor = response.meta?.next_cursor ?? undefined; + hasNext = response.meta?.has_next ?? false; + } + // snippet.end + assert.ok( + lines.some((line) => line.startsWith('Page 1:')), + 'the loop ran at least once', + ); + assert.ok( + lines.some((line) => line.includes(ALICE)), + 'a page lists Alice', + ); + }); + }); + + it('updateUserBasicUsage: setUser with ifMatchesEtag replaces Alice, and the eTag it was guarded by is stale afterwards', async () => { + const pubnub = server('datasync'); + const before = (await pubnub.dataSync.getUser({ id: ALICE })).data.eTag; + // The sample shows an example eTag. The example itself is refused by the guard. + await assert.rejects( + () => + pubnub.dataSync.setUser({ + id: ALICE, + data: { classVersion: 1, payload: { name: 'x' } }, + ifMatchesEtag: 'AbQdEfGhIjKlMn', + }), + (error: unknown) => /\b(412|403)\b/.test(errorText(error)), + 'the example eTag is rejected', + ); + // The sample text keeps the example eTag. The request carries the real one, so the guarded write can succeed. + useRealEtag(pubnub.dataSync, 'setUser', 'AbQdEfGhIjKlMn', before); + // snippet.updateUserBasicUsage + const response = await pubnub.dataSync.setUser({ + id: 'user-alice', + data: { + classVersion: 1, + payload: { name: 'Alice B.', type: 'shopper' }, + }, + ifMatchesEtag: 'AbQdEfGhIjKlMn', + }); + console.log('setUser response:', response.data); + // snippet.end + assert.equal(response.data.payload?.name, 'Alice B.'); + assert.notEqual(response.data.eTag, before, 'the write produced a new eTag'); + await assert.rejects( + () => + pubnub.dataSync.setUser({ + id: ALICE, + data: { classVersion: 1, payload: { name: 'stale' } }, + ifMatchesEtag: before, + }), + (error: unknown) => /\b(412|403)\b/.test(errorText(error)), + 'the old eTag is now stale', + ); + }); + + it('patchUserBasicUsage: updateUser applies a patch to the user name and status', async () => { + const pubnub = server('datasync'); + // snippet.patchUserBasicUsage + const response = await pubnub.dataSync.updateUser({ + id: 'user-alice', + replace: { '/payload/name': 'Alice B.', '/status': 'active' }, + }); + console.log('updateUser response:', response.data); + // snippet.end + assert.equal(response.data.payload?.name, 'Alice B.'); + assert.equal(response.data.status, 'active'); + }); + }); + + describe('channels', () => { + it('createDataSyncChannelBasicUsage: createChannel creates the drop channel and echoes the payload', async () => { + const pubnub = server('datasync'); + // snippet.createDataSyncChannelBasicUsage + const response = await pubnub.dataSync.createChannel({ + id: 'channel-summer-sale', + data: { + classVersion: 1, + payload: { name: 'Summer Sale', type: 'promotion' }, + }, + }); + console.log('createChannel response:', response.data); + // snippet.end + assert.equal(response.data.id, SUMMER); + assert.deepEqual(response.data.payload, { name: 'Summer Sale', type: 'promotion' }); + }); + + it('getChannelBasicUsage: getChannel reads the DataSync channel back by id', async () => { + const pubnub = server('datasync'); + // snippet.getChannelBasicUsage + const response = await pubnub.dataSync.getChannel({ id: 'channel-summer-sale' }); + console.log('getChannel response:', response.data); + // snippet.end + assert.equal(response.data.id, SUMMER); + assert.equal(response.data.payload?.name, 'Summer Sale'); + }); + + it('getChannelsBasicUsage: getChannels lists channels with the data array and meta.has_next envelope', async () => { + const pubnub = server('datasync'); + // snippet.getChannelsBasicUsage + const response = await pubnub.dataSync.getChannels({ limit: 20 }); + console.log('getChannels response:', response.data); + console.log('getChannels meta:', response.meta); + // snippet.end + assert.ok(Array.isArray(response.data)); + assert.equal(typeof response.meta?.has_next, 'boolean'); + }); + + it('getChannelsFilterExample: getChannels with filter finds the new channel once the index has caught up', async () => { + const pubnub = server('datasync'); + // `filter` is eventually consistent, so wait until the service indexed the object before running the sample. + await eventually( + async () => { + assert.ok(ids(await pubnub.dataSync.getChannels({ filter: 'name LIKE "*Sale*"' })).includes(SUMMER)); + }, + { timeout: 45_000, label: 'filter index caught up with the new channel' }, + ); + // snippet.getChannelsFilterExample + const response = await pubnub.dataSync.getChannels({ + filter: 'name LIKE "*Sale*"', + }); + console.log('getChannels (filter) response:', response.data); + // snippet.end + assert.ok(ids(response).includes(SUMMER)); + }).timeout(60000); + + it('getChannelsFilterFastExample: getChannels with filterFast finds the channel at once', async () => { + const pubnub = server('datasync'); + // snippet.getChannelsFilterFastExample + const response = await pubnub.dataSync.getChannels({ + filterFast: 'type == "promotion"', + }); + console.log('getChannels (filterFast) response:', response.data); + // snippet.end + assert.ok(ids(response).includes(SUMMER)); + }); + + it('getChannelsCursorPagingExample: the cursor paging loop for channels runs and lists the channel', async () => { + const pubnub = server('datasync'); + await withLogs(async (lines) => { + // snippet.getChannelsCursorPagingExample + let cursor: string | undefined; + let hasNext = true; + let page = 0; + + while (hasNext) { + const response = await pubnub.dataSync.getChannels({ + filterFast: 'type == "promotion"', + limit: 20, + cursor, + }); + console.log(`Page ${++page}:`, response.data); + cursor = response.meta?.next_cursor ?? undefined; + hasNext = response.meta?.has_next ?? false; + } + // snippet.end + assert.ok( + lines.some((line) => line.includes(SUMMER)), + 'a page lists the channel', + ); + }); + }); + + it('updateChannelBasicUsage: setChannel replaces the channel payload', async () => { + const pubnub = server('datasync'); + // snippet.updateChannelBasicUsage + const response = await pubnub.dataSync.setChannel({ + id: 'channel-summer-sale', + data: { + classVersion: 1, + payload: { name: 'Summer Sale 2026', type: 'promotion' }, + }, + }); + console.log('setChannel response:', response.data); + // snippet.end + assert.equal(response.data.payload?.name, 'Summer Sale 2026'); + }); + + it('patchChannelBasicUsage: updateChannel patches one field and the other payload fields survive', async () => { + const pubnub = server('datasync'); + // snippet.patchChannelBasicUsage + const response = await pubnub.dataSync.updateChannel({ + id: 'channel-summer-sale', + replace: { '/payload/name': 'Summer Sale 2026' }, + }); + console.log('updateChannel response:', response.data); + // snippet.end + assert.equal(response.data.payload?.name, 'Summer Sale 2026'); + assert.equal(response.data.payload?.type, 'promotion', 'the patch leaves other fields alone'); + }); + + // Not a sample: paging one item at a time returns next_cursor and has_next correctly and never repeats an item. + it('cursor paging by one item returns next_cursor and has_next correctly and never repeats an item', async () => { + const api = server('datasync').dataSync; + const seen: string[] = []; + let cursor: string | undefined; + let pages = 0; + for (;;) { + const response = await api.getChannels({ filterFast: 'type == "promotion"', limit: 1, cursor }); + pages += 1; + seen.push(...ids(response)); + cursor = response.meta?.next_cursor ?? undefined; + if (!response.meta?.has_next) break; + assert.ok(cursor, 'has_next true comes with a next_cursor'); + assert.ok(pages < 50, 'paging terminates'); + } + assert.ok(seen.includes(SUMMER), 'paging by one still reaches the channel'); + assert.equal(new Set(seen).size, seen.length, 'no object appears twice'); + }); + }); + + describe('memberships', () => { + it('createMembershipBasicUsage: createMembership links Alice to the channel with the fixed id that later tests use', async () => { + const pubnub = server('datasync'); + // snippet.createMembershipBasicUsage + const response = await pubnub.dataSync.createMembership({ + id: 'membership-alice-summer-sale', + userId: 'user-alice', + channelId: 'channel-summer-sale', + data: { + classVersion: 1, + payload: { role: 'viewer' }, + }, + }); + console.log('createMembership response:', response.data); + // snippet.end + assert.equal(response.data.id, MEMBERSHIP); + assert.equal(response.data.userId, ALICE); + assert.equal(response.data.channelId, SUMMER); + assert.equal(response.data.relationshipClass, 'Membership'); + assert.deepEqual(response.data.payload, { role: 'viewer' }); + }); + + it('getDataSyncMembershipBasicUsage: getMembership reads the membership by its fixed id', async () => { + const pubnub = server('datasync'); + // snippet.getDataSyncMembershipBasicUsage + const response = await pubnub.dataSync.getMembership({ + id: 'membership-alice-summer-sale', + }); + console.log('getMembership response:', response.data); + // snippet.end + assert.equal(response.data.id, MEMBERSHIP); + assert.equal(response.data.payload?.role, 'viewer'); + }); + + it('getMembershipsBasicUsage: getMemberships lists exactly the membership of Alice', async () => { + const pubnub = server('datasync'); + // snippet.getMembershipsBasicUsage + const response = await pubnub.dataSync.getMemberships({ + userId: 'user-alice', + limit: 20, + }); + console.log('getMemberships response:', response.data); + console.log('getMemberships meta:', response.meta); + // snippet.end + // Exact list: legitimate because the before hook removed Alice (and with her every membership) before this run created her. + assert.deepEqual(ids(response), [MEMBERSHIP]); + assert.equal(response.data[0].channelId, SUMMER); + }); + + it('getMembershipsByChannelIdExample: getMemberships by channel id lists Alice as the only member', async () => { + const pubnub = server('datasync'); + // snippet.getMembershipsByChannelIdExample + // Pass `channelId` instead of `userId` to list the members of a channel rather than a user's + // memberships. + const response = await pubnub.dataSync.getMemberships({ + channelId: 'channel-summer-sale', + limit: 20, + }); + console.log('getMemberships (channelId) response:', response.data); + // snippet.end + // Exact list: the before hook removed the channel (and its memberships) before this run created it. + assert.deepEqual( + response.data.map((m) => m.userId), + [ALICE], + ); + }); + + // Filters on membership fields. The Global Membership class declares no payload properties and cannot be extended, so a + // membership payload is never indexed and filters cannot reach it. The built-in fields (id, createdAt, updatedAt, status, + // createdBy, modifiedBy) are filterable. Both samples need a second membership of Alice that matches (status "archived"), + // so the helper adds one on a second channel and removes the channel (and with it the membership) afterwards. + + it('getMembershipsFilterExample: getMemberships with filter excludes a seeded archived membership once the index has caught up', async () => { + const pubnub = server('datasync'); + await seedArchivedMembership(pubnub); + try { + // `filter` is eventually consistent, so wait until the service indexed the new membership. + await eventually( + async () => { + const found = await pubnub.dataSync.getMemberships({ userId: ALICE, filter: 'status LIKE "arch*"' }); + assert.ok(ids(found).includes(WINTER_MEMBERSHIP)); + }, + { timeout: 45_000, label: 'filter index caught up with the new membership' }, + ); + // snippet.getMembershipsFilterExample + // `status` is a built-in field, so it is filterable without any declaration. The Membership class + // declares no `payload` properties, which means a filter cannot reference anything inside `payload`. + // `LIKE` takes `*` as a wildcard. This call returns Alice's memberships whose status starts with `arch`. + const response = await pubnub.dataSync.getMemberships({ + userId: 'user-alice', + filter: 'status LIKE "arch*"', + }); + console.log('getMemberships (filter) response:', response.data); + // snippet.end + // Exact list: legitimate because the before hook reset Alice's memberships and this test seeded the only archived one. + assert.deepEqual(ids(response), [WINTER_MEMBERSHIP], 'only the archived membership matches'); + assert.ok(!ids(response).includes(MEMBERSHIP), 'the membership without a status is filtered out'); + const unfiltered = await pubnub.dataSync.getMemberships({ userId: ALICE }); + assert.ok( + ids(unfiltered).includes(WINTER_MEMBERSHIP) && ids(unfiltered).includes(MEMBERSHIP), + 'without the filter both memberships are listed', + ); + } finally { + await pubnub.dataSync.removeChannel({ id: WINTER }).catch(() => undefined); + } + }).timeout(60000); + + it('getMembershipsFilterFastExample: getMemberships with filterFast excludes the seeded archived membership at once', async () => { + const pubnub = server('datasync'); + await seedArchivedMembership(pubnub); + try { + // snippet.getMembershipsFilterFastExample + // `filterFast` is evaluated against strongly consistent storage, so it matches a membership you just + // wrote. It accepts the same built-in fields. This call returns Alice's archived memberships. + const response = await pubnub.dataSync.getMemberships({ + userId: 'user-alice', + filterFast: 'status == "archived"', + }); + console.log('getMemberships (filterFast) response:', response.data); + // snippet.end + assert.deepEqual( + ids(response), + [WINTER_MEMBERSHIP], + 'only the archived membership matches, right after the write', + ); + assert.equal(response.data[0].status, 'archived'); + assert.ok(!ids(response).includes(MEMBERSHIP), 'the membership without a status is filtered out'); + } finally { + await pubnub.dataSync.removeChannel({ id: WINTER }).catch(() => undefined); + } + }); + + // Not a sample: documents the limit the two samples above work around. A field of the membership payload is not filterable. + it('filtering on a field of the membership payload is refused by the service', async () => { + const api = server('datasync').dataSync; + for (const parameters of [{ filter: 'role LIKE "*mod*"' }, { filterFast: 'role == "viewer"' }]) { + await assert.rejects( + () => api.getMemberships({ userId: ALICE, ...parameters }), + (error: unknown) => + /400/.test(errorText(error)) && + /DS-1001/.test(errorText(error)) && + /Unknown field: role/.test(errorText(error)), + `the service rejects ${JSON.stringify(parameters)}`, + ); + } + }); + + it('getMembershipsCursorPagingExample: the cursor paging loop for memberships runs and lists the membership', async () => { + const pubnub = server('datasync'); + await withLogs(async (lines) => { + // snippet.getMembershipsCursorPagingExample + let cursor: string | undefined; + let hasNext = true; + let page = 0; + + while (hasNext) { + const response = await pubnub.dataSync.getMemberships({ + userId: 'user-alice', + limit: 20, + cursor, + }); + console.log(`Page ${++page}:`, response.data); + cursor = response.meta?.next_cursor ?? undefined; + hasNext = response.meta?.has_next ?? false; + } + // snippet.end + assert.ok( + lines.some((line) => line.includes(MEMBERSHIP)), + 'a page lists the membership', + ); + }); + }); + + it('updateMembershipBasicUsage: setMembership replaces the membership payload', async () => { + const pubnub = server('datasync'); + // snippet.updateMembershipBasicUsage + const response = await pubnub.dataSync.setMembership({ + id: 'membership-alice-summer-sale', + userId: 'user-alice', + channelId: 'channel-summer-sale', + data: { + classVersion: 1, + payload: { role: 'moderator' }, + }, + }); + console.log('setMembership response:', response.data); + // snippet.end + assert.equal(response.data.payload?.role, 'moderator'); + }); + + it('patchMembershipBasicUsage: updateMembership patches one payload field', async () => { + const pubnub = server('datasync'); + // snippet.patchMembershipBasicUsage + const response = await pubnub.dataSync.updateMembership({ + id: 'membership-alice-summer-sale', + replace: { '/payload/role': 'moderator' }, + }); + console.log('updateMembership response:', response.data); + // snippet.end + assert.equal(response.data.payload?.role, 'moderator'); + }); + }); + + describe('entities (class product)', () => { + it('createEntityBasicUsage: createEntity creates the product entity that later tests use', async () => { + const pubnub = server('datasync'); + // snippet.createEntityBasicUsage + const response = await pubnub.dataSync.createEntity({ + id: 'product-sneaker-42', + class: 'product', + data: { + classVersion: 1, + payload: { name: 'Retro Sneaker', price: 89.99, stock: 12 }, + }, + }); + console.log('createEntity response:', response.data); + // snippet.end + assert.equal(response.data.id, PRODUCT); + assert.equal(response.data.entityClass, 'product'); + assert.deepEqual(response.data.payload, { name: 'Retro Sneaker', price: 89.99, stock: 12 }); + }); + + it('getEntityBasicUsage: getEntity reads the product by id', async () => { + const pubnub = server('datasync'); + // snippet.getEntityBasicUsage + const response = await pubnub.dataSync.getEntity({ id: 'product-sneaker-42' }); + console.log('getEntity response:', response.data); + // snippet.end + assert.equal(response.data.id, PRODUCT); + assert.equal(response.data.payload?.price, 89.99); + }); + + it('getEntitiesBasicUsage: getEntities lists products sorted by price descending, verified on the returned prices', async () => { + const pubnub = server('datasync'); + // snippet.getEntitiesBasicUsage + const response = await pubnub.dataSync.getEntities({ + class: 'product', + sort: { price: 'desc' }, + limit: 20, + }); + console.log('getEntities response:', response.data); + console.log('getEntities meta:', response.meta); + // snippet.end + assert.ok(ids(response).includes(PRODUCT)); + const prices = response.data.map((e) => Number(e.payload?.price)); + assert.deepEqual( + prices, + [...prices].sort((a, b) => b - a), + 'sorted by price descending', + ); + }); + + it('getEntitiesFilterExample: getEntities with filter finds the product once the index has caught up', async () => { + const pubnub = server('datasync'); + // `filter` is eventually consistent, so wait until the service indexed the object before running the sample. + await eventually( + async () => { + assert.ok( + ids( + await pubnub.dataSync.getEntities({ + class: 'product', + filter: 'name LIKE "*sneaker*" && !(status == "discontinued")', + }), + ).includes(PRODUCT), + ); + }, + { timeout: 45_000, label: 'filter index caught up with the new product' }, + ); + // snippet.getEntitiesFilterExample + // Reference a declared property by its `name` (not its `path`), and combine conditions with `&&` + // and `||`. + const response = await pubnub.dataSync.getEntities({ + class: 'product', + filter: 'name LIKE "*sneaker*" && !(status == "discontinued")', + }); + console.log('getEntities (filter) response:', response.data); + // snippet.end + assert.ok(ids(response).includes(PRODUCT)); + }).timeout(60000); + + it('getEntitiesFilterFastExample: getEntities with filterFast finds the product at once', async () => { + const pubnub = server('datasync'); + // snippet.getEntitiesFilterFastExample + const response = await pubnub.dataSync.getEntities({ + class: 'product', + filterFast: 'price < 100 && stock > 0', + }); + console.log('getEntities (filterFast) response:', response.data); + // snippet.end + assert.ok(ids(response).includes(PRODUCT)); + }); + + it('getEntitiesCursorPagingExample: the cursor paging loop for products runs and lists the product', async () => { + const pubnub = server('datasync'); + await withLogs(async (lines) => { + // snippet.getEntitiesCursorPagingExample + let cursor: string | undefined; + let hasNext = true; + let page = 0; + + while (hasNext) { + const response = await pubnub.dataSync.getEntities({ + class: 'product', + filterFast: 'price < 100', + limit: 20, + cursor, + }); + console.log(`Page ${++page}:`, response.data); + cursor = response.meta?.next_cursor ?? undefined; + hasNext = response.meta?.has_next ?? false; + } + // snippet.end + assert.ok( + lines.some((line) => line.includes(PRODUCT)), + 'a page lists the product', + ); + }); + }); + + it('updateEntityBasicUsage: setEntity replaces the product and sets the price and stock later tests rely on', async () => { + const pubnub = server('datasync'); + // snippet.updateEntityBasicUsage + const response = await pubnub.dataSync.setEntity({ + id: 'product-sneaker-42', + data: { + classVersion: 1, + payload: { name: 'Retro Sneaker', price: 79.99, stock: 8 }, + }, + }); + console.log('setEntity response:', response.data); + // snippet.end + assert.deepEqual(response.data.payload, { name: 'Retro Sneaker', price: 79.99, stock: 8 }); + }); + + it('patchEntityBasicUsage: updateEntity patches the price and the stock stays the same', async () => { + const pubnub = server('datasync'); + // snippet.patchEntityBasicUsage + const response = await pubnub.dataSync.updateEntity({ + id: 'product-sneaker-42', + replace: { '/payload/price': 79.99 }, + }); + console.log('updateEntity response:', response.data); + // snippet.end + assert.equal(response.data.payload?.price, 79.99); + assert.equal(response.data.payload?.stock, 8); + }); + + it('updateEntityCombinedPatchExample: one patch call with all six JSON Patch operations applies, and the eTag it was guarded by is stale afterwards', async () => { + const pubnub = server('datasync'); + const seeded = await pubnub.dataSync.setEntity({ + id: PRODUCT, + data: { + classVersion: 1, + payload: { + name: 'Retro Sneaker', + price: 79.99, + stock: 8, + tags: [], + legacy: { field: 'old' }, + legacyName: 'Retro', + }, + }, + }); + const before = seeded.data.eTag; + await assert.rejects( + () => + pubnub.dataSync.updateEntity({ + id: PRODUCT, + replace: { '/payload/price': 70 }, + ifMatchesEtag: 'StUvWxYzAbCdEf', + }), + (error: unknown) => /\b(412|403)\b/.test(errorText(error)), + 'the example eTag is rejected', + ); + // The sample text keeps the example eTag. The request carries the real one, so the guarded patch can succeed. + useRealEtag(pubnub.dataSync, 'updateEntity', 'StUvWxYzAbCdEf', before); + // snippet.updateEntityCombinedPatchExample + // A single `updateEntity` call can mix `add`, `replace`, `remove`, `move`, `copy`, and `test`. Add + // `ifMatchesEtag` (the `eTag` from a prior read) to reject the patch with a 412 if the entity + // changed since you read it, instead of silently overwriting a concurrent change. + const response = await pubnub.dataSync.updateEntity({ + id: 'product-sneaker-42', + test: { '/payload/stock': 8 }, + replace: { '/payload/price': 74.99 }, + add: { '/payload/tags/0': 'clearance' }, + remove: ['/payload/legacy/field'], + move: [{ from: '/payload/legacyName', path: '/payload/displayName' }], + copy: [{ from: '/payload/displayName', path: '/payload/previousName' }], + ifMatchesEtag: 'StUvWxYzAbCdEf', + }); + console.log('updateEntity (combined patch) response:', response.data); + // snippet.end + const payload = response.data.payload as Record; + assert.equal(payload.price, 74.99); + assert.deepEqual(payload.tags, ['clearance']); + assert.deepEqual(payload.legacy, {}); + assert.equal(payload.displayName, 'Retro'); + assert.equal(payload.previousName, 'Retro'); + assert.ok(!('legacyName' in payload), 'move removed the source'); + await assert.rejects( + () => pubnub.dataSync.updateEntity({ id: PRODUCT, replace: { '/payload/price': 70 }, ifMatchesEtag: before }), + (error: unknown) => /\b(412|403)\b/.test(errorText(error)), + 'the old eTag is rejected after the patch', + ); + }); + + // Not samples: behaviour the DataSync page promises in prose. + it('entity ids with a dot are stored verbatim, returned verbatim and found by filterFast', async () => { + const api = server('datasync').dataSync; + const id = 'product.676'; + await api.removeEntity({ id }).catch(() => undefined); + try { + const created = await api.createEntity({ + id, + class: 'product', + data: { classVersion: 1, payload: { name: 'Dot Sneaker', price: 10, stock: 1 } }, + }); + assert.equal(created.data.id, id, 'the service keeps a dotted id verbatim'); + assert.equal((await api.getEntity({ id })).data.id, id); + // `id` is a built-in filterable field, so a strongly consistent filter finds the dotted id right away. + const found = await api.getEntities({ class: 'product', filterFast: `id == "${id}"` }); + assert.deepEqual(ids(found), [id]); + } finally { + await api.removeEntity({ id }).catch(() => undefined); + } + }); + + it('an entity can be created again with the same id right after it was deleted', async () => { + // (addDataSyncListener recreates the product it deletes without a retry loop.) + const api = server('datasync').dataSync; + const id = 'product.678'; + const body = { classVersion: 1, payload: { name: 'Recreate Sneaker', price: 12, stock: 2 } }; + await api.removeEntity({ id }).catch(() => undefined); + try { + await api.createEntity({ id, class: 'product', data: body }); + await api.removeEntity({ id }); + const again = await api.createEntity({ id, class: 'product', data: body }); + assert.equal(again.data.id, id, 'the id can be reused straight after the delete'); + } finally { + await api.removeEntity({ id }).catch(() => undefined); + } + }); + + it('a duplicate create of an existing id is refused with 409 and leaves the stored object unchanged', async () => { + const api = server('datasync').dataSync; + const before = (await api.getEntity({ id: PRODUCT })).data; + await assert.rejects( + () => + api.createEntity({ + id: PRODUCT, + class: 'product', + data: { classVersion: 1, payload: { name: 'Other', price: 1, stock: 1 } }, + }), + (error: unknown) => /409/.test(errorText(error)) && /DS-0301/.test(errorText(error)), + ); + assert.deepEqual((await api.getEntity({ id: PRODUCT })).data.payload, before.payload); + }); + + it('every filter, sort and patch line shown in the DataSync page fragments is accepted or refused by the service as documented', async () => { + const api = server('datasync').dataSync; + const accepted = async ( + query: Pick, + ): Promise => { + await api.getEntities({ ...query, class: 'product', limit: 5 }); + }; + for (const fragment of filterFragments) { + await accepted(fragment).catch((error) => { + throw new Error(`filter fragment ${JSON.stringify(fragment)} was rejected: ${errorText(error)}`); + }); + } + // Two sort fragments order by `type`, a property the sample class `product` does not declare (the docs example implies a + // class that has both `type` and `price`). The service refuses a sort field that no property declares, with 403 from the + // Access Manager service (the same answer as for an undeclared projection). That is a docs example written for a + // different class, not a syntax error, so the test expects the refusal and keeps the finding visible. Every other sort + // fragment must be accepted. + const needsUndeclaredType = (fragment: (typeof sortFragments)[number]): boolean => + JSON.stringify(fragment.sort).includes('type'); + for (const fragment of sortFragments) { + if (needsUndeclaredType(fragment)) { + await assert.rejects( + accepted(fragment), + (error: unknown) => /\b(400|403)\b/.test(errorText(error)), + `undeclared sort field ${JSON.stringify(fragment)}`, + ); + } else { + await accepted(fragment).catch((error) => { + throw new Error(`sort fragment ${JSON.stringify(fragment)} was rejected: ${errorText(error)}`); + }); + } + } + + // Patch forms: apply each line to a scratch entity that holds every field the lines touch. + const id = 'product.679'; + await api.removeEntity({ id }).catch(() => undefined); + try { + await api.createEntity({ + id, + class: 'product', + data: { + classVersion: 1, + status: 'active', + payload: { + name: 'Patch Sneaker', + price: 99, + stock: 3, + tags: [], + tempFlag: true, + legacy: { field: 'old' }, + legacyName: 'Retro', + displayName: 'Retro', + }, + }, + }); + for (const fragment of patchFragments) { + const response = await api.updateEntity({ id, ...fragment }).catch((error) => { + throw new Error(`patch fragment ${JSON.stringify(fragment)} was rejected: ${errorText(error)}`); + }); + assert.equal(response.status, 200, `patch fragment ${JSON.stringify(fragment)}`); + } + // Spot check the result of the chain: add, replace, remove, move and copy all took effect. + const payload = (await api.getEntity({ id })).data.payload as Record; + assert.deepEqual(payload.tags, ['featured']); + assert.equal(payload.price, 79.99); + assert.ok(!('tempFlag' in payload)); + assert.ok(!('legacyName' in payload)); + assert.equal(payload.previousName, 'Retro'); + } finally { + await api.removeEntity({ id }).catch(() => undefined); + } + }).timeout(120_000); + }); + + // The event rules of the keyset must be on for every object kind used below (Admin Portal, DataSync tab). A kind without a rule + // makes the matching test fail on its missing event. + describe('events and subscriptions', () => { + it('dataSyncEntitySubscribeExample: the entity subscribe sample connects, and an updated entity arrives as an event', async () => { + const pubnub = server('datasync'); + await withLogs(async (lines) => { + const ready = connected(pubnub, { channels: [PRODUCT] }); + // snippet.dataSyncEntitySubscribeExample + // DataSync objects publish create/update/delete events over a `dataSync` listener. Create a + // DataSync SDK entity for the object, subscribe to it, and attach the listener. + const productSubscription = pubnub.dataSyncEntity('product-sneaker-42').subscription(); + productSubscription.onDataSync = (event) => console.log(event.message.event, event.message.data); + productSubscription.subscribe(); + // snippet.end + await ready; + try { + await pubnub.dataSync.updateEntity({ id: PRODUCT, replace: { '/payload/stock': 7 } }); + await eventually( + () => + assert.ok(lines.some((l) => l.startsWith('update') && l.includes(PRODUCT) && l.includes('"stock":7'))), + { timeout: 30_000, label: 'update event logged' }, + ); + } finally { + productSubscription.unsubscribe(); + } + }); + }).timeout(60000); + + it('subscribeToDataSyncEntity: subscribing to one entity delivers the new stock in the event payload', async () => { + const pubnub = server('datasync'); + await withLogs(async (lines) => { + const ready = connected(pubnub, { channels: [PRODUCT] }); + // snippet.subscribeToDataSyncEntity + // Observe one entity. + const entitySubscription = pubnub.dataSyncEntity('product-sneaker-42').subscription(); + + entitySubscription.onDataSync = (event) => { + console.log(event.message.event, event.message.data); + }; + + entitySubscription.subscribe(); + // snippet.end + await ready; + try { + await pubnub.dataSync.updateEntity({ id: PRODUCT, replace: { '/payload/stock': 6 } }); + await eventually(() => assert.ok(lines.some((l) => l.startsWith('update') && l.includes('"stock":6'))), { + timeout: 30_000, + label: 'update event with the new stock', + }); + } finally { + entitySubscription.unsubscribe(); + } + }); + }).timeout(60000); + + it('observeDataSyncEntitiesByPattern: a wildcard subscription on the admin projection channel receives the event for a newly created dotted id', async () => { + const pubnub = server('datasync'); + const member = 'product.676'; + await pubnub.dataSync.removeEntity({ id: member }).catch(() => undefined); + await withLogs(async (lines) => { + const ready = connected(pubnub, { channels: ['__admin__product.*'] }); + // snippet.observeDataSyncEntitiesByPattern + // Because the identifier is used verbatim, a wildcard identifier subscribes to the matching + // channel pattern and one subscription serves the whole set. `event.channel` reports the concrete + // delivery channel, while `event.subscription` reports the pattern that matched it. + const patternSubscription = pubnub.dataSyncEntity('product.*').subscription({ projection: 'admin' }); + + patternSubscription.onDataSync = (event) => { + // event.channel -> '__admin__product.676' + // event.subscription -> '__admin__product.*' + console.log('changed:', event.message.data.id); + }; + + patternSubscription.subscribe(); + // snippet.end + await ready; + try { + await pubnub.dataSync.createEntity({ + id: member, + class: 'product', + data: { classVersion: 1, payload: { name: 'Pattern Sneaker', price: 10, stock: 1 } }, + }); + await eventually(() => assert.ok(lines.includes('changed: ' + member)), { + timeout: 30_000, + label: 'event for an entity that matches the pattern', + }); + } finally { + patternSubscription.unsubscribe(); + await pubnub.dataSync.removeEntity({ id: member }).catch(() => undefined); + } + }); + }).timeout(60000); + + it('combineDataSyncSubscriptions: a user and an entity subscription combined in one set both deliver events to one listener', async () => { + const pubnub = server('datasync'); + const member = 'product.677'; + await pubnub.dataSync.removeEntity({ id: member }).catch(() => undefined); + await withLogs(async (lines) => { + const ready = connected(pubnub, { channels: ['product.*', ALICE] }); + // snippet.combineDataSyncSubscriptions + // A DataSync subscription is an ordinary `subscription`, so it composes into a set with any other + // subscription. + const productsSubscription = pubnub.dataSyncEntity('product.*').subscription(); + const aliceSubscription = pubnub.dataSyncUser('user-alice').subscription(); + + const dataSyncSubscriptionSet = productsSubscription.addSubscription(aliceSubscription); + dataSyncSubscriptionSet.onDataSync = (event) => console.log(event.message.objectType, event.message.event); + dataSyncSubscriptionSet.subscribe(); + // snippet.end + await ready; + try { + await pubnub.dataSync.updateUser({ id: ALICE, replace: { '/payload/name': 'Alice C.' } }); + await pubnub.dataSync.createEntity({ + id: member, + class: 'product', + data: { classVersion: 1, payload: { name: 'Set Sneaker', price: 11, stock: 1 } }, + }); + await eventually(() => assert.ok(lines.includes('user update') && lines.includes('entity create')), { + timeout: 30_000, + label: 'one set receives user and entity events', + }); + } finally { + dataSyncSubscriptionSet.unsubscribe(); + await pubnub.dataSync.removeEntity({ id: member }).catch(() => undefined); + } + }); + }).timeout(60000); + + it('addDataSyncListener: the listener switch sees update, delete and create events for one product', async () => { + const pubnub = server('datasync'); + await withLogs(async (lines) => { + const ready = connected(pubnub, { channels: [PRODUCT] }); + // snippet.addDataSyncListener + // Create a DataSync SDK entity for the object and attach the listener. + const dataSyncObjectId = 'product-sneaker-42'; + const dataSyncListenerSubscription = pubnub.dataSyncEntity(dataSyncObjectId).subscription(); + + dataSyncListenerSubscription.onDataSync = (event) => { + const change = event.message; + + // This channel can also receive relationship events and events about connected entities. Only + // handle events for the observed product. `Product` is a class of your own, so its events + // report `type: 'entity'`. + if (change.type !== 'entity' || change.data.id !== dataSyncObjectId) return; + + switch (change.event) { + case 'create': + console.log('Product created:', change.data.id); + break; + case 'update': { + // `payload` is typed as the generic SDK `Payload` union (it can be a string, number, array, + // and so on for other message types), but a DataSync entity payload is always the JSON + // object declared by its class, so it is safe to read it as a record here. + const payload = + 'payload' in change.data ? (change.data.payload as Record | undefined) : undefined; + console.log('New price:', payload?.price); + break; + } + case 'delete': + console.log('Product deleted:', change.data.id); + break; + } + }; + + dataSyncListenerSubscription.subscribe(); + // snippet.end + await ready; + try { + await pubnub.dataSync.updateEntity({ id: PRODUCT, replace: { '/payload/price': 69.5 } }); + await eventually(() => assert.ok(lines.some((l) => l.includes('New price: 69.5'))), { + timeout: 30_000, + label: 'update branch', + }); + await pubnub.dataSync.removeEntity({ id: PRODUCT }); + await eventually(() => assert.ok(lines.some((l) => l.includes('Product deleted: ' + PRODUCT))), { + timeout: 30_000, + label: 'delete branch', + }); + // The service accepts the same id straight after a delete (see the recreate test above), so no retry loop here. + await pubnub.dataSync.createEntity({ + id: PRODUCT, + class: 'product', + data: { classVersion: 1, payload: { name: 'Retro Sneaker', price: 74.99, stock: 8 } }, + }); + await eventually(() => assert.ok(lines.some((l) => l.includes('Product created: ' + PRODUCT))), { + timeout: 30_000, + label: 'create branch', + }); + } finally { + dataSyncListenerSubscription.unsubscribe(); + } + }); + }).timeout(60000); + + it('dataSyncProjectionSubscriptions: two projection subscriptions connect, and unsubscribing one leaves the other subscribed', async () => { + const pubnub = server('datasync'); + // The sample is written without semicolons, so the formatter must not touch it. + /* eslint-disable prettier/prettier */ + // snippet.dataSyncProjectionSubscriptions + const user = pubnub.dataSyncUser('user-alice') + + const base = user.subscription() // observes `user-alice` + const admin = user.subscription({ projection: 'admin' }) // observes `__admin__user-alice` + // snippet.end + /* eslint-enable prettier/prettier */ + // The docs promise two independent data channels: the object itself and `____`. + const ready = connected(pubnub, { channels: [ALICE, '__admin__' + ALICE] }); + base.subscribe(); + admin.subscribe(); + await ready; + // "Unsubscribing one leaves the other subscribed": drop the base one and check the admin channel is still active. + base.unsubscribe(); + assert.ok(pubnub.getSubscribedChannels().includes('__admin__' + ALICE), 'the admin projection stays subscribed'); + assert.ok(!pubnub.getSubscribedChannels().includes(ALICE), 'the base channel is gone'); + admin.unsubscribe(); + }); + }); + + describe('relationships (class ProductOwner)', () => { + // Bob is the owning end of the link. No sample creates him, so he is set up here and removed by the after hook. + before(async () => { + const created = await server('datasync').dataSync.createUser({ + id: BOB, + data: { classVersion: 1, payload: { name: 'Bob', type: 'seller' } }, + }); + assert.equal(created.data.id, BOB); + }); + + it('createRelationshipBasicUsage: createRelationship links the seller to the product with the fixed id that later tests use', async () => { + const pubnub = server('datasync'); + // snippet.createRelationshipBasicUsage + const response = await pubnub.dataSync.createRelationship({ + id: 'rel-bob-owns-sneaker-42', + class: 'ProductOwner', + entityAId: 'seller-bob', + entityBId: 'product-sneaker-42', + data: { + classVersion: 1, + payload: { since: '2026-07-13' }, + }, + }); + console.log('createRelationship response:', response.data); + // snippet.end + assert.equal(response.data.id, REL); + assert.equal(response.data.relationshipClass, 'ProductOwner'); + assert.equal(response.data.entityAId, BOB); + assert.equal(response.data.entityBId, PRODUCT); + assert.deepEqual(response.data.payload, { since: '2026-07-13' }); + }); + + it('getRelationshipBasicUsage: getRelationship reads the link by its fixed id', async () => { + const pubnub = server('datasync'); + // snippet.getRelationshipBasicUsage + const response = await pubnub.dataSync.getRelationship({ + id: 'rel-bob-owns-sneaker-42', + }); + console.log('getRelationship response:', response.data); + // snippet.end + assert.equal(response.data.id, REL); + assert.equal(response.data.entityBId, PRODUCT); + }); + + it('getRelationshipsBasicUsage: getRelationships lists exactly the link from the seller side', async () => { + const pubnub = server('datasync'); + // snippet.getRelationshipsBasicUsage + const response = await pubnub.dataSync.getRelationships({ + class: 'ProductOwner', + entityAId: 'seller-bob', + limit: 20, + }); + console.log('getRelationships response:', response.data); + console.log('getRelationships meta:', response.meta); + // snippet.end + // Exact list: legitimate because the before hook removed Bob (and every link of his) before this run created him. + assert.deepEqual(ids(response), [REL]); + }); + + it('getRelationshipsByEntityBIdExample: getRelationships by entity B id lists the same link from the product side', async () => { + const pubnub = server('datasync'); + // snippet.getRelationshipsByEntityBIdExample + // Pass `entityBId` instead of `entityAId` to list relationships where the entity is on the second + // side of the link. + const response = await pubnub.dataSync.getRelationships({ + class: 'ProductOwner', + entityBId: 'product-sneaker-42', + limit: 20, + }); + console.log('getRelationships (entityBId) response:', response.data); + // snippet.end + // Exact list: the product was removed by the before hook and recreated by this run, with every link it had. + assert.deepEqual(ids(response), [REL]); + }); + + it('updateRelationshipBasicUsage: setRelationship replaces the link and adds a tier to its payload', async () => { + const pubnub = server('datasync'); + // snippet.updateRelationshipBasicUsage + const response = await pubnub.dataSync.setRelationship({ + id: 'rel-bob-owns-sneaker-42', + entityAId: 'seller-bob', + entityBId: 'product-sneaker-42', + data: { + classVersion: 1, + payload: { since: '2026-07-13', tier: 'gold' }, + }, + }); + console.log('setRelationship response:', response.data); + // snippet.end + assert.deepEqual(response.data.payload, { since: '2026-07-13', tier: 'gold' }); + }); + + it('getRelationshipsFilterExample: a negated filter keeps the gold link once the index has caught up', async () => { + const pubnub = server('datasync'); + // `filter` is eventually consistent, so wait until the service indexed the object before running the sample. + await eventually( + async () => { + assert.ok( + ids( + await pubnub.dataSync.getRelationships({ class: 'ProductOwner', filter: '!(tier == "platinum")' }), + ).includes(REL), + ); + }, + { timeout: 45_000, label: 'filter index caught up with the relationship' }, + ); + // snippet.getRelationshipsFilterExample + const response = await pubnub.dataSync.getRelationships({ + class: 'ProductOwner', + filter: '!(tier == "platinum")', + }); + console.log('getRelationships (filter) response:', response.data); + // snippet.end + assert.ok(ids(response).includes(REL), 'tier is gold, so the negated filter keeps it'); + }).timeout(60000); + + it('getRelationshipsFilterFastExample: filterFast on tier finds the link at once', async () => { + const pubnub = server('datasync'); + // snippet.getRelationshipsFilterFastExample + const response = await pubnub.dataSync.getRelationships({ + class: 'ProductOwner', + filterFast: 'tier == "gold"', + }); + console.log('getRelationships (filterFast) response:', response.data); + // snippet.end + assert.ok(ids(response).includes(REL)); + }); + + it('getRelationshipsCursorPagingExample: the cursor paging loop for relationships runs and lists the link', async () => { + const pubnub = server('datasync'); + await withLogs(async (lines) => { + // snippet.getRelationshipsCursorPagingExample + let cursor: string | undefined; + let hasNext = true; + let page = 0; + + while (hasNext) { + const response = await pubnub.dataSync.getRelationships({ + class: 'ProductOwner', + entityAId: 'seller-bob', + limit: 20, + cursor, + }); + console.log(`Page ${++page}:`, response.data); + cursor = response.meta?.next_cursor ?? undefined; + hasNext = response.meta?.has_next ?? false; + } + // snippet.end + assert.ok( + lines.some((line) => line.includes(REL)), + 'a page lists the relationship', + ); + }); + }); + + it('patchRelationshipBasicUsage: updateRelationship patches the tier of the link', async () => { + const pubnub = server('datasync'); + // snippet.patchRelationshipBasicUsage + const response = await pubnub.dataSync.updateRelationship({ + id: 'rel-bob-owns-sneaker-42', + replace: { '/payload/tier': 'platinum' }, + }); + console.log('updateRelationship response:', response.data); + // snippet.end + assert.equal(response.data.payload?.tier, 'platinum'); + assert.equal(response.data.payload?.since, '2026-07-13'); + }); + }); + + // Removal comes last: each sample removes an object the blocks above created, in dependency order. + describe('removal', () => { + it('removeRelationshipBasicUsage: removeRelationship deletes the link and a read afterwards answers 404', async () => { + const pubnub = server('datasync'); + // snippet.removeRelationshipBasicUsage + const response = await pubnub.dataSync.removeRelationship({ + id: 'rel-bob-owns-sneaker-42', + }); + console.log('removeRelationship response status:', response.status); + // snippet.end + assert.ok(response.status >= 200 && response.status < 300); + await assert.rejects( + () => pubnub.dataSync.getRelationship({ id: REL }), + (error: unknown) => /404/.test(errorText(error)), + ); + }); + + it('removeMembershipBasicUsage: removeMembership deletes the membership and a read afterwards answers 404', async () => { + const pubnub = server('datasync'); + // snippet.removeMembershipBasicUsage + const response = await pubnub.dataSync.removeMembership({ + id: 'membership-alice-summer-sale', + }); + console.log('removeMembership response status:', response.status); + // snippet.end + assert.ok(response.status >= 200 && response.status < 300); + await assert.rejects( + () => pubnub.dataSync.getMembership({ id: MEMBERSHIP }), + (error: unknown) => /404/.test(errorText(error)), + ); + }); + + it('removeEntityBasicUsage: removeEntity deletes the product and a read afterwards answers 404', async () => { + const pubnub = server('datasync'); + // snippet.removeEntityBasicUsage + const response = await pubnub.dataSync.removeEntity({ id: 'product-sneaker-42' }); + console.log('removeEntity response status:', response.status); + // snippet.end + assert.ok(response.status >= 200 && response.status < 300); + await assert.rejects( + () => pubnub.dataSync.getEntity({ id: PRODUCT }), + (error: unknown) => /404/.test(errorText(error)), + ); + }); + + it('removeChannelBasicUsage: removeChannel deletes the DataSync channel and a read afterwards answers 404', async () => { + const pubnub = server('datasync'); + // snippet.removeChannelBasicUsage + const response = await pubnub.dataSync.removeChannel({ id: 'channel-summer-sale' }); + console.log('removeChannel response status:', response.status); + // snippet.end + assert.ok(response.status >= 200 && response.status < 300); + await assert.rejects( + () => pubnub.dataSync.getChannel({ id: SUMMER }), + (error: unknown) => /404/.test(errorText(error)), + ); + }); + + it('removeUserBasicUsage: removeUser deletes Alice and a read afterwards answers 404', async () => { + const pubnub = server('datasync'); + // snippet.removeUserBasicUsage + const response = await pubnub.dataSync.removeUser({ id: 'user-alice' }); + console.log('removeUser response status:', response.status); + // snippet.end + assert.ok(response.status >= 200 && response.status < 300); + await assert.rejects( + () => pubnub.dataSync.getUser({ id: ALICE }), + (error: unknown) => /404/.test(errorText(error)), + ); + }); + }); +}); diff --git a/test/integration/live/api-reference/encryption-legacy.test.ts b/test/integration/live/api-reference/encryption-legacy.test.ts new file mode 100644 index 000000000..6ddbdf12c --- /dev/null +++ b/test/integration/live/api-reference/encryption-legacy.test.ts @@ -0,0 +1,62 @@ +import assert from 'node:assert/strict'; +import PubNub from 'pubnub'; +import { createClient } from '../support/clients'; + +/** + * Examples for the deprecated `encrypt()` and `decrypt()` methods and the cipher key arguments. Both methods log a + * deprecation warning on every call. Port of docs-snippets/api-reference/encryption-legacy.ts. Everything runs locally. + */ +describe('api-reference/encryption-legacy', () => { + // Open keyset client with an AES-CBC crypto module, the setup the legacy methods fall back to. + const withModule = (label: string): PubNub => + createClient('open', label, { + cryptoModule: PubNub.CryptoModule.aesCbcCryptoModule({ cipherKey: 'pubnubenigma' }), + }); + + it('decryptBasicUsage: the legacy decrypt returns the original JSON for text encrypted with the same cipher key', () => { + const pubnub = withModule('legacy-decrypt'); + // `decrypt()` JSON-parses the plain text, so the input has to be a JSON document. + const encrypted = pubnub.encrypt(JSON.stringify({ item: 'sneaker' })) as string; + // snippet.decryptBasicUsage + const decrypted = pubnub.decrypt(encrypted); // Pass the encrypted data as the first parameter in decrypt Method + // snippet.end + assert.deepEqual(decrypted, { item: 'sneaker' }); + }); + + it('encryptionLegacyCipherKeyArguments: the optional customCipherKey argument of encrypt and decrypt behaves as the page states', () => { + const pubnub = withModule('legacy-cipher-key'); + const secret = { item: 'sneaker', size: 42 }; + // customCipherKey overrides the configured module and uses the legacy encryption. + const encrypted = pubnub.encrypt(JSON.stringify(secret), 'another-key'); + assert.equal(typeof encrypted, 'string'); + assert.throws(() => pubnub.encrypt(secret as never, 'another-key'), TypeError); + assert.deepEqual(pubnub.decrypt(encrypted, 'another-key'), secret); + // A wrong key does not throw with the legacy encryption: decrypt() returns null. + assert.equal(pubnub.decrypt(encrypted, 'wrong-key'), null); + // Without a configured module or a key, the data comes back unchanged. + const bare = createClient('open', 'legacy-bare'); + assert.equal(bare.encrypt('plain'), 'plain'); + }); + + // The error row of the decrypt section: unreadable input throws instead of returning garbage. + it('encryptionLegacyDecryptError: decrypting unreadable input throws instead of returning garbage', () => { + const pubnub = withModule('legacy-decrypt-error'); + // cryptoModule rejects data it cannot read with an Error that describes the reason. + assert.throws(() => pubnub.decrypt(''), /Decryption error/); + }); + + it('encryptionLegacyFileCipherKey: legacy file encryption with an explicit key works, and still needs a configured module to decrypt', async () => { + const pubnub = createClient('open', 'legacy-file', { cipherKey: 'pubnubenigma' }); + const content = Buffer.from('legacy file content'); + const file = pubnub.File.create({ data: content, name: 'legacy.txt', mimeType: 'text/plain' }); + const encrypted = await pubnub.encryptFile('file-key', file); + assert.notDeepEqual(Buffer.from(await encrypted.toArrayBuffer()), content); + const decrypted = await pubnub.decryptFile('file-key', encrypted); + assert.deepEqual(Buffer.from(await decrypted.toArrayBuffer()), content); + const noModule = createClient('open', 'legacy-file-none'); + await assert.rejects( + noModule.decryptFile('file-key', encrypted), + /File decryption error\. Crypto module not configured\./, + ); + }); +}); diff --git a/test/integration/live/api-reference/encryption.test.ts b/test/integration/live/api-reference/encryption.test.ts new file mode 100644 index 000000000..4036e8648 --- /dev/null +++ b/test/integration/live/api-reference/encryption.test.ts @@ -0,0 +1,119 @@ +import assert from 'node:assert/strict'; +import crypto from 'node:crypto'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import PubNub from 'pubnub'; +import { createClient } from '../support/clients'; + +/** + * Encryption examples. Port of docs-snippets/api-reference/encryption.ts. Everything runs locally (no request is made), + * the clients only carry a crypto module. Each file test works in its own temp directory. + */ +describe('api-reference/encryption', () => { + let workDir = ''; + const picture = crypto.randomBytes(2048); + + beforeEach(() => { + workDir = fs.mkdtempSync(path.join(os.tmpdir(), 'pn-docs-encryption-')); + }); + afterEach(() => { + fs.rmSync(workDir, { recursive: true, force: true }); + }); + + /** Runs `fn` with `workDir` as the working directory, as the docs file samples assume. */ + const inWorkDir = async (fn: () => Promise): Promise => { + const previous = process.cwd(); + process.chdir(workDir); + try { + return await fn(); + } finally { + process.chdir(previous); + } + }; + + const cryptoClient = (label: string): PubNub => + createClient('open', label, { + cryptoModule: PubNub.CryptoModule.aesCbcCryptoModule({ cipherKey: 'pubnubenigma' }), + }); + + it('encryptMessageBasicUsage: encrypt turns a message into cipher text that differs from the plain text, and decrypt reads it back', () => { + // snippet.encryptMessageBasicUsage + // Create a crypto module instance with AES-CBC encryption + const cryptoModule = PubNub.CryptoModule.aesCbcCryptoModule({ + cipherKey: 'pubnubenigma', + }); + + const msgContent = 'This is the data I wish to encrypt.'; + console.log(`Original Message: ${msgContent}`); + + // Encrypt the message + const encryptedMessage = cryptoModule.encrypt(JSON.stringify(msgContent)); + console.log('message encrypted successfully'); + + // Decrypt with the same instance. decrypt() returns an ArrayBuffer with the bytes of the JSON string + const decryptedBytes = cryptoModule.decrypt(encryptedMessage) as ArrayBuffer; + const decryptedMessage = JSON.parse(new TextDecoder().decode(decryptedBytes)); + console.log(`Decrypted Message: ${decryptedMessage}`); + // snippet.end + assert.notEqual(encryptedMessage, JSON.stringify(msgContent), 'the output is not the plain text'); + assert.ok(decryptedBytes instanceof ArrayBuffer, 'decrypt returns an ArrayBuffer'); + assert.equal(decryptedMessage, msgContent, 'the decoded bytes are the original message'); + }); + + it('encryptFileBasicUsage: encryptFile produces an encrypted copy of the picture', async () => { + const pubnub = cryptoClient('crypto-file'); + fs.writeFileSync(path.join(workDir, 'cat_picture.jpg'), picture); + await inWorkDir(async () => { + // snippet.encryptFileBasicUsage + // Node.js example + // import fs from 'fs'; + + const fileBuffer = fs.readFileSync('./cat_picture.jpg'); + + const file = pubnub.File.create({ data: fileBuffer, name: 'cat_picture.jpg', mimeType: 'image/jpeg' }); + + const encryptedFile = await pubnub.encryptFile(file); + // snippet.end + const encrypted = Buffer.from(await encryptedFile.toArrayBuffer()); + assert.notDeepEqual(encrypted, picture, 'the file content changed'); + assert.ok(encrypted.length > picture.length, 'the cipher text carries a header and padding'); + }); + }); + + it('decryptFileBasicUsage: decryptFile restores the picture byte for byte with a second client that has the same cipher key', async () => { + // Setup outside the sample: an encrypted copy of the picture, made by another client with the same cipher key. + const writer = cryptoClient('crypto-file'); + const encryptedFile = await writer.encryptFile( + writer.File.create({ data: picture, name: 'cat_picture.jpg', mimeType: 'image/jpeg' }), + ); + fs.writeFileSync(path.join(workDir, 'cat_picture_encrypted.jpg'), Buffer.from(await encryptedFile.toArrayBuffer())); + const pubnub = cryptoClient('crypto-file-reader'); + await inWorkDir(async () => { + // snippet.decryptFileBasicUsage + // Node.js example. + // import fs from 'fs'; + const fileBufferData = fs.readFileSync('./cat_picture_encrypted.jpg'); + + const fileData = pubnub.File.create({ data: fileBufferData, name: 'cat_picture.jpg', mimeType: 'image/jpeg' }); + + const decryptedFile = await pubnub.decryptFile(fileData); + // snippet.end + const decrypted = Buffer.from(await decryptedFile.toArrayBuffer()); + assert.deepEqual(decrypted, picture, 'the round trip returns the original bytes'); + }); + }); + + it('encryptionFileErrors: each row of the file encryption error table is raised for the matching bad call', async () => { + const withModule = cryptoClient('crypto-errors'); + const file = withModule.File.create({ data: Buffer.from('abc'), name: 'a.txt', mimeType: 'text/plain' }); + await assert.rejects( + withModule.encryptFile(undefined as never), + /File encryption error\. Source file is missing\./, + ); + + const noModule = createClient('open', 'crypto-none'); + await assert.rejects(noModule.encryptFile(file), /File encryption error\. Crypto module not configured\./); + assert.equal(await noModule.decryptFile(file), undefined); + }); +}); diff --git a/test/integration/live/api-reference/file-sharing-legacy.test.ts b/test/integration/live/api-reference/file-sharing-legacy.test.ts new file mode 100644 index 000000000..c409592b4 --- /dev/null +++ b/test/integration/live/api-reference/file-sharing-legacy.test.ts @@ -0,0 +1,88 @@ +import assert from 'node:assert/strict'; +import fs from 'node:fs'; +import { createClient, shopper } from '../support/clients'; +import { withPlaceholders } from '../support/placeholders'; +import { enterWorkdir, FIXTURE_FILE } from '../support/file-workdir'; + +/** + * Example for the deprecated per-call `cipherKey` parameter of file sharing. Port of + * docs-snippets/api-reference/file-sharing-legacy.ts. Files persist on the shared channel 'my_channel', so each test asserts + * on the id it got back and an `afterEach` deletes the files the test uploaded. The download sample shows the id placeholder + * '...': it runs unchanged through a client wrapper (`withPlaceholders`) that swaps it for the real id. + */ +describe('api-reference/file-sharing-legacy', () => { + // Plain bytes of the fixture, the reference for the round trip comparison. + const original = fs.readFileSync(FIXTURE_FILE); + let uploads: string[] = []; + + afterEach(async () => { + const cleaner = createClient('open', 'file-cleaner'); + const pending = uploads; + uploads = []; + for (const id of pending) { + try { + await cleaner.deleteFile({ channel: 'my_channel', id, name: 'cat_picture.jpg' }); + } catch { + // Already deleted by the test itself. + } + } + }); + + it('sendFileCustomCipherKey: sendFile with a custom cipher key uploads the picture from a working directory', async function () { + this.timeout(60_000); + const pubnub = shopper(); + const workdir = enterWorkdir(); + try { + // snippet.sendFileCustomCipherKey + // in Node.js + + const myFile = fs.readFileSync('./cat_picture.jpg'); + + const response = await pubnub.sendFile({ + channel: 'my_channel', + message: 'Look at this picture!', + file: { data: myFile, name: 'cat_picture.jpg', mimeType: 'image/jpeg' }, + cipherKey: 'myCipherKey', + }); + console.log('File sent successfully:', response); + // snippet.end + assert.ok(response.id); + uploads.push(response.id); + // The upload is encrypted: downloading with the key returns the picture, without it the stored payload differs. + const withKey = await pubnub.downloadFile({ + channel: 'my_channel', + id: response.id, + name: 'cat_picture.jpg', + cipherKey: 'myCipherKey', + }); + assert.deepEqual(await withKey.toBuffer(), original, 'the uploaded file decrypts with the cipher key'); + } finally { + workdir.restore(); + } + }); + + it('downloadFileCustomCipherKey: downloadFile with the right custom cipher key returns the file and without the key it does not', async function () { + this.timeout(60_000); + const uploader = shopper(); + const sent = await uploader.sendFile({ + channel: 'my_channel', + message: 'Look at this picture!', + file: { data: original, name: 'cat_picture.jpg', mimeType: 'image/jpeg' }, + cipherKey: 'myCipherKey', + }); + uploads.push(sent.id); + const pubnub = withPlaceholders(shopper(), { '...': sent.id }); + // snippet.downloadFileCustomCipherKey + const file = await pubnub.downloadFile({ + channel: 'my_channel', + id: '...', + name: 'cat_picture.jpg', + cipherKey: 'myCipherKey', + }); + // snippet.end + assert.deepEqual(await file.toBuffer(), original, 'the cipher key decrypts the upload'); + // Without the key the content is the encrypted payload. + const raw = await uploader.downloadFile({ channel: 'my_channel', id: sent.id, name: 'cat_picture.jpg' }); + assert.notDeepEqual(await raw.toBuffer(), original, 'no key, no plain file'); + }); +}); diff --git a/test/integration/live/api-reference/file-sharing.test.ts b/test/integration/live/api-reference/file-sharing.test.ts new file mode 100644 index 000000000..043a1ffdd --- /dev/null +++ b/test/integration/live/api-reference/file-sharing.test.ts @@ -0,0 +1,263 @@ +import assert from 'node:assert/strict'; +import fs from 'node:fs'; +import { Readable } from 'node:stream'; +import type PubNub from 'pubnub'; +import { assertTimetoken } from '../support/assertions'; +import { captureConsole } from '../support/console'; +import { createClient, observer, shopper } from '../support/clients'; +import { withPlaceholders } from '../support/placeholders'; +import { enterWorkdir, FIXTURE_FILE } from '../support/file-workdir'; +import { eventually, watch } from '../support/wait'; + +/** + * File sharing examples for Node.js. Port of docs-snippets/api-reference/file-sharing.ts. + * The open keyset needs File Sharing and Message Persistence enabled. + * + * Files persist on the shared channel 'my_channel', so every test asserts on the file id it got back and an `afterEach` + * deletes every file the test uploaded. Samples that show the id placeholder '...' run unchanged: the test passes them a + * client wrapper (`withPlaceholders`) that swaps '...' for the id of the file the test uploaded. + */ +describe('api-reference/file-sharing', () => { + // Plain bytes of the fixture, the reference every download is compared against. + const original = fs.readFileSync(FIXTURE_FILE); + const CHANNEL = 'my_channel'; + const NAME = 'cat_picture.jpg'; + let uploads: Array<{ id: string; name: string }> = []; + + /** Uploads the fixture outside any sample and registers it for deletion. */ + const upload = async (client: PubNub): Promise<{ id: string; name: string }> => { + const response = await client.sendFile({ + channel: CHANNEL, + file: { data: original, name: NAME, mimeType: 'image/jpeg' }, + customMessageType: 'file-message', + }); + const entry = { id: response.id, name: response.name }; + uploads.push(entry); + return entry; + }; + + afterEach(async () => { + const cleaner = createClient('open', 'file-cleaner'); + const pending = uploads; + uploads = []; + for (const file of pending) { + try { + await cleaner.deleteFile({ channel: CHANNEL, id: file.id, name: file.name }); + } catch { + // Already deleted by the test itself. + } + } + }); + + it('sendFileBasicUsage: sendFile uploads the picture and a second client receives the file event', async function () { + this.timeout(60_000); + const pubnub = shopper(); + const listening = await watch(observer(), [CHANNEL]); + const workdir = enterWorkdir(); + try { + // snippet.sendFileBasicUsage + // Function to send a file to a channel + // in node.js, make sure to import 'fs' + // and use the createReadStream method to read the file + const myFile = fs.createReadStream('./cat_picture.jpg'); + + const response = await pubnub.sendFile({ + channel: 'my_channel', + file: { stream: myFile, name: 'cat_picture.jpg', mimeType: 'image/jpeg' }, + customMessageType: 'file-message', + }); + + console.log('File sent successfully:', response); + // snippet.end + assert.ok(response.id, 'the response carries the file id'); + uploads.push({ id: response.id, name: response.name }); + assert.equal(response.name, 'cat_picture.jpg'); + const event = await listening.next('file', (e) => e.file?.id === response.id); + assert.equal(event.channel, CHANNEL); + assert.equal(event.file?.name, 'cat_picture.jpg'); + assert.equal(event.publisher, pubnub.getUserId()); + } finally { + listening.stop(); + workdir.restore(); + } + }); + + it('listFilesBasicUsage: listFiles returns the file this test uploaded', async function () { + this.timeout(60_000); + const { id } = await upload(shopper()); + const pubnub = shopper(); + // snippet.listFilesBasicUsage + const response = await pubnub.listFiles({ channel: 'my_channel' }); + console.log('Files listed successfully:', response); + // snippet.end + // The channel is shared, so look for this run's file by id. The sample lists the default page only, which holds the newest files. + const entry = response.data.find((f) => f.id === id); + assert.ok(entry, 'the uploaded file is listed'); + assert.equal(entry.name, 'cat_picture.jpg'); + }); + + it('getFileUrlBasicUsage: getFileUrl builds a URL that serves the uploaded bytes', async function () { + this.timeout(60_000); + const { id, name } = await upload(shopper()); + const pubnub = withPlaceholders(shopper(), { '...': id }); + // snippet.getFileUrlBasicUsage + const response = pubnub.getFileUrl({ channel: 'my_channel', id: '...', name: 'cat_picture.jpg' }); + // snippet.end + assert.match(response, /^https:\/\//); + assert.ok(response.includes(`/files/${id}/`), 'the URL carries the file id'); + assert.ok(response.includes(`/${name}?`), 'the URL carries the file name'); + assert.ok(response.includes(encodeURIComponent(CHANNEL)), 'the URL carries the channel'); + const served = await fetch(response); + assert.equal(served.status, 200); + assert.deepEqual(Buffer.from(await served.arrayBuffer()), original); + }); + + it('downloadFileNodeBasicUsage: downloadFile in Node writes the downloaded content to the output file', async function () { + this.timeout(60_000); + const { id } = await upload(shopper()); + const pubnub = withPlaceholders(shopper(), { '...': id }); + const workdir = enterWorkdir(); + const logs = captureConsole('log'); + try { + fs.writeFileSync('./cat_picture.jpg', 'placeholder to be overwritten'); + // snippet.downloadFileNodeBasicUsage + // In Node.js using streams: + // import fs from 'fs' + const downloadFileResponse = await pubnub.downloadFile({ + channel: 'my_channel', + id: '...', + name: 'cat_picture.jpg', + }); + + const output = fs.createWriteStream('./cat_picture.jpg'); + const fileStream = await downloadFileResponse.toStream(); + + fileStream.pipe(output); + + output.once('finish', () => { + console.log('File saved to ./cat_picture.jpg'); + }); + // snippet.end + await eventually( + () => { + assert.deepEqual(fs.readFileSync('./cat_picture.jpg'), original); + }, + { label: 'downloaded file written' }, + ); + await eventually( + () => { + assert.ok( + logs.lines.includes('File saved to ./cat_picture.jpg'), + 'the finish handler logged the confirmation', + ); + }, + { label: 'finish event logged' }, + ); + assert.equal(downloadFileResponse.name, 'cat_picture.jpg'); + } finally { + logs.restore(); + workdir.restore(); + } + }); + + it('publishFileMessageBasicUsage: publishFile announces the uploaded file and the event is matched by timetoken', async function () { + this.timeout(60_000); + const { id } = await upload(shopper()); + const pubnub = withPlaceholders(shopper(), { '...': id }); + const listening = await watch(observer(), [CHANNEL]); + // snippet.publishFileMessageBasicUsage + const fileMessageResponse = await pubnub.publishFile({ + channel: 'my_channel', + fileId: '...', + fileName: 'cat_picture.jpg', + message: { field: 'value' }, + customMessageType: 'file-message', + }); + console.log('File message published successfully:', fileMessageResponse); + // snippet.end + assertTimetoken(fileMessageResponse.timetoken); + const event = await listening.next('file', (e) => e.timetoken === String(fileMessageResponse.timetoken)); + assert.equal(event.file?.id, id); + assert.deepEqual(event.message, { field: 'value' }); + listening.stop(); + }); + + // ---- FileInput forms and PubNubFile in Node --------------------------------------------------------------------- + + it('fileInputNodeForms: every Node file input form listed on the page uploads and downloads correctly', async function () { + this.timeout(90_000); + // The docs list stream, buffer, ArrayBuffer and string forms for Node.js. Each one must upload and come back unchanged. + const pubnub = shopper(); + const text = 'sneaker drop receipt'; + const forms: Array<{ label: string; file: Parameters[0]['file']; expected: Buffer }> = [ + { + label: 'stream', + file: { stream: Readable.from([original]), name: 'stream.jpg', mimeType: 'image/jpeg' }, + expected: original, + }, + { label: 'buffer', file: { data: original, name: 'buffer.jpg', mimeType: 'image/jpeg' }, expected: original }, + { + label: 'ArrayBuffer', + file: { data: new Uint8Array(original).buffer, name: 'arraybuffer.jpg', mimeType: 'image/jpeg' }, + expected: original, + }, + { + label: 'string', + file: { data: text, name: 'string.txt', mimeType: 'text/plain' }, + expected: Buffer.from(text), + }, + ]; + for (const form of forms) { + const name = (form.file as { name: string }).name; + const sent = await pubnub.sendFile({ channel: CHANNEL, file: form.file }); + uploads.push({ id: sent.id, name }); + const downloaded = await pubnub.downloadFile({ channel: CHANNEL, id: sent.id, name }); + assert.deepEqual(await downloaded.toBuffer(), form.expected, `${form.label} round trip`); + } + }); + + it('pubnubFileNodeMethods: the PubNubFile conversions that work in Node work and the others reject as the page table says', async function () { + this.timeout(60_000); + // Docs, "Methods supported in Node.js": toBuffer, toStream, toString, toArrayBuffer work. toFile, toBlob and toFileUri reject. + const pubnub = shopper(); + const { id, name } = await upload(pubnub); + const read = async () => pubnub.downloadFile({ channel: CHANNEL, id, name }); + const stream = await (await read()).toStream(); + const chunks: Buffer[] = []; + for await (const chunk of stream) chunks.push(Buffer.from(chunk)); + assert.deepEqual(Buffer.concat(chunks), original, 'toStream'); + assert.deepEqual(await (await read()).toBuffer(), original, 'toBuffer'); + assert.deepEqual(Buffer.from(await (await read()).toArrayBuffer()), original, 'toArrayBuffer'); + assert.equal(await (await read()).toString('base64'), original.toString('base64'), 'toString with an encoding'); + for (const method of ['toFile', 'toBlob', 'toFileUri'] as const) { + const target = await read(); + await assert.rejects( + () => (target as unknown as Record Promise>)[method](), + /only supported in/, + `${method} rejects in Node`, + ); + } + }); + + it('deleteFileBasicUsage: deleteFile removes the file and a later listing no longer shows it', async function () { + this.timeout(60_000); + const { id } = await upload(shopper()); + const pubnub = withPlaceholders(shopper(), { '...': id }); + // snippet.deleteFileBasicUsage + const deleteFileResponse = await pubnub.deleteFile({ + channel: 'my_channel', + id: '...', + name: 'cat_picture.jpg', + }); + console.log('File deleted successfully:', deleteFileResponse); + // snippet.end + assert.equal(deleteFileResponse.status, 200); + await eventually( + async () => { + const left = await observer().listFiles({ channel: CHANNEL }); + assert.ok(!left.data.some((f) => f.id === id), 'the file is no longer listed'); + }, + { label: 'file removed from the list' }, + ); + }); +}); diff --git a/test/integration/live/api-reference/message-actions.test.ts b/test/integration/live/api-reference/message-actions.test.ts new file mode 100644 index 000000000..854f5beb9 --- /dev/null +++ b/test/integration/live/api-reference/message-actions.test.ts @@ -0,0 +1,136 @@ +import assert from 'node:assert/strict'; +import { addToTimetoken } from '../support/assertions'; +import { friend, observer, shopper } from '../support/clients'; +import { retryTransient } from '../support/retry'; +import { withPlaceholders } from '../support/placeholders'; +import { eventually, watch } from '../support/wait'; + +/** + * Message action examples. Port of docs-snippets/api-reference/message-actions.ts. Message Persistence must be on for the open + * keyset. The samples carry placeholder literals (`replace_with_message_timetoken` and so on): the client is wrapped with + * `withPlaceholders`, so the sample text stays unchanged and the real timetokens reach the service. Every test publishes its + * own message first, so the tests do not depend on each other. + */ +describe('api-reference/message-actions', () => { + it('addMessageActionBasicUsage: addMessageAction returns the action and a subscriber receives an added event', async () => { + const listening = await watch(observer(), ['channel_name']); + const published = await retryTransient( + () => friend().publish({ channel: 'channel_name', message: { text: 'react to me' } }), + { label: 'seed message-actions publish' }, + ); + const pubnub = withPlaceholders(shopper(), { replace_with_message_timetoken: published.timetoken }); + + // snippet.addMessageActionBasicUsage + // first publish a message using publish() method to get the message timetoken + const response = await pubnub.addMessageAction({ + channel: 'channel_name', + messageTimetoken: 'replace_with_message_timetoken', // Replace with actual message timetoken + action: { + type: 'reaction', + value: 'smiley_face', + }, + }); + console.log('Message reaction added successfully:', response); + + // snippet.end + assert.equal(response.data.type, 'reaction'); + assert.equal(response.data.value, 'smiley_face'); + assert.equal(response.data.messageTimetoken, published.timetoken); + const event = await listening.next( + 'messageAction', + (e) => e.data.actionTimetoken === response.data.actionTimetoken, + ); + assert.equal(event.event, 'added'); + assert.equal(event.data.uuid, pubnub.getUserId()); + listening.stop(); + // Leave no action behind on the persistent message. + await friend().removeMessageAction({ + channel: 'channel_name', + messageTimetoken: published.timetoken, + actionTimetoken: response.data.actionTimetoken, + }); + }); + + it('removeMessageActionBasicUsage: removeMessageAction emits a removed event and the action disappears from the stored list', async () => { + const channel = 'channel_name'; + const published = await retryTransient( + () => friend().publish({ channel, message: { text: 'remove my reaction' } }), + { label: 'seed message-actions publish' }, + ); + const added = await friend().addMessageAction({ + channel, + messageTimetoken: published.timetoken, + action: { type: 'reaction', value: 'smiley_face' }, + }); + const actionTimetoken = added.data.actionTimetoken; + const pubnub = withPlaceholders(shopper(), { + replace_with_message_timetoken: published.timetoken, + replace_with_action_timetoken: actionTimetoken, + }); + const listening = await watch(observer(), [channel]); + + // snippet.removeMessageActionBasicUsage + const response = await pubnub.removeMessageAction({ + channel: 'channel_name', + messageTimetoken: 'replace_with_message_timetoken', + actionTimetoken: 'replace_with_action_timetoken', + }); + console.log('Message action removed successfully:', response); + // snippet.end + const event = await listening.next('messageAction', (e) => e.data.actionTimetoken === actionTimetoken); + assert.equal(event.event, 'removed'); + await eventually( + async () => { + // Window of one action: start is the exclusive upper bound, end the inclusive lower bound. + const left = await observer().getMessageActions({ + channel, + start: addToTimetoken(actionTimetoken, 1), + end: actionTimetoken, + }); + assert.ok(!left.data.some((a) => a.actionTimetoken === actionTimetoken), 'action is gone'); + }, + { label: 'removed action is no longer listed' }, + ); + listening.stop(); + }); + + it('getMessageActionsBasicUsage: getMessageActions returns the action added during setup with the right type, value and message timetoken', async () => { + const channel = 'channel_name'; + const published = await retryTransient( + () => friend().publish({ channel, message: { text: 'count my reactions' } }), + { label: 'seed message-actions publish' }, + ); + const added = await friend().addMessageAction({ + channel, + messageTimetoken: published.timetoken, + action: { type: 'reaction', value: 'thumbs_up' }, + }); + // Actions are listed newest first: `start` is the exclusive upper bound, `end` the inclusive lower bound. The window + // holds exactly this run's action, so the list can be compared in full although the channel is shared. + const pubnub = withPlaceholders(shopper(), { + replace_with_start_timetoken: addToTimetoken(added.data.actionTimetoken, 1), + replace_with_end_timetoken: added.data.actionTimetoken, + }); + + // snippet.getMessageActionsBasicUsage + // to get some data in response, first publish a message and then add a message action + // using addMessageAction() method. + const response = await pubnub.getMessageActions({ + channel: 'channel_name', + start: 'replace_with_start_timetoken', + end: 'replace_with_end_timetoken', + limit: 100, + }); + console.log('Message actions retrieved successfully:', response); + // snippet.end + assert.equal(response.data.length, 1); + assert.equal(response.data[0].actionTimetoken, added.data.actionTimetoken); + assert.equal(response.data[0].value, 'thumbs_up'); + assert.equal(response.data[0].messageTimetoken, published.timetoken); + await friend().removeMessageAction({ + channel, + messageTimetoken: published.timetoken, + actionTimetoken: added.data.actionTimetoken, + }); + }); +}); diff --git a/test/integration/live/api-reference/message-persistence-legacy.test.ts b/test/integration/live/api-reference/message-persistence-legacy.test.ts new file mode 100644 index 000000000..ffded8a9e --- /dev/null +++ b/test/integration/live/api-reference/message-persistence-legacy.test.ts @@ -0,0 +1,585 @@ +import assert from 'node:assert/strict'; +import type PubNub from 'pubnub'; +import { addToTimetoken } from '../support/assertions'; +import { captureConsole } from '../support/console'; +import { friend, observer, shopper } from '../support/clients'; +import { withPlaceholders } from '../support/placeholders'; +import { newSeedRun, seedPublish, settleSeeded, storedMessage, untilStored } from '../support/history'; +import { retryTransient } from '../support/retry'; +import { eventually, sleep } from '../support/wait'; + +/** + * Examples for the deprecated `history()` method and the deprecated `data` property of `actions` on `fetchMessages()`. + * Port of docs-snippets/api-reference/message-persistence-legacy.ts. Message Persistence and Delete-From-History must be on + * for the open keyset. + * + * Channels are shared and persistent, and the samples read "the latest 100" of a channel. So every test reads its own + * window: the messages it just published are the newest on the channel, and the placeholder literals of the samples + * (`13406746780720711`, `123123123133`, `123123123123`) are replaced by this run's timetokens through `withPlaceholders`, + * which keeps the sample text unchanged. + * + * The one test that clears a channel is historyPagingExample: the sample walks the WHOLE channel `history_test` and the text + * leaves no way to bound that, so it deletes that channel's history at its start. + * Residue: seeded messages on `history_channel`, `my_channel`, `persist_channel`, `history_documented_behavior` (105 + * messages, written once) and `history_test` (120 messages, cleared at the start of each run). + */ +describe('api-reference/message-persistence-legacy', () => { + // Hands the result out of the sample's try block (set on a hidden line of the sample). + let capturedResult: PubNub.History.GetHistoryResponse | undefined; + beforeEach(() => { + capturedResult = undefined; + }); + + // Publishes `count` messages as the friend, optionally with `meta`, and returns their timetokens, oldest first. Setup, so a + // transient publish failure is repeated (seedPublish). A repeated publish may leave a stored copy under a timetoken this test + // never saw, which the exact-order assertions below would trip over: settleSeeded removes such copies before the sample runs. + const seed = async ( + channel: string, + count: number, + extra: { meta?: Record } = {}, + ): Promise => { + const run = newSeedRun(); + const tokens: string[] = []; + for (let i = 0; i < count; i += 1) { + const { timetoken } = await seedPublish(run, `seed ${channel} publish`, () => + friend().publish({ channel, message: { legacy: 'history', n: i, run: run.id }, ...extra }), + ); + tokens.push(timetoken); + } + await settleSeeded(channel, tokens, run); + return tokens; + }; + + /** + * Waits until every one of `tokens` is stored, however many there are. `untilStored` reads one page of at most 100 + * messages, so with more it can only vouch for the newest 100. This one walks the channel page by page from the newest + * message to the oldest and checks the whole set of tokens at the end. Storage is eventually consistent, so the walk + * repeats until nothing is missing or the timeout passes. + */ + const untilAllStored = (channel: string, tokens: string[]): Promise => + eventually( + async () => { + const have = new Set(); + let before: string | undefined; + for (;;) { + const page = await observer().fetchMessages({ + channels: [channel], + count: 100, + ...(before ? { start: before } : {}), + }); + const items = page.channels[channel] ?? []; + for (const item of items) have.add(String(item.timetoken)); + // A page shorter than the cap is the last one. Messages come oldest first, so items[0] is the oldest of the page. + if (items.length < 100) break; + before = String(items[0].timetoken); + } + const missing = tokens.filter((token) => !have.has(token)); + assert.equal( + missing.length, + 0, + `${missing.length} of ${tokens.length} seeded messages are not stored yet (first missing: ${missing[0]})`, + ); + }, + { timeout: 45_000, label: `all ${tokens.length} messages stored on ${channel}` }, + ); + + // Normalizes timetokens to strings, history() returns numbers unless stringifiedTimeToken is set. + const tokensOf = (result: { messages: Array<{ timetoken: string | number }> }): string[] => + result.messages.map((m) => String(m.timetoken)); + + // history() always returns oldest first, whatever `reverse` is. + const isAscending = (tokens: string[]): boolean => + tokens.every((t, i) => i === 0 || BigInt(tokens[i - 1]) < BigInt(t)); + + // history() with one retry loop for network hiccups, as the old harness did. + const withRetry = (pubnub: PubNub): PubNub => { + const originalHistory = pubnub.history.bind(pubnub) as (...args: unknown[]) => Promise; + (pubnub as unknown as { history: unknown }).history = async (...args: unknown[]): Promise => { + const attempts = 3; + for (let attempt = 1; ; attempt += 1) { + try { + return await originalHistory(...args); + } catch (error) { + const category = (error as { status?: { category?: string } }).status?.category; + if (category !== 'PNNetworkIssuesCategory' || attempt >= attempts) throw error; + await sleep(1_000 * attempt); + } + } + }; + return pubnub; + }; + + it('historyBasicUsage: history returns the three seeded messages as the newest, oldest first, with the start and end timetokens', async () => { + const pubnub = shopper(); + const tokens = await seed('history_channel', 3); + await untilStored('history_channel', tokens); + + /* eslint-disable prettier/prettier, max-len -- sample text is copied as readers see it */ + + // snippet.historyBasicUsage + try { + const result = await pubnub.history({ + channel: "history_channel", + count: 100, // how many items to fetch + stringifiedTimeToken: true, // false is the default + }); + // snippet.hide + capturedResult = result; + // snippet.show + } catch (status) { + console.log(status); + } + // snippet.end + /* eslint-enable prettier/prettier, max-len */ + assert.ok(capturedResult, 'the history call returned a result (its catch only logs)'); + const result = capturedResult; + + // The channel is shared, so earlier messages may precede them: the three just published are the newest, in order. + assert.deepEqual(tokensOf(result).slice(-3), tokens, 'the three seeded messages are the newest, oldest first'); + assert.ok(isAscending(tokensOf(result))); + assert.deepEqual( + result.messages.slice(-3).map((m) => (m.entry as { n: number }).n), + [0, 1, 2], + ); + assert.equal(result.startTimeToken, tokensOf(result)[0]); + assert.equal(result.endTimeToken, tokens[2]); + }); + + it('historyOldestThree: history with reverse true and count 3 returns the three oldest messages', async () => { + const pubnub = shopper(); + const tokens = await seed('my_channel', 3); + await untilStored('my_channel', tokens); + + /* eslint-disable prettier/prettier, max-len -- sample text is copied as readers see it */ + + // snippet.historyOldestThree + try { + const result = await pubnub.history({ + channel: "my_channel", + reverse: true, // Setting to true will traverse the time line in reverse starting with the oldest message first. + count: 3, // how many items to fetch + stringifiedTimeToken: true, // false is the default + }); + // snippet.hide + capturedResult = result; + // snippet.show + } catch (status) { + console.log(status); + } + // snippet.end + /* eslint-enable prettier/prettier, max-len */ + assert.ok(capturedResult, 'the history call returned a result (its catch only logs)'); + const result = capturedResult; + + // The oldest messages of a shared channel are not this run's, so check them against an independent reader (fetchMessages). + const got = tokensOf(result); + assert.equal(got.length, 3); + assert.ok(isAscending(got)); + const older = await observer().fetchMessages({ channels: ['my_channel'], start: got[0], count: 1 }); + assert.equal( + older.channels['my_channel']?.length ?? 0, + 0, + 'nothing on the channel is older than the first returned message', + ); + const window = await observer().fetchMessages({ + channels: ['my_channel'], + start: addToTimetoken(got[2], 1), + end: got[0], + count: 100, + }); + assert.deepEqual( + (window.channels['my_channel'] ?? []).map((m) => String(m.timetoken)), + got, + 'the three messages are exactly the oldest ones of the channel', + ); + }); + + it('historyNewerThanTimetoken: history with reverse and start returns only messages newer than the anchor, and start is exclusive', async () => { + const [, middle, newest] = await seed('my_channel', 3); + await untilStored('my_channel', [newest]); + const pubnub = withPlaceholders(shopper(), { '13406746780720711': middle }); + + /* eslint-disable prettier/prettier, max-len -- sample text is copied as readers see it */ + + // snippet.historyNewerThanTimetoken + try { + const result = await pubnub.history({ + channel: "my_channel", + reverse: true, // Setting to true will traverse the time line in reverse starting with the oldest message first. + stringifiedTimeToken: true, // false is the default + start: "13406746780720711", // start timetoken to fetch + }); + // snippet.hide + capturedResult = result; + // snippet.show + } catch (status) { + console.log(status); + } + // snippet.end + /* eslint-enable prettier/prettier, max-len */ + assert.ok(capturedResult, 'the history call returned a result (its catch only logs)'); + const result = capturedResult; + + const tokens = tokensOf(result); + assert.equal(tokens[0], newest, 'start is exclusive: the first message is the one after the anchor'); + assert.ok(!tokens.includes(middle), 'the anchor itself is not returned'); + assert.ok(tokens.every((t) => BigInt(t) > BigInt(middle))); + assert.ok(isAscending(tokens)); + }); + + it('historyUntilTimetoken: history with only end returns messages down to the anchor, and end is inclusive', async () => { + const [oldest, middle, newest] = await seed('my_channel', 3); + await untilStored('my_channel', [newest]); + const pubnub = withPlaceholders(shopper(), { '13406746780720711': middle }); + + /* eslint-disable prettier/prettier, max-len -- sample text is copied as readers see it */ + + // snippet.historyUntilTimetoken + try { + const result = await pubnub.history({ + channel: "my_channel", + stringifiedTimeToken: true, // false is the default + end: "13406746780720711", // end timetoken to fetch + }); + // snippet.hide + capturedResult = result; + // snippet.show + } catch (status) { + console.log(status); + } + // snippet.end + /* eslint-enable prettier/prettier, max-len */ + assert.ok(capturedResult, 'the history call returned a result (its catch only logs)'); + const result = capturedResult; + + const tokens = tokensOf(result); + assert.equal(tokens[0], middle, 'end is inclusive: the anchor is the oldest returned message'); + assert.ok(tokens.includes(newest)); + assert.ok(!tokens.includes(oldest), 'older than the anchor is not returned'); + assert.ok(isAscending(tokens)); + }); + + it('historyPagingExample: the paging helper of the page collects all 120 messages across two pages', async function () { + this.timeout(240_000); + const pubnub = withRetry(shopper()); + // Records every page the sample reads, so a count mismatch can say which message went missing and how the pages looked. + const pages: string[] = []; + const historyOfSample = pubnub.history.bind(pubnub) as ( + ...args: unknown[] + ) => Promise; + (pubnub as unknown as { history: unknown }).history = async (...args: unknown[]) => { + const page = await historyOfSample(...args); + const first = page.messages[0]?.timetoken; + const last = page.messages[page.messages.length - 1]?.timetoken; + pages.push( + `${page.messages.length} messages (${first}..${last}), startTimeToken ${page.startTimeToken}, endTimeToken ${page.endTimeToken}`, + ); + return page; + }; + // The sample walks the whole channel and cannot be bounded, so start from an empty channel: clear everything older than + // now (an upper bound, so a message published afterwards is never caught by a late delete) and wait until it is empty. + // Both bounds are needed: observed on the open keyset, a delete with only `end` is accepted ({}) but removes nothing, so + // the previous run's 120 messages stayed and the wait below timed out (6 of 13 runs). With `start` too it clears within seconds. + const now = (await observer().time()).timetoken; + const deleteBound = now; + await shopper().deleteMessages({ channel: 'history_test', start: '1', end: now }); + await eventually( + async () => { + const probe = await observer().fetchMessages({ channels: ['history_test'], count: 1 }); + assert.equal(probe.channels['history_test']?.length ?? 0, 0, 'history_test is empty'); + }, + { timeout: 60_000, label: 'history_test cleared' }, + ); + // More than one page of 100 messages, so the loop has to page. Observed: two concurrent publishes can come back with the SAME + // timetoken, and the service then stores only one of them (a run stored 119 of 120, and the 120 returned tokens held a + // duplicate). So the seeding publishes in parallel for speed, then replaces every message whose timetoken repeats until 120 + // distinct timetokens are stored, and records each collision so the log shows it. + // A publish that fails transiently is repeated (seedPublish), and a failed attempt may still have been stored. The repeat + // returns its own timetoken, so such a message would make 121 on the channel and break the exact count below. Every + // payload carries the run id, and settleSeeded deletes stored copies of this run whose timetoken is not in `tokens`. + const seedRun = newSeedRun(); + const tokens: string[] = []; + const collisions: string[] = []; + let nextN = 0; + while (tokens.length < 120) { + const wanted = Math.min(20, 120 - tokens.length); + const sent = await Promise.all( + Array.from({ length: wanted }, () => { + const n = nextN++; + return seedPublish(seedRun, 'seed history_test publish', () => + friend().publish({ channel: 'history_test', message: { legacy: 'paging', n, run: seedRun.id } }), + ); + }), + ); + for (const { timetoken } of sent) { + if (tokens.includes(timetoken)) collisions.push(timetoken); + else tokens.push(timetoken); + } + } + if (collisions.length > 0) + console.log( + `history_test seeding: replaced ${collisions.length} publish(es) with a repeated timetoken: ${collisions}`, + ); + // Wait for all 120 seeded messages, not only the newest 100. The oldest 20 sit on the second page. + await untilAllStored('history_test', tokens); + // Exactly 120 must be stored before the sample runs, however many publishes were repeated. + await settleSeeded('history_test', tokens, seedRun); + + /* eslint-disable prettier/prettier, max-len -- sample text is copied as readers see it */ + + // snippet.historyPagingExample + async function getAllMessages(initialTimetoken = '') { + const allMessages = []; + let latestCount = 100; + let timetoken = initialTimetoken; + + while (latestCount === 100) { + const { messages, startTimeToken, endTimeToken } = await pubnub.history( + { + channel: 'history_test', + stringifiedTimeToken: true, // false is the default + start: timetoken, // start timetoken to fetch + } + ); + + latestCount = messages.length; + timetoken = String(startTimeToken); + + if (messages && messages.length > 0) { + allMessages.push(...messages); + } + } + + return allMessages; + } + + // Usage examples: + // await getAllMessages(); + // await getAllMessages("12345678901234"); + // snippet.end + /* eslint-enable prettier/prettier, max-len */ + + // History is eventually consistent, so wait until the oldest messages are stored too. On a failure the message names the + // seeded timetokens the sample did not return and whether a tight fetchMessages window finds each of them, which tells a + // paging defect of the sample (the message is stored) from a storage defect (it is not). + let lastFound: Array<{ timetoken: string | number }> = []; + let all: Array<{ timetoken: string | number }>; + try { + all = await eventually( + async () => { + pages.length = 0; + const found = await getAllMessages(); + lastFound = found; + assert.equal(found.length, 120, 'paging returns every message of the cleared channel'); + return found; + }, + { timeout: 45_000, label: 'paging collects all seeded messages' }, + ); + } catch (error) { + const returned = new Set(lastFound.map((m) => String(m.timetoken))); + const missing = tokens.filter((t) => !returned.has(t)); + const lines: string[] = []; + for (const token of missing) { + const stored = await storedMessage(observer(), 'history_test', token).catch(() => undefined); + lines.push( + `${token} (seeded #${tokens.indexOf(token)}): fetchMessages ${stored ? 'FINDS it' : 'does NOT find it'}`, + ); + } + const sorted = [...tokens].sort((a, b) => (BigInt(a) < BigInt(b) ? -1 : 1)); + const unexpected = lastFound.filter((m) => !tokens.includes(String(m.timetoken))).length; + throw new Error( + `${(error as Error).message}\nsample returned ${lastFound.length} (${unexpected} not seeded), missing: ${lines.join('; ') || 'none'}\n` + + `${collisions.length} repeated publish timetokens replaced during seeding, delete bound ${deleteBound}, first seeded ${sorted[0]}, last seeded ${sorted[sorted.length - 1]}, seeded in publish order #0 ${tokens[0]}\n` + + `pages of the last attempt: ${pages.join(' | ')}`, + ); + } + assert.deepEqual(new Set(all.map((m) => m.timetoken)), new Set(tokens), 'each seeded message exactly once'); + // A valid timetoken starts the walk there: nothing newer than it is returned. + const sortedTokens = [...tokens].sort((a, b) => (BigInt(a) < BigInt(b) ? -1 : 1)); + const fromMiddle = await getAllMessages(sortedTokens[59]); + assert.equal(fromMiddle.length, 59, 'start is exclusive, so only the 59 messages before the 60th are returned'); + }); + + it('historyIncludeMeta: includeMeta returns meta for a message published with metadata and omits it for others', async () => { + const pubnub = shopper(); + const [withMeta] = await seed('my_channel', 1, { meta: { legacy: 'meta' } }); + const [plain] = await seed('my_channel', 1); + await untilStored('my_channel', [withMeta, plain]); + + /* eslint-disable prettier/prettier, max-len -- sample text is copied as readers see it */ + + // snippet.historyIncludeMeta + try { + const result = await pubnub.history({ + channel: "my_channel", + stringifiedTimeToken: true, + includeMeta: true, + }); + // snippet.hide + capturedResult = result; + // snippet.show + } catch (status) { + console.log(status); + } + // snippet.end + /* eslint-enable prettier/prettier, max-len */ + assert.ok(capturedResult, 'the history call returned a result (its catch only logs)'); + const result = capturedResult; + + const found = result.messages.find((m) => m.timetoken === withMeta); + assert.ok(found, 'the seeded message is returned'); + assert.deepEqual(found.meta, { legacy: 'meta' }); + // The meta property is present only when the publisher set metadata. + const without = result.messages.find((m) => m.timetoken === plain); + assert.ok(without, 'the message without metadata is returned'); + assert.equal(without.meta, undefined, 'a message without metadata has no meta property'); + }); + + it('historyWithPromises: the promise form of history logs the response with the expected messages', async () => { + const tokens = await seed('history_channel', 3); + await untilStored('history_channel', tokens); + // The docs literals are 1970 timetokens. The live values are the newest boundary (start) and the oldest one (end), so + // the window holds exactly the three seeded messages. + const pubnub = withPlaceholders(shopper(), { + '123123123133': addToTimetoken(tokens[2], 1), + '123123123123': tokens[0], + }); + + const logs = captureConsole('log'); + try { + /* eslint-disable prettier/prettier, max-len -- sample text is copied as readers see it */ + // snippet.historyWithPromises + pubnub.history({ + channel: 'history_channel', + reverse: true, // Setting to true will traverse the time line in reverse starting with the oldest message first. + count: 100, // how many items to fetch + stringifiedTimeToken: true, // false is the default + start: '123123123133', // start timetoken to fetch + end: '123123123123', // end timetoken to fetch + }).then((response) => { + console.log(response); + }).catch((error) => { + console.log(error); + }); + // snippet.end + /* eslint-enable prettier/prettier, max-len */ + + await eventually( + () => { + assert.ok( + logs.lines.some((line) => line.includes('"messages"')), + `the response was logged. Logged: ${logs.lines.join(' | ').slice(0, 300)}`, + ); + }, + { label: 'promise resolved and logged' }, + ); + const response = JSON.parse(logs.lines.find((line) => line.includes('"messages"'))!); + assert.deepEqual( + response.messages.map((m: { timetoken: string }) => String(m.timetoken)), + tokens, + ); + } finally { + logs.restore(); + } + }); + + it('historyDocumentedBehavior: the prose claims of the history page hold: the cap of 100, ordering, returned fields and timetoken types', async function () { + this.timeout(180_000); + const pubnub = shopper(); + const channel = 'history_documented_behavior'; + // The claims need a channel with more than 100 messages. It is written once and then reused by later runs. + const probeCount = async (): Promise => + (await observer().fetchMessages({ channels: [channel], count: 100 })).channels[channel]?.length ?? 0; + if ((await probeCount()) < 100) { + for (let batch = 0; batch < 5; batch += 1) { + // Setup with retry. No de-duplication: the claims need at least 100 messages, and an extra stored copy changes none of them. + await Promise.all( + Array.from({ length: 21 }, (_, i) => + retryTransient(() => friend().publish({ channel, message: { legacy: 'documented', n: batch * 21 + i } }), { + label: `seed ${channel} publish`, + }), + ), + ); + } + await eventually(async () => assert.equal(await probeCount(), 100), { + timeout: 45_000, + label: 'channel holds 100 messages', + }); + } + const probe = (parameters: Partial[0]>) => + pubnub.history({ channel, stringifiedTimeToken: true, ...parameters }); + + // "The maximum is 100, and history() lowers a larger value to 100." + assert.equal((await probe({ count: 150 })).messages.length, 100); + + // "history() always returns the messages sorted from the oldest to the newest, whatever reverse is." + const newestFive = await probe({ count: 5 }); + const oldestFive = await probe({ count: 5, reverse: true }); + for (const result of [newestFive, oldestFive]) assert.ok(isAscending(tokensOf(result))); + // reverse decides which end of the interval the retrieval starts from. + assert.ok( + BigInt(oldestFive.messages[4].timetoken) < BigInt(newestFive.messages[0].timetoken), + 'reverse starts from the oldest end', + ); + + // Returns: startTimeToken and endTimeToken hold the timetokens of the first and last returned message, as strings. + assert.equal(newestFive.startTimeToken, newestFive.messages[0].timetoken); + assert.equal(newestFive.endTimeToken, newestFive.messages[4].timetoken); + assert.equal(typeof newestFive.startTimeToken, 'string'); + for (const message of newestFive.messages) { + assert.deepEqual( + Object.keys(message).sort(), + ['entry', 'timetoken'], + 'an item has entry and timetoken, and meta only when set', + ); + } + + // Without stringifiedTimeToken the timetokens are numbers. + const plain = await pubnub.history({ channel, count: 1 }); + assert.equal(typeof plain.messages[0].timetoken, 'number'); + + // "With start, start must be a higher timetoken than end." The docs sample passes start below end. Record what the service does with that order. + const older = String(oldestFive.messages[0].timetoken); + const newer = String(newestFive.messages[4].timetoken); + const inverted = await probe({ count: 100, start: older, end: newer }); + const ordered = await probe({ count: 100, start: newer, end: older }); + console.log( + `history with start below end returned ${inverted.messages.length} messages, with start above end ${ordered.messages.length}`, + ); + assert.ok(ordered.messages.length > 0, 'start above end returns the messages in between'); + }); + + it('historyActionsDataDeprecated: the deprecated data property of actions equals actions, and both are absent without includeMessageActions', async () => { + const pubnub = shopper(); + const channel = 'persist_channel'; + const [token] = await seed(channel, 1); + await friend().addMessageAction({ + channel, + messageTimetoken: token, + action: { type: 'reaction', value: 'smiley_face' }, + }); + type WithActions = { + timetoken: string | number; + actions?: Record>; + data?: unknown; + }; + const found = await eventually( + async () => { + const response = await pubnub.fetchMessages({ channels: [channel], includeMessageActions: true }); + const item = response.channels[channel]?.find((m) => String(m.timetoken) === token) as WithActions | undefined; + assert.ok(item?.actions, 'the message has actions'); + return item; + }, + { timeout: 30_000, label: 'message with action stored' }, + ); + assert.deepEqual(found.data, found.actions, 'the deprecated data property holds the same value as actions'); + assert.ok(found.actions?.reaction?.smiley_face, 'grouped by action type, then by action value'); + + // "The response has them only with includeMessageActions and when the message has actions." + const without = await pubnub.fetchMessages({ channels: [channel] }); + const plainItem = without.channels[channel]?.find((m) => String(m.timetoken) === token) as WithActions | undefined; + assert.ok(plainItem, 'the message is returned without actions'); + assert.equal(plainItem.actions, undefined); + assert.equal(plainItem.data, undefined); + }); +}); diff --git a/test/integration/live/api-reference/message-persistence.test.ts b/test/integration/live/api-reference/message-persistence.test.ts new file mode 100644 index 000000000..85258c930 --- /dev/null +++ b/test/integration/live/api-reference/message-persistence.test.ts @@ -0,0 +1,246 @@ +import assert from 'node:assert/strict'; +import type PubNub from 'pubnub'; +import { addToTimetoken } from '../support/assertions'; +import { friend, observer, shopper } from '../support/clients'; +import { retryTransient } from '../support/retry'; +import { withPlaceholders } from '../support/placeholders'; +import { newSeedRun, seedPublish, settleSeeded, untilStored } from '../support/history'; +import { eventually } from '../support/wait'; + +/** + * Message persistence examples. Port of docs-snippets/api-reference/message-persistence.ts. Message Persistence and + * Delete-From-History must be on for the open keyset. + * + * The samples carry placeholder literals (`replace-with-start-timetoken` and so on). The client is wrapped with + * `withPlaceholders`, so the sample text stays unchanged and the real timetokens reach the service. Channels are shared and + * persistent: every assertion is bounded to a timetoken window that holds only what this test published. + * Residue: seeded messages (customMessageType `seed`) stay on `my-channel`, `chats.room1`, `chats.room2`, `ch2`, `ch3` + * and `my_channel` (one message with meta and a reaction). `ch1` messages are deleted by the delete tests. + */ +describe('api-reference/message-persistence', () => { + // Publishes `count` messages as the friend and returns their timetokens, oldest first. Setup, so a transient publish failure + // is repeated (seedPublish); settleSeeded then removes any stored copy a repeat left behind, so the windows below hold exactly + // the returned messages. + const seed = async (channel: string, count: number): Promise => { + const run = newSeedRun(); + const tokens: string[] = []; + for (let i = 0; i < count; i += 1) { + const { timetoken } = await seedPublish(run, `seed ${channel} publish`, () => + friend().publish({ channel, message: { seed: i, run: run.id }, customMessageType: 'seed' }), + ); + tokens.push(timetoken); + } + await settleSeeded(channel, tokens, run); + return tokens; + }; + + type WithActions = { actions?: Record> }; + + it('fetchMessagesBasicUsage: fetchMessages with count, start and end returns one message of the window', async () => { + const tokens = await seed('my-channel', 3); + await untilStored('my-channel', tokens); + // `start` is the exclusive newer bound and `end` the inclusive older bound: the window holds exactly the three messages. + const pubnub = withPlaceholders(shopper(), { + 'replace-with-start-timetoken': addToTimetoken(tokens[2], 1), + 'replace-with-end-timetoken': tokens[0], + }); + + // snippet.fetchMessagesBasicUsage + // Function to fetch message history + const result = await pubnub.fetchMessages({ + channels: ['my-channel'], + count: 1, // Number of messages to retrieve + includeCustomMessageType: true, // if you want to include custom message type in the response + start: 'replace-with-start-timetoken', // start timetoken + end: 'replace-with-end-timetoken', // end timetoken + }); + console.log('Fetched Messages:', result); + // snippet.end + const messages = result.channels['my-channel'] ?? []; + assert.equal(messages.length, 1, 'count: 1 returns one message'); + assert.ok(tokens.includes(String(messages[0].timetoken)), 'the message is one of the seeded ones'); + assert.equal(messages[0].customMessageType, 'seed'); + }); + + it('fetchMessagesWithMetaActions: fetchMessages returns meta, message actions and the custom message type when asked for', async () => { + const pubnub = shopper(); + // Setup, so the publish is repeated on a transient failure. A stored copy of a repeated publish is harmless: the test reads + // its message by timetoken. The action is not repeated (a second add could count twice). + const { timetoken } = await retryTransient( + () => + friend().publish({ + channel: 'my_channel', + message: { text: 'with meta and an action' }, + meta: { source: 'docs-snippets' }, + customMessageType: 'text-message', + }), + { label: 'seed my_channel publish' }, + ); + await friend().addMessageAction({ + channel: 'my_channel', + messageTimetoken: timetoken, + action: { type: 'reaction', value: 'smiley_face' }, + }); + await eventually( + async () => { + const probe = await observer().fetchMessages({ + channels: ['my_channel'], + includeMessageActions: true, + includeMeta: true, + }); + const found = probe.channels['my_channel']?.find((m) => String(m.timetoken) === timetoken); + assert.ok((found as WithActions | undefined)?.actions, 'message and its action are stored'); + }, + { timeout: 30_000, label: 'message with action stored' }, + ); + + // snippet.fetchMessagesWithMetaActions + const response = await pubnub.fetchMessages({ + channels: ['my_channel'], + stringifiedTimeToken: true, + includeMeta: true, + includeMessageActions: true, + includeCustomMessageType: true, + }); + console.log('fetch messages response:', response); + // snippet.end + // The sample asks for the newest messages of a shared channel, so look up this run's message by its timetoken. + const found = response.channels['my_channel']?.find((m) => String(m.timetoken) === timetoken); + assert.ok(found, 'the seeded message is returned'); + assert.deepEqual(found.meta, { source: 'docs-snippets' }); + assert.equal(found.customMessageType, 'text-message'); + assert.ok((found as WithActions).actions?.reaction?.smiley_face, 'the reaction is attached to the message'); + }); + + it('messageCountBasicUsage: messageCounts with one timetoken counts the messages per channel since that time', async () => { + const before = (await observer().time()).timetoken; + const first = await seed('chats.room1', 2); + const second = await seed('chats.room2', 1); + await untilStored('chats.room1', first); + await untilStored('chats.room2', second); + const pubnub = withPlaceholders(shopper(), { 'replace-with-timetoken': before }); + + // snippet.messageCountBasicUsage + const result = await pubnub.messageCounts({ + channels: ['chats.room1', 'chats.room2'], + channelTimetokens: ['replace-with-timetoken'], // single timetoken applied to all channels + }); + console.log('Message counts retrieved successfully:', result); + // snippet.end + // The count covers the window since `before`, which holds this test's messages. The channels are shared, so allow more + // than were seeded, but never fewer. + assert.ok( + result.channels['chats.room1'] >= 2, + `chats.room1 counts the 2 seeded messages (got ${result.channels['chats.room1']})`, + ); + assert.ok( + result.channels['chats.room2'] >= 1, + `chats.room2 counts the seeded message (got ${result.channels['chats.room2']})`, + ); + }); + + it('messageCountTimetokensForChannels: messageCounts with one timetoken per channel gives the per-channel counts', async () => { + const channels = ['ch1', 'ch2', 'ch3']; + // One timetoken per channel, taken just before that channel gets 1, 2 and 3 messages. + const froms: string[] = []; + for (const [index, channel] of channels.entries()) { + froms.push((await observer().time()).timetoken); + await untilStored(channel, await seed(channel, index + 1)); + } + const pubnub = withPlaceholders(shopper(), { + 'replace-with-channel-timetoken-ch1': froms[0], + 'replace-with-channel-timetoken-ch2': froms[1], + 'replace-with-channel-timetoken-ch3': froms[2], + }); + + // snippet.messageCountTimetokensForChannels + const response = await pubnub.messageCounts({ + channels: ['ch1', 'ch2', 'ch3'], + channelTimetokens: [ + 'replace-with-channel-timetoken-ch1', // timetoken for channel ch1 + 'replace-with-channel-timetoken-ch2', // timetoken for channel ch2 + 'replace-with-channel-timetoken-ch3', // timetoken for channel ch3 + ], + }); + console.log('message count response:', response); + // snippet.end + // Each window starts just before this test's own messages. Shared channels may hold more, never fewer. + channels.forEach((channel, index) => { + assert.ok( + response.channels[channel] >= index + 1, + `${channel} counts its ${index + 1} seeded messages (got ${response.channels[channel]})`, + ); + }); + }); + + // `start` is exclusive and `end` inclusive, so start = T - 1 and end = T delete exactly the message at T. + const deleteOne = async (): Promise<{ token: string; placeholders: Record }> => { + const [token] = await seed('ch1', 1); + await untilStored('ch1', [token]); + return { + token, + placeholders: { + 'replace-with-start-timetoken': addToTimetoken(token, -1), + 'replace-with-end-timetoken': token, + }, + }; + }; + + // Waits until a deleted message is gone for readers, deletion is also eventually consistent. + const untilDeleted = (channel: string, token: string): Promise => + eventually( + async () => { + const history = await observer().fetchMessages({ channels: [channel], count: 100 }); + assert.ok( + !(history.channels[channel] ?? []).some((m) => String(m.timetoken) === token), + 'message is still stored', + ); + }, + { timeout: 30_000, label: 'message deleted' }, + ); + + it('deleteMessagesBasicUsage: deleteMessages removes exactly the message at the boundary timetoken', async () => { + const { token, placeholders } = await deleteOne(); + // A neighbour published right after must survive the delete of `token`. + const [neighbour] = await seed('ch1', 1); + await untilStored('ch1', [neighbour]); + const pubnub = withPlaceholders(shopper(), placeholders); + + // snippet.deleteMessagesBasicUsage + const result = await pubnub.deleteMessages({ + channel: 'ch1', + start: 'replace-with-start-timetoken', + end: 'replace-with-end-timetoken', + }); + console.log('Messages deleted successfully:', result); + // snippet.end + await untilDeleted('ch1', token); + const left = await observer().fetchMessages({ channels: ['ch1'], count: 100 }); + assert.ok( + left.channels['ch1']?.some((m) => String(m.timetoken) === neighbour), + 'the neighbouring message is kept', + ); + }); + + it('deleteSpecificMessages: the second deleteMessages sample removes exactly the message at its boundary timetoken', async () => { + const { token, placeholders } = await deleteOne(); + const [neighbour] = await seed('ch1', 1); + await untilStored('ch1', [neighbour]); + const pubnub = withPlaceholders(shopper(), placeholders); + + // snippet.deleteSpecificMessages + const response = await pubnub.deleteMessages({ + channel: 'ch1', + start: 'replace-with-start-timetoken', + end: 'replace-with-end-timetoken', + }); + console.log('delete messages response:', response); + // snippet.end + await untilDeleted('ch1', token); + const left = await observer().fetchMessages({ channels: ['ch1'], count: 100 }); + assert.ok( + left.channels['ch1']?.some((m) => String(m.timetoken) === neighbour), + 'the neighbouring message is kept', + ); + }); +}); diff --git a/test/integration/live/api-reference/misc.test.ts b/test/integration/live/api-reference/misc.test.ts new file mode 100644 index 000000000..7dff1e118 --- /dev/null +++ b/test/integration/live/api-reference/misc.test.ts @@ -0,0 +1,94 @@ +import assert from 'node:assert/strict'; +import { randomUUID } from 'node:crypto'; +import { assertServerTime } from '../support/assertions'; +import { friend, shopper } from '../support/clients'; +import { LocalProxy } from '../support/proxy'; +import { expectStatus, sleep, watch } from '../support/wait'; + +/** + * Network examples: `disconnect()`, `reconnect()` and `setProxy()`. Port of docs-snippets/api-reference/misc.ts. + * The samples name no channel, so each test listens on a channel of its own name. + */ +describe('api-reference/misc', () => { + const proxy = new LocalProxy(); + after(() => proxy.stop()); + + it('disconnectBasicUsage: disconnect() reports a disconnected status and no message arrives afterwards', async () => { + const pubnub = shopper(); + const run = randomUUID(); + const channel = `misc_disconnect_${run}`; + const listening = await watch(pubnub, [channel]); + const disconnected = expectStatus(pubnub, ['PNDisconnectedCategory']); + + // snippet.disconnectBasicUsage + pubnub.disconnect(); + // snippet.end + + assert.equal(await disconnected, 'PNDisconnectedCategory'); + // While disconnected the client receives nothing. A fixed wait is right here: the test proves that nothing arrived. + await friend().publish({ channel, message: { while: 'disconnected', run } }); + await sleep(3000); + assert.ok( + !listening.events.some((e) => (e.event as { message?: { while?: string } }).message?.while === 'disconnected'), + 'no message arrives while disconnected', + ); + }); + + it('reconnectBasicUsage: reconnect() restores delivery after a disconnect', async () => { + const pubnub = shopper(); + const run = randomUUID(); + const channel = `misc_reconnect_${run}`; + const listening = await watch(pubnub, [channel]); + const disconnected = expectStatus(pubnub, ['PNDisconnectedCategory']); + pubnub.disconnect(); + await disconnected; + const back = expectStatus(pubnub, ['PNReconnectedCategory', 'PNConnectedCategory']); + + // snippet.reconnectBasicUsage + pubnub.reconnect(); + // snippet.end + + await back; + await friend().publish({ channel, message: { after: 'reconnect', run } }); + await listening.next('message', (m) => (m.message as { after?: string }).after === 'reconnect'); + }); + + it('setProxyBasicUsage: setProxy routes the client traffic through the proxy, seen as a tunnel request', async () => { + await proxy.start(); + const pubnub = shopper(); + + // snippet.setProxyBasicUsage + // This method is only available for NodeJS. + pubnub.setProxy({ + hostname: 'YOUR_HOSTNAME', + port: 8080, + protocol: 'YOUR_PROTOCOL', + // snippet.hide + ...proxy.config, + // snippet.show + }); + // snippet.end + + const tunnelsBefore = proxy.connects; + assertServerTime((await pubnub.time()).timetoken); + assert.ok(proxy.connects > tunnelsBefore, 'the request went through the local proxy'); + }); + + it('deleteProxy: setProxy with no argument removes the proxy so the next request adds no tunnel', async () => { + await proxy.start(); + const pubnub = shopper(); + // Set up by hand what setProxyBasicUsage does, so this test does not depend on it. + pubnub.setProxy(proxy.config); + const tunnelsBefore = proxy.connects; + assertServerTime((await pubnub.time()).timetoken); + assert.ok(proxy.connects > tunnelsBefore, 'precondition: the request went through the local proxy'); + + // snippet.deleteProxy + pubnub.setProxy(); + // snippet.end + + const tunnelsAfterDelete = proxy.connects; + assertServerTime((await pubnub.time()).timetoken); + assert.equal(proxy.connects, tunnelsAfterDelete, 'no request goes through the proxy any more'); + }); +}); diff --git a/test/integration/live/api-reference/mobile-push.test.ts b/test/integration/live/api-reference/mobile-push.test.ts new file mode 100644 index 000000000..4c15fa8ed --- /dev/null +++ b/test/integration/live/api-reference/mobile-push.test.ts @@ -0,0 +1,363 @@ +import assert from 'node:assert/strict'; +import PubNub from 'pubnub'; +import { observer, friend, shopper } from '../support/clients'; +import { retryTransient } from '../support/retry'; +import { captureConsole } from '../support/console'; +import { eventually, watch } from '../support/wait'; +import { sorted } from '../support/util'; + +/** + * Mobile push notification examples. Port of docs-snippets/api-reference/mobile-push.ts. + * Registering a device token does not send a push. Delivery needs a real device and APNs or FCM credentials. + * Needs the Push Notifications add-on on the open keyset (no APNs or FCM credentials are needed). + * + * The device token of the docs is shared and persistent, so a hook brings it to a known state (every registration removed) + * before each test that asserts exact channel lists (reset approach). + */ +describe('api-reference/mobile-push', () => { + // Registration order is not guaranteed by the service, so every list comparison sorts first. + const DEVICE = '740f4707bebcf74f9b7c25d48e3358945f6aa01da5ddb387462c7eaf61bb78ad'; + const apns2Options = { pushGateway: 'apns2', environment: 'production', topic: 'com.example.bundle_id' } as const; + + // The service text for "add-on not enabled". Anything else is a real failure. + const PUSH_OFF = /requires Push Notifications which is not enabled/i; + + /** Channels of the device on both gateways, read through a given client. */ + async function registrations(pubnub: PubNub, device = DEVICE): Promise<{ apns2: string[]; fcm: string[] }> { + const apns2 = await pubnub.push.listChannels({ device, ...apns2Options }); + const fcm = await pubnub.push.listChannels({ device, pushGateway: 'fcm' }); + return { apns2: sorted(apns2.channels), fcm: sorted(fcm.channels) }; + } + + /** Removes every registration of the device on both gateways and waits until another client sees none. */ + async function resetDevice(): Promise { + const client = friend(); + try { + await client.push.deleteDevice({ device: DEVICE, ...apns2Options }); + await client.push.deleteDevice({ device: DEVICE, pushGateway: 'fcm' }); + } catch (error) { + const status = (error as { status?: { statusCode?: unknown; errorData?: unknown } } | undefined)?.status; + const detail = `${error instanceof Error ? error.message : String(error)} ${JSON.stringify(status?.errorData ?? '')}`; + if (PUSH_OFF.test(detail)) { + throw new Error( + `Push Notifications is not enabled for the open keyset. Enable it in the Admin Portal, no APNs or FCM credentials are needed. ${detail}`, + ); + } + throw error; + } + const reader = observer(); + await eventually(async () => assert.deepEqual(await registrations(reader), { apns2: [], fcm: [] }), { + label: 'device has no registrations', + timeout: 30_000, + }); + } + + /** Resets the device, registers `channels` on both gateways and waits until another client lists them. */ + async function seedDevice(channels: string[]): Promise { + await resetDevice(); + const client = friend(); + await retryTransient(() => client.push.addChannels({ channels, device: DEVICE, ...apns2Options }), { + label: 'seed push registration', + }); + await retryTransient(() => client.push.addChannels({ channels, device: DEVICE, pushGateway: 'fcm' }), { + label: 'seed push registration', + }); + const reader = observer(); + await eventually( + async () => assert.deepEqual(await registrations(reader), { apns2: sorted(channels), fcm: sorted(channels) }), + { label: 'device registrations visible', timeout: 30_000 }, + ); + } + + after(async () => { + // The samples register the device. Leave it without registrations. + await resetDevice(); + }); + + it('addDeciveToChannelBasicUsage: addChannels registers a device token for a channel and another client reads the registration back', async function () { + this.timeout(60_000); + const pubnub = shopper(); + // Reset approach: the device starts without registrations, so the lists below are exactly the channels of the sample. + await resetDevice(); + + // snippet.addDeciveToChannelBasicUsage + // Function to add a device to a channel for APNs2 + const response = await pubnub.push.addChannels({ + channels: ['a', 'b'], + device: '740f4707bebcf74f9b7c25d48e3358945f6aa01da5ddb387462c7eaf61bb78ad', + pushGateway: 'apns2', + environment: 'production', + topic: 'com.example.bundle_id', + }); + console.log('device added to channels response:', response); + + // Function to add a device to a channel for FCM + const response2 = await pubnub.push.addChannels({ + channels: ['a', 'b'], + device: '740f4707bebcf74f9b7c25d48e3358945f6aa01da5ddb387462c7eaf61bb78ad', + pushGateway: 'fcm', + }); + console.log('device added to channels response:', response2); + // snippet.end + + const reader = observer(); + await eventually( + async () => assert.deepEqual(await registrations(reader), { apns2: ['a', 'b'], fcm: ['a', 'b'] }), + { + label: 'registrations visible', + timeout: 30_000, + }, + ); + }); + + it('listChannelsForDeviceBasicUsage: listChannels returns the channels registered for the device', async function () { + this.timeout(60_000); + const pubnub = shopper(); + // Reset approach: register exactly a and b on both gateways first. + await seedDevice(['a', 'b']); + + // snippet.listChannelsForDeviceBasicUsage + // for APNs2 + const response = await pubnub.push.listChannels({ + device: '740f4707bebcf74f9b7c25d48e3358945f6aa01da5ddb387462c7eaf61bb78ad', + pushGateway: 'apns2', + environment: 'production', + topic: 'com.example.bundle_id', + }); + console.log('listing channels for device response:', response); + response.channels.forEach((channel) => { + console.log(channel); + }); + + // for FCM + const response2 = await pubnub.push.listChannels({ + device: '740f4707bebcf74f9b7c25d48e3358945f6aa01da5ddb387462c7eaf61bb78ad', + pushGateway: 'fcm', + }); + + console.log('listing channels for device response:', response2); + + response2.channels.forEach((channel) => { + console.log(channel); + }); + // snippet.end + + assert.deepEqual(sorted(response.channels), ['a', 'b'], 'apns2 list'); + assert.deepEqual(sorted(response2.channels), ['a', 'b'], 'fcm list'); + }); + + it('removeDeviceFromChannelBasicUsage: removeChannels unregisters the device and both gateways list nothing afterwards', async function () { + this.timeout(60_000); + const pubnub = shopper(); + await seedDevice(['a', 'b']); + + // snippet.removeDeviceFromChannelBasicUsage + // for APNs2 + const response = await pubnub.push.removeChannels({ + channels: ['a', 'b'], + device: '740f4707bebcf74f9b7c25d48e3358945f6aa01da5ddb387462c7eaf61bb78ad', + pushGateway: 'apns2', + environment: 'production', + topic: 'com.example.bundle_id', + }); + + console.log('removing device from channel response:', response); + + // for FCM + const response2 = await pubnub.push.removeChannels({ + channels: ['a', 'b'], + device: '740f4707bebcf74f9b7c25d48e3358945f6aa01da5ddb387462c7eaf61bb78ad', + pushGateway: 'fcm', + }); + + console.log('removing device from channel response:', response2); + // snippet.end + + // Read-back: after removeChannels a fresh list per gateway (other client) must be empty. + const reader = observer(); + await eventually(async () => assert.deepEqual(await registrations(reader), { apns2: [], fcm: [] }), { + label: 'registrations removed', + timeout: 30_000, + }); + }); + + it('removeAllMobilePushNotificationsBasicUsage: deleteDevice removes every registration of the device and the list is empty afterwards', async function () { + this.timeout(60_000); + const pubnub = shopper(); + await seedDevice(['a']); + + // snippet.removeAllMobilePushNotificationsBasicUsage + + // for APNs2 + const response = await pubnub.push.deleteDevice({ + device: '740f4707bebcf74f9b7c25d48e3358945f6aa01da5ddb387462c7eaf61bb78ad', + pushGateway: 'apns2', + environment: 'production', + topic: 'com.example.bundle_id', + }); + + console.log('deleteDevice response:', response); + + // for FCM + const response2 = await pubnub.push.deleteDevice({ + device: '740f4707bebcf74f9b7c25d48e3358945f6aa01da5ddb387462c7eaf61bb78ad', + pushGateway: 'fcm', + }); + + console.log('deleteDevice response:', response2); + + // snippet.end + + // Read-back: deleteDevice must leave no channel on either gateway. + const reader = observer(); + await eventually(async () => assert.deepEqual(await registrations(reader), { apns2: [], fcm: [] }), { + label: 'device deleted', + timeout: 30_000, + }); + }); + + // Replaces the old check pushDeviceTokenFormat: it pins which device token formats each gateway accepts. The docs show + // the token 'niceDevice', which only fcm accepts. + it('pushDeviceTokenFormat: apns and apns2 reject the non-hex docs token with 400 and fcm registers it', async function () { + this.timeout(60_000); + const client = shopper(); + const reader = observer(); + const docsToken = 'niceDevice'; + const fcmOptions = { device: docsToken, pushGateway: 'fcm' } as const; + // Reset approach for this token as well. + await client.push.deleteDevice(fcmOptions); + + for (const [label, options] of [ + ['apns', { pushGateway: 'apns' }], + ['apns2', apns2Options], + ] as const) { + let rejected: unknown; + try { + await retryTransient(() => client.push.addChannels({ channels: ['a'], device: docsToken, ...options }), { + label: 'seed push registration', + }); + } catch (error) { + rejected = error; + } + assert.ok(rejected, `${label} accepted the docs token "${docsToken}", the docs bug is gone, update the docs`); + const status = (rejected as { status?: { statusCode?: unknown; errorData?: unknown } }).status; + const detail = `${rejected instanceof Error ? rejected.message : String(rejected)} status=${String( + status?.statusCode ?? '', + )} body=${JSON.stringify(status?.errorData ?? '')}`; + assert.match(detail, /status=400/, `${label}: expected status 400, got ${detail}`); + assert.match(detail, /Invalid device token/i, `${label}: expected "Invalid device token", got ${detail}`); + } + + try { + await retryTransient(() => client.push.addChannels({ channels: ['a'], ...fcmOptions }), { + label: 'seed push registration', + }); + await eventually(async () => assert.deepEqual((await reader.push.listChannels(fcmOptions)).channels, ['a']), { + label: 'fcm registered a non-hex token', + timeout: 30_000, + }); + } finally { + await client.push.deleteDevice(fcmOptions).catch(() => undefined); + } + }); + + it('buildNotificationPayloadBasicUsage: the notification payload builder output is published and a subscriber receives the message', async () => { + const pubnub = shopper(); + const listening = await watch(observer(), ['chat-bot']); + + // snippet.buildNotificationPayloadBasicUsage + + const builder = PubNub.notificationPayload('Chat invitation', "You have been invited to 'quiz' chat"); + builder.apns.configurations = [{ targets: [{ topic: 'com.example.bundle_id' }] }]; + const messagePayload = builder.buildPayload(['apns2', 'fcm']); + // add required fields to the payload + const response = await pubnub.publish({ + message: messagePayload, + channel: 'chat-bot', + }); + console.log('publish response:', response); + // snippet.end + + assert.match(String(response.timetoken), /^\d{17}$/); + assert.ok(messagePayload.pn_apns && messagePayload.pn_fcm, 'the payload carries the APNs and FCM sections'); + // The channel is shared, so match this run's own message by timetoken. + const received = await listening.next('message', (m) => m.timetoken === String(response.timetoken)); + assert.ok(received.message, 'subscribers get the message'); + listening.stop(); + }); + + // ---- payload builders, local --------------------------------------------------------------------------------------- + + it('simpleNotificationPayloadFCMandAPNS: the payload builder prints a payload with the FCM and APNs sections the sample describes', async () => { + const logs = captureConsole('log'); + try { + // snippet.simpleNotificationPayloadFCMandAPNS + const builder = PubNub.notificationPayload('Chat invitation', "You have been invited to 'quiz' chat"); + builder.sound = 'default'; + + console.log(JSON.stringify(builder.buildPayload(['apns', 'fcm']), null, 2)); + // snippet.end + } finally { + logs.restore(); + } + const alert = { title: 'Chat invitation', body: "You have been invited to 'quiz' chat" }; + assert.deepEqual(JSON.parse(logs.lines[0]), { + pn_apns: { aps: { alert, sound: 'default' } }, + pn_fcm: { notification: alert, android: { notification: { sound: 'default' } } }, + }); + }); + + it('simpleNotificationPayloadFCMandAPNSH/2: the payload builder produces the APNs HTTP/2 section as documented', async () => { + const logs = captureConsole('log'); + try { + // snippet.simpleNotificationPayloadFCMandAPNSH/2 + const payloadBuilder = PubNub.notificationPayload('Chat invitation', "You have been invited to 'quiz' chat"); + payloadBuilder.apns.configurations = [{ targets: [{ topic: 'com.meetings.chat.app' }] }]; + payloadBuilder.sound = 'default'; + + console.log(JSON.stringify(payloadBuilder.buildPayload(['apns2', 'fcm']), null, 2)); + // snippet.end + } finally { + logs.restore(); + } + const payload = JSON.parse(logs.lines[0]); + assert.equal(payload.pn_apns.pn_push[0].targets[0].topic, 'com.meetings.chat.app'); + assert.equal(payload.pn_apns.pn_push[0].version, 'v2'); + assert.equal(payload.pn_apns.aps.sound, 'default'); + assert.equal(payload.pn_fcm.android.notification.sound, 'default'); + }); + + // Checks that collapseId and expirationDate reach the payload in the snake case wire format. + it('simpleNotificationPayloadFCMandAPNSH/2CustomConfiguration: collapseId and expirationDate reach the payload in the wire format', async () => { + const logs = captureConsole('log'); + // The sample builds its own configuration, so check the expiration against the time window around the run. + const startedAt = Date.now(); + try { + // snippet.simpleNotificationPayloadFCMandAPNSH/2CustomConfiguration + const configuration = [ + { + collapseId: 'invitations', + expirationDate: new Date(Date.now() + 10000), + targets: [{ topic: 'com.meetings.chat.app' }], + }, + ]; + const customConfigurationBuilder = PubNub.notificationPayload( + 'Chat invitation', + "You have been invited to 'quiz' chat", + ); + customConfigurationBuilder.apns.configurations = configuration; + + console.log(JSON.stringify(customConfigurationBuilder.buildPayload(['apns2', 'fcm']), null, 2)); + // snippet.end + } finally { + logs.restore(); + } + const payload = JSON.parse(logs.lines[0]); + assert.equal(payload.pn_apns.pn_push[0].collapse_id, 'invitations'); + const expiration = Date.parse(payload.pn_apns.pn_push[0].expiration); + assert.ok( + expiration >= startedAt + 10000 && expiration <= Date.now() + 10000, + `expiration ${expiration} is 10 s ahead`, + ); + }); +}); diff --git a/test/integration/live/api-reference/objects.test.ts b/test/integration/live/api-reference/objects.test.ts new file mode 100644 index 000000000..a4970ae01 --- /dev/null +++ b/test/integration/live/api-reference/objects.test.ts @@ -0,0 +1,726 @@ +import assert from 'node:assert/strict'; +import type PubNub from 'pubnub'; +import { friend, observer, shopper } from '../support/clients'; +import { retryTransient } from '../support/retry'; +import { captureConsole } from '../support/console'; +import { eventually, watch } from '../support/wait'; +import { sorted } from '../support/util'; + +/** + * App Context examples. Port of docs-snippets/api-reference/objects.ts. + * Needs App Context on the open keyset. Users, channels, memberships and members are shared literal names, so every test + * resets the specific resources its sample works on first (reset approach) and then asserts exact lists. The `after` hook + * removes what the samples leave behind. + */ +describe('api-reference/objects', () => { + // Sorted ids, so list assertions do not depend on service ordering. + const ids = (list: Array<{ id: string }>): string[] => list.map((o) => o.id).sort(); + + const SHOPPER = 'shopper'; + + /** Removes every membership of the user and waits until another client sees none. */ + async function clearMemberships(uuid: string): Promise { + const admin = friend(); + const current = await admin.objects.getMemberships({ uuid, limit: 100 }); + if (current.data.length > 0) { + await admin.objects.removeMemberships({ uuid, channels: current.data.map((m) => m.channel.id) }); + } + const reader = observer(); + await eventually(async () => assert.deepEqual((await reader.objects.getMemberships({ uuid })).data, []), { + label: `memberships of ${uuid} cleared`, + }); + } + + /** Removes every member of the channel and waits until another client sees none. */ + async function clearMembers(channel: string): Promise { + const admin = friend(); + const current = await admin.objects.getChannelMembers({ channel, limit: 100 }); + if (current.data.length > 0) { + await admin.objects.removeChannelMembers({ channel, uuids: current.data.map((m) => m.uuid.id) }); + } + const reader = observer(); + await eventually(async () => assert.deepEqual((await reader.objects.getChannelMembers({ channel })).data, []), { + label: `members of ${channel} cleared`, + }); + } + + /** Creates or overwrites the channel `team.blue` that the docs read (its name matches the `*Team` filter). */ + async function seedTeamBlue(pubnub: PubNub): Promise { + await retryTransient( + () => + pubnub.objects.setChannelMetadata({ + channel: 'team.blue', + data: { name: 'Blue Team', description: 'The channel for Blue team.' }, + }), + { label: 'seed app context' }, + ); + const reader = observer(); + await eventually( + async () => + assert.equal((await reader.objects.getChannelMetadata({ channel: 'team.blue' })).data.name, 'Blue Team'), + { + label: 'team.blue visible', + }, + ); + } + + const ignoreMissing = (promise: Promise): Promise => promise.catch(() => undefined); + + before(async () => { + // Preflight: the open keyset must have App Context enabled. + try { + await observer().objects.getAllUUIDMetadata({ limit: 1 }); + } catch (error) { + const detail = error instanceof Error ? error.message : String(error); + throw new Error(`The open keyset does not have App Context. Enable "App Context" for it. ${detail}`); + } + }); + + after(async () => { + const admin = friend(); + for (const uuid of [SHOPPER, 'myUuid', 'my-uuid']) { + await ignoreMissing(clearMemberships(uuid)); + await ignoreMissing(admin.objects.removeUUIDMetadata({ uuid })); + } + for (const channel of ['myChannel']) await ignoreMissing(clearMembers(channel)); + for (const uuid of ['uuid-1', 'uuid-2', 'uuid-3']) await ignoreMissing(admin.objects.removeUUIDMetadata({ uuid })); + for (const channel of [ + 'team.blue', + 'team.red', + 'myChannel', + 'my-channel', + 'channel-with-status-type', + 'ch-1', + 'ch-2', + ]) { + await ignoreMissing(admin.objects.removeChannelMetadata({ channel })); + } + }); + + // ---- users ---------------------------------------------------------------------------------------------------- + + it('setUUIDMetadataBasicUsage: setUUIDMetadata stores the user and another client reads it back', async () => { + const pubnub = shopper(); + + // snippet.setUUIDMetadataBasicUsage + // Using UUID from the config - default when uuid is not passed in the method + const response = await pubnub.objects.setUUIDMetadata({ + data: { + name: 'John Doe', + }, + }); + console.log('setUUIDMetadata response:', response); + + // Using the passed in UUID + const response2 = await pubnub.objects.setUUIDMetadata({ + uuid: 'myUuid', + data: { + email: 'john.doe@example.com', + }, + }); + console.log('setUUIDMetadata response:', response2); + // snippet.end + + assert.equal(response.data.id, pubnub.getUserId()); + assert.equal(response.data.name, 'John Doe'); + assert.equal(response2.data.id, 'myUuid'); + assert.equal(response2.data.email, 'john.doe@example.com'); + const reader = observer(); + await eventually( + async () => { + const stored = await reader.objects.getUUIDMetadata({ uuid: 'myUuid' }); + assert.equal(stored.data.email, 'john.doe@example.com'); + }, + { label: 'stored user visible to another client' }, + ); + }); + + it('getUUIDMetadataBasicUsage: getUUIDMetadata reads the users stored beforehand', async () => { + const pubnub = shopper(); + const admin = friend(); + // Store the two users the sample reads (own data, so the exact values are legitimate). + await retryTransient(() => admin.objects.setUUIDMetadata({ uuid: SHOPPER, data: { name: 'John Doe' } }), { + label: 'seed app context', + }); + await retryTransient( + () => admin.objects.setUUIDMetadata({ uuid: 'myUuid', data: { email: 'john.doe@example.com' } }), + { label: 'seed app context' }, + ); + const reader = observer(); + await eventually( + async () => { + assert.equal((await reader.objects.getUUIDMetadata({ uuid: SHOPPER })).data.name, 'John Doe'); + assert.equal((await reader.objects.getUUIDMetadata({ uuid: 'myUuid' })).data.email, 'john.doe@example.com'); + }, + { label: 'users visible' }, + ); + + // snippet.getUUIDMetadataBasicUsage + // Using UUID from the config - default when uuid is not passed in the method + const response = await pubnub.objects.getUUIDMetadata(); + console.log('getUUIDMetadata response:', response); + + // Using the passed in UUID + const response2 = await pubnub.objects.getUUIDMetadata({ + uuid: 'myUuid', + }); + console.log('getUUIDMetadata response:', response2); + // snippet.end + + assert.equal(response.data.id, pubnub.getUserId()); + assert.equal(response.data.name, 'John Doe'); + assert.equal(response2.data.id, 'myUuid'); + assert.equal(response2.data.email, 'john.doe@example.com'); + }); + + it('getAllUUIDMetadataBasicUsage: getAllUUIDMetadata runs the sample function and logs the user list', async () => { + const pubnub = shopper(); + // The sample says to add user metadata first. + await retryTransient( + () => friend().objects.setUUIDMetadata({ uuid: 'myUuid', data: { email: 'john.doe@example.com' } }), + { label: 'seed app context' }, + ); + const logs = captureConsole('log'); + try { + // snippet.getAllUUIDMetadataBasicUsage + // Function to get all UUID metadata + // to get some data in response, add user metadata using setUUIDMetadata method + async function getAllUUIDMetadata() { + const response = await pubnub.objects.getAllUUIDMetadata(); + console.log('getAllUUIDMetadata response:', response); + } + + // Execute the function to get UUID metadata + await getAllUUIDMetadata(); + // snippet.end + } finally { + logs.restore(); + } + assert.ok( + logs.lines.some((line) => line.startsWith('getAllUUIDMetadata response:')), + 'the function logged its response', + ); + // The list is shared and can be long: look at the 100 most recently updated users, where the one just written is. + await eventually( + async () => { + const listed = await pubnub.objects.getAllUUIDMetadata({ limit: 100, sort: { updated: 'desc' } }); + assert.ok( + listed.data.some((u) => u.id === 'myUuid'), + 'the stored user is part of the list', + ); + }, + { label: 'stored user listed' }, + ); + }); + + it('removeUUIDMetadataBasicUsage: removeUUIDMetadata deletes the users and reads fail with not found afterwards', async () => { + const pubnub = shopper(); + const admin = friend(); + await retryTransient(() => admin.objects.setUUIDMetadata({ uuid: SHOPPER, data: { name: 'John Doe' } }), { + label: 'seed app context', + }); + await retryTransient( + () => admin.objects.setUUIDMetadata({ uuid: 'myUuid', data: { email: 'john.doe@example.com' } }), + { label: 'seed app context' }, + ); + + // snippet.removeUUIDMetadataBasicUsage + // Using UUID from the config - default when uuid is not passed in the method + const response = await pubnub.objects.removeUUIDMetadata(); + + // Using the passed in UUID + const response2 = await pubnub.objects.removeUUIDMetadata({ + uuid: 'myUuid', + }); + // snippet.end + + assert.equal(response.status, 200); + assert.equal(response2.status, 200); + const reader = observer(); + await eventually( + async () => { + for (const uuid of [pubnub.getUserId(), 'myUuid']) { + await assert.rejects(reader.objects.getUUIDMetadata({ uuid }), `${uuid} is gone`); + } + }, + { label: 'users removed' }, + ); + }); + + // ---- channels ------------------------------------------------------------------------------------------------- + + // The `*Team` filter needs a channel named `... Team`, which team.blue provides with the name `Blue Team`. + it('getAllChannelMetadataBasicUsage: getAllChannelMetadata with the Team filter returns the seeded channel', async () => { + const pubnub = shopper(); + await seedTeamBlue(friend()); + + // snippet.getAllChannelMetadataBasicUsage + // Get the total number of channels included in the response. + const response = await pubnub.objects.getAllChannelMetadata({ + include: { + totalCount: true, + }, + }); + + // Get all channels with the filter option. To get all channel which has Id ending 'Team'. + const response2 = await pubnub.objects.getAllChannelMetadata({ + filter: 'name LIKE "*Team"', + }); + console.log('Get all channel metadata response:', response2); + // snippet.end + + assert.ok((response.totalCount ?? 0) >= 1, 'totalCount is part of the response'); + assert.ok(response2.data.length >= 1, 'the filter finds the team channels'); + assert.ok( + response2.data.every((c) => c.name?.endsWith('Team')), + 'every match ends with Team', + ); + assert.ok(ids(response2.data).includes('team.blue'), 'team.blue is in the filtered list'); + }); + + it('getChannelMetadataBasicUsage: getChannelMetadata reads the seeded channel by id', async () => { + const pubnub = shopper(); + await seedTeamBlue(friend()); + + // snippet.getChannelMetadataBasicUsage + const response = await pubnub.objects.getChannelMetadata({ + // `channel` is the `id` in the _metadata_, not `name` + channel: 'team.blue', + }); + console.log('Get channel metadata response:', response); + // snippet.end + + assert.equal(response.data.id, 'team.blue'); + assert.equal(response.data.name, 'Blue Team'); + }); + + it('setChannelMetadataBasicUsage: setChannelMetadata stores the channel, an observer gets the set event and the hidden custom data was stored', async () => { + const pubnub = shopper(); + // Channel scoped objects events arrive on a subscription to the channel itself. + const listening = await watch(observer(), ['team.red']); + + // snippet.setChannelMetadataBasicUsage + const response = await pubnub.objects.setChannelMetadata({ + channel: 'team.red', + data: { + name: 'Red Team', + description: 'The channel for Red team and no other teams.', + custom: { + owner: 'Red Leader', + }, + }, + include: { + customFields: false, + }, + }); + console.log('Set channel metadata response:', response); + // snippet.end + + assert.equal(response.data.id, 'team.red'); + assert.equal(response.data.name, 'Red Team'); + assert.equal(response.data.custom, undefined, 'customFields: false leaves custom out of the response'); + const reader = observer(); + await eventually( + async () => { + const stored = await reader.objects.getChannelMetadata({ + channel: 'team.red', + include: { customFields: true }, + }); + assert.deepEqual(stored.data.custom, { owner: 'Red Leader' }); + }, + { label: 'stored channel custom data visible to another client' }, + ); + // The sample's own write is the newest set event for the channel, and its description is unique to this sample. + const event = await listening.next( + 'objects', + (e) => e.channel === 'team.red' && e.message.event === 'set' && e.message.type === 'channel', + ); + assert.equal(event.message.event, 'set'); + assert.equal(event.message.type, 'channel'); + listening.stop(); + }); + + it('iterativelyUpdateExistingMetadata: the read-modify-write pattern merges custom data and the merged object survives on the service', async () => { + const pubnub = shopper(); + + // snippet.iterativelyUpdateExistingMetadata + const channel = 'team.red'; + const name = 'Red Team'; + const description = 'The channel for Red team.'; + const customField = { visible: 'team' }; + + // Function to set and then update channel metadata + const response = await pubnub.objects.setChannelMetadata({ + channel: channel, + data: { + name: name, + description: description, + custom: customField, + }, + }); + console.log('The channel has been created with name and description.\n'); + + // Fetch current object with custom fields + const currentObjectResponse = await pubnub.objects.getChannelMetadata({ + channel: channel, + include: { + customFields: true, + }, + }); + const currentObject = currentObjectResponse.data; + + // Initialize the custom field object + const custom = currentObject.custom || {}; + + // Add or update the field + custom['edit'] = 'admin'; + + // Writing the updated object back to the server + const setResponse = await pubnub.objects.setChannelMetadata({ + channel: channel, + data: { + name: currentObject.name || '', + description: currentObject.description || '', + custom: custom, + }, + }); + console.log('Object has been updated', setResponse); + // snippet.end + + assert.equal(response.status, 200); + // The sample's first write replaces the whole custom object, so the final state is exact whatever was there before. + const reader = observer(); + await eventually( + async () => { + const stored = await reader.objects.getChannelMetadata({ + channel: 'team.red', + include: { customFields: true }, + }); + assert.deepEqual(stored.data.custom, { visible: 'team', edit: 'admin' }); + assert.equal(stored.data.description, 'The channel for Red team.'); + }, + { label: 'updated channel metadata visible to another client' }, + ); + }); + + it('removeChannelMetadataBasicUsage: removeChannelMetadata deletes the channel and reads fail afterwards', async () => { + const pubnub = shopper(); + await retryTransient( + () => friend().objects.setChannelMetadata({ channel: 'team.red', data: { name: 'Red Team' } }), + { label: 'seed app context' }, + ); + + // snippet.removeChannelMetadataBasicUsage + const response = await pubnub.objects.removeChannelMetadata({ + channel: 'team.red', + }); + // snippet.end + + assert.equal(response.status, 200); + const reader = observer(); + await eventually( + async () => { + await assert.rejects(reader.objects.getChannelMetadata({ channel: 'team.red' })); + }, + { label: 'team.red removed' }, + ); + }); + + // ---- memberships ---------------------------------------------------------------------------------------------- + + it('setMembershipBasicUsage: setMemberships stores a membership, and a second read with include flags shows the type and custom data', async () => { + const pubnub = shopper(); + // Reset approach: both users start without memberships, so the response lists are exactly the two channels. + await clearMemberships(SHOPPER); + await clearMemberships('my-uuid'); + + // snippet.setMembershipBasicUsage + // Using UUID from the config + const response = await pubnub.objects.setMemberships({ + channels: [ + 'my-channel', + { id: 'channel-with-status-type', custom: { hello: 'World' }, status: 'helloStatus', type: 'helloType' }, + ], + }); + console.log('Set memberships response:', response); + + // Using the passed in UUID + const response2 = await pubnub.objects.setMemberships({ + uuid: 'my-uuid', + channels: [ + 'my-channel', + { id: 'channel-with-status-type', custom: { hello: 'World' }, status: 'helloStatus', type: 'helloType' }, + ], + include: { + // To include channel fields in response + channelFields: true, + }, + }); + console.log('Set memberships response:', response2); + // snippet.end + + const expected = ['channel-with-status-type', 'my-channel']; + assert.deepEqual(response.data.map((m) => m.channel.id).sort(), expected); + assert.deepEqual(response2.data.map((m) => m.channel.id).sort(), expected); + assert.equal(response.data.find((m) => m.channel.id === 'channel-with-status-type')?.status, 'helloStatus'); + const reader = observer(); + await eventually( + async () => { + const stored = await reader.objects.getMemberships({ + uuid: 'my-uuid', + include: { typeField: true, statusField: true, customFields: true }, + }); + assert.deepEqual(stored.data.map((m) => m.channel.id).sort(), expected); + const special = stored.data.find((m) => m.channel.id === 'channel-with-status-type'); + assert.equal(special?.status, 'helloStatus'); + assert.equal(special?.type, 'helloType'); + assert.deepEqual(special?.custom, { hello: 'World' }); + }, + { label: 'stored memberships visible to another client' }, + ); + }); + + it('getMembershipBasicUsage: getMemberships with the starred filter returns exactly the one starred membership', async () => { + const pubnub = shopper(); + const admin = friend(); + // Reset approach, then seed: `myUuid` has one starred membership, the shopper has the same two channels. + await clearMemberships(SHOPPER); + await clearMemberships('myUuid'); + await retryTransient( + () => + admin.objects.setMemberships({ + uuid: 'myUuid', + channels: [ + { id: 'my-channel', custom: { starred: true } }, + { id: 'channel-with-status-type', custom: { starred: false } }, + ], + }), + { label: 'seed app context' }, + ); + await retryTransient( + () => admin.objects.setMemberships({ uuid: SHOPPER, channels: ['my-channel', 'channel-with-status-type'] }), + { label: 'seed app context' }, + ); + const reader = observer(); + await eventually( + async () => { + assert.equal((await reader.objects.getMemberships({ uuid: 'myUuid' })).data.length, 2); + assert.equal((await reader.objects.getMemberships({ uuid: SHOPPER })).data.length, 2); + }, + { label: 'memberships seeded' }, + ); + + // snippet.getMembershipBasicUsage + // Using UUID from the config + const response = await pubnub.objects.getMemberships(); + + // Using the passed in UUID + const response2 = await pubnub.objects.getMemberships({ + uuid: 'myUuid', + include: { + channelFields: true, + }, + }); + + // Get all memberships that are starred by the user + const response3 = await pubnub.objects.getMemberships({ + uuid: 'myUuid', + filter: 'custom.starred == true', + }); + // snippet.end + + assert.deepEqual(response.data.map((m) => m.channel.id).sort(), ['channel-with-status-type', 'my-channel']); + assert.equal(response2.data.length, 2); + assert.ok( + response2.data.every((m) => m.channel.id), + 'channel ids are part of the response', + ); + assert.deepEqual( + response3.data.map((m) => m.channel.id), + ['my-channel'], + ); + }); + + it('removeMembershipsBasicUsage: removeMemberships removes the named memberships and keeps the others', async () => { + const pubnub = shopper(); + const admin = friend(); + // Reset approach, then seed three memberships per user. + for (const uuid of [SHOPPER, 'myUuid']) { + await clearMemberships(uuid); + await retryTransient(() => admin.objects.setMemberships({ uuid, channels: ['ch-1', 'ch-2', 'my-channel'] }), { + label: 'seed app context', + }); + } + const reader = observer(); + await eventually( + async () => { + for (const uuid of [SHOPPER, 'myUuid']) { + assert.equal((await reader.objects.getMemberships({ uuid })).data.length, 3); + } + }, + { label: 'memberships seeded' }, + ); + + // snippet.removeMembershipsBasicUsage + // Using UUID from the config + const response = await pubnub.objects.removeMemberships({ + channels: ['ch-1', 'ch-2'], + }); + + // Using the passed in UUID + const response2 = await pubnub.objects.removeMemberships({ + uuid: 'myUuid', + channels: ['ch-1', 'ch-2'], + }); + // snippet.end + + for (const result of [response, response2]) { + assert.deepEqual( + result.data.map((m) => m.channel.id), + ['my-channel'], + 'only my-channel stays', + ); + } + await eventually( + async () => { + const left = await reader.objects.getMemberships({ uuid: 'myUuid' }); + assert.deepEqual( + left.data.map((m) => m.channel.id), + ['my-channel'], + ); + }, + { label: 'memberships removed' }, + ); + }); + + // ---- channel members ------------------------------------------------------------------------------------------ + + /** Stores the two users of the member samples, so the `name` filter and the user fields have something to match. */ + async function seedUsers(): Promise { + const admin = friend(); + await retryTransient( + () => admin.objects.setUUIDMetadata({ uuid: 'uuid-1', data: { name: 'Member One', custom: { role: 'Member' } } }), + { label: 'seed app context' }, + ); + await retryTransient(() => admin.objects.setUUIDMetadata({ uuid: 'uuid-2', data: { name: 'Team admin' } }), { + label: 'seed app context', + }); + } + + it('getChannelMembersBasicUsage: getChannelMembers with the admin filter returns only the matching member', async () => { + const pubnub = shopper(); + // Reset approach: myChannel starts without members, then gets exactly uuid-1 and uuid-2. + await clearMembers('myChannel'); + await seedUsers(); + await retryTransient( + () => friend().objects.setChannelMembers({ channel: 'myChannel', uuids: ['uuid-1', 'uuid-2'] }), + { label: 'seed app context' }, + ); + const reader = observer(); + await eventually( + async () => { + assert.equal((await reader.objects.getChannelMembers({ channel: 'myChannel' })).data.length, 2); + }, + { label: 'members seeded' }, + ); + + // snippet.getChannelMembersBasicUsage + const response = await pubnub.objects.getChannelMembers({ + channel: 'myChannel', + include: { + UUIDFields: true, + }, + }); + console.log('getChannelMembers response:', response); + + // Get all channel members with "admin" in the user name + const response2 = await pubnub.objects.getChannelMembers({ + channel: 'myChannel', + filter: 'uuid.name LIKE "*admin*"', + }); + console.log('getChannelMembers response:', response2); + // snippet.end + + assert.deepEqual(response.data.map((m) => m.uuid.id).sort(), ['uuid-1', 'uuid-2']); + assert.equal( + (response.data.find((m) => m.uuid.id === 'uuid-1')?.uuid as { name?: string } | undefined)?.name, + 'Member One', + ); + assert.deepEqual( + response2.data.map((m) => m.uuid.id), + ['uuid-2'], + 'the name filter keeps the admin only', + ); + }); + + it('setChannelMembersBasicUsage: setChannelMembers adds a member with a custom role and another client reads it back', async () => { + const pubnub = shopper(); + await clearMembers('myChannel'); + await seedUsers(); + + // snippet.setChannelMembersBasicUsage + const response = await pubnub.objects.setChannelMembers({ + channel: 'myChannel', + uuids: ['uuid-1', 'uuid-2', { id: 'uuid-3', custom: { role: 'Super Admin' } }], + }); + console.log('setChannelMembers response:', response); + // snippet.end + + assert.deepEqual(response.data.map((m) => m.uuid.id).sort(), ['uuid-1', 'uuid-2', 'uuid-3']); + const reader = observer(); + await eventually( + async () => { + const stored = await reader.objects.getChannelMembers({ + channel: 'myChannel', + include: { customFields: true }, + }); + assert.deepEqual(stored.data.find((m) => m.uuid.id === 'uuid-3')?.custom, { role: 'Super Admin' }); + assert.deepEqual(sorted(stored.data.map((m) => m.uuid.id)), ['uuid-1', 'uuid-2', 'uuid-3']); + }, + { label: 'stored channel members visible to another client' }, + ); + }); + + it('removeChannelMembersBasicUsage: removeChannelMembers removes members by id and leaves the others', async () => { + const pubnub = shopper(); + await clearMembers('myChannel'); + await seedUsers(); + await retryTransient( + () => + friend().objects.setChannelMembers({ + channel: 'myChannel', + uuids: ['uuid-1', 'uuid-2', { id: 'uuid-3', custom: { role: 'Super Admin' } }], + }), + { label: 'seed app context' }, + ); + const reader = observer(); + await eventually( + async () => { + assert.equal((await reader.objects.getChannelMembers({ channel: 'myChannel' })).data.length, 3); + }, + { label: 'members seeded' }, + ); + + // snippet.removeChannelMembersBasicUsage + const response = await pubnub.objects.removeChannelMembers({ + channel: 'myChannel', + uuids: ['uuid-1', 'uuid-2'], + }); + // snippet.end + + assert.deepEqual( + response.data.map((m) => m.uuid.id), + ['uuid-3'], + ); + await eventually( + async () => { + const left = await reader.objects.getChannelMembers({ channel: 'myChannel' }); + assert.deepEqual( + left.data.map((m) => m.uuid.id), + ['uuid-3'], + ); + }, + { label: 'members removed' }, + ); + }); +}); diff --git a/test/integration/live/api-reference/presence.test.ts b/test/integration/live/api-reference/presence.test.ts new file mode 100644 index 000000000..95f084dc8 --- /dev/null +++ b/test/integration/live/api-reference/presence.test.ts @@ -0,0 +1,325 @@ +import assert from 'node:assert/strict'; +import PubNub from 'pubnub'; +import { friend, observer } from '../support/clients'; +import { keysOf } from '../support/env'; +import { connected, eventually, PRESENCE_TIMEOUT } from '../support/wait'; +import { sorted } from '../support/util'; + +/** + * Presence examples. Port of docs-snippets/api-reference/presence.ts. + * + * The samples run as the user id `uuid` on the channels `ch1`, `my_channel` and the channel group `cg1`. One occupant client + * joins them once in `before` and leaves in `after`, because joining is the slow part of presence. It is built by hand + * (not through `createClient`) so the per-test client cleanup does not destroy it between tests. Channels and groups are + * shared and persistent, so the assertions look for the occupant `uuid` in a list, never for exact occupancy. + * Residue: channel groups `cg1` and `my_channel_group` (both hold the channel `cg1_member`). + */ +describe('api-reference/presence', () => { + let occupant: PubNub; + const here = (query: Parameters[0]) => observer().hereNow(query); + + before(async function () { + this.timeout(120_000); + // Two groups share one member channel: `cg1` is subscribed by the occupant, `my_channel_group` is only read. + for (const channelGroup of ['cg1', 'my_channel_group']) { + await friend().channelGroups.addChannels({ channelGroup, channels: ['cg1_member'] }); + } + const { publishKey, subscribeKey } = keysOf('open'); + // Short presence timeout, so a client that dies without leaving does not linger for the default 300 s. + occupant = new PubNub({ publishKey, subscribeKey, userId: 'uuid', presenceTimeout: 20, heartbeatInterval: 8 }); + const set = occupant.subscriptionSet({ channels: ['ch1', 'my_channel'], channelGroups: ['cg1'] }); + const ready = connected(occupant, { channels: ['ch1', 'my_channel'], channelGroups: ['cg1'] }); + set.subscribe(); + await ready; + await occupant.setState({ state: { status: 'online' }, channels: ['my_channel', 'ch1'] }); + }); + + after(() => { + occupant?.unsubscribeAll(); + occupant?.destroy(); + }); + + it('fixture: the occupant is visible on all its channels, through both channel groups, with its state', async function () { + this.timeout(120_000); + const channels = ['ch1', 'my_channel', 'cg1_member']; + await eventually( + async () => { + const result = await here({ channels }); + for (const channel of channels) { + assert.ok( + result.channels[channel]?.occupants.some((o) => o.uuid === 'uuid'), + `${channel} lists the occupant`, + ); + } + }, + { timeout: PRESENCE_TIMEOUT, label: 'occupant visible in hereNow' }, + ); + const reads: Array<[string, { channels?: string[]; channelGroups: string[] }]> = [ + ['cg1', { channelGroups: ['cg1'] }], + ['my_channel_group', { channelGroups: ['my_channel_group'] }], + ['ch1 and cg1', { channels: ['ch1'], channelGroups: ['cg1'] }], + ]; + for (const [label, query] of reads) { + await eventually( + async () => { + const result = await here(query); + assert.ok( + result.channels['cg1_member']?.occupants.some((o) => o.uuid === 'uuid'), + `${label} lists the occupant on cg1_member`, + ); + }, + { timeout: PRESENCE_TIMEOUT, label: `occupant visible through ${label}` }, + ); + } + await eventually( + async () => { + const result = await here({ channels: ['my_channel'], includeState: true }); + assert.deepEqual( + result.channels['my_channel']?.occupants.find((o) => o.uuid === 'uuid')?.state, + { status: 'online' }, + 'hereNow lists the state on the channel', + ); + const where = await observer().whereNow({ uuid: 'uuid' }); + assert.ok( + ['ch1', 'my_channel'].every((c) => where.channels.includes(c)), + 'whereNow lists both channels', + ); + }, + { timeout: PRESENCE_TIMEOUT, label: 'state and channel list visible' }, + ); + }); + + it('setStateBasicUsage: setState stores state that another client can read back', async () => { + const pubnub = occupant; + // Start from a different state, so the read-back below proves the sample changed it. + await pubnub.setState({ state: { status: 'offline' }, channels: ['ch1'] }); + await eventually( + async () => { + const stored = await observer().getState({ uuid: 'uuid', channels: ['ch1'] }); + assert.deepEqual(stored.channels['ch1'], { status: 'offline' }); + }, + { label: 'initial state visible' }, + ); + + // snippet.setStateBasicUsage + const response = await pubnub.setState({ + state: { status: 'online' }, + channels: ['ch1'], + channelGroups: ['cg1'], + }); + console.log('State set successfully:', response); + // snippet.end + assert.deepEqual(response.state, { status: 'online' }); + await eventually( + async () => { + const stored = await observer().getState({ uuid: 'uuid', channels: ['ch1'] }); + assert.deepEqual(stored.channels['ch1'], { status: 'online' }); + }, + { label: 'state visible to another client' }, + ); + // hereNowBasicUsage reads the same state through hereNow in one shot, so wait for that read path too. + await eventually( + async () => { + const result = await here({ channels: ['ch1'], includeState: true }); + assert.deepEqual( + result.channels['ch1']?.occupants.find((o) => o.uuid === 'uuid')?.state, + { status: 'online' }, + 'hereNow lists the state on ch1', + ); + }, + { label: 'state visible through hereNow' }, + ); + }); + + it('getStateBasicUsage: getState returns the state set for the channel and the group', async function () { + this.timeout(60_000); + const pubnub = occupant; + await pubnub.setState({ state: { status: 'online' }, channels: ['ch1'], channelGroups: ['cg1'] }); + + // snippet.getStateBasicUsage + const response = await pubnub.getState({ + uuid: 'uuid', + channels: ['ch1'], + channelGroups: ['cg1'], + }); + console.log('State retrieved successfully:', response); + // snippet.end + const nested = (response.channels as unknown as { channels?: Record }).channels; + assert.deepEqual(nested?.['ch1'], { status: 'online' }); + }); + + it('hereNowBasicUsage: hereNow lists the occupant with its state and the group member channel appears through the group', async function () { + this.timeout(60_000); + const pubnub = occupant; + await pubnub.setState({ state: { status: 'online' }, channels: ['ch1'] }); + await eventually( + async () => { + const result = await here({ channels: ['ch1'], includeState: true }); + assert.deepEqual(result.channels['ch1']?.occupants.find((o) => o.uuid === 'uuid')?.state, { status: 'online' }); + }, + { timeout: 30_000, label: 'state visible before the sample reads it' }, + ); + + // snippet.hereNowBasicUsage + // Function to get presence information for a channel + const result = await pubnub.hereNow({ + channels: ['ch1'], + channelGroups: ['cg1'], + includeUUIDs: true, + includeState: true, + }); + console.log('Here Now Result:', result); + // snippet.end + const entry = result.channels['ch1']?.occupants.find((o) => o.uuid === 'uuid'); + assert.ok(entry, 'ch1 lists the occupant'); + assert.deepEqual(entry.state, { status: 'online' }); + // The group member shows up a moment after the occupant joins it, so re-read until it does. + await eventually( + async () => { + const again = await pubnub.hereNow({ + channels: ['ch1'], + channelGroups: ['cg1'], + includeUUIDs: true, + includeState: true, + }); + assert.ok( + again.channels['cg1_member']?.occupants.some((o) => o.uuid === 'uuid'), + 'the channel group member is part of the result', + ); + }, + { timeout: 30_000, label: 'hereNow lists the channel group member' }, + ); + }); + + it('hereNowWithState: hereNow with includeState returns the state set in the fixture', async () => { + const pubnub = occupant; + + // snippet.hereNowWithState + const response = await pubnub.hereNow({ + channels: ['my_channel'], + includeState: true, + }); + console.log('hereNow response:', response); + // snippet.end + const entry = response.channels['my_channel']?.occupants.find((o) => o.uuid === 'uuid'); + assert.deepEqual(entry?.state, { status: 'online' }); + }); + + it('hereNowFetchOccupancyOnly: hereNow with includeUUIDs false gives a count and no user list', async () => { + const pubnub = occupant; + + // snippet.hereNowFetchOccupancyOnly + const response = await pubnub.hereNow({ + channels: ['my_channel'], + includeUUIDs: false, + }); + console.log('hereNow response:', response); + // snippet.end + const entry = response.channels['my_channel']; + assert.ok(entry.occupancy >= 1, 'occupancy counts the occupant'); + assert.equal(entry.occupants?.length ?? 0, 0, 'no user ids when includeUUIDs is false'); + }); + + it('hereNowChannelGroup: hereNow on a channel group reports presence for each member channel', async () => { + const pubnub = occupant; + + // snippet.hereNowChannelGroup + const response = await pubnub.hereNow({ + channelGroups: ['my_channel_group'], + }); + console.log('hereNow response:', response); + // snippet.end + const entry = response.channels['cg1_member']; + assert.ok(entry, 'the group member channel is in the result'); + assert.ok(entry.occupants.some((o) => o.uuid === 'uuid')); + }); + + it('basicUsageWithPromises: the promise form of hereNow resolves with the occupants, seen through a spy', async () => { + const pubnub = occupant; + await pubnub.setState({ state: { status: 'online' }, channels: ['ch1'] }); + // Spy: keep the promise the sample does not return. + const results: Array> = []; + const original = pubnub.hereNow.bind(pubnub); + pubnub.hereNow = ((...args: Parameters) => { + const result = original(...args); + results.push(result); + return result; + }) as PubNub['hereNow']; + + // snippet.basicUsageWithPromises + pubnub + .hereNow({ + channels: ['ch1'], + channelGroups: ['cg1'], + includeUUIDs: true, + includeState: true, + }) + .then((response) => { + console.log(response); + }) + .catch((error) => { + console.log(error); + }); + // snippet.end + assert.equal(results.length, 1, 'the snippet called hereNow once'); + const response = await results[0]; + assert.ok(response.channels['ch1']?.occupants.some((o) => o.uuid === 'uuid')); + }); + + it('whereNowBasicUsage: whereNow lists the channels the occupant is in', async () => { + const pubnub = observer(); + + // snippet.whereNowBasicUsage + const response = await pubnub.whereNow({ + uuid: 'uuid', + }); + console.log('whereNow response:', response); + // snippet.end + assert.deepEqual( + sorted(response.channels).filter((c) => ['ch1', 'my_channel'].includes(c)), + ['ch1', 'my_channel'], + ); + }); + + it('after the occupant unsubscribes the channel empties of it in hereNow', async function () { + this.timeout(90_000); + // Own user and a channel unique to this test, so it does not disturb the shared occupant of the other tests. + const channel = `presence-leave-${Date.now().toString(36)}`; + const { publishKey, subscribeKey } = keysOf('open'); + const leaver = new PubNub({ + publishKey, + subscribeKey, + userId: `leaver-${channel}`, + presenceTimeout: 20, + heartbeatInterval: 8, + }); + try { + const ready = connected(leaver, { channels: [channel] }); + leaver.subscribe({ channels: [channel] }); + await ready; + await eventually( + async () => { + const result = await here({ channels: [channel] }); + assert.ok( + result.channels[channel]?.occupants.some((o) => o.uuid === leaver.getUserId()), + 'occupant is listed', + ); + }, + { timeout: PRESENCE_TIMEOUT, label: 'occupant joined' }, + ); + leaver.unsubscribeAll(); + await eventually( + async () => { + const result = await here({ channels: [channel] }); + assert.ok( + !result.channels[channel]?.occupants.some((o) => o.uuid === leaver.getUserId()), + 'occupant is still listed', + ); + }, + { timeout: PRESENCE_TIMEOUT, label: 'occupant left' }, + ); + } finally { + leaver.destroy(); + } + }); +}); diff --git a/test/integration/live/api-reference/publish-legacy.test.ts b/test/integration/live/api-reference/publish-legacy.test.ts new file mode 100644 index 000000000..1f8c3c060 --- /dev/null +++ b/test/integration/live/api-reference/publish-legacy.test.ts @@ -0,0 +1,59 @@ +import assert from 'node:assert/strict'; +import PubNub from 'pubnub'; +import { assertTimetoken, rejectsValidation } from '../support/assertions'; +import { keysOf } from '../support/env'; +import { unique } from '../support/util'; +import { observer, shopper, track } from '../support/clients'; +import { sleep, watch } from '../support/wait'; + +/** + * Examples for the deprecated `fire()` method and its replacement. Port of docs-snippets/api-reference/publish-legacy.ts. + * The page has no snippet markers: every test is a behaviour check. Channels are unique per run (nothing is stored on them, + * and a unique name keeps the "nothing arrived" checks free of other runs' traffic). + */ +describe('api-reference/publish-legacy', () => { + it('publishLegacyFireResolvesUndefined: fire() resolves with undefined although the TypeScript type promises a response', async () => { + const response = await shopper().fire({ channel: unique('fire_undefined'), message: { legacy: 'fire' } }); + // The TypeScript signature promises a PublishResponse, the runtime value is undefined. + assert.equal(response, undefined); + }); + + it('publishLegacyFireDiscardsErrors: publish rejects with a validation error on a bad client while fire swallows it', async () => { + // A client without publishKey cannot publish. publish() reports it, fire() does not. + const noPublishKey = track( + new PubNub({ subscribeKey: keysOf('open').subscribeKey, userId: unique('fire-no-key') }), + ); + const channel = unique('fire_errors'); + await rejectsValidation(noPublishKey.publish({ channel, message: { legacy: 'publish' } }), /Missing 'publishKey'/); + assert.equal( + await noPublishKey.fire({ channel, message: { legacy: 'fire' } }), + undefined, + 'fire() resolves undefined and discards the error', + ); + }); + + it('publishLegacyReplacementDoesNotReplicateOrStore: the fire replacement call is neither delivered to subscribers nor stored in history', async function () { + this.timeout(45_000); + const channel = unique('fire_replacement'); + const listening = await watch(observer(), [channel]); + const response = await shopper().publish({ + channel, + message: { legacy: 'not-replicated' }, + storeInHistory: false, + replicate: false, + }); + assertTimetoken(response.timetoken); + // Proves that nothing arrived, so a fixed wait is right. + await sleep(4000); + assert.ok( + !listening.events.some( + (e) => (e.event as { message?: { legacy?: string } }).message?.legacy === 'not-replicated', + ), + 'with replicate: false the message does not reach subscribers', + ); + // The channel name is unique to this run, so any stored message here would be this one. + const history = await observer().fetchMessages({ channels: [channel], count: 25 }); + assert.equal(history.channels[channel]?.length ?? 0, 0, 'with storeInHistory: false nothing is stored'); + listening.stop(); + }); +}); diff --git a/test/integration/live/api-reference/publish.test.ts b/test/integration/live/api-reference/publish.test.ts new file mode 100644 index 000000000..ce5a262af --- /dev/null +++ b/test/integration/live/api-reference/publish.test.ts @@ -0,0 +1,255 @@ +import assert from 'node:assert/strict'; +import PubNub from 'pubnub'; +import { assertTimetoken, rejectsValidation, rejectsWithCategory } from '../support/assertions'; +import { createClient, observer, server, shopper, unreachable } from '../support/clients'; +import { captureConsole } from '../support/console'; +import { storedMessage } from '../support/history'; +import { eventually, sleep, watch } from '../support/wait'; + +/** + * Publish and signal examples, including `fire()`. Port of docs-snippets/api-reference/publish.ts. + * Message Persistence must be on for the open keyset (publishBasicUsage and the ttl test read history). + */ +describe('api-reference/publish', () => { + it('publishBasicUsage: publish delivers a message with meta and a custom message type to a subscriber and stores it', async () => { + const pubnub = shopper(); + const listening = await watch(observer(), ['my_channel']); + + // snippet.publishBasicUsage + const response = await pubnub.publish({ + message: { text: 'Hello World' }, + channel: 'my_channel', + sendByPost: false, + storeInHistory: true, + meta: { sender: 'user123' }, + customMessageType: 'text-message', + }); + console.log('Publish Success:', response); + // snippet.end + + assertTimetoken(response.timetoken); + // Match this run's own message by timetoken: the channel is shared and persistent. + const received = await listening.next('message', (m) => m.timetoken === response.timetoken); + assert.deepEqual(received.message, { text: 'Hello World' }); + assert.equal(received.publisher, pubnub.getUserId()); + assert.equal(received.userMetadata?.sender, 'user123'); + assert.equal(received.customMessageType, 'text-message'); + // storeInHistory: true, so Message Persistence has it too. + await eventually( + async () => { + assert.ok(await storedMessage(observer(), 'my_channel', response.timetoken), 'message is stored'); + }, + { label: 'published message is persisted' }, + ); + listening.stop(); + }); + + it('publishStoreThePublishedMessagefor10Hours: publish with a ttl still delivers and stores the message', async () => { + const pubnub = shopper(); + const listening = await watch(observer(), ['my_channel']); + + // snippet.publishStoreThePublishedMessagefor10Hours + const response = await pubnub.publish({ + message: 'hello!', + channel: 'my_channel', + storeInHistory: true, + ttl: 10, + customMessageType: 'text-message', + }); + + console.log('message published with server response:', response); + // snippet.end + + assertTimetoken(response.timetoken); + const received = await listening.next('message', (m) => m.timetoken === response.timetoken); + assert.equal(received.message, 'hello!'); + await eventually( + async () => { + assert.ok(await storedMessage(observer(), 'my_channel', response.timetoken), 'message is stored'); + }, + { label: 'message with ttl is persisted' }, + ); + listening.stop(); + }); + + it('publishMinimal: the minimal publish returns a string timetoken and delivers the message', async () => { + const pubnub = shopper(); + const listening = await watch(observer(), ['my_channel']); + + // snippet.publishMinimal + const { timetoken } = await pubnub.publish({ + channel: 'my_channel', + message: { text: 'Hello' }, + }); + + console.log(timetoken); // "17918851187695440" (a string) + // snippet.end + + assert.equal(typeof timetoken, 'string'); + assertTimetoken(timetoken); + const received = await listening.next('message', (m) => m.timetoken === timetoken); + assert.deepEqual(received.message, { text: 'Hello' }); + listening.stop(); + }); + + it('publishFalsyMessage: a wrapped value 0 is delivered as a message', async () => { + const pubnub = shopper(); + const listening = await watch(observer(), ['my_channel']); + + // snippet.publishFalsyMessage + // message: 0 is rejected with "Missing 'message'", so wrap the value + await pubnub.publish({ + channel: 'my_channel', + message: { value: 0 }, + }); + // snippet.end + + // The sample does not keep the response, so the message cannot be matched by timetoken. The watch subscribed just before + // the publish and this tier runs alone on its keyset, so the first `{ value: 0 }` from this publisher is ours. + const received = await listening.next( + 'message', + (m) => (m.message as { value?: number }).value === 0 && m.publisher === pubnub.getUserId(), + ); + assert.deepEqual(received.message, { value: 0 }); + await rejectsValidation(pubnub.publish({ channel: 'my_channel', message: 0 }), /Missing 'message'/); + listening.stop(); + }); + + it('signalBasicUsage: signal reaches a subscriber as a signal event with the custom message type, not as a message', async () => { + const pubnub = shopper(); + const listening = await watch(observer(), ['foo']); + + // snippet.signalBasicUsage + const response = await pubnub.signal({ + message: 'hello', + channel: 'foo', + customMessageType: 'text-message', + }); + console.log('signal response:', response); + // snippet.end + + assertTimetoken(response.timetoken); + const received = await listening.next('signal', (s) => s.timetoken === response.timetoken); + assert.equal(received.message, 'hello'); + assert.equal(received.publisher, pubnub.getUserId()); + assert.equal(received.customMessageType, 'text-message'); + listening.stop(); + }); + + it('fireBasicUsage: fire sends to Functions only, so subscribers receive nothing and history stores nothing', async () => { + const pubnub = shopper(); + const listening = await watch(observer(), ['my_channel']); + // Everything fired by the sample is newer than this timetoken. Shared history may hold older messages, so the "not + // stored" check looks only at the window after it. + const { timetoken: before } = await observer().time(); + + // snippet.fireBasicUsage + // Initialize PubNub with your keys + await pubnub.fire({ + message: { + such: 'object', + }, + channel: 'my_channel', + sendByPost: false, // true to send via post + meta: { + cool: 'meta', + }, // fire extra meta with the request + }); + + console.log('message fired'); + // snippet.end + + // fire() sends to Functions only: subscribers do not receive it and Message Persistence does not store it. This is the + // one place a fixed sleep is right, because the test proves that nothing arrived. + await sleep(4000); + assert.ok( + !listening.events.some((e) => (e.event as { message?: { such?: string } }).message?.such === 'object'), + 'a fired message must not reach subscribers', + ); + const history = await observer().fetchMessages({ channels: ['my_channel'], end: String(before), count: 25 }); + assert.ok( + !history.channels['my_channel']?.some((m) => (m.message as { such?: string } | undefined)?.such === 'object'), + 'a fired message is not stored', + ); + listening.stop(); + }); + + // Needs the pam keyset (PAM_* env names): Access Manager on, and no grant that lets a client without a token publish. + it('publishHandleErrors: the same error handling code logs the right line for success, access denied (403), validation and network failure', async function () { + this.timeout(240_000); + type PubNubError = PubNub.PubNubError; + + const attempt = async (pubnub: PubNub): Promise => { + const errors = captureConsole('error'); + try { + // snippet.publishHandleErrors + try { + await pubnub.publish({ + channel: 'my_channel', + message: { text: 'Hello' }, + }); + } catch (e) { + const status = (e as PubNubError).status; + + if (status?.category === 'PNValidationErrorCategory') { + console.error('The call is invalid:', status.message); + } else if (status?.category === 'PNAccessDeniedCategory') { + console.error('403: the token or authKey has no write permission on the channel'); + } else if (status?.category === 'PNNetworkIssuesCategory') { + console.error('The network is unreachable. Retry the publish yourself.'); + } else { + console.error(status?.category, status?.statusCode, status?.errorData); + } + } + // snippet.end + } finally { + errors.restore(); + } + return errors.lines; + }; + + // Wait until the keyset enforces Access Manager on this channel for a stranger. Without this wait the 403 branch below + // is only as reliable as the propagation of earlier grants and revokes on the keyset. A keyset that never denies fails + // here with a message that names the cause, not with an empty list further down. + await eventually( + () => + rejectsWithCategory( + createClient('pam', 'no-token-probe').publish({ channel: 'my_channel', message: { probe: true } }), + 'PNAccessDeniedCategory', + ), + { + timeout: 120_000, + interval: 3_000, + label: + 'a client without a token is refused on the channel (Access Manager grants and revokes take up to 60 s to propagate)', + }, + ); + + // Success: the server client may publish, so no error line. + assert.deepEqual(await attempt(server()), [], 'a valid publish logs nothing'); + + let deniedAttempts = 0; + await eventually( + async () => { + deniedAttempts += 1; + const denied = await attempt(createClient('pam', 'no-token-user')); + assert.deepEqual(denied, ['403: the token or authKey has no write permission on the channel']); + }, + { + timeout: 90_000, + interval: 3_000, + label: 'a tokenless publish logs the 403 line (Access Manager enforcement reaches every node)', + }, + ); + + // Validation: no publish key, the SDK refuses before it sends. + const invalid = await attempt(createClient('pam', 'no-publish-key-user', { publishKey: '' })); + assert.equal(invalid.length, 1); + assert.match(invalid[0], /^The call is invalid: /); + + // Network: nothing listens on the target port, so the connection is refused. Publish has no retry, so the call fails at once. + assert.deepEqual(await attempt(createClient('pam', 'unreachable-user', unreachable)), [ + 'The network is unreachable. Retry the publish yourself.', + ]); + }); +}); diff --git a/test/integration/live/api-reference/subscribe-legacy.test.ts b/test/integration/live/api-reference/subscribe-legacy.test.ts new file mode 100644 index 000000000..b63fdcdbd --- /dev/null +++ b/test/integration/live/api-reference/subscribe-legacy.test.ts @@ -0,0 +1,632 @@ +import assert from 'node:assert/strict'; +import { randomUUID } from 'node:crypto'; +import { createClient, friend, observer, shopper } from '../support/clients'; +import { connected, eventually, expectEvent, expectStatus, PRESENCE_EVENT_TIMEOUT, sleep } from '../support/wait'; +import { ensureGroups, memberOf, removeGroupMembers } from '../support/subscribe-groups'; +import { sorted } from '../support/util'; + +/** + * Examples for the older `pubnub.subscribe()`, `pubnub.unsubscribe()` and `pubnub.unsubscribeAll()` methods. + * Port of docs-snippets/api-reference/subscribe-legacy.ts. Presence must be on for the open keyset. + */ +describe('api-reference/subscribe-legacy', () => { + const tagOf = (m: { message: unknown }): string | undefined => (m.message as { run?: string }).run; + // Subscription state lists have no guaranteed order. + + // The groups the samples subscribe to, each with one member channel. A group without a channel carries no messages. + const GROUPS = [ + 'my_channelGroup', + 'family', + 'my_channelGroup1', + 'my_channelGroup2', + 'my_channelGroup3', + 'demo_group1', + 'demo_group2', + ]; + before(() => ensureGroups(GROUPS)); + after(() => removeGroupMembers(GROUPS)); + + it('OLDsubscribeBasicUsage: the legacy subscribe call connects and delivers a message', async () => { + const pubnub = shopper(); + const run = randomUUID(); + const ready = connected(pubnub, { channels: ['my_channel'] }); + const received = expectEvent(pubnub, 'message', (m) => tagOf(m) === run && m.channel === 'my_channel'); + + // snippet.OLDsubscribeBasicUsage + pubnub.subscribe({ + channels: ['my_channel'], + }); + // snippet.end + + await ready; + assert.deepEqual(pubnub.getSubscribedChannels(), ['my_channel']); + await friend().publish({ channel: 'my_channel', message: { run } }); + await received; + pubnub.unsubscribeAll(); + }); + + it('OLDsubscribeMultipleChannels: one legacy subscribe call covers three channels, shown by a message on the middle one', async () => { + const pubnub = shopper(); + const run = randomUUID(); + const channels = ['my_channel_1', 'my_channel_2', 'my_channel_3']; + const ready = connected(pubnub, { channels }); + const received = expectEvent(pubnub, 'message', (m) => tagOf(m) === run && m.channel === 'my_channel_2'); + + // snippet.OLDsubscribeMultipleChannels + pubnub.subscribe({ + channels: ['my_channel_1', 'my_channel_2', 'my_channel_3'], + }); + // snippet.end + + await ready; + assert.deepEqual(sorted(pubnub.getSubscribedChannels()), channels); + await friend().publish({ channel: 'my_channel_2', message: { run } }); + await received; + pubnub.unsubscribeAll(); + }); + + it('OLDsubscribeWithPresence: withPresence true produces the own join event with the fields the page table lists', async function () { + this.timeout(60_000); + // A unique user id: a fixed one could still be present from an earlier test, and then no join event is emitted. + const pubnub = createClient('open', `oldpresence-${randomUUID()}`); + const ready = connected(pubnub, { channels: ['my_channel'] }); + const joined = expectEvent( + pubnub, + 'presence', + (p) => p.action === 'join' && p.channel === 'my_channel' && p.uuid === pubnub.getUserId(), + { timeout: PRESENCE_EVENT_TIMEOUT }, + ); + + // snippet.OLDsubscribeWithPresence + pubnub.subscribe({ + channels: ['my_channel'], + withPresence: true, + }); + // snippet.end + + await ready; + const event = await joined; + // The docs presence payload table promises these fields. + assert.equal(typeof (event as { occupancy?: number }).occupancy, 'number'); + assert.ok(event.timetoken); + pubnub.unsubscribeAll(); + }); + + it('OLDsubscribeWithWildCardChannels: a wildcard subscription receives a message on a matching channel', async () => { + const pubnub = shopper(); + const run = randomUUID(); + const ready = connected(pubnub, { channels: ['ab.*'] }); + const received = expectEvent(pubnub, 'message', (m) => tagOf(m) === run && m.channel === 'ab.news'); + + // snippet.OLDsubscribeWithWildCardChannels + pubnub.subscribe({ + channels: ['ab.*'], + }); + // snippet.end + + await ready; + await friend().publish({ channel: 'ab.news', message: { run } }); + const message = await received; + // The wildcard match is reported as the subscription. + assert.equal(message.subscription, 'ab.*'); + pubnub.unsubscribeAll(); + }); + + it('OLDsubscribeWithState: the connected status handler can set state, and the observer reads it back per channel', async function () { + this.timeout(60_000); + // A unique user id keeps the state of this run apart from any earlier one on the shared channels. + const pubnub = createClient('open', `oldstate-${randomUUID()}`); + const ready = connected(pubnub, { channels: ['ch1', 'ch2', 'ch3'] }); + + // snippet.OLDsubscribeWithState + pubnub.addListener({ + status: async (statusEvent) => { + if (statusEvent.category === 'PNConnectedCategory') { + await pubnub.setState({ + state: { + some: 'state', + }, + channels: ['ch1', 'ch2', 'ch3'], + }); + } + }, + message: (messageEvent) => { + // handle message + }, + presence: (presenceEvent) => { + // handle presence + }, + }); + + pubnub.subscribe({ + channels: ['ch1', 'ch2', 'ch3'], + }); + // snippet.end + + await ready; + // The observer reads the state back. The docs sample called setState without channels, which the SDK rejects (an intended difference from the old docs body). + const reader = observer(); + await eventually( + async () => { + // One channel per call: with several channels getState nests the result one level deeper (docs finding). + for (const channel of ['ch1', 'ch2', 'ch3']) { + const state = await reader.getState({ uuid: pubnub.getUserId(), channels: [channel] }); + assert.deepEqual(state.channels[channel], { some: 'state' }, `state on ${channel}`); + } + }, + { timeout: 30_000, label: 'state set from the connected status handler' }, + ); + pubnub.unsubscribeAll(); + }); + + it('OLDsubscribeChannelGroup: subscribing to a channel group delivers a message from its member channel', async () => { + const pubnub = shopper(); + const run = randomUUID(); + const ready = connected(pubnub, { channelGroups: ['my_channelGroup'] }); + const received = expectEvent(pubnub, 'message', (m) => tagOf(m) === run && m.subscription === 'my_channelGroup'); + + // snippet.OLDsubscribeChannelGroup + pubnub.subscribe({ + channelGroups: ['my_channelGroup'], + }); + // snippet.end + + await ready; + assert.deepEqual(pubnub.getSubscribedChannelGroups(), ['my_channelGroup']); + await friend().publish({ channel: memberOf('my_channelGroup'), message: { run } }); + const message = await received; + assert.equal(message.channel, memberOf('my_channelGroup')); + pubnub.unsubscribeAll(); + }); + + it('OLDsubscribeChannelGroupWithPresence: a group subscription with presence yields a join event that belongs to the group', async function () { + this.timeout(60_000); + const pubnub = createClient('open', `oldgrouppresence-${randomUUID()}`); + const ready = connected(pubnub, { channelGroups: ['family'] }); + const joined = expectEvent(pubnub, 'presence', (p) => p.action === 'join' && p.uuid === pubnub.getUserId(), { + timeout: PRESENCE_EVENT_TIMEOUT, + }); + + // snippet.OLDsubscribeChannelGroupWithPresence + pubnub.subscribe({ + channelGroups: ['family'], + withPresence: true, + }); + // snippet.end + + await ready; + const event = await joined; + assert.ok( + event.channel === memberOf('family') || event.subscription === 'family', + `presence event belongs to the group (channel ${event.channel}, subscription ${event.subscription})`, + ); + pubnub.unsubscribeAll(); + }); + + it('OLDsubscribeMultipleChannelGroup: one call subscribes to three groups, shown by a message through the last group', async () => { + const pubnub = shopper(); + const run = randomUUID(); + const groups = ['my_channelGroup1', 'my_channelGroup2', 'my_channelGroup3']; + const ready = connected(pubnub, { channelGroups: groups }); + const received = expectEvent(pubnub, 'message', (m) => tagOf(m) === run && m.subscription === 'my_channelGroup3'); + + // snippet.OLDsubscribeMultipleChannelGroup + pubnub.subscribe({ + channelGroups: ['my_channelGroup1', 'my_channelGroup2', 'my_channelGroup3'], + }); + // snippet.end + + await ready; + assert.deepEqual(sorted(pubnub.getSubscribedChannelGroups()), groups); + await friend().publish({ channel: memberOf('my_channelGroup3'), message: { run } }); + await received; + pubnub.unsubscribeAll(); + }); + + it('OLDsubscribeChannelGroupAndChannels: channels and groups can be mixed in one call, shown by a message through each', async () => { + const pubnub = shopper(); + const run = randomUUID(); + const ready = connected(pubnub, { channels: ['my_channel'], channelGroups: ['my_channelGroup'] }); + const onChannel = expectEvent( + pubnub, + 'message', + (m) => tagOf(m) === `${run}-channel` && m.channel === 'my_channel', + ); + const onGroup = expectEvent( + pubnub, + 'message', + (m) => tagOf(m) === `${run}-group` && m.subscription === 'my_channelGroup', + ); + + // snippet.OLDsubscribeChannelGroupAndChannels + pubnub.subscribe({ + channels: ['my_channel'], + channelGroups: ['my_channelGroup'], + }); + // snippet.end + + await ready; + const sender = friend(); + await sender.publish({ channel: 'my_channel', message: { run: `${run}-channel` } }); + await sender.publish({ channel: memberOf('my_channelGroup'), message: { run: `${run}-group` } }); + await onChannel; + await onGroup; + pubnub.unsubscribeAll(); + }); + + it('OLDaddListeners: the legacy listener sample handlers see status, message and presence events whose fields exist on live events', async function () { + this.timeout(120_000); + const run = randomUUID(); + // The samples's handlers do not mention a channel, so the test picks a unique one. App Context metadata is set on it and removed at the end. + const channel = `legacy_listeners_${run}`; + const pubnub = createClient('open', `oldlisteners-${run}`); + const joiner = createClient('open', `oldlisteners-joiner-${run}`); + const sender = friend(); + // Everything the client sees, in arrival order (other tests' leftovers cannot reach a channel with a unique name). + // The listener payload types are a union per kind. The test reads fields that only some members have, so it keeps them loose. + /* eslint-disable @typescript-eslint/no-explicit-any */ + const seen = { + message: [] as any[], + presence: [] as any[], + signal: [] as any[], + objects: [] as any[], + messageAction: [] as any[], + status: [] as any[], + }; + /* eslint-enable @typescript-eslint/no-explicit-any */ + pubnub.addListener({ + message: (m) => seen.message.push(m), + presence: (p) => seen.presence.push(p), + signal: (s) => seen.signal.push(s), + objects: (o) => seen.objects.push(o), + messageAction: (a) => seen.messageAction.push(a), + status: (s) => seen.status.push(s), + }); + + /* eslint-disable prettier/prettier */ + // snippet.OLDaddListeners + pubnub.addListener({ + // Messages + message: function (m) { + const channelName = m.channel; // Channel on which the message was published + const channelGroup = m.subscription; // Channel group or wildcard subscription match (if exists) + const pubTT = m.timetoken; // Publish timetoken + const msg = m.message; // Message payload + const publisher = m.publisher; // Message publisher + }, + // Presence + // requires a subscription with presence + presence: function (p) { + const action = p.action; // Can be join, leave, timeout, state-change, or interval + const channelName = p.channel; // Channel to which the message belongs + const channelGroup = p.subscription; // Channel group or wildcard subscription match, if any + const publishTime = p.timestamp; // Publish timetoken + const timetoken = p.timetoken; // Current timetoken + if (p.action !== 'state-change') { + const occupancy = p.occupancy; // Number of users subscribed to the channel + } + if (p.action === 'state-change') { + const state = p.state; // User state + } + if (p.action !== 'interval') { + const uuid = p.uuid; // User ID of the user who joined, left, timed out, or changed state + } + }, + // Signals + signal: function (s) { + const channelName = s.channel; // Channel to which the signal belongs + const channelGroup = s.subscription; // Channel group or wildcard subscription match, if any + const pubTT = s.timetoken; // Publish timetoken + const msg = s.message; // Payload + const publisher = s.publisher; // Message publisher + }, + // App Context + objects: (objectEvent) => { + const channel = objectEvent.channel; // Channel to which the event belongs + const channelGroup = objectEvent.subscription; // Channel group + const timetoken = objectEvent.timetoken; // Event timetoken + const event = objectEvent.message.event; // Name of the event that occurred + const type = objectEvent.message.type; // Type of the object that changed + const data = objectEvent.message.data; // Data from the event that occurred + }, + // Message Actions + messageAction: function (ma) { + const channelName = ma.channel; // Channel to which the message belongs + const publisher = ma.publisher; // Message publisher + const event = ma.event; // Message action added or removed + const type = ma.data.type; // Message action type + const value = ma.data.value; // Message action value + const messageTimetoken = ma.data.messageTimetoken; // Timetoken of the original message + const actionTimetoken = ma.data.actionTimetoken; // Timetoken of the message action + }, + // File Sharing + file: function (event) { + const channelName = event.channel; // Channel to which the file belongs + const channelGroup = event.subscription; // Channel group or wildcard subscription match (if exists) + const publisher = event.publisher; // File publisher + const timetoken = event.timetoken; // Event timetoken + + const message = event.message; // Optional message attached to the file + const fileId = event.file?.id; // File unique id + const fileName = event.file?.name; // File name + const fileUrl = event.file?.url; // File direct URL + }, + status: function (s) { + const affectedChannelGroups = s.affectedChannelGroups; // Array of channel groups affected in the operation + const affectedChannels = s.affectedChannels; // Array of channels affected in the operation + const category = s.category; // Returns PNConnectedCategory + const operation = s.operation; // Returns PNSubscribeOperation + const lastTimetoken = s.lastTimetoken; // Last timetoken used in the subscribe request (type long), only with enableEventEngine: false + const currentTimetoken = s.currentTimetoken; /* Current timetoken fetched in subscribe response, + * to be used in the next request (type long) */ + const subscribedChannels = s.subscribedChannels; // Array of all currently subscribed channels, only with enableEventEngine: false + }, + }); + // snippet.end + /* eslint-enable prettier/prettier */ + + try { + const ready = connected(pubnub, { channels: [channel] }); + pubnub.subscribe({ channels: [channel], withPresence: true }); + await ready; + // The second client joins only after the first one is connected, so its join is published after the first client's + // subscribe loop is running and must reach it. + const joinerReady = connected(joiner, { channels: [channel] }); + joiner.subscribe({ channels: [channel] }); + await joinerReady; + // The service can lose or delay one join. Wait briefly for the joiner's join. If it does not arrive, subscribe the joiner + // again, at most twice, and log it so a lost join stays visible. + const joinerJoined = () => seen.presence.some((p) => p.action === 'join' && p.uuid === joiner.getUserId()); + for (let attempt = 1; attempt <= 2 && !joinerJoined(); attempt += 1) { + try { + await eventually(() => assert.ok(joinerJoined(), 'the joiner join arrived'), { + timeout: 8_000, + label: 'joiner join arrived', + }); + } catch { + console.log(`joiner join not delivered, subscribed again (attempt ${attempt})`); + joiner.unsubscribeAll(); + const again = connected(joiner, { channels: [channel] }); + joiner.subscribe({ channels: [channel] }); + await again; + } + } + const published = await sender.publish({ channel, message: { run } }); + await sender.signal({ channel, message: 'ping' }); + await sender.addMessageAction({ + channel, + messageTimetoken: published.timetoken, + action: { type: 'reaction', value: 'smiley_face' }, + }); + await sender.objects.setChannelMetadata({ channel, data: { name: 'legacy listeners' } }); + await eventually( + () => { + assert.ok( + seen.message.some((m) => tagOf(m) === run), + 'message arrived', + ); + assert.ok( + seen.presence.some((p) => p.action === 'join' && p.uuid === joiner.getUserId()), + 'joiner join arrived', + ); + assert.ok( + seen.signal.some((s) => s.message === 'ping'), + 'signal arrived', + ); + assert.ok( + seen.objects.some((o) => o.message?.event === 'set' && o.message?.data?.id === channel), + 'objects event arrived', + ); + assert.ok( + seen.messageAction.some((a) => a.event === 'added'), + 'message action arrived', + ); + }, + { timeout: PRESENCE_EVENT_TIMEOUT, label: 'every event kind arrived' }, + ); + // The joiner leaves and the leave event must follow. + joiner.unsubscribeAll(); + await eventually( + () => + assert.ok( + seen.presence.some((p) => p.action === 'leave' && p.uuid === joiner.getUserId()), + `the joiner's leave event arrived. Presence events seen: ${seen.presence.map((p) => `${p.action} ${p.uuid}`).join(', ')}`, + ), + { timeout: PRESENCE_EVENT_TIMEOUT, label: 'leave event arrived' }, + ); + // The own join is not asserted: whether it is delivered depends on where the join falls against the first subscribe timetoken. + + const message = seen.message.find((m) => tagOf(m) === run)!; + assert.equal(message.channel, channel); + assert.equal(message.publisher, sender.getUserId()); + assert.equal(message.timetoken, published.timetoken); + const presence = seen.presence.find((p) => p.action === 'join' && p.uuid === joiner.getUserId())!; + assert.equal(presence.channel, channel); + assert.equal(typeof presence.occupancy, 'number'); + assert.ok(presence.timestamp, 'presence timestamp'); + assert.ok(presence.timetoken, 'presence timetoken'); + const leave = seen.presence.find((p) => p.action === 'leave' && p.uuid === joiner.getUserId())!; + assert.equal(typeof leave.occupancy, 'number'); + const signal = seen.signal.find((s) => s.message === 'ping')!; + assert.equal(signal.publisher, sender.getUserId()); + const objectEvent = seen.objects.find((o) => o.message?.event === 'set' && o.message?.data?.id === channel)!; + assert.equal(objectEvent.message.type, 'channel'); + // The old sample read event, type, data and publisher from the event itself. They are under message, and objects events have no publisher. + assert.equal((objectEvent as { event?: unknown }).event, undefined); + assert.equal((objectEvent as { publisher?: unknown }).publisher, undefined); + const action = seen.messageAction.find((a) => a.event === 'added')!; + assert.equal(action.data.type, 'reaction'); + assert.equal(action.data.value, 'smiley_face'); + assert.equal(action.data.messageTimetoken, published.timetoken); + assert.ok(action.data.actionTimetoken); + assert.ok(action.publisher, 'message action publisher (the docs read ma.publisher)'); + const connectedStatus = seen.status.find((s) => s.category === 'PNConnectedCategory'); + assert.ok(connectedStatus, 'connected status'); + assert.equal(connectedStatus.operation, 'PNSubscribeOperation'); + // The new-subscription-loop status carries neither lastTimetoken nor subscribedChannels, see the status fields test below. + assert.ok(connectedStatus.affectedChannels.includes(channel)); + assert.ok(connectedStatus.currentTimetoken); + pubnub.unsubscribeAll(); + } finally { + await sender.objects.removeChannelMetadata({ channel }).catch(() => undefined); + } + }); + + it('status fields: lastTimetoken and subscribedChannels are set on the status only by the legacy subscription manager', async () => { + for (const enableEventEngine of [true, false]) { + const pubnub = shopper({ enableEventEngine }); + const statuses: Array> = []; + pubnub.addListener({ status: (s) => statuses.push(s as unknown as Record) }); + pubnub.subscribe({ channels: ['my_channel'] }); + await eventually(() => assert.ok(statuses.some((s) => s.category === 'PNConnectedCategory')), { + label: 'connected', + }); + const status = statuses.find((s) => s.category === 'PNConnectedCategory')!; + assert.equal(status.operation, 'PNSubscribeOperation'); + assert.deepEqual(status.affectedChannels, ['my_channel']); + assert.ok(status.currentTimetoken, 'currentTimetoken'); + assert.equal( + 'lastTimetoken' in status, + !enableEventEngine, + `lastTimetoken is present only with enableEventEngine: false (enableEventEngine ${enableEventEngine})`, + ); + assert.equal( + 'subscribedChannels' in status, + !enableEventEngine, + `subscribedChannels is present only with enableEventEngine: false (enableEventEngine ${enableEventEngine})`, + ); + pubnub.unsubscribeAll(); + } + }); + + it('OLDremoveListeners: removeListener leaves the other listeners of the client working, and the removed object receives nothing', async () => { + const pubnub = shopper(); + const run = randomUUID(); + const kept: unknown[] = []; + pubnub.addListener({ message: (m) => tagOf(m) === run && kept.push(m) }); + + /* eslint-disable no-var, prettier/prettier */ + // snippet.OLDremoveListeners + var existingListener = { + message: function () { + }, + }; + + pubnub.removeListener(existingListener); + // snippet.end + /* eslint-enable no-var, prettier/prettier */ + let removedCalls = 0; + existingListener.message = () => { + removedCalls += 1; + }; + const ready = connected(pubnub, { channels: ['my_channel'] }); + pubnub.subscribe({ channels: ['my_channel'] }); + await ready; + try { + await observer().publish({ channel: 'my_channel', message: { text: 'after removeListener', run } }); + await eventually(() => assert.equal(kept.length, 1, 'the other listener still receives'), { + label: 'kept listener receives', + }); + assert.equal(removedCalls, 0, 'the removed listener object received nothing'); + } finally { + pubnub.unsubscribeAll(); + } + }); + + it('removeListener effect: a removed listener gets no more events and a kept one does', async () => { + const pubnub = shopper(); + const run = randomUUID(); + const removed: unknown[] = []; + const kept: unknown[] = []; + const removable = { message: (m: { message: unknown }) => tagOf(m) === run && removed.push(m) }; + pubnub.addListener(removable); + pubnub.addListener({ message: (m) => tagOf(m) === run && kept.push(m) }); + const ready = connected(pubnub, { channels: ['my_channel'] }); + pubnub.subscribe({ channels: ['my_channel'] }); + await ready; + pubnub.removeListener(removable); + await friend().publish({ channel: 'my_channel', message: { run } }); + await eventually(() => assert.equal(kept.length, 1), { label: 'kept listener got the message' }); + // Proves that something did NOT arrive, so a short fixed wait is the right tool. + await sleep(500); + assert.equal(removed.length, 0, 'the removed listener got nothing'); + pubnub.unsubscribeAll(); + }); + + it('OLDUnsubscribeBasicUsage: unsubscribe with channels drops that channel', async () => { + const pubnub = shopper(); + const ready = connected(pubnub, { channels: ['my_channel'] }); + pubnub.subscribe({ channels: ['my_channel'] }); + await ready; + assert.deepEqual(pubnub.getSubscribedChannels(), ['my_channel']); + + // snippet.OLDUnsubscribeBasicUsage + pubnub.unsubscribe({ + channels: ['my_channel'], + }); + // snippet.end + + await eventually(() => assert.deepEqual(pubnub.getSubscribedChannels(), []), { label: 'channel dropped' }); + }); + + it('OLDUnsubscribeMultipleChannels: unsubscribing three of four channels leaves exactly the fourth', async () => { + const pubnub = shopper(); + const channels = ['chan1', 'chan2', 'chan3', 'ch1']; + const ready = connected(pubnub, { channels }); + pubnub.subscribe({ channels }); + await ready; + + // snippet.OLDUnsubscribeMultipleChannels + pubnub.unsubscribe({ + channels: ['chan1', 'chan2', 'chan3'], + }); + // snippet.end + + // Only the channel that the sample did not name stays subscribed. + await eventually(() => assert.deepEqual(pubnub.getSubscribedChannels(), ['ch1']), { + label: 'three channels dropped', + }); + pubnub.unsubscribeAll(); + }); + + it('OLDUnsubscribeMultipleChannelGroup: unsubscribing two of three groups leaves exactly the third', async () => { + const pubnub = shopper(); + const groups = ['demo_group1', 'demo_group2', 'family']; + const ready = connected(pubnub, { channelGroups: groups }); + pubnub.subscribe({ channelGroups: groups }); + await ready; + + // snippet.OLDUnsubscribeMultipleChannelGroup + pubnub.unsubscribe({ + channelGroups: ['demo_group1', 'demo_group2'], + }); + // snippet.end + + await eventually(() => assert.deepEqual(pubnub.getSubscribedChannelGroups(), ['family']), { + label: 'two groups dropped', + }); + pubnub.unsubscribeAll(); + }); + + it('OLDUnsubscribeAllBasicUsage: unsubscribeAll clears all channels and groups', async () => { + const pubnub = shopper(); + const channels = ['ch1', 'ch2']; + const ready = connected(pubnub, { channels, channelGroups: ['family'] }); + pubnub.subscribe({ channels, channelGroups: ['family'] }); + await ready; + const disconnected = expectStatus(pubnub, ['PNDisconnectedCategory']); + + // snippet.OLDUnsubscribeAllBasicUsage + pubnub.unsubscribeAll(); + // snippet.end + + await eventually( + () => { + assert.deepEqual(pubnub.getSubscribedChannels(), []); + assert.deepEqual(pubnub.getSubscribedChannelGroups(), []); + }, + { label: 'everything dropped' }, + ); + void disconnected.catch(() => undefined); + }); +}); diff --git a/test/integration/live/api-reference/subscribe.test.ts b/test/integration/live/api-reference/subscribe.test.ts new file mode 100644 index 000000000..25cf2cb62 --- /dev/null +++ b/test/integration/live/api-reference/subscribe.test.ts @@ -0,0 +1,467 @@ +import assert from 'node:assert/strict'; +import { randomUUID } from 'node:crypto'; +import { createClient, friend, shopper } from '../support/clients'; +import { captureConsole } from '../support/console'; +import { connected, eventually, expectEvent, PRESENCE_EVENT_TIMEOUT } from '../support/wait'; +import { ensureGroups, memberOf, removeGroupMembers } from '../support/subscribe-groups'; +import { sorted } from '../support/util'; + +/** + * Subscribe examples, without the DataSync examples. Port of docs-snippets/api-reference/subscribe.ts. + * Presence must be on for the open keyset (eventListenerBasicUsage reads join events). + */ +describe('api-reference/subscribe', () => { + // Subscription state lists have no guaranteed order. + const hasLine = (lines: string[], prefix: string, containing: string): boolean => + lines.some((line) => line.startsWith(prefix) && line.includes(containing)); + + // Three samples subscribe to channelGroup_1, and a group without a channel carries no messages. + before(() => ensureGroups(['channelGroup_1'])); + after(() => removeGroupMembers(['channelGroup_1'])); + + it('createChannelBasicUsage: the channel entity factory returns the channel name and does not subscribe', () => { + const pubnub = shopper(); + // snippet.createChannelBasicUsage + const channel = pubnub.channel('my_channel'); + // snippet.end + assert.equal(channel.name, 'my_channel'); + assert.deepEqual(pubnub.getSubscribedChannels(), [], 'creating an entity does not subscribe'); + }); + + it('createChannelGroupBasicUsage: the channel group entity carries the group name', () => { + const pubnub = shopper(); + // snippet.createChannelGroupBasicUsage + // Initialize PubNub with demo keys + const channelGroup = pubnub.channelGroup('channelGroup_1'); + // snippet.end + assert.equal(channelGroup.name, 'channelGroup_1'); + }); + + it('createChannelMetadataBasicUsage: the channel metadata entity carries the id and makes no network call', () => { + const pubnub = shopper(); + // snippet.createChannelMetadataBasicUsage + // Initialize PubNub with demo keys + + const channelMetadata = pubnub.channelMetadata('channel_1'); + // snippet.end + assert.equal(channelMetadata.id, 'channel_1'); + }); + + it('createUserMetadataBasicUsage: the user metadata entity carries the id and makes no network call', () => { + const pubnub = shopper(); + // snippet.createUserMetadataBasicUsage + // Initialize PubNub with demo keys + + const userMetadata = pubnub.userMetadata('user_meta1'); + // snippet.end + assert.equal(userMetadata.id, 'user_meta1'); + }); + + it('createSubscription: creating a subscription object does not subscribe anything', () => { + const pubnub = shopper(); + // snippet.createSubscription + const channel = pubnub.channel('my_channel'); + + const subscriptionOptions = { receivePresenceEvents: true }; + channel.subscription(subscriptionOptions); + // snippet.end + // Creating a subscription object does not subscribe. + assert.deepEqual(pubnub.getSubscribedChannels(), []); + }); + + it('subscriptionSetSubscribeBasicUsage: subscribing a set connects both channels and delivers a message from a friend', async () => { + const pubnub = shopper(); + const run = randomUUID(); + const ready = connected(pubnub, { channels: ['ch1', 'ch2'] }); + const received = expectEvent( + pubnub, + 'message', + (m) => m.channel === 'ch1' && (m.message as { run?: string }).run === run, + ); + + // snippet.subscriptionSetSubscribeBasicUsage + const channelsSubscriptionSet = pubnub.subscriptionSet({ channels: ['ch1', 'ch2'] }); + channelsSubscriptionSet.subscribe(); + // snippet.end + + await ready; + assert.deepEqual(sorted(pubnub.getSubscribedChannels()), ['ch1', 'ch2']); + await friend().publish({ channel: 'ch1', message: { via: 'set', run } }); + await received; + channelsSubscriptionSet.unsubscribe(); + }); + + it('eventListenerAddListeners: the message, signal and message action handlers of the sample log when the events arrive', async () => { + const pubnub = shopper(); + const run = randomUUID(); + const logs = captureConsole('log'); + try { + // snippet.eventListenerAddListeners + + // create a subscription from a channel entity + const channel = pubnub.channel('channel_1'); + const subscription = channel.subscription(); + + // Event-specific listeners + subscription.onMessage = (message) => { + console.log('Message event: ', message); + }; + subscription.onPresence = (presence) => { + console.log('Presence event: ', presence); + }; + subscription.onSignal = (signal) => { + console.log('Signal event: ', signal); + }; + subscription.onObjects = (objectsEvent) => { + console.log('Objects event: ', objectsEvent); + }; + subscription.onMessageAction = (messageActionEvent) => { + console.log('Message Reaction event: ', messageActionEvent); + }; + subscription.onFile = (fileEvent) => { + console.log('File event: ', fileEvent); + }; + + // Generic listeners + subscription.addListener({ + // Messages + message: function (m) { + const channelName = m.channel; // Channel on which the message was published + const channelGroup = m.subscription; // Channel group or wildcard subscription match (if exists) + const pubTT = m.timetoken; // Publish timetoken + const msg = m.message; // Message payload + const publisher = m.publisher; // Message publisher + }, + // Presence + // requires a subscription with presence + presence: function (p) { + const action = p.action; // Can be join, leave, timeout, state-change, or interval + const channelName = p.channel; // Channel to which the message belongs + const channelGroup = p.subscription; // Channel group or wildcard subscription match, if any + const publishTime = p.timestamp; // Publish timetoken + const timetoken = p.timetoken; // Current timetoken + }, + // Signals + signal: function (s) { + const channelName = s.channel; // Channel to which the signal belongs + const channelGroup = s.subscription; // Channel group or wildcard subscription match, if any + const pubTT = s.timetoken; // Publish timetoken + const msg = s.message; // Payload + const publisher = s.publisher; // Message publisher + }, + // App Context + objects: (objectEvent) => { + const channel = objectEvent.channel; // channel Id or a User Id of updated/set app context metadata object + const channelGroup = objectEvent.subscription; // Channel group + const timetoken = objectEvent.timetoken; // Event timetoken + const event = objectEvent.message.data.type; // Event name + }, + // Message Actions + messageAction: function (ma) { + const channelName = ma.channel; // Channel to which the message belongs + const publisher = ma.data.uuid; // Message publisher + const event = ma.event; // Message action added or removed + const type = ma.data.type; // Message action type + const value = ma.data.value; // Message action value + const messageTimetoken = ma.data.messageTimetoken; // Timetoken of the original message + const actionTimetoken = ma.data.actionTimetoken; // Timetoken of the message action + }, + // File Sharing + file: function (event) { + const channelName = event.channel; // Channel to which the file belongs + const channelGroup = event.subscription; // Channel group or wildcard subscription match (if exists) + const publisher = event.publisher; // File publisher + const timetoken = event.timetoken; // Event timetoken + const message = event.message; // Optional message attached to the file + }, + }); + + // snippet.end + + const ready = connected(pubnub, { channels: ['channel_1'] }); + subscription.subscribe(); + await ready; + const sender = friend(); + const published = await sender.publish({ channel: 'channel_1', message: { n: 1, run } }); + await sender.signal({ channel: 'channel_1', message: `ping-${run}` }); + await sender.addMessageAction({ + channel: 'channel_1', + messageTimetoken: published.timetoken, + action: { type: 'reaction', value: 'smiley_face' }, + }); + // Each line must carry this run's own data: the channel is shared and persistent. + await eventually( + () => { + assert.ok(hasLine(logs.lines, 'Message event: ', run), 'onMessage ran for this run'); + assert.ok(hasLine(logs.lines, 'Signal event: ', run), 'onSignal ran for this run'); + assert.ok( + hasLine(logs.lines, 'Message Reaction event: ', published.timetoken), + 'onMessageAction ran for this run', + ); + }, + { label: 'event-specific listeners ran' }, + ); + subscription.unsubscribe(); + } finally { + logs.restore(); + } + }); + + it('eventListenerBasicUsage: status, message, presence and set level message action listeners all fire from one sample', async function () { + this.timeout(90_000); + const run = randomUUID(); + const pubnub = createClient('open', `listenerbasic-${run}`); + const joiner = createClient('open', `listenerbasic-joiner-${run}`); + const logs = captureConsole('log'); + try { + const ready = connected(pubnub, { channels: ['channel_1', 'ch1', 'ch2'] }); + + // snippet.eventListenerBasicUsage + // create a subscription from a channel entity + const channel = pubnub.channel('channel_1'); + const subscription1 = channel.subscription({ receivePresenceEvents: true }); + + // create a subscription set with multiple channels + const subscriptionSet1 = pubnub.subscriptionSet({ channels: ['ch1', 'ch2'] }); + + // add a status listener + pubnub.addListener({ + status: (s) => { + console.log('Status', s.category); + }, + }); + + // add message and presence listeners + subscription1.addListener({ + // Messages + message: (m) => { + console.log('Received message', m); + }, + // Presence + presence: (p) => { + console.log('Presence event', p); + }, + }); + + // add event-specific message actions listener + subscriptionSet1.onMessageAction = (p) => { + console.log('Message action event:', p); + }; + + subscription1.subscribe(); + subscriptionSet1.subscribe(); + // snippet.end + + await ready; + const joinerReady = connected(joiner, { channels: ['channel_1'] }); + joiner.subscribe({ channels: ['channel_1'] }); + await joinerReady; + const sender = friend(); + await sender.publish({ channel: 'channel_1', message: { n: 1, run } }); + const onSet = await sender.publish({ channel: 'ch1', message: { n: 2, run } }); + await sender.addMessageAction({ + channel: 'ch1', + messageTimetoken: onSet.timetoken, + action: { type: 'reaction', value: 'smiley_face' }, + }); + // Every line is matched on this run's data (the run id, the joiner's unique user id, the set message's timetoken). + await eventually( + () => { + assert.ok(logs.lines.includes('Status PNConnectedCategory'), 'status listener saw the connection'); + assert.ok(hasLine(logs.lines, 'Received message', run), 'subscription message listener ran'); + assert.ok( + hasLine(logs.lines, 'Presence event', joiner.getUserId()), + 'presence listener saw the join of the joiner', + ); + assert.ok( + hasLine(logs.lines, 'Message action event:', onSet.timetoken), + 'subscription set message action listener ran', + ); + }, + { label: 'listeners from the snippet ran', timeout: PRESENCE_EVENT_TIMEOUT }, + ); + joiner.unsubscribeAll(); + subscription1.unsubscribe(); + subscriptionSet1.unsubscribe(); + } finally { + logs.restore(); + } + }); + + it('eventListenerAddConnectionStatusListenersBasicUsage: the status listener fires PNConnectedCategory', async () => { + const pubnub = shopper(); + const logs = captureConsole('log'); + try { + // snippet.eventListenerAddConnectionStatusListenersBasicUsage + // add a status listener + pubnub.addListener({ + status: (s) => { + console.log('Status', s.category); + }, + }); + // snippet.end + + const subscription = pubnub.channel('ch1').subscription(); + subscription.subscribe(); + await eventually(() => assert.ok(logs.lines.includes('Status PNConnectedCategory')), { + label: 'connected status logged', + }); + subscription.unsubscribe(); + } finally { + logs.restore(); + } + }); + + it('unsubscribeBasicUsage: unsubscribing a subscription and a set leaves nothing subscribed', async () => { + const pubnub = shopper(); + // snippet.unsubscribeBasicUsage + // create a subscription from a channel entity + const channel1 = pubnub.channel('channel_1'); + const subscription1 = channel1.subscription({ receivePresenceEvents: true }); + + // create a subscription set with multiple channels + const subscriptionSet1 = pubnub.subscriptionSet({ channels: ['ch1', 'ch2'] }); + + subscription1.subscribe(); + subscriptionSet1.subscribe(); + + subscription1.unsubscribe(); + subscriptionSet1.unsubscribe(); + // snippet.end + await eventually(() => assert.deepEqual(pubnub.getSubscribedChannels(), []), { label: 'nothing left subscribed' }); + }); + + it('unsubscribeAllBasicUsage: unsubscribeAll clears channels and groups', async () => { + const pubnub = shopper(); + // snippet.unsubscribeAllBasicUsage + // create a subscription set with multiple channels + const subscriptionSet = pubnub.subscriptionSet({ channels: ['ch1', 'ch2'] }); + subscriptionSet.subscribe(); + + // create a subscription from a channel entity + const channelGroup1 = pubnub.channelGroup('channelGroup_1'); + const groupSubscription1 = channelGroup1.subscription({ receivePresenceEvents: true }); + groupSubscription1.subscribe(); + + // unsubscribe all active subscriptions + pubnub.unsubscribeAll(); + // snippet.end + await eventually( + () => { + assert.deepEqual(pubnub.getSubscribedChannels(), []); + assert.deepEqual(pubnub.getSubscribedChannelGroups(), []); + }, + { label: 'unsubscribeAll clears channels and groups' }, + ); + }); + + it('subscriptionSetFrom2IndividualSubscriptions: addSubscription and removeSubscription change the set before subscribing', async () => { + const pubnub = shopper(); + const ready = connected(pubnub, { channels: ['channel_1'], channelGroups: ['channelGroup_1'] }); + + // snippet.subscriptionSetFrom2IndividualSubscriptions + // create a subscription from a channel entity + const channel = pubnub.channel('channel_1'); + const subscription1 = channel.subscription({ receivePresenceEvents: true }); + + // create a subscription from a channel group entity + const channelGroup = pubnub.channelGroup('channelGroup_1'); + const subscription2 = channelGroup.subscription(); + + // add 2 subscriptions to create a subscription set + const subscriptionSet = subscription1.addSubscription(subscription2); + + // add another subscription to the set + const subscription3 = pubnub.channel('channel_3').subscription({ receivePresenceEvents: false }); + subscriptionSet.addSubscription(subscription3); + + // remove a subscription from a subscription set + subscriptionSet.removeSubscription(subscription3); + + subscriptionSet.subscribe(); + // snippet.end + + await ready; + const channels = pubnub.getSubscribedChannels(); + assert.ok(channels.includes('channel_1'), 'channel_1 is subscribed'); + assert.ok(!channels.includes('channel_3'), 'channel_3 was removed before subscribing'); + assert.ok(pubnub.getSubscribedChannelGroups().includes('channelGroup_1'), 'channelGroup_1 is subscribed'); + pubnub.unsubscribeAll(); + }); + + it('SubscriptionSetFrom2Sets: addSubscriptionSet and removeSubscriptionSet are local operations and subscribe nothing', () => { + const pubnub = shopper(); + // snippet.SubscriptionSetFrom2Sets + // create a subscription set with multiple channels + const subscriptionSet1 = pubnub.subscriptionSet({ channels: ['ch1', 'ch2'] }); + + // create another subscription set with multiple channels and options + const subscriptionSet2 = pubnub.subscriptionSet({ + channels: ['ch3', 'ch4'], + subscriptionOptions: { receivePresenceEvents: true }, + }); + + // add a subscription set to another subscription set + subscriptionSet1.addSubscriptionSet(subscriptionSet2); + + // remove a subscription set from another subscription set + subscriptionSet1.removeSubscriptionSet(subscriptionSet2); + // snippet.end + // Nothing subscribed, and the first set is back to its own two channels. + assert.deepEqual(pubnub.getSubscribedChannels(), []); + assert.deepEqual(sorted(subscriptionSet1.channels), ['ch1', 'ch2']); + }); + + it('AddToExistingSubscriptionSet: adding sets and subscriptions to an already subscribed set takes effect at once', async () => { + const pubnub = shopper(); + const run = randomUUID(); + const member = memberOf('channelGroup_1'); + const ready = connected(pubnub, { + channels: ['ch1', 'ch2', 'ch3', 'ch4'], + channelGroups: ['channelGroup_1'], + }); + const onLateChannel = expectEvent( + pubnub, + 'message', + (m) => m.channel === 'ch4' && (m.message as { run?: string }).run === run, + ); + const onGroup = expectEvent( + pubnub, + 'message', + (m) => m.channel === member && m.subscription === 'channelGroup_1' && (m.message as { run?: string }).run === run, + ); + + // snippet.AddToExistingSubscriptionSet + // create a subscription set with multiple channels + const subscriptionSet1 = pubnub.subscriptionSet({ channels: ['ch1', 'ch2'] }); + + // subscribe to the set + // start receiving events from ch1 and ch2 + subscriptionSet1.subscribe(); + + // create another subscription set with multiple channels + const subscriptionSet2 = pubnub.subscriptionSet({ channels: ['ch3', 'ch4'] }); + + // add the new set to the initial set + subscriptionSet1.addSubscriptionSet(subscriptionSet2); + + // you're now receiving events from ch1, ch2, ch3, and ch4 + // because the set has been subscribed to previously + + // create and add another subscription to the set + const channelGroup = pubnub.channelGroup('channelGroup_1'); + const subscription2 = channelGroup.subscription(); + subscriptionSet1.addSubscription(subscription2); + + // you're now receiving events from ch1, ch2, ch3, and ch4 and channelGroup_1 + // snippet.end + + await ready; + const sender = friend(); + await sender.publish({ channel: 'ch4', message: { n: 4, run } }); + await sender.publish({ channel: member, message: { n: 5, run } }); + await onLateChannel; + await onGroup; + subscriptionSet1.unsubscribe(); + }); +}); diff --git a/test/integration/live/coverage.json b/test/integration/live/coverage.json new file mode 100644 index 000000000..9381ec5d8 --- /dev/null +++ b/test/integration/live/coverage.json @@ -0,0 +1,1587 @@ +{ + "$comment": "GENERATED by npm run test:integration:live:manifest from the markers in test/integration/live (and the mocked allow-list in support/snippet-scan.ts). DO NOT EDIT. One entry per snippet id. Deliberate exceptions live in snippet-exceptions.json.", + "snippets": { + "AddToExistingSubscriptionSet": { + "file": "test/integration/live/api-reference/subscribe.test.ts", + "test": "api-reference/subscribe AddToExistingSubscriptionSet: adding sets and subscriptions to an already subscribed set takes effect at once", + "role": "open", + "mode": "live" + }, + "OLDUnsubscribeAllBasicUsage": { + "file": "test/integration/live/api-reference/subscribe-legacy.test.ts", + "test": "api-reference/subscribe-legacy OLDUnsubscribeAllBasicUsage: unsubscribeAll clears all channels and groups", + "role": "open", + "mode": "live" + }, + "OLDUnsubscribeBasicUsage": { + "file": "test/integration/live/api-reference/subscribe-legacy.test.ts", + "test": "api-reference/subscribe-legacy OLDUnsubscribeBasicUsage: unsubscribe with channels drops that channel", + "role": "open", + "mode": "live" + }, + "OLDUnsubscribeMultipleChannelGroup": { + "file": "test/integration/live/api-reference/subscribe-legacy.test.ts", + "test": "api-reference/subscribe-legacy OLDUnsubscribeMultipleChannelGroup: unsubscribing two of three groups leaves exactly the third", + "role": "open", + "mode": "live" + }, + "OLDUnsubscribeMultipleChannels": { + "file": "test/integration/live/api-reference/subscribe-legacy.test.ts", + "test": "api-reference/subscribe-legacy OLDUnsubscribeMultipleChannels: unsubscribing three of four channels leaves exactly the fourth", + "role": "open", + "mode": "live" + }, + "OLDaddListeners": { + "file": "test/integration/live/api-reference/subscribe-legacy.test.ts", + "test": "api-reference/subscribe-legacy OLDaddListeners: the legacy listener sample handlers see status, message and presence events whose fields exist on live events", + "role": "open", + "mode": "live" + }, + "OLDremoveListeners": { + "file": "test/integration/live/api-reference/subscribe-legacy.test.ts", + "test": "api-reference/subscribe-legacy OLDremoveListeners: removeListener leaves the other listeners of the client working, and the removed object receives nothing", + "role": "open", + "mode": "live" + }, + "OLDsubscribeBasicUsage": { + "file": "test/integration/live/api-reference/subscribe-legacy.test.ts", + "test": "api-reference/subscribe-legacy OLDsubscribeBasicUsage: the legacy subscribe call connects and delivers a message", + "role": "open", + "mode": "live" + }, + "OLDsubscribeChannelGroup": { + "file": "test/integration/live/api-reference/subscribe-legacy.test.ts", + "test": "api-reference/subscribe-legacy OLDsubscribeChannelGroup: subscribing to a channel group delivers a message from its member channel", + "role": "open", + "mode": "live" + }, + "OLDsubscribeChannelGroupAndChannels": { + "file": "test/integration/live/api-reference/subscribe-legacy.test.ts", + "test": "api-reference/subscribe-legacy OLDsubscribeChannelGroupAndChannels: channels and groups can be mixed in one call, shown by a message through each", + "role": "open", + "mode": "live" + }, + "OLDsubscribeChannelGroupWithPresence": { + "file": "test/integration/live/api-reference/subscribe-legacy.test.ts", + "test": "api-reference/subscribe-legacy OLDsubscribeChannelGroupWithPresence: a group subscription with presence yields a join event that belongs to the group", + "role": "open", + "mode": "live" + }, + "OLDsubscribeMultipleChannelGroup": { + "file": "test/integration/live/api-reference/subscribe-legacy.test.ts", + "test": "api-reference/subscribe-legacy OLDsubscribeMultipleChannelGroup: one call subscribes to three groups, shown by a message through the last group", + "role": "open", + "mode": "live" + }, + "OLDsubscribeMultipleChannels": { + "file": "test/integration/live/api-reference/subscribe-legacy.test.ts", + "test": "api-reference/subscribe-legacy OLDsubscribeMultipleChannels: one legacy subscribe call covers three channels, shown by a message on the middle one", + "role": "open", + "mode": "live" + }, + "OLDsubscribeWithPresence": { + "file": "test/integration/live/api-reference/subscribe-legacy.test.ts", + "test": "api-reference/subscribe-legacy OLDsubscribeWithPresence: withPresence true produces the own join event with the fields the page table lists", + "role": "open", + "mode": "live" + }, + "OLDsubscribeWithState": { + "file": "test/integration/live/api-reference/subscribe-legacy.test.ts", + "test": "api-reference/subscribe-legacy OLDsubscribeWithState: the connected status handler can set state, and the observer reads it back per channel", + "role": "open", + "mode": "live" + }, + "OLDsubscribeWithWildCardChannels": { + "file": "test/integration/live/api-reference/subscribe-legacy.test.ts", + "test": "api-reference/subscribe-legacy OLDsubscribeWithWildCardChannels: a wildcard subscription receives a message on a matching channel", + "role": "open", + "mode": "live" + }, + "PubNubinitBasicUsage": { + "file": "test/integration/live/shared/import-pubnub.test.ts", + "test": "shared/import-pubnub PubNubinitBasicUsage: the generic init sample builds a working client and time() succeeds", + "mode": "live" + }, + "SubscriptionSetFrom2Sets": { + "file": "test/integration/live/api-reference/subscribe.test.ts", + "test": "api-reference/subscribe SubscriptionSetFrom2Sets: addSubscriptionSet and removeSubscriptionSet are local operations and subscribe nothing", + "role": "open", + "mode": "live" + }, + "accessManagerBasicUsage": { + "file": "test/integration/live/api-reference/access-manager.test.ts", + "test": "api-reference/access-manager accessManagerBasicUsage: grantToken returns a token whose parsed form carries the user id, channel permissions and ttl that were asked for", + "role": "pam", + "mode": "live" + }, + "addChannelsToGroupBasicUsage": { + "file": "test/integration/live/api-reference/channel-groups.test.ts", + "test": "api-reference/channel-groups addChannelsToGroupBasicUsage: addChannels adds two channels to a group, and a second client reads them back from the group", + "role": "open", + "mode": "live" + }, + "addDataSyncListener": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync events and subscriptions addDataSyncListener: the listener switch sees update, delete and create events for one product", + "role": "datasync", + "mode": "live" + }, + "addDeciveToChannelBasicUsage": { + "file": "test/integration/live/api-reference/mobile-push.test.ts", + "test": "api-reference/mobile-push addDeciveToChannelBasicUsage: addChannels registers a device token for a channel and another client reads the registration back", + "role": "open", + "mode": "live" + }, + "addMessageActionBasicUsage": { + "file": "test/integration/live/api-reference/message-actions.test.ts", + "test": "api-reference/message-actions addMessageActionBasicUsage: addMessageAction returns the action and a subscriber receives an added event", + "role": "open", + "mode": "live" + }, + "automatedPollingPublishTriggeredPoll": { + "file": "test/integration/live/use-cases/automated-polling.test.ts", + "test": "use-cases/automated-polling automatedPollingPublishTriggeredPoll, automatedPollingThrottleTriggers, automatedPollingReceiveTrigger: a trigger opens the poll for its reaction, a second trigger inside the cooldown does not", + "role": "open", + "mode": "live" + }, + "automatedPollingReceiveTrigger": { + "file": "test/integration/live/use-cases/automated-polling.test.ts", + "test": "use-cases/automated-polling automatedPollingPublishTriggeredPoll, automatedPollingThrottleTriggers, automatedPollingReceiveTrigger: a trigger opens the poll for its reaction, a second trigger inside the cooldown does not", + "role": "open", + "mode": "live" + }, + "automatedPollingThrottleTriggers": { + "file": "test/integration/live/use-cases/automated-polling.test.ts", + "test": "use-cases/automated-polling automatedPollingPublishTriggeredPoll, automatedPollingThrottleTriggers, automatedPollingReceiveTrigger: a trigger opens the poll for its reaction, a second trigger inside the cooldown does not", + "role": "open", + "mode": "live" + }, + "basicUsageWithPromises": { + "file": "test/integration/live/api-reference/presence.test.ts", + "test": "api-reference/presence basicUsageWithPromises: the promise form of hereNow resolves with the occupants, seen through a spy", + "mode": "live" + }, + "buildNotificationPayloadBasicUsage": { + "file": "test/integration/live/api-reference/mobile-push.test.ts", + "test": "api-reference/mobile-push buildNotificationPayloadBasicUsage: the notification payload builder output is published and a subscriber receives the message", + "role": "open", + "mode": "live" + }, + "chatModerationApplyModeratorToken": { + "file": "test/integration/live/use-cases/game-chat-moderation.test.ts", + "test": "use-cases/game-chat-moderation chatModerationGrantControlChannelAccess, chatModerationApplyModeratorToken, chatModerationFlagMessage, chatModerationReceiveModerationDecisions, chatModerationLoadHistoryWithFlags, chatModerationDeleteMessage: a moderator token lets moderator-7 flag a message, clients hide it live and from history, and the server deletes it for good", + "role": "pam", + "mode": "live" + }, + "chatModerationDeleteMessage": { + "file": "test/integration/live/use-cases/game-chat-moderation.test.ts", + "test": "use-cases/game-chat-moderation chatModerationGrantControlChannelAccess, chatModerationApplyModeratorToken, chatModerationFlagMessage, chatModerationReceiveModerationDecisions, chatModerationLoadHistoryWithFlags, chatModerationDeleteMessage: a moderator token lets moderator-7 flag a message, clients hide it live and from history, and the server deletes it for good", + "role": "pam", + "mode": "live" + }, + "chatModerationFlagMessage": { + "file": "test/integration/live/use-cases/game-chat-moderation.test.ts", + "test": "use-cases/game-chat-moderation chatModerationGrantControlChannelAccess, chatModerationApplyModeratorToken, chatModerationFlagMessage, chatModerationReceiveModerationDecisions, chatModerationLoadHistoryWithFlags, chatModerationDeleteMessage: a moderator token lets moderator-7 flag a message, clients hide it live and from history, and the server deletes it for good", + "role": "pam", + "mode": "live" + }, + "chatModerationGrantControlChannelAccess": { + "file": "test/integration/live/use-cases/game-chat-moderation.test.ts", + "test": "use-cases/game-chat-moderation chatModerationGrantControlChannelAccess, chatModerationApplyModeratorToken, chatModerationFlagMessage, chatModerationReceiveModerationDecisions, chatModerationLoadHistoryWithFlags, chatModerationDeleteMessage: a moderator token lets moderator-7 flag a message, clients hide it live and from history, and the server deletes it for good", + "role": "pam", + "mode": "live" + }, + "chatModerationLoadHistoryWithFlags": { + "file": "test/integration/live/use-cases/game-chat-moderation.test.ts", + "test": "use-cases/game-chat-moderation chatModerationGrantControlChannelAccess, chatModerationApplyModeratorToken, chatModerationFlagMessage, chatModerationReceiveModerationDecisions, chatModerationLoadHistoryWithFlags, chatModerationDeleteMessage: a moderator token lets moderator-7 flag a message, clients hide it live and from history, and the server deletes it for good", + "role": "pam", + "mode": "live" + }, + "chatModerationReceiveModerationDecisions": { + "file": "test/integration/live/use-cases/game-chat-moderation.test.ts", + "test": "use-cases/game-chat-moderation chatModerationGrantControlChannelAccess, chatModerationApplyModeratorToken, chatModerationFlagMessage, chatModerationReceiveModerationDecisions, chatModerationLoadHistoryWithFlags, chatModerationDeleteMessage: a moderator token lets moderator-7 flag a message, clients hide it live and from history, and the server deletes it for good", + "role": "pam", + "mode": "live" + }, + "combineDataSyncSubscriptions": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync events and subscriptions combineDataSyncSubscriptions: a user and an entity subscription combined in one set both deliver events to one listener", + "role": "datasync", + "mode": "live" + }, + "configurationBasicUsage": { + "file": "test/integration/live/api-reference/configuration.test.ts", + "test": "api-reference/configuration configurationBasicUsage: the full constructor sample yields a working client with its user id, auth key, crypto module and Debug logging, and time() succeeds", + "mode": "live" + }, + "configurationCryptoModule": { + "file": "test/integration/live/api-reference/configuration.test.ts", + "test": "api-reference/configuration configurationCryptoModule: two clients with different crypto modules decrypt each other's data as the sample comments claim", + "mode": "live" + }, + "configurationRealOnlyClient": { + "file": "test/integration/live/api-reference/configuration.test.ts", + "test": "api-reference/configuration configurationRealOnlyClient: a client with only a subscribe key works for time() and refuses to publish locally", + "mode": "live" + }, + "configurationServerOpertaion": { + "file": "test/integration/live/api-reference/configuration.test.ts", + "test": "api-reference/configuration configurationServerOpertaion: the server configuration sample builds a client with the secret key and its user id, and time() succeeds", + "mode": "live" + }, + "createChannelBasicUsage": { + "file": "test/integration/live/api-reference/subscribe.test.ts", + "test": "api-reference/subscribe createChannelBasicUsage: the channel entity factory returns the channel name and does not subscribe", + "role": "open", + "mode": "live" + }, + "createChannelGroupBasicUsage": { + "file": "test/integration/live/api-reference/subscribe.test.ts", + "test": "api-reference/subscribe createChannelGroupBasicUsage: the channel group entity carries the group name", + "role": "open", + "mode": "live" + }, + "createChannelMetadataBasicUsage": { + "file": "test/integration/live/api-reference/subscribe.test.ts", + "test": "api-reference/subscribe createChannelMetadataBasicUsage: the channel metadata entity carries the id and makes no network call", + "role": "open", + "mode": "live" + }, + "createDataSyncChannelBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync channels createDataSyncChannelBasicUsage: createChannel creates the drop channel and echoes the payload", + "role": "datasync", + "mode": "live" + }, + "createEntityBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync entities (class product) createEntityBasicUsage: createEntity creates the product entity that later tests use", + "role": "datasync", + "mode": "live" + }, + "createMembershipBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync memberships createMembershipBasicUsage: createMembership links Alice to the channel with the fixed id that later tests use", + "role": "datasync", + "mode": "live" + }, + "createRelationshipBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync relationships (class ProductOwner) createRelationshipBasicUsage: createRelationship links the seller to the product with the fixed id that later tests use", + "role": "datasync", + "mode": "live" + }, + "createSubscription": { + "file": "test/integration/live/api-reference/subscribe.test.ts", + "test": "api-reference/subscribe createSubscription: creating a subscription object does not subscribe anything", + "role": "open", + "mode": "live" + }, + "createUserBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync users createUserBasicUsage: createUser creates Alice, echoes the payload and returns an eTag", + "role": "datasync", + "mode": "live" + }, + "createUserMetadataBasicUsage": { + "file": "test/integration/live/api-reference/subscribe.test.ts", + "test": "api-reference/subscribe createUserMetadataBasicUsage: the user metadata entity carries the id and makes no network call", + "role": "open", + "mode": "live" + }, + "dataSyncEntitySubscribeExample": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync events and subscriptions dataSyncEntitySubscribeExample: the entity subscribe sample connects, and an updated entity arrives as an event", + "role": "datasync", + "mode": "live" + }, + "dataSyncProjectionSubscriptions": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync events and subscriptions dataSyncProjectionSubscriptions: two projection subscriptions connect, and unsubscribing one leaves the other subscribed", + "role": "datasync", + "mode": "live" + }, + "decryptBasicUsage": { + "file": "test/integration/live/api-reference/encryption-legacy.test.ts", + "test": "api-reference/encryption-legacy decryptBasicUsage: the legacy decrypt returns the original JSON for text encrypted with the same cipher key", + "role": "open", + "mode": "live" + }, + "decryptFileBasicUsage": { + "file": "test/integration/live/api-reference/encryption.test.ts", + "test": "api-reference/encryption decryptFileBasicUsage: decryptFile restores the picture byte for byte with a second client that has the same cipher key", + "role": "open", + "mode": "live" + }, + "deleteChannelGroupBasicUsage": { + "file": "test/integration/live/api-reference/channel-groups.test.ts", + "test": "api-reference/channel-groups deleteChannelGroupBasicUsage: deleteGroup removes the group, and a deleted group lists no channels", + "role": "open", + "mode": "live" + }, + "deleteFileBasicUsage": { + "file": "test/integration/live/api-reference/file-sharing.test.ts", + "test": "api-reference/file-sharing deleteFileBasicUsage: deleteFile removes the file and a later listing no longer shows it", + "role": "open", + "mode": "live" + }, + "deleteMessagesBasicUsage": { + "file": "test/integration/live/api-reference/message-persistence.test.ts", + "test": "api-reference/message-persistence deleteMessagesBasicUsage: deleteMessages removes exactly the message at the boundary timetoken", + "role": "open", + "mode": "live" + }, + "deleteProxy": { + "file": "test/integration/live/api-reference/misc.test.ts", + "test": "api-reference/misc deleteProxy: setProxy with no argument removes the proxy so the next request adds no tunnel", + "role": "open", + "mode": "live" + }, + "deleteSpecificMessages": { + "file": "test/integration/live/api-reference/message-persistence.test.ts", + "test": "api-reference/message-persistence deleteSpecificMessages: the second deleteMessages sample removes exactly the message at its boundary timetoken", + "role": "open", + "mode": "live" + }, + "destroyBasicUsage": { + "file": "test/integration/live/api-reference/configuration.test.ts", + "test": "api-reference/configuration destroyBasicUsage: destroy() stops a subscribed client and clears its subscriptions", + "role": "open", + "mode": "live" + }, + "disconnectBasicUsage": { + "file": "test/integration/live/api-reference/misc.test.ts", + "test": "api-reference/misc disconnectBasicUsage: disconnect() reports a disconnected status and no message arrives afterwards", + "role": "open", + "mode": "live" + }, + "downloadFileCustomCipherKey": { + "file": "test/integration/live/api-reference/file-sharing-legacy.test.ts", + "test": "api-reference/file-sharing-legacy downloadFileCustomCipherKey: downloadFile with the right custom cipher key returns the file and without the key it does not", + "role": "open", + "mode": "live" + }, + "downloadFileNodeBasicUsage": { + "file": "test/integration/live/api-reference/file-sharing.test.ts", + "test": "api-reference/file-sharing downloadFileNodeBasicUsage: downloadFile in Node writes the downloaded content to the output file", + "role": "open", + "mode": "live" + }, + "encryptFileBasicUsage": { + "file": "test/integration/live/api-reference/encryption.test.ts", + "test": "api-reference/encryption encryptFileBasicUsage: encryptFile produces an encrypted copy of the picture", + "role": "open", + "mode": "live" + }, + "encryptMessageBasicUsage": { + "file": "test/integration/live/api-reference/encryption.test.ts", + "test": "api-reference/encryption encryptMessageBasicUsage: encrypt turns a message into cipher text that differs from the plain text, and decrypt reads it back", + "mode": "live" + }, + "eventListenerAddConnectionStatusListenersBasicUsage": { + "file": "test/integration/live/api-reference/subscribe.test.ts", + "test": "api-reference/subscribe eventListenerAddConnectionStatusListenersBasicUsage: the status listener fires PNConnectedCategory", + "role": "open", + "mode": "live" + }, + "eventListenerAddListeners": { + "file": "test/integration/live/api-reference/subscribe.test.ts", + "test": "api-reference/subscribe eventListenerAddListeners: the message, signal and message action handlers of the sample log when the events arrive", + "role": "open", + "mode": "live" + }, + "eventListenerBasicUsage": { + "file": "test/integration/live/api-reference/subscribe.test.ts", + "test": "api-reference/subscribe eventListenerBasicUsage: status, message, presence and set level message action listeners all fire from one sample", + "role": "open", + "mode": "live" + }, + "fanBehaviorApplyToken": { + "file": "test/integration/live/use-cases/fan-behavior-management.test.ts", + "test": "use-cases/fan-behavior-management fanBehaviorStateStore, fanBehaviorGrantChatAccess, fanBehaviorHandleAccessDenied, fanBehaviorApplyToken, fanBehaviorSubscribeAndPublish, fanBehaviorMuteFan, fanBehaviorBanFan, fanBehaviorUnmuteFan, fanBehaviorServerDispatch, fanBehaviorInspectToken: a fan is granted, muted, banned and let back in, and the fan client sees each change", + "role": "pam", + "mode": "live" + }, + "fanBehaviorBanFan": { + "file": "test/integration/live/use-cases/fan-behavior-management.test.ts", + "test": "use-cases/fan-behavior-management fanBehaviorStateStore, fanBehaviorGrantChatAccess, fanBehaviorHandleAccessDenied, fanBehaviorApplyToken, fanBehaviorSubscribeAndPublish, fanBehaviorMuteFan, fanBehaviorBanFan, fanBehaviorUnmuteFan, fanBehaviorServerDispatch, fanBehaviorInspectToken: a fan is granted, muted, banned and let back in, and the fan client sees each change", + "role": "pam", + "mode": "live" + }, + "fanBehaviorGrantChatAccess": { + "file": "test/integration/live/use-cases/fan-behavior-management.test.ts", + "test": "use-cases/fan-behavior-management fanBehaviorStateStore, fanBehaviorGrantChatAccess, fanBehaviorHandleAccessDenied, fanBehaviorApplyToken, fanBehaviorSubscribeAndPublish, fanBehaviorMuteFan, fanBehaviorBanFan, fanBehaviorUnmuteFan, fanBehaviorServerDispatch, fanBehaviorInspectToken: a fan is granted, muted, banned and let back in, and the fan client sees each change", + "role": "pam", + "mode": "live" + }, + "fanBehaviorHandleAccessDenied": { + "file": "test/integration/live/use-cases/fan-behavior-management.test.ts", + "test": "use-cases/fan-behavior-management fanBehaviorStateStore, fanBehaviorGrantChatAccess, fanBehaviorHandleAccessDenied, fanBehaviorApplyToken, fanBehaviorSubscribeAndPublish, fanBehaviorMuteFan, fanBehaviorBanFan, fanBehaviorUnmuteFan, fanBehaviorServerDispatch, fanBehaviorInspectToken: a fan is granted, muted, banned and let back in, and the fan client sees each change", + "role": "pam", + "mode": "live" + }, + "fanBehaviorInspectToken": { + "file": "test/integration/live/use-cases/fan-behavior-management.test.ts", + "test": "use-cases/fan-behavior-management fanBehaviorStateStore, fanBehaviorGrantChatAccess, fanBehaviorHandleAccessDenied, fanBehaviorApplyToken, fanBehaviorSubscribeAndPublish, fanBehaviorMuteFan, fanBehaviorBanFan, fanBehaviorUnmuteFan, fanBehaviorServerDispatch, fanBehaviorInspectToken: a fan is granted, muted, banned and let back in, and the fan client sees each change", + "role": "pam", + "mode": "live" + }, + "fanBehaviorMuteFan": { + "file": "test/integration/live/use-cases/fan-behavior-management.test.ts", + "test": "use-cases/fan-behavior-management fanBehaviorStateStore, fanBehaviorGrantChatAccess, fanBehaviorHandleAccessDenied, fanBehaviorApplyToken, fanBehaviorSubscribeAndPublish, fanBehaviorMuteFan, fanBehaviorBanFan, fanBehaviorUnmuteFan, fanBehaviorServerDispatch, fanBehaviorInspectToken: a fan is granted, muted, banned and let back in, and the fan client sees each change", + "role": "pam", + "mode": "live" + }, + "fanBehaviorServerDispatch": { + "file": "test/integration/live/use-cases/fan-behavior-management.test.ts", + "test": "use-cases/fan-behavior-management fanBehaviorStateStore, fanBehaviorGrantChatAccess, fanBehaviorHandleAccessDenied, fanBehaviorApplyToken, fanBehaviorSubscribeAndPublish, fanBehaviorMuteFan, fanBehaviorBanFan, fanBehaviorUnmuteFan, fanBehaviorServerDispatch, fanBehaviorInspectToken: a fan is granted, muted, banned and let back in, and the fan client sees each change", + "role": "pam", + "mode": "live" + }, + "fanBehaviorStateStore": { + "file": "test/integration/live/use-cases/fan-behavior-management.test.ts", + "test": "use-cases/fan-behavior-management fanBehaviorStateStore, fanBehaviorGrantChatAccess, fanBehaviorHandleAccessDenied, fanBehaviorApplyToken, fanBehaviorSubscribeAndPublish, fanBehaviorMuteFan, fanBehaviorBanFan, fanBehaviorUnmuteFan, fanBehaviorServerDispatch, fanBehaviorInspectToken: a fan is granted, muted, banned and let back in, and the fan client sees each change", + "role": "pam", + "mode": "live" + }, + "fanBehaviorSubscribeAndPublish": { + "file": "test/integration/live/use-cases/fan-behavior-management.test.ts", + "test": "use-cases/fan-behavior-management fanBehaviorStateStore, fanBehaviorGrantChatAccess, fanBehaviorHandleAccessDenied, fanBehaviorApplyToken, fanBehaviorSubscribeAndPublish, fanBehaviorMuteFan, fanBehaviorBanFan, fanBehaviorUnmuteFan, fanBehaviorServerDispatch, fanBehaviorInspectToken: a fan is granted, muted, banned and let back in, and the fan client sees each change", + "role": "pam", + "mode": "live" + }, + "fanBehaviorUnmuteFan": { + "file": "test/integration/live/use-cases/fan-behavior-management.test.ts", + "test": "use-cases/fan-behavior-management fanBehaviorStateStore, fanBehaviorGrantChatAccess, fanBehaviorHandleAccessDenied, fanBehaviorApplyToken, fanBehaviorSubscribeAndPublish, fanBehaviorMuteFan, fanBehaviorBanFan, fanBehaviorUnmuteFan, fanBehaviorServerDispatch, fanBehaviorInspectToken: a fan is granted, muted, banned and let back in, and the fan client sees each change", + "role": "pam", + "mode": "live" + }, + "fanChatAddReaction": { + "file": "test/integration/live/use-cases/real-time-chat.test.ts", + "test": "use-cases/real-time-chat fanChatCreateChannelMetadata, fanChatSetFanProfile, fanChatSubscribeWithPresence, fanChatSendMessage, fanChatLoadRecentMessages, fanChatCountFansOnline, fanChatReceiveReactions, fanChatAddReaction: two fans chat, see presence and history, and a reaction reaches the subscriber", + "role": "open", + "mode": "live" + }, + "fanChatCountFansOnline": { + "file": "test/integration/live/use-cases/real-time-chat.test.ts", + "test": "use-cases/real-time-chat fanChatCreateChannelMetadata, fanChatSetFanProfile, fanChatSubscribeWithPresence, fanChatSendMessage, fanChatLoadRecentMessages, fanChatCountFansOnline, fanChatReceiveReactions, fanChatAddReaction: two fans chat, see presence and history, and a reaction reaches the subscriber", + "role": "open", + "mode": "live" + }, + "fanChatCreateChannelMetadata": { + "file": "test/integration/live/use-cases/real-time-chat.test.ts", + "test": "use-cases/real-time-chat fanChatCreateChannelMetadata, fanChatSetFanProfile, fanChatSubscribeWithPresence, fanChatSendMessage, fanChatLoadRecentMessages, fanChatCountFansOnline, fanChatReceiveReactions, fanChatAddReaction: two fans chat, see presence and history, and a reaction reaches the subscriber", + "role": "open", + "mode": "live" + }, + "fanChatLoadRecentMessages": { + "file": "test/integration/live/use-cases/real-time-chat.test.ts", + "test": "use-cases/real-time-chat fanChatCreateChannelMetadata, fanChatSetFanProfile, fanChatSubscribeWithPresence, fanChatSendMessage, fanChatLoadRecentMessages, fanChatCountFansOnline, fanChatReceiveReactions, fanChatAddReaction: two fans chat, see presence and history, and a reaction reaches the subscriber", + "role": "open", + "mode": "live" + }, + "fanChatReceiveReactions": { + "file": "test/integration/live/use-cases/real-time-chat.test.ts", + "test": "use-cases/real-time-chat fanChatCreateChannelMetadata, fanChatSetFanProfile, fanChatSubscribeWithPresence, fanChatSendMessage, fanChatLoadRecentMessages, fanChatCountFansOnline, fanChatReceiveReactions, fanChatAddReaction: two fans chat, see presence and history, and a reaction reaches the subscriber", + "role": "open", + "mode": "live" + }, + "fanChatSendMessage": { + "file": "test/integration/live/use-cases/real-time-chat.test.ts", + "test": "use-cases/real-time-chat fanChatCreateChannelMetadata, fanChatSetFanProfile, fanChatSubscribeWithPresence, fanChatSendMessage, fanChatLoadRecentMessages, fanChatCountFansOnline, fanChatReceiveReactions, fanChatAddReaction: two fans chat, see presence and history, and a reaction reaches the subscriber", + "role": "open", + "mode": "live" + }, + "fanChatSetFanProfile": { + "file": "test/integration/live/use-cases/real-time-chat.test.ts", + "test": "use-cases/real-time-chat fanChatCreateChannelMetadata, fanChatSetFanProfile, fanChatSubscribeWithPresence, fanChatSendMessage, fanChatLoadRecentMessages, fanChatCountFansOnline, fanChatReceiveReactions, fanChatAddReaction: two fans chat, see presence and history, and a reaction reaches the subscriber", + "role": "open", + "mode": "live" + }, + "fanChatSubscribeWithPresence": { + "file": "test/integration/live/use-cases/real-time-chat.test.ts", + "test": "use-cases/real-time-chat fanChatCreateChannelMetadata, fanChatSetFanProfile, fanChatSubscribeWithPresence, fanChatSendMessage, fanChatLoadRecentMessages, fanChatCountFansOnline, fanChatReceiveReactions, fanChatAddReaction: two fans chat, see presence and history, and a reaction reaches the subscriber", + "role": "open", + "mode": "live" + }, + "fetchMessagesBasicUsage": { + "file": "test/integration/live/api-reference/message-persistence.test.ts", + "test": "api-reference/message-persistence fetchMessagesBasicUsage: fetchMessages with count, start and end returns one message of the window", + "role": "open", + "mode": "live" + }, + "fetchMessagesWithMetaActions": { + "file": "test/integration/live/api-reference/message-persistence.test.ts", + "test": "api-reference/message-persistence fetchMessagesWithMetaActions: fetchMessages returns meta, message actions and the custom message type when asked for", + "role": "open", + "mode": "live" + }, + "fireBasicUsage": { + "file": "test/integration/live/api-reference/publish.test.ts", + "test": "api-reference/publish fireBasicUsage: fire sends to Functions only, so subscribers receive nothing and history stores nothing", + "role": "open", + "mode": "live" + }, + "getAllChannelMetadataBasicUsage": { + "file": "test/integration/live/api-reference/objects.test.ts", + "test": "api-reference/objects getAllChannelMetadataBasicUsage: getAllChannelMetadata with the Team filter returns the seeded channel", + "role": "open", + "mode": "live" + }, + "getAllUUIDMetadataBasicUsage": { + "file": "test/integration/live/api-reference/objects.test.ts", + "test": "api-reference/objects getAllUUIDMetadataBasicUsage: getAllUUIDMetadata runs the sample function and logs the user list", + "role": "open", + "mode": "live" + }, + "getAuthKeyBasicUsage": { + "file": "test/integration/live/api-reference/configuration.test.ts", + "test": "api-reference/configuration getAuthKeyBasicUsage: getAuthKey returns the auth key that was set", + "role": "open", + "mode": "live" + }, + "getChannelBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync channels getChannelBasicUsage: getChannel reads the DataSync channel back by id", + "role": "datasync", + "mode": "live" + }, + "getChannelMembersBasicUsage": { + "file": "test/integration/live/api-reference/objects.test.ts", + "test": "api-reference/objects getChannelMembersBasicUsage: getChannelMembers with the admin filter returns only the matching member", + "role": "open", + "mode": "live" + }, + "getChannelMetadataBasicUsage": { + "file": "test/integration/live/api-reference/objects.test.ts", + "test": "api-reference/objects getChannelMetadataBasicUsage: getChannelMetadata reads the seeded channel by id", + "role": "open", + "mode": "live" + }, + "getChannelsBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync channels getChannelsBasicUsage: getChannels lists channels with the data array and meta.has_next envelope", + "role": "datasync", + "mode": "live" + }, + "getChannelsCursorPagingExample": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync channels getChannelsCursorPagingExample: the cursor paging loop for channels runs and lists the channel", + "role": "datasync", + "mode": "live" + }, + "getChannelsFilterExample": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync channels getChannelsFilterExample: getChannels with filter finds the new channel once the index has caught up", + "role": "datasync", + "mode": "live" + }, + "getChannelsFilterFastExample": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync channels getChannelsFilterFastExample: getChannels with filterFast finds the channel at once", + "role": "datasync", + "mode": "live" + }, + "getDataSyncMembershipBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync memberships getDataSyncMembershipBasicUsage: getMembership reads the membership by its fixed id", + "role": "datasync", + "mode": "live" + }, + "getEntitiesBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync entities (class product) getEntitiesBasicUsage: getEntities lists products sorted by price descending, verified on the returned prices", + "role": "datasync", + "mode": "live" + }, + "getEntitiesCursorPagingExample": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync entities (class product) getEntitiesCursorPagingExample: the cursor paging loop for products runs and lists the product", + "role": "datasync", + "mode": "live" + }, + "getEntitiesFilterExample": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync entities (class product) getEntitiesFilterExample: getEntities with filter finds the product once the index has caught up", + "role": "datasync", + "mode": "live" + }, + "getEntitiesFilterFastExample": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync entities (class product) getEntitiesFilterFastExample: getEntities with filterFast finds the product at once", + "role": "datasync", + "mode": "live" + }, + "getEntityBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync entities (class product) getEntityBasicUsage: getEntity reads the product by id", + "role": "datasync", + "mode": "live" + }, + "getFileUrlBasicUsage": { + "file": "test/integration/live/api-reference/file-sharing.test.ts", + "test": "api-reference/file-sharing getFileUrlBasicUsage: getFileUrl builds a URL that serves the uploaded bytes", + "role": "open", + "mode": "live" + }, + "getFilterExpressionBasicUsage": { + "file": "test/integration/live/api-reference/configuration.test.ts", + "test": "api-reference/configuration getFilterExpressionBasicUsage: getFilterExpression returns the expression that was set", + "role": "open", + "mode": "live" + }, + "getMembershipBasicUsage": { + "file": "test/integration/live/api-reference/objects.test.ts", + "test": "api-reference/objects getMembershipBasicUsage: getMemberships with the starred filter returns exactly the one starred membership", + "role": "open", + "mode": "live" + }, + "getMembershipsBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync memberships getMembershipsBasicUsage: getMemberships lists exactly the membership of Alice", + "role": "datasync", + "mode": "live" + }, + "getMembershipsByChannelIdExample": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync memberships getMembershipsByChannelIdExample: getMemberships by channel id lists Alice as the only member", + "role": "datasync", + "mode": "live" + }, + "getMembershipsCursorPagingExample": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync memberships getMembershipsCursorPagingExample: the cursor paging loop for memberships runs and lists the membership", + "role": "datasync", + "mode": "live" + }, + "getMembershipsFilterExample": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync memberships getMembershipsFilterExample: getMemberships with filter excludes a seeded archived membership once the index has caught up", + "role": "datasync", + "mode": "live" + }, + "getMembershipsFilterFastExample": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync memberships getMembershipsFilterFastExample: getMemberships with filterFast excludes the seeded archived membership at once", + "role": "datasync", + "mode": "live" + }, + "getMessageActionsBasicUsage": { + "file": "test/integration/live/api-reference/message-actions.test.ts", + "test": "api-reference/message-actions getMessageActionsBasicUsage: getMessageActions returns the action added during setup with the right type, value and message timetoken", + "role": "open", + "mode": "live" + }, + "getRelationshipBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync relationships (class ProductOwner) getRelationshipBasicUsage: getRelationship reads the link by its fixed id", + "role": "datasync", + "mode": "live" + }, + "getRelationshipsBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync relationships (class ProductOwner) getRelationshipsBasicUsage: getRelationships lists exactly the link from the seller side", + "role": "datasync", + "mode": "live" + }, + "getRelationshipsByEntityBIdExample": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync relationships (class ProductOwner) getRelationshipsByEntityBIdExample: getRelationships by entity B id lists the same link from the product side", + "role": "datasync", + "mode": "live" + }, + "getRelationshipsCursorPagingExample": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync relationships (class ProductOwner) getRelationshipsCursorPagingExample: the cursor paging loop for relationships runs and lists the link", + "role": "datasync", + "mode": "live" + }, + "getRelationshipsFilterExample": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync relationships (class ProductOwner) getRelationshipsFilterExample: a negated filter keeps the gold link once the index has caught up", + "role": "datasync", + "mode": "live" + }, + "getRelationshipsFilterFastExample": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync relationships (class ProductOwner) getRelationshipsFilterFastExample: filterFast on tier finds the link at once", + "role": "datasync", + "mode": "live" + }, + "getStateBasicUsage": { + "file": "test/integration/live/api-reference/presence.test.ts", + "test": "api-reference/presence getStateBasicUsage: getState returns the state set for the channel and the group", + "mode": "live" + }, + "getUUIDMetadataBasicUsage": { + "file": "test/integration/live/api-reference/objects.test.ts", + "test": "api-reference/objects getUUIDMetadataBasicUsage: getUUIDMetadata reads the users stored beforehand", + "role": "open", + "mode": "live" + }, + "getUserBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync users getUserBasicUsage: getUser reads Alice back by id", + "role": "datasync", + "mode": "live" + }, + "getUserIdBasicUsage": { + "file": "test/integration/live/api-reference/configuration.test.ts", + "test": "api-reference/configuration getUserIdBasicUsage: getUserId returns the user id given at construction", + "role": "open", + "mode": "live" + }, + "getUsersBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync users getUsersBasicUsage: getUsers lists users sorted by createdAt, verified on the returned timestamps", + "role": "datasync", + "mode": "live" + }, + "getUsersCursorPagingExample": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync users getUsersCursorPagingExample: the cursor paging loop for users runs and lists Alice", + "role": "datasync", + "mode": "live" + }, + "getUsersFilterExample": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync users getUsersFilterExample: getUsers with filter finds Alice once the eventually consistent index has caught up", + "role": "datasync", + "mode": "live" + }, + "getUsersFilterFastExample": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync users getUsersFilterFastExample: getUsers with filterFast finds Alice at once (strongly consistent)", + "role": "datasync", + "mode": "live" + }, + "gettingStartedCreateSubscription": { + "file": "test/integration/live/getting-started/quickstart-node.test.ts", + "test": "getting-started/quickstart-node gettingStartedInitPubnub, quickstartNodeListeners, gettingStartedCreateSubscription, quickstartNodePublish: the Node quickstart script connects, publishes and receives, and prints the lines the page promises", + "mode": "live" + }, + "gettingStartedInitPubnub": { + "file": "test/integration/live/getting-started/quickstart-node.test.ts", + "test": "getting-started/quickstart-node gettingStartedInitPubnub, quickstartNodeListeners, gettingStartedCreateSubscription, quickstartNodePublish: the Node quickstart script connects, publishes and receives, and prints the lines the page promises", + "mode": "live" + }, + "grantTokenDataSyncResources": { + "file": "test/integration/live/api-reference/access-manager.test.ts", + "test": "api-reference/access-manager grantTokenDataSyncResources: a token can carry DataSync entity scopes and the top-level users scope, and parseToken returns them", + "role": "datasync", + "mode": "live" + }, + "grantTokenRegExAndResources": { + "file": "test/integration/live/api-reference/access-manager.test.ts", + "test": "api-reference/access-manager grantTokenRegExAndResources: a token with resources and patterns returns both sections when parsed", + "role": "pam", + "mode": "live" + }, + "grantTokenSpacesBasicUsage": { + "file": "test/integration/live/api-reference/access-manager-legacy.test.ts", + "test": "api-reference/access-manager-legacy grantTokenSpacesBasicUsage: a grant that uses the deprecated spaces scope returns a token whose parsed form lists the space under channels", + "role": "pam", + "mode": "live" + }, + "grantTokenSpacesRegEx": { + "file": "test/integration/live/api-reference/access-manager-legacy.test.ts", + "test": "api-reference/access-manager-legacy grantTokenSpacesRegEx: a pattern-only grant with the deprecated spaces scope keeps the regular expression unchanged under patterns.channels", + "role": "pam", + "mode": "live" + }, + "grantTokenSpacesRegExAndResources": { + "file": "test/integration/live/api-reference/access-manager-legacy.test.ts", + "test": "api-reference/access-manager-legacy grantTokenSpacesRegExAndResources: resources and patterns in one deprecated-scope grant both survive in the parsed token", + "role": "pam", + "mode": "live" + }, + "grantTokenSpacesVariousResources": { + "file": "test/integration/live/api-reference/access-manager-legacy.test.ts", + "test": "api-reference/access-manager-legacy grantTokenSpacesVariousResources: a token with mixed spaces and users resources keeps one space entry and one user entry with their permissions", + "role": "pam", + "mode": "live" + }, + "grantTokenUsingRegEx": { + "file": "test/integration/live/api-reference/access-manager.test.ts", + "test": "api-reference/access-manager grantTokenUsingRegEx: a pattern-only token keeps the regular expression unchanged under patterns.channels", + "role": "pam", + "mode": "live" + }, + "grantTokenVariousResources": { + "file": "test/integration/live/api-reference/access-manager.test.ts", + "test": "api-reference/access-manager grantTokenVariousResources: a token with channels, groups and uuids keeps each resource with its permissions in the parsed form", + "role": "pam", + "mode": "live" + }, + "grantTokenWithCategories": { + "file": "test/integration/live/api-reference/access-manager.test.ts", + "test": "api-reference/access-manager grantTokenWithCategories: a category token carries get on all channel metadata and all uuid metadata", + "role": "pam", + "mode": "live" + }, + "grantTokenWithProjection": { + "file": "test/integration/live/api-reference/access-manager.test.ts", + "test": "api-reference/access-manager grantTokenWithProjection: a token with dataSyncProjections is created and parsed, and carries the projection assignment of the entity", + "role": "datasync", + "mode": "live" + }, + "handleRequestErrorStatusCode": { + "file": "test/integration/endpoints/time.test.ts", + "test": "time endpoints reports the HTTP status code of a failed request without retrying it when retries are disabled", + "mode": "mocked" + }, + "heartbeatFailureStatusListener": { + "file": "test/integration/components/subscription_manager.test.ts", + "test": "#components/subscription_manager reports a failed heartbeat to the status listener with category, operation and status code", + "mode": "mocked" + }, + "hereNowBasicUsage": { + "file": "test/integration/live/api-reference/presence.test.ts", + "test": "api-reference/presence hereNowBasicUsage: hereNow lists the occupant with its state and the group member channel appears through the group", + "role": "open", + "mode": "live" + }, + "hereNowChannelGroup": { + "file": "test/integration/live/api-reference/presence.test.ts", + "test": "api-reference/presence hereNowChannelGroup: hereNow on a channel group reports presence for each member channel", + "mode": "live" + }, + "hereNowFetchOccupancyOnly": { + "file": "test/integration/live/api-reference/presence.test.ts", + "test": "api-reference/presence hereNowFetchOccupancyOnly: hereNow with includeUUIDs false gives a count and no user list", + "mode": "live" + }, + "hereNowWithState": { + "file": "test/integration/live/api-reference/presence.test.ts", + "test": "api-reference/presence hereNowWithState: hereNow with includeState returns the state set in the fixture", + "mode": "live" + }, + "historyBasicUsage": { + "file": "test/integration/live/api-reference/message-persistence-legacy.test.ts", + "test": "api-reference/message-persistence-legacy historyBasicUsage: history returns the three seeded messages as the newest, oldest first, with the start and end timetokens", + "role": "open", + "mode": "live" + }, + "historyIncludeMeta": { + "file": "test/integration/live/api-reference/message-persistence-legacy.test.ts", + "test": "api-reference/message-persistence-legacy historyIncludeMeta: includeMeta returns meta for a message published with metadata and omits it for others", + "role": "open", + "mode": "live" + }, + "historyNewerThanTimetoken": { + "file": "test/integration/live/api-reference/message-persistence-legacy.test.ts", + "test": "api-reference/message-persistence-legacy historyNewerThanTimetoken: history with reverse and start returns only messages newer than the anchor, and start is exclusive", + "role": "open", + "mode": "live" + }, + "historyOldestThree": { + "file": "test/integration/live/api-reference/message-persistence-legacy.test.ts", + "test": "api-reference/message-persistence-legacy historyOldestThree: history with reverse true and count 3 returns the three oldest messages", + "role": "open", + "mode": "live" + }, + "historyPagingExample": { + "file": "test/integration/live/api-reference/message-persistence-legacy.test.ts", + "test": "api-reference/message-persistence-legacy historyPagingExample: the paging helper of the page collects all 120 messages across two pages", + "role": "open", + "mode": "live" + }, + "historyUntilTimetoken": { + "file": "test/integration/live/api-reference/message-persistence-legacy.test.ts", + "test": "api-reference/message-persistence-legacy historyUntilTimetoken: history with only end returns messages down to the anchor, and end is inclusive", + "role": "open", + "mode": "live" + }, + "historyWithPromises": { + "file": "test/integration/live/api-reference/message-persistence-legacy.test.ts", + "test": "api-reference/message-persistence-legacy historyWithPromises: the promise form of history logs the response with the expected messages", + "role": "open", + "mode": "live" + }, + "importFS": { + "file": "test/integration/live/shared/import-pubnub.test.ts", + "mode": "compile-only" + }, + "importPubNub": { + "file": "test/integration/live/shared/import-pubnub.test.ts", + "mode": "compile-only" + }, + "iterativelyUpdateExistingMetadata": { + "file": "test/integration/live/api-reference/objects.test.ts", + "test": "api-reference/objects iterativelyUpdateExistingMetadata: the read-modify-write pattern merges custom data and the merged object survives on the service", + "role": "open", + "mode": "live" + }, + "listChannelsForDeviceBasicUsage": { + "file": "test/integration/live/api-reference/mobile-push.test.ts", + "test": "api-reference/mobile-push listChannelsForDeviceBasicUsage: listChannels returns the channels registered for the device", + "role": "open", + "mode": "live" + }, + "listChannelsInGroupBasicUsage": { + "file": "test/integration/live/api-reference/channel-groups.test.ts", + "test": "api-reference/channel-groups listChannelsInGroupBasicUsage: listChannels returns the channels that were added to the group", + "role": "open", + "mode": "live" + }, + "listFilesBasicUsage": { + "file": "test/integration/live/api-reference/file-sharing.test.ts", + "test": "api-reference/file-sharing listFilesBasicUsage: listFiles returns the file this test uploaded", + "role": "open", + "mode": "live" + }, + "liveCommentaryLoadBacklog": { + "file": "test/integration/live/use-cases/live-commentary.test.ts", + "test": "use-cases/live-commentary liveCommentaryPublishRemark, liveCommentarySubscribe, liveCommentaryLoadBacklog, liveCommentaryUnsubscribe: a remark is stored, a viewer receives the next one live, catches up from the backlog and stops listening", + "role": "open", + "mode": "live" + }, + "liveCommentaryPublishRemark": { + "file": "test/integration/live/use-cases/live-commentary.test.ts", + "test": "use-cases/live-commentary liveCommentaryPublishRemark, liveCommentarySubscribe, liveCommentaryLoadBacklog, liveCommentaryUnsubscribe: a remark is stored, a viewer receives the next one live, catches up from the backlog and stops listening", + "role": "open", + "mode": "live" + }, + "liveCommentarySubscribe": { + "file": "test/integration/live/use-cases/live-commentary.test.ts", + "test": "use-cases/live-commentary liveCommentaryPublishRemark, liveCommentarySubscribe, liveCommentaryLoadBacklog, liveCommentaryUnsubscribe: a remark is stored, a viewer receives the next one live, catches up from the backlog and stops listening", + "role": "open", + "mode": "live" + }, + "liveCommentaryUnsubscribe": { + "file": "test/integration/live/use-cases/live-commentary.test.ts", + "test": "use-cases/live-commentary liveCommentaryPublishRemark, liveCommentarySubscribe, liveCommentaryLoadBacklog, liveCommentaryUnsubscribe: a remark is stored, a viewer receives the next one live, catches up from the backlog and stops listening", + "role": "open", + "mode": "live" + }, + "livePollsFetchOpenPoll": { + "file": "test/integration/live/use-cases/live-polls.test.ts", + "test": "use-cases/live-polls livePollsPublishPoll, livePollsSubscribeToPolls, livePollsFetchOpenPoll, livePollsPublishVote, livePollsSubscribeToResults, livePollsPublishResults: a poll opens, a late fan fetches it, votes, and both see the tally", + "role": "open", + "mode": "live" + }, + "livePollsPublishPoll": { + "file": "test/integration/live/use-cases/live-polls.test.ts", + "test": "use-cases/live-polls livePollsPublishPoll, livePollsSubscribeToPolls, livePollsFetchOpenPoll, livePollsPublishVote, livePollsSubscribeToResults, livePollsPublishResults: a poll opens, a late fan fetches it, votes, and both see the tally", + "role": "open", + "mode": "live" + }, + "livePollsPublishResults": { + "file": "test/integration/live/use-cases/live-polls.test.ts", + "test": "use-cases/live-polls livePollsPublishPoll, livePollsSubscribeToPolls, livePollsFetchOpenPoll, livePollsPublishVote, livePollsSubscribeToResults, livePollsPublishResults: a poll opens, a late fan fetches it, votes, and both see the tally", + "role": "open", + "mode": "live" + }, + "livePollsPublishVote": { + "file": "test/integration/live/use-cases/live-polls.test.ts", + "test": "use-cases/live-polls livePollsPublishPoll, livePollsSubscribeToPolls, livePollsFetchOpenPoll, livePollsPublishVote, livePollsSubscribeToResults, livePollsPublishResults: a poll opens, a late fan fetches it, votes, and both see the tally", + "role": "open", + "mode": "live" + }, + "livePollsSubscribeToPolls": { + "file": "test/integration/live/use-cases/live-polls.test.ts", + "test": "use-cases/live-polls livePollsPublishPoll, livePollsSubscribeToPolls, livePollsFetchOpenPoll, livePollsPublishVote, livePollsSubscribeToResults, livePollsPublishResults: a poll opens, a late fan fetches it, votes, and both see the tally", + "role": "open", + "mode": "live" + }, + "livePollsSubscribeToResults": { + "file": "test/integration/live/use-cases/live-polls.test.ts", + "test": "use-cases/live-polls livePollsPublishPoll, livePollsSubscribeToPolls, livePollsFetchOpenPoll, livePollsPublishVote, livePollsSubscribeToResults, livePollsPublishResults: a poll opens, a late fan fetches it, votes, and both see the tally", + "role": "open", + "mode": "live" + }, + "loggingBasicUsage": { + "file": "test/integration/live/getting-started/logging.test.ts", + "test": "getting-started/logging loggingBasicUsage: a client built with Debug level logging works and its logger output is captured", + "mode": "live" + }, + "loggingCustomLoggerExample": { + "file": "test/integration/live/getting-started/logging.test.ts", + "test": "getting-started/logging loggingCustomLoggerExample: a custom logger receives config, error, failed request and deprecation entries", + "mode": "live" + }, + "loggingFileLogger": { + "file": "test/integration/live/getting-started/logging-file-logger.test.ts", + "test": "getting-started/logging-file-logger loggingFileLogger: a file logger writes the SDK log lines to the log file", + "mode": "live" + }, + "loggingFileLoggerImports": { + "file": "test/integration/live/getting-started/logging-file-logger.test.ts", + "mode": "compile-only" + }, + "loggingImportLogger": { + "file": "test/integration/live/getting-started/logging.test.ts", + "mode": "compile-only" + }, + "loggingImportPubNub": { + "file": "test/integration/live/getting-started/logging.test.ts", + "mode": "compile-only" + }, + "loggingMultipleLoggers": { + "file": "test/integration/live/getting-started/logging.test.ts", + "test": "getting-started/logging loggingMultipleLoggers: the SDK calls every logger in the loggers array", + "mode": "live" + }, + "matchStatsFetchCurrentValues": { + "file": "test/integration/live/use-cases/match-stats.test.ts", + "test": "use-cases/match-stats matchStatsPublishOneStat, matchStatsSubscribeToAllStats, matchStatsFetchCurrentValues: a published stat reaches the wildcard subscription and a late reader gets the current value of every stat", + "role": "open", + "mode": "live" + }, + "matchStatsPublishOneStat": { + "file": "test/integration/live/use-cases/match-stats.test.ts", + "test": "use-cases/match-stats matchStatsPublishOneStat, matchStatsSubscribeToAllStats, matchStatsFetchCurrentValues: a published stat reaches the wildcard subscription and a late reader gets the current value of every stat", + "role": "open", + "mode": "live" + }, + "matchStatsSubscribeToAllStats": { + "file": "test/integration/live/use-cases/match-stats.test.ts", + "test": "use-cases/match-stats matchStatsPublishOneStat, matchStatsSubscribeToAllStats, matchStatsFetchCurrentValues: a published stat reaches the wildcard subscription and a late reader gets the current value of every stat", + "role": "open", + "mode": "live" + }, + "messageCountBasicUsage": { + "file": "test/integration/live/api-reference/message-persistence.test.ts", + "test": "api-reference/message-persistence messageCountBasicUsage: messageCounts with one timetoken counts the messages per channel since that time", + "role": "open", + "mode": "live" + }, + "messageCountTimetokensForChannels": { + "file": "test/integration/live/api-reference/message-persistence.test.ts", + "test": "api-reference/message-persistence messageCountTimetokensForChannels: messageCounts with one timetoken per channel gives the per-channel counts", + "role": "open", + "mode": "live" + }, + "migrationV11LegacySubscriptionManager": { + "file": "test/integration/live/migration-guides/v11.test.ts", + "test": "migration-guides/v11 migrationV11LegacySubscriptionManager: the legacy manager fires PNConnectedCategory again on a second subscribe and never PNSubscriptionChangedCategory", + "mode": "live" + }, + "migrationV11RestoreFalse": { + "file": "test/integration/live/migration-guides/v11.test.ts", + "test": "migration-guides/v11 migrationV11RestoreFalse: a client built with restore false is valid and works", + "mode": "live" + }, + "migrationV8AfterImport": { + "file": "test/integration/live/migration-guides/v8.test.ts", + "mode": "compile-only" + }, + "migrationV8AfterParams": { + "file": "test/integration/live/migration-guides/v8.test.ts", + "test": "migration-guides/v8 migrationV8AfterImport, migrationV8AfterParams: the typed parameters constant of the v8 guide compiles against the SDK types", + "mode": "live" + }, + "observeDataSyncEntitiesByPattern": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync events and subscriptions observeDataSyncEntitiesByPattern: a wildcard subscription on the admin projection channel receives the event for a newly created dotted id", + "role": "datasync", + "mode": "live" + }, + "parseTokenBasicUsage": { + "file": "test/integration/live/api-reference/access-manager.test.ts", + "test": "api-reference/access-manager parseTokenBasicUsage: parseToken decodes a token locally, and the fixture token shows the right owner and permissions", + "role": "pam", + "mode": "live" + }, + "parseTokenWithCategories": { + "file": "test/integration/live/api-reference/access-manager.test.ts", + "test": "api-reference/access-manager parseTokenWithCategories: parseToken reads the category permissions of a category token", + "role": "pam", + "mode": "live" + }, + "patchChannelBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync channels patchChannelBasicUsage: updateChannel patches one field and the other payload fields survive", + "role": "datasync", + "mode": "live" + }, + "patchEntityBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync entities (class product) patchEntityBasicUsage: updateEntity patches the price and the stock stays the same", + "role": "datasync", + "mode": "live" + }, + "patchMembershipBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync memberships patchMembershipBasicUsage: updateMembership patches one payload field", + "role": "datasync", + "mode": "live" + }, + "patchRelationshipBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync relationships (class ProductOwner) patchRelationshipBasicUsage: updateRelationship patches the tier of the link", + "role": "datasync", + "mode": "live" + }, + "patchUserBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync users patchUserBasicUsage: updateUser applies a patch to the user name and status", + "role": "datasync", + "mode": "live" + }, + "publishBasicUsage": { + "file": "test/integration/live/api-reference/publish.test.ts", + "test": "api-reference/publish publishBasicUsage: publish delivers a message with meta and a custom message type to a subscriber and stores it", + "role": "open", + "mode": "live" + }, + "publishFalsyMessage": { + "file": "test/integration/live/api-reference/publish.test.ts", + "test": "api-reference/publish publishFalsyMessage: a wrapped value 0 is delivered as a message", + "role": "open", + "mode": "live" + }, + "publishFileMessageBasicUsage": { + "file": "test/integration/live/api-reference/file-sharing.test.ts", + "test": "api-reference/file-sharing publishFileMessageBasicUsage: publishFile announces the uploaded file and the event is matched by timetoken", + "role": "open", + "mode": "live" + }, + "publishHandleErrors": { + "file": "test/integration/live/api-reference/publish.test.ts", + "test": "api-reference/publish publishHandleErrors: the same error handling code logs the right line for success, access denied (403), validation and network failure", + "role": "pam", + "mode": "live" + }, + "publishMinimal": { + "file": "test/integration/live/api-reference/publish.test.ts", + "test": "api-reference/publish publishMinimal: the minimal publish returns a string timetoken and delivers the message", + "role": "open", + "mode": "live" + }, + "publishStoreThePublishedMessagefor10Hours": { + "file": "test/integration/live/api-reference/publish.test.ts", + "test": "api-reference/publish publishStoreThePublishedMessagefor10Hours: publish with a ttl still delivers and stores the message", + "role": "open", + "mode": "live" + }, + "quickstartNodeListeners": { + "file": "test/integration/live/getting-started/quickstart-node.test.ts", + "test": "getting-started/quickstart-node gettingStartedInitPubnub, quickstartNodeListeners, gettingStartedCreateSubscription, quickstartNodePublish: the Node quickstart script connects, publishes and receives, and prints the lines the page promises", + "mode": "live" + }, + "quickstartNodePublish": { + "file": "test/integration/live/getting-started/quickstart-node.test.ts", + "test": "getting-started/quickstart-node gettingStartedInitPubnub, quickstartNodeListeners, gettingStartedCreateSubscription, quickstartNodePublish: the Node quickstart script connects, publishes and receives, and prints the lines the page promises", + "mode": "live" + }, + "rateLimitingPickShardForFan": { + "file": "test/integration/live/use-cases/live-event-rate-limiting.test.ts", + "test": "use-cases/live-event-rate-limiting rateLimitingReadShardOccupancy, rateLimitingPickShardForFan, rateLimitingThrottleFanPublishes: occupancy per shard is read, a fan is placed in a shard with room and its publishes are throttled on the client", + "role": "open", + "mode": "live" + }, + "rateLimitingReadShardOccupancy": { + "file": "test/integration/live/use-cases/live-event-rate-limiting.test.ts", + "test": "use-cases/live-event-rate-limiting rateLimitingReadShardOccupancy, rateLimitingPickShardForFan, rateLimitingThrottleFanPublishes: occupancy per shard is read, a fan is placed in a shard with room and its publishes are throttled on the client", + "role": "open", + "mode": "live" + }, + "rateLimitingThrottleFanPublishes": { + "file": "test/integration/live/use-cases/live-event-rate-limiting.test.ts", + "test": "use-cases/live-event-rate-limiting rateLimitingReadShardOccupancy, rateLimitingPickShardForFan, rateLimitingThrottleFanPublishes: occupancy per shard is read, a fan is placed in a shard with room and its publishes are throttled on the client", + "role": "open", + "mode": "live" + }, + "reEngagementCheckOneFan": { + "file": "test/integration/live/use-cases/fan-re-engagement.test.ts", + "test": "use-cases/fan-re-engagement reEngagementCountFansWatching, reEngagementPublishMomentAlert, reEngagementCheckOneFan, reEngagementWatchFansLeave, reEngagementFanViewer: the service counts fans, alerts only a fan who left, and a viewer stays reachable after it stops watching", + "role": "open", + "mode": "live" + }, + "reEngagementCountFansWatching": { + "file": "test/integration/live/use-cases/fan-re-engagement.test.ts", + "test": "use-cases/fan-re-engagement reEngagementCountFansWatching, reEngagementPublishMomentAlert, reEngagementCheckOneFan, reEngagementWatchFansLeave, reEngagementFanViewer: the service counts fans, alerts only a fan who left, and a viewer stays reachable after it stops watching", + "role": "open", + "mode": "live" + }, + "reEngagementFanViewer": { + "file": "test/integration/live/use-cases/fan-re-engagement.test.ts", + "test": "use-cases/fan-re-engagement reEngagementCountFansWatching, reEngagementPublishMomentAlert, reEngagementCheckOneFan, reEngagementWatchFansLeave, reEngagementFanViewer: the service counts fans, alerts only a fan who left, and a viewer stays reachable after it stops watching", + "role": "open", + "mode": "live" + }, + "reEngagementPublishMomentAlert": { + "file": "test/integration/live/use-cases/fan-re-engagement.test.ts", + "test": "use-cases/fan-re-engagement reEngagementCountFansWatching, reEngagementPublishMomentAlert, reEngagementCheckOneFan, reEngagementWatchFansLeave, reEngagementFanViewer: the service counts fans, alerts only a fan who left, and a viewer stays reachable after it stops watching", + "role": "open", + "mode": "live" + }, + "reEngagementWatchFansLeave": { + "file": "test/integration/live/use-cases/fan-re-engagement.test.ts", + "test": "use-cases/fan-re-engagement reEngagementCountFansWatching, reEngagementPublishMomentAlert, reEngagementCheckOneFan, reEngagementWatchFansLeave, reEngagementFanViewer: the service counts fans, alerts only a fan who left, and a viewer stays reachable after it stops watching", + "role": "open", + "mode": "live" + }, + "realTimeAdsPublishReaction": { + "file": "test/integration/live/use-cases/real-time-ads.test.ts", + "test": "use-cases/real-time-ads realTimeAdsPublishReaction, realTimeAdsReceiveAdDecision, realTimeAdsReceiveReactionUpgrade: a fan publishes a reaction, then shows an ad, clears the slot and renders an upgraded reaction on decisions", + "role": "open", + "mode": "live" + }, + "realTimeAdsReceiveAdDecision": { + "file": "test/integration/live/use-cases/real-time-ads.test.ts", + "test": "use-cases/real-time-ads realTimeAdsPublishReaction, realTimeAdsReceiveAdDecision, realTimeAdsReceiveReactionUpgrade: a fan publishes a reaction, then shows an ad, clears the slot and renders an upgraded reaction on decisions", + "role": "open", + "mode": "live" + }, + "realTimeAdsReceiveReactionUpgrade": { + "file": "test/integration/live/use-cases/real-time-ads.test.ts", + "test": "use-cases/real-time-ads realTimeAdsPublishReaction, realTimeAdsReceiveAdDecision, realTimeAdsReceiveReactionUpgrade: a fan publishes a reaction, then shows an ad, clears the slot and renders an upgraded reaction on decisions", + "role": "open", + "mode": "live" + }, + "reconnectBasicUsage": { + "file": "test/integration/live/api-reference/misc.test.ts", + "test": "api-reference/misc reconnectBasicUsage: reconnect() restores delivery after a disconnect", + "role": "open", + "mode": "live" + }, + "removeAllMobilePushNotificationsBasicUsage": { + "file": "test/integration/live/api-reference/mobile-push.test.ts", + "test": "api-reference/mobile-push removeAllMobilePushNotificationsBasicUsage: deleteDevice removes every registration of the device and the list is empty afterwards", + "role": "open", + "mode": "live" + }, + "removeChannelBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync removal removeChannelBasicUsage: removeChannel deletes the DataSync channel and a read afterwards answers 404", + "role": "datasync", + "mode": "live" + }, + "removeChannelMembersBasicUsage": { + "file": "test/integration/live/api-reference/objects.test.ts", + "test": "api-reference/objects removeChannelMembersBasicUsage: removeChannelMembers removes members by id and leaves the others", + "role": "open", + "mode": "live" + }, + "removeChannelMetadataBasicUsage": { + "file": "test/integration/live/api-reference/objects.test.ts", + "test": "api-reference/objects removeChannelMetadataBasicUsage: removeChannelMetadata deletes the channel and reads fail afterwards", + "role": "open", + "mode": "live" + }, + "removeChannelsFromGroupBasicUsage": { + "file": "test/integration/live/api-reference/channel-groups.test.ts", + "test": "api-reference/channel-groups removeChannelsFromGroupBasicUsage: removeChannels removes one channel from a group and leaves only the other one", + "role": "open", + "mode": "live" + }, + "removeDeviceFromChannelBasicUsage": { + "file": "test/integration/live/api-reference/mobile-push.test.ts", + "test": "api-reference/mobile-push removeDeviceFromChannelBasicUsage: removeChannels unregisters the device and both gateways list nothing afterwards", + "role": "open", + "mode": "live" + }, + "removeEntityBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync removal removeEntityBasicUsage: removeEntity deletes the product and a read afterwards answers 404", + "role": "datasync", + "mode": "live" + }, + "removeMembershipBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync removal removeMembershipBasicUsage: removeMembership deletes the membership and a read afterwards answers 404", + "role": "datasync", + "mode": "live" + }, + "removeMembershipsBasicUsage": { + "file": "test/integration/live/api-reference/objects.test.ts", + "test": "api-reference/objects removeMembershipsBasicUsage: removeMemberships removes the named memberships and keeps the others", + "role": "open", + "mode": "live" + }, + "removeMessageActionBasicUsage": { + "file": "test/integration/live/api-reference/message-actions.test.ts", + "test": "api-reference/message-actions removeMessageActionBasicUsage: removeMessageAction emits a removed event and the action disappears from the stored list", + "role": "open", + "mode": "live" + }, + "removeRelationshipBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync removal removeRelationshipBasicUsage: removeRelationship deletes the link and a read afterwards answers 404", + "role": "datasync", + "mode": "live" + }, + "removeUUIDMetadataBasicUsage": { + "file": "test/integration/live/api-reference/objects.test.ts", + "test": "api-reference/objects removeUUIDMetadataBasicUsage: removeUUIDMetadata deletes the users and reads fail with not found afterwards", + "role": "open", + "mode": "live" + }, + "removeUserBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync removal removeUserBasicUsage: removeUser deletes Alice and a read afterwards answers 404", + "role": "datasync", + "mode": "live" + }, + "retryPolicyExponential": { + "file": "test/integration/endpoints/objects/channel.test.ts", + "test": "objects channel retry policy should retry on 503 error with exponential retry policy until maximum retry count is reached", + "mode": "mocked" + }, + "retryPolicyLinear": { + "file": "test/integration/endpoints/objects/channel.test.ts", + "test": "objects channel retry policy should not retry on 404 Not Found error with linear retry policy", + "mode": "mocked" + }, + "retryPolicyNone": { + "file": "test/integration/endpoints/time.test.ts", + "test": "time endpoints reports the HTTP status code of a failed request without retrying it when retries are disabled", + "mode": "mocked" + }, + "revokeTokenBasicUsage": { + "file": "test/integration/live/api-reference/access-manager.test.ts", + "test": "api-reference/access-manager revokeTokenBasicUsage: revokeToken makes a working token stop working, for the client that was allowed before and for fresh clients with the same token", + "role": "pam", + "mode": "live" + }, + "scoreAlertsBuildNotification": { + "file": "test/integration/live/use-cases/score-alerts.test.ts", + "test": "use-cases/score-alerts scoreAlertsEnvironmentConstant, scoreAlertsRegisterDeviceFCM, scoreAlertsRegisterDeviceAPNs, scoreAlertsBuildNotification, scoreAlertsPublishNotification, scoreAlertsListDeviceRegistrationsFCM, scoreAlertsListDeviceRegistrationsAPNs, scoreAlertsRemoveDeviceRegistrationFCM, scoreAlertsRemoveDeviceRegistrationAPNs: a device registers, an alert is built and published, the registration is listed and removed", + "role": "open", + "mode": "live" + }, + "scoreAlertsEnvironmentConstant": { + "file": "test/integration/live/use-cases/score-alerts.test.ts", + "test": "use-cases/score-alerts scoreAlertsEnvironmentConstant, scoreAlertsRegisterDeviceFCM, scoreAlertsRegisterDeviceAPNs, scoreAlertsBuildNotification, scoreAlertsPublishNotification, scoreAlertsListDeviceRegistrationsFCM, scoreAlertsListDeviceRegistrationsAPNs, scoreAlertsRemoveDeviceRegistrationFCM, scoreAlertsRemoveDeviceRegistrationAPNs: a device registers, an alert is built and published, the registration is listed and removed", + "role": "open", + "mode": "live" + }, + "scoreAlertsListDeviceRegistrationsAPNs": { + "file": "test/integration/live/use-cases/score-alerts.test.ts", + "test": "use-cases/score-alerts scoreAlertsEnvironmentConstant, scoreAlertsRegisterDeviceFCM, scoreAlertsRegisterDeviceAPNs, scoreAlertsBuildNotification, scoreAlertsPublishNotification, scoreAlertsListDeviceRegistrationsFCM, scoreAlertsListDeviceRegistrationsAPNs, scoreAlertsRemoveDeviceRegistrationFCM, scoreAlertsRemoveDeviceRegistrationAPNs: a device registers, an alert is built and published, the registration is listed and removed", + "role": "open", + "mode": "live" + }, + "scoreAlertsListDeviceRegistrationsFCM": { + "file": "test/integration/live/use-cases/score-alerts.test.ts", + "test": "use-cases/score-alerts scoreAlertsEnvironmentConstant, scoreAlertsRegisterDeviceFCM, scoreAlertsRegisterDeviceAPNs, scoreAlertsBuildNotification, scoreAlertsPublishNotification, scoreAlertsListDeviceRegistrationsFCM, scoreAlertsListDeviceRegistrationsAPNs, scoreAlertsRemoveDeviceRegistrationFCM, scoreAlertsRemoveDeviceRegistrationAPNs: a device registers, an alert is built and published, the registration is listed and removed", + "role": "open", + "mode": "live" + }, + "scoreAlertsPublishNotification": { + "file": "test/integration/live/use-cases/score-alerts.test.ts", + "test": "use-cases/score-alerts scoreAlertsEnvironmentConstant, scoreAlertsRegisterDeviceFCM, scoreAlertsRegisterDeviceAPNs, scoreAlertsBuildNotification, scoreAlertsPublishNotification, scoreAlertsListDeviceRegistrationsFCM, scoreAlertsListDeviceRegistrationsAPNs, scoreAlertsRemoveDeviceRegistrationFCM, scoreAlertsRemoveDeviceRegistrationAPNs: a device registers, an alert is built and published, the registration is listed and removed", + "role": "open", + "mode": "live" + }, + "scoreAlertsRegisterDeviceAPNs": { + "file": "test/integration/live/use-cases/score-alerts.test.ts", + "test": "use-cases/score-alerts scoreAlertsEnvironmentConstant, scoreAlertsRegisterDeviceFCM, scoreAlertsRegisterDeviceAPNs, scoreAlertsBuildNotification, scoreAlertsPublishNotification, scoreAlertsListDeviceRegistrationsFCM, scoreAlertsListDeviceRegistrationsAPNs, scoreAlertsRemoveDeviceRegistrationFCM, scoreAlertsRemoveDeviceRegistrationAPNs: a device registers, an alert is built and published, the registration is listed and removed", + "role": "open", + "mode": "live" + }, + "scoreAlertsRegisterDeviceFCM": { + "file": "test/integration/live/use-cases/score-alerts.test.ts", + "test": "use-cases/score-alerts scoreAlertsEnvironmentConstant, scoreAlertsRegisterDeviceFCM, scoreAlertsRegisterDeviceAPNs, scoreAlertsBuildNotification, scoreAlertsPublishNotification, scoreAlertsListDeviceRegistrationsFCM, scoreAlertsListDeviceRegistrationsAPNs, scoreAlertsRemoveDeviceRegistrationFCM, scoreAlertsRemoveDeviceRegistrationAPNs: a device registers, an alert is built and published, the registration is listed and removed", + "role": "open", + "mode": "live" + }, + "scoreAlertsRemoveDeviceRegistrationAPNs": { + "file": "test/integration/live/use-cases/score-alerts.test.ts", + "test": "use-cases/score-alerts scoreAlertsEnvironmentConstant, scoreAlertsRegisterDeviceFCM, scoreAlertsRegisterDeviceAPNs, scoreAlertsBuildNotification, scoreAlertsPublishNotification, scoreAlertsListDeviceRegistrationsFCM, scoreAlertsListDeviceRegistrationsAPNs, scoreAlertsRemoveDeviceRegistrationFCM, scoreAlertsRemoveDeviceRegistrationAPNs: a device registers, an alert is built and published, the registration is listed and removed", + "role": "open", + "mode": "live" + }, + "scoreAlertsRemoveDeviceRegistrationFCM": { + "file": "test/integration/live/use-cases/score-alerts.test.ts", + "test": "use-cases/score-alerts scoreAlertsEnvironmentConstant, scoreAlertsRegisterDeviceFCM, scoreAlertsRegisterDeviceAPNs, scoreAlertsBuildNotification, scoreAlertsPublishNotification, scoreAlertsListDeviceRegistrationsFCM, scoreAlertsListDeviceRegistrationsAPNs, scoreAlertsRemoveDeviceRegistrationFCM, scoreAlertsRemoveDeviceRegistrationAPNs: a device registers, an alert is built and published, the registration is listed and removed", + "role": "open", + "mode": "live" + }, + "sendFileBasicUsage": { + "file": "test/integration/live/api-reference/file-sharing.test.ts", + "test": "api-reference/file-sharing sendFileBasicUsage: sendFile uploads the picture and a second client receives the file event", + "role": "open", + "mode": "live" + }, + "sendFileCustomCipherKey": { + "file": "test/integration/live/api-reference/file-sharing-legacy.test.ts", + "test": "api-reference/file-sharing-legacy sendFileCustomCipherKey: sendFile with a custom cipher key uploads the picture from a working directory", + "role": "open", + "mode": "live" + }, + "setAuthKeyBasicUsage": { + "file": "test/integration/live/api-reference/configuration.test.ts", + "test": "api-reference/configuration setAuthKeyBasicUsage: setAuthKey stores the auth key on the client", + "role": "open", + "mode": "live" + }, + "setChannelMembersBasicUsage": { + "file": "test/integration/live/api-reference/objects.test.ts", + "test": "api-reference/objects setChannelMembersBasicUsage: setChannelMembers adds a member with a custom role and another client reads it back", + "role": "open", + "mode": "live" + }, + "setChannelMetadataBasicUsage": { + "file": "test/integration/live/api-reference/objects.test.ts", + "test": "api-reference/objects setChannelMetadataBasicUsage: setChannelMetadata stores the channel, an observer gets the set event and the hidden custom data was stored", + "role": "open", + "mode": "live" + }, + "setFilterExpressionBasicUsage": { + "file": "test/integration/live/api-reference/configuration.test.ts", + "test": "api-reference/configuration setFilterExpressionBasicUsage: setFilterExpression stores the filter expression on the client", + "role": "open", + "mode": "live" + }, + "setMembershipBasicUsage": { + "file": "test/integration/live/api-reference/objects.test.ts", + "test": "api-reference/objects setMembershipBasicUsage: setMemberships stores a membership, and a second read with include flags shows the type and custom data", + "role": "open", + "mode": "live" + }, + "setProxyBasicUsage": { + "file": "test/integration/live/api-reference/misc.test.ts", + "test": "api-reference/misc setProxyBasicUsage: setProxy routes the client traffic through the proxy, seen as a tunnel request", + "role": "open", + "mode": "live" + }, + "setStateBasicUsage": { + "file": "test/integration/live/api-reference/presence.test.ts", + "test": "api-reference/presence setStateBasicUsage: setState stores state that another client can read back", + "role": "open", + "mode": "live" + }, + "setTokenBasicUsage": { + "file": "test/integration/live/api-reference/access-manager.test.ts", + "test": "api-reference/access-manager setTokenBasicUsage: setToken lets Alice publish to her room but not to another, while Bob stays read only", + "role": "pam", + "mode": "live" + }, + "setUUIDMetadataBasicUsage": { + "file": "test/integration/live/api-reference/objects.test.ts", + "test": "api-reference/objects setUUIDMetadataBasicUsage: setUUIDMetadata stores the user and another client reads it back", + "role": "open", + "mode": "live" + }, + "setUpTypeScriptImport": { + "file": "test/integration/live/getting-started/set-up-your-environment.test.ts", + "mode": "compile-only" + }, + "setUpTypeScriptInit": { + "file": "test/integration/live/getting-started/set-up-your-environment.test.ts", + "test": "getting-started/set-up-your-environment setUpTypeScriptImport, setUpTypeScriptInit: the TypeScript init sample builds a working client and time() succeeds", + "mode": "live" + }, + "setUserIdBasicUsage": { + "file": "test/integration/live/api-reference/configuration.test.ts", + "test": "api-reference/configuration setUserIdBasicUsage: setUserId renames the user and the client keeps working", + "role": "open", + "mode": "live" + }, + "signalBasicUsage": { + "file": "test/integration/live/api-reference/publish.test.ts", + "test": "api-reference/publish signalBasicUsage: signal reaches a subscriber as a signal event with the custom message type, not as a message", + "role": "open", + "mode": "live" + }, + "simpleNotificationPayloadFCMandAPNS": { + "file": "test/integration/live/api-reference/mobile-push.test.ts", + "test": "api-reference/mobile-push simpleNotificationPayloadFCMandAPNS: the payload builder prints a payload with the FCM and APNs sections the sample describes", + "mode": "live" + }, + "simpleNotificationPayloadFCMandAPNSH/2": { + "file": "test/integration/live/api-reference/mobile-push.test.ts", + "test": "api-reference/mobile-push simpleNotificationPayloadFCMandAPNSH/2: the payload builder produces the APNs HTTP/2 section as documented", + "mode": "live" + }, + "simpleNotificationPayloadFCMandAPNSH/2CustomConfiguration": { + "file": "test/integration/live/api-reference/mobile-push.test.ts", + "test": "api-reference/mobile-push simpleNotificationPayloadFCMandAPNSH/2CustomConfiguration: collapseId and expirationDate reach the payload in the wire format", + "mode": "live" + }, + "statusEventsMigrationAfter": { + "file": "test/integration/live/getting-started/status-events.test.ts", + "test": "getting-started/status-events statusEventsMigrationAfter: the new subscribe loop reports PNConnectedCategory once and PNSubscriptionChangedCategory for later changes.", + "role": "open", + "mode": "live" + }, + "statusEventsMigrationBefore": { + "file": "test/integration/live/getting-started/status-events.test.ts", + "test": "getting-started/status-events statusEventsMigrationBefore: the legacy subscription manager reports PNConnectedCategory on every change of the channel mix.", + "role": "open", + "mode": "live" + }, + "subscribeToDataSyncEntity": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync events and subscriptions subscribeToDataSyncEntity: subscribing to one entity delivers the new stock in the event payload", + "role": "datasync", + "mode": "live" + }, + "subscriptionSetFrom2IndividualSubscriptions": { + "file": "test/integration/live/api-reference/subscribe.test.ts", + "test": "api-reference/subscribe subscriptionSetFrom2IndividualSubscriptions: addSubscription and removeSubscription change the set before subscribing", + "role": "open", + "mode": "live" + }, + "subscriptionSetSubscribeBasicUsage": { + "file": "test/integration/live/api-reference/subscribe.test.ts", + "test": "api-reference/subscribe subscriptionSetSubscribeBasicUsage: subscribing a set connects both channels and delivers a message from a friend", + "role": "open", + "mode": "live" + }, + "unsubscribeAllBasicUsage": { + "file": "test/integration/live/api-reference/subscribe.test.ts", + "test": "api-reference/subscribe unsubscribeAllBasicUsage: unsubscribeAll clears channels and groups", + "role": "open", + "mode": "live" + }, + "unsubscribeBasicUsage": { + "file": "test/integration/live/api-reference/subscribe.test.ts", + "test": "api-reference/subscribe unsubscribeBasicUsage: unsubscribing a subscription and a set leaves nothing subscribed", + "role": "open", + "mode": "live" + }, + "updateChannelBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync channels updateChannelBasicUsage: setChannel replaces the channel payload", + "role": "datasync", + "mode": "live" + }, + "updateEntityBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync entities (class product) updateEntityBasicUsage: setEntity replaces the product and sets the price and stock later tests rely on", + "role": "datasync", + "mode": "live" + }, + "updateEntityCombinedPatchExample": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync entities (class product) updateEntityCombinedPatchExample: one patch call with all six JSON Patch operations applies, and the eTag it was guarded by is stale afterwards", + "role": "datasync", + "mode": "live" + }, + "updateMembershipBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync memberships updateMembershipBasicUsage: setMembership replaces the membership payload", + "role": "datasync", + "mode": "live" + }, + "updateRelationshipBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync relationships (class ProductOwner) updateRelationshipBasicUsage: setRelationship replaces the link and adds a tier to its payload", + "role": "datasync", + "mode": "live" + }, + "updateUserBasicUsage": { + "file": "test/integration/live/api-reference/datasync.test.ts", + "test": "api-reference/datasync users updateUserBasicUsage: setUser with ifMatchesEtag replaces Alice, and the eTag it was guarded by is stale afterwards", + "role": "datasync", + "mode": "live" + }, + "whereNowBasicUsage": { + "file": "test/integration/live/api-reference/presence.test.ts", + "test": "api-reference/presence whereNowBasicUsage: whereNow lists the channels the occupant is in", + "role": "open", + "mode": "live" + } + }, + "files": { + "test/integration/live/getting-started/quickstart-node.mjs": { + "mode": "live", + "test": "getting-started/quickstart-node quickstartNodeRun: the complete quickstart file runs in a Node process, prints Connected and Published, receives its own message, and receives a message from another client", + "testFile": "test/integration/live/getting-started/quickstart-node.test.ts" + } + } +} diff --git a/test/integration/endpoints/data_sync/events.channel.test.ts b/test/integration/live/endpoints/data_sync/events.channel.test.ts similarity index 81% rename from test/integration/endpoints/data_sync/events.channel.test.ts rename to test/integration/live/endpoints/data_sync/events.channel.test.ts index fac7c4d9b..79cea0787 100644 --- a/test/integration/endpoints/data_sync/events.channel.test.ts +++ b/test/integration/live/endpoints/data_sync/events.channel.test.ts @@ -6,9 +6,7 @@ * */ -import nock from 'nock'; - -import type * as Subscription from '../../../../src/core/types/api/subscription'; +import type PubNub from 'pubnub'; import { CLASS_VERSION, @@ -19,28 +17,17 @@ import { channelPayload, freshId, freshPubNub, -} from './helpers'; +} from '../../support/datasync-live-helpers'; -describe('DataSync events — Channel', function () { +describe('endpoints/data_sync/events.channel: DataSync events — Channel', function () { this.timeout(60000); - // Earlier nocked Data Sync files call `disableNetConnect()` and never re-enable it. This suite - // talks to the live service, so lift that block before any REST / subscribe call. - before(() => { - nock.cleanAll(); - nock.enableNetConnect(); - }); - let pubnub: ReturnType; beforeEach(() => { pubnub = freshPubNub(); }); - afterEach(() => { - pubnub.destroy(); - }); - const createChannel = (id: string, payload = channelPayload()) => pubnub.dataSync .createChannel({ id, data: { classVersion: CLASS_VERSION, status: 'active', payload } }) @@ -64,7 +51,7 @@ describe('DataSync events — Channel', function () { className: 'Channel', classLevel: 'Global', }); - assertEventEntityData(event.message.data as Subscription.DataSyncEntityData, { + assertEventEntityData(event.message.data as PubNub.DataSyncEntityData, { id, status: 'active', payload: { name: 'engineering', kind: 'public' }, @@ -101,7 +88,7 @@ describe('DataSync events — Channel', function () { className: 'Channel', classLevel: 'Global', }); - assertEventEntityData(event.message.data as Subscription.DataSyncEntityData, { + assertEventEntityData(event.message.data as PubNub.DataSyncEntityData, { id, status: 'updated', payload: { memberCount: 5 }, @@ -130,7 +117,7 @@ describe('DataSync events — Channel', function () { className: 'Channel', classLevel: 'Global', }); - assertEventEntityData(event.message.data as Subscription.DataSyncEntityData, { + assertEventEntityData(event.message.data as PubNub.DataSyncEntityData, { id, payload: { memberCount: 9 }, }); @@ -156,6 +143,6 @@ describe('DataSync events — Channel', function () { className: 'Channel', classLevel: 'Global', }); - assertEventDeleteData(event.message.data as Subscription.DataSyncDeleteData, id); + assertEventDeleteData(event.message.data as PubNub.DataSyncDeleteData, id); }); }); diff --git a/test/integration/endpoints/data_sync/events.entity.test.ts b/test/integration/live/endpoints/data_sync/events.entity.test.ts similarity index 82% rename from test/integration/endpoints/data_sync/events.entity.test.ts rename to test/integration/live/endpoints/data_sync/events.entity.test.ts index 8a17ef957..c778a0375 100644 --- a/test/integration/endpoints/data_sync/events.entity.test.ts +++ b/test/integration/live/endpoints/data_sync/events.entity.test.ts @@ -7,9 +7,7 @@ * */ -import nock from 'nock'; - -import type * as Subscription from '../../../../src/core/types/api/subscription'; +import type PubNub from 'pubnub'; import { CLASS_VERSION, @@ -21,28 +19,17 @@ import { customerPayload, freshId, freshPubNub, -} from './helpers'; +} from '../../support/datasync-live-helpers'; -describe('DataSync events — Entity Customer', function () { +describe('endpoints/data_sync/events.entity: DataSync events — Entity Customer', function () { this.timeout(60000); - // Earlier nocked Data Sync files call `disableNetConnect()` and never re-enable it. This suite - // talks to the live service, so lift that block before any REST / subscribe call. - before(() => { - nock.cleanAll(); - nock.enableNetConnect(); - }); - let pubnub: ReturnType; beforeEach(() => { pubnub = freshPubNub(); }); - afterEach(() => { - pubnub.destroy(); - }); - const createCustomer = (id: string) => pubnub.dataSync .createEntity({ @@ -74,7 +61,7 @@ describe('DataSync events — Entity Customer', function () { className: ENTITY_CLASS_CUSTOMER, classLevel: 'SubKey', }); - assertEventEntityData(event.message.data as Subscription.DataSyncEntityData, { + assertEventEntityData(event.message.data as PubNub.DataSyncEntityData, { id, status: 'active', payload: { customerId: id, firstName: 'Alice', creditScore: 720, city: 'Pune' }, @@ -111,7 +98,7 @@ describe('DataSync events — Entity Customer', function () { className: ENTITY_CLASS_CUSTOMER, classLevel: 'SubKey', }); - assertEventEntityData(event.message.data as Subscription.DataSyncEntityData, { + assertEventEntityData(event.message.data as PubNub.DataSyncEntityData, { id, status: 'updated', payload: { creditScore: 780 }, @@ -138,7 +125,7 @@ describe('DataSync events — Entity Customer', function () { className: ENTITY_CLASS_CUSTOMER, classLevel: 'SubKey', }); - assertEventEntityData(event.message.data as Subscription.DataSyncEntityData, { + assertEventEntityData(event.message.data as PubNub.DataSyncEntityData, { id, payload: { creditScore: 810 }, }); @@ -164,6 +151,6 @@ describe('DataSync events — Entity Customer', function () { className: ENTITY_CLASS_CUSTOMER, classLevel: 'SubKey', }); - assertEventDeleteData(event.message.data as Subscription.DataSyncDeleteData, id); + assertEventDeleteData(event.message.data as PubNub.DataSyncDeleteData, id); }); }); diff --git a/test/integration/endpoints/data_sync/events.membership.test.ts b/test/integration/live/endpoints/data_sync/events.membership.test.ts similarity index 86% rename from test/integration/endpoints/data_sync/events.membership.test.ts rename to test/integration/live/endpoints/data_sync/events.membership.test.ts index dc402aa16..28022ce6a 100644 --- a/test/integration/endpoints/data_sync/events.membership.test.ts +++ b/test/integration/live/endpoints/data_sync/events.membership.test.ts @@ -8,10 +8,9 @@ * `userId`, not the relationship's `entityAId` / `entityBId`. */ -import assert from 'assert'; -import nock from 'nock'; +import assert from 'node:assert'; -import type * as Subscription from '../../../../src/core/types/api/subscription'; +import type PubNub from 'pubnub'; import { CLASS_VERSION, @@ -24,28 +23,17 @@ import { freshPubNub, safeRemove, userPayload, -} from './helpers'; +} from '../../support/datasync-live-helpers'; -describe('DataSync events — Membership', function () { +describe('endpoints/data_sync/events.membership: DataSync events — Membership', function () { this.timeout(60000); - // Earlier nocked Data Sync files call `disableNetConnect()` and never re-enable it. This suite - // talks to the live service, so lift that block before any REST / subscribe call. - before(() => { - nock.cleanAll(); - nock.enableNetConnect(); - }); - let pubnub: ReturnType; beforeEach(() => { pubnub = freshPubNub(); }); - afterEach(() => { - pubnub.destroy(); - }); - /** Seed the User + Channel a Membership joins. */ async function seedEndpoints(): Promise<{ userId: string; channelId: string }> { const userId = freshId('user'); @@ -95,7 +83,7 @@ describe('DataSync events — Membership', function () { className: 'Membership', classLevel: 'Global', }); - assertEventMembershipData(event.message.data as Subscription.DataSyncMembershipData, { + assertEventMembershipData(event.message.data as PubNub.DataSyncMembershipData, { id, channelId, userId, @@ -137,7 +125,7 @@ describe('DataSync events — Membership', function () { className: 'Membership', classLevel: 'Global', }); - assertEventMembershipData(event.message.data as Subscription.DataSyncMembershipData, { + assertEventMembershipData(event.message.data as PubNub.DataSyncMembershipData, { id, channelId, userId, @@ -166,7 +154,7 @@ describe('DataSync events — Membership', function () { className: 'Membership', classLevel: 'Global', }); - assertEventMembershipData(event.message.data as Subscription.DataSyncMembershipData, { + assertEventMembershipData(event.message.data as PubNub.DataSyncMembershipData, { id, channelId, userId, @@ -195,7 +183,7 @@ describe('DataSync events — Membership', function () { className: 'Membership', classLevel: 'Global', }); - assertEventDeleteData(event.message.data as Subscription.DataSyncDeleteData, id, { channelId, userId }); + assertEventDeleteData(event.message.data as PubNub.DataSyncDeleteData, id, { channelId, userId }); }); it('routing proof — a subscriber to the membership id receives nothing', async () => { diff --git a/test/integration/endpoints/data_sync/events.relationship.test.ts b/test/integration/live/endpoints/data_sync/events.relationship.test.ts similarity index 89% rename from test/integration/endpoints/data_sync/events.relationship.test.ts rename to test/integration/live/endpoints/data_sync/events.relationship.test.ts index 744a1c750..e99f84da5 100644 --- a/test/integration/endpoints/data_sync/events.relationship.test.ts +++ b/test/integration/live/endpoints/data_sync/events.relationship.test.ts @@ -8,10 +8,9 @@ * */ -import assert from 'assert'; -import nock from 'nock'; +import assert from 'node:assert'; -import type * as Subscription from '../../../../src/core/types/api/subscription'; +import type PubNub from 'pubnub'; import { CLASS_VERSION, @@ -27,28 +26,17 @@ import { freshPubNub, loanQuotePayload, safeRemove, -} from './helpers'; +} from '../../support/datasync-live-helpers'; -describe('DataSync events Relationship REQUESTED_BY', function () { +describe('endpoints/data_sync/events.relationship: DataSync events Relationship REQUESTED_BY', function () { this.timeout(60000); - // Earlier nocked Data Sync files call `disableNetConnect()` and never re-enable it. This suite - // talks to the live service, so lift that block before any REST / subscribe call. - before(() => { - nock.cleanAll(); - nock.enableNetConnect(); - }); - let pubnub: ReturnType; beforeEach(() => { pubnub = freshPubNub(); }); - afterEach(() => { - pubnub.destroy(); - }); - /** Seed the two endpoint entities a REQUESTED_BY relationship links. */ async function seedEndpoints(): Promise<{ customerId: string; loanQuoteId: string }> { const customerId = freshId('customer'); @@ -109,7 +97,7 @@ describe('DataSync events Relationship REQUESTED_BY', function () { className: RELATIONSHIP_CLASS_REQUESTED_BY, classLevel: 'SubKey', }); - assertEventRelationshipData(event.message.data as Subscription.DataSyncRelationshipData, { + assertEventRelationshipData(event.message.data as PubNub.DataSyncRelationshipData, { id: relId, entityAId: customerId, entityBId: loanQuoteId, @@ -154,7 +142,7 @@ describe('DataSync events Relationship REQUESTED_BY', function () { className: RELATIONSHIP_CLASS_REQUESTED_BY, classLevel: 'SubKey', }); - assertEventRelationshipData(event.message.data as Subscription.DataSyncRelationshipData, { + assertEventRelationshipData(event.message.data as PubNub.DataSyncRelationshipData, { id: relId, entityAId: customerId, entityBId: loanQuoteId, @@ -189,7 +177,7 @@ describe('DataSync events Relationship REQUESTED_BY', function () { className: RELATIONSHIP_CLASS_REQUESTED_BY, classLevel: 'SubKey', }); - assertEventRelationshipData(event.message.data as Subscription.DataSyncRelationshipData, { + assertEventRelationshipData(event.message.data as PubNub.DataSyncRelationshipData, { id: relId, entityAId: customerId, entityBId: loanQuoteId, @@ -221,7 +209,7 @@ describe('DataSync events Relationship REQUESTED_BY', function () { className: RELATIONSHIP_CLASS_REQUESTED_BY, classLevel: 'SubKey', }); - assertEventDeleteData(event.message.data as Subscription.DataSyncDeleteData, relId, { + assertEventDeleteData(event.message.data as PubNub.DataSyncDeleteData, relId, { entityAId: customerId, entityBId: loanQuoteId, }); diff --git a/test/integration/endpoints/data_sync/events.user.test.ts b/test/integration/live/endpoints/data_sync/events.user.test.ts similarity index 81% rename from test/integration/endpoints/data_sync/events.user.test.ts rename to test/integration/live/endpoints/data_sync/events.user.test.ts index 989d67f12..65f4dcbbf 100644 --- a/test/integration/endpoints/data_sync/events.user.test.ts +++ b/test/integration/live/endpoints/data_sync/events.user.test.ts @@ -5,9 +5,7 @@ * `objectType` matches the wire type. Delivered on `channel === id`. */ -import nock from 'nock'; - -import type * as Subscription from '../../../../src/core/types/api/subscription'; +import type PubNub from 'pubnub'; import { CLASS_VERSION, @@ -18,28 +16,17 @@ import { freshId, freshPubNub, userPayload, -} from './helpers'; +} from '../../support/datasync-live-helpers'; -describe('DataSync events — User', function () { +describe('endpoints/data_sync/events.user: DataSync events — User', function () { this.timeout(60000); - // Earlier nocked Data Sync files call `disableNetConnect()` and never re-enable it. This suite - // talks to the live service, so lift that block before any REST / subscribe call. - before(() => { - nock.cleanAll(); - nock.enableNetConnect(); - }); - let pubnub: ReturnType; beforeEach(() => { pubnub = freshPubNub(); }); - afterEach(() => { - pubnub.destroy(); - }); - const createUser = (id: string, payload = userPayload()) => pubnub.dataSync .createUser({ id, data: { classVersion: CLASS_VERSION, status: 'active', payload } }) @@ -63,7 +50,7 @@ describe('DataSync events — User', function () { className: 'User', classLevel: 'Global', }); - assertEventEntityData(event.message.data as Subscription.DataSyncEntityData, { + assertEventEntityData(event.message.data as PubNub.DataSyncEntityData, { id, status: 'active', payload: { firstName: 'Alice', lastName: 'Verma' }, @@ -96,7 +83,7 @@ describe('DataSync events — User', function () { className: 'User', classLevel: 'Global', }); - assertEventEntityData(event.message.data as Subscription.DataSyncEntityData, { + assertEventEntityData(event.message.data as PubNub.DataSyncEntityData, { id, status: 'updated', payload: { level: 'L4' }, @@ -125,7 +112,7 @@ describe('DataSync events — User', function () { className: 'User', classLevel: 'Global', }); - assertEventEntityData(event.message.data as Subscription.DataSyncEntityData, { + assertEventEntityData(event.message.data as PubNub.DataSyncEntityData, { id, payload: { level: 'L5' }, }); @@ -151,6 +138,6 @@ describe('DataSync events — User', function () { className: 'User', classLevel: 'Global', }); - assertEventDeleteData(event.message.data as Subscription.DataSyncDeleteData, id); + assertEventDeleteData(event.message.data as PubNub.DataSyncDeleteData, id); }); }); diff --git a/test/integration/endpoints/data_sync/projections.test.ts b/test/integration/live/endpoints/data_sync/projections.test.ts similarity index 92% rename from test/integration/endpoints/data_sync/projections.test.ts rename to test/integration/live/endpoints/data_sync/projections.test.ts index 6e2e2d352..3cee3764c 100644 --- a/test/integration/endpoints/data_sync/projections.test.ts +++ b/test/integration/live/endpoints/data_sync/projections.test.ts @@ -16,28 +16,26 @@ * */ -import assert from 'assert'; -import nock from 'nock'; +import assert from 'node:assert'; +import PubNub from 'pubnub'; -import PubNub from '../../../../src/node/index'; -import type * as PAM from '../../../../src/core/types/api/access-manager'; -import type * as Subscription from '../../../../src/core/types/api/subscription'; -import type { Status, StatusEvent } from '../../../../src/core/types/api'; +import { createClient } from '../../support/clients'; +import { sleep } from '../../support/wait'; import { CLASS_VERSION, - DATA_SYNC_KEYSET, + dataSyncKeyset, + freshPubNub, + untrackedPubNub, ENTITY_CLASS_CUSTOMER, ENTITY_CLASS_LOAN_QUOTE, RELATIONSHIP_CLASS_REQUESTED_BY, assertEventCommon, assertEventDeleteData, customerPayload, - delay, - freshPubNub, loanQuotePayload, safeRemove, -} from './helpers'; +} from '../../support/datasync-live-helpers'; // -------------------------------------------------------- // ------------------- Projection facts ------------------- @@ -100,22 +98,21 @@ function projectionId(prefix: string): string { /** Token-only client: same keyset as `freshPubNub()` but deliberately **without** `secretKey`. */ function receiverClient(userId: string): PubNub { - const { secretKey: _superUserKey, ...publicKeyset } = DATA_SYNC_KEYSET; - return new PubNub({ + // Created through the tier's `createClient`: the root hook destroys it after the test. + const { secretKey: _superUserKey, ...publicKeyset } = dataSyncKeyset(); + return createClient('datasync', userId, { ...publicKeyset, - userId, - // @ts-expect-error Force override default value. useRequestId: false, - }); + } as Partial); } /** DataSync CRUD wide open — this suite isolates *projections*, never CRUD permission bits. */ -const DATA_SYNC_FULL_ACCESS: PAM.DataSyncTokenScopes = { +const DATA_SYNC_FULL_ACCESS: PubNub.PAM.DataSyncTokenScopes = { entities: { '.*': { get: true, create: true, update: true, delete: true } }, relationships: { '.*': { get: true, create: true, update: true, delete: true } }, }; -const channelReadMap = (names: readonly string[]): Record => +const channelReadMap = (names: readonly string[]): Record => Object.fromEntries(names.map((name) => [name, { read: true, join: true }])); /** @@ -127,7 +124,7 @@ function grantReaderToken( opts: { channelPatterns?: readonly string[]; channelResources?: readonly string[]; - projections?: PAM.DataSyncProjections; + projections?: PubNub.PAM.DataSyncProjections; } = {}, ): Promise { return pnPam.grantToken({ @@ -141,7 +138,7 @@ function grantReaderToken( }); } -/** A reader wired to a freshly minted token. Caller owns `destroy()`. */ +/** A reader wired to a freshly minted token. The root hook destroys it after the test. */ async function readerWithToken( pnPam: PubNub, userId: string, @@ -202,7 +199,7 @@ function rejectsWithDataSyncError(expected: { /** Entity / set subscription — the surface `dataSyncEntity(id).subscription()` returns. */ type EntitySubscription = { - addListener(listener: { dataSync?: (event: Subscription.DataSyncObject) => void }): void; + addListener(listener: { dataSync?: (event: PubNub.Subscription.DataSyncObject) => void }): void; removeListener(listener: object): void; subscribe(): void; unsubscribe(): void; @@ -235,7 +232,7 @@ function adminProjection(reader: PubNub, id: string): EntitySubscription { function captureEvents( pn: PubNub, subscription: EntitySubscription, - predicate: (event: Subscription.DataSyncObject) => boolean, + predicate: (event: PubNub.Subscription.DataSyncObject) => boolean, expectedCount: number, trigger: () => Promise, opts: { @@ -243,21 +240,21 @@ function captureEvents( connectTimeoutMs?: number; timeoutMs?: number; /** When set, wins over `expectedCount` — used when fan-out order is not stable. */ - completeWhen?: (events: Subscription.DataSyncObject[]) => boolean; + completeWhen?: (events: PubNub.Subscription.DataSyncObject[]) => boolean; } = {}, -): Promise { +): Promise { const graceMs = opts.graceMs ?? 1000; const connectTimeoutMs = opts.connectTimeoutMs ?? 20000; const timeoutMs = opts.timeoutMs ?? 45000; - const isComplete = (events: Subscription.DataSyncObject[]): boolean => + const isComplete = (events: PubNub.Subscription.DataSyncObject[]): boolean => opts.completeWhen ? opts.completeWhen(events) : events.length >= expectedCount; - return new Promise((resolve, reject) => { - const collected: Subscription.DataSyncObject[] = []; + return new Promise((resolve, reject) => { + const collected: PubNub.Subscription.DataSyncObject[] = []; let settled = false; const listener = { - dataSync: (event: Subscription.DataSyncObject) => { + dataSync: (event: PubNub.Subscription.DataSyncObject) => { if (settled || !predicate(event)) return; collected.push(event); if (!isComplete(collected)) return; @@ -270,7 +267,7 @@ function captureEvents( /** Resolves on the first connect status; rejects if the subscribe never comes up. */ const connected = new Promise((onConnected, onConnectFailure) => { const statusListener = { - status: (status: Status | StatusEvent) => { + status: (status: PubNub.Status | PubNub.StatusEvent) => { if ( status.category !== PubNub.CATEGORIES.PNConnectedCategory && status.category !== PubNub.CATEGORIES.PNSubscriptionChangedCategory @@ -316,7 +313,7 @@ function captureEvents( subscription.subscribe(); connected - .then(() => delay(graceMs)) + .then(() => sleep(graceMs)) .then(trigger) .catch((error) => { if (settled) return; @@ -336,22 +333,22 @@ async function observeSubscribe( subscription: EntitySubscription, trigger: () => Promise, opts: { settleMs?: number; windowMs?: number } = {}, -): Promise<{ events: Subscription.DataSyncObject[]; statuses: Array }> { +): Promise<{ events: PubNub.Subscription.DataSyncObject[]; statuses: Array }> { const settleMs = opts.settleMs ?? 4000; const windowMs = opts.windowMs ?? 10000; - const events: Subscription.DataSyncObject[] = []; - const statuses: Array = []; - const dataSyncListener = { dataSync: (event: Subscription.DataSyncObject) => events.push(event) }; - const statusListener = { status: (status: Status | StatusEvent) => statuses.push(status) }; + const events: PubNub.Subscription.DataSyncObject[] = []; + const statuses: Array = []; + const dataSyncListener = { dataSync: (event: PubNub.Subscription.DataSyncObject) => events.push(event) }; + const statusListener = { status: (status: PubNub.Status | PubNub.StatusEvent) => statuses.push(status) }; pn.addListener(statusListener); subscription.addListener(dataSyncListener); subscription.subscribe(); try { - await delay(settleMs); + await sleep(settleMs); await trigger(); - await delay(windowMs); + await sleep(windowMs); } finally { subscription.removeListener(dataSyncListener); subscription.unsubscribe(); @@ -361,7 +358,7 @@ async function observeSubscribe( } /** Pick the single event delivered on `channel` out of a fan-out batch. */ -function eventOn(events: Subscription.DataSyncObject[], channel: string): Subscription.DataSyncObject { +function eventOn(events: PubNub.Subscription.DataSyncObject[], channel: string): PubNub.Subscription.DataSyncObject { const matches = events.filter((e) => e.channel === channel); assert.strictEqual(matches.length, 1, `exactly one event on ${channel} (got ${matches.length})`); return matches[0]; @@ -414,16 +411,9 @@ async function seedRequestedBy(pnPam: PubNub, id: string, customerId: string, lo // ------------------------- Tests ------------------------ // -------------------------------------------------------- -describe('DataSync projections admin vs __default__', function () { +describe('endpoints/data_sync/projections: DataSync projections admin vs __default__', function () { this.timeout(120000); - // Earlier nocked Data Sync files call `disableNetConnect()` and never re-enable it. This suite - // talks to the live service, so lift that block before any REST / subscribe call. - before(() => { - nock.cleanAll(); - nock.enableNetConnect(); - }); - // ====================================================== // grantToken / parseToken encoding // ====================================================== @@ -434,10 +424,6 @@ describe('DataSync projections admin vs __default__', function () { pnPam = freshPubNub(); }); - afterEach(() => { - pnPam.destroy(); - }); - const projectionsMeta = (token: string): Record> | undefined => { const meta = pnPam.parseToken(token)?.meta as Record | undefined; return meta?.['pn-projections'] as Record> | undefined; @@ -525,11 +511,11 @@ describe('DataSync projections admin vs __default__', function () { let customerId: string; let loanQuoteId: string; let relationshipId: string; - const readers: PubNub[] = []; /** Seeded once — there is nothing to isolate per test. */ before(async () => { - pnPam = freshPubNub(); + // Built in a `before` hook, so it must outlive the first test: not tracked by the root hook, destroyed in `after`. + pnPam = untrackedPubNub(); customerId = projectionId('customer'); loanQuoteId = projectionId('loanquote'); relationshipId = projectionId('requestedby'); @@ -545,18 +531,11 @@ describe('DataSync projections admin vs __default__', function () { pnPam.destroy(); }); - afterEach(() => { - while (readers.length) readers.pop()!.destroy(); - }); - - /** Track for teardown. */ - const track = (reader: PubNub): PubNub => { - readers.push(reader); - return reader; - }; + /** Readers come from the tier's `createClient`, so the root hook destroys them after each test. */ + const track = (reader: PubNub): PubNub => reader; /** Projection assignments covering every object this section seeded. */ - const allAdmin = (): PAM.DataSyncProjections => ({ + const allAdmin = (): PubNub.PAM.DataSyncProjections => ({ patterns: { entities: { 'customer.*': PROJECTION_ADMIN, 'loanquote.*': PROJECTION_ADMIN }, relationships: { 'requestedby.*': PROJECTION_ADMIN }, @@ -739,8 +718,6 @@ describe('DataSync projections admin vs __default__', function () { afterEach(async () => { await safeRemove(() => pnPam.dataSync.removeEntity({ id: customerId })); - reader?.destroy(); - pnPam.destroy(); }); /** Wildcard grant covering both the id channel and its mirror, spelled out explicitly. */ @@ -773,8 +750,8 @@ describe('DataSync projections admin vs __default__', function () { } // Same object, same revision — only the payload projection differs. - const plainData = onPlain.message.data as Subscription.DataSyncEntityData; - const mirrorData = onMirror.message.data as Subscription.DataSyncEntityData; + const plainData = onPlain.message.data as PubNub.DataSyncEntityData; + const mirrorData = onMirror.message.data as PubNub.DataSyncEntityData; assert.strictEqual(mirrorData.eTag, plainData.eTag, 'both events describe the same revision'); assert.strictEqual(mirrorData.updatedAt, plainData.updatedAt, 'same updatedAt'); @@ -798,7 +775,7 @@ describe('DataSync projections admin vs __default__', function () { () => seedCustomer(pnPam, customerId), ); - const mirrorData = eventOn(events, mirrorChannel).message.data as Subscription.DataSyncEntityData; + const mirrorData = eventOn(events, mirrorChannel).message.data as PubNub.DataSyncEntityData; assertPayloadFields(payloadOf(mirrorData), CUSTOMER_ADMIN_FIELDS, 'mirror event, projection-less token'); assert.strictEqual(payloadOf(mirrorData)!.private, CUSTOMER_SECRET, '`private` delivered regardless'); @@ -820,7 +797,7 @@ describe('DataSync projections admin vs __default__', function () { () => seedCustomer(pnPam, customerId), ); - const mirrorData = events[0].message.data as Subscription.DataSyncEntityData; + const mirrorData = events[0].message.data as PubNub.DataSyncEntityData; assert.strictEqual(events[0].channel, mirrorChannel, 'delivered on the mirror channel'); assert.strictEqual(payloadOf(mirrorData)!.private, CUSTOMER_SECRET, '`private` leaked via the pattern grant'); }); @@ -845,7 +822,7 @@ describe('DataSync projections admin vs __default__', function () { (status) => status.category === PubNub.CATEGORIES.PNAccessDeniedCategory || status.category === PubNub.CATEGORIES.PNConnectionErrorCategory || - (status as Status).statusCode === 403, + (status as PubNub.Status).statusCode === 403, ); assert.ok(refused, `subscribe must be refused; saw [${statuses.map((s) => s.category).join(', ')}]`); }); @@ -866,12 +843,12 @@ describe('DataSync projections admin vs __default__', function () { .then(() => undefined), ); - const plainData = eventOn(events, plainChannel).message.data as Subscription.DataSyncEntityData; + const plainData = eventOn(events, plainChannel).message.data as PubNub.DataSyncEntityData; assertPayloadFields(payloadOf(plainData), CUSTOMER_DEFAULT_FIELDS, 'id-channel update'); assert.strictEqual(payloadOf(plainData)!.city, 'Mumbai', 'default field updated'); assert.ok(!('private' in payloadOf(plainData)!), 'id channel still withholds `private`'); - const mirrorData = eventOn(events, mirrorChannel).message.data as Subscription.DataSyncEntityData; + const mirrorData = eventOn(events, mirrorChannel).message.data as PubNub.DataSyncEntityData; assertPayloadFields(payloadOf(mirrorData), CUSTOMER_ADMIN_FIELDS, 'mirror-channel update'); assert.strictEqual(payloadOf(mirrorData)!.private, 'rotated', 'rotated `private` visible on the mirror'); assert.strictEqual(payloadOf(mirrorData)!.city, 'Mumbai', 'shared field updated on the mirror too'); @@ -901,7 +878,7 @@ describe('DataSync projections admin vs __default__', function () { className: ENTITY_CLASS_CUSTOMER, classLevel: 'SubKey', }); - assertEventDeleteData(event.message.data as Subscription.DataSyncDeleteData, customerId); + assertEventDeleteData(event.message.data as PubNub.DataSyncDeleteData, customerId); } }); }); @@ -928,8 +905,6 @@ describe('DataSync projections admin vs __default__', function () { await safeRemove(() => pnPam.dataSync.removeRelationship({ id: relationshipId })); await safeRemove(() => pnPam.dataSync.removeEntity({ id: customerId })); await safeRemove(() => pnPam.dataSync.removeEntity({ id: loanQuoteId })); - reader?.destroy(); - pnPam.destroy(); }); itWhenAllClassesEmit('LoanQuote create — mirror narrows to {quoteId, private}', async () => { @@ -945,10 +920,10 @@ describe('DataSync projections admin vs __default__', function () { () => seedLoanQuote(pnPam, loanQuoteId), ); - const plainData = eventOn(events, loanQuoteId).message.data as Subscription.DataSyncEntityData; + const plainData = eventOn(events, loanQuoteId).message.data as PubNub.DataSyncEntityData; assertPayloadFields(payloadOf(plainData), LOAN_QUOTE_DEFAULT_FIELDS, 'LoanQuote id-channel event'); - const mirrorData = eventOn(events, adminChannel(loanQuoteId)).message.data as Subscription.DataSyncEntityData; + const mirrorData = eventOn(events, adminChannel(loanQuoteId)).message.data as PubNub.DataSyncEntityData; assertPayloadFields(payloadOf(mirrorData), LOAN_QUOTE_ADMIN_FIELDS, 'LoanQuote mirror event'); assert.strictEqual(payloadOf(mirrorData)!.private, LOAN_QUOTE_SECRET, 'LoanQuote `private` on the mirror'); }); @@ -1006,11 +981,11 @@ describe('DataSync projections admin vs __default__', function () { assert.ok(mirrorEvents.length >= 1, 'relationship event on at least one endpoint mirror channel'); for (const event of plainEvents) { - const data = event.message.data as Subscription.DataSyncRelationshipData; + const data = event.message.data as PubNub.DataSyncRelationshipData; assertPayloadFields(payloadOf(data), REQUESTED_BY_DEFAULT_FIELDS, `${event.channel} (default)`); } for (const event of mirrorEvents) { - const data = event.message.data as Subscription.DataSyncRelationshipData; + const data = event.message.data as PubNub.DataSyncRelationshipData; assertPayloadFields(payloadOf(data), REQUESTED_BY_ADMIN_FIELDS, `${event.channel} (admin)`); assert.strictEqual(payloadOf(data)!.private, REQUESTED_BY_SECRET, 'relationship `private` on the mirror'); } diff --git a/test/integration/live/endpoints/fetch_messages.test.ts b/test/integration/live/endpoints/fetch_messages.test.ts new file mode 100644 index 000000000..fc92992af --- /dev/null +++ b/test/integration/live/endpoints/fetch_messages.test.ts @@ -0,0 +1,145 @@ +/** + * Real-network counterpart of test/integration/endpoints/fetch_messages.test.ts: the four tests that publish to the service + * and read the messages back (the nock-mocked tests of that file stay there). Not documentation samples. + * + * Needs the `open` keyset with Message Persistence and message actions (SUBSCRIBE_KEY, PUBLISH_KEY). Every test uses a channel + * named after a fresh UUID, so runs on a shared, persistent keyset never see each other's messages. Reads that depend on + * eventually consistent storage are polled with `eventually` instead of waiting a fixed time. + */ +import type PubNub from 'pubnub'; + +import assert from 'node:assert'; +import { randomUUID } from 'node:crypto'; + +import { createClient } from '../support/clients'; +import { eventually } from '../support/wait'; + +/** Published test message shape. */ +type TestMessage = { messageIdx: string; time: number }; + +type Published = { message: TestMessage; timetoken: string }; + +const uniqueChannel = (): string => `fetch-messages-${randomUUID()}`; + +/** Publishes `count` messages one after another (even indexes carry metadata), returns them sorted by timetoken. */ +async function publishMessagesToChannel( + client: PubNub, + count: number, + channel: string, + customMessageType: string, +): Promise { + const messages: Published[] = []; + + for (let messageIdx = 0; messageIdx < count; messageIdx += 1) { + const message: TestMessage = { messageIdx: [channel, messageIdx].join(': '), time: Date.now() }; + const payload: PubNub.Publish.PublishParameters = { message, channel }; + + if (customMessageType.length) payload.customMessageType = customMessageType; + if (messageIdx % 2 === 0) payload.meta = { time: message.time }; + + const response = await client.publish(payload); + messages.push({ message, timetoken: response.timetoken }); + } + + return messages.sort((left, right) => parseInt(left.timetoken, 10) - parseInt(right.timetoken, 10)); +} + +/** Attaches `count` message actions to each referenced message and returns the added actions sorted by action timetoken. */ +async function addActionsInChannel( + client: PubNub, + count: number, + messageTimetokens: string[], + channel: string, +): Promise { + const types = ['reaction', 'receipt', 'custom']; + const values = Array.from({ length: 10 }, () => randomUUID()); + const actions: PubNub.MessageAction.MessageAction[] = []; + + for (const messageTimetoken of messageTimetokens) { + for (let messageActionIdx = 0; messageActionIdx < count; messageActionIdx += 1) { + const action = { type: types[(messageActionIdx + 1) % 3], value: values[(messageActionIdx + 1) % 10] }; + const response = await client.addMessageAction({ channel, messageTimetoken, action }); + actions.push(response.data); + } + } + + return actions.sort((left, right) => parseInt(left.actionTimetoken, 10) - parseInt(right.actionTimetoken, 10)); +} + +describe('endpoints/fetch_messages: fetchMessages against the service', function () { + this.timeout(60000); + + let pubnub: PubNub; + + beforeEach(() => { + // The root hook destroys the client after the test. + pubnub = createClient('open', 'myUUID'); + }); + + it('supports metadata', async () => { + const channel = uniqueChannel(); + const messages = await publishMessagesToChannel(pubnub, 10, channel, ''); + + await eventually( + async () => { + const response = await pubnub.fetchMessages({ channels: [channel], count: 25, includeMeta: true }); + const channelMessages = response.channels[channel]; + + assert.deepEqual(channelMessages[0].meta, { time: messages[0].message.time }); + assert(!channelMessages[1].meta); + }, + { timeout: 30_000, label: 'fetchMessages with includeMeta' }, + ); + }); + + it('supports custom message type', async () => { + const channel = uniqueChannel(); + await publishMessagesToChannel(pubnub, 2, channel, 'test-message-type'); + + const response = await pubnub.fetchMessages({ channels: [channel], includeCustomMessageType: true }); + response.channels[channel].forEach((message) => { + assert.equal(message.customMessageType, 'test-message-type'); + }); + }); + + for (const field of ['data', 'actions'] as const) { + it(`supports actions (stored as '${field}' field)`, async () => { + const channel = uniqueChannel(); + const expectedActionsCount = 4; + const messages = await publishMessagesToChannel(pubnub, 2, channel, ''); + const messageTimetokens = messages.map((message) => message.timetoken); + const actions = await addActionsInChannel(pubnub, expectedActionsCount, messageTimetokens, channel); + + // The mocked original waited a fixed 2 s before reading; poll until the actions are visible instead. + await eventually( + async () => { + const response = await pubnub.fetchMessages({ channels: [channel], includeMessageActions: true }); + const fetchedMessages = response.channels[channel]; + // TypeScript types system now requires to figure out type of object before using it. + assert('actions' in fetchedMessages[0]); + const actionsByType = (field === 'data' ? fetchedMessages[0].data : fetchedMessages[0].actions) ?? {}; + let historyActionsCount = 0; + + Object.keys(actionsByType).forEach((actionType) => { + Object.keys(actionsByType[actionType]).forEach((actionValue) => { + historyActionsCount += 1; + + assert.equal( + actions.some((action) => action.value === actionValue), + true, + ); + }); + }); + + assert.equal(historyActionsCount, expectedActionsCount); + assert.equal(fetchedMessages[0].timetoken, messageTimetokens[0]); + assert.equal( + fetchedMessages[fetchedMessages.length - 1].timetoken, + messageTimetokens[messageTimetokens.length - 1], + ); + }, + { timeout: 30_000, label: `fetchMessages with actions (${field})` }, + ); + }); + } +}); diff --git a/test/integration/live/endpoints/history.test.ts b/test/integration/live/endpoints/history.test.ts new file mode 100644 index 000000000..05fb66799 --- /dev/null +++ b/test/integration/live/endpoints/history.test.ts @@ -0,0 +1,65 @@ +/** + * Real-network counterpart of test/integration/endpoints/history.test.ts: the one test that publishes to the service and reads + * the history back (the nock-mocked tests of that file stay there). Not a documentation sample. + * + * Needs the `open` keyset with Message Persistence (SUBSCRIBE_KEY, PUBLISH_KEY). The channel is named after a fresh UUID, so runs + * on a shared, persistent keyset never see each other's messages. The read is polled with `eventually`, because history is + * eventually consistent. + */ +import type PubNub from 'pubnub'; + +import assert from 'node:assert'; +import { randomUUID } from 'node:crypto'; + +import { createClient } from '../support/clients'; +import { eventually } from '../support/wait'; + +/** Published test message shape. */ +type TestMessage = { messageIdx: string; time: number }; + +/** Publishes `count` messages one after another (even indexes carry metadata), returns them sorted by timetoken. */ +async function publishMessagesToChannel( + client: PubNub, + count: number, + channel: string, +): Promise<{ message: TestMessage; timetoken: string }[]> { + const messages: { message: TestMessage; timetoken: string }[] = []; + + for (let messageIdx = 0; messageIdx < count; messageIdx += 1) { + const message: TestMessage = { messageIdx: [channel, messageIdx].join(': '), time: Date.now() }; + const payload: { channel: string; message: TestMessage; meta?: PubNub.Payload } = { message, channel }; + + if (messageIdx % 2 === 0) payload.meta = { time: message.time }; + + const response = await client.publish(payload); + messages.push({ message, timetoken: response.timetoken }); + } + + return messages.sort((left, right) => parseInt(left.timetoken, 10) - parseInt(right.timetoken, 10)); +} + +describe('endpoints/history: history against the service', function () { + this.timeout(60000); + + let pubnub: PubNub; + + beforeEach(() => { + // The root hook destroys the client after the test. + pubnub = createClient('open', 'myUUID'); + }); + + it('supports metadata', async () => { + const channel = `history-${randomUUID()}`; + const messages = await publishMessagesToChannel(pubnub, 10, channel); + + await eventually( + async () => { + const response = await pubnub.history({ channel, includeMeta: true }); + + assert.deepEqual(response.messages[0].meta, { time: messages[0].message.time }); + assert(!response.messages[1].meta); + }, + { timeout: 30_000, label: 'history with includeMeta' }, + ); + }); +}); diff --git a/test/integration/live/endpoints/message_actions.test.ts b/test/integration/live/endpoints/message_actions.test.ts new file mode 100644 index 000000000..354a63c0f --- /dev/null +++ b/test/integration/live/endpoints/message_actions.test.ts @@ -0,0 +1,235 @@ +/** + * Real-network counterpart of test/integration/endpoints/message_actions.test.ts: the nine tests that publish to the service and + * add, read or remove message actions (the nock-mocked tests of that file stay there). Not documentation samples. + * + * Needs the `open` keyset with Message Persistence and message actions (SUBSCRIBE_KEY, PUBLISH_KEY). Every test uses a channel + * named after a fresh UUID, so runs on a shared, persistent keyset never see each other's messages or actions. The mocked + * original waited a fixed 500 ms or 2 s before reading actions back; here the read is polled with `eventually`, and the + * message action events are awaited with `expectEvent`. + */ +import type PubNub from 'pubnub'; + +import assert from 'node:assert'; +import { randomUUID } from 'node:crypto'; + +import { createClient } from '../support/clients'; +import { connected, eventually, expectEvent } from '../support/wait'; + +/** Message action object. */ +type MessageAction = PubNub.MessageAction.AddMessageActionParameters['action']; + +const uniqueChannel = (suffix = ''): string => `message-actions-${randomUUID()}${suffix}`; + +/** Publishes `count` messages one after another and returns their timetokens, sorted. */ +async function publishMessages(client: PubNub, count: number, channel: string): Promise { + const timetokens: string[] = []; + + for (let messageIdx = 0; messageIdx < count; messageIdx += 1) { + const response = await client.publish({ message: { messageIdx, time: Date.now() }, channel }); + timetokens.push(response.timetoken); + } + + return timetokens.sort((left, right) => parseInt(left, 10) - parseInt(right, 10)); +} + +/** Attaches `count` message actions to each referenced message and returns the action timetokens, sorted. */ +async function addActions( + client: PubNub, + count: number, + messageTimetokens: string[], + channel: string, +): Promise { + const types = ['reaction', 'receipt', 'custom']; + const values = Array.from({ length: 10 }, () => randomUUID()); + const timetokens: string[] = []; + + for (const messageTimetoken of messageTimetokens) { + for (let messageActionIdx = 0; messageActionIdx < count; messageActionIdx += 1) { + const action: MessageAction = { + type: types[(messageActionIdx + 1) % 3], + value: values[(messageActionIdx + 1) % 10], + }; + const response = await client.addMessageAction({ channel, messageTimetoken, action }); + timetokens.push(response.data.actionTimetoken); + } + } + + return timetokens.sort((left, right) => parseInt(left, 10) - parseInt(right, 10)); +} + +describe('endpoints/message_actions: message actions against the service', function () { + this.timeout(60000); + + let pubnub: PubNub; + + beforeEach(() => { + // The root hook destroys the client after the test. + pubnub = createClient('open', 'myUUID'); + }); + + describe('addMessageAction', () => { + const addsAction = async (channel: string): Promise => { + const messageAction: MessageAction = { type: 'custom', value: randomUUID() }; + const [messageTimetoken] = await publishMessages(pubnub, 1, channel); + + const response = await pubnub.addMessageAction({ channel, messageTimetoken, action: messageAction }); + + assert.equal(response.data.type, messageAction.type); + assert.equal(response.data.value, messageAction.value); + assert.equal(response.data.uuid, pubnub.getUUID()); + assert.equal(response.data.messageTimetoken, messageTimetoken); + assert(response.data.actionTimetoken); + }; + + it('add message action', () => addsAction(uniqueChannel())); + + it('add message action with encoded channel', () => addsAction(uniqueChannel('#1'))); + + it('add message action should trigger event', async () => { + const messageAction: MessageAction = { type: 'custom', value: randomUUID() }; + const channel = uniqueChannel(); + + const ready = connected(pubnub, { channels: [channel] }); + const event = expectEvent(pubnub, 'messageAction', (e) => e.data.value === messageAction.value); + pubnub.subscribe({ channels: [channel] }); + await ready; + + const published = await pubnub.publish({ channel, message: { hello: 'test' }, sendByPost: true }); + await pubnub.addMessageAction({ channel, messageTimetoken: published.timetoken, action: messageAction }); + + const messageActionEvent = await event; + assert(messageActionEvent.data); + assert.equal(messageActionEvent.data.type, messageAction.type); + assert.equal(messageActionEvent.data.value, messageAction.value); + assert.equal(messageActionEvent.data.uuid, pubnub.getUUID()); + assert.equal(messageActionEvent.data.messageTimetoken, published.timetoken); + assert(messageActionEvent.data.actionTimetoken); + assert.equal(messageActionEvent.event, 'added'); + pubnub.unsubscribeAll(); + }); + }); + + describe('removeMessageAction', () => { + const removesAction = async (channel: string): Promise => { + const messageTimetokens = await publishMessages(pubnub, 1, channel); + const actionTimetokens = await addActions(pubnub, 1, messageTimetokens, channel); + + await eventually( + async () => { + const response = await pubnub.getMessageActions({ channel }); + assert.equal(response.data.length, actionTimetokens.length); + }, + { timeout: 30_000, label: 'the added action is listed' }, + ); + + // A rejected promise is the failed status of the callback form (`status.error`). + await pubnub.removeMessageAction({ + channel, + actionTimetoken: actionTimetokens[0], + messageTimetoken: messageTimetokens[0], + }); + + await eventually( + async () => { + const response = await pubnub.getMessageActions({ channel }); + assert.equal(response.data.length, 0); + }, + { timeout: 30_000, label: 'the removed action is gone' }, + ); + }; + + it('remove message action', () => removesAction(uniqueChannel())); + + it('remove message action with encoded channel', () => removesAction(uniqueChannel('#1'))); + + it('remove message action should trigger event', async () => { + const channel = uniqueChannel(); + const messageTimetokens = await publishMessages(pubnub, 1, channel); + const actionTimetokens = await addActions(pubnub, 1, messageTimetokens, channel); + + const ready = connected(pubnub, { channels: [channel] }); + const event = expectEvent(pubnub, 'messageAction', (e) => e.event === 'removed'); + pubnub.subscribe({ channels: [channel] }); + await ready; + + await pubnub.removeMessageAction({ + channel, + actionTimetoken: actionTimetokens[0], + messageTimetoken: messageTimetokens[0], + }); + + const messageActionEvent = await event; + assert(messageActionEvent.data); + assert.equal(messageActionEvent.data.uuid, pubnub.getUUID()); + assert.equal(messageActionEvent.data.messageTimetoken, messageTimetokens[0]); + assert.equal(messageActionEvent.data.actionTimetoken, actionTimetokens[0]); + assert.equal(messageActionEvent.event, 'removed'); + pubnub.unsubscribeAll(); + }); + }); + + describe('getMessageAction', () => { + const fetchesActions = async (channel: string): Promise => { + const messageTimetokens = await publishMessages(pubnub, 2, channel); + const actionTimetokens = await addActions(pubnub, 3, messageTimetokens, channel); + const lastPublishedActionTimetoken = actionTimetokens[actionTimetokens.length - 1]; + const firstPublishedActionTimetoken = actionTimetokens[0]; + + await eventually( + async () => { + const response = await pubnub.getMessageActions({ channel }); + const firstFetchedActionTimetoken = response.data[0].actionTimetoken; + const lastFetchedActionTimetoken = response.data[response.data.length - 1].actionTimetoken; + assert.equal(firstFetchedActionTimetoken, firstPublishedActionTimetoken); + assert.equal(lastFetchedActionTimetoken, lastPublishedActionTimetoken); + assert.equal(response.data.length, actionTimetokens.length); + assert.equal(response.start, firstPublishedActionTimetoken); + assert.equal(response.end, lastPublishedActionTimetoken); + }, + { timeout: 30_000, label: 'getMessageActions lists every added action' }, + ); + }; + + it('fetch message actions', () => fetchesActions(uniqueChannel())); + + it('fetch message actions with encoded channel', () => fetchesActions(uniqueChannel('#1'))); + + it('fetch next message actions page', async () => { + const channel = uniqueChannel(); + const messageTimetokens = await publishMessages(pubnub, 2, channel); + const actionTimetokens = await addActions(pubnub, 5, messageTimetokens, channel); + const lastPublishedActionTimetoken = actionTimetokens[actionTimetokens.length - 1]; + const halfSize = Math.floor(actionTimetokens.length * 0.5); + const firstPublishedActionTimetoken = actionTimetokens[0]; + const middleMinusOnePublishedActionTimetoken = actionTimetokens[halfSize - 1]; + const middlePublishedActionTimetoken = actionTimetokens[halfSize]; + + await eventually( + async () => { + const response = await pubnub.getMessageActions({ channel, limit: halfSize }); + let firstFetchedActionTimetoken = response.data[0].actionTimetoken; + let lastFetchedActionTimetoken = response.data[response.data.length - 1].actionTimetoken; + assert.equal(firstFetchedActionTimetoken, middlePublishedActionTimetoken); + assert.equal(lastFetchedActionTimetoken, lastPublishedActionTimetoken); + assert.equal(response.data.length, halfSize); + assert.equal(response.start, middlePublishedActionTimetoken); + assert.equal(response.end, lastPublishedActionTimetoken); + + const next = await pubnub.getMessageActions({ + channel, + start: middlePublishedActionTimetoken, + limit: halfSize, + }); + firstFetchedActionTimetoken = next.data[0].actionTimetoken; + lastFetchedActionTimetoken = next.data[next.data.length - 1].actionTimetoken; + assert.equal(firstFetchedActionTimetoken, firstPublishedActionTimetoken); + assert.equal(lastFetchedActionTimetoken, middleMinusOnePublishedActionTimetoken); + assert.equal(next.data.length, halfSize); + assert.equal(next.start, firstPublishedActionTimetoken); + assert.equal(next.end, middleMinusOnePublishedActionTimetoken); + }, + { timeout: 30_000, label: 'getMessageActions pages' }, + ); + }); + }); +}); diff --git a/test/integration/live/fixtures/cat_picture.jpg b/test/integration/live/fixtures/cat_picture.jpg new file mode 100644 index 0000000000000000000000000000000000000000..6878c904f713ce1d34fc1d712382be106c76ca79 GIT binary patch literal 134 zcmex= { + // The sample writes `./pubnub.log`, so the test changes into a scratch directory under the OS temp dir and removes it afterwards. + // The file is read back with `eventually` because the write stream flushes asynchronously. + it('loggingFileLogger: a file logger writes the SDK log lines to the log file', async () => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'pn-logging-')); + const previous = process.cwd(); + process.chdir(dir); + // Debug level makes the console logger print the configuration, which holds the keys. Keep it out of the test output. + const quiet = quietConsole(); + try { + /* eslint-disable prettier/prettier */ + // snippet.loggingFileLogger + class FileLogger implements Logger { + private readonly stream: fs.WriteStream; + + constructor(private readonly logFilePath: string) { + this.stream = fs.createWriteStream(logFilePath, { flags: 'a' }); + } + + log(logMessage: LogMessage) { + const level = PubNub.LogLevel[logMessage.level]; + const timestamp = logMessage.timestamp.toISOString(); + const location = logMessage.location || 'Unknown'; + const message = this.formatMessage(logMessage); + + this.stream.write( + `${timestamp} [${level.toUpperCase()}] ${location}: ${message}\n` + ); + } + + formatMessage(logMessage: LogMessage) { + if (logMessage.messageType === 'text') { + return logMessage.message; + } else if (logMessage.messageType === 'object') { + return `${logMessage.details || ''}\n${JSON.stringify(logMessage.message, null, 2)}`; + } else if (logMessage.messageType === 'error') { + return `${logMessage.message.name}: ${logMessage.message.message}`; + } + // Handle other message types + return JSON.stringify(logMessage.message); + } + + trace(logMessage: LogMessage) { this.log(logMessage); } + debug(logMessage: LogMessage) { this.log(logMessage); } + info(logMessage: LogMessage) { this.log(logMessage); } + warn(logMessage: LogMessage) { this.log(logMessage); } + error(logMessage: LogMessage) { this.log(logMessage); } + } + + // Use FileLogger with PubNub + const pubnub = new PubNub({ + subscribeKey: 'mySubscribeKey', + publishKey: 'myPublishKey', + userId: 'myUniqueUserId', + logLevel: PubNub.LogLevel.Debug, + // snippet.hide + ...openKeys(), + // snippet.show + loggers: [new FileLogger('./pubnub.log')] + }); + // snippet.end + /* eslint-enable prettier/prettier */ + track(pubnub); + // The SDK returns the timetoken of time() as a number, so compare its digits. + assert.match(String((await pubnub.time()).timetoken), /^\d{17}$/); + const content = await eventually( + () => { + const text = fs.readFileSync(path.join(dir, 'pubnub.log'), 'utf8'); + assert.match(text, /\[DEBUG\]/, 'debug entries are in the file'); + return text; + }, + { label: 'file has log entries' }, + ); + for (const line of content.split('\n').filter((l) => /^\d{4}-\d{2}-\d{2}T/.test(l))) { + assert.match(line, /^\d{4}-\d{2}-\d{2}T[\d:.]+Z \[(TRACE|DEBUG|INFO|WARN|ERROR)\] \S+: /); + } + assert.ok(/network|GET|\/time\//i.test(content), 'the time() request appears in the log'); + } finally { + quiet.restore(); + process.chdir(previous); + fs.rmSync(dir, { recursive: true, force: true }); + } + }); +}); diff --git a/test/integration/live/getting-started/logging.test.ts b/test/integration/live/getting-started/logging.test.ts new file mode 100644 index 000000000..790525e0b --- /dev/null +++ b/test/integration/live/getting-started/logging.test.ts @@ -0,0 +1,457 @@ +import assert from 'node:assert/strict'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +/* eslint-disable prettier/prettier */ +// snippet.loggingImportLogger +import { Logger, LogMessage } from 'pubnub' + +// snippet.end +/* eslint-enable prettier/prettier */ +/* eslint-disable prettier/prettier */ +// snippet.loggingImportPubNub +import PubNub from 'pubnub'; + +// snippet.end +/* eslint-enable prettier/prettier */ +import { createClient, track, unreachable } from '../support/clients'; +import { quietConsole } from '../support/console'; +import { openKeys } from '../support/env'; +import { eventually } from '../support/wait'; + +/** + * Logging examples. Port of docs-snippets/getting-started/logging.ts. + * The samples show placeholder keys. Hidden lines spread the real keys of the open keyset over them. + */ +const LogLevel = PubNub.LogLevel; + +describe('getting-started/logging', () => { + type Payload = Record; + /** Records what a logger receives, per level and per message type. */ + const recorder = () => { + const seen: Array<{ level: string; message: LogMessage }> = []; + const logger: Logger = { + trace: (m) => seen.push({ level: 'trace', message: m }), + debug: (m) => seen.push({ level: 'debug', message: m }), + info: (m) => seen.push({ level: 'info', message: m }), + warn: (m) => seen.push({ level: 'warn', message: m }), + error: (m) => seen.push({ level: 'error', message: m }), + }; + return { seen, logger }; + }; + + it('loggingBasicUsage: a client built with Debug level logging works and its logger output is captured', async () => { + const quiet = quietConsole(); + try { + /* eslint-disable prettier/prettier */ + // snippet.loggingBasicUsage + // Initialize PubNub with debug-level logging + const pubnub = new PubNub({ + subscribeKey: 'mySubscribeKey', + publishKey: 'myPublishKey', + userId: 'myUniqueUserId', + // snippet.hide + ...openKeys(), + // snippet.show + logLevel: PubNub.LogLevel.Debug + }); + // snippet.end + /* eslint-enable prettier/prettier */ + track(pubnub); + assert.equal(pubnub.getUserId(), 'myUniqueUserId'); + } finally { + quiet.restore(); + } + }); + + it('loggingCustomLoggerExample: a custom logger receives config, error, failed request and deprecation entries', async () => { + // "The example assumes that monitoringService and analyticsService are objects in your application." + const reported = { + errors: [] as Payload[], + failures: [] as Payload[], + deprecations: [] as Payload[], + configs: [] as Payload[], + }; + const monitoringService = { + reportError: (payload: Payload) => reported.errors.push(payload), + reportNetworkFailure: (payload: Payload) => reported.failures.push(payload), + }; + const analyticsService = { + trackDeprecation: (payload: Payload) => reported.deprecations.push(payload), + trackConfigChange: (payload: Payload) => reported.configs.push(payload), + }; + const quiet = quietConsole(); + try { + /* eslint-disable prettier/prettier */ + // snippet.loggingCustomLoggerExample + class MonitoringLogger implements Logger { + /** + * Process error-level messages + */ + error(logMessage: LogMessage) { + const { timestamp, pubNubId, location, messageType } = logMessage; + + if (messageType === 'error') { + // TypeScript narrows type to ErrorLogMessage + // IDE provides autocomplete for PubNubError fields + monitoringService.reportError({ + message: logMessage.message.message, + name: logMessage.message.name, + stack: logMessage.message.stack, + category: logMessage.message.status?.category, + operation: logMessage.message.status?.operation, + instanceId: pubNubId, + timestamp: timestamp.toISOString(), + location + }); + } + } + + /** + * Process warning messages and failed requests + */ + warn(logMessage: LogMessage) { + if (logMessage.messageType === 'text') { + // TypeScript narrows type to TextLogMessage + // IDE provides autocomplete for string methods on message + if (logMessage.message.includes('deprecated')) { + analyticsService.trackDeprecation({ + warning: logMessage.message, + instanceId: logMessage.pubNubId, + timestamp: logMessage.timestamp.toISOString() + }); + } + } else if (logMessage.messageType === 'network-request') { + // TypeScript narrows type to NetworkRequestLogMessage + // The SDK logs a failed request at the warn level + if (logMessage.failed) { + const url = `${logMessage.message.origin}${logMessage.message.path}`; + + monitoringService.reportNetworkFailure({ + url: url, + method: logMessage.message.method, + reason: logMessage.details, + instanceId: logMessage.pubNubId, + timestamp: logMessage.timestamp.toISOString() + }); + } + } + } + + /** + * Process debug messages for configuration tracking + */ + debug(logMessage: LogMessage) { + if (logMessage.messageType === 'object') { + // TypeScript narrows type to ObjectLogMessage + // IDE provides autocomplete for details and ignoredKeys fields + if (logMessage.details?.includes('configuration')) { + analyticsService.trackConfigChange({ + config: logMessage.message, + instanceId: logMessage.pubNubId, + timestamp: logMessage.timestamp.toISOString() + }); + } + } + } + + // No-op implementations for other levels if not needed + trace() {} + info() {} + } + + // Configure SDK with custom logger + const pubnub = new PubNub({ + subscribeKey: 'mySubscribeKey', + publishKey: 'myPublishKey', + userId: 'myUniqueUserId', + logLevel: PubNub.LogLevel.Debug, + // snippet.hide + ...openKeys(), ...unreachable, + // snippet.show + loggers: [new MonitoringLogger()] + }); + // snippet.end + /* eslint-enable prettier/prettier */ + track(pubnub); + + // Debug entry about the configuration reaches trackConfigChange. + assert.ok(reported.configs.length > 0, 'trackConfigChange received the configuration entry'); + assert.ok(reported.configs[0].instanceId, 'instanceId (pubNubId) is set'); + assert.match(String(reported.configs[0].timestamp), /^\d{4}-\d{2}-\d{2}T/, 'timestamp is an ISO string'); + + // The client points at a port where nothing listens, so a request fails. The failure reaches reportError (error entry) + // and reportNetworkFailure (failed network-request entry). The deprecated stop() reaches trackDeprecation (warn entry). + await assert.rejects(pubnub.publish({ channel: 'x', message: 'm' })); + await eventually(() => assert.ok(reported.errors.length > 0, 'reportError received an error entry'), { + label: 'error entry', + }); + assert.equal(reported.errors[0].name, 'PubNubError'); + assert.ok(reported.errors[0].message); + assert.equal(reported.errors[0].category, 'PNNetworkIssuesCategory'); + assert.equal(reported.errors[0].operation, 'PNPublishOperation'); + assert.ok(reported.errors[0].stack, 'stack is set'); + // The SDK logs a failed network-request at the warn level, and the sample handles it in warn(), so + // reportNetworkFailure receives the failed publish request. + await eventually( + () => assert.ok(reported.failures.length > 0, 'reportNetworkFailure received a failed request'), + { label: 'failed request entry' }, + ); + assert.ok(reported.failures[0].method, 'method is set'); + assert.match(String(reported.failures[0].url), /\/publish\//, 'the url names the publish path'); + assert.ok(reported.failures[0].reason, 'reason (details) is set'); + assert.ok(reported.failures[0].instanceId, 'instanceId (pubNubId) is set'); + pubnub.stop(); + assert.ok(reported.deprecations.length > 0, 'a deprecation warning reaches trackDeprecation'); + assert.match(String(reported.deprecations[0].warning), /deprecated/); + } finally { + quiet.restore(); + } + }); + + it('loggingMultipleLoggers: the SDK calls every logger in the loggers array', async () => { + class MonitoringLogger implements Logger { + trace() {} + debug() {} + info() {} + warn() {} + error() {} + } + class AnalyticsLogger implements Logger { + trace() {} + debug() {} + info() {} + warn() {} + error() {} + } + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'pn-logging-')); + const file = path.join(dir, 'multi.log'); + class FileLogger implements Logger { + private readonly stream = fs.createWriteStream(file, { flags: 'a' }); + trace(m: LogMessage) { + this.stream.write(`${m.level}\n`); + } + debug(m: LogMessage) { + this.stream.write(`${m.level}\n`); + } + info(m: LogMessage) { + this.stream.write(`${m.level}\n`); + } + warn(m: LogMessage) { + this.stream.write(`${m.level}\n`); + } + error(m: LogMessage) { + this.stream.write(`${m.level}\n`); + } + } + const quiet = quietConsole(); + try { + /* eslint-disable prettier/prettier */ + // snippet.loggingMultipleLoggers + const pubnub = new PubNub({ + subscribeKey: 'mySubscribeKey', + publishKey: 'myPublishKey', + userId: 'myUniqueUserId', + logLevel: PubNub.LogLevel.Debug, + // snippet.hide + ...openKeys(), + // snippet.show + loggers: [ + new MonitoringLogger(), // Send errors to monitoring service + new AnalyticsLogger(), // Track usage patterns + new FileLogger() // Write detailed logs to file (Node.js) + ] + }); + // snippet.end + /* eslint-enable prettier/prettier */ + track(pubnub); + // Each custom logger is called next to the console logger: the file logger saw the initialization entries. + await eventually(() => assert.ok(fs.readFileSync(file, 'utf8').length > 0), { label: 'file logger wrote' }); + } finally { + quiet.restore(); + fs.rmSync(dir, { recursive: true, force: true }); + } + }); + + // ---- the prose and the LogMessage reference ------------------------------------------------------------------ + + it('loggingLevels: each log level records only the entry levels it should, and the default level is None', async function () { + this.timeout(60_000); + // The same traffic at every level. A level records its own entries and those of the less verbose levels listed after it. + const levels: Array<[string, PubNub.LogLevel, string[]]> = [ + ['trace', LogLevel.Trace, ['trace', 'debug', 'info', 'warn', 'error']], + ['debug', LogLevel.Debug, ['debug', 'info', 'warn', 'error']], + ['info', LogLevel.Info, ['info', 'warn', 'error']], + ['warn', LogLevel.Warn, ['warn', 'error']], + ['error', LogLevel.Error, ['error']], + ['none', LogLevel.None, []], + ]; + const seenByLevel = new Map>(); + for (const [name, logLevel, allowed] of levels) { + const { logger, seen } = recorder(); + const quiet = quietConsole(); + try { + const pubnub = createClient('open', `logging-${name}`, { logLevel, loggers: [logger] }); + await pubnub.time(); + await assert.rejects( + pubnub.publish({ channel: '', message: 'x' } as unknown as PubNub.Publish.PublishParameters), + ); + pubnub.stop(); // warn: deprecated + } finally { + quiet.restore(); + } + const levelsSeen = new Set(seen.map((s) => s.level)); + for (const level of levelsSeen) + assert.ok(allowed.includes(level), `logLevel ${name} must not record ${level} entries`); + for (const level of ['error', 'warn', 'info', 'debug'].filter((l) => allowed.includes(l))) { + assert.ok(levelsSeen.has(level), `logLevel ${name} records ${level} entries (saw ${[...levelsSeen]})`); + } + seenByLevel.set(name, levelsSeen); + } + assert.equal(seenByLevel.get('none')!.size, 0, 'LogLevel.None records nothing'); + // Default: logging is disabled. + const { logger, seen } = recorder(); + createClient('open', 'logging-default', { loggers: [logger] }); + assert.equal(seen.length, 0, 'the default logLevel is None'); + }); + + it('loggingConsoleLogger: the console logger writes each level to the matching console method, with Trace going to console.debug', async () => { + // The built-in console logger runs next to custom loggers and prints ISO timestamp, PubNub-, level, location, message. + const lines: Array<{ method: string; text: string }> = []; + const captures = (['log', 'info', 'debug', 'warn', 'error'] as const).map((method) => { + const original = console[method]; + console[method] = (...args: unknown[]) => lines.push({ method, text: args.map(String).join(' ') }); + return () => { + console[method] = original; + }; + }); + try { + const pubnub = createClient('open', 'logging-console', { logLevel: LogLevel.Trace }); + await pubnub.time(); + pubnub.stop(); + } finally { + captures.forEach((restore) => restore()); + } + assert.ok(lines.length > 0, 'the console logger printed'); + const format = /^\d{4}-\d{2}-\d{2}T[\d:.]+Z PubNub-[0-9a-f]+ (Trace|Debug|Info|Warn|Error)\s+\S+ /; + assert.ok( + lines.some((l) => format.test(l.text)), + `an entry matches the documented format. First: ${lines[0]?.text.slice(0, 120)}`, + ); + assert.ok( + lines.filter((l) => / Trace /.test(l.text)).every((l) => l.method === 'debug'), + 'Trace entries are written with console.debug', + ); + }); + + it('loggingMessageTypes: every log message type has the shape the page reference lists', async function () { + this.timeout(60_000); + const { logger, seen } = recorder(); + const quiet = quietConsole(); + try { + const pubnub = createClient('open', 'logging-types', { logLevel: LogLevel.Trace, loggers: [logger] }); + await pubnub.time(); + const down = createClient('open', 'logging-types-down', { + ...unreachable, + logLevel: LogLevel.Trace, + loggers: [logger], + }); + await assert.rejects(down.publish({ channel: 'x', message: 'm' })); + } finally { + quiet.restore(); + } + const byType = (type: string) => seen.map((s) => s.message).filter((m) => m.messageType === type); + // Every entry carries the BaseLogMessage fields. + for (const { message, level } of seen) { + assert.ok(message.timestamp instanceof Date, 'timestamp is a Date'); + assert.match(message.pubNubId, /^[0-9a-f]+$/); + assert.equal(typeof message.level, 'number'); + assert.equal(message.minimumLevel, LogLevel.Trace); + assert.ok(['text', 'object', 'error', 'network-request', 'network-response'].includes(message.messageType)); + assert.ok(level); + } + const text = byType('text')[0] as Extract; + assert.equal(typeof text.message, 'string'); + const object = byType('object')[0] as Extract; + assert.ok(object.message !== undefined); + const error = byType('error')[0] as Extract; + assert.ok(error, 'an error entry from the failed publish'); + assert.ok(error.message.name && error.message.message, 'error entry has name and message'); + assert.ok(error.message.stack, 'error entry has a stack'); + assert.equal(error.message.status?.category, 'PNNetworkIssuesCategory', 'error entry has status.category'); + const failed = byType('network-request').find((m) => (m as { failed?: boolean }).failed) as + | Extract + | undefined; + assert.ok(failed, 'a network-request entry with failed: true for the unreachable origin'); + assert.ok(!failed.canceled); + const request = byType('network-request').find((m) => + /pndsn\.com/.test((m as { message: { origin?: string } }).message.origin ?? ''), + ) as Extract; + assert.ok(request, 'a network-request entry from time()'); + assert.equal(request.message.method, 'GET'); + assert.match(request.message.origin ?? '', /pndsn\.com/); + assert.match(request.message.path, /\/time\//); + assert.ok(request.message.queryParameters, 'queryParameters'); + const response = byType('network-response')[0] as Extract; + assert.ok(response, 'a network-response entry from time()'); + assert.equal(response.message.status, 200); + assert.match(response.message.url, /\/time\//); + assert.ok(response.message.headers, 'headers'); + assert.ok(response.message.body instanceof ArrayBuffer, 'the body is an ArrayBuffer, as the page shows'); + // Node.js: the details field carries a protocol hint. + assert.match( + String(response.details ?? ''), + /Negotiated protocol: \S+/, + 'network-response details carry the negotiated protocol on Node.js', + ); + }); + + it('loggingValidationErrorEntryShape: a logged validation failure is a plain status object and not a PubNubError', async () => { + const { logger, seen } = recorder(); + const pubnub = createClient('open', 'logging-validation', { logLevel: LogLevel.Error, loggers: [logger] }); + const quiet = quietConsole(); + try { + await assert.rejects( + pubnub.publish({ channel: '', message: 'x' } as unknown as PubNub.Publish.PublishParameters), + ); + } finally { + quiet.restore(); + } + const entry = seen.find((s) => s.message.messageType === 'error')!.message as Extract< + LogMessage, + { messageType: 'error' } + >; + const message = entry.message as unknown as Record; + assert.equal(message.message, "Missing 'channel'"); + assert.equal(message.category, 'PNValidationErrorCategory'); + assert.equal(message.name, undefined, 'no name on the logged validation failure'); + assert.equal(message.stack, undefined, 'no stack on the logged validation failure'); + assert.equal(message.status, undefined, 'category sits on the message itself, not under status'); + }); + + // The sample reads failed and canceled requests in error(). Record the level at which the SDK logs them. + it('loggingFailedRequestLevel: a failed request is logged at the warn level', async () => { + const { logger, seen } = recorder(); + const quiet = quietConsole(); + try { + const down = createClient('open', 'logging-failed', { + ...unreachable, + logLevel: LogLevel.Trace, + loggers: [logger], + }); + await assert.rejects(down.publish({ channel: 'x', message: 'm' })); + } finally { + quiet.restore(); + } + const failed = seen.filter( + (s) => s.message.messageType === 'network-request' && (s.message as { failed?: boolean }).failed, + ); + assert.equal(failed.length, 1); + assert.equal(failed[0].level, 'warn', 'a failed request is logged at the warn level, not error'); + assert.equal((failed[0].message as { details?: string }).details, 'Network error'); + assert.ok( + !seen.some((s) => s.level === 'error' && s.message.messageType === 'network-request'), + 'error() receives no network-request entries', + ); + }); +}); diff --git a/test/integration/live/getting-started/quickstart-node.mjs b/test/integration/live/getting-started/quickstart-node.mjs new file mode 100644 index 000000000..c505e731b --- /dev/null +++ b/test/integration/live/getting-started/quickstart-node.mjs @@ -0,0 +1,44 @@ +import PubNub from 'pubnub'; + +const pubnub = new PubNub({ + publishKey: 'YOUR_PUBLISH_KEY', + subscribeKey: 'YOUR_SUBSCRIBE_KEY', + userId: 'YOUR_USER_ID', +}); + +pubnub.addListener({ + message: (event) => { + console.log(`Received on ${event.channel}: ${event.message}`); + }, + status: (event) => { + if (event.category === 'PNConnectedCategory') { + console.log('Connected to PubNub'); + } + }, +}); + +// Create a channel entity +const channel = pubnub.channel('hello_world'); + +// Create a subscription +const subscription = channel.subscription({ + receivePresenceEvents: true, // to receive presence events +}); + +// Subscribe +subscription.subscribe(); + +pubnub.addListener({ + status: async (event) => { + if (event.category !== 'PNConnectedCategory') return; + try { + const result = await pubnub.publish({ + channel: 'hello_world', + message: 'Hello, world!', + }); + console.log(`Published, timetoken: ${result.timetoken}`); + } catch (error) { + console.error(`Publish failed: ${error}`); + } + }, +}); diff --git a/test/integration/live/getting-started/quickstart-node.test.ts b/test/integration/live/getting-started/quickstart-node.test.ts new file mode 100644 index 000000000..76616a7ff --- /dev/null +++ b/test/integration/live/getting-started/quickstart-node.test.ts @@ -0,0 +1,171 @@ +import assert from 'node:assert/strict'; +import { spawn } from 'node:child_process'; +import fs from 'node:fs'; +import path from 'node:path'; +import PubNub from 'pubnub'; +import { observer, track } from '../support/clients'; +import { captureConsole } from '../support/console'; +import { keysOf, openKeys } from '../support/env'; +import { makeScratch, SDK_ROOT } from '../support/tsc-scratch'; +import { eventually, watch } from '../support/wait'; + +/** + * Node.js quickstart examples. Port of docs-snippets/getting-started/quickstart-node.ts. + * The docs embed quickstart-node.mjs as a whole file, and the test runs that file in a child process. It sits next to this + * test and is a byte for byte copy of the file of the docs. + */ + +describe('getting-started/quickstart-node', () => { + it('gettingStartedInitPubnub, quickstartNodeListeners, gettingStartedCreateSubscription, quickstartNodePublish: the Node quickstart script connects, publishes and receives, and prints the lines the page promises', async () => { + const logs = captureConsole('log'); + try { + // snippet.gettingStartedInitPubnub + const pubnub = new PubNub({ + publishKey: 'YOUR_PUBLISH_KEY', + subscribeKey: 'YOUR_SUBSCRIBE_KEY', + userId: 'YOUR_USER_ID', + // snippet.hide + ...openKeys(), + // snippet.show + }); + // snippet.end + track(pubnub); + + // snippet.quickstartNodeListeners + pubnub.addListener({ + message: (event) => { + console.log(`Received on ${event.channel}: ${event.message}`); + }, + status: (event) => { + if (event.category === 'PNConnectedCategory') { + console.log('Connected to PubNub'); + } + }, + }); + // snippet.end + + // snippet.gettingStartedCreateSubscription + // Create a channel entity + const channel = pubnub.channel('hello_world'); + + // Create a subscription + const subscription = channel.subscription({ + receivePresenceEvents: true, // to receive presence events + }); + + // Subscribe + subscription.subscribe(); + // snippet.end + + // snippet.quickstartNodePublish + pubnub.addListener({ + status: async (event) => { + if (event.category !== 'PNConnectedCategory') return; + try { + const result = await pubnub.publish({ + channel: 'hello_world', + message: 'Hello, world!', + }); + console.log(`Published, timetoken: ${result.timetoken}`); + } catch (error) { + console.error(`Publish failed: ${error}`); + } + }, + }); + // snippet.end + + // The channel is shared, so bind the check to this run's own message: the timetoken in the "Published" line. This + // listener is added after the sample's, so by the time it sees the message the sample has printed its "Received" line. + const receivedTimetokens: string[] = []; + pubnub.addListener({ message: (event) => receivedTimetokens.push(event.timetoken) }); + + assert.equal(pubnub.getUserId(), 'YOUR_USER_ID'); + const published = await eventually( + () => { + const line = logs.lines.find((l) => l.startsWith('Published, timetoken: ')) ?? ''; + assert.match(line, /^Published, timetoken: \d{17}$/); + return line.slice('Published, timetoken: '.length); + }, + { label: 'published line printed' }, + ); + await eventually(() => assert.ok(receivedTimetokens.includes(published), 'own message received'), { + label: 'own message received', + }); + assert.ok(logs.lines.includes('Received on hello_world: Hello, world!'), `Received line. Saw: ${logs.lines}`); + assert.ok(logs.lines.includes('Connected to PubNub'), `Connected line. Saw: ${logs.lines}`); + subscription.unsubscribe(); + } finally { + logs.restore(); + } + }); + + it('quickstartNodeRun: the complete quickstart file runs in a Node process, prints Connected and Published, receives its own message, and receives a message from another client', async function () { + this.timeout(60_000); + const open = keysOf('open'); + // The file holds the placeholder keys the readers see. A copy in a scratch directory gets the real keys, the way the old + // suite ran it. The scratch directory has node_modules/pubnub linked to this checkout so that `import 'pubnub'` resolves. + const source = fs + .readFileSync(path.join(__dirname, 'quickstart-node.mjs'), 'utf8') + .replace('YOUR_PUBLISH_KEY', open.publishKey) + .replace('YOUR_SUBSCRIBE_KEY', open.subscribeKey); + assert.ok(!source.includes('YOUR_PUBLISH_KEY') && !source.includes('YOUR_SUBSCRIBE_KEY'), 'both keys replaced'); + const dir = makeScratch('pn-live-quickstart-'); + const file = path.join(dir, 'index.mjs'); + fs.writeFileSync(file, source); + // Independent view of the channel: proves the service got the script's message. + const listening = await watch(observer(), ['hello_world']); + const child = spawn(process.execPath, [file], { stdio: ['ignore', 'pipe', 'pipe'], cwd: dir }); + let out = ''; + let err = ''; + child.stdout.on('data', (chunk) => (out += chunk)); + child.stderr.on('data', (chunk) => (err += chunk)); + try { + // The script publishes only after the connected status, so its own message reaches an active subscription. All three + // documented lines are asserted. The channel is shared, so the "Received" line cannot be bound to the timetoken + // (the script does not print it): the message published by the script is bound through the observer below. + const published = await eventually( + () => { + const lines = out.split('\n'); + assert.ok(lines.includes('Connected to PubNub'), `Connected line. Output: ${out} ${err}`); + const line = lines.find((l) => /^Published, timetoken: \d{17}$/.test(l)); + assert.ok(line, `Published line. Output: ${out} ${err}`); + assert.ok(lines.includes('Received on hello_world: Hello, world!'), `Received line. Output: ${out} ${err}`); + return line.slice('Published, timetoken: '.length); + }, + { timeout: 30_000, label: 'quickstart script connected, published and received its own message' }, + ); + const seen = await listening.next('message', (m) => m.timetoken === published); + assert.equal(seen.message, 'Hello, world!'); + assert.equal(seen.publisher, 'YOUR_USER_ID'); + // The listener keeps working: a message from another client arrives too. The text is unique to this run. + const text = `Hello from the test ${Date.now().toString(36)}${Math.random().toString(36).slice(2, 6)}`; + await observer().publish({ channel: 'hello_world', message: text }); + await eventually( + () => + assert.ok( + out.split('\n').includes(`Received on hello_world: ${text}`), + `Received line. Output: ${out} ${err}`, + ), + { + timeout: 30_000, + label: 'quickstart script receives a message from another client', + }, + ); + // "The script keeps running because the subscription stays open." + assert.equal(child.exitCode, null, 'the script keeps running'); + } finally { + const exited = new Promise((resolve) => child.once('exit', () => resolve())); + child.kill('SIGKILL'); + await exited; + fs.rmSync(dir, { recursive: true, force: true }); + listening.stop(); + } + }); + + // The "Before you begin" claim: Node.js 22 or later. The SDK declares the same minimum. + it('quickstartNodeVersion: the SDK declares the same minimum Node.js version as the quickstart states', () => { + const engines = JSON.parse(fs.readFileSync(path.join(SDK_ROOT, 'package.json'), 'utf8')).engines; + assert.equal(engines.node, '>=22.0.0'); + assert.ok(Number(process.versions.node.split('.')[0]) >= 22, 'the app itself runs on Node.js 22 or later'); + }); +}); diff --git a/test/integration/live/getting-started/set-up-your-environment.test.ts b/test/integration/live/getting-started/set-up-your-environment.test.ts new file mode 100644 index 000000000..a9c16b12a --- /dev/null +++ b/test/integration/live/getting-started/set-up-your-environment.test.ts @@ -0,0 +1,174 @@ +import assert from 'node:assert/strict'; +import { execFileSync } from 'node:child_process'; +import fs from 'node:fs'; +import path from 'node:path'; +import ts from 'typescript'; +// snippet.setUpTypeScriptImport +import PubNub from 'pubnub'; + +// snippet.end +import { assertServerTime } from '../support/assertions'; +import { track } from '../support/clients'; +import { openKeys } from '../support/env'; + +import { makeScratch, SDK_ROOT, tscScratch } from '../support/tsc-scratch'; + +/** + * Examples for setting up your environment. Port of docs-snippets/getting-started/set-up-your-environment.ts. + * The import snippet sits at the top because an import cannot sit inside a test. It is type checked by the tier's tsconfig and + * loaded when the file runs. + */ + +describe('getting-started/set-up-your-environment', () => { + it('setUpTypeScriptImport, setUpTypeScriptInit: the TypeScript init sample builds a working client and time() succeeds', async () => { + assert.equal(typeof PubNub, 'function'); + /* eslint-disable prettier/prettier -- the sample text is shown to readers as written */ + // snippet.setUpTypeScriptInit + const pubnub = new PubNub({ + publishKey: 'your-publish-key', + subscribeKey: 'your-subscribe-key', + // snippet.hide + ...openKeys(), + // snippet.show + userId: 'your-user-id' + }); + // snippet.end + /* eslint-enable prettier/prettier */ + track(pubnub); + assert.equal(pubnub.getUserId(), 'your-user-id'); + assertServerTime((await pubnub.time()).timetoken); + }); + + it('setUpTypeScriptProject: the TypeScript sample file and the documented tsconfig.json compile under strict settings', function () { + this.timeout(90_000); + // "you do not need an additional @types package": tsc resolves the types that the pubnub package ships. + const result = tscScratch({ + 'tsconfig.json': JSON.stringify( + { + compilerOptions: { + target: 'ES2020', + module: 'ESNext', + moduleResolution: 'bundler', + strict: true, + esModuleInterop: true, + skipLibCheck: true, + forceConsistentCasingInFileNames: true, + }, + include: ['index.ts'], + }, + null, + 2, + ), + 'index.ts': `import PubNub from 'pubnub'; + +const pubnub = new PubNub({ + publishKey: 'your-publish-key', + subscribeKey: 'your-subscribe-key', + userId: 'your-user-id' +}); +`, + }); + assert.ok(result.ok, `the documented project compiles.\n${result.output}`); + }); + + it('setUpModuleFormats: each documented module format loads the SDK in a real Node process', function () { + this.timeout(60_000); + const dir = makeScratch('pn-live-modules-'); + const run = (args: string[]): string => + execFileSync(process.execPath, args, { cwd: dir, encoding: 'utf8', stdio: 'pipe' }).trim(); + const probe = + "typeof PubNub === 'function' ? new PubNub({ subscribeKey: 'k', userId: 'u' }).getUserId() : 'no constructor'"; + try { + // CommonJS: const PubNub = require('pubnub'); + fs.writeFileSync(path.join(dir, 'cjs.cjs'), `const PubNub = require('pubnub');\nconsole.log(${probe});\n`); + assert.equal(run([path.join(dir, 'cjs.cjs')]), 'u', 'require("pubnub") returns the constructor'); + // ES modules through the .mjs extension. + fs.writeFileSync(path.join(dir, 'esm.mjs'), `import PubNub from 'pubnub';\nconsole.log(${probe});\n`); + assert.equal(run([path.join(dir, 'esm.mjs')]), 'u', '.mjs files use import'); + // "type": "module" in package.json makes .js an ES module. + fs.writeFileSync(path.join(dir, 'package.json'), JSON.stringify({ type: 'module' })); + fs.writeFileSync(path.join(dir, 'esm.js'), `import PubNub from 'pubnub';\nconsole.log(${probe});\n`); + assert.equal(run([path.join(dir, 'esm.js')]), 'u', '"type": "module" makes .js an ES module'); + } finally { + fs.rmSync(dir, { recursive: true, force: true }); + } + }); + + // Two-way drift guard against the SDK checkout: every documented variable and default must be in rollup.config.js, and every + // `*_MODULE` variable the SDK source reads must be documented, so adding a module to the SDK without the table fails here. + // Build a custom bundle: the module table matches rollup.config.js, names and defaults. + it('setUpBundleModuleTable: the custom bundle module table matches the SDK rollup configuration in both directions', () => { + const documented: Record = { + CRYPTO_MODULE: 'enabled', + SHARED_WORKER: 'enabled', + PUBLISH_MODULE: 'enabled', + SUBSCRIBE_MODULE: 'enabled', + SUBSCRIBE_EVENT_ENGINE_MODULE: 'enabled', + SUBSCRIBE_MANAGER_MODULE: 'enabled', + PRESENCE_MODULE: 'enabled', + PAM_MODULE: 'disabled', + CHANNEL_GROUPS_MODULE: 'enabled', + MESSAGE_PERSISTENCE_MODULE: 'enabled', + MOBILE_PUSH_MODULE: 'enabled', + APP_CONTEXT_MODULE: 'enabled', + DATA_SYNC_MODULE: 'enabled', + FILE_SHARING_MODULE: 'enabled', + MESSAGE_REACTIONS_MODULE: 'enabled', + }; + const rollup = fs.readFileSync(path.join(SDK_ROOT, 'rollup.config.js'), 'utf8'); + const actual: Record = {}; + for (const match of rollup.matchAll( + /'process\.env\.([A-Z_]+)':\s*JSON\.stringify\(process\.env\.\1 \?\? '(enabled|disabled)'\)/g, + )) + actual[match[1]] = match[2]; + assert.deepEqual(actual, documented, 'documented build variables and defaults equal rollup.config.js'); + const used = new Set(); + const walk = (dir: string): void => { + for (const entry of fs.readdirSync(dir, { withFileTypes: true })) { + const full = path.join(dir, entry.name); + if (entry.isDirectory()) walk(full); + else if (entry.name.endsWith('.ts')) + for (const m of fs.readFileSync(full, 'utf8').matchAll(/process\.env\.([A-Z_]+_MODULE|SHARED_WORKER)\b/g)) + used.add(m[1]); + } + }; + walk(path.join(SDK_ROOT, 'src')); + assert.deepEqual( + [...used].sort(), + Object.keys(documented).sort(), + 'every build variable that the source reads is documented', + ); + // "Set enableTreeShaking to true": the constant exists. + assert.match(rollup, /const enableTreeShaking = (false|true);/); + }); + + it('setUpWebpackConfigSyntax: the documented Webpack configuration is valid JavaScript and defines the variables of the table', () => { + const source = `const webpack = require('webpack'); + +module.exports = { + // Other configuration... + plugins: [ + new webpack.DefinePlugin({ + 'process.env.PRESENCE_MODULE': JSON.stringify(process.env.PRESENCE_MODULE ?? 'enabled'), + 'process.env.PUBLISH_MODULE': JSON.stringify(process.env.PUBLISH_MODULE ?? 'enabled'), + // One entry for every variable in the table. + }), + ], +}; +`; + const result = ts.transpileModule(source, { + reportDiagnostics: true, + compilerOptions: { module: ts.ModuleKind.CommonJS, target: ts.ScriptTarget.ES2020 }, + }); + assert.deepEqual(result.diagnostics ?? [], [], 'the Webpack configuration parses'); + }); + + it('setUpCdnUrls: the pinned CDN URLs for the linked SDK version exist', async function () { + this.timeout(30_000); + const version = JSON.parse(fs.readFileSync(path.join(SDK_ROOT, 'package.json'), 'utf8')).version as string; + for (const file of [`pubnub.${version}.js`, `pubnub.worker.${version}.js`]) { + const response = await fetch(`https://cdn.pubnub.com/sdk/javascript/${file}`, { method: 'HEAD' }); + assert.equal(response.status, 200, `${file} exists on the CDN`); + } + }); +}); diff --git a/test/integration/live/getting-started/status-events.test.ts b/test/integration/live/getting-started/status-events.test.ts new file mode 100644 index 000000000..2295fa8e6 --- /dev/null +++ b/test/integration/live/getting-started/status-events.test.ts @@ -0,0 +1,443 @@ +import assert from 'node:assert/strict'; +import http from 'node:http'; +import type { AddressInfo } from 'node:net'; +import PubNub from 'pubnub'; +import { createClient, friend, shopper, track } from '../support/clients'; +import { captureConsole } from '../support/console'; +import { keysOf } from '../support/env'; +import { LocalProxy } from '../support/proxy'; +import { connected, eventually, expectEvent, sleep } from '../support/wait'; + +/** + * Status event examples. Port of docs-snippets/getting-started/status-events.ts. + * Outages are simulated with a local CONNECT proxy. The fixed sleeps prove that a call is still pending or that nothing arrived. + */ +describe('getting-started/status-events', () => { + const proxy = new LocalProxy(); + // Channels are shared and persistent. The names below are unique per run and are generated outside the snippets. + const runId = Date.now().toString(36) + Math.random().toString(36).slice(2, 6); + const uniqueChannel = (name: string): string => `${name}-${runId}`; + const fresh = (label: string, extra: Partial = {}): PubNub => + createClient('open', label, extra); + + /** Runs `fn` with console.log captured and restores it afterwards. */ + async function withConsole(fn: (logs: ReturnType) => Promise): Promise { + const logs = captureConsole('log'); + try { + return await fn(logs); + } finally { + logs.restore(); + } + } + + after(async () => { + proxy.outage(false); + await proxy.stop(); + }); + + const nextStatus = ( + client: PubNub, + categories: string[], + timeout = 20_000, + ): Promise => + new Promise((resolve, reject) => { + const listener: PubNub.Listener = { + status: (status) => { + if (!categories.includes(String(status.category))) return; + clearTimeout(timer); + setImmediate(() => client.removeListener(listener)); + resolve(status); + }, + }; + const timer = setTimeout(() => { + client.removeListener(listener); + reject(new Error(`no status in [${categories.join(', ')}] within ${timeout}ms`)); + }, timeout); + client.addListener(listener); + }); + + const lines = (logs: string[], prefix: string): string[] => logs.filter((line) => line.startsWith(prefix)); + + // ---- before and after: PNConnectedCategory on every change versus PNSubscriptionChangedCategory --------------- + + it('statusEventsMigrationBefore: the legacy subscription manager reports PNConnectedCategory on every change of the channel mix.', async () => { + const pubnub = fresh('status-before', { enableEventEngine: false }); + const [c1, c2] = [uniqueChannel('ch1'), uniqueChannel('ch2')]; + await withConsole(async (logs) => { + /* eslint-disable prettier/prettier */ + // snippet.statusEventsMigrationBefore + pubnub.addListener({ + status: (statusEvent) => { + if (statusEvent.category === 'PNConnectedCategory') { + // This ran on first connect AND every time channels/groups were added or removed + console.log('Subscription updated:', statusEvent.affectedChannels, statusEvent.affectedChannelGroups); + } + } + }); + // snippet.end + /* eslint-enable prettier/prettier */ + + pubnub.subscribe({ channels: [c1] }); + await eventually(() => assert.equal(lines(logs.lines, 'Subscription updated:').length, 1), { + label: 'first connect logged', + }); + pubnub.subscribe({ channels: [c2] }); + await eventually(() => assert.equal(lines(logs.lines, 'Subscription updated:').length, 2), { + label: 'channel change logged as PNConnectedCategory again', + }); + assert.match(lines(logs.lines, 'Subscription updated:')[1], new RegExp(c2)); + pubnub.unsubscribeAll(); + }); + }); + + // After: the same two subscribes report PNConnectedCategory once and PNSubscriptionChangedCategory for the change. The final + // `sleep` is a negative wait, it gives a wrongly repeated PNConnectedCategory time to arrive before asserting it did not. + it('statusEventsMigrationAfter: the new subscribe loop reports PNConnectedCategory once and PNSubscriptionChangedCategory for later changes.', async () => { + const pubnub = fresh('status-after'); + const [c1, c2] = [uniqueChannel('ch1'), uniqueChannel('ch2')]; + await withConsole(async (logs) => { + /* eslint-disable prettier/prettier */ + // snippet.statusEventsMigrationAfter + pubnub.addListener({ + status: (statusEvent) => { + if (statusEvent.category === 'PNConnectedCategory') { + console.log('Subscribed and ready'); + } + if (statusEvent.category === 'PNSubscriptionChangedCategory') { + console.log('Subscription updated:', statusEvent.affectedChannels, statusEvent.affectedChannelGroups); + } + } + }); + // snippet.end + /* eslint-enable prettier/prettier */ + + const subscription1 = pubnub.channel(c1).subscription(); + subscription1.subscribe(); + await eventually(() => assert.equal(lines(logs.lines, 'Subscribed and ready').length, 1), { + label: 'connected logged', + }); + const subscription2 = pubnub.channel(c2).subscription(); + subscription2.subscribe(); + await eventually(() => assert.equal(lines(logs.lines, 'Subscription updated:').length, 1), { + label: 'change logged as PNSubscriptionChangedCategory', + }); + // The change reports every channel of the new mix, and PNConnectedCategory does not fire again. + assert.match(lines(logs.lines, 'Subscription updated:')[0], new RegExp(c1)); + assert.match(lines(logs.lines, 'Subscription updated:')[0], new RegExp(c2)); + await sleep(1000); + assert.equal( + lines(logs.lines, 'Subscribed and ready').length, + 1, + 'PNConnectedCategory fires only when the subscription starts', + ); + pubnub.unsubscribeAll(); + }); + }); + + // ---- request statuses against a scripted origin ---------------------------------------------------------------- + + it('statusEventsRequestCategories: each documented status category is produced by the matching HTTP response from a local server.', async function () { + this.timeout(60_000); + let respond: (res: http.ServerResponse) => void = (res) => res.end('[]'); + const server = http.createServer((_req, res) => respond(res)); + await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve)); + const origin = `127.0.0.1:${(server.address() as AddressInfo).port}`; + try { + const client = (extra: Partial = {}): PubNub => + fresh('status-origin', { + origin, + ssl: false, + keepAlive: false, + retryConfiguration: PubNub.NoneRetryPolicy(), + ...extra, + }); + const categoryOf = async (pubnub: PubNub): Promise<{ category: string; statusCode?: number }> => { + try { + await pubnub.time(); + } catch (error) { + const status = (error as { status?: { category?: string; statusCode?: number } }).status; + return { category: String(status?.category), statusCode: status?.statusCode }; + } + return { category: 'none (resolved)' }; + }; + const reply = (code: number, body: string) => (res: http.ServerResponse) => { + res.writeHead(code, { 'content-type': 'application/json' }).end(body); + }; + + respond = reply(400, '{"error":true,"message":"bad"}'); + assert.equal((await categoryOf(client())).category, 'PNBadRequestCategory', '400'); + respond = reply(403, '{"error":true,"message":"Forbidden"}'); + assert.equal((await categoryOf(client())).category, 'PNAccessDeniedCategory', '403'); + respond = reply(500, '{"error":true}'); + assert.equal((await categoryOf(client())).category, 'PNServerErrorCategory', '500'); + respond = reply(503, '{"error":true}'); + assert.equal((await categoryOf(client())).category, 'PNServerErrorCategory', '503'); + respond = reply(418, '{"error":true}'); + assert.equal((await categoryOf(client())).category, 'PNUnknownCategory', 'unhandled status code'); + respond = reply(200, 'this is not json'); + assert.equal((await categoryOf(client())).category, 'PNUnknownCategory', '200 with a body that is not JSON'); + respond = (res) => res.writeHead(200, { 'content-type': 'text/html' }).end('proxy login'); + assert.equal((await categoryOf(client())).category, 'PNUnknownCategory', '200 with an HTML page'); + // A server that never answers: PNTimeoutCategory after transactionalRequestTimeout seconds. + respond = () => undefined; + const timedOut = client({ transactionalRequestTimeout: 1 }); + await assert.rejects(timedOut.publish({ channel: 'x', message: 'm' }), (error: unknown) => { + assert.equal( + String((error as { status?: { category?: string } }).status?.category), + 'PNTimeoutCategory', + 'no answer within the timeout', + ); + return true; + }); + timedOut.destroy(); + // Nothing listens on the origin any more: PNNetworkIssuesCategory. + server.closeAllConnections(); + await new Promise((resolve) => server.close(() => resolve())); + // publish() is excluded from the default retry policy, so it rejects at once. time() would keep retrying. + await assert.rejects(client().publish({ channel: 'x', message: 'm' }), (error: unknown) => { + assert.equal( + String((error as { status?: { category?: string } }).status?.category), + 'PNNetworkIssuesCategory', + 'connection refused', + ); + return true; + }); + } finally { + if (server.listening) { + server.closeAllConnections(); + server.close(); + } + } + + const withoutPublishKey = track(new PubNub({ subscribeKey: keysOf('open').subscribeKey, userId: 'status-nokey' })); + for (const call of [ + () => withoutPublishKey.publish({ channel: 'x', message: 'm' }), + () => shopper().publish({ message: 'm' } as unknown as PubNub.Publish.PublishParameters), + () => shopper().publish({ channel: 'x' } as unknown as PubNub.Publish.PublishParameters), + ]) { + await assert.rejects(call(), (error: unknown) => { + assert.equal( + String((error as { status?: { category?: string } }).status?.category), + 'PNValidationErrorCategory', + ); + return true; + }); + } + }); + + it('statusEventsDefaultRetryAlsoRetriesTime: the default retry policy also retries time(), so it stays pending against a server that answers 500.', async function () { + this.timeout(30_000); + let hits = 0; + const server = http.createServer((_req, res) => { + hits += 1; + res.writeHead(500, { 'content-type': 'application/json' }).end('{"error":true}'); + }); + await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve)); + const origin = `127.0.0.1:${(server.address() as AddressInfo).port}`; + try { + const pubnub = fresh('status-default-retry', { origin, ssl: false, keepAlive: false }); + const settled = pubnub.time().then( + () => 'settled', + () => 'settled', + ); + assert.equal( + await Promise.race([settled, sleep(4500).then(() => 'pending')]), + 'pending', + 'time() is still retrying after 4.5 seconds', + ); + assert.ok(hits >= 2, `time() was retried (${hits} requests reached the server)`); + // publish() is excluded from the default policy: one request, one failure. + const before = hits; + await assert.rejects(pubnub.publish({ channel: 'x', message: 'm' })); + assert.equal(hits - before, 1, 'publish() is not retried'); + pubnub.destroy(); + } finally { + server.closeAllConnections(); + server.close(); + } + }); + + // ---- subscription statuses under an outage --------------------------------------------------------------------- + + const proxiedSubscriber = async (label: string, extra: Partial) => { + await proxy.start(); + proxy.outage(false); + const pubnub = fresh(label, extra); + pubnub.setProxy(proxy.config); + const channel = uniqueChannel('outage_channel'); + const ready = connected(pubnub, { channels: [channel] }); + const subscription = pubnub.channel(channel).subscription(); + subscription.subscribe(); + await ready; + return { pubnub, channel, subscription }; + }; + + it('statusEventsUnexpectedDisconnectAndReconnect: with no retry policy a lost request reports PNDisconnectedUnexpectedlyCategory and reconnect() restores delivery.', async function () { + this.timeout(90_000); + const { pubnub, channel, subscription } = await proxiedSubscriber('outage-none', { + retryConfiguration: PubNub.NoneRetryPolicy(), + }); + try { + const before = proxy.connects; + const lost = nextStatus(pubnub, ['PNDisconnectedUnexpectedlyCategory', 'PNConnectionErrorCategory']); + proxy.outage(true); + const status = await lost; + // With the new subscription loop and no retries the loss is reported as PNDisconnectedUnexpectedlyCategory. + assert.equal(status.category, 'PNDisconnectedUnexpectedlyCategory'); + assert.ok((status as { error?: unknown }).error, 'the status carries an error field'); + console.log(`unexpected disconnect status: ${JSON.stringify(status)}`); + assert.ok(proxy.connects - before <= 1, 'NoneRetryPolicy makes no retry attempts'); + + // The outage ends and the app reconnects. + proxy.outage(false); + const back = nextStatus(pubnub, ['PNConnectedCategory', 'PNReconnectedCategory']); + pubnub.reconnect(); + const reconnected = await back; + console.log(`reconnect status category: ${reconnected.category}`); + const received = expectEvent( + pubnub, + 'message', + (m) => (m.message as { outage?: string }).outage === 'after' && m.channel === channel, + ); + await friend().publish({ channel, message: { outage: 'after' } }); + await received; + } finally { + proxy.outage(false); + subscription.unsubscribe(); + } + }); + + // The outage starts before the first subscribe, so the very first connect fails: PNConnectionErrorCategory, not a disconnect. + it('statusEventsConnectionErrorOnInitialConnect: a failure on the first subscribe is PNConnectionErrorCategory and not a disconnect.', async function () { + this.timeout(60_000); + await proxy.start(); + proxy.outage(true); + const pubnub = fresh('outage-initial', { retryConfiguration: PubNub.NoneRetryPolicy() }); + pubnub.setProxy(proxy.config); + try { + const failed = nextStatus(pubnub, ['PNConnectionErrorCategory', 'PNDisconnectedUnexpectedlyCategory']); + pubnub.channel(uniqueChannel('outage_channel')).subscription().subscribe(); + const status = await failed; + // Failed to establish the initial connection. + assert.equal(status.category, 'PNConnectionErrorCategory'); + assert.ok((status as { error?: unknown }).error, 'the status carries an error field'); + console.log(`connection error status: ${JSON.stringify(status)}`); + } finally { + proxy.outage(false); + pubnub.unsubscribeAll(); + } + }); + + // Measures, not mocks: counts CONNECT attempts the proxy sees while a policy retries during one outage. Bounds are loose on + // purpose (attempts at least 2, linear at least 3.5 s) because timing on a live network varies. The long timeout covers three + // subscribe and outage cycles. + it('statusEventsRetryPolicies: during an outage each retry policy makes repeated connect attempts, counted at the proxy, and the linear policy spaces them out.', async function () { + this.timeout(120_000); + // Count the tunnel attempts that a policy makes during one outage until the client gives up. + const attemptsDuring = async ( + label: string, + retryConfiguration: PubNub.RequestRetryPolicy, + ): Promise<{ attempts: number; ms: number }> => { + const { pubnub, subscription } = await proxiedSubscriber(label, { retryConfiguration }); + try { + const lost = nextStatus(pubnub, ['PNDisconnectedUnexpectedlyCategory', 'PNConnectionErrorCategory'], 60_000); + const before = proxy.connects; + const started = Date.now(); + proxy.outage(true); + await lost; + return { attempts: proxy.connects - before, ms: Date.now() - started }; + } finally { + proxy.outage(false); + subscription.unsubscribe(); + } + }; + + const none = await attemptsDuring('retry-none', PubNub.NoneRetryPolicy()); + const linear = await attemptsDuring('retry-linear', PubNub.LinearRetryPolicy({ delay: 2, maximumRetry: 2 })); + const exponential = await attemptsDuring( + 'retry-exponential', + PubNub.ExponentialRetryPolicy({ minimumDelay: 2, maximumDelay: 4, maximumRetry: 2 }), + ); + console.log( + `retry attempts during an outage: none ${none.attempts} (${none.ms} ms), linear ${linear.attempts} (${linear.ms} ms), exponential ${exponential.attempts} (${exponential.ms} ms)`, + ); + assert.ok(none.attempts <= 1, 'NoneRetryPolicy does not retry'); + assert.ok(linear.attempts >= 2, `LinearRetryPolicy({ maximumRetry: 2 }) retries (saw ${linear.attempts} attempts)`); + assert.ok(linear.ms >= 3500, `Linear waits delay seconds between attempts (took ${linear.ms} ms)`); + assert.ok( + exponential.attempts >= 2, + `ExponentialRetryPolicy({ maximumRetry: 2 }) retries (saw ${exponential.attempts} attempts)`, + ); + assert.ok(exponential.ms > none.ms, 'retries take time'); + }); + + // Default policy: a three second outage is retried inside the SDK. The client never reports a status for the in-between state and + // delivers a message published after the outage. + it('statusEventsDefaultRetryRecoversSilently: a short outage is retried inside the SDK with no status and later messages still arrive.', async function () { + this.timeout(90_000); + // Default policy (no retryConfiguration, enableEventEngine true): retries subscribe requests. A short outage ends without an unexpected-disconnect status. + const { pubnub, channel, subscription } = await proxiedSubscriber('outage-default', {}); + const categories: string[] = []; + pubnub.addListener({ status: (s) => categories.push(String(s.category)) }); + try { + const before = proxy.connects; + proxy.outage(true); + await sleep(3000); + assert.ok(proxy.connects > before, 'the default policy retried the subscribe request during the outage'); + proxy.outage(false); + const received = expectEvent( + pubnub, + 'message', + (m) => (m.message as { outage?: string }).outage === 'default' && m.channel === channel, + { timeout: 30_000 }, + ); + await friend().publish({ channel, message: { outage: 'default' } }); + await received; + assert.deepEqual(categories, [], `no status for intermediate states (saw ${categories.join(', ')})`); + } finally { + proxy.outage(false); + subscription.unsubscribe(); + } + }); + + // Legacy manager (`enableEventEngine: false`) reports an outage as PNNetworkIssuesCategory and the recovery as PNReconnectedCategory. + it('statusEventsLegacyManagerOutage: the legacy manager reports an outage as PNNetworkIssuesCategory and recovery as PNReconnectedCategory.', async function () { + this.timeout(90_000); + await proxy.start(); + proxy.outage(false); + const pubnub = fresh('outage-legacy', { enableEventEngine: false }); + pubnub.setProxy(proxy.config); + const channel = uniqueChannel('outage_channel'); + const seen: string[] = []; + pubnub.addListener({ status: (s) => seen.push(String(s.category)) }); + const ready = connected(pubnub, { channels: [channel] }); + pubnub.subscribe({ channels: [channel] }); + await ready; + try { + const issues = nextStatus(pubnub, ['PNNetworkIssuesCategory'], 30_000); + proxy.outage(true); + await issues; + const recovered = nextStatus(pubnub, ['PNReconnectedCategory'], 60_000); + proxy.outage(false); + await recovered; + console.log(`legacy manager statuses: ${seen.join(', ')}`); + } finally { + proxy.outage(false); + pubnub.unsubscribeAll(); + } + }); + + // Access Manager: a 403 on the subscribe loop is a PNConnectionErrorCategory with PNAccessDeniedCategory in the nested error. + it('statusEventsAccessDeniedOnSubscribe: a 403 on the subscribe loop is PNConnectionErrorCategory with PNAccessDeniedCategory nested.', async function () { + this.timeout(45_000); + const pubnub = createClient('pam', 'status-denied'); + const denied = nextStatus(pubnub, ['PNConnectionErrorCategory', 'PNAccessDeniedCategory']); + pubnub.channel(uniqueChannel('am_room')).subscription().subscribe(); + const status = await denied; + assert.equal(status.category, 'PNConnectionErrorCategory'); + console.log(`access denied status: ${JSON.stringify(status)}`); + // The nested error is the category name as a string. + assert.equal((status as { error?: unknown }).error, 'PNAccessDeniedCategory'); + pubnub.unsubscribeAll(); + }); +}); diff --git a/test/integration/live/getting-started/troubleshooting.test.ts b/test/integration/live/getting-started/troubleshooting.test.ts new file mode 100644 index 000000000..f7df7ba26 --- /dev/null +++ b/test/integration/live/getting-started/troubleshooting.test.ts @@ -0,0 +1,86 @@ +import assert from 'node:assert/strict'; +import PubNub from 'pubnub'; +import { rejectsValidation, rejectsWithCategory } from '../support/assertions'; +import { createClient, shopper, track } from '../support/clients'; +import { keysOf } from '../support/env'; + +/** + * Troubleshooting examples. Port of docs-snippets/getting-started/troubleshooting.ts. The page has no snippet markers. + * The access denied test needs a secured (pam) keyset on which the secret-less client has no token. + */ +describe('getting-started/troubleshooting', () => { + // Each row is one documented cause of "the constructor throws". The SDK throws a PubNubError synchronously, before any request. + it('troubleshootingConstructorThrows: each documented cause of a constructor failure throws a PubNubError before any request', () => { + const keys = { subscribeKey: keysOf('open').subscribeKey }; + // "userId is omitted, is empty or only whitespace, or is set together with uuid." + for (const [config, pattern] of [ + [{ ...keys }, /'userId' not set/], + [{ ...keys, userId: '' }, /'userId' not set/], + [{ ...keys, userId: ' ' }, /'userId' is empty/], + [{ ...keys, userId: 'a', uuid: 'b' }, /use only 'userId'/], + ] as const) { + assert.throws( + () => new PubNub(config as unknown as PubNub.PubNubConfiguration), + (error: unknown) => (error as Error).name === 'PubNubError' && pattern.test((error as Error).message), + ); + } + }); + + it('troubleshootingPublishValidation: a publish without a publish key fails local validation, and the category differs from what the page states', async () => { + const withoutPublishKey = track(new PubNub({ subscribeKey: keysOf('open').subscribeKey, userId: 'trouble-nokey' })); + await rejectsValidation(withoutPublishKey.publish({ channel: 'x', message: 'm' }), /Missing 'publishKey'/); + await rejectsValidation( + shopper().publish({ message: 'm' } as unknown as PubNub.Publish.PublishParameters), + /Missing 'channel'/, + ); + await rejectsValidation( + shopper().publish({ channel: 'x' } as unknown as PubNub.Publish.PublishParameters), + /Missing 'message'/, + ); + }); + + // "Publish fails with Message Too Large ... PubNub rejects the publish with HTTP 400." Record what the service does. + it('troubleshootingMessageTooLarge: the service rejects an oversized publish, with 413 for POST and 414 for GET', async function () { + this.timeout(60_000); + // A channel of this run: the oversized payloads are junk that no other test should receive. Nothing is stored. + const channel = `too_large-${Date.now().toString(36)}${Math.random().toString(36).slice(2, 6)}`; + const publisher = shopper(); + const outcome = async ( + size: number, + sendByPost: boolean, + ): Promise<{ statusCode?: number; category?: string; message?: string } | 'published'> => { + try { + await publisher.publish({ channel, message: 'a'.repeat(size), sendByPost, storeInHistory: false }); + return 'published'; + } catch (error) { + const status = ( + error as { status?: { statusCode?: number; category?: string; errorData?: { message?: string } } } + ).status; + return { statusCode: status?.statusCode, category: status?.category, message: status?.errorData?.message }; + } + }; + assert.equal(await outcome(30_000, true), 'published', 'a message below 32 KiB is published'); + const post = await outcome(33_000, true); + const get = await outcome(33_000, false); + console.log(`message above the limit: POST ${JSON.stringify(post)}, GET ${JSON.stringify(get)}`); + assert.notEqual(post, 'published'); + assert.notEqual(get, 'published'); + assert.equal((post as { statusCode?: number }).statusCode, 413); + assert.equal((get as { statusCode?: number }).statusCode, 414); + }); + + it('troubleshootingAccessDenied: a client without a token is denied on the secured keyset, and the error names Access Manager as the source', async function () { + this.timeout(30_000); + const pubnub = createClient('pam', 'trouble-denied'); + await assert.rejects(pubnub.publish({ channel: 'am_room', message: 'm' }), (error: unknown) => { + const status = ( + error as { status?: { category?: string; statusCode?: number; errorData?: { service?: string } } } + ).status; + assert.equal(status?.category, 'PNAccessDeniedCategory'); + assert.equal(status?.statusCode, 403); + assert.equal(status?.errorData?.service, 'Access Manager'); + return true; + }); + await rejectsWithCategory(pubnub.fetchMessages({ channels: ['am_room'] }), 'PNAccessDeniedCategory'); + }); +}); diff --git a/test/integration/live/migration-guides/v11.test.ts b/test/integration/live/migration-guides/v11.test.ts new file mode 100644 index 000000000..d0d04f272 --- /dev/null +++ b/test/integration/live/migration-guides/v11.test.ts @@ -0,0 +1,120 @@ +import assert from 'node:assert/strict'; +import PubNub from 'pubnub'; +import { assertServerTime } from '../support/assertions'; +import { createClient, observer, track } from '../support/clients'; +import { unique } from '../support/util'; +import { keysOf, openKeys } from '../support/env'; + +import { connected, eventually } from '../support/wait'; + +/** + * Examples for the version 11 migration guide. Port of docs-snippets/migration-guides/v11.ts. + * The presence test needs Presence on for the open keyset. + */ + +describe('migration-guides/v11', () => { + it('migrationV11LegacySubscriptionManager: the legacy manager fires PNConnectedCategory again on a second subscribe and never PNSubscriptionChangedCategory', async () => { + // snippet.migrationV11LegacySubscriptionManager + const pubnub = new PubNub({ + subscribeKey: 'mySubscribeKey', + publishKey: 'myPublishKey', + userId: 'myUserId', + enableEventEngine: false, // set explicitly to keep legacy behavior + // snippet.hide + ...openKeys(), + // snippet.show + }); + // snippet.end + track(pubnub); + // Legacy behavior: PNConnectedCategory fires again when the channel mix changes, and there is no PNSubscriptionChangedCategory. + const categories: string[] = []; + pubnub.addListener({ status: (s) => categories.push(String(s.category)) }); + const first = connected(pubnub, { channels: ['ch1'] }); + pubnub.subscribe({ channels: ['ch1'] }); + await first; + pubnub.subscribe({ channels: ['ch2'] }); + await eventually(() => assert.equal(categories.filter((c) => c === 'PNConnectedCategory').length, 2), { + label: 'second PNConnectedCategory', + }); + assert.ok(!categories.includes('PNSubscriptionChangedCategory')); + pubnub.unsubscribeAll(); + }); + + it('migrationV11RestoreFalse: a client built with restore false is valid and works', async () => { + // snippet.migrationV11RestoreFalse + const pubnub = new PubNub({ + subscribeKey: 'mySubscribeKey', + publishKey: 'myPublishKey', + userId: 'myUserId', + restore: false, // set explicitly to keep legacy behavior + // snippet.hide + ...openKeys(), + // snippet.show + }); + // snippet.end + track(pubnub); + // The option is accepted and the client works. The network-loss behavior itself needs a browser. + assertServerTime((await pubnub.time()).timetoken); + }); + + // Offline: only the constructor is exercised. Version 11 lifted the upper bounds, so values the old SDK rejected must be accepted. + it('migrationV11RetryPolicyBounds: retry policy values the old SDK rejected are accepted since version 11', () => { + const config = (retryConfiguration: PubNub.RequestRetryPolicy): PubNub.PubNubConfiguration => ({ + subscribeKey: keysOf('open').subscribeKey, + userId: 'v11-bounds', + retryConfiguration, + }); + // "Version 11.0.0 accepts" maximumRetry above 10 (linear), above 6 and maximumDelay above 150 (exponential). + track(new PubNub(config(PubNub.LinearRetryPolicy({ delay: 2, maximumRetry: 50 })))); + track(new PubNub(config(PubNub.ExponentialRetryPolicy({ minimumDelay: 2, maximumDelay: 600, maximumRetry: 20 })))); + // The minimum delay of the exponential policy is still checked. + assert.throws( + () => new PubNub(config(PubNub.ExponentialRetryPolicy({ minimumDelay: 1, maximumDelay: 10, maximumRetry: 2 }))), + /minimum|delay/i, + ); + }); + + it('migrationV11MaintainPresenceState: with maintainPresenceState true the client re-sends its state after leaving and rejoining, and with false the state is gone', async function () { + this.timeout(90_000); + // Own channel and user ids of this run: the test asserts on exact presence state. + const channel = unique('v11_state'); + // A short presence timeout keeps a client that leaves without unsubscribing out of hereNow quickly. + const presenceOptions = { presenceTimeout: 20, heartbeatInterval: 8 }; + const kept = createClient('open', unique('v11-state-kept'), presenceOptions); + const dropped = createClient('open', unique('v11-state-dropped'), { + ...presenceOptions, + maintainPresenceState: false, + }); + const reader = observer(); + const stateOn = async (client: PubNub): Promise => + (await reader.getState({ uuid: client.getUserId(), channels: [channel] })).channels[channel]; + for (const client of [kept, dropped]) { + const first = connected(client, { channels: [channel] }); + client.subscribe({ channels: [channel], withPresence: true }); + await first; + await client.setState({ state: { mood: 'happy' }, channels: [channel] }); + await eventually(async () => assert.deepEqual(await stateOn(client), { mood: 'happy' }), { + timeout: 30_000, + label: 'state set', + }); + client.disconnect(); + await eventually(async () => assert.deepEqual(await stateOn(client), {}), { + timeout: 30_000, + label: 'state cleared by the leave', + }); + client.reconnect(); + } + await eventually(async () => assert.deepEqual(await stateOn(kept), { mood: 'happy' }), { + timeout: 30_000, + label: 'state re-sent when the client joins again', + }); + // The other client joined at the same time, so its state would be back by now if it were sent. + assert.notDeepEqual( + await stateOn(dropped), + { mood: 'happy' }, + 'with maintainPresenceState: false the state is gone after leaving', + ); + kept.unsubscribeAll(); + dropped.unsubscribeAll(); + }); +}); diff --git a/test/integration/live/migration-guides/v12.test.ts b/test/integration/live/migration-guides/v12.test.ts new file mode 100644 index 000000000..80c181248 --- /dev/null +++ b/test/integration/live/migration-guides/v12.test.ts @@ -0,0 +1,59 @@ +import assert from 'node:assert/strict'; +import fs from 'node:fs'; +import path from 'node:path'; +import PubNub from 'pubnub'; +import { assertServerTime } from '../support/assertions'; +import { createClient, shopper } from '../support/clients'; + +import { SDK_ROOT } from '../support/tsc-scratch'; +import { LocalProxy } from '../support/proxy'; + +/** + * Examples for the version 12 migration guide. Port of docs-snippets/migration-guides/v12.ts. The page has no snippet markers. + */ +describe('migration-guides/v12', () => { + it('migrationV12ProxyEnvironmentVariablesIgnored: the SDK ignores proxy environment variables, shown by zero tunnel requests at the local proxy', async function () { + this.timeout(45_000); + const proxy = new LocalProxy(); + await proxy.start(); + const names = ['HTTP_PROXY', 'HTTPS_PROXY', 'ALL_PROXY', 'http_proxy', 'https_proxy', 'all_proxy']; + const saved = Object.fromEntries(names.map((n) => [n, process.env[n]])); + const uri = `http://${proxy.host}:${proxy.port}`; + for (const n of names) process.env[n] = uri; + try { + const pubnub = shopper(); + const before = proxy.connects; + assertServerTime((await pubnub.time()).timetoken); + assert.equal(proxy.connects, before, 'proxy environment variables are not read'); + + // "read the variable in your own code and pass the proxy address to setProxy()": a URI string works. + pubnub.setProxy(process.env.HTTPS_PROXY!); + assertServerTime((await pubnub.time()).timetoken); + assert.ok(proxy.connects > before, 'setProxy(uri) routes requests through the proxy'); + pubnub.setProxy(); + } finally { + await proxy.stop(); + for (const n of names) { + if (saved[n] === undefined) delete process.env[n]; + else process.env[n] = saved[n]; + } + } + }); + + // Offline: the throw happens inside `setProxy()`, no request is sent. + it('migrationV12SetProxyKeepAlive: setProxy throws when the client was built with keepAlive true, before any request', () => { + // "setProxy() ... throws if keepAlive is true." + const pubnub = createClient('open', 'v12-keepalive', { keepAlive: true }); + assert.throws(() => pubnub.setProxy({ hostname: '127.0.0.1', port: 8080, protocol: 'http' }), /keepAlive/); + }); + + it('migrationV12ReactNativePolyfill: the SDK manifest lists react-native-url-polyfill as an optional peer dependency and requires Node 22 or later', () => { + // "Version 12.0.0 lists react-native-url-polyfill as an optional peer dependency, so installing pubnub no longer installs it." + const manifest = JSON.parse(fs.readFileSync(path.join(SDK_ROOT, 'package.json'), 'utf8')); + assert.ok(manifest.peerDependencies?.['react-native-url-polyfill']); + assert.equal(manifest.peerDependenciesMeta?.['react-native-url-polyfill']?.optional, true); + assert.ok(!manifest.dependencies?.['react-native-url-polyfill']); + assert.equal(manifest.engines.node, '>=22.0.0'); + void PubNub; + }); +}); diff --git a/test/integration/live/migration-guides/v8.test.ts b/test/integration/live/migration-guides/v8.test.ts new file mode 100644 index 000000000..3e7e1a347 --- /dev/null +++ b/test/integration/live/migration-guides/v8.test.ts @@ -0,0 +1,54 @@ +import assert from 'node:assert/strict'; +/* eslint-disable prettier/prettier -- the sample text is shown to readers as written */ +// snippet.migrationV8AfterImport +import PubNub, { AppContext } from 'pubnub' + +// snippet.end +/* eslint-enable prettier/prettier */ +import { tscScratch } from '../support/tsc-scratch'; + +/** + * Examples for the version 8 migration guide. Port of docs-snippets/migration-guides/v8.ts. + * The import snippet sits at the top because an import cannot sit inside a test. It is type checked by the tier's tsconfig + * (npm run test:integration:live:types) and loaded when the file runs. The scratch projects of the second test show what the + * guide claims about the "before" import. + */ +describe('migration-guides/v8', () => { + it('migrationV8AfterImport, migrationV8AfterParams: the typed parameters constant of the v8 guide compiles against the SDK types', () => { + void PubNub; + // snippet.migrationV8AfterParams + // other code + + const params: AppContext.GetMembershipsParameters = {}; + // snippet.end + assert.deepEqual(params, {}); + }); + + it('migrationV8TypeImports: the before sample for type imports fails to compile and the after sample compiles', function () { + this.timeout(90_000); + const before = tscScratch({ + 'before.ts': `import PubNub, { GetMembershipsParameters } from 'pubnub'\n\nconst params: GetMembershipsParameters = {};\n`, + }); + assert.equal(before.ok, false, 'importing GetMembershipsParameters directly fails in 8.x.x'); + assert.match(before.output, /GetMembershipsParameters/); + + const namespaces = [ + 'Subscription', + 'AppContext', + 'ChannelGroups', + 'Publish', + 'Signal', + 'Presence', + 'History', + 'MessageAction', + 'FileSharing', + 'PAM', + 'Time', + 'Push', + ]; + const after = tscScratch({ + 'after.ts': `import PubNub, { ${namespaces.join(', ')} } from 'pubnub'\n\nconst params: AppContext.GetMembershipsParameters = {};\nvoid PubNub; void params;\n`, + }); + assert.ok(after.ok, `the 8.x.x import and every namespace compile.\n${after.output}`); + }); +}); diff --git a/test/integration/live/provision/datasync-classes.json b/test/integration/live/provision/datasync-classes.json new file mode 100644 index 000000000..9cd93dc1f --- /dev/null +++ b/test/integration/live/provision/datasync-classes.json @@ -0,0 +1,63 @@ +{ + "$comment": "Desired DataSync classes on the secured docs keyset. Applied by provision-datasync.ts (dry run by default). Only SubKey-level classes are managed here. The Global User, Channel and Membership classes are seeded by PubNub and cannot be changed. Event publishing per class version has no Admin API endpoint and is switched on in the Admin Portal (DataSync tab, Select entities to sync): User v1, Channel v1, Membership v1, product v1, ProductOwner v1.", + "entityClasses": [ + { + "name": "product", + "version": 1, + "description": "Sneaker product used by the JavaScript SDK docs snippet tests. The public projection exposes name and price, admin exposes everything.", + "config": { "ttlSec": 2678400 }, + "properties": [ + { + "name": "name", + "path": "/payload/name", + "valueKind": "string", + "filtering": "full", + "isNullable": true, + "projections": [{ "name": "__default__" }, { "name": "public" }, { "name": "admin" }] + }, + { + "name": "price", + "path": "/payload/price", + "valueKind": "number", + "filtering": "simple", + "isNullable": false, + "projections": [{ "name": "__default__" }, { "name": "public" }, { "name": "admin" }] + }, + { + "name": "stock", + "path": "/payload/stock", + "valueKind": "number", + "filtering": "simple", + "isNullable": true, + "projections": [{ "name": "__default__" }, { "name": "admin" }] + } + ] + } + ], + "relationshipClasses": [ + { + "name": "ProductOwner", + "version": 1, + "description": "Links a seller (entity A, any class) to a product (entity B) in the JavaScript SDK docs snippet tests.", + "cardinality": "many-to-many", + "entityBClass": "product", + "properties": [ + { + "name": "tier", + "path": "/payload/tier", + "valueKind": "string", + "filtering": "full", + "isNullable": true, + "projections": [{ "name": "__default__" }, { "name": "public" }, { "name": "admin" }] + } + ] + } + ], + "expectedBuiltIns": { + "entityClasses": [ + { "name": "User", "version": 1, "level": "Global" }, + { "name": "Channel", "version": 1, "level": "Global" } + ], + "relationshipClasses": [{ "name": "Membership", "version": 1 }] + } +} diff --git a/test/integration/live/provision/provision-datasync.ts b/test/integration/live/provision/provision-datasync.ts new file mode 100644 index 000000000..c52f4cff1 --- /dev/null +++ b/test/integration/live/provision/provision-datasync.ts @@ -0,0 +1,359 @@ +import fs from 'node:fs'; +import path from 'node:path'; +import { describeRoles, env as currentEnv, redact, type Env } from '../support/env'; + +/** + * Sets up the DataSync classes of the datasync keyset from `datasync-classes.json`. + * + * npm run test:integration:live:provision creates missing classes, updates classes whose mutable fields drifted + * npm run test:integration:live:provision -- --check no-write diff, exit code 1 when anything differs + * + * The live tier does NOT run it: run it by hand when a keyset is new or the classes changed (the preflight cannot see + * classes). Working copy of docs-snippets/provision/provision-datasync.ts. + * + * Rules: + * - Never deletes a class. A class whose immutable fields differ (relationship cardinality or entity sides) fails with the + * diff, because only a person may recreate it. + * - Idempotent. A second run reports every class as unchanged and writes nothing. + * - Touches only the classes in datasync-classes.json. Classes other tests use on a shared keyset (for example the `JS*` + * classes of test/integration/live/support/datasync-live-helpers.ts) are never read for changes and never written. + * - Without PUBNUB_ADMIN_API_KEY it still checks, through the SDK with the datasync keys, and a missing class fails with its + * name. The DataSync steps are never skipped silently. + * - The per class event switch ("Select entities to sync") has no Admin API endpoint. It stays a feature probe + * (`needs-feature:datasync-events`, step datasyncEvents). + * - Output never contains a key or part of one. + * + * Admin API auth is the raw key in the `Authorization` header (no `Bearer`) plus `PubNub-Version`. + */ + +const HERE = __dirname; +const BASE = 'https://admin-api.pubnub.com/v2/datasync/subkeys'; +const API_VERSION = '2026-09-17'; +const ENTITY_TYPE = 'application/vnd.pubnub.objects.entity-class+json;version=1'; +const RELATIONSHIP_TYPE = 'application/vnd.pubnub.objects.relationship-class+json;version=1'; + +type Json = null | boolean | number | string | Json[] | { [key: string]: Json }; +type Obj = { [key: string]: Json }; + +interface Desired { + entityClasses: Obj[]; + relationshipClasses: Obj[]; + expectedBuiltIns: { entityClasses: Obj[]; relationshipClasses: Obj[] }; +} + +interface Action { + kind: 'entity' | 'relationship'; + op: 'create' | 'replace' | 'unchanged' | 'blocked'; + name: string; + version: number; + diff: string[]; + body?: Obj; +} + +export interface ProvisionResult { + ok: boolean; + /** Lines for the console. No key material. */ + log: string[]; + /** What failed the setup, one line each. */ + problems: string[]; + /** Number of POST and PUT calls made. 0 on a second run. */ + writes: number; +} + +async function request( + method: string, + url: string, + apiKey: string, + contentType?: string, + body?: Obj, +): Promise<{ status: number; json: Obj }> { + const headers: Record = { Authorization: apiKey, 'PubNub-Version': API_VERSION }; + if (contentType) headers['Content-Type'] = contentType; + const response = await fetch(url, { method, headers, body: body ? JSON.stringify(body) : undefined }); + const text = await response.text(); + let json: Obj = {}; + try { + json = text ? (JSON.parse(text) as Obj) : {}; + } catch { + // Redact first, then cut: a key cut in half would match neither its literal value nor its shape. + json = { raw: redact(text).slice(0, 200) }; + } + return { status: response.status, json }; +} + +/** Subset comparison: every key in `desired` must match in `actual`. Arrays compare element by element. */ +function differences(actual: Json | undefined, desired: Json, where: string): string[] { + if (Array.isArray(desired)) { + if (!Array.isArray(actual)) return [`${where}: expected an array, found ${JSON.stringify(actual)}`]; + const out: string[] = []; + if (actual.length !== desired.length) + out.push(`${where}: expected ${desired.length} item(s), found ${actual.length}`); + desired.forEach((item, i) => { + const named = typeof item === 'object' && item !== null && 'name' in item; + const label = named ? `${where}[${(item as Obj).name}]` : `${where}[${i}]`; + const match = named ? actual.find((a) => (a as Obj)?.name === (item as Obj).name) : actual[i]; + out.push(...differences(match, item, label)); + }); + return out; + } + if (desired !== null && typeof desired === 'object') { + if (actual === null || typeof actual !== 'object' || Array.isArray(actual)) + return [`${where}: expected an object, found ${JSON.stringify(actual)}`]; + return Object.entries(desired).flatMap(([key, value]) => + differences((actual as Obj)[key], value, where ? `${where}.${key}` : key), + ); + } + return actual === desired ? [] : [`${where}: ${JSON.stringify(actual)} -> ${JSON.stringify(desired)}`]; +} + +function listOf(json: Obj): Obj[] { + return Array.isArray(json.data) ? (json.data as Obj[]) : []; +} + +function entityBody(d: Obj): Obj { + const data: Obj = { config: d.config as Json, properties: d.properties as Json }; + if (d.description) data.description = d.description; + return { data }; +} + +function relationshipCreateBody(d: Obj): Obj { + const data: Obj = { cardinality: d.cardinality as Json }; + for (const key of [ + 'description', + 'entityAClass', + 'entityAClassLevel', + 'entityBClass', + 'entityBClassLevel', + 'properties', + ]) { + if (d[key] !== undefined) data[key] = d[key] as Json; + } + return { data }; +} + +function relationshipReplaceBody(d: Obj): Obj { + const data: Obj = {}; + if (d.description) data.description = d.description; + if (d.properties) data.properties = d.properties; + return { data }; +} + +function desiredClasses(): Desired { + return JSON.parse(fs.readFileSync(path.join(HERE, 'datasync-classes.json'), 'utf8')) as Desired; +} + +/** Without an admin key: check through the SDK that each class exists. Never writes. */ +async function checkWithSdk(env: Env, desired: Desired): Promise { + const log: string[] = [ + 'PUBNUB_ADMIN_API_KEY is not set: checking the classes through the SDK with the datasync keys (no writes)', + ]; + const problems: string[] = []; + const keys = env.datasync; + if (!keys) + return { + ok: false, + log, + problems: ['the datasync keyset is not configured (DS_PUBLISH_KEY, DS_SUBSCRIBE_KEY, DS_SECRET_KEY)'], + writes: 0, + }; + const { default: PubNub } = await import('pubnub'); + const client = new PubNub({ + ...keys, + userId: 'live-provision-check', + logLevel: PubNub.LogLevel.None, + heartbeatInterval: 0, + }); + try { + for (const d of desired.entityClasses) { + try { + await client.dataSync.getEntities({ class: String(d.name), classVersion: Number(d.version), limit: 1 }); + log.push(` = entity class ${d.name} v${d.version} exists`); + } catch (error) { + problems.push( + `DataSync entity class ${d.name} v${d.version} is missing on the datasync keyset (${(error as { status?: { statusCode?: number } }).status?.statusCode ?? (error as Error).message}). Set PUBNUB_ADMIN_API_KEY and run npm run test:integration:live:provision.`, + ); + } + } + for (const d of desired.relationshipClasses) { + try { + await client.dataSync.getRelationships({ class: String(d.name), classVersion: Number(d.version), limit: 1 }); + log.push(` = relationship class ${d.name} v${d.version} exists`); + } catch (error) { + problems.push( + `DataSync relationship class ${d.name} v${d.version} is missing on the datasync keyset (${(error as { status?: { statusCode?: number } }).status?.statusCode ?? (error as Error).message}). Set PUBNUB_ADMIN_API_KEY and run npm run test:integration:live:provision.`, + ); + } + } + } finally { + client.destroy(); + } + return { ok: problems.length === 0, log, problems, writes: 0 }; +} + +/** + * Reads the live classes, compares them with datasync-classes.json and, unless `check`, creates missing classes and updates + * drifted mutable fields. See the header for the rules. + */ +export async function provisionDataSync(env: Env, options: { check: boolean }): Promise { + const desired = desiredClasses(); + if (!env.adminKey) return checkWithSdk(env, desired); + const log: string[] = []; + const problems: string[] = []; + const subKey = env.datasync?.subscribeKey; + if (!subKey) + return { ok: false, log, problems: ['the datasync keyset is not configured (DS_SUBSCRIBE_KEY)'], writes: 0 }; + const apiKey = env.adminKey; + const entities = await request('GET', `${BASE}/${subKey}/entity-classes`, apiKey); + const relationships = await request('GET', `${BASE}/${subKey}/relationship-classes`, apiKey); + if (entities.status !== 200 || relationships.status !== 200) { + const hint = + entities.status === 403 || relationships.status === 403 + ? ' The Service Integration needs a DataSync permission row (Keyset level, Read & write) for the datasync keyset.' + : ''; + return { + ok: false, + log, + problems: [ + `could not read the live classes: entity-classes HTTP ${entities.status}, relationship-classes HTTP ${relationships.status}.${hint}`, + ], + writes: 0, + }; + } + const liveEntities = listOf(entities.json); + const liveRelationships = listOf(relationships.json); + log.push( + `datasync keyset: ${liveEntities.length} entity class(es), ${liveRelationships.length} relationship class(es) live`, + ); + + for (const builtIn of desired.expectedBuiltIns.entityClasses) { + if (!liveEntities.some((c) => c.name === builtIn.name && c.version === builtIn.version)) + problems.push( + `built-in entity class ${builtIn.name} v${builtIn.version} is missing, so DataSync is off on the datasync keyset`, + ); + } + for (const builtIn of desired.expectedBuiltIns.relationshipClasses) { + if (!liveRelationships.some((c) => c.name === builtIn.name && c.version === builtIn.version)) + problems.push( + `built-in relationship class ${builtIn.name} v${builtIn.version} is missing, so DataSync is off on the datasync keyset`, + ); + } + + const actions: Action[] = []; + for (const d of desired.entityClasses) { + const live = liveEntities.find((c) => c.name === d.name && c.version === d.version && c.level === 'SubKey'); + const { name, version, ...comparable } = d; + void name; + void version; + if (!live) + actions.push({ + kind: 'entity', + op: 'create', + name: String(d.name), + version: Number(d.version), + diff: ['(new class)'], + body: entityBody(d), + }); + else { + const diff = differences(live as Json, comparable as Json, ''); + actions.push({ + kind: 'entity', + op: diff.length ? 'replace' : 'unchanged', + name: String(d.name), + version: Number(d.version), + diff, + body: entityBody(d), + }); + } + } + for (const d of desired.relationshipClasses) { + const live = liveRelationships.find((c) => c.name === d.name && c.version === d.version); + const { name, version, ...comparable } = d; + void name; + void version; + if (!live) + actions.push({ + kind: 'relationship', + op: 'create', + name: String(d.name), + version: Number(d.version), + diff: ['(new class)'], + body: relationshipCreateBody(d), + }); + else { + const diff = differences(live as Json, comparable as Json, ''); + const immutable = diff.filter((line) => /^(cardinality|entityA|entityB)/.test(line)); + if (immutable.length) + actions.push({ + kind: 'relationship', + op: 'blocked', + name: String(d.name), + version: Number(d.version), + diff: immutable, + }); + else + actions.push({ + kind: 'relationship', + op: diff.length ? 'replace' : 'unchanged', + name: String(d.name), + version: Number(d.version), + diff, + body: relationshipReplaceBody(d), + }); + } + } + + for (const a of actions) { + const symbol = { create: '+', replace: '~', unchanged: '=', blocked: '!' }[a.op]; + log.push(` ${symbol} ${a.kind} class ${a.name} v${a.version} [${a.op}]`); + if (a.op !== 'unchanged') for (const line of a.diff) log.push(` ${line}`); + } + for (const a of actions.filter((x) => x.op === 'blocked')) { + problems.push( + `${a.kind} class ${a.name} v${a.version} differs in an immutable field (${a.diff.join('; ')}). Only a person may delete and recreate it.`, + ); + } + + const pending = actions.filter((a) => a.op === 'create' || a.op === 'replace'); + let writes = 0; + if (options.check) { + log.push( + pending.length + ? `check: ${pending.length} change(s) would be written, nothing was written` + : 'check: every class is unchanged', + ); + for (const a of pending) + problems.push(`${a.kind} class ${a.name} v${a.version} would be ${a.op === 'create' ? 'created' : 'updated'}`); + } else { + for (const a of pending) { + const dir = a.kind === 'entity' ? 'entity-classes' : 'relationship-classes'; + const url = `${BASE}/${subKey}/${dir}/${encodeURIComponent(a.name)}/versions/${a.version}`; + const type = a.kind === 'entity' ? ENTITY_TYPE : RELATIONSHIP_TYPE; + const result = await request(a.op === 'create' ? 'POST' : 'PUT', url, apiKey, type, a.body); + writes += 1; + const ok = result.status === 200 || result.status === 201; + log.push(` ${a.op === 'create' ? 'POST' : 'PUT'} ${a.kind} ${a.name} v${a.version}: HTTP ${result.status}`); + if (!ok) + problems.push( + `${a.op} of ${a.kind} class ${a.name} v${a.version} failed with HTTP ${result.status}: ${redact(JSON.stringify(result.json)).slice(0, 300)}`, + ); + } + log.push(`${writes} write(s), ${actions.filter((a) => a.op === 'unchanged').length} class(es) unchanged`); + } + return { ok: problems.length === 0, log, problems, writes }; +} + +if (require.main === module) { + const env = currentEnv(); + console.log(`keysets: ${describeRoles()}`); + provisionDataSync(env, { check: process.argv.includes('--check') }).then( + (result) => { + result.log.forEach((line) => console.log(redact(line))); + result.problems.forEach((p) => console.error(`problem: ${redact(p)}`)); + process.exitCode = result.ok ? 0 : 1; + }, + (error) => { + console.error(redact(error instanceof Error ? error.message : String(error))); + process.exitCode = 1; + }, + ); +} diff --git a/test/integration/live/shared/import-pubnub.test.ts b/test/integration/live/shared/import-pubnub.test.ts new file mode 100644 index 000000000..02d019d24 --- /dev/null +++ b/test/integration/live/shared/import-pubnub.test.ts @@ -0,0 +1,47 @@ +import assert from 'node:assert/strict'; +// snippet.importPubNub +import PubNub from 'pubnub'; +// snippet.end +// snippet.importFS +import fs from 'fs'; +// snippet.end +import { assertServerTime } from '../support/assertions'; +import { track } from '../support/clients'; +import { openKeys } from '../support/env'; + +/** + * Shared examples used next to the examples of many pages: the SDK import line, the `fs` import for Node.js file sharing and + * the generic client constructor. Port of docs-snippets/shared/import-pubnub.ts. + * The two import snippets sit at the top of the file because an import cannot sit inside a test. They run when the file loads + * and the tests below prove that the imported names are the real thing. + */ + +describe('shared/import-pubnub', () => { + it('importPubNub: the import line gives the PubNub constructor', () => { + assert.equal(typeof PubNub, 'function'); + assert.equal(typeof PubNub.CryptoModule.aesCbcCryptoModule, 'function'); + }); + + it('importFS: the fs import line gives the Node.js file system module', () => { + assert.equal(typeof fs.readFileSync, 'function'); + assert.equal(fs.existsSync(__filename), true); + }); + + it('PubNubinitBasicUsage: the generic init sample builds a working client and time() succeeds', async () => { + // snippet.PubNubinitBasicUsage + // Initialize PubNub with your keys + const pubnub = new PubNub({ + publishKey: 'YOUR_PUBLISH_KEY', + subscribeKey: 'YOUR_SUBSCRIBE_KEY', + userId: 'YOUR_USER_ID', + // snippet.hide + ...openKeys(), + // snippet.show + }); + // snippet.end + + track(pubnub); + assert.equal(pubnub.getUserId(), 'YOUR_USER_ID'); + assertServerTime((await pubnub.time()).timetoken); + }); +}); diff --git a/test/integration/live/snippet-exceptions.json b/test/integration/live/snippet-exceptions.json new file mode 100644 index 000000000..cfeb16712 --- /dev/null +++ b/test/integration/live/snippet-exceptions.json @@ -0,0 +1,132 @@ +{ + "$comment": "Hand maintained. Things the public docs embed from this repository that are deliberately NOT run against the service, each with a reason. An entry is keyed by the docs-facing path (the path in the EmbeddedCode url, as the docs use it today) and the snippet ids; an entry without \"ids\" covers every embed of that path (whole-file embeds). Seeded from docs-snippets/coverage.json (modes stub and compile-only of the quickstarts). npm run test:integration:live:docs-check reads it.", + "exceptions": [ + { + "path": "docs-snippets/quickstarts/browser-download-file.ts", + "ids": [ + "downloadFileWebBasicUsage" + ], + "reason": "Browser-only sample (needs window and a bundler or a script tag), so it cannot run in Node. Stays in docs-snippets/quickstarts/ and is type-checked by docs-snippets/tsconfig.browser.json (npm run test:docs-snippets:types); it is not run against the service. Old mode: compile-only." + }, + { + "path": "docs-snippets/quickstarts/browser-shared-worker.ts", + "ids": [ + "configurationBasicUsageSubscriptionWorkerUrl" + ], + "reason": "Browser-only sample (needs window and a bundler or a script tag), so it cannot run in Node. Stays in docs-snippets/quickstarts/ and is type-checked by docs-snippets/tsconfig.browser.json (npm run test:docs-snippets:types); it is not run against the service. Old mode: compile-only." + }, + { + "path": "docs-snippets/quickstarts/quickstart-react-native.tsx", + "ids": [ + "quickstartReactNativeCleanup", + "quickstartReactNativeImport", + "quickstartReactNativeInit", + "quickstartReactNativePublish", + "quickstartReactNativeSubscription" + ], + "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in docs-snippets/quickstarts/. Old mode: stub." + }, + { + "path": "docs-snippets/quickstarts/quickstart-react.tsx", + "ids": [ + "quickstartReactImports", + "quickstartReactInit", + "quickstartReactPublish", + "quickstartReactSubscribe" + ], + "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in docs-snippets/quickstarts/. Old mode: stub." + }, + { + "path": "docs-snippets/quickstarts/quickstart-web.ts", + "ids": [ + "quickstartWebInit", + "quickstartWebListeners", + "quickstartWebPublish", + "quickstartWebSubscribe" + ], + "reason": "Browser-only sample (needs window and a bundler or a script tag), so it cannot run in Node. Stays in docs-snippets/quickstarts/ and is type-checked by docs-snippets/tsconfig.browser.json (npm run test:docs-snippets:types); it is not run against the service. Old mode: compile-only." + }, + { + "path": "docs-snippets/quickstarts/react-configuration.jsx", + "ids": [ + "reactConfigurationLifecycle" + ], + "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in docs-snippets/quickstarts/. Old mode: stub." + }, + { + "path": "docs-snippets/quickstarts/react-fetch-messages.jsx", + "ids": [ + "reactFetchMessages" + ], + "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in docs-snippets/quickstarts/. Old mode: stub." + }, + { + "path": "docs-snippets/quickstarts/react-get-state.jsx", + "ids": [ + "reactGetState" + ], + "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in docs-snippets/quickstarts/. Old mode: stub." + }, + { + "path": "docs-snippets/quickstarts/react-here-now.jsx", + "ids": [ + "reactHereNow" + ], + "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in docs-snippets/quickstarts/. Old mode: stub." + }, + { + "path": "docs-snippets/quickstarts/react-listeners.jsx", + "ids": [ + "reactListeners" + ], + "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in docs-snippets/quickstarts/. Old mode: stub." + }, + { + "path": "docs-snippets/quickstarts/react-native-download-file.ts", + "ids": [ + "downloadFileReactNativeBasicUsage" + ], + "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in docs-snippets/quickstarts/. Old mode: stub." + }, + { + "path": "docs-snippets/quickstarts/react-presence-events.jsx", + "ids": [ + "reactPresenceEvents" + ], + "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in docs-snippets/quickstarts/. Old mode: stub." + }, + { + "path": "docs-snippets/quickstarts/react-set-state.jsx", + "ids": [ + "reactSetState" + ], + "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in docs-snippets/quickstarts/. Old mode: stub." + }, + { + "path": "docs-snippets/quickstarts/react-subscribe.jsx", + "ids": [ + "reactSubscribe" + ], + "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in docs-snippets/quickstarts/. Old mode: stub." + }, + { + "path": "docs-snippets/quickstarts/react-where-now.jsx", + "ids": [ + "reactWhereNow" + ], + "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in docs-snippets/quickstarts/. Old mode: stub." + }, + { + "path": "docs-snippets/quickstarts/quickstart-web.html", + "reason": "Whole-file embed of the browser quickstart page (script tag, DOM, CDN build): needs a browser. Stays in docs-snippets/quickstarts/; not run, not type-checked. Its snippets are the same code as quickstart-web.ts." + }, + { + "path": "docs-snippets/quickstarts/quickstart-react-app.tsx", + "reason": "Whole-file embed of the React quickstart app: React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in docs-snippets/quickstarts/. Old mode: stub." + }, + { + "path": "docs-snippets/quickstarts/quickstart-react-native-app.tsx", + "reason": "Whole-file embed of the React Native quickstart app: React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in docs-snippets/quickstarts/. Old mode: stub." + } + ] +} diff --git a/test/integration/live/support/access-manager.ts b/test/integration/live/support/access-manager.ts new file mode 100644 index 000000000..f05875b75 --- /dev/null +++ b/test/integration/live/support/access-manager.ts @@ -0,0 +1,3 @@ +/** Parsed tokens list every permission flag. The docs show the granted ones, so compare only those. */ +export const granted = (permissions: object | undefined): Record => + Object.fromEntries(Object.entries(permissions ?? {}).filter(([, value]) => value === true)); diff --git a/test/integration/live/support/assertions.ts b/test/integration/live/support/assertions.ts new file mode 100644 index 000000000..c09cc222a --- /dev/null +++ b/test/integration/live/support/assertions.ts @@ -0,0 +1,45 @@ +/** + * Small assertion helpers. The SDK reports failures as a rejected promise carrying a `status` object (category, statusCode, + * errorData) rather than as a typed error class, so "this call must be refused for reason X" is the same few lines everywhere. + */ +import assert from 'node:assert/strict'; + +/** Adds `delta` to a PubNub timetoken (17-digit string) without losing precision (BigInt, not Number). */ +export function addToTimetoken(timetoken: string, delta: number): string { + return (BigInt(timetoken) + BigInt(delta)).toString(); +} + +/** A timetoken as `publish()` and the other writes return it: a string of 17 digits (10 ns ticks since 1970). */ +export function assertTimetoken(value: unknown): void { + assert.equal(typeof value, 'string', `expected a timetoken string, got ${typeof value} ${String(value)}`); + assert.match(String(value), /^\d{17}$/, `expected a 17 digit timetoken, got ${String(value)}`); +} + +/** + * The timetoken of `time()`. The types declare `timetoken: string`, but the SDK returns the parsed JSON number, which is above + * Number.MAX_SAFE_INTEGER and can lose its last digit. This pins that behaviour: when the SDK starts returning a string, + * it fails and the special case can go. + */ +export function assertServerTime(value: unknown): void { + assert.equal(typeof value, 'number', `time() timetoken: expected the number the SDK returns, got ${typeof value}`); + assert.match(String(value), /^\d{17}$/, `expected a 17 digit timetoken, got ${String(value)}`); +} + +/** Resolves when `promise` rejects with the SDK's validation error and the status message matches. */ +export async function rejectsValidation(promise: Promise, pattern: RegExp): Promise { + await assert.rejects(promise, (error: unknown) => { + const status = (error as { status?: { category?: string; message?: unknown } }).status; + assert.equal(String(status?.category), 'PNValidationErrorCategory', 'expected a validation error'); + assert.match(String(status?.message ?? ''), pattern); + return true; + }); +} + +/** Resolves when `promise` rejects with an SDK error whose category matches (for example access denied). */ +export async function rejectsWithCategory(promise: Promise, category: string): Promise { + await assert.rejects(promise, (error: unknown) => { + const status = (error as { status?: { category?: string } }).status; + assert.equal(String(status?.category), category); + return true; + }); +} diff --git a/test/integration/live/support/clients.ts b/test/integration/live/support/clients.ts new file mode 100644 index 000000000..9a51a7c58 --- /dev/null +++ b/test/integration/live/support/clients.ts @@ -0,0 +1,110 @@ +/** + * Plain client factories. Every client is tracked and destroyed by the root hook (hooks.ts) after each test, so no test + * leaks a subscribed client into the next one. User ids are literal and shared across runs on purpose: the tier runs one + * live run per keyset at a time. + */ +import PubNub from 'pubnub'; +import { quietConsole } from './console'; +import { keysOf, type Role } from './env'; + +const tracked = new Set(); + +/** + * Every helper client talks to one host and shares one connection pool. The SDK gives each client its own undici pool and a + * random host of ps1..ps20, so each short-lived client pays a TCP and TLS handshake (3 round trips) before its first request. + * At a 190 ms round trip that is 0.9 s for the first request of a client against 0.35 s for the next. Most tests build several + * clients (`observer()` is a new client every call), so the handshakes were about half of the run. + */ +const SHARED_ORIGIN = 'ps.pndsn.com'; +let sharedPool: unknown; + +/** True when `extra` leaves the connection settings alone, so the client may use the shared host and pool. */ +const usesDefaultConnection = (extra: Partial): boolean => + !extra.origin && + !('proxy' in extra) && + extra.keepAlive === undefined && + !extra.keepAliveSettings && + extra.ssl !== false; + +type Pooled = { nodeTransport?: { keepAliveAgent?: unknown; makeKeepAliveAgent?: () => unknown } }; + +function shareConnectionPool(client: PubNub): void { + // The transport and its pool are SDK internals (no public option takes a pool), so this stays defensive: when the shape + // changes the client keeps its own pool and only runs slower. + const transport = (client as unknown as Pooled).nodeTransport; + if (!transport) return; + // The pool of a client built with keepAlive false does not reuse sockets, so the shared one comes from a keep-alive client + // that is never used. The helper clients stay keepAlive false because the SDK refuses setProxy on a keepAlive client. + if (!sharedPool) { + const { publishKey, subscribeKey } = keysOf('open'); + const donor = new PubNub({ publishKey, subscribeKey, userId: 'pool-donor', keepAlive: true }); + sharedPool = (donor as unknown as Pooled).nodeTransport?.makeKeepAliveAgent?.(); + donor.destroy(); + } + if (sharedPool) transport.keepAliveAgent = sharedPool; +} + +/** + * A client on the keyset of `role`, destroyed after the current test. `extra` overrides any configuration field. A client whose + * `extra` sets no origin, proxy, keep-alive or ssl option runs on the shared host and connection pool (see above). + */ +export function createClient(role: Role, userId: string, extra: Partial = {}): PubNub { + const { publishKey, subscribeKey } = keysOf(role); + const pooled = usesDefaultConnection(extra); + const client = new PubNub({ + publishKey, + subscribeKey, + userId, + ...(pooled ? { origin: SHARED_ORIGIN } : {}), + ...extra, + }); + if (pooled) shareConnectionPool(client); + tracked.add(client); + return client; +} + +/** Open keyset, plain user. Most samples run as this client. */ +export const shopper = (extra?: Partial): PubNub => createClient('open', 'shopper', extra); +export const friend = (extra?: Partial): PubNub => createClient('open', 'friend', extra); +/** Never runs sample code. Subscribes and reads to verify effects. Every call returns a new client. */ +export const observer = (extra?: Partial): PubNub => + createClient('open', 'observer', extra); + +/** Secured keyset with the secret key: grants, revokes. Defaults to the `pam` keyset. */ +export function server(role: 'pam' | 'datasync' = 'pam', extra: Partial = {}): PubNub { + // heartbeatInterval 0: a server that only signs grants has no use for presence heartbeats. + return createClient(role, 'staff-server', { secretKey: keysOf(role).secretKey, heartbeatInterval: 0, ...extra }); +} + +/** Spread into a configuration to point a client at a port where nothing listens. Every request fails at once with a network error. */ +export const unreachable = { origin: '127.0.0.1:1', ssl: false } as const; + +/** Registers a client the test built by hand (for example the `new PubNub(...)` of a sample). */ +export function track(client: T): T { + tracked.add(client); + return client; +} + +/** Destroys every tracked client. The root hook calls it after each test. */ +export function destroyClients(): void { + // A client at Debug or Trace level logs a dump of its state while it is destroyed, long after its test printed its summary. + const quiet = quietConsole(); + try { + destroyAll(); + } finally { + quiet.restore(); + } +} + +function destroyAll(): void { + for (const client of tracked) { + try { + // Drop listeners first so the final unsubscribe status does not fire into a test that already finished. + client.removeAllListeners(); + client.destroy(); + } catch { + // A client that failed to build or was destroyed by its test is fine. + } + } + tracked.clear(); +} diff --git a/test/integration/live/support/compare-snippets.ts b/test/integration/live/support/compare-snippets.ts new file mode 100644 index 000000000..689e8fb92 --- /dev/null +++ b/test/integration/live/support/compare-snippets.ts @@ -0,0 +1,51 @@ +/** + * Equivalence check of a ported page: extracts every snippet of the old file and of the new file and compares the text per id. + * Every id must exist in both and be byte-identical. Exit code 0 when all match, 1 otherwise. + * + * npx tsx test/integration/live/support/compare-snippets.ts + * + * It will grow into the marker lint. + */ +import fs from 'node:fs'; +import { extractSnippets } from './snippets'; + +function main(): number { + const [oldFile, newFile] = process.argv.slice(2); + if (!oldFile || !newFile) { + console.error('usage: compare-snippets.ts '); + return 2; + } + const before = extractSnippets(fs.readFileSync(oldFile, 'utf8')); + const after = extractSnippets(fs.readFileSync(newFile, 'utf8')); + let failures = 0; + + for (const [label, result] of [ + [oldFile, before], + [newFile, after], + ] as const) { + for (const problem of result.problems) { + failures += 1; + console.log(`PROBLEM ${label}: ${problem}`); + } + } + for (const id of new Set([...before.snippets.keys(), ...after.snippets.keys()])) { + const a = before.snippets.get(id); + const b = after.snippets.get(id); + if (a === undefined) { + failures += 1; + console.log(`EXTRA ${id}: only in the new file`); + } else if (b === undefined) { + failures += 1; + console.log(`MISSING ${id}: not in the new file`); + } else if (a !== b) { + failures += 1; + console.log(`DIFFERS ${id}\n--- old\n${a}\n--- new\n${b}`); + } else { + console.log(`SAME ${id} (${Buffer.byteLength(a)} bytes)`); + } + } + console.log(failures === 0 ? `OK: ${before.snippets.size} snippets identical` : `FAILED: ${failures} problem(s)`); + return failures === 0 ? 0 : 1; +} + +process.exitCode = main(); diff --git a/test/integration/live/support/console.ts b/test/integration/live/support/console.ts new file mode 100644 index 000000000..239acdf42 --- /dev/null +++ b/test/integration/live/support/console.ts @@ -0,0 +1,48 @@ +/** Console capture for samples that report through `console`. Call `restore()` when the sample is done (in a `finally` when it can throw). */ +export interface ConsoleCapture { + lines: string[]; + restore(): void; +} + +const active = new Set<() => void>(); + +export function captureConsole(method: 'log' | 'info' | 'debug' | 'warn' | 'error' = 'log'): ConsoleCapture { + const original = console[method]; + const lines: string[] = []; + console[method] = (...args: unknown[]) => { + lines.push(args.map((a) => (typeof a === 'string' ? a : safeJson(a))).join(' ')); + }; + const restore = (): void => { + console[method] = original; + active.delete(restore); + }; + active.add(restore); + return { lines, restore }; +} + +/** Undoes every capture that is still active, newest first. The root hook calls it after each test, so a test that failed before its own `restore()` cannot leave the console muted. */ +export function restoreAllConsoles(): void { + for (const restore of [...active].reverse()) restore(); +} + +function safeJson(value: unknown): string { + try { + return JSON.stringify(value); + } catch { + return String(value); + } +} + +/** + * Silences log, info, debug, warn and error. Use it while a client logs at Debug or Trace level: the SDK prints its + * configuration, which holds the keys. `lines` has everything printed meanwhile. Call `restore()` in a `finally`. + */ +export function quietConsole(): ConsoleCapture { + const captures = (['log', 'info', 'debug', 'warn', 'error'] as const).map((method) => captureConsole(method)); + return { + get lines() { + return captures.flatMap((capture) => capture.lines); + }, + restore: () => captures.forEach((capture) => capture.restore()), + }; +} diff --git a/test/integration/live/support/datasync-fragments.ts b/test/integration/live/support/datasync-fragments.ts new file mode 100644 index 000000000..a8091b138 --- /dev/null +++ b/test/integration/live/support/datasync-fragments.ts @@ -0,0 +1,48 @@ +import type PubNub from 'pubnub'; + +/** + * The four fragment blocks of the DataSync page (docs/sdks/javascript/api-reference/datasync.md), one object per docs + * line. The docs show these as bare option lines (`filterFast: 'price < 100'`) inside prose, so they cannot be a + * snippet: several lines repeat the same key and are not statements. They stay hardcoded in the docs and this file + * keeps them honest in two ways. + * + * 1. Types: every entry is typed against the real SDK parameter types, so a renamed option or a `sort` shape the SDK + * no longer accepts breaks `npm run test:integration:live:types`. + * 2. Behaviour: the + * live test of api-reference/datasync.test.ts sends each entry to the service. + * + * Keep each entry on one line, written exactly like the docs line without its trailing comment. The text check + * relies on that. + */ + +// Sneaker Drop context: `price`, `stock`, `status` and `name` are fields of the provisioned `product` class (provision/). +type Query = PubNub.DataSync.GetEntitiesParameters; +type Patch = PubNub.DataSync.UpdateEntityParameters; + +/** Block "Some examples" (filter and filterFast expressions). */ +export const filterFragments: Array> = [ + { filterFast: 'price < 100' }, + { filterFast: '(price < 100 && stock > 0) || price > 500' }, + { filter: 'name LIKE "*sneaker*"' }, + { filter: '!(status == "discontinued")' }, +]; + +/** The two `sort` blocks: object form first, then the raw string form. */ +export const sortFragments: Array> = [ + { sort: { price: 'desc' } }, + { sort: { type: 'asc', price: 'desc' } }, + { sort: { createdAt: null } }, + { sort: 'price:desc' }, + { sort: 'type,price:desc' }, + { sort: 'createdAt' }, +]; + +/** Block "Correct patch forms" (JSON Patch maps of `update*`). */ +export const patchFragments: Array> = [ + { add: { '/payload/tags/0': 'featured' } }, + { replace: { '/payload/price': 79.99 } }, + { remove: ['/payload/tempFlag', '/payload/legacy/field'] }, + { move: [{ from: '/payload/legacyName', path: '/payload/displayName' }] }, + { copy: [{ from: '/payload/displayName', path: '/payload/previousName' }] }, + { test: { '/status': 'active' } }, +]; diff --git a/test/integration/endpoints/data_sync/helpers.ts b/test/integration/live/support/datasync-live-helpers.ts similarity index 85% rename from test/integration/endpoints/data_sync/helpers.ts rename to test/integration/live/support/datasync-live-helpers.ts index bad840b33..4a1c423d2 100644 --- a/test/integration/endpoints/data_sync/helpers.ts +++ b/test/integration/live/support/datasync-live-helpers.ts @@ -1,20 +1,22 @@ -import assert from 'assert'; - -import PubNub from '../../../../src/node/index'; -import type * as DataSync from '../../../../src/core/types/api/data-sync'; -import type * as Subscription from '../../../../src/core/types/api/subscription'; -import type { Status, StatusEvent } from '../../../../src/core/types/api'; +/** + * Helpers of the live DataSync event and projection tests (endpoints/data_sync/*.test.ts). Moved here from the mocked + * test folder: the nock-mocked DataSync tests never used them. Keys come from the tier's env loading (the `datasync` role). + */ +import assert from 'node:assert'; +import PubNub from 'pubnub'; +import { createClient } from './clients'; +import { keysOf } from './env'; +import { connected } from './wait'; // -------------------------------------------------------- // ------------------- Config & basics -------------------- // -------------------------------------------------------- -export const DATA_SYNC_KEYSET = { - subscribeKey: process.env.DS_SUBSCRIBE_KEY ?? '', - publishKey: process.env.DS_PUBLISH_KEY ?? '', - secretKey: process.env.DS_SECRET_KEY ?? '', - userId: 'dsjs', -} as const; +/** Keys of the `datasync` role (DS_* variables or the tier's env file). Read on use, so a missing keyset fails the test that needs it. */ +export const dataSyncKeyset = (): { subscribeKey: string; publishKey: string; secretKey: string } => keysOf('datasync'); + +/** User id of every DataSync test client. */ +export const DATA_SYNC_USER_ID = 'dsjs'; /** Entity/relationship class schema version provisioned on this subkey. */ export const CLASS_VERSION = 1; @@ -27,14 +29,30 @@ export const RELATIONSHIP_CLASS_REQUESTED_BY = 'JSREQUESTED_BY'; /** Class the service pins for the typed `/memberships` endpoints (never sent by the client). */ export const RELATIONSHIP_CLASS_MEMBERSHIP = 'Membership'; -/** Fresh client per test — no shared mutable state, no cross-test bleed. */ +/** + * Fresh client per call on the datasync keyset, with the secret key. The root hook destroys it after the test. + * Pass `secretKey: undefined` for a token-only client. + */ export function freshPubNub(overrides: Record = {}): PubNub { + // `useRequestId: false` is an internal switch the public type does not declare. + return createClient('datasync', DATA_SYNC_USER_ID, { + secretKey: dataSyncKeyset().secretKey, + useRequestId: false, + ...overrides, + } as Partial); +} + +/** + * Like `freshPubNub`, but NOT tracked: for a client built in a `before` hook, which must outlive the first test. The caller + * destroys it in `after`. + */ +export function untrackedPubNub(overrides: Record = {}): PubNub { return new PubNub({ - ...DATA_SYNC_KEYSET, - // @ts-expect-error Force override default value. + ...dataSyncKeyset(), + userId: DATA_SYNC_USER_ID, useRequestId: false, ...overrides, - }); + } as PubNub.PubNubConfiguration); } /** Client-supplied id, underscore-free (service rejects `_` with SYN-0004). */ @@ -49,8 +67,6 @@ export function runMarker(): string { .slice(0, 12); } -export const delay = (ms: number): Promise => new Promise((resolve) => setTimeout(resolve, ms)); - /** Await `fn`, swallowing any error. Use only for cleanup — never in the assertion path. */ export async function safeRemove(fn: () => Promise): Promise { try { @@ -115,7 +131,7 @@ export function assertPayloadContains( } export function assertEntityObject( - obj: DataSync.EntityObject, + obj: PubNub.DataSync.EntityObject, expected: { id: string; entityClass: string; @@ -133,7 +149,7 @@ export function assertEntityObject( } export function assertRelationshipObject( - obj: DataSync.RelationshipObject, + obj: PubNub.DataSync.RelationshipObject, expected: { id: string; entityAId: string; @@ -156,7 +172,7 @@ export function assertRelationshipObject( /** UserObject/ChannelObject share the shape: id + entityClassVersion + status? + payload + timestamps. */ export function assertUserOrChannelObject( - obj: DataSync.UserObject | DataSync.ChannelObject, + obj: PubNub.DataSync.UserObject | PubNub.DataSync.ChannelObject, expected: { id: string; entityClassVersion: number; status?: string; payload?: Record }, ): void { assert.strictEqual(obj.id, expected.id, 'id'); @@ -167,7 +183,7 @@ export function assertUserOrChannelObject( } export function assertMembershipObject( - obj: DataSync.MembershipObject, + obj: PubNub.DataSync.MembershipObject, expected: { id: string; channelId: string; @@ -211,7 +227,7 @@ export type Fixture = { obj: T; dispose: () => Promise }; export async function createCustomerEntity( pn: PubNub, overrides: Record = {}, -): Promise> { +): Promise> { const id = freshId('customer'); const res = await pn.dataSync.createEntity({ id, @@ -228,7 +244,7 @@ export async function createCustomerEntity( export async function createLoanQuoteEntity( pn: PubNub, overrides: Record = {}, -): Promise> { +): Promise> { const id = freshId('loanquote'); const res = await pn.dataSync.createEntity({ id, @@ -247,7 +263,7 @@ export async function createRequestedByRelationship( customerId: string, loanQuoteId: string, overrides: Record = {}, -): Promise> { +): Promise> { const id = freshId('requested-by'); const res = await pn.dataSync.createRelationship({ id, @@ -266,7 +282,7 @@ export async function createRequestedByRelationship( export async function createUserObj( pn: PubNub, overrides: Record = {}, -): Promise> { +): Promise> { const id = freshId('user'); const res = await pn.dataSync.createUser({ id, @@ -278,7 +294,7 @@ export async function createUserObj( export async function createChannelObj( pn: PubNub, overrides: Record = {}, -): Promise> { +): Promise> { const id = freshId('JSchannel'); const res = await pn.dataSync.createChannel({ id, @@ -292,7 +308,7 @@ export async function createMembershipObj( userId: string, channelId: string, overrides: Record = {}, -): Promise> { +): Promise> { const id = freshId('membership'); const res = await pn.dataSync.createMembership({ id, @@ -312,7 +328,8 @@ export async function createMembershipObj( // -------------------------------------------------------- /** - * Subscribe to `channels`, wait for the connection to settle, run `trigger`, and resolve with the + * Subscribe to `channels`, wait until the subscription is connected (the tier's `connected` helper; it replaced a fixed + * 3 s settle delay), run `trigger`, and resolve with the * first DataSync event matching `predicate`. Rejects on timeout or if `trigger` throws. Always cleans * up (removes listener + unsubscribes), even on the happy path. * @@ -322,24 +339,23 @@ export async function createMembershipObj( export function captureEvent( pn: PubNub, channels: string[], - predicate: (event: Subscription.DataSyncObject) => boolean, + predicate: (event: PubNub.Subscription.DataSyncObject) => boolean, trigger: () => Promise, - opts: { settleMs?: number; timeoutMs?: number } = {}, -): Promise { - const settleMs = opts.settleMs ?? 3000; + opts: { timeoutMs?: number } = {}, +): Promise { const timeoutMs = opts.timeoutMs ?? 20000; - return new Promise((resolve, reject) => { + return new Promise((resolve, reject) => { let settled = false; const listener = { - dataSync: (event: Subscription.DataSyncObject) => { + dataSync: (event: PubNub.Subscription.DataSyncObject) => { if (settled || !predicate(event)) return; settled = true; cleanup(); resolve(event); }, - status: (_status: Status | StatusEvent) => { + status: (_status: PubNub.Status | PubNub.StatusEvent) => { /* connection lifecycle — not asserted here */ }, }; @@ -357,17 +373,17 @@ export function captureEvent( pn.unsubscribe({ channels }); } + // Register for the connected status BEFORE subscribing: statuses are not replayed. + const ready = connected(pn, { channels }, { timeout: timeoutMs }); pn.addListener(listener); pn.subscribe({ channels }); - delay(settleMs) - .then(trigger) - .catch((error) => { - if (settled) return; - settled = true; - cleanup(); - reject(error); - }); + ready.then(trigger).catch((error) => { + if (settled) return; + settled = true; + cleanup(); + reject(error); + }); }); } @@ -388,15 +404,15 @@ function assertNoDuplicatedClassFields(data: object): void { /** Assert the fields common to every parsed DataSync event. */ export function assertEventCommon( - event: Subscription.DataSyncObject, + event: PubNub.Subscription.DataSyncObject, expected: { event: 'create' | 'update' | 'delete'; - type: Subscription.DataSyncObjectType; - objectType: Subscription.DataSyncObjectType; + type: PubNub.DataSyncObjectType; + objectType: PubNub.DataSyncObjectType; id: string; channelOneOf: string[]; className?: string; - classLevel?: Subscription.DataSyncClassLevel; + classLevel?: PubNub.DataSyncClassLevel; }, ): void { assert.strictEqual(event.message.source, 'data-sync', 'source'); @@ -419,7 +435,7 @@ export function assertEventCommon( /** For create/update: assert the full non-delete entity data body. */ export function assertEventEntityData( - data: Subscription.DataSyncEntityData, + data: PubNub.DataSyncEntityData, exp: { id: string; status?: string; payload?: Record }, ): void { assert.strictEqual(data.id, exp.id, 'data.id'); @@ -434,7 +450,7 @@ export function assertEventEntityData( /** For create/update: assert the full non-delete relationship data body. */ export function assertEventRelationshipData( - data: Subscription.DataSyncRelationshipData, + data: PubNub.DataSyncRelationshipData, exp: { id: string; entityAId: string; @@ -461,7 +477,7 @@ export function assertEventRelationshipData( * (`channelId` / `userId`) — the same axis the REST `MembershipObject` uses. */ export function assertEventMembershipData( - data: Subscription.DataSyncMembershipData, + data: PubNub.DataSyncMembershipData, exp: { id: string; channelId: string; @@ -490,7 +506,7 @@ export function assertEventMembershipData( * those ids; otherwise every link key must be absent. */ export function assertEventDeleteData( - data: Subscription.DataSyncDeleteData & { + data: PubNub.DataSyncDeleteData & { channelId?: string; userId?: string; entityAId?: string; diff --git a/test/integration/live/support/datasync-state.ts b/test/integration/live/support/datasync-state.ts new file mode 100644 index 000000000..2be7ec907 --- /dev/null +++ b/test/integration/live/support/datasync-state.ts @@ -0,0 +1,32 @@ +/** + * State handling of the DataSync page (api-reference/datasync.test.ts). The samples create objects with literal ids that + * persist on the keyset, so the page removes exactly those objects before and after its run. It never touches a DataSync + * class (those are provisioned elsewhere) and never removes an object it does not list here. + */ +import type PubNub from 'pubnub'; +import { DATASYNC_PAGE_OBJECTS as PAGE_OBJECTS } from './resources'; + +/** The SDK error carries the service answer in `status`. Assertions match on this string, so a changed code shows in the failure. */ +export const errorText = (error: unknown): string => { + const status = (error as { status?: { errorData?: unknown; statusCode?: number } } | undefined)?.status; + return `${status?.statusCode ?? ''} ${JSON.stringify(status?.errorData ?? {})}`; +}; + +/** + * Removes the page's literal objects in dependency order: relationships and memberships first, then the entities, users and + * channels they reference. A missing object is fine (404 is swallowed), so it works as the reset before the run and as the + * best effort cleanup after it. Removing an entity, user or channel also removes the links that point at it. + */ +export async function removePageObjects(client: PubNub): Promise { + const api = client.dataSync; + const quietly = (run: () => Promise): Promise => + run().then( + () => undefined, + () => undefined, + ); + await Promise.all(PAGE_OBJECTS.relationships.map((id) => quietly(() => api.removeRelationship({ id })))); + await Promise.all(PAGE_OBJECTS.memberships.map((id) => quietly(() => api.removeMembership({ id })))); + await Promise.all(PAGE_OBJECTS.entities.map((id) => quietly(() => api.removeEntity({ id })))); + await Promise.all(PAGE_OBJECTS.users.map((id) => quietly(() => api.removeUser({ id })))); + await Promise.all(PAGE_OBJECTS.channels.map((id) => quietly(() => api.removeChannel({ id })))); +} diff --git a/test/integration/live/support/docs-check.ts b/test/integration/live/support/docs-check.ts new file mode 100644 index 000000000..28c6faeb1 --- /dev/null +++ b/test/integration/live/support/docs-check.ts @@ -0,0 +1,552 @@ +/** + * Docs cross-check: does every snippet the public docs embed from this repository map to a test that runs it? + * + * npm run test:integration:live:docs-check -- [] [--report-only] [--out ] + * + * The checkout comes from the first non-flag argument, then $DOCS_REPO, then ../documentation next to this repository. + * It parses every tag whose url points at pubnub/javascript and classifies each embedded id (or whole-file + * embed) as live / compile-only / mocked / exception (snippet-exceptions.json) / NOT COVERED. Two URL styles are understood: + * an old-style url (`docs-snippets//.ts`) is mapped to `test/integration/live//.test.ts`; a + * new-style url (a path under `test/integration/live/`, or one of the mocked test files) is checked as it stands: the file must + * exist and the coverage manifest must list each requested id for that very file. Exit code 1 when anything is NOT COVERED, + * unless --report-only. It writes the repoint list for a later mechanical edit of the docs URLs (default + * /tmp/pn-live/docs-repoint.json). The documentation checkout is only read. No network. + */ +import { spawnSync } from 'node:child_process'; +import fs from 'node:fs'; +import path from 'node:path'; +import { loadExceptions, type ExceptionEntry, type Manifest } from './manifest'; +import { buildManifest } from './manifest'; +import { LIVE_REL, MOCKED_FILES, REPO_ROOT } from './snippet-scan'; +import { docsExtract } from './snippets'; + +/** Pages nobody maintains (decided by the docs owner): their embeds are skipped with a note. Relative to the docs checkout. */ +export const OUT_OF_SCOPE_PAGES = [ + 'docs/sdks/javascript/api-reference/users.md', + 'docs/sdks/javascript/api-reference/spaces.md', + 'docs/sdks/javascript/api-reference/memberships.md', + 'docs/sdks/javascript/api-reference/access-manager-v2.md', +]; + +const TMP_DIR = '/tmp/pn-live'; + +export interface Embed { + docsFile: string; + /** Line of `` outside quotes and braces. */ +function tagEnd(text: string, start: number): number { + let depth = 0; + let quote: string | undefined; + for (let i = start; i < text.length; i++) { + const c = text[i]; + if (quote) { + if (c === '\\') i++; + else if (c === quote) quote = undefined; + } else if (c === "'" || c === '"' || c === '`') quote = c; + else if (c === '{') depth++; + else if (c === '}') depth--; + else if (c === '/' && text[i + 1] === '>' && depth <= 0) return i + 2; + } + return -1; +} + +export function parseEmbeds(docsFile: string, text: string): { embeds: Embed[]; problems: string[] } { + const embeds: Embed[] = []; + const problems: string[] = []; + const lineAt = (index: number): number => text.slice(0, index).split('\n').length; + let from = 0; + for (;;) { + const start = text.indexOf(' tag`); + break; + } + from = end; + const tag = text.slice(start, end); + const url = /\burl=(?:\{\s*)?(['"`])([^'"`]*)\1/.exec(tag); + if (!url) continue; + const m = /^https:\/\/raw\.githubusercontent\.com\/pubnub\/javascript\/refs\/heads\/[^/]+\/(.+)$/.exec(url[2]); + if (!m) { + if (/pubnub\/javascript\b/.test(url[2])) + problems.push(`${docsFile}:${lineAt(start)} unrecognised pubnub/javascript url ${url[2]}`); + continue; + } + const list = /\bsnippets=\{\s*\[([\s\S]*?)\]\s*\}/.exec(tag); + const ids = list + ? [...list[1].matchAll(/(['"`])([^'"`]*)\1/g)].map((x) => x[2]).filter((x) => x.trim() !== '') + : []; + if (/\b(start|end)=\{?\s*\d/.test(tag)) + problems.push(`${docsFile}:${lineAt(start)} embed uses line numbers (start/end): not stable, not checkable`); + embeds.push({ + docsFile, + line: lineAt(start), + urlLine: lineAt(start + (url.index ?? 0)), + oldUrl: url[2], + oldPath: m[1], + ids, + wholeFile: ids.length === 0, + }); + } + return { embeds, problems }; +} + +/** True when the embed already points at a file of the tested tier (a new-style url) rather than at `docs-snippets/`. */ +export function isNewStylePath(p: string): boolean { + return p.startsWith(`${LIVE_REL}/`) || MOCKED_FILES.includes(p); +} + +/** `docs-snippets//.ts` becomes `test/integration/live//.test.ts`; other extensions keep their name. */ +export function mapPath(oldPath: string): string | undefined { + const m = /^docs-snippets\/(.+)$/.exec(oldPath); + if (!m) return undefined; + const rest = m[1]; + return `${LIVE_REL}/${/\.ts$/.test(rest) && !/\.d\.ts$/.test(rest) ? rest.replace(/\.ts$/, '.test.ts') : rest}`; +} + +function readIfExists(rel: string): string | undefined { + const abs = path.join(REPO_ROOT, rel); + return fs.existsSync(abs) ? fs.readFileSync(abs, 'utf8') : undefined; +} + +/** The old copies wrap the call in `try { ... } catch (e) { console.error(...) }`. Removes that wrapper (and two spaces of indent). */ +function unwrapTry(text: string): string | undefined { + const lines = text.split('\n'); + const open = lines.findIndex((l) => l === 'try {'); + const close = lines.findIndex((l, i) => i > open && /^\} catch \(\w+\) \{$/.test(l)); + if (open === -1 || close === -1) return undefined; + const body = lines.slice(open + 1, close).map((l) => (l.startsWith(' ') ? l.slice(2) : l)); + return [...lines.slice(0, open), ...body].join('\n'); +} + +function slug(s: string): string { + return s.replace(/[^A-Za-z0-9_-]+/g, '_'); +} + +function exceptionFor( + exceptions: ExceptionEntry[], + oldPath: string, + id: string | null, + used: Set, +): ExceptionEntry | undefined { + const hit = exceptions.find( + (e) => e.path === oldPath && (e.ids === undefined || (id !== null && e.ids.includes(id))), + ); + if (hit) used.add(hit); + return hit; +} + +/** New-style url: the file is named by the url itself, so it is checked as it stands (no mapping, no old copy to compare). */ +function classifyNewStyle( + embed: Embed, + manifest: Manifest, + exceptions: ExceptionEntry[], + used: Set, +): IdResult[] { + const file = embed.oldPath; + const exists = fs.existsSync(path.join(REPO_ROOT, file)); + if (embed.wholeFile) { + const sample = manifest.files[file]; + if (exists && sample) { + return [{ id: null, status: 'live', detail: `${sample.testFile} :: ${sample.test}` }]; + } + const ex = exceptionFor(exceptions, file, null, used); + if (ex) return [{ id: null, status: 'exception', detail: ex.reason }]; + return [ + { + id: null, + status: 'NOT COVERED', + reason: exists + ? `whole-file embed of ${file}, which no test runs as a file (a whole-file embed needs a sample file such as quickstart-node.mjs that a test executes)` + : `whole-file embed: ${file} does not exist`, + }, + ]; + } + return embed.ids.map((id): IdResult => { + const entry = manifest.snippets[id]; + if (exists && entry && entry.file === file) { + if (docsExtract(readIfExists(file) ?? '', id) === undefined) { + return { + id, + status: 'NOT COVERED', + reason: `${file} has the marker but the docs extractor finds no text for "${id}" (empty or malformed)`, + }; + } + return { + id, + status: entry.mode, + detail: entry.test ?? 'import-only, loaded with the file', + }; + } + const ex = exceptionFor(exceptions, file, id, used); + if (ex) return { id, status: 'exception', detail: ex.reason }; + const reason = !exists + ? `${file} does not exist` + : entry + ? `id is not in ${file}; it is ${entry.mode} in ${entry.file}` + : `id is not in the coverage manifest (not found in ${file})`; + return { id, status: 'NOT COVERED', reason }; + }); +} + +export function classify( + embed: Embed, + manifest: Manifest, + exceptions: ExceptionEntry[], + used: Set, +): IdResult[] { + if (isNewStylePath(embed.oldPath)) return classifyNewStyle(embed, manifest, exceptions, used); + const target = mapPath(embed.oldPath); + const results: IdResult[] = []; + if (embed.wholeFile) { + const sample = target ? manifest.files[target] : undefined; + if (sample) { + const old = readIfExists(embed.oldPath); + const now = readIfExists(target!); + results.push({ + id: null, + status: 'live', + detail: `${sample.testFile} :: ${sample.test}`, + textMatch: old === undefined ? 'old-missing' : old === now ? 'identical' : 'different', + }); + return results; + } + const ex = exceptionFor(exceptions, embed.oldPath, null, used); + if (ex) return [{ id: null, status: 'exception', detail: ex.reason }]; + const reason = !target + ? `path is not under docs-snippets/ (${embed.oldPath})` + : fs.existsSync(path.join(REPO_ROOT, target)) + ? `whole-file embed of ${target}, which no test runs as a file (a whole-file embed needs a sample file such as quickstart-node.mjs that a test executes)` + : `whole-file embed: no sample file ${target}`; + return [{ id: null, status: 'NOT COVERED', reason }]; + } + for (const id of embed.ids) { + const entry = manifest.snippets[id]; + if (target && entry && entry.file === target) { + const old = readIfExists(embed.oldPath); + const oldText = old === undefined ? undefined : docsExtract(old, id); + const newText = docsExtract(readIfExists(target) ?? '', id); + if (newText === undefined) { + results.push({ + id, + status: 'NOT COVERED', + reason: `${target} has the marker but the docs extractor finds no text for "${id}" (empty or malformed)`, + }); + continue; + } + results.push({ + id, + status: entry.mode, + detail: entry.test ?? 'import-only, loaded with the file', + textMatch: oldText === undefined ? 'old-missing' : oldText === newText ? 'identical' : 'different', + }); + continue; + } + const ex = exceptionFor(exceptions, embed.oldPath, id, used); + if (ex) { + results.push({ id, status: 'exception', detail: ex.reason }); + continue; + } + let reason: string; + if (!target) reason = `path is not under docs-snippets/ (${embed.oldPath})`; + else if (!fs.existsSync(path.join(REPO_ROOT, target))) reason = `${target} does not exist`; + else reason = `id is not in ${target}`; + const result: IdResult = { id, status: 'NOT COVERED', reason }; + if (entry) { + // The same id is tested in another file: an old duplicate copy. + const oldText = (() => { + const old = readIfExists(embed.oldPath); + return old === undefined ? undefined : docsExtract(old, id); + })(); + const twinText = docsExtract(readIfExists(entry.file) ?? '', id); + const twin: NonNullable = { + file: entry.file, + identical: oldText === undefined || twinText === undefined ? null : oldText === twinText, + }; + if (twin.identical === false && oldText !== undefined) twin.tryOnly = unwrapTry(oldText) === twinText; + if (twin.identical === false) { + fs.mkdirSync(path.join(TMP_DIR, 'dup-diffs'), { recursive: true }); + const base = path.join( + TMP_DIR, + 'dup-diffs', + `${slug(embed.oldPath.replace(/^docs-snippets\//, ''))}__${slug(id)}`, + ); + fs.writeFileSync(`${base}.old.txt`, `${oldText}\n`); + fs.writeFileSync(`${base}.new.txt`, `${twinText}\n`); + const diff = spawnSync( + 'diff', + ['-u', `--label=old:${embed.oldPath}`, `--label=tested:${entry.file}`, `${base}.old.txt`, `${base}.new.txt`], + { encoding: 'utf8' }, + ); + fs.writeFileSync(`${base}.diff`, diff.stdout); + twin.diffFile = `${base}.diff`; + } + result.twin = twin; + result.reason = `${reason}; the same id is ${entry.mode} in ${entry.file}`; + } + results.push(result); + } + return results; +} + +function worst(results: IdResult[]): Status { + for (const s of ['NOT COVERED', 'exception', 'mocked', 'compile-only', 'live'] as Status[]) + if (results.some((r) => r.status === s)) return s; + return 'live'; +} + +export function main(argv: string[]): number { + const reportOnly = argv.includes('--report-only'); + const outIndex = argv.indexOf('--out'); + const out = outIndex >= 0 ? argv[outIndex + 1] : path.join(TMP_DIR, 'docs-repoint.json'); + const positional = argv.filter((a, i) => !a.startsWith('--') && argv[i - 1] !== '--out'); + const defaultDir = path.resolve(REPO_ROOT, '../documentation'); + const docsDir = path.resolve(positional[0] ?? process.env.DOCS_REPO ?? (fs.existsSync(defaultDir) ? defaultDir : '')); + if (!docsDir || !fs.existsSync(docsDir)) { + console.error( + 'docs-check: no documentation checkout. Pass its path, set DOCS_REPO, or put it at ../documentation.', + ); + return 2; + } + const scanRoot = fs.existsSync(path.join(docsDir, 'docs')) ? path.join(docsDir, 'docs') : docsDir; + const { manifest } = buildManifest(); + const exceptions = loadExceptions(); + const used = new Set(); + + const rows: Array<{ embed: Embed; results: IdResult[]; status: Status }> = []; + const skipped: Embed[] = []; + const problems: string[] = []; + for (const file of walkMd(scanRoot)) { + const rel = path.relative(docsDir, file).split(path.sep).join('/'); + const text = fs.readFileSync(file, 'utf8'); + if (!text.includes('pubnub/javascript/')) continue; + const parsed = parseEmbeds(rel, text); + if (OUT_OF_SCOPE_PAGES.includes(rel)) { + skipped.push(...parsed.embeds); + continue; + } + problems.push(...parsed.problems); + for (const embed of parsed.embeds) { + const results = classify(embed, manifest, exceptions, used); + rows.push({ embed, results, status: worst(results) }); + } + } + + // Summary. + const embedCount: Record = {}; + const refCount: Record = {}; + for (const s of STATUSES) embedCount[s] = refCount[s] = 0; + for (const row of rows) { + embedCount[row.status] += 1; + for (const r of row.results) refCount[r.status] += 1; + } + const pad = (s: string | number, n: number): string => String(s).padEnd(n); + console.log(`docs checkout: ${docsDir}`); + console.log( + `out of scope (skipped): ${OUT_OF_SCOPE_PAGES.map((p) => path.basename(p)).join(', ')}: ${skipped.length} embed(s)`, + ); + console.log(''); + console.log(`${pad('status', 14)}${pad('embeds', 8)}ids/whole files`); + for (const s of STATUSES) console.log(`${pad(s, 14)}${pad(embedCount[s], 8)}${refCount[s]}`); + console.log(`${pad('total', 14)}${pad(rows.length, 8)}${rows.reduce((n, r) => n + r.results.length, 0)}`); + const alreadyNew = rows.filter((r) => isNewStylePath(r.embed.oldPath)).length; + console.log( + `new-style urls (already point at ${LIVE_REL}/ or a mocked test file): ${alreadyNew} of ${rows.length} embeds`, + ); + problems.forEach((p) => console.log(`note: ${p}`)); + + // NOT COVERED, grouped by docs file. + const bad = rows.filter((r) => r.results.some((x) => x.status === 'NOT COVERED')); + if (bad.length > 0) { + console.log('\nNOT COVERED'); + const byFile = new Map(); + for (const r of bad) byFile.set(r.embed.docsFile, [...(byFile.get(r.embed.docsFile) ?? []), r]); + for (const [file, list] of [...byFile].sort()) { + console.log(` ${file}`); + for (const r of list) { + for (const x of r.results.filter((y) => y.status === 'NOT COVERED')) { + console.log(` :${r.embed.line} ${r.embed.oldPath} ${x.id ?? '(whole file)'} ${x.reason ?? ''}`); + } + } + } + } + + // Duplicate copies: NOT COVERED embeds whose id is tested in another file. + const twins = rows.flatMap((r) => r.results.filter((x) => x.twin).map((x) => ({ embed: r.embed, x }))); + if (twins.length > 0) { + const seen = new Set(); + console.log('\nDuplicate copies (old copy vs the tested twin, text extracted with the docs rules)'); + for (const { embed, x } of twins) { + const key = `${embed.oldPath}#${x.id}`; + if (seen.has(key)) continue; + seen.add(key); + const verdict = + x.twin!.identical === null + ? 'UNKNOWN (old copy missing)' + : x.twin!.identical + ? 'IDENTICAL' + : x.twin!.tryOnly + ? 'DIFFERENT (only the try/catch wrapper of the old copy)' + : 'DIFFERENT'; + console.log( + ` ${verdict} ${embed.oldPath} :: ${x.id} -> ${x.twin!.file}${x.twin!.diffFile ? ` (diff: ${x.twin!.diffFile})` : ''}`, + ); + } + } + + // Text of directly mapped embeds. + const differing = rows.flatMap((r) => + r.results.filter((x) => x.textMatch === 'different').map((x) => `${r.embed.oldPath} :: ${x.id ?? '(whole file)'}`), + ); + const compared = rows.flatMap((r) => + r.results.filter((x) => x.textMatch === 'identical' || x.textMatch === 'different'), + ); + console.log( + `\ntext of old-style embeds vs the file they map to (docs extractor rules; embeds that already use the new path are not compared): ${compared.length - differing.length} identical, ${differing.length} different`, + ); + [...new Set(differing)].forEach((d) => console.log(` DIFFERENT ${d}`)); + + const unused = exceptions.filter((e) => !used.has(e)); + unused.forEach((e) => + console.log(`note: exception for ${e.path}${e.ids ? ` [${e.ids.join(', ')}]` : ''} matches no embed in the docs`), + ); + + // Repoint file. + const repoint = rows.map(({ embed, results, status }) => { + if (isNewStylePath(embed.oldPath)) { + // Already repointed: nothing to change (an exception still reads `keep`). + const done = status === 'exception' ? 'keep' : status === 'NOT COVERED' ? 'blocked' : 'done'; + return { + docsFile: embed.docsFile, + line: embed.line, + urlLine: embed.urlLine, + oldUrl: embed.oldUrl, + oldPath: embed.oldPath, + newPath: done === 'blocked' ? null : embed.oldPath, + newUrl: done === 'blocked' ? null : embed.oldUrl, + ids: embed.ids, + wholeFile: embed.wholeFile, + status, + action: done, + ...(done === 'blocked' ? { wouldBePath: null, twinFile: null } : {}), + perId: results.map((r) => ({ id: r.id, status: r.status })), + }; + } + const target = mapPath(embed.oldPath); + const twinFiles = [...new Set(results.filter((r) => r.status === 'NOT COVERED').map((r) => r.twin?.file))]; + // An embed that points at an old duplicate copy is repointed to the tested twin (the docs owner decided so), when every id + // that is not covered has a twin and all twins sit in one file. Anything else stays blocked. + const toTwin = status === 'NOT COVERED' && twinFiles.length === 1 && twinFiles[0] !== undefined; + const twinFile = toTwin ? twinFiles[0] : results.find((r) => r.twin)?.twin?.file; + const action = toTwin + ? 'repoint-to-twin' + : status === 'NOT COVERED' + ? 'blocked' + : status === 'exception' + ? 'keep' + : 'repoint'; + const newPath = + action === 'repoint' + ? (target ?? null) + : action === 'keep' + ? embed.oldPath + : action === 'repoint-to-twin' + ? twinFile! + : null; + return { + docsFile: embed.docsFile, + line: embed.line, + urlLine: embed.urlLine, + oldUrl: embed.oldUrl, + oldPath: embed.oldPath, + newPath, + newUrl: newPath ? embed.oldUrl.replace(embed.oldPath, newPath) : null, + ids: embed.ids, + wholeFile: embed.wholeFile, + status, + action, + ...(action === 'blocked' ? { wouldBePath: target ?? null, twinFile: twinFile ?? null } : {}), + perId: results.map((r) => ({ + id: r.id, + status: r.status, + ...(r.twin + ? { + twin: r.twin.file, + twinText: + r.twin.identical === null + ? 'unknown' + : r.twin.identical + ? 'IDENTICAL' + : r.twin.tryOnly + ? 'DIFFERENT-try-catch-wrapper-only' + : 'DIFFERENT', + } + : {}), + })), + }; + }); + fs.mkdirSync(path.dirname(out), { recursive: true }); + fs.writeFileSync( + out, + `${JSON.stringify( + { + $comment: + 'Generated by test/integration/live/support/docs-check.ts. action: repoint = change oldPath to newPath in the url; repoint-to-twin = the embed points at an old duplicate copy, change oldPath to newPath, the path of the tested twin file (decided by the docs owner); keep = documented exception, url stays; done = the url already points at the tested tier; blocked = not covered and no twin.', + docsDir, + embeds: repoint, + outOfScope: skipped.map((e) => ({ docsFile: e.docsFile, line: e.line, oldPath: e.oldPath, ids: e.ids })), + }, + null, + 2, + )}\n`, + ); + console.log( + `\nrepoint list: ${out} (${repoint.filter((r) => r.action === 'repoint').length} repoint, ${repoint.filter((r) => r.action === 'repoint-to-twin').length} repoint-to-twin, ${repoint.filter((r) => r.action === 'keep').length} keep, ${repoint.filter((r) => r.action === 'done').length} done, ${repoint.filter((r) => r.action === 'blocked').length} blocked)`, + ); + + const failed = embedCount['NOT COVERED'] > 0; + if (failed) + console.log( + reportOnly + ? '\nNOT COVERED embeds present (--report-only: exit 0)' + : '\ndocs-check failed: NOT COVERED embeds present', + ); + return failed && !reportOnly ? 1 : 0; +} + +if (require.main === module) process.exitCode = main(process.argv.slice(2)); diff --git a/test/integration/live/support/env.ts b/test/integration/live/support/env.ts new file mode 100644 index 000000000..860bd2dfb --- /dev/null +++ b/test/integration/live/support/env.ts @@ -0,0 +1,173 @@ +/** + * Keysets of the live tier, read from the environment. The names are the ones of the old harness: + * + * role env names + * open SUBSCRIBE_KEY, PUBLISH_KEY + * pam PAM_SUBSCRIBE_KEY, PAM_PUBLISH_KEY, PAM_SECRET_KEY + * datasync DS_SUBSCRIBE_KEY, DS_PUBLISH_KEY, DS_SECRET_KEY + * + * Optional: PUBNUB_ADMIN_API_KEY and DOCS_SNIPPETS_OPEN_KEYSET_ID / DOCS_SNIPPETS_PAM_KEYSET_ID / DOCS_SNIPPETS_DS_KEYSET_ID (Admin + * API: keyset feature check, DataSync provisioning). LIVE_JANITOR=off turns the before/after cleanup off, LIVE_PREFLIGHT=off the + * preflight (see hooks.ts). + * + * Real environment variables win over the optional local file `test/integration/live/.env.local` (git ignores it), or the + * file `LIVE_ENV_FILE` names. The file is read in memory and nothing here prints a key. + */ +import fs from 'node:fs'; +import path from 'node:path'; + +export type Role = 'open' | 'pam' | 'datasync'; +export const ROLES: readonly Role[] = ['open', 'pam', 'datasync']; + +export interface OpenKeyset { + publishKey: string; + subscribeKey: string; +} + +export interface SecuredKeyset extends OpenKeyset { + secretKey: string; +} + +const KEY_NAMES = { + open: ['PUBLISH_KEY', 'SUBSCRIBE_KEY'], + pam: ['PAM_PUBLISH_KEY', 'PAM_SUBSCRIBE_KEY', 'PAM_SECRET_KEY'], + datasync: ['DS_PUBLISH_KEY', 'DS_SUBSCRIBE_KEY', 'DS_SECRET_KEY'], +} as const; + +export const DEFAULT_ENV_FILE = path.resolve(__dirname, '..', '.env.local'); + +/** Minimal env file reader (`NAME=value`, `#` comments, optional quotes). */ +export function readDotEnv(file: string): Record { + if (!file || !fs.existsSync(file)) return {}; + const out: Record = {}; + for (const raw of fs.readFileSync(file, 'utf8').split('\n')) { + const line = raw.trim(); + if (!line || line.startsWith('#')) continue; + const eq = line.indexOf('='); + if (eq === -1) continue; + const name = line + .slice(0, eq) + .trim() + .replace(/^export\s+/, ''); + let value = line.slice(eq + 1).trim(); + if ((value.startsWith('"') && value.endsWith('"')) || (value.startsWith("'") && value.endsWith("'"))) { + value = value.slice(1, -1); + } + out[name] = value; + } + return out; +} + +/** File values overlaid by the real environment. */ +function source(): Record { + const named = process.env.LIVE_ENV_FILE?.trim(); + const file = named ? path.resolve(named) : DEFAULT_ENV_FILE; + const out: Record = { ...readDotEnv(file) }; + for (const [name, value] of Object.entries(process.env)) if (value) out[name] = value; + return out; +} + +export interface Env { + open?: OpenKeyset; + pam?: SecuredKeyset; + datasync?: SecuredKeyset; + /** Admin API key (Service Integration). Optional: only the feature check and provisioning use it. */ + adminKey?: string; + /** Admin Portal keyset ids per role, for the feature check. Not secrets. */ + keysetIds: Partial>; + /** True when the pam and datasync roles are one keyset. */ + sharedSecured: boolean; + /** Missing env names per role. Empty when the role is usable. */ + missing: Record; +} + +export function loadEnv(vars: Record = source()): Env { + const read = (role: Role): { values: string[]; missing: string[] } => { + const values: string[] = []; + const missing: string[] = []; + for (const name of KEY_NAMES[role]) { + const value = vars[name]?.trim(); + if (value) values.push(value); + else missing.push(name); + } + return { values, missing }; + }; + const open = read('open'); + const pam = read('pam'); + const ds = read('datasync'); + const secured = (r: { values: string[]; missing: string[] }): SecuredKeyset | undefined => + r.missing.length === 0 ? { publishKey: r.values[0], subscribeKey: r.values[1], secretKey: r.values[2] } : undefined; + const pamKeys = secured(pam); + const dsKeys = secured(ds); + const id = (name: string): string | undefined => vars[name]?.trim() || undefined; + return { + open: open.missing.length === 0 ? { publishKey: open.values[0], subscribeKey: open.values[1] } : undefined, + pam: pamKeys, + datasync: dsKeys, + adminKey: id('PUBNUB_ADMIN_API_KEY'), + keysetIds: { + open: id('DOCS_SNIPPETS_OPEN_KEYSET_ID'), + pam: id('DOCS_SNIPPETS_PAM_KEYSET_ID'), + datasync: id('DOCS_SNIPPETS_DS_KEYSET_ID'), + }, + sharedSecured: Boolean( + pamKeys && dsKeys && pamKeys.subscribeKey === dsKeys.subscribeKey && pamKeys.publishKey === dsKeys.publishKey, + ), + missing: { open: open.missing, pam: pam.missing, datasync: ds.missing }, + }; +} + +let cached: Env | undefined; + +/** The environment of this process, read once. */ +export function env(): Env { + cached ??= loadEnv(); + return cached; +} + +/** The keys of a role, or an error that names the missing env names. */ +export function keysOf(role: 'open'): OpenKeyset; +export function keysOf(role: 'pam' | 'datasync'): SecuredKeyset; +export function keysOf(role: Role): OpenKeyset | SecuredKeyset; +export function keysOf(role: Role): OpenKeyset | SecuredKeyset { + const keys = env()[role]; + if (!keys) + throw new Error(`missing keys: the ${role} keyset is not configured, set ${env().missing[role].join(', ')}`); + return keys; +} + +/** "open set, pam missing, ...". Never a value, never a prefix of one. */ +export function describeRoles(): string { + return ROLES.map((role) => `${role} ${env()[role] ? 'set' : 'missing'}`).join(', '); +} + +/** False only when `LIVE_JANITOR=off`: the cleanup before and after the run is skipped (single-page `--grep` runs). */ +export function janitorEnabled(): boolean { + return source().LIVE_JANITOR?.trim().toLowerCase() !== 'off'; +} + +/** False only when `LIVE_PREFLIGHT=off`. */ +export function preflightEnabled(): boolean { + return source().LIVE_PREFLIGHT?.trim().toLowerCase() !== 'off'; +} + +/** + * Masks every key of the environment, the admin key, keys by shape and long token-like strings. Everything the janitor, the + * preflight and the provisioning print goes through it, because SDK error messages can carry request URLs. + */ +export function redact(text: string): string { + let out = text; + const e = env(); + const secrets = [e.open, e.pam, e.datasync].flatMap((k) => (k ? Object.values(k) : [])); + if (e.adminKey) secrets.push(e.adminKey); + for (const secret of secrets) if (secret.length >= 8) out = out.split(secret).join('***'); + return out.replace(/\b(?:sub|pub|sec)-c-[0-9a-f-]{20,}/gi, '***').replace(/\b[A-Za-z0-9_-]{60,}\b/g, '[long-token]'); +} + +/** + * Real keys to spread into a sample inside a `snippet.hide` block, over the placeholder keys readers see. Typed `Partial`, + * otherwise TypeScript reports the visible keys the spread overwrites (TS2783). + */ +export const openKeys = (): Partial => ({ ...keysOf('open') }); +export const securedKeys = (role: 'pam' | 'datasync' = 'pam'): Partial => ({ ...keysOf(role) }); +export const subscribeKeyOnly = (): Partial => ({ subscribeKey: keysOf('open').subscribeKey }); diff --git a/test/integration/live/support/file-workdir.ts b/test/integration/live/support/file-workdir.ts new file mode 100644 index 000000000..393c55d30 --- /dev/null +++ b/test/integration/live/support/file-workdir.ts @@ -0,0 +1,29 @@ +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; + +/** + * Scratch working directory for the file samples. They read and write `./cat_picture.jpg` relative to the working directory, + * and the download sample overwrites it, so each test works on a temp copy of the fixture and the tracked file stays untouched. + */ +export const FIXTURE_FILE = path.join(__dirname, '..', 'fixtures', 'cat_picture.jpg'); + +export interface Workdir { + dir: string; + restore(): void; +} + +/** Copies the fixture to a temp dir and `chdir`s into it. The caller must `restore()` (in a `finally`): the cwd is process-wide. */ +export function enterWorkdir(): Workdir { + const previous = process.cwd(); + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'pn-docs-files-')); + fs.copyFileSync(FIXTURE_FILE, path.join(dir, 'cat_picture.jpg')); + process.chdir(dir); + return { + dir, + restore() { + process.chdir(previous); + fs.rmSync(dir, { recursive: true, force: true }); + }, + }; +} diff --git a/test/integration/live/support/history.ts b/test/integration/live/support/history.ts new file mode 100644 index 000000000..88d4b2575 --- /dev/null +++ b/test/integration/live/support/history.ts @@ -0,0 +1,122 @@ +import assert from 'node:assert/strict'; +import type PubNub from 'pubnub'; +import { addToTimetoken } from './assertions'; +import { observer, shopper } from './clients'; +import { retryTransient } from './retry'; +import { eventually, sleep } from './wait'; +import { randomUUID } from 'node:crypto'; + +/** + * Reads the stored message published at exactly `timetoken`, or undefined. Channels are shared and persistent, so a test + * must look for its own message by timetoken, not take "the latest N". The window is one tick wide on both sides. + */ +export async function storedMessage(client: PubNub, channel: string, timetoken: string) { + const history = await client.fetchMessages({ + channels: [channel], + start: addToTimetoken(timetoken, 1), + end: addToTimetoken(timetoken, -1), + }); + return history.channels[channel]?.find((m) => m.timetoken === timetoken); +} + +/** + * Waits until every timetoken of `tokens` is visible to a reader on `channel` (newest 100 messages only; the write and the read + * model of the service are separate). For larger sets, page through the channel instead. + */ +export function untilStored(channel: string, tokens: string[]): Promise { + return eventually( + async () => { + const history = await observer().fetchMessages({ channels: [channel], count: 100 }); + const have = new Set((history.channels[channel] ?? []).map((m) => String(m.timetoken))); + for (const token of tokens) assert.ok(have.has(token), `${token} is not stored yet`); + }, + { timeout: 30_000, label: `messages stored on ${channel}` }, + ); +} + +/** + * Seeding with retry. Setup code only: a publish that fails with a transient error (5xx, 429, network) is repeated, because + * the failure says nothing about the sample. A failed attempt can still have reached the service, so the message may be stored + * twice: once under a timetoken the caller never saw. A caller that counts the messages of a channel therefore + * 1. puts `run: seedRun.id` into the payload of every seeded message, + * 2. publishes through `seedPublish(seedRun, ...)`, which counts the repeats, and + * 3. calls `settleSeeded(channel, tokens, seedRun)` before it relies on the count. When nothing was repeated it returns at + * once. Otherwise it waits until the returned timetokens are visible, lets the read model settle, deletes every stored + * message of this run whose timetoken the caller does not hold, and waits until none is left. + */ +export interface SeedRun { + readonly id: string; + /** How many publishes needed more than one attempt. */ + retried: number; +} + +export const newSeedRun = (): SeedRun => ({ id: randomUUID(), retried: 0 }); + +export function seedPublish(run: SeedRun, label: string, publish: () => Promise): Promise { + let attempts = 0; + return retryTransient( + () => { + attempts += 1; + return publish(); + }, + { label }, + ).finally(() => { + if (attempts > 1) run.retried += 1; + }); +} + +/** Every stored message of `channel` newer than `oldest` (all when omitted), read page by page from the newest. */ +async function scanChannel(channel: string, oldest?: string): Promise> { + const out: Array<{ timetoken: string; message: unknown }> = []; + let before: string | undefined; + for (let page = 0; page < 50; page += 1) { + const response = await observer().fetchMessages({ + channels: [channel], + count: 100, + ...(before ? { start: before } : {}), + ...(oldest ? { end: oldest } : {}), + }); + const items = response.channels[channel] ?? []; + for (const item of items) out.push({ timetoken: String(item.timetoken), message: item.message }); + if (items.length < 100) break; + before = String(items[0].timetoken); + } + return out; +} + +/** See `SeedRun`. Leaves exactly `tokens` of this run on `channel`. A no-op when no publish was repeated. */ +export async function settleSeeded(channel: string, tokens: string[], run: SeedRun): Promise { + if (run.retried === 0) return; + const wanted = new Set(tokens); + // A repeated publish happens within seconds of the first attempt, so two minutes before the oldest token is a safe bound. + const oldest = tokens.reduce((a, b) => (BigInt(a) < BigInt(b) ? a : b)); + const bound = addToTimetoken(oldest, -1_200_000_000); + const extras = async () => + (await scanChannel(channel, bound)).filter( + (m) => (m.message as { run?: string } | null)?.run === run.id && !wanted.has(m.timetoken), + ); + await eventually( + async () => { + const have = new Set((await scanChannel(channel, bound)).map((m) => m.timetoken)); + for (const token of tokens) assert.ok(have.has(token), `${token} is not stored yet`); + }, + { timeout: 45_000, label: `seeded messages stored on ${channel}` }, + ); + // The first attempt of a repeated publish is older than the one that succeeded, so it is visible by now. The pause is a + // margin for the eventually consistent read model, not a retry. + await sleep(3_000); + const duplicates = await extras(); + for (const { timetoken } of duplicates) { + await retryTransient( + () => shopper().deleteMessages({ channel, start: addToTimetoken(timetoken, -1), end: timetoken }), + { label: `delete duplicate seed on ${channel}` }, + ); + } + if (duplicates.length > 0) { + await eventually(async () => assert.equal((await extras()).length, 0, 'duplicates are gone'), { + timeout: 45_000, + label: `duplicate seeds removed from ${channel}`, + }); + } + console.log(`seeding ${channel}: ${run.retried} publish(es) repeated, ${duplicates.length} duplicate(s) removed`); +} diff --git a/test/integration/live/support/hooks.ts b/test/integration/live/support/hooks.ts new file mode 100644 index 000000000..61be4c1b6 --- /dev/null +++ b/test/integration/live/support/hooks.ts @@ -0,0 +1,121 @@ +/** + * Mocha root hooks of the live tier (loaded by .mocharc.cjs). + * + * before all: key check (the open keyset must be set; a run without --grep and without file arguments needs the pam and datasync + * keysets too, see `isFullRun`), then the janitor (cleans every resource of resources.ts), then, on a + * secured keyset, a wait until a client without a token is refused again, then the preflight (keyset features). + * A cleanup failure here FAILS the run with a message that lists what could not be removed: the tests assume a clean slate. + * after each: restores the console if a test left it captured, then destroys every tracked client. + * after all: the janitor again. Its failures are printed loudly but do NOT fail the run (the test results stay what they are, and + * the next run's "before all" cleanup retries and fails the run when the keyset is still dirty). One exception: a failed revoke of + * the application-level Access Manager grant leaves a secured keyset open, so it fails the run. + * + * The endpoint tests (`.mocharc.endpoints.cjs`, LIVE_TIER=endpoints) check no role up front: a test fails with a "missing keys" + * message (env names) when the role it builds a client for is not set. Janitor and preflight default to off there. + * + * `LIVE_JANITOR=off` skips both cleanups (use it for a single page with `--grep`: a full pass costs about the time + * reported in the README). `LIVE_PREFLIGHT=off` skips the preflight. + */ +import { destroyClients } from './clients'; +import { restoreAllConsoles } from './console'; +import { describeRoles, env, janitorEnabled, preflightEnabled, redact, ROLES } from './env'; +import { formatReport, runJanitor, verifyApplicationGrantClosed } from './janitor'; +import { runPreflight } from './preflight'; + +/** + * True when the command line selects everything: no --grep / --fgrep and no spec file argument. A full run needs every + * role up front, so a missing keyset fails it at once and not after minutes in the first test that needs it. A partial run + * checks only the roles its tests ask for, when they build their client. + */ +function isFullRun(argv: readonly string[] = process.argv.slice(2)): boolean { + const selects = (arg: string): boolean => /^(--grep|-g|--fgrep|-f)(=|$)/.test(arg) || /\.ts$/.test(arg); + return !argv.some(selects); +} + +/** True under `.mocharc.endpoints.cjs` (it sets LIVE_TIER=endpoints): no role is checked up front (each test needs its own, see `keysOf`), janitor and preflight default to off. */ +const endpointsTier = process.env.LIVE_TIER === 'endpoints'; + +const banner = (lines: string[]): void => { + const bar = '!'.repeat(78); + console.error(`\n${bar}\n${lines.map(redact).join('\n')}\n${bar}\n`); +}; + +export const mochaHooks = { + async beforeAll(this: Mocha.Context): Promise { + // The cleanup waits for Access Manager to propagate a revoke (up to about 60 s), so the default test timeout does not fit. + this.timeout(300_000); + console.log(`keysets: ${describeRoles()}`); + const { open, missing } = env(); + // Endpoint tier: no role is required up front. Each test asks for its role when it builds a client (`keysOf` throws an error that + // names the missing env variables), so a --grep run of the DataSync files needs only the DS_* keys and a run of the open-keyset + // files needs only SUBSCRIBE_KEY and PUBLISH_KEY. + if (!endpointsTier && !open) { + throw new Error( + `The live tier needs the open keyset. Missing: ${missing.open.join(', ')}. ` + + 'Set them in the environment or in test/integration/live/.env.local (see .env.example).', + ); + } + if (!endpointsTier && isFullRun()) { + const absent = ROLES.filter((role) => !env()[role]); + if (absent.length) { + throw new Error( + `A full run needs every keyset. Missing: ${absent.map((role) => `${role} (${env().missing[role].join(', ')})`).join('; ')}. ` + + 'Set them, or select tests with --grep (that run needs only the roles of those tests).', + ); + } + } + if (janitorEnabled()) { + const report = await runJanitor(); + formatReport(report).forEach((line) => console.log(line)); + if (report.critical.length || report.failed.length) { + throw new Error( + 'The cleanup before the run failed, so the tests would start on a dirty keyset. ' + + `${[...report.critical, ...report.failed].map(redact).join(' | ')}. ` + + 'Fix the cause, or run `npm run test:integration:live:janitor` to retry the cleanup alone.', + ); + } + const secured = env().pam ?? env().datasync; + if (secured) { + const ms = await verifyApplicationGrantClosed(secured); + console.log(`janitor: a client without a token is refused on the secured keyset (${ms} ms)`); + } + } else { + console.log('janitor: skipped before the run (LIVE_JANITOR=off)'); + } + if (preflightEnabled()) { + const { notes } = await runPreflight(); + notes.forEach((note) => console.log(`preflight: ${note}`)); + console.log('preflight: keyset features ok'); + } + }, + afterEach(): void { + restoreAllConsoles(); + destroyClients(); + }, + async afterAll(this: Mocha.Context): Promise { + if (!janitorEnabled()) return; + this.timeout(300_000); + let report; + try { + report = await runJanitor(); + } catch (error) { + banner([ + `JANITOR CRASHED after the run: ${error instanceof Error ? error.message : String(error)}`, + 'The keysets may hold leftovers.', + ]); + return; + } + formatReport(report).forEach((line) => console.log(line)); + if (report.failed.length) { + banner([ + 'JANITOR: the cleanup after the run could not remove everything. Test results are not affected.', + ...report.failed.map((f) => ` ${f}`), + 'The next run fails in its "before all" cleanup until this is fixed.', + ]); + } + if (report.critical.length) { + banner(['JANITOR CRITICAL: a secured keyset may be left OPEN.', ...report.critical]); + throw new Error(`cleanup after the run left a secured keyset open: ${report.critical.map(redact).join(' | ')}`); + } + }, +}; diff --git a/test/integration/live/support/janitor-cli.ts b/test/integration/live/support/janitor-cli.ts new file mode 100644 index 000000000..c323a32d7 --- /dev/null +++ b/test/integration/live/support/janitor-cli.ts @@ -0,0 +1,28 @@ +/** + * Runs the janitor alone: `npm run test:integration:live:janitor`. Reads the keys like the tier does (environment, then + * LIVE_ENV_FILE or test/integration/live/.env.local). Exit code 1 when something could not be removed or a grant stayed open. + * `--role open,pam` limits the roles. + */ +import { formatReport, runJanitor, verifyApplicationGrantClosed } from './janitor'; +import { env, redact, ROLES, type Role } from './env'; + +async function main(): Promise { + const flag = process.argv.indexOf('--role'); + const roles = + flag > 0 + ? (process.argv[flag + 1] ?? '').split(',').filter((r): r is Role => (ROLES as readonly string[]).includes(r)) + : ROLES; + const report = await runJanitor({ roles }); + formatReport(report).forEach((line) => console.log(line)); + const secured = roles.includes('pam') ? env().pam : roles.includes('datasync') ? env().datasync : undefined; + if (secured && report.critical.length === 0 && !process.argv.includes('--no-verify')) { + const ms = await verifyApplicationGrantClosed(secured); + console.log(`janitor: a client without a token is refused on the secured keyset (checked in ${ms} ms)`); + } + process.exitCode = report.failed.length || report.critical.length ? 1 : 0; +} + +main().catch((error: unknown) => { + console.error(redact(error instanceof Error ? error.message : String(error))); + process.exitCode = 1; +}); diff --git a/test/integration/live/support/janitor-datasync.ts b/test/integration/live/support/janitor-datasync.ts new file mode 100644 index 000000000..834a1d15e --- /dev/null +++ b/test/integration/live/support/janitor-datasync.ts @@ -0,0 +1,155 @@ +/** + * DataSync part of the janitor (called from janitor.ts for the datasync role). Removes only the object ids listed under + * `dataSync` in resources.ts, in dependency order: relationships and memberships first, then entities, users and channels. + * It never touches a DataSync class and never lists a class to delete what it finds. The only listings are the memberships and + * relationships of a LISTED user, channel or entity, and one is removed only when both sides are listed. + */ +import type PubNub from 'pubnub'; +import { CONCURRENCY, Ledger, MAX_PAGES, pool, retry, statusCode } from './janitor-util'; +import type { DataSyncResources } from './resources'; + +interface Page { + data: T[]; + meta?: { has_next?: boolean; next_cursor?: string | null }; +} + +async function allPages(fetch: (cursor?: string) => Promise>): Promise { + const out: T[] = []; + let cursor: string | undefined; + for (let page = 0; page < MAX_PAGES; page += 1) { + const response = await retry(() => fetch(cursor)); + out.push(...response.data); + cursor = response.meta?.next_cursor ?? undefined; + if (!response.meta?.has_next || !cursor) break; + } + return out; +} + +/** DS-0100 is the service code of an unknown class: a keyset without the `ProductOwner` class has no relationship to clean. */ +function missingClass(error: unknown): boolean { + const status = (error as { status?: { statusCode?: number; errorData?: unknown } } | undefined)?.status; + return status?.statusCode === 404 && /DS-0100/.test(JSON.stringify(status.errorData ?? {})); +} + +export async function cleanDataSync(client: PubNub, list: Partial, ledger: Ledger): Promise { + const ds = client.dataSync; + const users = list.users ?? []; + const channels = list.channels ?? []; + const entities = list.entities ?? []; + const userSet = new Set(users); + const channelSet = new Set(channels); + const entitySet = new Set(entities); + + /** Gets the object first, so that an absent one is not reported as removed. A 404 means absent. */ + const remove = async ( + kind: string, + id: string, + get: () => Promise, + del: () => Promise, + ): Promise => { + try { + await retry(get); + } catch (error) { + if (statusCode(error) !== 404) ledger.fail(`DataSync ${kind}`, id, error); + return; + } + try { + await retry(del); + ledger.remove(`DataSync ${kind}`, id); + } catch (error) { + if (statusCode(error) !== 404) ledger.fail(`DataSync ${kind}`, id, error); + } + }; + + // 1. Relationships. Listed ids first, then the `ProductOwner` relationships between a listed user and a listed entity + // (the DataSync event probe creates one without an id of its own). + await pool(list.relationships ?? [], CONCURRENCY, (id) => + remove( + 'relationship', + id, + () => ds.getRelationship({ id }), + () => ds.removeRelationship({ id }), + ), + ); + await pool(users, CONCURRENCY, async (entityAId) => { + try { + const found = await allPages((cursor) => + ds.getRelationships({ class: 'ProductOwner', entityAId, limit: 100, cursor }), + ); + for (const relationship of found) { + const other = (relationship as { entityBId?: string }).entityBId ?? ''; + if (!entitySet.has(other) && !userSet.has(other)) { + ledger.note(`DataSync relationship ${relationship.id} left in place, ${other} is not in resources.ts`); + continue; + } + await remove( + 'relationship', + relationship.id, + async () => undefined, + () => ds.removeRelationship({ id: relationship.id }), + ); + } + } catch (error) { + if (missingClass(error)) ledger.note('DataSync class ProductOwner is not provisioned, no relationships to clean'); + else if (statusCode(error) !== 404) ledger.fail('DataSync relationships of user', entityAId, error); + } + }); + + // 2. Memberships. Listed ids, then the memberships of a listed user to a listed channel. + await pool(list.memberships ?? [], CONCURRENCY, (id) => + remove( + 'membership', + id, + () => ds.getMembership({ id }), + () => ds.removeMembership({ id }), + ), + ); + await pool(users, CONCURRENCY, async (userId) => { + try { + const found = await allPages((cursor) => ds.getMemberships({ userId, limit: 100, cursor })); + for (const membership of found) { + const channelId = (membership as { channelId?: string }).channelId ?? ''; + if (!channelSet.has(channelId)) { + ledger.note( + `DataSync membership ${membership.id} left in place, channel ${channelId} is not in resources.ts`, + ); + continue; + } + await remove( + 'membership', + membership.id, + async () => undefined, + () => ds.removeMembership({ id: membership.id }), + ); + } + } catch (error) { + if (statusCode(error) !== 404) ledger.fail('DataSync memberships of user', userId, error); + } + }); + + // 3. Entities, users, channels. + await pool(entities, CONCURRENCY, (id) => + remove( + 'entity', + id, + () => ds.getEntity({ id }), + () => ds.removeEntity({ id }), + ), + ); + await pool(users, CONCURRENCY, (id) => + remove( + 'user', + id, + () => ds.getUser({ id }), + () => ds.removeUser({ id }), + ), + ); + await pool(channels, CONCURRENCY, (id) => + remove( + 'channel', + id, + () => ds.getChannel({ id }), + () => ds.removeChannel({ id }), + ), + ); +} diff --git a/test/integration/live/support/janitor-util.ts b/test/integration/live/support/janitor-util.ts new file mode 100644 index 000000000..47f2ff66e --- /dev/null +++ b/test/integration/live/support/janitor-util.ts @@ -0,0 +1,55 @@ +/** Small helpers shared by janitor.ts and janitor-datasync.ts. */ +import { redact, type Role } from './env'; +import { retryTransient, statusCode } from './retry'; + +export { statusCode }; + +export const CONCURRENCY = 8; +/** Bounds paging loops in case the service keeps returning cursors. */ +export const MAX_PAGES = 50; + +/** Collects what one pass did. Names go through `redact` before they are stored. */ +export class Ledger { + readonly removed: Array<{ role: Role; kind: string; name: string }> = []; + readonly failed: string[] = []; + readonly notes: string[] = []; + readonly critical: string[] = []; + private readonly seenNotes = new Set(); + + constructor(public role: Role) {} + remove(kind: string, name: string): void { + this.removed.push({ role: this.role, kind, name }); + } + fail(kind: string, name: string, error: unknown): void { + this.failed.push(`${this.role} ${kind} ${name}: ${describeError(error)}`); + } + note(text: string): void { + const line = `${this.role}: ${text}`; + if (!this.seenNotes.has(line)) { + this.seenNotes.add(line); + this.notes.push(line); + } + } +} + +export function describeError(error: unknown): string { + const e = error as { message?: string; status?: { statusCode?: number; errorData?: unknown } } | undefined; + const code = statusCode(error); + const data = e?.status?.errorData ? ` ${JSON.stringify(e.status.errorData)}` : ''; + return redact(`${e?.message ?? String(error)}${code ? ` [HTTP ${code}]` : ''}${data}`).slice(0, 300); +} + +export const pause = (ms: number): Promise => new Promise((resolve) => setTimeout(resolve, ms)); + +/** The janitor's calls are idempotent: transient failures (network, 5xx, 429) are retried by the shared retryTransient, a 4xx is final. */ +export const retry = (call: () => Promise, attempts = 3): Promise => + retryTransient(call, { attempts, label: 'janitor' }); + +/** Runs `fn` over `items` with at most `limit` calls at once. */ +export async function pool(items: readonly T[], limit: number, fn: (item: T) => Promise): Promise { + let next = 0; + const worker = async (): Promise => { + while (next < items.length) await fn(items[next++]); + }; + await Promise.all(Array.from({ length: Math.min(limit, items.length) }, worker)); +} diff --git a/test/integration/live/support/janitor.ts b/test/integration/live/support/janitor.ts new file mode 100644 index 000000000..f2658417d --- /dev/null +++ b/test/integration/live/support/janitor.ts @@ -0,0 +1,429 @@ +/** + * The janitor: cleans EXACTLY the resources listed in resources.ts, per keyset role. Best effort and idempotent: a second pass + * right after the first finds nothing to remove. + * + * Safety rules (do not relax): + * - It deletes only names from resources.ts. It never lists a keyset and deletes what it finds (no getAll*), never touches a + * DataSync class and never changes keyset configuration. + * - The only things it reads without a name are the memberships of a LISTED user or channel, and it removes one only when + * both sides are listed. Anything else is reported in `notes` as left in place. + * - A role whose keys are missing is skipped with a note. + * + * Scope per role: open: channel groups, App Context users/channels/memberships, push registrations, files, history. + * pam: revoke of the application-level Access Manager grant, plus the same lists as open when the role has any. + * datasync: the same revoke (once when pam and datasync are one keyset), DataSync objects (janitor-datasync.ts), history. + * + * Run alone: `npm run test:integration:live:janitor`. The root hooks (hooks.ts) call it before and after the whole run. + */ +import PubNub from 'pubnub'; +import { cleanDataSync } from './janitor-datasync'; +import { CONCURRENCY, describeError, Ledger, MAX_PAGES, pause, pool, retry, statusCode } from './janitor-util'; +import { env, redact, ROLES, type Env, type Role } from './env'; +import { RESOURCES, type RoleResources } from './resources'; + +export interface JanitorReport { + /** Everything removed. */ + removed: Array<{ role: Role; kind: string; name: string }>; + /** Resources that could not be removed (after retries). A failure before the run fails the run. */ + failed: string[]; + /** Skipped parts, resources left in place on purpose, features that are off. */ + notes: string[]; + /** Failures that leave a keyset unsafe (the application-level grant could not be revoked). Always fail the run. */ + critical: string[]; + ms: number; +} + +export interface JanitorOptions { + roles?: readonly Role[]; + /** Replace the resource list (tests of the janitor itself). Defaults to resources.ts. */ + resources?: Readonly>; + env?: Env; +} + +function clientFor(environment: Env, role: Role): PubNub | undefined { + const keys = environment[role]; + if (!keys) return undefined; + return new PubNub({ + ...keys, + userId: 'janitor', + logLevel: PubNub.LogLevel.None, + heartbeatInterval: 0, + }); +} + +/** + * `grant({ read: false, write: false, ttl: 1 })` on a secret-key client replaces the application-level Access Manager (v2) + * rights with none, which closes what the `grantAllChannelsAllAuthKeys` sample opened. Retried, then critical. The revoke takes up + * to about 60 seconds to take effect, `verifyApplicationGrantClosed` waits for it. + */ +async function revokeApplicationGrant(client: PubNub, ledger: Ledger): Promise { + try { + await retry(() => client.grant({ read: false, write: false, ttl: 1 }), 3); + ledger.remove('application-level Access Manager grant', 'revoked'); + } catch (error) { + ledger.critical.push( + `${ledger.role} keyset: the application-level grant could not be revoked (${describeError(error)}). The keyset may be OPEN ` + + 'for reads until the grant ttl runs out (up to 1440 minutes). Revoke it in the Admin Portal.', + ); + } +} + +/** + * Waits until a client WITHOUT a token is refused on the secured keyset (Access Manager enforces again). A revoke starts a clock of up + * to about 60 seconds, and a test that expects a 403 for a tokenless client would fail on a keyset that still behaves as open. + * The probe uses a channel no test ever grants. Returns the waiting time in ms, throws when the keyset stays open for `timeoutMs`. + */ +export async function verifyApplicationGrantClosed( + keys: { publishKey: string; subscribeKey: string }, + timeoutMs = 120_000, +): Promise { + const started = Date.now(); + let last = ''; + for (let attempt = 1; ; attempt += 1) { + // Only the publish and subscribe key: a client that holds the secret key would never be refused. + const probe = new PubNub({ + publishKey: keys.publishKey, + subscribeKey: keys.subscribeKey, + userId: `janitor-probe-${attempt}`, + logLevel: PubNub.LogLevel.None, + heartbeatInterval: 0, + }); + const channel = `janitor-probe-${Date.now().toString(36)}-${attempt}`; + const open: string[] = []; + const unclear: string[] = []; + const ask = async (label: string, call: () => Promise): Promise => { + try { + await call(); + open.push(label); + } catch (error) { + if (statusCode(error) !== 403) unclear.push(`${label}: ${describeError(error)}`); + } + }; + try { + await ask('publish', () => probe.publish({ channel, message: { probe: true } })); + await ask('read', () => probe.fetchMessages({ channels: [channel], count: 1 })); + } finally { + probe.destroy(); + } + // Closed only when both calls got a 403. An unclear answer (network, 5xx, 400) is retried, never taken as closed. + if (open.length === 0 && unclear.length === 0) return Date.now() - started; + last = [open.length ? `a client without a token could ${open.join(' and ')}` : '', unclear.join('; ')] + .filter(Boolean) + .join(', '); + if (Date.now() - started + 3_000 > timeoutMs) break; + await pause(3_000); + } + throw new Error( + `the secured keyset still lets a client without a token through ${Math.round((Date.now() - started) / 1000)} s after the ` + + `application-level grant was revoked (${last}). A grant from an earlier run is probably still effective (a revoke takes up ` + + 'to about 60 seconds to propagate). Check the keyset in the Admin Portal (Access Manager must be on), then run again.', + ); +} + +/** Channel groups: listed names only. A group with no channel does not exist for the service. */ +async function cleanChannelGroups(pn: PubNub, names: readonly string[], ledger: Ledger): Promise { + await pool(names, CONCURRENCY, async (channelGroup) => { + try { + const { channels } = await retry(() => pn.channelGroups.listChannels({ channelGroup })); + if (channels.length === 0) return; + await retry(() => pn.channelGroups.deleteGroup({ channelGroup })); + ledger.remove('channel group', channelGroup); + } catch (error) { + ledger.fail('channel group', channelGroup, error); + } + }); +} + +/** Pages `fetch` until the cursor ends. */ +async function allPages(fetch: (next?: string) => Promise<{ data: T[]; next?: string }>): Promise { + const out: T[] = []; + let next: string | undefined; + for (let page = 0; page < MAX_PAGES; page += 1) { + const response = await retry(() => fetch(next)); + out.push(...response.data); + next = response.next; + if (!next || response.data.length === 0) break; + } + return out; +} + +/** + * App Context. Memberships first (referential integrity is off, so deleting a user or channel would leave them behind), then the + * listed users and channels. Only memberships whose both sides are listed are removed. + */ +async function cleanAppContext( + pn: PubNub, + users: readonly string[], + channels: readonly string[], + ledger: Ledger, +): Promise { + const userSet = new Set(users); + const channelSet = new Set(channels); + const left = (text: string): void => + ledger.note(`App Context membership left in place, one side is not in resources.ts: ${text}`); + + await pool(users, CONCURRENCY, async (uuid) => { + try { + const memberships = await allPages((next) => + pn.objects.getMemberships({ uuid, limit: 100, page: next ? { next } : undefined }), + ); + const own = memberships.map((m) => m.channel.id).filter((id) => channelSet.has(id)); + memberships + .map((m) => m.channel.id) + .filter((id) => !channelSet.has(id)) + .forEach((id) => left(`${uuid} -> ${id}`)); + if (own.length === 0) return; + await retry(() => pn.objects.removeMemberships({ uuid, channels: own })); + own.forEach((id) => ledger.remove('App Context membership', `${uuid} -> ${id}`)); + } catch (error) { + if (statusCode(error) !== 404) ledger.fail('App Context memberships of user', uuid, error); + } + }); + await pool(channels, CONCURRENCY, async (channel) => { + try { + const members = await allPages((next) => + pn.objects.getChannelMembers({ channel, limit: 100, page: next ? { next } : undefined }), + ); + const own = members.map((m) => m.uuid.id).filter((id) => userSet.has(id)); + members + .map((m) => m.uuid.id) + .filter((id) => !userSet.has(id)) + .forEach((id) => left(`${id} -> ${channel}`)); + if (own.length === 0) return; + await retry(() => pn.objects.removeChannelMembers({ channel, uuids: own })); + own.forEach((id) => ledger.remove('App Context member', `${id} -> ${channel}`)); + } catch (error) { + if (statusCode(error) !== 404) ledger.fail('App Context members of channel', channel, error); + } + }); + + const removeObject = async ( + kind: string, + name: string, + get: () => Promise, + del: () => Promise, + ): Promise => { + try { + await retry(get); + } catch (error) { + if (statusCode(error) !== 404) ledger.fail(kind, name, error); + return; + } + try { + await retry(del); + ledger.remove(kind, name); + } catch (error) { + if (statusCode(error) !== 404) ledger.fail(kind, name, error); + } + }; + await pool(users, CONCURRENCY, (uuid) => + removeObject( + 'App Context user', + uuid, + () => pn.objects.getUUIDMetadata({ uuid }), + () => pn.objects.removeUUIDMetadata({ uuid }), + ), + ); + await pool(channels, CONCURRENCY, (channel) => + removeObject( + 'App Context channel', + channel, + () => pn.objects.getChannelMetadata({ channel }), + () => pn.objects.removeChannelMetadata({ channel }), + ), + ); +} + +/** Gateways the old janitor covered: fcm, apns (v1) and both apns2 environments with the docs topic. */ +function gatewaysOf(pn: PubNub): Array<{ + label: string; + list: (device: string) => Promise<{ channels: string[] }>; + remove: (device: string) => Promise; +}> { + const topic = 'com.example.bundle_id'; + return [ + { + label: 'fcm', + list: (device) => pn.push.listChannels({ device, pushGateway: 'fcm' }), + remove: (device) => pn.push.deleteDevice({ device, pushGateway: 'fcm' }), + }, + { + label: 'apns', + list: (device) => pn.push.listChannels({ device, pushGateway: 'apns' }), + remove: (device) => pn.push.deleteDevice({ device, pushGateway: 'apns' }), + }, + ...(['production', 'development'] as const).map((environment) => ({ + label: `apns2/${environment}`, + list: (device: string) => pn.push.listChannels({ device, pushGateway: 'apns2', environment, topic }), + remove: (device: string) => pn.push.deleteDevice({ device, pushGateway: 'apns2', environment, topic }), + })), + ]; +} + +/** + * Push registrations of the listed tokens on every gateway. A 400 or 403 answer to the listing means "not applicable" (push add-on + * off, or a token format the gateway rejects, such as `niceDevice` on apns): noted once, not a failure. + */ +async function cleanPush(pn: PubNub, devices: readonly string[], ledger: Ledger): Promise { + const jobs = devices.flatMap((device) => gatewaysOf(pn).map((gateway) => ({ device, gateway }))); + await pool(jobs, CONCURRENCY, async ({ device, gateway }) => { + const name = `${device.slice(0, 8)}... on ${gateway.label}`; + let channels: string[]; + try { + channels = (await retry(() => gateway.list(device))).channels; + } catch (error) { + const code = statusCode(error); + if (code === 400 || code === 403 || code === 404) + ledger.note(`push ${gateway.label} not applicable (${describeError(error)})`); + else ledger.fail('push device', name, error); + return; + } + if (channels.length === 0) return; + try { + await retry(() => gateway.remove(device)); + ledger.remove('push device', name); + } catch (error) { + ledger.fail('push device', name, error); + } + }); +} + +/** Files on the listed file channels. Pages through the listing (100 per page) before deleting, so deletes cannot shift pages. */ +async function cleanFiles(pn: PubNub, channels: readonly string[], ledger: Ledger): Promise { + await pool(channels, CONCURRENCY, async (channel) => { + let files: Array<{ id: string; name: string }>; + try { + files = await allPages((next) => + pn.listFiles({ channel, limit: 100, next }).then((r) => ({ data: r.data, next: r.next })), + ); + } catch (error) { + // Files off on the keyset, or a channel that never had a file, both leave nothing behind unless the call is a real failure. + if (statusCode(error) === 404) return; + if (statusCode(error) === 400 || statusCode(error) === 403) { + ledger.note(`files not available on this keyset (${describeError(error)})`); + return; + } + ledger.fail('files of channel', channel, error); + return; + } + await pool(files, 4, async (file) => { + try { + await retry(() => pn.deleteFile({ channel, id: file.id, name: file.name })); + ledger.remove('file', `${channel}/${file.name}`); + } catch (error) { + if (statusCode(error) !== 404) ledger.fail('file', `${channel}/${file.name}`, error); + } + }); + }); +} + +/** + * Message history of the listed channels, deleted with `deleteMessages` and no start/end (full range). Only channels that have a + * message are touched (one multi-channel `fetchMessages` per 50 channels finds them). A keyset that refuses it with 403 (Delete-From-History + * off) or has no Message Persistence is skipped with a note. + */ +async function cleanHistory(pn: PubNub, channels: readonly string[], ledger: Ledger): Promise { + const withMessages: string[] = []; + for (let i = 0; i < channels.length; i += 50) { + const chunk = channels.slice(i, i + 50); + try { + const response = await retry(() => pn.fetchMessages({ channels: chunk, count: 1 })); + for (const [channel, messages] of Object.entries(response.channels ?? {})) + if (messages.length > 0) withMessages.push(channel); + } catch (error) { + const code = statusCode(error); + if (code === 400 || code === 403) { + ledger.note(`history not readable, history deletion skipped (${describeError(error)})`); + return; + } + ledger.fail('history read of channels', chunk.join(','), error); + } + } + let refused = false; + await pool(withMessages, CONCURRENCY, async (channel) => { + if (refused) return; + try { + await retry(() => pn.deleteMessages({ channel })); + ledger.remove('history of channel', channel); + } catch (error) { + if (statusCode(error) === 403) { + refused = true; + ledger.note(`Delete-From-History is off, history deletion skipped (${describeError(error)})`); + } else { + ledger.fail('history of channel', channel, error); + } + } + }); +} + +const unique = (...lists: Array): string[] => [ + ...new Set(lists.flatMap((l) => l ?? [])), +]; + +/** Cleans every role (or `options.roles`). Never throws for an ordinary failure, they are collected in the report. */ +export async function runJanitor(options: JanitorOptions = {}): Promise { + const started = Date.now(); + const environment = options.env ?? env(); + const resources = options.resources ?? RESOURCES; + const report: JanitorReport = { removed: [], failed: [], notes: [], critical: [], ms: 0 }; + const take = (ledger: Ledger): void => { + report.removed.push(...ledger.removed); + report.failed.push(...ledger.failed); + report.notes.push(...ledger.notes); + report.critical.push(...ledger.critical); + }; + + for (const role of options.roles ?? ROLES) { + const ledger = new Ledger(role); + const client = clientFor(environment, role); + if (!client) { + ledger.note(`skipped, keys missing: ${environment.missing[role].join(', ')}`); + take(ledger); + continue; + } + const list = resources[role]; + try { + if (role !== 'open') { + // pam and datasync may be one keyset: revoke it once, on the pam role. + if (role === 'datasync' && environment.sharedSecured && environment.pam) + ledger.note('same keyset as pam, grant revoked there'); + else await revokeApplicationGrant(client, ledger); + } + const groups = list.channelGroups ?? []; + const files = list.fileChannels ?? []; + const steps: Array<() => Promise> = []; + if (groups.length) steps.push(() => cleanChannelGroups(client, groups, ledger)); + const users = list.appContext?.users ?? []; + const appChannels = list.appContext?.channels ?? []; + if (users.length || appChannels.length) steps.push(() => cleanAppContext(client, users, appChannels, ledger)); + if (list.pushDevices?.length) steps.push(() => cleanPush(client, list.pushDevices!, ledger)); + if (files.length) steps.push(() => cleanFiles(client, files, ledger)); + if (list.dataSync) steps.push(() => cleanDataSync(client, list.dataSync!, ledger)); + for (const step of steps) await step(); + // History last: files and objects above may publish nothing, but earlier steps are independent of it. + const history = unique(list.channels, list.fileChannels); + if (history.length) await cleanHistory(client, history, ledger); + } catch (error) { + ledger.fail('janitor step', role, error); + } finally { + client.destroy(); + take(ledger); + } + } + report.ms = Date.now() - started; + return report; +} + +/** Console lines of a report: counts per kind, names, notes, failures. Everything is redacted. */ +export function formatReport(report: JanitorReport): string[] { + const lines: string[] = []; + const byKind = new Map(); + for (const { role, kind, name } of report.removed) + byKind.set(`${role} ${kind}`, [...(byKind.get(`${role} ${kind}`) ?? []), name]); + lines.push(`janitor: removed ${report.removed.length} item(s) in ${(report.ms / 1000).toFixed(1)} s`); + for (const [kind, names] of byKind) lines.push(` removed ${kind}: ${names.length} (${names.join(', ')})`); + for (const note of report.notes) lines.push(` note: ${note}`); + for (const failure of report.failed) lines.push(` COULD NOT REMOVE ${failure}`); + for (const critical of report.critical) lines.push(` CRITICAL ${critical}`); + return lines.map(redact); +} diff --git a/test/integration/live/support/lint-markers.ts b/test/integration/live/support/lint-markers.ts new file mode 100644 index 000000000..48d7eb7d0 --- /dev/null +++ b/test/integration/live/support/lint-markers.ts @@ -0,0 +1,479 @@ +/** + * Marker lint of the live tier. Static, no keys, no network. + * + * npx tsx test/integration/live/support/lint-markers.ts (npm run test:integration:live:lint runs it) + * + * Scans every `*.test.ts` and sample file under test/integration/live/ plus the mocked allow-list of snippet-scan.ts. + * + * Marker rules (the docs extractor, documentation/src/helper/codeSnippetFetcher.ts, defines what a marker is): + * M1 marker syntax: `// snippet.`, `// snippet.end`, `// snippet.hide`, `// snippet.show`, with the space after the + * slashes, id charset [a-zA-Z0-9_-] (one known id with "/" is a warning, see snippet-scan.ts) + * M2 every snippet has snippet.end; no start marker inside an open snippet (the extractor would cut the outer one) + * M3 snippet.hide has snippet.show; a hidden block has at most MAX_HIDDEN_LINES lines and holds code + * M4 an id appears once in the tier (one file, one place): the docs address a snippet by file + id + * M5 a snippet is not empty + * M6 no test-only code in the visible text (assert, expect, eventually, watch, anything imported from ./support or + * node:assert) + * M7 no try/catch and no console.error in the visible text, except the ids in ERROR_HANDLING_SNIPPETS + * Honesty rules (replace the run-time injection of the old harness): + * H1 a snippet runs inside an it() callback (or a named helper that an it() calls), or is an import-only snippet at the + * top of a test file (mode compile-only) + * H2 the it() has an assertion call after the snippet's end line (assert*, rejects*, expect*, eventually), or the snippet + * sits in a helper that asserts after it, or the it() carries `// lint: asserted-via ` (listed in the output) + * H3 no `return` that belongs to the function holding the snippet inside the snippet (it could skip the rest) + * H4 a snippet is not in a skipped it() or describe() + * H5 a sample file with markers is run by a test (a test mentions its file name) + * Warnings (do not fail): the slash id; an it() title that does not name its snippet ids; an unused allow-list entry. + */ +import ts from 'typescript'; +import { + callsName, + isVisibleLine, + scanTier, + type Issue, + type ItBlock, + type Region, + type ScannedFile, + type SnippetInfo, + type TierScan, +} from './snippet-scan'; + +/** Hidden blocks hold a single line of live configuration (a key spread, a type suppression). */ +export const MAX_HIDDEN_LINES = 3; + +/** + * Snippets whose whole purpose is to show try/catch around a call, as the docs pages do (M7 does not apply to them). Ported + * from the old harness. Every other snippet lets errors throw, so a failed call fails the test. Keep this list short. + */ +export const ERROR_HANDLING_SNIPPETS: ReadonlySet = new Set([ + 'publishHandleErrors', + 'grantTokenSpacesBasicUsage', + 'grantTokenSpacesVariousResources', + 'grantTokenSpacesRegEx', + 'grantTokenSpacesRegExAndResources', + 'historyBasicUsage', + 'historyOldestThree', + 'historyNewerThanTimetoken', + 'historyUntilTimetoken', + 'historyIncludeMeta', + 'quickstartNodePublish', +]); + +/** + * Folders (relative to live/) whose samples are application code that the use-case pages show with their own error handling + * (try/catch around each call, console.error). M7 does not apply there. The old harness never linted use-cases/ (its scan + * list was api-reference, shared, getting-started, migration-guides), so this is a new decision, not a port: the + * protection against a swallowed error is rule H2 (an assertion on the effect after the snippet). + */ +export const ERROR_HANDLING_FOLDERS: readonly string[] = ['use-cases/']; + +const TEST_ONLY_NAMES = ['assert', 'expect', 'eventually', 'watch', 'expectEvent', 'expectStatus']; +const ASSERTION_NAME = /^(assert\w*|rejects\w*|expect\w*)$/; +const ESCAPE = /^\s*\/\/\s*lint:\s*asserted-via\s+([A-Za-z_$][\w$]*)\s*$/; + +export interface LintResult { + errors: Issue[]; + warnings: Issue[]; + /** Every use of the `lint: asserted-via` escape and every hidden type suppression: reported, never silent. */ + notes: string[]; + files: number; + snippets: number; + scan: TierScan; +} + +function leftmost(expr: ts.Expression): ts.Expression { + let e = expr; + while (true) { + if (ts.isPropertyAccessExpression(e) || ts.isElementAccessExpression(e)) e = e.expression; + else if (ts.isCallExpression(e)) e = e.expression; + else if (ts.isNonNullExpression(e) || ts.isParenthesizedExpression(e) || ts.isAwaitExpression(e)) e = e.expression; + else return e; + } +} + +/** Is `call` an assertion by name: assert.x(), assertFoo(), expect(...).…, expectEvent(), rejectsX(), eventually(). */ +export function isAssertionCall(call: ts.CallExpression): boolean { + const callee = call.expression; + const root = leftmost(callee); + if (ts.isIdentifier(root) && (root.text === 'assert' || root.text === 'expect')) return true; + const name = ts.isIdentifier(callee) + ? callee.text + : ts.isPropertyAccessExpression(callee) + ? callee.name.text + : undefined; + if (name === undefined) return false; + return ASSERTION_NAME.test(name) || name === 'eventually'; +} + +/** + * Variables of `node` initialised by an assertion call (`const back = expectStatus(...)`): the assertion is armed there and + * fires when the variable is awaited later. Returns their names. + */ +function armedAssertions(node: ts.Node): Set { + const names = new Set(); + const visit = (n: ts.Node): void => { + if (ts.isVariableDeclaration(n) && ts.isIdentifier(n.name) && n.initializer) { + const init = ts.isAwaitExpression(n.initializer) ? n.initializer.expression : n.initializer; + if (ts.isCallExpression(init) && isAssertionCall(init)) names.add(n.name.text); + } + ts.forEachChild(n, visit); + }; + visit(node); + return names; +} + +/** + * First line (1-based) of an assertion that runs after `afterLine` inside `node`: a call of assert*, rejects*, expect*, + * eventually, or an `await` of a variable that an earlier assertion call armed (`const back = expectStatus(...)` ... `await back`). + */ +function assertionAfter(file: ScannedFile, node: ts.Node, afterLine: number): number | undefined { + let found: number | undefined; + const armed = armedAssertions(node); + const visit = (n: ts.Node): void => { + if (found !== undefined) return; + if (ts.isCallExpression(n) && isAssertionCall(n)) { + const line = file.lineOf(n.getStart(file.source)); + if (line > afterLine) { + found = line; + return; + } + } + if (ts.isAwaitExpression(n) && armed.size > 0) { + const line = file.lineOf(n.getStart(file.source)); + let uses = false; + const look = (m: ts.Node): void => { + if (ts.isIdentifier(m) && armed.has(m.text)) uses = true; + else ts.forEachChild(m, look); + }; + look(n.expression); + if (uses && line > afterLine) { + found = line; + return; + } + } + ts.forEachChild(n, visit); + }; + visit(node); + return found; +} + +/** Line of the first call of `name` inside `node`. */ +function firstCallLine(file: ScannedFile, node: ts.Node, name: string): number | undefined { + let found: number | undefined; + const visit = (n: ts.Node): void => { + if (found !== undefined) return; + if (ts.isCallExpression(n) && ts.isIdentifier(n.expression) && n.expression.text === name) { + found = file.lineOf(n.getStart(file.source)); + return; + } + ts.forEachChild(n, visit); + }; + visit(node); + return found; +} + +function importedTestOnlyNames(file: ScannedFile): Set { + const names = new Set(TEST_ONLY_NAMES); + for (const s of file.source.statements) { + if (!ts.isImportDeclaration(s) || !ts.isStringLiteral(s.moduleSpecifier)) continue; + const spec = s.moduleSpecifier.text; + if (!/(^|\/)support(\/|$)/.test(spec) && !/^(node:)?assert(\/strict)?$/.test(spec)) continue; + const clause = s.importClause; + if (!clause) continue; + if (clause.name) names.add(clause.name.text); + const bindings = clause.namedBindings; + if (bindings && ts.isNamespaceImport(bindings)) names.add(bindings.name.text); + if (bindings && ts.isNamedImports(bindings)) bindings.elements.forEach((e) => names.add(e.name.text)); + } + return names; +} + +function isNamePosition(node: ts.Identifier): boolean { + const p = node.parent; + return ( + (ts.isPropertyAccessExpression(p) && p.name === node) || + (ts.isPropertyAssignment(p) && p.name === node) || + (ts.isBindingElement(p) && p.propertyName === node) || + (ts.isPropertySignature(p) && p.name === node) || + (ts.isMethodDeclaration(p) && p.name === node) || + (ts.isShorthandPropertyAssignment(p) && false) + ); +} + +function enclosingFunction(node: ts.Node): ts.Node | undefined { + for (let p = node.parent; p; p = p.parent) if (ts.isFunctionLike(p)) return p; + return undefined; +} + +function checkRegion( + file: ScannedFile, + snippet: SnippetInfo, + region: Region, + add: (line: number, msg: string) => void, + warn: (line: number, msg: string) => void, + notes: string[], + usedAllow: Set, +): void { + const lines = file.text.split('\n'); + const { source, lineOf } = file; + + // M3 hidden blocks. + for (const block of region.hidden) { + if (block.showLine === undefined) continue; + const body = lines.slice(block.hideLine, block.showLine - 1); + const count = body.length; + if (count > MAX_HIDDEN_LINES) + add(block.hideLine, `hidden block in "${region.id}" has ${count} lines, maximum is ${MAX_HIDDEN_LINES}`); + const code = body + .join('\n') + .replace(/\/\*[\s\S]*?\*\//g, '') + .replace(/(^|[^:'"`])\/\/.*$/gm, '$1') + .trim(); + if (body.join('').trim() === '') add(block.hideLine, `hidden block in "${region.id}" is empty`); + else if (code === '') { + if (/^\s*\/\/\s*@ts-expect-error\b/.test(body.find((l) => l.trim() !== '') ?? '') && count === 1) { + notes.push(`hidden @ts-expect-error: ${file.rel}:${block.hideLine} (${region.id})`); + } else + add(block.hideLine, `hidden block in "${region.id}" holds only comments: hiding text the reader should see`); + } + } + + // M5 empty. + const visible = lines.filter((_, i) => isVisibleLine(region, i + 1)); + if (visible.every((l) => l.trim() === '')) + add(region.startLine, `snippet "${region.id}" is empty: no visible line between its markers`); + + // M6 and M7: walk the file, look at nodes that start on a visible line of this region. + const testOnly = importedTestOnlyNames(file); + const allowTry = + ERROR_HANDLING_SNIPPETS.has(region.id) || + ERROR_HANDLING_FOLDERS.some((f) => file.rel.startsWith(`test/integration/live/${f}`)); + let sawTry = false; + const visit = (node: ts.Node): void => { + const start = lineOf(node.getStart(source)); + const visibleHere = isVisibleLine(region, start); + if (visibleHere) { + if ( + ts.isIdentifier(node) && + testOnly.has(node.text) && + !isNamePosition(node) && + !ts.isImportSpecifier(node.parent) + ) { + add(start, `test-only code in visible text of "${region.id}": \`${node.text}\``); + } + if ( + ts.isImportDeclaration(node) && + ts.isStringLiteral(node.moduleSpecifier) && + /(^|\/)support(\/|$)/.test(node.moduleSpecifier.text) + ) { + add(start, `test-only import from "${node.moduleSpecifier.text}" in visible text of "${region.id}"`); + } + if (ts.isTryStatement(node)) { + sawTry = true; + if (!allowTry) + add( + start, + `try/catch inside snippet "${region.id}" (a snippet lets errors throw; allow-list: ERROR_HANDLING_SNIPPETS in lint-markers.ts)`, + ); + } + if ( + ts.isCallExpression(node) && + ts.isPropertyAccessExpression(node.expression) && + ts.isIdentifier(node.expression.expression) && + node.expression.expression.text === 'console' && + node.expression.name.text === 'error' + ) { + sawTry = true; + if (!allowTry) + add( + start, + `console.error inside snippet "${region.id}" (a snippet lets errors throw; allow-list: ERROR_HANDLING_SNIPPETS in lint-markers.ts)`, + ); + } + } + if (start <= region.endLine) ts.forEachChild(node, visit); + }; + visit(source); + if (sawTry) usedAllow.add(region.id); + + // H1 to H4 by owner. + const owner = snippet.owner; + if (owner.type === 'none') { + if (file.kind === 'sample') { + if (snippet.mode === 'invalid') + add( + region.startLine, + `snippet "${region.id}" is in a sample file that no test runs (no it() in the same folder mentions its file name)`, + ); + } else add(region.startLine, `snippet "${region.id}" ${owner.reason}`); + return; + } + if (owner.type === 'import-only') return; + if (owner.type === 'file-level-error') { + add(owner.line, owner.message); + return; + } + if (owner.type === 'file-level') { + // The named it() must assert something at all; that it asserts THIS sample's effect is the author's claim, printed as a note. + if (assertionAfter(file, owner.it.fn, owner.it.startLine - 1) === undefined) + add(owner.line, `\`lint: asserted-by ${owner.needle}\`: the test "${owner.it.title}" has no assertion call`); + else + notes.push(`asserted-by "${owner.needle}": ${file.rel}:${owner.line} (${region.id}) -> "${owner.it.fullTitle}"`); + return; + } + + const fn = owner.type === 'it' ? owner.it.fn : owner.fn; + // H3: a return that belongs to the function holding the snippet. + const returns: number[] = []; + const findReturns = (node: ts.Node): void => { + if (ts.isReturnStatement(node)) { + const l = lineOf(node.getStart(source)); + if (l > region.startLine && l < region.endLine && enclosingFunction(node) === fn) returns.push(l); + } + if (node !== fn && ts.isFunctionLike(node)) return; + ts.forEachChild(node, findReturns); + }; + if (fn.body) ts.forEachChild(fn.body, findReturns); + returns.forEach((l) => + add(l, `return inside snippet "${region.id}" can skip the rest of the test (the assertions after it)`), + ); + + if (owner.type === 'it') { + const it = owner.it; + if (it.skipped) add(region.startLine, `snippet "${region.id}" sits in a skipped test ("${it.fullTitle}")`); + checkAssertion(file, it, region, add, notes); + } else { + // A named helper: it must be called by an it(), and the assertion comes from the helper or from the caller. + const callers = file.its.filter((it) => callsName(it.fn, owner.name)); + if (callers.length === 0) { + add(region.startLine, `snippet "${region.id}" is in helper ${owner.name}() that no it() calls`); + return; + } + if (assertionAfter(file, owner.fn, region.endLine) !== undefined) return; + for (const it of callers) { + const callLine = firstCallLine(file, it.fn, owner.name) ?? it.startLine; + if (assertionAfter(file, it.fn, callLine) === undefined && !escapeOf(file, it, region, notes, add)) { + add( + region.startLine, + `snippet "${region.id}" runs in helper ${owner.name}() but no assertion follows its call in "${it.fullTitle}" (add one, or \`// lint: asserted-via \`)`, + ); + } + } + } + // The title convention is the live tier's; the mocked tests outside it keep their own titles. + if (file.kind !== 'mocked' && titleLacksId(snippet, owner)) { + warn( + region.startLine, + `title of the test "${owner.type === 'it' ? owner.it.title : ''}" does not start with its snippet id "${region.id}" (README convention)`, + ); + } +} + +function titleLacksId(snippet: SnippetInfo, owner: SnippetInfo['owner']): boolean { + return owner.type === 'it' && !owner.it.title.includes(snippet.id); +} + +function escapeOf( + file: ScannedFile, + it: ItBlock, + region: Region, + notes: string[], + add: (line: number, msg: string) => void, +): boolean { + const lines = file.text.split('\n'); + for (let l = Math.max(region.endLine, it.startLine); l <= it.endLine; l++) { + const m = ESCAPE.exec(lines[l - 1] ?? ''); + if (!m) continue; + const helper = m[1]; + const callLine = firstCallLine(file, it.fn, helper); + if (callLine === undefined || callLine <= region.endLine) { + add( + l, + `\`lint: asserted-via ${helper}\` names a helper that is not called after the end of "${region.id}" in this test`, + ); + return true; + } + notes.push(`asserted-via ${helper}: ${file.rel}:${l} (${region.id})`); + return true; + } + return false; +} + +function checkAssertion( + file: ScannedFile, + it: ItBlock, + region: Region, + add: (line: number, msg: string) => void, + notes: string[], +): void { + if (assertionAfter(file, it.fn, region.endLine) !== undefined) return; + if (escapeOf(file, it, region, notes, add)) return; + add( + region.endLine, + `no assertion after snippet "${region.id}" in "${it.title}" (assert*, rejects*, expect*, eventually() after line ${region.endLine}; or \`// lint: asserted-via \`)`, + ); +} + +export function lintTier(scan: TierScan = scanTier()): LintResult { + const errors: Issue[] = []; + const warnings: Issue[] = []; + const notes: string[] = []; + const usedAllow = new Set(); + const seen = new Map(); + + for (const file of scan.files) { + errors.push(...file.problems); + warnings.push(...file.warnings); + const mine = scan.snippets.filter((s) => s.file === file.rel); + for (const snippet of mine) { + const { region } = snippet; + const add = (line: number, message: string): void => void errors.push({ file: file.rel, line, message }); + const warn = (line: number, message: string): void => void warnings.push({ file: file.rel, line, message }); + // M4 + const previous = seen.get(region.id); + if (previous) + add( + region.startLine, + `duplicate snippet id "${region.id}" (also at ${previous}): the docs address a snippet by file and id, keep one tested copy`, + ); + else seen.set(region.id, `${file.rel}:${region.startLine}`); + checkRegion(file, snippet, region, add, warn, notes, usedAllow); + } + } + // Allow-list hygiene: an entry whose snippet exists but has no try/console.error any more. + const present = new Set(scan.snippets.map((s) => s.id)); + for (const id of ERROR_HANDLING_SNIPPETS) { + if (present.has(id) && !usedAllow.has(id)) { + warnings.push({ + file: 'support/lint-markers.ts', + line: 0, + message: `ERROR_HANDLING_SNIPPETS lists "${id}" but its visible text has no try/catch or console.error: drop it from the list`, + }); + } + } + const order = (a: Issue, b: Issue): number => (a.file === b.file ? a.line - b.line : a.file < b.file ? -1 : 1); + errors.sort(order); + warnings.sort(order); + return { errors, warnings, notes, files: scan.files.length, snippets: scan.snippets.length, scan }; +} + +export function printLint(result: LintResult): void { + for (const w of result.warnings) console.warn(`warning ${w.file}:${w.line} ${w.message}`); + for (const n of result.notes) console.log(`note: ${n}`); + if (result.errors.length === 0) { + console.log( + `marker lint ok: ${result.files} files, ${result.snippets} snippets, ${result.warnings.length} warning(s).`, + ); + return; + } + for (const e of result.errors) console.error(`${e.file}:${e.line} ${e.message}`); + console.error( + `marker lint failed: ${result.errors.length} problem(s) in ${result.files} files (${result.snippets} snippets).`, + ); +} + +if (require.main === module) { + const result = lintTier(); + printLint(result); + process.exitCode = result.errors.length === 0 ? 0 : 1; +} diff --git a/test/integration/live/support/lint-selftest.ts b/test/integration/live/support/lint-selftest.ts new file mode 100644 index 000000000..1cc1db900 --- /dev/null +++ b/test/integration/live/support/lint-selftest.ts @@ -0,0 +1,250 @@ +/** + * Self-test of the marker lint: writes small broken files into a temporary directory outside the repository, lints them and + * checks that each defect is reported (and that a clean file is not). `npx tsx test/integration/live/support/lint-selftest.ts` + * Exit code 1 when a defect goes unreported. + */ +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { lintTier } from './lint-markers'; +import { buildManifest } from './manifest'; +import { scanTier } from './snippet-scan'; + +const GOOD = `import assert from 'node:assert/strict'; +describe('d', () => { + it('good: ok', async () => { + // snippet.good + const x = await Promise.resolve(1); + // snippet.end + assert.equal(x, 1); + }); +}); +`; + +const CASES: Array<{ name: string; source: string; expect: RegExp[] }> = [ + { name: 'clean', source: GOOD, expect: [] }, + { + name: 'nested start, no end, hide without show, loose marker', + source: `describe('d', () => { it('a', () => { + // snippet.outer + const a = 1; + // snippet.inner + const b = 2; + // snippet.hide + const c = 3; + // snippet.end + //snippet.nospace + // snippet.bad.id + // snippet.open + const d = 4; + assert.ok(a + b + c + d); +}); });`, + expect: [ + /"inner" starts inside snippet "outer"/, + /no snippet\.show/, + /needs a space/, + /characters outside/, + /"open" has no snippet\.end/, + ], + }, + { + name: 'hidden block too long, empty snippet', + source: `describe('d', () => { it('a', () => { + // snippet.big + const a = 1; + // snippet.hide + const b = 1; + const c = 1; + const d = 1; + const e = 1; + // snippet.show + // snippet.end + // snippet.empty + // snippet.end + assert.ok(a); +}); });`, + expect: [/hidden block in "big" has 4 lines/, /snippet "empty" is empty/], + }, + { + name: 'test-only code, try/catch, console.error, return', + source: `import { observer } from '../support/clients'; +import assert from 'node:assert/strict'; +describe('d', () => { it('a', async () => { + // snippet.dirty + assert.ok(true); + const p = observer(); + try { await p.time(); } catch (e) { console.error(e); } + if (p) return; + // snippet.end + assert.ok(p); +}); });`, + expect: [ + /test-only code .*`assert`/, + /test-only code .*`observer`/, + /try\/catch inside snippet "dirty"/, + /console\.error inside snippet "dirty"/, + /return inside snippet "dirty"/, + ], + }, + { + name: 'no assertion after the snippet, assertion only before', + source: `import assert from 'node:assert/strict'; +describe('d', () => { it('late', async () => { + assert.ok(true); + // snippet.late + const a = 1; + // snippet.end + console.log(a); +}); });`, + expect: [/no assertion after snippet "late"/], + }, + { + name: 'top level non-import snippet, snippet outside any it, skipped test', + source: `// snippet.topLevel +const a = 1; +// snippet.end +describe('d', () => { it.skip('s', () => { + // snippet.skipped + const b = 2; + // snippet.end + assert.ok(b); +}); });`, + expect: [/"topLevel" is neither inside one it\(\)/, /sits in a skipped test/], + }, + { + name: 'escape comment: valid and invalid', + source: `describe('d', () => { it('esc', async () => { + // snippet.esc + const a = 1; + // snippet.end + // lint: asserted-via checkIt + await checkIt(a); +}); +it('esc2', async () => { + // snippet.esc2 + const a = 1; + // snippet.end + // lint: asserted-via neverCalled +}); });`, + expect: [/"esc2"|asserted-via neverCalled/], + }, + { + name: 'asserted-by escape: valid is clean', + source: `import fs from 'node:fs'; +// snippet.fileLevel +import path from 'node:path'; +function load() { return path.sep + fs.existsSync('x'); } +// snippet.end +// lint: asserted-by reads it back +describe('d', () => { it('fileLevel: reads it back', () => { assert.ok(load()); }); });`, + expect: [], + }, + { + name: 'asserted-by escape: no match, ambiguous, no assertion, missing', + source: `// snippet.noMatch +function a() { return 1; } +// snippet.end +// lint: asserted-by nothing like this +// snippet.ambiguous +function b() { return 1; } +// snippet.end +// lint: asserted-by same +// snippet.noAssert +function c() { return 1; } +// snippet.end +// lint: asserted-by quiet +// snippet.missing +function d() { return 1; } +// snippet.end + +describe('d', () => { + it('same one', () => { assert.ok(a()); }); + it('same two', () => { assert.ok(b()); }); + it('quiet test', () => { c(); }); +});`, + expect: [ + /asserted-by nothing like this.* matches 0/, + /asserted-by same.* matches 2/, + /asserted-by quiet.*has no assertion call/, + /"missing" is neither inside one it\(\)/, + ], + }, + { + name: 'helper called by it, no assertion', + source: `describe('d', () => { + const run = async (): Promise => { + // snippet.helperSnippet + const a = 1; + // snippet.end + return a; + }; + it('h', async () => { await run(); }); +});`, + expect: [/runs in helper run\(\) but no assertion follows/], + }, + { + name: 'slash id is a warning only', + source: GOOD.replace('snippet.good', 'snippet.good/2'), + expect: [], + }, +]; + +let failed = 0; +const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'pn-lint-selftest-')); +try { + for (const c of CASES) { + const abs = path.join(dir, `${c.name.replace(/\W+/g, '-')}.test.ts`); + fs.writeFileSync(abs, c.source); + const result = lintTier(scanTier([{ abs, kind: 'test' }])); + const messages = result.errors.map((e) => `${e.line}: ${e.message}`); + const missing = c.expect.filter((re) => !messages.some((m) => re.test(m))); + const unexpected = c.expect.length === 0 && result.errors.length > 0; + const warned = c.name.startsWith('slash') ? result.warnings.some((w) => /contains "\/"/.test(w.message)) : true; + if (missing.length > 0 || unexpected || !warned) { + failed += 1; + console.error( + `FAIL ${c.name}: missing ${missing.map(String).join(' ')} ${unexpected ? `unexpected: ${messages.join(' | ')}` : ''} ${warned ? '' : 'no slash warning'}`, + ); + } else console.log(`ok ${c.name} (${result.errors.length} finding(s))`); + } + // Cross-file duplicate. + const a = path.join(dir, 'dup-a.test.ts'); + const b = path.join(dir, 'dup-b.test.ts'); + fs.writeFileSync(a, GOOD); + fs.writeFileSync(b, GOOD); + const dup = lintTier( + scanTier([ + { abs: a, kind: 'test' }, + { abs: b, kind: 'test' }, + ]), + ); + if (dup.errors.some((e) => /duplicate snippet id "good"/.test(e.message))) + console.log('ok duplicate id across files'); + else { + failed += 1; + console.error('FAIL duplicate id across files'); + } + // The valid escape is recorded as live with the named test. + const e = path.join(dir, 'escape.test.ts'); + fs.writeFileSync(e, CASES.find((c) => c.name === 'asserted-by escape: valid is clean')!.source); + const esc = buildManifest(scanTier([{ abs: e, kind: 'test' }])).manifest.snippets.fileLevel; + if (esc?.mode === 'live' && esc.test === 'd fileLevel: reads it back') + console.log('ok asserted-by is recorded as live with the test title'); + else { + failed += 1; + console.error(`FAIL asserted-by manifest entry: ${JSON.stringify(esc)}`); + } + // A file of the mocked allow-list: same rules, mode mocked. + const m = path.join(dir, 'mocked.test.ts'); + fs.writeFileSync(m, GOOD); + const mocked = buildManifest(scanTier([{ abs: m, kind: 'mocked' }])).manifest.snippets.good; + if (mocked?.mode === 'mocked' && mocked.test === 'd good: ok') console.log('ok mocked mode and full test title'); + else { + failed += 1; + console.error(`FAIL mocked mode: ${JSON.stringify(mocked)}`); + } +} finally { + fs.rmSync(dir, { recursive: true, force: true }); +} +console.log(failed === 0 ? 'lint self-test passed' : `lint self-test FAILED (${failed})`); +process.exitCode = failed === 0 ? 0 : 1; diff --git a/test/integration/live/support/manifest.ts b/test/integration/live/support/manifest.ts new file mode 100644 index 000000000..3c2afd552 --- /dev/null +++ b/test/integration/live/support/manifest.ts @@ -0,0 +1,199 @@ +/** + * Coverage manifest of the live tier: `test/integration/live/coverage.json`, generated, one entry per snippet id. + * + * npm run test:integration:live:manifest writes coverage.json + * npm run test:integration:live:manifest -- --check fails when coverage.json is stale (used by the lint script) + * + * mode: + * live the snippet sits inside an it() of the live tier (or in a helper such an it() calls) + * compile-only import-only snippet at the top of a test file: type-checked and loaded with the file, not run as an API call + * mocked the snippet sits inside an it() of a nock-mocked test listed in MOCKED_FILES (snippet-scan.ts) + * `files` lists whole-file samples the docs embed without snippet ids (for example quickstart-node.mjs) that a test runs. + * Things the docs embed that are deliberately not run are in snippet-exceptions.json (hand maintained). + */ +import fs from 'node:fs'; +import path from 'node:path'; +import { LIVE_DIR, scanTier, type Mode, type TierScan } from './snippet-scan'; + +export const COVERAGE_FILE = path.join(LIVE_DIR, 'coverage.json'); +export const EXCEPTIONS_FILE = path.join(LIVE_DIR, 'snippet-exceptions.json'); + +export interface SnippetEntry { + /** Repository relative path of the file that holds the markers. */ + file: string; + /** Full Mocha title (describe + it) of the test that runs it. Absent for compile-only. */ + test?: string; + /** Keyset roles the test uses, derived from createClient/keysOf/server calls (heuristic). Absent when not derivable. */ + role?: string; + mode: Mode; +} + +export interface FileEntry { + mode: 'live'; + test: string; + testFile: string; +} + +export interface Manifest { + $comment: string; + snippets: Record; + files: Record; +} + +export interface ExceptionEntry { + /** Docs-facing path, as in the EmbeddedCode url (for example docs-snippets/quickstarts/quickstart-web.ts). */ + path: string; + /** Snippet ids. Absent: every embed of the path, whole-file embeds included. */ + ids?: string[]; + reason: string; +} + +export function loadExceptions(): ExceptionEntry[] { + if (!fs.existsSync(EXCEPTIONS_FILE)) return []; + const raw = JSON.parse(fs.readFileSync(EXCEPTIONS_FILE, 'utf8')) as { exceptions?: ExceptionEntry[] }; + return raw.exceptions ?? []; +} + +/** Problems with the exceptions file itself (shape only; the docs cross-check reports unused entries). */ +export function checkExceptions(entries: ExceptionEntry[]): string[] { + const problems: string[] = []; + const seen = new Set(); + entries.forEach((e, i) => { + const where = `snippet-exceptions.json entry ${i + 1} (${e.path ?? '?'})`; + if (typeof e.path !== 'string' || !e.path.startsWith('docs-snippets/')) + problems.push(`${where}: "path" must be a docs-snippets/... path`); + if (typeof e.reason !== 'string' || e.reason.trim().length < 10) problems.push(`${where}: "reason" is required`); + if ( + e.ids !== undefined && + (!Array.isArray(e.ids) || e.ids.length === 0 || e.ids.some((x) => typeof x !== 'string')) + ) { + problems.push(`${where}: "ids" must be a non-empty array of strings, or absent`); + } + for (const id of e.ids ?? ['*']) { + const key = `${e.path}#${id}`; + if (seen.has(key)) problems.push(`${where}: ${id === '*' ? 'path' : `id ${id}`} is listed twice`); + seen.add(key); + } + }); + return problems; +} + +export function buildManifest(scan: TierScan = scanTier()): { manifest: Manifest; problems: string[] } { + const problems: string[] = []; + const snippets: Record = {}; + const origin = new Map(); + for (const s of scan.snippets) { + const where = `${s.file}:${s.line}`; + if (s.mode === 'invalid') { + problems.push( + `${where}: snippet "${s.id}" is not run by any test (see the marker lint), left out of the manifest`, + ); + continue; + } + if (origin.has(s.id)) { + problems.push( + `${where}: duplicate snippet id "${s.id}" (first at ${origin.get(s.id)}), left out of the manifest`, + ); + continue; + } + origin.set(s.id, where); + const entry: SnippetEntry = { file: s.file, mode: s.mode }; + if (s.test !== undefined) entry.test = s.test; + if (s.role !== undefined && s.mode !== 'compile-only') entry.role = s.role; + // Key order: file, test, role, mode (stable, readable). + snippets[s.id] = { + file: entry.file, + ...(entry.test ? { test: entry.test } : {}), + ...(entry.role ? { role: entry.role } : {}), + mode: entry.mode, + }; + } + const files: Record = {}; + for (const sample of scan.samples) { + if (sample.test) files[sample.file] = { mode: 'live', test: sample.test.title, testFile: sample.test.file }; + } + const sortKeys = (o: Record): Record => + Object.fromEntries( + Object.keys(o) + .sort() + .map((k) => [k, o[k]]), + ); + return { + manifest: { + $comment: + 'GENERATED by npm run test:integration:live:manifest from the markers in test/integration/live (and the mocked allow-list in support/snippet-scan.ts). DO NOT EDIT. One entry per snippet id. Deliberate exceptions live in snippet-exceptions.json.', + snippets: sortKeys(snippets), + files: sortKeys(files), + }, + problems, + }; +} + +export function serialize(manifest: Manifest): string { + return `${JSON.stringify(manifest, null, 2)}\n`; +} + +export function totals(manifest: Manifest): Record { + const t: Record = { live: 0, 'compile-only': 0, mocked: 0 }; + for (const e of Object.values(manifest.snippets)) t[e.mode] += 1; + return t; +} + +/** Reads coverage.json; undefined when absent. docs-check uses it to know what is covered. */ +export function readManifest(): Manifest | undefined { + return fs.existsSync(COVERAGE_FILE) ? (JSON.parse(fs.readFileSync(COVERAGE_FILE, 'utf8')) as Manifest) : undefined; +} + +function diffSummary(oldM: Manifest, newM: Manifest): string[] { + const out: string[] = []; + const ids = new Set([...Object.keys(oldM.snippets ?? {}), ...Object.keys(newM.snippets)]); + for (const id of [...ids].sort()) { + const a = oldM.snippets?.[id]; + const b = newM.snippets[id]; + if (!a) out.push(` + ${id} (${b.file})`); + else if (!b) out.push(` - ${id} (${a.file})`); + else if (JSON.stringify(a) !== JSON.stringify(b)) out.push(` ~ ${id}`); + } + const files = new Set([...Object.keys(oldM.files ?? {}), ...Object.keys(newM.files)]); + for (const f of [...files].sort()) { + if (JSON.stringify(oldM.files?.[f]) !== JSON.stringify(newM.files[f])) out.push(` ~ file ${f}`); + } + return out; +} + +export function main(argv: string[]): number { + const check = argv.includes('--check'); + const { manifest, problems } = buildManifest(); + const exceptionProblems = checkExceptions(loadExceptions()); + const text = serialize(manifest); + const t = totals(manifest); + const summary = `${Object.keys(manifest.snippets).length} snippets (live ${t.live}, compile-only ${t['compile-only']}, mocked ${t.mocked}), ${Object.keys(manifest.files).length} whole-file sample(s)`; + let failed = false; + for (const p of [...problems, ...exceptionProblems]) { + console.error(p); + failed = true; + } + if (check) { + const current = fs.existsSync(COVERAGE_FILE) ? fs.readFileSync(COVERAGE_FILE, 'utf8') : undefined; + if (current === text) console.log(`coverage.json is up to date: ${summary}.`); + else { + failed = true; + console.error('coverage.json is stale. Run: npm run test:integration:live:manifest'); + if (current) { + try { + diffSummary(JSON.parse(current) as Manifest, manifest) + .slice(0, 30) + .forEach((l) => console.error(l)); + } catch { + console.error(' (coverage.json does not parse)'); + } + } else console.error(' (coverage.json is missing)'); + } + } else { + fs.writeFileSync(COVERAGE_FILE, text); + console.log(`wrote ${path.relative(process.cwd(), COVERAGE_FILE)}: ${summary}.`); + } + return failed ? 1 : 0; +} + +if (require.main === module) process.exitCode = main(process.argv.slice(2)); diff --git a/test/integration/live/support/placeholders.ts b/test/integration/live/support/placeholders.ts new file mode 100644 index 000000000..34e88da98 --- /dev/null +++ b/test/integration/live/support/placeholders.ts @@ -0,0 +1,32 @@ +/** + * Lets a docs sample that carries a placeholder literal run with its text unchanged. Examples: a token such as + * 'use-token-string-generated-by-grantToken()', a file id such as '...', a timetoken such as 'replace-with-start-timetoken'. + * + * `withPlaceholders(client, { placeholder: liveValue })` returns the client wrapped so that every string argument equal to a key + * of `values`, also inside arrays and plain-object arguments, is replaced by the live value before the call reaches the SDK. + * Nothing else changes. Only the methods of the client itself are wrapped, not the ones of `client.objects` and the like. + */ +import type PubNub from 'pubnub'; + +function substitute(value: unknown, values: Record): unknown { + if (typeof value === 'string') return Object.prototype.hasOwnProperty.call(values, value) ? values[value] : value; + if (Array.isArray(value)) return value.map((item) => substitute(item, values)); + if (value && typeof value === 'object' && Object.getPrototypeOf(value) === Object.prototype) { + return Object.fromEntries(Object.entries(value).map(([key, item]) => [key, substitute(item, values)])); + } + return value; +} + +export function withPlaceholders(client: T, values: Record): T { + return new Proxy(client, { + get(target, property) { + const member = (target as unknown as Record)[property]; + if (typeof member !== 'function') return member; + return (...args: unknown[]) => + (member as (...a: unknown[]) => unknown).apply( + target, + args.map((arg) => substitute(arg, values)), + ); + }, + }); +} diff --git a/test/integration/live/support/preflight.ts b/test/integration/live/support/preflight.ts new file mode 100644 index 000000000..4a9d89901 --- /dev/null +++ b/test/integration/live/support/preflight.ts @@ -0,0 +1,204 @@ +/** + * Preflight: fails fast, with a message that names the missing keyset feature, before any test runs. The root `before all` hook + * (hooks.ts) runs it after the janitor. + * + * 1. Admin API check (read only, never changes a keyset). With PUBNUB_ADMIN_API_KEY and a keyset id per role + * (DOCS_SNIPPETS_OPEN_KEYSET_ID, DOCS_SNIPPETS_PAM_KEYSET_ID, DOCS_SNIPPETS_DS_KEYSET_ID) it reads GET /v2/keysets/{id} and + * /config, confirms that the keyset of that id holds the role's subscribe key, and names every feature the role needs and lacks. + * Without the key or the id the role is skipped here and only the probes below run. + * 2. Live probes on the open keyset, as the old harness did: keys and network (publish), Message Persistence (fetchMessages), + * Delete-From-History (deleteMessages on an empty range) and, when the Admin API did not vouch for it, Presence. + */ +import PubNub from 'pubnub'; +import { env, redact, ROLES, type Env, type Role } from './env'; +import { describeError, statusCode } from './janitor-util'; + +const BASE = 'https://admin-api.pubnub.com/v2/keysets'; +const API_VERSION = '2026-09-17'; + +/** The part of the keyset configuration the tier needs. Field names as the Admin API returns them. */ +interface KeysetConfig { + accessManager?: { enabled?: boolean; tokenRevoke?: boolean }; + files?: { enabled?: boolean }; + streamController?: { enabled?: boolean; wildcardSubscribe?: boolean }; + appContext?: { + enabled?: boolean; + userMetadataEvents?: boolean; + channelMetadataEvents?: boolean; + membershipEvents?: boolean; + referentialIntegrity?: boolean; + disallowGetAllChannelMetadata?: boolean; + disallowGetAllUserMetadata?: boolean; + }; + dataSync?: { enabled?: boolean }; + messagePersistence?: { enabled?: boolean; deleteFromHistory?: boolean }; + presence?: { enabled?: boolean; rules?: Array<{ pattern?: string; tracking?: { enabled?: boolean } }> }; +} + +type FeatureRead = + | { status: 'skipped'; why: string } + | { status: 'mismatch' | 'error'; why: string } + | { status: 'read'; config: KeysetConfig }; + +async function get(url: string, apiKey: string): Promise<{ status: number; json: unknown }> { + const response = await fetch(url, { headers: { Authorization: apiKey, 'PubNub-Version': API_VERSION } }); + const text = await response.text(); + try { + return { status: response.status, json: text ? JSON.parse(text) : {} }; + } catch { + return { status: response.status, json: {} }; + } +} + +/** Reads a role's keyset configuration. Never returns or logs key material. */ +export async function readFeatures(environment: Env, role: Role): Promise { + const keys = environment[role]; + const id = environment.keysetIds[role]; + if (!environment.adminKey) return { status: 'skipped', why: 'PUBNUB_ADMIN_API_KEY is not set' }; + if (!id) return { status: 'skipped', why: `no keyset id for role ${role}` }; + if (!keys) return { status: 'skipped', why: `role ${role} has no keys` }; + const keyset = await get(`${BASE}/${encodeURIComponent(id)}`, environment.adminKey); + if (keyset.status !== 200) return { status: 'error', why: `reading keyset ${id} answered HTTP ${keyset.status}` }; + // The only comparison of key material in this module. Only the boolean leaves it. + if ((keyset.json as { subscribeKey?: unknown }).subscribeKey !== keys.subscribeKey) + return { status: 'mismatch', why: `keyset ${id} does not hold the ${role} subscribe key` }; + const config = await get(`${BASE}/${encodeURIComponent(id)}/config`, environment.adminKey); + if (config.status !== 200) + return { status: 'error', why: `reading the configuration of keyset ${id} answered HTTP ${config.status}` }; + return { status: 'read', config: config.json as KeysetConfig }; +} + +/** Features a role needs (docs-snippets/README.md, "Features each keyset needs"). One sentence per missing feature. */ +export function missingFeatures(role: Role, c: KeysetConfig): string[] { + const out: string[] = []; + const on = (value: boolean | undefined, text: string): void => { + if (value !== true) out.push(`${text} is off on the ${role} keyset`); + }; + if (role === 'open') { + on(c.presence?.enabled, 'Presence'); + on( + c.presence?.rules?.some((r) => r.pattern === '*' && r.tracking?.enabled === true), + 'Presence tracking on all channels (rule pattern *)', + ); + on(c.messagePersistence?.enabled, 'Message Persistence'); + on(c.messagePersistence?.deleteFromHistory, 'Message Persistence Delete-From-History'); + on(c.appContext?.enabled, 'App Context'); + on(c.appContext?.userMetadataEvents, 'App Context user metadata events'); + on(c.appContext?.channelMetadataEvents, 'App Context channel metadata events'); + on(c.appContext?.membershipEvents, 'App Context membership events'); + if (c.appContext?.referentialIntegrity === true) + out.push('App Context referential integrity is on on the open keyset (it must be off)'); + if (c.appContext?.disallowGetAllUserMetadata === true || c.appContext?.disallowGetAllChannelMetadata === true) + out.push('App Context "disallow get all" is on on the open keyset (it must be off)'); + on(c.files?.enabled, 'Files'); + on(c.streamController?.enabled, 'Stream Controller'); + on(c.streamController?.wildcardSubscribe, 'Stream Controller wildcard subscribe'); + if (c.accessManager?.enabled === true) out.push('Access Manager is on on the open keyset (it must be off)'); + } else { + on(c.accessManager?.enabled, 'Access Manager'); + if (role === 'pam') on(c.accessManager?.tokenRevoke, 'Access Manager token revocation'); + on(c.messagePersistence?.enabled, 'Message Persistence'); + on(c.presence?.enabled, 'Presence'); + if (role === 'datasync') on(c.dataSync?.enabled, 'DataSync'); + } + return out; +} + +const sleep = (ms: number): Promise => new Promise((resolve) => setTimeout(resolve, ms)); + +/** Live probes on the open keyset. Returns one sentence per failed probe; a failed publish stops the rest (keys or network). */ +async function probeOpenKeyset(environment: Env, presenceVouched: boolean): Promise { + const keys = environment.open; + if (!keys) return []; + const problems: string[] = []; + const client = new PubNub({ + ...keys, + userId: 'preflight-observer', + logLevel: PubNub.LogLevel.None, + heartbeatInterval: 0, + }); + const channel = 'preflight_channel'; + const probe = async (feature: string, setting: string, call: () => Promise): Promise => { + try { + await call(); + return true; + } catch (error) { + problems.push( + `${feature} does not work on the open keyset (enable "${setting}" in the Admin Portal): ${describeError(error)}`, + ); + return false; + } + }; + const subscriber = presenceVouched + ? undefined + : new PubNub({ ...keys, userId: 'preflight-presence', logLevel: PubNub.LogLevel.None, heartbeatInterval: 0 }); + try { + if ( + !(await probe('Publish (working keys and network)', 'a publish and subscribe key pair', () => + client.publish({ channel, message: { preflight: true } }), + )) + ) + return problems; + await probe('Message Persistence', 'Message Persistence', () => + client.fetchMessages({ channels: [channel], count: 1 }), + ); + // Deleting an empty range has no side effect. 403 means the setting is off. + await probe('Delete-From-History', 'Message Persistence > Delete-From-History', async () => { + const now = (await client.time()).timetoken; + await client.deleteMessages({ channel, start: (BigInt(now) - 1n).toString(), end: now }); + }); + if (subscriber) { + subscriber.subscribe({ channels: [channel] }); + let seen = false; + for (let attempt = 0; attempt < 10 && !seen; attempt += 1) { + await sleep(1_500); + try { + const here = await client.hereNow({ channels: [channel] }); + seen = Boolean(here.channels[channel]?.occupants.some((o) => o.uuid === 'preflight-presence')); + } catch (error) { + if (statusCode(error) === 403) break; + } + } + if (!seen) + problems.push( + 'Presence does not work on the open keyset: hereNow does not list a subscribed client (enable "Presence")', + ); + } + } finally { + subscriber?.destroy(); + client.destroy(); + } + return problems; +} + +/** Throws one error that lists every problem. `notes` returns the roles the Admin API could not check. */ +export async function runPreflight(environment: Env = env()): Promise<{ notes: string[] }> { + const problems: string[] = []; + const notes: string[] = []; + let openVouched = false; + for (const role of ROLES) { + if (!environment[role]) continue; + const read = await readFeatures(environment, role).catch((error: unknown) => ({ + status: 'error' as const, + why: describeError(error), + })); + if (read.status === 'read') { + problems.push(...missingFeatures(role, read.config)); + if (role === 'open') openVouched = read.config.presence?.enabled === true; + } else if (read.status === 'skipped') { + notes.push(`${role}: Admin API feature check skipped (${read.why})`); + } else { + problems.push(`${role}: Admin API feature check failed: ${read.why}`); + } + } + problems.push(...(await probeOpenKeyset(environment, openVouched))); + if (problems.length > 0) { + throw new Error( + redact( + `Keyset preflight failed. Fix these, then run again:\n${problems.map((p) => ` - ${p}`).join('\n')}\n` + + 'The features each keyset needs are listed in docs-snippets/README.md ("Features each keyset needs").', + ), + ); + } + return { notes }; +} diff --git a/test/integration/live/support/proxy.ts b/test/integration/live/support/proxy.ts new file mode 100644 index 000000000..68ff68b63 --- /dev/null +++ b/test/integration/live/support/proxy.ts @@ -0,0 +1,75 @@ +import http from 'node:http'; +import net from 'node:net'; +import type { AddressInfo } from 'node:net'; + +/** + * Local HTTP CONNECT proxy on 127.0.0.1 (`setProxy`). It tunnels to the real host and + * counts CONNECT requests, so a step can prove traffic went through the proxy and stopped after `setProxy()`. + * `outage(true)` refuses new tunnels and destroys open ones, for the later status-event steps. + */ +// An HTTP CONNECT proxy is enough because the SDK talks TLS to the service. The proxy never sees plaintext, it only pipes bytes. +// `http.createServer` rejects plain requests with 405 for the same reason. +export class LocalProxy { + private server?: http.Server; + private sockets = new Set(); + private blocked = false; + connects = 0; + port = 0; + readonly host = '127.0.0.1'; + + /** Listens on an ephemeral port (`listen(0)`), so parallel runs never collide. Idempotent. */ + async start(): Promise { + if (this.server) return; + this.server = http.createServer((_req, res) => { + res.writeHead(405).end('CONNECT only'); + }); + this.server.on('connect', (req, clientSocket: net.Socket, head) => { + this.connects += 1; + if (this.blocked) { + clientSocket.end('HTTP/1.1 503 Service Unavailable\r\n\r\n'); + return; + } + const [hostname, port] = (req.url ?? '').split(':'); + const upstream = net.connect(Number(port) || 443, hostname, () => { + clientSocket.write('HTTP/1.1 200 Connection Established\r\n\r\n'); + upstream.write(head); + upstream.pipe(clientSocket); + clientSocket.pipe(upstream); + }); + // Track both ends so `outage(true)` and `stop()` can cut established tunnels, which a refused new CONNECT would not. + this.sockets.add(clientSocket); + this.sockets.add(upstream); + const drop = (): void => { + this.sockets.delete(clientSocket); + this.sockets.delete(upstream); + clientSocket.destroy(); + upstream.destroy(); + }; + upstream.on('error', drop); + clientSocket.on('error', drop); + upstream.on('close', drop); + clientSocket.on('close', drop); + }); + await new Promise((resolve) => this.server!.listen(0, this.host, resolve)); + this.port = (this.server.address() as AddressInfo).port; + } + + /** Spread into `setProxy({...})` to point a snippet at this proxy. */ + get config(): Partial<{ hostname: string; port: number; protocol: string }> { + return { hostname: this.host, port: this.port, protocol: 'http' }; + } + + /** Simulates a network outage: refuses new tunnels with 503 and cuts the open ones, so the SDK reports an unexpected disconnect. */ + outage(on: boolean): void { + this.blocked = on; + if (on) for (const socket of this.sockets) socket.destroy(); + } + + async stop(): Promise { + for (const socket of this.sockets) socket.destroy(); + this.sockets.clear(); + const server = this.server; + this.server = undefined; + if (server) await new Promise((resolve) => server.close(() => resolve())); + } +} diff --git a/test/integration/live/support/resources.ts b/test/integration/live/support/resources.ts new file mode 100644 index 000000000..d259086ce --- /dev/null +++ b/test/integration/live/support/resources.ts @@ -0,0 +1,218 @@ +/** + * The literal resources the live tier uses, per keyset role. DATA ONLY. + * + * The tests use the names the docs show, so these names are fixed, shared and persistent. The janitor (janitor.ts) cleans exactly + * what is listed here before and after every run, and nothing else. It never lists a keyset and deletes what it finds. + * + * To add a name: put it in the right list of the right role below (one string, no logic). A name that a test creates but that is + * not listed here is never cleaned and leaks from run to run. Seeded from docs-snippets/fixtures/names.ts and from the + * literals of the old page modules (snippet regions and `world.n(...)`). + * + * Which list does a name belong to? + * channels every channel a test publishes to or reads history from (history is deleted on these, and on `fileChannels`) + * fileChannels channels a test uploads files to (all files are deleted, then history) + * channelGroups channel groups (deleted as a whole; the member channels are NOT touched, list them under `channels`) + * appContext users and channels stored as App Context metadata, and their memberships (removed when both sides are listed) + * pushDevices device tokens registered for push (fcm, apns, apns2 production and development are all cleared) + * dataSync DataSync object ids by kind (the datasync keyset only; classes are never touched) + */ +import type { Role } from './env'; + +export interface DataSyncResources { + users: readonly string[]; + channels: readonly string[]; + /** Entities of the `product` class. Ids may contain dots. */ + entities: readonly string[]; + /** Membership ids. A membership between a listed user and a listed channel is removed even when its id is not listed. */ + memberships: readonly string[]; + /** `ProductOwner` relationship ids. Relationships between a listed user and a listed entity are removed even when not listed. */ + relationships: readonly string[]; +} + +export interface RoleResources { + channels?: readonly string[]; + fileChannels?: readonly string[]; + channelGroups?: readonly string[]; + appContext?: { users?: readonly string[]; channels?: readonly string[] }; + pushDevices?: readonly string[]; + dataSync?: Partial; +} + +/** A realistic 64-character hex device token (the shape of an APNs token), the one the push samples show. */ +export const DEVICE_TOKEN_EXAMPLE = '740f4707bebcf74f9b7c25d48e3358945f6aa01da5ddb387462c7eaf61bb78ad'; + +/** Groups the legacy subscribe page fills with one `_member` channel each. */ +const MEMBER_GROUPS = [ + 'my_channelGroup', + 'family', + 'my_channelGroup1', + 'my_channelGroup2', + 'my_channelGroup3', + 'demo_group1', + 'demo_group2', + 'cg1', + 'channelGroup_1', +]; + +/** + * The literal DataSync ids the page api-reference/datasync.test.ts creates, by kind. The page's own hooks + * (support/datasync-state.ts) reset exactly these, and the janitor cleans them too, so both read this one list. + */ +export const DATASYNC_PAGE_OBJECTS = { + relationships: ['rel-bob-owns-sneaker-42'], + memberships: ['membership-alice-summer-sale', 'membership-alice-winter-sale'], + entities: ['product-sneaker-42', 'product.676', 'product.677', 'product.678', 'product.679'], + users: ['user-alice', 'seller-bob'], + channels: ['channel-summer-sale', 'channel-winter-sale'], +} as const; + +export const RESOURCES: Readonly> = { + open: { + // Deliberately NOT listed: `history_documented_behavior` (api-reference/message-persistence-legacy). Its test needs more than + // 100 stored messages, writes about 105 once (5 batches of 21 publishes, then waits for the service to show them) and + // reuses them. Listing it would make the janitor wipe it before every run and the test refill it every run. It cannot + // grow: the test publishes only while the channel holds fewer than 100 messages. The assertions look at the cap, the order + // and the field types of whatever is stored, not at the content. Names that tests generate per run (a suffix from + // randomUUID or Date.now, such as `misc_disconnect_`) cannot be listed either; their messages expire with the + // retention of the keyset. + channels: [ + 'my_channel', + 'my-channel', + 'my_channel_1', + 'my_channel_2', + 'my_channel_3', + 'ch1', + 'ch2', + 'ch3', + 'ch4', + 'ch-1', + 'ch-2', + 'channel_1', + 'channel_3', + 'channel_name', + 'chan1', + 'chan2', + 'chan3', + 'chats.room1', + 'chats.room2', + 'team.blue', + 'team.red', + 'channel-with-status-type', + 'foo', + 'son', + 'daughter', + 'a', + 'b', + 'myChannel', + 'awesomeChannel', + 'history_channel', + 'history_test', + 'hello_world', + 'chat-bot', + 'ab.news', + 'outage_channel', + 'fire_replacement', + 'legacy_ivs', + 'legacy_cipher', + 'legacy_listeners_channel', + 'persist_channel', + 'filter_channel', + 'preflight_channel', + 'push_marker', + // Use cases (use-cases/*.test.ts), literal channel names of the sports example. + 'game.chat', + 'game.chat.shard-0', + 'game.chat.shard-1', + 'game.chat.shard-2', + 'game.match-stats.score', + 'game.match-stats.possession', + 'game.match-stats.shots', + 'game.match-stats.cards', + 'game.new-poll', + 'game.poll-votes', + 'game.poll-results', + 'game.poll-triggers', + 'game.score-alerts', + 'game.commentary', + 'game.stream', + 'game.stream-reactions', + 'game.ad-decisions', + 'game.reaction-upgrades', + 'game.moment-alerts.fan-a', + 'game.moment-alerts.fan-b', + // Members of the channel groups above (`_member`). + ...MEMBER_GROUPS.map((group) => `${group}_member`), + ], + fileChannels: ['my_channel', 'awesomeChannel', 'preflight_channel'], + channelGroups: [ + 'myChannelGroup', + 'family', + 'cg1', + 'my_channel_group', + 'channelGroup_1', + 'my_channelGroup', + 'my_channelGroup1', + 'my_channelGroup2', + 'my_channelGroup3', + 'demo_group1', + 'demo_group2', + ], + appContext: { + // Users are metadata of an App Context user id: the ids the objects samples set, and the clients' own user ids. + users: [ + 'myUuid', + 'my-uuid', + 'uuid', + 'uuid-1', + 'uuid-2', + 'uuid-3', + 'user_meta1', + 'shopper', + 'friend', + 'observer', + 'fan-42', + ], + channels: [ + 'team.blue', + 'team.red', + 'my-channel', + 'my_channel', + 'channel-with-status-type', + 'myChannel', + 'ch-1', + 'ch-2', + 'push_marker', + 'legacy_listeners_channel', + 'game.chat', + ], + }, + pushDevices: [DEVICE_TOKEN_EXAMPLE, 'niceDevice'], + }, + + // Access Manager keyset: the application-level grant is revoked by the janitor (not a list entry). Channels the grant and + // token samples publish to. + pam: { + channels: [ + 'am_room', + 'am_other', + 'my_channel', + 'my-channel', + 'channel-a', + 'channel-b', + 'channel-c', + 'channel-d', + 'game.chat', + 'game.chat.moderation', + ], + }, + + datasync: { + dataSync: { + users: DATASYNC_PAGE_OBJECTS.users, + channels: DATASYNC_PAGE_OBJECTS.channels, + entities: DATASYNC_PAGE_OBJECTS.entities, + memberships: DATASYNC_PAGE_OBJECTS.memberships, + relationships: DATASYNC_PAGE_OBJECTS.relationships, + }, + }, +}; diff --git a/test/integration/live/support/retry.ts b/test/integration/live/support/retry.ts new file mode 100644 index 000000000..dcae7c90f --- /dev/null +++ b/test/integration/live/support/retry.ts @@ -0,0 +1,86 @@ +/** + * Retry for SETUP and CLEANUP code only: seeding, resets, the janitor. It never wraps a snippet, an assertion about a sample's + * effect, or a call whose failure is what the test asserts. The tier runs with Mocha `retries: 0` on purpose. + * + * What counts as transient (shapes from src/errors/pubnub-api-error.ts): + * - a rejected PubNub call is a `PubNubError` whose `status` carries `category` and `statusCode`; a callback style status and + * a janitor `fetch` error carry them directly. HTTP 5xx and 429 are transient, any other code (400, 403, 404, 409, ...) is final. + * - without an HTTP code (statusCode 0 or missing) the category decides: PNNetworkIssuesCategory, PNTimeoutCategory, + * PNServerErrorCategory and PNUnknownCategory (a transport failure the SDK could not classify) are transient. Cancelled, + * bad request, access denied, validation and malformed response are not. + * - a plain Error that is a transport failure (undici "fetch failed", ECONNRESET, ETIMEDOUT, ...) is transient. Anything else is + * not: an AssertionError, a TypeError, a validation error thrown before a request is sent. + * + * A retried write can have reached the service although its answer was lost (a 5xx after the write, a timeout), so a retried + * publish may be stored twice. Callers that count what is stored must tolerate or remove such a duplicate (see seeding in + * api-reference/message-persistence-legacy.test.ts). + */ + +export interface RetryOptions { + /** Total tries including the first. Default 3. */ + attempts?: number; + /** Names the call in the log line. Never put keys, tokens or payloads in it. */ + label?: string; + /** Base of the backoff in milliseconds: base * attempt. Default 500. */ + backoff?: number; +} + +type ErrorLike = { + name?: string; + code?: unknown; + message?: string; + cause?: { code?: unknown }; + statusCode?: number; + category?: string; + status?: { statusCode?: number; category?: string }; +}; + +export function statusCode(error: unknown): number | undefined { + const e = error as ErrorLike | undefined; + return e?.status?.statusCode ?? e?.statusCode; +} + +export function categoryOf(error: unknown): string | undefined { + const e = error as ErrorLike | undefined; + return e?.status?.category ?? e?.category; +} + +const TRANSIENT_CATEGORIES = new Set([ + 'PNNetworkIssuesCategory', + 'PNTimeoutCategory', + 'PNServerErrorCategory', + 'PNUnknownCategory', +]); +const TRANSPORT = + /fetch failed|socket hang up|ECONNRESET|ECONNREFUSED|ETIMEDOUT|ENOTFOUND|EAI_AGAIN|ENETUNREACH|UND_ERR/i; + +export function isTransient(error: unknown): boolean { + const e = error as ErrorLike | undefined; + if (!e || e.name === 'AssertionError' || e.code === 'ERR_ASSERTION') return false; + const code = statusCode(error); + if (code !== undefined && code > 0) return code >= 500 || code === 429; + const category = categoryOf(error); + if (category !== undefined) return TRANSIENT_CATEGORIES.has(category); + return TRANSPORT.test(`${e.message ?? ''} ${String(e.code ?? '')} ${String(e.cause?.code ?? '')}`); +} + +const pause = (ms: number): Promise => new Promise((resolve) => setTimeout(resolve, ms)); + +/** Runs `fn`; on a transient failure waits `backoff * attempt` ms and runs it again, up to `attempts` tries. Logs one line per retry. */ +export async function retryTransient(fn: () => Promise, options: RetryOptions = {}): Promise { + const attempts = options.attempts ?? 3; + const backoff = options.backoff ?? 500; + for (let attempt = 1; ; attempt += 1) { + try { + return await fn(); + } catch (error) { + if (attempt >= attempts || !isTransient(error)) throw error; + const code = statusCode(error); + console.log( + `retryTransient: ${options.label ?? 'call'} failed (attempt ${attempt} of ${attempts}, ` + + `category ${categoryOf(error) ?? 'none'}, status ${code ?? 'none'}), retrying`, + ); + await pause(backoff * attempt); + } + } +} diff --git a/test/integration/live/support/snippet-scan.ts b/test/integration/live/support/snippet-scan.ts new file mode 100644 index 000000000..fe59c4a07 --- /dev/null +++ b/test/integration/live/support/snippet-scan.ts @@ -0,0 +1,541 @@ +/** + * Shared scanner of the snippet tier: finds the files that can hold snippet markers, scans the markers with the exact syntax + * of the docs extractor, and uses the TypeScript compiler API to find the `it()` callback (and the `describe()` chain) that + * runs each snippet. The marker lint (lint-markers.ts), the coverage manifest (manifest.ts) and the docs cross-check + * (docs-check.ts) all build on it. Pure file reads: no network, no keys, nothing is executed. + */ +import fs from 'node:fs'; +import path from 'node:path'; +import ts from 'typescript'; + +export const REPO_ROOT = path.resolve(__dirname, '../../../..'); +export const LIVE_DIR = path.join(REPO_ROOT, 'test/integration/live'); +export const LIVE_REL = 'test/integration/live'; + +/** + * The one place that lists nock-mocked tests outside `live/` which carry snippet markers (mode `mocked`). Repository relative + * paths. Keep it short: a snippet belongs here only when a real server cannot produce what it shows on demand (retry policies + * against a failing server, a heartbeat failure). + */ +export const MOCKED_FILES: readonly string[] = [ + 'test/integration/endpoints/objects/channel.test.ts', + 'test/integration/endpoints/time.test.ts', + 'test/integration/components/subscription_manager.test.ts', +]; + +/** Directories under live/ that never hold samples. */ +const SKIP_DIRS = new Set(['node_modules', 'support', 'provision', 'fixtures', '.run']); +const SAMPLE_EXT = /\.(mjs|cjs|js|jsx|tsx|ts|html)$/; + +export type Mode = 'live' | 'compile-only' | 'mocked'; + +export interface Issue { + file: string; + line: number; + message: string; +} + +export interface HiddenBlock { + hideLine: number; + showLine?: number; +} + +export interface Region { + id: string; + startLine: number; + /** Line of `snippet.end`, or of the marker that closed the region implicitly. */ + endLine: number; + explicitEnd: boolean; + hidden: HiddenBlock[]; +} + +export interface ItBlock { + /** Full Mocha title: the describe chain and the it title, joined by a space (Mocha's `fullTitle()`). */ + fullTitle: string; + title: string; + startLine: number; + endLine: number; + fn: ts.FunctionLikeDeclaration; + node: ts.CallExpression; + /** `it.skip`, `xit` and the like. */ + skipped: boolean; +} + +export interface ScannedFile { + /** Repository relative path with forward slashes. */ + rel: string; + abs: string; + kind: 'test' | 'sample' | 'mocked'; + text: string; + source: ts.SourceFile; + regions: Region[]; + /** Marker syntax problems (errors) found while scanning. */ + problems: Issue[]; + /** Warnings from the scan (the slash id). */ + warnings: Issue[]; + its: ItBlock[]; + lineOf: (pos: number) => number; +} + +/** Where a region runs, resolved from the AST. */ +export type Owner = + | { type: 'it'; it: ItBlock } + | { type: 'helper'; name: string; fn: ts.FunctionLikeDeclaration } + | { type: 'import-only' } + /** A file-level sample that a named `it()` asserts: `// lint: asserted-by ` on the line after `snippet.end`. */ + | { type: 'file-level'; it: ItBlock; needle: string; line: number } + | { type: 'file-level-error'; message: string; line: number } + | { type: 'none'; reason: string }; + +// The docs extractor matches `// ` (with the space, because its default prefix is "// "), then optional blanks, then +// `snippet.`. These are the same shapes. +const MARKER = /^\s*\/\/ \s*snippet\.([A-Za-z0-9_/-]+)\s*$/; +// Anything that starts like a marker: used to report near misses the extractor would silently ignore. +const ASSERTED_BY = /^\s*\/\/\s*lint:\s*asserted-by\s+(\S.*?)\s*$/; +const LOOSE = /^\s*\/\/\s*(?:snippet\.|end\.snippet)/; +const LOOSE_TOKEN = /^\s*\/\/\s*snippet\.(\S+)\s*$/; + +function toRel(abs: string): string { + return path.relative(REPO_ROOT, abs).split(path.sep).join('/'); +} + +function walk(dir: string): string[] { + const out: string[] = []; + for (const entry of fs.readdirSync(dir, { withFileTypes: true })) { + if (entry.name.startsWith('.')) continue; + const full = path.join(dir, entry.name); + if (entry.isDirectory()) { + if (!SKIP_DIRS.has(entry.name)) out.push(...walk(full)); + } else out.push(full); + } + return out.sort(); +} + +/** Every file that can hold markers: `*.test.ts` and sample files under live/, plus the mocked allow-list. Sorted. */ +export function listFiles(): Array<{ abs: string; kind: ScannedFile['kind'] }> { + const files: Array<{ abs: string; kind: ScannedFile['kind'] }> = []; + for (const abs of walk(LIVE_DIR)) { + if (/\.test\.ts$/.test(abs)) files.push({ abs, kind: 'test' }); + else if (SAMPLE_EXT.test(abs) && !/\.d\.ts$/.test(abs)) files.push({ abs, kind: 'sample' }); + } + for (const rel of MOCKED_FILES) files.push({ abs: path.join(REPO_ROOT, rel), kind: 'mocked' }); + return files; +} + +export function scanMarkers(rel: string, text: string): { regions: Region[]; problems: Issue[]; warnings: Issue[] } { + const regions: Region[] = []; + const problems: Issue[] = []; + const warnings: Issue[] = []; + let open: Region | undefined; + let openHide: HiddenBlock | undefined; + const bad = (line: number, message: string): void => void problems.push({ file: rel, line, message }); + const close = (line: number, explicit: boolean): void => { + if (!open) return; + open.endLine = line; + open.explicitEnd = explicit; + if (openHide) { + bad(openHide.hideLine, `snippet.hide in "${open.id}" has no snippet.show`); + openHide = undefined; + } + regions.push(open); + open = undefined; + }; + const lines = text.split('\n'); + lines.forEach((raw, index) => { + const line = index + 1; + const match = MARKER.exec(raw); + if (!match) { + if (LOOSE.test(raw)) { + const token = LOOSE_TOKEN.exec(raw)?.[1]; + if (/^\s*\/\/[^ ]/.test(raw)) { + bad( + line, + `marker needs a space after the slashes ("// snippet.…"); the docs extractor ignores this line: ${raw.trim()}`, + ); + } else if (token !== undefined && !/^[A-Za-z0-9_-]+$/.test(token)) { + bad( + line, + `snippet id "${token}" has characters outside [a-zA-Z0-9_-]; the docs extractor does not see it as a marker`, + ); + } else { + bad(line, `marker-like comment is not a valid marker (the docs extractor ignores it): ${raw.trim()}`); + } + } + return; + } + const word = match[1]; + if (word === 'end') { + if (!open) bad(line, 'snippet.end without an open snippet'); + else close(line, true); + } else if (word === 'hide') { + if (!open) bad(line, 'snippet.hide outside a snippet'); + else if (openHide) bad(line, 'snippet.hide inside a hidden block'); + else { + openHide = { hideLine: line }; + open.hidden.push(openHide); + } + } else if (word === 'show') { + if (!open || !openHide) bad(line, 'snippet.show without a matching snippet.hide'); + else { + openHide.showLine = line; + openHide = undefined; + } + } else { + const slash = word.includes('/'); + if (slash) { + warnings.push({ + file: rel, + line, + message: + `snippet id "${word}" contains "/": the docs extractor finds it as a start marker (it matches the requested id ` + + 'literally) but its implicit-end regex [a-zA-Z0-9_-]+ does not recognise it, so it never ends the snippet before it ' + + 'and, if the marker sits inside an open snippet, its line becomes part of that snippet text. Works only while the ' + + 'previous snippet has an explicit snippet.end. Rename the id in the docs when convenient.', + }); + } + if (open) { + bad( + line, + slash + ? `snippet "${word}" starts inside snippet "${open.id}": the marker line would show up in the text of "${open.id}"` + : `snippet "${word}" starts inside snippet "${open.id}" (the docs extractor silently cuts "${open.id}" here)`, + ); + close(line, false); + } + open = { id: word, startLine: line, endLine: line, explicitEnd: false, hidden: [] }; + } + }); + if (open) { + const dangling: Region = open; + bad( + dangling.startLine, + `snippet "${dangling.id}" has no snippet.end (the docs extractor would run to the end of the file)`, + ); + close(lines.length + 1, false); + } + return { regions, problems, warnings }; +} + +const DESCRIBE = new Set(['describe', 'context', 'suite']); +const IT = new Set(['it', 'specify', 'test']); + +function calleeParts(call: ts.CallExpression): { root: string; member?: string } | undefined { + const e = call.expression; + if (ts.isIdentifier(e)) return { root: e.text }; + if (ts.isPropertyAccessExpression(e) && ts.isIdentifier(e.expression)) + return { root: e.expression.text, member: e.name.text }; + return undefined; +} + +function titleOf(call: ts.CallExpression, source: ts.SourceFile): string | undefined { + const first = call.arguments[0]; + if (!first) return undefined; + if (ts.isStringLiteral(first) || ts.isNoSubstitutionTemplateLiteral(first)) return first.text; + if (ts.isTemplateExpression(first)) return first.getText(source); + return undefined; +} + +function callbackOf(call: ts.CallExpression): ts.FunctionLikeDeclaration | undefined { + const fn = [...call.arguments].reverse().find((a) => ts.isArrowFunction(a) || ts.isFunctionExpression(a)); + return fn as ts.FunctionLikeDeclaration | undefined; +} + +export function scanFile(abs: string, kind: ScannedFile['kind']): ScannedFile { + const rel = toRel(abs); + const text = fs.readFileSync(abs, 'utf8'); + const scriptKind = /\.(mjs|cjs|js)$/.test(abs) + ? ts.ScriptKind.JS + : /\.jsx$/.test(abs) + ? ts.ScriptKind.JSX + : /\.tsx$/.test(abs) + ? ts.ScriptKind.TSX + : ts.ScriptKind.TS; + const source = ts.createSourceFile(abs, text, ts.ScriptTarget.ES2022, true, scriptKind); + const lineOf = (pos: number): number => source.getLineAndCharacterOfPosition(pos).line + 1; + const { regions, problems, warnings } = scanMarkers(rel, text); + const its: ItBlock[] = []; + + const visit = (node: ts.Node): void => { + if (ts.isCallExpression(node)) { + const parts = calleeParts(node); + const isIt = + parts && + (IT.has(parts.root) || parts.root === 'xit') && + (!parts.member || ['only', 'skip'].includes(parts.member)); + const fn = callbackOf(node); + if (parts && isIt && fn) { + if (parts.member === 'only') + problems.push({ + file: rel, + line: lineOf(node.getStart(source)), + message: 'it.only left in the file: Mocha would skip every other test', + }); + const titles: string[] = []; + for (let p: ts.Node | undefined = node.parent; p; p = p.parent) { + if (ts.isCallExpression(p)) { + const pp = calleeParts(p); + if (pp && (DESCRIBE.has(pp.root) || pp.root === 'xdescribe')) { + const t = titleOf(p, source); + if (t !== undefined) titles.unshift(t); + } + } + } + const title = titleOf(node, source) ?? '(dynamic title)'; + const skippedDescribe = (() => { + for (let p: ts.Node | undefined = node.parent; p; p = p.parent) { + if (ts.isCallExpression(p)) { + const pp = calleeParts(p); + if (pp && (pp.root === 'xdescribe' || (DESCRIBE.has(pp.root) && pp.member === 'skip'))) return true; + } + } + return false; + })(); + its.push({ + fullTitle: [...titles, title].join(' '), + title, + startLine: lineOf(fn.getStart(source)), + endLine: lineOf(fn.end), + fn, + node, + skipped: parts.member === 'skip' || parts.root === 'xit' || skippedDescribe, + }); + } + } + ts.forEachChild(node, visit); + }; + visit(source); + return { rel, abs, kind, text, source, regions, problems, warnings, its, lineOf }; +} + +/** Innermost function-like node (not a describe/it callback of the harness) that contains the lines of `region`. */ +function helperAround(file: ScannedFile, region: Region): { name: string; fn: ts.FunctionLikeDeclaration } | undefined { + let best: ts.FunctionLikeDeclaration | undefined; + const find = (node: ts.Node): void => { + if ( + (ts.isFunctionDeclaration(node) || ts.isArrowFunction(node) || ts.isFunctionExpression(node)) && + file.lineOf(node.getStart(file.source)) <= region.startLine && + file.lineOf(node.end) >= region.endLine + ) { + best = node; + } + ts.forEachChild(node, find); + }; + find(file.source); + if (!best) return undefined; + if (ts.isFunctionDeclaration(best) && best.name) return { name: best.name.text, fn: best }; + const parent = best.parent; + if (parent && ts.isVariableDeclaration(parent) && ts.isIdentifier(parent.name)) + return { name: parent.name.text, fn: best }; + return undefined; +} + +export function resolveOwner(file: ScannedFile, region: Region): Owner { + const inside = file.its.filter((it) => it.startLine <= region.startLine && it.endLine >= region.endLine); + if (inside.length > 0) { + // The innermost it (the one that starts last). + return { type: 'it', it: inside.reduce((a, b) => (b.startLine >= a.startLine ? b : a)) }; + } + const helper = helperAround(file, region); + if (helper) return { type: 'helper', ...helper }; + const overlapping = file.source.statements.filter( + (s) => file.lineOf(s.getStart(file.source)) <= region.endLine && file.lineOf(s.end) >= region.startLine, + ); + const importOnly = + overlapping.length > 0 && + overlapping.every( + (s) => + ts.isImportDeclaration(s) && + file.lineOf(s.getStart(file.source)) > region.startLine && + file.lineOf(s.end) < region.endLine, + ); + if (importOnly) return { type: 'import-only' }; + // Narrow escape for a file-level sample that is not import-only (it contains functions and other code, so it cannot sit in an + // it()): the line right after `snippet.end` says which test asserts its effect. + const escapeLine = region.endLine + 1; + const escape = ASSERTED_BY.exec(file.text.split('\n')[escapeLine - 1] ?? ''); + if (escape) { + const needle = escape[1].trim(); + const matches = file.its.filter((it) => it.title.includes(needle)); + if (matches.length !== 1) { + return { + type: 'file-level-error', + line: escapeLine, + message: `\`lint: asserted-by ${needle}\` must match exactly one it() title in this file, it matches ${matches.length}`, + }; + } + return { type: 'file-level', it: matches[0], needle, line: escapeLine }; + } + return { + type: 'none', + reason: + 'is neither inside one it() callback (or a named helper that an it() calls) nor an import-only snippet at the top of the file', + }; +} + +/** True when `line` (1-based) is a visible, non-marker line of `region`. */ +export function isVisibleLine(region: Region, line: number): boolean { + if (line <= region.startLine || line >= region.endLine) return false; + return !region.hidden.some((h) => line >= h.hideLine && line <= (h.showLine ?? region.endLine)); +} + +const ROLE_FUNCS: Record = { + createClient: 'arg', + keysOf: 'arg', + shopper: 'open', + friend: 'open', + observer: 'open', +}; + +/** Keyset roles named in `node` (string role arguments of createClient/keysOf/server, defaults of shopper/friend/observer/server). */ +function rolesIn(node: ts.Node, namedHelpers: Map>, depth = 0): Set { + const roles = new Set(); + const visit = (n: ts.Node): void => { + if (ts.isCallExpression(n) && ts.isIdentifier(n.expression)) { + const name = n.expression.text; + const kind = ROLE_FUNCS[name]; + const first = n.arguments[0]; + if (kind === 'arg' && first && ts.isStringLiteral(first)) roles.add(first.text); + else if (kind === 'open') roles.add('open'); + else if (name === 'server') roles.add(first && ts.isStringLiteral(first) ? first.text : 'pam'); + else if (depth === 0 && namedHelpers.has(name)) namedHelpers.get(name)!.forEach((r) => roles.add(r)); + } + ts.forEachChild(n, visit); + }; + visit(node); + return roles; +} + +/** Keyset roles used by the `it` body, following calls of functions declared in the same file one level deep. Heuristic. */ +export function rolesOfIt(file: ScannedFile, it: ItBlock): string[] { + const helpers = new Map>(); + const collect = (node: ts.Node): void => { + if (ts.isFunctionDeclaration(node) && node.name && node.body) + helpers.set(node.name.text, rolesIn(node.body, new Map(), 1)); + if ( + ts.isVariableDeclaration(node) && + ts.isIdentifier(node.name) && + node.initializer && + (ts.isArrowFunction(node.initializer) || ts.isFunctionExpression(node.initializer)) + ) { + helpers.set(node.name.text, rolesIn(node.initializer, new Map(), 1)); + } + ts.forEachChild(node, collect); + }; + collect(file.source); + return [...rolesIn(it.fn, helpers)].sort(); +} + +/** Names of the files (basenames) that appear as string literals inside a given `it`: how a test points at a sample file it runs. */ +export function stringLiteralsOf(node: ts.Node): Set { + const out = new Set(); + const visit = (n: ts.Node): void => { + if (ts.isStringLiteral(n) || ts.isNoSubstitutionTemplateLiteral(n)) out.add(n.text); + ts.forEachChild(n, visit); + }; + visit(node); + return out; +} + +export interface SnippetInfo { + id: string; + file: string; + line: number; + mode: Mode | 'invalid'; + /** Full Mocha title of the test that runs it (live and mocked). */ + test?: string; + role?: string; + owner: Owner; + region: Region; +} + +export interface SampleFileInfo { + /** Repository relative path of the sample file. */ + file: string; + /** Test (file + full title) that mentions the sample file by name. */ + test?: { file: string; title: string }; +} + +export interface TierScan { + files: ScannedFile[]; + snippets: SnippetInfo[]; + samples: SampleFileInfo[]; +} + +/** Scans the whole tier once. Rules are applied by the callers. */ +export function scanTier(list: Array<{ abs: string; kind: ScannedFile['kind'] }> = listFiles()): TierScan { + const files = list.filter((f) => fs.existsSync(f.abs)).map((f) => scanFile(f.abs, f.kind)); + const snippets: SnippetInfo[] = []; + const samples: SampleFileInfo[] = []; + + // Sample files run by a test: the test mentions the file name as a string literal inside an it(). + const runnerOf = (sample: ScannedFile): { file: string; it: ItBlock } | undefined => { + const base = path.basename(sample.abs); + for (const f of files) { + if (f.kind === 'sample' || path.dirname(f.abs) !== path.dirname(sample.abs)) continue; + for (const it of f.its) { + if ([...stringLiteralsOf(it.fn)].some((s) => s === base || s.endsWith(`/${base}`))) return { file: f.rel, it }; + } + } + return undefined; + }; + + for (const f of files) { + if (f.kind === 'sample') { + const runner = runnerOf(f); + samples.push({ file: f.rel, test: runner ? { file: runner.file, title: runner.it.fullTitle } : undefined }); + for (const region of f.regions) { + snippets.push({ + id: region.id, + file: f.rel, + line: region.startLine, + mode: runner ? 'live' : 'invalid', + test: runner?.it.fullTitle, + owner: { type: 'none', reason: 'sample file' }, + region, + }); + } + continue; + } + for (const region of f.regions) { + const owner = resolveOwner(f, region); + let mode: SnippetInfo['mode'] = 'invalid'; + let test: string | undefined; + let role: string | undefined; + if (owner.type === 'it') { + mode = f.kind === 'mocked' ? 'mocked' : 'live'; + test = owner.it.fullTitle; + const roles = rolesOfIt(f, owner.it); + if (roles.length > 0) role = roles.join(','); + } else if (owner.type === 'file-level') { + mode = f.kind === 'mocked' ? 'mocked' : 'live'; + test = owner.it.fullTitle; + const roles = rolesOfIt(f, owner.it); + if (roles.length > 0) role = roles.join(','); + } else if (owner.type === 'import-only') { + mode = f.kind === 'mocked' ? 'mocked' : 'compile-only'; + } else if (owner.type === 'helper') { + // The tests that call the helper run the snippet. The first caller names it. + const caller = f.its.find((it) => callsName(it.fn, owner.name)); + if (caller) { + mode = f.kind === 'mocked' ? 'mocked' : 'live'; + test = caller.fullTitle; + const roles = rolesOfIt(f, caller); + if (roles.length > 0) role = roles.join(','); + } + } + snippets.push({ id: region.id, file: f.rel, line: region.startLine, mode, test, role, owner, region }); + } + } + return { files, snippets, samples }; +} + +export function callsName(node: ts.Node, name: string): boolean { + let found = false; + const visit = (n: ts.Node): void => { + if (found) return; + if (ts.isCallExpression(n) && ts.isIdentifier(n.expression) && n.expression.text === name) found = true; + else ts.forEachChild(n, visit); + }; + visit(node); + return found; +} diff --git a/test/integration/live/support/snippets.ts b/test/integration/live/support/snippets.ts new file mode 100644 index 000000000..38cb44688 --- /dev/null +++ b/test/integration/live/support/snippets.ts @@ -0,0 +1,126 @@ +/** + * Snippet extraction with the rules of the docs extractor (the docs repo, `src/helper/codeSnippetFetcher.ts`): + * + * // snippet. starts a snippet (ids may contain `/`) + * // snippet.end ends it + * // snippet.hide drops the lines up to `// snippet.show` + * // snippet.show + * another start marker silently ends the open snippet (reported here as a problem, never relied on) + * + * The extracted lines lose their common indentation. The marker lines themselves are not part of the text. + */ +const MARKER = /^\s*\/\/\s*snippet\.([A-Za-z0-9_/-]+)\s*$/; +const LOOSE_MARKER = /^\s*\/\/\s*(?:snippet\.|end\.snippet)/; + +export interface ExtractResult { + /** Extracted text per snippet id. */ + snippets: Map; + /** Marker problems: nesting, a missing end, a hide without show, a repeated id, a malformed marker. */ + problems: string[]; +} + +function unindent(lines: string[]): string { + const indents = lines.filter((l) => l.trim() !== '').map((l) => /^[ \t]*/.exec(l)![0].length); + const common = indents.length ? Math.min(...indents) : 0; + return lines.map((l) => l.slice(Math.min(common, /^[ \t]*/.exec(l)![0].length))).join('\n'); +} + +export function extractSnippets(text: string): ExtractResult { + const snippets = new Map(); + const problems: string[] = []; + let id: string | undefined; + let startLine = 0; + let hidden = false; + let body: string[] = []; + + const close = (): void => { + if (id === undefined) return; + if (hidden) problems.push(`line ${startLine}: snippet.hide in "${id}" has no snippet.show`); + if (snippets.has(id)) problems.push(`line ${startLine}: snippet id "${id}" is used twice`); + snippets.set(id, unindent(body)); + id = undefined; + hidden = false; + body = []; + }; + + text.split('\n').forEach((raw, index) => { + const line = index + 1; + const word = MARKER.exec(raw)?.[1]; + if (word === undefined) { + if (LOOSE_MARKER.test(raw)) + problems.push(`line ${line}: marker-like comment is not a valid marker: ${raw.trim()}`); + else if (id !== undefined && !hidden) body.push(raw); + return; + } + if (word === 'end') { + if (id === undefined) problems.push(`line ${line}: snippet.end without an open snippet`); + else close(); + } else if (word === 'hide') { + if (id === undefined) problems.push(`line ${line}: snippet.hide outside a snippet`); + else if (hidden) problems.push(`line ${line}: snippet.hide inside a hidden block`); + else hidden = true; + } else if (word === 'show') { + if (id === undefined || !hidden) problems.push(`line ${line}: snippet.show without a matching snippet.hide`); + else hidden = false; + } else { + if (id !== undefined) { + problems.push(`line ${line}: snippet "${word}" starts inside snippet "${id}" (the outer snippet is cut short)`); + close(); + } + id = word; + startLine = line; + } + }); + if (id !== undefined) { + problems.push(`line ${startLine}: snippet "${id}" has no snippet.end`); + close(); + } + return { snippets, problems }; +} + +/** + * Faithful port of `extractSnippets` of the docs repo (src/helper/codeSnippetFetcher.ts) for the default prefix list. The + * docs component shows exactly this text for `snippets={[id]}`. Differences from `extractSnippets` above, kept on purpose: + * + * - the start marker must be `// ` (slash slash SPACE) and then the literal id, so `//snippet.x` is invisible to the docs; + * - only the first start marker of the id counts, the explicit end is `// snippet.end`, and another marker whose id matches + * `[a-zA-Z0-9_-]+` ends the snippet implicitly (a marker with `/` in its id does NOT, it stays in the text); + * - whitespace-only lines keep their whitespace when the common indentation is removed; + * - no start marker, or nothing between the markers, means "not found" (returns undefined). + */ +export function docsExtract(content: string, snippetId: string): string | undefined { + const lines = content.split('\n'); + const escaped = snippetId.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); + const start = new RegExp(`^\\s*// \\s*snippet\\.${escaped}\\s*$`); + const end = /^\s*\/\/ \s*snippet\.end\s*$/; + const anyMarker = /^\s*\/\/ \s*snippet\.([a-zA-Z0-9_-]+)\s*$/; + const hide = /^\s*\/\/ \s*snippet\.hide\s*$/; + const show = /^\s*\/\/ \s*snippet\.show\s*$/; + const blueprint = /^\s*\/\/ \s*blueprint\.[a-zA-Z0-9_-]+\s*$/; + const first = lines.findIndex((l) => start.test(l)); + if (first === -1) return undefined; + let stop = lines.length; + for (let i = first + 1; i < lines.length; i++) { + if (end.test(lines[i])) { + stop = i; + break; + } + const match = anyMarker.exec(lines[i]); + if (match && match[1] !== 'hide' && match[1] !== 'show' && match[1] !== snippetId) { + stop = i; + break; + } + } + const kept: string[] = []; + let hidden = false; + for (const line of lines.slice(first + 1, stop)) { + if (hide.test(line)) hidden = true; + else if (show.test(line)) hidden = false; + else if (!blueprint.test(line) && !hidden) kept.push(line); + } + if (kept.length === 0) return undefined; + let min = Infinity; + for (const line of kept) if (line.trim().length > 0) min = Math.min(min, line.search(/\S/)); + if (min === Infinity || min === 0) return kept.join('\n'); + return kept.map((line) => (line.trim().length === 0 ? line : line.substring(min))).join('\n'); +} diff --git a/test/integration/live/support/subscribe-groups.ts b/test/integration/live/support/subscribe-groups.ts new file mode 100644 index 000000000..3d4006bfe --- /dev/null +++ b/test/integration/live/support/subscribe-groups.ts @@ -0,0 +1,38 @@ +/** + * Channel group fixtures for the subscribe pages. A group without a channel carries no messages, so the samples that subscribe to + * a group need one member channel. The member name is derived from the group, so it never collides with another page's channels. + */ +import assert from 'node:assert/strict'; +import { createClient, friend, observer } from './clients'; +import { eventually } from './wait'; + +export const memberOf = (group: string): string => `${group}_member`; + +/** Gives every group its member channel and waits until the service lists it. */ +export async function ensureGroups(groups: string[]): Promise { + const writer = friend(); + for (const group of groups) { + await writer.channelGroups.addChannels({ channelGroup: group, channels: [memberOf(group)] }); + } + const reader = observer(); + await eventually( + async () => { + for (const group of groups) { + const listed = await reader.channelGroups.listChannels({ channelGroup: group }); + assert.ok(listed.channels.includes(memberOf(group)), `${group} lists ${memberOf(group)}`); + } + }, + { label: 'channel groups have their member channel' }, + ); +} + +/** Removes only the member channels that `ensureGroups` added, so the groups do not keep them between runs. */ +export async function removeGroupMembers(groups: string[]): Promise { + const writer = createClient('open', 'friend'); + for (const group of groups) { + await writer.channelGroups + .removeChannels({ channelGroup: group, channels: [memberOf(group)] }) + .catch(() => undefined); + } + writer.destroy(); +} diff --git a/test/integration/live/support/tsc-scratch.ts b/test/integration/live/support/tsc-scratch.ts new file mode 100644 index 000000000..f9fa31a57 --- /dev/null +++ b/test/integration/live/support/tsc-scratch.ts @@ -0,0 +1,67 @@ +/** + * Scratch TypeScript projects for docs pages that make claims about types and module resolution (the v8 migration guide, the + * TypeScript setup page). A project lives in a temporary directory outside the repository with `node_modules/pubnub` + * linked to this checkout, so `import ... from 'pubnub'` resolves the way it does for a reader who installed the package + * (package.json `main` / `types`). Needs `npm run build:node` for `lib/types`. + */ +import { spawnSync } from 'node:child_process'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; + +/** Root of this checkout (the `pubnub` package). */ +export const SDK_ROOT = path.resolve(__dirname, '..', '..', '..', '..'); + +const TSC = require.resolve('typescript/lib/tsc.js'); + +/** Creates a temporary directory whose `node_modules/pubnub` points at this checkout. The caller removes it. */ +export function makeScratch(prefix = 'pn-live-'): string { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), prefix)); + fs.mkdirSync(path.join(dir, 'node_modules')); + fs.symlinkSync(SDK_ROOT, path.join(dir, 'node_modules', 'pubnub'), 'dir'); + return dir; +} + +export interface TscResult { + ok: boolean; + /** `tsc` output, one diagnostic per line. */ + output: string; +} + +/** + * Compiles small TypeScript files with the real `tsc` against the SDK types and returns the diagnostics. When `files` has a + * `tsconfig.json`, it is used as it is. Otherwise a strict one that includes every `.ts` file is written, with + * `compilerOptions` merged in. + */ +export function tscScratch(files: Record, compilerOptions: Record = {}): TscResult { + const dir = makeScratch('pn-live-tsc-'); + try { + for (const [name, content] of Object.entries(files)) { + fs.mkdirSync(path.dirname(path.join(dir, name)), { recursive: true }); + fs.writeFileSync(path.join(dir, name), content); + } + if (!('tsconfig.json' in files)) { + fs.writeFileSync( + path.join(dir, 'tsconfig.json'), + JSON.stringify({ + compilerOptions: { + target: 'ES2022', + module: 'ESNext', + moduleResolution: 'bundler', + strict: true, + noEmit: true, + esModuleInterop: true, + skipLibCheck: true, + types: [], + ...compilerOptions, + }, + include: Object.keys(files).filter((f) => /\.tsx?$/.test(f)), + }), + ); + } + const run = spawnSync(process.execPath, [TSC, '-p', dir, '--noEmit'], { encoding: 'utf8' }); + return { ok: run.status === 0, output: `${run.stdout}${run.stderr}`.trim() }; + } finally { + fs.rmSync(dir, { recursive: true, force: true }); + } +} diff --git a/test/integration/live/support/util.ts b/test/integration/live/support/util.ts new file mode 100644 index 000000000..859458c2e --- /dev/null +++ b/test/integration/live/support/util.ts @@ -0,0 +1,10 @@ +import { randomUUID } from 'node:crypto'; + +/** + * A name that no other run uses, for channels and users a test asserts on exactly. Names built this way cannot be listed in + * resources.ts, so the janitor never removes the messages on them; they expire with the retention of the keyset. + */ +export const unique = (base: string): string => `${base}-${randomUUID().slice(0, 8)}`; + +/** A sorted copy, for comparing lists whose order the service does not promise. */ +export const sorted = (values: string[]): string[] => [...values].sort(); diff --git a/test/integration/live/support/wait.ts b/test/integration/live/support/wait.ts new file mode 100644 index 000000000..e3c358414 --- /dev/null +++ b/test/integration/live/support/wait.ts @@ -0,0 +1,284 @@ +/** + * Waiting primitives. Poll a read until it holds (`eventually`) or listen for the exact event the action should cause. Use a + * fixed `sleep` only to prove that something did NOT arrive. + * + * - `eventually` poll a read (hereNow, history, App Context filter) that is eventually consistent on the service. + * - `expectEvent` register a listener BEFORE the action, resolve on the first matching event. + * - `connected` resolve when a subscribe is really active. A publish sent before that point is not delivered. + * - `watch` subscribe once and record everything, so the assertion can come after the publish without racing. + * - `expectStatus` resolve on a status category such as `PNConnectedCategory`. + * + * Every helper rejects with a message that says what was awaited and what was seen. + */ +import type PubNub from 'pubnub'; + +export interface EventuallyOptions { + /** Milliseconds. Default 15000. */ + timeout?: number; + /** Milliseconds between attempts. Default 500. */ + interval?: number; + label?: string; +} + +/** Presence is the slowest eventual state: what hereNow and similar reads show lags the heartbeat. */ +export const PRESENCE_TIMEOUT = 45_000; + +/** + * How long a test waits for a presence event it caused. The keyset announces a join or leave after a 2 s debounce when the + * channel holds at most 20 occupants, and falls back to one summary per 10 s interval above that. 30 s covers the debounce, + * one interval and a long-poll cycle with margin. + */ +export const PRESENCE_EVENT_TIMEOUT = 30_000; + +export const sleep = (ms: number): Promise => new Promise((resolve) => setTimeout(resolve, ms)); + +/** Polls a read until the callback stops throwing, then returns its value. On timeout the error carries the last failure. */ +export async function eventually(fn: () => Promise | T, options: EventuallyOptions = {}): Promise { + const timeout = options.timeout ?? 15_000; + const interval = options.interval ?? 500; + const deadline = Date.now() + timeout; + let last: unknown; + let attempts = 0; + for (;;) { + attempts += 1; + try { + return await fn(); + } catch (error) { + last = error; + } + if (Date.now() + interval > deadline) break; + await sleep(interval); + } + const detail = last instanceof Error ? last.message : String(last); + const error = new Error( + `eventually${options.label ? ` (${options.label})` : ''} timed out after ${timeout}ms, ${attempts} attempts. Last: ${detail}`, + ); + (error as Error & { cause?: unknown }).cause = last; + throw error; +} + +/** + * Removes a listener after the current dispatch. Removing it inside its own callback mutates the SDK's listener array while + * it iterates, which makes the SDK skip the next listener. + */ +function detach(target: { removeListener(listener: PubNub.Listener): void }, listener: PubNub.Listener): void { + setImmediate(() => target.removeListener(listener)); +} + +/** Anything with the listener API: the client, a subscription or a subscription set. */ +export interface ListenerTarget { + addListener(listener: PubNub.Listener): void; + removeListener(listener: PubNub.Listener): void; +} + +/** Listener callbacks the helpers can wait for. `status` has its own helpers (`connected`, `expectStatus`). */ +export type EventKind = 'message' | 'signal' | 'presence' | 'messageAction' | 'file' | 'objects'; + +type EventOf = Parameters>[0]; + +export interface ExpectOptions { + timeout?: number; +} + +/** Registers a listener now and resolves on the first event of `kind` that satisfies `predicate`. Call it before the action. */ +export function expectEvent( + target: ListenerTarget, + kind: K, + predicate: (event: EventOf) => boolean = () => true, + options: ExpectOptions = {}, +): Promise> { + const timeout = options.timeout ?? 15_000; + return new Promise>((resolve, reject) => { + const listener: PubNub.Listener = {}; + const timer = setTimeout(() => { + target.removeListener(listener); + reject(new Error(`no "${kind}" event matched within ${timeout}ms`)); + }, timeout); + // Mocha's own test timer keeps the process alive meanwhile. unref() only means an abandoned wait cannot hold it open. + timer.unref(); + (listener as Record)[kind] = (event: EventOf) => { + // A predicate that throws (reading a field of an event of another shape) means "not this one", not a test failure. + let matched = false; + try { + matched = predicate(event); + } catch { + matched = false; + } + if (!matched) return; + clearTimeout(timer); + detach(target, listener); + resolve(event); + }; + target.addListener(listener); + }); +} + +// Statuses that mean a subscribe loop is up. A re-subscribe reports `SubscriptionChanged`, a recovered network `Reconnected`. +const CONNECT_CATEGORIES = new Set(['PNConnectedCategory', 'PNSubscriptionChangedCategory', 'PNReconnectedCategory']); + +/** + * Resolves when the client reports a connected or subscription-changed status that covers every channel and group. Register + * it BEFORE the call that subscribes (statuses are not replayed). + */ +export function connected( + client: Pick, + targets: { channels?: string[]; channelGroups?: string[] }, + options: ExpectOptions = {}, +): Promise { + const timeout = options.timeout ?? 15_000; + const needChannels = new Set(targets.channels ?? []); + const needGroups = new Set(targets.channelGroups ?? []); + return new Promise((resolve, reject) => { + const seenChannels = new Set(); + const seenGroups = new Set(); + const listener: PubNub.Listener = { + status: (status) => { + if (!CONNECT_CATEGORIES.has(String(status.category))) return; + const event = status as PubNub.StatusEvent; + // A status without lists means the whole subscription is up. + event.affectedChannels?.forEach((c) => seenChannels.add(c)); + event.subscribedChannels?.forEach((c) => seenChannels.add(c)); + event.affectedChannelGroups?.forEach((g) => seenGroups.add(g)); + const done = + [...needChannels].every((c) => seenChannels.has(c)) && [...needGroups].every((g) => seenGroups.has(g)); + if (done) { + clearTimeout(timer); + detach(client, listener); + resolve(); + } + }, + }; + const timer = setTimeout(() => { + client.removeListener(listener); + reject( + new Error( + `not connected within ${timeout}ms. Wanted channels [${[...needChannels]}] groups [${[...needGroups]}], ` + + `saw channels [${[...seenChannels]}] groups [${[...seenGroups]}]`, + ), + ); + }, timeout); + timer.unref(); + client.addListener(listener); + }); +} + +/** One event a `Watch` saw, tagged with the listener it arrived on. */ +export interface RecordedEvent { + kind: EventKind; + event: unknown; +} + +/** A subscription that records every event it receives, so a test can subscribe once and assert later. Created with `watch()`. */ +export class Watch { + readonly events: RecordedEvent[] = []; + // Callers blocked in `next()`. Each is woken as soon as an event arrives. + private waiters: Array<() => void> = []; + + constructor(private readonly set: ReturnType) { + const record = (kind: EventKind) => (event: unknown) => { + this.events.push({ kind, event }); + this.waiters.splice(0).forEach((wake) => wake()); + }; + set.addListener({ + message: record('message'), + signal: record('signal'), + presence: record('presence'), + messageAction: record('messageAction'), + file: record('file'), + objects: record('objects'), + }); + } + + /** Resolves with the first recorded event of `kind` matching `predicate`, waiting if needed. */ + async next( + kind: K, + predicate: (event: EventOf) => boolean = () => true, + options: ExpectOptions = {}, + ): Promise> { + const timeout = options.timeout ?? 15_000; + const deadline = Date.now() + timeout; + for (;;) { + for (const recorded of this.events) { + if (recorded.kind !== kind) continue; + let ok = false; + try { + ok = predicate(recorded.event as EventOf); + } catch { + ok = false; + } + if (ok) return recorded.event as EventOf; + } + const remaining = deadline - Date.now(); + if (remaining <= 0) { + throw new Error( + `no "${kind}" event matched within ${timeout}ms. Recorded: ${this.events.map((e) => e.kind).join(',') || 'none'}`, + ); + } + // Sleep until the next event wakes us. The 250 ms cap is a safety net, not a poll interval. + await new Promise((resolve) => { + const timer = setTimeout(resolve, Math.min(remaining, 250)); + this.waiters.push(() => { + clearTimeout(timer); + resolve(); + }); + }); + } + } + + /** Unsubscribes and drops the listeners. */ + stop(): void { + this.set.unsubscribe(); + this.set.removeAllListeners(); + } +} + +export interface WatchOptions { + channelGroups?: string[]; + presence?: boolean; + timeout?: number; +} + +/** + * Subscribes `client` to channels (and groups), waits until connected and returns a recording `Watch`. Pass a client that has + * not subscribed before (`observer()` builds a new one on every call): a reused client can still report the status of an + * earlier unsubscribe, which would resolve `connected` before the new subscription is active. + */ +export async function watch(client: PubNub, channels: string[], options: WatchOptions = {}): Promise { + const set = client.subscriptionSet({ + channels, + channelGroups: options.channelGroups, + subscriptionOptions: options.presence ? { receivePresenceEvents: true } : undefined, + }); + // Register the connected listener first, then subscribe: statuses are not replayed. + const ready = connected(client, { channels, channelGroups: options.channelGroups }, { timeout: options.timeout }); + const handle = new Watch(set); + set.subscribe(); + await ready; + return handle; +} + +/** Resolves on the first status event whose category is one of `categories`. Register it before the action. */ +export function expectStatus( + client: Pick, + categories: string[], + options: ExpectOptions = {}, +): Promise { + const timeout = options.timeout ?? 15_000; + return new Promise((resolve, reject) => { + const listener: PubNub.Listener = { + status: (status) => { + const category = String(status.category); + if (!categories.includes(category)) return; + clearTimeout(timer); + detach(client, listener); + resolve(category); + }, + }; + const timer = setTimeout(() => { + client.removeListener(listener); + reject(new Error(`no status in [${categories.join(', ')}] within ${timeout}ms`)); + }, timeout); + timer.unref(); + client.addListener(listener); + }); +} diff --git a/test/integration/live/tsconfig.json b/test/integration/live/tsconfig.json new file mode 100644 index 000000000..736cf8a6c --- /dev/null +++ b/test/integration/live/tsconfig.json @@ -0,0 +1,22 @@ +{ + "//": "Type checking of the live tier (npm run test:integration:live:types). 'pubnub' maps to the shipped types, which need `npm run build:node` first. At run time tsconfig.run.json maps it to the built library instead.", + "compilerOptions": { + "target": "ES2022", + "module": "ESNext", + "moduleResolution": "Bundler", + "lib": ["ES2022"], + "types": ["node", "mocha"], + "strict": true, + "noEmit": true, + "skipLibCheck": true, + "esModuleInterop": true, + "allowSyntheticDefaultImports": true, + "forceConsistentCasingInFileNames": true, + "allowJs": false, + "baseUrl": "../../..", + "paths": { + "pubnub": ["lib/types/index.d.ts", "lib/node/index.js"] + } + }, + "include": ["./**/*.ts"] +} diff --git a/test/integration/live/tsconfig.run.json b/test/integration/live/tsconfig.run.json new file mode 100644 index 000000000..b8b2e3572 --- /dev/null +++ b/test/integration/live/tsconfig.run.json @@ -0,0 +1,9 @@ +{ + "//": "Read by tsx at run time (TSX_TSCONFIG_PATH). 'pubnub' resolves to the built library of this checkout. lib/node has no .d.ts, so type checking uses tsconfig.json.", + "extends": "./tsconfig.json", + "compilerOptions": { + "paths": { + "pubnub": ["lib/node/index.js"] + } + } +} diff --git a/test/integration/live/use-cases/automated-polling.test.ts b/test/integration/live/use-cases/automated-polling.test.ts new file mode 100644 index 000000000..486894209 --- /dev/null +++ b/test/integration/live/use-cases/automated-polling.test.ts @@ -0,0 +1,225 @@ +/* eslint-disable prettier/prettier, max-len -- sample text must stay character for character, so its long lines cannot be re-wrapped */ +import assert from 'node:assert/strict'; +import { createClient, observer } from '../support/clients'; +import { captureConsole } from '../support/console'; +import { storedMessage } from '../support/history'; +import { connected, eventually, sleep, watch } from '../support/wait'; + +/** + * Automated polling. Port of docs-snippets/use-cases/automated-polling.ts. + * The trigger messages come from an Illuminate Business Rule on the real service. The keyset of the tier has no such rule, so + * the test publishes the triggers itself with the payload the page describes and verifies the poll service side. + * Needs Message Persistence on the open keyset (the poll is published with storeInHistory). + */ +describe('use-cases/automated-polling', () => { + it('automatedPollingPublishTriggeredPoll, automatedPollingThrottleTriggers, automatedPollingReceiveTrigger: a trigger opens the poll for its reaction, a second trigger inside the cooldown does not', async function () { + this.timeout(60_000); + const pubnub = createClient('open', 'poll-service'); + const logs = captureConsole('log'); + const errors = captureConsole('error'); + const polls = await watch(observer(), ['game.new-poll']); + try { + // snippet.automatedPollingPublishTriggeredPoll + const pollsByReaction = new Map([ + [ + '\u{1F621}', + { + title: 'Which team is playing dirtiest?', + options: [ + { id: 1, text: 'Home team' }, + { id: 2, text: 'Away team' }, + ], + }, + ], + [ + '\u{1F389}', + { + title: 'Whose fans are celebrating hardest?', + options: [ + { id: 1, text: 'Home team' }, + { id: 2, text: 'Away team' }, + ], + }, + ], + ]); + + async function openPollForReaction(reaction = '') { + const template = pollsByReaction.get(reaction); + + if (!template) { + console.log('no poll is defined for', reaction); + return; + } + + try { + const response = await pubnub.publish({ + channel: 'game.new-poll', + message: { + id: `poll-${Date.now()}`, + title: template.title, + durationSeconds: 30, + options: template.options, + }, + customMessageType: 'poll-opened', + storeInHistory: true, + }); + console.log('triggered poll published at timetoken:', response.timetoken); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error( + `Publishing the triggered poll failed: ${error}${status ? ` Additional information: ${status}` : ''}`, + ); + } + } + // snippet.end + + // Open one poll directly, and ask for a reaction nobody defined a poll for. + await openPollForReaction('\u{1F621}'); + await openPollForReaction('\u{1F4A4}'); + assert.deepEqual(errors.lines, [], 'the error path was not taken'); + assert.ok(logs.lines.includes('no poll is defined for \u{1F4A4}')); + const direct = logs.lines.find((l) => l.startsWith('triggered poll published at timetoken: ')); + assert.ok(direct, 'the poll was published'); + const directTimetoken = direct.slice('triggered poll published at timetoken: '.length); + const directPoll = await polls.next('message', (m) => m.timetoken === directTimetoken); + assert.equal(directPoll.customMessageType, 'poll-opened'); + assert.equal(directPoll.publisher, 'poll-service'); + const directBody = directPoll.message as { id: string; title: string; durationSeconds: number; options: unknown[] }; + assert.equal(directBody.title, 'Which team is playing dirtiest?'); + assert.equal(directBody.durationSeconds, 30); + assert.deepEqual(directBody.options, [ + { id: 1, text: 'Home team' }, + { id: 2, text: 'Away team' }, + ]); + assert.match(directBody.id, /^poll-\d+$/); + await eventually( + async () => { + assert.ok(await storedMessage(observer(), 'game.new-poll', directTimetoken), 'poll is stored'); + }, + { label: 'poll stored' }, + ); + logs.lines.length = 0; + + // snippet.automatedPollingThrottleTriggers + const minimumMillisecondsBetweenPolls = 60000; + let lastPollOpenedAt = 0; + + function shouldOpenPoll() { + const now = Date.now(); + + if (now - lastPollOpenedAt < minimumMillisecondsBetweenPolls) { + console.log('that trigger arrived inside the cooldown window, so no new poll opened'); + return false; + } + + lastPollOpenedAt = now; + return true; + } + // snippet.end + + assert.equal(shouldOpenPoll(), true, 'the first trigger is allowed'); + assert.equal(shouldOpenPoll(), false, 'the second one inside the cooldown is refused'); + assert.deepEqual(logs.lines, ['that trigger arrived inside the cooldown window, so no new poll opened']); + // Reset the cooldown the assertions above consumed, as a fresh service would have it. + lastPollOpenedAt = 0; + logs.lines.length = 0; + + const ready = connected(pubnub, { channels: ['game.poll-triggers'] }); + // snippet.automatedPollingReceiveTrigger + const triggerSubscription = pubnub.channel('game.poll-triggers').subscription({ receivePresenceEvents: false }); + + triggerSubscription.onMessage = (event) => { + const trigger = event.message; + const rawReaction = + typeof trigger === 'object' && trigger !== null && !Array.isArray(trigger) && 'reaction' in trigger + ? trigger.reaction + : undefined; + + if (typeof rawReaction !== 'string') return; + + const reaction = rawReaction; + + console.log('open a poll because fans keep tapping', reaction); + + if (!pollsByReaction.has(reaction) || shouldOpenPoll()) { + void openPollForReaction(reaction); + } + }; + + triggerSubscription.subscribe(); + // snippet.end + + await ready; + const sender = createClient('open', 'poll-trigger-sender'); + // A message without a reaction is ignored. + await sender.publish({ channel: 'game.poll-triggers', message: { something: 'else' } }); + // First trigger: a poll for the celebrating reaction opens. + const trigger = await sender.publish({ + channel: 'game.poll-triggers', + message: { reaction: '\u{1F389}' }, + customMessageType: 'poll-trigger', + }); + const opened = await polls.next( + 'message', + (m) => m.timetoken > directTimetoken && (m.message as { title?: string }).title === 'Whose fans are celebrating hardest?', + ); + assert.ok(BigInt(opened.timetoken) > BigInt(trigger.timetoken)); + assert.equal(opened.publisher, 'poll-service'); + assert.equal(opened.customMessageType, 'poll-opened'); + assert.ok(logs.lines.includes('open a poll because fans keep tapping \u{1F389}')); + assert.equal(logs.lines.filter((l) => l.startsWith('open a poll')).length, 1, 'the message without a reaction was ignored'); + + // Second mapped trigger inside the cooldown: logged as a trigger, refused by the throttle, no new poll. A wait proves absence. + const pollsBefore = polls.events.filter((e) => e.kind === 'message').length; + await sender.publish({ + channel: 'game.poll-triggers', + message: { reaction: '\u{1F621}' }, + customMessageType: 'poll-trigger', + }); + await eventually(() => assert.ok(logs.lines.includes('open a poll because fans keep tapping \u{1F621}')), { + label: 'second trigger received', + }); + assert.ok(logs.lines.includes('that trigger arrived inside the cooldown window, so no new poll opened')); + await sleep(2000); + assert.equal(polls.events.filter((e) => e.kind === 'message').length, pollsBefore, 'no second poll opened'); + assert.deepEqual(errors.lines, [], 'the error path was not taken'); + + // An unmapped reaction must not use up the cooldown: reset it, send an unmapped trigger and at once a mapped one. + lastPollOpenedAt = 0; + logs.lines.length = 0; + const pollsBeforeUnmapped = polls.events.filter((e) => e.kind === 'message').length; + await sender.publish({ + channel: 'game.poll-triggers', + message: { reaction: '\u{1F4A4}' }, + customMessageType: 'poll-trigger', + }); + const afterUnmapped = await sender.publish({ + channel: 'game.poll-triggers', + message: { reaction: '\u{1F621}' }, + customMessageType: 'poll-trigger', + }); + const openedAfterUnmapped = await polls.next( + 'message', + (m) => BigInt(m.timetoken) > BigInt(afterUnmapped.timetoken) && (m.message as { title?: string }).title === 'Which team is playing dirtiest?', + ); + assert.equal(openedAfterUnmapped.publisher, 'poll-service'); + assert.ok(logs.lines.includes('open a poll because fans keep tapping \u{1F4A4}')); + assert.ok(logs.lines.includes('no poll is defined for \u{1F4A4}')); + assert.ok( + !logs.lines.includes('that trigger arrived inside the cooldown window, so no new poll opened'), + 'the unmapped trigger did not consume the cooldown', + ); + await sleep(1000); + assert.equal( + polls.events.filter((e) => e.kind === 'message').length, + pollsBeforeUnmapped + 1, + 'exactly one poll opened, for the mapped reaction', + ); + assert.deepEqual(errors.lines, [], 'the error path was not taken'); + } finally { + polls.stop(); + logs.restore(); + errors.restore(); + } + }); +}); diff --git a/test/integration/live/use-cases/fan-behavior-management.test.ts b/test/integration/live/use-cases/fan-behavior-management.test.ts new file mode 100644 index 000000000..2b7005f50 --- /dev/null +++ b/test/integration/live/use-cases/fan-behavior-management.test.ts @@ -0,0 +1,521 @@ +import assert from 'node:assert/strict'; +import PubNub from 'pubnub'; +import { fileURLToPath } from 'node:url'; +import { granted } from '../support/access-manager'; +import { assertTimetoken, rejectsWithCategory } from '../support/assertions'; +import { track } from '../support/clients'; +import { captureConsole } from '../support/console'; +import { keysOf } from '../support/env'; +import { withPlaceholders } from '../support/placeholders'; +import { eventually } from '../support/wait'; + +/** + * Port of docs-snippets/use-cases/fan-behavior-management.ts (page use-cases/sports-media-entertainment/fan-behavior-management). + * Runs on the secured keyset: the token service signs and revokes tokens, the fan client only applies them. + * + * The state-store snippet starts with an `import`, which only exists at module level, so it sits at file level, before the + * describe (the `lint: asserted-by` line after it names the test that asserts it), and the flow below reads its state file back. The sample keeps its state next to the script (`./fan-state.json`): + * under the runner that is this folder, and the hooks below remove the file before and after. + */ +// snippet.fanBehaviorStateStore +import fs from 'node:fs'; + +// A real token service tracks issued tokens and each fan's status in a database. +// This tutorial persists the same information in a JSON file next to the script, +// so `grant`, `mute`, `unmute`, and `ban` share state across separate `node server.js` +// invocations without needing a database just to run the tutorial. +const stateFilePath = new URL('./fan-state.json', import.meta.url); + +function loadState() { + try { + return JSON.parse(fs.readFileSync(stateFilePath, 'utf8')); + } catch { + return {}; + } +} + +function saveState(state = {}) { + fs.writeFileSync(stateFilePath, JSON.stringify(state, null, 2)); +} +// snippet.end +// lint: asserted-by a fan is granted, muted, banned and let back in + +interface IssuedToken { + token: string; + write: boolean; +} +interface FanState { + status: string; + tokens: IssuedToken[]; +} + +describe('use-cases/fan-behavior-management', () => { + const FAN = 'fan-42'; + const removeState = (): void => fs.rmSync(stateFilePath, { force: true }); + const readState = (): Record => JSON.parse(fs.readFileSync(stateFilePath, 'utf8')); + const originalArgv = process.argv; + before(removeState); + after(() => { + process.argv = originalArgv; + removeState(); + }); + + it('fanBehaviorStateStore, fanBehaviorGrantChatAccess, fanBehaviorHandleAccessDenied, fanBehaviorApplyToken, fanBehaviorSubscribeAndPublish, fanBehaviorMuteFan, fanBehaviorBanFan, fanBehaviorUnmuteFan, fanBehaviorServerDispatch, fanBehaviorInspectToken: a fan is granted, muted, banned and let back in, and the fan client sees each change', async function () { + // Revoking a token takes up to about a minute to reach every server, and the ban step waits for it. + this.timeout(400_000); + const { publishKey, subscribeKey, secretKey } = keysOf('pam'); + const log = captureConsole('log'); + const errors = captureConsole('error'); + try { + const server = track( + new PubNub({ publishKey, subscribeKey, secretKey, userId: 'moderation-service', heartbeatInterval: 0 }), + ); + const reset = (): void => { + log.lines.length = 0; + errors.lines.length = 0; + }; + + // snippet.fanBehaviorGrantChatAccess + async function grantChatAccess(userId = '') { + const state = loadState(); + const entry = state[userId] ?? { status: 'active', tokens: [] }; + + if (entry.status === 'banned') { + console.log(`${userId} is banned, so no token was issued`); + return; + } + + const canWrite = entry.status !== 'muted'; + + try { + const token = await server.grantToken({ + ttl: 60, + authorizedUserId: userId, + resources: { + channels: { + 'game.chat': { read: true, write: canWrite }, + }, + }, + }); + + entry.status = entry.status ?? 'active'; + entry.tokens = [...entry.tokens, { token, write: canWrite }]; + state[userId] = entry; + saveState(state); + + console.log(`token that allows ${canWrite ? 'reading and writing' : 'reading'} chat:`, token); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Granting chat access failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + } + // snippet.end + + /* eslint-disable prettier/prettier */ + // snippet.fanBehaviorMuteFan + async function muteFan(userId = '') { + const state = loadState(); + const entry = state[userId] ?? { status: 'active', tokens: [] }; + + if (entry.status === 'banned') { + console.log(`${userId} is already banned, so there is nothing left to mute`); + return; + } + + const writableTokens = entry.tokens.filter((issued = { token: '', write: false }) => issued.write); + const stillValid = []; + + for (const issued of writableTokens) { + try { + await server.revokeToken(issued.token); + console.log('revoked an outstanding writable token'); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Revoking a writable token failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + // Keep tracking a token you couldn't revoke, so the next mute or ban retries it. + stillValid.push(issued); + } + } + + // Record the mute before issuing anything new, so a later grant can't hand out write access. + entry.status = 'muted'; + entry.tokens = [...entry.tokens.filter((issued = { token: '', write: false }) => !issued.write), ...stillValid]; + state[userId] = entry; + saveState(state); + + if (stillValid.length > 0) { + console.error(`${stillValid.length} writable token(s) are still valid. Run mute again to retry revoking them.`); + } + + try { + const token = await server.grantToken({ + ttl: 15, + authorizedUserId: userId, + resources: { + channels: { + 'game.chat': { read: true }, + }, + }, + }); + + entry.tokens = [...entry.tokens, { token, write: false }]; + state[userId] = entry; + saveState(state); + + console.log('token that allows reading chat but not writing to it:', token); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Muting the fan failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + } + // snippet.end + /* eslint-enable prettier/prettier */ + + // snippet.fanBehaviorUnmuteFan + async function unmuteFan(userId = '') { + const state = loadState(); + const entry = state[userId] ?? { status: 'active', tokens: [] }; + + // This is the one command that clears a ban as well as a mute. Calling it is always + // a deliberate decision by whoever operates server.js, never a side effect of anything + // else in this tutorial. + try { + const token = await server.grantToken({ + ttl: 60, + authorizedUserId: userId, + resources: { + channels: { + 'game.chat': { read: true, write: true }, + }, + }, + }); + + entry.status = 'active'; + entry.tokens = [...entry.tokens, { token, write: true }]; + state[userId] = entry; + saveState(state); + + console.log('token that allows reading and writing chat again:', token); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Unmuting the fan failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + } + // snippet.end + + // snippet.fanBehaviorBanFan + async function banFan(userId = '') { + const state = loadState(); + const entry = state[userId] ?? { status: 'active', tokens: [] }; + + const stillValid = []; + + for (const issued of entry.tokens) { + try { + await server.revokeToken(issued.token); + console.log('revoked an outstanding token'); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Revoking a token failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + // Keep tracking a token you couldn't revoke, so the next ban retries it. + stillValid.push(issued); + } + } + + entry.status = 'banned'; + entry.tokens = stillValid; + state[userId] = entry; + saveState(state); + + if (stillValid.length > 0) { + console.error( + `${userId} is banned, but ${stillValid.length} token(s) are still valid. Run ban again to retry revoking them.`, + ); + } else { + console.log(`${userId} is banned. Every outstanding token, read-only and writable, is now revoked.`); + } + } + // snippet.end + + // The page runs `node server.js `: the same snippet runs once per command with that argv. + const runServer = async (argument: string, userId = FAN): Promise => { + process.argv = [originalArgv[0], 'server.js', argument, userId]; + // snippet.fanBehaviorServerDispatch + const [, , command, userIdArgument] = process.argv; + const targetUserId = userIdArgument ?? 'fan-42'; + + if (command === 'grant') { + await grantChatAccess(targetUserId); + } else if (command === 'mute') { + await muteFan(targetUserId); + } else if (command === 'unmute') { + await unmuteFan(targetUserId); + } else if (command === 'ban') { + await banFan(targetUserId); + } else { + console.log('usage: node server.js [userId]'); + } + // snippet.end + }; + + // The page runs `node fan.js `: listener first, then the token, then subscribe and publish. + const startFan = async (token: string, extra: Partial = {}): Promise => { + const fanClient = track(new PubNub({ publishKey, subscribeKey, userId: FAN, ...extra })); + process.argv = [originalArgv[0], 'fan.js', token]; + + // snippet.fanBehaviorHandleAccessDenied + fanClient.addListener({ + status: (event) => { + if (event.category === 'PNConnectedCategory') { + console.log('connected to game.chat'); + } else if (event.category === 'PNAccessDeniedCategory' || event.error === 'PNAccessDeniedCategory') { + // A revoked token arrives as the category, or as the error of an unexpected disconnect, depending on the subscribe loop. + console.log('this fan may no longer write to game.chat'); + } + }, + }); + // snippet.end + + // snippet.fanBehaviorApplyToken + const suppliedToken = process.argv[2]; + + if (!suppliedToken) { + console.error('usage: node fan.js '); + process.exit(1); + } + + fanClient.setToken(suppliedToken); + // snippet.end + + /* eslint-disable prettier/prettier */ + // snippet.fanBehaviorSubscribeAndPublish + const subscription = fanClient.channel('game.chat').subscription(); + subscription.subscribe(); + + try { + const response = await fanClient.publish({ + channel: 'game.chat', + message: { text: 'Come on!' }, + }); + console.log('chat message published at timetoken:', response.timetoken); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Publishing to game.chat failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + // snippet.end + /* eslint-enable prettier/prettier */ + return fanClient; + }; + + const lastToken = (): string => { + const tokens = readState()[FAN].tokens; + return tokens[tokens.length - 1].token; + }; + const parsedOf = (token: string): NonNullable> => { + const parsed = server.parseToken(token); + assert.ok(parsed, 'the token parses'); + return parsed; + }; + const grantedChat = (token: string): Record => + granted(parsedOf(token).resources?.channels?.['game.chat']); + const publishWith = (token: string): Promise => { + const other = track(new PubNub({ publishKey, subscribeKey, userId: FAN })); + other.setToken(token); + return other.publish({ channel: 'game.chat', message: { text: 'probe' } }); + }; + + // 1. grant: the dispatch snippet routes `grant` to grantChatAccess. + await runServer('grant'); + assert.deepEqual(errors.lines, [], 'grant took no error path'); + const grantLine = log.lines.find((line) => line.startsWith('token that allows reading and writing chat:')); + assert.ok(grantLine, `grant logged its token. Logged: ${log.lines.join(' | ')}`); + let state = readState(); + assert.equal(state[FAN].status, 'active'); + assert.equal(state[FAN].tokens.length, 1); + assert.equal(state[FAN].tokens[0].write, true); + const readWriteToken = lastToken(); + assert.ok(grantLine.endsWith(readWriteToken), 'the logged token is the stored token'); + assert.deepEqual(grantedChat(readWriteToken), { read: true, write: true }); + assert.equal(parsedOf(readWriteToken).authorized_uuid, FAN, 'the token is bound to the fan'); + assert.equal(parsedOf(readWriteToken).ttl, 60); + + // 2. the fan applies the token, subscribes and publishes. + reset(); + const fan = await startFan(readWriteToken); + await eventually(() => assert.ok(log.lines.includes('connected to game.chat'), `log: ${log.lines.join(' | ')}`)); + assert.deepEqual(errors.lines, [], 'neither applying the token nor publishing took the error path'); + const published = log.lines.find((line) => line.startsWith('chat message published at timetoken: ')); + assert.ok(published, `the publish was accepted. Logged: ${log.lines.join(' | ')}`); + assertTimetoken(published.split(': ')[1]); + + // 3. inspect the token the fan holds (placeholder token of the page swapped for the real one). + reset(); + { + const fanClient = withPlaceholders(track(new PubNub({ publishKey, subscribeKey, userId: FAN })), { + 'replace-with-the-token-to-inspect': readWriteToken, + }); + // snippet.fanBehaviorInspectToken + const parsed = fanClient.parseToken('replace-with-the-token-to-inspect'); + + if (parsed) { + console.log('token expires in', parsed.ttl, 'minutes'); + console.log('channel permissions:', parsed.resources?.channels); + } + // snippet.end + } + assert.ok(log.lines.includes('token expires in 60 minutes'), `log: ${log.lines.join(' | ')}`); + const permissionsLine = log.lines.find((line) => line.startsWith('channel permissions:')); + assert.ok(permissionsLine?.includes('"game.chat"'), `permissions line: ${permissionsLine}`); + assert.ok(permissionsLine); + assert.match(permissionsLine, /"read":true,"write":true/, 'the printed permissions carry read and write'); + + // 4. mute: every writable token is revoked, a read-only one is issued, the state records the mute. + reset(); + await runServer('mute'); + assert.deepEqual(errors.lines, [], 'mute took no error path (revoke must be enabled on the keyset)'); + assert.deepEqual( + log.lines.filter((line) => line === 'revoked an outstanding writable token'), + ['revoked an outstanding writable token'], + 'the one writable token was revoked', + ); + state = readState(); + assert.equal(state[FAN].status, 'muted'); + assert.equal(state[FAN].tokens.length, 1, 'the revoked writable token left the tracked list'); + assert.equal(state[FAN].tokens[0].write, false); + const mutedToken = lastToken(); + assert.notEqual(mutedToken, readWriteToken); + assert.ok( + log.lines.some((line) => line === `token that allows reading chat but not writing to it: ${mutedToken}`), + `log: ${log.lines.join(' | ')}`, + ); + assert.deepEqual(grantedChat(mutedToken), { read: true }); + assert.equal(parsedOf(mutedToken).ttl, 15); + + // The revoke reaches the old token: a fresh client holding it is refused (propagation takes up to about a minute). + await eventually(() => rejectsWithCategory(publishWith(readWriteToken), 'PNAccessDeniedCategory'), { + label: 'the revoked writable token is refused', + timeout: 120_000, + interval: 3000, + }); + + // A grant while muted can only hand out a read-only token. + reset(); + await runServer('grant'); + assert.deepEqual(errors.lines, []); + assert.ok( + log.lines.some((line) => line.startsWith('token that allows reading chat: ')), + log.lines.join(' | '), + ); + assert.equal(readState()[FAN].tokens.length, 2); + assert.equal(readState()[FAN].tokens[1].write, false, 'a muted fan is not granted write'); + + // 5. the fan restarts with the muted token: connect works, the publish is refused and caught at the call site. + fan.destroy(); + reset(); + const mutedFan = await startFan(mutedToken); + // Records every status of the running fan, with the error category the status carries (outside the sample). + const mutedFanStatuses: Array<{ category: string; error: string }> = []; + mutedFan.addListener({ + status: (status) => + mutedFanStatuses.push({ + category: String(status.category), + error: String((status as { error?: unknown }).error ?? ''), + }), + }); + await eventually(() => assert.ok(log.lines.includes('connected to game.chat'), `log: ${log.lines.join(' | ')}`)); + assert.equal(errors.lines.length, 1, `exactly one error line. Logged: ${errors.lines.join(' | ')}`); + assert.ok((errors.lines as string[])[0].startsWith('Publishing to game.chat failed: '), errors.lines[0]); + assert.ok(!log.lines.some((line) => line.startsWith('chat message published')), 'the publish was not accepted'); + await rejectsWithCategory(publishWith(mutedToken), 'PNAccessDeniedCategory'); + + // 6. ban with the muted fan still running: all tokens are revoked and the running client reports access denied. + reset(); + await runServer('ban'); + assert.deepEqual(errors.lines, [], 'ban took no error path'); + assert.equal( + log.lines.filter((line) => line === 'revoked an outstanding token').length, + 2, + 'both tracked tokens (the muted one and the read-only grant) were revoked', + ); + assert.ok( + log.lines.includes(`${FAN} is banned. Every outstanding token, read-only and writable, is now revoked.`), + ); + state = readState(); + assert.equal(state[FAN].status, 'banned'); + assert.deepEqual(state[FAN].tokens, []); + // With the default subscribe loop (event engine) a revoked token ends the running subscription with status category + // PNDisconnectedUnexpectedlyCategory and `error: 'PNAccessDeniedCategory'`; only the legacy loop uses the category itself. + // The sample handles both shapes, so its log line must appear on the default loop. + const deniedStatus = (): boolean => + mutedFanStatuses.some( + (s) => s.category === 'PNDisconnectedUnexpectedlyCategory' && s.error === 'PNAccessDeniedCategory', + ); + // First without any traffic (the page says "shortly after"), then with messages on the channel, which end the long poll. + const banStart = Date.now(); + try { + await eventually(() => assert.ok(deniedStatus(), JSON.stringify(mutedFanStatuses)), { + timeout: 75_000, + interval: 3000, + }); + process.stdout.write( + ` [fan-behavior-management] access denied seen after ${Date.now() - banStart} ms without traffic\n`, + ); + } catch { + await eventually( + async () => { + await server.publish({ channel: 'game.chat', message: { text: 'nudge' } }); + assert.ok(deniedStatus(), JSON.stringify(mutedFanStatuses)); + }, + { label: 'running fan learns about the ban', timeout: 120_000, interval: 5000 }, + ); + process.stdout.write( + ` [fan-behavior-management] access denied seen only with traffic, after ${Date.now() - banStart} ms\n`, + ); + } + assert.ok( + log.lines.includes('this fan may no longer write to game.chat'), + `the sample's access denied branch ran on the default subscribe loop. Log: ${log.lines.join(' | ')}`, + ); + mutedFan.destroy(); + + // The sample's branch does work with the legacy subscription manager: a fan with a revoked token is told at once. + reset(); + await startFan(mutedToken, { enableEventEngine: false }); + await eventually( + () => assert.ok(log.lines.includes('this fan may no longer write to game.chat'), log.lines.join(' | ')), + { label: 'legacy subscribe loop reports PNAccessDeniedCategory', timeout: 60_000 }, + ); + + // A banned fan gets no token at all, even from a fresh grant. + reset(); + await runServer('grant'); + assert.deepEqual(log.lines, [`${FAN} is banned, so no token was issued`]); + assert.deepEqual(readState()[FAN].tokens, []); + + // 7. unmute lifts the ban: the fan is active again and a fresh read-and-write token works. + reset(); + await runServer('unmute'); + assert.deepEqual(errors.lines, []); + state = readState(); + assert.equal(state[FAN].status, 'active'); + assert.equal(state[FAN].tokens.length, 1); + const backToken = lastToken(); + assert.ok( + log.lines.includes(`token that allows reading and writing chat again: ${backToken}`), + log.lines.join(' | '), + ); + assert.deepEqual(grantedChat(backToken), { read: true, write: true }); + reset(); + await startFan(backToken); + assert.deepEqual(errors.lines, [], 'the fan publishes again with the new token'); + assert.ok( + log.lines.some((line) => line.startsWith('chat message published at timetoken: ')), + log.lines.join(' | '), + ); + } finally { + log.restore(); + errors.restore(); + process.argv = originalArgv; + } + }); +}); diff --git a/test/integration/live/use-cases/fan-re-engagement.test.ts b/test/integration/live/use-cases/fan-re-engagement.test.ts new file mode 100644 index 000000000..58b2fb099 --- /dev/null +++ b/test/integration/live/use-cases/fan-re-engagement.test.ts @@ -0,0 +1,291 @@ +/* eslint-disable prettier/prettier, max-len -- sample text must stay character for character, so its long lines cannot be re-wrapped */ +import assert from 'node:assert/strict'; +import PubNub from 'pubnub'; +import { createClient, observer, track } from '../support/clients'; +import { captureConsole } from '../support/console'; +import { openKeys } from '../support/env'; +import { DEVICE_TOKEN_EXAMPLE } from '../support/resources'; +import { connected, eventually, PRESENCE_TIMEOUT, watch } from '../support/wait'; + +/** + * Fan re-engagement. Port of docs-snippets/use-cases/fan-re-engagement.ts. + * Needs Presence and Mobile Push Notifications on the open keyset. The test registers a device token for push but no real + * device exists, so it cannot verify that a push notification is DELIVERED; it verifies the publish to the fan's own alert + * channel, the payload, and the device registration. + */ +describe('use-cases/fan-re-engagement', () => { + it('reEngagementCountFansWatching, reEngagementPublishMomentAlert, reEngagementCheckOneFan, reEngagementWatchFansLeave, reEngagementFanViewer: the service counts fans, alerts only a fan who left, and a viewer stays reachable after it stops watching', async function () { + this.timeout(180_000); + const pubnub = createClient('open', 'match-service'); + const logs = captureConsole('log'); + const errors = captureConsole('error'); + const argvBefore = process.argv; + const sigintBefore = process.listeners('SIGINT'); + // Two stand-in fans that watch the stream: fan-b leaves in the middle of the flow, fan-a is the sample's viewer. + const fanB = createClient('open', 'fan-b'); + const fanBWatching = await watch(fanB, ['game.stream']); + const presenceSeen = async (userId: string, present = true) => { + await eventually( + async () => { + const here = await observer().hereNow({ channels: ['game.stream'], includeUUIDs: true }); + const listed = here.channels['game.stream']?.occupants.some((o) => o.uuid === userId) ?? false; + assert.equal(listed, present, `${userId} ${present ? 'is' : 'is not'} listed on game.stream`); + }, + { timeout: PRESENCE_TIMEOUT, label: `${userId} presence on game.stream` }, + ); + }; + try { + await presenceSeen('fan-b'); + + // snippet.reEngagementCountFansWatching + try { + const response = await pubnub.hereNow({ + channels: ['game.stream'], + includeUUIDs: false, + }); + console.log('fans currently watching the stream:', response.totalOccupancy); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Counting the fans watching failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + // snippet.end + + assert.deepEqual(errors.lines, [], 'the error path was not taken'); + const countLine = logs.lines.find((line) => line.startsWith('fans currently watching the stream: ')); + assert.ok(countLine, 'the sample printed the count'); + // Other users may watch too: the count includes at least fan-b. + assert.ok(Number(countLine.slice('fans currently watching the stream: '.length)) >= 1, countLine); + + // snippet.reEngagementPublishMomentAlert + async function sendMomentAlert(userId = '') { + const alertChannel = `game.moment-alerts.${userId}`; + const moment = PubNub.notificationPayload('Injury time', 'Two minutes left, and it is still 2-2.'); + + moment.sound = 'default'; + moment.apns.configurations = [{ targets: [{ topic: 'com.example.matchday' }] }]; + + try { + const response = await pubnub.publish({ + channel: alertChannel, + message: { + ...moment.buildPayload(['apns2', 'fcm']), + moment: 'injury-time', + }, + customMessageType: 'moment-alert', + }); + console.log(`moment alert published to ${alertChannel} at timetoken:`, response.timetoken); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Publishing the moment alert failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + } + // snippet.end + + const alerts = await watch(observer(), ['game.moment-alerts.fan-b']); + await sendMomentAlert('fan-b'); + assert.deepEqual(errors.lines, [], 'the error path was not taken'); + const alert = await alerts.next('message'); + assert.equal(alert.channel, 'game.moment-alerts.fan-b'); + assert.equal(alert.customMessageType, 'moment-alert'); + assert.equal(alert.publisher, 'match-service'); + const body = alert.message as { + moment?: string; + pn_fcm?: { notification?: { title?: string; body?: string } }; + pn_apns?: { aps?: { alert?: { title?: string } }; pn_push?: Array<{ targets?: Array<{ topic?: string }> }> }; + }; + assert.equal(body.moment, 'injury-time'); + assert.equal(body.pn_fcm?.notification?.title, 'Injury time'); + assert.equal(body.pn_fcm?.notification?.body, 'Two minutes left, and it is still 2-2.'); + assert.equal(body.pn_apns?.aps?.alert?.title, 'Injury time'); + assert.equal(body.pn_apns?.pn_push?.[0]?.targets?.[0]?.topic, 'com.example.matchday'); + assert.ok(logs.lines.some((line) => line.startsWith('moment alert published to game.moment-alerts.fan-b at timetoken: '))); + alerts.stop(); + + // snippet.reEngagementCheckOneFan + async function notifyIfAbsent(userId = '') { + try { + const response = await pubnub.whereNow({ uuid: userId }); + + if (response.channels.includes('game.stream')) { + console.log(`${userId} is still subscribed to game.stream, so no alert is needed`); + return; + } + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error( + `Checking where ${userId} is subscribed failed: ${error}${status ? ` Additional information: ${status}` : ''}`, + ); + return; + } + + console.log(`${userId} is not subscribed to game.stream, so sending a moment alert`); + await sendMomentAlert(userId); + } + // snippet.end + + // A fan who is watching gets no alert; an absent one does. Unique ids keep this run's alert channels apart. + logs.lines.length = 0; + const absent = `fan-absent-${Date.now()}`; + const absentAlerts = await watch(observer(), [`game.moment-alerts.${absent}`]); + const watchingAlerts = await watch(observer(), ['game.moment-alerts.fan-b']); + await notifyIfAbsent('fan-b'); + assert.ok(logs.lines.includes('fan-b is still subscribed to game.stream, so no alert is needed')); + await notifyIfAbsent(absent); + assert.ok(logs.lines.includes(`${absent} is not subscribed to game.stream, so sending a moment alert`)); + const absentAlert = await absentAlerts.next('message'); + assert.equal((absentAlert.message as { moment?: string }).moment, 'injury-time'); + // The absence of an alert for the watching fan is proven by the log above (no publish happens on that path) and by the + // channel staying silent over the time the absent fan's alert took to arrive. + assert.equal(watchingAlerts.events.filter((e) => e.kind === 'message').length, 0, 'the watching fan got no alert'); + assert.deepEqual(errors.lines, [], 'the error path was not taken'); + absentAlerts.stop(); + watchingAlerts.stop(); + + // snippet.reEngagementWatchFansLeave + const streamSubscription = pubnub.channel('game.stream').subscription({ receivePresenceEvents: true }); + + streamSubscription.onPresence = (event) => { + if (event.action === 'leave' || event.action === 'timeout') { + console.log(`${event.uuid} stopped watching, and ${event.occupancy} fans remain`); + void notifyIfAbsent(event.uuid); + } + }; + + streamSubscription.subscribe(); + // snippet.end + + // fan-b leaves: the service sees the leave, confirms through whereNow and alerts fan-b alone. + await eventually(async () => { + const here = await observer().hereNow({ channels: ['game.stream'], includeUUIDs: true }); + assert.ok(here.channels['game.stream']?.occupants.some((o) => o.uuid === 'match-service'), 'service is listed'); + }, { timeout: PRESENCE_TIMEOUT, label: 'service presence' }); + const leaveAlerts = await watch(observer(), ['game.moment-alerts.fan-b']); + logs.lines.length = 0; + fanBWatching.stop(); + await eventually(() => assert.ok(logs.lines.some((line) => line.startsWith('fan-b stopped watching, and '))), { + timeout: PRESENCE_TIMEOUT, + label: 'leave event handled', + }); + const leftAlert = await leaveAlerts.next('message', undefined, { timeout: 30_000 }); + assert.equal((leftAlert.message as { moment?: string }).moment, 'injury-time'); + assert.ok(logs.lines.includes('fan-b is not subscribed to game.stream, so sending a moment alert')); + assert.deepEqual(errors.lines, [], 'the error path was not taken'); + leaveAlerts.stop(); + + // The sample reads the fan from argv. Under mocha argv holds mocha's own arguments, so give it the shape of `node fan-viewer.js`. + process.argv = ['node', 'fan-viewer.js']; + logs.lines.length = 0; + // Setup of the viewer client the sample builds (kept out of the snippet so the hidden block stays one line): the test + // destroys it, and the sample shows a placeholder for the device token, so the real registration call gets a token of the + // right shape. + const prepareViewer = (viewerClient: PubNub): void => { + track(viewerClient); + const realAddChannels = viewerClient.push.addChannels.bind(viewerClient.push); + viewerClient.push.addChannels = (args) => + realAddChannels({ + ...args, + device: args.device === 'replace-with-the-fcm-registration-token' ? DEVICE_TOKEN_EXAMPLE : args.device, + }); + }; + // snippet.reEngagementFanViewer + const viewerUserId = process.argv[2] ?? 'fan-a'; + + const viewerClient = new PubNub({ + publishKey: 'demo', + subscribeKey: 'demo', + userId: viewerUserId, + // snippet.hide + ...openKeys(), + // snippet.show + }); + // snippet.hide + prepareViewer(viewerClient); + // snippet.show + + const watchSubscription = viewerClient.channel('game.stream').subscription({ receivePresenceEvents: false }); + watchSubscription.subscribe(); + + try { + const response = await viewerClient.push.addChannels({ + channels: [`game.moment-alerts.${viewerUserId}`], + device: 'replace-with-the-fcm-registration-token', + pushGateway: 'fcm', + }); + console.log(`${viewerUserId} registered its device for game.moment-alerts.${viewerUserId}:`, response); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Registering the device failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + + const alertSubscription = viewerClient.channel(`game.moment-alerts.${viewerUserId}`).subscription(); + + alertSubscription.onMessage = (event) => { + console.log(`${viewerUserId} received a moment alert:`, event.message); + }; + + alertSubscription.subscribe(); + + process.on('SIGINT', () => { + console.log(`${viewerUserId} left game.stream, but is still reachable for a moment alert`); + watchSubscription.unsubscribe(); + }); + // snippet.end + + process.argv = argvBefore; + assert.equal(viewerUserId, 'fan-a', 'the viewer defaults to fan-a'); + assert.deepEqual(errors.lines, [], 'the error path was not taken'); + assert.ok(logs.lines.some((line) => line.startsWith('fan-a registered its device for game.moment-alerts.fan-a:'))); + const registered = await observer().push.listChannels({ device: DEVICE_TOKEN_EXAMPLE, pushGateway: 'fcm' }); + assert.ok(registered.channels.includes('game.moment-alerts.fan-a'), 'the device is registered for the fan alert channel'); + await presenceSeen('fan-a'); + + // The fan's alert-channel subscription is live: an alert from the service reaches it. + await eventually(async () => { + const here = await observer().hereNow({ channels: ['game.moment-alerts.fan-a'], includeUUIDs: true }); + assert.ok(here.channels['game.moment-alerts.fan-a']?.occupants.some((o) => o.uuid === 'fan-a'), 'fan-a on alert channel'); + }, { timeout: PRESENCE_TIMEOUT, label: 'fan-a subscribed to its alert channel' }); + await sendMomentAlert('fan-a'); + await eventually(() => assert.ok(logs.lines.some((line) => line.startsWith('fan-a received a moment alert:'))), { + label: 'viewer received the alert', + }); + + // Ctrl+C: the viewer leaves game.stream; the service sees it, confirms and alerts; the viewer still receives it. + const handlers = process.listeners('SIGINT').filter((listener) => !sigintBefore.includes(listener)); + assert.equal(handlers.length, 1, 'the sample registered one SIGINT handler'); + logs.lines.length = 0; + handlers[0]('SIGINT'); + assert.ok(logs.lines.includes('fan-a left game.stream, but is still reachable for a moment alert')); + await eventually(() => assert.ok(logs.lines.some((line) => line.startsWith('fan-a stopped watching, and '))), { + timeout: PRESENCE_TIMEOUT, + label: 'service saw fan-a leave', + }); + await eventually(() => assert.ok(logs.lines.some((line) => line.startsWith('fan-a received a moment alert:'))), { + timeout: 30_000, + label: 'fan-a still receives the alert after leaving game.stream', + }); + assert.ok(logs.lines.includes('fan-a is not subscribed to game.stream, so sending a moment alert')); + assert.deepEqual(errors.lines, [], 'the error path was not taken'); + } finally { + process.argv = argvBefore; + process.listeners('SIGINT').forEach((listener) => { + if (!sigintBefore.includes(listener)) process.removeListener('SIGINT', listener); + }); + logs.restore(); + errors.restore(); + } + }); + + after(async () => { + // Best effort: drop the device registration the viewer sample made. + try { + const client = createClient('open', 'fan-a'); + await client.push.removeChannels({ + channels: ['game.moment-alerts.fan-a'], + device: DEVICE_TOKEN_EXAMPLE, + pushGateway: 'fcm', + }); + } catch { + // the janitor clears the device token as well + } + }); +}); diff --git a/test/integration/live/use-cases/game-chat-moderation.test.ts b/test/integration/live/use-cases/game-chat-moderation.test.ts new file mode 100644 index 000000000..110a00eb5 --- /dev/null +++ b/test/integration/live/use-cases/game-chat-moderation.test.ts @@ -0,0 +1,290 @@ +/* eslint-disable prettier/prettier, max-len -- sample text must stay character for character, so its long lines cannot be re-wrapped */ +import assert from 'node:assert/strict'; +import { assertTimetoken } from '../support/assertions'; +import { createClient } from '../support/clients'; +import { captureConsole } from '../support/console'; +import { keysOf } from '../support/env'; +import { withPlaceholders } from '../support/placeholders'; +import { connected, eventually, sleep } from '../support/wait'; + +/** + * Hide and remove chat messages. Port of docs-snippets/use-cases/game-chat-moderation.ts. + * Runs on the Access Manager keyset (grantToken needs the secret key); Message Persistence and Delete-From-History must be on. + */ +describe('use-cases/game-chat-moderation', () => { + const published: string[] = []; + const cleaner = () => + createClient('pam', 'moderation-cleaner', { secretKey: keysOf('pam').secretKey, heartbeatInterval: 0 }); + + after(async () => { + // Best effort: remove the messages this test left on the shared literal channels (one tick wide windows). + for (const [channel, timetoken] of published.map((entry) => entry.split(':') as [string, string])) { + try { + await cleaner().deleteMessages({ channel, start: (BigInt(timetoken) - BigInt(1)).toString(), end: timetoken }); + } catch { + // already deleted, or the keyset has no Delete-From-History + } + } + }); + + it('chatModerationGrantControlChannelAccess, chatModerationApplyModeratorToken, chatModerationFlagMessage, chatModerationReceiveModerationDecisions, chatModerationLoadHistoryWithFlags, chatModerationDeleteMessage: a moderator token lets moderator-7 flag a message, clients hide it live and from history, and the server deletes it for good', async function () { + this.timeout(120_000); + const { publishKey, subscribeKey, secretKey } = keysOf('pam'); + const logs = captureConsole('log'); + const errors = captureConsole('error'); + const realBigInt = globalThis.BigInt; + // The moderator's own client has no secret key; the server holds it. + // Placeholders of the samples (token, message timetoken) are filled in at call time from this object. + const placeholders: Record = {}; + const pubnub = withPlaceholders(createClient('pam', 'moderator-7'), placeholders); + const server = withPlaceholders( + createClient('pam', 'moderation-service', { secretKey, heartbeatInterval: 0 }), + placeholders, + ); + assert.ok(publishKey && subscribeKey); + try { + // snippet.chatModerationGrantControlChannelAccess + try { + const token = await server.grantToken({ + ttl: 60, + authorizedUserId: 'moderator-7', + resources: { + channels: { + 'game.chat': { read: true }, + 'game.chat.moderation': { read: true, write: true }, + }, + }, + }); + console.log('token that lets moderator-7 publish hide decisions:', token); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Granting moderator access failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + // snippet.end + + assert.deepEqual(errors.lines, [], 'the error path was not taken'); + const grantLine = logs.lines.find((line) => line.startsWith('token that lets moderator-7 publish hide decisions: ')); + assert.ok(grantLine, 'the sample printed the token'); + const token = grantLine.slice('token that lets moderator-7 publish hide decisions: '.length); + const parsed = server.parseToken(token); + assert.ok(parsed, 'the token parses'); + assert.equal(parsed.authorized_uuid, 'moderator-7'); + assert.equal(parsed.ttl, 60); + assert.deepEqual(parsed.resources?.channels?.['game.chat.moderation'], { + read: true, + write: true, + manage: false, + delete: false, + get: false, + update: false, + join: false, + }); + assert.equal(parsed.resources?.channels?.['game.chat']?.read, true); + assert.equal(parsed.resources?.channels?.['game.chat']?.write, false); + + // The sample shows a placeholder where the reader pastes the printed token, message timetoken and so on. + // This test chat message is what a moderator would flag. The server publishes it (fans' tokens are another page). + const flagged = await server.publish({ channel: 'game.chat', message: { text: `moderation test: flag me ${Date.now()}` } }); + const neighbour = await server.publish({ channel: 'game.chat', message: { text: `moderation test: keep me ${Date.now()}` } }); + published.push(`game.chat:${flagged.timetoken}`, `game.chat:${neighbour.timetoken}`); + placeholders['replace-with-the-token-server-js-printed'] = token; + placeholders['replace-with-message-timetoken'] = flagged.timetoken; + + // snippet.chatModerationApplyModeratorToken + pubnub.setToken('replace-with-the-token-server-js-printed'); + // snippet.end + + assert.equal(pubnub.getToken(), token, 'the moderator client now carries the granted token'); + // A grant takes a moment to apply on the service. Probe with a signal (not a message) until write is accepted. + await eventually(() => pubnub.signal({ channel: 'game.chat.moderation', message: 'probe' }), { + timeout: 30_000, + label: 'token accepted for write on game.chat.moderation', + }); + + // snippet.chatModerationFlagMessage + try { + const response = await pubnub.publish({ + channel: 'game.chat.moderation', + message: { + action: 'hide', + messageTimetoken: 'replace-with-message-timetoken', + }, + customMessageType: 'moderation-hide', + storeInHistory: true, + }); + console.log('message flagged at timetoken:', response.timetoken); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Flagging the message failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + // snippet.end + + assert.deepEqual(errors.lines, [], 'the error path was not taken'); + const flagLine = logs.lines.find((line) => line.startsWith('message flagged at timetoken: ')); + assert.ok(flagLine, 'the sample reported the flag'); + const flagTimetoken = flagLine.slice('message flagged at timetoken: '.length); + assertTimetoken(flagTimetoken); + published.push(`game.chat.moderation:${flagTimetoken}`); + + const ready = connected(pubnub, { channels: ['game.chat.moderation'] }); + // snippet.chatModerationReceiveModerationDecisions + const hiddenTimetokens = new Set(); + + const moderationSubscription = pubnub.channel('game.chat.moderation').subscription(); + + moderationSubscription.onMessage = (event) => { + const decision = event.message; + const action = + typeof decision === 'object' && decision !== null && !Array.isArray(decision) && 'action' in decision + ? decision.action + : undefined; + const messageTimetoken = + typeof decision === 'object' && decision !== null && !Array.isArray(decision) && 'messageTimetoken' in decision + ? decision.messageTimetoken + : undefined; + + if (action === 'hide' && typeof messageTimetoken === 'string') { + hiddenTimetokens.add(messageTimetoken); + console.log('hide the message published at', messageTimetoken); + } + }; + + moderationSubscription.subscribe(); + // snippet.end + + await ready; + // The page's own run order (everything is appended to moderator.js, in page order: apply token, flag, subscribe, history) + // publishes the flag before this subscription exists. A subscription starts at the current server time, so that earlier + // decision is NOT delivered live; moderator.js sees it only in the history fetch. Proved below: a decision published after + // the subscription is received, and the earlier flag was not received before it (messages arrive in order). + // A decision published after the subscription is delivered: flag a third message to prove the handler. + const live = await server.publish({ channel: 'game.chat', message: { text: `moderation test: live hide ${Date.now()}` } }); + published.push(`game.chat:${live.timetoken}`); + const liveDecision = await pubnub.publish({ + channel: 'game.chat.moderation', + message: { action: 'hide', messageTimetoken: live.timetoken }, + customMessageType: 'moderation-hide', + storeInHistory: true, + }); + published.push(`game.chat.moderation:${liveDecision.timetoken}`); + await eventually(() => assert.ok(hiddenTimetokens.has(live.timetoken), 'live decision handled'), { + label: 'live hide decision received', + }); + assert.ok(logs.lines.includes(`hide the message published at ${live.timetoken}`)); + assert.ok(!hiddenTimetokens.has(flagged.timetoken), 'the flag published before the subscription was not delivered live'); + assert.ok( + !logs.lines.includes(`hide the message published at ${flagged.timetoken}`), + 'the page order logs no live hide event for the flag published before the subscription', + ); + // A decision that is not a hide for a string timetoken is ignored. + await pubnub.publish({ channel: 'game.chat.moderation', message: { action: 'noop', messageTimetoken: neighbour.timetoken } }); + await pubnub.publish({ channel: 'game.chat.moderation', message: 'just text' }); + await sleep(1500); + assert.ok(!hiddenTimetokens.has(neighbour.timetoken), 'a non-hide decision hides nothing'); + assert.deepEqual(errors.lines, [], 'the error path was not taken'); + + // Wait until history holds the three chat messages and both hide decisions, so the history fetch sees them. + await eventually( + async () => { + const history = await server.fetchMessages({ channels: ['game.chat', 'game.chat.moderation'], count: 25 }); + const chat = (history.channels['game.chat'] ?? []).map((m) => m.timetoken); + const decisions = (history.channels['game.chat.moderation'] ?? []).map((m) => m.timetoken); + assert.ok([flagged, neighbour, live].every((m) => chat.includes(m.timetoken)), 'chat messages stored'); + assert.ok([flagTimetoken, liveDecision.timetoken].every((t) => decisions.includes(t)), 'decisions stored'); + }, + { timeout: 30_000, label: 'history holds this run' }, + ); + logs.lines.length = 0; + + // snippet.chatModerationLoadHistoryWithFlags + try { + const response = await pubnub.fetchMessages({ + channels: ['game.chat', 'game.chat.moderation'], + count: 25, + }); + + const moderationEntries = response.channels['game.chat.moderation'] ?? []; + + moderationEntries.forEach((entry) => { + const decision = entry.message; + const action = + typeof decision === 'object' && decision !== null && !Array.isArray(decision) && 'action' in decision + ? decision.action + : undefined; + const messageTimetoken = + typeof decision === 'object' && decision !== null && !Array.isArray(decision) && 'messageTimetoken' in decision + ? decision.messageTimetoken + : undefined; + + if (action === 'hide' && typeof messageTimetoken === 'string') { + hiddenTimetokens.add(messageTimetoken); + } + }); + + const chatEntries = response.channels['game.chat'] ?? []; + + chatEntries.forEach((entry) => { + const hidden = hiddenTimetokens.has(entry.timetoken.toString()); + console.log(entry.timetoken, hidden ? '[hidden by a moderator]' : entry.message); + }); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Loading the moderated history failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + // snippet.end + + assert.deepEqual(errors.lines, [], 'the error path was not taken'); + assert.ok(hiddenTimetokens.has(flagged.timetoken), 'history gave the hide decision of the flagged message'); + assert.ok(logs.lines.includes(`${flagged.timetoken} [hidden by a moderator]`), 'flagged message is rendered as hidden'); + assert.ok(logs.lines.includes(`${live.timetoken} [hidden by a moderator]`), 'live-flagged message is rendered as hidden'); + const neighbourLine = logs.lines.find((line) => line.startsWith(`${neighbour.timetoken} `)); + assert.ok(neighbourLine && neighbourLine.includes('keep me') && !neighbourLine.includes('hidden'), 'unflagged message keeps its text'); + + // The sample has a placeholder timetoken as a local variable that it passes to BigInt(). Substitute the placeholder + // inside BigInt for this run (the text readers see stays as it is); `end` goes through the placeholder wrapper. + globalThis.BigInt = Object.assign( + ((value: unknown) => realBigInt(value === 'replace-with-message-timetoken' ? flagged.timetoken : (value as string))) as typeof BigInt, + realBigInt, + ); + logs.lines.length = 0; + + // snippet.chatModerationDeleteMessage + try { + const messageTimetoken = 'replace-with-message-timetoken'; + const start = (BigInt(messageTimetoken) - BigInt(1)).toString(); + const end = messageTimetoken; + + const response = await server.deleteMessages({ + channel: 'game.chat', + start, + end, + }); + console.log('message deleted from Message Persistence:', response); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Deleting the message failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + // snippet.end + + globalThis.BigInt = realBigInt; + assert.deepEqual(errors.lines, [], 'the error path was not taken'); + assert.ok(logs.lines.some((line) => line.startsWith('message deleted from Message Persistence:'))); + // Only the flagged message is gone; the hide decision and the neighbouring messages stay. + await eventually( + async () => { + const history = await server.fetchMessages({ channels: ['game.chat', 'game.chat.moderation'], count: 25 }); + const chat = (history.channels['game.chat'] ?? []).map((m) => m.timetoken); + assert.ok(!chat.includes(flagged.timetoken), 'flagged message is deleted'); + assert.ok(chat.includes(neighbour.timetoken) && chat.includes(live.timetoken), 'the other messages stay'); + const decisions = (history.channels['game.chat.moderation'] ?? []).map((m) => m.timetoken); + assert.ok(decisions.includes(flagTimetoken), 'the hide decision stays in history'); + }, + { label: 'only the flagged message is deleted' }, + ); + } finally { + globalThis.BigInt = realBigInt; + logs.restore(); + errors.restore(); + } + }); +}); diff --git a/test/integration/live/use-cases/live-commentary.test.ts b/test/integration/live/use-cases/live-commentary.test.ts new file mode 100644 index 000000000..f2f83714d --- /dev/null +++ b/test/integration/live/use-cases/live-commentary.test.ts @@ -0,0 +1,175 @@ +/* eslint-disable prettier/prettier, max-len -- sample text must stay character for character, so its long lines cannot be re-wrapped */ +import assert from 'node:assert/strict'; +import { assertTimetoken } from '../support/assertions'; +import { createClient, observer } from '../support/clients'; +import { captureConsole } from '../support/console'; +import { storedMessage } from '../support/history'; +import { connected, eventually, sleep } from '../support/wait'; + +/** + * Live commentary tutorial. Port of docs-snippets/use-cases/live-commentary.ts. + * Needs Message Persistence on the open keyset (the backlog step reads stored remarks). + */ +describe('use-cases/live-commentary', () => { + it('liveCommentaryPublishRemark, liveCommentarySubscribe, liveCommentaryLoadBacklog, liveCommentaryUnsubscribe: a remark is stored, a viewer receives the next one live, catches up from the backlog and stops listening', async function () { + this.timeout(60_000); + const pubnub = createClient('open', 'fan-42'); + const logs = captureConsole('log'); + const errors = captureConsole('error'); + const sigintBefore = process.listeners('SIGINT'); + const realExit = process.exit; + const exitCodes: unknown[] = []; + try { + // snippet.liveCommentaryPublishRemark + try { + const response = await pubnub.publish({ + channel: 'game.commentary', + message: { + text: 'Long ball over the top, and the striker is clean through.', + clock: '48:07', + }, + customMessageType: 'commentary', + storeInHistory: true, + }); + console.log('commentary published at timetoken:', response.timetoken); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Publishing the commentary failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + // snippet.end + + const published = logs.lines.find((line) => line.startsWith('commentary published at timetoken: ')); + assert.ok(published, 'the sample reported the publish'); + const firstTimetoken = published.slice('commentary published at timetoken: '.length); + assertTimetoken(firstTimetoken); + assert.deepEqual(errors.lines, [], 'the error path was not taken'); + const stored = await eventually( + async () => { + const entry = await storedMessage(observer(), 'game.commentary', firstTimetoken); + assert.ok(entry, 'remark is stored'); + return entry; + }, + { label: 'commentary stored' }, + ); + assert.deepEqual(stored.message, { + text: 'Long ball over the top, and the striker is clean through.', + clock: '48:07', + }); + + const ready = connected(pubnub, { channels: ['game.commentary'] }); + // snippet.liveCommentarySubscribe + const commentaryChannel = pubnub.channel('game.commentary'); + const commentarySubscription = commentaryChannel.subscription({ receivePresenceEvents: false }); + + commentarySubscription.onMessage = (event) => { + console.log(`[${event.timetoken}] ${JSON.stringify(event.message)}`); + }; + + pubnub.addListener({ + status: (event) => { + if (event.category === 'PNConnectedCategory') { + console.log('connected and ready to receive commentary'); + } + }, + }); + + commentarySubscription.subscribe(); + // snippet.end + + await ready; + assert.ok(logs.lines.includes('connected and ready to receive commentary'), 'connected message printed'); + // A second remark from another client (the commentator), with a text unique to this run. + const liveText = `Corner kick, run ${Date.now()}`; + const live = await createClient('open', 'commentator').publish({ + channel: 'game.commentary', + message: { text: liveText, clock: '49:30' }, + customMessageType: 'commentary', + }); + await eventually( + () => { + assert.ok( + logs.lines.includes(`[${live.timetoken}] ${JSON.stringify({ text: liveText, clock: '49:30' })}`), + 'the viewer printed the live remark', + ); + }, + { label: 'live remark printed' }, + ); + assert.ok( + !logs.lines.some((line) => line.startsWith(`[${firstTimetoken}]`)), + 'the remark published before the subscription is not replayed live', + ); + + await eventually( + async () => { + assert.ok(await storedMessage(observer(), 'game.commentary', live.timetoken), 'live remark is stored'); + }, + { label: 'live remark stored' }, + ); + const beforeBacklog = logs.lines.length; + // snippet.liveCommentaryLoadBacklog + try { + const response = await pubnub.fetchMessages({ + channels: ['game.commentary'], + count: 25, + }); + + const entries = response.channels['game.commentary'] ?? []; + + entries.forEach((entry) => { + console.log(entry.timetoken, entry.message); + }); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Loading the commentary backlog failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + // snippet.end + + const backlog = logs.lines.slice(beforeBacklog); + assert.ok( + backlog.some((line) => line.startsWith(`${firstTimetoken} {`) && line.includes('"clock":"48:07"')), + 'the backlog has the first remark', + ); + assert.ok( + backlog.some((line) => line.startsWith(`${live.timetoken} {`) && line.includes(liveText)), + 'the backlog has the second remark', + ); + assert.deepEqual(errors.lines, [], 'the error path was not taken'); + + // snippet.liveCommentaryUnsubscribe + process.on('SIGINT', () => { + console.log('viewer shutting down, closing the commentary subscription'); + commentarySubscription.unsubscribe(); + process.exit(0); + }); + // snippet.end + + const handlers = process.listeners('SIGINT').filter((listener) => !sigintBefore.includes(listener)); + assert.equal(handlers.length, 1, 'the sample registered one SIGINT handler'); + // Run the handler as Ctrl+C would, without letting it stop the test process. + process.exit = ((code?: number) => { + exitCodes.push(code); + }) as typeof process.exit; + handlers[0]('SIGINT'); + process.exit = realExit; + assert.ok(logs.lines.includes('viewer shutting down, closing the commentary subscription')); + assert.deepEqual(exitCodes, [0]); + // The subscription is closed: a remark published now is not printed. Proving absence needs a wait. + const afterText = `After the whistle ${Date.now()}`; + await createClient('open', 'commentator').publish({ + channel: 'game.commentary', + message: { text: afterText, clock: '90:00' }, + customMessageType: 'commentary', + }); + await sleep(2500); + assert.ok(!logs.lines.some((line) => line.includes(afterText)), 'nothing is printed after unsubscribe'); + assert.deepEqual(errors.lines, [], 'the error path was not taken'); + } finally { + process.exit = realExit; + process.listeners('SIGINT').forEach((listener) => { + if (!sigintBefore.includes(listener)) process.removeListener('SIGINT', listener); + }); + logs.restore(); + errors.restore(); + } + }); +}); diff --git a/test/integration/live/use-cases/live-event-rate-limiting.test.ts b/test/integration/live/use-cases/live-event-rate-limiting.test.ts new file mode 100644 index 000000000..fdb27e4f1 --- /dev/null +++ b/test/integration/live/use-cases/live-event-rate-limiting.test.ts @@ -0,0 +1,137 @@ +/* eslint-disable prettier/prettier, max-len -- sample text must stay character for character, so its long lines cannot be re-wrapped */ +import assert from 'node:assert/strict'; +import { createClient, observer } from '../support/clients'; +import { captureConsole } from '../support/console'; +import { eventually, sleep, watch } from '../support/wait'; + +/** + * Live event rate limiting. Port of docs-snippets/use-cases/live-event-rate-limiting.ts. + * Needs Presence on the open keyset (the shard occupancy comes from Here Now). + */ +describe('use-cases/live-event-rate-limiting', () => { + it('rateLimitingReadShardOccupancy, rateLimitingPickShardForFan, rateLimitingThrottleFanPublishes: occupancy per shard is read, a fan is placed in a shard with room and its publishes are throttled on the client', async function () { + this.timeout(90_000); + const pubnub = createClient('open', 'fan-42'); + const logs = captureConsole('log'); + const errors = captureConsole('error'); + // Another fan sits in shard 0, so the shard has a known occupant. The hereNow of the open keyset is eventually consistent. + const occupant = await watch(observer(), ['game.chat.shard-0']); + try { + await eventually( + async () => { + const here = await observer().hereNow({ channels: ['game.chat.shard-0'], includeUUIDs: true }); + assert.ok(here.channels['game.chat.shard-0']?.occupants.some((o) => o.uuid === 'observer'), 'observer is listed'); + }, + { timeout: 30_000, label: 'occupant visible in hereNow' }, + ); + + // snippet.rateLimitingReadShardOccupancy + try { + const response = await pubnub.hereNow({ + channels: ['game.chat.shard-0', 'game.chat.shard-1', 'game.chat.shard-2'], + includeUUIDs: false, + }); + + Object.entries(response.channels).forEach(([channel, data]) => { + console.log(`${channel} holds ${data.occupancy} fans`); + }); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Reading the shard occupancy failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + // snippet.end + + assert.deepEqual(errors.lines, [], 'the error path was not taken'); + // Other tests may have fans in the shards, so only the known occupant is asserted: shard 0 holds at least one fan. + const shard0 = logs.lines.find((line) => line.startsWith('game.chat.shard-0 holds ')); + assert.ok(shard0, `a line for shard 0 in ${JSON.stringify(logs.lines)}`); + assert.match(shard0, /^game\.chat\.shard-0 holds [1-9]\d* fans$/); + logs.lines.length = 0; + + // snippet.rateLimitingPickShardForFan + async function pickShardForFan(shardCount = 0, maxFansPerShard = 0) { + const channels = Array.from({ length: shardCount }, (_, index) => `game.chat.shard-${index}`); + + const response = await pubnub.hereNow({ channels, includeUUIDs: false }); + + const openShard = channels.find((channel) => (response.channels[channel]?.occupancy ?? 0) < maxFansPerShard); + + if (!openShard) { + throw new Error('every shard is full, so add more shards before the next match'); + } + + return openShard; + } + + const shard = await pickShardForFan(10, 1000); + + console.log('this fan joins', shard); + // snippet.end + + // The first shard below the limit is shard 0: it holds one or a few fans, far below 1000. + assert.equal(shard, 'game.chat.shard-0'); + assert.deepEqual(logs.lines, ['this fan joins game.chat.shard-0']); + // A limit of 1 makes shard 0 (which holds the occupant) full, so the next shard with room is picked. + assert.notEqual(await pickShardForFan(3, 1), 'game.chat.shard-0'); + await assert.rejects(pickShardForFan(1, 1), /every shard is full/); + const watching = await watch(observer(), [shard]); + logs.lines.length = 0; + + // snippet.rateLimitingThrottleFanPublishes + const minimumMillisecondsBetweenMessages = 2000; + let lastPublishedAt = 0; + + async function sendChatMessage(text = '') { + const now = Date.now(); + + if (now - lastPublishedAt < minimumMillisecondsBetweenMessages) { + console.log('too soon, so this message is dropped before it reaches the network'); + return; + } + + lastPublishedAt = now; + + await pubnub.publish({ + channel: shard, + message: { text }, + customMessageType: 'text-message', + }); + } + + await sendChatMessage('Offside, surely?'); + // snippet.end + + // The sample does not keep the publish response, so match the message by publisher and text. + const first = await watching.next( + 'message', + (m) => (m.message as { text?: string }).text === 'Offside, surely?' && m.publisher === 'fan-42', + ); + assert.equal(first.channel, shard); + assert.equal(first.customMessageType, 'text-message'); + assert.deepEqual(logs.lines, [], 'the first message is not throttled'); + + // A second message inside the 2 s window is dropped before the network; a fixed wait proves it never arrives. + const tooSoonText = `too soon ${Date.now()}`; + await sendChatMessage(tooSoonText); + assert.deepEqual(logs.lines, ['too soon, so this message is dropped before it reaches the network']); + await sleep(1500); + assert.ok( + !watching.events.some((e) => (e.event as { message?: { text?: string } }).message?.text === tooSoonText), + 'the throttled message never reached the channel', + ); + + // After the window has passed, the next message goes out. + await sleep(2100 - 1500); + const laterText = `later ${Date.now()}`; + await sendChatMessage(laterText); + await watching.next('message', (m) => (m.message as { text?: string }).text === laterText); + assert.equal(logs.lines.length, 1, 'no further throttle log'); + assert.deepEqual(errors.lines, [], 'the error path was not taken'); + watching.stop(); + } finally { + occupant.stop(); + logs.restore(); + errors.restore(); + } + }); +}); diff --git a/test/integration/live/use-cases/live-polls.test.ts b/test/integration/live/use-cases/live-polls.test.ts new file mode 100644 index 000000000..6e2d5aa0e --- /dev/null +++ b/test/integration/live/use-cases/live-polls.test.ts @@ -0,0 +1,278 @@ +import assert from 'node:assert/strict'; +import { randomUUID } from 'node:crypto'; +import PubNub from 'pubnub'; +import { assertTimetoken } from '../support/assertions'; +import { createClient, observer, track } from '../support/clients'; +import { captureConsole } from '../support/console'; +import { keysOf } from '../support/env'; +import { connected, eventually, sleep, watch } from '../support/wait'; + +/** + * Port of docs-snippets/use-cases/live-polls.ts (page use-cases/sports-media-entertainment/live-polls). + * One flow over three channels: open a poll, a late fan fetches it, the fan votes, the service publishes the tally and the + * fan receives it. Needs Message Persistence on the open keyset. + */ +describe('use-cases/live-polls', () => { + const CHANNELS = ['game.new-poll', 'game.poll-votes', 'game.poll-results']; + // The three channels are shared literals: clear their history so the late-joiner fetch reads this run's poll. + const clearHistory = async (): Promise => { + const cleaner = createClient('open', 'live-polls-cleaner'); + for (const channel of CHANNELS) { + try { + await cleaner.deleteMessages({ channel }); + } catch { + // Best effort. + } + } + }; + before(clearHistory); + after(clearHistory); + + it('livePollsPublishPoll, livePollsSubscribeToPolls, livePollsFetchOpenPoll, livePollsPublishVote, livePollsSubscribeToResults, livePollsPublishResults: a poll opens, a late fan fetches it, votes, and both see the tally', async function () { + this.timeout(90_000); + const { publishKey, subscribeKey } = keysOf('open'); + const log = captureConsole('log'); + const errors = captureConsole('error'); + try { + // The poll service (open-poll.js) and the fan (vote.js) are two clients, as on the page. + const service = track(new PubNub({ publishKey, subscribeKey, userId: 'match-service' })); + const fan = track(new PubNub({ publishKey, subscribeKey, userId: 'fan-42' })); + // An independent client records every message on the three channels. + const seen = await watch(observer(), CHANNELS); + const timetokenOf = (prefix: string): string => { + const line = log.lines.find((l) => l.startsWith(prefix)); + assert.ok( + line, + `expected a log line "${prefix}". Logged: ${log.lines.join(' | ')}. Errors: ${errors.lines.join(' | ')}`, + ); + const timetoken = line.slice(prefix.length); + assertTimetoken(timetoken); + return timetoken; + }; + + // 1. open-poll.js publishes the poll. + { + const pubnub = service; + /* eslint-disable prettier/prettier */ + // snippet.livePollsPublishPoll + const poll = { + id: 'poll-1', + title: 'Who will win the match?', + durationSeconds: 60, + options: [ + { id: 1, text: 'Home team' }, + { id: 2, text: 'Away team' }, + { id: 3, text: 'Draw' }, + ], + }; + + try { + const response = await pubnub.publish({ + channel: 'game.new-poll', + message: poll, + customMessageType: 'poll-opened', + storeInHistory: true, + }); + console.log('poll published at timetoken:', response.timetoken); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Publishing the poll failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + // snippet.end + /* eslint-enable prettier/prettier */ + assert.deepEqual(errors.lines, [], 'the poll publish did not take the error path'); + assert.deepEqual(poll.options.length, 3); + } + const pollTimetoken = timetokenOf('poll published at timetoken: '); + const openedEvent = await seen.next('message', (m) => m.timetoken === pollTimetoken); + assert.equal(openedEvent.channel, 'game.new-poll'); + assert.equal(openedEvent.customMessageType, 'poll-opened'); + assert.equal(openedEvent.publisher, 'match-service'); + assert.equal((openedEvent.message as { id: string }).id, 'poll-1'); + + // 2. vote.js starts after the publish: its live subscription must not deliver the earlier poll. + { + const pubnub = fan; + const ready = connected(pubnub, { channels: ['game.new-poll'] }); + /* eslint-disable prettier/prettier */ + // snippet.livePollsSubscribeToPolls + const pollChannel = pubnub.channel('game.new-poll'); + const pollSubscription = pollChannel.subscription({ receivePresenceEvents: false }); + + pollSubscription.onMessage = (event) => { + console.log('new poll:', event.message); + }; + + pollSubscription.subscribe(); + // snippet.end + /* eslint-enable prettier/prettier */ + await ready; + assert.deepEqual(pubnub.getSubscribedChannels(), ['game.new-poll']); + } + // Proves an absence: a subscription only delivers what is published after it connects. + await sleep(2000); + assert.deepEqual( + log.lines.filter((line) => line.startsWith('new poll:')), + [], + 'the poll published before the subscription is not delivered live', + ); + + // 3. ... so the fan fetches the stored poll instead. Message Persistence lags the publish, so the sample runs again until + // it finds this run's poll. + const fetchOpenPoll = async (): Promise => { + const pubnub = fan; + /* eslint-disable prettier/prettier */ + // snippet.livePollsFetchOpenPoll + try { + const response = await pubnub.fetchMessages({ + channels: ['game.new-poll'], + count: 1, + }); + + const entries = response.channels['game.new-poll']; + + if (entries && entries.length > 0) { + console.log('poll that is already open:', entries[0].message); + } + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Fetching the open poll failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + // snippet.end + /* eslint-enable prettier/prettier */ + }; + await eventually( + async () => { + log.lines.length = 0; + await fetchOpenPoll(); + assert.deepEqual(errors.lines, [], 'the fetch did not take the error path'); + assert.equal(log.lines.length, 1, `logged: ${log.lines.join(' | ')}`); + }, + { label: 'the stored poll is fetched' }, + ); + // The service returns the keys in its own order, so compare the parsed object. + assert.ok(log.lines[0].startsWith('poll that is already open: ')); + assert.deepEqual(JSON.parse(log.lines[0].slice('poll that is already open: '.length)), { + id: 'poll-1', + title: 'Who will win the match?', + durationSeconds: 60, + options: [ + { id: 1, text: 'Home team' }, + { id: 2, text: 'Away team' }, + { id: 3, text: 'Draw' }, + ], + }); + // The fetched message is the one of this run (history of the channel was cleared first). + const stored = await observer().fetchMessages({ channels: ['game.new-poll'], count: 1 }); + assert.equal(stored.channels['game.new-poll'][0].timetoken, pollTimetoken); + + // The subscription is live: a poll opened now arrives (a unique poll, published outside the samples). + const liveId = `poll-live-${randomUUID()}`; + await service.publish({ channel: 'game.new-poll', message: { id: liveId, title: 'Live', options: [] } }); + await eventually(() => + assert.ok( + log.lines.some((l) => l.startsWith('new poll:') && l.includes(liveId)), + log.lines.join(' | '), + ), + ); + + // 4. the fan votes on game.poll-votes. + log.lines.length = 0; + { + const pubnub = fan; + /* eslint-disable prettier/prettier */ + // snippet.livePollsPublishVote + try { + const response = await pubnub.publish({ + channel: 'game.poll-votes', + message: { pollId: 'poll-1', optionId: 2 }, + customMessageType: 'poll-vote', + }); + console.log('vote published at timetoken:', response.timetoken); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Publishing the vote failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + // snippet.end + /* eslint-enable prettier/prettier */ + } + assert.deepEqual(errors.lines, [], 'the vote publish did not take the error path'); + const voteTimetoken = timetokenOf('vote published at timetoken: '); + const vote = await seen.next('message', (m) => m.timetoken === voteTimetoken); + assert.equal(vote.channel, 'game.poll-votes'); + assert.equal(vote.customMessageType, 'poll-vote'); + assert.deepEqual(vote.message, { pollId: 'poll-1', optionId: 2 }); + // The vote names no voter; PubNub records the publisher. + assert.equal(vote.publisher, 'fan-42'); + + // 5. both sides subscribe to the results before the tally is published (the page says to do it before the timer fires). + log.lines.length = 0; + const fanReady = connected(fan, { channels: ['game.poll-results'] }); + { + const pubnub = fan; + /* eslint-disable prettier/prettier */ + // snippet.livePollsSubscribeToResults + const resultsChannel = pubnub.channel('game.poll-results'); + const resultsSubscription = resultsChannel.subscription({ receivePresenceEvents: false }); + + resultsSubscription.onMessage = (event) => { + console.log('poll results:', event.message); + }; + + resultsSubscription.subscribe(); + // snippet.end + /* eslint-enable prettier/prettier */ + assert.ok(resultsSubscription); + } + await fanReady; + assert.ok(fan.getSubscribedChannels().includes('game.poll-results')); + + // 6. the service publishes the tally. + { + const pubnub = service; + /* eslint-disable prettier/prettier */ + // snippet.livePollsPublishResults + try { + const response = await pubnub.publish({ + channel: 'game.poll-results', + message: { + pollId: 'poll-1', + totals: { 1: 812, 2: 1043, 3: 219 }, + closed: true, + }, + customMessageType: 'poll-results', + storeInHistory: true, + }); + console.log('results published at timetoken:', response.timetoken); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Publishing the results failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + // snippet.end + /* eslint-enable prettier/prettier */ + } + assert.deepEqual(errors.lines, [], 'the results publish did not take the error path'); + const resultsTimetoken = timetokenOf('results published at timetoken: '); + const tally = await seen.next('message', (m) => m.timetoken === resultsTimetoken); + assert.equal(tally.channel, 'game.poll-results'); + assert.equal(tally.customMessageType, 'poll-results'); + assert.equal(tally.publisher, 'match-service'); + assert.deepEqual(tally.message, { pollId: 'poll-1', totals: { 1: 812, 2: 1043, 3: 219 }, closed: true }); + // The fan's subscription received the same tally. + await eventually(() => + assert.ok( + log.lines.includes('poll results: {"pollId":"poll-1","totals":{"1":812,"2":1043,"3":219},"closed":true}'), + log.lines.join(' | '), + ), + ); + // storeInHistory on the results: the tally is readable after the fact. + await eventually(async () => { + const history = await observer().fetchMessages({ channels: ['game.poll-results'], count: 1 }); + assert.equal(history.channels['game.poll-results']?.[0].timetoken, resultsTimetoken); + }); + } finally { + log.restore(); + errors.restore(); + } + }); +}); diff --git a/test/integration/live/use-cases/match-stats.test.ts b/test/integration/live/use-cases/match-stats.test.ts new file mode 100644 index 000000000..b9265b1fd --- /dev/null +++ b/test/integration/live/use-cases/match-stats.test.ts @@ -0,0 +1,158 @@ +import assert from 'node:assert/strict'; +import { randomUUID } from 'node:crypto'; +import PubNub from 'pubnub'; +import { assertTimetoken } from '../support/assertions'; +import { createClient, observer, track } from '../support/clients'; +import { captureConsole } from '../support/console'; +import { keysOf } from '../support/env'; +import { storedMessage } from '../support/history'; +import { connected, eventually, watch } from '../support/wait'; + +/** + * Port of docs-snippets/use-cases/match-stats.ts (page use-cases/sports-media-entertainment/match-stats). + * One flow: a scoreboard subscribes to the wildcard, a feed publishes one stat, a late scoreboard fills in current values. + * Needs wildcard subscribe (Stream Controller) and Message Persistence on the open keyset. + */ +describe('use-cases/match-stats', () => { + const STATS = ['score', 'possession', 'shots', 'cards']; + const channelOf = (stat: string): string => `game.match-stats.${stat}`; + // The sample channels are shared literals: clear their history so "currently" can be asserted exactly for this run. + const clearHistory = async (): Promise => { + const cleaner = createClient('open', 'match-stats-cleaner'); + for (const stat of STATS) { + try { + await cleaner.deleteMessages({ channel: channelOf(stat) }); + } catch { + // Best effort. + } + } + }; + before(clearHistory); + after(clearHistory); + + it('matchStatsPublishOneStat, matchStatsSubscribeToAllStats, matchStatsFetchCurrentValues: a published stat reaches the wildcard subscription and a late reader gets the current value of every stat', async function () { + this.timeout(90_000); + const { publishKey, subscribeKey } = keysOf('open'); + const log = captureConsole('log'); + const errors = captureConsole('error'); + try { + // Hidden from readers: the page shows its own `new PubNub` with placeholder keys. + const pubnub = track(new PubNub({ publishKey, subscribeKey, userId: 'fan-42' })); + // An independent client that records everything on the stat channels, to prove delivery and the publish options. + const seen = await watch(observer(), STATS.map(channelOf)); + + // The scoreboard starts first (page: "Run it"), then the feed publishes. + const ready = connected(pubnub, { channels: ['game.match-stats.*'] }); + /* eslint-disable prettier/prettier */ + // snippet.matchStatsSubscribeToAllStats + const statsSubscription = pubnub.subscriptionSet({ + channels: ['game.match-stats.*'], + subscriptionOptions: { receivePresenceEvents: false }, + }); + + statsSubscription.onMessage = (event) => { + const statName = event.channel.split('.').pop(); + console.log(`${statName} is now`, event.message); + }; + + statsSubscription.subscribe(); + // snippet.end + /* eslint-enable prettier/prettier */ + await ready; + assert.deepEqual(pubnub.getSubscribedChannels(), ['game.match-stats.*']); + + /* eslint-disable prettier/prettier */ + // snippet.matchStatsPublishOneStat + try { + const response = await pubnub.publish({ + channel: 'game.match-stats.score', + message: { value: '3-1' }, + customMessageType: 'match-stat', + storeInHistory: true, + }); + console.log('score published at timetoken:', response.timetoken); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Publishing the stat failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + // snippet.end + /* eslint-enable prettier/prettier */ + assert.deepEqual(errors.lines, [], 'the publish did not take the error path'); + const publishedLine = log.lines.find((line) => line.startsWith('score published at timetoken: ')); + assert.ok(publishedLine, `logged: ${log.lines.join(' | ')}`); + const scoreTimetoken = publishedLine.split(': ')[1]; + assertTimetoken(scoreTimetoken); + + // The wildcard subscription delivered it, and the channel name supplied the stat name. + await eventually(() => assert.ok(log.lines.includes('score is now {"value":"3-1"}'), log.lines.join(' | '))); + const delivered = await seen.next('message', (m) => m.timetoken === scoreTimetoken); + assert.equal(delivered.channel, 'game.match-stats.score'); + assert.equal(delivered.customMessageType, 'match-stat'); + assert.equal(delivered.publisher, 'fan-42'); + // storeInHistory: the value is in Message Persistence under that very timetoken. + await eventually(async () => { + const stored = await storedMessage(observer(), 'game.match-stats.score', scoreTimetoken); + assert.deepEqual(stored?.message, { value: '3-1' }); + }); + + // A stat that changes while nobody listens: the feed publishes it, a scoreboard starting later reads it back. + const feed = createClient('open', 'match-service'); + const possession = { value: `54%-${randomUUID()}` }; + const possessionTimetoken = ( + await feed.publish({ channel: channelOf('possession'), message: possession, storeInHistory: true }) + ).timetoken; + await eventually(async () => { + const stored = await storedMessage(observer(), channelOf('possession'), possessionTimetoken); + assert.deepEqual(stored?.message, possession); + }); + + // A late scoreboard: its own client, so the fetch does not rely on the live subscription above. + const lateScoreboard = track(new PubNub({ publishKey, subscribeKey, userId: 'fan-43' })); + const fetchCurrentValues = async (pubnub: PubNub): Promise => { + /* eslint-disable prettier/prettier */ + // snippet.matchStatsFetchCurrentValues + const statNames = ['score', 'possession', 'shots', 'cards']; + + try { + const response = await pubnub.fetchMessages({ + channels: statNames.map((statName) => `game.match-stats.${statName}`), + count: 1, + }); + + statNames.forEach((statName) => { + const entries = response.channels[`game.match-stats.${statName}`]; + + if (entries && entries.length > 0) { + console.log(`${statName} is currently`, entries[0].message); + } + }); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Fetching the current stats failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + // snippet.end + /* eslint-enable prettier/prettier */ + }; + // Message Persistence is eventually consistent, so the sample runs again until the stored values show up. + await eventually( + async () => { + log.lines.length = 0; + await fetchCurrentValues(lateScoreboard); + assert.deepEqual(errors.lines, [], 'the fetch did not take the error path'); + assert.ok(log.lines.includes('score is currently {"value":"3-1"}'), `logged: ${log.lines.join(' | ')}`); + assert.ok( + log.lines.includes(`possession is currently ${JSON.stringify(possession)}`), + `logged: ${log.lines.join(' | ')}`, + ); + }, + { label: 'current value of every stat', timeout: 30_000 }, + ); + // shots and cards were never published in this run (history cleared first): nothing is printed for them. + assert.ok(!log.lines.some((line) => line.startsWith('shots is currently'))); + assert.ok(!log.lines.some((line) => line.startsWith('cards is currently'))); + } finally { + log.restore(); + errors.restore(); + } + }); +}); diff --git a/test/integration/live/use-cases/real-time-ads.test.ts b/test/integration/live/use-cases/real-time-ads.test.ts new file mode 100644 index 000000000..64565ef8c --- /dev/null +++ b/test/integration/live/use-cases/real-time-ads.test.ts @@ -0,0 +1,123 @@ +/* eslint-disable prettier/prettier, max-len -- sample text must stay character for character, so its long lines cannot be re-wrapped */ +import assert from 'node:assert/strict'; +import { assertTimetoken } from '../support/assertions'; +import { createClient, observer } from '../support/clients'; +import { captureConsole } from '../support/console'; +import { connected, eventually, watch } from '../support/wait'; + +/** + * Real-time ads and reaction upgrades. Port of docs-snippets/use-cases/real-time-ads.ts. + * The ad decisions and reaction upgrades come from Illuminate Business Rules on the real service. The keyset used by the tier + * has no such rule, so the test publishes the decisions itself, with the payload shape the page describes, and verifies the + * client side: what the fan publishes and how the client reacts to a decision. + */ +describe('use-cases/real-time-ads', () => { + it('realTimeAdsPublishReaction, realTimeAdsReceiveAdDecision, realTimeAdsReceiveReactionUpgrade: a fan publishes a reaction, then shows an ad, clears the slot and renders an upgraded reaction on decisions', async function () { + this.timeout(60_000); + const pubnub = createClient('open', 'fan-42'); + const logs = captureConsole('log'); + const errors = captureConsole('error'); + try { + const reactions = await watch(observer(), ['game.stream-reactions']); + + // snippet.realTimeAdsPublishReaction + try { + const response = await pubnub.publish({ + channel: 'game.stream-reactions', + message: { reaction: '\u{1F525}' }, + customMessageType: 'reaction', + }); + console.log('reaction published at timetoken:', response.timetoken); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Publishing the reaction failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + // snippet.end + + const line = logs.lines.find((l) => l.startsWith('reaction published at timetoken: ')); + assert.ok(line, 'the sample reported the publish'); + const timetoken = line.slice('reaction published at timetoken: '.length); + assertTimetoken(timetoken); + assert.deepEqual(errors.lines, [], 'the error path was not taken'); + const received = await reactions.next('message', (m) => m.timetoken === timetoken); + assert.deepEqual(received.message, { reaction: '\u{1F525}' }); + assert.equal(received.publisher, 'fan-42'); + assert.equal(received.customMessageType, 'reaction'); + reactions.stop(); + + const adReady = connected(pubnub, { channels: ['game.ad-decisions'] }); + // snippet.realTimeAdsReceiveAdDecision + const adSubscription = pubnub.channel('game.ad-decisions').subscription({ receivePresenceEvents: false }); + + adSubscription.onMessage = (event) => { + const decision = event.message; + const adId = + typeof decision === 'object' && decision !== null && !Array.isArray(decision) && 'adId' in decision + ? decision.adId + : undefined; + const clickPoints = + typeof decision === 'object' && decision !== null && !Array.isArray(decision) && 'clickPoints' in decision + ? decision.clickPoints + : undefined; + + if (adId) { + console.log(`show ad ${adId}, worth ${clickPoints} points`); + } else { + console.log('no ad to show, so clear the ad slot'); + } + }; + + adSubscription.subscribe(); + // snippet.end + + await adReady; + // Stand-in for the Illuminate action: another client publishes the decisions. + const decider = createClient('open', 'illuminate-stand-in'); + const adId = `ad-${Date.now()}`; + await decider.publish({ channel: 'game.ad-decisions', message: { adId, clickPoints: 25 } }); + await eventually(() => assert.ok(logs.lines.includes(`show ad ${adId}, worth 25 points`)), { + label: 'ad shown', + }); + assert.ok(!logs.lines.includes('no ad to show, so clear the ad slot'), 'no clear before the empty decision'); + await decider.publish({ channel: 'game.ad-decisions', message: {} }); + await eventually(() => assert.ok(logs.lines.includes('no ad to show, so clear the ad slot')), { + label: 'ad slot cleared', + }); + + const upgradeReady = connected(pubnub, { channels: ['game.reaction-upgrades'] }); + // snippet.realTimeAdsReceiveReactionUpgrade + const upgradeSubscription = pubnub.channel('game.reaction-upgrades').subscription({ receivePresenceEvents: false }); + + upgradeSubscription.onMessage = (event) => { + const upgrade = event.message; + const reaction = + typeof upgrade === 'object' && upgrade !== null && !Array.isArray(upgrade) && 'reaction' in upgrade + ? upgrade.reaction + : undefined; + const replacement = + typeof upgrade === 'object' && upgrade !== null && !Array.isArray(upgrade) && 'replacement' in upgrade + ? upgrade.replacement + : undefined; + + console.log(`render ${reaction} as ${replacement} from now on`); + }; + + upgradeSubscription.subscribe(); + // snippet.end + + await upgradeReady; + await decider.publish({ + channel: 'game.reaction-upgrades', + message: { reaction: '\u{1F525}', replacement: '\u{1F680}' }, + }); + await eventually( + () => assert.ok(logs.lines.includes('render \u{1F525} as \u{1F680} from now on'), 'upgrade rendered'), + { label: 'reaction upgrade printed' }, + ); + assert.deepEqual(errors.lines, [], 'the error path was not taken'); + } finally { + logs.restore(); + errors.restore(); + } + }); +}); diff --git a/test/integration/live/use-cases/real-time-chat.test.ts b/test/integration/live/use-cases/real-time-chat.test.ts new file mode 100644 index 000000000..9d8e8aae1 --- /dev/null +++ b/test/integration/live/use-cases/real-time-chat.test.ts @@ -0,0 +1,286 @@ +import assert from 'node:assert/strict'; +import { randomUUID } from 'node:crypto'; +import PubNub from 'pubnub'; +import { assertTimetoken } from '../support/assertions'; +import { createClient, observer, track } from '../support/clients'; +import { captureConsole } from '../support/console'; +import { keysOf } from '../support/env'; +import { withPlaceholders } from '../support/placeholders'; +import { connected, eventually, PRESENCE_TIMEOUT } from '../support/wait'; + +/** + * Port of docs-snippets/use-cases/real-time-chat.ts (page use-cases/sports-media-entertainment/real-time-chat). + * One flow, as the page runs it: set channel and fan metadata, join with presence, send, load history, count fans, react. + * Needs Presence, Message Persistence and App Context on the open keyset. + */ +describe('use-cases/real-time-chat', () => { + const clean = async (): Promise => { + const cleaner = createClient('open', 'real-time-chat-cleaner'); + // Best effort: everything the sample writes under its literal names. + await cleaner.deleteMessages({ channel: 'game.chat' }).catch(() => undefined); + await cleaner.objects.removeChannelMetadata({ channel: 'game.chat' }).catch(() => undefined); + await cleaner.objects.removeUUIDMetadata({ uuid: 'fan-42' }).catch(() => undefined); + }; + before(clean); + after(clean); + + it('fanChatCreateChannelMetadata, fanChatSetFanProfile, fanChatSubscribeWithPresence, fanChatSendMessage, fanChatLoadRecentMessages, fanChatCountFansOnline, fanChatReceiveReactions, fanChatAddReaction: two fans chat, see presence and history, and a reaction reaches the subscriber', async function () { + this.timeout(120_000); + const { publishKey, subscribeKey } = keysOf('open'); + const log = captureConsole('log'); + const errors = captureConsole('error'); + try { + const reader = observer(); + // The second fan of the page (index2.js). + const fan108 = createClient('open', 'fan-108'); + // The page tells readers to replace the timetoken placeholder by the timetoken of the message they sent. Only that + // argument is swapped, the value is filled in below once the message exists. + const placeholders: Record = {}; + const pubnub = withPlaceholders(track(new PubNub({ publishKey, subscribeKey, userId: 'fan-42' })), placeholders); + + /* eslint-disable prettier/prettier */ + // snippet.fanChatCreateChannelMetadata + try { + const response = await pubnub.objects.setChannelMetadata({ + channel: 'game.chat', + data: { + name: 'Match chat', + description: 'Open chat for everyone watching the match', + }, + }); + console.log('channel metadata set:', response.data); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Setting the channel metadata failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + // snippet.end + /* eslint-enable prettier/prettier */ + assert.deepEqual(errors.lines, [], 'channel metadata: error path not taken'); + assert.ok(log.lines.some((l) => l.startsWith('channel metadata set: ') && l.includes('"name":"Match chat"'))); + // Another client reads the same header back. + await eventually(async () => { + const read = await reader.objects.getChannelMetadata({ channel: 'game.chat' }); + assert.equal(read.data.name, 'Match chat'); + assert.equal(read.data.description, 'Open chat for everyone watching the match'); + }); + + /* eslint-disable prettier/prettier */ + // snippet.fanChatSetFanProfile + try { + const response = await pubnub.objects.setUUIDMetadata({ + uuid: 'fan-42', + data: { + name: 'Alex Moreau', + profileUrl: 'https://example.com/avatars/fan-42.png', + custom: { supports: 'home' }, + }, + }); + console.log('fan profile set:', response.data); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Setting the fan profile failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + // snippet.end + /* eslint-enable prettier/prettier */ + assert.deepEqual(errors.lines, [], 'fan profile: error path not taken'); + assert.ok(log.lines.some((l) => l.startsWith('fan profile set: ') && l.includes('"supports":"home"'))); + await eventually(async () => { + const read = await reader.objects.getUUIDMetadata({ uuid: 'fan-42' }); + assert.equal(read.data.name, 'Alex Moreau'); + assert.equal(read.data.profileUrl, 'https://example.com/avatars/fan-42.png'); + assert.deepEqual(read.data.custom, { supports: 'home' }); + }); + + // Join with presence events. The second fan joins afterwards, so the presence handler has something to report. + const joined = connected(pubnub, { channels: ['game.chat'] }); + /* eslint-disable prettier/prettier */ + // snippet.fanChatSubscribeWithPresence + const chatChannel = pubnub.channel('game.chat'); + const chatSubscription = chatChannel.subscription({ receivePresenceEvents: true }); + + chatSubscription.onMessage = (event) => { + console.log(`${event.publisher}: ${JSON.stringify(event.message)}`); + }; + + chatSubscription.onPresence = (event) => { + if (event.action === 'interval' || event.action === 'join' || event.action === 'leave') { + console.log('fans in this chat:', event.occupancy); + } + }; + + chatSubscription.subscribe(); + // snippet.end + /* eslint-enable prettier/prettier */ + await joined; + assert.ok(pubnub.getSubscribedChannels().includes('game.chat')); + // fan-108 joins: the presence handler of the sample prints the occupancy, which now counts both fans. + log.lines.length = 0; + const join108 = connected(fan108, { channels: ['game.chat'] }); + fan108.subscriptionSet({ channels: ['game.chat'] }).subscribe(); + await join108; + await eventually( + () => + assert.ok( + log.lines.some((l) => /^fans in this chat: ([2-9]|\d\d+)$/.test(l)), + `logged: ${log.lines.join(' | ')}`, + ), + { label: 'presence handler of the sample prints an occupancy of at least 2', timeout: PRESENCE_TIMEOUT }, + ); + + /* eslint-disable prettier/prettier */ + // snippet.fanChatSendMessage + try { + const response = await pubnub.publish({ + channel: 'game.chat', + message: { text: 'What a save!' }, + customMessageType: 'text-message', + storeInHistory: true, + }); + console.log('chat message published at timetoken:', response.timetoken); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Publishing the chat message failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + // snippet.end + /* eslint-enable prettier/prettier */ + assert.deepEqual(errors.lines, [], 'send: error path not taken'); + const sentLine = log.lines.find((l) => l.startsWith('chat message published at timetoken: ')); + assert.ok(sentLine, `logged: ${log.lines.join(' | ')}`); + const messageTimetoken = sentLine.split(': ')[1]; + assertTimetoken(messageTimetoken); + placeholders['replace-with-message-timetoken'] = messageTimetoken; + // The message reaches the sample's own message handler (the sender is subscribed too). + await eventually(() => assert.ok(log.lines.includes('fan-42: {"text":"What a save!"}'), log.lines.join(' | '))); + + // The other fan answers: the message handler prints `: `. + const reply = `reply-${randomUUID()}`; + await fan108.publish({ channel: 'game.chat', message: { text: reply }, storeInHistory: true }); + await eventually(() => assert.ok(log.lines.includes(`fan-108: {"text":"${reply}"}`), log.lines.join(' | '))); + + // Load what was said: Message Persistence lags the publish, so run the sample again until this run's message shows. + const loadRecent = async (): Promise => { + /* eslint-disable prettier/prettier */ + // snippet.fanChatLoadRecentMessages + try { + const response = await pubnub.fetchMessages({ + channels: ['game.chat'], + count: 25, + includeMessageActions: true, + }); + + const entries = response.channels['game.chat'] ?? []; + + entries.forEach((entry) => { + console.log(entry.timetoken, entry.message); + }); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Loading recent messages failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + // snippet.end + /* eslint-enable prettier/prettier */ + }; + await eventually( + async () => { + log.lines.length = 0; + await loadRecent(); + assert.deepEqual(errors.lines, [], 'load: error path not taken'); + assert.ok( + log.lines.includes(`${messageTimetoken} {"text":"What a save!"}`), + `backlog: ${log.lines.join(' | ')}`, + ); + }, + { label: 'backlog contains the message of this run' }, + ); + + // Here Now in count-only mode: at least the two fans of this run are present (others may be, on the shared channel). + const countFans = async (): Promise => { + /* eslint-disable prettier/prettier */ + // snippet.fanChatCountFansOnline + try { + const response = await pubnub.hereNow({ + channels: ['game.chat'], + includeUUIDs: false, + }); + console.log('fans in the chat:', response.totalOccupancy); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Counting the fans online failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + // snippet.end + /* eslint-enable prettier/prettier */ + }; + await eventually( + async () => { + log.lines.length = 0; + await countFans(); + assert.deepEqual(errors.lines, [], 'count: error path not taken'); + const line = log.lines.find((l) => l.startsWith('fans in the chat: ')); + assert.ok(line, `logged: ${log.lines.join(' | ')}`); + assert.ok(Number(line.split(': ')[1]) >= 2, line); + }, + { label: 'Here Now counts both fans', timeout: PRESENCE_TIMEOUT }, + ); + + // Reactions: the handler is registered first, then a reaction is added to the message sent above. + log.lines.length = 0; + const reactionsReady = connected(pubnub, { channels: ['game.chat'] }, { timeout: 5000 }).catch(() => undefined); + /* eslint-disable prettier/prettier */ + // snippet.fanChatReceiveReactions + const reactionsSubscription = pubnub.channel('game.chat').subscription(); + + reactionsSubscription.onMessageAction = (event) => { + console.log(`${event.publisher} reacted with ${event.data.value} to the message at ${event.data.messageTimetoken}`); + }; + + reactionsSubscription.subscribe(); + // snippet.end + /* eslint-enable prettier/prettier */ + await reactionsReady; + assert.ok(reactionsSubscription); + + /* eslint-disable prettier/prettier */ + // snippet.fanChatAddReaction + try { + const response = await pubnub.addMessageAction({ + channel: 'game.chat', + messageTimetoken: 'replace-with-message-timetoken', + action: { + type: 'reaction', + value: '\u{1F44F}', + }, + }); + console.log('reaction added at timetoken:', response.data.actionTimetoken); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Adding the reaction failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + // snippet.end + /* eslint-enable prettier/prettier */ + assert.deepEqual(errors.lines, [], 'reaction: error path not taken'); + const reactionLine = log.lines.find((l) => l.startsWith('reaction added at timetoken: ')); + assert.ok(reactionLine, `logged: ${log.lines.join(' | ')}`); + const actionTimetoken = reactionLine.split(': ')[1]; + assertTimetoken(actionTimetoken); + await eventually( + () => + assert.ok( + log.lines.includes(`fan-42 reacted with \u{1F44F} to the message at ${messageTimetoken}`), + log.lines.join(' | '), + ), + { label: 'reaction event reaches the subscription' }, + ); + // The reaction is stored against the original message. + await eventually(async () => { + const actions = await reader.getMessageActions({ channel: 'game.chat' }); + const mine = actions.data.find((a) => a.actionTimetoken === actionTimetoken); + assert.equal(mine?.messageTimetoken, messageTimetoken); + assert.equal(mine?.type, 'reaction'); + assert.equal(mine?.value, '\u{1F44F}'); + }); + } finally { + log.restore(); + errors.restore(); + } + }); +}); diff --git a/test/integration/live/use-cases/score-alerts.test.ts b/test/integration/live/use-cases/score-alerts.test.ts new file mode 100644 index 000000000..2040bb131 --- /dev/null +++ b/test/integration/live/use-cases/score-alerts.test.ts @@ -0,0 +1,296 @@ +import assert from 'node:assert/strict'; +import { randomBytes, randomUUID } from 'node:crypto'; +import PubNub from 'pubnub'; +import { assertTimetoken } from '../support/assertions'; +import { createClient, observer, track } from '../support/clients'; +import { captureConsole } from '../support/console'; +import { keysOf } from '../support/env'; +import { watch, eventually } from '../support/wait'; + +/** + * Port of docs-snippets/use-cases/score-alerts.ts (page use-cases/sports-media-entertainment/real-time-score-alerts). + * One flow: register a device, build and publish the alert, list the registration, remove it. Needs the Push Notifications + * add-on on the open keyset. Delivery of the notification itself needs a real device and provider credentials: not verified. + */ +describe('use-cases/score-alerts', () => { + const sorted = (values: string[]): string[] => [...values].sort(); + // The device tokens of the page are placeholders. Registrations are shared and persistent, so every run uses its own tokens. + const APNS_TOKEN = randomBytes(32).toString('hex'); + const FCM_TOKEN = `fcm-registration-${randomUUID()}`; + const apnsOptions = { pushGateway: 'apns2', environment: 'development', topic: 'com.example.matchday' } as const; + + const removeDevices = async (): Promise => { + const cleaner = createClient('open', 'score-alerts-cleaner'); + await cleaner.push.deleteDevice({ device: APNS_TOKEN, ...apnsOptions }).catch(() => undefined); + await cleaner.push.deleteDevice({ device: FCM_TOKEN, pushGateway: 'fcm' }).catch(() => undefined); + await cleaner.deleteMessages({ channel: 'game.score-alerts' }).catch(() => undefined); + }; + before(removeDevices); + after(removeDevices); + + it('scoreAlertsEnvironmentConstant, scoreAlertsRegisterDeviceFCM, scoreAlertsRegisterDeviceAPNs, scoreAlertsBuildNotification, scoreAlertsPublishNotification, scoreAlertsListDeviceRegistrationsFCM, scoreAlertsListDeviceRegistrationsAPNs, scoreAlertsRemoveDeviceRegistrationFCM, scoreAlertsRemoveDeviceRegistrationAPNs: a device registers, an alert is built and published, the registration is listed and removed', async function () { + this.timeout(120_000); + const { publishKey, subscribeKey } = keysOf('open'); + const log = captureConsole('log'); + const errors = captureConsole('error'); + try { + const pubnub = track(new PubNub({ publishKey, subscribeKey, userId: 'fan-42' })); + // The page tells readers to replace the device token placeholders. Swap them for this run's tokens in the push calls. + const tokens: Record = { + 'replace-with-the-apns-device-token': APNS_TOKEN, + 'replace-with-the-fcm-registration-token': FCM_TOKEN, + }; + const swap = (value: unknown): unknown => + value && typeof value === 'object' && !Array.isArray(value) + ? Object.fromEntries( + Object.entries(value).map(([key, item]) => [ + key, + typeof item === 'string' ? (tokens[item] ?? item) : item, + ]), + ) + : value; + for (const method of ['addChannels', 'listChannels', 'removeChannels'] as const) { + const original = pubnub.push[method].bind(pubnub.push) as (arg: unknown) => Promise; + (pubnub.push as unknown as Record)[method] = (arg: unknown) => original(swap(arg)); + } + + // An independent client listens on the alert channel. + const seen = await watch(observer(), ['game.score-alerts']); + // Reader that lists registrations independently of the registering client. + const reader = observer(); + const registrations = async (): Promise<{ apns: string[]; fcm: string[] }> => ({ + apns: sorted((await reader.push.listChannels({ device: APNS_TOKEN, ...apnsOptions })).channels), + fcm: sorted((await reader.push.listChannels({ device: FCM_TOKEN, pushGateway: 'fcm' })).channels), + }); + + /* eslint-disable prettier/prettier */ + // snippet.scoreAlertsEnvironmentConstant + // Every APNs call below, device registration, the notification payload's target, + // listing, and removal, reads this same value. An iOS device token only works in the + // APNs environment that issued it: a development (sandbox) token comes from a + // debug or development-signed build and only works with environment: 'development'; + // a production token comes from a TestFlight or App Store build and only works with + // environment: 'production'. Registering with one value and publishing toward the + // other is why a registration can succeed while the notification it's supposed to + // produce never arrives. Change this one constant when you move from a development + // build to a TestFlight or App Store build, rather than editing every call below. + const APNS_ENVIRONMENT = 'development'; + // snippet.end + /* eslint-enable prettier/prettier */ + assert.equal(APNS_ENVIRONMENT, apnsOptions.environment, 'the helpers of this test use the same environment'); + + /* eslint-disable prettier/prettier */ + // snippet.scoreAlertsRegisterDeviceFCM + try { + const response = await pubnub.push.addChannels({ + channels: ['game.score-alerts'], + device: 'replace-with-the-fcm-registration-token', + pushGateway: 'fcm', + }); + console.log('Android device registered for score alerts:', response); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Registering the Android device failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + // snippet.end + /* eslint-enable prettier/prettier */ + assert.deepEqual(errors.lines, [], 'FCM registration: error path not taken'); + assert.ok( + log.lines.some((l) => l.startsWith('Android device registered for score alerts:')), + log.lines.join(' | '), + ); + + /* eslint-disable prettier/prettier */ + // snippet.scoreAlertsRegisterDeviceAPNs + try { + const response = await pubnub.push.addChannels({ + channels: ['game.score-alerts'], + device: 'replace-with-the-apns-device-token', + pushGateway: 'apns2', + environment: APNS_ENVIRONMENT, + topic: 'com.example.matchday', + }); + console.log('iOS device registered for score alerts:', response); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Registering the iOS device failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + // snippet.end + /* eslint-enable prettier/prettier */ + assert.deepEqual(errors.lines, [], 'APNs registration: error path not taken'); + assert.ok( + log.lines.some((l) => l.startsWith('iOS device registered for score alerts:')), + log.lines.join(' | '), + ); + // Another client sees both registrations: one channel per token (the tokens are this run's own). + await eventually( + async () => + assert.deepEqual(await registrations(), { apns: ['game.score-alerts'], fcm: ['game.score-alerts'] }), + { + label: 'registrations visible to another client', + timeout: 30_000, + }, + ); + + /* eslint-disable prettier/prettier */ + // snippet.scoreAlertsBuildNotification + const goal = PubNub.notificationPayload('Leeds score!', 'Southampton 0 - 2 Leeds'); + + goal.sound = 'default'; + goal.apns.configurations = [{ targets: [{ topic: 'com.example.matchday', environment: APNS_ENVIRONMENT }] }]; + + const payload = goal.buildPayload(['apns2', 'fcm']); + + console.log(JSON.stringify(payload, null, 2)); + // snippet.end + /* eslint-enable prettier/prettier */ + // The printed object is the one the next step publishes: APNs (v2 target with topic and environment) and FCM blocks. + const printed = JSON.parse(log.lines[log.lines.length - 1]); + assert.deepEqual(printed, payload); + const built = payload as unknown as { + pn_apns: { + aps: { alert: { title: string; body: string }; sound: string }; + pn_push: Array<{ targets: Array<{ topic: string; environment: string }>; version: string }>; + }; + pn_fcm: { notification: { title: string; body: string } }; + }; + assert.equal(built.pn_apns.aps.alert.title, 'Leeds score!'); + assert.equal(built.pn_apns.aps.alert.body, 'Southampton 0 - 2 Leeds'); + assert.equal(built.pn_apns.aps.sound, 'default'); + assert.deepEqual(built.pn_apns.pn_push[0].targets, [ + { topic: 'com.example.matchday', environment: 'development' }, + ]); + assert.equal(built.pn_apns.pn_push[0].version, 'v2'); + assert.equal(built.pn_fcm.notification.title, 'Leeds score!'); + assert.equal(built.pn_fcm.notification.body, 'Southampton 0 - 2 Leeds'); + + log.lines.length = 0; + /* eslint-disable prettier/prettier */ + // snippet.scoreAlertsPublishNotification + try { + const response = await pubnub.publish({ + channel: 'game.score-alerts', + message: { + ...payload, + score: '0-2', + scorer: 'Leeds', + }, + customMessageType: 'score-alert', + }); + console.log('score alert published at timetoken:', response.timetoken); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error(`Publishing the score alert failed: ${error}${status ? ` Additional information: ${status}` : ''}`); + } + // snippet.end + /* eslint-enable prettier/prettier */ + assert.deepEqual(errors.lines, [], 'publish: error path not taken'); + const timetoken = log.lines.find((l) => l.startsWith('score alert published at timetoken: '))?.split(': ')[1]; + assertTimetoken(timetoken); + // A subscriber with the app open receives it as an ordinary message. + const alert = await seen.next('message', (m) => m.timetoken === timetoken); + assert.equal(alert.channel, 'game.score-alerts'); + assert.equal(alert.customMessageType, 'score-alert'); + assert.equal(alert.publisher, 'fan-42'); + const delivered = alert.message as { score: string; scorer: string }; + assert.equal(delivered.score, '0-2'); + assert.equal(delivered.scorer, 'Leeds'); + + log.lines.length = 0; + /* eslint-disable prettier/prettier */ + // snippet.scoreAlertsListDeviceRegistrationsFCM + try { + const response = await pubnub.push.listChannels({ + device: 'replace-with-the-fcm-registration-token', + pushGateway: 'fcm', + }); + console.log('this device receives alerts on:', response.channels); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error( + `Listing the device registrations failed: ${error}${status ? ` Additional information: ${status}` : ''}`, + ); + } + // snippet.end + /* eslint-enable prettier/prettier */ + assert.deepEqual(errors.lines, [], 'FCM listing: error path not taken'); + assert.deepEqual(log.lines, ['this device receives alerts on: ["game.score-alerts"]']); + + log.lines.length = 0; + /* eslint-disable prettier/prettier */ + // snippet.scoreAlertsListDeviceRegistrationsAPNs + try { + const response = await pubnub.push.listChannels({ + device: 'replace-with-the-apns-device-token', + pushGateway: 'apns2', + environment: APNS_ENVIRONMENT, + topic: 'com.example.matchday', + }); + console.log('this device receives alerts on:', response.channels); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error( + `Listing the device registrations failed: ${error}${status ? ` Additional information: ${status}` : ''}`, + ); + } + // snippet.end + /* eslint-enable prettier/prettier */ + assert.deepEqual(errors.lines, [], 'APNs listing: error path not taken'); + assert.deepEqual(log.lines, ['this device receives alerts on: ["game.score-alerts"]']); + + log.lines.length = 0; + /* eslint-disable prettier/prettier */ + // snippet.scoreAlertsRemoveDeviceRegistrationFCM + try { + const response = await pubnub.push.removeChannels({ + channels: ['game.score-alerts'], + device: 'replace-with-the-fcm-registration-token', + pushGateway: 'fcm', + }); + console.log('device no longer receives score alerts:', response); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error( + `Removing the device registration failed: ${error}${status ? ` Additional information: ${status}` : ''}`, + ); + } + // snippet.end + /* eslint-enable prettier/prettier */ + assert.deepEqual(errors.lines, [], 'FCM removal: error path not taken'); + assert.ok( + log.lines.some((l) => l.startsWith('device no longer receives score alerts:')), + log.lines.join(' | '), + ); + + /* eslint-disable prettier/prettier */ + // snippet.scoreAlertsRemoveDeviceRegistrationAPNs + try { + const response = await pubnub.push.removeChannels({ + channels: ['game.score-alerts'], + device: 'replace-with-the-apns-device-token', + pushGateway: 'apns2', + environment: APNS_ENVIRONMENT, + topic: 'com.example.matchday', + }); + console.log('device no longer receives score alerts:', response); + } catch (error) { + const status = error instanceof Error && 'status' in error ? error.status : undefined; + console.error( + `Removing the device registration failed: ${error}${status ? ` Additional information: ${status}` : ''}`, + ); + } + // snippet.end + /* eslint-enable prettier/prettier */ + assert.deepEqual(errors.lines, [], 'APNs removal: error path not taken'); + assert.equal(log.lines.filter((l) => l.startsWith('device no longer receives score alerts:')).length, 2); + // Another client sees no registration left for either token. + await eventually(async () => assert.deepEqual(await registrations(), { apns: [], fcm: [] }), { + label: 'registrations removed', + timeout: 30_000, + }); + } finally { + log.restore(); + errors.restore(); + } + }); +}); diff --git a/test/setup-why.ts b/test/setup-why.ts index bffde2f1f..714d9cd58 100644 --- a/test/setup-why.ts +++ b/test/setup-why.ts @@ -1,8 +1,25 @@ -/* global after */ +/* global before, beforeEach, afterEach, after */ import whyIsNodeRunning from 'why-is-node-running'; import wtf from 'wtfnode'; import nock from 'nock'; +/** + * Offline guard: nothing in this run may reach the real network. Every request that no nock scope matches is refused + * locally (NetConnectNotAllowedError), which also covers requests of clients a test forgot to destroy. + * + * Several mocked test files call `nock.enableNetConnect()` in their own `after` / `afterEach`, which would reopen the + * network for everything that runs afterwards. Instead of editing every one of them, the block is re-applied by root + * hooks: before the run, before each test, and after each test (the root `afterEach` runs after the `afterEach` of the + * suite, so it undoes that call at once). A test that needs the loopback (test/unit/node-transport.test.ts) opts in + * with `nock.enableNetConnect('127.0.0.1')` inside the test. Real-network tests belong to test/integration/live/, + * which this run does not load. + */ +const blockNetwork = () => nock.disableNetConnect(); + +before(blockNetwork); +beforeEach(blockNetwork); +afterEach(blockNetwork); + after(function () { nock.enableNetConnect(); nock.restore(); From 1b39117a37b459760f033dbb57ef75482e69ab95 Mon Sep 17 00:00:00 2001 From: Mateusz Wiktor Date: Tue, 17 Nov 2026 09:48:51 +0100 Subject: [PATCH 02/13] ci: disable docs-snippets suite and update live integration conditions - Disabled the docs-snippets testing suite, indicating that samples now run in the live tier. - Updated the live integration workflow to allow testing on pull requests from branches within the same repository, while maintaining restrictions on forked pull requests. --- .github/workflows/docs-snippets.yml | 7 +++++++ .github/workflows/live-integration.yml | 7 +++++-- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/.github/workflows/docs-snippets.yml b/.github/workflows/docs-snippets.yml index 49a7be0d7..1268d09ce 100644 --- a/.github/workflows/docs-snippets.yml +++ b/.github/workflows/docs-snippets.yml @@ -60,6 +60,13 @@ jobs: live=false [ "$changed" = "true" ] && echo "live shards skipped: fork PR without secrets" >> "$GITHUB_STEP_SUMMARY" fi + # DISABLED: the samples now run in the live tier (.github/workflows/live-integration.yml). The dry and live jobs + # are switched off here, and the gate still reports success, so a branch protection rule that requires it is + # not left pending. Remove these three lines to switch this suite back on. The workflow goes away with + # docs-snippets/ at the cutover. + changed=false + live=false + echo "docs-snippets suite is disabled: the samples run in the live tier" >> "$GITHUB_STEP_SUMMARY" echo "changed=$changed" >> "$GITHUB_OUTPUT" echo "live=$live" >> "$GITHUB_OUTPUT" echo "relevant changes: $changed, live shards: $live" diff --git a/.github/workflows/live-integration.yml b/.github/workflows/live-integration.yml index 279bdeb56..4a6fdaa0b 100644 --- a/.github/workflows/live-integration.yml +++ b/.github/workflows/live-integration.yml @@ -61,8 +61,11 @@ jobs: live: name: Live tier needs: [offline] - # Never on a pull request: no secrets for fork code. Scheduled and manual runs only. - if: ${{ github.event_name != 'pull_request' }} + # TEMPORARY: also runs on pull requests from a branch of this repository, so the tier can be tested before it is merged + # (a workflow can only be run manually or on a schedule once its file is on the default branch). Never on a fork pull + # request: no secrets for fork code. Before or right after the merge, change this back to + # `github.event_name != 'pull_request'` so the tier runs nightly and on demand only. + if: ${{ github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository }} runs-on: group: organization/Default timeout-minutes: 40 From f705b9cc45cde64d314aa3df93273893e3530e74 Mon Sep 17 00:00:00 2001 From: Mateusz Wiktor Date: Tue, 17 Nov 2026 10:50:59 +0100 Subject: [PATCH 03/13] refactor: replace `eventually` with `settled` in tests for consistent state verification - Updated multiple test files to use `settled` instead of `eventually` for assertions that require consecutive reads to confirm state consistency. - Enhanced comments to clarify the need for consecutive reads in presence, data synchronization, and message action tests. - Adjusted README and coverage files to reflect the changes in testing methodology. --- test/integration/live/README.md | 3 +- .../live/api-reference/channel-groups.test.ts | 10 +-- .../configuration-legacy.test.ts | 16 +++- .../live/api-reference/datasync.test.ts | 82 ++++++++++++++++--- .../live/api-reference/file-sharing.test.ts | 14 +++- .../api-reference/message-actions.test.ts | 16 +++- .../message-persistence-legacy.test.ts | 16 ++-- .../api-reference/message-persistence.test.ts | 49 ++++++++--- .../live/api-reference/mobile-push.test.ts | 10 +-- .../live/api-reference/objects.test.ts | 56 +++++++++---- .../live/api-reference/presence.test.ts | 54 ++++++------ .../live/api-reference/publish.test.ts | 30 +++---- test/integration/live/coverage.json | 1 + .../live/migration-guides/v11.test.ts | 6 +- .../live/support/datasync-state.ts | 24 ++++++ test/integration/live/support/history.ts | 12 +-- test/integration/live/support/janitor.ts | 15 +++- .../live/support/subscribe-groups.ts | 10 ++- test/integration/live/support/wait.ts | 58 +++++++++++++ .../live/use-cases/fan-re-engagement.test.ts | 20 ++++- .../use-cases/game-chat-moderation.test.ts | 7 +- .../live/use-cases/live-commentary.test.ts | 13 ++- .../live-event-rate-limiting.test.ts | 11 ++- .../live/use-cases/live-polls.test.ts | 30 ++++--- .../live/use-cases/match-stats.test.ts | 69 ++++++++++------ .../live/use-cases/real-time-chat.test.ts | 46 ++++++----- .../live/use-cases/score-alerts.test.ts | 5 +- 27 files changed, 490 insertions(+), 193 deletions(-) diff --git a/test/integration/live/README.md b/test/integration/live/README.md index 176621d2c..f41598c66 100644 --- a/test/integration/live/README.md +++ b/test/integration/live/README.md @@ -45,6 +45,7 @@ Rebuild with `npm run build:node` after a change under `src/`, otherwise the tie - **Transient service errors: retry setup, never the sample.** Setup and cleanup code outside the markers (seeding, resets) may wrap a call in `retryTransient` (`support/retry.ts`). It repeats only HTTP 5xx, 429 and network, timeout or unclassified transport failures, never a 4xx, a validation error or an assertion error, and it logs one `retryTransient:` line per retry. Code inside the markers, the assertions about a sample's effect, and calls whose failure a test asserts are never retried or wrapped. Mocha `retries` stays 0 and timeouts are not lengthened to hide flakiness. - **Retried publishes can be stored twice.** A failed attempt may have reached the service. A seeder whose test counts messages puts `run: seedRun.id` in the payload, publishes through `seedPublish` and calls `settleSeeded` (`support/history.ts`), which deletes stored copies the caller never received a timetoken for. - **Waiting.** Poll with `eventually`, listen with `watch` / `expectEvent`. Use a fixed `sleep` only where the test proves that something did NOT arrive. `PRESENCE_EVENT_TIMEOUT` (30 s) is for a presence event the test caused; `PRESENCE_TIMEOUT` (45 s) is for polling `hereNow` and similar reads. +- **`settled` instead of `eventually`.** `eventually` returns on the first read that passes, which proves only that one replica caught up. Use `settled` (default: 3 passes in a row, 750 ms apart, so about 1.5 s once the state is visible) as the precondition of a sample that reads the same eventually consistent state ONCE and is asserted exactly: a filter or sort index, a list, a count, history, presence, message actions, file listing, push registrations, App Context members. Poll the same read the sample will make, from a helper client, and put it outside the markers. Keep `eventually` where the wait is itself the final assertion of the test, or where it waits for an event or local state. A fixed `sleep` or a longer timeout is never a substitute: they do not show that consecutive reads agree. Do not run a sample inside the wait to retry it. - Clients come from `support/clients.ts` and are destroyed after every test by the root hook. Do not call `destroy()` for them yourself. A client built by hand (the sample's own `new PubNub(...)`) goes through `track()`. ## Helpers (`support/`) @@ -53,7 +54,7 @@ Rebuild with `npm run build:node` after a change under `src/`, otherwise the tie | :--- | :--- | | `env.ts` | `env()`, `keysOf(role)`, `openKeys()`, `securedKeys(role?)`, `subscribeKeyOnly()`, `describeRoles()`, `loadEnv()`, `redact()` | | `clients.ts` | `createClient(role, userId, extra?)`, `shopper()`, `friend()`, `observer()`, `server(role?)`, `unreachable`, `track()`, `destroyClients()` | -| `wait.ts` | `eventually`, `watch` (returns a `Watch` with `events`, `next`, `stop`), `connected`, `expectEvent`, `expectStatus`, `sleep`, `PRESENCE_TIMEOUT`, `PRESENCE_EVENT_TIMEOUT` | +| `wait.ts` | `eventually`, `settled`, `watch` (returns a `Watch` with `events`, `next`, `stop`), `connected`, `expectEvent`, `expectStatus`, `sleep`, `PRESENCE_TIMEOUT`, `PRESENCE_EVENT_TIMEOUT` | | `assertions.ts` | `assertTimetoken` (a string, as `publish()` returns it), `assertServerTime` (a number, as `time()` returns it), `addToTimetoken`, `rejectsValidation`, `rejectsWithCategory` | | `placeholders.ts` | `withPlaceholders(client, values)` | | `util.ts` | `unique(base)`, `sorted(values)` | diff --git a/test/integration/live/api-reference/channel-groups.test.ts b/test/integration/live/api-reference/channel-groups.test.ts index 832acc83e..1eb5243de 100644 --- a/test/integration/live/api-reference/channel-groups.test.ts +++ b/test/integration/live/api-reference/channel-groups.test.ts @@ -2,7 +2,7 @@ import assert from 'node:assert/strict'; import type PubNub from 'pubnub'; import { observer, shopper } from '../support/clients'; import { retryTransient } from '../support/retry'; -import { eventually } from '../support/wait'; +import { eventually, settled } from '../support/wait'; import { sorted } from '../support/util'; /** @@ -16,23 +16,23 @@ describe('api-reference/channel-groups', () => { const listed = async (pubnub: PubNub, channelGroup: string): Promise => (await pubnub.channelGroups.listChannels({ channelGroup })).channels; - /** Deletes the group and waits until another client sees it empty. */ + /** Deletes the group and waits until another client sees it empty on consecutive reads. */ async function resetGroup(channelGroup: string): Promise { const reader = observer(); await reader.channelGroups.deleteGroup({ channelGroup }); - await eventually(async () => assert.deepEqual(await listed(reader, channelGroup), []), { + await settled(async () => assert.deepEqual(await listed(reader, channelGroup), []), { label: `group ${channelGroup} reset`, }); } - /** Resets a group, fills it with `channels` and waits until another client lists exactly those. */ + /** Resets a group, fills it with `channels` and waits until another client lists exactly those on consecutive reads. */ async function seedGroup(channelGroup: string, channels: string[]): Promise { await resetGroup(channelGroup); const reader = observer(); await retryTransient(() => reader.channelGroups.addChannels({ channelGroup, channels }), { label: 'seed channel group', }); - await eventually(async () => assert.deepEqual(sorted(await listed(reader, channelGroup)), sorted(channels)), { + await settled(async () => assert.deepEqual(sorted(await listed(reader, channelGroup)), sorted(channels)), { label: `group ${channelGroup} seeded`, }); } diff --git a/test/integration/live/api-reference/configuration-legacy.test.ts b/test/integration/live/api-reference/configuration-legacy.test.ts index 56b5eaa73..212dbe0d3 100644 --- a/test/integration/live/api-reference/configuration-legacy.test.ts +++ b/test/integration/live/api-reference/configuration-legacy.test.ts @@ -4,7 +4,7 @@ import { createClient, track } from '../support/clients'; import { quietConsole } from '../support/console'; import { keysOf } from '../support/env'; import { unique } from '../support/util'; -import { connected, eventually } from '../support/wait'; +import { connected, eventually, settled } from '../support/wait'; /** * Examples for the deprecated configuration parameters `uuid`, `logVerbosity`, `cipherKey` and `useRandomIVs`, and for the @@ -133,9 +133,17 @@ describe('api-reference/configuration-legacy', () => { const viaModule = createClient('open', 'legacy-cipher-module', { cryptoModule: PubNub.CryptoModule.legacyCryptoModule({ cipherKey }), }); - const read = await viaModule.fetchMessages({ channels: [channel], count: 25 }); - const item = read.channels[channel]?.find((m) => String(m.timetoken) === published.timetoken); - assert.deepEqual(item?.message, { secret: 'legacy' }); + // `rawMessage` saw the message once. This read goes to the service again, so it must hold on consecutive polls. + const item = await settled( + async () => { + const read = await viaModule.fetchMessages({ channels: [channel], count: 25 }); + const found = read.channels[channel]?.find((m) => String(m.timetoken) === published.timetoken); + assert.ok(found, 'the message is stored'); + return found; + }, + { timeout: 30_000, label: 'legacy message read through the module' }, + ); + assert.deepEqual(item.message, { secret: 'legacy' }); // The AES-CBC module opt-in produces the PNED format. const aes = createClient('open', 'legacy-aes', { diff --git a/test/integration/live/api-reference/datasync.test.ts b/test/integration/live/api-reference/datasync.test.ts index d285e686f..0bcd4283c 100644 --- a/test/integration/live/api-reference/datasync.test.ts +++ b/test/integration/live/api-reference/datasync.test.ts @@ -3,8 +3,8 @@ import type PubNub from 'pubnub'; import { server } from '../support/clients'; import { captureConsole } from '../support/console'; import { filterFragments, patchFragments, sortFragments } from '../support/datasync-fragments'; -import { errorText, removePageObjects } from '../support/datasync-state'; -import { connected, eventually } from '../support/wait'; +import { errorText, removePageObjects, untilPageObjectsGone } from '../support/datasync-state'; +import { connected, eventually, settled } from '../support/wait'; /** * DataSync examples, including the DataSync listener and subscription examples. Port of docs-snippets/api-reference/datasync.ts. @@ -69,8 +69,11 @@ describe('api-reference/datasync', () => { } }; before(async function () { - this.timeout(60_000); + this.timeout(90_000); await reset(); + // The exact-list samples below assume nothing of an earlier run is left. Removing a user, channel or entity cascades to its + // memberships and links, so wait until consecutive reads from the service show them gone. + await untilPageObjectsGone(server('datasync')); }); after(async function () { this.timeout(60_000); @@ -146,8 +149,9 @@ describe('api-reference/datasync', () => { it('getUsersFilterExample: getUsers with filter finds Alice once the eventually consistent index has caught up', async () => { const pubnub = server('datasync'); - // `filter` is eventually consistent, so wait until the service indexed the object before running the sample. - await eventually( + // `filter` is eventually consistent, so wait until the service indexed the object before running the sample. One read that + // sees it does not prove the next one will (replicas lag independently), so the read must hold several times in a row. + await settled( async () => { assert.ok(ids(await pubnub.dataSync.getUsers({ filter: 'name LIKE "*Alice*"' })).includes(ALICE)); }, @@ -304,8 +308,8 @@ describe('api-reference/datasync', () => { it('getChannelsFilterExample: getChannels with filter finds the new channel once the index has caught up', async () => { const pubnub = server('datasync'); - // `filter` is eventually consistent, so wait until the service indexed the object before running the sample. - await eventually( + // `filter` is eventually consistent: wait until it holds on consecutive reads before running the sample (see `settled`). + await settled( async () => { assert.ok(ids(await pubnub.dataSync.getChannels({ filter: 'name LIKE "*Sale*"' })).includes(SUMMER)); }, @@ -441,6 +445,15 @@ describe('api-reference/datasync', () => { it('getMembershipsBasicUsage: getMemberships lists exactly the membership of Alice', async () => { const pubnub = server('datasync'); + // The sample reads the list once and the assertion is exact. Nothing documents a plain list as strongly consistent, so + // wait until consecutive reads show the membership created a moment ago. + await settled( + async () => { + const listed = await pubnub.dataSync.getMemberships({ userId: ALICE, limit: 20 }); + assert.deepEqual(ids(listed), [MEMBERSHIP]); + }, + { timeout: 45_000, label: 'Alice lists exactly her membership' }, + ); // snippet.getMembershipsBasicUsage const response = await pubnub.dataSync.getMemberships({ userId: 'user-alice', @@ -456,6 +469,17 @@ describe('api-reference/datasync', () => { it('getMembershipsByChannelIdExample: getMemberships by channel id lists Alice as the only member', async () => { const pubnub = server('datasync'); + // Exact list again: wait until consecutive reads show the one member (see getMembershipsBasicUsage). + await settled( + async () => { + const listed = await pubnub.dataSync.getMemberships({ channelId: SUMMER, limit: 20 }); + assert.deepEqual( + listed.data.map((m) => m.userId), + [ALICE], + ); + }, + { timeout: 45_000, label: 'the channel lists exactly Alice' }, + ); // snippet.getMembershipsByChannelIdExample // Pass `channelId` instead of `userId` to list the members of a channel rather than a user's // memberships. @@ -481,8 +505,9 @@ describe('api-reference/datasync', () => { const pubnub = server('datasync'); await seedArchivedMembership(pubnub); try { - // `filter` is eventually consistent, so wait until the service indexed the new membership. - await eventually( + // `filter` is eventually consistent: wait until it holds on consecutive reads before running the sample. One read that saw the + // membership was not enough in CI: the sample's own read, milliseconds later, returned [] from a replica that lagged. + await settled( async () => { const found = await pubnub.dataSync.getMemberships({ userId: ALICE, filter: 'status LIKE "arch*"' }); assert.ok(ids(found).includes(WINTER_MEMBERSHIP)); @@ -639,6 +664,18 @@ describe('api-reference/datasync', () => { it('getEntitiesBasicUsage: getEntities lists products sorted by price descending, verified on the returned prices', async () => { const pubnub = server('datasync'); + // `sort` on a declared property is served from the same index as `filter` (eventually consistent), and the sample is + // asserted to list the product created a moment ago, so wait until consecutive reads list it. + await settled( + async () => { + assert.ok( + ids(await pubnub.dataSync.getEntities({ class: 'product', sort: { price: 'desc' }, limit: 20 })).includes( + PRODUCT, + ), + ); + }, + { timeout: 45_000, label: 'sorted product list includes the new product' }, + ); // snippet.getEntitiesBasicUsage const response = await pubnub.dataSync.getEntities({ class: 'product', @@ -659,8 +696,8 @@ describe('api-reference/datasync', () => { it('getEntitiesFilterExample: getEntities with filter finds the product once the index has caught up', async () => { const pubnub = server('datasync'); - // `filter` is eventually consistent, so wait until the service indexed the object before running the sample. - await eventually( + // `filter` is eventually consistent: wait until it holds on consecutive reads before running the sample (see `settled`). + await settled( async () => { assert.ok( ids( @@ -1201,6 +1238,14 @@ describe('api-reference/datasync', () => { it('getRelationshipsBasicUsage: getRelationships lists exactly the link from the seller side', async () => { const pubnub = server('datasync'); + // Exact list: wait until consecutive reads show the link created a moment ago (a plain list is not documented as strongly consistent). + await settled( + async () => { + const listed = await pubnub.dataSync.getRelationships({ class: 'ProductOwner', entityAId: BOB, limit: 20 }); + assert.deepEqual(ids(listed), [REL]); + }, + { timeout: 45_000, label: 'Bob lists exactly his link' }, + ); // snippet.getRelationshipsBasicUsage const response = await pubnub.dataSync.getRelationships({ class: 'ProductOwner', @@ -1216,6 +1261,17 @@ describe('api-reference/datasync', () => { it('getRelationshipsByEntityBIdExample: getRelationships by entity B id lists the same link from the product side', async () => { const pubnub = server('datasync'); + await settled( + async () => { + const listed = await pubnub.dataSync.getRelationships({ + class: 'ProductOwner', + entityBId: PRODUCT, + limit: 20, + }); + assert.deepEqual(ids(listed), [REL]); + }, + { timeout: 45_000, label: 'the product lists exactly the link' }, + ); // snippet.getRelationshipsByEntityBIdExample // Pass `entityBId` instead of `entityAId` to list relationships where the entity is on the second // side of the link. @@ -1249,8 +1305,8 @@ describe('api-reference/datasync', () => { it('getRelationshipsFilterExample: a negated filter keeps the gold link once the index has caught up', async () => { const pubnub = server('datasync'); - // `filter` is eventually consistent, so wait until the service indexed the object before running the sample. - await eventually( + // `filter` is eventually consistent: wait until it holds on consecutive reads before running the sample (see `settled`). + await settled( async () => { assert.ok( ids( diff --git a/test/integration/live/api-reference/file-sharing.test.ts b/test/integration/live/api-reference/file-sharing.test.ts index 043a1ffdd..bd97a6bb0 100644 --- a/test/integration/live/api-reference/file-sharing.test.ts +++ b/test/integration/live/api-reference/file-sharing.test.ts @@ -7,7 +7,7 @@ import { captureConsole } from '../support/console'; import { createClient, observer, shopper } from '../support/clients'; import { withPlaceholders } from '../support/placeholders'; import { enterWorkdir, FIXTURE_FILE } from '../support/file-workdir'; -import { eventually, watch } from '../support/wait'; +import { eventually, settled, watch } from '../support/wait'; /** * File sharing examples for Node.js. Port of docs-snippets/api-reference/file-sharing.ts. @@ -86,6 +86,18 @@ describe('api-reference/file-sharing', () => { this.timeout(60_000); const { id } = await upload(shopper()); const pubnub = shopper(); + // The file list is an index that trails the upload. The sample lists once and the assertion needs the file, so wait until + // the same listing, from another client, shows it on consecutive polls. + await settled( + async () => { + const probe = await observer().listFiles({ channel: CHANNEL }); + assert.ok( + probe.data.some((f) => f.id === id), + 'the uploaded file is listed', + ); + }, + { label: 'uploaded file listed' }, + ); // snippet.listFilesBasicUsage const response = await pubnub.listFiles({ channel: 'my_channel' }); console.log('Files listed successfully:', response); diff --git a/test/integration/live/api-reference/message-actions.test.ts b/test/integration/live/api-reference/message-actions.test.ts index 854f5beb9..8b0f03680 100644 --- a/test/integration/live/api-reference/message-actions.test.ts +++ b/test/integration/live/api-reference/message-actions.test.ts @@ -3,7 +3,7 @@ import { addToTimetoken } from '../support/assertions'; import { friend, observer, shopper } from '../support/clients'; import { retryTransient } from '../support/retry'; import { withPlaceholders } from '../support/placeholders'; -import { eventually, watch } from '../support/wait'; +import { eventually, settled, watch } from '../support/wait'; /** * Message action examples. Port of docs-snippets/api-reference/message-actions.ts. Message Persistence must be on for the open @@ -111,6 +111,20 @@ describe('api-reference/message-actions', () => { replace_with_start_timetoken: addToTimetoken(added.data.actionTimetoken, 1), replace_with_end_timetoken: added.data.actionTimetoken, }); + // The sample lists the window once and the assertion is exact, so wait until the same read, from another client, returns the + // action on consecutive polls. + await settled( + async () => { + const probe = await observer().getMessageActions({ + channel, + start: addToTimetoken(added.data.actionTimetoken, 1), + end: added.data.actionTimetoken, + limit: 100, + }); + assert.equal(probe.data.length, 1, 'the window lists the action'); + }, + { label: 'action listed' }, + ); // snippet.getMessageActionsBasicUsage // to get some data in response, first publish a message and then add a message action diff --git a/test/integration/live/api-reference/message-persistence-legacy.test.ts b/test/integration/live/api-reference/message-persistence-legacy.test.ts index ffded8a9e..45d063ad6 100644 --- a/test/integration/live/api-reference/message-persistence-legacy.test.ts +++ b/test/integration/live/api-reference/message-persistence-legacy.test.ts @@ -6,7 +6,7 @@ import { friend, observer, shopper } from '../support/clients'; import { withPlaceholders } from '../support/placeholders'; import { newSeedRun, seedPublish, settleSeeded, storedMessage, untilStored } from '../support/history'; import { retryTransient } from '../support/retry'; -import { eventually, sleep } from '../support/wait'; +import { eventually, settled, sleep } from '../support/wait'; /** * Examples for the deprecated `history()` method and the deprecated `data` property of `actions` on `fetchMessages()`. @@ -196,7 +196,7 @@ describe('api-reference/message-persistence-legacy', () => { it('historyNewerThanTimetoken: history with reverse and start returns only messages newer than the anchor, and start is exclusive', async () => { const [, middle, newest] = await seed('my_channel', 3); - await untilStored('my_channel', [newest]); + await untilStored('my_channel', [middle, newest]); const pubnub = withPlaceholders(shopper(), { '13406746780720711': middle }); /* eslint-disable prettier/prettier, max-len -- sample text is copied as readers see it */ @@ -229,7 +229,7 @@ describe('api-reference/message-persistence-legacy', () => { it('historyUntilTimetoken: history with only end returns messages down to the anchor, and end is inclusive', async () => { const [oldest, middle, newest] = await seed('my_channel', 3); - await untilStored('my_channel', [newest]); + await untilStored('my_channel', [middle, newest]); const pubnub = withPlaceholders(shopper(), { '13406746780720711': middle }); /* eslint-disable prettier/prettier, max-len -- sample text is copied as readers see it */ @@ -283,7 +283,7 @@ describe('api-reference/message-persistence-legacy', () => { const now = (await observer().time()).timetoken; const deleteBound = now; await shopper().deleteMessages({ channel: 'history_test', start: '1', end: now }); - await eventually( + await settled( async () => { const probe = await observer().fetchMessages({ channels: ['history_test'], count: 1 }); assert.equal(probe.channels['history_test']?.length ?? 0, 0, 'history_test is empty'); @@ -365,7 +365,8 @@ describe('api-reference/message-persistence-legacy', () => { let lastFound: Array<{ timetoken: string | number }> = []; let all: Array<{ timetoken: string | number }>; try { - all = await eventually( + // `fromMiddle` below reads the channel once more, so the walk must return everything on consecutive polls, not once. + all = await settled( async () => { pages.length = 0; const found = await getAllMessages(); @@ -501,7 +502,7 @@ describe('api-reference/message-persistence-legacy', () => { ), ); } - await eventually(async () => assert.equal(await probeCount(), 100), { + await settled(async () => assert.equal(await probeCount(), 100), { timeout: 45_000, label: 'channel holds 100 messages', }); @@ -563,7 +564,8 @@ describe('api-reference/message-persistence-legacy', () => { actions?: Record>; data?: unknown; }; - const found = await eventually( + // The reads after the wait look up the same message again, so the wait must hold on consecutive polls. + const found = await settled( async () => { const response = await pubnub.fetchMessages({ channels: [channel], includeMessageActions: true }); const item = response.channels[channel]?.find((m) => String(m.timetoken) === token) as WithActions | undefined; diff --git a/test/integration/live/api-reference/message-persistence.test.ts b/test/integration/live/api-reference/message-persistence.test.ts index 85258c930..a28bb78e3 100644 --- a/test/integration/live/api-reference/message-persistence.test.ts +++ b/test/integration/live/api-reference/message-persistence.test.ts @@ -5,7 +5,7 @@ import { friend, observer, shopper } from '../support/clients'; import { retryTransient } from '../support/retry'; import { withPlaceholders } from '../support/placeholders'; import { newSeedRun, seedPublish, settleSeeded, untilStored } from '../support/history'; -import { eventually } from '../support/wait'; +import { eventually, settled } from '../support/wait'; /** * Message persistence examples. Port of docs-snippets/api-reference/message-persistence.ts. Message Persistence and @@ -81,15 +81,19 @@ describe('api-reference/message-persistence', () => { messageTimetoken: timetoken, action: { type: 'reaction', value: 'smiley_face' }, }); - await eventually( + // The same read as the sample, from another client, until it holds on consecutive polls. + await settled( async () => { const probe = await observer().fetchMessages({ channels: ['my_channel'], - includeMessageActions: true, + stringifiedTimeToken: true, includeMeta: true, + includeMessageActions: true, + includeCustomMessageType: true, }); const found = probe.channels['my_channel']?.find((m) => String(m.timetoken) === timetoken); assert.ok((found as WithActions | undefined)?.actions, 'message and its action are stored'); + assert.deepEqual(found?.meta, { source: 'docs-snippets' }); }, { timeout: 30_000, label: 'message with action stored' }, ); @@ -118,6 +122,18 @@ describe('api-reference/message-persistence', () => { const second = await seed('chats.room2', 1); await untilStored('chats.room1', first); await untilStored('chats.room2', second); + // messageCounts is its own read path, so wait for the same count the sample reads, not only for the history. + await settled( + async () => { + const counts = await observer().messageCounts({ + channels: ['chats.room1', 'chats.room2'], + channelTimetokens: [before], + }); + assert.ok(counts.channels['chats.room1'] >= 2, `chats.room1 counts ${counts.channels['chats.room1']}`); + assert.ok(counts.channels['chats.room2'] >= 1, `chats.room2 counts ${counts.channels['chats.room2']}`); + }, + { label: 'message counts caught up' }, + ); const pubnub = withPlaceholders(shopper(), { 'replace-with-timetoken': before }); // snippet.messageCountBasicUsage @@ -145,8 +161,18 @@ describe('api-reference/message-persistence', () => { const froms: string[] = []; for (const [index, channel] of channels.entries()) { froms.push((await observer().time()).timetoken); - await untilStored(channel, await seed(channel, index + 1)); + await seed(channel, index + 1); } + // One wait for the three channels: the same count the sample reads, held on consecutive polls. + await settled( + async () => { + const counts = await observer().messageCounts({ channels, channelTimetokens: froms }); + channels.forEach((channel, index) => { + assert.ok(counts.channels[channel] >= index + 1, `${channel} counts ${counts.channels[channel]}`); + }); + }, + { label: 'message counts caught up on ch1, ch2, ch3' }, + ); const pubnub = withPlaceholders(shopper(), { 'replace-with-channel-timetoken-ch1': froms[0], 'replace-with-channel-timetoken-ch2': froms[1], @@ -174,11 +200,14 @@ describe('api-reference/message-persistence', () => { }); // `start` is exclusive and `end` inclusive, so start = T - 1 and end = T delete exactly the message at T. - const deleteOne = async (): Promise<{ token: string; placeholders: Record }> => { + // A neighbour is published right after the target, and must survive the delete. One wait covers both. + const deleteOne = async (): Promise<{ token: string; neighbour: string; placeholders: Record }> => { const [token] = await seed('ch1', 1); - await untilStored('ch1', [token]); + const [neighbour] = await seed('ch1', 1); + await untilStored('ch1', [token, neighbour]); return { token, + neighbour, placeholders: { 'replace-with-start-timetoken': addToTimetoken(token, -1), 'replace-with-end-timetoken': token, @@ -200,10 +229,8 @@ describe('api-reference/message-persistence', () => { ); it('deleteMessagesBasicUsage: deleteMessages removes exactly the message at the boundary timetoken', async () => { - const { token, placeholders } = await deleteOne(); // A neighbour published right after must survive the delete of `token`. - const [neighbour] = await seed('ch1', 1); - await untilStored('ch1', [neighbour]); + const { token, neighbour, placeholders } = await deleteOne(); const pubnub = withPlaceholders(shopper(), placeholders); // snippet.deleteMessagesBasicUsage @@ -223,9 +250,7 @@ describe('api-reference/message-persistence', () => { }); it('deleteSpecificMessages: the second deleteMessages sample removes exactly the message at its boundary timetoken', async () => { - const { token, placeholders } = await deleteOne(); - const [neighbour] = await seed('ch1', 1); - await untilStored('ch1', [neighbour]); + const { token, neighbour, placeholders } = await deleteOne(); const pubnub = withPlaceholders(shopper(), placeholders); // snippet.deleteSpecificMessages diff --git a/test/integration/live/api-reference/mobile-push.test.ts b/test/integration/live/api-reference/mobile-push.test.ts index 4c15fa8ed..583cb845e 100644 --- a/test/integration/live/api-reference/mobile-push.test.ts +++ b/test/integration/live/api-reference/mobile-push.test.ts @@ -3,7 +3,7 @@ import PubNub from 'pubnub'; import { observer, friend, shopper } from '../support/clients'; import { retryTransient } from '../support/retry'; import { captureConsole } from '../support/console'; -import { eventually, watch } from '../support/wait'; +import { eventually, settled, watch } from '../support/wait'; import { sorted } from '../support/util'; /** @@ -29,7 +29,7 @@ describe('api-reference/mobile-push', () => { return { apns2: sorted(apns2.channels), fcm: sorted(fcm.channels) }; } - /** Removes every registration of the device on both gateways and waits until another client sees none. */ + /** Removes every registration of the device on both gateways and waits until another client sees none on consecutive reads. */ async function resetDevice(): Promise { const client = friend(); try { @@ -46,13 +46,13 @@ describe('api-reference/mobile-push', () => { throw error; } const reader = observer(); - await eventually(async () => assert.deepEqual(await registrations(reader), { apns2: [], fcm: [] }), { + await settled(async () => assert.deepEqual(await registrations(reader), { apns2: [], fcm: [] }), { label: 'device has no registrations', timeout: 30_000, }); } - /** Resets the device, registers `channels` on both gateways and waits until another client lists them. */ + /** Resets the device, registers `channels` on both gateways and waits until another client lists them on consecutive reads. */ async function seedDevice(channels: string[]): Promise { await resetDevice(); const client = friend(); @@ -63,7 +63,7 @@ describe('api-reference/mobile-push', () => { label: 'seed push registration', }); const reader = observer(); - await eventually( + await settled( async () => assert.deepEqual(await registrations(reader), { apns2: sorted(channels), fcm: sorted(channels) }), { label: 'device registrations visible', timeout: 30_000 }, ); diff --git a/test/integration/live/api-reference/objects.test.ts b/test/integration/live/api-reference/objects.test.ts index a4970ae01..537d70899 100644 --- a/test/integration/live/api-reference/objects.test.ts +++ b/test/integration/live/api-reference/objects.test.ts @@ -3,7 +3,7 @@ import type PubNub from 'pubnub'; import { friend, observer, shopper } from '../support/clients'; import { retryTransient } from '../support/retry'; import { captureConsole } from '../support/console'; -import { eventually, watch } from '../support/wait'; +import { eventually, settled, watch } from '../support/wait'; import { sorted } from '../support/util'; /** @@ -18,7 +18,7 @@ describe('api-reference/objects', () => { const SHOPPER = 'shopper'; - /** Removes every membership of the user and waits until another client sees none. */ + /** Removes every membership of the user and waits until another client sees none on consecutive reads. */ async function clearMemberships(uuid: string): Promise { const admin = friend(); const current = await admin.objects.getMemberships({ uuid, limit: 100 }); @@ -26,12 +26,12 @@ describe('api-reference/objects', () => { await admin.objects.removeMemberships({ uuid, channels: current.data.map((m) => m.channel.id) }); } const reader = observer(); - await eventually(async () => assert.deepEqual((await reader.objects.getMemberships({ uuid })).data, []), { + await settled(async () => assert.deepEqual((await reader.objects.getMemberships({ uuid })).data, []), { label: `memberships of ${uuid} cleared`, }); } - /** Removes every member of the channel and waits until another client sees none. */ + /** Removes every member of the channel and waits until another client sees none on consecutive reads. */ async function clearMembers(channel: string): Promise { const admin = friend(); const current = await admin.objects.getChannelMembers({ channel, limit: 100 }); @@ -39,7 +39,7 @@ describe('api-reference/objects', () => { await admin.objects.removeChannelMembers({ channel, uuids: current.data.map((m) => m.uuid.id) }); } const reader = observer(); - await eventually(async () => assert.deepEqual((await reader.objects.getChannelMembers({ channel })).data, []), { + await settled(async () => assert.deepEqual((await reader.objects.getChannelMembers({ channel })).data, []), { label: `members of ${channel} cleared`, }); } @@ -54,13 +54,15 @@ describe('api-reference/objects', () => { }), { label: 'seed app context' }, ); + // The samples read team.blue by id and through the `*Team` name filter (an index), so both reads must hold on consecutive polls. const reader = observer(); - await eventually( - async () => - assert.equal((await reader.objects.getChannelMetadata({ channel: 'team.blue' })).data.name, 'Blue Team'), - { - label: 'team.blue visible', + await settled( + async () => { + assert.equal((await reader.objects.getChannelMetadata({ channel: 'team.blue' })).data.name, 'Blue Team'); + const filtered = await reader.objects.getAllChannelMetadata({ filter: 'name LIKE "*Team"' }); + assert.ok(ids(filtered.data).includes('team.blue'), 'team.blue is in the filtered list'); }, + { label: 'team.blue visible by id and in the name filter' }, ); } @@ -147,7 +149,7 @@ describe('api-reference/objects', () => { { label: 'seed app context' }, ); const reader = observer(); - await eventually( + await settled( async () => { assert.equal((await reader.objects.getUUIDMetadata({ uuid: SHOPPER })).data.name, 'John Doe'); assert.equal((await reader.objects.getUUIDMetadata({ uuid: 'myUuid' })).data.email, 'john.doe@example.com'); @@ -501,11 +503,17 @@ describe('api-reference/objects', () => { () => admin.objects.setMemberships({ uuid: SHOPPER, channels: ['my-channel', 'channel-with-status-type'] }), { label: 'seed app context' }, ); + // The sample lists both users and the starred filter (an index) once each, so wait until the same reads hold on consecutive polls. const reader = observer(); - await eventually( + await settled( async () => { assert.equal((await reader.objects.getMemberships({ uuid: 'myUuid' })).data.length, 2); assert.equal((await reader.objects.getMemberships({ uuid: SHOPPER })).data.length, 2); + const starred = await reader.objects.getMemberships({ uuid: 'myUuid', filter: 'custom.starred == true' }); + assert.deepEqual( + starred.data.map((m) => m.channel.id), + ['my-channel'], + ); }, { label: 'memberships seeded' }, ); @@ -552,7 +560,7 @@ describe('api-reference/objects', () => { }); } const reader = observer(); - await eventually( + await settled( async () => { for (const uuid of [SHOPPER, 'myUuid']) { assert.equal((await reader.objects.getMemberships({ uuid })).data.length, 3); @@ -616,12 +624,26 @@ describe('api-reference/objects', () => { () => friend().objects.setChannelMembers({ channel: 'myChannel', uuids: ['uuid-1', 'uuid-2'] }), { label: 'seed app context' }, ); + // The sample lists the members with their user fields and filters them by user name (an index), so wait until the same reads hold. const reader = observer(); - await eventually( + await settled( async () => { - assert.equal((await reader.objects.getChannelMembers({ channel: 'myChannel' })).data.length, 2); + const members = await reader.objects.getChannelMembers({ channel: 'myChannel', include: { UUIDFields: true } }); + assert.deepEqual(members.data.map((m) => m.uuid.id).sort(), ['uuid-1', 'uuid-2']); + assert.equal( + (members.data.find((m) => m.uuid.id === 'uuid-1')?.uuid as { name?: string } | undefined)?.name, + 'Member One', + ); + const admins = await reader.objects.getChannelMembers({ + channel: 'myChannel', + filter: 'uuid.name LIKE "*admin*"', + }); + assert.deepEqual( + admins.data.map((m) => m.uuid.id), + ['uuid-2'], + ); }, - { label: 'members seeded' }, + { label: 'members seeded and name filter indexed' }, ); // snippet.getChannelMembersBasicUsage @@ -694,7 +716,7 @@ describe('api-reference/objects', () => { { label: 'seed app context' }, ); const reader = observer(); - await eventually( + await settled( async () => { assert.equal((await reader.objects.getChannelMembers({ channel: 'myChannel' })).data.length, 3); }, diff --git a/test/integration/live/api-reference/presence.test.ts b/test/integration/live/api-reference/presence.test.ts index 95f084dc8..a55aadabf 100644 --- a/test/integration/live/api-reference/presence.test.ts +++ b/test/integration/live/api-reference/presence.test.ts @@ -2,7 +2,7 @@ import assert from 'node:assert/strict'; import PubNub from 'pubnub'; import { friend, observer } from '../support/clients'; import { keysOf } from '../support/env'; -import { connected, eventually, PRESENCE_TIMEOUT } from '../support/wait'; +import { connected, eventually, PRESENCE_TIMEOUT, settled } from '../support/wait'; import { sorted } from '../support/util'; /** @@ -42,7 +42,15 @@ describe('api-reference/presence', () => { it('fixture: the occupant is visible on all its channels, through both channel groups, with its state', async function () { this.timeout(120_000); const channels = ['ch1', 'my_channel', 'cg1_member']; - await eventually( + // The samples below read each of these views once and assert the occupant exactly, so one pass over all of them is not + // enough: every view must hold on consecutive passes (`settled`), because presence reads come from replicas that catch up + // independently. + const groupReads: Array<[string, { channels?: string[]; channelGroups: string[] }]> = [ + ['cg1', { channelGroups: ['cg1'] }], + ['my_channel_group', { channelGroups: ['my_channel_group'] }], + ['ch1 and cg1', { channels: ['ch1'], channelGroups: ['cg1'] }], + ]; + await settled( async () => { const result = await here({ channels }); for (const channel of channels) { @@ -51,41 +59,28 @@ describe('api-reference/presence', () => { `${channel} lists the occupant`, ); } - }, - { timeout: PRESENCE_TIMEOUT, label: 'occupant visible in hereNow' }, - ); - const reads: Array<[string, { channels?: string[]; channelGroups: string[] }]> = [ - ['cg1', { channelGroups: ['cg1'] }], - ['my_channel_group', { channelGroups: ['my_channel_group'] }], - ['ch1 and cg1', { channels: ['ch1'], channelGroups: ['cg1'] }], - ]; - for (const [label, query] of reads) { - await eventually( - async () => { - const result = await here(query); + for (const [label, query] of groupReads) { + const viaGroup = await here(query); assert.ok( - result.channels['cg1_member']?.occupants.some((o) => o.uuid === 'uuid'), + viaGroup.channels['cg1_member']?.occupants.some((o) => o.uuid === 'uuid'), `${label} lists the occupant on cg1_member`, ); - }, - { timeout: PRESENCE_TIMEOUT, label: `occupant visible through ${label}` }, - ); - } - await eventually( - async () => { - const result = await here({ channels: ['my_channel'], includeState: true }); + } + const withState = await here({ channels: ['my_channel'], includeState: true }); assert.deepEqual( - result.channels['my_channel']?.occupants.find((o) => o.uuid === 'uuid')?.state, + withState.channels['my_channel']?.occupants.find((o) => o.uuid === 'uuid')?.state, { status: 'online' }, 'hereNow lists the state on the channel', ); + const counted = await here({ channels: ['my_channel'], includeUUIDs: false }); + assert.ok((counted.channels['my_channel']?.occupancy ?? 0) >= 1, 'occupancy counts the occupant'); const where = await observer().whereNow({ uuid: 'uuid' }); assert.ok( ['ch1', 'my_channel'].every((c) => where.channels.includes(c)), 'whereNow lists both channels', ); }, - { timeout: PRESENCE_TIMEOUT, label: 'state and channel list visible' }, + { timeout: PRESENCE_TIMEOUT, label: 'occupant visible through every read the samples use' }, ); }); @@ -135,6 +130,15 @@ describe('api-reference/presence', () => { this.timeout(60_000); const pubnub = occupant; await pubnub.setState({ state: { status: 'online' }, channels: ['ch1'], channelGroups: ['cg1'] }); + // The sample reads the state once. Wait until consecutive reads from another client return it (the previous test left the + // same value, so this matters mostly when this test runs alone). + await settled( + async () => { + const stored = await observer().getState({ uuid: 'uuid', channels: ['ch1'] }); + assert.deepEqual(stored.channels['ch1'], { status: 'online' }); + }, + { label: 'state visible before the sample reads it' }, + ); // snippet.getStateBasicUsage const response = await pubnub.getState({ @@ -152,7 +156,7 @@ describe('api-reference/presence', () => { this.timeout(60_000); const pubnub = occupant; await pubnub.setState({ state: { status: 'online' }, channels: ['ch1'] }); - await eventually( + await settled( async () => { const result = await here({ channels: ['ch1'], includeState: true }); assert.deepEqual(result.channels['ch1']?.occupants.find((o) => o.uuid === 'uuid')?.state, { status: 'online' }); diff --git a/test/integration/live/api-reference/publish.test.ts b/test/integration/live/api-reference/publish.test.ts index ce5a262af..1b5e64fa9 100644 --- a/test/integration/live/api-reference/publish.test.ts +++ b/test/integration/live/api-reference/publish.test.ts @@ -4,7 +4,7 @@ import { assertTimetoken, rejectsValidation, rejectsWithCategory } from '../supp import { createClient, observer, server, shopper, unreachable } from '../support/clients'; import { captureConsole } from '../support/console'; import { storedMessage } from '../support/history'; -import { eventually, sleep, watch } from '../support/wait'; +import { eventually, settled, sleep, watch } from '../support/wait'; /** * Publish and signal examples, including `fire()`. Port of docs-snippets/api-reference/publish.ts. @@ -208,10 +208,11 @@ describe('api-reference/publish', () => { return errors.lines; }; - // Wait until the keyset enforces Access Manager on this channel for a stranger. Without this wait the 403 branch below - // is only as reliable as the propagation of earlier grants and revokes on the keyset. A keyset that never denies fails - // here with a message that names the cause, not with an empty list further down. - await eventually( + // Wait until the keyset enforces Access Manager on this channel for a stranger, on consecutive probes. One refusal only proves + // that one node knows the revoke (the nodes learn of it at different moments, up to about 60 s), and the sample below runs + // once, so a stranger that is let through by a lagging node would fail it. A keyset that never denies fails here with a + // message that names the cause, not with an empty list further down. + await settled( () => rejectsWithCategory( createClient('pam', 'no-token-probe').publish({ channel: 'my_channel', message: { probe: true } }), @@ -219,7 +220,7 @@ describe('api-reference/publish', () => { ), { timeout: 120_000, - interval: 3_000, + interval: 2_000, label: 'a client without a token is refused on the channel (Access Manager grants and revokes take up to 60 s to propagate)', }, @@ -228,19 +229,10 @@ describe('api-reference/publish', () => { // Success: the server client may publish, so no error line. assert.deepEqual(await attempt(server()), [], 'a valid publish logs nothing'); - let deniedAttempts = 0; - await eventually( - async () => { - deniedAttempts += 1; - const denied = await attempt(createClient('pam', 'no-token-user')); - assert.deepEqual(denied, ['403: the token or authKey has no write permission on the channel']); - }, - { - timeout: 90_000, - interval: 3_000, - label: 'a tokenless publish logs the 403 line (Access Manager enforcement reaches every node)', - }, - ); + // Access denied: the sample runs once, as written. + assert.deepEqual(await attempt(createClient('pam', 'no-token-user')), [ + '403: the token or authKey has no write permission on the channel', + ]); // Validation: no publish key, the SDK refuses before it sends. const invalid = await attempt(createClient('pam', 'no-publish-key-user', { publishKey: '' })); diff --git a/test/integration/live/coverage.json b/test/integration/live/coverage.json index 9381ec5d8..9d03b95f5 100644 --- a/test/integration/live/coverage.json +++ b/test/integration/live/coverage.json @@ -711,6 +711,7 @@ "getStateBasicUsage": { "file": "test/integration/live/api-reference/presence.test.ts", "test": "api-reference/presence getStateBasicUsage: getState returns the state set for the channel and the group", + "role": "open", "mode": "live" }, "getUUIDMetadataBasicUsage": { diff --git a/test/integration/live/migration-guides/v11.test.ts b/test/integration/live/migration-guides/v11.test.ts index d0d04f272..55e3e7fdc 100644 --- a/test/integration/live/migration-guides/v11.test.ts +++ b/test/integration/live/migration-guides/v11.test.ts @@ -5,7 +5,7 @@ import { createClient, observer, track } from '../support/clients'; import { unique } from '../support/util'; import { keysOf, openKeys } from '../support/env'; -import { connected, eventually } from '../support/wait'; +import { connected, eventually, settled } from '../support/wait'; /** * Examples for the version 11 migration guide. Port of docs-snippets/migration-guides/v11.ts. @@ -98,7 +98,9 @@ describe('migration-guides/v11', () => { label: 'state set', }); client.disconnect(); - await eventually(async () => assert.deepEqual(await stateOn(client), {}), { + // The assertion after the loop reads the dropped client's state once more and expects it NOT to be back, so a replica that still + // holds the old state must not be able to pass here: the cleared state has to hold on consecutive reads. + await settled(async () => assert.deepEqual(await stateOn(client), {}), { timeout: 30_000, label: 'state cleared by the leave', }); diff --git a/test/integration/live/support/datasync-state.ts b/test/integration/live/support/datasync-state.ts index 2be7ec907..3dcd761c7 100644 --- a/test/integration/live/support/datasync-state.ts +++ b/test/integration/live/support/datasync-state.ts @@ -5,6 +5,7 @@ */ import type PubNub from 'pubnub'; import { DATASYNC_PAGE_OBJECTS as PAGE_OBJECTS } from './resources'; +import { settled } from './wait'; /** The SDK error carries the service answer in `status`. Assertions match on this string, so a changed code shows in the failure. */ export const errorText = (error: unknown): string => { @@ -30,3 +31,26 @@ export async function removePageObjects(client: PubNub): Promise { await Promise.all(PAGE_OBJECTS.users.map((id) => quietly(() => api.removeUser({ id })))); await Promise.all(PAGE_OBJECTS.channels.map((id) => quietly(() => api.removeChannel({ id })))); } + +/** + * Waits until the lists the page's exact-list samples read show nothing of the page's objects: no membership of Alice or of the + * summer channel, no link of Bob or of the sneaker. `removePageObjects` does not observe its own effect, and the memberships and + * links disappear as a cascade of removing their end, so a list read right after could still show a leftover. The reads must be + * empty on consecutive polls (`settled`). + */ +export async function untilPageObjectsGone(client: PubNub): Promise { + const api = client.dataSync; + await settled( + async () => { + const [byUser, byChannel, bySeller, byProduct] = await Promise.all([ + api.getMemberships({ userId: 'user-alice', limit: 20 }), + api.getMemberships({ channelId: 'channel-summer-sale', limit: 20 }), + api.getRelationships({ class: 'ProductOwner', entityAId: 'seller-bob', limit: 20 }), + api.getRelationships({ class: 'ProductOwner', entityBId: 'product-sneaker-42', limit: 20 }), + ]); + const left = [...byUser.data, ...byChannel.data, ...bySeller.data, ...byProduct.data].map((o) => o.id); + if (left.length > 0) throw new Error(`still listed after the reset: ${[...new Set(left)].join(', ')}`); + }, + { timeout: 45_000, label: 'page objects gone from the lists' }, + ); +} diff --git a/test/integration/live/support/history.ts b/test/integration/live/support/history.ts index 88d4b2575..55659af5d 100644 --- a/test/integration/live/support/history.ts +++ b/test/integration/live/support/history.ts @@ -3,7 +3,7 @@ import type PubNub from 'pubnub'; import { addToTimetoken } from './assertions'; import { observer, shopper } from './clients'; import { retryTransient } from './retry'; -import { eventually, sleep } from './wait'; +import { settled, sleep } from './wait'; import { randomUUID } from 'node:crypto'; /** @@ -21,10 +21,12 @@ export async function storedMessage(client: PubNub, channel: string, timetoken: /** * Waits until every timetoken of `tokens` is visible to a reader on `channel` (newest 100 messages only; the write and the read - * model of the service are separate). For larger sets, page through the channel instead. + * model of the service are separate). For larger sets, page through the channel instead. One read that shows the messages does + * not vouch for the next one (replicas lag independently), and the samples read the window once and assert it exactly, so the + * read must hold on consecutive polls (`settled`, about 1.5 s once the messages are visible). */ export function untilStored(channel: string, tokens: string[]): Promise { - return eventually( + return settled( async () => { const history = await observer().fetchMessages({ channels: [channel], count: 100 }); const have = new Set((history.channels[channel] ?? []).map((m) => String(m.timetoken))); @@ -95,7 +97,7 @@ export async function settleSeeded(channel: string, tokens: string[], run: SeedR (await scanChannel(channel, bound)).filter( (m) => (m.message as { run?: string } | null)?.run === run.id && !wanted.has(m.timetoken), ); - await eventually( + await settled( async () => { const have = new Set((await scanChannel(channel, bound)).map((m) => m.timetoken)); for (const token of tokens) assert.ok(have.has(token), `${token} is not stored yet`); @@ -113,7 +115,7 @@ export async function settleSeeded(channel: string, tokens: string[], run: SeedR ); } if (duplicates.length > 0) { - await eventually(async () => assert.equal((await extras()).length, 0, 'duplicates are gone'), { + await settled(async () => assert.equal((await extras()).length, 0, 'duplicates are gone'), { timeout: 45_000, label: `duplicate seeds removed from ${channel}`, }); diff --git a/test/integration/live/support/janitor.ts b/test/integration/live/support/janitor.ts index f2658417d..561b4fac6 100644 --- a/test/integration/live/support/janitor.ts +++ b/test/integration/live/support/janitor.ts @@ -68,10 +68,16 @@ async function revokeApplicationGrant(client: PubNub, ledger: Ledger): Promise { const started = Date.now(); let last = ''; + let closedRounds = 0; for (let attempt = 1; ; attempt += 1) { // Only the publish and subscribe key: a client that holds the secret key would never be refused. const probe = new PubNub({ @@ -106,7 +113,13 @@ export async function verifyApplicationGrantClosed( probe.destroy(); } // Closed only when both calls got a 403. An unclear answer (network, 5xx, 400) is retried, never taken as closed. - if (open.length === 0 && unclear.length === 0) return Date.now() - started; + if (open.length === 0 && unclear.length === 0) { + closedRounds += 1; + if (closedRounds >= CLOSED_ROUNDS) return Date.now() - started; + await pause(CONFIRM_PAUSE); + continue; + } + closedRounds = 0; last = [open.length ? `a client without a token could ${open.join(' and ')}` : '', unclear.join('; ')] .filter(Boolean) .join(', '); diff --git a/test/integration/live/support/subscribe-groups.ts b/test/integration/live/support/subscribe-groups.ts index 3d4006bfe..3aefe1a18 100644 --- a/test/integration/live/support/subscribe-groups.ts +++ b/test/integration/live/support/subscribe-groups.ts @@ -4,18 +4,22 @@ */ import assert from 'node:assert/strict'; import { createClient, friend, observer } from './clients'; -import { eventually } from './wait'; +import { settled } from './wait'; export const memberOf = (group: string): string => `${group}_member`; -/** Gives every group its member channel and waits until the service lists it. */ +/** + * Gives every group its member channel and waits until the service lists it. A sample subscribes to the group right after and the + * test expects a message on the member channel to arrive, so one listing that shows the member is not enough: the listing must hold + * on consecutive polls (`settled`), because the subscribe request is served by whichever node answers it. + */ export async function ensureGroups(groups: string[]): Promise { const writer = friend(); for (const group of groups) { await writer.channelGroups.addChannels({ channelGroup: group, channels: [memberOf(group)] }); } const reader = observer(); - await eventually( + await settled( async () => { for (const group of groups) { const listed = await reader.channelGroups.listChannels({ channelGroup: group }); diff --git a/test/integration/live/support/wait.ts b/test/integration/live/support/wait.ts index e3c358414..79fc80087 100644 --- a/test/integration/live/support/wait.ts +++ b/test/integration/live/support/wait.ts @@ -3,6 +3,8 @@ * fixed `sleep` only to prove that something did NOT arrive. * * - `eventually` poll a read (hereNow, history, App Context filter) that is eventually consistent on the service. + * - `settled` like `eventually`, but the read must hold on several consecutive polls. Use it as the precondition of a + * sample that reads the same state once and asserts it exactly (see README, Waiting). * - `expectEvent` register a listener BEFORE the action, resolve on the first matching event. * - `connected` resolve when a subscribe is really active. A publish sent before that point is not delivered. * - `watch` subscribe once and record everything, so the assertion can come after the publish without racing. @@ -57,6 +59,62 @@ export async function eventually(fn: () => Promise | T, options: Eventuall throw error; } +export interface SettledOptions extends EventuallyOptions { + /** How many polls in a row must pass. Default 3. */ + consecutive?: number; +} + +/** + * Polls a read until it passes `consecutive` times in a row, then returns the value of the last pass. A failed poll resets + * the count. Use it where a later request reads the same state ONCE and an exact assertion depends on it: the service answers + * reads from replicas that do not all see a write at the same moment, so one passing read only proves that one replica caught + * up (a read right after it can still land on a lagging one, as `getMembershipsFilterExample` did in CI). + * + * Defaults: 3 passes, 750 ms apart. The spacing matters as much as the count. Reads a few milliseconds apart can be served from the + * same replica or cache, so they confirm little. 750 ms gives another replica a real chance to answer, and three passes cover two independent confirmations of the first, at a fixed cost of + * (consecutive - 1) * interval = 1.5 s once the state is visible. `interval` is also the pause between failed polls. The + * default `timeout` is 15 s like `eventually`; pass the same longer timeout the old `eventually` call had. + * A fresh client does not help: the tier's helper clients share one connection pool (support/clients.ts), so only time moves + * a read to another replica. + */ +export async function settled(fn: () => Promise | T, options: SettledOptions = {}): Promise { + const timeout = options.timeout ?? 15_000; + const interval = options.interval ?? 750; + const consecutive = options.consecutive ?? 3; + if (!Number.isInteger(consecutive) || consecutive < 1) { + throw new Error('settled: consecutive must be an integer >= 1'); + } + const deadline = Date.now() + timeout; + let last: unknown; + let lastValue: T | undefined; + let streak = 0; + let best = 0; + let polls = 0; + for (;;) { + polls += 1; + try { + lastValue = await fn(); + streak += 1; + best = Math.max(best, streak); + if (streak >= consecutive) return lastValue; + } catch (error) { + last = error; + streak = 0; + } + if (Date.now() + interval > deadline) break; + await sleep(interval); + } + let detail: string; + if (last === undefined) detail = 'none (no poll failed, the timeout was shorter than the confirmations)'; + else detail = last instanceof Error ? last.message : String(last); + const error = new Error( + `settled${options.label ? ` (${options.label})` : ''} timed out after ${timeout}ms, ${polls} polls. ` + + `Reached ${streak} consecutive successes at the end, at most ${best}, of ${consecutive} needed. Last failure: ${detail}`, + ); + (error as Error & { cause?: unknown }).cause = last; + throw error; +} + /** * Removes a listener after the current dispatch. Removing it inside its own callback mutates the SDK's listener array while * it iterates, which makes the SDK skip the next listener. diff --git a/test/integration/live/use-cases/fan-re-engagement.test.ts b/test/integration/live/use-cases/fan-re-engagement.test.ts index 58b2fb099..11cdaf796 100644 --- a/test/integration/live/use-cases/fan-re-engagement.test.ts +++ b/test/integration/live/use-cases/fan-re-engagement.test.ts @@ -5,7 +5,7 @@ import { createClient, observer, track } from '../support/clients'; import { captureConsole } from '../support/console'; import { openKeys } from '../support/env'; import { DEVICE_TOKEN_EXAMPLE } from '../support/resources'; -import { connected, eventually, PRESENCE_TIMEOUT, watch } from '../support/wait'; +import { connected, eventually, PRESENCE_TIMEOUT, settled, watch } from '../support/wait'; /** * Fan re-engagement. Port of docs-snippets/use-cases/fan-re-engagement.ts. @@ -24,12 +24,16 @@ describe('use-cases/fan-re-engagement', () => { // Two stand-in fans that watch the stream: fan-b leaves in the middle of the flow, fan-a is the sample's viewer. const fanB = createClient('open', 'fan-b'); const fanBWatching = await watch(fanB, ['game.stream']); + // The samples read the presence of the fan once (hereNow for the count, whereNow for the check of one fan) and the assertions + // after them need the fan listed, so both reads must hold on consecutive polls. const presenceSeen = async (userId: string, present = true) => { - await eventually( + await settled( async () => { const here = await observer().hereNow({ channels: ['game.stream'], includeUUIDs: true }); const listed = here.channels['game.stream']?.occupants.some((o) => o.uuid === userId) ?? false; assert.equal(listed, present, `${userId} ${present ? 'is' : 'is not'} listed on game.stream`); + const where = await observer().whereNow({ uuid: userId }); + assert.equal(where.channels.includes('game.stream'), present, `whereNow ${present ? 'lists' : 'omits'} game.stream for ${userId}`); }, { timeout: PRESENCE_TIMEOUT, label: `${userId} presence on game.stream` }, ); @@ -235,8 +239,16 @@ describe('use-cases/fan-re-engagement', () => { assert.equal(viewerUserId, 'fan-a', 'the viewer defaults to fan-a'); assert.deepEqual(errors.lines, [], 'the error path was not taken'); assert.ok(logs.lines.some((line) => line.startsWith('fan-a registered its device for game.moment-alerts.fan-a:'))); - const registered = await observer().push.listChannels({ device: DEVICE_TOKEN_EXAMPLE, pushGateway: 'fcm' }); - assert.ok(registered.channels.includes('game.moment-alerts.fan-a'), 'the device is registered for the fan alert channel'); + await settled( + async () => { + const registered = await observer().push.listChannels({ device: DEVICE_TOKEN_EXAMPLE, pushGateway: 'fcm' }); + assert.ok( + registered.channels.includes('game.moment-alerts.fan-a'), + 'the device is registered for the fan alert channel', + ); + }, + { label: 'device registration listed' }, + ); await presenceSeen('fan-a'); // The fan's alert-channel subscription is live: an alert from the service reaches it. diff --git a/test/integration/live/use-cases/game-chat-moderation.test.ts b/test/integration/live/use-cases/game-chat-moderation.test.ts index 110a00eb5..fafd70569 100644 --- a/test/integration/live/use-cases/game-chat-moderation.test.ts +++ b/test/integration/live/use-cases/game-chat-moderation.test.ts @@ -5,7 +5,7 @@ import { createClient } from '../support/clients'; import { captureConsole } from '../support/console'; import { keysOf } from '../support/env'; import { withPlaceholders } from '../support/placeholders'; -import { connected, eventually, sleep } from '../support/wait'; +import { connected, eventually, settled, sleep } from '../support/wait'; /** * Hide and remove chat messages. Port of docs-snippets/use-cases/game-chat-moderation.ts. @@ -183,8 +183,9 @@ describe('use-cases/game-chat-moderation', () => { assert.ok(!hiddenTimetokens.has(neighbour.timetoken), 'a non-hide decision hides nothing'); assert.deepEqual(errors.lines, [], 'the error path was not taken'); - // Wait until history holds the three chat messages and both hide decisions, so the history fetch sees them. - await eventually( + // Wait until history holds the three chat messages and both hide decisions, so the history fetch sees them. The sample reads + // this once and the assertions are exact, so the same read must hold on consecutive polls. + await settled( async () => { const history = await server.fetchMessages({ channels: ['game.chat', 'game.chat.moderation'], count: 25 }); const chat = (history.channels['game.chat'] ?? []).map((m) => m.timetoken); diff --git a/test/integration/live/use-cases/live-commentary.test.ts b/test/integration/live/use-cases/live-commentary.test.ts index f2f83714d..bb1448281 100644 --- a/test/integration/live/use-cases/live-commentary.test.ts +++ b/test/integration/live/use-cases/live-commentary.test.ts @@ -4,7 +4,7 @@ import { assertTimetoken } from '../support/assertions'; import { createClient, observer } from '../support/clients'; import { captureConsole } from '../support/console'; import { storedMessage } from '../support/history'; -import { connected, eventually, sleep } from '../support/wait'; +import { connected, eventually, settled, sleep } from '../support/wait'; /** * Live commentary tutorial. Port of docs-snippets/use-cases/live-commentary.ts. @@ -99,11 +99,16 @@ describe('use-cases/live-commentary', () => { 'the remark published before the subscription is not replayed live', ); - await eventually( + // The sample loads the newest 25 once and the assertions need both remarks, so wait until the same read, from another client, + // lists both on consecutive polls. + await settled( async () => { - assert.ok(await storedMessage(observer(), 'game.commentary', live.timetoken), 'live remark is stored'); + const backlog = await observer().fetchMessages({ channels: ['game.commentary'], count: 25 }); + const tokens = (backlog.channels['game.commentary'] ?? []).map((m) => String(m.timetoken)); + assert.ok(tokens.includes(firstTimetoken), 'first remark is in the backlog'); + assert.ok(tokens.includes(live.timetoken), 'live remark is in the backlog'); }, - { label: 'live remark stored' }, + { label: 'both remarks in the newest 25' }, ); const beforeBacklog = logs.lines.length; // snippet.liveCommentaryLoadBacklog diff --git a/test/integration/live/use-cases/live-event-rate-limiting.test.ts b/test/integration/live/use-cases/live-event-rate-limiting.test.ts index fdb27e4f1..1e14c53c6 100644 --- a/test/integration/live/use-cases/live-event-rate-limiting.test.ts +++ b/test/integration/live/use-cases/live-event-rate-limiting.test.ts @@ -2,7 +2,7 @@ import assert from 'node:assert/strict'; import { createClient, observer } from '../support/clients'; import { captureConsole } from '../support/console'; -import { eventually, sleep, watch } from '../support/wait'; +import { settled, sleep, watch } from '../support/wait'; /** * Live event rate limiting. Port of docs-snippets/use-cases/live-event-rate-limiting.ts. @@ -17,10 +17,17 @@ describe('use-cases/live-event-rate-limiting', () => { // Another fan sits in shard 0, so the shard has a known occupant. The hereNow of the open keyset is eventually consistent. const occupant = await watch(observer(), ['game.chat.shard-0']); try { - await eventually( + // The sample reads the three shards once and the assertion needs shard 0 to hold the occupant, so wait until the same + // read, from another client, shows it on consecutive polls. + await settled( async () => { const here = await observer().hereNow({ channels: ['game.chat.shard-0'], includeUUIDs: true }); assert.ok(here.channels['game.chat.shard-0']?.occupants.some((o) => o.uuid === 'observer'), 'observer is listed'); + const counted = await observer().hereNow({ + channels: ['game.chat.shard-0', 'game.chat.shard-1', 'game.chat.shard-2'], + includeUUIDs: false, + }); + assert.ok((counted.channels['game.chat.shard-0']?.occupancy ?? 0) >= 1, 'shard 0 counts the occupant'); }, { timeout: 30_000, label: 'occupant visible in hereNow' }, ); diff --git a/test/integration/live/use-cases/live-polls.test.ts b/test/integration/live/use-cases/live-polls.test.ts index 6e2d5aa0e..b6c234ddd 100644 --- a/test/integration/live/use-cases/live-polls.test.ts +++ b/test/integration/live/use-cases/live-polls.test.ts @@ -5,7 +5,7 @@ import { assertTimetoken } from '../support/assertions'; import { createClient, observer, track } from '../support/clients'; import { captureConsole } from '../support/console'; import { keysOf } from '../support/env'; -import { connected, eventually, sleep, watch } from '../support/wait'; +import { connected, eventually, settled, sleep, watch } from '../support/wait'; /** * Port of docs-snippets/use-cases/live-polls.ts (page use-cases/sports-media-entertainment/live-polls). @@ -117,8 +117,19 @@ describe('use-cases/live-polls', () => { 'the poll published before the subscription is not delivered live', ); - // 3. ... so the fan fetches the stored poll instead. Message Persistence lags the publish, so the sample runs again until - // it finds this run's poll. + // 3. ... so the fan fetches the stored poll instead. Message Persistence lags the publish, so wait until the same read, from + // another client, returns this run's poll on consecutive polls. The sample then runs once. + await settled( + async () => { + const probe = await observer().fetchMessages({ channels: ['game.new-poll'], count: 1 }); + assert.equal( + probe.channels['game.new-poll']?.[0]?.timetoken, + pollTimetoken, + "the newest stored poll is this run's", + ); + }, + { label: 'the stored poll is fetched', timeout: 30_000 }, + ); const fetchOpenPoll = async (): Promise => { const pubnub = fan; /* eslint-disable prettier/prettier */ @@ -141,15 +152,10 @@ describe('use-cases/live-polls', () => { // snippet.end /* eslint-enable prettier/prettier */ }; - await eventually( - async () => { - log.lines.length = 0; - await fetchOpenPoll(); - assert.deepEqual(errors.lines, [], 'the fetch did not take the error path'); - assert.equal(log.lines.length, 1, `logged: ${log.lines.join(' | ')}`); - }, - { label: 'the stored poll is fetched' }, - ); + log.lines.length = 0; + await fetchOpenPoll(); + assert.deepEqual(errors.lines, [], 'the fetch did not take the error path'); + assert.equal(log.lines.length, 1, `logged: ${log.lines.join(' | ')}`); // The service returns the keys in its own order, so compare the parsed object. assert.ok(log.lines[0].startsWith('poll that is already open: ')); assert.deepEqual(JSON.parse(log.lines[0].slice('poll that is already open: '.length)), { diff --git a/test/integration/live/use-cases/match-stats.test.ts b/test/integration/live/use-cases/match-stats.test.ts index b9265b1fd..8044681d9 100644 --- a/test/integration/live/use-cases/match-stats.test.ts +++ b/test/integration/live/use-cases/match-stats.test.ts @@ -6,7 +6,7 @@ import { createClient, observer, track } from '../support/clients'; import { captureConsole } from '../support/console'; import { keysOf } from '../support/env'; import { storedMessage } from '../support/history'; -import { connected, eventually, watch } from '../support/wait'; +import { connected, eventually, settled, watch } from '../support/wait'; /** * Port of docs-snippets/use-cases/match-stats.ts (page use-cases/sports-media-entertainment/match-stats). @@ -27,7 +27,23 @@ describe('use-cases/match-stats', () => { } } }; - before(clearHistory); + before(async function () { + this.timeout(60_000); + await clearHistory(); + // The sample prints nothing for a stat without a stored value, and the test asserts that for shots and cards. A read that + // still showed an old message after the delete would break that, so wait until their history reads empty on consecutive polls. + await settled( + async () => { + const left = await observer().fetchMessages({ channels: ['shots', 'cards'].map(channelOf), count: 1 }); + assert.deepEqual( + ['shots', 'cards'].map((stat) => left.channels[channelOf(stat)]?.length ?? 0), + [0, 0], + 'shots and cards have no stored value', + ); + }, + { timeout: 30_000, label: 'shots and cards history cleared' }, + ); + }); after(clearHistory); it('matchStatsPublishOneStat, matchStatsSubscribeToAllStats, matchStatsFetchCurrentValues: a published stat reaches the wildcard subscription and a late reader gets the current value of every stat', async function () { @@ -89,11 +105,7 @@ describe('use-cases/match-stats', () => { assert.equal(delivered.channel, 'game.match-stats.score'); assert.equal(delivered.customMessageType, 'match-stat'); assert.equal(delivered.publisher, 'fan-42'); - // storeInHistory: the value is in Message Persistence under that very timetoken. - await eventually(async () => { - const stored = await storedMessage(observer(), 'game.match-stats.score', scoreTimetoken); - assert.deepEqual(stored?.message, { value: '3-1' }); - }); + // storeInHistory: the value is in Message Persistence under that very timetoken (waited for together with the possession below). // A stat that changes while nobody listens: the feed publishes it, a scoreboard starting later reads it back. const feed = createClient('open', 'match-service'); @@ -101,10 +113,24 @@ describe('use-cases/match-stats', () => { const possessionTimetoken = ( await feed.publish({ channel: channelOf('possession'), message: possession, storeInHistory: true }) ).timetoken; - await eventually(async () => { - const stored = await storedMessage(observer(), channelOf('possession'), possessionTimetoken); - assert.deepEqual(stored?.message, possession); - }); + // The scoreboard sample below reads the newest value of every stat once and the assertions are exact. So wait until the same + // read, from another client, returns the score and the possession on consecutive polls (storeInHistory: they are in + // Message Persistence under their timetokens). + await settled( + async () => { + const current = await observer().fetchMessages({ + channels: ['score', 'possession', 'shots', 'cards'].map(channelOf), + count: 1, + }); + const score = current.channels[channelOf('score')]?.[0]; + assert.equal(score?.timetoken, scoreTimetoken, 'the score is the newest stored value'); + assert.deepEqual(score?.message, { value: '3-1' }); + const stat = current.channels[channelOf('possession')]?.[0]; + assert.equal(stat?.timetoken, possessionTimetoken, 'the possession is the newest stored value'); + assert.deepEqual(stat?.message, possession); + }, + { label: 'current value of every stat stored', timeout: 30_000 }, + ); // A late scoreboard: its own client, so the fetch does not rely on the live subscription above. const lateScoreboard = track(new PubNub({ publishKey, subscribeKey, userId: 'fan-43' })); @@ -133,19 +159,14 @@ describe('use-cases/match-stats', () => { // snippet.end /* eslint-enable prettier/prettier */ }; - // Message Persistence is eventually consistent, so the sample runs again until the stored values show up. - await eventually( - async () => { - log.lines.length = 0; - await fetchCurrentValues(lateScoreboard); - assert.deepEqual(errors.lines, [], 'the fetch did not take the error path'); - assert.ok(log.lines.includes('score is currently {"value":"3-1"}'), `logged: ${log.lines.join(' | ')}`); - assert.ok( - log.lines.includes(`possession is currently ${JSON.stringify(possession)}`), - `logged: ${log.lines.join(' | ')}`, - ); - }, - { label: 'current value of every stat', timeout: 30_000 }, + // The sample runs once. Message Persistence is eventually consistent, which is what the wait above is for. + log.lines.length = 0; + await fetchCurrentValues(lateScoreboard); + assert.deepEqual(errors.lines, [], 'the fetch did not take the error path'); + assert.ok(log.lines.includes('score is currently {"value":"3-1"}'), `logged: ${log.lines.join(' | ')}`); + assert.ok( + log.lines.includes(`possession is currently ${JSON.stringify(possession)}`), + `logged: ${log.lines.join(' | ')}`, ); // shots and cards were never published in this run (history cleared first): nothing is printed for them. assert.ok(!log.lines.some((line) => line.startsWith('shots is currently'))); diff --git a/test/integration/live/use-cases/real-time-chat.test.ts b/test/integration/live/use-cases/real-time-chat.test.ts index 9d8e8aae1..dc8986175 100644 --- a/test/integration/live/use-cases/real-time-chat.test.ts +++ b/test/integration/live/use-cases/real-time-chat.test.ts @@ -6,7 +6,7 @@ import { createClient, observer, track } from '../support/clients'; import { captureConsole } from '../support/console'; import { keysOf } from '../support/env'; import { withPlaceholders } from '../support/placeholders'; -import { connected, eventually, PRESENCE_TIMEOUT } from '../support/wait'; +import { connected, eventually, PRESENCE_TIMEOUT, settled } from '../support/wait'; /** * Port of docs-snippets/use-cases/real-time-chat.ts (page use-cases/sports-media-entertainment/real-time-chat). @@ -157,7 +157,16 @@ describe('use-cases/real-time-chat', () => { await fan108.publish({ channel: 'game.chat', message: { text: reply }, storeInHistory: true }); await eventually(() => assert.ok(log.lines.includes(`fan-108: {"text":"${reply}"}`), log.lines.join(' | '))); - // Load what was said: Message Persistence lags the publish, so run the sample again until this run's message shows. + // Load what was said: Message Persistence lags the publish, so wait until the same read, from another client, lists this run's + // message on consecutive polls. The sample then runs once. + await settled( + async () => { + const probe = await reader.fetchMessages({ channels: ['game.chat'], count: 25, includeMessageActions: true }); + const tokens = (probe.channels['game.chat'] ?? []).map((m) => String(m.timetoken)); + assert.ok(tokens.includes(messageTimetoken), 'the message of this run is in the newest 25'); + }, + { label: 'backlog contains the message of this run', timeout: 30_000 }, + ); const loadRecent = async (): Promise => { /* eslint-disable prettier/prettier */ // snippet.fanChatLoadRecentMessages @@ -180,18 +189,10 @@ describe('use-cases/real-time-chat', () => { // snippet.end /* eslint-enable prettier/prettier */ }; - await eventually( - async () => { - log.lines.length = 0; - await loadRecent(); - assert.deepEqual(errors.lines, [], 'load: error path not taken'); - assert.ok( - log.lines.includes(`${messageTimetoken} {"text":"What a save!"}`), - `backlog: ${log.lines.join(' | ')}`, - ); - }, - { label: 'backlog contains the message of this run' }, - ); + log.lines.length = 0; + await loadRecent(); + assert.deepEqual(errors.lines, [], 'load: error path not taken'); + assert.ok(log.lines.includes(`${messageTimetoken} {"text":"What a save!"}`), `backlog: ${log.lines.join(' | ')}`); // Here Now in count-only mode: at least the two fans of this run are present (others may be, on the shared channel). const countFans = async (): Promise => { @@ -210,17 +211,20 @@ describe('use-cases/real-time-chat', () => { // snippet.end /* eslint-enable prettier/prettier */ }; - await eventually( + // Presence lags the join: wait until the same count, read from another client, holds on consecutive polls. The sample runs once. + await settled( async () => { - log.lines.length = 0; - await countFans(); - assert.deepEqual(errors.lines, [], 'count: error path not taken'); - const line = log.lines.find((l) => l.startsWith('fans in the chat: ')); - assert.ok(line, `logged: ${log.lines.join(' | ')}`); - assert.ok(Number(line.split(': ')[1]) >= 2, line); + const here = await reader.hereNow({ channels: ['game.chat'], includeUUIDs: false }); + assert.ok((here.totalOccupancy ?? 0) >= 2, `occupancy ${here.totalOccupancy}`); }, { label: 'Here Now counts both fans', timeout: PRESENCE_TIMEOUT }, ); + log.lines.length = 0; + await countFans(); + assert.deepEqual(errors.lines, [], 'count: error path not taken'); + const countLine = log.lines.find((l) => l.startsWith('fans in the chat: ')); + assert.ok(countLine, `logged: ${log.lines.join(' | ')}`); + assert.ok(Number(countLine.split(': ')[1]) >= 2, countLine); // Reactions: the handler is registered first, then a reaction is added to the message sent above. log.lines.length = 0; diff --git a/test/integration/live/use-cases/score-alerts.test.ts b/test/integration/live/use-cases/score-alerts.test.ts index 2040bb131..7f85aacba 100644 --- a/test/integration/live/use-cases/score-alerts.test.ts +++ b/test/integration/live/use-cases/score-alerts.test.ts @@ -5,7 +5,7 @@ import { assertTimetoken } from '../support/assertions'; import { createClient, observer, track } from '../support/clients'; import { captureConsole } from '../support/console'; import { keysOf } from '../support/env'; -import { watch, eventually } from '../support/wait'; +import { watch, eventually, settled } from '../support/wait'; /** * Port of docs-snippets/use-cases/score-alerts.ts (page use-cases/sports-media-entertainment/real-time-score-alerts). @@ -123,7 +123,8 @@ describe('use-cases/score-alerts', () => { log.lines.join(' | '), ); // Another client sees both registrations: one channel per token (the tokens are this run's own). - await eventually( + // The listing samples below read each registration once and assert it exactly, so the same reads must hold on consecutive polls. + await settled( async () => assert.deepEqual(await registrations(), { apns: ['game.score-alerts'], fcm: ['game.score-alerts'] }), { From 5600ce128359c8db259ce92a26f1c43ef4135e27 Mon Sep 17 00:00:00 2001 From: Mateusz Wiktor Date: Tue, 17 Nov 2026 11:26:27 +0100 Subject: [PATCH 04/13] feat: enhance test reliability with unique ID creation and error explanation - Introduced `createWithRunUniqueId` for creating DataSync objects with unique IDs, ensuring retries handle transient errors correctly. - Updated multiple test files to utilize `seedOnFreshChannel` for seeding channels and messages, improving test stability. - Added `explainSdkError` functionality to provide detailed error messages for PubNub SDK failures in test outputs. - Enhanced README documentation to reflect new testing methodologies and error handling improvements. --- test/integration/live/README.md | 5 +- .../data_sync/events.channel.test.ts | 10 +- .../endpoints/data_sync/events.entity.test.ts | 27 ++-- .../data_sync/events.membership.test.ts | 56 +++++--- .../data_sync/events.relationship.test.ts | 78 ++++++----- .../endpoints/data_sync/events.user.test.ts | 10 +- .../endpoints/data_sync/projections.test.ts | 76 +++++++---- .../live/endpoints/fetch_messages.test.ts | 36 +++-- .../live/endpoints/history.test.ts | 10 +- .../live/endpoints/message_actions.test.ts | 73 +++++++--- .../live/support/datasync-live-helpers.ts | 125 +++++++++++------- .../integration/live/support/explain-error.ts | 125 ++++++++++++++++++ test/integration/live/support/hooks.ts | 32 ++++- test/integration/live/support/retry.ts | 90 +++++++++++++ 14 files changed, 576 insertions(+), 177 deletions(-) create mode 100644 test/integration/live/support/explain-error.ts diff --git a/test/integration/live/README.md b/test/integration/live/README.md index f41598c66..440beee38 100644 --- a/test/integration/live/README.md +++ b/test/integration/live/README.md @@ -43,6 +43,8 @@ Rebuild with `npm run build:node` after a change under `src/`, otherwise the tie - **Unique run ids.** Anything a test asserts on exactly and does not own (a channel it reads back, a payload marker) carries a per-run suffix: `unique('name')` from `support/util.ts`, or `randomUUID()` in the payload. Such names cannot be listed in `support/resources.ts`, so the janitor never clears them. Their messages expire with the retention of the keyset. - **Console.** A client that logs at Debug or Trace level prints its configuration with the real keys. Wrap the test in `quietConsole()` (`support/console.ts`) and call `restore()` in a `finally`. The root hook also mutes the console while it destroys clients, so the destroy-time dumps stay out of the log. Sample `console.log` output is fine, but a sample that prints a granted token (`console.log('Granted Token:', token)`) runs under `captureConsole('log')`. The root hook restores any capture a failed test left behind (`restoreAllConsoles()`). - **Transient service errors: retry setup, never the sample.** Setup and cleanup code outside the markers (seeding, resets) may wrap a call in `retryTransient` (`support/retry.ts`). It repeats only HTTP 5xx, 429 and network, timeout or unclassified transport failures, never a 4xx, a validation error or an assertion error, and it logs one `retryTransient:` line per retry. Code inside the markers, the assertions about a sample's effect, and calls whose failure a test asserts are never retried or wrapped. Mocha `retries` stays 0 and timeouts are not lengthened to hide flakiness. +- **Creates with a run-unique id, and seeding on a fresh channel (endpoint tests only).** A create is not idempotent, so `retryTransient` alone is wrong for it: a retry after a first attempt that the service stored answers 409. `createWithRunUniqueId(create, readBack, { label })` (`support/retry.ts`) is for ids no one else can hold (`freshId`, `projectionId`): it retries transient failures, treats a 409 with DS-0301 on a RETRY as "the earlier attempt was stored" and returns `readBack()` (a get by id), and throws a 409 on the first attempt. `seedOnFreshChannel(newChannel, seed)` seeds a run-unique channel and, on a transient failure, starts the whole seeding again on a new channel (tests that assert exact counts must never reuse a half-seeded channel). Neither belongs in a documentation sample, which uses fixed literal ids and runs once. +- **Failures explain themselves.** `support/hooks.ts` wraps Mocha's `Runner#fail` once, so every failed test or hook of both npm scripts whose error is (or wraps, through `.cause`) a PubNub SDK error gets two lines appended to its message in the failure list: `category`, `statusCode`, `operation` and the service's error data (`support/explain-error.ts`). Keys, tokens and URL queries are masked. No test opts in, and nothing about pass or fail changes. - **Retried publishes can be stored twice.** A failed attempt may have reached the service. A seeder whose test counts messages puts `run: seedRun.id` in the payload, publishes through `seedPublish` and calls `settleSeeded` (`support/history.ts`), which deletes stored copies the caller never received a timetoken for. - **Waiting.** Poll with `eventually`, listen with `watch` / `expectEvent`. Use a fixed `sleep` only where the test proves that something did NOT arrive. `PRESENCE_EVENT_TIMEOUT` (30 s) is for a presence event the test caused; `PRESENCE_TIMEOUT` (45 s) is for polling `hereNow` and similar reads. - **`settled` instead of `eventually`.** `eventually` returns on the first read that passes, which proves only that one replica caught up. Use `settled` (default: 3 passes in a row, 750 ms apart, so about 1.5 s once the state is visible) as the precondition of a sample that reads the same eventually consistent state ONCE and is asserted exactly: a filter or sort index, a list, a count, history, presence, message actions, file listing, push registrations, App Context members. Poll the same read the sample will make, from a helper client, and put it outside the markers. Keep `eventually` where the wait is itself the final assertion of the test, or where it waits for an event or local state. A fixed `sleep` or a longer timeout is never a substitute: they do not show that consecutive reads agree. Do not run a sample inside the wait to retry it. @@ -59,7 +61,8 @@ Rebuild with `npm run build:node` after a change under `src/`, otherwise the tie | `placeholders.ts` | `withPlaceholders(client, values)` | | `util.ts` | `unique(base)`, `sorted(values)` | | `history.ts` | `storedMessage(client, channel, timetoken)`, `untilStored(channel, tokens)`, `newSeedRun`, `seedPublish`, `settleSeeded` (seeding with retry and duplicate removal) | -| `retry.ts` | `retryTransient(fn, { attempts, label })`, `isTransient(error)`; the janitor's `retry` calls it too | +| `retry.ts` | `retryTransient(fn, { attempts, label })`, `isTransient(error)`; the janitor's `retry` calls it too; also `createWithRunUniqueId`, `isAlreadyExists`, `seedOnFreshChannel` | +| `explain-error.ts` | `explainSdkError(err)`: appends the SDK status summary to a failure's message; installed by `hooks.ts` | | `console.ts` | `captureConsole(method)`, `quietConsole()`, `restoreAllConsoles()` | | `resources.ts` | the janitor's resource list; `DATASYNC_PAGE_OBJECTS` is shared with `datasync-state.ts` | | `hooks.ts` | root hooks: role check before a full run, janitor, preflight, client cleanup after each test | diff --git a/test/integration/live/endpoints/data_sync/events.channel.test.ts b/test/integration/live/endpoints/data_sync/events.channel.test.ts index 79cea0787..dac888e14 100644 --- a/test/integration/live/endpoints/data_sync/events.channel.test.ts +++ b/test/integration/live/endpoints/data_sync/events.channel.test.ts @@ -18,6 +18,7 @@ import { freshId, freshPubNub, } from '../../support/datasync-live-helpers'; +import { createWithRunUniqueId } from '../../support/retry'; describe('endpoints/data_sync/events.channel: DataSync events — Channel', function () { this.timeout(60000); @@ -28,10 +29,13 @@ describe('endpoints/data_sync/events.channel: DataSync events — Channel', func pubnub = freshPubNub(); }); + // The id comes from `freshId` and is unique to this run, so a 409 on a retry means the first attempt was stored (see retry.ts). const createChannel = (id: string, payload = channelPayload()) => - pubnub.dataSync - .createChannel({ id, data: { classVersion: CLASS_VERSION, status: 'active', payload } }) - .then(() => undefined); + createWithRunUniqueId( + () => pubnub.dataSync.createChannel({ id, data: { classVersion: CLASS_VERSION, status: 'active', payload } }), + () => pubnub.dataSync.getChannel({ id }), + { label: 'createChannel' }, + ).then(() => undefined); it('channel create → "create" event (objectType "channel")', async () => { const id = freshId('channel'); diff --git a/test/integration/live/endpoints/data_sync/events.entity.test.ts b/test/integration/live/endpoints/data_sync/events.entity.test.ts index c778a0375..8c9e61318 100644 --- a/test/integration/live/endpoints/data_sync/events.entity.test.ts +++ b/test/integration/live/endpoints/data_sync/events.entity.test.ts @@ -20,6 +20,7 @@ import { freshId, freshPubNub, } from '../../support/datasync-live-helpers'; +import { createWithRunUniqueId } from '../../support/retry'; describe('endpoints/data_sync/events.entity: DataSync events — Entity Customer', function () { this.timeout(60000); @@ -30,18 +31,22 @@ describe('endpoints/data_sync/events.entity: DataSync events — Entity Customer pubnub = freshPubNub(); }); + // The id comes from `freshId` and is unique to this run, so a 409 on a retry means the first attempt was stored (see retry.ts). const createCustomer = (id: string) => - pubnub.dataSync - .createEntity({ - id, - class: ENTITY_CLASS_CUSTOMER, - data: { - classVersion: CLASS_VERSION, - status: 'active', - payload: customerPayload(id), - }, - }) - .then(() => undefined); + createWithRunUniqueId( + () => + pubnub.dataSync.createEntity({ + id, + class: ENTITY_CLASS_CUSTOMER, + data: { + classVersion: CLASS_VERSION, + status: 'active', + payload: customerPayload(id), + }, + }), + () => pubnub.dataSync.getEntity({ id }), + { label: 'createEntity' }, + ).then(() => undefined); it('entity create - create event', async () => { const id = freshId('customer'); diff --git a/test/integration/live/endpoints/data_sync/events.membership.test.ts b/test/integration/live/endpoints/data_sync/events.membership.test.ts index 28022ce6a..fe303babe 100644 --- a/test/integration/live/endpoints/data_sync/events.membership.test.ts +++ b/test/integration/live/endpoints/data_sync/events.membership.test.ts @@ -24,6 +24,7 @@ import { safeRemove, userPayload, } from '../../support/datasync-live-helpers'; +import { createWithRunUniqueId } from '../../support/retry'; describe('endpoints/data_sync/events.membership: DataSync events — Membership', function () { this.timeout(60000); @@ -38,30 +39,45 @@ describe('endpoints/data_sync/events.membership: DataSync events — Membership' async function seedEndpoints(): Promise<{ userId: string; channelId: string }> { const userId = freshId('user'); const channelId = freshId('channel'); - await pubnub.dataSync.createUser({ - id: userId, - data: { classVersion: CLASS_VERSION, status: 'active', payload: userPayload() }, - }); - await pubnub.dataSync.createChannel({ - id: channelId, - data: { classVersion: CLASS_VERSION, status: 'active', payload: channelPayload() }, - }); + // Both ids come from `freshId`, unique to this run: a 409 on a retry means the first attempt was stored (see retry.ts). + await createWithRunUniqueId( + () => + pubnub.dataSync.createUser({ + id: userId, + data: { classVersion: CLASS_VERSION, status: 'active', payload: userPayload() }, + }), + () => pubnub.dataSync.getUser({ id: userId }), + { label: 'createUser' }, + ); + await createWithRunUniqueId( + () => + pubnub.dataSync.createChannel({ + id: channelId, + data: { classVersion: CLASS_VERSION, status: 'active', payload: channelPayload() }, + }), + () => pubnub.dataSync.getChannel({ id: channelId }), + { label: 'createChannel' }, + ); return { userId, channelId }; } + // The id comes from `freshId` and is unique to this run, so a 409 on a retry means the first attempt was stored (see retry.ts). const createMembership = (id: string, userId: string, channelId: string, role = 'member') => - pubnub.dataSync - .createMembership({ - id, - userId, - channelId, - data: { - classVersion: CLASS_VERSION, - status: 'active', - payload: { role, joinedAt: '2026-07-06T10:00:00.000Z' }, - }, - }) - .then(() => undefined); + createWithRunUniqueId( + () => + pubnub.dataSync.createMembership({ + id, + userId, + channelId, + data: { + classVersion: CLASS_VERSION, + status: 'active', + payload: { role, joinedAt: '2026-07-06T10:00:00.000Z' }, + }, + }), + () => pubnub.dataSync.getMembership({ id }), + { label: 'createMembership' }, + ).then(() => undefined); it('membership create → "create" event (delivered on channel + user)', async () => { const { userId, channelId } = await seedEndpoints(); diff --git a/test/integration/live/endpoints/data_sync/events.relationship.test.ts b/test/integration/live/endpoints/data_sync/events.relationship.test.ts index e99f84da5..18af39873 100644 --- a/test/integration/live/endpoints/data_sync/events.relationship.test.ts +++ b/test/integration/live/endpoints/data_sync/events.relationship.test.ts @@ -27,6 +27,7 @@ import { loanQuotePayload, safeRemove, } from '../../support/datasync-live-helpers'; +import { createWithRunUniqueId } from '../../support/retry'; describe('endpoints/data_sync/events.relationship: DataSync events Relationship REQUESTED_BY', function () { this.timeout(60000); @@ -41,41 +42,56 @@ describe('endpoints/data_sync/events.relationship: DataSync events Relationship async function seedEndpoints(): Promise<{ customerId: string; loanQuoteId: string }> { const customerId = freshId('customer'); const loanQuoteId = freshId('loanquote'); - await pubnub.dataSync.createEntity({ - id: customerId, - class: ENTITY_CLASS_CUSTOMER, - data: { - classVersion: CLASS_VERSION, - status: 'active', - payload: customerPayload(customerId), - }, - }); - await pubnub.dataSync.createEntity({ - id: loanQuoteId, - class: ENTITY_CLASS_LOAN_QUOTE, - data: { - classVersion: CLASS_VERSION, - status: 'active', - payload: loanQuotePayload(loanQuoteId), - }, - }); + // Both ids come from `freshId`, unique to this run: a 409 on a retry means the first attempt was stored (see retry.ts). + await createWithRunUniqueId( + () => + pubnub.dataSync.createEntity({ + id: customerId, + class: ENTITY_CLASS_CUSTOMER, + data: { + classVersion: CLASS_VERSION, + status: 'active', + payload: customerPayload(customerId), + }, + }), + () => pubnub.dataSync.getEntity({ id: customerId }), + { label: 'createEntity customer' }, + ); + await createWithRunUniqueId( + () => + pubnub.dataSync.createEntity({ + id: loanQuoteId, + class: ENTITY_CLASS_LOAN_QUOTE, + data: { + classVersion: CLASS_VERSION, + status: 'active', + payload: loanQuotePayload(loanQuoteId), + }, + }), + () => pubnub.dataSync.getEntity({ id: loanQuoteId }), + { label: 'createEntity loanQuote' }, + ); return { customerId, loanQuoteId }; } + // The id comes from `freshId` and is unique to this run, so a 409 on a retry means the first attempt was stored (see retry.ts). const createRel = (relId: string, customerId: string, loanQuoteId: string, linkedAt = '2026-07-06T10:00:00.000Z') => - pubnub.dataSync - .createRelationship({ - id: relId, - class: RELATIONSHIP_CLASS_REQUESTED_BY, - entityAId: customerId, - entityBId: loanQuoteId, - data: { - classVersion: CLASS_VERSION, - status: 'active', - payload: { linkedAt }, - }, - }) - .then(() => undefined); + createWithRunUniqueId( + () => + pubnub.dataSync.createRelationship({ + id: relId, + class: RELATIONSHIP_CLASS_REQUESTED_BY, + entityAId: customerId, + entityBId: loanQuoteId, + data: { + classVersion: CLASS_VERSION, + status: 'active', + payload: { linkedAt }, + }, + }), + () => pubnub.dataSync.getRelationship({ id: relId }), + { label: 'createRelationship' }, + ).then(() => undefined); it('relationship create create event delivered on endpoint channels', async () => { const { customerId, loanQuoteId } = await seedEndpoints(); diff --git a/test/integration/live/endpoints/data_sync/events.user.test.ts b/test/integration/live/endpoints/data_sync/events.user.test.ts index 65f4dcbbf..48832bb21 100644 --- a/test/integration/live/endpoints/data_sync/events.user.test.ts +++ b/test/integration/live/endpoints/data_sync/events.user.test.ts @@ -17,6 +17,7 @@ import { freshPubNub, userPayload, } from '../../support/datasync-live-helpers'; +import { createWithRunUniqueId } from '../../support/retry'; describe('endpoints/data_sync/events.user: DataSync events — User', function () { this.timeout(60000); @@ -27,10 +28,13 @@ describe('endpoints/data_sync/events.user: DataSync events — User', function ( pubnub = freshPubNub(); }); + // The id comes from `freshId` and is unique to this run, so a 409 on a retry means the first attempt was stored (see retry.ts). const createUser = (id: string, payload = userPayload()) => - pubnub.dataSync - .createUser({ id, data: { classVersion: CLASS_VERSION, status: 'active', payload } }) - .then(() => undefined); + createWithRunUniqueId( + () => pubnub.dataSync.createUser({ id, data: { classVersion: CLASS_VERSION, status: 'active', payload } }), + () => pubnub.dataSync.getUser({ id }), + { label: 'createUser' }, + ).then(() => undefined); it('user create → "create" event (objectType "user")', async () => { const id = freshId('user'); diff --git a/test/integration/live/endpoints/data_sync/projections.test.ts b/test/integration/live/endpoints/data_sync/projections.test.ts index 3cee3764c..26c68c054 100644 --- a/test/integration/live/endpoints/data_sync/projections.test.ts +++ b/test/integration/live/endpoints/data_sync/projections.test.ts @@ -20,6 +20,7 @@ import assert from 'node:assert'; import PubNub from 'pubnub'; import { createClient } from '../../support/clients'; +import { createWithRunUniqueId } from '../../support/retry'; import { sleep } from '../../support/wait'; import { @@ -369,42 +370,59 @@ function eventOn(events: PubNub.Subscription.DataSyncObject[], channel: string): // -------------------------------------------------------- /** Every seeded object carries a `private` field, which only the `admin` projection exposes. */ +// The ids come from `projectionId` and are unique to this run, so a 409 on a retry means the first attempt was stored (see +// createWithRunUniqueId in support/retry.ts). The read-back is a plain get with the super-user client. async function seedCustomer(pnPam: PubNub, id: string, overrides: Record = {}): Promise { - await pnPam.dataSync.createEntity({ - id, - class: ENTITY_CLASS_CUSTOMER, - data: { - classVersion: CLASS_VERSION, - status: 'active', - payload: customerPayload(id, { private: CUSTOMER_SECRET, ...overrides }), - }, - }); + await createWithRunUniqueId( + () => + pnPam.dataSync.createEntity({ + id, + class: ENTITY_CLASS_CUSTOMER, + data: { + classVersion: CLASS_VERSION, + status: 'active', + payload: customerPayload(id, { private: CUSTOMER_SECRET, ...overrides }), + }, + }), + () => pnPam.dataSync.getEntity({ id }), + { label: 'createEntity customer' }, + ); } async function seedLoanQuote(pnPam: PubNub, id: string): Promise { - await pnPam.dataSync.createEntity({ - id, - class: ENTITY_CLASS_LOAN_QUOTE, - data: { - classVersion: CLASS_VERSION, - status: 'active', - payload: loanQuotePayload(id, { private: LOAN_QUOTE_SECRET }), - }, - }); + await createWithRunUniqueId( + () => + pnPam.dataSync.createEntity({ + id, + class: ENTITY_CLASS_LOAN_QUOTE, + data: { + classVersion: CLASS_VERSION, + status: 'active', + payload: loanQuotePayload(id, { private: LOAN_QUOTE_SECRET }), + }, + }), + () => pnPam.dataSync.getEntity({ id }), + { label: 'createEntity loanQuote' }, + ); } async function seedRequestedBy(pnPam: PubNub, id: string, customerId: string, loanQuoteId: string): Promise { - await pnPam.dataSync.createRelationship({ - id, - class: RELATIONSHIP_CLASS_REQUESTED_BY, - entityAId: customerId, - entityBId: loanQuoteId, - data: { - classVersion: CLASS_VERSION, - status: 'active', - payload: { linkedAt: LINKED_AT, private: REQUESTED_BY_SECRET }, - }, - }); + await createWithRunUniqueId( + () => + pnPam.dataSync.createRelationship({ + id, + class: RELATIONSHIP_CLASS_REQUESTED_BY, + entityAId: customerId, + entityBId: loanQuoteId, + data: { + classVersion: CLASS_VERSION, + status: 'active', + payload: { linkedAt: LINKED_AT, private: REQUESTED_BY_SECRET }, + }, + }), + () => pnPam.dataSync.getRelationship({ id }), + { label: 'createRelationship' }, + ); } // -------------------------------------------------------- diff --git a/test/integration/live/endpoints/fetch_messages.test.ts b/test/integration/live/endpoints/fetch_messages.test.ts index fc92992af..8c512ea3a 100644 --- a/test/integration/live/endpoints/fetch_messages.test.ts +++ b/test/integration/live/endpoints/fetch_messages.test.ts @@ -12,6 +12,7 @@ import assert from 'node:assert'; import { randomUUID } from 'node:crypto'; import { createClient } from '../support/clients'; +import { seedOnFreshChannel } from '../support/retry'; import { eventually } from '../support/wait'; /** Published test message shape. */ @@ -77,8 +78,12 @@ describe('endpoints/fetch_messages: fetchMessages against the service', function }); it('supports metadata', async () => { - const channel = uniqueChannel(); - const messages = await publishMessagesToChannel(pubnub, 10, channel, ''); + // Seeding (publishing) is retried on a new channel after a transient failure, see seedOnFreshChannel in support/retry.ts. + const { channel, result: messages } = await seedOnFreshChannel( + uniqueChannel, + (name) => publishMessagesToChannel(pubnub, 10, name, ''), + { label: 'fetch_messages metadata seed' }, + ); await eventually( async () => { @@ -93,8 +98,11 @@ describe('endpoints/fetch_messages: fetchMessages against the service', function }); it('supports custom message type', async () => { - const channel = uniqueChannel(); - await publishMessagesToChannel(pubnub, 2, channel, 'test-message-type'); + const { channel } = await seedOnFreshChannel( + uniqueChannel, + (name) => publishMessagesToChannel(pubnub, 2, name, 'test-message-type'), + { label: 'fetch_messages message type seed' }, + ); const response = await pubnub.fetchMessages({ channels: [channel], includeCustomMessageType: true }); response.channels[channel].forEach((message) => { @@ -104,11 +112,23 @@ describe('endpoints/fetch_messages: fetchMessages against the service', function for (const field of ['data', 'actions'] as const) { it(`supports actions (stored as '${field}' field)`, async () => { - const channel = uniqueChannel(); const expectedActionsCount = 4; - const messages = await publishMessagesToChannel(pubnub, 2, channel, ''); - const messageTimetokens = messages.map((message) => message.timetoken); - const actions = await addActionsInChannel(pubnub, expectedActionsCount, messageTimetokens, channel); + // Publishing and adding the actions are both seeding here (the assertion is on what fetchMessages returns). + const { + channel, + result: { messageTimetokens, actions }, + } = await seedOnFreshChannel( + uniqueChannel, + async (name) => { + const messages = await publishMessagesToChannel(pubnub, 2, name, ''); + const timetokens = messages.map((message) => message.timetoken); + return { + messageTimetokens: timetokens, + actions: await addActionsInChannel(pubnub, expectedActionsCount, timetokens, name), + }; + }, + { label: `fetch_messages actions seed (${field})` }, + ); // The mocked original waited a fixed 2 s before reading; poll until the actions are visible instead. await eventually( diff --git a/test/integration/live/endpoints/history.test.ts b/test/integration/live/endpoints/history.test.ts index 05fb66799..88948bc06 100644 --- a/test/integration/live/endpoints/history.test.ts +++ b/test/integration/live/endpoints/history.test.ts @@ -12,6 +12,7 @@ import assert from 'node:assert'; import { randomUUID } from 'node:crypto'; import { createClient } from '../support/clients'; +import { seedOnFreshChannel } from '../support/retry'; import { eventually } from '../support/wait'; /** Published test message shape. */ @@ -49,8 +50,13 @@ describe('endpoints/history: history against the service', function () { }); it('supports metadata', async () => { - const channel = `history-${randomUUID()}`; - const messages = await publishMessagesToChannel(pubnub, 10, channel); + // A transient publish failure discards the channel and seeds a new one (nothing else depends on its name), so the read + // below never sees a channel that a lost-answer publish left with an extra message. + const { channel, result: messages } = await seedOnFreshChannel( + () => `history-${randomUUID()}`, + (name) => publishMessagesToChannel(pubnub, 10, name), + { label: 'history seed' }, + ); await eventually( async () => { diff --git a/test/integration/live/endpoints/message_actions.test.ts b/test/integration/live/endpoints/message_actions.test.ts index 354a63c0f..6beb98882 100644 --- a/test/integration/live/endpoints/message_actions.test.ts +++ b/test/integration/live/endpoints/message_actions.test.ts @@ -13,6 +13,7 @@ import assert from 'node:assert'; import { randomUUID } from 'node:crypto'; import { createClient } from '../support/clients'; +import { retryTransient, seedOnFreshChannel } from '../support/retry'; import { connected, eventually, expectEvent } from '../support/wait'; /** Message action object. */ @@ -67,10 +68,27 @@ describe('endpoints/message_actions: message actions against the service', funct pubnub = createClient('open', 'myUUID'); }); + /** Seeding for `seedOnFreshChannel`: publishes `messages` messages, then adds `actionsPerMessage` actions to each. */ + const seedActions = + (messages: number, actionsPerMessage: number) => + async (channel: string): Promise<{ messageTimetokens: string[]; actionTimetokens: string[] }> => { + const messageTimetokens = await publishMessages(pubnub, messages, channel); + const actionTimetokens = await addActions(pubnub, actionsPerMessage, messageTimetokens, channel); + return { messageTimetokens, actionTimetokens }; + }; + describe('addMessageAction', () => { - const addsAction = async (channel: string): Promise => { + const addsAction = async (suffix?: string): Promise => { const messageAction: MessageAction = { type: 'custom', value: randomUUID() }; - const [messageTimetoken] = await publishMessages(pubnub, 1, channel); + // Only the publish is seeding. The addMessageAction call below is what the test asserts on and is never retried. + const { + channel, + result: [messageTimetoken], + } = await seedOnFreshChannel( + () => uniqueChannel(suffix), + (name) => publishMessages(pubnub, 1, name), + { label: 'addMessageAction seed' }, + ); const response = await pubnub.addMessageAction({ channel, messageTimetoken, action: messageAction }); @@ -81,9 +99,9 @@ describe('endpoints/message_actions: message actions against the service', funct assert(response.data.actionTimetoken); }; - it('add message action', () => addsAction(uniqueChannel())); + it('add message action', () => addsAction()); - it('add message action with encoded channel', () => addsAction(uniqueChannel('#1'))); + it('add message action with encoded channel', () => addsAction('#1')); it('add message action should trigger event', async () => { const messageAction: MessageAction = { type: 'custom', value: randomUUID() }; @@ -94,7 +112,12 @@ describe('endpoints/message_actions: message actions against the service', funct pubnub.subscribe({ channels: [channel] }); await ready; - const published = await pubnub.publish({ channel, message: { hello: 'test' }, sendByPost: true }); + // Seeding: the message the action is attached to. A duplicate stored by a retried publish is harmless, the assertions use the + // timetoken of the publish that answered. The addMessageAction call below is the trigger and is not retried. + const published = await retryTransient( + () => pubnub.publish({ channel, message: { hello: 'test' }, sendByPost: true }), + { label: 'publish seed' }, + ); await pubnub.addMessageAction({ channel, messageTimetoken: published.timetoken, action: messageAction }); const messageActionEvent = await event; @@ -110,9 +133,13 @@ describe('endpoints/message_actions: message actions against the service', funct }); describe('removeMessageAction', () => { - const removesAction = async (channel: string): Promise => { - const messageTimetokens = await publishMessages(pubnub, 1, channel); - const actionTimetokens = await addActions(pubnub, 1, messageTimetokens, channel); + const removesAction = async (suffix?: string): Promise => { + const { + channel, + result: { messageTimetokens, actionTimetokens }, + } = await seedOnFreshChannel(() => uniqueChannel(suffix), seedActions(1, 1), { + label: 'removeMessageAction seed', + }); await eventually( async () => { @@ -138,14 +165,15 @@ describe('endpoints/message_actions: message actions against the service', funct ); }; - it('remove message action', () => removesAction(uniqueChannel())); + it('remove message action', () => removesAction()); - it('remove message action with encoded channel', () => removesAction(uniqueChannel('#1'))); + it('remove message action with encoded channel', () => removesAction('#1')); it('remove message action should trigger event', async () => { - const channel = uniqueChannel(); - const messageTimetokens = await publishMessages(pubnub, 1, channel); - const actionTimetokens = await addActions(pubnub, 1, messageTimetokens, channel); + const { + channel, + result: { messageTimetokens, actionTimetokens }, + } = await seedOnFreshChannel(uniqueChannel, seedActions(1, 1), { label: 'removeMessageAction event seed' }); const ready = connected(pubnub, { channels: [channel] }); const event = expectEvent(pubnub, 'messageAction', (e) => e.event === 'removed'); @@ -169,9 +197,11 @@ describe('endpoints/message_actions: message actions against the service', funct }); describe('getMessageAction', () => { - const fetchesActions = async (channel: string): Promise => { - const messageTimetokens = await publishMessages(pubnub, 2, channel); - const actionTimetokens = await addActions(pubnub, 3, messageTimetokens, channel); + const fetchesActions = async (suffix?: string): Promise => { + const { + channel, + result: { actionTimetokens }, + } = await seedOnFreshChannel(() => uniqueChannel(suffix), seedActions(2, 3), { label: 'getMessageActions seed' }); const lastPublishedActionTimetoken = actionTimetokens[actionTimetokens.length - 1]; const firstPublishedActionTimetoken = actionTimetokens[0]; @@ -190,14 +220,15 @@ describe('endpoints/message_actions: message actions against the service', funct ); }; - it('fetch message actions', () => fetchesActions(uniqueChannel())); + it('fetch message actions', () => fetchesActions()); - it('fetch message actions with encoded channel', () => fetchesActions(uniqueChannel('#1'))); + it('fetch message actions with encoded channel', () => fetchesActions('#1')); it('fetch next message actions page', async () => { - const channel = uniqueChannel(); - const messageTimetokens = await publishMessages(pubnub, 2, channel); - const actionTimetokens = await addActions(pubnub, 5, messageTimetokens, channel); + const { + channel, + result: { actionTimetokens }, + } = await seedOnFreshChannel(uniqueChannel, seedActions(2, 5), { label: 'getMessageActions page seed' }); const lastPublishedActionTimetoken = actionTimetokens[actionTimetokens.length - 1]; const halfSize = Math.floor(actionTimetokens.length * 0.5); const firstPublishedActionTimetoken = actionTimetokens[0]; diff --git a/test/integration/live/support/datasync-live-helpers.ts b/test/integration/live/support/datasync-live-helpers.ts index 4a1c423d2..196ace5dc 100644 --- a/test/integration/live/support/datasync-live-helpers.ts +++ b/test/integration/live/support/datasync-live-helpers.ts @@ -6,6 +6,7 @@ import assert from 'node:assert'; import PubNub from 'pubnub'; import { createClient } from './clients'; import { keysOf } from './env'; +import { createWithRunUniqueId } from './retry'; import { connected } from './wait'; // -------------------------------------------------------- @@ -229,15 +230,20 @@ export async function createCustomerEntity( overrides: Record = {}, ): Promise> { const id = freshId('customer'); - const res = await pn.dataSync.createEntity({ - id, - class: ENTITY_CLASS_CUSTOMER, - data: { - classVersion: CLASS_VERSION, - status: 'active', - payload: customerPayload(id, overrides), - }, - }); + const res = await createWithRunUniqueId( + () => + pn.dataSync.createEntity({ + id, + class: ENTITY_CLASS_CUSTOMER, + data: { + classVersion: CLASS_VERSION, + status: 'active', + payload: customerPayload(id, overrides), + }, + }), + () => pn.dataSync.getEntity({ id }), + { label: 'createEntity' }, + ); return { obj: res.data, dispose: () => safeRemove(() => pn.dataSync.removeEntity({ id: res.data.id })) }; } @@ -246,15 +252,20 @@ export async function createLoanQuoteEntity( overrides: Record = {}, ): Promise> { const id = freshId('loanquote'); - const res = await pn.dataSync.createEntity({ - id, - class: ENTITY_CLASS_LOAN_QUOTE, - data: { - classVersion: CLASS_VERSION, - status: 'active', - payload: loanQuotePayload(id, overrides), - }, - }); + const res = await createWithRunUniqueId( + () => + pn.dataSync.createEntity({ + id, + class: ENTITY_CLASS_LOAN_QUOTE, + data: { + classVersion: CLASS_VERSION, + status: 'active', + payload: loanQuotePayload(id, overrides), + }, + }), + () => pn.dataSync.getEntity({ id }), + { label: 'createEntity' }, + ); return { obj: res.data, dispose: () => safeRemove(() => pn.dataSync.removeEntity({ id: res.data.id })) }; } @@ -265,17 +276,22 @@ export async function createRequestedByRelationship( overrides: Record = {}, ): Promise> { const id = freshId('requested-by'); - const res = await pn.dataSync.createRelationship({ - id, - class: RELATIONSHIP_CLASS_REQUESTED_BY, - entityAId: customerId, - entityBId: loanQuoteId, - data: { - classVersion: CLASS_VERSION, - status: 'active', - payload: { linkedAt: '2026-07-06T10:00:00.000Z', ...overrides }, - }, - }); + const res = await createWithRunUniqueId( + () => + pn.dataSync.createRelationship({ + id, + class: RELATIONSHIP_CLASS_REQUESTED_BY, + entityAId: customerId, + entityBId: loanQuoteId, + data: { + classVersion: CLASS_VERSION, + status: 'active', + payload: { linkedAt: '2026-07-06T10:00:00.000Z', ...overrides }, + }, + }), + () => pn.dataSync.getRelationship({ id }), + { label: 'createRelationship' }, + ); return { obj: res.data, dispose: () => safeRemove(() => pn.dataSync.removeRelationship({ id: res.data.id })) }; } @@ -284,10 +300,15 @@ export async function createUserObj( overrides: Record = {}, ): Promise> { const id = freshId('user'); - const res = await pn.dataSync.createUser({ - id, - data: { classVersion: CLASS_VERSION, status: 'active', payload: userPayload(overrides) }, - }); + const res = await createWithRunUniqueId( + () => + pn.dataSync.createUser({ + id, + data: { classVersion: CLASS_VERSION, status: 'active', payload: userPayload(overrides) }, + }), + () => pn.dataSync.getUser({ id }), + { label: 'createUser' }, + ); return { obj: res.data, dispose: () => safeRemove(() => pn.dataSync.removeUser({ id: res.data.id })) }; } @@ -296,10 +317,15 @@ export async function createChannelObj( overrides: Record = {}, ): Promise> { const id = freshId('JSchannel'); - const res = await pn.dataSync.createChannel({ - id, - data: { classVersion: CLASS_VERSION, status: 'active', payload: channelPayload(overrides) }, - }); + const res = await createWithRunUniqueId( + () => + pn.dataSync.createChannel({ + id, + data: { classVersion: CLASS_VERSION, status: 'active', payload: channelPayload(overrides) }, + }), + () => pn.dataSync.getChannel({ id }), + { label: 'createChannel' }, + ); return { obj: res.data, dispose: () => safeRemove(() => pn.dataSync.removeChannel({ id: res.data.id })) }; } @@ -310,16 +336,21 @@ export async function createMembershipObj( overrides: Record = {}, ): Promise> { const id = freshId('membership'); - const res = await pn.dataSync.createMembership({ - id, - userId, - channelId, - data: { - classVersion: CLASS_VERSION, - status: 'active', - payload: { role: 'member', joinedAt: '2026-07-06T10:00:00.000Z', ...overrides }, - }, - }); + const res = await createWithRunUniqueId( + () => + pn.dataSync.createMembership({ + id, + userId, + channelId, + data: { + classVersion: CLASS_VERSION, + status: 'active', + payload: { role: 'member', joinedAt: '2026-07-06T10:00:00.000Z', ...overrides }, + }, + }), + () => pn.dataSync.getMembership({ id }), + { label: 'createMembership' }, + ); return { obj: res.data, dispose: () => safeRemove(() => pn.dataSync.removeMembership({ id: res.data.id })) }; } diff --git a/test/integration/live/support/explain-error.ts b/test/integration/live/support/explain-error.ts new file mode 100644 index 000000000..f0f1d4d4c --- /dev/null +++ b/test/integration/live/support/explain-error.ts @@ -0,0 +1,125 @@ +/** + * Makes a rejected PubNub SDK call explain itself in the failure list. + * + * The SDK rejects most service failures with the generic message "REST API request processing error, check status for details". + * What explains the failure (category, HTTP status, operation, the service's own error body) sits on `error.status`, which the + * Mocha reporter never prints. `explainSdkError(err)` finds the SDK error (the error itself, or one on its `.cause` chain, which + * is where `eventually` and `settled` keep the last failure) and appends a two line summary to the message of the error that + * Mocha reports. It also puts the new message into the stack header, because Mocha's reporter splits message from stack by + * looking the message up in the stack and drops the "Caused by" section when it cannot find it. + * + * Nothing else about the error changes: same object, same class, same stack frames, same `.cause`. The summary goes through + * `redact()` (keys of the environment, key-shaped and token-like strings) and through `scrub()` below (URLs are cut to their + * path without the query, values of key/token/auth/signature fields are masked). + * + * It is wired once, for every test and every hook of both npm scripts, in support/hooks.ts. + */ +import { redact } from './env'; + +type SdkStatusLike = { + category?: unknown; + statusCode?: unknown; + operation?: unknown; + errorData?: unknown; +}; + +const MAX_DETAIL = 500; +const MAX_DEPTH = 6; + +/** Errors already annotated, so one error object that Mocha reports twice (test and hook) is annotated once. */ +const annotated = new WeakSet(); + +const isObject = (value: unknown): value is Record => typeof value === 'object' && value !== null; + +/** The status an SDK error carries: `error.status` (PubNubError) or the error itself when it has the status fields (PubNubAPIError). */ +function sdkStatusOf(error: Record): SdkStatusLike | undefined { + const status = error.status; + if (isObject(status) && ('category' in status || typeof status.statusCode === 'number')) return status; + if (typeof error.category === 'string' && typeof error.statusCode === 'number') return error; + return undefined; +} + +/** Walks `err` and its `.cause` chain and returns the first SDK status found. */ +function findSdkStatus(err: unknown): SdkStatusLike | undefined { + const seen = new Set(); + let current: unknown = err; + for (let depth = 0; depth < MAX_DEPTH && isObject(current) && !seen.has(current); depth += 1) { + seen.add(current); + const status = sdkStatusOf(current); + if (status) return status; + current = current.cause; + } + return undefined; +} + +/** Cuts every URL to its path (no host, no query, no fragment: queries carry keys and signatures). */ +function stripUrls(text: string): string { + return text.replace(/\bhttps?:\/\/[^\s"'<>)\]]+/gi, (url) => { + try { + return ``; + } catch { + return ''; + } + }); +} + +/** Masks the value of fields and query parameters whose name says they are credentials. */ +function maskCredentialFields(text: string): string { + const name = + '(?:auth(?:Key|Token)?|token|signature|sig|secret(?:Key)?|publish(?:Key)?|subscribe(?:Key)?|cipher(?:Key)?|password|api[-_]?key|access[-_]?key)'; + return text + .replace(new RegExp(`("${name}"\\s*:\\s*)"[^"]*"`, 'gi'), '$1"***"') + .replace(new RegExp(`(\\b${name}\\s*[=:]\\s*)[^\\s,&"'}]+`, 'gi'), '$1***'); +} + +/** `redact()` first (it knows the exact keys of this run), then the pattern based scrubbing. */ +function scrub(text: string): string { + return maskCredentialFields(stripUrls(redact(text))); +} + +function compact(value: unknown): string { + if (value === undefined || value === null) return ''; + if (typeof value === 'string') return value; + if (value instanceof Error) return `${value.name}: ${value.message}`; + try { + return JSON.stringify(value) ?? ''; + } catch { + return '[unserializable error data]'; + } +} + +/** The two line summary of the SDK status, or undefined when `err` is not (and does not wrap) an SDK error. */ +export function summarizeSdkError(err: unknown): string | undefined { + const status = findSdkStatus(err); + if (!status) return undefined; + const head = + `PubNub SDK status: category=${String(status.category ?? 'n/a')} statusCode=${String(status.statusCode ?? 'n/a')} ` + + `operation=${String(status.operation ?? 'n/a')}`; + let detail = compact(status.errorData).replace(/\s+/g, ' ').trim(); + if (detail.length > MAX_DETAIL) detail = `${detail.slice(0, MAX_DETAIL)}... [truncated]`; + return scrub(detail ? `${head}\n service said: ${detail}` : `${head}\n service said: (no error data)`); +} + +/** + * Appends the SDK summary to `err.message` and to the message in `err.stack`. No-op for anything that is not an SDK error or + * does not wrap one, for an error already annotated, and when the error cannot be modified. Never throws. + */ +export function explainSdkError(err: unknown): void { + try { + if (!(err instanceof Error) || annotated.has(err)) return; + const summary = summarizeSdkError(err); + if (!summary) return; + annotated.add(err); + const original = err.message; + // Read the stack before touching the message: V8 may format the header lazily, on the first read. + const stackBefore = err.stack; + // Mocha prints the first line of the message after 5 spaces, so the continuation lines line up under it. + const message = `${original}\n${summary.replace(/^/gm, ' ')}`; + err.message = message; + if (typeof stackBefore === 'string' && original && !String(err.stack).includes(message)) { + err.stack = stackBefore.replace(original, () => message); + } + } catch { + /* explaining a failure must never change or hide it */ + } +} diff --git a/test/integration/live/support/hooks.ts b/test/integration/live/support/hooks.ts index 61be4c1b6..0b4395168 100644 --- a/test/integration/live/support/hooks.ts +++ b/test/integration/live/support/hooks.ts @@ -10,6 +10,10 @@ * the next run's "before all" cleanup retries and fails the run when the keyset is still dirty). One exception: a failed revoke of * the application-level Access Manager grant leaves a secured keyset open, so it fails the run. * + * failure explanation: every failure of a test or of a hook (before, beforeEach, after, ...) that is, or wraps through `.cause`, a + * PubNub SDK error gets the SDK's status (category, status code, operation, the service's error data) appended to its message, + * so the failure list at the end of the run explains itself. See explain-error.ts. Installed for both npm scripts. + * * The endpoint tests (`.mocharc.endpoints.cjs`, LIVE_TIER=endpoints) check no role up front: a test fails with a "missing keys" * message (env names) when the role it builds a client for is not set. Janitor and preflight default to off there. * @@ -19,6 +23,7 @@ import { destroyClients } from './clients'; import { restoreAllConsoles } from './console'; import { describeRoles, env, janitorEnabled, preflightEnabled, redact, ROLES } from './env'; +import { explainSdkError } from './explain-error'; import { formatReport, runJanitor, verifyApplicationGrantClosed } from './janitor'; import { runPreflight } from './preflight'; @@ -35,6 +40,30 @@ function isFullRun(argv: readonly string[] = process.argv.slice(2)): boolean { /** True under `.mocharc.endpoints.cjs` (it sets LIVE_TIER=endpoints): no role is checked up front (each test needs its own, see `keysOf`), janitor and preflight default to off. */ const endpointsTier = process.env.LIVE_TIER === 'endpoints'; +/** + * Every failure of the run, test or hook, passes through `Runner#fail` before the reporter sees it, so wrapping it once covers + * all of them, including `before` and `beforeEach` hooks, which `afterEach` cannot reach (a failed hook is reported against the + * hook, not `currentTest`). The wrapper only annotates the error and then calls the original with the same arguments: results, + * counts and the error object stay as they are. `Runner#fail` is not part of Mocha's documented API, so when it is missing (a + * Mocha upgrade) the install reports false and the `afterEach` fallback below annotates failed tests (hook failures stay plain). + */ +function installFailureExplainer(): boolean { + try { + // eslint-disable-next-line @typescript-eslint/no-require-imports + const Runner = require('mocha/lib/runner') as { prototype: { fail?: (...args: unknown[]) => unknown } }; + const original = Runner.prototype.fail; + if (typeof original !== 'function') return false; + Runner.prototype.fail = function (this: unknown, ...args: unknown[]): unknown { + explainSdkError(args[1]); + return original.apply(this, args); + }; + return true; + } catch { + return false; + } +} +const explainerInstalled = installFailureExplainer(); + const banner = (lines: string[]): void => { const bar = '!'.repeat(78); console.error(`\n${bar}\n${lines.map(redact).join('\n')}\n${bar}\n`); @@ -88,7 +117,8 @@ export const mochaHooks = { console.log('preflight: keyset features ok'); } }, - afterEach(): void { + afterEach(this: Mocha.Context): void { + if (!explainerInstalled) explainSdkError(this.currentTest?.err); restoreAllConsoles(); destroyClients(); }, diff --git a/test/integration/live/support/retry.ts b/test/integration/live/support/retry.ts index dcae7c90f..084928e47 100644 --- a/test/integration/live/support/retry.ts +++ b/test/integration/live/support/retry.ts @@ -84,3 +84,93 @@ export async function retryTransient(fn: () => Promise, options: RetryOpti } } } + +/** + * True for the answer a DataSync create gets when the id already exists: HTTP 409 with the service code DS-0301 in the error + * body (`status.errorData`, for example `{ errors: [{ errorCode: 'DS-0301', ... }] }`; see the duplicate create test in + * api-reference/datasync.test.ts). + */ +export function isAlreadyExists(error: unknown): boolean { + if (statusCode(error) !== 409) return false; + const data = (error as { status?: { errorData?: unknown } } | undefined)?.status?.errorData; + try { + return JSON.stringify(data ?? {}).includes('DS-0301'); + } catch { + return false; + } +} + +/** + * Creates a DataSync object whose id is unique to this run (`freshId`, `projectionId`) and tolerates a transient service error. + * + * A create is not idempotent, so a plain retry is unsafe: when the first attempt was stored but its answer was lost (a 5xx after + * the write, a timeout), the retry meets the object it just created and gets 409 (DS-0301). With an id nobody else can hold, + * that 409 can only mean "the earlier attempt was stored", so it counts as success, and the object is read back (`readBack`, a get + * by id) so the caller still gets the eTag and timestamps the create would have returned. + * + * - failure of the FIRST attempt that is not transient (including a 409, which means the id was not unique): thrown at once. + * - transient failure (`isTransient`): retried like `retryTransient`, up to `attempts` tries (default 3). + * - 409 / DS-0301 on a RETRY: resolved through `readBack` (itself retried on transient errors). Never on the first attempt. + * - any other failure of a retry (403, 400, ...): thrown. + * + * Use it for setup, and for a create that only triggers an event the test then asserts on. Never inside a documentation sample + * (those use fixed literal ids, where a 409 means a leftover and must fail). Logs one line per retry and one when the 409 path + * resolves; ids, payloads and keys are never logged. + */ +export async function createWithRunUniqueId( + create: () => Promise, + readBack: () => Promise, + options: RetryOptions = {}, +): Promise { + const attempts = options.attempts ?? 3; + const backoff = options.backoff ?? 500; + const label = options.label ?? 'create'; + for (let attempt = 1; ; attempt += 1) { + try { + return await create(); + } catch (error) { + if (attempt > 1 && isAlreadyExists(error)) { + console.log( + `createWithRunUniqueId: ${label} answered 409 DS-0301 on attempt ${attempt} of ${attempts}: ` + + 'an earlier attempt was stored, reading the object back', + ); + return retryTransient(readBack, { ...options, label: `${label} read-back` }); + } + if (attempt >= attempts || !isTransient(error)) throw error; + console.log( + `createWithRunUniqueId: ${label} failed (attempt ${attempt} of ${attempts}, ` + + `category ${categoryOf(error) ?? 'none'}, status ${statusCode(error) ?? 'none'}), retrying`, + ); + await pause(backoff * attempt); + } + } +} + +/** + * Seeds state on a channel with a run-unique name and, when a seeding call fails transiently, throws that channel away and runs + * the whole seeding again on a new one (up to `attempts` tries, default 3). Nothing else depends on the channel name, and the + * tests that use it assert exact counts of what the seeding wrote, so a half-written channel must never be reused: a publish that + * was stored although its answer was lost would add one message, and a plain retry would add another. Only the seeding is + * covered. The call a test asserts on stays outside and is never retried. Returns the channel that was seeded and what `seed` returned. + */ +export async function seedOnFreshChannel( + newChannel: () => string, + seed: (channel: string) => Promise, + options: RetryOptions = {}, +): Promise<{ channel: string; result: T }> { + const attempts = options.attempts ?? 3; + const backoff = options.backoff ?? 500; + for (let attempt = 1; ; attempt += 1) { + const channel = newChannel(); + try { + return { channel, result: await seed(channel) }; + } catch (error) { + if (attempt >= attempts || !isTransient(error)) throw error; + console.log( + `seedOnFreshChannel: ${options.label ?? 'seed'} failed (attempt ${attempt} of ${attempts}, ` + + `category ${categoryOf(error) ?? 'none'}, status ${statusCode(error) ?? 'none'}), starting again on a new channel`, + ); + await pause(backoff * attempt); + } + } +} From 4c773aedafb881e9763979076dc0fd6c965d5312 Mon Sep 17 00:00:00 2001 From: Mateusz Wiktor Date: Tue, 17 Nov 2026 12:02:51 +0100 Subject: [PATCH 05/13] feat: implement per-test cleanup for DataSync objects to enhance test reliability - Introduced `removeAfterTest` functionality to register the removal of DataSync objects created during tests, ensuring they are cleaned up regardless of test outcomes. - Updated multiple test files to utilize `removeAfterTest`, improving test isolation and preventing leftover objects from affecting subsequent tests. - Added `cleanupAfterTest` and `runCleanups` methods to manage cleanup operations efficiently after each test. - Enhanced README documentation to reflect the new cleanup methodology and its benefits for test reliability. --- test/integration/live/README.md | 8 +- .../data_sync/events.channel.test.ts | 9 +- .../endpoints/data_sync/events.entity.test.ts | 9 +- .../data_sync/events.membership.test.ts | 15 +- .../data_sync/events.relationship.test.ts | 15 +- .../endpoints/data_sync/events.user.test.ts | 9 +- .../endpoints/data_sync/projections.test.ts | 8 +- .../live/endpoints/message_actions.test.ts | 22 ++- test/integration/live/support/cleanup.ts | 40 +++++ .../live/support/datasync-live-helpers.ts | 50 +++++-- test/integration/live/support/hooks.ts | 15 +- test/integration/live/support/retry.ts | 9 +- test/integration/live/support/util.ts | 26 +++- test/integration/live/support/wait.ts | 139 +++++++++++++++++- 14 files changed, 316 insertions(+), 58 deletions(-) create mode 100644 test/integration/live/support/cleanup.ts diff --git a/test/integration/live/README.md b/test/integration/live/README.md index 440beee38..fe621f75f 100644 --- a/test/integration/live/README.md +++ b/test/integration/live/README.md @@ -43,7 +43,8 @@ Rebuild with `npm run build:node` after a change under `src/`, otherwise the tie - **Unique run ids.** Anything a test asserts on exactly and does not own (a channel it reads back, a payload marker) carries a per-run suffix: `unique('name')` from `support/util.ts`, or `randomUUID()` in the payload. Such names cannot be listed in `support/resources.ts`, so the janitor never clears them. Their messages expire with the retention of the keyset. - **Console.** A client that logs at Debug or Trace level prints its configuration with the real keys. Wrap the test in `quietConsole()` (`support/console.ts`) and call `restore()` in a `finally`. The root hook also mutes the console while it destroys clients, so the destroy-time dumps stay out of the log. Sample `console.log` output is fine, but a sample that prints a granted token (`console.log('Granted Token:', token)`) runs under `captureConsole('log')`. The root hook restores any capture a failed test left behind (`restoreAllConsoles()`). - **Transient service errors: retry setup, never the sample.** Setup and cleanup code outside the markers (seeding, resets) may wrap a call in `retryTransient` (`support/retry.ts`). It repeats only HTTP 5xx, 429 and network, timeout or unclassified transport failures, never a 4xx, a validation error or an assertion error, and it logs one `retryTransient:` line per retry. Code inside the markers, the assertions about a sample's effect, and calls whose failure a test asserts are never retried or wrapped. Mocha `retries` stays 0 and timeouts are not lengthened to hide flakiness. -- **Creates with a run-unique id, and seeding on a fresh channel (endpoint tests only).** A create is not idempotent, so `retryTransient` alone is wrong for it: a retry after a first attempt that the service stored answers 409. `createWithRunUniqueId(create, readBack, { label })` (`support/retry.ts`) is for ids no one else can hold (`freshId`, `projectionId`): it retries transient failures, treats a 409 with DS-0301 on a RETRY as "the earlier attempt was stored" and returns `readBack()` (a get by id), and throws a 409 on the first attempt. `seedOnFreshChannel(newChannel, seed)` seeds a run-unique channel and, on a transient failure, starts the whole seeding again on a new channel (tests that assert exact counts must never reuse a half-seeded channel). Neither belongs in a documentation sample, which uses fixed literal ids and runs once. +- **Creates with a run-unique id, and seeding on a fresh channel (endpoint tests only).** A create is not idempotent, so `retryTransient` alone is wrong for it: a retry after a first attempt that the service stored answers 409. `createWithRunUniqueId(create, readBack, { label })` (`support/retry.ts`) is for ids no one else can hold (`freshId`, `projectionId`; both end in `uniqueSuffix()` of `support/util.ts`: the time in base 36 plus 6 random base-36 characters, so an id cannot meet an object that earlier runs left on a keyset that is years old): it retries transient failures, treats a 409 with DS-0301 on a RETRY as "the earlier attempt was stored" and returns `readBack()` (a get by id), and throws a 409 on the first attempt. `seedOnFreshChannel(newChannel, seed)` seeds a run-unique channel and, on a transient failure, starts the whole seeding again on a new channel (tests that assert exact counts must never reuse a half-seeded channel). Neither belongs in a documentation sample, which uses fixed literal ids and runs once. +- **Objects a test creates are removed whatever the outcome (endpoint tests).** Register the id before the create: `removeAfterTest(client, 'entity' | 'user' | 'channel' | 'relationship' | 'membership', freshId(...))` (`support/datasync-live-helpers.ts`, backed by `support/cleanup.ts`). The root `afterEach` removes the registered objects, newest first, before it destroys the clients, whether the test passed, failed on an assertion or failed in the create itself. A 404 is expected and silent. Nothing lists or sweeps the keyset: only ids this test generated are removed. A hook that creates objects (`before`) removes them in its own `after`. - **Failures explain themselves.** `support/hooks.ts` wraps Mocha's `Runner#fail` once, so every failed test or hook of both npm scripts whose error is (or wraps, through `.cause`) a PubNub SDK error gets two lines appended to its message in the failure list: `category`, `statusCode`, `operation` and the service's error data (`support/explain-error.ts`). Keys, tokens and URL queries are masked. No test opts in, and nothing about pass or fail changes. - **Retried publishes can be stored twice.** A failed attempt may have reached the service. A seeder whose test counts messages puts `run: seedRun.id` in the payload, publishes through `seedPublish` and calls `settleSeeded` (`support/history.ts`), which deletes stored copies the caller never received a timetoken for. - **Waiting.** Poll with `eventually`, listen with `watch` / `expectEvent`. Use a fixed `sleep` only where the test proves that something did NOT arrive. `PRESENCE_EVENT_TIMEOUT` (30 s) is for a presence event the test caused; `PRESENCE_TIMEOUT` (45 s) is for polling `hereNow` and similar reads. @@ -56,10 +57,11 @@ Rebuild with `npm run build:node` after a change under `src/`, otherwise the tie | :--- | :--- | | `env.ts` | `env()`, `keysOf(role)`, `openKeys()`, `securedKeys(role?)`, `subscribeKeyOnly()`, `describeRoles()`, `loadEnv()`, `redact()` | | `clients.ts` | `createClient(role, userId, extra?)`, `shopper()`, `friend()`, `observer()`, `server(role?)`, `unreachable`, `track()`, `destroyClients()` | -| `wait.ts` | `eventually`, `settled`, `watch` (returns a `Watch` with `events`, `next`, `stop`), `connected`, `expectEvent`, `expectStatus`, `sleep`, `PRESENCE_TIMEOUT`, `PRESENCE_EVENT_TIMEOUT` | +| `wait.ts` | `eventually`, `settled`, `watch` (returns a `Watch` with `events`, `next`, `stop`), `connected`, `expectEvent`, `expectStatus`, `sleep`, `PRESENCE_TIMEOUT`, `PRESENCE_EVENT_TIMEOUT`. A timed-out `expectEvent`, `connected`, `expectStatus` or `Watch.next` says what the client saw meanwhile: statuses (category, operation, the cursor of a connected status), the count and kinds of events with their channels, and the channels it was subscribed to. Pass `{ describe: () => '...' }` to add what the test knows (timetokens). | | `assertions.ts` | `assertTimetoken` (a string, as `publish()` returns it), `assertServerTime` (a number, as `time()` returns it), `addToTimetoken`, `rejectsValidation`, `rejectsWithCategory` | | `placeholders.ts` | `withPlaceholders(client, values)` | -| `util.ts` | `unique(base)`, `sorted(values)` | +| `util.ts` | `unique(base)`, `uniqueSuffix()`, `sorted(values)` | +| `cleanup.ts` | `cleanupAfterTest(label, run)`, `runCleanups()` (called by the root `afterEach`) | | `history.ts` | `storedMessage(client, channel, timetoken)`, `untilStored(channel, tokens)`, `newSeedRun`, `seedPublish`, `settleSeeded` (seeding with retry and duplicate removal) | | `retry.ts` | `retryTransient(fn, { attempts, label })`, `isTransient(error)`; the janitor's `retry` calls it too; also `createWithRunUniqueId`, `isAlreadyExists`, `seedOnFreshChannel` | | `explain-error.ts` | `explainSdkError(err)`: appends the SDK status summary to a failure's message; installed by `hooks.ts` | diff --git a/test/integration/live/endpoints/data_sync/events.channel.test.ts b/test/integration/live/endpoints/data_sync/events.channel.test.ts index dac888e14..4ef0f286f 100644 --- a/test/integration/live/endpoints/data_sync/events.channel.test.ts +++ b/test/integration/live/endpoints/data_sync/events.channel.test.ts @@ -17,6 +17,7 @@ import { channelPayload, freshId, freshPubNub, + removeAfterTest, } from '../../support/datasync-live-helpers'; import { createWithRunUniqueId } from '../../support/retry'; @@ -38,7 +39,7 @@ describe('endpoints/data_sync/events.channel: DataSync events — Channel', func ).then(() => undefined); it('channel create → "create" event (objectType "channel")', async () => { - const id = freshId('channel'); + const id = removeAfterTest(pubnub, 'channel', freshId('channel')); const event = await captureEvent( pubnub, [id], @@ -63,7 +64,7 @@ describe('endpoints/data_sync/events.channel: DataSync events — Channel', func }); it('channel update (PUT) → "update" event', async () => { - const id = freshId('channel'); + const id = removeAfterTest(pubnub, 'channel', freshId('channel')); await createChannel(id); const event = await captureEvent( @@ -100,7 +101,7 @@ describe('endpoints/data_sync/events.channel: DataSync events — Channel', func }); it('channel update → "update" event', async () => { - const id = freshId('channel'); + const id = removeAfterTest(pubnub, 'channel', freshId('channel')); // Seed `memberCount` first — the service rejects update-replace of a property that does not already // exist (`SYN-0006: Property does not exist`). await createChannel(id, channelPayload({ memberCount: 1 })); @@ -128,7 +129,7 @@ describe('endpoints/data_sync/events.channel: DataSync events — Channel', func }); it('channel delete → "delete" event with only {id, deletedAt}', async () => { - const id = freshId('channel'); + const id = removeAfterTest(pubnub, 'channel', freshId('channel')); await createChannel(id); const event = await captureEvent( diff --git a/test/integration/live/endpoints/data_sync/events.entity.test.ts b/test/integration/live/endpoints/data_sync/events.entity.test.ts index 8c9e61318..f53a095f1 100644 --- a/test/integration/live/endpoints/data_sync/events.entity.test.ts +++ b/test/integration/live/endpoints/data_sync/events.entity.test.ts @@ -19,6 +19,7 @@ import { customerPayload, freshId, freshPubNub, + removeAfterTest, } from '../../support/datasync-live-helpers'; import { createWithRunUniqueId } from '../../support/retry'; @@ -49,7 +50,7 @@ describe('endpoints/data_sync/events.entity: DataSync events — Entity Customer ).then(() => undefined); it('entity create - create event', async () => { - const id = freshId('customer'); + const id = removeAfterTest(pubnub, 'entity', freshId('customer')); const event = await captureEvent( pubnub, [id], @@ -74,7 +75,7 @@ describe('endpoints/data_sync/events.entity: DataSync events — Entity Customer }); it('entity update (PUT) - update event', async () => { - const id = freshId('customer'); + const id = removeAfterTest(pubnub, 'entity', freshId('customer')); await createCustomer(id); const event = await captureEvent( @@ -111,7 +112,7 @@ describe('endpoints/data_sync/events.entity: DataSync events — Entity Customer }); it('entity update update event', async () => { - const id = freshId('customer'); + const id = removeAfterTest(pubnub, 'entity', freshId('customer')); await createCustomer(id); const event = await captureEvent( @@ -137,7 +138,7 @@ describe('endpoints/data_sync/events.entity: DataSync events — Entity Customer }); it('entity delete - delete event with only id, deletedAt', async () => { - const id = freshId('customer'); + const id = removeAfterTest(pubnub, 'entity', freshId('customer')); await createCustomer(id); const event = await captureEvent( diff --git a/test/integration/live/endpoints/data_sync/events.membership.test.ts b/test/integration/live/endpoints/data_sync/events.membership.test.ts index fe303babe..34b091538 100644 --- a/test/integration/live/endpoints/data_sync/events.membership.test.ts +++ b/test/integration/live/endpoints/data_sync/events.membership.test.ts @@ -21,6 +21,7 @@ import { channelPayload, freshId, freshPubNub, + removeAfterTest, safeRemove, userPayload, } from '../../support/datasync-live-helpers'; @@ -37,8 +38,8 @@ describe('endpoints/data_sync/events.membership: DataSync events — Membership' /** Seed the User + Channel a Membership joins. */ async function seedEndpoints(): Promise<{ userId: string; channelId: string }> { - const userId = freshId('user'); - const channelId = freshId('channel'); + const userId = removeAfterTest(pubnub, 'user', freshId('user')); + const channelId = removeAfterTest(pubnub, 'channel', freshId('channel')); // Both ids come from `freshId`, unique to this run: a 409 on a retry means the first attempt was stored (see retry.ts). await createWithRunUniqueId( () => @@ -81,7 +82,7 @@ describe('endpoints/data_sync/events.membership: DataSync events — Membership' it('membership create → "create" event (delivered on channel + user)', async () => { const { userId, channelId } = await seedEndpoints(); - const id = freshId('membership'); + const id = removeAfterTest(pubnub, 'membership', freshId('membership')); const event = await captureEvent( pubnub, @@ -110,7 +111,7 @@ describe('endpoints/data_sync/events.membership: DataSync events — Membership' it('membership update (PUT) → "update" event', async () => { const { userId, channelId } = await seedEndpoints(); - const id = freshId('membership'); + const id = removeAfterTest(pubnub, 'membership', freshId('membership')); await createMembership(id, userId, channelId); const event = await captureEvent( @@ -151,7 +152,7 @@ describe('endpoints/data_sync/events.membership: DataSync events — Membership' it('membership update → "update" event', async () => { const { userId, channelId } = await seedEndpoints(); - const id = freshId('membership'); + const id = removeAfterTest(pubnub, 'membership', freshId('membership')); await createMembership(id, userId, channelId); const event = await captureEvent( @@ -180,7 +181,7 @@ describe('endpoints/data_sync/events.membership: DataSync events — Membership' it('membership delete → "delete" event with {id, deletedAt, channelId, userId}', async () => { const { userId, channelId } = await seedEndpoints(); - const id = freshId('membership'); + const id = removeAfterTest(pubnub, 'membership', freshId('membership')); await createMembership(id, userId, channelId); const event = await captureEvent( @@ -204,7 +205,7 @@ describe('endpoints/data_sync/events.membership: DataSync events — Membership' it('routing proof — a subscriber to the membership id receives nothing', async () => { const { userId, channelId } = await seedEndpoints(); - const id = freshId('membership'); + const id = removeAfterTest(pubnub, 'membership', freshId('membership')); try { await assert.rejects( () => diff --git a/test/integration/live/endpoints/data_sync/events.relationship.test.ts b/test/integration/live/endpoints/data_sync/events.relationship.test.ts index 18af39873..9c2c6f2ef 100644 --- a/test/integration/live/endpoints/data_sync/events.relationship.test.ts +++ b/test/integration/live/endpoints/data_sync/events.relationship.test.ts @@ -24,6 +24,7 @@ import { customerPayload, freshId, freshPubNub, + removeAfterTest, loanQuotePayload, safeRemove, } from '../../support/datasync-live-helpers'; @@ -40,8 +41,8 @@ describe('endpoints/data_sync/events.relationship: DataSync events Relationship /** Seed the two endpoint entities a REQUESTED_BY relationship links. */ async function seedEndpoints(): Promise<{ customerId: string; loanQuoteId: string }> { - const customerId = freshId('customer'); - const loanQuoteId = freshId('loanquote'); + const customerId = removeAfterTest(pubnub, 'entity', freshId('customer')); + const loanQuoteId = removeAfterTest(pubnub, 'entity', freshId('loanquote')); // Both ids come from `freshId`, unique to this run: a 409 on a retry means the first attempt was stored (see retry.ts). await createWithRunUniqueId( () => @@ -95,7 +96,7 @@ describe('endpoints/data_sync/events.relationship: DataSync events Relationship it('relationship create create event delivered on endpoint channels', async () => { const { customerId, loanQuoteId } = await seedEndpoints(); - const relId = freshId('requested-by'); + const relId = removeAfterTest(pubnub, 'relationship', freshId('requested-by')); try { const event = await captureEvent( pubnub, @@ -127,7 +128,7 @@ describe('endpoints/data_sync/events.relationship: DataSync events Relationship it('relationship set (PUT) "update" event', async () => { const { customerId, loanQuoteId } = await seedEndpoints(); - const relId = freshId('requested-by'); + const relId = removeAfterTest(pubnub, 'relationship', freshId('requested-by')); await createRel(relId, customerId, loanQuoteId); try { const event = await captureEvent( @@ -171,7 +172,7 @@ describe('endpoints/data_sync/events.relationship: DataSync events Relationship it('relationship update (PATCH) event', async () => { const { customerId, loanQuoteId } = await seedEndpoints(); - const relId = freshId('requested-by'); + const relId = removeAfterTest(pubnub, 'relationship', freshId('requested-by')); await createRel(relId, customerId, loanQuoteId); try { const event = await captureEvent( @@ -206,7 +207,7 @@ describe('endpoints/data_sync/events.relationship: DataSync events Relationship it('relationship delete event with {id, deletedAt, entityAId, entityBId}', async () => { const { customerId, loanQuoteId } = await seedEndpoints(); - const relId = freshId('requested-by'); + const relId = removeAfterTest(pubnub, 'relationship', freshId('requested-by')); await createRel(relId, customerId, loanQuoteId); const event = await captureEvent( @@ -233,7 +234,7 @@ describe('endpoints/data_sync/events.relationship: DataSync events Relationship it('routing proof — a subscriber to the relationship id receives nothing', async () => { const { customerId, loanQuoteId } = await seedEndpoints(); - const relId = freshId('requested-by'); + const relId = removeAfterTest(pubnub, 'relationship', freshId('requested-by')); try { // Subscribing ONLY to the relationship's own id must time out — the event routes to the // endpoint channels, never the relationship id. diff --git a/test/integration/live/endpoints/data_sync/events.user.test.ts b/test/integration/live/endpoints/data_sync/events.user.test.ts index 48832bb21..e0b4685e5 100644 --- a/test/integration/live/endpoints/data_sync/events.user.test.ts +++ b/test/integration/live/endpoints/data_sync/events.user.test.ts @@ -15,6 +15,7 @@ import { captureEvent, freshId, freshPubNub, + removeAfterTest, userPayload, } from '../../support/datasync-live-helpers'; import { createWithRunUniqueId } from '../../support/retry'; @@ -37,7 +38,7 @@ describe('endpoints/data_sync/events.user: DataSync events — User', function ( ).then(() => undefined); it('user create → "create" event (objectType "user")', async () => { - const id = freshId('user'); + const id = removeAfterTest(pubnub, 'user', freshId('user')); const event = await captureEvent( pubnub, [id], @@ -62,7 +63,7 @@ describe('endpoints/data_sync/events.user: DataSync events — User', function ( }); it('user update (PUT) → "update" event', async () => { - const id = freshId('user'); + const id = removeAfterTest(pubnub, 'user', freshId('user')); await createUser(id); const event = await captureEvent( @@ -95,7 +96,7 @@ describe('endpoints/data_sync/events.user: DataSync events — User', function ( }); it('user update → "update" event', async () => { - const id = freshId('user'); + const id = removeAfterTest(pubnub, 'user', freshId('user')); // Seed `level` first — the service rejects update-replace of a property that does not already // exist (`SYN-0006: Property does not exist`). await createUser(id, userPayload({ level: 'L3' })); @@ -123,7 +124,7 @@ describe('endpoints/data_sync/events.user: DataSync events — User', function ( }); it('user delete → "delete" event with only {id, deletedAt}', async () => { - const id = freshId('user'); + const id = removeAfterTest(pubnub, 'user', freshId('user')); await createUser(id); const event = await captureEvent( diff --git a/test/integration/live/endpoints/data_sync/projections.test.ts b/test/integration/live/endpoints/data_sync/projections.test.ts index 26c68c054..c5584f400 100644 --- a/test/integration/live/endpoints/data_sync/projections.test.ts +++ b/test/integration/live/endpoints/data_sync/projections.test.ts @@ -21,6 +21,7 @@ import PubNub from 'pubnub'; import { createClient } from '../../support/clients'; import { createWithRunUniqueId } from '../../support/retry'; +import { uniqueSuffix } from '../../support/util'; import { sleep } from '../../support/wait'; import { @@ -91,10 +92,11 @@ const itWhenAllClassesEmit = LOAN_QUOTE_AND_RELATIONSHIP_EVENTS_ENABLED ? it : i /** * Dot-separated id, so `dataSyncEntity('.*')` matches the object's own channel — and * `subscription({ projection: 'admin' })` matches `__admin__.*`. (helpers' `freshId` uses - * `-`, which no wildcard matches.) + * `-`, which no wildcard matches.) The part after `.p` is `uniqueSuffix()` (time in base 36 plus 6 random base-36 characters), + * so the id cannot meet an object an earlier run left on the keyset; no `_` (the service rejects it), at most 27 characters here. */ function projectionId(prefix: string): string { - return `${prefix}.p${Date.now().toString(36)}${Math.floor(Math.random() * 90000) + 10000}`; + return `${prefix}.p${uniqueSuffix()}`; } /** Token-only client: same keyset as `freshPubNub()` but deliberately **without** `secretKey`. */ @@ -546,7 +548,7 @@ describe('endpoints/data_sync/projections: DataSync projections admin vs __defau await safeRemove(() => pnPam.dataSync.removeRelationship({ id: relationshipId })); await safeRemove(() => pnPam.dataSync.removeEntity({ id: customerId })); await safeRemove(() => pnPam.dataSync.removeEntity({ id: loanQuoteId })); - pnPam.destroy(); + pnPam?.destroy(); }); /** Readers come from the tier's `createClient`, so the root hook destroys them after each test. */ diff --git a/test/integration/live/endpoints/message_actions.test.ts b/test/integration/live/endpoints/message_actions.test.ts index 6beb98882..6d86026ef 100644 --- a/test/integration/live/endpoints/message_actions.test.ts +++ b/test/integration/live/endpoints/message_actions.test.ts @@ -107,8 +107,13 @@ describe('endpoints/message_actions: message actions against the service', funct const messageAction: MessageAction = { type: 'custom', value: randomUUID() }; const channel = uniqueChannel(); + // Timetokens the test knows, for the timeout message: an action whose timetoken is not after the cursor of the connected + // status (`tt=` in the statuses listed) is one a subscribe request that starts at that cursor does not return. + const known: Record = {}; const ready = connected(pubnub, { channels: [channel] }); - const event = expectEvent(pubnub, 'messageAction', (e) => e.data.value === messageAction.value); + const event = expectEvent(pubnub, 'messageAction', (e) => e.data.value === messageAction.value, { + describe: () => `Timetokens the test knows: ${JSON.stringify(known)}.`, + }); pubnub.subscribe({ channels: [channel] }); await ready; @@ -118,7 +123,13 @@ describe('endpoints/message_actions: message actions against the service', funct () => pubnub.publish({ channel, message: { hello: 'test' }, sendByPost: true }), { label: 'publish seed' }, ); - await pubnub.addMessageAction({ channel, messageTimetoken: published.timetoken, action: messageAction }); + known.published = published.timetoken; + const added = await pubnub.addMessageAction({ + channel, + messageTimetoken: published.timetoken, + action: messageAction, + }); + known.actionAdded = added.data.actionTimetoken; const messageActionEvent = await event; assert(messageActionEvent.data); @@ -175,8 +186,12 @@ describe('endpoints/message_actions: message actions against the service', funct result: { messageTimetokens, actionTimetokens }, } = await seedOnFreshChannel(uniqueChannel, seedActions(1, 1), { label: 'removeMessageAction event seed' }); + // Timetokens and steps the test knows, for the timeout message (see the add test). + const known: Record = { seededAction: actionTimetokens[0] }; const ready = connected(pubnub, { channels: [channel] }); - const event = expectEvent(pubnub, 'messageAction', (e) => e.event === 'removed'); + const event = expectEvent(pubnub, 'messageAction', (e) => e.event === 'removed', { + describe: () => `What the test knows: ${JSON.stringify(known)}.`, + }); pubnub.subscribe({ channels: [channel] }); await ready; @@ -185,6 +200,7 @@ describe('endpoints/message_actions: message actions against the service', funct actionTimetoken: actionTimetokens[0], messageTimetoken: messageTimetokens[0], }); + known.removeAnswered = true; const messageActionEvent = await event; assert(messageActionEvent.data); diff --git a/test/integration/live/support/cleanup.ts b/test/integration/live/support/cleanup.ts new file mode 100644 index 000000000..d1c29454d --- /dev/null +++ b/test/integration/live/support/cleanup.ts @@ -0,0 +1,40 @@ +/** + * Per-test cleanup list. A test registers the removal of an object it is about to create (`cleanupAfterTest`, or the typed + * `removeAfterTest` of datasync-live-helpers.ts), and the root `afterEach` (hooks.ts) runs `runCleanups()` once the test is over, + * passed or failed, before the clients are destroyed. So a failed assertion, a failed seed or a timeout does not leave the object + * on a keyset that outlives the run. + * + * Only what a test registered is removed: nothing here lists, sweeps or matches by prefix. Removals run newest first (a + * relationship or a membership goes before the entities it links). A failure never fails the test, because the test is already + * decided: a 404 is silent (the test removed the object itself, or the create never stored it), any other failure prints one line. + */ +import { categoryOf, retryTransient, statusCode } from './retry'; + +interface Cleanup { + label: string; + run: () => Promise; +} + +let pending: Cleanup[] = []; + +/** Registers `run` to execute after the current test. `label` names it in the log line of a failed removal: never put keys or tokens in it. */ +export function cleanupAfterTest(label: string, run: () => Promise): void { + pending.push({ label, run }); +} + +/** Runs and clears the registered removals, newest first. Never throws. Called by the root `afterEach`. */ +export async function runCleanups(): Promise { + const batch = pending.reverse(); + pending = []; + for (const { label, run } of batch) { + try { + // A removal is idempotent (a second one answers 404), so a transient failure may be retried. + await retryTransient(run, { attempts: 2, backoff: 300, label }); + } catch (error) { + if (statusCode(error) === 404) continue; + console.log( + `cleanup: ${label} failed (category ${categoryOf(error) ?? 'none'}, status ${statusCode(error) ?? 'none'}), the object may be left on the keyset`, + ); + } + } +} diff --git a/test/integration/live/support/datasync-live-helpers.ts b/test/integration/live/support/datasync-live-helpers.ts index 196ace5dc..562ce56f9 100644 --- a/test/integration/live/support/datasync-live-helpers.ts +++ b/test/integration/live/support/datasync-live-helpers.ts @@ -6,7 +6,9 @@ import assert from 'node:assert'; import PubNub from 'pubnub'; import { createClient } from './clients'; import { keysOf } from './env'; +import { cleanupAfterTest } from './cleanup'; import { createWithRunUniqueId } from './retry'; +import { uniqueSuffix } from './util'; import { connected } from './wait'; // -------------------------------------------------------- @@ -56,9 +58,16 @@ export function untrackedPubNub(overrides: Record = {}): PubNub } as PubNub.PubNubConfiguration); } -/** Client-supplied id, underscore-free (service rejects `_` with SYN-0004). */ +/** + * Client-supplied id for an object this run creates: `-`, for example `customer-mh3k9x2a4f7q1z`. The + * suffix (support/util.ts) is the time in base 36 plus 6 random base-36 characters, so it cannot meet an object that earlier + * runs left on the keyset. Underscore-free (the service rejects `_` with SYN-0004), lower-case apart from the caller's prefix, and + * at most 28 characters with the prefixes used here (the longest id known to be accepted is 28 characters). + * `createWithRunUniqueId` (retry.ts) relies on this: a 409 on a retry may only mean that our own first attempt was stored. + * Register the id with `removeAfterTest` before creating the object, so a red test does not leave it behind. + */ export function freshId(prefix: string): string { - return `${prefix}-${String(Math.floor(Math.random() * 90000) + 10000)}`; + return `${prefix}-${uniqueSuffix()}`; } /** Per-run marker used to isolate list/filter assertions against a shared live subkey. */ @@ -68,6 +77,28 @@ export function runMarker(): string { .slice(0, 12); } +/** What `removeAfterTest` knows how to remove. */ +export type DataSyncKind = 'entity' | 'user' | 'channel' | 'relationship' | 'membership'; + +/** + * Registers the removal of an object THIS test creates and returns its id, so it reads `const id = removeAfterTest(pubnub, 'entity', freshId('customer'))`. + * Call it before the create: the root `afterEach` (hooks.ts, `support/cleanup.ts`) then removes the object whether the test passed, + * failed on an assertion, or failed in the create itself (a create whose answer was lost can still have stored the object). The + * removals run newest first, so a relationship or membership goes before the entities it links, and a 404 (the test removed it itself, + * or it was never stored) is expected. It only ever removes the ids registered here, which this run generated. + */ +export function removeAfterTest(pn: PubNub, kind: DataSyncKind, id: string): string { + const remove: Record Promise> = { + entity: () => pn.dataSync.removeEntity({ id }), + user: () => pn.dataSync.removeUser({ id }), + channel: () => pn.dataSync.removeChannel({ id }), + relationship: () => pn.dataSync.removeRelationship({ id }), + membership: () => pn.dataSync.removeMembership({ id }), + }; + cleanupAfterTest(`remove ${kind} ${id}`, remove[kind]); + return id; +} + /** Await `fn`, swallowing any error. Use only for cleanup — never in the assertion path. */ export async function safeRemove(fn: () => Promise): Promise { try { @@ -221,7 +252,8 @@ export function assertMembershipObject( // -------------- REST fixture factories ------------------ // -------------------------------------------------------- // Each factory creates a valid object and returns `{ obj, dispose }`. `dispose()` is idempotent and -// swallows errors (already-removed objects 404, which is fine for cleanup). +// swallows errors (already-removed objects 404, which is fine for cleanup). The id is also registered with `removeAfterTest`, so a +// test that forgets `dispose()` does not leave the object behind. export type Fixture = { obj: T; dispose: () => Promise }; @@ -229,7 +261,7 @@ export async function createCustomerEntity( pn: PubNub, overrides: Record = {}, ): Promise> { - const id = freshId('customer'); + const id = removeAfterTest(pn, 'entity', freshId('customer')); const res = await createWithRunUniqueId( () => pn.dataSync.createEntity({ @@ -251,7 +283,7 @@ export async function createLoanQuoteEntity( pn: PubNub, overrides: Record = {}, ): Promise> { - const id = freshId('loanquote'); + const id = removeAfterTest(pn, 'entity', freshId('loanquote')); const res = await createWithRunUniqueId( () => pn.dataSync.createEntity({ @@ -275,7 +307,7 @@ export async function createRequestedByRelationship( loanQuoteId: string, overrides: Record = {}, ): Promise> { - const id = freshId('requested-by'); + const id = removeAfterTest(pn, 'relationship', freshId('requested-by')); const res = await createWithRunUniqueId( () => pn.dataSync.createRelationship({ @@ -299,7 +331,7 @@ export async function createUserObj( pn: PubNub, overrides: Record = {}, ): Promise> { - const id = freshId('user'); + const id = removeAfterTest(pn, 'user', freshId('user')); const res = await createWithRunUniqueId( () => pn.dataSync.createUser({ @@ -316,7 +348,7 @@ export async function createChannelObj( pn: PubNub, overrides: Record = {}, ): Promise> { - const id = freshId('JSchannel'); + const id = removeAfterTest(pn, 'channel', freshId('JSchannel')); const res = await createWithRunUniqueId( () => pn.dataSync.createChannel({ @@ -335,7 +367,7 @@ export async function createMembershipObj( channelId: string, overrides: Record = {}, ): Promise> { - const id = freshId('membership'); + const id = removeAfterTest(pn, 'membership', freshId('membership')); const res = await createWithRunUniqueId( () => pn.dataSync.createMembership({ diff --git a/test/integration/live/support/hooks.ts b/test/integration/live/support/hooks.ts index 0b4395168..5e8ececd0 100644 --- a/test/integration/live/support/hooks.ts +++ b/test/integration/live/support/hooks.ts @@ -5,7 +5,8 @@ * keysets too, see `isFullRun`), then the janitor (cleans every resource of resources.ts), then, on a * secured keyset, a wait until a client without a token is refused again, then the preflight (keyset features). * A cleanup failure here FAILS the run with a message that lists what could not be removed: the tests assume a clean slate. - * after each: restores the console if a test left it captured, then destroys every tracked client. + * after each: removes what the test registered with `cleanupAfterTest` (cleanup.ts; the clients are still alive), restores the + * console if a test left it captured, then destroys every tracked client. * after all: the janitor again. Its failures are printed loudly but do NOT fail the run (the test results stay what they are, and * the next run's "before all" cleanup retries and fails the run when the keyset is still dirty). One exception: a failed revoke of * the application-level Access Manager grant leaves a secured keyset open, so it fails the run. @@ -21,6 +22,7 @@ * reported in the README). `LIVE_PREFLIGHT=off` skips the preflight. */ import { destroyClients } from './clients'; +import { runCleanups } from './cleanup'; import { restoreAllConsoles } from './console'; import { describeRoles, env, janitorEnabled, preflightEnabled, redact, ROLES } from './env'; import { explainSdkError } from './explain-error'; @@ -117,10 +119,15 @@ export const mochaHooks = { console.log('preflight: keyset features ok'); } }, - afterEach(this: Mocha.Context): void { + async afterEach(this: Mocha.Context): Promise { if (!explainerInstalled) explainSdkError(this.currentTest?.err); - restoreAllConsoles(); - destroyClients(); + // Before the clients are destroyed: the registered removals use them. Never throws. + try { + await runCleanups(); + } finally { + restoreAllConsoles(); + destroyClients(); + } }, async afterAll(this: Mocha.Context): Promise { if (!janitorEnabled()) return; diff --git a/test/integration/live/support/retry.ts b/test/integration/live/support/retry.ts index 084928e47..51986a86e 100644 --- a/test/integration/live/support/retry.ts +++ b/test/integration/live/support/retry.ts @@ -101,13 +101,20 @@ export function isAlreadyExists(error: unknown): boolean { } /** - * Creates a DataSync object whose id is unique to this run (`freshId`, `projectionId`) and tolerates a transient service error. + * Creates a DataSync object whose id is unique to this run (`freshId`, `projectionId`: time in base 36 plus 6 random base-36 + * characters, see `uniqueSuffix` in util.ts) and tolerates a transient service error. * * A create is not idempotent, so a plain retry is unsafe: when the first attempt was stored but its answer was lost (a 5xx after * the write, a timeout), the retry meets the object it just created and gets 409 (DS-0301). With an id nobody else can hold, * that 409 can only mean "the earlier attempt was stored", so it counts as success, and the object is read back (`readBack`, a get * by id) so the caller still gets the eTag and timestamps the create would have returned. * + * The rule is only as good as the id. With the old `prefix-NNNNN` ids (90,000 values) every object left on a keyset made a first + * attempt fail with 409 (DS-0301) with probability 1 in 90,000 per leftover of that prefix, and CI hit it. An id with a time part + * and 36^6 random values can meet an old object only by drawing the same random part in the same millisecond. What remains is a + * collision between a retry and another run in the same millisecond with the same draw: the 409 path would then read back that + * other run's object, and the test's own assertions on the id and payload are what would catch it. + * * - failure of the FIRST attempt that is not transient (including a 409, which means the id was not unique): thrown at once. * - transient failure (`isTransient`): retried like `retryTransient`, up to `attempts` tries (default 3). * - 409 / DS-0301 on a RETRY: resolved through `readBack` (itself retried on transient errors). Never on the first attempt. diff --git a/test/integration/live/support/util.ts b/test/integration/live/support/util.ts index 859458c2e..a4eaf36ea 100644 --- a/test/integration/live/support/util.ts +++ b/test/integration/live/support/util.ts @@ -1,4 +1,4 @@ -import { randomUUID } from 'node:crypto'; +import { randomInt, randomUUID } from 'node:crypto'; /** * A name that no other run uses, for channels and users a test asserts on exactly. Names built this way cannot be listed in @@ -8,3 +8,27 @@ export const unique = (base: string): string => `${base}-${randomUUID().slice(0, /** A sorted copy, for comparing lists whose order the service does not promise. */ export const sorted = (values: string[]): string[] => [...values].sort(); + +/** + * A suffix for the id of an object a test creates on a keyset that other runs have used for years (DataSync objects, which a + * test run can leave behind). 14 characters of `[0-9a-z]`: the time in milliseconds in base 36 (8 characters until the year 2059), + * then 6 random base-36 characters (36^6, about 2.2 billion values), so two ids collide only when two runs draw the same random + * part in the same millisecond (within one process they never do). No `_` (the service rejects it in ids), no upper case, no separator of its own. Ids built from it + * stay at or under 28 characters with the prefixes of the endpoint tests (the longest id the docs samples create is 28). + */ +export function uniqueSuffix(): string { + const now = Date.now(); + if (now !== drawnAt) { + drawnAt = now; + drawn.clear(); + } + // Two ids of this process never share a millisecond and a draw, however fast a loop asks. + let random: number; + do random = randomInt(36 ** 6); + while (drawn.has(random)); + drawn.add(random); + return `${now.toString(36)}${random.toString(36).padStart(6, '0')}`; +} + +let drawnAt = 0; +const drawn = new Set(); diff --git a/test/integration/live/support/wait.ts b/test/integration/live/support/wait.ts index 79fc80087..27df8b9ed 100644 --- a/test/integration/live/support/wait.ts +++ b/test/integration/live/support/wait.ts @@ -10,7 +10,10 @@ * - `watch` subscribe once and record everything, so the assertion can come after the publish without racing. * - `expectStatus` resolve on a status category such as `PNConnectedCategory`. * - * Every helper rejects with a message that says what was awaited and what was seen. + * Every helper rejects with a message that says what was awaited and what was seen. The event waits (`expectEvent`, `connected`, + * `expectStatus`, `Watch.next`) also say what the CLIENT saw while they waited: the statuses (category, operation, seconds since the wait + * began), the count and kinds of events with their channels, and the channels it was subscribed to when the wait gave up (`observe` + * below). No payloads, keys or tokens. It costs one extra listener that only records, so a wait that succeeds behaves as before. */ import type PubNub from 'pubnub'; @@ -115,6 +118,85 @@ export async function settled(fn: () => Promise | T, options: SettledOptio throw error; } +/** What `observe` needs from a client to report its subscriptions. A subscription or a set has neither. */ +interface SubscriptionInfo { + getSubscribedChannels?: () => string[]; + getSubscribedChannelGroups?: () => string[]; +} + +const OBSERVED_EVENTS = ['message', 'signal', 'presence', 'messageAction', 'file', 'objects', 'dataSync'] as const; +const MAX_STATUSES = 12; +const MAX_CHANNELS = 6; + +const joinList = (items: Iterable): string => [...items].join(', ') || 'none'; + +/** + * Records what a client delivers to its listeners from now on, to explain a timeout: statuses (category and operation, plus the + * cursor of a connected status, which is a timetoken and not a secret), and for every event only its kind and channel. It never keeps + * a payload. Attach `listener` to the same target the wait listens on (the client, for statuses) and call `stop()` when the wait is over. + */ +export class Observation { + readonly listener: PubNub.Listener = {}; + private readonly startedAt = Date.now(); + private readonly statuses: string[] = []; + private statusCount = 0; + private readonly events = new Map }>(); + /** How many events of the awaited kind arrived and were refused by the predicate. */ + rejected = 0; + + constructor(private readonly info: SubscriptionInfo | undefined) { + this.listener.status = (status) => { + const event = status as PubNub.StatusEvent & { operation?: string; error?: unknown }; + this.statusCount += 1; + const operation = event.operation ? `/${event.operation}` : ''; + const cursor = event.currentTimetoken !== undefined ? ` tt=${event.currentTimetoken}` : ''; + const failed = event.error ? ' (error)' : ''; + const entry = `${String(event.category)}${operation}${cursor}${failed} at +${Date.now() - this.startedAt}ms`; + this.statuses.push(entry); + if (this.statuses.length > MAX_STATUSES) this.statuses.shift(); + }; + for (const kind of OBSERVED_EVENTS) { + (this.listener as Record)[kind] = (event: { channel?: string }) => + this.note(kind, event?.channel); + } + } + + /** Counts one event of `kind` on `channel`. Nothing else about the event is kept. */ + note(kind: string, channel: unknown): void { + const entry = this.events.get(kind) ?? { count: 0, channels: new Set() }; + entry.count += 1; + if (typeof channel === 'string' && entry.channels.size < MAX_CHANNELS) entry.channels.add(channel); + this.events.set(kind, entry); + } + + /** One paragraph for an error message. */ + summary(): string { + const statuses = this.statusCount + ? `${this.statusCount} status(es)${this.statusCount > this.statuses.length ? `, last ${this.statuses.length}` : ''}: ${this.statuses.join('; ')}` + : `no status${this.info?.getSubscribedChannels ? '' : ' (the wait listens on something that is not a client, which reports none)'}`; + const total = [...this.events.values()].reduce((sum, entry) => sum + entry.count, 0); + const kinds = [...this.events].map(([kind, e]) => `${kind} x${e.count} on [${joinList(e.channels)}]`); + const events = total ? `${total} event(s): ${kinds.join('; ')}` : '0 events'; + const rejected = this.rejected ? `, ${this.rejected} of the awaited kind failed the predicate` : ''; + let subscribed = 'subscribed channels unknown'; + try { + if (this.info?.getSubscribedChannels) { + subscribed = + `subscribed channels [${joinList(this.info.getSubscribedChannels())}] ` + + `groups [${joinList(this.info.getSubscribedChannelGroups?.() ?? [])}]`; + } + } catch { + /* a destroyed client cannot say */ + } + return `In that window the client saw ${statuses}. ${events}${rejected}. At timeout: ${subscribed}.`; + } + + /** Removes the listener from `target` after the current dispatch (see `detach`). */ + stop(target: { removeListener(listener: PubNub.Listener): void }): void { + detach(target, this.listener); + } +} + /** * Removes a listener after the current dispatch. Removing it inside its own callback mutates the SDK's listener array while * it iterates, which makes the SDK skip the next listener. @@ -136,6 +218,19 @@ type EventOf = Parameters>[ export interface ExpectOptions { timeout?: number; + /** Adds the caller's own facts to the timeout error (timetokens the test knows, for example). Keys, tokens and payloads stay out. */ + describe?: () => string; +} + +/** Text a timeout error appends: what the client saw (`Observation.summary`) and what the caller says about the wait. */ +function timeoutDetail(seen: Observation, options: ExpectOptions): string { + let extra = ''; + try { + extra = options.describe ? ` ${options.describe()}` : ''; + } catch { + /* the description is optional */ + } + return ` ${seen.summary()}${extra}`; } /** Registers a listener now and resolves on the first event of `kind` that satisfies `predicate`. Call it before the action. */ @@ -148,9 +243,11 @@ export function expectEvent( const timeout = options.timeout ?? 15_000; return new Promise>((resolve, reject) => { const listener: PubNub.Listener = {}; + const seen = new Observation(target as SubscriptionInfo); const timer = setTimeout(() => { target.removeListener(listener); - reject(new Error(`no "${kind}" event matched within ${timeout}ms`)); + target.removeListener(seen.listener); + reject(new Error(`no "${kind}" event matched within ${timeout}ms.${timeoutDetail(seen, options)}`)); }, timeout); // Mocha's own test timer keeps the process alive meanwhile. unref() only means an abandoned wait cannot hold it open. timer.unref(); @@ -162,11 +259,16 @@ export function expectEvent( } catch { matched = false; } - if (!matched) return; + if (!matched) { + seen.rejected += 1; + return; + } clearTimeout(timer); detach(target, listener); + seen.stop(target); resolve(event); }; + target.addListener(seen.listener); target.addListener(listener); }); } @@ -187,6 +289,7 @@ export function connected( const needChannels = new Set(targets.channels ?? []); const needGroups = new Set(targets.channelGroups ?? []); return new Promise((resolve, reject) => { + const seen = new Observation(client as SubscriptionInfo); const seenChannels = new Set(); const seenGroups = new Set(); const listener: PubNub.Listener = { @@ -202,20 +305,23 @@ export function connected( if (done) { clearTimeout(timer); detach(client, listener); + seen.stop(client); resolve(); } }, }; const timer = setTimeout(() => { client.removeListener(listener); + client.removeListener(seen.listener); reject( new Error( `not connected within ${timeout}ms. Wanted channels [${[...needChannels]}] groups [${[...needGroups]}], ` + - `saw channels [${[...seenChannels]}] groups [${[...seenGroups]}]`, + `saw channels [${[...seenChannels]}] groups [${[...seenGroups]}].${timeoutDetail(seen, options)}`, ), ); }, timeout); timer.unref(); + client.addListener(seen.listener); client.addListener(listener); }); } @@ -231,10 +337,21 @@ export class Watch { readonly events: RecordedEvent[] = []; // Callers blocked in `next()`. Each is woken as soon as an event arrives. private waiters: Array<() => void> = []; + // Statuses are per client, so the client (when given) is what explains a `next()` that times out. + private readonly seen: Observation; + private readonly statusListener: PubNub.Listener; - constructor(private readonly set: ReturnType) { + constructor( + private readonly set: ReturnType, + private readonly client?: PubNub, + ) { + this.seen = new Observation(client as SubscriptionInfo | undefined); + // Only the statuses of the client: this class records the events itself. + this.statusListener = { status: this.seen.listener.status }; + client?.addListener(this.statusListener); const record = (kind: EventKind) => (event: unknown) => { this.events.push({ kind, event }); + this.seen.note(kind, (event as { channel?: string } | undefined)?.channel); this.waiters.splice(0).forEach((wake) => wake()); }; set.addListener({ @@ -269,7 +386,8 @@ export class Watch { const remaining = deadline - Date.now(); if (remaining <= 0) { throw new Error( - `no "${kind}" event matched within ${timeout}ms. Recorded: ${this.events.map((e) => e.kind).join(',') || 'none'}`, + `no "${kind}" event matched within ${timeout}ms. Recorded: ${this.events.map((e) => e.kind).join(',') || 'none'}. ` + + `${this.seen.summary()}${options.describe ? ` ${options.describe()}` : ''}`, ); } // Sleep until the next event wakes us. The 250 ms cap is a safety net, not a poll interval. @@ -285,6 +403,7 @@ export class Watch { /** Unsubscribes and drops the listeners. */ stop(): void { + this.client?.removeListener(this.statusListener); this.set.unsubscribe(); this.set.removeAllListeners(); } @@ -309,7 +428,7 @@ export async function watch(client: PubNub, channels: string[], options: WatchOp }); // Register the connected listener first, then subscribe: statuses are not replayed. const ready = connected(client, { channels, channelGroups: options.channelGroups }, { timeout: options.timeout }); - const handle = new Watch(set); + const handle = new Watch(set, client); set.subscribe(); await ready; return handle; @@ -323,20 +442,24 @@ export function expectStatus( ): Promise { const timeout = options.timeout ?? 15_000; return new Promise((resolve, reject) => { + const seen = new Observation(client as SubscriptionInfo); const listener: PubNub.Listener = { status: (status) => { const category = String(status.category); if (!categories.includes(category)) return; clearTimeout(timer); detach(client, listener); + seen.stop(client); resolve(category); }, }; const timer = setTimeout(() => { client.removeListener(listener); - reject(new Error(`no status in [${categories.join(', ')}] within ${timeout}ms`)); + client.removeListener(seen.listener); + reject(new Error(`no status in [${categories.join(', ')}] within ${timeout}ms.${timeoutDetail(seen, options)}`)); }, timeout); timer.unref(); + client.addListener(seen.listener); client.addListener(listener); }); } From c05d3483bff8df36d57f302287f9b105f192633a Mon Sep 17 00:00:00 2001 From: Mateusz Wiktor Date: Tue, 17 Nov 2026 12:27:21 +0100 Subject: [PATCH 06/13] test: enhance DataSync tests with improved state verification and diagnostics - Added waiting mechanisms to ensure that DataSync reads return expected results after create and delete operations, improving test reliability. - Introduced detailed diagnostics for capturing statuses and events during DataSync operations, aiding in troubleshooting. - Updated multiple test files to utilize new waiting and diagnostic features, ensuring consistent state verification across tests. - Enhanced comments for clarity on the purpose of waiting for state stabilization in tests. --- .../live/api-reference/datasync.test.ts | 61 ++++++++++++++----- .../live/api-reference/objects.test.ts | 29 +++++++++ .../endpoints/data_sync/projections.test.ts | 26 +++++++- 3 files changed, 100 insertions(+), 16 deletions(-) diff --git a/test/integration/live/api-reference/datasync.test.ts b/test/integration/live/api-reference/datasync.test.ts index 0bcd4283c..1faa2743c 100644 --- a/test/integration/live/api-reference/datasync.test.ts +++ b/test/integration/live/api-reference/datasync.test.ts @@ -1226,6 +1226,12 @@ describe('api-reference/datasync', () => { it('getRelationshipBasicUsage: getRelationship reads the link by its fixed id', async () => { const pubnub = server('datasync'); + // The sample reads once, right after the create above: wait until consecutive reads find the link (a replica that has not + // seen the write answers 404 DS-0100). + await settled(() => pubnub.dataSync.getRelationship({ id: REL }), { + timeout: 45_000, + label: 'the link is readable', + }); // snippet.getRelationshipBasicUsage const response = await pubnub.dataSync.getRelationship({ id: 'rel-bob-owns-sneaker-42', @@ -1390,9 +1396,14 @@ describe('api-reference/datasync', () => { console.log('removeRelationship response status:', response.status); // snippet.end assert.ok(response.status >= 200 && response.status < 300); - await assert.rejects( - () => pubnub.dataSync.getRelationship({ id: REL }), - (error: unknown) => /404/.test(errorText(error)), + // A replica that has not seen the delete can still answer the read, so wait for the 404. + await eventually( + () => + assert.rejects( + () => pubnub.dataSync.getRelationship({ id: REL }), + (error: unknown) => /404/.test(errorText(error)), + ), + { timeout: 30_000, label: 'REL reads 404 after removal' }, ); }); @@ -1405,9 +1416,14 @@ describe('api-reference/datasync', () => { console.log('removeMembership response status:', response.status); // snippet.end assert.ok(response.status >= 200 && response.status < 300); - await assert.rejects( - () => pubnub.dataSync.getMembership({ id: MEMBERSHIP }), - (error: unknown) => /404/.test(errorText(error)), + // A replica that has not seen the delete can still answer the read, so wait for the 404. + await eventually( + () => + assert.rejects( + () => pubnub.dataSync.getMembership({ id: MEMBERSHIP }), + (error: unknown) => /404/.test(errorText(error)), + ), + { timeout: 30_000, label: 'MEMBERSHIP reads 404 after removal' }, ); }); @@ -1418,9 +1434,14 @@ describe('api-reference/datasync', () => { console.log('removeEntity response status:', response.status); // snippet.end assert.ok(response.status >= 200 && response.status < 300); - await assert.rejects( - () => pubnub.dataSync.getEntity({ id: PRODUCT }), - (error: unknown) => /404/.test(errorText(error)), + // A replica that has not seen the delete can still answer the read, so wait for the 404. + await eventually( + () => + assert.rejects( + () => pubnub.dataSync.getEntity({ id: PRODUCT }), + (error: unknown) => /404/.test(errorText(error)), + ), + { timeout: 30_000, label: 'PRODUCT reads 404 after removal' }, ); }); @@ -1431,9 +1452,14 @@ describe('api-reference/datasync', () => { console.log('removeChannel response status:', response.status); // snippet.end assert.ok(response.status >= 200 && response.status < 300); - await assert.rejects( - () => pubnub.dataSync.getChannel({ id: SUMMER }), - (error: unknown) => /404/.test(errorText(error)), + // A replica that has not seen the delete can still answer the read, so wait for the 404. + await eventually( + () => + assert.rejects( + () => pubnub.dataSync.getChannel({ id: SUMMER }), + (error: unknown) => /404/.test(errorText(error)), + ), + { timeout: 30_000, label: 'SUMMER reads 404 after removal' }, ); }); @@ -1444,9 +1470,14 @@ describe('api-reference/datasync', () => { console.log('removeUser response status:', response.status); // snippet.end assert.ok(response.status >= 200 && response.status < 300); - await assert.rejects( - () => pubnub.dataSync.getUser({ id: ALICE }), - (error: unknown) => /404/.test(errorText(error)), + // A replica that has not seen the delete can still answer the read, so wait for the 404. + await eventually( + () => + assert.rejects( + () => pubnub.dataSync.getUser({ id: ALICE }), + (error: unknown) => /404/.test(errorText(error)), + ), + { timeout: 30_000, label: 'ALICE reads 404 after removal' }, ); }); }); diff --git a/test/integration/live/api-reference/objects.test.ts b/test/integration/live/api-reference/objects.test.ts index 537d70899..674e1fdbc 100644 --- a/test/integration/live/api-reference/objects.test.ts +++ b/test/integration/live/api-reference/objects.test.ts @@ -345,6 +345,35 @@ describe('api-reference/objects', () => { it('iterativelyUpdateExistingMetadata: the read-modify-write pattern merges custom data and the merged object survives on the service', async () => { const pubnub = shopper(); + // The sample reads the channel back once, right after its own first write, and merges into what it reads. An earlier test + // left other custom data on team.red, so a replica that is behind would hand the sample that old data and the merge would + // drop `visible`. Store exactly what the sample's first write stores, and wait until it is stable, so that old and new + // state read the same. + await retryTransient( + () => + pubnub.objects.setChannelMetadata({ + channel: 'team.red', + data: { + name: 'Red Team', + description: 'The channel for Red team.', + custom: { visible: 'team' }, + }, + }), + { label: 'seed team.red' }, + ); + await settled( + async () => { + const stored = await observer().objects.getChannelMetadata({ + channel: 'team.red', + include: { customFields: true }, + }); + assert.deepEqual(stored.data.custom, { visible: 'team' }); + assert.equal(stored.data.name, 'Red Team'); + assert.equal(stored.data.description, 'The channel for Red team.'); + }, + { timeout: 45_000, label: 'team.red holds the state the sample writes first' }, + ); + // snippet.iterativelyUpdateExistingMetadata const channel = 'team.red'; const name = 'Red Team'; diff --git a/test/integration/live/endpoints/data_sync/projections.test.ts b/test/integration/live/endpoints/data_sync/projections.test.ts index c5584f400..e0c9df34f 100644 --- a/test/integration/live/endpoints/data_sync/projections.test.ts +++ b/test/integration/live/endpoints/data_sync/projections.test.ts @@ -256,6 +256,28 @@ function captureEvents( const collected: PubNub.Subscription.DataSyncObject[] = []; let settled = false; + // Diagnostics only: what the client saw while it waited, reported when the wait gives up. Categories, operations, + // channels and timetokens; no payloads, keys or tokens. + const startedAt = Date.now(); + const seenStatuses: string[] = []; + const seenEvents: string[] = []; + const recorder = { + status: (status: PubNub.Status | PubNub.StatusEvent) => { + const tt = (status as { currentTimetoken?: string }).currentTimetoken; + const operation = (status as { operation?: string }).operation; + seenStatuses.push( + `${status.category}${operation ? `/${operation}` : ''}${tt ? ` tt=${tt}` : ''} +${((Date.now() - startedAt) / 1000).toFixed(1)}s`, + ); + }, + dataSync: (event: PubNub.Subscription.DataSyncObject) => { + seenEvents.push(`${(event.message as { event?: string }).event ?? '?'} on ${event.channel}`); + }, + }; + pn.addListener(recorder); + const diagnostics = (): string => + ` Statuses seen: [${seenStatuses.join('; ') || 'none'}]. Events seen (any, before the predicate): ` + + `[${seenEvents.join('; ') || 'none'}].`; + const listener = { dataSync: (event: PubNub.Subscription.DataSyncObject) => { if (settled || !predicate(event)) return; @@ -301,13 +323,15 @@ function captureEvents( new Error( `Timed out waiting for DataSync event(s)` + `${opts.completeWhen ? '' : ` (expected ${expectedCount})`}; ` + - `got ${collected.length} on [${collected.map((e) => e.channel).join(', ')}].`, + `got ${collected.length} on [${collected.map((e) => e.channel).join(', ')}].` + + diagnostics(), ), ); }, timeoutMs); function cleanup(): void { clearTimeout(timer); + pn.removeListener(recorder); subscription.removeListener(listener); subscription.unsubscribe(); } From 97c2e1a3a1a245faf7a250f644763fe66fc8c0a6 Mon Sep 17 00:00:00 2001 From: Mateusz Wiktor Date: Tue, 17 Nov 2026 13:03:20 +0100 Subject: [PATCH 07/13] refactor: remove docs-snippets and update related configurations - Deleted the docs-snippets directory and its associated files to streamline the project structure. - Updated .codacy.yml to exclude paths related to the removed docs-snippets. - Adjusted .gitignore and .npmignore to reflect the removal of docs-snippets. - Enhanced package.json scripts for integration tests by refining linting commands and removing references to docs-snippets. - Updated live integration workflow to ensure compatibility with the new structure. --- .codacy.yml | 2 +- .github/workflows/docs-snippets.yml | 223 -- .github/workflows/live-integration.yml | 8 +- .github/workflows/release/versions.json | 4 +- .gitignore | 5 - .npmignore | 1 - docs-snippets/.env.example | 25 - docs-snippets/.mocharc.cjs | 15 - docs-snippets/README.md | 161 -- docs-snippets/access-manager.ts | 283 --- .../api-reference/access-manager-legacy.ts | 241 --- .../api-reference/access-manager-v2.ts | 241 --- docs-snippets/api-reference/access-manager.ts | 412 ---- docs-snippets/api-reference/channel-groups.ts | 96 - .../api-reference/configuration-legacy.ts | 139 -- docs-snippets/api-reference/configuration.ts | 212 -- docs-snippets/api-reference/datasync.ts | 1306 ----------- .../api-reference/encryption-legacy.ts | 57 - docs-snippets/api-reference/encryption.ts | 101 - .../api-reference/file-sharing-legacy.ts | 56 - docs-snippets/api-reference/file-sharing.ts | 219 -- .../api-reference/message-actions.ts | 92 - .../message-persistence-legacy.ts | 384 ---- .../api-reference/message-persistence.ts | 182 -- docs-snippets/api-reference/misc.ts | 85 - docs-snippets/api-reference/mobile-push.ts | 336 --- docs-snippets/api-reference/objects.ts | 431 ---- docs-snippets/api-reference/presence.ts | 208 -- docs-snippets/api-reference/publish-legacy.ts | 50 - docs-snippets/api-reference/publish.ts | 238 --- .../api-reference/subscribe-legacy.ts | 552 ----- docs-snippets/api-reference/subscribe.ts | 423 ---- docs-snippets/app-context.ts | 59 - docs-snippets/basic-usage/access-manager.ts | 52 - docs-snippets/basic-usage/app-context.ts | 287 --- docs-snippets/basic-usage/channel-groups.ts | 62 - docs-snippets/basic-usage/configuration.ts | 43 - docs-snippets/basic-usage/data-sync.ts | 508 ----- .../basic-usage/download-file-web.ts | 35 - docs-snippets/basic-usage/event-listener.ts | 52 - docs-snippets/basic-usage/file-sharing.ts | 106 - docs-snippets/basic-usage/message-actions.ts | 66 - .../basic-usage/message-persistence.ts | 48 - docs-snippets/basic-usage/miscellaneous.ts | 69 - docs-snippets/basic-usage/mobile-push.ts | 146 -- docs-snippets/basic-usage/presence.ts | 75 - .../basic-usage/publish-subscribe.ts | 142 -- docs-snippets/configuration.ts | 54 - docs-snippets/coverage.json | 1901 ----------------- docs-snippets/data-sync.ts | 440 ---- docs-snippets/event-listener.ts | 95 - docs-snippets/file-sharing.ts | 39 - docs-snippets/fixtures/datasync-fragments.ts | 48 - docs-snippets/fixtures/files/cat_picture.jpg | Bin 134 -> 0 bytes docs-snippets/fixtures/names.ts | 182 -- docs-snippets/fixtures/workdir.ts | 32 - docs-snippets/getting-started-example.ts | 102 - docs-snippets/getting-started.ts | 93 - .../getting-started/logging-file-logger.ts | 92 - docs-snippets/getting-started/logging.ts | 375 ---- .../getting-started/quickstart-node.mjs | 44 - .../getting-started/quickstart-node.ts | 132 -- .../set-up-your-environment.ts | 164 -- .../getting-started/status-events.ts | 344 --- .../getting-started/troubleshooting.ts | 67 - docs-snippets/harness/assert.ts | 116 - docs-snippets/harness/assertions.ts | 33 - docs-snippets/harness/cli.ts | 840 -------- docs-snippets/harness/env.ts | 294 --- docs-snippets/harness/guard.ts | 56 - docs-snippets/harness/janitor-datasync.ts | 134 -- docs-snippets/harness/janitor.ts | 255 --- docs-snippets/harness/keyset-features.ts | 103 - docs-snippets/harness/lint-markers.ts | 331 --- docs-snippets/harness/manifest.ts | 145 -- docs-snippets/harness/markers.ts | 117 - docs-snippets/harness/mocha-entry.ts | 351 --- docs-snippets/harness/policy.ts | 63 - docs-snippets/harness/preflight-errors.ts | 75 - docs-snippets/harness/preflight-summary.ts | 190 -- docs-snippets/harness/preflight.ts | 64 - docs-snippets/harness/proxy.ts | 75 - docs-snippets/harness/quiet.ts | 64 - docs-snippets/harness/recorder.ts | 123 -- docs-snippets/harness/redact.cjs | 162 -- docs-snippets/harness/redact.d.cts | 5 - docs-snippets/harness/registry.ts | 284 --- docs-snippets/harness/reporter.cjs | 220 -- docs-snippets/harness/revoke.ts | 152 -- docs-snippets/harness/signals.ts | 65 - docs-snippets/harness/snippet-summary.cjs | 144 -- docs-snippets/harness/stage.ts | 385 ---- docs-snippets/harness/step.ts | 93 - docs-snippets/harness/tsc-project.ts | 48 - docs-snippets/harness/wait.ts | 397 ---- docs-snippets/harness/world.ts | 403 ---- docs-snippets/import-pubnub.ts | 16 - docs-snippets/message-persistence.ts | 51 - docs-snippets/migration-guides/v11.ts | 88 - docs-snippets/migration-guides/v12.ts | 58 - docs-snippets/migration-guides/v8.ts | 40 - docs-snippets/mobile-push.ts | 39 - docs-snippets/package.json | 4 - docs-snippets/presence.ts | 42 - docs-snippets/provision/datasync-classes.json | 63 - docs-snippets/provision/provision-datasync.ts | 363 ---- docs-snippets/publish-subscribe.ts | 263 --- docs-snippets/run.ts | 107 - docs-snippets/shared/import-pubnub.ts | 32 - docs-snippets/tsconfig.json | 43 - docs-snippets/tsconfig.live.json | 33 - docs-snippets/tsconfig.run.json | 9 - docs-snippets/use-cases/automated-polling.ts | 102 - .../use-cases/fan-behavior-management.ts | 264 --- docs-snippets/use-cases/fan-re-engagement.ts | 121 -- .../use-cases/game-chat-moderation.ts | 139 -- docs-snippets/use-cases/live-commentary.ts | 70 - .../use-cases/live-event-rate-limiting.ts | 67 - docs-snippets/use-cases/live-polls.ts | 106 - docs-snippets/use-cases/match-stats.ts | 58 - docs-snippets/use-cases/real-time-ads.ts | 65 - docs-snippets/use-cases/real-time-chat.ts | 131 -- docs-snippets/use-cases/score-alerts.ts | 145 -- package.json | 12 +- test/integration/live/README.md | 22 +- .../access-manager-legacy.test.ts | 2 +- .../live/api-reference/access-manager.test.ts | 1 - .../live/api-reference/channel-groups.test.ts | 2 +- .../configuration-legacy.test.ts | 2 +- .../live/api-reference/configuration.test.ts | 2 +- .../live/api-reference/datasync.test.ts | 4 +- .../api-reference/encryption-legacy.test.ts | 2 +- .../live/api-reference/encryption.test.ts | 2 +- .../api-reference/file-sharing-legacy.test.ts | 5 +- .../live/api-reference/file-sharing.test.ts | 2 +- .../api-reference/message-actions.test.ts | 2 +- .../message-persistence-legacy.test.ts | 2 +- .../api-reference/message-persistence.test.ts | 2 +- .../live/api-reference/misc.test.ts | 2 +- .../live/api-reference/mobile-push.test.ts | 2 +- .../live/api-reference/objects.test.ts | 2 +- .../live/api-reference/presence.test.ts | 2 +- .../live/api-reference/publish-legacy.test.ts | 2 +- .../live/api-reference/publish.test.ts | 2 +- .../api-reference/subscribe-legacy.test.ts | 2 +- .../live/api-reference/subscribe.test.ts | 2 +- .../data_sync/events.channel.test.ts | 1 - .../endpoints/data_sync/events.entity.test.ts | 1 - .../data_sync/events.relationship.test.ts | 1 - .../endpoints/data_sync/projections.test.ts | 1 - .../logging-file-logger.test.ts | 2 +- .../live/getting-started/logging.test.ts | 2 +- .../getting-started/quickstart-node.test.ts | 2 +- .../set-up-your-environment.test.ts | 2 +- .../getting-started/status-events.test.ts | 2 +- .../getting-started/troubleshooting.test.ts | 2 +- .../live/migration-guides/v11.test.ts | 2 +- .../live/migration-guides/v12.test.ts | 2 +- .../live/migration-guides/v8.test.ts | 2 +- .../live/provision/provision-datasync.ts | 2 +- .../quickstarts/browser-download-file.ts | 0 .../quickstarts/browser-shared-worker.ts | 0 .../quickstarts/quickstart-react-app.tsx | 0 .../quickstart-react-native-app.tsx | 0 .../quickstarts/quickstart-react-native.tsx | 0 .../live}/quickstarts/quickstart-react.tsx | 0 .../live}/quickstarts/quickstart-web.html | 0 .../live}/quickstarts/quickstart-web.ts | 0 .../live}/quickstarts/react-configuration.jsx | 0 .../quickstarts/react-fetch-messages.jsx | 0 .../live}/quickstarts/react-get-state.jsx | 0 .../live}/quickstarts/react-here-now.jsx | 0 .../live}/quickstarts/react-listeners.jsx | 0 .../quickstarts/react-native-download-file.ts | 0 .../quickstarts/react-presence-events.jsx | 0 .../live}/quickstarts/react-set-state.jsx | 0 .../live}/quickstarts/react-subscribe.jsx | 0 .../live}/quickstarts/react-where-now.jsx | 0 .../live/shared/import-pubnub.test.ts | 2 +- test/integration/live/snippet-exceptions.json | 74 +- .../live/support/compare-snippets.ts | 51 - test/integration/live/support/docs-check.ts | 302 +-- test/integration/live/support/manifest.ts | 6 +- test/integration/live/support/preflight.ts | 4 +- test/integration/live/support/resources.ts | 4 +- test/integration/live/support/snippet-scan.ts | 8 +- .../integration/live}/tsconfig.browser.json | 4 +- test/integration/live/tsconfig.json | 3 +- .../live/use-cases/automated-polling.test.ts | 2 +- .../use-cases/fan-behavior-management.test.ts | 2 +- .../live/use-cases/fan-re-engagement.test.ts | 2 +- .../use-cases/game-chat-moderation.test.ts | 2 +- .../live/use-cases/live-commentary.test.ts | 2 +- .../live-event-rate-limiting.test.ts | 2 +- .../live/use-cases/live-polls.test.ts | 2 +- .../live/use-cases/match-stats.test.ts | 2 +- .../live/use-cases/real-time-ads.test.ts | 2 +- .../live/use-cases/real-time-chat.test.ts | 2 +- .../live/use-cases/score-alerts.test.ts | 2 +- 199 files changed, 137 insertions(+), 21495 deletions(-) delete mode 100644 .github/workflows/docs-snippets.yml delete mode 100644 docs-snippets/.env.example delete mode 100644 docs-snippets/.mocharc.cjs delete mode 100644 docs-snippets/README.md delete mode 100644 docs-snippets/access-manager.ts delete mode 100644 docs-snippets/api-reference/access-manager-legacy.ts delete mode 100644 docs-snippets/api-reference/access-manager-v2.ts delete mode 100644 docs-snippets/api-reference/access-manager.ts delete mode 100644 docs-snippets/api-reference/channel-groups.ts delete mode 100644 docs-snippets/api-reference/configuration-legacy.ts delete mode 100644 docs-snippets/api-reference/configuration.ts delete mode 100644 docs-snippets/api-reference/datasync.ts delete mode 100644 docs-snippets/api-reference/encryption-legacy.ts delete mode 100644 docs-snippets/api-reference/encryption.ts delete mode 100644 docs-snippets/api-reference/file-sharing-legacy.ts delete mode 100644 docs-snippets/api-reference/file-sharing.ts delete mode 100644 docs-snippets/api-reference/message-actions.ts delete mode 100644 docs-snippets/api-reference/message-persistence-legacy.ts delete mode 100644 docs-snippets/api-reference/message-persistence.ts delete mode 100644 docs-snippets/api-reference/misc.ts delete mode 100644 docs-snippets/api-reference/mobile-push.ts delete mode 100644 docs-snippets/api-reference/objects.ts delete mode 100644 docs-snippets/api-reference/presence.ts delete mode 100644 docs-snippets/api-reference/publish-legacy.ts delete mode 100644 docs-snippets/api-reference/publish.ts delete mode 100644 docs-snippets/api-reference/subscribe-legacy.ts delete mode 100644 docs-snippets/api-reference/subscribe.ts delete mode 100644 docs-snippets/app-context.ts delete mode 100644 docs-snippets/basic-usage/access-manager.ts delete mode 100644 docs-snippets/basic-usage/app-context.ts delete mode 100644 docs-snippets/basic-usage/channel-groups.ts delete mode 100644 docs-snippets/basic-usage/configuration.ts delete mode 100644 docs-snippets/basic-usage/data-sync.ts delete mode 100644 docs-snippets/basic-usage/download-file-web.ts delete mode 100644 docs-snippets/basic-usage/event-listener.ts delete mode 100644 docs-snippets/basic-usage/file-sharing.ts delete mode 100644 docs-snippets/basic-usage/message-actions.ts delete mode 100644 docs-snippets/basic-usage/message-persistence.ts delete mode 100644 docs-snippets/basic-usage/miscellaneous.ts delete mode 100644 docs-snippets/basic-usage/mobile-push.ts delete mode 100644 docs-snippets/basic-usage/presence.ts delete mode 100644 docs-snippets/basic-usage/publish-subscribe.ts delete mode 100644 docs-snippets/configuration.ts delete mode 100644 docs-snippets/coverage.json delete mode 100644 docs-snippets/data-sync.ts delete mode 100644 docs-snippets/event-listener.ts delete mode 100644 docs-snippets/file-sharing.ts delete mode 100644 docs-snippets/fixtures/datasync-fragments.ts delete mode 100644 docs-snippets/fixtures/files/cat_picture.jpg delete mode 100644 docs-snippets/fixtures/names.ts delete mode 100644 docs-snippets/fixtures/workdir.ts delete mode 100644 docs-snippets/getting-started-example.ts delete mode 100644 docs-snippets/getting-started.ts delete mode 100644 docs-snippets/getting-started/logging-file-logger.ts delete mode 100644 docs-snippets/getting-started/logging.ts delete mode 100644 docs-snippets/getting-started/quickstart-node.mjs delete mode 100644 docs-snippets/getting-started/quickstart-node.ts delete mode 100644 docs-snippets/getting-started/set-up-your-environment.ts delete mode 100644 docs-snippets/getting-started/status-events.ts delete mode 100644 docs-snippets/getting-started/troubleshooting.ts delete mode 100644 docs-snippets/harness/assert.ts delete mode 100644 docs-snippets/harness/assertions.ts delete mode 100644 docs-snippets/harness/cli.ts delete mode 100644 docs-snippets/harness/env.ts delete mode 100644 docs-snippets/harness/guard.ts delete mode 100644 docs-snippets/harness/janitor-datasync.ts delete mode 100644 docs-snippets/harness/janitor.ts delete mode 100644 docs-snippets/harness/keyset-features.ts delete mode 100644 docs-snippets/harness/lint-markers.ts delete mode 100644 docs-snippets/harness/manifest.ts delete mode 100644 docs-snippets/harness/markers.ts delete mode 100644 docs-snippets/harness/mocha-entry.ts delete mode 100644 docs-snippets/harness/policy.ts delete mode 100644 docs-snippets/harness/preflight-errors.ts delete mode 100644 docs-snippets/harness/preflight-summary.ts delete mode 100644 docs-snippets/harness/preflight.ts delete mode 100644 docs-snippets/harness/proxy.ts delete mode 100644 docs-snippets/harness/quiet.ts delete mode 100644 docs-snippets/harness/recorder.ts delete mode 100644 docs-snippets/harness/redact.cjs delete mode 100644 docs-snippets/harness/redact.d.cts delete mode 100644 docs-snippets/harness/registry.ts delete mode 100644 docs-snippets/harness/reporter.cjs delete mode 100644 docs-snippets/harness/revoke.ts delete mode 100644 docs-snippets/harness/signals.ts delete mode 100644 docs-snippets/harness/snippet-summary.cjs delete mode 100644 docs-snippets/harness/stage.ts delete mode 100644 docs-snippets/harness/step.ts delete mode 100644 docs-snippets/harness/tsc-project.ts delete mode 100644 docs-snippets/harness/wait.ts delete mode 100644 docs-snippets/harness/world.ts delete mode 100644 docs-snippets/import-pubnub.ts delete mode 100644 docs-snippets/message-persistence.ts delete mode 100644 docs-snippets/migration-guides/v11.ts delete mode 100644 docs-snippets/migration-guides/v12.ts delete mode 100644 docs-snippets/migration-guides/v8.ts delete mode 100644 docs-snippets/mobile-push.ts delete mode 100644 docs-snippets/package.json delete mode 100644 docs-snippets/presence.ts delete mode 100644 docs-snippets/provision/datasync-classes.json delete mode 100644 docs-snippets/provision/provision-datasync.ts delete mode 100644 docs-snippets/publish-subscribe.ts delete mode 100644 docs-snippets/run.ts delete mode 100644 docs-snippets/shared/import-pubnub.ts delete mode 100644 docs-snippets/tsconfig.json delete mode 100644 docs-snippets/tsconfig.live.json delete mode 100644 docs-snippets/tsconfig.run.json delete mode 100644 docs-snippets/use-cases/automated-polling.ts delete mode 100644 docs-snippets/use-cases/fan-behavior-management.ts delete mode 100644 docs-snippets/use-cases/fan-re-engagement.ts delete mode 100644 docs-snippets/use-cases/game-chat-moderation.ts delete mode 100644 docs-snippets/use-cases/live-commentary.ts delete mode 100644 docs-snippets/use-cases/live-event-rate-limiting.ts delete mode 100644 docs-snippets/use-cases/live-polls.ts delete mode 100644 docs-snippets/use-cases/match-stats.ts delete mode 100644 docs-snippets/use-cases/real-time-ads.ts delete mode 100644 docs-snippets/use-cases/real-time-chat.ts delete mode 100644 docs-snippets/use-cases/score-alerts.ts rename {docs-snippets => test/integration/live}/quickstarts/browser-download-file.ts (100%) rename {docs-snippets => test/integration/live}/quickstarts/browser-shared-worker.ts (100%) rename {docs-snippets => test/integration/live}/quickstarts/quickstart-react-app.tsx (100%) rename {docs-snippets => test/integration/live}/quickstarts/quickstart-react-native-app.tsx (100%) rename {docs-snippets => test/integration/live}/quickstarts/quickstart-react-native.tsx (100%) rename {docs-snippets => test/integration/live}/quickstarts/quickstart-react.tsx (100%) rename {docs-snippets => test/integration/live}/quickstarts/quickstart-web.html (100%) rename {docs-snippets => test/integration/live}/quickstarts/quickstart-web.ts (100%) rename {docs-snippets => test/integration/live}/quickstarts/react-configuration.jsx (100%) rename {docs-snippets => test/integration/live}/quickstarts/react-fetch-messages.jsx (100%) rename {docs-snippets => test/integration/live}/quickstarts/react-get-state.jsx (100%) rename {docs-snippets => test/integration/live}/quickstarts/react-here-now.jsx (100%) rename {docs-snippets => test/integration/live}/quickstarts/react-listeners.jsx (100%) rename {docs-snippets => test/integration/live}/quickstarts/react-native-download-file.ts (100%) rename {docs-snippets => test/integration/live}/quickstarts/react-presence-events.jsx (100%) rename {docs-snippets => test/integration/live}/quickstarts/react-set-state.jsx (100%) rename {docs-snippets => test/integration/live}/quickstarts/react-subscribe.jsx (100%) rename {docs-snippets => test/integration/live}/quickstarts/react-where-now.jsx (100%) delete mode 100644 test/integration/live/support/compare-snippets.ts rename {docs-snippets => test/integration/live}/tsconfig.browser.json (90%) diff --git a/.codacy.yml b/.codacy.yml index 31479edde..9488e747c 100644 --- a/.codacy.yml +++ b/.codacy.yml @@ -1,2 +1,2 @@ exclude_paths: - - "docs-snippets/**" + - "test/integration/live/quickstarts/**" diff --git a/.github/workflows/docs-snippets.yml b/.github/workflows/docs-snippets.yml deleted file mode 100644 index 1268d09ce..000000000 --- a/.github/workflows/docs-snippets.yml +++ /dev/null @@ -1,223 +0,0 @@ -name: Docs snippets - -# Live tests of the JavaScript docs code samples (docs-snippets/README.md). Separate from run-tests.yml on purpose: this -# suite calls the real PubNub service and is never part of npm test, ci:node or ci:web. -# -# No workflow-level paths filter: a required check behind a paths filter stays pending. The `changes` job decides instead, -# and the `gate` job is the one to require in branch protection. - -on: - pull_request: - push: - branches: [master] - schedule: - - cron: '17 3 * * *' - workflow_dispatch: - -defaults: - run: - shell: bash - -permissions: - contents: read - -jobs: - changes: - name: Relevant changes - runs-on: - group: organization/Default - outputs: - changed: ${{ steps.diff.outputs.changed }} - live: ${{ steps.diff.outputs.live }} - steps: - - name: Checkout repository - uses: actions/checkout@v7 - with: - fetch-depth: 0 - persist-credentials: false - - name: Decide with git diff - id: diff - env: - EVENT: ${{ github.event_name }} - PR_BASE: ${{ github.event.pull_request.base.sha }} - PUSH_BEFORE: ${{ github.event.before }} - FORK: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name != github.repository }} - run: | - changed=false - if [ "$EVENT" = "schedule" ] || [ "$EVENT" = "workflow_dispatch" ]; then - changed=true - else - base="$PR_BASE" - [ "$EVENT" = "push" ] && base="$PUSH_BEFORE" - if [ -z "$base" ] || [ "$base" = "0000000000000000000000000000000000000000" ] || ! git cat-file -e "$base^{commit}" 2>/dev/null; then - changed=true - elif git diff --name-only "$base" HEAD | grep -qE '^(docs-snippets/|src/|lib/|package\.json$|package-lock\.json$|\.github/workflows/docs-snippets\.yml$)'; then - changed=true - fi - fi - live=$changed - if [ "$FORK" = "true" ]; then - live=false - [ "$changed" = "true" ] && echo "live shards skipped: fork PR without secrets" >> "$GITHUB_STEP_SUMMARY" - fi - # DISABLED: the samples now run in the live tier (.github/workflows/live-integration.yml). The dry and live jobs - # are switched off here, and the gate still reports success, so a branch protection rule that requires it is - # not left pending. Remove these three lines to switch this suite back on. The workflow goes away with - # docs-snippets/ at the cutover. - changed=false - live=false - echo "docs-snippets suite is disabled: the samples run in the live tier" >> "$GITHUB_STEP_SUMMARY" - echo "changed=$changed" >> "$GITHUB_OUTPUT" - echo "live=$live" >> "$GITHUB_OUTPUT" - echo "relevant changes: $changed, live shards: $live" - - dry: - name: Dry run, types and manifest - needs: [changes] - if: needs.changes.outputs.changed == 'true' - runs-on: - group: organization/Default - timeout-minutes: 15 - steps: - - name: Checkout repository - uses: actions/checkout@v7 - with: - persist-credentials: false - - name: Setup NodeJS 22 - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4 - with: - node-version: 22 - cache: 'npm' - - name: Install dependencies - run: npm ci - - name: Build the SDK for Node - run: npm run build:node - - name: Dry run (no network) - run: npm run test:docs-snippets:dry - - name: Type check the suite and the browser samples - run: npm run test:docs-snippets:types - - name: Check the coverage manifest - run: npm run docs-snippets:manifest -- --check - - live: - name: Live shards (open, pam, datasync) - needs: [changes, dry] - if: needs.changes.outputs.live == 'true' - runs-on: - group: organization/Default - # The three shards run one after another in this one job, never in parallel. The pam and datasync roles may share one - # keyset, and the pam shard opens it with an application-level grant for a moment. Run in parallel, that grant and the - # janitor's revoke would break the other shard. - timeout-minutes: 35 - # One live job at a time across the repository, so two workflow runs (a pull request and master, say) never share the - # keysets either. A newer waiting run replaces an older waiting one, a running one is never cancelled. - concurrency: - group: docs-snippets-live - cancel-in-progress: false - # Keys and secrets. The one place that maps CI secrets onto the env names the suite reads (docs-snippets/README.md, "Keys in - # CI"). They are set on the steps that run the suite only, never on the job, so `npm ci` (install scripts of - # dependencies) and the build never see them. - env: - DOCS_SNIPPETS_DS_KEYSET_ID: ${{ vars.SDK_DOCS_DS_KEYSET_ID }} - steps: - - name: Checkout repository - uses: actions/checkout@v7 - with: - persist-credentials: false - - name: Setup NodeJS 22 - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4 - with: - node-version: 22 - cache: 'npm' - - name: Install dependencies - run: npm ci - - name: Build the SDK for Node - id: build - run: npm run build:node - # Each shard runs when the build succeeded, even when an earlier shard failed, but not after a cancel. Each gets the secret - # key of its own role only. The janitor step after them needs every role, so it keeps all of them. - - name: Shard open - id: open - if: '!cancelled() && steps.build.outcome == ''success''' - env: - SUBSCRIBE_KEY: ${{ secrets.SDK_DOCS_OPEN_SUBSCRIBE_KEY }} - PUBLISH_KEY: ${{ secrets.SDK_DOCS_OPEN_PUBLISH_KEY }} - run: npm run test:docs-snippets -- --shard open - - name: Keep the open report - if: always() && steps.open.outcome != 'skipped' - run: mkdir -p docs-snippets/report/open && mv docs-snippets/report/run.json docs-snippets/report/summary.md docs-snippets/report/open/ 2>/dev/null || true - - name: Shard pam - id: pam - if: '!cancelled() && steps.build.outcome == ''success''' - env: - SUBSCRIBE_KEY: ${{ secrets.SDK_DOCS_OPEN_SUBSCRIBE_KEY }} - PUBLISH_KEY: ${{ secrets.SDK_DOCS_OPEN_PUBLISH_KEY }} - PAM_SUBSCRIBE_KEY: ${{ secrets.SDK_DOCS_PAM_SUBSCRIBE_KEY }} - PAM_PUBLISH_KEY: ${{ secrets.SDK_DOCS_PAM_PUBLISH_KEY }} - PAM_SECRET_KEY: ${{ secrets.SDK_DOCS_PAM_SECRET_KEY }} - run: npm run test:docs-snippets -- --shard pam - - name: Keep the pam report - if: always() && steps.pam.outcome != 'skipped' - run: mkdir -p docs-snippets/report/pam && mv docs-snippets/report/run.json docs-snippets/report/summary.md docs-snippets/report/pam/ 2>/dev/null || true - - name: Shard datasync - id: datasync - if: '!cancelled() && steps.build.outcome == ''success''' - env: - SUBSCRIBE_KEY: ${{ secrets.SDK_DOCS_OPEN_SUBSCRIBE_KEY }} - PUBLISH_KEY: ${{ secrets.SDK_DOCS_OPEN_PUBLISH_KEY }} - DS_SUBSCRIBE_KEY: ${{ secrets.SDK_DOCS_DS_SUBSCRIBE_KEY }} - DS_PUBLISH_KEY: ${{ secrets.SDK_DOCS_DS_PUBLISH_KEY }} - DS_SECRET_KEY: ${{ secrets.SDK_DOCS_DS_SECRET_KEY }} - # Only this shard gets the Admin API key: it sets up the DataSync classes before the tests. - PUBNUB_ADMIN_API_KEY: ${{ secrets.SDK_DOCS_ADMIN_API_KEY }} - run: npm run test:docs-snippets -- --shard datasync - - name: Keep the datasync report - if: always() && steps.datasync.outcome != 'skipped' - run: mkdir -p docs-snippets/report/datasync && mv docs-snippets/report/run.json docs-snippets/report/summary.md docs-snippets/report/datasync/ 2>/dev/null || true - # One block on the run page with a row per snippet id (passed, FAILED, skipped, not run, type checked only). It reads the - # shard reports and coverage.json, needs no secrets, and runs after a failure or a cancel too. - - name: Result per snippet id - if: always() - run: node docs-snippets/harness/snippet-summary.cjs docs-snippets/report/open docs-snippets/report/pam docs-snippets/report/datasync - # After a failure or a cancel: revoke the application-level Access Manager grants and remove leftovers, so a run that - # died between a grant and its revoke never leaves a keyset open until the next run. - - name: Revoke grants and clean up - if: (cancelled() || failure()) && steps.build.outcome == 'success' - env: - SUBSCRIBE_KEY: ${{ secrets.SDK_DOCS_OPEN_SUBSCRIBE_KEY }} - PUBLISH_KEY: ${{ secrets.SDK_DOCS_OPEN_PUBLISH_KEY }} - PAM_SUBSCRIBE_KEY: ${{ secrets.SDK_DOCS_PAM_SUBSCRIBE_KEY }} - PAM_PUBLISH_KEY: ${{ secrets.SDK_DOCS_PAM_PUBLISH_KEY }} - PAM_SECRET_KEY: ${{ secrets.SDK_DOCS_PAM_SECRET_KEY }} - DS_SUBSCRIBE_KEY: ${{ secrets.SDK_DOCS_DS_SUBSCRIBE_KEY }} - DS_PUBLISH_KEY: ${{ secrets.SDK_DOCS_DS_PUBLISH_KEY }} - DS_SECRET_KEY: ${{ secrets.SDK_DOCS_DS_SECRET_KEY }} - run: npm run test:docs-snippets:janitor - - name: Upload the reports - if: always() - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 - with: - name: docs-snippets-reports - path: docs-snippets/report/ - if-no-files-found: warn - - gate: - name: Docs snippets - if: always() - needs: [changes, dry, live] - runs-on: - group: organization/Default - steps: - - name: Decide - env: - RESULTS: ${{ needs.changes.result }} ${{ needs.dry.result }} ${{ needs.live.result }} - CHANGED: ${{ needs.changes.outputs.changed }} - run: | - echo "changes, dry, live: $RESULTS (relevant changes: $CHANGED)" - for result in $RESULTS; do - if [ "$result" = "failure" ] || [ "$result" = "cancelled" ]; then - echo "a needed job ended with $result" - exit 1 - fi - done - echo "docs snippets gate passed" diff --git a/.github/workflows/live-integration.yml b/.github/workflows/live-integration.yml index 4a6fdaa0b..327df7ebf 100644 --- a/.github/workflows/live-integration.yml +++ b/.github/workflows/live-integration.yml @@ -2,8 +2,7 @@ name: Live integration # Live tier (test/integration/live/README.md): the public documentation's code samples run against the real PubNub service. # Nightly and on demand only. The live job never runs on a pull request, so fork PRs never see secrets. Pull requests get the -# offline job only (lint, manifest freshness, types), and it is not a required check. The old harness workflow -# (docs-snippets.yml) stays until the cutover. +# offline job only (lint, manifest freshness, types), and it is not a required check. on: # Pull requests: offline job only, and only when something the tier or its mocked samples depend on changed. @@ -15,7 +14,6 @@ on: - 'test/integration/components/subscription_manager.test.ts' - 'package.json' - '.github/workflows/live-integration.yml' - # 04:43 UTC, away from docs-snippets.yml (03:17 UTC), so the two never queue behind each other for the shared keysets. schedule: - cron: '43 4 * * *' workflow_dispatch: @@ -69,8 +67,8 @@ jobs: runs-on: group: organization/Default timeout-minutes: 40 - # Same group name as the live job of docs-snippets.yml on purpose: both use the same keysets, and this tier uses fixed - # resource names, so the two must never overlap. A running job is never cancelled. + # One live run per keyset at a time: this tier uses fixed resource names, so two runs must never overlap. A running job is + # never cancelled. concurrency: group: docs-snippets-live cancel-in-progress: false diff --git a/.github/workflows/release/versions.json b/.github/workflows/release/versions.json index e06aec85f..60ae32a39 100644 --- a/.github/workflows/release/versions.json +++ b/.github/workflows/release/versions.json @@ -39,14 +39,14 @@ "clearedSuffix": false } ], - "docs-snippets/quickstarts/quickstart-web.html": [ + "test/integration/live/quickstarts/quickstart-web.html": [ { "pattern": "cdn\\.pubnub\\.com/sdk/javascript/pubnub\\.(v?(\\.?\\d+){2,}([a-zA-Z0-9-]+(\\.?\\d+)?)?)\\.js\\\"", "clearedPrefix": true, "clearedSuffix": false } ], - "docs-snippets/quickstarts/browser-shared-worker.ts": [ + "test/integration/live/quickstarts/browser-shared-worker.ts": [ { "pattern": "using PubNub JS SDK (v?(\\.?\\d+){2,}([a-zA-Z0-9-]+(\\.?\\d+)?)?), make sure the versions match\\.", "clearedPrefix": true, diff --git a/.gitignore b/.gitignore index ec59be7a2..45ca59117 100644 --- a/.gitignore +++ b/.gitignore @@ -25,13 +25,8 @@ release-artifacts-* .vscode -# Live docs snippet suite (docs-snippets/README.md) # -#################################################### -docs-snippets/.run/ -docs-snippets/report/ .env .env.* -!docs-snippets/.env.example # Live integration tier (test/integration/live/README.md) # ########################################################### diff --git a/.npmignore b/.npmignore index 1f35c7b34..b18dfac80 100644 --- a/.npmignore +++ b/.npmignore @@ -19,4 +19,3 @@ test/* .github release-artifacts-* resources/* -docs-snippets/* \ No newline at end of file diff --git a/docs-snippets/.env.example b/docs-snippets/.env.example deleted file mode 100644 index cb0289edc..000000000 --- a/docs-snippets/.env.example +++ /dev/null @@ -1,25 +0,0 @@ -# Keys for the live docs snippet tests (docs-snippets/README.md, "Use your own keys"). -# Copy this file to docs-snippets/.env.local and fill in the values. Git ignores .env.local. -# Real environment variables win over this file. Leave a role empty to skip its steps (missing-env:). - -# open: a keyset without Access Manager -SUBSCRIBE_KEY= -PUBLISH_KEY= - -# pam: a keyset with Access Manager and token revocation -PAM_SUBSCRIBE_KEY= -PAM_PUBLISH_KEY= -PAM_SECRET_KEY= - -# datasync: a keyset with Access Manager and DataSync. It may be the same keyset as pam. -DS_SUBSCRIBE_KEY= -DS_PUBLISH_KEY= -DS_SECRET_KEY= - -# Optional. A Service Integration key with DataSync read and write on the datasync keyset. It creates the DataSync classes. -PUBNUB_ADMIN_API_KEY= - -# Optional, not secret. Admin Portal keyset ids, so the preflight can read each keyset's features through the Admin API. -DOCS_SNIPPETS_OPEN_KEYSET_ID= -DOCS_SNIPPETS_PAM_KEYSET_ID= -DOCS_SNIPPETS_DS_KEYSET_ID= diff --git a/docs-snippets/.mocharc.cjs b/docs-snippets/.mocharc.cjs deleted file mode 100644 index e817492b3..000000000 --- a/docs-snippets/.mocharc.cjs +++ /dev/null @@ -1,15 +0,0 @@ -'use strict'; -/** - * Mocha options of the live docs snippet suite. harness/cli.ts reads them and adds the staged spec file, which only exists - * after staging (docs-snippets/.run//harness/mocha-entry.ts). This file is separate from the SDK's root .mocharc.json - * on purpose: the suite never loads test/setup-why.ts (it restores nock) and the root spec never picks up the suite. - * - * retries 0: a retry would hide a flaky docs sample, waits are explicit (harness/wait.ts). bail false: one failing step must - * not hide the others. - */ -module.exports = { - reporter: require('node:path').join(__dirname, 'harness', 'reporter.cjs'), - retries: 0, - timeout: 30000, - bail: false, -}; diff --git a/docs-snippets/README.md b/docs-snippets/README.md deleted file mode 100644 index 057fb9f88..000000000 --- a/docs-snippets/README.md +++ /dev/null @@ -1,161 +0,0 @@ -# Live docs snippet tests - -These tests run the code samples of the JavaScript SDK (software development kit) docs against the real PubNub service and check that each sample does what its docs page says. - -## What only these tests catch - -A mocked test never runs the docs code and never talks to the service, so it cannot find a wrong docs sample. - -## Why they do not duplicate the integration tests - -- A test here checks only what the docs page claims, through the service. It does not check request encoding, retries or parsing. `test/integration` owns that. -- No step extends a sample to cover more parameters. A passing step means the documented behavior held, not that every option of the method works. -- The code readers see stays simple. All test code sits outside the snippet markers or on `snippet.hide` lines, and a marker lint enforces it. - -## Tech stack - -The suite adds no dependency. It runs on the Mocha 10.4 and tsx that the existing Node tests use, and checks types with the repository's TypeScript. It has its own Mocha config (`.mocharc.cjs`), its own tsconfigs and no nock, the HTTP mocking library of the integration tests. - -## What a run does - -| Stage | What happens | -| :--- | :--- | -| 1. Load keys | The loader reads the env names below and prints "set" or "missing" per role | -| 2. Stage | The command line tool, [`harness/cli.ts`](harness/cli.ts), copies the suite to `docs-snippets/.run//` and adds the run suffix to resource names | -| 3. Set up DataSync | Only when the run includes the `datasync` role. The setup creates missing classes | -| 4. Preflight | Checks that each keyset has the features its role needs, and names every missing one | -| 5. Steps | Each step runs its docs snippet, then asserts the effect through the service | -| 6. Clean up | The janitor (the cleanup routine) deletes what the run created and proves that no Access Manager grant is left open | -| 7. Report | The reporter writes `report/run.json` and `report/summary.md`, then the run policy decides the exit code | - -A shard is the set of steps of one role. CI, the GitHub Actions workflow [`docs-snippets.yml`](../.github/workflows/docs-snippets.yml), runs the three shards one after another in one job, and each shard's report goes to `report//`. - -## Run the tests locally - -You need Node.js 22 or later and your own PubNub keysets, as "Use your own keys" describes. Then, from the repository root: - -```bash -npm ci -npm run build:node # the tests use lib/node/index.js of this checkout -cp docs-snippets/.env.example docs-snippets/.env.local # then fill in your keys -npm run test:docs-snippets # everything -``` - -| Command | What it does | -| :--- | :--- | -| `npm run test:docs-snippets` | Stages the suite, sets up the DataSync classes when the run includes the `datasync` role, then runs every step live | -| `npm run test:docs-snippets -- --shard ` | Runs one role: `open` (about 8 min), `pam` (about 1.5 min) or `datasync` (about 3.5 min) | -| `npm run test:docs-snippets -- --grep ` | Runs the steps whose title matches, for example `--grep publishBasicUsage`. Each title carries its snippet ids. The steps a selected step needs run first, and the report lists them as prerequisites. Refused in CI | -| `npm run test:docs-snippets:dry` | No network. Checks the registry, staging, the marker lint, the static mapping, the manifest, the selection, the run policy and the reporter | -| `npm run test:docs-snippets:types` | Type checks the suite (`tsconfig.live.json`) and the browser samples (`tsconfig.browser.json`) | -| `npm run docs-snippets:manifest` | Writes `coverage.json`. With `-- --check` it verifies it instead | -| `npm run docs-snippets:provision` | Creates missing DataSync classes and updates drifted ones. With `-- --check` it shows the difference and writes nothing | -| `npm run test:docs-snippets:janitor` | Removes leftovers of runs older than six hours and revokes the application-level Access Manager grants | - -Rebuild with `npm run build:node` after a change under `src/`, otherwise the suite tests the previous build. If `pam` and `datasync` point at the same keyset, do not run those two shards at the same time. - -Each run writes two files to `docs-snippets/report/`. `run.json` has one record per step with its snippet ids, page, role, status, skip reason, duration and assertion count, plus the SDK version and commit. `summary.md` has the counts, failures, skips, time per phase and the slowest steps. In GitHub Actions the summary also goes to the job summary. - -## How a test runs the exact docs code - -A test is a `step()` in a page module that has the same base name as its docs page, for example [`api-reference/publish.ts`](api-reference/publish.ts) for the publish page. The docs embed the code between the markers from this file, so the test and the docs page show the same code. - -### Checks that keep the tests honest - -| Check | Rule | -| :--- | :--- | -| The snippet ran | In the staged copy every snippet start and end line records itself. A step fails when one of its snippets did not reach its `snippet.end` line | -| It asserted something | A step fails when it recorded zero assertions after its snippet started. Fixture setup does not count | -| Static mapping | Every marker id belongs to exactly one step or stub, every step's `snippets` list equals the markers in its body, and no id appears twice | -| Mutation | `DOCS_SNIPPETS_MUTATE=` throws at the start marker in the staged copy. That step must fail. Refused in CI | -| Same lines | Staging rewrites only registered names and never moves a line, so the line number of a failure matches the source file | - -## Keysets, roles and secrets - -The suite reads one fixed set of env names, whatever fills them. - -| Role | Env names | Used for | -| :--- | :--- | :--- | -| `open` | `SUBSCRIBE_KEY`, `PUBLISH_KEY` | Publish, subscribe, presence, history, message actions, App Context, files, push, channel groups, encryption, misc | -| `pam` | `PAM_SUBSCRIBE_KEY`, `PAM_PUBLISH_KEY`, `PAM_SECRET_KEY` | Access Manager v2 and v3 samples, `publishHandleErrors` and other 403 branches | -| `datasync` | `DS_SUBSCRIBE_KEY`, `DS_PUBLISH_KEY`, `DS_SECRET_KEY` | DataSync samples, including the ones that also need Access Manager | -| admin | `PUBNUB_ADMIN_API_KEY` | DataSync class setup before the tests, and a read-only check of keyset features | - -### Use your own keys - -1. In the [Admin Portal](https://admin.pubnub.com), create the keysets of "Features each keyset needs" below. One keyset can serve both `pam` and `datasync`. -2. Copy [`docs-snippets/.env.example`](.env.example) to `docs-snippets/.env.local` and fill in the keys. Git ignores `.env.local`. -3. Run `npm run test:docs-snippets -- --shard open` first. It needs only the open keyset. - -Leave a role empty only when you select the other roles with `--shard `. A full run with a role missing fails, because its steps would skip as `missing-env:`. You can also keep the file elsewhere and set `DOCS_SNIPPETS_ENV_FILE=`, or export the env names in your shell. Real environment variables always win over the file. The file may also use the older `PN_DOCS_*` names, which the loader maps onto the names above. - -The loader reads the file in memory and prints only "set" or "missing" per role. It never prints a key or part of one, and the reporter redacts every file before it writes it. - -### Keys in CI - -The workflow sets the same names from repository secrets in one `env:` block of the `live` job: - -| Secret | Env name | -| :--- | :--- | -| `SDK_DOCS_OPEN_SUBSCRIBE_KEY`, `SDK_DOCS_OPEN_PUBLISH_KEY` | `SUBSCRIBE_KEY`, `PUBLISH_KEY` | -| `SDK_DOCS_PAM_SUBSCRIBE_KEY`, `SDK_DOCS_PAM_PUBLISH_KEY`, `SDK_DOCS_PAM_SECRET_KEY` | `PAM_*` | -| `SDK_DOCS_DS_SUBSCRIBE_KEY`, `SDK_DOCS_DS_PUBLISH_KEY`, `SDK_DOCS_DS_SECRET_KEY` | `DS_*` | -| `SDK_DOCS_ADMIN_API_KEY` (only the `datasync` shard gets it) | `PUBNUB_ADMIN_API_KEY` | - -Optional and not secret: `DOCS_SNIPPETS_OPEN_KEYSET_ID`, `DOCS_SNIPPETS_PAM_KEYSET_ID` and `DOCS_SNIPPETS_DS_KEYSET_ID` (the workflow reads `vars.SDK_DOCS_DS_KEYSET_ID`). With the admin key the preflight reads each keyset's feature configuration through the Admin API, after it confirms that the keyset of that id has the role's subscribe key. - -Give this suite keysets of its own, not the keysets of `run-tests.yml`. Some Access Manager v2 samples grant on the whole keyset for a moment. The suite closes every grant when its step ends and checks before and after the run that a client without a token is refused, but a test that runs in parallel on the same keyset could still see the grant. - -### Features each keyset needs - -The suite never changes a keyset. A preflight names every missing feature in one message, for example "Presence is off on the open keyset". - -| Role | Features | -| :--- | :--- | -| `open` | Presence with tracking on all channels (pattern `*`), Message Persistence with Delete-From-History, App Context with user, channel and membership events on and referential integrity off, Files, Mobile Push Notifications (no credentials needed), Stream Controller with wildcard subscribe. Access Manager off | -| `pam` | Access Manager with token revocation, Message Persistence, Presence | -| `datasync` | Access Manager, DataSync, the classes `product` v1 and `ProductOwner` v1 with a `public` projection (the setup creates them), DataSync event rules for `User`, `Channel`, `Membership`, `product` and `ProductOwner` v1 (Admin Portal only, DataSync tab, "Select entities to sync") | - -### DataSync classes - -[`provision/datasync-classes.json`](provision/datasync-classes.json) defines the classes. Every live run that includes the `datasync` role sets them up first: - -- It creates missing classes and updates classes whose mutable fields drifted. It never deletes a class. -- A class whose immutable fields differ (relationship cardinality or entity sides) fails the run with the difference, because only a person may recreate it. -- A second run reports every class as unchanged and writes nothing. -- It touches only the classes of that file. Classes of other tests on a shared keyset stay as they are. -- Without `PUBNUB_ADMIN_API_KEY` it checks through the SDK, and a missing class fails the run with its name. - -## Skips - -A skip fails the run, always: locally, on a pull request, on master and nightly. A run must run every valid test, so a skip never counts as a pass. There is no skip budget, no allowed list and no opt-out. Every skip prints its reason and the report lists it. - -| Reason | What it means | How to fix it | -| :--- | :--- | :--- | -| `needs-feature:push-notifications` | The keyset has no Mobile Push add-on | Turn Mobile Push on for the keyset | -| `needs-feature:datasync-events` | DataSync event rules are off (Admin Portal only) | Turn on "Select entities to sync" for the classes in "Features each keyset needs" | -| `missing-env:` | The role's keys are not set | Set the keys, or select only the roles you have keys for with `--shard ` | -| `blocked-by:` | A step this one needs did not pass | Fix the step it names. The steps that need it skip until then | - -The browser, React and React Native sources in `quickstarts/` stay here because the docs embed them. `tsconfig.browser.json` type checks the browser ones. Nothing compiles the React and React Native ones here, because the SDK has no React or React Native dependencies, so these snippets cannot be run in this repository. The browser ones are type checked only, because this repository has no browser to run them in. - -## Control variables - -| Variable | Effect | -| :--- | :--- | -| `DOCS_SNIPPETS_ENV_FILE` | Env file to read keys from. Default: `docs-snippets/.env.local` | -| `DOCS_SNIPPETS_SHARD` | Same as `--shard` | -| `DOCS_SNIPPETS_ONLY` | Comma-separated step ids, like `--grep` with exact ids. Refused in CI | -| `DOCS_SNIPPETS_MUTATE` | See "Checks that keep the tests honest". Refused in CI | -| `DOCS_SNIPPETS_INJECT_FAILURE` | Makes a step fail with an SDK-style 403 after it ran. Refused in CI | -| `DOCS_SNIPPETS_LOG_LEVEL` | SDK log level of the samples. `trace` and `debug` are refused in CI, because the SDK logs its configuration at those levels | -| `DOCS_SNIPPETS_VERBOSE` | `1` prints the console output of every step | -| `DOCS_SNIPPETS_RUN_ID` | Fixed run id. In CI it is derived from the time, `GITHUB_RUN_ID`, `GITHUB_RUN_ATTEMPT` and the shard | -| `DOCS_SNIPPETS_DOCS_DIR` | Docs checkout for the manifest's docs id report. Default: a `documentation` folder next to this repository | - -## Add a snippet - -1. Write the docs code inside a `step()` of the page module, between `// snippet.` and `// snippet.end`. -2. Register new resource names in `fixtures/names.ts`. -3. Assert the effect the page claims, outside the markers. Use a second client or a read back, not "it did not throw". -4. Run `npm run test:docs-snippets:dry`, then `npm run docs-snippets:manifest`, then `npm run test:docs-snippets -- --grep `. diff --git a/docs-snippets/access-manager.ts b/docs-snippets/access-manager.ts deleted file mode 100644 index 4c62519a7..000000000 --- a/docs-snippets/access-manager.ts +++ /dev/null @@ -1,283 +0,0 @@ -import PubNub, { PubNubError } from '../lib/types'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'myUniqueUserId', -}); - -// snippet.grantTokenVariousResources -try { - const token = await pubnub.grantToken({ - ttl: 15, - authorized_uuid: 'my-authorized-uuid', - resources: { - channels: { - 'channel-a': { - read: true, - }, - 'channel-b': { - read: true, - write: true, - }, - 'channel-c': { - read: true, - write: true, - }, - 'channel-d': { - read: true, - write: true, - }, - }, - groups: { - 'channel-group-b': { - read: true, - }, - }, - uuids: { - 'uuid-c': { - get: true, - }, - 'uuid-d': { - get: true, - update: true, - }, - }, - }, - }); -} catch (error) { - console.error( - `Grant token error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.grantTokenUsingRegEx -try { - const token = await pubnub.grantToken({ - ttl: 15, - authorized_uuid: 'my-authorized-uuid', - patterns: { - channels: { - '^channel-[A-Za-z0-9]$': { - read: true, - }, - }, - }, - }); -} catch (error) { - console.error( - `Grant token error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.grantTokenDataSyncResources -// Grants `user-alice`: -// - Get access to the entity `product-sneaker-42`, and to all entities matching the -// `product-.*` RegEx pattern. -// - Get access to the DataSync user `user-bob` through the top-level `users` scope, because -// DataSync users are not permissioned under `dataSync`. -try { - const token = await pubnub.grantToken({ - ttl: 15, - authorized_uuid: 'user-alice', - resources: { - users: { - 'user-bob': { - get: true, - }, - }, - dataSync: { - entities: { - 'product-sneaker-42': { - get: true, - }, - }, - }, - }, - patterns: { - dataSync: { - entities: { - 'product-.*': { - get: true, - }, - }, - }, - }, - }); - console.log('Granted Token:', token); -} catch (error) { - console.error( - `Grant token error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.grantTokenWithProjection -// Grants `user-alice` the same access as above and assigns the `public` projection to the entity -// `product-sneaker-42` and to the user `user-bob`. With those projections, Alice reads and writes -// only the fields that the `public` projection exposes on each object. -try { - const token = await pubnub.grantToken({ - ttl: 15, - authorized_uuid: 'user-alice', - resources: { - users: { - 'user-bob': { - get: true, - }, - }, - dataSync: { - entities: { - 'product-sneaker-42': { - get: true, - }, - }, - }, - }, - patterns: { - dataSync: { - entities: { - 'product-.*': { - get: true, - }, - }, - }, - }, - dataSyncProjections: { - resources: { - entities: { - 'product-sneaker-42': 'public', - }, - users: { - 'user-bob': 'public', - }, - }, - patterns: { - entities: { - 'product-.*': 'public', - }, - }, - }, - }); - console.log('Granted Token:', token); -} catch (error) { - console.error( - `Grant token error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.grantTokenRegExAndResources -try { - const token = await pubnub.grantToken({ - ttl: 15, - authorized_uuid: 'my-authorized-uuid', - resources: { - channels: { - 'channel-a': { - read: true, - }, - 'channel-b': { - read: true, - write: true, - }, - 'channel-c': { - read: true, - write: true, - }, - 'channel-d': { - read: true, - write: true, - }, - }, - groups: { - 'channel-group-b': { - read: true, - }, - }, - uuids: { - 'uuid-c': { - get: true, - }, - 'uuid-d': { - get: true, - update: true, - }, - }, - }, - patterns: { - channels: { - '^channel-[A-Za-z0-9]$': { - read: true, - }, - }, - }, - }); -} catch (error) { - console.error( - `Grant token error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.grantTokenWithCategories -// Grants `my-authorized-uuid` permission to list all channel metadata and all uuid metadata. -// Category permissions support only the `get` operation. -try { - const token = await pubnub.grantToken({ - ttl: 15, - authorizedUserId: 'my-authorized-uuid', - categories: { - channels: { get: true }, - uuids: { get: true }, - }, - }); - console.log('Granted token:', token); -} catch (error) { - console.error( - `Grant token error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.parseTokenWithCategories -// Decodes a token that was granted with category permissions. -const parsedToken = pubnub.parseToken('use-token-string-generated-by-grantToken()'); - -console.log('Can list all channel metadata:', parsedToken?.categories?.channels?.get); -console.log('Can list all uuid metadata:', parsedToken?.categories?.uuids?.get); -// snippet.end - -// snippet.grantWithCategories -// Grants the auth key permission to list all channel metadata and all uuid metadata. -// This legacy `grant` method requires the secret key to be set on the client. -try { - const response = await pubnub.grant({ - authKeys: ['my-auth-key'], - categories: ['channels', 'uuids'], - get: true, - ttl: 15, - }); - console.log('Grant response:', response); -} catch (error) { - console.error( - `Grant error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end diff --git a/docs-snippets/api-reference/access-manager-legacy.ts b/docs-snippets/api-reference/access-manager-legacy.ts deleted file mode 100644 index 1a419516a..000000000 --- a/docs-snippets/api-reference/access-manager-legacy.ts +++ /dev/null @@ -1,241 +0,0 @@ -import assert from '../harness/assert.js'; -import type PubNub from 'pubnub'; -import { page, step, check } from '../harness/step.js'; -import { rejectsValidation } from '../harness/assertions.js'; -import { granted } from './access-manager.js'; - -/** - * Access Manager token examples that use the deprecated `spaces`, `users` and `uuids` terms. The SDK maps them to the - * current terms, so a `spaces` grant appears as `channels` in the parsed token. - */ -export default page('api-reference/access-manager-legacy', 'P8', (world) => { - const secured = { keyset: 'pam' } as const; - - interface LegacyGrantClient { - grantToken(parameters: PubNub.PAM.ObjectsGrantTokenParameters): Promise; - parseToken: PubNub['parseToken']; - } - const legacy = (): LegacyGrantClient => world.server as unknown as LegacyGrantClient; - - step('grantTokenSpacesBasicUsage', { ...secured, proves: 'A grant that uses the deprecated spaces scope returns a token whose parsed form lists the space under channels.' }, async () => { - const pubnub = legacy(); - // snippet.grantTokenSpacesBasicUsage - try { - const token = await pubnub.grantToken({ - ttl: 15, - authorizedUserId: "my-authorized-userId", - resources: { - spaces: { - "my-space": { - read: true, - }, - }, - }, - }); - // snippet.hide - world.keep('token', token); - // snippet.show - } catch (status) { - console.log(status); - } - // snippet.end - const token = world.take('token'); - const parsed = pubnub.parseToken(token); - assert.equal(parsed?.authorized_uuid, world.n('my-authorized-userId')); - assert.deepEqual(granted(parsed?.resources?.channels?.['my-space']), { read: true }); - }); - - // Mixed `spaces` and `users` resources with different permission sets. Asserts one space and one user entry - // in the parsed token (`users` keeps its name, `get` and `update` survive). - step('grantTokenSpacesVariousResources', { ...secured, proves: 'A token with mixed spaces and users resources keeps one space entry and one user entry with their permissions.' }, async () => { - const pubnub = legacy(); - // snippet.grantTokenSpacesVariousResources - try { - const token = await pubnub.grantToken({ - ttl: 15, - authorizedUserId: "my-authorized-userId", - resources: { - spaces: { - "space-a": { - read: true, - }, - "space-b": { - read: true, - write: true, - }, - "space-c": { - read: true, - write: true, - }, - "space-d": { - read: true, - write: true, - }, - }, - users: { - "userId-c": { - get: true, - }, - "userId-d": { - get: true, - update: true, - }, - }, - }, - }); - // snippet.hide - world.keep('token', token); - // snippet.show - } catch (status) { - console.log(status); - } - // snippet.end - const token = world.take('token'); - const parsed = pubnub.parseToken(token); - assert.deepEqual(granted(parsed?.resources?.channels?.['space-b']), { read: true, write: true }); - assert.deepEqual(granted(parsed?.resources?.users?.['userId-d']), { get: true, update: true }); - }); - - // Pattern-only grant. The regular expression key must appear unchanged under `patterns.channels`. - step('grantTokenSpacesRegEx', { ...secured, proves: 'A pattern-only grant with the deprecated spaces scope keeps the regular expression unchanged under patterns.channels.' }, async () => { - const pubnub = legacy(); - // snippet.grantTokenSpacesRegEx - try { - const token = await pubnub.grantToken({ - ttl: 15, - authorizedUserId: "my-authorized-userId", - patterns: { - spaces: { - "^space-[A-Za-z0-9]$": { - read: true, - }, - }, - }, - }); - // snippet.hide - world.keep('token', token); - // snippet.show - } catch (status) { - console.log(status); - } - // snippet.end - const token = world.take('token'); - const parsed = pubnub.parseToken(token); - assert.deepEqual(granted(parsed?.patterns?.channels?.['^space-[A-Za-z0-9]$']), { read: true }); - }); - - // Resources and patterns in one token. Checks both sections of the parsed token survive side by side. - step('grantTokenSpacesRegExAndResources', { ...secured, proves: 'Resources and patterns in one deprecated-scope grant both survive in the parsed token.' }, async () => { - const pubnub = legacy(); - // snippet.grantTokenSpacesRegExAndResources - try { - const token = await pubnub.grantToken({ - ttl: 15, - authorizedUserId: "my-authorized-userId", - resources: { - spaces: { - "space-a": { - read: true, - }, - "space-b": { - read: true, - write: true, - }, - "space-c": { - read: true, - write: true, - }, - "space-d": { - read: true, - write: true, - }, - }, - users: { - "userId-c": { - get: true, - }, - "userId-d": { - get: true, - update: true, - }, - }, - }, - patterns: { - spaces: { - "^space-[A-Za-z0-9]$": { - read: true - }, - }, - }, - }); - // snippet.hide - world.keep('token', token); - // snippet.show - } catch (status) { - console.log(status); - } - // snippet.end - const token = world.take('token'); - const parsed = pubnub.parseToken(token); - assert.deepEqual(granted(parsed?.resources?.channels?.['space-a']), { read: true }); - assert.deepEqual(granted(parsed?.patterns?.channels?.['^space-[A-Za-z0-9]$']), { read: true }); - }); - - check('accessManagerLegacyAuthorizedUuidAlias', { ...secured, proves: 'authorized_uuid and authorizedUserId produce the same token owner, and passing both is a validation error.' }, async () => { - const server = world.server; - const user = world.n('my-authorized-userId'); - const resources = { channels: { 'legacy-alias': { read: true } } }; - const viaAlias = await server.grantToken({ ttl: 15, authorized_uuid: user, resources }); - const viaCurrent = await server.grantToken({ ttl: 15, authorizedUserId: user, resources }); - assert.equal(server.parseToken(viaAlias)?.authorized_uuid, user); - assert.equal(server.parseToken(viaCurrent)?.authorized_uuid, user); - await rejectsValidation( - server.grantToken({ ttl: 15, authorizedUserId: user, authorized_uuid: user, resources } as never), - /Cannot mix `authorizedUserId` with `authorized_uuid`/, - ); - }); - - // The `uuids` scope is the old name of `users`. The parsed token keeps it under `uuids`, for resources and patterns. - check('accessManagerLegacyUuidsScope', { ...secured, proves: 'The uuids scope is the old name of users and the parsed token keeps it under uuids for resources and patterns.' }, async () => { - const server = world.server; - const token = await server.grantToken({ - ttl: 15, - authorizedUserId: world.n('my-authorized-userId'), - resources: { uuids: { 'uuid-1': { get: true, update: true, delete: true } } }, - patterns: { uuids: { 'uuid-.*': { get: true } } }, - }); - const parsed = server.parseToken(token); - assert.deepEqual(granted(parsed?.resources?.uuids?.['uuid-1']), { get: true, update: true, delete: true }); - assert.deepEqual(granted(parsed?.patterns?.uuids?.['uuid-.*']), { get: true }); - }); - - // The SDK refuses to mix old and new scope names in one grant. These are client-side validation errors, so no - // request leaves the process and the secured keyset sees no traffic. - check('accessManagerLegacyMixingErrors', { ...secured, proves: 'Mixing old and new scope names in one grant is refused locally with a validation error and sends no request.' }, async () => { - const server = world.server; - const ttl = 15; - // users and uuids: the check spans resources and patterns together. - await rejectsValidation( - server.grantToken({ - ttl, - resources: { users: { a: { get: true } } }, - patterns: { uuids: { 'a.*': { get: true } } }, - }), - /Cannot mix `users` with `uuids`/, - ); - await rejectsValidation( - server.grantToken({ - ttl, - resources: { spaces: { a: { read: true } }, channels: { b: { read: true } } }, - } as never), - /Cannot mix `spaces` with `channels`/, - ); - await rejectsValidation( - server.grantToken({ - ttl, - resources: { spaces: { a: { read: true } }, uuids: { b: { get: true } } }, - } as never), - /Cannot mix `spaces` with `uuids`/, - ); - }); -}); diff --git a/docs-snippets/api-reference/access-manager-v2.ts b/docs-snippets/api-reference/access-manager-v2.ts deleted file mode 100644 index b898553e8..000000000 --- a/docs-snippets/api-reference/access-manager-v2.ts +++ /dev/null @@ -1,241 +0,0 @@ -import assert from '../harness/assert.js'; -import type PubNub from 'pubnub'; -import { page, step, check } from '../harness/step.js'; -import { eventually, expectStatus } from '../harness/wait.js'; -import { rejectsValidation } from '../harness/assertions.js'; -import { revokeApplicationGrant } from '../harness/revoke.js'; - -/** - * Access Manager v2 `grant` examples. These calls require the secret key. - */ -export default page('api-reference/access-manager-v2', 'P1', (world) => { - const secured = { keyset: 'pam' } as const; - /** Loose view of a grant response. The union in the types is wide, the assertions read plain fields. */ - const loose = (response: unknown): Record => response as Record; - - /** A client that sends `authKey` with every request. */ - const withAuthKey = (label: string, authKey: string): PubNub => - world.client('secured', label, { authKey: world.n(authKey) }); - - const publishTo = async (client: PubNub, channel: string): Promise => - (await client.publish({ channel, message: { text: 'am-v2' } })).timetoken; - - const closeGrant = (params: Record): Promise => - world.server.grant({ read: false, write: false, ttl: 1, ...params } as never); - - step('grantBasicUsage', { ...secured, proves: 'grant returns the permissions asked for on one auth key and channel, and a client carrying that auth key can publish there.' }, async () => { - const pubnub = world.server; - // snippet.grantBasicUsage - const result = await pubnub.grant({ - authKeys: ['my_authkey'], - channels: ['my_channel'], - ttl: 12313, // 0 for infinite - read: true, // false to disallow - write: true, // false to disallow - manage: true, // false to disallow - delete: true, // false to disallow - }); - // snippet.end - try { - const response = loose(result); - console.log(JSON.stringify(response)); - const channel = world.n('my_channel'); - const auth = world.n('my_authkey'); - // The SDK resolves with the payload itself, not with the status envelope the docs show. - // The docs Response block shows exactly these keys and the ttl the request sent. - assert.deepEqual(Object.keys(response).sort(), ['auths', 'channel', 'level', 'subscribe_key', 'ttl']); - assert.equal(response.ttl, 12313); - assert.equal(response.channel, channel); - assert.equal(response.level, 'user'); - const entry = response.auths?.[auth]; - assert.deepEqual([entry?.r, entry?.w, entry?.m, entry?.d], [1, 1, 1, 1], 'all four permissions are granted'); - // Effect: a client with that auth key can publish to the channel. - world.check.timetoken(await publishTo(withAuthKey('am-v2-user', 'my_authkey'), channel)); - } finally { - await closeGrant({ authKeys: [world.n('my_authkey')], channels: [world.n('my_channel')], manage: false, delete: false }); - } - }); - - // Downgrade of the same pair to read only. `needs` the first grant so the response describes a change. - // No effect check, see the comment at the end of the body. The long timeout is headroom for slow grant propagation. - step('grantChannelOnAuthKey', { ...secured, needs: ['grantBasicUsage'], timeout: 60_000, proves: 'A second grant on the same auth key and channel reports the downgrade to read only in its response.' }, async () => { - const pubnub = world.server; - // snippet.grantChannelOnAuthKey - const result = await pubnub.grant({ - authKeys: ['my_authkey'], - channels: ['my_channel'], - ttl: 5, - read: true, // false to disallow - write: false, // false to disallow - }); - // snippet.end - try { - const entry = loose(result).auths?.[world.n('my_authkey')]; - assert.deepEqual([entry?.r, entry?.w], [1, 0]); - } finally { - await closeGrant({ authKeys: [world.n('my_authkey')], channels: [world.n('my_channel')] }); - } - }); - - step('grantPresenceChannelOnAuthKey', { ...secured, proves: 'Granting the -pnpres companion channel returns the permissions for that presence channel.' }, async () => { - const pubnub = world.server; - world.setPh('myChannelPnpres', `${world.n('my_channel')}-pnpres`); - // snippet.grantPresenceChannelOnAuthKey - const result = await pubnub.grant({ - authKeys: ['my_authkey'], - channels: ['my_channel-pnpres'], - ttl: 5, - read: true, // false to disallow - write: true, // false to disallow - }); - // snippet.end - try { - const response = loose(result); - assert.equal(response.channel, `${world.n('my_channel')}-pnpres`); - const entry = response.auths?.[world.n('my_authkey')]; - assert.deepEqual([entry?.r, entry?.w], [1, 1]); - } finally { - await closeGrant({ authKeys: [world.n('my_authkey')], channels: [`${world.n('my_channel')}-pnpres`] }); - } - }); - - // User level grant (`get`, `update`, `delete` on user metadata). Response only, no effect check. - step('grantUuidOnAuthKey', { ...secured, proves: 'A user level grant returns get, update and delete permissions for the uuid.' }, async () => { - const pubnub = world.server; - // snippet.grantUuidOnAuthKey - const result = await pubnub.grant({ - authKeys: ['key1'], - uuids: ['uuid1'], - ttl: 1440, // 0 for infinite - get: true, // false to disallow - update: true, // false to disallow - delete: true, // false to disallow - }); - // snippet.end - try { - const response = loose(result); - console.log(JSON.stringify(response)); - // The docs describe a uuid grant as `uuids: { : { auths } }` next to `level`, `ttl` and `subscribe_key`. - assert.deepEqual(Object.keys(response).sort(), ['level', 'subscribe_key', 'ttl', 'uuids']); - assert.equal(response.level, 'uuid'); - const entry = response.uuids?.[world.n('uuid1')]?.auths?.[world.n('key1')]; - assert.deepEqual([entry?.g, entry?.u, entry?.d], [1, 1, 1]); - } finally { - await world.server.grant({ authKeys: [world.n('key1')], uuids: [world.n('uuid1')], get: false, update: false, delete: false, ttl: 1 }); - } - }); - - step('grantWithCategories', { ...secured, proves: 'A category grant returns the permissions for all channel metadata and all uuid metadata.' }, async () => { - const pubnub = world.server; - // snippet.grantWithCategories - // Grants the auth key permission to list all channel metadata and all uuid metadata. - // This legacy `grant` method requires the secret key to be set on the client. - const response = await pubnub.grant({ - authKeys: ['my-auth-key'], - categories: ['channels', 'uuids'], - get: true, - ttl: 15, - }); - console.log('Grant response:', response); - // snippet.end - try { - const result = loose(response); - assert.equal(result.level, 'category'); - const auth = world.n('my-auth-key'); - assert.equal(result.categories?.channels?.auths?.[auth]?.g, 1); - assert.equal(result.categories?.uuids?.auths?.[auth]?.g, 1); - } finally { - await world.server.grant({ authKeys: [world.n('my-auth-key')], categories: ['channels', 'uuids'], get: false, ttl: 2 }); - } - }); - - step('grantChannelAllAuthKeys', { ...secured, proves: 'A channel level grant without auth keys opens the channel to a client with no auth key, and the grant is closed again at the end.' }, async () => { - const pubnub = world.server; - // snippet.grantChannelAllAuthKeys - const result = await pubnub.grant({ - channels: ['my_channel'], - read: true, // false to disallow - write: true, // false to disallow - }); - // snippet.end - // From here on the channel is open to everybody, so everything below runs inside try and the finally closes it again. - try { - const response = loose(result); - assert.equal(response.level, 'channel'); - const entry = response.channels?.[world.n('my_channel')]; - assert.deepEqual([entry?.r, entry?.w], [1, 1]); - // Effect: a client without any auth key can use the channel now. - world.check.timetoken(await publishTo(world.client('secured', 'am-denied'), world.n('my_channel'))); - } finally { - // Closes the channel for everybody again (read and write off). Runs also when an assertion above fails. - await pubnub.grant({ channels: [world.n('my_channel')], read: false, write: false, ttl: 1 }); - } - }); - - // Categories: documented validation errors, raised locally before anything is sent. - check('accessManagerV2CategoryErrors', { ...secured, proves: 'The documented category grant mistakes are refused locally with validation errors before anything is sent.' }, async () => { - const server = world.server; - await rejectsValidation( - server.grant({ categories: ['channels'], get: true, ttl: 15 }), - /authKeys are required for grant request on categories/, - ); - await rejectsValidation( - server.grant({ authKeys: ['k'], categories: ['groups' as never], get: true, ttl: 15 }), - /Invalid category: only channels and uuids are supported/, - ); - await rejectsValidation( - server.grant({ authKeys: ['k'], categories: ['channels'], get: true, read: true, ttl: 15 }), - /Category permissions must be exactly the get permission/, - ); - await rejectsValidation( - server.grant({ authKeys: ['k'], categories: ['channels'], get: true, ttl: 1 }), - /One-minute category grants are not supported/, - ); - }); - - // The application level sample opens every channel for every client, so it is the last thing this module does - // and the revoke runs even when an assertion fails. The probe is a stranger with no token and no auth key that - // subscribes: it connects while the application level read is on, otherwise the SDK reports a connection error - // or access denied. - step('grantAllChannelsAllAuthKeys', { ...secured, phase: 'P9', timeout: 120_000, proves: 'The application level grant opens the keyset to a stranger, and the revoke closes it again and the keyset stops serving the stranger.' }, async () => { - const pubnub = world.server; - let probes = 0; - const subscribeResult = async (): Promise => { - probes += 1; - const stranger = world.client('secured', `am-denied-${probes}`); - const result = expectStatus(stranger, ['PNConnectedCategory', 'PNAccessDeniedCategory', 'PNConnectionErrorCategory'], { timeout: 10_000 }); - const subscription = stranger.channel(world.n('am_other')).subscription(); - subscription.subscribe(); - try { - return await result; - } finally { - subscription.unsubscribe(); - stranger.destroy(); - } - }; - try { - // snippet.grantAllChannelsAllAuthKeys - const result = await pubnub.grant({ - read: true, - }); - // snippet.end - const response = loose(result); - assert.equal(response.level, 'subkey'); - assert.equal(response.r, 1); - assert.equal(response.ttl, 1440, 'the default ttl is 1440 minutes'); - await eventually(async () => assert.equal(await subscribeResult(), 'PNConnectedCategory'), { - timeout: 30_000, - interval: 500, - label: 'a stranger can read now', - }); - } finally { - await revokeApplicationGrant(pubnub); - } - // The revoke took, so the keyset is not left open for other modules. - await eventually(async () => assert.notEqual(await subscribeResult(), 'PNConnectedCategory'), { - timeout: 45_000, - interval: 500, - label: 'application level grant is revoked', - }); - }); -}); diff --git a/docs-snippets/api-reference/access-manager.ts b/docs-snippets/api-reference/access-manager.ts deleted file mode 100644 index ba3860485..000000000 --- a/docs-snippets/api-reference/access-manager.ts +++ /dev/null @@ -1,412 +0,0 @@ -import assert from '../harness/assert.js'; -import type PubNub from 'pubnub'; -import { page, step, check } from '../harness/step.js'; -import { eventually, connected } from '../harness/wait.js'; -import { rejectsValidation, rejectsWithCategory } from '../harness/assertions.js'; - -/** Parsed tokens list every permission flag. The docs show the granted ones, so compare only those. */ -export const granted = (permissions: object | undefined): Record => - Object.fromEntries(Object.entries(permissions ?? {}).filter(([, value]) => value === true)); - -/** - * Access Manager v3 token examples. The `grantToken` calls run on a server that holds the secret key. - */ -export default page('api-reference/access-manager', 'P1', (world) => { - const secured = { keyset: 'pam' } as const; - let aliceToken = ''; - let bobToken = ''; - - /** Publishes to a channel and resolves with the timetoken. */ - const publishTo = async (client: PubNub, channel: string): Promise => { - const response = await client.publish({ channel, message: { text: 'am-check' } }); - return response.timetoken; - }; - - check('accessManagerTokenFixture', { keyset: 'pam', timeout: 60_000, proves: 'Fixture: Alice gets read and write and Bob read only on the room, and Bob is denied on publish but can subscribe.' }, async () => { - const server = world.server; - const room = world.n('am_room'); - aliceToken = await server.grantToken({ - ttl: 15, - authorizedUserId: world.alice.getUserId(), - resources: { channels: { [room]: { read: true, write: true } } }, - }); - bobToken = await server.grantToken({ - ttl: 15, - authorizedUserId: world.bob.getUserId(), - resources: { channels: { [room]: { read: true } } }, - }); - world.setToken('bob', bobToken); - assert.equal(world.bob.getToken(), bobToken); - // Bob holds a read-only token: he can subscribe to the room but not publish to it. - await rejectsWithCategory(world.bob.publish({ channel: room, message: 'x' }), 'PNAccessDeniedCategory'); - const subscription = world.bob.channel(room).subscription(); - const ready = connected(world.bob, { channels: [room] }); - subscription.subscribe(); - await ready; - subscription.unsubscribe(); - }); - - step('accessManagerBasicUsage', { ...secured, proves: 'grantToken returns a token whose parsed form carries the user id, channel permissions and ttl that were asked for.' }, async () => { - const pubnub = world.server; - // snippet.accessManagerBasicUsage - // Function to use grantToken method - const token = await pubnub.grantToken({ - ttl: 15, - authorizedUserId: 'my-authorized-uuid', - resources: { - channels: { - 'my-channel': { - read: true, - write: true, - }, - }, - }, - }); - console.log('Granted Token:', token); - // snippet.end - const parsed = pubnub.parseToken(token); - assert.equal(parsed?.authorized_uuid, world.n('my-authorized-uuid')); - assert.deepEqual(granted(parsed?.resources?.channels?.['my-channel']), { read: true, write: true }); - assert.equal(parsed?.ttl, 15); - world.setPh('grantedToken', token); - }); - - step('revokeTokenBasicUsage', { ...secured, timeout: 240_000, proves: 'revokeToken makes a working token stop working, observed by a client that was allowed before and by two fresh clients with the same token.' }, async () => { - // Fixture: a user with a working token, so the effect of the revoke is visible. - const room = world.n('am_other'); - const revokee = world.client('secured', 'am-revokee'); - const token = await world.server.grantToken({ - ttl: 15, - authorizedUserId: revokee.getUserId(), - resources: { channels: { [room]: { read: true, write: true } } }, - }); - revokee.setToken(token); - world.check.timetoken(await publishTo(revokee, room)); - world.setPh('tokenToRevoke', token); - - const pubnub = world.server; - // snippet.revokeTokenBasicUsage - const response = await pubnub.revokeToken('p0AkFl043rhDdHRsple3KgQ3NwY6BDcENnctokenVzcqBDczaWdYIGOAeTyWGJI'); - // snippet.end - assert.deepEqual(response, {}, 'revokeToken resolves with an empty object'); - await eventually( - () => rejectsWithCategory(publishTo(revokee, room), 'PNAccessDeniedCategory'), - // The docs say a revoke can take up to one minute, because the non-revoked state is cached for that long. - { timeout: 100_000, interval: 2_000, label: 'revoked token is rejected' }, - ); - for (const attempt of [1, 2]) { - const fresh = world.client('secured', 'am-revokee'); - fresh.setToken(token); - await eventually( - () => rejectsWithCategory(publishTo(fresh, room), 'PNAccessDeniedCategory'), - { timeout: 60_000, interval: 2_000, label: `revoked token is rejected for fresh client ${attempt}` }, - ); - } - }); - - step('parseTokenBasicUsage', { ...secured, needs: ['accessManagerTokenFixture'], proves: 'parseToken decodes a token locally, and the fixture token shows the right owner and permissions.' }, () => { - const pubnub = world.server; - world.setPh('grantedToken', aliceToken); - // snippet.parseTokenBasicUsage - pubnub.parseToken('use-token-string-generated-by-grantToken()'); - // snippet.end - const parsed = pubnub.parseToken(aliceToken); - assert.equal(parsed?.authorized_uuid, world.alice.getUserId()); - assert.deepEqual(granted(parsed?.resources?.channels?.[world.n('am_room')]), { read: true, write: true }); - }); - - step('setTokenBasicUsage', { ...secured, needs: ['accessManagerTokenFixture'], timeout: 60_000, proves: 'setToken lets Alice publish to her room but not to another, while Bob stays read only.' }, async () => { - const pubnub = world.alice; - const room = world.n('am_room'); - world.setPh('grantedToken', aliceToken); - // Without a token the secured keyset refuses Alice. - pubnub.setToken(undefined); - await rejectsWithCategory(publishTo(pubnub, room), 'PNAccessDeniedCategory'); - // snippet.setTokenBasicUsage - pubnub.setToken('use-token-string-generated-by-grantToken()'); - // snippet.end - assert.equal(pubnub.getToken(), aliceToken); - world.check.timetoken(await publishTo(pubnub, room)); - // The token names one room only. - await rejectsWithCategory(publishTo(pubnub, world.n('am_other')), 'PNAccessDeniedCategory'); - // Bob (read only) still cannot publish, he reads what Alice sent. - await rejectsWithCategory(publishTo(world.bob, room), 'PNAccessDeniedCategory'); - }); - - // Channels, groups and uuids in one token. Property keys are not suffixed (see header), so the parsed token is - // read with the plain keys. - step('grantTokenVariousResources', { ...secured, proves: 'A token with channels, groups and uuids keeps each resource with its permissions in the parsed form.' }, async () => { - const pubnub = world.server; - // snippet.grantTokenVariousResources - const token = await pubnub.grantToken({ - ttl: 15, - authorizedUserId: 'my-authorized-uuid', - resources: { - channels: { - 'channel-a': { - read: true, - }, - 'channel-b': { - read: true, - write: true, - }, - 'channel-c': { - read: true, - write: true, - }, - 'channel-d': { - read: true, - write: true, - }, - }, - groups: { - 'channel-group-b': { - read: true, - }, - }, - uuids: { - 'uuid-c': { - get: true, - }, - 'uuid-d': { - get: true, - update: true, - }, - }, - }, - }); - // snippet.end - const parsed = pubnub.parseToken(token); - assert.deepEqual(granted(parsed?.resources?.channels?.['channel-a']), { read: true }); - assert.deepEqual(granted(parsed?.resources?.channels?.['channel-d']), { read: true, write: true }); - assert.deepEqual(granted(parsed?.resources?.groups?.['channel-group-b']), { read: true }); - assert.deepEqual(granted(parsed?.resources?.uuids?.['uuid-d']), { get: true, update: true }); - }); - - // Pattern-only token: the regular expression key must survive unchanged in `patterns.channels`. - step('grantTokenUsingRegEx', { ...secured, proves: 'A pattern-only token keeps the regular expression unchanged under patterns.channels.' }, async () => { - const pubnub = world.server; - // snippet.grantTokenUsingRegEx - const token = await pubnub.grantToken({ - ttl: 15, - authorizedUserId: 'my-authorized-uuid', - patterns: { - channels: { - '^channel-[A-Za-z0-9]$': { - read: true, - }, - }, - }, - }); - // snippet.end - const parsed = pubnub.parseToken(token); - assert.deepEqual(granted(parsed?.patterns?.channels?.['^channel-[A-Za-z0-9]$']), { read: true }); - }); - - // Resources and patterns together, both read back from the parsed token. - step('grantTokenRegExAndResources', { ...secured, proves: 'A token with resources and patterns returns both sections when parsed.' }, async () => { - const pubnub = world.server; - // snippet.grantTokenRegExAndResources - const token = await pubnub.grantToken({ - ttl: 15, - authorizedUserId: 'my-authorized-uuid', - resources: { - channels: { - 'channel-a': { - read: true, - }, - 'channel-b': { - read: true, - write: true, - }, - 'channel-c': { - read: true, - write: true, - }, - 'channel-d': { - read: true, - write: true, - }, - }, - groups: { - 'channel-group-b': { - read: true, - }, - }, - uuids: { - 'uuid-c': { - get: true, - }, - 'uuid-d': { - get: true, - update: true, - }, - }, - }, - patterns: { - channels: { - '^channel-[A-Za-z0-9]$': { - read: true, - }, - }, - }, - }); - // snippet.end - const parsed = pubnub.parseToken(token); - assert.deepEqual(granted(parsed?.resources?.channels?.['channel-b']), { read: true, write: true }); - assert.deepEqual(granted(parsed?.patterns?.channels?.['^channel-[A-Za-z0-9]$']), { read: true }); - }); - - // Category permissions (`get` on all channel metadata and all uuid metadata). Stores the token as the - // `grantedToken` placeholder for `parseTokenWithCategories`. - step('grantTokenWithCategories', { ...secured, proves: 'A category token carries get on all channel metadata and all uuid metadata.' }, async () => { - const pubnub = world.server; - // snippet.grantTokenWithCategories - // Grants `my-authorized-uuid` permission to list all channel metadata and all uuid metadata. - // Category permissions support only the `get` operation. - const token = await pubnub.grantToken({ - ttl: 15, - authorizedUserId: 'my-authorized-uuid', - categories: { - channels: { get: true }, - uuids: { get: true }, - }, - }); - console.log('Granted token:', token); - // snippet.end - const parsed = pubnub.parseToken(token); - assert.equal(parsed?.categories?.channels?.get, true); - assert.equal(parsed?.categories?.uuids?.get, true); - world.setPh('grantedToken', token); - }); - - step('parseTokenWithCategories', { ...secured, needs: ['grantTokenWithCategories'], proves: 'parseToken reads the category permissions of the token created by the previous step.' }, () => { - const pubnub = world.server; - // snippet.parseTokenWithCategories - // Decodes a token that was granted with category permissions. - const parsedToken = pubnub.parseToken('use-token-string-generated-by-grantToken()'); - - console.log('Can list all channel metadata:', parsedToken?.categories?.channels?.get); - console.log('Can list all uuid metadata:', parsedToken?.categories?.uuids?.get); - // snippet.end - assert.equal(parsedToken?.categories?.channels?.get, true); - assert.equal(parsedToken?.categories?.uuids?.get, true); - }); - - step('grantTokenDataSyncResources', { keyset: 'datasync', proves: 'A token can carry DataSync entity scopes and the top-level users scope, and parseToken returns them.' }, async () => { - const pubnub = world.server; - // snippet.grantTokenDataSyncResources - // Grants `user-alice`: - // - Get access to the entity `product-sneaker-42`, and to all entities matching the - // `product-.*` RegEx pattern. - // - Get access to the DataSync user `user-bob` through the top-level `users` scope, because - // DataSync users are not permissioned under `dataSync`. - const token = await pubnub.grantToken({ - ttl: 15, - authorizedUserId: 'user-alice', - resources: { - users: { - 'user-bob': { - get: true, - }, - }, - dataSync: { - entities: { - 'product-sneaker-42': { - get: true, - }, - }, - }, - }, - patterns: { - dataSync: { - entities: { - 'product-.*': { - get: true, - }, - }, - }, - }, - }); - console.log('Granted Token:', token); - // snippet.end - const parsed = pubnub.parseToken(token); - assert.equal(parsed?.authorized_uuid, world.alice.getUserId()); - assert.ok(parsed?.resources, 'the token carries resources'); - }); - - step('grantTokenWithProjection', { keyset: 'datasync', proves: 'A token with dataSyncProjections is created and parsed, and the projection assignment for an entity of a class that declares it is carried in the token.' }, async () => { - const pubnub = world.server; - // snippet.grantTokenWithProjection - // Grants `user-alice` the same access as above and assigns the `public` projection to the entity - // `product-sneaker-42` and to all entities matching `product-.*`. The entity class must declare - // a `public` projection. With it, Alice reads and writes only the fields that the projection exposes. - const token = await pubnub.grantToken({ - ttl: 15, - authorizedUserId: 'user-alice', - resources: { - users: { - 'user-bob': { - get: true, - }, - }, - dataSync: { - entities: { - 'product-sneaker-42': { - get: true, - }, - }, - }, - }, - patterns: { - dataSync: { - entities: { - 'product-.*': { - get: true, - }, - }, - }, - }, - dataSyncProjections: { - resources: { - entities: { - 'product-sneaker-42': 'public', - }, - }, - patterns: { - entities: { - 'product-.*': 'public', - }, - }, - }, - }); - console.log('Granted Token:', token); - // snippet.end - const parsed = pubnub.parseToken(token) as { meta?: Record; pat?: Record }> } | undefined; - assert.ok(parsed, 'the token parses'); - assert.equal(parsed.meta?.['pn-projections']?.res?.['datasync:entities:product-sneaker-42'], 'public'); - assert.equal(parsed.meta?.['pn-projections']?.pat?.['datasync:entities:product-.*'], 'public'); - }); - - check('accessManagerGrantTokenErrors', { ...secured, proves: 'The documented grantToken mistakes are refused locally with validation errors and create no token.' }, async () => { - const server = world.server; - await rejectsValidation(server.grantToken({ ttl: 15 } as never), /Missing values for either Resources or Patterns/); - await rejectsValidation( - server.grantToken({ ttl: 15, resources: { channels: {} } }), - /Missing values for either Resources or Patterns/, - ); - await rejectsValidation(server.revokeToken(''), /token can't be empty/); - await rejectsValidation(world.shopper.revokeToken('some-token'), /Missing Secret Key/); - // The open client has no secret key, so it cannot grant. - await rejectsValidation( - world.shopper.grantToken({ ttl: 15, resources: { channels: { x: { read: true } } } }), - /Missing Secret Key/, - ); - }); - - // A client that never received a token is rejected on the secured keyset. - check('accessManagerDeniedWithoutToken', { ...secured, proves: 'A client that never received a token is refused with a 403 on the secured keyset.' }, async () => { - const stranger = world.client('secured', 'am-denied'); - await rejectsWithCategory(publishTo(stranger, world.n('am_room')), 'PNAccessDeniedCategory'); - }); -}); diff --git a/docs-snippets/api-reference/channel-groups.ts b/docs-snippets/api-reference/channel-groups.ts deleted file mode 100644 index 435ad6140..000000000 --- a/docs-snippets/api-reference/channel-groups.ts +++ /dev/null @@ -1,96 +0,0 @@ -import assert from '../harness/assert.js'; -import { page, step } from '../harness/step.js'; -import { eventually } from '../harness/wait.js'; - -/** - * Channel group examples. - */ -export default page('api-reference/channel-groups', 'P3', (world) => { - // Channel order in a group is not guaranteed, so compare sorted copies. - const sorted = (values: string[]): string[] => [...values].sort(); - - step('addChannelsToGroupBasicUsage', { proves: 'addChannels adds two channels to a group, and a second client reads them back from the group.' }, async () => { - const pubnub = world.shopper; - // snippet.addChannelsToGroupBasicUsage - const response = await pubnub.channelGroups.addChannels({ - channels: ['ch1', 'ch2'], - channelGroup: 'myChannelGroup', - }); - console.log('addChannels to Group response:', response); - // snippet.end - // Another client confirms the effect. - await eventually(async () => { - const listed = await world.observer.channelGroups.listChannels({ channelGroup: world.n('myChannelGroup') }); - assert.deepEqual(sorted(listed.channels), sorted([world.n('ch1'), world.n('ch2')])); - }, { label: 'group lists ch1 and ch2' }); - }); - - step('listChannelsInGroupBasicUsage', { needs: ['addChannelsToGroupBasicUsage'], proves: 'listChannels returns the channels the previous step added to the group.' }, async () => { - const pubnub = world.shopper; - // snippet.listChannelsInGroupBasicUsage - // assuming an intialized PubNub instance already exists - // to get some data in response, first add some channels to the group using addChannels() method. - const response = await pubnub.channelGroups.listChannels({ - channelGroup: 'myChannelGroup', - }); - console.log('Channels in the group:', response); - response.channels.forEach((channel) => { - console.log(channel); - }); - // snippet.end - assert.deepEqual(sorted(response.channels), sorted([world.n('ch1'), world.n('ch2')])); - }); - - step('removeChannelsFromGroupBasicUsage', { proves: 'removeChannels removes one channel from a group and leaves only the other one.' }, async () => { - const pubnub = world.shopper; - // The docs say the channel must already be in the group to observe the removal. - await world.observer.channelGroups.addChannels({ - channelGroup: world.n('family'), - channels: [world.n('son'), world.n('daughter')], - }); - await eventually(async () => { - const listed = await world.observer.channelGroups.listChannels({ channelGroup: world.n('family') }); - assert.deepEqual(sorted(listed.channels), sorted([world.n('son'), world.n('daughter')])); - }, { label: 'family group seeded' }); - - // snippet.removeChannelsFromGroupBasicUsage - // assuming an initialized PubNub instance already exists - // and channel which is going to be removed from the group is aredaly added to the group to observe the removal - const response = await pubnub.channelGroups.removeChannels({ - channels: ['son'], - channelGroup: 'family', - }); - console.log('removeChannels from group response:', response); - // snippet.end - await eventually(async () => { - const listed = await world.observer.channelGroups.listChannels({ channelGroup: world.n('family') }); - assert.deepEqual(listed.channels, [world.n('daughter')]); - }, { label: 'son removed, daughter stays' }); - await world.observer.channelGroups.deleteGroup({ channelGroup: world.n('family') }); - await eventually(async () => { - const listed = await world.observer.channelGroups.listChannels({ channelGroup: world.n('family') }); - assert.deepEqual(listed.channels, []); - }, { label: 'family group cleaned up after the removal sample' }); - }); - - step('deleteChannelGroupBasicUsage', { phase: 'P9', needs: ['removeChannelsFromGroupBasicUsage'], proves: 'deleteChannelGroup removes the group, and a deleted group lists no channels.' }, async () => { - const pubnub = world.shopper; - await world.observer.channelGroups.addChannels({ channelGroup: world.n('family'), channels: [world.n('son')] }); - await eventually(async () => { - const listed = await world.observer.channelGroups.listChannels({ channelGroup: world.n('family') }); - assert.ok(listed.channels.includes(world.n('son')), `family lists son, got ${listed.channels.join(', ')}`); - }, { label: 'family group seeded for deletion' }); - // snippet.deleteChannelGroupBasicUsage - // assuming an initialized PubNub instance already exists - // and channel group which is getting deleted already exist to see the deletion effect. - const response = await pubnub.channelGroups.deleteGroup({ - channelGroup: 'family', - }); - console.log('deleteChannelGroup response:', response); - // snippet.end - await eventually(async () => { - const listed = await world.observer.channelGroups.listChannels({ channelGroup: world.n('family') }); - assert.deepEqual(listed.channels, []); - }, { label: 'family group is empty after deletion' }); - }); -}); diff --git a/docs-snippets/api-reference/configuration-legacy.ts b/docs-snippets/api-reference/configuration-legacy.ts deleted file mode 100644 index aa31ab3b0..000000000 --- a/docs-snippets/api-reference/configuration-legacy.ts +++ /dev/null @@ -1,139 +0,0 @@ -import assert from '../harness/assert.js'; -import PubNub from 'pubnub'; -import { check, page } from '../harness/step.js'; -import { connected, eventually } from '../harness/wait.js'; - -/** - * Examples for the deprecated configuration parameters `uuid`, `logVerbosity`, `cipherKey` and `useRandomIVs`, and for - * the deprecated `stop()` method. - */ -export default page('api-reference/configuration-legacy', 'P8', (world) => { - // Reads the keys from the env on purpose: these checks build clients with raw (deprecated) configuration objects. - const keys = (): { publishKey: string; subscribeKey: string } => { - const open = world.env.open; - assert.ok(open, 'open keyset'); - return { publishKey: open.publishKey, subscribeKey: open.subscribeKey }; - }; - - // The logger records what the SDK logs so the checks can assert deprecation warnings without parsing stdout. - /** A logger that records the levels it is asked to handle. */ - const recorder = (): { logger: PubNub.Logger; seen: Array<{ level: string; text: string }> } => { - const seen: Array<{ level: string; text: string }> = []; - const handle = (level: string) => (m: PubNub.LogMessage) => { - seen.push({ level, text: m.messageType === 'text' ? m.message : 'details' in m ? (m.details ?? '') : '' }); - }; - return { - seen, - logger: { trace: handle('trace'), debug: handle('debug'), info: handle('info'), warn: handle('warn'), error: handle('error') }, - }; - }; - - // Row `uuid`: still read when `userId` is missing, and both together are an error. - check('configurationLegacyUuid', { proves: 'The deprecated uuid option is still read when userId is missing, and passing both is an error.' }, () => { - // "The SDK reads uuid when userId is not set." - const client = world.track(new PubNub({ ...keys(), uuid: world.n('legacy-uuid') } as unknown as PubNub.PubNubConfiguration)); - assert.equal(client.getUserId(), world.n('legacy-uuid')); - // "Setting both throws a PubNubError." - assert.throws( - () => new PubNub({ ...keys(), userId: 'a', uuid: 'b' } as unknown as PubNub.PubNubConfiguration), - (error: unknown) => (error as Error).name === 'PubNubError' && /use only 'userId'/.test((error as Error).message), - ); - }); - - // Row `logVerbosity`. The logger is injected, so no real log output and no timing is involved. - check('configurationLegacyLogVerbosity', { proves: 'logVerbosity: true switches on debug logging and logs a deprecation warning, and logLevel Debug does it without the warning.' }, () => { - // "true sets logLevel to LogLevel.Debug", also when a different logLevel is passed, and logs a deprecation warning. - const { logger, seen } = recorder(); - const client = world.track( - new PubNub({ ...keys(), userId: world.n('legacy-verbosity'), logVerbosity: true, logLevel: PubNub.LogLevel.Error, loggers: [logger] } as PubNub.PubNubConfiguration), - ); - assert.ok(client); - assert.ok(seen.some((s) => s.level === 'warn' && /logVerbosity/.test(s.text) && /deprecated/.test(s.text)), `deprecation warning logged. Saw: ${JSON.stringify(seen)}`); - assert.ok(seen.some((s) => s.level === 'debug'), 'logVerbosity: true turns on debug logging although logLevel is Error'); - - // The replacement: logLevel Debug records debug entries without the deprecation warning. - const replacement = recorder(); - world.track(new PubNub({ ...keys(), userId: world.n('legacy-verbosity-new'), logLevel: PubNub.LogLevel.Debug, loggers: [replacement.logger] })); - assert.ok(replacement.seen.some((s) => s.level === 'debug')); - assert.ok(!replacement.seen.some((s) => /logVerbosity/.test(s.text))); - }); - - // Reads a stored message with a client that has no crypto, so the cipher text is visible as a string. History is - // eventually consistent after a publish, hence `eventually`. - // Ciphertext of one message, read by a client that has no crypto at all. - const rawMessage = async (channel: string, timetoken: string): Promise => { - const raw = world.fresh('legacy-raw'); - const history = await eventually(async () => { - const response = await raw.fetchMessages({ channels: [channel], count: 25 }); - const found = response.channels[channel]?.find((m) => String(m.timetoken) === timetoken); - assert.ok(found, 'message stored'); - return found; - }, { timeout: 30_000, label: 'encrypted message stored' }); - assert.equal(typeof history.message, 'string', 'an encrypted message arrives as a string for a client without crypto'); - return history.message as string; - }; - - // Row `cipherKey`. Needs persistence because the cipher text is read back from history. The PNED header (base64 - // `UE5FRA`) tells the AES-CBC format apart from the legacy one. - check('configurationLegacyCipherKey', { needs: ['preflightPersistence'], timeout: 60_000, proves: 'The deprecated cipherKey option encrypts messages in the AES-CBC format (PNED header), read back from history.' }, async () => { - const channel = world.n('legacy_cipher'); - const cipherKey = 'legacy-cipher-key'; - const legacyClient = world.fresh('legacy-cipher', { cipherKey } as Partial); - const published = await legacyClient.publish({ channel, message: { secret: 'legacy' } }); - const cipherText = await rawMessage(channel, published.timetoken); - // "Defaults to legacy 128-bit encryption." The legacy format has no PNED header, the AES-CBC format has one. - assert.ok(!cipherText.startsWith('UE5FRA'), 'cipherKey without a crypto module produces the legacy format (no PNED header)'); - // "Pass the key to cryptoModule instead": the legacy crypto module reads it back, and so does the cipherKey client. - const viaModule = world.fresh('legacy-cipher-module', { cryptoModule: PubNub.CryptoModule.legacyCryptoModule({ cipherKey }) }); - const read = await viaModule.fetchMessages({ channels: [channel], count: 25 }); - const item = read.channels[channel]?.find((m) => String(m.timetoken) === published.timetoken); - assert.deepEqual(item?.message, { secret: 'legacy' }); - - // The AES-CBC module opt-in produces the PNED format. - const aes = world.fresh('legacy-aes', { cryptoModule: PubNub.CryptoModule.aesCbcCryptoModule({ cipherKey }) }); - const aesPublished = await aes.publish({ channel, message: { secret: 'aes' } }); - assert.ok((await rawMessage(channel, aesPublished.timetoken)).startsWith('UE5FRA'), 'aesCbcCryptoModule writes the PNED header'); - }); - - // Row `useRandomIVs`. Equal cipher text for equal input is only possible with a fixed IV, which is what makes the - // comparison a proof and not a guess. - check('configurationLegacyUseRandomIVs', { needs: ['preflightPersistence'], timeout: 60_000, proves: 'useRandomIVs false gives equal cipher text for equal input and true gives different cipher text.' }, async () => { - const channel = world.n('legacy_ivs'); - const cipherKey = 'legacy-cipher-key'; - const send = async (client: PubNub): Promise => rawMessage(channel, (await client.publish({ channel, message: { same: 'message' } })).timetoken); - - // Default (true): the initialization vector is random, so the same message encrypts differently each time. - const random = world.fresh('legacy-iv-random', { cipherKey } as Partial); - const [r1, r2] = [await send(random), await send(random)]; - assert.notEqual(r1, r2, 'random IVs make the cipher text differ'); - - // useRandomIVs: false: a fixed IV, so the cipher text repeats. - const fixed = world.fresh('legacy-iv-fixed', { cipherKey, useRandomIVs: false } as Partial); - const [f1, f2] = [await send(fixed), await send(fixed)]; - assert.equal(f1, f2, 'useRandomIVs: false repeats the cipher text'); - - // The replacement: useRandomIVs on legacyCryptoModule() behaves the same. - const replacement = world.fresh('legacy-iv-module', { cryptoModule: PubNub.CryptoModule.legacyCryptoModule({ cipherKey, useRandomIVs: false }) }); - const [m1, m2] = [await send(replacement), await send(replacement)]; - assert.equal(m1, m2); - assert.equal(m1, f1, 'the module option gives the same cipher text as the deprecated parameter'); - }); - - check('configurationLegacyStop', { proves: 'stop() unsubscribes the client and logs a deprecation warning.' }, async () => { - const client = world.fresh('legacy-stop'); - const channel = world.n('ch1'); - const ready = connected(client, { channels: [channel] }); - const subscription = client.channel(channel).subscription(); - subscription.subscribe(); - await ready; - const { logger, seen } = recorder(); - const watched = world.fresh('legacy-stop-warn', { logLevel: PubNub.LogLevel.Warn, loggers: [logger] }); - - // "stop() is an alias of destroy()": it invalidates subscriptions and unsubscribes from everything. - client.stop(); - await eventually(() => assert.deepEqual(client.getSubscribedChannels(), []), { label: 'stop unsubscribes' }); - - watched.stop(); - assert.ok(seen.some((s) => s.level === 'warn' && /stop/.test(s.text) && /deprecated/.test(s.text)), `stop() logs a deprecation warning. Saw: ${JSON.stringify(seen)}`); - }); -}); diff --git a/docs-snippets/api-reference/configuration.ts b/docs-snippets/api-reference/configuration.ts deleted file mode 100644 index 0697fb496..000000000 --- a/docs-snippets/api-reference/configuration.ts +++ /dev/null @@ -1,212 +0,0 @@ -import assert from '../harness/assert.js'; -import PubNub from 'pubnub'; -import { page, step, check } from '../harness/step.js'; -import { watch } from '../harness/wait.js'; -import { eventually } from '../harness/wait.js'; - -/** - * Configuration examples. - */ -export default page('api-reference/configuration', 'P0', (world) => { - step('configurationBasicUsage', { proves: 'The full constructor sample yields a working client with the run user id, auth key, crypto module and Debug logging, and time() succeeds.' }, async () => { - const debugLog = world.captureConsole('debug'); - // snippet.configurationBasicUsage - // Initialize PubNub with your keys - const pubnubConfig = new PubNub({ - subscribeKey: 'YOUR_SUBSCRIBE_KEY', - publishKey: 'YOUR_PUBLISH_KEY', - userId: 'YOUR_USER_ID', - cryptoModule: PubNub.CryptoModule?.aesCbcCryptoModule({ cipherKey: 'YOUR_CIPHER_KEY' }), - authKey: 'accessMangerToken', - logLevel: PubNub.LogLevel.Debug, - ssl: true, - presenceTimeout: 130, - // snippet.hide - ...world.live.openKeys, - // snippet.show - }); - - // snippet.end - world.track(pubnubConfig); - try { - assert.equal(pubnubConfig.getUserId(), world.n('YOUR_USER_ID')); - assert.equal(pubnubConfig.getAuthKey(), 'accessMangerToken'); - // cryptoModule is active: encrypt and decrypt round trip. encrypt() with a crypto module takes a string and decrypt() JSON-parses the plain text, so pass stringified JSON. - assert.deepEqual(pubnubConfig.decrypt(pubnubConfig.encrypt(JSON.stringify({ greeting: 'hello' }))), { greeting: 'hello' }); - world.check.serverTime((await pubnubConfig.time()).timetoken); - } finally { - debugLog.restore(); - } - assert.ok(debugLog.lines.length > 0, 'the Debug logger wrote no output, so logLevel: Debug did not take effect'); - }); - - step('configurationServerOpertaion', { keyset: 'pam', proves: 'The server configuration sample builds a client with the secret key and the run user id, and time() succeeds.' }, async () => { - // snippet.configurationServerOpertaion - const pubnubServer = new PubNub({ - subscribeKey: 'mySubscribeKey', - publishKey: 'myPublishKey', - userId: 'myUniqueUserId', - secretKey: 'secretKey', - heartbeatInterval: 0, - // snippet.hide - ...world.live.secured, - // snippet.show - }); - // snippet.end - world.track(pubnubServer); - assert.equal(pubnubServer.getUserId(), world.n('myUniqueUserId')); - world.check.serverTime((await pubnubServer.time()).timetoken); - }); - - // The id keeps the docs' spelling ("Real"). A subscribe-only client has no publishKey, so the SDK refuses to - // publish locally. The `time()` call shows the subscribe key itself is valid. - step('configurationRealOnlyClient', { proves: 'A client with only a subscribe key works for time() and refuses to publish locally.' }, async () => { - // snippet.configurationRealOnlyClient - // Initialize for Read Only Client - - const pubnubReadOnly = new PubNub({ - subscribeKey: 'mySubscribeKey', - userId: 'myUniqueUserId', - // snippet.hide - ...world.live.openSubscribeOnly, - // snippet.show - }); - // snippet.end - world.track(pubnubReadOnly); - // Read-only means no publishKey: publishing is refused before any request is sent. - await assert.rejects(pubnubReadOnly.publish({ channel: world.n('my_channel'), message: { text: 'x' } }), (error: unknown) => { - assert.match(JSON.stringify((error as { status?: unknown }).status), /publishKey/); - return true; - }); - world.check.serverTime((await pubnubReadOnly.time()).timetoken); - }); - - step('configurationCryptoModule', { proves: 'Two clients with different crypto modules decrypt each other\'s data as the sample comments claim.' }, async () => { - // snippet.configurationCryptoModule - // encrypts using 256-bit AES-CBC cipher (recommended) - // decrypts data encrypted with the legacy and the 256 bit AES-CBC ciphers - const pubnub = new PubNub({ - subscribeKey: 'YOUR_SUBSCRIBE_KEY', - userId: 'YOUR_USER_ID', - cryptoModule: PubNub.CryptoModule.aesCbcCryptoModule({ cipherKey: 'pubnubenigma' }), - // snippet.hide - ...world.live.openSubscribeOnly, - // snippet.show - }); - - // encrypts with 128-bit cipher key entropy (legacy) - // decrypts data encrypted with the legacy and the 256-bit AES-CBC ciphers - const pubnubLegacy = new PubNub({ - subscribeKey: 'YOUR_SUBSCRIBE_KEY', - userId: 'YOUR_USER_ID', - cryptoModule: PubNub.CryptoModule.legacyCryptoModule({ cipherKey: 'pubnubenigma' }), - // snippet.hide - ...world.live.openSubscribeOnly, - // snippet.show - }); - // snippet.end - world.track(pubnub); - world.track(pubnubLegacy); - const payload = JSON.stringify({ note: 'same input' }); - assert.deepEqual(pubnub.decrypt(pubnubLegacy.encrypt(payload)), { note: 'same input' }); - assert.deepEqual(pubnubLegacy.decrypt(pubnub.encrypt(payload)), { note: 'same input' }); - // The two modules produce different ciphertext (random IVs, different headers), so the choice is applied. - assert.notEqual(pubnub.encrypt(payload), pubnubLegacy.encrypt(payload)); - }); - - step('setUserIdBasicUsage', { proves: 'setUserId renames the user and the client keeps working.' }, async () => { - const pubnub = world.fresh('setuserid'); - // snippet.setUserIdBasicUsage - pubnub.setUserId('myUniqueUserId'); - // snippet.end - assert.equal(pubnub.getUserId(), world.n('myUniqueUserId')); - world.check.serverTime((await pubnub.time()).timetoken); - }); - - step('getUserIdBasicUsage', { proves: 'getUserId returns the user id given at construction, including the run suffix.' }, () => { - const pubnub = world.fresh('getuserid'); - // snippet.getUserIdBasicUsage - const currentUserId = pubnub.getUserId(); - // snippet.end - assert.equal(currentUserId, pubnub.userId); - assert.ok(currentUserId.endsWith(`-r${world.runId}`), 'user id carries the run suffix'); - }); - - // Local state only: the auth key matters when Access Manager is on, which is covered by the access manager modules. - step('setAuthKeyBasicUsage', { proves: 'setAuthKey stores the auth key on the client.' }, () => { - const pubnub = world.fresh('setauthkey'); - // snippet.setAuthKeyBasicUsage - pubnub.setAuthKey('my_authkey'); - // snippet.end - assert.equal(pubnub.getAuthKey(), world.n('my_authkey')); - }); - - step('getAuthKeyBasicUsage', { proves: 'getAuthKey returns the auth key that was set.' }, () => { - const pubnub = world.fresh('getauthkey'); - pubnub.setAuthKey('my_authkey'); - // snippet.getAuthKeyBasicUsage - const currentAuthKey = pubnub.getAuthKey(); - // snippet.end - assert.equal(currentAuthKey, 'my_authkey'); - }); - - // Local check of the stored expression. The wire effect is proven by the check `filterExpressionDropsMessage` below. - // The older docs literal `such=wow` is not valid filter syntax: a live probe (2026-11-03) showed the server delivers - // no message at all for it, while `such == 'wow'` delivers only the matching one. - step('setFilterExpressionBasicUsage', { proves: 'setFilterExpression stores the filter expression on the client.' }, () => { - const pubnub = world.fresh('setfilter'); - // snippet.setFilterExpressionBasicUsage - pubnub.setFilterExpression("such == 'wow'"); - // snippet.end - assert.equal(pubnub.getFilterExpression(), "such == 'wow'"); - }); - - step('getFilterExpressionBasicUsage', { proves: 'getFilterExpression returns the expression that was set.' }, () => { - const pubnub = world.fresh('getfilter'); - pubnub.setFilterExpression("such == 'wow'"); - const spy = world.spy(pubnub, 'getFilterExpression'); - // snippet.getFilterExpressionBasicUsage - pubnub.getFilterExpression(); - // snippet.end - assert.deepEqual(spy.results, ["such == 'wow'"]); - }); - - step('destroyBasicUsage', { phase: 'P9', proves: 'destroy() stops a subscribed client and clears its subscriptions.' }, async () => { - const pubnub = world.fresh('destroy'); - const channel = world.n('my_channel'); - await watch(pubnub, [channel]); - assert.ok(pubnub.getSubscribedChannels().includes(channel), 'subscribed before destroy'); - // snippet.destroyBasicUsage - pubnub.destroy(); - // snippet.end - await eventually(() => assert.deepEqual(pubnub.getSubscribedChannels(), []), { label: 'destroy unsubscribes' }); - }); - - check('configurationInitializationErrors', { proves: 'Each documented constructor error is thrown for the matching bad configuration.' }, () => { - const base = { subscribeKey: world.live.openSubscribeOnly.subscribeKey as string }; - assert.throws(() => new PubNub({ ...base } as unknown as PubNub.PubNubConfiguration), /'userId' not set/); - assert.throws(() => new PubNub({ ...base, userId: ' ' }), /'userId' is empty/); - assert.throws( - () => new PubNub({ ...base, userId: 'a', uuid: 'b' } as unknown as PubNub.PubNubConfiguration), - /use only 'userId'/, - ); - assert.throws(() => new PubNub({ ...base, userId: '' }), /'userId' not set/); - }); - - check('filterExpressionDropsMessage', { phase: 'P4', timeout: 45_000, proves: 'A subscribe filter expression makes the server drop messages that do not match, observed on the wire.' }, async () => { - const channel = world.n('filter_channel'); - const subscriber = world.fresh('filtered'); - subscriber.setFilterExpression("such == 'wow'"); - const listening = await watch(subscriber, [channel]); - await world.friend.publish({ channel, message: { n: 1 }, meta: { such: 'nope' } }); - await world.friend.publish({ channel, message: { n: 2 }, meta: { such: 'wow' } }); - const delivered = await listening.next('message', (m) => (m.message as { n?: number }).n === 2); - assert.equal((delivered.userMetadata as { such?: string }).such, 'wow'); - assert.equal( - listening.events.filter((e) => e.kind === 'message').length, - 1, - 'the message with meta such=nope must be dropped by the filter', - ); - listening.stop(); - }); -}); diff --git a/docs-snippets/api-reference/datasync.ts b/docs-snippets/api-reference/datasync.ts deleted file mode 100644 index 7daf969e1..000000000 --- a/docs-snippets/api-reference/datasync.ts +++ /dev/null @@ -1,1306 +0,0 @@ -import assert from '../harness/assert.js'; -import type PubNub from 'pubnub'; -import { page, step, check, skipStep } from '../harness/step.js'; -import { connected, eventually } from '../harness/wait.js'; -import { filterFragments, patchFragments, sortFragments } from '../fixtures/datasync-fragments.js'; - -/** - * DataSync examples, including the DataSync listener and subscription examples. - */ -export default page('api-reference/datasync', 'P2', (world) => { - const secured = { keyset: 'datasync' as const }; - const api = (): PubNub['dataSync'] => world.server.dataSync; - - // ---- helpers ----------------------------------------------------------------------------------------------- - - // The SDK error carries the service answer in `status`. Assertions match on this one string so a changed status - // code or DS- error code shows up in the failure message. - const errorText = (error: unknown): string => { - const status = (error as { status?: { errorData?: unknown; statusCode?: number } } | undefined)?.status; - return `${status?.statusCode ?? ''} ${JSON.stringify(status?.errorData ?? {})}`; - }; - const missingClass = (error: unknown): boolean => /404/.test(errorText(error)) && /DS-0100/.test(errorText(error)); - - let productClass: Promise | undefined; - let ownerClass: Promise | undefined; - const hasProductClass = (): Promise => - (productClass ??= api() - .getEntities({ class: 'product', limit: 1 }) - .then(() => true, (error) => (missingClass(error) ? false : Promise.reject(error)))); - const hasOwnerClass = (): Promise => - (ownerClass ??= api() - .getRelationships({ class: 'ProductOwner', limit: 1 }) - .then(() => true, (error) => (missingClass(error) ? false : Promise.reject(error)))); - - type EventKind = 'user' | 'channel' | 'membership' | 'entity' | 'relationship'; - const EVENT_LABEL: Record = { - user: 'User v1', - channel: 'Channel v1', - membership: 'Membership v1', - entity: 'product v1', - relationship: 'ProductOwner v1', - }; - /** How long the probe waits for the create events of its own writes. */ - const PROBE_WINDOW_MS = 20_000; - let eventProbe: Promise> | undefined; - - /** Creates one object of each kind while listening, and reports the kinds that produced a create event. */ - async function probeEvents(): Promise> { - const client = world.server; - const found = new Set(); - const [userId, channelId, productId] = [world.n('ds-probe-user'), world.n('ds-probe-channel'), world.n('ds-probe-product')]; - const subscriptions = [ - client.dataSyncUser(userId).subscription(), - client.dataSyncChannel(channelId).subscription(), - client.dataSyncEntity(productId).subscription(), - ]; - for (const subscription of subscriptions) { - subscription.onDataSync = (event) => { - if (event.message.event === 'create') found.add(event.message.objectType as EventKind); - }; - } - const ready = connected(client, { channels: [userId, channelId, productId] }); - subscriptions.forEach((s) => s.subscribe()); - await ready; - const withProduct = await hasProductClass(); - const withOwner = withProduct && (await hasOwnerClass()); - const expected = new Set(['user', 'channel', 'membership']); - if (withProduct) expected.add('entity'); - if (withOwner) expected.add('relationship'); - const dataSync = client.dataSync; - await dataSync.createUser({ id: userId, data: { classVersion: 1, payload: { name: 'Probe' } } }); - await dataSync.createChannel({ id: channelId, data: { classVersion: 1, payload: { name: 'Probe' } } }); - await dataSync.createMembership({ userId, channelId, data: { classVersion: 1 } }); - if (withProduct) { - await dataSync.createEntity({ id: productId, class: 'product', data: { classVersion: 1, payload: { name: 'Probe', price: 1 } } }); - } - if (withOwner) { - await dataSync.createRelationship({ class: 'ProductOwner', entityAId: userId, entityBId: productId, data: { classVersion: 1 } }); - } - try { - await eventually(() => assert.ok([...expected].every((kind) => found.has(kind))), { timeout: PROBE_WINDOW_MS, interval: 500, label: 'probe events' }); - } catch { - // Whatever arrived is what is switched on. - } - for (const subscription of subscriptions) subscription.unsubscribe(); - // Deleting an entity removes its relationships, deleting a user or channel removes its memberships. - if (withProduct) await dataSync.removeEntity({ id: productId }).catch(() => undefined); - await dataSync.removeUser({ id: userId }).catch(() => undefined); - await dataSync.removeChannel({ id: channelId }).catch(() => undefined); - return found; - } - const events = (): Promise> => (eventProbe ??= probeEvents()); - - /** The kinds among `kinds` whose create event the probe did not see. Empty when every needed event rule is on. */ - async function offKinds(...kinds: EventKind[]): Promise { - const on = await events(); - return kinds.filter((kind) => !on.has(kind)); - } - - function skipUnlessEvents(off: EventKind[]): void { - if (off.length > 0) { - skipStep( - 'needs-feature:datasync-events', - `subscription connected, but no create event arrived within ${PROBE_WINDOW_MS / 1000} s for ${off.map((k) => EVENT_LABEL[k]).join(', ')}, ` + - `so event delivery was not verified. Switch the event rule on for each in the Admin Portal, DataSync tab, then rerun.`, - ); - } - } - - const user = (): string => world.n('user-alice'); - const channel = (): string => world.n('channel-summer-sale'); - const membership = (): string => world.n('membership-alice-summer-sale'); - const product = (): string => world.n('product-sneaker-42'); - const seller = (): string => world.n('seller-bob'); - const relationship = (): string => world.n('rel-bob-owns-sneaker-42'); - - const ids = (response: { data: Array<{ id: string }> }): string[] => response.data.map((o) => o.id); - - // ---- preconditions ----------------------------------------------------------------------------------------- - - check('datasyncClassProduct', { ...secured, proves: 'Precondition: the product class exists on the keyset, otherwise dependent DataSync steps skip.' }, async () => { - if (!(await hasProductClass())) { - skipStep('blocked-by:datasync-class-missing (product v1), run provision/provision-datasync.ts --apply after approval'); - } - const klass = await api().getEntities({ class: 'product', classVersion: 1, limit: 1 }); - assert.equal(klass.status, 200); - }); - - // Same for the `ProductOwner` relationship class. `needs` the product class because a relationship links entities. - check('datasyncClassProductOwner', { ...secured, needs: ['datasyncClassProduct'], proves: 'Precondition: the ProductOwner relationship class exists on the keyset.' }, async () => { - if (!(await hasOwnerClass())) { - skipStep('blocked-by:datasync-class-missing (ProductOwner v1), run provision/provision-datasync.ts --apply after approval'); - } - }); - - check('datasyncEvents', { ...secured, timeout: 90_000, proves: 'Probes which DataSync object kinds publish events, so event steps can skip when event rules are off.' }, async () => { - const on = await events(); - console.log(`DataSync event rules on: ${[...on].map((k) => EVENT_LABEL[k]).join(', ') || 'none'}`); - }); - - // ---- users ------------------------------------------------------------------------------------------------- - - step('createUserBasicUsage', { ...secured, proves: 'createUser creates Alice, echoes the payload and returns an eTag.' }, async () => { - const pubnub = world.server; - // snippet.createUserBasicUsage - const response = await pubnub.dataSync.createUser({ - id: 'user-alice', - data: { - classVersion: 1, - payload: { name: 'Alice', type: 'shopper' }, - }, - }); - console.log('createUser response:', response.data); - // snippet.end - assert.ok(response.status >= 200 && response.status < 300, `status ${response.status}`); - assert.equal(response.data.id, user()); - assert.equal(response.data.entityClassVersion, 1); - assert.deepEqual(response.data.payload, { name: 'Alice', type: 'shopper' }); - assert.ok(response.data.eTag, 'response carries an eTag'); - }); - - // Reads Alice back by id. - step('getUserBasicUsage', { ...secured, needs: ['createUserBasicUsage'], proves: 'getUser reads Alice back by id.' }, async () => { - const pubnub = world.server; - // snippet.getUserBasicUsage - const response = await pubnub.dataSync.getUser({ id: 'user-alice' }); - console.log('getUser response:', response.data); - // snippet.end - assert.equal(response.data.id, user()); - assert.equal(response.data.payload?.name, 'Alice'); - }); - - step('getUsersBasicUsage', { ...secured, needs: ['createUserBasicUsage'], proves: 'getUsers lists users sorted by createdAt, verified on the returned timestamps.' }, async () => { - const pubnub = world.server; - // snippet.getUsersBasicUsage - const response = await pubnub.dataSync.getUsers({ - limit: 20, - sort: { createdAt: 'asc' }, - }); - console.log('getUsers response:', response.data); - console.log('getUsers meta:', response.meta); - // snippet.end - assert.ok(Array.isArray(response.data)); - assert.ok(response.data.length <= 20); - assert.equal(typeof response.meta?.has_next, 'boolean'); - const created = response.data.map((u) => Date.parse(u.createdAt)); - assert.deepEqual(created, [...created].sort((a, b) => a - b), 'sorted by createdAt ascending'); - }); - - step('getUsersFilterExample', { ...secured, needs: ['createUserBasicUsage'], timeout: 60_000, proves: 'getUsers with filter finds Alice once the eventually consistent index has caught up.' }, async () => { - const pubnub = world.server; - // `filter` is eventually consistent, so wait until the service indexed the user before running the sample. - await eventually(async () => { - assert.ok(ids(await api().getUsers({ filter: 'name LIKE "*Alice*"' })).includes(user())); - }, { timeout: 45_000, label: 'filter index caught up with the new user' }); - // snippet.getUsersFilterExample - // `filter` runs over properties declared with filtering mode `full` and is evaluated against - // eventually consistent storage. - const response = await pubnub.dataSync.getUsers({ - filter: 'name LIKE "*Alice*"', - }); - console.log('getUsers (filter) response:', response.data); - // snippet.end - assert.ok(ids(response).includes(user()), 'the filter finds Alice'); - }); - - // `filterFast` is strongly consistent, so no wait is needed right after the create. - step('getUsersFilterFastExample', { ...secured, needs: ['createUserBasicUsage'], proves: 'getUsers with filterFast finds Alice at once.' }, async () => { - const pubnub = world.server; - // snippet.getUsersFilterFastExample - // `filterFast` is evaluated against strongly consistent storage, so it matches an object you just - // wrote. - const response = await pubnub.dataSync.getUsers({ - filterFast: 'type == "shopper"', - }); - console.log('getUsers (filterFast) response:', response.data); - // snippet.end - assert.ok(ids(response).includes(user()), 'filterFast sees the user right after the write'); - }); - - step('getUsersCursorPagingExample', { ...secured, needs: ['createUserBasicUsage'], proves: 'The cursor paging loop for users runs and lists Alice.' }, async () => { - const pubnub = world.server; - await world.withConsole(async (logs) => { - // snippet.getUsersCursorPagingExample - // Pass no `cursor` on the first call. Take `meta.next_cursor` from the response and pass it back - // as `cursor` on the next call. Stop when `meta.has_next` is `false`. - let cursor: string | undefined; - let hasNext = true; - let page = 0; - - while (hasNext) { - const response = await pubnub.dataSync.getUsers({ - filterFast: 'type == "shopper"', - limit: 20, - cursor, - }); - console.log(`Page ${++page}:`, response.data); - cursor = response.meta?.next_cursor ?? undefined; - hasNext = response.meta?.has_next ?? false; - } - // snippet.end - assert.ok(logs.lines.some((line) => line.startsWith('Page 1:')), 'the loop ran at least once'); - assert.ok(logs.lines.some((line) => line.includes(user())), 'a page lists Alice'); - }); - }); - - step('updateUserBasicUsage', { ...secured, needs: ['createUserBasicUsage'], proves: 'setUser with ifMatchesEtag replaces Alice and a stale eTag is rejected.' }, async () => { - const pubnub = world.server; - // The sample shows an example eTag. Use the real one, otherwise the guard (correctly) rejects the write. - world.setPh('userEtag', (await api().getUser({ id: user() })).data.eTag); - // snippet.updateUserBasicUsage - const response = await pubnub.dataSync.setUser({ - id: 'user-alice', - data: { - classVersion: 1, - payload: { name: 'Alice B.', type: 'shopper' }, - }, - ifMatchesEtag: 'AbQdEfGhIjKlMn', - }); - console.log('setUser response:', response.data); - // snippet.end - assert.equal(response.data.payload?.name, 'Alice B.'); - assert.notEqual(response.data.eTag, world.ph('userEtag'), 'the write produced a new eTag'); - await assert.rejects( - () => api().setUser({ id: user(), data: { classVersion: 1, payload: { name: 'stale' } }, ifMatchesEtag: world.ph('userEtag') }), - (error: unknown) => /\b(412|403)\b/.test(errorText(error)), - 'the old eTag is now stale', - ); - }); - - step('patchUserBasicUsage', { ...secured, needs: ['updateUserBasicUsage'], proves: 'updateUser applies a patch to the user name.' }, async () => { - const pubnub = world.server; - // snippet.patchUserBasicUsage - const response = await pubnub.dataSync.updateUser({ - id: 'user-alice', - replace: { '/payload/name': 'Alice B.', '/status': 'active' }, - }); - console.log('updateUser response:', response.data); - // snippet.end - assert.equal(response.data.payload?.name, 'Alice B.'); - assert.equal(response.data.status, 'active'); - }); - - // ---- channels ---------------------------------------------------------------------------------------------- - - step('createDataSyncChannelBasicUsage', { ...secured, proves: 'createChannel creates the drop channel and echoes the payload.' }, async () => { - const pubnub = world.server; - // snippet.createDataSyncChannelBasicUsage - const response = await pubnub.dataSync.createChannel({ - id: 'channel-summer-sale', - data: { - classVersion: 1, - payload: { name: 'Summer Sale', type: 'promotion' }, - }, - }); - console.log('createChannel response:', response.data); - // snippet.end - assert.equal(response.data.id, channel()); - assert.deepEqual(response.data.payload, { name: 'Summer Sale', type: 'promotion' }); - }); - - // Reads the channel back. - step('getChannelBasicUsage', { ...secured, needs: ['createDataSyncChannelBasicUsage'], proves: 'getChannel reads the DataSync channel back by id.' }, async () => { - const pubnub = world.server; - // snippet.getChannelBasicUsage - const response = await pubnub.dataSync.getChannel({ id: 'channel-summer-sale' }); - console.log('getChannel response:', response.data); - // snippet.end - assert.equal(response.data.id, channel()); - assert.equal(response.data.payload?.name, 'Summer Sale'); - }); - - step('getChannelsBasicUsage', { ...secured, needs: ['createDataSyncChannelBasicUsage'], proves: 'getChannels lists channels with the data array and meta.has_next envelope.' }, async () => { - const pubnub = world.server; - // snippet.getChannelsBasicUsage - const response = await pubnub.dataSync.getChannels({ limit: 20 }); - console.log('getChannels response:', response.data); - console.log('getChannels meta:', response.meta); - // snippet.end - assert.ok(Array.isArray(response.data)); - assert.equal(typeof response.meta?.has_next, 'boolean'); - }); - - step('getChannelsFilterExample', { ...secured, needs: ['createDataSyncChannelBasicUsage'], timeout: 60_000, proves: 'getChannels with filter finds the new channel once the index has caught up.' }, async () => { - const pubnub = world.server; - await eventually(async () => { - assert.ok(ids(await api().getChannels({ filter: 'name LIKE "*Sale*"' })).includes(channel())); - }, { timeout: 45_000, label: 'filter index caught up with the new channel' }); - // snippet.getChannelsFilterExample - const response = await pubnub.dataSync.getChannels({ - filter: 'name LIKE "*Sale*"', - }); - console.log('getChannels (filter) response:', response.data); - // snippet.end - assert.ok(ids(response).includes(channel())); - }); - - // Strongly consistent filter, no wait. - step('getChannelsFilterFastExample', { ...secured, needs: ['createDataSyncChannelBasicUsage'], proves: 'getChannels with filterFast finds the channel at once.' }, async () => { - const pubnub = world.server; - // snippet.getChannelsFilterFastExample - const response = await pubnub.dataSync.getChannels({ - filterFast: 'type == "promotion"', - }); - console.log('getChannels (filterFast) response:', response.data); - // snippet.end - assert.ok(ids(response).includes(channel())); - }); - - // Cursor loop with console capture, same pattern as for users. - step('getChannelsCursorPagingExample', { ...secured, needs: ['createDataSyncChannelBasicUsage'], proves: 'The cursor paging loop for channels runs and lists the channel.' }, async () => { - const pubnub = world.server; - await world.withConsole(async (logs) => { - // snippet.getChannelsCursorPagingExample - let cursor: string | undefined; - let hasNext = true; - let page = 0; - - while (hasNext) { - const response = await pubnub.dataSync.getChannels({ - filterFast: 'type == "promotion"', - limit: 20, - cursor, - }); - console.log(`Page ${++page}:`, response.data); - cursor = response.meta?.next_cursor ?? undefined; - hasNext = response.meta?.has_next ?? false; - } - // snippet.end - assert.ok(logs.lines.some((line) => line.includes(channel())), 'a page lists the channel'); - }); - }); - - // Full replace with `setChannel`. - step('updateChannelBasicUsage', { ...secured, needs: ['createDataSyncChannelBasicUsage'], proves: 'setChannel replaces the channel payload.' }, async () => { - const pubnub = world.server; - // snippet.updateChannelBasicUsage - const response = await pubnub.dataSync.setChannel({ - id: 'channel-summer-sale', - data: { - classVersion: 1, - payload: { name: 'Summer Sale 2026', type: 'promotion' }, - }, - }); - console.log('setChannel response:', response.data); - // snippet.end - assert.equal(response.data.payload?.name, 'Summer Sale 2026'); - }); - - // Patches one field and checks the other payload fields survive. - step('patchChannelBasicUsage', { ...secured, needs: ['updateChannelBasicUsage'], proves: 'updateChannel patches one field and the other payload fields survive.' }, async () => { - const pubnub = world.server; - // snippet.patchChannelBasicUsage - const response = await pubnub.dataSync.updateChannel({ - id: 'channel-summer-sale', - replace: { '/payload/name': 'Summer Sale 2026' }, - }); - console.log('updateChannel response:', response.data); - // snippet.end - assert.equal(response.data.payload?.name, 'Summer Sale 2026'); - assert.equal(response.data.payload?.type, 'promotion', 'the patch leaves other fields alone'); - }); - - // ---- memberships ------------------------------------------------------------------------------------------- - - // Creates Alice's membership with the fixed id that the later samples read, set, patch and remove. - step('createMembershipBasicUsage', { ...secured, needs: ['createUserBasicUsage', 'createDataSyncChannelBasicUsage'], proves: 'createMembership links Alice to the channel with the fixed id that later steps use.' }, async () => { - const pubnub = world.server; - // snippet.createMembershipBasicUsage - const response = await pubnub.dataSync.createMembership({ - id: 'membership-alice-summer-sale', - userId: 'user-alice', - channelId: 'channel-summer-sale', - data: { - classVersion: 1, - payload: { role: 'viewer' }, - }, - }); - console.log('createMembership response:', response.data); - // snippet.end - assert.equal(response.data.id, membership()); - assert.equal(response.data.userId, user()); - assert.equal(response.data.channelId, channel()); - assert.equal(response.data.relationshipClass, 'Membership'); - assert.deepEqual(response.data.payload, { role: 'viewer' }); - }); - - step('getDataSyncMembershipBasicUsage', { ...secured, needs: ['createMembershipBasicUsage'], proves: 'getMembership reads the membership by its fixed id.' }, async () => { - const pubnub = world.server; - // snippet.getDataSyncMembershipBasicUsage - const response = await pubnub.dataSync.getMembership({ - id: 'membership-alice-summer-sale', - }); - console.log('getMembership response:', response.data); - // snippet.end - assert.equal(response.data.id, membership()); - assert.equal(response.data.payload?.role, 'viewer'); - }); - - // Lists Alice's memberships. Expects exactly the one from the fixture. - step('getMembershipsBasicUsage', { ...secured, needs: ['createMembershipBasicUsage'], proves: 'getMemberships lists exactly the membership of Alice.' }, async () => { - const pubnub = world.server; - // snippet.getMembershipsBasicUsage - const response = await pubnub.dataSync.getMemberships({ - userId: 'user-alice', - limit: 20, - }); - console.log('getMemberships response:', response.data); - console.log('getMemberships meta:', response.meta); - // snippet.end - assert.deepEqual(ids(response), [membership()]); - assert.equal(response.data[0].channelId, channel()); - }); - - // Same list from the channel side, expects Alice as the only member. - step('getMembershipsByChannelIdExample', { ...secured, needs: ['createMembershipBasicUsage'], proves: 'getMemberships by channel id lists Alice as the only member.' }, async () => { - const pubnub = world.server; - // snippet.getMembershipsByChannelIdExample - // Pass `channelId` instead of `userId` to list the members of a channel rather than a user's - // memberships. - const response = await pubnub.dataSync.getMemberships({ - channelId: 'channel-summer-sale', - limit: 20, - }); - console.log('getMemberships (channelId) response:', response.data); - // snippet.end - assert.deepEqual(response.data.map((m) => m.userId), [user()]); - }); - - // ---- membership filters -------------------------------------------------------------------------------------- - // - // The Global Membership class declares no payload properties and cannot be extended, so a membership `payload` - // is never indexed and filters cannot reach it. What a membership offers are the built-in fields: `id`, - // `createdAt`, `updatedAt`, `status`, `createdBy` and `modifiedBy`. Of these `status` is the one an application - // sets itself (a free-form lifecycle string), which makes it the natural field to filter memberships on. Both - // filter tiers accept the built-in fields without any class declaration (probed live: both answer on a fresh write). - // - // Both samples need two memberships of Alice, one that matches and one that does not. The memberships created by - // `createMembershipBasicUsage` carry no `status`. The helper below adds a second membership of Alice, on a second - // channel, with `status: 'archived'`. The two samples then have to return exactly that one and leave the - // original membership out. - - /** Creates the channel `channel-winter-sale` and Alice's archived membership on it. Returns the cleanup. */ - async function seedArchivedMembership(): Promise<{ id: string; cleanup: () => Promise }> { - const channelId = world.n('channel-winter-sale'); - const id = world.n('membership-alice-winter-sale'); - await api().createChannel({ id: channelId, data: { classVersion: 1, payload: { name: 'Winter Sale 2026' } } }); - await api().createMembership({ id, userId: user(), channelId, data: { classVersion: 1, status: 'archived' } }); - // Removing the channel also removes the memberships that point at it. - return { id, cleanup: () => api().removeChannel({ id: channelId }).then(() => undefined) }; - } - - // Seeds a second, archived membership (see the section comment) so the filter has something to exclude, waits for - // the eventually consistent index, then always cleans the seed in `finally`. - step('getMembershipsFilterExample', { ...secured, needs: ['createMembershipBasicUsage'], timeout: 60_000, proves: 'getMemberships with filter excludes a seeded archived membership once the index has caught up.' }, async () => { - const pubnub = world.server; - const seeded = await seedArchivedMembership(); - try { - // `filter` is eventually consistent, so wait until the service indexed the new membership. - await eventually(async () => { - assert.ok(ids(await api().getMemberships({ userId: user(), filter: 'status LIKE "arch*"' })).includes(seeded.id)); - }, { timeout: 45_000, label: 'filter index caught up with the new membership' }); - // snippet.getMembershipsFilterExample - // `status` is a built-in field, so it is filterable without any declaration. The Membership class - // declares no `payload` properties, which means a filter cannot reference anything inside `payload`. - // `LIKE` takes `*` as a wildcard. This call returns Alice's memberships whose status starts with `arch`. - const response = await pubnub.dataSync.getMemberships({ - userId: 'user-alice', - filter: 'status LIKE "arch*"', - }); - console.log('getMemberships (filter) response:', response.data); - // snippet.end - assert.deepEqual(ids(response), [seeded.id], 'only the archived membership matches'); - assert.ok(!ids(response).includes(membership()), 'the membership without a status is filtered out'); - const unfiltered = await api().getMemberships({ userId: user() }); - assert.ok(ids(unfiltered).length >= 2 && ids(unfiltered).includes(membership()), 'without the filter both memberships are listed'); - } finally { - await seeded.cleanup(); - } - }); - - // Same seed, strongly consistent filter, so no wait. - step('getMembershipsFilterFastExample', { ...secured, needs: ['createMembershipBasicUsage'], proves: 'getMemberships with filterFast excludes the seeded archived membership at once.' }, async () => { - const pubnub = world.server; - const seeded = await seedArchivedMembership(); - try { - // snippet.getMembershipsFilterFastExample - // `filterFast` is evaluated against strongly consistent storage, so it matches a membership you just - // wrote. It accepts the same built-in fields. This call returns Alice's archived memberships. - const response = await pubnub.dataSync.getMemberships({ - userId: 'user-alice', - filterFast: 'status == "archived"', - }); - console.log('getMemberships (filterFast) response:', response.data); - // snippet.end - assert.deepEqual(ids(response), [seeded.id], 'only the archived membership matches, right after the write'); - assert.equal(response.data[0].status, 'archived'); - assert.ok(!ids(response).includes(membership()), 'the membership without a status is filtered out'); - } finally { - await seeded.cleanup(); - } - }); - - // Documents the limit the two samples above work around: a field of the membership `payload` is not filterable. - check('datasyncMembershipFilterRejected', { ...secured, proves: 'Filtering on a field of the membership payload is refused, which is the limit the filter samples work around.' }, async () => { - for (const parameters of [{ filter: 'role LIKE "*mod*"' }, { filterFast: 'role == "viewer"' }]) { - await assert.rejects( - () => api().getMemberships({ userId: user(), ...parameters }), - (error: unknown) => /400/.test(errorText(error)) && /DS-1001/.test(errorText(error)) && /Unknown field: role/.test(errorText(error)), - `the service rejects ${JSON.stringify(parameters)}`, - ); - } - }); - - // Cursor loop over Alice's memberships with console capture. - step('getMembershipsCursorPagingExample', { ...secured, needs: ['createMembershipBasicUsage'], proves: 'The cursor paging loop for memberships runs and lists the membership.' }, async () => { - const pubnub = world.server; - await world.withConsole(async (logs) => { - // snippet.getMembershipsCursorPagingExample - let cursor: string | undefined; - let hasNext = true; - let page = 0; - - while (hasNext) { - const response = await pubnub.dataSync.getMemberships({ - userId: 'user-alice', - limit: 20, - cursor, - }); - console.log(`Page ${++page}:`, response.data); - cursor = response.meta?.next_cursor ?? undefined; - hasNext = response.meta?.has_next ?? false; - } - // snippet.end - assert.ok(logs.lines.some((line) => line.includes(membership())), 'a page lists the membership'); - }); - }); - - // Full replace of the membership payload. - step('updateMembershipBasicUsage', { ...secured, needs: ['createMembershipBasicUsage'], proves: 'setMembership replaces the membership payload.' }, async () => { - const pubnub = world.server; - // snippet.updateMembershipBasicUsage - const response = await pubnub.dataSync.setMembership({ - id: 'membership-alice-summer-sale', - userId: 'user-alice', - channelId: 'channel-summer-sale', - data: { - classVersion: 1, - payload: { role: 'moderator' }, - }, - }); - console.log('setMembership response:', response.data); - // snippet.end - assert.equal(response.data.payload?.role, 'moderator'); - }); - - // Patch of one payload field. `needs` the update so the starting value is known. - step('patchMembershipBasicUsage', { ...secured, needs: ['updateMembershipBasicUsage'], proves: 'updateMembership patches one payload field.' }, async () => { - const pubnub = world.server; - // snippet.patchMembershipBasicUsage - const response = await pubnub.dataSync.updateMembership({ - id: 'membership-alice-summer-sale', - replace: { '/payload/role': 'moderator' }, - }); - console.log('updateMembership response:', response.data); - // snippet.end - assert.equal(response.data.payload?.role, 'moderator'); - }); - - // ---- entities (class `product`) ---------------------------------------------------------------------------- - - const entity = { ...secured, needs: ['datasyncClassProduct', 'createEntityBasicUsage'] }; - - step('createEntityBasicUsage', { ...secured, needs: ['datasyncClassProduct'], proves: 'createEntity creates the product entity that later steps use.' }, async () => { - const pubnub = world.server; - // snippet.createEntityBasicUsage - const response = await pubnub.dataSync.createEntity({ - id: 'product-sneaker-42', - class: 'product', - data: { - classVersion: 1, - payload: { name: 'Retro Sneaker', price: 89.99, stock: 12 }, - }, - }); - console.log('createEntity response:', response.data); - // snippet.end - assert.equal(response.data.id, product()); - assert.equal(response.data.entityClass, 'product'); - assert.deepEqual(response.data.payload, { name: 'Retro Sneaker', price: 89.99, stock: 12 }); - }); - - // Reads the product by id. - step('getEntityBasicUsage', { ...entity, proves: 'getEntity reads the product by id.' }, async () => { - const pubnub = world.server; - // snippet.getEntityBasicUsage - const response = await pubnub.dataSync.getEntity({ id: 'product-sneaker-42' }); - console.log('getEntity response:', response.data); - // snippet.end - assert.equal(response.data.id, product()); - assert.equal(response.data.payload?.price, 89.99); - }); - - // Lists products sorted by price descending, verified on the returned prices. - step('getEntitiesBasicUsage', { ...entity, proves: 'getEntities lists products sorted by price descending, verified on the returned prices.' }, async () => { - const pubnub = world.server; - // snippet.getEntitiesBasicUsage - const response = await pubnub.dataSync.getEntities({ - class: 'product', - sort: { price: 'desc' }, - limit: 20, - }); - console.log('getEntities response:', response.data); - console.log('getEntities meta:', response.meta); - // snippet.end - assert.ok(ids(response).includes(product())); - const prices = response.data.map((e) => Number(e.payload?.price)); - assert.deepEqual(prices, [...prices].sort((a, b) => b - a), 'sorted by price descending'); - }); - - step('getEntitiesFilterExample', { ...entity, timeout: 60_000, proves: 'getEntities with filter finds the product once the index has caught up.' }, async () => { - const pubnub = world.server; - await eventually(async () => { - const found = await api().getEntities({ class: 'product', filter: 'name LIKE "*sneaker*" && !(status == "discontinued")' }); - assert.ok(ids(found).includes(product())); - }, { timeout: 45_000, label: 'filter index caught up with the new product' }); - // snippet.getEntitiesFilterExample - // Reference a declared property by its `name` (not its `path`), and combine conditions with `&&` - // and `||`. - const response = await pubnub.dataSync.getEntities({ - class: 'product', - filter: 'name LIKE "*sneaker*" && !(status == "discontinued")', - }); - console.log('getEntities (filter) response:', response.data); - // snippet.end - assert.ok(ids(response).includes(product())); - }); - - // Strongly consistent filter on declared properties, no wait. - step('getEntitiesFilterFastExample', { ...entity, proves: 'getEntities with filterFast finds the product at once.' }, async () => { - const pubnub = world.server; - // snippet.getEntitiesFilterFastExample - const response = await pubnub.dataSync.getEntities({ - class: 'product', - filterFast: 'price < 100 && stock > 0', - }); - console.log('getEntities (filterFast) response:', response.data); - // snippet.end - assert.ok(ids(response).includes(product())); - }); - - // Cursor loop over products with console capture. - step('getEntitiesCursorPagingExample', { ...entity, proves: 'The cursor paging loop for products runs and lists the product.' }, async () => { - const pubnub = world.server; - await world.withConsole(async (logs) => { - // snippet.getEntitiesCursorPagingExample - let cursor: string | undefined; - let hasNext = true; - let page = 0; - - while (hasNext) { - const response = await pubnub.dataSync.getEntities({ - class: 'product', - filterFast: 'price < 100', - limit: 20, - cursor, - }); - console.log(`Page ${++page}:`, response.data); - cursor = response.meta?.next_cursor ?? undefined; - hasNext = response.meta?.has_next ?? false; - } - // snippet.end - assert.ok(logs.lines.some((line) => line.includes(product())), 'a page lists the product'); - }); - }); - - step('updateEntityBasicUsage', { ...entity, proves: 'setEntity replaces the product and sets the price and stock later steps rely on.' }, async () => { - const pubnub = world.server; - // snippet.updateEntityBasicUsage - const response = await pubnub.dataSync.setEntity({ - id: 'product-sneaker-42', - data: { - classVersion: 1, - payload: { name: 'Retro Sneaker', price: 79.99, stock: 8 }, - }, - }); - console.log('setEntity response:', response.data); - // snippet.end - assert.deepEqual(response.data.payload, { name: 'Retro Sneaker', price: 79.99, stock: 8 }); - }); - - // Patch of the price only, stock must stay 8. - step('patchEntityBasicUsage', { ...secured, needs: ['updateEntityBasicUsage'], proves: 'updateEntity patches the price and the stock stays the same.' }, async () => { - const pubnub = world.server; - // snippet.patchEntityBasicUsage - const response = await pubnub.dataSync.updateEntity({ - id: 'product-sneaker-42', - replace: { '/payload/price': 79.99 }, - }); - console.log('updateEntity response:', response.data); - // snippet.end - assert.equal(response.data.payload?.price, 79.99); - assert.equal(response.data.payload?.stock, 8); - }); - - step('updateEntityCombinedPatchExample', { ...secured, needs: ['patchEntityBasicUsage'], proves: 'One patch call with all six JSON Patch operations applies, and a stale eTag is rejected.' }, async () => { - const pubnub = world.server; - const seeded = await api().setEntity({ - id: product(), - data: { - classVersion: 1, - payload: { name: 'Retro Sneaker', price: 79.99, stock: 8, tags: [], legacy: { field: 'old' }, legacyName: 'Retro' }, - }, - }); - world.setPh('entityEtag', seeded.data.eTag); - // snippet.updateEntityCombinedPatchExample - // A single `updateEntity` call can mix `add`, `replace`, `remove`, `move`, `copy`, and `test`. Add - // `ifMatchesEtag` (the `eTag` from a prior read) to reject the patch with a 412 if the entity - // changed since you read it, instead of silently overwriting a concurrent change. - const response = await pubnub.dataSync.updateEntity({ - id: 'product-sneaker-42', - test: { '/payload/stock': 8 }, - replace: { '/payload/price': 74.99 }, - add: { '/payload/tags/0': 'clearance' }, - remove: ['/payload/legacy/field'], - move: [{ from: '/payload/legacyName', path: '/payload/displayName' }], - copy: [{ from: '/payload/displayName', path: '/payload/previousName' }], - ifMatchesEtag: 'StUvWxYzAbCdEf', - }); - console.log('updateEntity (combined patch) response:', response.data); - // snippet.end - const payload = response.data.payload as Record; - assert.equal(payload.price, 74.99); - assert.deepEqual(payload.tags, ['clearance']); - assert.deepEqual(payload.legacy, {}); - assert.equal(payload.displayName, 'Retro'); - assert.equal(payload.previousName, 'Retro'); - assert.ok(!('legacyName' in payload), 'move removed the source'); - await assert.rejects( - () => api().updateEntity({ id: product(), replace: { '/payload/price': 70 }, ifMatchesEtag: world.ph('entityEtag') }), - (error: unknown) => /\b(412|403)\b/.test(errorText(error)), - 'the old eTag is rejected after the patch', - ); - }); - - - check('datasyncDotIds', { ...secured, needs: ['datasyncClassProduct'], proves: 'Entity ids with a dot are stored verbatim, returned verbatim and found by filterFast.' }, async () => { - const id = world.n('product.676'); - try { - const created = await api().createEntity({ id, class: 'product', data: { classVersion: 1, payload: { name: 'Dot Sneaker', price: 10, stock: 1 } } }); - assert.equal(created.data.id, id, 'the service keeps a dotted id verbatim'); - assert.equal((await api().getEntity({ id })).data.id, id); - // `id` is a built-in filterable field, so a strongly consistent filter finds the dotted id right away. - const found = await api().getEntities({ class: 'product', filterFast: `id == "${id}"` }); - assert.deepEqual(ids(found), [id]); - } finally { - await api().removeEntity({ id }).catch(() => undefined); - } - }); - - check('datasyncDeleteRecreate', { ...secured, needs: ['datasyncClassProduct'], proves: 'An entity can be created again with the same id right after it was deleted.' }, async () => { - // (Used by addDataSyncListener, which recreates the product it deletes without a retry loop.) - const id = world.n('product.678'); - const body = { classVersion: 1, payload: { name: 'Recreate Sneaker', price: 12, stock: 2 } }; - try { - await api().createEntity({ id, class: 'product', data: body }); - await api().removeEntity({ id }); - const again = await api().createEntity({ id, class: 'product', data: body }); - assert.equal(again.data.id, id, 'the id can be reused straight after the delete'); - } finally { - await api().removeEntity({ id }).catch(() => undefined); - } - }); - - check('datasyncFragments', { ...secured, needs: ['datasyncClassProduct'], timeout: 120_000, proves: 'Every filter, sort and patch line shown in the DataSync page fragments is accepted or refused by the service as documented.' }, async () => { - const accepted = async (query: Pick): Promise => { - await api().getEntities({ ...query, class: 'product', limit: 5 }); - }; - for (const fragment of filterFragments) { - await accepted(fragment).catch((error) => { - throw new Error(`filter fragment ${JSON.stringify(fragment)} was rejected: ${errorText(error)}`); - }); - } - // Two sort fragments order by `type`, a property the sample class `product` does not declare (the docs example - // implies a class that has both `type` and `price`). The service refuses a sort field that no property declares, - // with 403 from the Access Manager service (the same answer as for an undeclared projection). That is a docs - // example written for a different class, not a syntax error, so the check expects the refusal and keeps the - // finding visible. Every other sort fragment must be accepted. - const needsUndeclaredType = (fragment: (typeof sortFragments)[number]): boolean => JSON.stringify(fragment.sort).includes('type'); - for (const fragment of sortFragments) { - if (needsUndeclaredType(fragment)) { - await assert.rejects(accepted(fragment), (error: unknown) => /\b(400|403)\b/.test(errorText(error)), `undeclared sort field ${JSON.stringify(fragment)}`); - } else { - await accepted(fragment).catch((error) => { - throw new Error(`sort fragment ${JSON.stringify(fragment)} was rejected: ${errorText(error)}`); - }); - } - } - - // Patch forms: apply each line to a scratch entity that holds every field the lines touch. - const id = world.n('product.679'); - try { - await api().createEntity({ - id, - class: 'product', - data: { - classVersion: 1, - status: 'active', - payload: { name: 'Patch Sneaker', price: 99, stock: 3, tags: [], tempFlag: true, legacy: { field: 'old' }, legacyName: 'Retro', displayName: 'Retro' }, - }, - }); - for (const fragment of patchFragments) { - const response = await api().updateEntity({ id, ...fragment }).catch((error) => { - throw new Error(`patch fragment ${JSON.stringify(fragment)} was rejected: ${errorText(error)}`); - }); - assert.equal(response.status, 200, `patch fragment ${JSON.stringify(fragment)}`); - } - // Spot check the result of the chain: add, replace, remove, move and copy all took effect. - const payload = (await api().getEntity({ id })).data.payload as Record; - assert.deepEqual(payload.tags, ['featured']); - assert.equal(payload.price, 79.99); - assert.ok(!('tempFlag' in payload)); - assert.ok(!('legacyName' in payload)); - assert.equal(payload.previousName, 'Retro'); - } finally { - await api().removeEntity({ id }).catch(() => undefined); - } - }); - - - const eventStep = { ...secured, needs: ['datasyncClassProduct', 'createEntityBasicUsage', 'datasyncEvents'], timeout: 60_000 }; - const logged = (lines: string[], text: string): boolean => lines.some((line) => line.includes(text)); - - step('dataSyncEntitySubscribeExample', { ...eventStep, proves: 'The entity subscribe sample connects, and a changed entity arrives as an event when event rules are on.' }, async () => { - // The probe result is cached, so this decides the skip up front but the skip itself is thrown only after the - // subscription connected (below). - const off = await offKinds('entity'); - const pubnub = world.server; - await world.withConsole(async (logs) => { - const ready = connected(pubnub, { channels: [product()] }); - // snippet.dataSyncEntitySubscribeExample - // DataSync objects publish create/update/delete events over a `dataSync` listener. Create a - // DataSync SDK entity for the object, subscribe to it, and attach the listener. - const productSubscription = pubnub.dataSyncEntity('product-sneaker-42').subscription(); - productSubscription.onDataSync = (event) => console.log(event.message.event, event.message.data); - productSubscription.subscribe(); - // snippet.end - await ready; - try { - skipUnlessEvents(off); - await api().updateEntity({ id: product(), replace: { '/payload/stock': 7 } }); - await eventually(() => assert.ok(logged(logs.lines, 'update') && logged(logs.lines, product())), { label: 'update event logged' }); - } finally { - productSubscription.unsubscribe(); - } - }); - }); - - step('subscribeToDataSyncEntity', { ...eventStep, proves: 'Subscribing to one entity delivers the new stock in the event payload when event rules are on.' }, async () => { - const off = await offKinds('entity'); - const pubnub = world.server; - await world.withConsole(async (logs) => { - const ready = connected(pubnub, { channels: [product()] }); - // snippet.subscribeToDataSyncEntity - // Observe one entity. - const entitySubscription = pubnub.dataSyncEntity('product-sneaker-42').subscription(); - - entitySubscription.onDataSync = (event) => { - console.log(event.message.event, event.message.data); - }; - - entitySubscription.subscribe(); - // snippet.end - await ready; - try { - skipUnlessEvents(off); - await api().updateEntity({ id: product(), replace: { '/payload/stock': 6 } }); - await eventually(() => assert.ok(logs.lines.some((l) => l.startsWith('update') && l.includes('"stock":6'))), { label: 'update event with the new stock' }); - } finally { - entitySubscription.unsubscribe(); - } - }); - }); - - // Wildcard subscription on the `admin` projection channel (`__admin__`). Creates a dotted id that matches the - // pattern and waits for its event. Cleans up the extra entity in `finally`. - step('observeDataSyncEntitiesByPattern', { ...eventStep, proves: 'A wildcard subscription on the admin projection channel receives the event for a newly created dotted id.' }, async () => { - const off = await offKinds('entity'); - const pubnub = world.server; - const member = world.n('product.676'); - await world.withConsole(async (logs) => { - const ready = connected(pubnub, { channels: [`__admin__${world.n('product.*')}`] }); - // snippet.observeDataSyncEntitiesByPattern - // Because the identifier is used verbatim, a wildcard identifier subscribes to the matching - // channel pattern and one subscription serves the whole set. `event.channel` reports the concrete - // delivery channel, while `event.subscription` reports the pattern that matched it. - const patternSubscription = pubnub.dataSyncEntity('product.*').subscription({ projection: 'admin' }); - - patternSubscription.onDataSync = (event) => { - // event.channel -> '__admin__product.676' - // event.subscription -> '__admin__product.*' - console.log('changed:', event.message.data.id); - }; - - patternSubscription.subscribe(); - // snippet.end - await ready; - try { - skipUnlessEvents(off); - await api().createEntity({ id: member, class: 'product', data: { classVersion: 1, payload: { name: 'Pattern Sneaker', price: 10, stock: 1 } } }); - await eventually(() => assert.ok(logged(logs.lines, `changed: ${member}`)), { label: 'event for an entity that matches the pattern' }); - } finally { - patternSubscription.unsubscribe(); - await api().removeEntity({ id: member }).catch(() => undefined); - } - }); - }); - - // Combines a user and an entity subscription into one set. Needs events on for both kinds. Triggers one update of - // each and expects both on the one listener. - step('combineDataSyncSubscriptions', { ...eventStep, proves: 'A user and an entity subscription combined in one set both deliver events to one listener.' }, async () => { - const off = await offKinds('entity', 'user'); - const pubnub = world.server; - const member = world.n('product.677'); - await world.withConsole(async (logs) => { - const ready = connected(pubnub, { channels: [world.n('product.*'), user()] }); - // snippet.combineDataSyncSubscriptions - // A DataSync subscription is an ordinary `subscription`, so it composes into a set with any other - // subscription. - const productsSubscription = pubnub.dataSyncEntity('product.*').subscription(); - const aliceSubscription = pubnub.dataSyncUser('user-alice').subscription(); - - const dataSyncSubscriptionSet = productsSubscription.addSubscription(aliceSubscription); - dataSyncSubscriptionSet.onDataSync = (event) => console.log(event.message.objectType, event.message.event); - dataSyncSubscriptionSet.subscribe(); - // snippet.end - await ready; - try { - skipUnlessEvents(off); - await api().updateUser({ id: user(), replace: { '/payload/name': 'Alice C.' } }); - await api().createEntity({ id: member, class: 'product', data: { classVersion: 1, payload: { name: 'Set Sneaker', price: 11, stock: 1 } } }); - await eventually(() => assert.ok(logs.lines.includes('user update') && logs.lines.includes('entity create')), { label: 'one set receives user and entity events' }); - } finally { - dataSyncSubscriptionSet.unsubscribe(); - await api().removeEntity({ id: member }).catch(() => undefined); - } - }); - }); - - step('addDataSyncListener', { ...eventStep, proves: 'The listener switch sees update, delete and create events in that order for one product.' }, async () => { - const off = await offKinds('entity'); - const pubnub = world.server; - await world.withConsole(async (logs) => { - const ready = connected(pubnub, { channels: [product()] }); - // snippet.addDataSyncListener - // Create a DataSync SDK entity for the object and attach the listener. - const dataSyncObjectId = 'product-sneaker-42'; - const dataSyncListenerSubscription = pubnub.dataSyncEntity(dataSyncObjectId).subscription(); - - dataSyncListenerSubscription.onDataSync = (event) => { - const change = event.message; - - // This channel can also receive relationship events and events about connected entities. Only - // handle events for the observed product. `Product` is a class of your own, so its events - // report `type: 'entity'`. - if (change.type !== 'entity' || change.data.id !== dataSyncObjectId) return; - - switch (change.event) { - case 'create': - console.log('Product created:', change.data.id); - break; - case 'update': { - // `payload` is typed as the generic SDK `Payload` union (it can be a string, number, array, - // and so on for other message types), but a DataSync entity payload is always the JSON - // object declared by its class, so it is safe to read it as a record here. - const payload = - 'payload' in change.data ? (change.data.payload as Record | undefined) : undefined; - console.log('New price:', payload?.price); - break; - } - case 'delete': - console.log('Product deleted:', change.data.id); - break; - } - }; - - dataSyncListenerSubscription.subscribe(); - // snippet.end - await ready; - try { - skipUnlessEvents(off); - await api().updateEntity({ id: product(), replace: { '/payload/price': 69.5 } }); - await eventually(() => assert.ok(logged(logs.lines, 'New price: 69.5')), { label: 'update branch' }); - await api().removeEntity({ id: product() }); - await eventually(() => assert.ok(logged(logs.lines, `Product deleted: ${product()}`)), { label: 'delete branch' }); - // The check datasyncDeleteRecreate proved that the service accepts the same id straight after a delete, so no retry - // loop here: a failure should show up as a failure. - await api().createEntity({ id: product(), class: 'product', data: { classVersion: 1, payload: { name: 'Retro Sneaker', price: 74.99, stock: 8 } } }); - await eventually(() => assert.ok(logged(logs.lines, `Product created: ${product()}`)), { label: 'create branch' }); - } finally { - dataSyncListenerSubscription.unsubscribe(); - } - }); - }); - - step('dataSyncProjectionSubscriptions', { ...secured, proves: 'Two projection subscriptions connect, and unsubscribing one leaves the other subscribed.' }, async () => { - const pubnub = world.server; - // snippet.dataSyncProjectionSubscriptions - const user = pubnub.dataSyncUser('user-alice') - - const base = user.subscription() // observes `user-alice` - const admin = user.subscription({ projection: 'admin' }) // observes `__admin__user-alice` - // snippet.end - // The docs promise two independent data channels: the object itself and `____`. - const ready = connected(pubnub, { channels: [world.n('user-alice'), `__admin__${world.n('user-alice')}`] }); - base.subscribe(); - admin.subscribe(); - await ready; - // "Unsubscribing one leaves the other subscribed": drop the base one and check the admin channel is still active. - base.unsubscribe(); - assert.ok(pubnub.getSubscribedChannels().includes(`__admin__${world.n('user-alice')}`), 'the admin projection stays subscribed'); - assert.ok(!pubnub.getSubscribedChannels().includes(world.n('user-alice')), 'the base channel is gone'); - admin.unsubscribe(); - }); - - // ---- relationships (class `ProductOwner`) ------------------------------------------------------------------ - - const link = { ...secured, needs: ['datasyncClassProductOwner', 'createRelationshipBasicUsage'] }; - - check('datasyncSellerFixture', { ...secured, needs: ['datasyncClassProductOwner', 'createEntityBasicUsage'], proves: 'Fixture: Bob exists as a DataSync user, the owning end of the ProductOwner link.' }, async () => { - const created = await api().createUser({ id: seller(), data: { classVersion: 1, payload: { name: 'Bob', type: 'seller' } } }); - assert.equal(created.data.id, seller()); - }); - - // Creates the `ProductOwner` link from the seller to the product with the fixed id that the later samples use. - // `needs` the seller fixture because both ends must exist. - step('createRelationshipBasicUsage', { ...secured, needs: ['datasyncSellerFixture'], proves: 'createRelationship links the seller to the product with the fixed id that later steps use.' }, async () => { - const pubnub = world.server; - // snippet.createRelationshipBasicUsage - const response = await pubnub.dataSync.createRelationship({ - id: 'rel-bob-owns-sneaker-42', - class: 'ProductOwner', - entityAId: 'seller-bob', - entityBId: 'product-sneaker-42', - data: { - classVersion: 1, - payload: { since: '2026-07-13' }, - }, - }); - console.log('createRelationship response:', response.data); - // snippet.end - assert.equal(response.data.id, relationship()); - assert.equal(response.data.relationshipClass, 'ProductOwner'); - assert.equal(response.data.entityAId, seller()); - assert.equal(response.data.entityBId, product()); - assert.deepEqual(response.data.payload, { since: '2026-07-13' }); - }); - - // Reads the link by its fixed id. - step('getRelationshipBasicUsage', { ...link, proves: 'getRelationship reads the link by its fixed id.' }, async () => { - const pubnub = world.server; - // snippet.getRelationshipBasicUsage - const response = await pubnub.dataSync.getRelationship({ - id: 'rel-bob-owns-sneaker-42', - }); - console.log('getRelationship response:', response.data); - // snippet.end - assert.equal(response.data.id, relationship()); - assert.equal(response.data.entityBId, product()); - }); - - // Lists links from the seller side, expects exactly the fixture. - step('getRelationshipsBasicUsage', { ...link, proves: 'getRelationships lists exactly the fixture link from the seller side.' }, async () => { - const pubnub = world.server; - // snippet.getRelationshipsBasicUsage - const response = await pubnub.dataSync.getRelationships({ - class: 'ProductOwner', - entityAId: 'seller-bob', - limit: 20, - }); - console.log('getRelationships response:', response.data); - console.log('getRelationships meta:', response.meta); - // snippet.end - assert.deepEqual(ids(response), [relationship()]); - }); - - // Same list from the product side. - step('getRelationshipsByEntityBIdExample', { ...link, proves: 'getRelationships by entity B id lists the same link from the product side.' }, async () => { - const pubnub = world.server; - // snippet.getRelationshipsByEntityBIdExample - // Pass `entityBId` instead of `entityAId` to list relationships where the entity is on the second - // side of the link. - const response = await pubnub.dataSync.getRelationships({ - class: 'ProductOwner', - entityBId: 'product-sneaker-42', - limit: 20, - }); - console.log('getRelationships (entityBId) response:', response.data); - // snippet.end - assert.deepEqual(ids(response), [relationship()]); - }); - - step('updateRelationshipBasicUsage', { ...link, proves: 'setRelationship replaces the link and adds a tier to its payload.' }, async () => { - const pubnub = world.server; - // snippet.updateRelationshipBasicUsage - const response = await pubnub.dataSync.setRelationship({ - id: 'rel-bob-owns-sneaker-42', - entityAId: 'seller-bob', - entityBId: 'product-sneaker-42', - data: { - classVersion: 1, - payload: { since: '2026-07-13', tier: 'gold' }, - }, - }); - console.log('setRelationship response:', response.data); - // snippet.end - assert.deepEqual(response.data.payload, { since: '2026-07-13', tier: 'gold' }); - }); - - // Negated eventually consistent filter. Polls until the link is indexed. Tier is `gold` at this point, so the link - // must be kept. - step('getRelationshipsFilterExample', { ...secured, needs: ['updateRelationshipBasicUsage'], timeout: 60_000, proves: 'A negated filter keeps the gold link once the index has caught up.' }, async () => { - const pubnub = world.server; - await eventually(async () => { - const found = await api().getRelationships({ class: 'ProductOwner', filter: '!(tier == "platinum")' }); - assert.ok(ids(found).includes(relationship())); - }, { timeout: 45_000, label: 'filter index caught up with the relationship' }); - // snippet.getRelationshipsFilterExample - const response = await pubnub.dataSync.getRelationships({ - class: 'ProductOwner', - filter: '!(tier == "platinum")', - }); - console.log('getRelationships (filter) response:', response.data); - // snippet.end - assert.ok(ids(response).includes(relationship()), 'tier is gold, so the negated filter keeps it'); - }); - - // Strongly consistent filter on `tier`, no wait. - step('getRelationshipsFilterFastExample', { ...secured, needs: ['updateRelationshipBasicUsage'], proves: 'filterFast on tier finds the link at once.' }, async () => { - const pubnub = world.server; - // snippet.getRelationshipsFilterFastExample - const response = await pubnub.dataSync.getRelationships({ - class: 'ProductOwner', - filterFast: 'tier == "gold"', - }); - console.log('getRelationships (filterFast) response:', response.data); - // snippet.end - assert.ok(ids(response).includes(relationship())); - }); - - // Cursor loop over the seller's links with console capture. - step('getRelationshipsCursorPagingExample', { ...secured, needs: ['updateRelationshipBasicUsage'], proves: 'The cursor paging loop for relationships runs and lists the link.' }, async () => { - const pubnub = world.server; - await world.withConsole(async (logs) => { - // snippet.getRelationshipsCursorPagingExample - let cursor: string | undefined; - let hasNext = true; - let page = 0; - - while (hasNext) { - const response = await pubnub.dataSync.getRelationships({ - class: 'ProductOwner', - entityAId: 'seller-bob', - limit: 20, - cursor, - }); - console.log(`Page ${++page}:`, response.data); - cursor = response.meta?.next_cursor ?? undefined; - hasNext = response.meta?.has_next ?? false; - } - // snippet.end - assert.ok(logs.lines.some((line) => line.includes(relationship())), 'a page lists the relationship'); - }); - }); - - step('patchRelationshipBasicUsage', { ...secured, needs: ['updateRelationshipBasicUsage'], proves: 'updateRelationship patches the tier of the link.' }, async () => { - const pubnub = world.server; - // snippet.patchRelationshipBasicUsage - const response = await pubnub.dataSync.updateRelationship({ - id: 'rel-bob-owns-sneaker-42', - replace: { '/payload/tier': 'platinum' }, - }); - console.log('updateRelationship response:', response.data); - // snippet.end - assert.equal(response.data.payload?.tier, 'platinum'); - assert.equal(response.data.payload?.since, '2026-07-13'); - }); - - - check('datasyncCursorPaging', { ...secured, needs: ['createUserBasicUsage', 'createDataSyncChannelBasicUsage'], proves: 'Paging one item at a time returns next_cursor and has_next correctly and never repeats an item.' }, async () => { - const seen: string[] = []; - let cursor: string | undefined; - let pages = 0; - for (;;) { - const response = await api().getChannels({ filterFast: 'type == "promotion"', limit: 1, cursor }); - pages += 1; - seen.push(...ids(response)); - cursor = response.meta?.next_cursor ?? undefined; - if (!response.meta?.has_next) break; - assert.ok(cursor, 'has_next true comes with a next_cursor'); - assert.ok(pages < 50, 'paging terminates'); - } - assert.ok(seen.includes(channel()), 'paging by one still reaches the channel'); - assert.equal(new Set(seen).size, seen.length, 'no object appears twice'); - }); - - // ---- removal, last -------------------------------------------------------------------------------------------- - - step('removeRelationshipBasicUsage', { ...secured, needs: ['patchRelationshipBasicUsage'], proves: 'removeRelationship deletes the link and a read afterwards answers 404.' }, async () => { - const pubnub = world.server; - // snippet.removeRelationshipBasicUsage - const response = await pubnub.dataSync.removeRelationship({ - id: 'rel-bob-owns-sneaker-42', - }); - console.log('removeRelationship response status:', response.status); - // snippet.end - assert.ok(response.status >= 200 && response.status < 300); - await assert.rejects(() => api().getRelationship({ id: relationship() }), (error: unknown) => /404/.test(errorText(error))); - }); - - step('removeMembershipBasicUsage', { ...secured, needs: ['patchMembershipBasicUsage'], proves: 'removeMembership deletes the membership and a read afterwards answers 404.' }, async () => { - const pubnub = world.server; - // snippet.removeMembershipBasicUsage - const response = await pubnub.dataSync.removeMembership({ - id: 'membership-alice-summer-sale', - }); - console.log('removeMembership response status:', response.status); - // snippet.end - assert.ok(response.status >= 200 && response.status < 300); - await assert.rejects(() => api().getMembership({ id: membership() }), (error: unknown) => /404/.test(errorText(error))); - }); - - step('removeEntityBasicUsage', { ...secured, needs: ['datasyncClassProduct', 'createEntityBasicUsage'], proves: 'removeEntity deletes the product and a read afterwards answers 404.' }, async () => { - const pubnub = world.server; - // snippet.removeEntityBasicUsage - const response = await pubnub.dataSync.removeEntity({ id: 'product-sneaker-42' }); - console.log('removeEntity response status:', response.status); - // snippet.end - assert.ok(response.status >= 200 && response.status < 300); - await assert.rejects(() => api().getEntity({ id: product() }), (error: unknown) => /404/.test(errorText(error))); - }); - - step('removeChannelBasicUsage', { ...secured, needs: ['createDataSyncChannelBasicUsage'], proves: 'removeChannel deletes the DataSync channel and a read afterwards answers 404.' }, async () => { - const pubnub = world.server; - // snippet.removeChannelBasicUsage - const response = await pubnub.dataSync.removeChannel({ id: 'channel-summer-sale' }); - console.log('removeChannel response status:', response.status); - // snippet.end - assert.ok(response.status >= 200 && response.status < 300); - await assert.rejects(() => api().getChannel({ id: channel() }), (error: unknown) => /404/.test(errorText(error))); - }); - - step('removeUserBasicUsage', { ...secured, needs: ['createUserBasicUsage'], proves: 'removeUser deletes Alice and a read afterwards answers 404.' }, async () => { - const pubnub = world.server; - // snippet.removeUserBasicUsage - const response = await pubnub.dataSync.removeUser({ id: 'user-alice' }); - console.log('removeUser response status:', response.status); - // snippet.end - assert.ok(response.status >= 200 && response.status < 300); - await assert.rejects(() => api().getUser({ id: user() }), (error: unknown) => /404/.test(errorText(error))); - }); - - // Removes the one fixture no sample removes (Bob). - check('datasyncTeardown', { ...secured, proves: 'Teardown removes the one fixture no sample removes (Bob).' }, async () => { - await api().removeUser({ id: seller() }).catch(() => undefined); - }); -}); diff --git a/docs-snippets/api-reference/encryption-legacy.ts b/docs-snippets/api-reference/encryption-legacy.ts deleted file mode 100644 index b4d7b111a..000000000 --- a/docs-snippets/api-reference/encryption-legacy.ts +++ /dev/null @@ -1,57 +0,0 @@ -import assert from '../harness/assert.js'; -import PubNub from 'pubnub'; -import { page, step, check } from '../harness/step.js'; - -/** - * Examples for the deprecated `encrypt()` and `decrypt()` methods and the cipher key arguments. Both methods log a - * deprecation warning on every call. - */ -export default page('api-reference/encryption-legacy', 'P8', (world) => { - // Open keyset client with an AES-CBC crypto module, the setup the legacy methods fall back to. - const withModule = (label: string): PubNub => - world.client('open', label, { cryptoModule: PubNub.CryptoModule.aesCbcCryptoModule({ cipherKey: 'pubnubenigma' }) }); - - step('decryptBasicUsage', { proves: 'The legacy decrypt returns the original JSON for text encrypted with the same cipher key.' }, () => { - const pubnub = withModule('legacy-decrypt'); - // `decrypt()` JSON-parses the plain text, so the input has to be a JSON document. - const encrypted = pubnub.encrypt(JSON.stringify({ item: 'sneaker' })) as string; - // snippet.decryptBasicUsage - const decrypted = pubnub.decrypt(encrypted); // Pass the encrypted data as the first parameter in decrypt Method - // snippet.end - assert.deepEqual(decrypted, { item: 'sneaker' }); - }); - - check('encryptionLegacyCipherKeyArguments', { proves: 'The optional customCipherKey argument of encrypt and decrypt behaves as the page states.' }, () => { - const pubnub = withModule('legacy-cipher-key'); - const secret = { item: 'sneaker', size: 42 }; - // customCipherKey overrides the configured module and uses the legacy encryption. - const encrypted = pubnub.encrypt(JSON.stringify(secret), 'another-key'); - assert.equal(typeof encrypted, 'string'); - assert.throws(() => pubnub.encrypt(secret as never, 'another-key'), TypeError); - assert.deepEqual(pubnub.decrypt(encrypted, 'another-key'), secret); - // A wrong key does not throw with the legacy encryption: decrypt() returns null. - assert.equal(pubnub.decrypt(encrypted, 'wrong-key'), null); - // Without a configured module or a key, the data comes back unchanged. - const bare = world.client('open', 'legacy-bare'); - assert.equal(bare.encrypt('plain'), 'plain'); - }); - - // The error row of the decrypt section: unreadable input throws instead of returning garbage. - check('encryptionLegacyDecryptError', { proves: 'Decrypting unreadable input throws instead of returning garbage.' }, () => { - const pubnub = withModule('legacy-decrypt-error'); - // cryptoModule rejects data it cannot read with an Error that describes the reason. - assert.throws(() => pubnub.decrypt(''), /Decryption error/); - }); - - check('encryptionLegacyFileCipherKey', { proves: 'Legacy file encryption with an explicit key works, and records that it still needs a configured module.' }, async () => { - const pubnub = world.client('open', 'legacy-file', { cipherKey: 'pubnubenigma' }); - const content = Buffer.from('legacy file content'); - const file = pubnub.File.create({ data: content, name: 'legacy.txt', mimeType: 'text/plain' }); - const encrypted = await pubnub.encryptFile('file-key', file); - assert.notDeepEqual(Buffer.from(await encrypted.toArrayBuffer()), content); - const decrypted = await pubnub.decryptFile('file-key', encrypted); - assert.deepEqual(Buffer.from(await decrypted.toArrayBuffer()), content); - const noModule = world.client('open', 'legacy-file-none'); - await assert.rejects(noModule.decryptFile('file-key', encrypted), /File decryption error\. Crypto module not configured\./); - }); -}); diff --git a/docs-snippets/api-reference/encryption.ts b/docs-snippets/api-reference/encryption.ts deleted file mode 100644 index 6698e7f4e..000000000 --- a/docs-snippets/api-reference/encryption.ts +++ /dev/null @@ -1,101 +0,0 @@ -import assert from '../harness/assert.js'; -import fs from 'node:fs'; -import os from 'node:os'; -import path from 'node:path'; -import crypto from 'node:crypto'; -import PubNub from 'pubnub'; -import { page, step, check } from '../harness/step.js'; - -/** - * Encryption examples. - */ -export default page('api-reference/encryption', 'P4', (world) => { - const workDir = fs.mkdtempSync(path.join(os.tmpdir(), 'pn-docs-encryption-')); - const picture = crypto.randomBytes(2048); - - /** Runs `fn` with `workDir` as the working directory, as the docs file samples assume. */ - const inWorkDir = async (fn: () => Promise): Promise => { - const previous = process.cwd(); - process.chdir(workDir); - try { - return await fn(); - } finally { - process.chdir(previous); - } - }; - - const cryptoClient = (label: string): PubNub => - world.client('open', label, { cryptoModule: PubNub.CryptoModule.aesCbcCryptoModule({ cipherKey: 'pubnubenigma' }) }); - - step('encryptMessageBasicUsage', { proves: 'encrypt turns a message into cipher text that differs from the plain text, and decrypt reads it back.' }, () => { - // snippet.encryptMessageBasicUsage - // Create a crypto module instance with AES-CBC encryption - const cryptoModule = PubNub.CryptoModule.aesCbcCryptoModule({ - cipherKey: 'pubnubenigma', - }); - - const msgContent = 'This is the data I wish to encrypt.'; - console.log(`Original Message: ${msgContent}`); - - // Encrypt the message - const encryptedMessage = cryptoModule.encrypt(JSON.stringify(msgContent)); - console.log('message encrypted successfully'); - - // Decrypt with the same instance. decrypt() returns an ArrayBuffer with the bytes of the JSON string - const decryptedBytes = cryptoModule.decrypt(encryptedMessage) as ArrayBuffer; - const decryptedMessage = JSON.parse(new TextDecoder().decode(decryptedBytes)); - console.log(`Decrypted Message: ${decryptedMessage}`); - // snippet.end - assert.notEqual(encryptedMessage, JSON.stringify(msgContent), 'the output is not the plain text'); - assert.ok(decryptedBytes instanceof ArrayBuffer, 'decrypt returns an ArrayBuffer'); - assert.equal(decryptedMessage, msgContent, 'the decoded bytes are the original message'); - }); - - step('encryptFileBasicUsage', { timeout: 30_000, proves: 'encryptFile produces an encrypted copy of the picture that is saved for the decrypt step.' }, async () => { - const pubnub = cryptoClient('crypto-file'); - fs.writeFileSync(path.join(workDir, 'cat_picture.jpg'), picture); - await inWorkDir(async () => { - // snippet.encryptFileBasicUsage - // Node.js example - // import fs from 'fs'; - - const fileBuffer = fs.readFileSync('./cat_picture.jpg'); - - const file = pubnub.File.create({ data: fileBuffer, name: 'cat_picture.jpg', mimeType: 'image/jpeg' }); - - const encryptedFile = await pubnub.encryptFile(file); - // snippet.end - const encrypted = Buffer.from(await encryptedFile.toArrayBuffer()); - assert.notDeepEqual(encrypted, picture, 'the file content changed'); - assert.ok(encrypted.length > picture.length, 'the cipher text carries a header and padding'); - fs.writeFileSync('./cat_picture_encrypted.jpg', encrypted); - }); - }); - - step('decryptFileBasicUsage', { needs: ['encryptFileBasicUsage'], timeout: 30_000, proves: 'decryptFile restores the picture byte for byte with a second client that has the same cipher key.' }, async () => { - const pubnub = cryptoClient('crypto-file-reader'); - await inWorkDir(async () => { - // snippet.decryptFileBasicUsage - // Node.js example. - // import fs from 'fs'; - const fileBufferData = fs.readFileSync('./cat_picture_encrypted.jpg'); - - const fileData = pubnub.File.create({ data: fileBufferData, name: 'cat_picture.jpg', mimeType: 'image/jpeg' }); - - const decryptedFile = await pubnub.decryptFile(fileData); - // snippet.end - const decrypted = Buffer.from(await decryptedFile.toArrayBuffer()); - assert.deepEqual(decrypted, picture, 'the round trip returns the original bytes'); - }); - }); - - check('encryptionFileErrors', { proves: 'Each row of the file encryption error table is raised for the matching bad call.' }, async () => { - const withModule = cryptoClient('crypto-errors'); - const file = withModule.File.create({ data: Buffer.from('abc'), name: 'a.txt', mimeType: 'text/plain' }); - await assert.rejects(withModule.encryptFile(undefined as never), /File encryption error\. Source file is missing\./); - - const noModule = world.client('open', 'crypto-none'); - await assert.rejects(noModule.encryptFile(file), /File encryption error\. Crypto module not configured\./); - assert.equal(await noModule.decryptFile(file), undefined); - }); -}); diff --git a/docs-snippets/api-reference/file-sharing-legacy.ts b/docs-snippets/api-reference/file-sharing-legacy.ts deleted file mode 100644 index 9c43d09b1..000000000 --- a/docs-snippets/api-reference/file-sharing-legacy.ts +++ /dev/null @@ -1,56 +0,0 @@ -import assert from '../harness/assert.js'; -import fs from 'node:fs'; -import { page, step } from '../harness/step.js'; -import { enterWorkdir, FIXTURE_FILE } from '../fixtures/workdir.js'; - -/** - * Example for the deprecated per-call `cipherKey` parameter of file sharing. - */ -export default page('api-reference/file-sharing-legacy', 'P8', (world) => { - // Plain bytes of the fixture, the reference for the round trip comparison. - const original = fs.readFileSync(FIXTURE_FILE); - let uploaded: { id: string } | undefined; - - step('sendFileCustomCipherKey', { timeout: 60_000, proves: 'sendFile with a custom cipher key uploads the picture from a working directory.' }, async () => { - const pubnub = world.shopper; - const workdir = enterWorkdir(); - try { - // snippet.sendFileCustomCipherKey - // in Node.js - - const myFile = fs.readFileSync('./cat_picture.jpg'); - - const response = await pubnub.sendFile({ - channel: 'my_channel', - message: 'Look at this picture!', - file: { data: myFile, name: 'cat_picture.jpg', mimeType: 'image/jpeg' }, - cipherKey: 'myCipherKey', - }); - console.log('File sent successfully:', response); - // snippet.end - assert.ok(response.id); - uploaded = { id: response.id }; - } finally { - workdir.restore(); - } - }); - - step('downloadFileCustomCipherKey', { needs: ['sendFileCustomCipherKey'], timeout: 60_000, proves: 'downloadFile with the right custom cipher key returns the file and the wrong key does not.' }, async () => { - const pubnub = world.shopper; - assert.ok(uploaded, 'sendFileCustomCipherKey has not run'); - world.setPh('fileId', uploaded.id); - // snippet.downloadFileCustomCipherKey - const file = await pubnub.downloadFile({ - channel: 'my_channel', - id: '...', - name: 'cat_picture.jpg', - cipherKey: 'myCipherKey', - }); - // snippet.end - assert.deepEqual(await file.toBuffer(), original, 'the cipher key decrypts the upload'); - // Without the key the content is the encrypted payload. - const raw = await pubnub.downloadFile({ channel: world.n('my_channel'), id: uploaded.id, name: 'cat_picture.jpg' }); - assert.notDeepEqual(await raw.toBuffer(), original, 'no key, no plain file'); - await pubnub.deleteFile({ channel: world.n('my_channel'), id: uploaded.id, name: 'cat_picture.jpg' }); - }); -}); diff --git a/docs-snippets/api-reference/file-sharing.ts b/docs-snippets/api-reference/file-sharing.ts deleted file mode 100644 index 737fc1601..000000000 --- a/docs-snippets/api-reference/file-sharing.ts +++ /dev/null @@ -1,219 +0,0 @@ -import assert from '../harness/assert.js'; -import { Readable } from 'node:stream'; -import { page, step, check } from '../harness/step.js'; -import { eventually, watch } from '../harness/wait.js'; -import { explainError } from '../harness/guard.js'; -import { enterWorkdir, FIXTURE_FILE } from '../fixtures/workdir.js'; - -import fs from 'fs'; - -/** - * File sharing examples for Node.js. - */ -export default page('api-reference/file-sharing', 'P4', (world) => { - // Plain bytes of the fixture, the reference every download is compared against. - const original = fs.readFileSync(FIXTURE_FILE); - let uploaded: { id: string; name: string } | undefined; - const uploadedFile = (): { id: string; name: string } => { - assert.ok(uploaded, 'sendFileBasicUsage has not run'); - world.setPh('fileId', uploaded.id); - return uploaded; - }; - - check('preflightFiles', { phase: 'P0', timeout: 60_000, proves: 'Preflight: the open keyset has Files enabled, checked by uploading and deleting a tiny file.' }, async () => { - const channel = world.n('preflight_channel'); - try { - const sent = await world.friend.sendFile({ channel, file: { data: Buffer.from('preflight'), name: 'preflight.txt', mimeType: 'text/plain' } }); - await world.friend.deleteFile({ channel, id: sent.id, name: 'preflight.txt' }); - } catch (error) { - explainError(error); - const detail = error instanceof Error ? error.message : String(error); - throw new Error(`The open keyset cannot send files. Enable "File Sharing" (and Message Persistence) for it. ${detail}`); - } - }); - - step('sendFileBasicUsage', { needs: ['preflightFiles'], timeout: 60_000, proves: 'sendFile uploads the picture and a second client receives the file event.' }, async () => { - const pubnub = world.shopper; - const listening = await watch(world.fresh('fileobserver'), [world.n('my_channel')]); - const workdir = enterWorkdir(); - try { - // snippet.sendFileBasicUsage - // Function to send a file to a channel - // in node.js, make sure to import 'fs' - // and use the createReadStream method to read the file - const myFile = fs.createReadStream('./cat_picture.jpg'); - - const response = await pubnub.sendFile({ - channel: 'my_channel', - file: { stream: myFile, name: 'cat_picture.jpg', mimeType: 'image/jpeg' }, - customMessageType: 'file-message', - }); - - console.log('File sent successfully:', response); - // snippet.end - assert.ok(response.id, 'the response carries the file id'); - assert.equal(response.name, 'cat_picture.jpg'); - uploaded = { id: response.id, name: response.name }; - const event = await listening.next('file', (e) => e.file?.id === response.id); - assert.equal(event.channel, world.n('my_channel')); - assert.equal(event.file?.name, 'cat_picture.jpg'); - assert.equal(event.publisher, pubnub.getUserId()); - } finally { - listening.stop(); - workdir.restore(); - } - }); - - step('listFilesBasicUsage', { needs: ['sendFileBasicUsage'], proves: 'listFiles returns the file the send step uploaded.' }, async () => { - const pubnub = world.shopper; - const { id } = uploadedFile(); - // snippet.listFilesBasicUsage - const response = await pubnub.listFiles({ channel: 'my_channel' }); - console.log('Files listed successfully:', response); - // snippet.end - const entry = response.data.find((f) => f.id === id); - assert.ok(entry, 'the uploaded file is listed'); - assert.equal(entry.name, 'cat_picture.jpg'); - }); - - step('getFileUrlBasicUsage', { needs: ['sendFileBasicUsage'], proves: 'getFileUrl builds a URL that serves the uploaded bytes.' }, async () => { - const pubnub = world.shopper; - const { id, name } = uploadedFile(); - // snippet.getFileUrlBasicUsage - const response = pubnub.getFileUrl({ channel: 'my_channel', id: '...', name: 'cat_picture.jpg' }); - // snippet.end - assert.match(response, /^https:\/\//); - assert.ok(response.includes(`/files/${id}/`), 'the URL carries the file id'); - assert.ok(response.includes(`/${name}?`), 'the URL carries the file name'); - assert.ok(response.includes(encodeURIComponent(world.n('my_channel'))), 'the URL carries the channel'); - // The URL with the real file name serves the uploaded bytes. - const real = pubnub.getFileUrl({ channel: world.n('my_channel'), id, name }); - const served = await fetch(real); - assert.equal(served.status, 200); - assert.deepEqual(Buffer.from(await served.arrayBuffer()), original); - }); - - step('downloadFileNodeBasicUsage', { needs: ['sendFileBasicUsage'], timeout: 60_000, proves: 'downloadFile in Node writes the downloaded content to the output file.' }, async () => { - const pubnub = world.shopper; - uploadedFile(); - const workdir = enterWorkdir(); - const logs = world.captureConsole('log'); - try { - fs.writeFileSync('./cat_picture.jpg', 'placeholder to be overwritten'); - // snippet.downloadFileNodeBasicUsage - // In Node.js using streams: - // import fs from 'fs' - const downloadFileResponse = await pubnub.downloadFile({ - channel: 'my_channel', - id: '...', - name: 'cat_picture.jpg', - }); - - const output = fs.createWriteStream('./cat_picture.jpg'); - const fileStream = await downloadFileResponse.toStream(); - - fileStream.pipe(output); - - output.once('finish', () => { - console.log('File saved to ./cat_picture.jpg'); - }); - // snippet.end - await eventually(() => { - assert.deepEqual(fs.readFileSync('./cat_picture.jpg'), original); - }, { label: 'downloaded file written' }); - await eventually(() => { - assert.ok(logs.lines.includes('File saved to ./cat_picture.jpg'), 'the finish handler logged the confirmation'); - }, { label: 'finish event logged' }); - assert.equal(downloadFileResponse.name, 'cat_picture.jpg'); - } finally { - logs.restore(); - workdir.restore(); - } - }); - - - step('publishFileMessageBasicUsage', { needs: ['sendFileBasicUsage'], proves: 'publishFileMessage announces the uploaded file and the event is matched by timetoken.' }, async () => { - const pubnub = world.shopper; - const { id } = uploadedFile(); - const listening = await watch(world.fresh('fileobserver'), [world.n('my_channel')]); - // snippet.publishFileMessageBasicUsage - const fileMessageResponse = await pubnub.publishFile({ - channel: 'my_channel', - fileId: '...', - fileName: 'cat_picture.jpg', - message: { field: 'value' }, - customMessageType: 'file-message', - }); - console.log('File message published successfully:', fileMessageResponse); - // snippet.end - world.check.timetoken(fileMessageResponse.timetoken); - const event = await listening.next('file', (e) => e.timetoken === String(fileMessageResponse.timetoken)); - assert.equal(event.file?.id, id); - assert.deepEqual(event.message, { field: 'value' }); - listening.stop(); - }); - - // ---- FileInput forms and PubNubFile in Node --------------------------------------------------------------------- - - check('fileInputNodeForms', { needs: ['preflightFiles'], timeout: 90_000, proves: 'Every Node file input form listed on the page uploads and downloads correctly.' }, async () => { - // The docs list stream, buffer, ArrayBuffer and string forms for Node.js. Each one must upload and come back unchanged. - const pubnub = world.shopper; - const channel = world.n('my_channel'); - const text = 'sneaker drop receipt'; - const forms: Array<{ label: string; file: Parameters[0]['file']; expected: Buffer }> = [ - { label: 'stream', file: { stream: Readable.from([original]), name: 'stream.jpg', mimeType: 'image/jpeg' }, expected: original }, - { label: 'buffer', file: { data: original, name: 'buffer.jpg', mimeType: 'image/jpeg' }, expected: original }, - { label: 'ArrayBuffer', file: { data: new Uint8Array(original).buffer, name: 'arraybuffer.jpg', mimeType: 'image/jpeg' }, expected: original }, - { label: 'string', file: { data: text, name: 'string.txt', mimeType: 'text/plain' }, expected: Buffer.from(text) }, - ]; - for (const form of forms) { - const sent = await pubnub.sendFile({ channel, file: form.file }); - try { - const name = (form.file as { name: string }).name; - const downloaded = await pubnub.downloadFile({ channel, id: sent.id, name }); - assert.deepEqual(await downloaded.toBuffer(), form.expected, `${form.label} round trip`); - } finally { - await pubnub.deleteFile({ channel, id: sent.id, name: (form.file as { name: string }).name }); - } - } - }); - - check('pubnubFileNodeMethods', { needs: ['sendFileBasicUsage'], timeout: 60_000, proves: 'The PubNubFile conversions that work in Node work and the others reject as the page table says.' }, async () => { - // Docs, "Methods supported in Node.js": toBuffer, toStream, toString, toArrayBuffer work. toFile, toBlob and toFileUri reject. - const pubnub = world.shopper; - const { id, name } = uploadedFile(); - const channel = world.n('my_channel'); - const read = async () => pubnub.downloadFile({ channel, id, name }); - const stream = await (await read()).toStream(); - const chunks: Buffer[] = []; - for await (const chunk of stream) chunks.push(Buffer.from(chunk)); - assert.deepEqual(Buffer.concat(chunks), original, 'toStream'); - assert.deepEqual(await (await read()).toBuffer(), original, 'toBuffer'); - assert.deepEqual(Buffer.from(await (await read()).toArrayBuffer()), original, 'toArrayBuffer'); - assert.equal((await (await read()).toString('base64')), original.toString('base64'), 'toString with an encoding'); - for (const method of ['toFile', 'toBlob', 'toFileUri'] as const) { - const target = await read(); - await assert.rejects(() => (target as unknown as Record Promise>)[method](), /only supported in/, `${method} rejects in Node`); - } - }); - - // ---- teardown --------------------------------------------------------------------------------------------------- - - step('deleteFileBasicUsage', { phase: 'P9', needs: ['publishFileMessageBasicUsage', 'downloadFileNodeBasicUsage', 'getFileUrlBasicUsage', 'listFilesBasicUsage'], proves: 'deleteFile removes the file and a later listing no longer shows it.' }, async () => { - const pubnub = world.shopper; - uploadedFile(); - // snippet.deleteFileBasicUsage - const deleteFileResponse = await pubnub.deleteFile({ - channel: 'my_channel', - id: '...', - name: 'cat_picture.jpg', - }); - console.log('File deleted successfully:', deleteFileResponse); - // snippet.end - assert.equal(deleteFileResponse.status, 200); - await eventually(async () => { - const left = await world.observer.listFiles({ channel: world.n('my_channel') }); - assert.ok(!left.data.some((f) => f.id === uploaded!.id), 'the file is no longer listed'); - }, { label: 'file removed from the list' }); - }); -}); diff --git a/docs-snippets/api-reference/message-actions.ts b/docs-snippets/api-reference/message-actions.ts deleted file mode 100644 index d488a4b89..000000000 --- a/docs-snippets/api-reference/message-actions.ts +++ /dev/null @@ -1,92 +0,0 @@ -import assert from '../harness/assert.js'; -import { page, step } from '../harness/step.js'; -import { eventually, watch } from '../harness/wait.js'; -import { addToTimetoken } from '../harness/assertions.js'; - -/** - * Message action examples. - */ -export default page('api-reference/message-actions', 'P4', (world) => { - step('addMessageActionBasicUsage', { needs: ['preflightPersistence'], proves: 'addMessageAction returns the action and a subscriber receives an added event.' }, async () => { - const pubnub = world.shopper; - const channel = world.n('channel_name'); - const listening = await watch(world.fresh('observer'), [channel]); - const published = await world.friend.publish({ channel, message: { text: 'react to me' } }); - world.setPh('messageTimetoken', published.timetoken); - - // snippet.addMessageActionBasicUsage - // first publish a message using publish() method to get the message timetoken - const response = await pubnub.addMessageAction({ - channel: 'channel_name', - messageTimetoken: 'replace_with_message_timetoken', // Replace with actual message timetoken - action: { - type: 'reaction', - value: 'smiley_face', - }, - }); - console.log('Message reaction added successfully:', response); - - // snippet.end - world.setPh('actionTimetoken', response.data.actionTimetoken); - assert.equal(response.data.type, 'reaction'); - assert.equal(response.data.value, 'smiley_face'); - assert.equal(response.data.messageTimetoken, published.timetoken); - const event = await listening.next('messageAction', (e) => e.data.actionTimetoken === response.data.actionTimetoken); - assert.equal(event.event, 'added'); - assert.equal(event.data.uuid, pubnub.getUserId()); - listening.stop(); - }); - - step('removeMessageActionBasicUsage', { needs: ['addMessageActionBasicUsage', 'preflightPersistence'], proves: 'removeMessageAction emits a removed event and the action disappears from the stored list.' }, async () => { - const pubnub = world.shopper; - const channel = world.n('channel_name'); - const listening = await watch(world.fresh('observer'), [channel]); - const actionTimetoken = world.ph('actionTimetoken'); - - // snippet.removeMessageActionBasicUsage - const response = await pubnub.removeMessageAction({ - channel: 'channel_name', - messageTimetoken: 'replace_with_message_timetoken', - actionTimetoken: 'replace_with_action_timetoken', - }); - console.log('Message action removed successfully:', response); - // snippet.end - const event = await listening.next('messageAction', (e) => e.data.actionTimetoken === actionTimetoken); - assert.equal(event.event, 'removed'); - await eventually(async () => { - const left = await world.observer.getMessageActions({ channel }); - assert.ok(!left.data.some((a) => a.actionTimetoken === actionTimetoken), 'action is gone'); - }, { label: 'removed action is no longer listed' }); - listening.stop(); - }); - - step('getMessageActionsBasicUsage', { needs: ['preflightPersistence'], proves: 'getMessageActions returns the one action added during setup with the right type, value and message timetoken.' }, async () => { - const pubnub = world.shopper; - const channel = world.n('channel_name'); - const published = await world.friend.publish({ channel, message: { text: 'count my reactions' } }); - const added = await world.friend.addMessageAction({ - channel, - messageTimetoken: published.timetoken, - action: { type: 'reaction', value: 'thumbs_up' }, - }); - // Actions are listed newest first: `start` is the exclusive upper bound, `end` the inclusive lower bound. - world.setPh('actionsStart', addToTimetoken(added.data.actionTimetoken, 1)); - world.setPh('actionsEnd', added.data.actionTimetoken); - - // snippet.getMessageActionsBasicUsage - // to get some data in response, first publish a message and then add a message action - // using addMessageAction() method. - const response = await pubnub.getMessageActions({ - channel: 'channel_name', - start: 'replace_with_start_timetoken', - end: 'replace_with_end_timetoken', - limit: 100, - }); - console.log('Message actions retrieved successfully:', response); - // snippet.end - assert.equal(response.data.length, 1); - assert.equal(response.data[0].actionTimetoken, added.data.actionTimetoken); - assert.equal(response.data[0].value, 'thumbs_up'); - assert.equal(response.data[0].messageTimetoken, published.timetoken); - }); -}); diff --git a/docs-snippets/api-reference/message-persistence-legacy.ts b/docs-snippets/api-reference/message-persistence-legacy.ts deleted file mode 100644 index f656cd510..000000000 --- a/docs-snippets/api-reference/message-persistence-legacy.ts +++ /dev/null @@ -1,384 +0,0 @@ -import type PubNub from 'pubnub'; -import assert from '../harness/assert.js'; -import { page, step, check } from '../harness/step.js'; -import { eventually, sleep } from '../harness/wait.js'; -import { recorder } from '../harness/recorder.js'; -import { addToTimetoken } from '../harness/assertions.js'; - -/** - * Examples for the deprecated `history()` method and the deprecated `data` property of `actions` on `fetchMessages()`. - */ -export default page('api-reference/message-persistence-legacy', 'P8', (world) => { - // Publishes `count` messages as the friend, optionally with `meta`, and returns their timetokens, oldest first. - const seed = async (channel: string, count: number, extra: { meta?: Record } = {}): Promise => { - const tokens: string[] = []; - for (let i = 0; i < count; i += 1) { - const { timetoken } = await world.friend.publish({ channel, message: { legacy: 'history', n: i }, ...extra }); - tokens.push(timetoken); - } - return tokens; - }; - - // Fine for up to 100 messages. For larger sets use untilAllStored below. - const untilStored = (channel: string, tokens: string[]): Promise => - eventually(async () => { - const history = await world.observer.fetchMessages({ channels: [channel], count: 100 }); - const have = new Set((history.channels[channel] ?? []).map((m) => String(m.timetoken))); - for (const token of tokens) assert.ok(have.has(token), `${token} is not stored yet`); - }, { timeout: 30_000, label: `messages stored on ${channel}` }); - - /** - * Waits until every one of `tokens` is stored, however many there are. `untilStored` reads one page of at most 100 - * messages (the cap of `fetchMessages`), so with more than 100 seeded messages it can only vouch for the newest - * 100. This one walks the channel page by page from the newest message to the oldest, always asking for the page - * that ends before the oldest message of the previous page, and checks the whole set of tokens at the end. - * Storage is eventually consistent, so the walk repeats until nothing is missing or the timeout passes. - */ - const untilAllStored = (channel: string, tokens: string[]): Promise => - eventually(async () => { - const have = new Set(); - let before: string | undefined; - for (;;) { - const page = await world.observer.fetchMessages({ channels: [channel], count: 100, ...(before ? { start: before } : {}) }); - const items = page.channels[channel] ?? []; - for (const item of items) have.add(String(item.timetoken)); - // A page shorter than the cap is the last one. Messages come oldest first, so items[0] is the oldest of the page. - if (items.length < 100) break; - before = String(items[0].timetoken); - } - const missing = tokens.filter((token) => !have.has(token)); - assert.equal(missing.length, 0, `${missing.length} of ${tokens.length} seeded messages are not stored yet (first missing: ${missing[0]})`); - }, { timeout: 45_000, label: `all ${tokens.length} messages stored on ${channel}` }); - - /** Every stored message of a channel, oldest first, through the non-deprecated reader. */ - // Reference reader for expected values: the non-deprecated fetchMessages, so history() is compared against an independent source. - const stored = async (channel: string): Promise => { - const history = await world.observer.fetchMessages({ channels: [channel], count: 100 }); - return (history.channels[channel] ?? []).map((m) => String(m.timetoken)); - }; - - // Normalizes timetokens to strings, history() returns numbers unless stringifiedTimeToken is set. - const tokensOf = (result: { messages: Array<{ timetoken: string | number }> }): string[] => result.messages.map((m) => String(m.timetoken)); - - // history() always returns oldest first, whatever `reverse` is. - const isAscending = (tokens: string[]): boolean => tokens.every((t, i) => i === 0 || BigInt(tokens[i - 1]) < BigInt(t)); - - step('historyBasicUsage', { needs: ['preflightPersistence'], proves: 'history returns the three seeded messages oldest first with the start and end timetokens.' }, async () => { - const pubnub = world.shopper; - const channel = world.n('history_channel'); - const tokens = await seed(channel, 3); - await untilStored(channel, tokens); - - // snippet.historyBasicUsage - try { - const result = await pubnub.history({ - channel: "history_channel", - count: 100, // how many items to fetch - stringifiedTimeToken: true, // false is the default - }); - // snippet.hide - world.keep('result', result); - // snippet.show - } catch (status) { - console.log(status); - } - // snippet.end - const result = world.take('result'); - - assert.deepEqual(tokensOf(result), tokens, 'the three seeded messages, oldest first'); - assert.deepEqual((result.messages[0].entry as { n: number }).n, 0); - assert.equal(result.startTimeToken, tokens[0]); - assert.equal(result.endTimeToken, tokens[2]); - }); - - step('historyOldestThree', { needs: ['preflightPersistence'], proves: 'history with reverse true and count 3 returns the three oldest messages.' }, async () => { - const pubnub = world.shopper; - const channel = world.n('my_channel'); - const tokens = await seed(channel, 3); - await untilStored(channel, tokens); - const expected = (await stored(channel)).slice(0, 3); - assert.equal(expected.length, 3); - - // snippet.historyOldestThree - try { - const result = await pubnub.history({ - channel: "my_channel", - reverse: true, // Setting to true will traverse the time line in reverse starting with the oldest message first. - count: 3, // how many items to fetch - stringifiedTimeToken: true, // false is the default - }); - // snippet.hide - world.keep('result', result); - // snippet.show - } catch (status) { - console.log(status); - } - // snippet.end - const result = world.take('result'); - - assert.deepEqual(tokensOf(result), expected, 'the three oldest messages of the channel'); - }); - - step('historyNewerThanTimetoken', { needs: ['preflightPersistence'], proves: 'history with reverse and start returns only messages newer than the anchor, and start is exclusive.' }, async () => { - const pubnub = world.shopper; - const channel = world.n('my_channel'); - const [, middle, newest] = await seed(channel, 3); - await untilStored(channel, [newest]); - world.setPh('historyAnchor', middle); - - // snippet.historyNewerThanTimetoken - try { - const result = await pubnub.history({ - channel: "my_channel", - reverse: true, // Setting to true will traverse the time line in reverse starting with the oldest message first. - stringifiedTimeToken: true, // false is the default - start: "13406746780720711", // start timetoken to fetch - }); - // snippet.hide - world.keep('result', result); - // snippet.show - } catch (status) { - console.log(status); - } - // snippet.end - const result = world.take('result'); - - const tokens = tokensOf(result); - assert.equal(tokens[0], newest, 'start is exclusive: the first message is the one after the anchor'); - assert.ok(!tokens.includes(middle), 'the anchor itself is not returned'); - assert.ok(tokens.every((t) => BigInt(t) > BigInt(middle))); - assert.ok(isAscending(tokens)); - }); - - step('historyUntilTimetoken', { needs: ['preflightPersistence'], proves: 'history with only end returns messages down to the anchor, and end is inclusive.' }, async () => { - const pubnub = world.shopper; - const channel = world.n('my_channel'); - const [oldest, middle, newest] = await seed(channel, 3); - await untilStored(channel, [newest]); - world.setPh('historyAnchor', middle); - - // snippet.historyUntilTimetoken - try { - const result = await pubnub.history({ - channel: "my_channel", - stringifiedTimeToken: true, // false is the default - end: "13406746780720711", // end timetoken to fetch - }); - // snippet.hide - world.keep('result', result); - // snippet.show - } catch (status) { - console.log(status); - } - // snippet.end - const result = world.take('result'); - - const tokens = tokensOf(result); - assert.equal(tokens[0], middle, 'end is inclusive: the anchor is the oldest returned message'); - assert.ok(tokens.includes(newest)); - assert.ok(!tokens.includes(oldest), 'older than the anchor is not returned'); - assert.ok(isAscending(tokens)); - }); - - step('historyPagingExample', { needs: ['preflightPersistence'], timeout: 120_000, proves: 'The paging helper of the page collects all 120 messages across two pages.' }, async () => { - const pubnub = world.fresh('history-paging'); - const originalHistory = pubnub.history.bind(pubnub) as (...args: unknown[]) => Promise; - (pubnub as unknown as { history: unknown }).history = async (...args: unknown[]): Promise => { - const attempts = 3; - for (let attempt = 1; ; attempt += 1) { - try { - return await originalHistory(...args); - } catch (error) { - const category = (error as { status?: { category?: string } }).status?.category; - if (category !== 'PNNetworkIssuesCategory') throw error; - if (attempt >= attempts) { - (error as Error).message = `${(error as Error).message} (history failed ${attempts} times with a network error)`; - throw error; - } - recorder.record({ kind: 'retry', label: `history retried after a network error (attempt ${attempt} of ${attempts})`, ok: true }); - await sleep(1_000 * attempt); - } - } - }; - const channel = world.n('history_test'); - // More than one page of 100 messages, so the loop has to page. - const tokens: string[] = []; - for (let batch = 0; batch < 6; batch += 1) { - const sent = await Promise.all( - Array.from({ length: 20 }, (_, i) => world.friend.publish({ channel, message: { legacy: 'paging', n: batch * 20 + i } })), - ); - tokens.push(...sent.map((s) => s.timetoken)); - } - // Wait for all 120 seeded messages, not only the newest 100. The oldest 20 sit on the second page. - await untilAllStored(channel, tokens); - await eventually(async () => { - const probe = await world.observer.fetchMessages({ channels: [channel], count: 100 }); - assert.equal(probe.channels[channel]?.length, 100); - }, { timeout: 30_000, label: 'first page is full' }); - - // snippet.historyPagingExample - async function getAllMessages(initialTimetoken = '') { - const allMessages = []; - let latestCount = 100; - let timetoken = initialTimetoken; - - while (latestCount === 100) { - const { messages, startTimeToken, endTimeToken } = await pubnub.history( - { - channel: 'history_test', - stringifiedTimeToken: true, // false is the default - start: timetoken, // start timetoken to fetch - } - ); - - latestCount = messages.length; - timetoken = String(startTimeToken); - - if (messages && messages.length > 0) { - allMessages.push(...messages); - } - } - - return allMessages; - } - - // Usage examples: - // await getAllMessages(); - // await getAllMessages("12345678901234"); - // snippet.end - - // History is eventually consistent, so wait until the oldest messages are stored too. - const all = await eventually(async () => { - const found = await getAllMessages(); - assert.equal(found.length, 120, 'paging returns every message'); - return found; - }, { timeout: 45_000, label: 'paging collects all seeded messages' }); - assert.deepEqual(new Set(all.map((m) => m.timetoken)), new Set(tokens), 'each seeded message exactly once'); - // A valid timetoken starts the walk there: nothing newer than it is returned. - const sortedTokens = [...tokens].sort((a, b) => (BigInt(a) < BigInt(b) ? -1 : 1)); - const fromMiddle = await getAllMessages(sortedTokens[59]); - assert.equal(fromMiddle.length, 59, 'start is exclusive, so only the 59 messages before the 60th are returned'); - }); - - step('historyIncludeMeta', { needs: ['preflightPersistence'], proves: 'includeMeta returns meta for a message published with metadata and omits it for others.' }, async () => { - const pubnub = world.shopper; - const channel = world.n('my_channel'); - const [withMeta] = await seed(channel, 1, { meta: { legacy: 'meta' } }); - await untilStored(channel, [withMeta]); - - // snippet.historyIncludeMeta - try { - const result = await pubnub.history({ - channel: "my_channel", - stringifiedTimeToken: true, - includeMeta: true, - }); - // snippet.hide - world.keep('result', result); - // snippet.show - } catch (status) { - console.log(status); - } - // snippet.end - const result = world.take('result'); - - const found = result.messages.find((m) => m.timetoken === withMeta); - assert.ok(found, 'the seeded message is returned'); - assert.deepEqual(found.meta, { legacy: 'meta' }); - // The meta property is present only when the publisher set metadata. - assert.ok(result.messages.some((m) => m.meta === undefined), 'messages without metadata have no meta property'); - }); - - step('historyWithPromises', { needs: ['preflightPersistence'], proves: 'The promise form of history logs the response with the expected messages.' }, async () => { - const pubnub = world.shopper; - const channel = world.n('history_channel'); - const tokens = await seed(channel, 3); - await untilStored(channel, tokens); - // The docs literals are 1970 timetokens. The live values are the newest boundary (start) and the oldest one (end). - world.setPh('historyPromiseStart', addToTimetoken(tokens[2], 1)); - world.setPh('historyPromiseEnd', tokens[0]); - - await world.withConsole(async (logs) => { - // snippet.historyWithPromises - pubnub.history({ - channel: 'history_channel', - reverse: true, // Setting to true will traverse the time line in reverse starting with the oldest message first. - count: 100, // how many items to fetch - stringifiedTimeToken: true, // false is the default - start: '123123123133', // start timetoken to fetch - end: '123123123123', // end timetoken to fetch - }).then((response) => { - console.log(response); - }).catch((error) => { - console.log(error); - }); - // snippet.end - - await eventually(() => { - assert.ok(logs.lines.some((line) => line.includes('"messages"')), `the response was logged. Logged: ${logs.lines.join(' | ').slice(0, 300)}`); - }, { label: 'promise resolved and logged' }); - const response = JSON.parse(logs.lines.find((line) => line.includes('"messages"'))!); - assert.deepEqual(response.messages.map((m: { timetoken: string }) => String(m.timetoken)), tokens); - }); - }); - - - check('historyDocumentedBehavior', { needs: ['historyPagingExample'], timeout: 60_000, proves: 'The prose claims of the history page hold: the cap of 100, ordering, returned fields and timetoken types.' }, async () => { - const pubnub = world.shopper; - const channel = world.n('history_test'); - const probe = (parameters: Partial[0]>) => - pubnub.history({ channel, stringifiedTimeToken: true, ...parameters }); - - // "The maximum is 100, and history() lowers a larger value to 100." - assert.equal((await probe({ count: 150 })).messages.length, 100); - - // "history() always returns the messages sorted from the oldest to the newest, whatever reverse is." - const newestFive = await probe({ count: 5 }); - const oldestFive = await probe({ count: 5, reverse: true }); - for (const result of [newestFive, oldestFive]) assert.ok(isAscending(tokensOf(result))); - // reverse decides which end of the interval the retrieval starts from. - assert.ok(BigInt(oldestFive.messages[4].timetoken) < BigInt(newestFive.messages[0].timetoken), 'reverse starts from the oldest end'); - - // Returns: startTimeToken and endTimeToken hold the timetokens of the first and last returned message, as strings. - assert.equal(newestFive.startTimeToken, newestFive.messages[0].timetoken); - assert.equal(newestFive.endTimeToken, newestFive.messages[4].timetoken); - assert.equal(typeof newestFive.startTimeToken, 'string'); - for (const message of newestFive.messages) { - assert.deepEqual(Object.keys(message).sort(), ['entry', 'timetoken'], 'an item has entry and timetoken, and meta only when set'); - } - - // Without stringifiedTimeToken the timetokens are numbers. - const plain = await pubnub.history({ channel, count: 1 }); - assert.equal(typeof plain.messages[0].timetoken, 'number'); - - // "With start, start must be a higher timetoken than end." The docs promises sample passes start below end. Record what the service does with that order. - const older = String(oldestFive.messages[0].timetoken); - const newer = String(newestFive.messages[4].timetoken); - const inverted = await probe({ count: 100, start: older, end: newer }); - const ordered = await probe({ count: 100, start: newer, end: older }); - console.log(`history with start below end returned ${inverted.messages.length} messages, with start above end ${ordered.messages.length}`); - assert.ok(ordered.messages.length > 0, 'start above end returns the messages in between'); - }); - - check('historyActionsDataDeprecated', { needs: ['preflightPersistence'], proves: 'The deprecated data property of actions equals actions, and both are absent without includeMessageActions.' }, async () => { - const pubnub = world.shopper; - const channel = world.n('persist_channel'); - const [token] = await seed(channel, 1); - await world.friend.addMessageAction({ channel, messageTimetoken: token, action: { type: 'reaction', value: 'smiley_face' } }); - type WithActions = { timetoken: string | number; actions?: Record>; data?: unknown }; - const found = await eventually(async () => { - const response = await pubnub.fetchMessages({ channels: [channel], includeMessageActions: true }); - const item = response.channels[channel]?.find((m) => String(m.timetoken) === token) as WithActions | undefined; - assert.ok(item?.actions, 'the message has actions'); - return item; - }, { timeout: 30_000, label: 'message with action stored' }); - assert.deepEqual(found.data, found.actions, 'the deprecated data property holds the same value as actions'); - assert.ok(found.actions?.reaction?.smiley_face, 'grouped by action type, then by action value'); - - // "The response has them only with includeMessageActions and when the message has actions." - const without = await pubnub.fetchMessages({ channels: [channel] }); - const plain = without.channels[channel]?.find((m) => String(m.timetoken) === token) as WithActions | undefined; - assert.equal(plain?.actions, undefined); - assert.equal(plain?.data, undefined); - }); -}); diff --git a/docs-snippets/api-reference/message-persistence.ts b/docs-snippets/api-reference/message-persistence.ts deleted file mode 100644 index 8d0166d49..000000000 --- a/docs-snippets/api-reference/message-persistence.ts +++ /dev/null @@ -1,182 +0,0 @@ -import assert from '../harness/assert.js'; -import { page, step } from '../harness/step.js'; -import { eventually } from '../harness/wait.js'; -import { addToTimetoken } from '../harness/assertions.js'; - -/** - * Message persistence examples. - */ -export default page('api-reference/message-persistence', 'P5', (world) => { - // Publishes `count` messages as the friend and returns their timetokens, oldest first. - const seed = async (channel: string, count: number): Promise => { - const tokens: string[] = []; - for (let i = 0; i < count; i += 1) { - const { timetoken } = await world.friend.publish({ channel, message: { seed: i }, customMessageType: 'seed' }); - tokens.push(timetoken); - } - return tokens; - }; - - type WithActions = { actions?: Record> }; - - // Waits until every seeded timetoken is visible to a reader. The write and the read model are separate, so one read is not enough. - const untilStored = (channel: string, tokens: string[]): Promise => - eventually(async () => { - const history = await world.observer.fetchMessages({ channels: [channel], count: 100 }); - const have = new Set((history.channels[channel] ?? []).map((m) => String(m.timetoken))); - for (const token of tokens) assert.ok(have.has(token), `${token} is not stored yet`); - }, { timeout: 30_000, label: `messages stored on ${channel}` }); - - step('fetchMessagesBasicUsage', { needs: ['preflightPersistence'], proves: 'fetchMessages with count, start and end returns exactly one seeded message.' }, async () => { - const pubnub = world.shopper; - const channel = world.n('my-channel'); - const tokens = await seed(channel, 3); - await untilStored(channel, tokens); - // `start` is the exclusive newer bound and `end` the inclusive older bound. - world.setPh('historyStart', addToTimetoken(tokens[2], 1)); - world.setPh('historyEnd', tokens[0]); - - // snippet.fetchMessagesBasicUsage - // Function to fetch message history - const result = await pubnub.fetchMessages({ - channels: ['my-channel'], - count: 1, // Number of messages to retrieve - includeCustomMessageType: true, // if you want to include custom message type in the response - start: 'replace-with-start-timetoken', // start timetoken - end: 'replace-with-end-timetoken', // end timetoken - }); - console.log('Fetched Messages:', result); - // snippet.end - const messages = result.channels[channel] ?? []; - assert.equal(messages.length, 1, 'count: 1 returns one message'); - assert.ok(tokens.includes(String(messages[0].timetoken)), 'the message is one of the seeded ones'); - assert.equal(messages[0].customMessageType, 'seed'); - }); - - step('fetchMessagesWithMetaActions', { needs: ['preflightPersistence'], proves: 'fetchMessages returns meta, message actions and the custom message type when asked for.' }, async () => { - const pubnub = world.shopper; - const channel = world.n('my_channel'); - const { timetoken } = await world.friend.publish({ - channel, - message: { text: 'with meta and an action' }, - meta: { source: 'docs-snippets' }, - customMessageType: 'text-message', - }); - await world.friend.addMessageAction({ - channel, - messageTimetoken: timetoken, - action: { type: 'reaction', value: 'smiley_face' }, - }); - await eventually(async () => { - const probe = await world.observer.fetchMessages({ channels: [channel], includeMessageActions: true, includeMeta: true }); - const found = probe.channels[channel]?.find((m) => String(m.timetoken) === timetoken); - assert.ok((found as WithActions | undefined)?.actions, 'message and its action are stored'); - }, { timeout: 30_000, label: 'message with action stored' }); - - // snippet.fetchMessagesWithMetaActions - const response = await pubnub.fetchMessages({ - channels: ['my_channel'], - stringifiedTimeToken: true, - includeMeta: true, - includeMessageActions: true, - includeCustomMessageType: true, - }); - console.log('fetch messages response:', response); - // snippet.end - const found = response.channels[channel]?.find((m) => String(m.timetoken) === timetoken); - assert.ok(found, 'the seeded message is returned'); - assert.deepEqual(found.meta, { source: 'docs-snippets' }); - assert.equal(found.customMessageType, 'text-message'); - assert.ok((found as WithActions).actions?.reaction?.smiley_face, 'the reaction is attached to the message'); - }); - - step('messageCountBasicUsage', { needs: ['preflightPersistence'], proves: 'messageCounts with one timetoken counts the messages per channel since that time.' }, async () => { - const pubnub = world.shopper; - const [room1, room2] = [world.n('chats.room1'), world.n('chats.room2')]; - const before = (await world.observer.time()).timetoken; - const first = await seed(room1, 2); - const second = await seed(room2, 1); - await untilStored(room1, first); - await untilStored(room2, second); - world.setPh('countFrom', before); - - // snippet.messageCountBasicUsage - const result = await pubnub.messageCounts({ - channels: ['chats.room1', 'chats.room2'], - channelTimetokens: ['replace-with-timetoken'], // single timetoken applied to all channels - }); - console.log('Message counts retrieved successfully:', result); - // snippet.end - assert.deepEqual(result.channels, { [room1]: 2, [room2]: 1 }); - }); - - step('messageCountTimetokensForChannels', { needs: ['preflightPersistence'], proves: 'messageCounts with one timetoken per channel gives the per-channel counts.' }, async () => { - const pubnub = world.shopper; - const channels = [world.n('ch1'), world.n('ch2'), world.n('ch3')]; - // One timetoken per channel, taken just before that channel gets 1, 2 and 3 messages. - for (const [index, channel] of channels.entries()) { - world.setPh(`countFromCh${index + 1}`, (await world.observer.time()).timetoken); - await untilStored(channel, await seed(channel, index + 1)); - } - - // snippet.messageCountTimetokensForChannels - const response = await pubnub.messageCounts({ - channels: ['ch1', 'ch2', 'ch3'], - channelTimetokens: [ - 'replace-with-channel-timetoken-ch1', // timetoken for channel ch1 - 'replace-with-channel-timetoken-ch2', // timetoken for channel ch2 - 'replace-with-channel-timetoken-ch3', // timetoken for channel ch3 - ], - }); - console.log('message count response:', response); - // snippet.end - assert.deepEqual(response.channels, { [channels[0]]: 1, [channels[1]]: 2, [channels[2]]: 3 }); - }); - - // `start` is exclusive and `end` inclusive, so start = T - 1 and end = T delete exactly the message at T. - const deleteOne = async (): Promise<{ channel: string; token: string }> => { - const channel = world.n('ch1'); - const [token] = await seed(channel, 1); - await untilStored(channel, [token]); - world.setPh('historyStart', addToTimetoken(token, -1)); - world.setPh('historyEnd', token); - return { channel, token }; - }; - - // Waits until a deleted message is gone for readers, deletion is also eventually consistent. - const untilDeleted = (channel: string, token: string): Promise => - eventually(async () => { - const history = await world.observer.fetchMessages({ channels: [channel], count: 100 }); - assert.ok(!(history.channels[channel] ?? []).some((m) => String(m.timetoken) === token), 'message is still stored'); - }, { timeout: 30_000, label: 'message deleted' }); - - step('deleteMessagesBasicUsage', { needs: ['preflightDeleteFromHistory'], proves: 'deleteMessages removes exactly the message at the boundary timetoken.' }, async () => { - const pubnub = world.shopper; - const { channel, token } = await deleteOne(); - - // snippet.deleteMessagesBasicUsage - const result = await pubnub.deleteMessages({ - channel: 'ch1', - start: 'replace-with-start-timetoken', - end: 'replace-with-end-timetoken', - }); - console.log('Messages deleted successfully:', result); - // snippet.end - await untilDeleted(channel, token); - }); - - step('deleteSpecificMessages', { needs: ['preflightDeleteFromHistory'], proves: 'The second deleteMessages sample removes exactly the message at its boundary timetoken.' }, async () => { - const pubnub = world.shopper; - const { channel, token } = await deleteOne(); - - // snippet.deleteSpecificMessages - const response = await pubnub.deleteMessages({ - channel: 'ch1', - start: 'replace-with-start-timetoken', - end: 'replace-with-end-timetoken', - }); - console.log('delete messages response:', response); - // snippet.end - await untilDeleted(channel, token); - }); -}); diff --git a/docs-snippets/api-reference/misc.ts b/docs-snippets/api-reference/misc.ts deleted file mode 100644 index c6a9db84d..000000000 --- a/docs-snippets/api-reference/misc.ts +++ /dev/null @@ -1,85 +0,0 @@ -import assert from '../harness/assert.js'; -import type PubNub from 'pubnub'; -import { page, step } from '../harness/step.js'; -import { expectStatus, sleep, watch } from '../harness/wait.js'; - -/** - * Network examples: `disconnect()`, `reconnect()` and `setProxy()`. - */ -export default page('api-reference/misc', 'P7', (world) => { - // Shared between setProxyBasicUsage and deleteProxy, which must act on the same client. - let proxied: PubNub | undefined; - - step('disconnectBasicUsage', { proves: 'disconnect() reports a disconnected status and no message arrives afterwards.' }, async () => { - const pubnub = world.fresh('disconnect'); - const channel = world.n('ch1'); - const listening = await watch(pubnub, [channel]); - const disconnected = expectStatus(pubnub, ['PNDisconnectedCategory']); - - // snippet.disconnectBasicUsage - pubnub.disconnect(); - // snippet.end - - assert.equal(await disconnected, 'PNDisconnectedCategory'); - // While disconnected the client receives nothing. - await world.friend.publish({ channel, message: { while: 'disconnected' } }); - await sleep(3000); - assert.ok( - !listening.events.some((e) => (e.event as { message?: { while?: string } }).message?.while === 'disconnected'), - 'no message arrives while disconnected', - ); - }); - - step('reconnectBasicUsage', { proves: 'reconnect() restores delivery after a disconnect.' }, async () => { - const pubnub = world.fresh('reconnect'); - const channel = world.n('ch1'); - const listening = await watch(pubnub, [channel]); - const disconnected = expectStatus(pubnub, ['PNDisconnectedCategory']); - pubnub.disconnect(); - await disconnected; - const back = expectStatus(pubnub, ['PNReconnectedCategory', 'PNConnectedCategory']); - - // snippet.reconnectBasicUsage - pubnub.reconnect(); - // snippet.end - - await back; - await world.friend.publish({ channel, message: { after: 'reconnect' } }); - await listening.next('message', (m) => (m.message as { after?: string }).after === 'reconnect'); - }); - - step('setProxyBasicUsage', { proves: 'setProxy routes the client traffic through the proxy, seen as a tunnel request.' }, async () => { - await world.proxy.start(); - const pubnub = world.fresh('proxy'); - proxied = pubnub; - - // snippet.setProxyBasicUsage - // This method is only available for NodeJS. - pubnub.setProxy({ - hostname: 'YOUR_HOSTNAME', - port: 8080, - protocol: 'YOUR_PROTOCOL', - // snippet.hide - ...world.proxy.config, - // snippet.show - }); - // snippet.end - - const tunnelsBefore = world.proxy.connects; - world.check.serverTime((await pubnub.time()).timetoken); - assert.ok(world.proxy.connects > tunnelsBefore, 'the request went through the local proxy'); - }); - - step('deleteProxy', { needs: ['setProxyBasicUsage'], proves: 'setProxy with no argument removes the proxy so the next request adds no tunnel.' }, async () => { - const pubnub = proxied; - assert.ok(pubnub, 'setProxyBasicUsage has not run'); - - // snippet.deleteProxy - pubnub.setProxy(); - // snippet.end - - const tunnelsBefore = world.proxy.connects; - world.check.serverTime((await pubnub.time()).timetoken); - assert.equal(world.proxy.connects, tunnelsBefore, 'no request goes through the proxy any more'); - }); -}); diff --git a/docs-snippets/api-reference/mobile-push.ts b/docs-snippets/api-reference/mobile-push.ts deleted file mode 100644 index 53aa1cb67..000000000 --- a/docs-snippets/api-reference/mobile-push.ts +++ /dev/null @@ -1,336 +0,0 @@ -import assert from '../harness/assert.js'; -import PubNub from 'pubnub'; -import { page, step, check, skipStep } from '../harness/step.js'; -import { eventually, watch } from '../harness/wait.js'; -import { explainError } from '../harness/guard.js'; -import { DEVICE_TOKEN_EXAMPLE } from '../fixtures/names.js'; -import { registry } from '../harness/registry.js'; - -/** - * Mobile push notification examples. Registering a device token does not send a push. Delivery needs a real device and - * APNs or FCM credentials. - */ -export default page('api-reference/mobile-push', 'P6', (world) => { - // Registration order is not guaranteed by the service, so every list comparison sorts first. - const sorted = (values: string[]): string[] => [...values].sort(); - const ab = (): string[] => sorted([world.n('a'), world.n('b')]); - const device = (): string => world.n(DEVICE_TOKEN_EXAMPLE); - const apns2Options = { pushGateway: 'apns2', environment: 'production', topic: 'com.example.bundle_id' } as const; - - function serviceText(error: unknown): string { - explainError(error); - const status = (error as { status?: { statusCode?: unknown; errorData?: unknown } } | undefined)?.status; - const body = status?.errorData === undefined ? '' : JSON.stringify(status.errorData); - const message = error instanceof Error ? error.message : String(error); - return `${message} status=${String(status?.statusCode ?? '')} body=${body}`; - } - // The service text for "add-on not enabled". Anything else is a real failure. - const PUSH_OFF = /requires Push Notifications which is not enabled/i; - let probe: Promise<{ off: false } | { off: true; detail: string }> | undefined; - - /** Registers and removes a hex token on fcm and apns2 once per run, and tells whether the add-on is off. */ - function pushAvailability(): Promise<{ off: false } | { off: true; detail: string }> { - probe ??= (async () => { - const client = world.shopper; - const channel = world.n('preflight_channel'); - try { - // fcm accepts any string, apns2 needs the hex format. Both use this run's hex token. - await client.push.addChannels({ channels: [channel], device: device(), pushGateway: 'fcm' }); - await client.push.addChannels({ channels: [channel], device: device(), ...apns2Options }); - return { off: false } as const; - } catch (error) { - const detail = serviceText(error); - if (PUSH_OFF.test(detail)) return { off: true, detail } as const; - throw new Error( - `A push registration with a synthetic ${DEVICE_TOKEN_EXAMPLE.length}-character hex device token was rejected, and the answer is not the "add-on not enabled" 403. ` + - `Push Notifications is on for this keyset, so this is a service or SDK problem. ${detail}`, - ); - } finally { - await client.push.deleteDevice({ device: device(), pushGateway: 'fcm' }).catch(() => undefined); - await client.push.deleteDevice({ device: device(), ...apns2Options }).catch(() => undefined); - } - })(); - return probe; - } - - async function requirePush(): Promise { - const result = await pushAvailability(); - if (result.off) { - skipStep( - 'needs-feature:push-notifications', - `the add-on is off for the open keyset, so registration was not exercised (${result.detail}). Enable Push Notifications for it in the Admin Portal, no APNs or FCM credentials are needed.`, - ); - } - await world.observer.objects - .setChannelMetadata({ channel: world.n('push_marker'), data: { name: 'mobile push run marker' } }) - .catch(() => undefined); - } - - // Wired through requirePush so a keyset without the add-on skips here as needs-feature and not as a failure. - check('preflightMobilePush', { phase: 'P0', timeout: 60_000, proves: 'Preflight: the keyset has the Mobile Push add-on, otherwise push steps skip as needs-feature.' }, async () => { - await requirePush(); - }); - - check('pushDeviceTokenFormat', { timeout: 60_000, proves: 'Pins which device token formats each push gateway accepts, behind a documented docs problem.' }, async () => { - await requirePush(); - const client = world.shopper; - const channel = world.n('a'); - const docsToken = 'niceDevice'; - for (const [label, options] of [['apns', { pushGateway: 'apns' }], ['apns2', apns2Options]] as const) { - let rejected: unknown; - try { - await client.push.addChannels({ channels: [channel], device: docsToken, ...options }); - } catch (error) { - rejected = error; - } - assert.ok(rejected, `${label} accepted the docs token "${docsToken}", the docs bug is gone, update the docs`); - const detail = serviceText(rejected); - assert.match(detail, /status=400/, `${label}: expected status 400, got ${detail}`); - assert.match(detail, /Invalid device token/i, `${label}: expected "Invalid device token", got ${detail}`); - } - const fcmToken = world.n(docsToken); - try { - await client.push.addChannels({ channels: [channel], device: fcmToken, pushGateway: 'fcm' }); - const listed = await world.observer.push.listChannels({ device: fcmToken, pushGateway: 'fcm' }); - assert.deepEqual(listed.channels, [channel], 'fcm registered a non-hex token'); - } finally { - await client.push.deleteDevice({ device: fcmToken, pushGateway: 'fcm' }).catch(() => undefined); - } - }); - - function requireStep(id: string): void { - const result = registry.result(id); - if (result && result.status !== 'passed') skipStep(`blocked-by:${id}`); - } - - step('addDeciveToChannelBasicUsage', { timeout: 60_000, proves: 'addChannels registers a device token for a channel and another client reads the registration back.' }, async () => { - const pubnub = world.shopper; - await requirePush(); - // snippet.addDeciveToChannelBasicUsage - // Function to add a device to a channel for APNs2 - const response = await pubnub.push.addChannels({ - channels: ['a', 'b'], - device: '740f4707bebcf74f9b7c25d48e3358945f6aa01da5ddb387462c7eaf61bb78ad', - pushGateway: 'apns2', - environment: 'production', - topic: 'com.example.bundle_id', - }); - console.log('device added to channels response:', response); - - // Function to add a device to a channel for FCM - const response2 = await pubnub.push.addChannels({ - channels: ['a', 'b'], - device: '740f4707bebcf74f9b7c25d48e3358945f6aa01da5ddb387462c7eaf61bb78ad', - pushGateway: 'fcm', - }); - console.log('device added to channels response:', response2); - // snippet.end - await eventually(async () => { - const apns2 = await world.observer.push.listChannels({ - device: device(), - pushGateway: 'apns2', - environment: 'production', - topic: 'com.example.bundle_id', - }); - assert.deepEqual(sorted(apns2.channels), ab(), 'apns2 registration'); - const fcm = await world.observer.push.listChannels({ device: device(), pushGateway: 'fcm' }); - assert.deepEqual(sorted(fcm.channels), ab(), 'fcm registration'); - }, { label: 'registrations visible' }); - }); - - step('listChannelsForDeviceBasicUsage', { timeout: 60_000, proves: 'listChannels returns the channels registered for the device.' }, async () => { - const pubnub = world.shopper; - await requirePush(); - requireStep('addDeciveToChannelBasicUsage'); - // snippet.listChannelsForDeviceBasicUsage - // for APNs2 - const response = await pubnub.push.listChannels({ - device: '740f4707bebcf74f9b7c25d48e3358945f6aa01da5ddb387462c7eaf61bb78ad', - pushGateway: 'apns2', - environment: 'production', - topic: 'com.example.bundle_id', - }); - console.log('listing channels for device response:', response); - response.channels.forEach((channel) => { - console.log(channel); - }); - - // for FCM - const response2 = await pubnub.push.listChannels({ - device: '740f4707bebcf74f9b7c25d48e3358945f6aa01da5ddb387462c7eaf61bb78ad', - pushGateway: 'fcm', - }); - - console.log('listing channels for device response:', response2); - - response2.channels.forEach((channel) => { - console.log(channel); - }); - // snippet.end - assert.deepEqual(sorted(response.channels), ab(), 'apns2 list'); - assert.deepEqual(sorted(response2.channels), ab(), 'fcm list'); - }); - - step('removeDeviceFromChannelBasicUsage', { timeout: 60_000, proves: 'removeChannels unregisters the device and both gateways list nothing afterwards.' }, async () => { - const pubnub = world.shopper; - await requirePush(); - requireStep('listChannelsForDeviceBasicUsage'); - // snippet.removeDeviceFromChannelBasicUsage - // for APNs2 - const response = await pubnub.push.removeChannels({ - channels: ['a', 'b'], - device: '740f4707bebcf74f9b7c25d48e3358945f6aa01da5ddb387462c7eaf61bb78ad', - pushGateway: 'apns2', - environment: 'production', - topic: 'com.example.bundle_id', - }); - - console.log('removing device from channel response:', response); - - // for FCM - const response2 = await pubnub.push.removeChannels({ - channels: ['a', 'b'], - device: '740f4707bebcf74f9b7c25d48e3358945f6aa01da5ddb387462c7eaf61bb78ad', - pushGateway: 'fcm', - }); - - console.log('removing device from channel response:', response2); - // snippet.end - // Read-back: after removeChannels a fresh list per gateway (other client) must be empty. - await eventually(async () => { - const apns2 = await world.observer.push.listChannels({ - device: device(), - pushGateway: 'apns2', - environment: 'production', - topic: 'com.example.bundle_id', - }); - assert.deepEqual(apns2.channels, []); - const fcm = await world.observer.push.listChannels({ device: device(), pushGateway: 'fcm' }); - assert.deepEqual(fcm.channels, []); - }, { label: 'registrations removed' }); - }); - - step('removeAllMobilePushNotificationsBasicUsage', { timeout: 60_000, proves: 'deleteDevice removes every registration of the device and the list is empty afterwards.' }, async () => { - const pubnub = world.shopper; - await requirePush(); - requireStep('removeDeviceFromChannelBasicUsage'); - await world.friend.push.addChannels({ channels: [world.n('a')], device: device(), ...apns2Options }); - await world.friend.push.addChannels({ channels: [world.n('a')], device: device(), pushGateway: 'fcm' }); - await eventually(async () => { - const apns2 = await world.observer.push.listChannels({ device: device(), ...apns2Options }); - assert.deepEqual(apns2.channels, [world.n('a')], 'apns2 registered before deleteDevice'); - const fcm = await world.observer.push.listChannels({ device: device(), pushGateway: 'fcm' }); - assert.deepEqual(fcm.channels, [world.n('a')], 'fcm registered before deleteDevice'); - }, { label: 'registrations visible before deleteDevice' }); - // snippet.removeAllMobilePushNotificationsBasicUsage - - // for APNs2 - const response = await pubnub.push.deleteDevice({ - device: '740f4707bebcf74f9b7c25d48e3358945f6aa01da5ddb387462c7eaf61bb78ad', - pushGateway: 'apns2', - environment: 'production', - topic: 'com.example.bundle_id', - }); - - console.log('deleteDevice response:', response); - - // for FCM - const response2 = await pubnub.push.deleteDevice({ - device: '740f4707bebcf74f9b7c25d48e3358945f6aa01da5ddb387462c7eaf61bb78ad', - pushGateway: 'fcm', - }); - - console.log('deleteDevice response:', response2); - - // snippet.end - // Read-back: deleteDevice must leave no channel on either gateway. - await eventually(async () => { - const apns2 = await world.observer.push.listChannels({ device: device(), ...apns2Options }); - assert.deepEqual(apns2.channels, [], 'apns2 device deleted'); - const fcm = await world.observer.push.listChannels({ device: device(), pushGateway: 'fcm' }); - assert.deepEqual(fcm.channels, [], 'fcm device deleted'); - }, { label: 'device deleted' }); - }); - - step('buildNotificationPayloadBasicUsage', { proves: 'The notification payload builder output is published and a subscriber receives the message.' }, async () => { - const pubnub = world.shopper; - const listening = await watch(world.fresh('pushobserver'), [world.n('chat-bot')]); - // snippet.buildNotificationPayloadBasicUsage - - const builder = PubNub.notificationPayload('Chat invitation', "You have been invited to 'quiz' chat"); - builder.apns.configurations = [{ targets: [{ topic: 'com.example.bundle_id' }] }]; - const messagePayload = builder.buildPayload(['apns2', 'fcm']); - // add required fields to the payload - const response = await pubnub.publish({ - message: messagePayload, - channel: 'chat-bot', - }); - console.log('publish response:', response); - // snippet.end - world.check.timetoken(response.timetoken); - assert.ok(messagePayload.pn_apns && messagePayload.pn_fcm, 'the payload carries the APNs and FCM sections'); - const received = await listening.next('message', (m) => m.timetoken === String(response.timetoken)); - assert.ok(received.message, 'subscribers get the message'); - listening.stop(); - }); - - // ---- payload builders, local --------------------------------------------------------------------------------------- - - step('simpleNotificationPayloadFCMandAPNS', { proves: 'The payload builder prints a payload with the FCM and APNs sections the sample describes.' }, async () => { - const logs = world.captureConsole('log'); - // snippet.simpleNotificationPayloadFCMandAPNS - const builder = PubNub.notificationPayload('Chat invitation', "You have been invited to 'quiz' chat"); - builder.sound = 'default'; - - console.log(JSON.stringify(builder.buildPayload(['apns', 'fcm']), null, 2)); - // snippet.end - logs.restore(); - const alert = { title: 'Chat invitation', body: "You have been invited to 'quiz' chat" }; - assert.deepEqual(JSON.parse(logs.lines[0]), { - pn_apns: { aps: { alert, sound: 'default' } }, - pn_fcm: { notification: alert, android: { notification: { sound: 'default' } } }, - }); - }); - - step('simpleNotificationPayloadFCMandAPNSH/2', { proves: 'The payload builder produces the APNs HTTP/2 section as documented.' }, async () => { - const logs = world.captureConsole('log'); - // snippet.simpleNotificationPayloadFCMandAPNSH/2 - const payloadBuilder = PubNub.notificationPayload('Chat invitation', "You have been invited to 'quiz' chat"); - payloadBuilder.apns.configurations = [{ targets: [{ topic: 'com.meetings.chat.app' }] }]; - payloadBuilder.sound = 'default'; - - console.log(JSON.stringify(payloadBuilder.buildPayload(['apns2', 'fcm']), null, 2)); - // snippet.end - logs.restore(); - const payload = JSON.parse(logs.lines[0]); - assert.equal(payload.pn_apns.pn_push[0].targets[0].topic, 'com.meetings.chat.app'); - assert.equal(payload.pn_apns.pn_push[0].version, 'v2'); - assert.equal(payload.pn_apns.aps.sound, 'default'); - assert.equal(payload.pn_fcm.android.notification.sound, 'default'); - }); - - // Checks that collapseId and expirationDate reach the payload in the snake case wire format. - step('simpleNotificationPayloadFCMandAPNSH/2CustomConfiguration', { proves: 'collapseId and expirationDate reach the payload in the wire format.' }, async () => { - const logs = world.captureConsole('log'); - // snippet.simpleNotificationPayloadFCMandAPNSH/2CustomConfiguration - const configuration = [ - { - collapseId: 'invitations', - expirationDate: new Date(Date.now() + 10000), - targets: [{ topic: 'com.meetings.chat.app' }], - }, - ]; - const customConfigurationBuilder = PubNub.notificationPayload( - 'Chat invitation', - "You have been invited to 'quiz' chat", - ); - customConfigurationBuilder.apns.configurations = configuration; - - console.log(JSON.stringify(customConfigurationBuilder.buildPayload(['apns2', 'fcm']), null, 2)); - // snippet.end - logs.restore(); - const payload = JSON.parse(logs.lines[0]); - assert.equal(payload.pn_apns.pn_push[0].collapse_id, 'invitations'); - assert.equal(payload.pn_apns.pn_push[0].expiration, configuration[0].expirationDate.toISOString()); - }); -}); diff --git a/docs-snippets/api-reference/objects.ts b/docs-snippets/api-reference/objects.ts deleted file mode 100644 index 89581e902..000000000 --- a/docs-snippets/api-reference/objects.ts +++ /dev/null @@ -1,431 +0,0 @@ -import assert from '../harness/assert.js'; -import { page, step, check } from '../harness/step.js'; -import { eventually, watch } from '../harness/wait.js'; -import { explainError } from '../harness/guard.js'; - -/** - * App Context examples. - */ -export default page('api-reference/objects', 'P3', (world) => { - // Sorted ids, so list assertions do not depend on service ordering. - const ids = (list: Array<{ id: string }>): string[] => list.map((o) => o.id).sort(); - - check('preflightAppContext', { phase: 'P0', proves: 'Preflight: the open keyset has App Context enabled, otherwise the App Context steps are blocked.' }, async () => { - try { - await world.observer.objects.getAllUUIDMetadata({ limit: 1 }); - } catch (error) { - explainError(error); - const detail = error instanceof Error ? error.message : String(error); - throw new Error(`The open keyset does not have App Context. Enable "App Context" for it. ${detail}`); - } - }); - - check('appContextFixture', { needs: ['preflightAppContext'], proves: 'Fixture: a seed channel exists and is readable from another client.' }, async () => { - // The channel the docs read (`team.blue`), named so that the "*Team" filter finds it. - await world.friend.objects.setChannelMetadata({ - channel: world.n('team.blue'), - data: { name: 'Blue Team', description: 'The channel for Blue team.' }, - }); - await eventually(async () => { - const blue = await world.observer.objects.getChannelMetadata({ channel: world.n('team.blue') }); - assert.equal(blue.data.name, 'Blue Team'); - }, { label: 'team.blue visible' }); - }); - - // ---- users ---------------------------------------------------------------------------------------------------- - - step('setUUIDMetadataBasicUsage', { needs: ['appContextFixture'], proves: 'setUUIDMetadata stores the user and another client reads it back.' }, async () => { - const pubnub = world.shopper; - // snippet.setUUIDMetadataBasicUsage - // Using UUID from the config - default when uuid is not passed in the method - const response = await pubnub.objects.setUUIDMetadata({ - data: { - name: 'John Doe', - }, - }); - console.log('setUUIDMetadata response:', response); - - // Using the passed in UUID - const response2 = await pubnub.objects.setUUIDMetadata({ - uuid: 'myUuid', - data: { - email: 'john.doe@example.com', - }, - }); - console.log('setUUIDMetadata response:', response2); - // snippet.end - assert.equal(response.data.id, pubnub.getUserId()); - assert.equal(response.data.name, 'John Doe'); - assert.equal(response2.data.id, world.n('myUuid')); - assert.equal(response2.data.email, 'john.doe@example.com'); - await eventually(async () => { - const stored = await world.observer.objects.getUUIDMetadata({ uuid: world.n('myUuid') }); - assert.equal(stored.data.email, 'john.doe@example.com'); - }, { label: 'stored user visible to another client' }); - }); - - step('getUUIDMetadataBasicUsage', { needs: ['setUUIDMetadataBasicUsage'], proves: 'getUUIDMetadata reads the user the previous step stored.' }, async () => { - const pubnub = world.shopper; - // snippet.getUUIDMetadataBasicUsage - // Using UUID from the config - default when uuid is not passed in the method - const response = await pubnub.objects.getUUIDMetadata(); - console.log('getUUIDMetadata response:', response); - - // Using the passed in UUID - const response2 = await pubnub.objects.getUUIDMetadata({ - uuid: 'myUuid', - }); - console.log('getUUIDMetadata response:', response2); - // snippet.end - assert.equal(response.data.id, pubnub.getUserId()); - assert.equal(response.data.name, 'John Doe'); - assert.equal(response2.data.id, world.n('myUuid')); - assert.equal(response2.data.email, 'john.doe@example.com'); - }); - - step('getAllUUIDMetadataBasicUsage', { needs: ['setUUIDMetadataBasicUsage'], proves: 'getAllUUIDMetadata runs the sample function and logs the user list.' }, async () => { - const pubnub = world.shopper; - const logs = world.captureConsole('log'); - try { - // snippet.getAllUUIDMetadataBasicUsage - // Function to get all UUID metadata - // to get some data in response, add user metadata using setUUIDMetadata method - async function getAllUUIDMetadata() { - const response = await pubnub.objects.getAllUUIDMetadata(); - console.log('getAllUUIDMetadata response:', response); - } - - // Execute the function to get UUID metadata - await getAllUUIDMetadata(); - // snippet.end - assert.ok(logs.lines.some((line) => line.startsWith('getAllUUIDMetadata response:')), 'the function logged its response'); - } finally { - logs.restore(); - } - const listed = await pubnub.objects.getAllUUIDMetadata({ limit: 100, sort: { updated: 'desc' } }); - assert.ok(listed.data.some((u) => u.id === world.n('myUuid')), 'the stored user is part of the list'); - }); - - step('removeUUIDMetadataBasicUsage', { phase: 'P9', needs: ['getUUIDMetadataBasicUsage', 'getAllUUIDMetadataBasicUsage'], proves: 'removeUUIDMetadata deletes the users and reads fail with not found afterwards.' }, async () => { - const pubnub = world.shopper; - // snippet.removeUUIDMetadataBasicUsage - // Using UUID from the config - default when uuid is not passed in the method - const response = await pubnub.objects.removeUUIDMetadata(); - - // Using the passed in UUID - const response2 = await pubnub.objects.removeUUIDMetadata({ - uuid: 'myUuid', - }); - // snippet.end - assert.equal(response.status, 200); - assert.equal(response2.status, 200); - await eventually(async () => { - for (const uuid of [pubnub.getUserId(), world.n('myUuid')]) { - await assert.rejects(world.observer.objects.getUUIDMetadata({ uuid }), `${uuid} is gone`); - } - }, { label: 'users removed' }); - }); - - // ---- channels ------------------------------------------------------------------------------------------------- - - // The `*Team` filter needs a channel named `... Team`, which the fixture created with the name `Blue Team`. - step('getAllChannelMetadataBasicUsage', { needs: ['appContextFixture'], proves: 'getAllChannelMetadata with the Team filter returns the seeded channel.' }, async () => { - const pubnub = world.shopper; - // snippet.getAllChannelMetadataBasicUsage - // Get the total number of channels included in the response. - const response = await pubnub.objects.getAllChannelMetadata({ - include: { - totalCount: true, - }, - }); - - // Get all channels with the filter option. To get all channel which has Id ending 'Team'. - const response2 = await pubnub.objects.getAllChannelMetadata({ - filter: 'name LIKE "*Team"', - }); - console.log('Get all channel metadata response:', response2); - // snippet.end - assert.ok((response.totalCount ?? 0) >= 1, 'totalCount is part of the response'); - assert.ok(response2.data.length >= 1, 'the filter finds the team channels'); - assert.ok(response2.data.every((c) => c.name?.endsWith('Team')), 'every match ends with Team'); - assert.ok(ids(response2.data).includes(world.n('team.blue')), 'team.blue is in the filtered list'); - }); - - step('getChannelMetadataBasicUsage', { needs: ['appContextFixture'], proves: 'getChannelMetadata reads the seeded channel by id.' }, async () => { - const pubnub = world.shopper; - // snippet.getChannelMetadataBasicUsage - const response = await pubnub.objects.getChannelMetadata({ - // `channel` is the `id` in the _metadata_, not `name` - channel: 'team.blue', - }); - console.log('Get channel metadata response:', response); - // snippet.end - assert.equal(response.data.id, world.n('team.blue')); - assert.equal(response.data.name, 'Blue Team'); - }); - - step('setChannelMetadataBasicUsage', { needs: ['appContextFixture'], proves: 'setChannelMetadata stores the channel, an observer gets the set event and the hidden custom data was stored.' }, async () => { - const pubnub = world.shopper; - // Channel scoped objects events arrive on a subscription to the channel itself. - const listening = await watch(world.fresh('objectsobserver'), [world.n('team.red')]); - // snippet.setChannelMetadataBasicUsage - const response = await pubnub.objects.setChannelMetadata({ - channel: 'team.red', - data: { - name: 'Red Team', - description: 'The channel for Red team and no other teams.', - custom: { - owner: 'Red Leader', - }, - }, - include: { - customFields: false, - }, - }); - console.log('Set channel metadata response:', response); - // snippet.end - assert.equal(response.data.id, world.n('team.red')); - assert.equal(response.data.name, 'Red Team'); - assert.equal(response.data.custom, undefined, 'customFields: false leaves custom out of the response'); - await eventually(async () => { - const stored = await world.observer.objects.getChannelMetadata({ channel: world.n('team.red'), include: { customFields: true } }); - assert.deepEqual(stored.data.custom, { owner: 'Red Leader' }); - }, { label: 'stored channel custom data visible to another client' }); - const event = await listening.next('objects', (e) => e.channel === world.n('team.red')); - assert.equal(event.message.event, 'set'); - assert.equal(event.message.type, 'channel'); - listening.stop(); - }); - - step('iterativelyUpdateExistingMetadata', { needs: ['setChannelMetadataBasicUsage'], proves: 'The read-modify-write pattern merges custom data and the merged object survives on the service.' }, async () => { - const pubnub = world.shopper; - // snippet.iterativelyUpdateExistingMetadata - const channel = 'team.red'; - const name = 'Red Team'; - const description = 'The channel for Red team.'; - const customField = { visible: 'team' }; - - // Function to set and then update channel metadata - const response = await pubnub.objects.setChannelMetadata({ - channel: channel, - data: { - name: name, - description: description, - custom: customField, - }, - }); - console.log('The channel has been created with name and description.\n'); - - // Fetch current object with custom fields - const currentObjectResponse = await pubnub.objects.getChannelMetadata({ - channel: channel, - include: { - customFields: true, - }, - }); - const currentObject = currentObjectResponse.data; - - // Initialize the custom field object - const custom = currentObject.custom || {}; - - // Add or update the field - custom['edit'] = 'admin'; - - // Writing the updated object back to the server - const setResponse = await pubnub.objects.setChannelMetadata({ - channel: channel, - data: { - name: currentObject.name || '', - description: currentObject.description || '', - custom: custom, - }, - }); - console.log('Object has been updated', setResponse); - // snippet.end - assert.equal(response.status, 200); - await eventually(async () => { - const stored = await world.observer.objects.getChannelMetadata({ channel: world.n('team.red'), include: { customFields: true } }); - assert.deepEqual(stored.data.custom, { visible: 'team', edit: 'admin' }); - assert.equal(stored.data.description, 'The channel for Red team.'); - }, { label: 'updated channel metadata visible to another client' }); - }); - - step('removeChannelMetadataBasicUsage', { phase: 'P9', needs: ['iterativelyUpdateExistingMetadata'], proves: 'removeChannelMetadata deletes the channel and reads fail afterwards.' }, async () => { - const pubnub = world.shopper; - // snippet.removeChannelMetadataBasicUsage - const response = await pubnub.objects.removeChannelMetadata({ - channel: 'team.red', - }); - // snippet.end - assert.equal(response.status, 200); - await eventually(async () => { - await assert.rejects(world.observer.objects.getChannelMetadata({ channel: world.n('team.red') })); - }, { label: 'team.red removed' }); - }); - - // ---- memberships ---------------------------------------------------------------------------------------------- - - step('setMembershipBasicUsage', { needs: ['appContextFixture'], proves: 'setMemberships stores a membership, and a second read with include flags shows the type and custom data.' }, async () => { - const pubnub = world.shopper; - // snippet.setMembershipBasicUsage - // Using UUID from the config - const response = await pubnub.objects.setMemberships({ - channels: [ - 'my-channel', - { id: 'channel-with-status-type', custom: { hello: 'World' }, status: 'helloStatus', type: 'helloType' }, - ], - }); - console.log('Set memberships response:', response); - - // Using the passed in UUID - const response2 = await pubnub.objects.setMemberships({ - uuid: 'my-uuid', - channels: [ - 'my-channel', - { id: 'channel-with-status-type', custom: { hello: 'World' }, status: 'helloStatus', type: 'helloType' }, - ], - include: { - // To include channel fields in response - channelFields: true, - }, - }); - console.log('Set memberships response:', response2); - // snippet.end - const expected = [world.n('channel-with-status-type'), world.n('my-channel')].sort(); - assert.deepEqual(response.data.map((m) => m.channel.id).sort(), expected); - assert.deepEqual(response2.data.map((m) => m.channel.id).sort(), expected); - assert.equal(response.data.find((m) => m.channel.id === world.n('channel-with-status-type'))?.status, 'helloStatus'); - await eventually(async () => { - const stored = await world.observer.objects.getMemberships({ - uuid: world.n('my-uuid'), - include: { typeField: true, statusField: true, customFields: true }, - }); - assert.deepEqual(stored.data.map((m) => m.channel.id).sort(), expected); - const special = stored.data.find((m) => m.channel.id === world.n('channel-with-status-type')); - assert.equal(special?.status, 'helloStatus'); - assert.equal(special?.type, 'helloType'); - assert.deepEqual(special?.custom, { hello: 'World' }); - }, { label: 'stored memberships visible to another client' }); - }); - - step('getMembershipBasicUsage', { needs: ['setMembershipBasicUsage'], proves: 'getMemberships with the starred filter returns exactly the one starred membership.' }, async () => { - const pubnub = world.shopper; - // Seed the user the docs read: `myUuid` has one starred membership. - await world.friend.objects.setMemberships({ - uuid: world.n('myUuid'), - channels: [{ id: world.n('my-channel'), custom: { starred: true } }, { id: world.n('channel-with-status-type'), custom: { starred: false } }], - }); - // snippet.getMembershipBasicUsage - // Using UUID from the config - const response = await pubnub.objects.getMemberships(); - - // Using the passed in UUID - const response2 = await pubnub.objects.getMemberships({ - uuid: 'myUuid', - include: { - channelFields: true, - }, - }); - - // Get all memberships that are starred by the user - const response3 = await pubnub.objects.getMemberships({ - uuid: 'myUuid', - filter: 'custom.starred == true', - }); - // snippet.end - assert.deepEqual(response.data.map((m) => m.channel.id).sort(), [world.n('channel-with-status-type'), world.n('my-channel')].sort()); - assert.equal(response2.data.length, 2); - assert.ok(response2.data.every((m) => m.channel.id), 'channel ids are part of the response'); - assert.deepEqual(response3.data.map((m) => m.channel.id), [world.n('my-channel')]); - }); - - step('removeMembershipsBasicUsage', { phase: 'P9', needs: ['getMembershipBasicUsage'], proves: 'removeMemberships removes the named memberships and keeps the others.' }, async () => { - const pubnub = world.shopper; - for (const uuid of [pubnub.getUserId(), world.n('myUuid')]) { - await world.friend.objects.setMemberships({ uuid, channels: [world.n('ch-1'), world.n('ch-2'), world.n('my-channel')] }); - } - // snippet.removeMembershipsBasicUsage - // Using UUID from the config - const response = await pubnub.objects.removeMemberships({ - channels: ['ch-1', 'ch-2'], - }); - - // Using the passed in UUID - const response2 = await pubnub.objects.removeMemberships({ - uuid: 'myUuid', - channels: ['ch-1', 'ch-2'], - }); - // snippet.end - for (const result of [response, response2]) { - assert.ok(!result.data.some((m) => [world.n('ch-1'), world.n('ch-2')].includes(m.channel.id)), 'ch-1 and ch-2 are gone'); - assert.ok(result.data.some((m) => m.channel.id === world.n('my-channel')), 'my-channel stays'); - } - await eventually(async () => { - const left = await world.observer.objects.getMemberships({ uuid: world.n('myUuid') }); - assert.ok(!left.data.some((m) => m.channel.id === world.n('ch-1'))); - }, { label: 'memberships removed' }); - }); - - // ---- channel members ------------------------------------------------------------------------------------------ - - step('getChannelMembersBasicUsage', { needs: ['appContextFixture'], proves: 'getChannelMembers with the admin filter returns only the matching member.' }, async () => { - const pubnub = world.shopper; - await world.friend.objects.setUUIDMetadata({ uuid: world.n('uuid-1'), data: { name: 'Member One', custom: { role: 'Member' } } }); - await world.friend.objects.setUUIDMetadata({ uuid: world.n('uuid-2'), data: { name: 'Team admin' } }); - await world.friend.objects.setChannelMembers({ - channel: world.n('myChannel'), - uuids: [world.n('uuid-1'), world.n('uuid-2')], - }); - // snippet.getChannelMembersBasicUsage - const response = await pubnub.objects.getChannelMembers({ - channel: 'myChannel', - include: { - UUIDFields: true, - }, - }); - console.log('getChannelMembers response:', response); - - // Get all channel members with "admin" in the user name - const response2 = await pubnub.objects.getChannelMembers({ - channel: 'myChannel', - filter: 'uuid.name LIKE "*admin*"', - }); - console.log('getChannelMembers response:', response2); - // snippet.end - assert.deepEqual(response.data.map((m) => m.uuid.id).sort(), [world.n('uuid-1'), world.n('uuid-2')].sort()); - assert.equal((response.data.find((m) => m.uuid.id === world.n('uuid-1'))?.uuid as { name?: string } | undefined)?.name, 'Member One'); - assert.deepEqual(response2.data.map((m) => m.uuid.id), [world.n('uuid-2')], 'the name filter keeps the admin only'); - }); - - // Adds `uuid-3` with a custom role, then reads it back through the observer. - step('setChannelMembersBasicUsage', { needs: ['getChannelMembersBasicUsage'], proves: 'setChannelMembers adds a member with a custom role and another client reads it back.' }, async () => { - const pubnub = world.shopper; - // snippet.setChannelMembersBasicUsage - const response = await pubnub.objects.setChannelMembers({ - channel: 'myChannel', - uuids: ['uuid-1', 'uuid-2', { id: 'uuid-3', custom: { role: 'Super Admin' } }], - }); - console.log('setChannelMembers response:', response); - // snippet.end - assert.deepEqual(response.data.map((m) => m.uuid.id).sort(), [world.n('uuid-1'), world.n('uuid-2'), world.n('uuid-3')].sort()); - await eventually(async () => { - const stored = await world.observer.objects.getChannelMembers({ channel: world.n('myChannel'), include: { customFields: true } }); - assert.deepEqual(stored.data.find((m) => m.uuid.id === world.n('uuid-3'))?.custom, { role: 'Super Admin' }); - }, { label: 'stored channel members visible to another client' }); - }); - - step('removeChannelMembersBasicUsage', { phase: 'P9', needs: ['setChannelMembersBasicUsage'], proves: 'removeChannelMembers removes one member by id and leaves the others.' }, async () => { - const pubnub = world.shopper; - // snippet.removeChannelMembersBasicUsage - const response = await pubnub.objects.removeChannelMembers({ - channel: 'myChannel', - uuids: ['uuid-1', 'uuid-2'], - }); - // snippet.end - assert.deepEqual(response.data.map((m) => m.uuid.id), [world.n('uuid-3')]); - await eventually(async () => { - const left = await world.observer.objects.getChannelMembers({ channel: world.n('myChannel') }); - assert.deepEqual(left.data.map((m) => m.uuid.id), [world.n('uuid-3')]); - }, { label: 'members removed' }); - }); -}); diff --git a/docs-snippets/api-reference/presence.ts b/docs-snippets/api-reference/presence.ts deleted file mode 100644 index 9e80d6239..000000000 --- a/docs-snippets/api-reference/presence.ts +++ /dev/null @@ -1,208 +0,0 @@ -import assert from '../harness/assert.js'; -import type PubNub from 'pubnub'; -import { page, step, check } from '../harness/step.js'; -import { connected, eventually, PRESENCE_TIMEOUT } from '../harness/wait.js'; - -/** - * Presence examples. - */ -export default page('api-reference/presence', 'P4', (world) => { - let occupant: PubNub | undefined; - const user = (): PubNub => { - assert.ok(occupant, 'presenceFixture has not run'); - return occupant; - }; - const sorted = (values: string[]): string[] => [...values].sort(); - - check('presenceFixture', { needs: ['preflightPresence'], timeout: 120_000, proves: 'Fixture: an occupant is visible on all three channels, two channels sharing one group member.' }, async () => { - const [ch1, myChannel, member] = [world.n('ch1'), world.n('my_channel'), world.n('cg1_member')]; - const [cg1, otherGroup] = [world.n('cg1'), world.n('my_channel_group')]; - // Two groups share one member channel: `cg1` is subscribed by the occupant, `my_channel_group` is only read. - for (const channelGroup of [cg1, otherGroup]) { - await world.friend.channelGroups.addChannels({ channelGroup, channels: [member] }); - } - occupant = world.client('open', 'uuid', world.presenceOptions); - const set = occupant.subscriptionSet({ channels: [ch1, myChannel], channelGroups: [cg1] }); - const ready = connected(occupant, { channels: [ch1, myChannel], channelGroups: [cg1] }); - set.subscribe(); - await ready; - await occupant.setState({ state: { status: 'online' }, channels: [myChannel] }); - await eventually(async () => { - const here = await world.observer.hereNow({ channels: [ch1, myChannel, member] }); - for (const channel of [ch1, myChannel, member]) { - assert.ok(here.channels[channel]?.occupants.some((o) => o.uuid === occupant!.getUserId()), `${channel} lists the occupant`); - } - }, { timeout: PRESENCE_TIMEOUT, label: 'occupant visible in hereNow' }); - const reads: Array<[string, { channels?: string[]; channelGroups: string[] }]> = [ - [cg1, { channelGroups: [cg1] }], - [otherGroup, { channelGroups: [otherGroup] }], - [`${ch1} and ${cg1}`, { channels: [ch1], channelGroups: [cg1] }], - ]; - for (const [label, query] of reads) { - await eventually(async () => { - const here = await world.observer.hereNow(query); - assert.ok(here.channels[member]?.occupants.some((o) => o.uuid === occupant!.getUserId()), `${label} lists the occupant on ${member}`); - }, { timeout: PRESENCE_TIMEOUT, label: `occupant visible through ${label}` }); - } - await eventually(async () => { - const here = await world.observer.hereNow({ channels: [myChannel], includeState: true }); - assert.deepEqual(here.channels[myChannel]?.occupants.find((o) => o.uuid === occupant!.getUserId())?.state, { status: 'online' }, 'hereNow lists the state on the channel'); - const where = await world.observer.whereNow({ uuid: occupant!.getUserId() }); - assert.ok([ch1, myChannel].every((c) => where.channels.includes(c)), `whereNow lists both channels (got ${where.channels.length})`); - }, { timeout: PRESENCE_TIMEOUT, label: 'state and channel list visible' }); - }); - - step('setStateBasicUsage', { needs: ['presenceFixture'], proves: 'setState stores state that another client can read back.' }, async () => { - const pubnub = user(); - // snippet.setStateBasicUsage - const response = await pubnub.setState({ - state: { status: 'online' }, - channels: ['ch1'], - channelGroups: ['cg1'], - }); - console.log('State set successfully:', response); - // snippet.end - assert.deepEqual(response.state, { status: 'online' }); - await eventually(async () => { - const stored = await world.observer.getState({ uuid: pubnub.getUserId(), channels: [world.n('ch1')] }); - assert.deepEqual(stored.channels[world.n('ch1')], { status: 'online' }); - }, { label: 'state visible to another client' }); - // hereNowBasicUsage reads the same state through hereNow in one shot, so wait for that read path too. - await eventually(async () => { - const here = await world.observer.hereNow({ channels: [world.n('ch1')], includeState: true }); - assert.deepEqual(here.channels[world.n('ch1')]?.occupants.find((o) => o.uuid === pubnub.getUserId())?.state, { status: 'online' }, 'hereNow lists the state on ch1'); - }, { label: 'state visible through hereNow' }); - }); - - step('getStateBasicUsage', { needs: ['setStateBasicUsage'], proves: 'getState returns the state set earlier.' }, async () => { - const pubnub = user(); - // snippet.getStateBasicUsage - const response = await pubnub.getState({ - uuid: 'uuid', - channels: ['ch1'], - channelGroups: ['cg1'], - }); - console.log('State retrieved successfully:', response); - // snippet.end - const nested = (response.channels as unknown as { channels?: Record }).channels; - assert.deepEqual(nested?.[world.n('ch1')], { status: 'online' }); - }); - - step('hereNowBasicUsage', { needs: ['setStateBasicUsage'], timeout: 60_000, proves: 'hereNow lists the occupant with its state and the group member channel appears through the group.' }, async () => { - const pubnub = user(); - // snippet.hereNowBasicUsage - // Function to get presence information for a channel - const result = await pubnub.hereNow({ - channels: ['ch1'], - channelGroups: ['cg1'], - includeUUIDs: true, - includeState: true, - }); - console.log('Here Now Result:', result); - // snippet.end - const entry = result.channels[world.n('ch1')]?.occupants.find((o) => o.uuid === pubnub.getUserId()); - assert.ok(entry, 'ch1 lists the occupant'); - assert.deepEqual(entry.state, { status: 'online' }); - // The group member shows up a moment after the fixture joins it, so re-read until it does. - await eventually( - async () => { - const again = await pubnub.hereNow({ - channels: [world.n('ch1')], - channelGroups: [world.n('cg1')], - includeUUIDs: true, - includeState: true, - }); - assert.ok(again.channels[world.n('cg1_member')], 'the channel group member is part of the result'); - }, - { timeout: 30_000, label: 'hereNow lists the channel group member' }, - ); - }); - - step('hereNowWithState', { needs: ['presenceFixture'], proves: 'hereNow with includeState returns the state set in the fixture.' }, async () => { - const pubnub = user(); - // snippet.hereNowWithState - const response = await pubnub.hereNow({ - channels: ['my_channel'], - includeState: true, - }); - console.log('hereNow response:', response); - // snippet.end - const entry = response.channels[world.n('my_channel')]?.occupants.find((o) => o.uuid === pubnub.getUserId()); - assert.deepEqual(entry?.state, { status: 'online' }); - }); - - step('hereNowFetchOccupancyOnly', { needs: ['presenceFixture'], proves: 'hereNow with includeUUIDs false gives a count and no user list.' }, async () => { - const pubnub = user(); - // snippet.hereNowFetchOccupancyOnly - const response = await pubnub.hereNow({ - channels: ['my_channel'], - includeUUIDs: false, - }); - console.log('hereNow response:', response); - // snippet.end - const entry = response.channels[world.n('my_channel')]; - assert.ok(entry.occupancy >= 1, 'occupancy counts the occupant'); - assert.equal(entry.occupants?.length ?? 0, 0, 'no user ids when includeUUIDs is false'); - }); - - step('hereNowChannelGroup', { needs: ['presenceFixture'], proves: 'hereNow on a channel group reports presence for each member channel.' }, async () => { - const pubnub = user(); - // snippet.hereNowChannelGroup - const response = await pubnub.hereNow({ - channelGroups: ['my_channel_group'], - }); - console.log('hereNow response:', response); - // snippet.end - const entry = response.channels[world.n('cg1_member')]; - assert.ok(entry, 'the group member channel is in the result'); - assert.ok(entry.occupants.some((o) => o.uuid === pubnub.getUserId())); - }); - - step('basicUsageWithPromises', { needs: ['setStateBasicUsage'], proves: 'The promise form of hereNow resolves with the occupants, seen through a spy.' }, async () => { - const pubnub = user(); - const spy = world.spy(pubnub, 'hereNow'); - // snippet.basicUsageWithPromises - pubnub - .hereNow({ - channels: ['ch1'], - channelGroups: ['cg1'], - includeUUIDs: true, - includeState: true, - }) - .then((response) => { - console.log(response); - }) - .catch((error) => { - console.log(error); - }); - // snippet.end - assert.equal(spy.results.length, 1, 'the snippet called hereNow once'); - const response = (await spy.results[0]) as PubNub.Presence.HereNowResponse; - assert.ok(response.channels[world.n('ch1')]?.occupants.some((o) => o.uuid === pubnub.getUserId())); - }); - - step('whereNowBasicUsage', { needs: ['presenceFixture'], proves: 'whereNow lists the channels the occupant is in.' }, async () => { - const pubnub = world.observer; - // snippet.whereNowBasicUsage - const response = await pubnub.whereNow({ - uuid: 'uuid', - }); - console.log('whereNow response:', response); - // snippet.end - assert.deepEqual( - sorted(response.channels).filter((c) => [world.n('ch1'), world.n('my_channel')].includes(c)), - sorted([world.n('ch1'), world.n('my_channel')]), - ); - }); - - // Leaving is part of presence too: the occupant unsubscribes and the channels empty out. - check('presenceTeardown', { phase: 'P9', needs: ['presenceFixture'], timeout: 90_000, proves: 'After the occupant unsubscribes the channels empty out of hereNow.' }, async () => { - const pubnub = user(); - const ch1 = world.n('ch1'); - pubnub.unsubscribeAll(); - await eventually(async () => { - const here = await world.observer.hereNow({ channels: [ch1] }); - assert.ok(!here.channels[ch1]?.occupants.some((o) => o.uuid === pubnub.getUserId()), 'occupant is still listed'); - }, { timeout: PRESENCE_TIMEOUT, label: 'occupant left ch1' }); - }); -}); diff --git a/docs-snippets/api-reference/publish-legacy.ts b/docs-snippets/api-reference/publish-legacy.ts deleted file mode 100644 index a5393ea4c..000000000 --- a/docs-snippets/api-reference/publish-legacy.ts +++ /dev/null @@ -1,50 +0,0 @@ -import assert from '../harness/assert.js'; -import PubNub from 'pubnub'; -import { check, page } from '../harness/step.js'; -import { sleep, watch } from '../harness/wait.js'; -import { rejectsValidation } from '../harness/assertions.js'; - -/** - * Examples for the deprecated `fire()` method and its replacement, `publish()` with `storeInHistory: false` and - * `replicate: false`. - */ -export default page('api-reference/publish-legacy', 'P8', (world) => { - // Records a docs/SDK mismatch: the TypeScript type promises a PublishResponse but fire() resolves with undefined. - check('publishLegacyFireResolvesUndefined', { proves: 'fire() resolves with undefined although the TypeScript type promises a response.' }, async () => { - const channel = world.n('my_channel'); - const response = await world.shopper.fire({ channel, message: { legacy: 'fire' } }); - // The TypeScript signature promises a PublishResponse, the runtime value is undefined. - assert.equal(response, undefined); - }); - - // Same client twice: publish() rejects with a validation error, fire() swallows it. A client without publishKey is the cheapest reproducible error. - check('publishLegacyFireDiscardsErrors', { proves: 'publish rejects with a validation error on a bad client while fire swallows it.' }, async () => { - // A client without publishKey cannot publish. publish() reports it, fire() does not. - const noPublishKey = world.track( - new PubNub({ subscribeKey: world.env.open!.subscribeKey, userId: world.n('fire-no-key') }), - ); - const channel = world.n('my_channel'); - await rejectsValidation(noPublishKey.publish({ channel, message: { legacy: 'publish' } }), /Missing 'publishKey'/); - assert.equal(await noPublishKey.fire({ channel, message: { legacy: 'fire' } }), undefined, 'fire() resolves undefined and discards the error'); - }); - - check('publishLegacyReplacementDoesNotReplicateOrStore', { needs: ['preflightPersistence'], timeout: 45_000, proves: 'The fire replacement call is neither delivered to subscribers nor stored in history.' }, async () => { - const channel = world.n('fire_replacement'); - const listening = await watch(world.fresh('observer'), [channel]); - const response = await world.shopper.publish({ - channel, - message: { legacy: 'not-replicated' }, - storeInHistory: false, - replicate: false, - }); - world.check.timetoken(response.timetoken); - await sleep(4000); - assert.ok( - !listening.events.some((e) => (e.event as { message?: { legacy?: string } }).message?.legacy === 'not-replicated'), - 'with replicate: false the message does not reach subscribers', - ); - const history = await world.observer.fetchMessages({ channels: [channel], count: 25 }); - assert.equal(history.channels[channel]?.length ?? 0, 0, 'with storeInHistory: false nothing is stored'); - listening.stop(); - }); -}); diff --git a/docs-snippets/api-reference/publish.ts b/docs-snippets/api-reference/publish.ts deleted file mode 100644 index 7223452c7..000000000 --- a/docs-snippets/api-reference/publish.ts +++ /dev/null @@ -1,238 +0,0 @@ -import assert from '../harness/assert.js'; -import type PubNub from 'pubnub'; -import { page, step } from '../harness/step.js'; -import { recorder } from '../harness/recorder.js'; -import { eventually, sleep, watch } from '../harness/wait.js'; -import { rejectsValidation, rejectsWithCategory } from '../harness/assertions.js'; - -/** - * Publish and signal examples, including `fire()`. - */ -export default page('api-reference/publish', 'P4', (world) => { - step('publishBasicUsage', { needs: ['preflightPersistence'], proves: 'publish delivers a message with meta and a custom message type to a subscriber and stores it.' }, async () => { - const pubnub = world.shopper; - const channel = world.n('my_channel'); - const listening = await watch(world.fresh('observer'), [channel]); - - // snippet.publishBasicUsage - const response = await pubnub.publish({ - message: { text: 'Hello World' }, - channel: 'my_channel', - sendByPost: false, - storeInHistory: true, - meta: { sender: 'user123' }, - customMessageType: 'text-message', - }); - console.log('Publish Success:', response); - // snippet.end - - world.check.timetoken(response.timetoken); - const received = await listening.next('message', (m) => (m.message as { text?: string }).text === 'Hello World'); - assert.equal(received.timetoken, response.timetoken); - assert.equal(received.publisher, pubnub.getUserId()); - assert.equal(received.userMetadata?.sender, 'user123'); - assert.equal(received.customMessageType, 'text-message'); - // storeInHistory: true, so Message Persistence has it too. - await eventually(async () => { - const history = await world.observer.fetchMessages({ channels: [channel], count: 10 }); - assert.ok(history.channels[channel]?.some((m) => m.timetoken === response.timetoken), 'message is stored'); - }, { label: 'published message is persisted' }); - listening.stop(); - }); - - step('publishStoreThePublishedMessagefor10Hours', { needs: ['preflightPersistence'], proves: 'publish with a ttl still delivers and stores the message.' }, async () => { - const pubnub = world.shopper; - const channel = world.n('my_channel'); - const listening = await watch(world.fresh('observer'), [channel]); - - // snippet.publishStoreThePublishedMessagefor10Hours - const response = await pubnub.publish({ - message: 'hello!', - channel: 'my_channel', - storeInHistory: true, - ttl: 10, - customMessageType: 'text-message', - }); - - console.log('message published with server response:', response); - // snippet.end - - world.check.timetoken(response.timetoken); - const received = await listening.next('message', (m) => m.message === 'hello!'); - assert.equal(received.timetoken, response.timetoken); - await eventually(async () => { - const history = await world.observer.fetchMessages({ channels: [channel], count: 10 }); - assert.ok(history.channels[channel]?.some((m) => m.timetoken === response.timetoken), 'message is stored'); - }, { label: 'message with ttl is persisted' }); - listening.stop(); - }); - - step('publishMinimal', { proves: 'The minimal publish returns a string timetoken and delivers the message.' }, async () => { - const pubnub = world.shopper; - const channel = world.n('my_channel'); - const listening = await watch(world.fresh('observer'), [channel]); - - // snippet.publishMinimal - const { timetoken } = await pubnub.publish({ - channel: 'my_channel', - message: { text: 'Hello' }, - }); - - console.log(timetoken); // "17918851187695440" (a string) - // snippet.end - - assert.equal(typeof timetoken, 'string'); - world.check.timetoken(timetoken); - const received = await listening.next('message', (m) => (m.message as { text?: string }).text === 'Hello'); - assert.equal(received.timetoken, timetoken); - listening.stop(); - }); - - step('publishFalsyMessage', { proves: 'A wrapped value 0 is delivered as a message.' }, async () => { - const pubnub = world.shopper; - const channel = world.n('my_channel'); - const listening = await watch(world.fresh('observer'), [channel]); - - // snippet.publishFalsyMessage - // message: 0 is rejected with "Missing 'message'", so wrap the value - await pubnub.publish({ - channel: 'my_channel', - message: { value: 0 }, - }); - // snippet.end - - const received = await listening.next('message', (m) => (m.message as { value?: number }).value === 0); - assert.deepEqual(received.message, { value: 0 }); - await rejectsValidation(pubnub.publish({ channel, message: 0 }), /Missing 'message'/); - listening.stop(); - }); - - step('signalBasicUsage', { proves: 'signal reaches a subscriber as a signal event with the custom message type, not as a message.' }, async () => { - const pubnub = world.shopper; - const channel = world.n('foo'); - const listening = await watch(world.fresh('observer'), [channel]); - - // snippet.signalBasicUsage - const response = await pubnub.signal({ - message: 'hello', - channel: 'foo', - customMessageType: 'text-message', - }); - console.log('signal response:', response); - // snippet.end - - world.check.timetoken(response.timetoken); - const received = await listening.next('signal', (s) => s.message === 'hello'); - assert.equal(received.timetoken, response.timetoken); - assert.equal(received.publisher, pubnub.getUserId()); - assert.equal(received.customMessageType, 'text-message'); - listening.stop(); - }); - - step('fireBasicUsage', { phase: 'P8', proves: 'fire sends to Functions only, so subscribers receive nothing and history stores nothing.' }, async () => { - const pubnub = world.shopper; - const channel = world.n('my_channel'); - const listening = await watch(world.fresh('observer'), [channel]); - - // snippet.fireBasicUsage - // Initialize PubNub with your keys - await pubnub.fire({ - message: { - such: 'object', - }, - channel: 'my_channel', - sendByPost: false, // true to send via post - meta: { - cool: 'meta', - }, // fire extra meta with the request - }); - - console.log('message fired'); - // snippet.end - - // fire() sends to Functions only: subscribers do not receive it and Message Persistence does not store it. - await sleep(4000); - assert.ok( - !listening.events.some((e) => (e.event as { message?: { such?: string } }).message?.such === 'object'), - 'a fired message must not reach subscribers', - ); - const history = await world.observer.fetchMessages({ channels: [channel], count: 25 }); - assert.ok( - !history.channels[channel]?.some((m) => (m.message as { such?: string } | undefined)?.such === 'object'), - 'a fired message is not stored', - ); - listening.stop(); - }); - - step('publishHandleErrors', { keyset: 'pam', timeout: 240_000, proves: 'The same error handling code logs the right line for success, access denied (403), validation and network failure.' }, async () => { - type PubNubError = PubNub.PubNubError; - - const attempt = async (pubnub: PubNub): Promise => { - const errors = world.captureConsole('error'); - try { - // snippet.publishHandleErrors - try { - await pubnub.publish({ - channel: 'my_channel', - message: { text: 'Hello' }, - }); - } catch (e) { - const status = (e as PubNubError).status; - - if (status?.category === 'PNValidationErrorCategory') { - console.error('The call is invalid:', status.message); - } else if (status?.category === 'PNAccessDeniedCategory') { - console.error('403: the token or authKey has no write permission on the channel'); - } else if (status?.category === 'PNNetworkIssuesCategory') { - console.error('The network is unreachable. Retry the publish yourself.'); - } else { - console.error(status?.category, status?.statusCode, status?.errorData); - } - } - // snippet.end - } finally { - errors.restore(); - } - return errors.lines; - }; - - // Wait until the keyset enforces Access Manager on this channel for a stranger. Without this wait the 403 branch below - // is only as reliable as the propagation of earlier grants and revokes on the keyset. A keyset that never denies fails - // here with a message that names the cause, not with an empty list further down. - await eventually( - () => - rejectsWithCategory( - world.client('secured', 'no-token-probe').publish({ channel: world.n('my_channel'), message: { probe: true } }), - 'PNAccessDeniedCategory', - ), - { timeout: 120_000, interval: 3_000, label: 'a client without a token is refused on the channel (Access Manager grants and revokes take up to 60 s to propagate)' }, - ); - - // Success: the server client may publish, so no error line. - assert.deepEqual(await attempt(world.server), [], 'a valid publish logs nothing'); - - let deniedAttempts = 0; - await eventually(async () => { - deniedAttempts += 1; - const denied = await attempt(world.client('secured', `no-token-user-${deniedAttempts}`)); - assert.deepEqual(denied, ['403: the token or authKey has no write permission on the channel']); - }, { timeout: 90_000, interval: 3_000, label: 'a tokenless publish logs the 403 line (Access Manager enforcement reaches every node)' }); - recorder.record({ kind: 'event', label: `tokenless publish logged the 403 line on attempt ${deniedAttempts}`, ok: true }); - - // Validation: no publish key, the SDK refuses before it sends. - const invalid = await attempt(world.client('secured', 'no-publish-key-user', { publishKey: '' })); - assert.equal(invalid.length, 1); - assert.match(invalid[0], /^The call is invalid: /); - - // Network: the proxy refuses every tunnel. Publish has no retry, so the call fails at once. - await world.proxy.start(); - const proxied = world.client('secured', 'proxied-user'); - proxied.setProxy(world.proxy.config); - world.proxy.outage(true); - try { - assert.deepEqual(await attempt(proxied), ['The network is unreachable. Retry the publish yourself.']); - } finally { - world.proxy.outage(false); - } - }); -}); diff --git a/docs-snippets/api-reference/subscribe-legacy.ts b/docs-snippets/api-reference/subscribe-legacy.ts deleted file mode 100644 index 62017ad06..000000000 --- a/docs-snippets/api-reference/subscribe-legacy.ts +++ /dev/null @@ -1,552 +0,0 @@ -import assert from '../harness/assert.js'; -import { page, step, check } from '../harness/step.js'; -import { recorder } from '../harness/recorder.js'; -import { connected, eventually, expectEvent, expectStatus, KindLog, PRESENCE_EVENT_TIMEOUT, sleep } from '../harness/wait.js'; - -/** - * Examples for the older `pubnub.subscribe()`, `pubnub.unsubscribe()` and `pubnub.unsubscribeAll()` methods. - */ -export default page('api-reference/subscribe-legacy', 'P8', (world) => { - const tag = (m: { message: unknown }): string | undefined => (m.message as { legacy?: string }).legacy; - // Subscription state lists have no guaranteed order. - const sorted = (values: string[]): string[] => [...values].sort(); - - /** The groups the samples subscribe to, each with one member channel. The groups must exist and hold a channel to carry messages. */ - // The groups the samples subscribe to, each with one member channel. Groups must hold a channel to carry messages. - const GROUPS = ['my_channelGroup', 'family', 'my_channelGroup1', 'my_channelGroup2', 'my_channelGroup3', 'demo_group1', 'demo_group2']; - const member = (group: string): string => `${world.n(group)}_member`; - - // Creates every group above with its member channel. - check('legacyGroupsFixture', { phase: 'P3', proves: 'Fixture: every channel group of the legacy subscribe page exists with a member channel.' }, async () => { - for (const group of GROUPS) { - await world.friend.channelGroups.addChannels({ channelGroup: world.n(group), channels: [member(group)] }); - } - await eventually(async () => { - for (const group of GROUPS) { - const listed = await world.observer.channelGroups.listChannels({ channelGroup: world.n(group) }); - assert.deepEqual(listed.channels, [member(group)], `${group} lists its member`); - } - }, { label: 'legacy groups have their members' }); - }); - - step('OLDsubscribeBasicUsage', { proves: 'The legacy subscribe call connects and delivers a message.' }, async () => { - const pubnub = world.fresh('oldsub'); - const channel = world.n('my_channel'); - const ready = connected(pubnub, { channels: [channel] }); - const received = expectEvent(pubnub, 'message', (m) => tag(m) === 'basic' && m.channel === channel); - - // snippet.OLDsubscribeBasicUsage - pubnub.subscribe({ - channels: ['my_channel'], - }); - // snippet.end - - await ready; - assert.deepEqual(pubnub.getSubscribedChannels(), [channel]); - await world.friend.publish({ channel, message: { legacy: 'basic' } }); - await received; - pubnub.unsubscribeAll(); - }); - - step('OLDsubscribeMultipleChannels', { proves: 'One legacy subscribe call covers three channels, shown by a message on the middle one.' }, async () => { - const pubnub = world.fresh('oldmulti'); - const channels = ['my_channel_1', 'my_channel_2', 'my_channel_3'].map((c) => world.n(c)); - const ready = connected(pubnub, { channels }); - const received = expectEvent(pubnub, 'message', (m) => tag(m) === 'multi' && m.channel === channels[1]); - - // snippet.OLDsubscribeMultipleChannels - pubnub.subscribe({ - channels: ['my_channel_1', 'my_channel_2', 'my_channel_3'], - }); - // snippet.end - - await ready; - assert.deepEqual(sorted(pubnub.getSubscribedChannels()), sorted(channels)); - await world.friend.publish({ channel: channels[1], message: { legacy: 'multi' } }); - await received; - pubnub.unsubscribeAll(); - }); - - step('OLDsubscribeWithPresence', { needs: ['preflightPresence'], proves: 'withPresence true produces the own join event with the fields the page table lists.' }, async () => { - const pubnub = world.fresh('oldpresence'); - const channel = world.n('my_channel'); - const ready = connected(pubnub, { channels: [channel] }); - const joined = expectEvent(pubnub, 'presence', (p) => p.action === 'join' && p.channel === channel && p.uuid === pubnub.getUserId()); - - // snippet.OLDsubscribeWithPresence - pubnub.subscribe({ - channels: ['my_channel'], - withPresence: true, - }); - // snippet.end - - await ready; - const event = await joined; - // The docs presence payload table promises these fields. - assert.equal(typeof (event as { occupancy?: number }).occupancy, 'number'); - assert.ok(event.timetoken); - pubnub.unsubscribeAll(); - }); - - step('OLDsubscribeWithWildCardChannels', { proves: 'A wildcard subscription receives a message on a matching channel.' }, async () => { - const pubnub = world.fresh('oldwild'); - const root = world.n('ab.*'); - const actual = root.replace('.*', '.news'); - const ready = connected(pubnub, { channels: [root] }); - const received = expectEvent(pubnub, 'message', (m) => tag(m) === 'wild' && m.channel === actual); - - // snippet.OLDsubscribeWithWildCardChannels - pubnub.subscribe({ - channels: ['ab.*'], - }); - // snippet.end - - await ready; - await world.friend.publish({ channel: actual, message: { legacy: 'wild' } }); - const message = await received; - // The wildcard match is reported as the subscription. - assert.equal(message.subscription, root); - pubnub.unsubscribeAll(); - }); - - step('OLDsubscribeWithState', { needs: ['preflightPresence'], timeout: 60_000, proves: 'The connected status handler can set state, and the observer reads it back per channel.' }, async () => { - const pubnub = world.fresh('oldstate'); - const channels = ['ch1', 'ch2', 'ch3'].map((c) => world.n(c)); - const ready = connected(pubnub, { channels }); - - // snippet.OLDsubscribeWithState - pubnub.addListener({ - status: async (statusEvent) => { - if (statusEvent.category === 'PNConnectedCategory') { - await pubnub.setState({ - state: { - some: 'state', - }, - channels: ['ch1', 'ch2', 'ch3'], - }); - } - }, - message: (messageEvent) => { - // handle message - }, - presence: (presenceEvent) => { - // handle presence - }, - }); - - pubnub.subscribe({ - channels: ['ch1', 'ch2', 'ch3'], - }); - // snippet.end - - await ready; - // The observer reads the state back. The docs sample called setState without channels, which the SDK rejects (an intended difference from the old docs body). - await eventually(async () => { - // One channel per call: with several channels getState nests the result one level deeper (docs finding). - for (const channel of channels) { - const state = await world.observer.getState({ uuid: pubnub.getUserId(), channels: [channel] }); - assert.deepEqual(state.channels[channel], { some: 'state' }, `state on ${channel}`); - } - }, { timeout: 30_000, label: 'state set from the connected status handler' }); - pubnub.unsubscribeAll(); - }); - - step('OLDsubscribeChannelGroup', { needs: ['legacyGroupsFixture'], proves: 'Subscribing to a channel group delivers a message from its member channel.' }, async () => { - const pubnub = world.fresh('oldgroup'); - const group = world.n('my_channelGroup'); - const ready = connected(pubnub, { channelGroups: [group] }); - const received = expectEvent(pubnub, 'message', (m) => tag(m) === 'group' && m.subscription === group); - - // snippet.OLDsubscribeChannelGroup - pubnub.subscribe({ - channelGroups: ['my_channelGroup'], - }); - // snippet.end - - await ready; - assert.deepEqual(pubnub.getSubscribedChannelGroups(), [group]); - await world.friend.publish({ channel: member('my_channelGroup'), message: { legacy: 'group' } }); - const message = await received; - assert.equal(message.channel, member('my_channelGroup')); - pubnub.unsubscribeAll(); - }); - - step('OLDsubscribeChannelGroupWithPresence', { needs: ['legacyGroupsFixture', 'preflightPresence'], timeout: 60_000, proves: 'A group subscription with presence yields a join event that belongs to the group.' }, async () => { - const pubnub = world.fresh('oldgrouppresence'); - const group = world.n('family'); - const ready = connected(pubnub, { channelGroups: [group] }); - const joined = expectEvent(pubnub, 'presence', (p) => p.action === 'join' && p.uuid === pubnub.getUserId()); - - // snippet.OLDsubscribeChannelGroupWithPresence - pubnub.subscribe({ - channelGroups: ['family'], - withPresence: true, - }); - // snippet.end - - await ready; - const event = await joined; - assert.ok(event.channel === member('family') || event.subscription === group, `presence event belongs to the group (channel ${event.channel}, subscription ${event.subscription})`); - pubnub.unsubscribeAll(); - }); - - step('OLDsubscribeMultipleChannelGroup', { needs: ['legacyGroupsFixture'], proves: 'One call subscribes to three groups, shown by a message through the last group.' }, async () => { - const pubnub = world.fresh('oldgroups'); - const groups = ['my_channelGroup1', 'my_channelGroup2', 'my_channelGroup3'].map((g) => world.n(g)); - const ready = connected(pubnub, { channelGroups: groups }); - const received = expectEvent(pubnub, 'message', (m) => tag(m) === 'groups' && m.subscription === groups[2]); - - // snippet.OLDsubscribeMultipleChannelGroup - pubnub.subscribe({ - channelGroups: ['my_channelGroup1', 'my_channelGroup2', 'my_channelGroup3'], - }); - // snippet.end - - await ready; - assert.deepEqual(sorted(pubnub.getSubscribedChannelGroups()), sorted(groups)); - await world.friend.publish({ channel: member('my_channelGroup3'), message: { legacy: 'groups' } }); - await received; - pubnub.unsubscribeAll(); - }); - - step('OLDsubscribeChannelGroupAndChannels', { needs: ['legacyGroupsFixture'], proves: 'Channels and groups can be mixed in one call, shown by a message through each.' }, async () => { - const pubnub = world.fresh('oldmixed'); - const [channel, group] = [world.n('my_channel'), world.n('my_channelGroup')]; - const ready = connected(pubnub, { channels: [channel], channelGroups: [group] }); - const onChannel = expectEvent(pubnub, 'message', (m) => tag(m) === 'mixed-channel' && m.channel === channel); - const onGroup = expectEvent(pubnub, 'message', (m) => tag(m) === 'mixed-group' && m.subscription === group); - - // snippet.OLDsubscribeChannelGroupAndChannels - pubnub.subscribe({ - channels: ['my_channel'], - channelGroups: ['my_channelGroup'], - }); - // snippet.end - - await ready; - await world.friend.publish({ channel, message: { legacy: 'mixed-channel' } }); - await world.friend.publish({ channel: member('my_channelGroup'), message: { legacy: 'mixed-group' } }); - await onChannel; - await onGroup; - pubnub.unsubscribeAll(); - }); - - step('OLDaddListeners', { needs: ['preflightPresence'], timeout: 90_000, proves: 'The legacy listener sample handlers see status, message and presence events whose fields exist on live events.' }, async () => { - const pubnub = world.fresh('oldlisteners'); - const joiner = world.fresh('oldlistenersjoiner'); - const channel = world.n('legacy_listeners_channel'); - const log = new KindLog(['message', 'presence', 'signal', 'objects', 'messageAction', 'status'], { - message: (m) => (m.message as { legacy?: string }).legacy === 'listeners', - presence: (p) => p.action === 'join' && p.uuid === joiner.getUserId(), - signal: (s) => s.message === 'ping', - objects: (o) => o.message?.event === 'set' && o.message?.data?.id === channel, - messageAction: (a) => a.event === 'added', - }); - const seen = Object.fromEntries(log.kinds.map((kind) => [kind, log.of(kind)])) as Record; - pubnub.addListener({ - message: log.on('message'), - presence: log.on('presence'), - signal: log.on('signal'), - objects: log.on('objects'), - messageAction: log.on('messageAction'), - status: log.on('status'), - }); - - // snippet.OLDaddListeners - pubnub.addListener({ - // Messages - message: function (m) { - const channelName = m.channel; // Channel on which the message was published - const channelGroup = m.subscription; // Channel group or wildcard subscription match (if exists) - const pubTT = m.timetoken; // Publish timetoken - const msg = m.message; // Message payload - const publisher = m.publisher; // Message publisher - }, - // Presence - // requires a subscription with presence - presence: function (p) { - const action = p.action; // Can be join, leave, timeout, state-change, or interval - const channelName = p.channel; // Channel to which the message belongs - const channelGroup = p.subscription; // Channel group or wildcard subscription match, if any - const publishTime = p.timestamp; // Publish timetoken - const timetoken = p.timetoken; // Current timetoken - if (p.action !== 'state-change') { - const occupancy = p.occupancy; // Number of users subscribed to the channel - } - if (p.action === 'state-change') { - const state = p.state; // User state - } - if (p.action !== 'interval') { - const uuid = p.uuid; // User ID of the user who joined, left, timed out, or changed state - } - }, - // Signals - signal: function (s) { - const channelName = s.channel; // Channel to which the signal belongs - const channelGroup = s.subscription; // Channel group or wildcard subscription match, if any - const pubTT = s.timetoken; // Publish timetoken - const msg = s.message; // Payload - const publisher = s.publisher; // Message publisher - }, - // App Context - objects: (objectEvent) => { - const channel = objectEvent.channel; // Channel to which the event belongs - const channelGroup = objectEvent.subscription; // Channel group - const timetoken = objectEvent.timetoken; // Event timetoken - const event = objectEvent.message.event; // Name of the event that occurred - const type = objectEvent.message.type; // Type of the object that changed - const data = objectEvent.message.data; // Data from the event that occurred - }, - // Message Actions - messageAction: function (ma) { - const channelName = ma.channel; // Channel to which the message belongs - const publisher = ma.publisher; // Message publisher - const event = ma.event; // Message action added or removed - const type = ma.data.type; // Message action type - const value = ma.data.value; // Message action value - const messageTimetoken = ma.data.messageTimetoken; // Timetoken of the original message - const actionTimetoken = ma.data.actionTimetoken; // Timetoken of the message action - }, - // File Sharing - file: function (event) { - const channelName = event.channel; // Channel to which the file belongs - const channelGroup = event.subscription; // Channel group or wildcard subscription match (if exists) - const publisher = event.publisher; // File publisher - const timetoken = event.timetoken; // Event timetoken - - const message = event.message; // Optional message attached to the file - const fileId = event.file?.id; // File unique id - const fileName = event.file?.name; // File name - const fileUrl = event.file?.url; // File direct URL - }, - status: function (s) { - const affectedChannelGroups = s.affectedChannelGroups; // Array of channel groups affected in the operation - const affectedChannels = s.affectedChannels; // Array of channels affected in the operation - const category = s.category; // Returns PNConnectedCategory - const operation = s.operation; // Returns PNSubscribeOperation - const lastTimetoken = s.lastTimetoken; // Last timetoken used in the subscribe request (type long), only with enableEventEngine: false - const currentTimetoken = s.currentTimetoken; /* Current timetoken fetched in subscribe response, - * to be used in the next request (type long) */ - const subscribedChannels = s.subscribedChannels; // Array of all currently subscribed channels, only with enableEventEngine: false - }, - }); - // snippet.end - - const ready = connected(pubnub, { channels: [channel] }); - pubnub.subscribe({ channels: [channel], withPresence: true }); - await ready; - // The second client joins only after the first one is connected, so its join is published after the first client's - // subscribe loop is running and must reach it. - const joinerReady = connected(joiner, { channels: [channel] }); - joiner.subscribe({ channels: [channel] }); - await joinerReady; - // The service can lose or delay one join. Wait briefly for the joiner's join. If it does not arrive, subscribe the joiner - // again, at most twice. Every re-subscribe is recorded in the report, so a lost join stays visible and is never hidden. - const joinerJoined = () => seen.presence.some((p) => p.action === 'join' && p.uuid === joiner.getUserId()); - for (let attempt = 1; attempt <= 2 && !joinerJoined(); attempt += 1) { - try { - await eventually(() => assert.ok(joinerJoined(), 'the joiner join arrived'), { timeout: 8_000, label: 'joiner join arrived' }); - } catch { - recorder.record({ kind: 'event', label: `joiner join not delivered, subscribed again (attempt ${attempt})`, ok: true }); - joiner.unsubscribeAll(); - const again = connected(joiner, { channels: [channel] }); - joiner.subscribe({ channels: [channel] }); - await again; - } - } - const published = await world.friend.publish({ channel, message: { legacy: 'listeners' } }); - await world.friend.signal({ channel, message: 'ping' }); - await world.friend.addMessageAction({ channel, messageTimetoken: published.timetoken, action: { type: 'reaction', value: 'smiley_face' } }); - await world.friend.objects.setChannelMetadata({ channel, data: { name: 'legacy listeners' } }); - // On timeout the error names the kinds that arrived (with counts and times) and the ones still missing, and adds what - // the service reports for the channel, so a missing presence event can be told from a client that is not subscribed. - try { - await eventually(() => log.assertArrived(['message', 'presence', 'signal', 'objects', 'messageAction']), { timeout: PRESENCE_EVENT_TIMEOUT, label: 'every event kind arrived' }); - } catch (error) { - const here = await world.observer.hereNow({ channels: [channel] }).then((r) => `hereNow occupancy ${r.channels[channel]?.occupancy ?? 0}`, (e) => `hereNow failed: ${(e as Error).message}`); - throw new Error(`${(error as Error).message}. ${here}. All presence events seen: ${seen.presence.map((p) => `${p.action} ${p.uuid}`).join(', ') || 'none'}`); - } - // Keep the arrival times in the report: they show how late the slowest kind was. - recorder.record({ kind: 'event', label: `event kinds: ${log.summary()}`, ok: true }); - // The joiner leaves and the leave event must follow. Same wait, same reasoning. - joiner.unsubscribeAll(); - await eventually(() => assert.ok(seen.presence.some((p) => p.action === 'leave' && p.uuid === joiner.getUserId()), `the joiner's leave event arrived. Presence events seen: ${seen.presence.map((p) => `${p.action} ${p.uuid}`).join(', ')}`), { timeout: PRESENCE_EVENT_TIMEOUT, label: 'leave event arrived' }); - // The own join is not asserted: whether it is delivered depends on where the join falls against the first subscribe timetoken. - // Recording it keeps the question open in the report instead of hiding it. - recorder.record({ kind: 'event', label: `own join event ${seen.presence.some((p) => p.action === 'join' && p.uuid === pubnub.getUserId()) ? 'was' : 'was NOT'} delivered to the subscribing client (not asserted)`, ok: true }); - - const [message] = seen.message; - assert.equal(message.channel, channel); - assert.equal(message.publisher, world.friend.getUserId()); - assert.ok(message.timetoken); - const [presence] = log.matching('presence'); - assert.equal(presence.channel, channel); - assert.equal(typeof presence.occupancy, 'number'); - assert.ok(presence.timestamp, 'presence timestamp'); - assert.ok(presence.timetoken, 'presence timetoken'); - assert.equal(presence.uuid, joiner.getUserId()); - const leave = seen.presence.find((p) => p.action === 'leave' && p.uuid === joiner.getUserId())!; - assert.equal(typeof leave.occupancy, 'number'); - const [signal] = seen.signal; - assert.equal(signal.publisher, world.friend.getUserId()); - const [objectEvent] = seen.objects; - assert.equal(objectEvent.message.event, 'set'); - assert.equal(objectEvent.message.type, 'channel'); - assert.equal(objectEvent.message.data.id, channel); - // The old sample read event, type, data and publisher from the event itself. They are under message, and objects events have no publisher. - assert.equal(objectEvent.event, undefined); - assert.equal(objectEvent.publisher, undefined); - const [action] = seen.messageAction; - assert.equal(action.event, 'added'); - assert.equal(action.data.type, 'reaction'); - assert.equal(action.data.value, 'smiley_face'); - assert.equal(action.data.messageTimetoken, published.timetoken); - assert.ok(action.data.actionTimetoken); - assert.ok(action.publisher, 'message action publisher (the docs read ma.publisher)'); - const connectedStatus = seen.status.find((s) => s.category === 'PNConnectedCategory'); - assert.ok(connectedStatus, 'connected status'); - assert.equal(connectedStatus.operation, 'PNSubscribeOperation'); - // The new-subscription-loop status carries neither lastTimetoken nor subscribedChannels, see legacyStatusFieldsByEventEngine. - assert.ok(connectedStatus.affectedChannels.includes(channel)); - assert.ok(connectedStatus.currentTimetoken); - pubnub.unsubscribeAll(); - await world.friend.objects.removeChannelMetadata({ channel }); - }); - - check('legacyStatusFieldsByEventEngine', { proves: 'lastTimetoken and subscribedChannels are set on the status only by the legacy subscription manager.' }, async () => { - const channel = world.n('my_channel'); - for (const enableEventEngine of [true, false]) { - const pubnub = world.fresh(`oldstatus-${enableEventEngine}`, { enableEventEngine }); - const statuses: Array> = []; - pubnub.addListener({ status: (s) => statuses.push(s as unknown as Record) }); - pubnub.subscribe({ channels: [channel] }); - await eventually(() => assert.ok(statuses.some((s) => s.category === 'PNConnectedCategory')), { label: 'connected' }); - const status = statuses.find((s) => s.category === 'PNConnectedCategory')!; - assert.equal(status.operation, 'PNSubscribeOperation'); - assert.deepEqual(status.affectedChannels, [channel]); - assert.ok(status.currentTimetoken, 'currentTimetoken'); - assert.equal('lastTimetoken' in status, !enableEventEngine, `lastTimetoken is present only with enableEventEngine: false (enableEventEngine ${enableEventEngine})`); - assert.equal('subscribedChannels' in status, !enableEventEngine, `subscribedChannels is present only with enableEventEngine: false (enableEventEngine ${enableEventEngine})`); - pubnub.unsubscribeAll(); - } - }); - - step('OLDremoveListeners', { timeout: 45_000, proves: 'removeListener leaves the other listeners of the client working, and the removed object receives nothing.' }, async () => { - const pubnub = world.fresh('oldremove'); - const channel = world.n('my_channel'); - const kept: unknown[] = []; - pubnub.addListener({ message: (m) => kept.push(m) }); - - // snippet.OLDremoveListeners - var existingListener = { - message: function () { - }, - }; - - pubnub.removeListener(existingListener); - // snippet.end - const removedCalls = world.spy(existingListener, 'message'); - const ready = connected(pubnub, { channels: [channel] }); - pubnub.subscribe({ channels: [channel] }); - await ready; - try { - await world.observer.publish({ channel, message: { text: 'after removeListener' } }); - await eventually(() => assert.equal(kept.length, 1, 'the other listener still receives'), { label: 'kept listener receives' }); - assert.equal(removedCalls.calls.length, 0, 'the removed listener object received nothing'); - } finally { - pubnub.unsubscribeAll(); - } - }); - - // Effect of removeListener: a removed listener gets no more events, a kept one does. - check('legacyRemoveListenerStopsDelivery', { proves: 'A removed listener gets no more events and a kept one does.' }, async () => { - const pubnub = world.fresh('oldremoveeffect'); - const channel = world.n('my_channel'); - const removed: unknown[] = []; - const kept: unknown[] = []; - const removable = { message: (m: unknown) => removed.push(m) }; - pubnub.addListener(removable); - pubnub.addListener({ message: (m) => kept.push(m) }); - const ready = connected(pubnub, { channels: [channel] }); - pubnub.subscribe({ channels: [channel] }); - await ready; - pubnub.removeListener(removable); - await world.friend.publish({ channel, message: { legacy: 'after-remove' } }); - await eventually(() => assert.equal(kept.length, 1), { label: 'kept listener got the message' }); - await sleep(500); - assert.equal(removed.length, 0, 'the removed listener got nothing'); - pubnub.unsubscribeAll(); - }); - - step('OLDUnsubscribeBasicUsage', { proves: 'unsubscribe with channels drops that channel.' }, async () => { - const pubnub = world.fresh('oldunsub'); - const channel = world.n('my_channel'); - const ready = connected(pubnub, { channels: [channel] }); - pubnub.subscribe({ channels: ['my_channel'].map((c) => world.n(c)) }); - await ready; - assert.deepEqual(pubnub.getSubscribedChannels(), [channel]); - - // snippet.OLDUnsubscribeBasicUsage - pubnub.unsubscribe({ - channels: ['my_channel'], - }); - // snippet.end - - await eventually(() => assert.deepEqual(pubnub.getSubscribedChannels(), []), { label: 'channel dropped' }); - }); - - step('OLDUnsubscribeMultipleChannels', { proves: 'Unsubscribing three of four channels leaves exactly the fourth.' }, async () => { - const pubnub = world.fresh('oldunsubmulti'); - const channels = ['chan1', 'chan2', 'chan3', 'ch1'].map((c) => world.n(c)); - const ready = connected(pubnub, { channels }); - pubnub.subscribe({ channels }); - await ready; - - // snippet.OLDUnsubscribeMultipleChannels - pubnub.unsubscribe({ - channels: ['chan1', 'chan2', 'chan3'], - }); - // snippet.end - - // Only the channel that the sample did not name stays subscribed. - await eventually(() => assert.deepEqual(pubnub.getSubscribedChannels(), [world.n('ch1')]), { label: 'three channels dropped' }); - pubnub.unsubscribeAll(); - }); - - step('OLDUnsubscribeMultipleChannelGroup', { needs: ['legacyGroupsFixture'], proves: 'Unsubscribing two of three groups leaves exactly the third.' }, async () => { - const pubnub = world.fresh('oldunsubgroups'); - const groups = ['demo_group1', 'demo_group2', 'family'].map((g) => world.n(g)); - const ready = connected(pubnub, { channelGroups: groups }); - pubnub.subscribe({ channelGroups: groups }); - await ready; - - // snippet.OLDUnsubscribeMultipleChannelGroup - pubnub.unsubscribe({ - channelGroups: ['demo_group1', 'demo_group2'], - }); - // snippet.end - - await eventually(() => assert.deepEqual(pubnub.getSubscribedChannelGroups(), [world.n('family')]), { label: 'two groups dropped' }); - pubnub.unsubscribeAll(); - }); - - step('OLDUnsubscribeAllBasicUsage', { needs: ['legacyGroupsFixture'], proves: 'unsubscribeAll clears all channels and groups.' }, async () => { - const pubnub = world.fresh('oldunsuball'); - const channels = ['ch1', 'ch2'].map((c) => world.n(c)); - const group = world.n('family'); - const ready = connected(pubnub, { channels, channelGroups: [group] }); - pubnub.subscribe({ channels, channelGroups: [group] }); - await ready; - const disconnected = expectStatus(pubnub, ['PNDisconnectedCategory']); - - // snippet.OLDUnsubscribeAllBasicUsage - pubnub.unsubscribeAll(); - // snippet.end - - await eventually(() => { - assert.deepEqual(pubnub.getSubscribedChannels(), []); - assert.deepEqual(pubnub.getSubscribedChannelGroups(), []); - }, { label: 'everything dropped' }); - void disconnected.catch(() => undefined); - }); -}); diff --git a/docs-snippets/api-reference/subscribe.ts b/docs-snippets/api-reference/subscribe.ts deleted file mode 100644 index 0eb9ae78c..000000000 --- a/docs-snippets/api-reference/subscribe.ts +++ /dev/null @@ -1,423 +0,0 @@ -import assert from '../harness/assert.js'; -import { page, step, check } from '../harness/step.js'; -import { assertAll, connected, eventually, expectEvent, PRESENCE_EVENT_TIMEOUT } from '../harness/wait.js'; - -/** - * Subscribe examples, without the DataSync examples, which are in the DataSync examples file. - */ -export default page('api-reference/subscribe', 'P4', (world) => { - // Subscription state lists have no guaranteed order. - const sorted = (values: string[]): string[] => [...values].sort(); - const hasLine = (lines: string[], prefix: string): boolean => lines.some((line) => line.startsWith(prefix)); - - // Gives `channelGroup_1` one member channel, because three samples subscribe to it. A group without channels carries no messages. - // Channel group `channelGroup_1` is subscribed by three samples. Give it one member channel. - check('subscribeGroupFixture', { phase: 'P3', proves: 'Fixture: channelGroup_1 has a member channel so that group subscriptions carry messages.' }, async () => { - const group = world.n('channelGroup_1'); - const member = world.n('channelGroup_1_member'); - await world.friend.channelGroups.addChannels({ channelGroup: group, channels: [member] }); - await eventually(async () => { - const listed = await world.observer.channelGroups.listChannels({ channelGroup: group }); - assert.deepEqual(listed.channels, [member]); - }, { label: 'channelGroup_1 has its member' }); - }); - - step('createChannelBasicUsage', { proves: 'The channel entity factory returns the run channel name and does not subscribe.' }, () => { - const pubnub = world.shopper; - // snippet.createChannelBasicUsage - const channel = pubnub.channel('my_channel'); - // snippet.end - assert.equal(channel.name, world.n('my_channel')); - assert.deepEqual(pubnub.getSubscribedChannels(), [], 'creating an entity does not subscribe'); - }); - - step('createChannelGroupBasicUsage', { proves: 'The channel group entity carries the run group name.' }, () => { - const pubnub = world.shopper; - // snippet.createChannelGroupBasicUsage - // Initialize PubNub with demo keys - const channelGroup = pubnub.channelGroup('channelGroup_1'); - // snippet.end - assert.equal(channelGroup.name, world.n('channelGroup_1')); - }); - - step('createChannelMetadataBasicUsage', { proves: 'The channel metadata entity carries the run id and makes no network call.' }, () => { - const pubnub = world.shopper; - // snippet.createChannelMetadataBasicUsage - // Initialize PubNub with demo keys - - const channelMetadata = pubnub.channelMetadata('channel_1'); - // snippet.end - assert.equal(channelMetadata.id, world.n('channel_1')); - }); - - step('createUserMetadataBasicUsage', { proves: 'The user metadata entity carries the run id and makes no network call.' }, () => { - const pubnub = world.shopper; - // snippet.createUserMetadataBasicUsage - // Initialize PubNub with demo keys - - const userMetadata = pubnub.userMetadata('user_meta1'); - // snippet.end - assert.equal(userMetadata.id, world.n('user_meta1')); - }); - - step('createSubscription', { proves: 'Creating a subscription object does not subscribe anything.' }, () => { - const pubnub = world.fresh('createsub'); - // snippet.createSubscription - const channel = pubnub.channel('my_channel'); - - const subscriptionOptions = { receivePresenceEvents: true }; - channel.subscription(subscriptionOptions); - // snippet.end - // Creating a subscription object does not subscribe. - assert.deepEqual(pubnub.getSubscribedChannels(), []); - }); - - step('subscriptionSetSubscribeBasicUsage', { proves: 'Subscribing a set connects both channels and delivers a message from a friend.' }, async () => { - const pubnub = world.fresh('setsubscribe'); - const [ch1, ch2] = [world.n('ch1'), world.n('ch2')]; - const ready = connected(pubnub, { channels: [ch1, ch2] }); - const received = expectEvent(pubnub, 'message', (m) => m.channel === ch1 && (m.message as { via?: string }).via === 'set'); - - // snippet.subscriptionSetSubscribeBasicUsage - const channelsSubscriptionSet = pubnub.subscriptionSet({ channels: ['ch1', 'ch2'] }); - channelsSubscriptionSet.subscribe(); - // snippet.end - - await ready; - assert.deepEqual(sorted(pubnub.getSubscribedChannels()), sorted([ch1, ch2])); - await world.friend.publish({ channel: ch1, message: { via: 'set' } }); - await received; - channelsSubscriptionSet.unsubscribe(); - }); - - step('eventListenerAddListeners', { proves: 'The message, signal and message action handlers of the sample log when the events arrive.' }, async () => { - const pubnub = world.fresh('addlisteners'); - const channelName = world.n('channel_1'); - await world.withConsole(async (logs) => { - // snippet.eventListenerAddListeners - - // create a subscription from a channel entity - const channel = pubnub.channel('channel_1'); - const subscription = channel.subscription(); - - // Event-specific listeners - subscription.onMessage = (message) => { - console.log('Message event: ', message); - }; - subscription.onPresence = (presence) => { - console.log('Presence event: ', presence); - }; - subscription.onSignal = (signal) => { - console.log('Signal event: ', signal); - }; - subscription.onObjects = (objectsEvent) => { - console.log('Objects event: ', objectsEvent); - }; - subscription.onMessageAction = (messageActionEvent) => { - console.log('Message Reaction event: ', messageActionEvent); - }; - subscription.onFile = (fileEvent) => { - console.log('File event: ', fileEvent); - }; - - // Generic listeners - subscription.addListener({ - // Messages - message: function (m) { - const channelName = m.channel; // Channel on which the message was published - const channelGroup = m.subscription; // Channel group or wildcard subscription match (if exists) - const pubTT = m.timetoken; // Publish timetoken - const msg = m.message; // Message payload - const publisher = m.publisher; // Message publisher - }, - // Presence - // requires a subscription with presence - presence: function (p) { - const action = p.action; // Can be join, leave, timeout, state-change, or interval - const channelName = p.channel; // Channel to which the message belongs - const channelGroup = p.subscription; // Channel group or wildcard subscription match, if any - const publishTime = p.timestamp; // Publish timetoken - const timetoken = p.timetoken; // Current timetoken - }, - // Signals - signal: function (s) { - const channelName = s.channel; // Channel to which the signal belongs - const channelGroup = s.subscription; // Channel group or wildcard subscription match, if any - const pubTT = s.timetoken; // Publish timetoken - const msg = s.message; // Payload - const publisher = s.publisher; // Message publisher - }, - // App Context - objects: (objectEvent) => { - const channel = objectEvent.channel; // channel Id or a User Id of updated/set app context metadata object - const channelGroup = objectEvent.subscription; // Channel group - const timetoken = objectEvent.timetoken; // Event timetoken - const event = objectEvent.message.data.type; // Event name - }, - // Message Actions - messageAction: function (ma) { - const channelName = ma.channel; // Channel to which the message belongs - const publisher = ma.data.uuid; // Message publisher - const event = ma.event; // Message action added or removed - const type = ma.data.type; // Message action type - const value = ma.data.value; // Message action value - const messageTimetoken = ma.data.messageTimetoken; // Timetoken of the original message - const actionTimetoken = ma.data.actionTimetoken; // Timetoken of the message action - }, - // File Sharing - file: function (event) { - const channelName = event.channel; // Channel to which the file belongs - const channelGroup = event.subscription; // Channel group or wildcard subscription match (if exists) - const publisher = event.publisher; // File publisher - const timetoken = event.timetoken; // Event timetoken - const message = event.message; // Optional message attached to the file - }, - }); - - // snippet.end - - const ready = connected(pubnub, { channels: [channelName] }); - subscription.subscribe(); - await ready; - const published = await world.friend.publish({ channel: channelName, message: { n: 1 } }); - await world.friend.signal({ channel: channelName, message: 'ping' }); - await world.friend.addMessageAction({ - channel: channelName, - messageTimetoken: published.timetoken, - action: { type: 'reaction', value: 'smiley_face' }, - }); - await eventually(() => assertAll({ - 'onMessage ran': hasLine(logs.lines, 'Message event: '), - 'onSignal ran': hasLine(logs.lines, 'Signal event: '), - 'onMessageAction ran': hasLine(logs.lines, 'Message Reaction event: '), - }), { label: 'event-specific listeners ran' }); - subscription.unsubscribe(); - }); - }); - - step('eventListenerBasicUsage', { needs: ['preflightPresence'], timeout: 60_000, proves: 'Status, message, presence and set level message action listeners all fire from one sample.' }, async () => { - const pubnub = world.fresh('listenerbasic'); - const [c1, ch1, ch2] = [world.n('channel_1'), world.n('ch1'), world.n('ch2')]; - await world.withConsole(async (logs) => { - const ready = connected(pubnub, { channels: [c1, ch1, ch2] }); - - // snippet.eventListenerBasicUsage - // create a subscription from a channel entity - const channel = pubnub.channel('channel_1'); - const subscription1 = channel.subscription({ receivePresenceEvents: true }); - - // create a subscription set with multiple channels - const subscriptionSet1 = pubnub.subscriptionSet({ channels: ['ch1', 'ch2'] }); - - // add a status listener - pubnub.addListener({ - status: (s) => { - console.log('Status', s.category); - }, - }); - - // add message and presence listeners - subscription1.addListener({ - // Messages - message: (m) => { - console.log('Received message', m); - }, - // Presence - presence: (p) => { - console.log('Presence event', p); - }, - }); - - // add event-specific message actions listener - subscriptionSet1.onMessageAction = (p) => { - console.log('Message action event:', p); - }; - - subscription1.subscribe(); - subscriptionSet1.subscribe(); - // snippet.end - - await ready; - const joiner = world.fresh('listenerbasicjoiner'); - const joinerReady = connected(joiner, { channels: [c1] }); - joiner.subscribe({ channels: [c1] }); - await joinerReady; - const onChannel = await world.friend.publish({ channel: c1, message: { n: 1 } }); - const onSet = await world.friend.publish({ channel: ch1, message: { n: 2 } }); - await world.friend.addMessageAction({ - channel: ch1, - messageTimetoken: onSet.timetoken, - action: { type: 'reaction', value: 'smiley_face' }, - }); - assert.ok(onChannel.timetoken); - // One error that lists every handler that has not logged yet, not only the first one. - await eventually(() => assertAll({ - 'status listener saw the connection': hasLine(logs.lines, 'Status PNConnectedCategory'), - 'subscription message listener ran': hasLine(logs.lines, 'Received message'), - 'presence listener saw a presence event (the joiner)': hasLine(logs.lines, 'Presence event'), - 'subscription set message action listener ran': hasLine(logs.lines, 'Message action event:'), - }), { label: 'listeners from the snippet ran', timeout: PRESENCE_EVENT_TIMEOUT }); - joiner.unsubscribeAll(); - subscription1.unsubscribe(); - subscriptionSet1.unsubscribe(); - }); - }); - - step('eventListenerAddConnectionStatusListenersBasicUsage', { proves: 'The status listener fires PNConnectedCategory.' }, async () => { - const pubnub = world.fresh('statuslistener'); - await world.withConsole(async (logs) => { - // snippet.eventListenerAddConnectionStatusListenersBasicUsage - // add a status listener - pubnub.addListener({ - status: (s) => { - console.log('Status', s.category); - }, - }); - // snippet.end - - const subscription = pubnub.channel(world.n('ch1')).subscription(); - subscription.subscribe(); - await eventually(() => assert.ok(hasLine(logs.lines, 'Status PNConnectedCategory')), { label: 'connected status logged' }); - subscription.unsubscribe(); - }); - }); - - step('unsubscribeBasicUsage', { proves: 'Unsubscribing a subscription and a set leaves nothing subscribed.' }, async () => { - const pubnub = world.fresh('unsubscribe'); - // snippet.unsubscribeBasicUsage - // create a subscription from a channel entity - const channel1 = pubnub.channel('channel_1'); - const subscription1 = channel1.subscription({ receivePresenceEvents: true }); - - // create a subscription set with multiple channels - const subscriptionSet1 = pubnub.subscriptionSet({ channels: ['ch1', 'ch2'] }); - - subscription1.subscribe(); - subscriptionSet1.subscribe(); - - subscription1.unsubscribe(); - subscriptionSet1.unsubscribe(); - // snippet.end - await eventually(() => assert.deepEqual(pubnub.getSubscribedChannels(), []), { label: 'nothing left subscribed' }); - }); - - step('unsubscribeAllBasicUsage', { needs: ['subscribeGroupFixture'], proves: 'unsubscribeAll clears channels and groups.' }, async () => { - const pubnub = world.fresh('unsubscribeall'); - // snippet.unsubscribeAllBasicUsage - // create a subscription set with multiple channels - const subscriptionSet = pubnub.subscriptionSet({ channels: ['ch1', 'ch2'] }); - subscriptionSet.subscribe(); - - // create a subscription from a channel entity - const channelGroup1 = pubnub.channelGroup('channelGroup_1'); - const groupSubscription1 = channelGroup1.subscription({ receivePresenceEvents: true }); - groupSubscription1.subscribe(); - - // unsubscribe all active subscriptions - pubnub.unsubscribeAll(); - // snippet.end - await eventually(() => { - assert.deepEqual(pubnub.getSubscribedChannels(), []); - assert.deepEqual(pubnub.getSubscribedChannelGroups(), []); - }, { label: 'unsubscribeAll clears channels and groups' }); - }); - - step('subscriptionSetFrom2IndividualSubscriptions', { needs: ['subscribeGroupFixture'], proves: 'addSubscription and removeSubscription change the set before subscribing.' }, async () => { - const pubnub = world.fresh('setfromsubs'); - const [c1, c3, group] = [world.n('channel_1'), world.n('channel_3'), world.n('channelGroup_1')]; - const ready = connected(pubnub, { channels: [c1], channelGroups: [group] }); - - // snippet.subscriptionSetFrom2IndividualSubscriptions - // create a subscription from a channel entity - const channel = pubnub.channel('channel_1'); - const subscription1 = channel.subscription({ receivePresenceEvents: true }); - - // create a subscription from a channel group entity - const channelGroup = pubnub.channelGroup('channelGroup_1'); - const subscription2 = channelGroup.subscription(); - - // add 2 subscriptions to create a subscription set - const subscriptionSet = subscription1.addSubscription(subscription2); - - // add another subscription to the set - const subscription3 = pubnub.channel('channel_3').subscription({ receivePresenceEvents: false }); - subscriptionSet.addSubscription(subscription3); - - // remove a subscription from a subscription set - subscriptionSet.removeSubscription(subscription3); - - subscriptionSet.subscribe(); - // snippet.end - - await ready; - const channels = pubnub.getSubscribedChannels(); - assert.ok(channels.includes(c1), 'channel_1 is subscribed'); - assert.ok(!channels.includes(c3), 'channel_3 was removed before subscribing'); - assert.ok(pubnub.getSubscribedChannelGroups().includes(group), 'channelGroup_1 is subscribed'); - pubnub.unsubscribeAll(); - }); - - step('SubscriptionSetFrom2Sets', { proves: 'addSubscriptionSet and removeSubscriptionSet are local operations and subscribe nothing.' }, () => { - const pubnub = world.fresh('setfromsets'); - // snippet.SubscriptionSetFrom2Sets - // create a subscription set with multiple channels - const subscriptionSet1 = pubnub.subscriptionSet({ channels: ['ch1', 'ch2'] }); - - // create another subscription set with multiple channels and options - const subscriptionSet2 = pubnub.subscriptionSet({ - channels: ['ch3', 'ch4'], - subscriptionOptions: { receivePresenceEvents: true }, - }); - - // add a subscription set to another subscription set - subscriptionSet1.addSubscriptionSet(subscriptionSet2); - - // remove a subscription set from another subscription set - subscriptionSet1.removeSubscriptionSet(subscriptionSet2); - // snippet.end - // Nothing subscribed, and the first set is back to its own two channels. - assert.deepEqual(pubnub.getSubscribedChannels(), []); - assert.deepEqual(sorted(subscriptionSet1.channels), sorted([world.n('ch1'), world.n('ch2')])); - }); - - step('AddToExistingSubscriptionSet', { needs: ['subscribeGroupFixture'], proves: 'Adding sets and subscriptions to an already subscribed set takes effect at once.' }, async () => { - const pubnub = world.fresh('addtoset'); - const [ch1, ch2, ch3, ch4] = ['ch1', 'ch2', 'ch3', 'ch4'].map((c) => world.n(c)); - const [group, member] = [world.n('channelGroup_1'), world.n('channelGroup_1_member')]; - const ready = connected(pubnub, { channels: [ch1, ch2, ch3, ch4], channelGroups: [group] }); - const onLateChannel = expectEvent(pubnub, 'message', (m) => m.channel === ch4 && (m.message as { n?: number }).n === 4); - const onGroup = expectEvent(pubnub, 'message', (m) => m.channel === member && m.subscription === group); - - // snippet.AddToExistingSubscriptionSet - // create a subscription set with multiple channels - const subscriptionSet1 = pubnub.subscriptionSet({ channels: ['ch1', 'ch2'] }); - - // subscribe to the set - // start receiving events from ch1 and ch2 - subscriptionSet1.subscribe(); - - // create another subscription set with multiple channels - const subscriptionSet2 = pubnub.subscriptionSet({ channels: ['ch3', 'ch4'] }); - - // add the new set to the initial set - subscriptionSet1.addSubscriptionSet(subscriptionSet2); - - // you're now receiving events from ch1, ch2, ch3, and ch4 - // because the set has been subscribed to previously - - // create and add another subscription to the set - const channelGroup = pubnub.channelGroup('channelGroup_1'); - const subscription2 = channelGroup.subscription(); - subscriptionSet1.addSubscription(subscription2); - - // you're now receiving events from ch1, ch2, ch3, and ch4 and channelGroup_1 - // snippet.end - - await ready; - await world.friend.publish({ channel: ch4, message: { n: 4 } }); - await world.friend.publish({ channel: member, message: { n: 5 } }); - await onLateChannel; - await onGroup; - subscriptionSet1.unsubscribe(); - }); -}); diff --git a/docs-snippets/app-context.ts b/docs-snippets/app-context.ts deleted file mode 100644 index 8aa80eda8..000000000 --- a/docs-snippets/app-context.ts +++ /dev/null @@ -1,59 +0,0 @@ -import PubNub, { PubNubError } from '../lib/types'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'myUniqueUserId', -}); - -// snippet.iterativelyUpdateExistingMetadata -const channel = 'team.red'; -const name = 'Red Team'; -const description = 'The channel for Red team.'; -const customField = { visible: 'team' }; - -// Function to set and then update channel metadata -try { - const response = await pubnub.objects.setChannelMetadata({ - channel: channel, - data: { - name: name, - description: description, - custom: customField, - }, - }); - console.log('The channel has been created with name and description.\n'); - - // Fetch current object with custom fields - const currentObjectResponse = await pubnub.objects.getChannelMetadata({ - channel: channel, - include: { - customFields: true, - }, - }); - const currentObject = currentObjectResponse.data; - - // Initialize the custom field object - const custom = currentObject.custom || {}; - - // Add or update the field - custom['edit'] = 'admin'; - - // Writing the updated object back to the server - const setResponse = await pubnub.objects.setChannelMetadata({ - channel: channel, - data: { - name: currentObject.name || '', - description: currentObject.description || '', - custom: custom, - }, - }); - console.log('Object has been updated', setResponse); -} catch (error) { - console.error( - `Set channel metadata error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end diff --git a/docs-snippets/basic-usage/access-manager.ts b/docs-snippets/basic-usage/access-manager.ts deleted file mode 100644 index 356b58107..000000000 --- a/docs-snippets/basic-usage/access-manager.ts +++ /dev/null @@ -1,52 +0,0 @@ -import PubNub, { PubNubError } from '../../lib/types'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'myUniqueUserId', -}); - -// snippet.accessManagerBasicUsage -// Function to use grantToken method -try { - const token = await pubnub.grantToken({ - ttl: 15, - authorized_uuid: 'my-authorized-uuid', - resources: { - channels: { - 'my-channel': { - read: true, - write: true, - }, - }, - }, - }); - console.log('Granted Token:', token); -} catch (error) { - console.error( - `Grant token error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.revokeTokenBasicUsage -try { - const response = await pubnub.revokeToken('p0AkFl043rhDdHRsple3KgQ3NwY6BDcENnctokenVzcqBDczaWdYIGOAeTyWGJI'); -} catch (error) { - console.error( - `Revoke token error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.parseTokenBasicUsage -pubnub.parseToken('use-token-string-generated-by-grantToken()'); -// snippet.end - -// snippet.setTokenBasicUsage -pubnub.setToken('use-token-string-generated-by-grantToken()'); -// snippet.end diff --git a/docs-snippets/basic-usage/app-context.ts b/docs-snippets/basic-usage/app-context.ts deleted file mode 100644 index 04bf02645..000000000 --- a/docs-snippets/basic-usage/app-context.ts +++ /dev/null @@ -1,287 +0,0 @@ -import PubNub from '../../lib/types'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'myUniqueUserId', -}); - -// snippet.getAllUUIDMetadataBasicUsage -// Function to get all UUID metadata -// to get some data in response, add user metadata using setUUIDMetadata method -async function getAllUUIDMetadata() { - try { - const response = await pubnub.objects.getAllUUIDMetadata(); - console.log('getAllUUIDMetadata response:', response); - } catch (error) { - console.error(`Get all UUID metadata error: ${error}`); - } -} - -// Execute the function to get UUID metadata -getAllUUIDMetadata(); -// snippet.end - -// snippet.getUUIDMetadataBasicUsage -// Using UUID from the config - default when uuid is not passed in the method -try { - const response = await pubnub.objects.getUUIDMetadata(); - console.log('getUUIDMetadata response:', response); -} catch (error) { - console.error(`Get UUID metadata error: ${error}`); -} - -// Using the passed in UUID -try { - const response = await pubnub.objects.getUUIDMetadata({ - uuid: 'myUuid', - }); - console.log('getUUIDMetadata response:', response); -} catch (error) { - console.error(`Get UUID metadata error: ${error}`); -} -// snippet.end - -// snippet.setUUIDMetadataBasicUsage -// Using UUID from the config - default when uuid is not passed in the method -try { - const response = await pubnub.objects.setUUIDMetadata({ - data: { - name: 'John Doe', - }, - }); - console.log('setUUIDMetadata response:', response); -} catch (error) { - console.error(`Set UUID metadata error: ${error}`); -} - -// Using the passed in UUID -try { - const response = await pubnub.objects.setUUIDMetadata({ - uuid: 'myUuid', - data: { - email: 'john.doe@example.com', - }, - }); - console.log('setUUIDMetadata response:', response); -} catch (error) { - console.error(`Set UUID metadata error: ${error}`); -} -// snippet.end - -// snippet.removeUUIDMetadataBasicUsage -// Using UUID from the config - default when uuid is not passed in the method -try { - const response = await pubnub.objects.removeUUIDMetadata(); -} catch (error) { - console.error(`Remove UUID metadata error: ${error}`); -} - -// Using the passed in UUID -try { - const response = await pubnub.objects.removeUUIDMetadata({ - uuid: 'myUuid', - }); -} catch (error) { - console.error(`Remove UUID metadata error: ${error}`); -} -// snippet.end - -// snippet.getAllChannelMetadataBasicUsage -// Get the total number of channels included in the response. -try { - const response = await pubnub.objects.getAllChannelMetadata({ - include: { - totalCount: true, - }, - }); -} catch (error) { - console.error(`Get all channel metadata error: ${error}`); -} - -// Get all channels with the filter option. To get all channel which has Id ending 'Team'. -try { - const response = await pubnub.objects.getAllChannelMetadata({ - filter: 'name LIKE "*Team"', - }); - console.log('Get all channel metadata response:', response); -} catch (error) { - console.error(`Get all channel metadata error: ${error}`); -} -// snippet.end - -// snippet.getChannelMetadataBasicUsage -try { - const response = await pubnub.objects.getChannelMetadata({ - // `channel` is the `id` in the _metadata_, not `name` - channel: 'team.blue', - }); - console.log('Get channel metadata response:', response); -} catch (error) { - console.error(`Get channel metadata error: ${error}`); -} -// snippet.end - -// snippet.setChannelMetadataBasicUsage -try { - const response = await pubnub.objects.setChannelMetadata({ - channel: 'team.red', - data: { - name: 'Red Team', - description: 'The channel for Red team and no other teams.', - custom: { - owner: 'Red Leader', - }, - }, - include: { - customFields: false, - }, - }); - console.log('Set channel metadata response:', response); -} catch (error) { - console.error(`Set channel metadata error: ${error}`); -} -// snippet.end - -// snippet.removeChannelMetadataBasicUsage -try { - const response = await pubnub.objects.removeChannelMetadata({ - channel: 'team.red', - }); -} catch (error) { - console.error(`Remove channel metadata error: ${error}`); -} -// snippet.end - -// snippet.getMembershipBasicUsage -// Using UUID from the config -try { - const response = await pubnub.objects.getMemberships(); -} catch (error) { - console.error(`Get memberships error: ${error}`); -} - -// Using the passed in UUID -try { - const response = await pubnub.objects.getMemberships({ - uuid: 'myUuid', - include: { - channelFields: true, - }, - }); -} catch (error) { - console.error(`Get memberships with channels fields included error: ${error}`); -} - -// Get all memberships that are starred by the user -try { - const response = await pubnub.objects.getMemberships({ - uuid: 'myUuid', - filter: 'custom.starred == true', - }); -} catch (error) { - console.error(`Get filtered memberships error: ${error}`); -} -// snippet.end - -// snippet.setMembershipBasicUsage -// Using UUID from the config -try { - const response = await pubnub.objects.setMemberships({ - channels: [ - 'my-channel', - { id: 'channel-with-status-type', custom: { hello: 'World' }, status: 'helloStatus', type: 'helloType' }, - ], - }); - console.log('Set memberships response:', response); -} catch (error) { - console.error(`Set memberships error: ${error}`); -} - -// Using the passed in UUID -try { - const response = await pubnub.objects.setMemberships({ - uuid: 'my-uuid', - channels: [ - 'my-channel', - { id: 'channel-with-status-type', custom: { hello: 'World' }, status: 'helloStatus', type: 'helloType' }, - ], - include: { - // To include channel fields in response - channelFields: true, - }, - }); - console.log('Set memberships response:', response); -} catch (error) { - console.error(`Set memberships error: ${error}`); -} -// snippet.end - -// snippet.removeMembershipsBasicUsage -// Using UUID from the config -try { - const response = await pubnub.objects.removeMemberships({ - channels: ['ch-1', 'ch-2'], - }); -} catch (error) { - console.error(`Remove memberships error: ${error}`); -} - -// Using the passed in UUID -try { - const response = await pubnub.objects.removeMemberships({ - uuid: 'myUuid', - channels: ['ch-1', 'ch-2'], - }); -} catch (error) { - console.error(`Remove memberships for given uuids error: ${error}`); -} -// snippet.end - -// snippet.getChannelMembersBasicUsage -try { - const response = await pubnub.objects.getChannelMembers({ - channel: 'myChannel', - include: { - UUIDFields: true, - }, - }); - console.log('getChannelMembers response:', response); -} catch (error) { - console.error(`Get channel members error: ${error}`); -} - -// Get all channel members with "admin" in the description -try { - const response = await pubnub.objects.getChannelMembers({ - channel: 'myChannel', - filter: 'description LIKE "*admin*"', - }); - console.log('getChannelMembers response:', response); -} catch (error) { - console.error(`Get channel members error: ${error}`); -} -// snippet.end - -// snippet.setChannelMembersBasicUsage -try { - const response = await pubnub.objects.setChannelMembers({ - channel: 'myChannel', - uuids: ['uuid-1', 'uuid-2', { id: 'uuid-3', custom: { role: 'Super Admin' } }], - }); - console.log('setChannelMembers response:', response); -} catch (error) { - console.error(`Set channel members error: ${error}`); -} -// snippet.end - -// snippet.removeChannelMembersBasicUsage -try { - const response = await pubnub.objects.removeChannelMembers({ - channel: 'myChannel', - uuids: ['uuid-1', 'uuid-2'], - }); -} catch (error) { - console.error(`Remove channel members error: ${error}`); -} -// snippet.end diff --git a/docs-snippets/basic-usage/channel-groups.ts b/docs-snippets/basic-usage/channel-groups.ts deleted file mode 100644 index 817f7d622..000000000 --- a/docs-snippets/basic-usage/channel-groups.ts +++ /dev/null @@ -1,62 +0,0 @@ -import PubNub from '../../lib/types'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'myUniqueUserId', -}); - -// snippet.addChannelsToGroupBasicUsage -try { - const response = await pubnub.channelGroups.addChannels({ - channels: ['ch1', 'ch2'], - channelGroup: 'myChannelGroup', - }); - console.log('addChannels to Group response:', response); -} catch (error) { - console.error(`Add channels to group error: ${error}`); -} -// snippet.end - -// snippet.listChannelsInGroupBasicUsage -// assuming an intialized PubNub instance already exists -// to get some data in response, first add some channels to the group using addChannels() method. -try { - const response = await pubnub.channelGroups.listChannels({ - channelGroup: 'myChannelGroup', - }); - console.log('Listing push channels for the device:', response); - response.channels.forEach((channel) => { - console.log(channel); - }); -} catch (error) { - console.error(`List channels of group error: ${error}`); -} -// snippet.end - -// snippet.removeChannelsFromGroupBasicUsage -// assuming an initialized PubNub instance already exists -// and channel which is going to be removed from the group is aredaly added to the group to observe the removal -try { - const response = await pubnub.channelGroups.removeChannels({ - channels: ['son'], - channelGroup: 'family', - }); - console.log('removeChannels from group response:', response); -} catch (error) { - console.error(`Remove channels from group error: ${error}`); -} -// snippet.end - -// snippet.deleteChannelGroupBasicUsage -// assuming an initialized PubNub instance already exists -// and channel group which is getting deleted already exist to see the deletion effect. -try { - const response = await pubnub.channelGroups.deleteGroup({ - channelGroup: 'family', - }); - console.log('deleteChannelGroup response:', response); -} catch (error) { - console.error(`Delete channel group error: ${error}`); -} -// snippet.end diff --git a/docs-snippets/basic-usage/configuration.ts b/docs-snippets/basic-usage/configuration.ts deleted file mode 100644 index 6d8f7583a..000000000 --- a/docs-snippets/basic-usage/configuration.ts +++ /dev/null @@ -1,43 +0,0 @@ -import PubNub from '../../src/web/index'; - -// snippet.configurationBasicUsageSubscriptionWorkerUrl -const pubnub = new PubNub({ - subscribeKey: 'demo', - publishKey: 'demo', - userId: 'unique-user-id', - // using PubNub JS SDK v9.6.0, make sure the versions match. - // NOTE: 'subscriptionWorkerUrl' is only needed if you need to use SharedWorker. - subscriptionWorkerUrl: 'https://www.my-domain.com/static/js/pubnub.worker.9.6.0.js', -}); -// snippet.end - -// snippet.setAuthKeyBasicUsage -pubnub.setAuthKey('my_authkey'); -// snippet.end - -// snippet.setFilterExpressionBasicUsage -pubnub.setFilterExpression('such=wow'); -// snippet.end - -// snippet.getFilterExpressionBasicUsage -pubnub.getFilterExpression(); -// snippet.end - -// snippet.generateUUIDBasicUsage(deprecated) -PubNub.generateUUID(); -// snippet.end - -// snippet.configurationBasicUsage -// Initialize PubNub with your keys -const pubnubConfig = new PubNub({ - subscribeKey: 'YOUR_SUBSCRIBE_KEY', - publishKey: 'YOUR_PUBLISH_KEY', - userId: 'YOUR_USER_ID', - cryptoModule: PubNub.CryptoModule?.aesCbcCryptoModule({ cipherKey: 'YOUR_CIPHER_KEY' }), - authKey: 'accessMangerToken', - logLevel: PubNub.LogLevel.Debug, - ssl: true, - presenceTimeout: 130, -}); - -// snippet.end diff --git a/docs-snippets/basic-usage/data-sync.ts b/docs-snippets/basic-usage/data-sync.ts deleted file mode 100644 index 8bb82975a..000000000 --- a/docs-snippets/basic-usage/data-sync.ts +++ /dev/null @@ -1,508 +0,0 @@ -import PubNub, { PubNubError } from '../../lib/types'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'myUniqueUserId', -}); - -// snippet.createUserBasicUsage -try { - const response = await pubnub.dataSync.createUser({ - id: 'user-alice', - data: { - classVersion: 1, - payload: { name: 'Alice', type: 'shopper' }, - }, - }); - console.log('createUser response:', response.data); -} catch (error) { - console.error( - `Create user error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.getUserBasicUsage -try { - const response = await pubnub.dataSync.getUser({ id: 'user-alice' }); - console.log('getUser response:', response.data); -} catch (error) { - console.error( - `Get user error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.getUsersBasicUsage -try { - const response = await pubnub.dataSync.getUsers({ - limit: 20, - sort: { createdAt: 'asc' }, - }); - console.log('getUsers response:', response.data); - console.log('getUsers meta:', response.meta); -} catch (error) { - console.error( - `Get users error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.updateUserBasicUsage -try { - const response = await pubnub.dataSync.setUser({ - id: 'user-alice', - data: { - classVersion: 1, - payload: { name: 'Alice B.', type: 'shopper' }, - }, - ifMatchesEtag: 'AbQdEfGhIjKlMn', - }); - console.log('setUser response:', response.data); -} catch (error) { - console.error( - `Set user error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.patchUserBasicUsage -try { - const response = await pubnub.dataSync.updateUser({ - id: 'user-alice', - replace: { '/payload/name': 'Alice B.', '/status': 'active' }, - }); - console.log('updateUser response:', response.data); -} catch (error) { - console.error( - `Update user error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.removeUserBasicUsage -try { - const response = await pubnub.dataSync.removeUser({ id: 'user-alice' }); - console.log('removeUser response status:', response.status); -} catch (error) { - console.error( - `Remove user error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.createChannelBasicUsage -try { - const response = await pubnub.dataSync.createChannel({ - id: 'channel-summer-sale', - data: { - classVersion: 1, - payload: { name: 'Summer Sale', type: 'promotion' }, - }, - }); - console.log('createChannel response:', response.data); -} catch (error) { - console.error( - `Create channel error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.getChannelBasicUsage -try { - const response = await pubnub.dataSync.getChannel({ id: 'channel-summer-sale' }); - console.log('getChannel response:', response.data); -} catch (error) { - console.error( - `Get channel error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.getChannelsBasicUsage -try { - const response = await pubnub.dataSync.getChannels({ limit: 20 }); - console.log('getChannels response:', response.data); - console.log('getChannels meta:', response.meta); -} catch (error) { - console.error( - `Get channels error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.updateChannelBasicUsage -try { - const response = await pubnub.dataSync.setChannel({ - id: 'channel-summer-sale', - data: { - classVersion: 1, - payload: { name: 'Summer Sale 2026', type: 'promotion' }, - }, - }); - console.log('setChannel response:', response.data); -} catch (error) { - console.error( - `Set channel error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.patchChannelBasicUsage -try { - const response = await pubnub.dataSync.updateChannel({ - id: 'channel-summer-sale', - replace: { '/payload/name': 'Summer Sale 2026' }, - }); - console.log('updateChannel response:', response.data); -} catch (error) { - console.error( - `Update channel error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.removeChannelBasicUsage -try { - const response = await pubnub.dataSync.removeChannel({ id: 'channel-summer-sale' }); - console.log('removeChannel response status:', response.status); -} catch (error) { - console.error( - `Remove channel error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.createMembershipBasicUsage -try { - const response = await pubnub.dataSync.createMembership({ - userId: 'user-alice', - channelId: 'channel-summer-sale', - data: { - classVersion: 1, - payload: { role: 'viewer' }, - }, - }); - console.log('createMembership response:', response.data); -} catch (error) { - console.error( - `Create membership error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.getMembershipBasicUsage -try { - const response = await pubnub.dataSync.getMembership({ - id: 'membership-alice-summer-sale', - }); - console.log('getMembership response:', response.data); -} catch (error) { - console.error( - `Get membership error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.getMembershipsBasicUsage -try { - const response = await pubnub.dataSync.getMemberships({ - userId: 'user-alice', - limit: 20, - }); - console.log('getMemberships response:', response.data); - console.log('getMemberships meta:', response.meta); -} catch (error) { - console.error( - `Get memberships error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.updateMembershipBasicUsage -try { - const response = await pubnub.dataSync.setMembership({ - id: 'membership-alice-summer-sale', - userId: 'user-alice', - channelId: 'channel-summer-sale', - data: { - classVersion: 1, - payload: { role: 'moderator' }, - }, - }); - console.log('setMembership response:', response.data); -} catch (error) { - console.error( - `Set membership error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.patchMembershipBasicUsage -try { - const response = await pubnub.dataSync.updateMembership({ - id: 'membership-alice-summer-sale', - replace: { '/payload/role': 'moderator' }, - }); - console.log('updateMembership response:', response.data); -} catch (error) { - console.error( - `Update membership error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.removeMembershipBasicUsage -try { - const response = await pubnub.dataSync.removeMembership({ - id: 'membership-alice-summer-sale', - }); - console.log('removeMembership response status:', response.status); -} catch (error) { - console.error( - `Remove membership error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.createEntityBasicUsage -try { - const response = await pubnub.dataSync.createEntity({ - id: 'product-sneaker-42', - class: 'product', - data: { - classVersion: 1, - payload: { name: 'Retro Sneaker', price: 89.99, stock: 12 }, - }, - }); - console.log('createEntity response:', response.data); -} catch (error) { - console.error( - `Create entity error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.getEntityBasicUsage -try { - const response = await pubnub.dataSync.getEntity({ id: 'product-sneaker-42' }); - console.log('getEntity response:', response.data); -} catch (error) { - console.error( - `Get entity error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.getEntitiesBasicUsage -try { - const response = await pubnub.dataSync.getEntities({ - class: 'product', - sort: { price: 'desc' }, - limit: 20, - }); - console.log('getEntities response:', response.data); - console.log('getEntities meta:', response.meta); -} catch (error) { - console.error( - `Get entities error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.updateEntityBasicUsage -try { - const response = await pubnub.dataSync.setEntity({ - id: 'product-sneaker-42', - data: { - classVersion: 1, - payload: { name: 'Retro Sneaker', price: 79.99, stock: 8 }, - }, - }); - console.log('setEntity response:', response.data); -} catch (error) { - console.error( - `Set entity error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.patchEntityBasicUsage -try { - const response = await pubnub.dataSync.updateEntity({ - id: 'product-sneaker-42', - replace: { '/payload/price': 79.99 }, - }); - console.log('updateEntity response:', response.data); -} catch (error) { - console.error( - `Update entity error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.removeEntityBasicUsage -try { - const response = await pubnub.dataSync.removeEntity({ id: 'product-sneaker-42' }); - console.log('removeEntity response status:', response.status); -} catch (error) { - console.error( - `Remove entity error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.createRelationshipBasicUsage -try { - const response = await pubnub.dataSync.createRelationship({ - class: 'ProductOwner', - entityAId: 'seller-bob', - entityBId: 'product-sneaker-42', - data: { - classVersion: 1, - payload: { since: '2026-07-13' }, - }, - }); - console.log('createRelationship response:', response.data); -} catch (error) { - console.error( - `Create relationship error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.getRelationshipBasicUsage -try { - const response = await pubnub.dataSync.getRelationship({ - id: 'rel-bob-owns-sneaker-42', - }); - console.log('getRelationship response:', response.data); -} catch (error) { - console.error( - `Get relationship error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.getRelationshipsBasicUsage -try { - const response = await pubnub.dataSync.getRelationships({ - class: 'ProductOwner', - entityAId: 'seller-bob', - limit: 20, - }); - console.log('getRelationships response:', response.data); - console.log('getRelationships meta:', response.meta); -} catch (error) { - console.error( - `Get relationships error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.updateRelationshipBasicUsage -try { - const response = await pubnub.dataSync.setRelationship({ - id: 'rel-bob-owns-sneaker-42', - entityAId: 'seller-bob', - entityBId: 'product-sneaker-42', - data: { - classVersion: 1, - payload: { since: '2026-07-13', tier: 'gold' }, - }, - }); - console.log('setRelationship response:', response.data); -} catch (error) { - console.error( - `Set relationship error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.patchRelationshipBasicUsage -try { - const response = await pubnub.dataSync.updateRelationship({ - id: 'rel-bob-owns-sneaker-42', - replace: { '/payload/tier': 'platinum' }, - }); - console.log('updateRelationship response:', response.data); -} catch (error) { - console.error( - `Update relationship error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.removeRelationshipBasicUsage -try { - const response = await pubnub.dataSync.removeRelationship({ - id: 'rel-bob-owns-sneaker-42', - }); - console.log('removeRelationship response status:', response.status); -} catch (error) { - console.error( - `Remove relationship error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end diff --git a/docs-snippets/basic-usage/download-file-web.ts b/docs-snippets/basic-usage/download-file-web.ts deleted file mode 100644 index 501459edd..000000000 --- a/docs-snippets/basic-usage/download-file-web.ts +++ /dev/null @@ -1,35 +0,0 @@ -import { PubNubError } from 'lib/types'; -import PubNub from '../../src/web/index'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'myUniqueUserId', -}); - -// snippet.downloadFileWebBasicUsage -// In browser -// download the intended file -const downloadFile = async () => { - try { - const file = await pubnub.downloadFile({ - channel: 'my_channel', - id: '...', - name: 'cat_picture.jpg', - }); - - // have proper html element to display the file - const myImageTag = document.createElement('img'); - myImageTag.src = URL.createObjectURL(await file!.toFile()); - - // attach the file content to the html element - document.body.appendChild(myImageTag); - } catch (error) { - console.error( - `Download file error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); - } -}; -// snippet.end diff --git a/docs-snippets/basic-usage/event-listener.ts b/docs-snippets/basic-usage/event-listener.ts deleted file mode 100644 index 287e02c32..000000000 --- a/docs-snippets/basic-usage/event-listener.ts +++ /dev/null @@ -1,52 +0,0 @@ -import PubNub from '../../lib/types'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'myUniqueUserId', -}); - -// snippet.eventListenerBasicUsage -// create a subscription from a channel entity -const channel = pubnub.channel('channel_1'); -const subscription1 = channel.subscription({ receivePresenceEvents: true }); - -// create a subscription set with multiple channels -const subscriptionSet1 = pubnub.subscriptionSet({ channels: ['ch1', 'ch2'] }); - -// add a status listener -pubnub.addListener({ - status: (s) => { - console.log('Status', s.category); - }, -}); - -// add message and presence listeners -subscription1.addListener({ - // Messages - message: (m) => { - console.log('Received message', m); - }, - // Presence - presence: (p) => { - console.log('Presence event', p); - }, -}); - -// add event-specific message actions listener -subscriptionSet1.onMessageAction = (p) => { - console.log('Message action event:', p); -}; - -subscription1.subscribe(); -subscriptionSet1.subscribe(); -// snippet.end - -// snippet.eventListenerAddConnectionStatusListenersBasicUsage -// add a status listener -pubnub.addListener({ - status: (s) => { - console.log('Status', s.category); - }, -}); -// snippet.end diff --git a/docs-snippets/basic-usage/file-sharing.ts b/docs-snippets/basic-usage/file-sharing.ts deleted file mode 100644 index 4f8df43a2..000000000 --- a/docs-snippets/basic-usage/file-sharing.ts +++ /dev/null @@ -1,106 +0,0 @@ -import PubNub from '../../lib/types'; -import fs from 'fs'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'myUniqueUserId', -}); - -// snippet.sendFileBasicUsage -// Function to send a file to a channel -try { - // in node.js, make sure to import 'fs' - // and use the createReadStream method to read the file - const myFile = fs.createReadStream('./cat_picture.jpg'); - - const response = await pubnub.sendFile({ - channel: 'my_channel', - file: { stream: myFile, name: 'cat_picture.jpg', mimeType: 'image/jpeg' }, - customMessageType: 'file-message', - }); - - console.log('File sent successfully:', response); -} catch (error) { - console.error(`Error sending file: ${error}`); -} -// snippet.end - -// snippet.listFilesBasicUsage -try { - const response = await pubnub.listFiles({ channel: 'my_channel' }); - console.log('Files listed successfully:', response); -} catch (error) { - console.error(`Error listing files: ${error}`); -} -// snippet.end - -// snippet.getFileUrlBasicUsage -const response = pubnub.getFileUrl({ channel: 'my_channel', id: '...', name: '...' }); -// snippet.end - -// snippet.downloadFileNodeBasicUsage -// In Node.js using streams: -// import fs from 'fs' -try { - const downloadFileResponse = await pubnub.downloadFile({ - channel: 'my_channel', - id: '...', - name: 'cat_picture.jpg', - }); - - const output = fs.createWriteStream('./cat_picture.jpg'); - const fileStream = await downloadFileResponse.toStream(); - - fileStream.pipe(output); - - output.once('end', () => { - console.log('File saved to ./cat_picture.jpg'); - }); -} catch (error) { - console.error(`Error downloading file: ${error}`); -} -// snippet.end - -// snippet.downloadFileReactNativeBasicUsage -// in React and React Native -let fileContent; -try { - const file = await pubnub.downloadFile({ - channel: 'awesomeChannel', - id: 'imageId', - name: 'cat_picture.jpg', - }); - fileContent = await file.toBlob(); -} catch (error) { - console.error(`Error downloading file: ${error}`); -} -// snippet.end - -// snippet.deleteFileBasicUsage -try { - const deleteFileResponse = await pubnub.deleteFile({ - channel: 'my_channel', - id: '...', - name: 'cat_picture.jpg', - }); - console.log('File deleted successfully:', deleteFileResponse); -} catch (error) { - console.error(`Error deleting file: ${error}`); -} -// snippet.end - -// snippet.publishFileMessageBasicUsage -try { - const fileMessageResponse = await pubnub.publishFile({ - channel: 'my_channel', - fileId: '...', - fileName: 'cat_picture.jpg', - message: { field: 'value' }, - customMessageType: 'file-message', - }); - console.log('File message published successfully:', fileMessageResponse); -} catch (error) { - console.error(`Error publishing file message: ${error}`); -} -// snippet.end diff --git a/docs-snippets/basic-usage/message-actions.ts b/docs-snippets/basic-usage/message-actions.ts deleted file mode 100644 index 44b965789..000000000 --- a/docs-snippets/basic-usage/message-actions.ts +++ /dev/null @@ -1,66 +0,0 @@ -import PubNub, { PubNubError } from '../../lib/types'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'myUniqueUserId', -}); - -// snippet.addMessageActionBasicUsage -// first publish a message using publish() method to get the message timetoken -try { - const response = await pubnub.addMessageAction({ - channel: 'channel_name', - messageTimetoken: 'replace_with_message_timetoken', // Replace with actual message timetoken - action: { - type: 'reaction', - value: 'smiley_face', - }, - }); - console.log('Message reaction added successfully:', response); -} catch (error) { - console.error( - `Error adding reaction: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} - -// snippet.end - -// snippet.removeMessageActionBasicUsage -try { - const response = await pubnub.removeMessageAction({ - channel: 'channel_name', - messageTimetoken: 'replace_with_message_timetoken', - actionTimetoken: 'replace_with_action_timetoken', - }); - console.log('Message action removed successfully:', response); -} catch (error) { - console.error( - `Error removing message action: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.getMessageActionsBasicUsage -// to get some data in response, first publish a message and then add a message action -// using addMessageAction() method. -try { - const response = await pubnub.getMessageActions({ - channel: 'channel_name', - start: 'replace_with_start_timetoken', - end: 'replace_with_end_timetoken', - limit: 100, - }); - console.log('Message actions retrieved successfully:', response); -} catch (error) { - console.error( - `Error retrieving message actions: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end diff --git a/docs-snippets/basic-usage/message-persistence.ts b/docs-snippets/basic-usage/message-persistence.ts deleted file mode 100644 index 284e1bed9..000000000 --- a/docs-snippets/basic-usage/message-persistence.ts +++ /dev/null @@ -1,48 +0,0 @@ -import PubNub from '../../lib/types'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'myUniqueUserId', -}); - -// snippet.fetchMessagesBasicUsage -// Function to fetch message history -try { - const result = await pubnub.fetchMessages({ - channels: ['my-channel'], - count: 1, // Number of messages to retrieve - includeCustomMessageType: true, // if you want to include custom message type in the response - start: 'replace-with-start-timetoken', // start timetoken - end: 'replace-with-end-timetoken', // end timetoken - }); - console.log('Fetched Messages:', result); -} catch (error) { - console.error(`Messages fetch failed: ${error}`); -} -// snippet.end - -// snippet.deleteMessagesBasicUsage -try { - const result = await pubnub.deleteMessages({ - channel: 'ch1', - start: 'replace-with-start-timetoken', - end: 'replace-with-end-timetoken', - }); - console.log('Messages deleted successfully:', result); -} catch (error) { - console.error(`Messages delete failed: ${error}`); -} -// snippet.end - -// snippet.messageCountBasicUsage -try { - const result = await pubnub.messageCounts({ - channels: ['chats.room1', 'chats.room2'], - channelTimetokens: ['replace-with-timetoken'], // single timetoken applied to all channels - }); - console.log('Message counts retrieved successfully:', result); -} catch (error) { - console.error(`Message counts retrieval failed: ${error}`); -} -// snippet.end diff --git a/docs-snippets/basic-usage/miscellaneous.ts b/docs-snippets/basic-usage/miscellaneous.ts deleted file mode 100644 index 56ab7d566..000000000 --- a/docs-snippets/basic-usage/miscellaneous.ts +++ /dev/null @@ -1,69 +0,0 @@ -import PubNub from '../../lib/types'; -import fs from 'fs'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'myUniqueUserId', -}); - -// snippet.encryptMessageBasicUsage -// Create a crypto module instance with AES-CBC encryption -const cryptoModule = PubNub.CryptoModule.aesCbcCryptoModule({ - cipherKey: 'pubnubenigma', -}); - -const msgContent = 'This is the data I wish to encrypt.'; -console.log(`Original Message: ${msgContent}`); - -// Encrypt the message -const encryptedMessage = cryptoModule.encrypt(JSON.stringify(msgContent)); -console.log('message encrypted successfully'); -// snippet.end - -// snippet.encryptFileBasicUsage -// Node.js example -// import fs from 'fs'; - -const fileBuffer = fs.readFileSync('./cat_picture.jpg'); - -const file = pubnub.File.create({ data: fileBuffer, name: 'cat_picture.jpg', mimeType: 'image/jpeg' }); - -const encryptedFile = await pubnub.encryptFile(file); -// snippet.end - -const encrypted = '..'; -// snippet.decryptBasicUsage -const decrypted = pubnub.decrypt(encrypted); // Pass the encrypted data as the first parameter in decrypt Method -// snippet.end - -// snippet.decryptFileBasicUsage -// Node.js example. -// import fs from 'fs'; -const fileBufferData = fs.readFileSync('./cat_picture_encrypted.jpg'); - -const fileData = pubnub.File.create({ data: fileBuffer, name: 'cat_picture.jpg', mimeType: 'image/jpeg' }); - -const decryptedFile = await pubnub.decryptFile(fileData); -// snippet.end - -// snippet.setProxyBasicUsage -// This method is only available for NodeJS. -pubnub.setProxy({ - hostname: 'YOUR_HOSTNAME', - port: 8080, - protocol: 'YOUR_PROTOCOL', -}); -// snippet.end - -// snippet.disconnectBasicUsage -pubnub.disconnect(); -// snippet.end - -// snippet.reconnectBasicUsage -pubnub.reconnect(); -// snippet.end - -// snippet.deleteProxy -pubnub.setProxy(); -// snippet.end diff --git a/docs-snippets/basic-usage/mobile-push.ts b/docs-snippets/basic-usage/mobile-push.ts deleted file mode 100644 index 1980a5931..000000000 --- a/docs-snippets/basic-usage/mobile-push.ts +++ /dev/null @@ -1,146 +0,0 @@ -import PubNub from '../../lib/types'; - -// Initialize PubNub with demo keys -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'myUniqueUserId', -}); - -// snippet.addDeciveToChannelBasicUsage -// Function to add a device to a channel for APNs2 -try { - const response = await pubnub.push.addChannels({ - channels: ['a', 'b'], - device: 'niceDevice', - pushGateway: 'apns2', - environment: 'production', - topic: 'com.example.bundle_id', - }); - console.log('device added to channels response:', response); -} catch (error) { - console.error(`Error adding device to channels: ${error}`); -} - -// Function to add a device to a channel for FCM -try { - const response = await pubnub.push.addChannels({ - channels: ['a', 'b'], - device: 'niceDevice', - pushGateway: 'fcm', - }); - console.log('device added to channels response:', response); -} catch (error) { - console.error(`Error adding device to channels: ${error}`); -} -// snippet.end - -// snippet.listChannelsForDeviceBasicUsage -// for APNs2 -try { - const response = await pubnub.push.listChannels({ - device: 'niceDevice', - pushGateway: 'apns2', - environment: 'production', - topic: 'com.example.bundle_id', - }); - console.log('listing channels for device response:', response); - response.channels.forEach((channel) => { - console.log(channel); - }); -} catch (error) { - console.error(`Error listing channels for device: ${error}`); -} - -// for FCM -try { - const response = await pubnub.push.listChannels({ - device: 'niceDevice', - pushGateway: 'fcm', - }); - - console.log('listing channels for device response:', response); - - response.channels.forEach((channel) => { - console.log(channel); - }); -} catch (error) { - console.error(`Error listing channels for device: ${error}`); -} -// snippet.end - -// snippet.removeDeviceFromChannelBasicUsage -// for APNs2 -try { - const response = await pubnub.push.removeChannels({ - channels: ['a', 'b'], - device: 'niceDevice', - pushGateway: 'apns2', - environment: 'production', - topic: 'com.example.bundle_id', - }); - - console.log('removing device from channel response:', response); -} catch (error) { - console.error(`Error removing device from channel: ${error}`); -} - -// for FCM -try { - const response = await pubnub.push.removeChannels({ - channels: ['a', 'b'], - device: 'niceDevice', - pushGateway: 'fcm', - }); - - console.log('removing device from channel response:', response); -} catch (error) { - console.error(`Error removing device from channel: ${error}`); -} -// snippet.end - -// snippet.removeAllMobilePushNotificationsBasicUsage - -// for APNs2 -try { - const response = await pubnub.push.deleteDevice({ - device: 'niceDevice', - pushGateway: 'apns2', - environment: 'production', - topic: 'com.example.bundle_id', - }); - - console.log('deleteDevice response:', response); -} catch (error) { - console.error(`Error deleting device: ${error}`); -} - -// for FCM -try { - const response = await pubnub.push.deleteDevice({ - device: 'niceDevice', - pushGateway: 'fcm', - }); - - console.log('deleteDevice response:', response); -} catch (error) { - console.error(`Error deleting device: ${error}`); -} - -// snippet.end - -// snippet.buildNotificationPayloadBasicUsage - -const builder = PubNub.notificationPayload('Chat invitation', "You have been invited to 'quiz' chat"); -const messagePayload = builder.buildPayload(['apns2', 'fcm']); -// add required fields to the payload -try { - const response = await pubnub.publish({ - message: messagePayload, - channel: 'chat-bot', - }); - console.log('publish response:', response); -} catch (error) { - console.error(`Error publishing message: ${error}`); -} -// snippet.end diff --git a/docs-snippets/basic-usage/presence.ts b/docs-snippets/basic-usage/presence.ts deleted file mode 100644 index 70b0a1449..000000000 --- a/docs-snippets/basic-usage/presence.ts +++ /dev/null @@ -1,75 +0,0 @@ -import PubNub from '../../lib/types'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'myUniqueUserId', -}); - -// snippet.hereNowBasicUsage -// Function to get presence information for a channel -try { - const result = await pubnub.hereNow({ - channels: ['ch1'], - channelGroups: ['cg1'], - includeUUIDs: true, - includeState: true, - }); - console.log('Here Now Result:', result); -} catch (error) { - console.error(`Here Now failed with error: ${error}`); -} -// snippet.end - -// snippet.whereNowBasicUsage -try { - const response = await pubnub.whereNow({ - uuid: 'uuid', - }); - console.log('State set successfully:', response); -} catch (error) { - console.error(`State set failed: ${error}`); -} -// snippet.end - -// snippet.setStateBasicUsage -try { - const response = await pubnub.setState({ - state: { status: 'online' }, - channels: ['ch1'], - channelGroups: ['cg1'], - }); - console.log('State set successfully:', response); -} catch (error) { - console.error(`State set failed: ${error}`); -} -// snippet.end - -// snippet.getStateBasicUsage -try { - const response = await pubnub.getState({ - uuid: 'uuid', - channels: ['ch1'], - channelGroups: ['cg1'], - }); - console.log('State retrieved successfully:', response); -} catch (error) { - console.error(`State retrieval failed: ${error}`); -} -// snippet.end - -// snippet.basicUsageWithPromises -pubnub - .hereNow({ - channels: ['ch1'], - channelGroups: ['cg1'], - includeUUIDs: true, - includeState: true, - }) - .then((response) => { - console.log(response); - }) - .catch((error) => { - console.log(error); - }); -// snippet.end diff --git a/docs-snippets/basic-usage/publish-subscribe.ts b/docs-snippets/basic-usage/publish-subscribe.ts deleted file mode 100644 index 5d514c770..000000000 --- a/docs-snippets/basic-usage/publish-subscribe.ts +++ /dev/null @@ -1,142 +0,0 @@ -import PubNub, { PubNubError } from '../../lib/types'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'myUniqueUserId', -}); - -// snippet.publishBasicUsage -try { - const response = await pubnub.publish({ - message: { text: 'Hello World' }, - channel: 'my_channel', - sendByPost: false, - storeInHistory: true, - meta: { sender: 'user123' }, - customMessageType: 'text-message', - }); - console.log('Publish Success:', response); -} catch (error) { - console.error( - `Publish Failed: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.signalBasicUsage -try { - const response = await pubnub.signal({ - message: 'hello', - channel: 'foo', - customMessageType: 'text-message', - }); - console.log('signal response:', response); -} catch (error) { - // handle error - console.error( - `Signal failed with error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.fireBasicUsage -// Initialize PubNub with your keys -try { - const response = await pubnub.fire({ - message: { - such: 'object', - }, - channel: 'my_channel', - sendByPost: false, // true to send via post - meta: { - cool: 'meta', - }, // fire extra meta with the request - }); - - console.log(`message published with timetoken: ${response.timetoken}`); -} catch (error) { - // handle error - console.error( - `Fire failed with error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.createChannelBasicUsage -const channel = pubnub.channel('my_channel'); -// snippet.end - -// snippet.createChannelGroupBasicUsage -// Initialize PubNub with demo keys -const channelGroup = pubnub.channelGroup('channelGroup_1'); -// snippet.end - -// snippet.createChannelMetadataBasicUsage -// Initialize PubNub with demo keys - -const channelMetadata = pubnub.channelMetadata('channel_1'); -// snippet.end - -// snippet.createUserMetadataBasicUsage -// Initialize PubNub with demo keys - -const userMetadata = pubnub.userMetadata('user_meta1'); -// snippet.end - -// snippet.unsubscribeBasicUsage -// create a subscription from a channel entity -const channel1 = pubnub.channel('channel_1'); -const subscription1 = channel1.subscription({ receivePresenceEvents: true }); - -// create a subscription set with multiple channels -const subscriptionSet1 = pubnub.subscriptionSet({ channels: ['ch1', 'ch2'] }); - -subscription1.subscribe(); -subscriptionSet1.subscribe(); - -subscription1.unsubscribe(); -subscriptionSet1.unsubscribe(); -// snippet.end - -// snippet.subscriptionSetSubscribeBasicUsage -const channelsSubscriptionSet = pubnub.subscriptionSet({ channels: ['ch1', 'ch2'] }); -channelsSubscriptionSet.subscribe(); -// snippet.end - -// snippet.unsubscribeAllBasicUsage -// create a subscription set with multiple channels -const subscriptionSet = pubnub.subscriptionSet({ channels: ['ch1', 'ch2'] }); -subscriptionSet.subscribe(); - -// create a subscription from a channel entity -const channelGroup1 = pubnub.channelGroup('channelGroup_1'); -const groupSubscription1 = channelGroup1.subscription({ receivePresenceEvents: true }); -groupSubscription1.subscribe(); - -// unsubscribe all active subscriptions -pubnub.unsubscribeAll(); -// snippet.end - -// *********** OLD SYNTAX *********** -// snippet.OLDsubscribeBasicUsage -pubnub.subscribe({ - channels: ['my_channel'], -}); -// snippet.end - -// snippet.OLDUnsubscribeBasicUsage -pubnub.unsubscribe({ - channels: ['my_channel'], -}); -// snippet.end - -// snippet.OLDUnsubscribeAllBasicUsage -pubnub.unsubscribeAll(); -// snippet.end diff --git a/docs-snippets/configuration.ts b/docs-snippets/configuration.ts deleted file mode 100644 index 5c53d7128..000000000 --- a/docs-snippets/configuration.ts +++ /dev/null @@ -1,54 +0,0 @@ -import PubNub from '../lib/types'; - -// snippet.configurationCryptoModule -// encrypts using 256-bit AES-CBC cipher (recommended) -// decrypts data encrypted with the legacy and the 256 bit AES-CBC ciphers -const pubnub = new PubNub({ - subscribeKey: 'YOUR_SUBSCRIBE_KEY', - userId: 'YOUR_USER_ID', - cryptoModule: PubNub.CryptoModule.aesCbcCryptoModule({ cipherKey: 'pubnubenigma' }), -}); - -// encrypts with 128-bit cipher key entropy (legacy) -// decrypts data encrypted with the legacy and the 256-bit AES-CBC ciphers -const pubnubLegacy = new PubNub({ - subscribeKey: 'YOUR_SUBSCRIBE_KEY', - userId: 'YOUR_USER_ID', - cryptoModule: PubNub.CryptoModule.legacyCryptoModule({ cipherKey: 'pubnubenigma' }), -}); -// snippet.end - -// snippet.configurationServerOpertaion -const pubnubServer = new PubNub({ - subscribeKey: 'mySubscribeKey', - publishKey: 'myPublishKey', - userId: 'myUniqueUserId', - secretKey: 'secretKey', - heartbeatInterval: 0, -}); -// snippet.end - -// snippet.configurationRealOnlyClient -// Initialize for Read Only Client - -const pubnubReadOnly = new PubNub({ - subscribeKey: 'mySubscribeKey', - userId: 'myUniqueUserId', -}); -// snippet.end - -// snippet.configurationSSLEnabled -const pubnubSSL = new PubNub({ - subscribeKey: 'mySubscribeKey', - publishKey: 'myPublishKey', - cryptoModule: PubNub.CryptoModule.aesCbcCryptoModule({ cipherKey: 'pubnubenigma' }), - authKey: 'myAuthKey', - logLevel: PubNub.LogLevel.Debug, - userId: 'myUniqueUserId', - ssl: true, -}); -// snippet.end - -// snippet.generateUUIDdeprected -const uuid = PubNub.generateUUID(); -// snippet.end diff --git a/docs-snippets/coverage.json b/docs-snippets/coverage.json deleted file mode 100644 index f186841ba..000000000 --- a/docs-snippets/coverage.json +++ /dev/null @@ -1,1901 +0,0 @@ -{ - "$comment": "Generated by npm run docs-snippets:manifest. Do not edit. One entry per snippet id: how the live suite covers it.", - "snippets": { - "AddToExistingSubscriptionSet": { - "file": "api-reference/subscribe.ts", - "step": "AddToExistingSubscriptionSet", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "OLDUnsubscribeAllBasicUsage": { - "file": "api-reference/subscribe-legacy.ts", - "step": "OLDUnsubscribeAllBasicUsage", - "role": "open", - "phase": "P8", - "mode": "live", - "reason": null - }, - "OLDUnsubscribeBasicUsage": { - "file": "api-reference/subscribe-legacy.ts", - "step": "OLDUnsubscribeBasicUsage", - "role": "open", - "phase": "P8", - "mode": "live", - "reason": null - }, - "OLDUnsubscribeMultipleChannelGroup": { - "file": "api-reference/subscribe-legacy.ts", - "step": "OLDUnsubscribeMultipleChannelGroup", - "role": "open", - "phase": "P8", - "mode": "live", - "reason": null - }, - "OLDUnsubscribeMultipleChannels": { - "file": "api-reference/subscribe-legacy.ts", - "step": "OLDUnsubscribeMultipleChannels", - "role": "open", - "phase": "P8", - "mode": "live", - "reason": null - }, - "OLDaddListeners": { - "file": "api-reference/subscribe-legacy.ts", - "step": "OLDaddListeners", - "role": "open", - "phase": "P8", - "mode": "live", - "reason": null - }, - "OLDremoveListeners": { - "file": "api-reference/subscribe-legacy.ts", - "step": "OLDremoveListeners", - "role": "open", - "phase": "P8", - "mode": "live", - "reason": null - }, - "OLDsubscribeBasicUsage": { - "file": "api-reference/subscribe-legacy.ts", - "step": "OLDsubscribeBasicUsage", - "role": "open", - "phase": "P8", - "mode": "live", - "reason": null - }, - "OLDsubscribeChannelGroup": { - "file": "api-reference/subscribe-legacy.ts", - "step": "OLDsubscribeChannelGroup", - "role": "open", - "phase": "P8", - "mode": "live", - "reason": null - }, - "OLDsubscribeChannelGroupAndChannels": { - "file": "api-reference/subscribe-legacy.ts", - "step": "OLDsubscribeChannelGroupAndChannels", - "role": "open", - "phase": "P8", - "mode": "live", - "reason": null - }, - "OLDsubscribeChannelGroupWithPresence": { - "file": "api-reference/subscribe-legacy.ts", - "step": "OLDsubscribeChannelGroupWithPresence", - "role": "open", - "phase": "P8", - "mode": "live", - "reason": null - }, - "OLDsubscribeMultipleChannelGroup": { - "file": "api-reference/subscribe-legacy.ts", - "step": "OLDsubscribeMultipleChannelGroup", - "role": "open", - "phase": "P8", - "mode": "live", - "reason": null - }, - "OLDsubscribeMultipleChannels": { - "file": "api-reference/subscribe-legacy.ts", - "step": "OLDsubscribeMultipleChannels", - "role": "open", - "phase": "P8", - "mode": "live", - "reason": null - }, - "OLDsubscribeWithPresence": { - "file": "api-reference/subscribe-legacy.ts", - "step": "OLDsubscribeWithPresence", - "role": "open", - "phase": "P8", - "mode": "live", - "reason": null - }, - "OLDsubscribeWithState": { - "file": "api-reference/subscribe-legacy.ts", - "step": "OLDsubscribeWithState", - "role": "open", - "phase": "P8", - "mode": "live", - "reason": null - }, - "OLDsubscribeWithWildCardChannels": { - "file": "api-reference/subscribe-legacy.ts", - "step": "OLDsubscribeWithWildCardChannels", - "role": "open", - "phase": "P8", - "mode": "live", - "reason": null - }, - "PubNubinitBasicUsage": { - "file": "shared/import-pubnub.ts", - "step": "PubNubinitBasicUsage", - "role": "open", - "phase": "P0", - "mode": "live", - "reason": null - }, - "SubscriptionSetFrom2Sets": { - "file": "api-reference/subscribe.ts", - "step": "SubscriptionSetFrom2Sets", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "accessManagerBasicUsage": { - "file": "api-reference/access-manager.ts", - "step": "accessManagerBasicUsage", - "role": "pam", - "phase": "P1", - "mode": "live", - "reason": null - }, - "addChannelsToGroupBasicUsage": { - "file": "api-reference/channel-groups.ts", - "step": "addChannelsToGroupBasicUsage", - "role": "open", - "phase": "P3", - "mode": "live", - "reason": null - }, - "addDataSyncListener": { - "file": "api-reference/datasync.ts", - "step": "addDataSyncListener", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "addDeciveToChannelBasicUsage": { - "file": "api-reference/mobile-push.ts", - "step": "addDeciveToChannelBasicUsage", - "role": "open", - "phase": "P6", - "mode": "live", - "reason": null - }, - "addMessageActionBasicUsage": { - "file": "api-reference/message-actions.ts", - "step": "addMessageActionBasicUsage", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "basicUsageWithPromises": { - "file": "api-reference/presence.ts", - "step": "basicUsageWithPromises", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "buildNotificationPayloadBasicUsage": { - "file": "api-reference/mobile-push.ts", - "step": "buildNotificationPayloadBasicUsage", - "role": "open", - "phase": "P6", - "mode": "live", - "reason": null - }, - "combineDataSyncSubscriptions": { - "file": "api-reference/datasync.ts", - "step": "combineDataSyncSubscriptions", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "configurationBasicUsage": { - "file": "api-reference/configuration.ts", - "step": "configurationBasicUsage", - "role": "open", - "phase": "P0", - "mode": "live", - "reason": null - }, - "configurationBasicUsageSubscriptionWorkerUrl": { - "file": "quickstarts/browser-shared-worker.ts", - "step": null, - "role": null, - "phase": null, - "mode": "compile-only", - "reason": "browser-only, type checked by tsconfig.browser.json" - }, - "configurationCryptoModule": { - "file": "api-reference/configuration.ts", - "step": "configurationCryptoModule", - "role": "open", - "phase": "P0", - "mode": "live", - "reason": null - }, - "configurationRealOnlyClient": { - "file": "api-reference/configuration.ts", - "step": "configurationRealOnlyClient", - "role": "open", - "phase": "P0", - "mode": "live", - "reason": null - }, - "configurationServerOpertaion": { - "file": "api-reference/configuration.ts", - "step": "configurationServerOpertaion", - "role": "pam", - "phase": "P0", - "mode": "live", - "reason": null - }, - "createChannelBasicUsage": { - "file": "api-reference/subscribe.ts", - "step": "createChannelBasicUsage", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "createChannelGroupBasicUsage": { - "file": "api-reference/subscribe.ts", - "step": "createChannelGroupBasicUsage", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "createChannelMetadataBasicUsage": { - "file": "api-reference/subscribe.ts", - "step": "createChannelMetadataBasicUsage", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "createDataSyncChannelBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "createDataSyncChannelBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "createEntityBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "createEntityBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "createMembershipBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "createMembershipBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "createRelationshipBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "createRelationshipBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "createSubscription": { - "file": "api-reference/subscribe.ts", - "step": "createSubscription", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "createUserBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "createUserBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "createUserMetadataBasicUsage": { - "file": "api-reference/subscribe.ts", - "step": "createUserMetadataBasicUsage", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "dataSyncEntitySubscribeExample": { - "file": "api-reference/datasync.ts", - "step": "dataSyncEntitySubscribeExample", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "dataSyncProjectionSubscriptions": { - "file": "api-reference/datasync.ts", - "step": "dataSyncProjectionSubscriptions", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "decryptBasicUsage": { - "file": "api-reference/encryption-legacy.ts", - "step": "decryptBasicUsage", - "role": "open", - "phase": "P8", - "mode": "live", - "reason": null - }, - "decryptFileBasicUsage": { - "file": "api-reference/encryption.ts", - "step": "decryptFileBasicUsage", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "deleteChannelGroupBasicUsage": { - "file": "api-reference/channel-groups.ts", - "step": "deleteChannelGroupBasicUsage", - "role": "open", - "phase": "P9", - "mode": "live", - "reason": null - }, - "deleteFileBasicUsage": { - "file": "api-reference/file-sharing.ts", - "step": "deleteFileBasicUsage", - "role": "open", - "phase": "P9", - "mode": "live", - "reason": null - }, - "deleteMessagesBasicUsage": { - "file": "api-reference/message-persistence.ts", - "step": "deleteMessagesBasicUsage", - "role": "open", - "phase": "P5", - "mode": "live", - "reason": null - }, - "deleteProxy": { - "file": "api-reference/misc.ts", - "step": "deleteProxy", - "role": "open", - "phase": "P7", - "mode": "live", - "reason": null - }, - "deleteSpecificMessages": { - "file": "api-reference/message-persistence.ts", - "step": "deleteSpecificMessages", - "role": "open", - "phase": "P5", - "mode": "live", - "reason": null - }, - "destroyBasicUsage": { - "file": "api-reference/configuration.ts", - "step": "destroyBasicUsage", - "role": "open", - "phase": "P9", - "mode": "live", - "reason": null - }, - "disconnectBasicUsage": { - "file": "api-reference/misc.ts", - "step": "disconnectBasicUsage", - "role": "open", - "phase": "P7", - "mode": "live", - "reason": null - }, - "downloadFileCustomCipherKey": { - "file": "api-reference/file-sharing-legacy.ts", - "step": "downloadFileCustomCipherKey", - "role": "open", - "phase": "P8", - "mode": "live", - "reason": null - }, - "downloadFileNodeBasicUsage": { - "file": "api-reference/file-sharing.ts", - "step": "downloadFileNodeBasicUsage", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "downloadFileReactNativeBasicUsage": { - "file": "quickstarts/react-native-download-file.ts", - "step": null, - "role": null, - "phase": null, - "mode": "stub", - "reason": "react or react-native source, kept for the docs URLs, cannot be run in this repository" - }, - "downloadFileWebBasicUsage": { - "file": "quickstarts/browser-download-file.ts", - "step": null, - "role": null, - "phase": null, - "mode": "compile-only", - "reason": "browser-only, type checked by tsconfig.browser.json" - }, - "encryptFileBasicUsage": { - "file": "api-reference/encryption.ts", - "step": "encryptFileBasicUsage", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "encryptMessageBasicUsage": { - "file": "api-reference/encryption.ts", - "step": "encryptMessageBasicUsage", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "eventListenerAddConnectionStatusListenersBasicUsage": { - "file": "api-reference/subscribe.ts", - "step": "eventListenerAddConnectionStatusListenersBasicUsage", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "eventListenerAddListeners": { - "file": "api-reference/subscribe.ts", - "step": "eventListenerAddListeners", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "eventListenerBasicUsage": { - "file": "api-reference/subscribe.ts", - "step": "eventListenerBasicUsage", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "fetchMessagesBasicUsage": { - "file": "api-reference/message-persistence.ts", - "step": "fetchMessagesBasicUsage", - "role": "open", - "phase": "P5", - "mode": "live", - "reason": null - }, - "fetchMessagesWithMetaActions": { - "file": "api-reference/message-persistence.ts", - "step": "fetchMessagesWithMetaActions", - "role": "open", - "phase": "P5", - "mode": "live", - "reason": null - }, - "fireBasicUsage": { - "file": "api-reference/publish.ts", - "step": "fireBasicUsage", - "role": "open", - "phase": "P8", - "mode": "live", - "reason": null - }, - "getAllChannelMetadataBasicUsage": { - "file": "api-reference/objects.ts", - "step": "getAllChannelMetadataBasicUsage", - "role": "open", - "phase": "P3", - "mode": "live", - "reason": null - }, - "getAllUUIDMetadataBasicUsage": { - "file": "api-reference/objects.ts", - "step": "getAllUUIDMetadataBasicUsage", - "role": "open", - "phase": "P3", - "mode": "live", - "reason": null - }, - "getAuthKeyBasicUsage": { - "file": "api-reference/configuration.ts", - "step": "getAuthKeyBasicUsage", - "role": "open", - "phase": "P0", - "mode": "live", - "reason": null - }, - "getChannelBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "getChannelBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "getChannelMembersBasicUsage": { - "file": "api-reference/objects.ts", - "step": "getChannelMembersBasicUsage", - "role": "open", - "phase": "P3", - "mode": "live", - "reason": null - }, - "getChannelMetadataBasicUsage": { - "file": "api-reference/objects.ts", - "step": "getChannelMetadataBasicUsage", - "role": "open", - "phase": "P3", - "mode": "live", - "reason": null - }, - "getChannelsBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "getChannelsBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "getChannelsCursorPagingExample": { - "file": "api-reference/datasync.ts", - "step": "getChannelsCursorPagingExample", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "getChannelsFilterExample": { - "file": "api-reference/datasync.ts", - "step": "getChannelsFilterExample", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "getChannelsFilterFastExample": { - "file": "api-reference/datasync.ts", - "step": "getChannelsFilterFastExample", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "getDataSyncMembershipBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "getDataSyncMembershipBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "getEntitiesBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "getEntitiesBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "getEntitiesCursorPagingExample": { - "file": "api-reference/datasync.ts", - "step": "getEntitiesCursorPagingExample", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "getEntitiesFilterExample": { - "file": "api-reference/datasync.ts", - "step": "getEntitiesFilterExample", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "getEntitiesFilterFastExample": { - "file": "api-reference/datasync.ts", - "step": "getEntitiesFilterFastExample", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "getEntityBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "getEntityBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "getFileUrlBasicUsage": { - "file": "api-reference/file-sharing.ts", - "step": "getFileUrlBasicUsage", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "getFilterExpressionBasicUsage": { - "file": "api-reference/configuration.ts", - "step": "getFilterExpressionBasicUsage", - "role": "open", - "phase": "P0", - "mode": "live", - "reason": null - }, - "getMembershipBasicUsage": { - "file": "api-reference/objects.ts", - "step": "getMembershipBasicUsage", - "role": "open", - "phase": "P3", - "mode": "live", - "reason": null - }, - "getMembershipsBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "getMembershipsBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "getMembershipsByChannelIdExample": { - "file": "api-reference/datasync.ts", - "step": "getMembershipsByChannelIdExample", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "getMembershipsCursorPagingExample": { - "file": "api-reference/datasync.ts", - "step": "getMembershipsCursorPagingExample", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "getMembershipsFilterExample": { - "file": "api-reference/datasync.ts", - "step": "getMembershipsFilterExample", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "getMembershipsFilterFastExample": { - "file": "api-reference/datasync.ts", - "step": "getMembershipsFilterFastExample", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "getMessageActionsBasicUsage": { - "file": "api-reference/message-actions.ts", - "step": "getMessageActionsBasicUsage", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "getRelationshipBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "getRelationshipBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "getRelationshipsBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "getRelationshipsBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "getRelationshipsByEntityBIdExample": { - "file": "api-reference/datasync.ts", - "step": "getRelationshipsByEntityBIdExample", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "getRelationshipsCursorPagingExample": { - "file": "api-reference/datasync.ts", - "step": "getRelationshipsCursorPagingExample", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "getRelationshipsFilterExample": { - "file": "api-reference/datasync.ts", - "step": "getRelationshipsFilterExample", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "getRelationshipsFilterFastExample": { - "file": "api-reference/datasync.ts", - "step": "getRelationshipsFilterFastExample", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "getStateBasicUsage": { - "file": "api-reference/presence.ts", - "step": "getStateBasicUsage", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "getUUIDMetadataBasicUsage": { - "file": "api-reference/objects.ts", - "step": "getUUIDMetadataBasicUsage", - "role": "open", - "phase": "P3", - "mode": "live", - "reason": null - }, - "getUserBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "getUserBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "getUserIdBasicUsage": { - "file": "api-reference/configuration.ts", - "step": "getUserIdBasicUsage", - "role": "open", - "phase": "P0", - "mode": "live", - "reason": null - }, - "getUsersBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "getUsersBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "getUsersCursorPagingExample": { - "file": "api-reference/datasync.ts", - "step": "getUsersCursorPagingExample", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "getUsersFilterExample": { - "file": "api-reference/datasync.ts", - "step": "getUsersFilterExample", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "getUsersFilterFastExample": { - "file": "api-reference/datasync.ts", - "step": "getUsersFilterFastExample", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "gettingStartedCreateSubscription": { - "file": "getting-started/quickstart-node.ts", - "step": "gettingStartedInitPubnub", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "gettingStartedInitPubnub": { - "file": "getting-started/quickstart-node.ts", - "step": "gettingStartedInitPubnub", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "grantAllChannelsAllAuthKeys": { - "file": "api-reference/access-manager-v2.ts", - "step": "grantAllChannelsAllAuthKeys", - "role": "pam", - "phase": "P9", - "mode": "live", - "reason": null - }, - "grantBasicUsage": { - "file": "api-reference/access-manager-v2.ts", - "step": "grantBasicUsage", - "role": "pam", - "phase": "P1", - "mode": "live", - "reason": null - }, - "grantChannelAllAuthKeys": { - "file": "api-reference/access-manager-v2.ts", - "step": "grantChannelAllAuthKeys", - "role": "pam", - "phase": "P1", - "mode": "live", - "reason": null - }, - "grantChannelOnAuthKey": { - "file": "api-reference/access-manager-v2.ts", - "step": "grantChannelOnAuthKey", - "role": "pam", - "phase": "P1", - "mode": "live", - "reason": null - }, - "grantPresenceChannelOnAuthKey": { - "file": "api-reference/access-manager-v2.ts", - "step": "grantPresenceChannelOnAuthKey", - "role": "pam", - "phase": "P1", - "mode": "live", - "reason": null - }, - "grantTokenDataSyncResources": { - "file": "api-reference/access-manager.ts", - "step": "grantTokenDataSyncResources", - "role": "datasync", - "phase": "P1", - "mode": "live", - "reason": null - }, - "grantTokenRegExAndResources": { - "file": "api-reference/access-manager.ts", - "step": "grantTokenRegExAndResources", - "role": "pam", - "phase": "P1", - "mode": "live", - "reason": null - }, - "grantTokenSpacesBasicUsage": { - "file": "api-reference/access-manager-legacy.ts", - "step": "grantTokenSpacesBasicUsage", - "role": "pam", - "phase": "P8", - "mode": "live", - "reason": null - }, - "grantTokenSpacesRegEx": { - "file": "api-reference/access-manager-legacy.ts", - "step": "grantTokenSpacesRegEx", - "role": "pam", - "phase": "P8", - "mode": "live", - "reason": null - }, - "grantTokenSpacesRegExAndResources": { - "file": "api-reference/access-manager-legacy.ts", - "step": "grantTokenSpacesRegExAndResources", - "role": "pam", - "phase": "P8", - "mode": "live", - "reason": null - }, - "grantTokenSpacesVariousResources": { - "file": "api-reference/access-manager-legacy.ts", - "step": "grantTokenSpacesVariousResources", - "role": "pam", - "phase": "P8", - "mode": "live", - "reason": null - }, - "grantTokenUsingRegEx": { - "file": "api-reference/access-manager.ts", - "step": "grantTokenUsingRegEx", - "role": "pam", - "phase": "P1", - "mode": "live", - "reason": null - }, - "grantTokenVariousResources": { - "file": "api-reference/access-manager.ts", - "step": "grantTokenVariousResources", - "role": "pam", - "phase": "P1", - "mode": "live", - "reason": null - }, - "grantTokenWithCategories": { - "file": "api-reference/access-manager.ts", - "step": "grantTokenWithCategories", - "role": "pam", - "phase": "P1", - "mode": "live", - "reason": null - }, - "grantTokenWithProjection": { - "file": "api-reference/access-manager.ts", - "step": "grantTokenWithProjection", - "role": "datasync", - "phase": "P1", - "mode": "live", - "reason": null - }, - "grantUuidOnAuthKey": { - "file": "api-reference/access-manager-v2.ts", - "step": "grantUuidOnAuthKey", - "role": "pam", - "phase": "P1", - "mode": "live", - "reason": null - }, - "grantWithCategories": { - "file": "api-reference/access-manager-v2.ts", - "step": "grantWithCategories", - "role": "pam", - "phase": "P1", - "mode": "live", - "reason": null - }, - "hereNowBasicUsage": { - "file": "api-reference/presence.ts", - "step": "hereNowBasicUsage", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "hereNowChannelGroup": { - "file": "api-reference/presence.ts", - "step": "hereNowChannelGroup", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "hereNowFetchOccupancyOnly": { - "file": "api-reference/presence.ts", - "step": "hereNowFetchOccupancyOnly", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "hereNowWithState": { - "file": "api-reference/presence.ts", - "step": "hereNowWithState", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "historyBasicUsage": { - "file": "api-reference/message-persistence-legacy.ts", - "step": "historyBasicUsage", - "role": "open", - "phase": "P8", - "mode": "live", - "reason": null - }, - "historyIncludeMeta": { - "file": "api-reference/message-persistence-legacy.ts", - "step": "historyIncludeMeta", - "role": "open", - "phase": "P8", - "mode": "live", - "reason": null - }, - "historyNewerThanTimetoken": { - "file": "api-reference/message-persistence-legacy.ts", - "step": "historyNewerThanTimetoken", - "role": "open", - "phase": "P8", - "mode": "live", - "reason": null - }, - "historyOldestThree": { - "file": "api-reference/message-persistence-legacy.ts", - "step": "historyOldestThree", - "role": "open", - "phase": "P8", - "mode": "live", - "reason": null - }, - "historyPagingExample": { - "file": "api-reference/message-persistence-legacy.ts", - "step": "historyPagingExample", - "role": "open", - "phase": "P8", - "mode": "live", - "reason": null - }, - "historyUntilTimetoken": { - "file": "api-reference/message-persistence-legacy.ts", - "step": "historyUntilTimetoken", - "role": "open", - "phase": "P8", - "mode": "live", - "reason": null - }, - "historyWithPromises": { - "file": "api-reference/message-persistence-legacy.ts", - "step": "historyWithPromises", - "role": "open", - "phase": "P8", - "mode": "live", - "reason": null - }, - "importFS": { - "file": "shared/import-pubnub.ts", - "step": null, - "role": null, - "phase": null, - "mode": "compile-only", - "reason": "import-only snippet, type checked by tsconfig.live.json" - }, - "importPubNub": { - "file": "shared/import-pubnub.ts", - "step": null, - "role": null, - "phase": null, - "mode": "compile-only", - "reason": "import-only snippet, type checked by tsconfig.live.json" - }, - "iterativelyUpdateExistingMetadata": { - "file": "api-reference/objects.ts", - "step": "iterativelyUpdateExistingMetadata", - "role": "open", - "phase": "P3", - "mode": "live", - "reason": null - }, - "listChannelsForDeviceBasicUsage": { - "file": "api-reference/mobile-push.ts", - "step": "listChannelsForDeviceBasicUsage", - "role": "open", - "phase": "P6", - "mode": "live", - "reason": null - }, - "listChannelsInGroupBasicUsage": { - "file": "api-reference/channel-groups.ts", - "step": "listChannelsInGroupBasicUsage", - "role": "open", - "phase": "P3", - "mode": "live", - "reason": null - }, - "listFilesBasicUsage": { - "file": "api-reference/file-sharing.ts", - "step": "listFilesBasicUsage", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "loggingBasicUsage": { - "file": "getting-started/logging.ts", - "step": "loggingBasicUsage", - "role": "open", - "phase": "P0", - "mode": "live", - "reason": null - }, - "loggingCustomLoggerExample": { - "file": "getting-started/logging.ts", - "step": "loggingCustomLoggerExample", - "role": "open", - "phase": "P0", - "mode": "live", - "reason": null - }, - "loggingFileLogger": { - "file": "getting-started/logging-file-logger.ts", - "step": "loggingFileLogger", - "role": "open", - "phase": "P0", - "mode": "live", - "reason": null - }, - "loggingFileLoggerImports": { - "file": "getting-started/logging-file-logger.ts", - "step": null, - "role": null, - "phase": null, - "mode": "compile-only", - "reason": "import-only snippet, type checked by tsconfig.live.json" - }, - "loggingImportLogger": { - "file": "getting-started/logging.ts", - "step": null, - "role": null, - "phase": null, - "mode": "compile-only", - "reason": "import-only snippet, type checked by tsconfig.live.json" - }, - "loggingImportPubNub": { - "file": "getting-started/logging.ts", - "step": null, - "role": null, - "phase": null, - "mode": "compile-only", - "reason": "import-only snippet, type checked by tsconfig.live.json" - }, - "loggingMultipleLoggers": { - "file": "getting-started/logging.ts", - "step": "loggingMultipleLoggers", - "role": "open", - "phase": "P0", - "mode": "live", - "reason": null - }, - "messageCountBasicUsage": { - "file": "api-reference/message-persistence.ts", - "step": "messageCountBasicUsage", - "role": "open", - "phase": "P5", - "mode": "live", - "reason": null - }, - "messageCountTimetokensForChannels": { - "file": "api-reference/message-persistence.ts", - "step": "messageCountTimetokensForChannels", - "role": "open", - "phase": "P5", - "mode": "live", - "reason": null - }, - "migrationV11LegacySubscriptionManager": { - "file": "migration-guides/v11.ts", - "step": "migrationV11LegacySubscriptionManager", - "role": "open", - "phase": "P0", - "mode": "live", - "reason": null - }, - "migrationV11RestoreFalse": { - "file": "migration-guides/v11.ts", - "step": "migrationV11RestoreFalse", - "role": "open", - "phase": "P0", - "mode": "live", - "reason": null - }, - "migrationV8AfterImport": { - "file": "migration-guides/v8.ts", - "step": null, - "role": null, - "phase": null, - "mode": "compile-only", - "reason": "import-only snippet, type checked by tsconfig.live.json" - }, - "migrationV8AfterParams": { - "file": "migration-guides/v8.ts", - "step": "migrationV8AfterParams", - "role": "open", - "phase": "P0", - "mode": "live", - "reason": null - }, - "observeDataSyncEntitiesByPattern": { - "file": "api-reference/datasync.ts", - "step": "observeDataSyncEntitiesByPattern", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "parseTokenBasicUsage": { - "file": "api-reference/access-manager.ts", - "step": "parseTokenBasicUsage", - "role": "pam", - "phase": "P1", - "mode": "live", - "reason": null - }, - "parseTokenWithCategories": { - "file": "api-reference/access-manager.ts", - "step": "parseTokenWithCategories", - "role": "pam", - "phase": "P1", - "mode": "live", - "reason": null - }, - "patchChannelBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "patchChannelBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "patchEntityBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "patchEntityBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "patchMembershipBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "patchMembershipBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "patchRelationshipBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "patchRelationshipBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "patchUserBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "patchUserBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "publishBasicUsage": { - "file": "api-reference/publish.ts", - "step": "publishBasicUsage", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "publishFalsyMessage": { - "file": "api-reference/publish.ts", - "step": "publishFalsyMessage", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "publishFileMessageBasicUsage": { - "file": "api-reference/file-sharing.ts", - "step": "publishFileMessageBasicUsage", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "publishHandleErrors": { - "file": "api-reference/publish.ts", - "step": "publishHandleErrors", - "role": "pam", - "phase": "P4", - "mode": "live", - "reason": null - }, - "publishMinimal": { - "file": "api-reference/publish.ts", - "step": "publishMinimal", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "publishStoreThePublishedMessagefor10Hours": { - "file": "api-reference/publish.ts", - "step": "publishStoreThePublishedMessagefor10Hours", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "quickstartNodeListeners": { - "file": "getting-started/quickstart-node.ts", - "step": "gettingStartedInitPubnub", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "quickstartNodePublish": { - "file": "getting-started/quickstart-node.ts", - "step": "gettingStartedInitPubnub", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "quickstartReactImports": { - "file": "quickstarts/quickstart-react.tsx", - "step": null, - "role": null, - "phase": null, - "mode": "stub", - "reason": "react or react-native source, kept for the docs URLs, cannot be run in this repository" - }, - "quickstartReactInit": { - "file": "quickstarts/quickstart-react.tsx", - "step": null, - "role": null, - "phase": null, - "mode": "stub", - "reason": "react or react-native source, kept for the docs URLs, cannot be run in this repository" - }, - "quickstartReactNativeCleanup": { - "file": "quickstarts/quickstart-react-native.tsx", - "step": null, - "role": null, - "phase": null, - "mode": "stub", - "reason": "react or react-native source, kept for the docs URLs, cannot be run in this repository" - }, - "quickstartReactNativeImport": { - "file": "quickstarts/quickstart-react-native.tsx", - "step": null, - "role": null, - "phase": null, - "mode": "stub", - "reason": "react or react-native source, kept for the docs URLs, cannot be run in this repository" - }, - "quickstartReactNativeInit": { - "file": "quickstarts/quickstart-react-native.tsx", - "step": null, - "role": null, - "phase": null, - "mode": "stub", - "reason": "react or react-native source, kept for the docs URLs, cannot be run in this repository" - }, - "quickstartReactNativePublish": { - "file": "quickstarts/quickstart-react-native.tsx", - "step": null, - "role": null, - "phase": null, - "mode": "stub", - "reason": "react or react-native source, kept for the docs URLs, cannot be run in this repository" - }, - "quickstartReactNativeSubscription": { - "file": "quickstarts/quickstart-react-native.tsx", - "step": null, - "role": null, - "phase": null, - "mode": "stub", - "reason": "react or react-native source, kept for the docs URLs, cannot be run in this repository" - }, - "quickstartReactPublish": { - "file": "quickstarts/quickstart-react.tsx", - "step": null, - "role": null, - "phase": null, - "mode": "stub", - "reason": "react or react-native source, kept for the docs URLs, cannot be run in this repository" - }, - "quickstartReactSubscribe": { - "file": "quickstarts/quickstart-react.tsx", - "step": null, - "role": null, - "phase": null, - "mode": "stub", - "reason": "react or react-native source, kept for the docs URLs, cannot be run in this repository" - }, - "quickstartWebInit": { - "file": "quickstarts/quickstart-web.ts", - "step": null, - "role": null, - "phase": null, - "mode": "compile-only", - "reason": "browser-only, type checked by tsconfig.browser.json" - }, - "quickstartWebListeners": { - "file": "quickstarts/quickstart-web.ts", - "step": null, - "role": null, - "phase": null, - "mode": "compile-only", - "reason": "browser-only, type checked by tsconfig.browser.json" - }, - "quickstartWebPublish": { - "file": "quickstarts/quickstart-web.ts", - "step": null, - "role": null, - "phase": null, - "mode": "compile-only", - "reason": "browser-only, type checked by tsconfig.browser.json" - }, - "quickstartWebSubscribe": { - "file": "quickstarts/quickstart-web.ts", - "step": null, - "role": null, - "phase": null, - "mode": "compile-only", - "reason": "browser-only, type checked by tsconfig.browser.json" - }, - "reactConfigurationLifecycle": { - "file": "quickstarts/react-configuration.jsx", - "step": null, - "role": null, - "phase": null, - "mode": "stub", - "reason": "react or react-native source, kept for the docs URLs, cannot be run in this repository" - }, - "reactFetchMessages": { - "file": "quickstarts/react-fetch-messages.jsx", - "step": null, - "role": null, - "phase": null, - "mode": "stub", - "reason": "react or react-native source, kept for the docs URLs, cannot be run in this repository" - }, - "reactGetState": { - "file": "quickstarts/react-get-state.jsx", - "step": null, - "role": null, - "phase": null, - "mode": "stub", - "reason": "react or react-native source, kept for the docs URLs, cannot be run in this repository" - }, - "reactHereNow": { - "file": "quickstarts/react-here-now.jsx", - "step": null, - "role": null, - "phase": null, - "mode": "stub", - "reason": "react or react-native source, kept for the docs URLs, cannot be run in this repository" - }, - "reactListeners": { - "file": "quickstarts/react-listeners.jsx", - "step": null, - "role": null, - "phase": null, - "mode": "stub", - "reason": "react or react-native source, kept for the docs URLs, cannot be run in this repository" - }, - "reactPresenceEvents": { - "file": "quickstarts/react-presence-events.jsx", - "step": null, - "role": null, - "phase": null, - "mode": "stub", - "reason": "react or react-native source, kept for the docs URLs, cannot be run in this repository" - }, - "reactSetState": { - "file": "quickstarts/react-set-state.jsx", - "step": null, - "role": null, - "phase": null, - "mode": "stub", - "reason": "react or react-native source, kept for the docs URLs, cannot be run in this repository" - }, - "reactSubscribe": { - "file": "quickstarts/react-subscribe.jsx", - "step": null, - "role": null, - "phase": null, - "mode": "stub", - "reason": "react or react-native source, kept for the docs URLs, cannot be run in this repository" - }, - "reactWhereNow": { - "file": "quickstarts/react-where-now.jsx", - "step": null, - "role": null, - "phase": null, - "mode": "stub", - "reason": "react or react-native source, kept for the docs URLs, cannot be run in this repository" - }, - "reconnectBasicUsage": { - "file": "api-reference/misc.ts", - "step": "reconnectBasicUsage", - "role": "open", - "phase": "P7", - "mode": "live", - "reason": null - }, - "removeAllMobilePushNotificationsBasicUsage": { - "file": "api-reference/mobile-push.ts", - "step": "removeAllMobilePushNotificationsBasicUsage", - "role": "open", - "phase": "P6", - "mode": "live", - "reason": null - }, - "removeChannelBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "removeChannelBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "removeChannelMembersBasicUsage": { - "file": "api-reference/objects.ts", - "step": "removeChannelMembersBasicUsage", - "role": "open", - "phase": "P9", - "mode": "live", - "reason": null - }, - "removeChannelMetadataBasicUsage": { - "file": "api-reference/objects.ts", - "step": "removeChannelMetadataBasicUsage", - "role": "open", - "phase": "P9", - "mode": "live", - "reason": null - }, - "removeChannelsFromGroupBasicUsage": { - "file": "api-reference/channel-groups.ts", - "step": "removeChannelsFromGroupBasicUsage", - "role": "open", - "phase": "P3", - "mode": "live", - "reason": null - }, - "removeDeviceFromChannelBasicUsage": { - "file": "api-reference/mobile-push.ts", - "step": "removeDeviceFromChannelBasicUsage", - "role": "open", - "phase": "P6", - "mode": "live", - "reason": null - }, - "removeEntityBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "removeEntityBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "removeMembershipBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "removeMembershipBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "removeMembershipsBasicUsage": { - "file": "api-reference/objects.ts", - "step": "removeMembershipsBasicUsage", - "role": "open", - "phase": "P9", - "mode": "live", - "reason": null - }, - "removeMessageActionBasicUsage": { - "file": "api-reference/message-actions.ts", - "step": "removeMessageActionBasicUsage", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "removeRelationshipBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "removeRelationshipBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "removeUUIDMetadataBasicUsage": { - "file": "api-reference/objects.ts", - "step": "removeUUIDMetadataBasicUsage", - "role": "open", - "phase": "P9", - "mode": "live", - "reason": null - }, - "removeUserBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "removeUserBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "revokeTokenBasicUsage": { - "file": "api-reference/access-manager.ts", - "step": "revokeTokenBasicUsage", - "role": "pam", - "phase": "P1", - "mode": "live", - "reason": null - }, - "sendFileBasicUsage": { - "file": "api-reference/file-sharing.ts", - "step": "sendFileBasicUsage", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "sendFileCustomCipherKey": { - "file": "api-reference/file-sharing-legacy.ts", - "step": "sendFileCustomCipherKey", - "role": "open", - "phase": "P8", - "mode": "live", - "reason": null - }, - "setAuthKeyBasicUsage": { - "file": "api-reference/configuration.ts", - "step": "setAuthKeyBasicUsage", - "role": "open", - "phase": "P0", - "mode": "live", - "reason": null - }, - "setChannelMembersBasicUsage": { - "file": "api-reference/objects.ts", - "step": "setChannelMembersBasicUsage", - "role": "open", - "phase": "P3", - "mode": "live", - "reason": null - }, - "setChannelMetadataBasicUsage": { - "file": "api-reference/objects.ts", - "step": "setChannelMetadataBasicUsage", - "role": "open", - "phase": "P3", - "mode": "live", - "reason": null - }, - "setFilterExpressionBasicUsage": { - "file": "api-reference/configuration.ts", - "step": "setFilterExpressionBasicUsage", - "role": "open", - "phase": "P0", - "mode": "live", - "reason": null - }, - "setMembershipBasicUsage": { - "file": "api-reference/objects.ts", - "step": "setMembershipBasicUsage", - "role": "open", - "phase": "P3", - "mode": "live", - "reason": null - }, - "setProxyBasicUsage": { - "file": "api-reference/misc.ts", - "step": "setProxyBasicUsage", - "role": "open", - "phase": "P7", - "mode": "live", - "reason": null - }, - "setStateBasicUsage": { - "file": "api-reference/presence.ts", - "step": "setStateBasicUsage", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "setTokenBasicUsage": { - "file": "api-reference/access-manager.ts", - "step": "setTokenBasicUsage", - "role": "pam", - "phase": "P1", - "mode": "live", - "reason": null - }, - "setUUIDMetadataBasicUsage": { - "file": "api-reference/objects.ts", - "step": "setUUIDMetadataBasicUsage", - "role": "open", - "phase": "P3", - "mode": "live", - "reason": null - }, - "setUpTypeScriptImport": { - "file": "getting-started/set-up-your-environment.ts", - "step": null, - "role": null, - "phase": null, - "mode": "compile-only", - "reason": "import-only snippet, type checked by tsconfig.live.json" - }, - "setUpTypeScriptInit": { - "file": "getting-started/set-up-your-environment.ts", - "step": "setUpTypeScriptInit", - "role": "open", - "phase": "P0", - "mode": "live", - "reason": null - }, - "setUserIdBasicUsage": { - "file": "api-reference/configuration.ts", - "step": "setUserIdBasicUsage", - "role": "open", - "phase": "P0", - "mode": "live", - "reason": null - }, - "signalBasicUsage": { - "file": "api-reference/publish.ts", - "step": "signalBasicUsage", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "simpleNotificationPayloadFCMandAPNS": { - "file": "api-reference/mobile-push.ts", - "step": "simpleNotificationPayloadFCMandAPNS", - "role": "open", - "phase": "P6", - "mode": "live", - "reason": null - }, - "simpleNotificationPayloadFCMandAPNSH/2": { - "file": "api-reference/mobile-push.ts", - "step": "simpleNotificationPayloadFCMandAPNSH/2", - "role": "open", - "phase": "P6", - "mode": "live", - "reason": null - }, - "simpleNotificationPayloadFCMandAPNSH/2CustomConfiguration": { - "file": "api-reference/mobile-push.ts", - "step": "simpleNotificationPayloadFCMandAPNSH/2CustomConfiguration", - "role": "open", - "phase": "P6", - "mode": "live", - "reason": null - }, - "statusEventsMigrationAfter": { - "file": "getting-started/status-events.ts", - "step": "statusEventsMigrationAfter", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "statusEventsMigrationBefore": { - "file": "getting-started/status-events.ts", - "step": "statusEventsMigrationBefore", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "subscribeToDataSyncEntity": { - "file": "api-reference/datasync.ts", - "step": "subscribeToDataSyncEntity", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "subscriptionSetFrom2IndividualSubscriptions": { - "file": "api-reference/subscribe.ts", - "step": "subscriptionSetFrom2IndividualSubscriptions", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "subscriptionSetSubscribeBasicUsage": { - "file": "api-reference/subscribe.ts", - "step": "subscriptionSetSubscribeBasicUsage", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "unsubscribeAllBasicUsage": { - "file": "api-reference/subscribe.ts", - "step": "unsubscribeAllBasicUsage", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "unsubscribeBasicUsage": { - "file": "api-reference/subscribe.ts", - "step": "unsubscribeBasicUsage", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - }, - "updateChannelBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "updateChannelBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "updateEntityBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "updateEntityBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "updateEntityCombinedPatchExample": { - "file": "api-reference/datasync.ts", - "step": "updateEntityCombinedPatchExample", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "updateMembershipBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "updateMembershipBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "updateRelationshipBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "updateRelationshipBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "updateUserBasicUsage": { - "file": "api-reference/datasync.ts", - "step": "updateUserBasicUsage", - "role": "datasync", - "phase": "P2", - "mode": "live", - "reason": null - }, - "whereNowBasicUsage": { - "file": "api-reference/presence.ts", - "step": "whereNowBasicUsage", - "role": "open", - "phase": "P4", - "mode": "live", - "reason": null - } - } -} diff --git a/docs-snippets/data-sync.ts b/docs-snippets/data-sync.ts deleted file mode 100644 index 46c4dda15..000000000 --- a/docs-snippets/data-sync.ts +++ /dev/null @@ -1,440 +0,0 @@ -import PubNub, { PubNubError } from '../lib/types'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'myUniqueUserId', -}); - -// snippet.getUsersFilterExample -// `filter` runs over properties declared with filtering mode `full` and is evaluated against -// eventually consistent storage. -try { - const response = await pubnub.dataSync.getUsers({ - filter: 'name LIKE "*Alice*"', - }); - console.log('getUsers (filter) response:', response.data); -} catch (error) { - console.error( - `Get users error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.getUsersFilterFastExample -// `filterFast` is evaluated against strongly consistent storage, so it matches an object you just -// wrote. -try { - const response = await pubnub.dataSync.getUsers({ - filterFast: 'type == "shopper"', - }); - console.log('getUsers (filterFast) response:', response.data); -} catch (error) { - console.error( - `Get users error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.getUsersCursorPagingExample -// Pass no `cursor` on the first call. Take `meta.next_cursor` from the response and pass it back -// as `cursor` on the next call. Stop when `meta.has_next` is `false`. -try { - let cursor: string | undefined; - let hasNext = true; - let page = 0; - - while (hasNext) { - const response = await pubnub.dataSync.getUsers({ - filterFast: 'type == "shopper"', - limit: 20, - cursor, - }); - console.log(`Page ${++page}:`, response.data); - cursor = response.meta?.next_cursor ?? undefined; - hasNext = response.meta?.has_next ?? false; - } -} catch (error) { - console.error( - `Get users error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.getChannelsFilterExample -try { - const response = await pubnub.dataSync.getChannels({ - filter: 'name LIKE "*Sale*"', - }); - console.log('getChannels (filter) response:', response.data); -} catch (error) { - console.error( - `Get channels error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.getChannelsFilterFastExample -try { - const response = await pubnub.dataSync.getChannels({ - filterFast: 'type == "promotion"', - }); - console.log('getChannels (filterFast) response:', response.data); -} catch (error) { - console.error( - `Get channels error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.getChannelsCursorPagingExample -try { - let cursor: string | undefined; - let hasNext = true; - let page = 0; - - while (hasNext) { - const response = await pubnub.dataSync.getChannels({ - filterFast: 'type == "promotion"', - limit: 20, - cursor, - }); - console.log(`Page ${++page}:`, response.data); - cursor = response.meta?.next_cursor ?? undefined; - hasNext = response.meta?.has_next ?? false; - } -} catch (error) { - console.error( - `Get channels error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.getMembershipsByChannelIdExample -// Pass `channelId` instead of `userId` to list the members of a channel rather than a user's -// memberships. -try { - const response = await pubnub.dataSync.getMemberships({ - channelId: 'channel-summer-sale', - limit: 20, - }); - console.log('getMemberships (channelId) response:', response.data); -} catch (error) { - console.error( - `Get memberships error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.getMembershipsFilterExample -try { - const response = await pubnub.dataSync.getMemberships({ - userId: 'user-alice', - filter: 'role LIKE "*mod*"', - }); - console.log('getMemberships (filter) response:', response.data); -} catch (error) { - console.error( - `Get memberships error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.getMembershipsFilterFastExample -try { - const response = await pubnub.dataSync.getMemberships({ - channelId: 'channel-summer-sale', - filterFast: 'role == "viewer"', - }); - console.log('getMemberships (filterFast) response:', response.data); -} catch (error) { - console.error( - `Get memberships error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.getMembershipsCursorPagingExample -try { - let cursor: string | undefined; - let hasNext = true; - let page = 0; - - while (hasNext) { - const response = await pubnub.dataSync.getMemberships({ - userId: 'user-alice', - limit: 20, - cursor, - }); - console.log(`Page ${++page}:`, response.data); - cursor = response.meta?.next_cursor ?? undefined; - hasNext = response.meta?.has_next ?? false; - } -} catch (error) { - console.error( - `Get memberships error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.getEntitiesFilterExample -// Reference a declared property by its `name` (not its `path`), and combine conditions with `&&` -// and `||`. -try { - const response = await pubnub.dataSync.getEntities({ - class: 'product', - filter: 'name LIKE "*sneaker*" && !(status == "discontinued")', - }); - console.log('getEntities (filter) response:', response.data); -} catch (error) { - console.error( - `Get entities error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.getEntitiesFilterFastExample -try { - const response = await pubnub.dataSync.getEntities({ - class: 'product', - filterFast: 'price < 100 && stock > 0', - }); - console.log('getEntities (filterFast) response:', response.data); -} catch (error) { - console.error( - `Get entities error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.getEntitiesCursorPagingExample -try { - let cursor: string | undefined; - let hasNext = true; - let page = 0; - - while (hasNext) { - const response = await pubnub.dataSync.getEntities({ - class: 'product', - filterFast: 'price < 100', - limit: 20, - cursor, - }); - console.log(`Page ${++page}:`, response.data); - cursor = response.meta?.next_cursor ?? undefined; - hasNext = response.meta?.has_next ?? false; - } -} catch (error) { - console.error( - `Get entities error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.updateEntityCombinedPatchExample -// A single `updateEntity` call can mix `add`, `replace`, `remove`, `move`, `copy`, and `test`. Add -// `ifMatchesEtag` (the `eTag` from a prior read) to reject the patch with a 412 if the entity -// changed since you read it, instead of silently overwriting a concurrent change. -try { - const response = await pubnub.dataSync.updateEntity({ - id: 'product-sneaker-42', - test: { '/payload/stock': 8 }, - replace: { '/payload/price': 74.99 }, - add: { '/payload/tags/0': 'clearance' }, - remove: ['/payload/legacy/field'], - move: [{ from: '/payload/legacyName', path: '/payload/displayName' }], - copy: [{ from: '/payload/displayName', path: '/payload/previousName' }], - ifMatchesEtag: 'StUvWxYzAbCdEf', - }); - console.log('updateEntity (combined patch) response:', response.data); -} catch (error) { - console.error( - `Update entity error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.getRelationshipsByEntityBIdExample -// Pass `entityBId` instead of `entityAId` to list relationships where the entity is on the second -// side of the link. -try { - const response = await pubnub.dataSync.getRelationships({ - class: 'ProductOwner', - entityBId: 'product-sneaker-42', - limit: 20, - }); - console.log('getRelationships (entityBId) response:', response.data); -} catch (error) { - console.error( - `Get relationships error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.getRelationshipsFilterExample -try { - const response = await pubnub.dataSync.getRelationships({ - class: 'ProductOwner', - filter: '!(tier == "platinum")', - }); - console.log('getRelationships (filter) response:', response.data); -} catch (error) { - console.error( - `Get relationships error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.getRelationshipsFilterFastExample -try { - const response = await pubnub.dataSync.getRelationships({ - class: 'ProductOwner', - filterFast: 'tier == "gold"', - }); - console.log('getRelationships (filterFast) response:', response.data); -} catch (error) { - console.error( - `Get relationships error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.getRelationshipsCursorPagingExample -try { - let cursor: string | undefined; - let hasNext = true; - let page = 0; - - while (hasNext) { - const response = await pubnub.dataSync.getRelationships({ - class: 'ProductOwner', - entityAId: 'seller-bob', - limit: 20, - cursor, - }); - console.log(`Page ${++page}:`, response.data); - cursor = response.meta?.next_cursor ?? undefined; - hasNext = response.meta?.has_next ?? false; - } -} catch (error) { - console.error( - `Get relationships error: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.dataSyncEntitySubscribeExample -// DataSync objects publish create/update/delete events over a `dataSync` listener. Create a -// DataSync SDK entity for the object, subscribe to it, and attach the listener. -const productSubscription = pubnub.dataSyncEntity('product-sneaker-42').subscription(); -productSubscription.onDataSync = (event) => console.log(event.message.event, event.message.data); -productSubscription.subscribe(); -// snippet.end - -// snippet.subscribeToDataSyncEntity -// Observe one entity. -const entitySubscription = pubnub.dataSyncEntity('product-sneaker-42').subscription(); - -entitySubscription.onDataSync = (event) => { - console.log(event.message.event, event.message.data); -}; - -entitySubscription.subscribe(); -// snippet.end - -// snippet.observeDataSyncEntitiesByPattern -// Because the identifier is used verbatim, a wildcard identifier subscribes to the matching -// channel pattern and one subscription serves the whole set. `event.channel` reports the concrete -// delivery channel, while `event.subscription` reports the pattern that matched it. -const patternSubscription = pubnub.dataSyncEntity('product.*').subscription({ projection: 'admin' }); - -patternSubscription.onDataSync = (event) => { - // event.channel -> '__admin__product.676' - // event.subscription -> '__admin__product.*' - console.log('changed:', event.message.data.id); -}; - -patternSubscription.subscribe(); -// snippet.end - -// snippet.combineDataSyncSubscriptions -// A DataSync subscription is an ordinary `subscription`, so it composes into a set with any other -// subscription. -const productsSubscription = pubnub.dataSyncEntity('product.*').subscription(); -const aliceSubscription = pubnub.dataSyncUser('user-alice').subscription(); - -const dataSyncSubscriptionSet = productsSubscription.addSubscription(aliceSubscription); -dataSyncSubscriptionSet.onDataSync = (event) => console.log(event.message.objectType, event.message.event); -dataSyncSubscriptionSet.subscribe(); -// snippet.end - -// snippet.addDataSyncListener -// Create a DataSync SDK entity for the object and attach the listener. -const dataSyncObjectId = 'product-sneaker-42'; -const dataSyncListenerSubscription = pubnub.dataSyncEntity(dataSyncObjectId).subscription(); - -dataSyncListenerSubscription.onDataSync = (event) => { - const change = event.message; - - // This channel can also receive relationship events and events about connected entities. Only - // handle events for the observed product. `Product` is a class of your own, so its events - // report `type: 'entity'`. - if (change.type !== 'entity' || change.data.id !== dataSyncObjectId) return; - - switch (change.event) { - case 'create': - console.log('Product created:', change.data.id); - break; - case 'update': { - // `payload` is typed as the generic SDK `Payload` union (it can be a string, number, array, - // and so on for other message types), but a DataSync entity payload is always the JSON - // object declared by its class, so it is safe to read it as a record here. - const payload = - 'payload' in change.data ? (change.data.payload as Record | undefined) : undefined; - console.log('New price:', payload?.price); - break; - } - case 'delete': - console.log('Product deleted:', change.data.id); - break; - } -}; - -dataSyncListenerSubscription.subscribe(); -// snippet.end diff --git a/docs-snippets/event-listener.ts b/docs-snippets/event-listener.ts deleted file mode 100644 index 8c1a4c671..000000000 --- a/docs-snippets/event-listener.ts +++ /dev/null @@ -1,95 +0,0 @@ -import PubNub from '../lib/types'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'myUniqueUserId', -}); - -// snippet.eventListenerAddListeners - -// create a subscription from a channel entity -const channel = pubnub.channel('channel_1'); -const subscription = channel.subscription(); - -// Event-specific listeners -subscription.onMessage = (message) => { - console.log('Message event: ', message); -}; -subscription.onPresence = (presence) => { - console.log('Presence event: ', presence); -}; -subscription.onSignal = (signal) => { - console.log('Signal event: ', signal); -}; -subscription.onObjects = (objectsEvent) => { - console.log('Objects event: ', objectsEvent); -}; -subscription.onMessageAction = (messageActionEvent) => { - console.log('Message Reaction event: ', messageActionEvent); -}; -subscription.onFile = (fileEvent) => { - console.log('File event: ', fileEvent); -}; - -// Generic listeners -subscription.addListener({ - // Messages - message: function (m) { - const channelName = m.channel; // Channel on which the message was published - const channelGroup = m.subscription; // Channel group or wildcard subscription match (if exists) - const pubTT = m.timetoken; // Publish timetoken - const msg = m.message; // Message payload - const publisher = m.publisher; // Message publisher - }, - // Presence - // requires a subscription with presence - presence: function (p) { - const action = p.action; // Can be join, leave, timeout, state-change, or interval - const channelName = p.channel; // Channel to which the message belongs - const channelGroup = p.subscription; // Channel group or wildcard subscription match, if any - const publishTime = p.timestamp; // Publish timetoken - const timetoken = p.timetoken; // Current timetoken - }, - // Signals - signal: function (s) { - const channelName = s.channel; // Channel to which the signal belongs - const channelGroup = s.subscription; // Channel group or wildcard subscription match, if any - const pubTT = s.timetoken; // Publish timetoken - const msg = s.message; // Payload - const publisher = s.publisher; // Message publisher - }, - // App Context - objects: (objectEvent) => { - const channel = objectEvent.channel; // channel Id or a User Id of updated/set app context metadata object - const channelGroup = objectEvent.subscription; // Channel group - const timetoken = objectEvent.timetoken; // Event timetoken - const event = objectEvent.message.data.type; // Event name - }, - // Message Actions - messageAction: function (ma) { - const channelName = ma.channel; // Channel to which the message belongs - const publisher = ma.data.uuid; // Message publisher - const event = ma.event; // Message action added or removed - const type = ma.data.type; // Message action type - const value = ma.data.value; // Message action value - const messageTimetoken = ma.data.messageTimetoken; // Timetoken of the original message - const actionTimetoken = ma.data.actionTimetoken; // Timetoken of the message action - }, - // File Sharing - file: function (event) { - const channelName = event.channel; // Channel to which the file belongs - const channelGroup = event.subscription; // Channel group or wildcard subscription match (if exists) - const publisher = event.publisher; // File publisher - const timetoken = event.timetoken; // Event timetoken - const message = event.message; // Optional message attached to the file - }, -}); - -// snippet.end - -// snippet.AddConnectionStatusListener -pubnub.addListener({ - status: (s) => s.category, -}); -// snippet.end diff --git a/docs-snippets/file-sharing.ts b/docs-snippets/file-sharing.ts deleted file mode 100644 index 2b7df6dac..000000000 --- a/docs-snippets/file-sharing.ts +++ /dev/null @@ -1,39 +0,0 @@ -import PubNub, { PubNubError } from '../lib/types'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'myUniqueUserId', -}); - -// snippet.sendFileCustomCipherKey -// in Node.js -import fs from 'fs'; - -try { - const myFile = fs.readFileSync('./cat_picture.jpg'); - - const response = await pubnub.sendFile({ - channel: 'my_channel', - message: 'Look at this picture!', - file: { data: myFile, name: 'cat_picture.jpg', mimeType: 'application/json' }, - cipherKey: 'myCipherKey', - }); - console.log('File sent successfully:', response); -} catch (error) { - console.error( - `Error sending file: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.downloadFileCustomCipherKey -const file = await pubnub.downloadFile({ - channel: 'my_channel', - id: '...', - name: 'cat_picture.jpg', - cipherKey: 'myCipherKey', -}); -// snippet.end diff --git a/docs-snippets/fixtures/datasync-fragments.ts b/docs-snippets/fixtures/datasync-fragments.ts deleted file mode 100644 index 79138f172..000000000 --- a/docs-snippets/fixtures/datasync-fragments.ts +++ /dev/null @@ -1,48 +0,0 @@ -import type PubNub from 'pubnub'; - -/** - * The four fragment blocks of the DataSync page (docs/sdks/javascript/api-reference/datasync.md), one object per docs - * line. The docs show these as bare option lines (`filterFast: 'price < 100'`) inside prose, so they cannot be a - * snippet: several lines repeat the same key and are not statements. They stay hardcoded in the docs and this file - * keeps them honest in two ways. - * - * 1. Types: every entry is typed against the real SDK parameter types, so a renamed option or a `sort` shape the SDK - * no longer accepts breaks `npm run test:docs-snippets:types`. - * 2. Behaviour: the - * live check `datasyncFragments` (api-reference/datasync.ts) sends each entry to the service. - * - * Keep each entry on one line, written exactly like the docs line without its trailing comment. The text check - * relies on that. - */ - -// Sneaker Drop context: `price`, `stock`, `status` and `name` are fields of the provisioned `product` class (provision/). -type Query = PubNub.DataSync.GetEntitiesParameters; -type Patch = PubNub.DataSync.UpdateEntityParameters; - -/** Block "Some examples" (filter and filterFast expressions). */ -export const filterFragments: Array> = [ - { filterFast: 'price < 100' }, - { filterFast: '(price < 100 && stock > 0) || price > 500' }, - { filter: 'name LIKE "*sneaker*"' }, - { filter: '!(status == "discontinued")' }, -]; - -/** The two `sort` blocks: object form first, then the raw string form. */ -export const sortFragments: Array> = [ - { sort: { price: 'desc' } }, - { sort: { type: 'asc', price: 'desc' } }, - { sort: { createdAt: null } }, - { sort: 'price:desc' }, - { sort: 'type,price:desc' }, - { sort: 'createdAt' }, -]; - -/** Block "Correct patch forms" (JSON Patch maps of `update*`). */ -export const patchFragments: Array> = [ - { add: { '/payload/tags/0': 'featured' } }, - { replace: { '/payload/price': 79.99 } }, - { remove: ['/payload/tempFlag', '/payload/legacy/field'] }, - { move: [{ from: '/payload/legacyName', path: '/payload/displayName' }] }, - { copy: [{ from: '/payload/displayName', path: '/payload/previousName' }] }, - { test: { '/status': 'active' } }, -]; diff --git a/docs-snippets/fixtures/files/cat_picture.jpg b/docs-snippets/fixtures/files/cat_picture.jpg deleted file mode 100644 index 6878c904f713ce1d34fc1d712382be106c76ca79..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 134 zcmex=-r<6 base36>`. A name with dots gets the suffix on its - * first dot segment so wildcard roots stay intact (`chats.room1` becomes `chats-rk3x9q.room1`). - * - `placeholders`: literals such as `replace_with_message_timetoken`. Rewritten to `world.ph('')`, so the - * step that produced the real value can supply it at run time. - * - `harnessOnly`: names the harness itself creates (outside snippet markers). They are never rewritten, the - * harness calls `world.n(name)` for them. Listed so the janitor knows what to clean. - */ - -import { createHash } from 'node:crypto'; -import { runIdAgeMs } from '../harness/env.js'; - -/** A realistic 64-character hex device token, the shape of an APNs token. Shown in the push snippets. */ -export const DEVICE_TOKEN_EXAMPLE = '740f4707bebcf74f9b7c25d48e3358945f6aa01da5ddb387462c7eaf61bb78ad'; - -/** Groups the registry by what the name identifies. Only used to organize `names`. Rewriting treats all kinds alike. */ -export type NameKind = 'channel' | 'channelGroup' | 'userId' | 'metadata' | 'device' | 'dataSync'; - -/** The literals the stage rewrite suffixes. Add a literal here when a snippet names a new live resource. */ -export const names: Readonly> = { - channel: [ - 'my_channel', - 'my-channel', - 'ch1', - 'ch2', - 'ch3', - 'ch4', - 'channel_1', - 'channel_3', - 'channel_name', - 'chats.room1', - 'chats.room2', - 'foo', - 'son', - 'team.blue', - 'team.red', - 'channel-with-status-type', - 'ch-1', - 'ch-2', - // Channels of the legacy subscribe, history and quickstart pages. - 'my_channel_1', - 'my_channel_2', - 'my_channel_3', - 'ab.*', - 'chan1', - 'chan2', - 'chan3', - 'history_channel', - 'history_test', - 'hello_world', - 'myChannel', - 'awesomeChannel', - 'chat-bot', - 'a', - 'b', - ], - channelGroup: ['myChannelGroup', 'family', 'cg1', 'my_channel_group', 'channelGroup_1', 'my_channelGroup', 'my_channelGroup1', 'my_channelGroup2', 'my_channelGroup3', 'demo_group1', 'demo_group2'], - userId: ['YOUR_USER_ID', 'myUniqueUserId', 'uuid', 'myUuid', 'my-uuid', 'uuid-1', 'uuid-2', 'uuid-3', 'my-authorized-uuid', 'my-authorized-userId', 'my_authkey', 'key1', 'my-auth-key', 'uuid1', 'user-alice', 'your-user-id', 'unique-user-id', 'myUserId'], - metadata: ['user_meta1'], - /** - * Push device tokens. The janitor removes their registrations. - * - * The literal must be a realistic token that the real service accepts on EVERY gateway. The docs sample uses - * `niceDevice`, which the service rejects for apns and apns2 (400 "Invalid device token", verified live 2026-11-02), - * because it checks the token FORMAT for those gateways: a hex string of 64 characters (40 also passes) is accepted, - * and FCM accepts any string. So the suite's snippet bodies show a 64-character hex token. The staging rewrite - * replaces it with another 64-character hex token derived from the run id (see `deviceToken`), so the rewritten - * value stays valid for apns and apns2. Tokens are not checked against Apple or Google until a push is sent. - */ - device: [DEVICE_TOKEN_EXAMPLE], - /** DataSync object ids on the secured keyset. Ids allow hyphens and dots but no underscores, so the run suffix fits. `user-alice` is a userId above. */ - dataSync: [ - 'channel-summer-sale', - 'membership-alice-summer-sale', - 'product-sneaker-42', - 'seller-bob', - 'rel-bob-owns-sneaker-42', - 'product.*', - ], -}; - -/** Literal found in a snippet, mapped to the `world.ph()` key that supplies the live value. */ -export const placeholders: Readonly> = { - replace_with_message_timetoken: 'messageTimetoken', - replace_with_action_timetoken: 'actionTimetoken', - replace_with_start_timetoken: 'actionsStart', - replace_with_end_timetoken: 'actionsEnd', - 'replace-with-start-timetoken': 'historyStart', - 'replace-with-end-timetoken': 'historyEnd', - 'replace-with-timetoken': 'countFrom', - // history (legacy): the docs use these literals as start and end anchors. - '13406746780720711': 'historyAnchor', - '123123123133': 'historyPromiseStart', - '123123123123': 'historyPromiseEnd', - 'replace-with-channel-timetoken-ch1': 'countFromCh1', - 'replace-with-channel-timetoken-ch2': 'countFromCh2', - 'replace-with-channel-timetoken-ch3': 'countFromCh3', - // File id and the image id of the React Native sample, both supplied by the step that sends the file. - '...': 'fileId', - imageId: 'imageId', - // Access Manager: a token the step granted, a token to revoke, the presence channel of my_channel. - 'use-token-string-generated-by-grantToken()': 'grantedToken', - p0AkFl043rhDdHRsple3KgQ3NwY6BDcENnctokenVzcqBDczaWdYIGOAeTyWGJI: 'tokenToRevoke', - 'my_channel-pnpres': 'myChannelPnpres', - // DataSync: eTags the docs show as examples. A step reads the real eTag first and sets these. - AbQdEfGhIjKlMn: 'userEtag', - StUvWxYzAbCdEf: 'entityEtag', -}; - -/** Names the harness creates itself. Used through `world.n()`, cleaned by the janitor. */ -export const harnessOnly: Readonly> = { - channel: ['cg1_member', 'channelGroup_1_member', 'daughter', 'filter_channel', 'persist_channel', 'preflight_channel', 'legacy_listeners_channel', 'am_room', 'am_other', 'push_marker'], - channelGroup: [], - userId: ['shopper', 'friend', 'observer', 'user-alice', 'user-bob', 'staff-server', 'am-revokee', 'am-v2-user', 'am-v2-reader', 'am-denied'], -}; - -/** `RUN_ID_PATTERN` validates a bare run id. `SUFFIX_PATTERN` finds the `-r` suffix inside a resource name (at the end or before a dot). */ -export const RUN_ID_PATTERN = /^[0-9a-z]{6}$/; -export const SUFFIX_PATTERN = /-r[0-9a-z]{6}(?=\.|$)/; - -/** - * True only when a name carries a run suffix that decodes to a run of the last year. The pattern alone also matches ordinary - * words (`test-results`, `app-reviews`), so the janitors use this before they delete anything on a shared keyset. - */ -export function hasRunSuffix(name: string): boolean { - const match = SUFFIX_PATTERN.exec(name); - return Boolean(match && runIdAgeMs(match[0].slice(2)) !== undefined); -} - -/** - * Per-run device token of the same length as `literal` and still valid hex (lowercase 0-9a-f). Derived from - * SHA-256 of the literal and the run id, so it is deterministic for one run (the janitor recomputes it for a crashed - * run from the run id alone) and different between runs. Plain "-r" suffixes would not be hex, and the - * service rejects non-hex tokens on apns and apns2. - */ -export function deviceToken(literal: string, runId: string): string { - let hex = ''; - for (let round = 0; hex.length < literal.length; round += 1) { - hex += createHash('sha256').update(`${literal}:${runId}:${round}`).digest('hex'); - } - return hex.slice(0, literal.length); -} - -/** - * Applies the run suffix: on the first dot segment when the name has dots, at the end otherwise. - * Device tokens are the exception: they get a derived hex token instead (see `deviceToken`). - */ -export function suffixName(name: string, runId: string): string { - if (names.device.includes(name)) return deviceToken(name, runId); - const suffix = `-r${runId}`; - const dot = name.indexOf('.'); - return dot === -1 ? `${name}${suffix}` : `${name.slice(0, dot)}${suffix}${name.slice(dot)}`; -} - -/** Every rewritable live-resource literal. */ -/** Used by the stage report (unused names) and by `isRegisteredName`. */ -export function allRewritableNames(): string[] { - return Object.values(names).flat(); -} - -/** Channel group names the janitor deletes per run id. */ -export function allGroupNames(): string[] { - return [...names.channelGroup, ...harnessOnly.channelGroup]; -} - -export function isRegisteredName(literal: string): boolean { - return allRewritableNames().includes(literal); -} - -export function isPlaceholder(literal: string): boolean { - return Object.prototype.hasOwnProperty.call(placeholders, literal); -} - -/** Literal that looks like a placeholder (so a missing registry entry is caught by the marker lint). */ -export const PLACEHOLDER_LOOKALIKE = /replace[-_]with/i; diff --git a/docs-snippets/fixtures/workdir.ts b/docs-snippets/fixtures/workdir.ts deleted file mode 100644 index 3dd20ee9f..000000000 --- a/docs-snippets/fixtures/workdir.ts +++ /dev/null @@ -1,32 +0,0 @@ -import fs from 'node:fs'; -import os from 'node:os'; -import path from 'node:path'; -import { fileURLToPath } from 'node:url'; - -/** - * Scratch directory for the file sharing samples. They read and write `./cat_picture.jpg` relative to the - * working directory, and `downloadFileNodeBasicUsage` overwrites it. The step changes into a temp directory that - * holds a copy of the fixture, so the tracked fixture stays untouched. - */ -const HERE = path.dirname(fileURLToPath(import.meta.url)); -export const FIXTURE_FILE = path.join(HERE, 'files', 'cat_picture.jpg'); - -export interface Workdir { - dir: string; - restore(): void; -} - -/** Copies the fixture to a temp dir and `chdir`s into it. The caller must `restore()` (in a `finally`), it is process-wide state. */ -export function enterWorkdir(): Workdir { - const previous = process.cwd(); - const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'pn-docs-files-')); - fs.copyFileSync(FIXTURE_FILE, path.join(dir, 'cat_picture.jpg')); - process.chdir(dir); - return { - dir, - restore() { - process.chdir(previous); - fs.rmSync(dir, { recursive: true, force: true }); - }, - }; -} diff --git a/docs-snippets/getting-started-example.ts b/docs-snippets/getting-started-example.ts deleted file mode 100644 index 4aef88cd5..000000000 --- a/docs-snippets/getting-started-example.ts +++ /dev/null @@ -1,102 +0,0 @@ -import PubNub, { PubNubError } from '../lib/types'; - -// snippet.gettingStartedCompleteExample -// Save this file as index.js/.ts - -// 1. Import pubnub -// import PubNub from 'pubnub'; - -// 2. Initialize PubNub with demo keys and a random user ID -const pubnub = new PubNub({ - publishKey: 'YOUR_PUBLISH_KEY', - subscribeKey: 'YOUR_SUBSCRIBE_KEY', - userId: 'YOUR_USER_ID', -}); - -// 3. Add listener to handle messages, presence events, and connection status -pubnub.addListener({ - // Handle incoming messages - message: function (event) { - // Handle message event - console.log('New message:', event.message); - // Format and display received message - const displayText = (() => { - if (typeof event.message === 'object' && event.message && 'text' in event.message) { - const messageObj = event.message as { text?: string; sender?: string }; - return `${messageObj.sender || 'User'}: ${messageObj.text}`; - } else { - return `Message: ${JSON.stringify(event.message)}`; - } - })(); - console.log(displayText); - }, - - // Handle presence events (join, leave, timeout) - presence: function (event) { - // Handle presence event - console.log('Presence event:', event); - console.log('Action:', event.action); // join, leave, timeout - console.log('Channel:', event.channel); - }, - - // Handle connection status events - status: function (event) { - // Handle status event - if (event.category === 'PNConnectedCategory') { - console.log('Connected to PubNub chat!'); - console.log('Your user ID is:', pubnub.userId); - } else if (event.category === 'PNNetworkIssuesCategory') { - // if eventEngine is not enabled, this event will be triggered when subscription encounter network issues. - console.log('Connection lost'); - // handle reconnection - } else if (event.category === 'PNDisconnectedUnexpectedlyCategory') { - // If enableEventEngine: true set in the constructor, this event will be triggered when the connection is lost. - console.log('Disconnected unexpectedly.'); - } - }, -}); - -// 4. Create a channel entity and subscription with presence -const channel = pubnub.channel('hello_world'); -const subscription = channel.subscription({ - receivePresenceEvents: true, // to receive presence events -}); - -// 5. Subscribe to the channel -subscription.subscribe(); - -// 6. Function to publish messages -async function publishMessage(text: string) { - if (!text.trim()) return; - - try { - // Create a message object with text, timestamp, and sender ID - const message = { - text: text, - time: new Date().toISOString(), - sender: pubnub.userId, - }; - - // Publish the message to the channel - const response = await pubnub.publish({ - message: message, - channel: 'hello_world', - storeInHistory: true, // Save this message in history - }); - - // Success message (timetoken is the unique ID for this message) - console.log(`\n✅ Message sent successfully!`); - } catch (error) { - // Handle publish errors - console.error( - `\n❌ Failed to send message: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); - } -} - -// 7. define the message and Publish that message. -const text_message = 'Hello, world!'; -publishMessage(text_message); -// snippet.end diff --git a/docs-snippets/getting-started.ts b/docs-snippets/getting-started.ts deleted file mode 100644 index 29d17a19c..000000000 --- a/docs-snippets/getting-started.ts +++ /dev/null @@ -1,93 +0,0 @@ -import PubNub, { PubNubError, Subscription } from '../lib/types'; - -// snippet.gettingStartedInitPubnub -const pubnub = new PubNub({ - publishKey: 'YOUR_PUBLISH_KEY', - subscribeKey: 'YOUR_SUBSCRIBE_KEY', - userId: 'YOUR_USER_ID', -}); -// snippet.end - -// snippet.gettingStartedEventListeners -// Add listener to handle messages, presence events, and connection status -pubnub.addListener({ - message: function (event: Subscription.Message) { - // Handle message event - console.log('New message:', event.message); - // Format and display received message - const displayText = (() => { - if (typeof event.message === 'object' && event.message && 'text' in event.message) { - const messageObj = event.message as { text?: string; sender?: string }; - return `${messageObj.sender || 'User'}: ${messageObj.text}`; - } else { - return `Message: ${JSON.stringify(event.message)}`; - } - })(); - console.log(displayText); - }, - presence: function (event: Subscription.Presence) { - // Handle presence event - console.log('Presence event:', event); - console.log('Action:', event.action); // join, leave, timeout - console.log('Channel:', event.channel); - }, - status: function (event) { - // Handle status event - if (event.category === 'PNConnectedCategory') { - console.log('Connected to PubNub chat!'); - console.log('Your user ID is:', pubnub.userId); - } else if (event.category === 'PNNetworkIssuesCategory') { - // if eventEngine is not enabled, this event will be triggered when subscription encounter network issues. - console.log('Connection lost'); - // handle reconnection - } else if (event.category === 'PNDisconnectedUnexpectedlyCategory') { - // If enableEventEngine: true set in the constructor, this event will be triggered when the connection is lost. - console.log('Disconnected unexpectedly.'); - } - }, -}); -// snippet.end - -// snippet.gettingStartedCreateSubscription -// Create a channel entity -const channel = pubnub.channel('hello_world'); - -// Create a subscription -const subscription = channel.subscription({ - receivePresenceEvents: true, // to receive presence events -}); - -// Subscribe -subscription.subscribe(); -// snippet.end - -// snippet.gettingStartedPublishMessages -// Function to publish a message -async function publishMessage(text: string) { - if (!text.trim()) return; - - try { - const result = await pubnub.publish({ - message: { - text: text, - sender: pubnub.userId, - time: new Date().toISOString(), - }, - channel: 'hello_world', - }); - console.log(`Message published with timetoken: ${result.timetoken}`); - console.log(`You: ${text}`); - } catch (error) { - console.error( - `Publish failed: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); - } -} - -// Example: publish a message -const text_message = 'Hello, world!'; -publishMessage(text_message); - -// snippet.end diff --git a/docs-snippets/getting-started/logging-file-logger.ts b/docs-snippets/getting-started/logging-file-logger.ts deleted file mode 100644 index 5df6728cf..000000000 --- a/docs-snippets/getting-started/logging-file-logger.ts +++ /dev/null @@ -1,92 +0,0 @@ -// snippet.loggingFileLoggerImports -import PubNub, { Logger, LogMessage } from 'pubnub' -import * as fs from 'fs'; - -// snippet.end -import assert from '../harness/assert.js'; -import os from 'node:os'; -import path from 'node:path'; -import { page, step } from '../harness/step.js'; -import { eventually } from '../harness/wait.js'; - -/** - * File logger example for Node.js. It reads the level name from `PubNub.LogLevel` because the SDK is a CommonJS - * package. `Logger` and `LogMessage` are types, so they work as named imports. - */ -export default page('logging', 'P0', (world) => { - // Changes into a temp directory because the sample writes `./pubnub.log`. The previous directory is restored in `finally`. - // The file is read back with `eventually` because the write stream flushes asynchronously. - step('loggingFileLogger', { proves: 'A file logger writes the SDK log lines to the log file.' }, async () => { - const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'pn-logging-')); - const previous = process.cwd(); - process.chdir(dir); - const quiet = world.captureConsole('log'); - try { - // snippet.loggingFileLogger - class FileLogger implements Logger { - private readonly stream: fs.WriteStream; - - constructor(private readonly logFilePath: string) { - this.stream = fs.createWriteStream(logFilePath, { flags: 'a' }); - } - - log(logMessage: LogMessage) { - const level = PubNub.LogLevel[logMessage.level]; - const timestamp = logMessage.timestamp.toISOString(); - const location = logMessage.location || 'Unknown'; - const message = this.formatMessage(logMessage); - - this.stream.write( - `${timestamp} [${level.toUpperCase()}] ${location}: ${message}\n` - ); - } - - formatMessage(logMessage: LogMessage) { - if (logMessage.messageType === 'text') { - return logMessage.message; - } else if (logMessage.messageType === 'object') { - return `${logMessage.details || ''}\n${JSON.stringify(logMessage.message, null, 2)}`; - } else if (logMessage.messageType === 'error') { - return `${logMessage.message.name}: ${logMessage.message.message}`; - } - // Handle other message types - return JSON.stringify(logMessage.message); - } - - trace(logMessage: LogMessage) { this.log(logMessage); } - debug(logMessage: LogMessage) { this.log(logMessage); } - info(logMessage: LogMessage) { this.log(logMessage); } - warn(logMessage: LogMessage) { this.log(logMessage); } - error(logMessage: LogMessage) { this.log(logMessage); } - } - - // Use FileLogger with PubNub - const pubnub = new PubNub({ - subscribeKey: 'mySubscribeKey', - publishKey: 'myPublishKey', - userId: 'myUniqueUserId', - logLevel: PubNub.LogLevel.Debug, - // snippet.hide - ...world.live.openKeys, - // snippet.show - loggers: [new FileLogger('./pubnub.log')] - }); - // snippet.end - world.track(pubnub); - world.check.serverTime((await pubnub.time()).timetoken); - const content = await eventually(() => { - const text = fs.readFileSync(path.join(dir, 'pubnub.log'), 'utf8'); - assert.match(text, /\[DEBUG\]/, 'debug entries are in the file'); - return text; - }, { label: 'file has log entries' }); - for (const line of content.split('\n').filter((l) => /^\d{4}-\d{2}-\d{2}T/.test(l))) { - assert.match(line, /^\d{4}-\d{2}-\d{2}T[\d:.]+Z \[(TRACE|DEBUG|INFO|WARN|ERROR)\] \S+: /); - } - assert.ok(/network|GET|\/time\//i.test(content), 'the time() request appears in the log'); - } finally { - quiet.restore(); - process.chdir(previous); - fs.rmSync(dir, { recursive: true, force: true }); - } - }); -}); diff --git a/docs-snippets/getting-started/logging.ts b/docs-snippets/getting-started/logging.ts deleted file mode 100644 index a49c2524c..000000000 --- a/docs-snippets/getting-started/logging.ts +++ /dev/null @@ -1,375 +0,0 @@ -// snippet.loggingImportLogger -import { Logger, LogMessage } from 'pubnub' - -// snippet.end -import assert from '../harness/assert.js'; -import fs from 'node:fs'; -import os from 'node:os'; -import path from 'node:path'; -// snippet.loggingImportPubNub -import PubNub from 'pubnub'; - -// snippet.end -import { page, step, check } from '../harness/step.js'; -import { eventually } from '../harness/wait.js'; - -/** - * Logging examples. - */ -const LogLevel = PubNub.LogLevel; - -export default page('logging', 'P0', (world) => { - /** Records what a logger receives, per level and per message type. */ - const recorder = () => { - const seen: Array<{ level: string; message: LogMessage }> = []; - const logger: Logger = { - trace: (m) => seen.push({ level: 'trace', message: m }), - debug: (m) => seen.push({ level: 'debug', message: m }), - info: (m) => seen.push({ level: 'info', message: m }), - warn: (m) => seen.push({ level: 'warn', message: m }), - error: (m) => seen.push({ level: 'error', message: m }), - }; - return { seen, logger }; - }; - - // Debug level makes the SDK's console logger print its configuration, which holds the keys. The capture swallows that output - // while the client is built and is restored right after. redact.cjs masks keys in the run log as a second line of defense. - step('loggingBasicUsage', { proves: 'A client built with Debug level logging works, and its logger output is captured.' }, async () => { - const quiet = world.captureConsole('info'); - // snippet.loggingBasicUsage - // Initialize PubNub with debug-level logging - const pubnub = new PubNub({ - subscribeKey: 'mySubscribeKey', - publishKey: 'myPublishKey', - userId: 'myUniqueUserId', - // snippet.hide - ...world.live.openKeys, - // snippet.show - logLevel: PubNub.LogLevel.Debug - }); - // snippet.end - world.track(pubnub); - quiet.restore(); - assert.equal(pubnub.getUserId(), world.n('myUniqueUserId')); - assert.ok(quiet.lines.length >= 0); - }); - - step('loggingCustomLoggerExample', { proves: 'A custom logger receives the error entry and the sample reports it to the stand-in services.' }, async () => { - // "The example assumes that monitoringService and analyticsService are objects in your application." - const reported = { errors: [] as any[], failures: [] as any[], deprecations: [] as any[], configs: [] as any[] }; - const monitoringService = { - reportError: (payload: any) => reported.errors.push(payload), - reportNetworkFailure: (payload: any) => reported.failures.push(payload), - }; - const analyticsService = { - trackDeprecation: (payload: any) => reported.deprecations.push(payload), - trackConfigChange: (payload: any) => reported.configs.push(payload), - }; - const quiet = world.captureConsole('log'); - // snippet.loggingCustomLoggerExample - class MonitoringLogger implements Logger { - /** - * Process error-level messages - */ - error(logMessage: LogMessage) { - const { timestamp, pubNubId, location, messageType } = logMessage; - - if (messageType === 'error') { - // TypeScript narrows type to ErrorLogMessage - // IDE provides autocomplete for PubNubError fields - monitoringService.reportError({ - message: logMessage.message.message, - name: logMessage.message.name, - stack: logMessage.message.stack, - category: logMessage.message.status?.category, - operation: logMessage.message.status?.operation, - instanceId: pubNubId, - timestamp: timestamp.toISOString(), - location - }); - } - } - - /** - * Process warning messages and failed requests - */ - warn(logMessage: LogMessage) { - if (logMessage.messageType === 'text') { - // TypeScript narrows type to TextLogMessage - // IDE provides autocomplete for string methods on message - if (logMessage.message.includes('deprecated')) { - analyticsService.trackDeprecation({ - warning: logMessage.message, - instanceId: logMessage.pubNubId, - timestamp: logMessage.timestamp.toISOString() - }); - } - } else if (logMessage.messageType === 'network-request') { - // TypeScript narrows type to NetworkRequestLogMessage - // The SDK logs a failed request at the warn level - if (logMessage.failed) { - const url = `${logMessage.message.origin}${logMessage.message.path}`; - - monitoringService.reportNetworkFailure({ - url: url, - method: logMessage.message.method, - reason: logMessage.details, - instanceId: logMessage.pubNubId, - timestamp: logMessage.timestamp.toISOString() - }); - } - } - } - - /** - * Process debug messages for configuration tracking - */ - debug(logMessage: LogMessage) { - if (logMessage.messageType === 'object') { - // TypeScript narrows type to ObjectLogMessage - // IDE provides autocomplete for details and ignoredKeys fields - if (logMessage.details?.includes('configuration')) { - analyticsService.trackConfigChange({ - config: logMessage.message, - instanceId: logMessage.pubNubId, - timestamp: logMessage.timestamp.toISOString() - }); - } - } - } - - // No-op implementations for other levels if not needed - trace() {} - info() {} - } - - // Configure SDK with custom logger - const pubnub = new PubNub({ - subscribeKey: 'mySubscribeKey', - publishKey: 'myPublishKey', - userId: 'myUniqueUserId', - logLevel: PubNub.LogLevel.Debug, - // snippet.hide - ...world.live.openKeys, ...world.unreachable, - // snippet.show - loggers: [new MonitoringLogger()] - }); - // snippet.end - world.track(pubnub); - quiet.restore(); - - // Debug entry about the configuration reaches trackConfigChange. - assert.ok(reported.configs.length > 0, 'trackConfigChange received the configuration entry'); - assert.ok(reported.configs[0].instanceId, 'instanceId (pubNubId) is set'); - assert.match(reported.configs[0].timestamp, /^\d{4}-\d{2}-\d{2}T/, 'timestamp is an ISO string'); - - // The client points at a port where nothing listens, so a request fails. The failure reaches reportError (error entry) - // and reportNetworkFailure (failed network-request entry). The deprecated stop() reaches trackDeprecation (warn entry). - await assert.rejects(pubnub.publish({ channel: 'x', message: 'm' })); - await eventually(() => assert.ok(reported.errors.length > 0, 'reportError received an error entry'), { label: 'error entry' }); - assert.equal(reported.errors[0].name, 'PubNubError'); - assert.ok(reported.errors[0].message); - assert.equal(reported.errors[0].category, 'PNNetworkIssuesCategory'); - assert.equal(reported.errors[0].operation, 'PNPublishOperation'); - assert.ok(reported.errors[0].stack, 'stack is set'); - // The SDK logs a failed network-request at the warn level, and the sample handles it in warn(), so - // reportNetworkFailure receives the failed publish request. - await eventually(() => assert.ok(reported.failures.length > 0, 'reportNetworkFailure received a failed request'), { label: 'failed request entry' }); - assert.ok(reported.failures[0].method, 'method is set'); - assert.match(reported.failures[0].url, /\/publish\//, 'the url names the publish path'); - assert.ok(reported.failures[0].reason, 'reason (details) is set'); - assert.ok(reported.failures[0].instanceId, 'instanceId (pubNubId) is set'); - pubnub.stop(); - assert.ok(reported.deprecations.length > 0, 'a deprecation warning reaches trackDeprecation'); - assert.match(reported.deprecations[0].warning, /deprecated/); - }); - - step('loggingMultipleLoggers', { proves: 'The SDK calls every logger in the loggers array.' }, async () => { - class MonitoringLogger implements Logger { trace() {} debug() {} info() {} warn() {} error() {} } - class AnalyticsLogger implements Logger { trace() {} debug() {} info() {} warn() {} error() {} } - const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'pn-logging-')); - const file = path.join(dir, 'multi.log'); - class FileLogger implements Logger { - private readonly stream = fs.createWriteStream(file, { flags: 'a' }); - trace(m: LogMessage) { this.stream.write(`${m.level}\n`); } - debug(m: LogMessage) { this.stream.write(`${m.level}\n`); } - info(m: LogMessage) { this.stream.write(`${m.level}\n`); } - warn(m: LogMessage) { this.stream.write(`${m.level}\n`); } - error(m: LogMessage) { this.stream.write(`${m.level}\n`); } - } - const quiet = world.captureConsole('log'); - // snippet.loggingMultipleLoggers - const pubnub = new PubNub({ - subscribeKey: 'mySubscribeKey', - publishKey: 'myPublishKey', - userId: 'myUniqueUserId', - logLevel: PubNub.LogLevel.Debug, - // snippet.hide - ...world.live.openKeys, - // snippet.show - loggers: [ - new MonitoringLogger(), // Send errors to monitoring service - new AnalyticsLogger(), // Track usage patterns - new FileLogger() // Write detailed logs to file (Node.js) - ] - }); - // snippet.end - world.track(pubnub); - quiet.restore(); - try { - // Each custom logger is called next to the console logger: the file logger saw the initialization entries. - await eventually(() => assert.ok(fs.readFileSync(file, 'utf8').length > 0), { label: 'file logger wrote' }); - } finally { - pubnub.destroy(); - fs.rmSync(dir, { recursive: true, force: true }); - } - }); - - - // ---- the prose and the LogMessage reference ------------------------------------------------------------------ - - check('loggingLevels', { timeout: 60_000, proves: 'Each log level records only the entry levels it should, and the default level is None.' }, async () => { - const open = world.env.open!; - // The same traffic at every level. A level records its own entries and those of the less verbose levels listed after it. - const levels: Array<[string, PubNub.LogLevel, string[]]> = [ - ['trace', LogLevel.Trace, ['trace', 'debug', 'info', 'warn', 'error']], - ['debug', LogLevel.Debug, ['debug', 'info', 'warn', 'error']], - ['info', LogLevel.Info, ['info', 'warn', 'error']], - ['warn', LogLevel.Warn, ['warn', 'error']], - ['error', LogLevel.Error, ['error']], - ['none', LogLevel.None, []], - ]; - const seenByLevel = new Map>(); - for (const [name, logLevel, allowed] of levels) { - const { logger, seen } = recorder(); - const quiet = world.captureConsole('log'); - const infoQuiet = world.captureConsole('info'); - const warnQuiet = world.captureConsole('warn'); - try { - const pubnub = world.track(new PubNub({ ...open, userId: world.n(`logging-${name}`), logLevel, loggers: [logger] })); - await pubnub.time(); - await assert.rejects(pubnub.publish({ channel: '', message: 'x' } as unknown as PubNub.Publish.PublishParameters)); - pubnub.stop(); // warn: deprecated - } finally { - quiet.restore(); - infoQuiet.restore(); - warnQuiet.restore(); - } - const levelsSeen = new Set(seen.map((s) => s.level)); - for (const level of levelsSeen) assert.ok(allowed.includes(level), `logLevel ${name} must not record ${level} entries`); - for (const level of ['error', 'warn', 'info', 'debug'].filter((l) => allowed.includes(l))) { - assert.ok(levelsSeen.has(level), `logLevel ${name} records ${level} entries (saw ${[...levelsSeen]})`); - } - seenByLevel.set(name, levelsSeen); - } - assert.equal(seenByLevel.get('none')!.size, 0, 'LogLevel.None records nothing'); - // Default: logging is disabled. - const { logger, seen } = recorder(); - world.track(new PubNub({ ...open, userId: world.n('logging-default'), loggers: [logger] })); - assert.equal(seen.length, 0, 'the default logLevel is None'); - }); - - check('loggingConsoleLogger', { proves: 'The console logger writes each level to the matching console method, with Trace going to console.debug.' }, async () => { - // The built-in console logger runs next to custom loggers and prints ISO timestamp, PubNub-, level, location, message. - const original = { log: console.log, info: console.info, debug: console.debug, warn: console.warn, error: console.error }; - const lines: Array<{ method: string; text: string }> = []; - for (const method of ['log', 'info', 'debug', 'warn', 'error'] as const) { - console[method] = (...args: unknown[]) => lines.push({ method, text: args.map(String).join(' ') }); - } - try { - const open = world.env.open!; - const pubnub = world.track(new PubNub({ ...open, userId: world.n('logging-console'), logLevel: LogLevel.Trace })); - await pubnub.time(); - pubnub.stop(); - } finally { - Object.assign(console, original); - } - assert.ok(lines.length > 0, 'the console logger printed'); - const format = /^\d{4}-\d{2}-\d{2}T[\d:.]+Z PubNub-[0-9a-f]+ (Trace|Debug|Info|Warn|Error)\s+\S+ /; - assert.ok(lines.some((l) => format.test(l.text)), `an entry matches the documented format. First: ${lines[0]?.text.slice(0, 120)}`); - assert.ok(lines.filter((l) => / Trace /.test(l.text)).every((l) => l.method === 'debug'), 'Trace entries are written with console.debug'); - }); - - check('loggingMessageTypes', { timeout: 60_000, proves: 'Every log message type has the shape the page reference lists.' }, async () => { - const { logger, seen } = recorder(); - const quiet = world.captureConsole('log'); - const debugQuiet = world.captureConsole('info'); - try { - const pubnub = world.track(new PubNub({ ...world.env.open!, userId: world.n('logging-types'), logLevel: LogLevel.Trace, loggers: [logger] })); - await pubnub.time(); - const unreachable = world.track(new PubNub({ ...world.env.open!, ...world.unreachable, userId: world.n('logging-types-down'), logLevel: LogLevel.Trace, loggers: [logger] })); - await assert.rejects(unreachable.publish({ channel: 'x', message: 'm' })); - } finally { - quiet.restore(); - debugQuiet.restore(); - } - const byType = (type: string) => seen.map((s) => s.message).filter((m) => m.messageType === type); - // Every entry carries the BaseLogMessage fields. - for (const { message, level } of seen) { - assert.ok(message.timestamp instanceof Date, 'timestamp is a Date'); - assert.match(message.pubNubId, /^[0-9a-f]+$/); - assert.equal(typeof message.level, 'number'); - assert.equal(message.minimumLevel, LogLevel.Trace); - assert.ok(['text', 'object', 'error', 'network-request', 'network-response'].includes(message.messageType)); - assert.ok(level); - } - const text = byType('text')[0] as Extract; - assert.equal(typeof text.message, 'string'); - const object = byType('object')[0] as Extract; - assert.ok(object.message !== undefined); - const error = byType('error')[0] as Extract; - assert.ok(error, 'an error entry from the failed publish'); - assert.ok(error.message.name && error.message.message, 'error entry has name and message'); - assert.ok(error.message.stack, 'error entry has a stack'); - assert.equal(error.message.status?.category, 'PNNetworkIssuesCategory', 'error entry has status.category'); - const failed = byType('network-request').find((m) => (m as { failed?: boolean }).failed) as Extract | undefined; - assert.ok(failed, 'a network-request entry with failed: true for the unreachable origin'); - assert.ok(!failed.canceled); - const request = byType('network-request').find((m) => /pndsn\.com/.test((m as { message: { origin?: string } }).message.origin ?? '')) as Extract; - assert.ok(request, 'a network-request entry from time()'); - assert.equal(request.message.method, 'GET'); - assert.match(request.message.origin ?? '', /pndsn\.com/); - assert.match(request.message.path, /\/time\//); - assert.ok(request.message.queryParameters, 'queryParameters'); - const response = byType('network-response')[0] as Extract; - assert.ok(response, 'a network-response entry from time()'); - assert.equal(response.message.status, 200); - assert.match(response.message.url, /\/time\//); - assert.ok(response.message.headers, 'headers'); - assert.ok(response.message.body instanceof ArrayBuffer, 'the body is an ArrayBuffer, as the page shows'); - // Node.js: the details field carries a protocol hint. - assert.match(String(response.details ?? ''), /Negotiated protocol: \S+/, 'network-response details carry the negotiated protocol on Node.js'); - console.log(`first error entry: ${JSON.stringify({ ...error, timestamp: undefined }).slice(0, 300)}`); - }); - - check('loggingValidationErrorEntryShape', { proves: 'Records the shape of a logged validation failure, a plain status object and not a PubNubError.' }, () => { - const { logger, seen } = recorder(); - const pubnub = world.track(new PubNub({ ...world.env.open!, userId: world.n('logging-validation'), logLevel: LogLevel.Error, loggers: [logger] })); - return assert.rejects(pubnub.publish({ channel: '', message: 'x' } as unknown as PubNub.Publish.PublishParameters)).then(() => { - const entry = seen.find((s) => s.message.messageType === 'error')!.message as Extract; - const message = entry.message as unknown as Record; - assert.equal(message.message, "Missing 'channel'"); - assert.equal(message.category, 'PNValidationErrorCategory'); - assert.equal(message.name, undefined, 'no name on the logged validation failure'); - assert.equal(message.stack, undefined, 'no stack on the logged validation failure'); - assert.equal(message.status, undefined, 'category sits on the message itself, not under status'); - }); - }); - - // The sample reads failed and canceled requests in error(). Record the level at which the SDK logs them. - check('loggingFailedRequestLevel', { proves: 'Records the log level of a failed request, which is warn.' }, async () => { - const { logger, seen } = recorder(); - const quiet = world.captureConsole('log'); - try { - const down = world.track(new PubNub({ ...world.env.open!, ...world.unreachable, userId: world.n('logging-failed'), logLevel: LogLevel.Trace, loggers: [logger] })); - await assert.rejects(down.publish({ channel: 'x', message: 'm' })); - } finally { - quiet.restore(); - } - const failed = seen.filter((s) => s.message.messageType === 'network-request' && (s.message as { failed?: boolean }).failed); - assert.equal(failed.length, 1); - assert.equal(failed[0].level, 'warn', 'a failed request is logged at the warn level, not error'); - assert.equal((failed[0].message as { details?: string }).details, 'Network error'); - assert.ok(!seen.some((s) => s.level === 'error' && s.message.messageType === 'network-request'), 'error() receives no network-request entries'); - }); -}); diff --git a/docs-snippets/getting-started/quickstart-node.mjs b/docs-snippets/getting-started/quickstart-node.mjs deleted file mode 100644 index c505e731b..000000000 --- a/docs-snippets/getting-started/quickstart-node.mjs +++ /dev/null @@ -1,44 +0,0 @@ -import PubNub from 'pubnub'; - -const pubnub = new PubNub({ - publishKey: 'YOUR_PUBLISH_KEY', - subscribeKey: 'YOUR_SUBSCRIBE_KEY', - userId: 'YOUR_USER_ID', -}); - -pubnub.addListener({ - message: (event) => { - console.log(`Received on ${event.channel}: ${event.message}`); - }, - status: (event) => { - if (event.category === 'PNConnectedCategory') { - console.log('Connected to PubNub'); - } - }, -}); - -// Create a channel entity -const channel = pubnub.channel('hello_world'); - -// Create a subscription -const subscription = channel.subscription({ - receivePresenceEvents: true, // to receive presence events -}); - -// Subscribe -subscription.subscribe(); - -pubnub.addListener({ - status: async (event) => { - if (event.category !== 'PNConnectedCategory') return; - try { - const result = await pubnub.publish({ - channel: 'hello_world', - message: 'Hello, world!', - }); - console.log(`Published, timetoken: ${result.timetoken}`); - } catch (error) { - console.error(`Publish failed: ${error}`); - } - }, -}); diff --git a/docs-snippets/getting-started/quickstart-node.ts b/docs-snippets/getting-started/quickstart-node.ts deleted file mode 100644 index efd0909bd..000000000 --- a/docs-snippets/getting-started/quickstart-node.ts +++ /dev/null @@ -1,132 +0,0 @@ -import assert from '../harness/assert.js'; -import { spawn } from 'node:child_process'; -import fs from 'node:fs'; -import path from 'node:path'; -import { fileURLToPath } from 'node:url'; -import PubNub from 'pubnub'; -import { page, step, check } from '../harness/step.js'; -import { eventually } from '../harness/wait.js'; - -/** - * Node.js quickstart examples. - */ -export default page('getting-started/quickstart-node', 'P4', (world) => { - const HERE = path.dirname(fileURLToPath(import.meta.url)); - - step('gettingStartedInitPubnub', { snippets: ['gettingStartedInitPubnub', 'quickstartNodeListeners', 'gettingStartedCreateSubscription', 'quickstartNodePublish'], proves: 'The Node quickstart script connects, publishes and receives, and prints the lines the page promises.' }, async () => { - await world.withConsole(async (logs) => { - // snippet.gettingStartedInitPubnub - const pubnub = new PubNub({ - publishKey: 'YOUR_PUBLISH_KEY', - subscribeKey: 'YOUR_SUBSCRIBE_KEY', - userId: 'YOUR_USER_ID', - // snippet.hide - ...world.live.open, - // snippet.show - }); - // snippet.end - world.track(pubnub); - - // snippet.quickstartNodeListeners - pubnub.addListener({ - message: (event) => { - console.log(`Received on ${event.channel}: ${event.message}`); - }, - status: (event) => { - if (event.category === 'PNConnectedCategory') { - console.log('Connected to PubNub'); - } - }, - }); - // snippet.end - - // snippet.gettingStartedCreateSubscription - // Create a channel entity - const channel = pubnub.channel('hello_world'); - - // Create a subscription - const subscription = channel.subscription({ - receivePresenceEvents: true, // to receive presence events - }); - - // Subscribe - subscription.subscribe(); - // snippet.end - - // snippet.quickstartNodePublish - pubnub.addListener({ - status: async (event) => { - if (event.category !== 'PNConnectedCategory') return; - try { - const result = await pubnub.publish({ - channel: 'hello_world', - message: 'Hello, world!', - }); - console.log(`Published, timetoken: ${result.timetoken}`); - } catch (error) { - console.error(`Publish failed: ${error}`); - } - }, - }); - // snippet.end - - assert.equal(pubnub.getUserId(), world.n('YOUR_USER_ID')); - await eventually(() => assert.match(logs.lines.find((l) => l.startsWith('Published, timetoken: ')) ?? '', /^Published, timetoken: \d{17}$/), { label: 'published line printed' }); - await eventually(() => assert.ok(logs.lines.includes(`Received on ${world.n('hello_world')}: Hello, world!`)), { label: 'own message received' }); - subscription.unsubscribe(); - }); - }); - - check('quickstartNodeRun', { timeout: 60_000, proves: 'The complete quickstart file runs in a Node process, prints Connected and Published, receives its own message, and receives a message from another client.' }, async () => { - const open = world.env.open!; - const source = fs - .readFileSync(path.join(HERE, 'quickstart-node.mjs'), 'utf8') - .replace('YOUR_PUBLISH_KEY', open.publishKey) - .replace('YOUR_SUBSCRIBE_KEY', open.subscribeKey) - .replace('YOUR_USER_ID', world.n('YOUR_USER_ID')) - .replaceAll("'hello_world'", `'${world.n('hello_world')}'`); - // The file sits next to the app so that `import 'pubnub'` resolves the same installation. - const dir = fs.mkdtempSync(path.join(HERE, '.quickstart-')); - const file = path.join(dir, 'index.mjs'); - fs.writeFileSync(file, source); - try { - const child = world.trackChild(spawn(process.execPath, [file], { stdio: ['ignore', 'pipe', 'pipe'] })); - let out = ''; - let err = ''; - child.stdout.on('data', (chunk) => (out += chunk)); - child.stderr.on('data', (chunk) => (err += chunk)); - try { - // The script publishes only after the connected status, so its own message reaches an active subscription and the - // "Received" line is deterministic. All three documented lines are asserted. - await eventually( - () => { - const lines = out.split('\n'); - assert.ok(lines.includes('Connected to PubNub'), `Connected line. Output: ${out} ${err}`); - assert.ok(lines.some((l) => /^Published, timetoken: \d{17}$/.test(l)), `Published line. Output: ${out} ${err}`); - assert.ok(lines.includes(`Received on ${world.n('hello_world')}: Hello, world!`), `Received line. Output: ${out} ${err}`); - }, - { timeout: 30_000, label: 'quickstart script connected, published and received its own message' }, - ); - // The listener keeps working: a message from another client arrives too. - await world.friend.publish({ channel: world.n('hello_world'), message: 'Hello from the harness' }); - await eventually( - () => assert.ok(out.split('\n').includes(`Received on ${world.n('hello_world')}: Hello from the harness`), `Received line. Output: ${out} ${err}`), - { timeout: 30_000, label: 'quickstart script receives a message from another client' }, - ); - // "The script keeps running because the subscription stays open." - assert.equal(child.exitCode, null, 'the script keeps running'); - } finally { - child.kill('SIGINT'); - } - } finally { - fs.rmSync(dir, { recursive: true, force: true }); - } - }); - - // The "Before you begin" claim: Node.js 22 or later. The SDK declares the same minimum. - check('quickstartNodeVersion', { proves: 'The SDK declares the same minimum Node.js version as the quickstart states.' }, () => { - const engines = JSON.parse(fs.readFileSync(path.join(path.dirname(fileURLToPath(import.meta.resolve('pubnub'))), '..', '..', 'package.json'), 'utf8')).engines; - assert.equal(engines.node, '>=22.0.0'); - assert.ok(Number(process.versions.node.split('.')[0]) >= 22, 'the app itself runs on Node.js 22 or later'); - }); -}); diff --git a/docs-snippets/getting-started/set-up-your-environment.ts b/docs-snippets/getting-started/set-up-your-environment.ts deleted file mode 100644 index e366ca655..000000000 --- a/docs-snippets/getting-started/set-up-your-environment.ts +++ /dev/null @@ -1,164 +0,0 @@ -import assert from '../harness/assert.js'; -import { execFileSync } from 'node:child_process'; -import fs from 'node:fs'; -import path from 'node:path'; -import { fileURLToPath } from 'node:url'; -import ts from 'typescript'; -// snippet.setUpTypeScriptImport -import PubNub from 'pubnub'; - -// snippet.end -import { page, step, check } from '../harness/step.js'; - -/** - * Examples for setting up your environment. - */ -const HERE = path.dirname(fileURLToPath(import.meta.url)); -const SDK_ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.resolve('pubnub'))), '..', '..'); -const TSC = path.resolve(path.dirname(fileURLToPath(import.meta.resolve('typescript'))), 'tsc.js'); - -export default page('getting-started/set-up-your-environment', 'P0', (world) => { - const scratch = (): string => fs.mkdtempSync(path.join(HERE, '.setup-')); - - step('setUpTypeScriptInit', { proves: 'The TypeScript init sample builds a working client and time() succeeds.' }, async () => { - // snippet.setUpTypeScriptInit - const pubnub = new PubNub({ - publishKey: 'your-publish-key', - subscribeKey: 'your-subscribe-key', - // snippet.hide - ...world.live.open, - // snippet.show - userId: 'your-user-id' - }); - // snippet.end - world.track(pubnub); - assert.equal(pubnub.getUserId(), world.n('your-user-id')); - world.check.serverTime((await pubnub.time()).timetoken); - }); - - check('setUpTypeScriptProject', { timeout: 90_000, proves: 'The TypeScript sample file and the documented tsconfig.json compile under strict settings.' }, () => { - const dir = scratch(); - try { - fs.writeFileSync( - path.join(dir, 'tsconfig.json'), - JSON.stringify( - { - compilerOptions: { - target: 'ES2020', - module: 'ESNext', - moduleResolution: 'bundler', - strict: true, - esModuleInterop: true, - skipLibCheck: true, - forceConsistentCasingInFileNames: true, - }, - include: ['index.ts'], - }, - null, - 2, - ), - ); - fs.writeFileSync( - path.join(dir, 'index.ts'), - `import PubNub from 'pubnub'; - -const pubnub = new PubNub({ - publishKey: 'your-publish-key', - subscribeKey: 'your-subscribe-key', - userId: 'your-user-id' -}); -`, - ); - // "you do not need an additional @types package": tsc resolves the types that the pubnub package ships. - execFileSync(process.execPath, [TSC, '-p', dir, '--noEmit'], { stdio: 'pipe' }); - } finally { - fs.rmSync(dir, { recursive: true, force: true }); - } - }); - - check('setUpModuleFormats', { timeout: 60_000, proves: 'Each documented module format loads the SDK in a real Node process.' }, () => { - const dir = scratch(); - const run = (args: string[], cwd = dir): string => execFileSync(process.execPath, args, { cwd, encoding: 'utf8', stdio: 'pipe' }).trim(); - const probe = "typeof PubNub === 'function' ? new PubNub({ subscribeKey: 'k', userId: 'u' }).getUserId() : 'no constructor'"; - try { - // CommonJS: const PubNub = require('pubnub'); - fs.writeFileSync(path.join(dir, 'cjs.cjs'), `const PubNub = require('pubnub');\nconsole.log(${probe});\n`); - assert.equal(run([path.join(dir, 'cjs.cjs')]), 'u', 'require("pubnub") returns the constructor'); - // ES modules through the .mjs extension. - fs.writeFileSync(path.join(dir, 'esm.mjs'), `import PubNub from 'pubnub';\nconsole.log(${probe});\n`); - assert.equal(run([path.join(dir, 'esm.mjs')]), 'u', '.mjs files use import'); - // "type": "module" in package.json makes .js an ES module. - fs.writeFileSync(path.join(dir, 'package.json'), JSON.stringify({ type: 'module' })); - fs.writeFileSync(path.join(dir, 'esm.js'), `import PubNub from 'pubnub';\nconsole.log(${probe});\n`); - assert.equal(run([path.join(dir, 'esm.js')]), 'u', '"type": "module" makes .js an ES module'); - } finally { - fs.rmSync(dir, { recursive: true, force: true }); - } - }); - - // Two-way drift guard against the SDK checkout: every documented variable and default must be in rollup.config.js, and every - // `*_MODULE` variable the SDK source reads must be documented, so adding a module to the SDK without the table fails here. - // Build a custom bundle: the module table matches rollup.config.js, names and defaults. - check('setUpBundleModuleTable', { proves: 'The custom bundle module table matches the SDK rollup configuration in both directions.' }, () => { - const documented: Record = { - CRYPTO_MODULE: 'enabled', - SHARED_WORKER: 'enabled', - PUBLISH_MODULE: 'enabled', - SUBSCRIBE_MODULE: 'enabled', - SUBSCRIBE_EVENT_ENGINE_MODULE: 'enabled', - SUBSCRIBE_MANAGER_MODULE: 'enabled', - PRESENCE_MODULE: 'enabled', - PAM_MODULE: 'disabled', - CHANNEL_GROUPS_MODULE: 'enabled', - MESSAGE_PERSISTENCE_MODULE: 'enabled', - MOBILE_PUSH_MODULE: 'enabled', - APP_CONTEXT_MODULE: 'enabled', - DATA_SYNC_MODULE: 'enabled', - FILE_SHARING_MODULE: 'enabled', - MESSAGE_REACTIONS_MODULE: 'enabled', - }; - const rollup = fs.readFileSync(path.join(SDK_ROOT, 'rollup.config.js'), 'utf8'); - const actual: Record = {}; - for (const match of rollup.matchAll(/'process\.env\.([A-Z_]+)':\s*JSON\.stringify\(process\.env\.\1 \?\? '(enabled|disabled)'\)/g)) actual[match[1]] = match[2]; - assert.deepEqual(actual, documented, 'documented build variables and defaults equal rollup.config.js'); - const used = new Set(); - const walk = (dir: string): void => { - for (const entry of fs.readdirSync(dir, { withFileTypes: true })) { - const full = path.join(dir, entry.name); - if (entry.isDirectory()) walk(full); - else if (entry.name.endsWith('.ts')) for (const m of fs.readFileSync(full, 'utf8').matchAll(/process\.env\.([A-Z_]+_MODULE|SHARED_WORKER)\b/g)) used.add(m[1]); - } - }; - walk(path.join(SDK_ROOT, 'src')); - assert.deepEqual([...used].sort(), Object.keys(documented).sort(), 'every build variable that the source reads is documented'); - // "Set enableTreeShaking to true": the constant exists. - assert.match(rollup, /const enableTreeShaking = (false|true);/); - }); - - check('setUpWebpackConfigSyntax', { proves: 'The documented Webpack configuration is valid JavaScript and defines the variables of the table.' }, () => { - const source = `const webpack = require('webpack'); - -module.exports = { - // Other configuration... - plugins: [ - new webpack.DefinePlugin({ - 'process.env.PRESENCE_MODULE': JSON.stringify(process.env.PRESENCE_MODULE ?? 'enabled'), - 'process.env.PUBLISH_MODULE': JSON.stringify(process.env.PUBLISH_MODULE ?? 'enabled'), - // One entry for every variable in the table. - }), - ], -}; -`; - const result = ts.transpileModule(source, { reportDiagnostics: true, compilerOptions: { module: ts.ModuleKind.CommonJS, target: ts.ScriptTarget.ES2020 } }); - assert.deepEqual(result.diagnostics ?? [], [], 'the Webpack configuration parses'); - }); - - check('setUpCdnUrls', { timeout: 30_000, proves: 'The pinned CDN URLs for the linked SDK version exist.' }, async () => { - const version = JSON.parse(fs.readFileSync(path.join(SDK_ROOT, 'package.json'), 'utf8')).version as string; - for (const file of [`pubnub.${version}.js`, `pubnub.worker.${version}.js`]) { - const response = await fetch(`https://cdn.pubnub.com/sdk/javascript/${file}`, { method: 'HEAD' }); - assert.equal(response.status, 200, `${file} exists on the CDN`); - } - }); - -}); diff --git a/docs-snippets/getting-started/status-events.ts b/docs-snippets/getting-started/status-events.ts deleted file mode 100644 index f92dc3a9a..000000000 --- a/docs-snippets/getting-started/status-events.ts +++ /dev/null @@ -1,344 +0,0 @@ -import assert from '../harness/assert.js'; -import http from 'node:http'; -import type { AddressInfo } from 'node:net'; -import PubNub from 'pubnub'; -import { page, step, check } from '../harness/step.js'; -import { connected, eventually, expectEvent, sleep } from '../harness/wait.js'; - -/** - * Status event examples. - */ -export default page('status-events', 'P7', (world) => { - const nextStatus = (client: PubNub, categories: string[], timeout = 20_000): Promise => - new Promise((resolve, reject) => { - const listener: PubNub.Listener = { - status: (status) => { - if (!categories.includes(String(status.category))) return; - clearTimeout(timer); - setImmediate(() => client.removeListener(listener)); - resolve(status); - }, - }; - const timer = setTimeout(() => { - client.removeListener(listener); - reject(new Error(`no status in [${categories.join(', ')}] within ${timeout}ms`)); - }, timeout); - client.addListener(listener); - }); - - const lines = (logs: string[], prefix: string): string[] => logs.filter((line) => line.startsWith(prefix)); - - // ---- before and after: PNConnectedCategory on every change versus PNSubscriptionChangedCategory --------------- - - step('statusEventsMigrationBefore', { phase: 'P4', proves: 'The legacy subscription manager reports PNConnectedCategory on every change of the channel mix.' }, async () => { - const pubnub = world.fresh('status-before', { enableEventEngine: false }); - const [c1, c2] = [world.n('ch1'), world.n('ch2')]; - await world.withConsole(async (logs) => { - // snippet.statusEventsMigrationBefore - pubnub.addListener({ - status: (statusEvent) => { - if (statusEvent.category === 'PNConnectedCategory') { - // This ran on first connect AND every time channels/groups were added or removed - console.log('Subscription updated:', statusEvent.affectedChannels, statusEvent.affectedChannelGroups); - } - } - }); - // snippet.end - - pubnub.subscribe({ channels: [c1] }); - await eventually(() => assert.equal(lines(logs.lines, 'Subscription updated:').length, 1), { label: 'first connect logged' }); - pubnub.subscribe({ channels: [c2] }); - await eventually(() => assert.equal(lines(logs.lines, 'Subscription updated:').length, 2), { label: 'channel change logged as PNConnectedCategory again' }); - assert.match(lines(logs.lines, 'Subscription updated:')[1], new RegExp(c2)); - pubnub.unsubscribeAll(); - }); - }); - - // After: the same two subscribes report PNConnectedCategory once and PNSubscriptionChangedCategory for the change. The final - // `sleep` is a negative wait, it gives a wrongly repeated PNConnectedCategory time to arrive before asserting it did not. - step('statusEventsMigrationAfter', { phase: 'P4', proves: 'The new subscribe loop reports PNConnectedCategory once and PNSubscriptionChangedCategory for later changes.' }, async () => { - const pubnub = world.fresh('status-after'); - const [c1, c2] = [world.n('ch1'), world.n('ch2')]; - await world.withConsole(async (logs) => { - // snippet.statusEventsMigrationAfter - pubnub.addListener({ - status: (statusEvent) => { - if (statusEvent.category === 'PNConnectedCategory') { - console.log('Subscribed and ready'); - } - if (statusEvent.category === 'PNSubscriptionChangedCategory') { - console.log('Subscription updated:', statusEvent.affectedChannels, statusEvent.affectedChannelGroups); - } - } - }); - // snippet.end - - const subscription1 = pubnub.channel(c1).subscription(); - subscription1.subscribe(); - await eventually(() => assert.equal(lines(logs.lines, 'Subscribed and ready').length, 1), { label: 'connected logged' }); - const subscription2 = pubnub.channel(c2).subscription(); - subscription2.subscribe(); - await eventually(() => assert.equal(lines(logs.lines, 'Subscription updated:').length, 1), { label: 'change logged as PNSubscriptionChangedCategory' }); - // The change reports every channel of the new mix, and PNConnectedCategory does not fire again. - assert.match(lines(logs.lines, 'Subscription updated:')[0], new RegExp(c1)); - assert.match(lines(logs.lines, 'Subscription updated:')[0], new RegExp(c2)); - await sleep(1000); - assert.equal(lines(logs.lines, 'Subscribed and ready').length, 1, 'PNConnectedCategory fires only when the subscription starts'); - pubnub.unsubscribeAll(); - }); - }); - - // ---- request statuses against a scripted origin ---------------------------------------------------------------- - - check('statusEventsRequestCategories', { phase: 'P0', timeout: 60_000, proves: 'Each documented status category is produced by the matching HTTP response from a local server.' }, async () => { - let respond: (res: http.ServerResponse) => void = (res) => res.end('[]'); - const server = http.createServer((_req, res) => respond(res)); - await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve)); - const origin = `127.0.0.1:${(server.address() as AddressInfo).port}`; - try { - const client = (extra: Partial = {}): PubNub => - world.fresh('status-origin', { origin, ssl: false, keepAlive: false, retryConfiguration: PubNub.NoneRetryPolicy(), ...extra }); - const categoryOf = async (pubnub: PubNub): Promise<{ category: string; statusCode?: number }> => { - try { - await pubnub.time(); - } catch (error) { - const status = (error as { status?: { category?: string; statusCode?: number } }).status; - return { category: String(status?.category), statusCode: status?.statusCode }; - } - return { category: 'none (resolved)' }; - }; - const reply = (code: number, body: string) => (res: http.ServerResponse) => { - res.writeHead(code, { 'content-type': 'application/json' }).end(body); - }; - - respond = reply(400, '{"error":true,"message":"bad"}'); - assert.equal((await categoryOf(client())).category, 'PNBadRequestCategory', '400'); - respond = reply(403, '{"error":true,"message":"Forbidden"}'); - assert.equal((await categoryOf(client())).category, 'PNAccessDeniedCategory', '403'); - respond = reply(500, '{"error":true}'); - assert.equal((await categoryOf(client())).category, 'PNServerErrorCategory', '500'); - respond = reply(503, '{"error":true}'); - assert.equal((await categoryOf(client())).category, 'PNServerErrorCategory', '503'); - respond = reply(418, '{"error":true}'); - assert.equal((await categoryOf(client())).category, 'PNUnknownCategory', 'unhandled status code'); - respond = reply(200, 'this is not json'); - assert.equal((await categoryOf(client())).category, 'PNUnknownCategory', '200 with a body that is not JSON'); - respond = (res) => res.writeHead(200, { 'content-type': 'text/html' }).end('proxy login'); - assert.equal((await categoryOf(client())).category, 'PNUnknownCategory', '200 with an HTML page'); - // A server that never answers: PNTimeoutCategory after transactionalRequestTimeout seconds. - respond = () => undefined; - const timedOut = client({ transactionalRequestTimeout: 1 }); - await assert.rejects(timedOut.publish({ channel: 'x', message: 'm' }), (error: unknown) => { - assert.equal(String((error as { status?: { category?: string } }).status?.category), 'PNTimeoutCategory', 'no answer within the timeout'); - return true; - }); - timedOut.destroy(); - // Nothing listens on the origin any more: PNNetworkIssuesCategory. - server.closeAllConnections(); - await new Promise((resolve) => server.close(() => resolve())); - // publish() is excluded from the default retry policy, so it rejects at once. time() would keep retrying. - await assert.rejects(client().publish({ channel: 'x', message: 'm' }), (error: unknown) => { - assert.equal(String((error as { status?: { category?: string } }).status?.category), 'PNNetworkIssuesCategory', 'connection refused'); - return true; - }); - } finally { - if (server.listening) { - server.closeAllConnections(); - server.close(); - } - } - - const open = world.env.open!; - const withoutPublishKey = world.track(new PubNub({ subscribeKey: open.subscribeKey, userId: world.n('status-nokey') })); - for (const call of [ - () => withoutPublishKey.publish({ channel: 'x', message: 'm' }), - () => world.shopper.publish({ message: 'm' } as unknown as PubNub.Publish.PublishParameters), - () => world.shopper.publish({ channel: 'x' } as unknown as PubNub.Publish.PublishParameters), - ]) { - await assert.rejects(call(), (error: unknown) => { - assert.equal(String((error as { status?: { category?: string } }).status?.category), 'PNValidationErrorCategory'); - return true; - }); - } - }); - - check('statusEventsDefaultRetryAlsoRetriesTime', { timeout: 30_000, proves: 'The default retry policy also retries time(), so it stays pending against a server that answers 500.' }, async () => { - let hits = 0; - const server = http.createServer((_req, res) => { - hits += 1; - res.writeHead(500, { 'content-type': 'application/json' }).end('{"error":true}'); - }); - await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve)); - const origin = `127.0.0.1:${(server.address() as AddressInfo).port}`; - try { - const pubnub = world.fresh('status-default-retry', { origin, ssl: false, keepAlive: false }); - const settled = pubnub.time().then(() => 'settled', () => 'settled'); - assert.equal(await Promise.race([settled, sleep(4500).then(() => 'pending')]), 'pending', 'time() is still retrying after 4.5 seconds'); - assert.ok(hits >= 2, `time() was retried (${hits} requests reached the server)`); - // publish() is excluded from the default policy: one request, one failure. - const before = hits; - await assert.rejects(pubnub.publish({ channel: 'x', message: 'm' })); - assert.equal(hits - before, 1, 'publish() is not retried'); - pubnub.destroy(); - } finally { - server.closeAllConnections(); - server.close(); - } - }); - - // ---- subscription statuses under an outage --------------------------------------------------------------------- - - const proxiedSubscriber = async (label: string, extra: Partial) => { - await world.proxy.start(); - world.proxy.outage(false); - const pubnub = world.fresh(label, extra); - pubnub.setProxy(world.proxy.config); - const channel = world.n('outage_channel'); - const ready = connected(pubnub, { channels: [channel] }); - const subscription = pubnub.channel(channel).subscription(); - subscription.subscribe(); - await ready; - return { pubnub, channel, subscription }; - }; - - check('statusEventsUnexpectedDisconnectAndReconnect', { timeout: 90_000, proves: 'With no retry policy a lost request reports PNDisconnectedUnexpectedlyCategory and reconnect() restores delivery.' }, async () => { - const { pubnub, channel, subscription } = await proxiedSubscriber('outage-none', { retryConfiguration: PubNub.NoneRetryPolicy() }); - try { - const before = world.proxy.connects; - const lost = nextStatus(pubnub, ['PNDisconnectedUnexpectedlyCategory', 'PNConnectionErrorCategory']); - world.proxy.outage(true); - const status = await lost; - // With the new subscription loop and no retries the loss is reported as PNDisconnectedUnexpectedlyCategory. - assert.equal(status.category, 'PNDisconnectedUnexpectedlyCategory'); - assert.ok((status as { error?: unknown }).error, 'the status carries an error field'); - console.log(`unexpected disconnect status: ${JSON.stringify(status)}`); - assert.ok(world.proxy.connects - before <= 1, 'NoneRetryPolicy makes no retry attempts'); - - // The outage ends and the app reconnects. - world.proxy.outage(false); - const back = nextStatus(pubnub, ['PNConnectedCategory', 'PNReconnectedCategory']); - pubnub.reconnect(); - const reconnected = await back; - console.log(`reconnect status category: ${reconnected.category}`); - const received = expectEvent(pubnub, 'message', (m) => (m.message as { outage?: string }).outage === 'after' && m.channel === channel); - await world.friend.publish({ channel, message: { outage: 'after' } }); - await received; - } finally { - world.proxy.outage(false); - subscription.unsubscribe(); - } - }); - - // The outage starts before the first subscribe, so the very first connect fails: PNConnectionErrorCategory, not a disconnect. - check('statusEventsConnectionErrorOnInitialConnect', { timeout: 60_000, proves: 'A failure on the first subscribe is PNConnectionErrorCategory and not a disconnect.' }, async () => { - await world.proxy.start(); - world.proxy.outage(true); - const pubnub = world.fresh('outage-initial', { retryConfiguration: PubNub.NoneRetryPolicy() }); - pubnub.setProxy(world.proxy.config); - try { - const failed = nextStatus(pubnub, ['PNConnectionErrorCategory', 'PNDisconnectedUnexpectedlyCategory']); - pubnub.channel(world.n('outage_channel')).subscription().subscribe(); - const status = await failed; - // Failed to establish the initial connection. - assert.equal(status.category, 'PNConnectionErrorCategory'); - assert.ok((status as { error?: unknown }).error, 'the status carries an error field'); - console.log(`connection error status: ${JSON.stringify(status)}`); - } finally { - world.proxy.outage(false); - pubnub.unsubscribeAll(); - } - }); - - // Measures, not mocks: counts CONNECT attempts the proxy sees while a policy retries during one outage. Bounds are loose on - // purpose (attempts at least 2, linear at least 3.5 s) because timing on a live network varies. The long timeout covers three - // subscribe and outage cycles. - check('statusEventsRetryPolicies', { timeout: 120_000, proves: 'During an outage each retry policy makes repeated connect attempts, counted at the proxy, and the linear policy spaces them out.' }, async () => { - // Count the tunnel attempts that a policy makes during one outage until the client gives up. - const attemptsDuring = async (label: string, retryConfiguration: PubNub.RequestRetryPolicy): Promise<{ attempts: number; ms: number }> => { - const { pubnub, subscription } = await proxiedSubscriber(label, { retryConfiguration }); - try { - const lost = nextStatus(pubnub, ['PNDisconnectedUnexpectedlyCategory', 'PNConnectionErrorCategory'], 60_000); - const before = world.proxy.connects; - const started = Date.now(); - world.proxy.outage(true); - await lost; - return { attempts: world.proxy.connects - before, ms: Date.now() - started }; - } finally { - world.proxy.outage(false); - subscription.unsubscribe(); - } - }; - - const none = await attemptsDuring('retry-none', PubNub.NoneRetryPolicy()); - const linear = await attemptsDuring('retry-linear', PubNub.LinearRetryPolicy({ delay: 2, maximumRetry: 2 })); - const exponential = await attemptsDuring('retry-exponential', PubNub.ExponentialRetryPolicy({ minimumDelay: 2, maximumDelay: 4, maximumRetry: 2 })); - console.log(`retry attempts during an outage: none ${none.attempts} (${none.ms} ms), linear ${linear.attempts} (${linear.ms} ms), exponential ${exponential.attempts} (${exponential.ms} ms)`); - assert.ok(none.attempts <= 1, 'NoneRetryPolicy does not retry'); - assert.ok(linear.attempts >= 2, `LinearRetryPolicy({ maximumRetry: 2 }) retries (saw ${linear.attempts} attempts)`); - assert.ok(linear.ms >= 3500, `Linear waits delay seconds between attempts (took ${linear.ms} ms)`); - assert.ok(exponential.attempts >= 2, `ExponentialRetryPolicy({ maximumRetry: 2 }) retries (saw ${exponential.attempts} attempts)`); - assert.ok(exponential.ms > none.ms, 'retries take time'); - }); - - // Default policy: a three second outage is retried inside the SDK. The client never reports a status for the in-between state and - // delivers a message published after the outage. - check('statusEventsDefaultRetryRecoversSilently', { timeout: 90_000, proves: 'A short outage is retried inside the SDK with no status and later messages still arrive.' }, async () => { - // Default policy (no retryConfiguration, enableEventEngine true): retries subscribe requests. A short outage ends without an unexpected-disconnect status. - const { pubnub, channel, subscription } = await proxiedSubscriber('outage-default', {}); - const categories: string[] = []; - pubnub.addListener({ status: (s) => categories.push(String(s.category)) }); - try { - const before = world.proxy.connects; - world.proxy.outage(true); - await sleep(3000); - assert.ok(world.proxy.connects > before, 'the default policy retried the subscribe request during the outage'); - world.proxy.outage(false); - const received = expectEvent(pubnub, 'message', (m) => (m.message as { outage?: string }).outage === 'default' && m.channel === channel, { timeout: 30_000 }); - await world.friend.publish({ channel, message: { outage: 'default' } }); - await received; - assert.deepEqual(categories, [], `no status for intermediate states (saw ${categories.join(', ')})`); - } finally { - world.proxy.outage(false); - subscription.unsubscribe(); - } - }); - - // Legacy manager (`enableEventEngine: false`) reports an outage as PNNetworkIssuesCategory and the recovery as PNReconnectedCategory. - check('statusEventsLegacyManagerOutage', { timeout: 90_000, proves: 'The legacy manager reports an outage as PNNetworkIssuesCategory and recovery as PNReconnectedCategory.' }, async () => { - await world.proxy.start(); - world.proxy.outage(false); - const pubnub = world.fresh('outage-legacy', { enableEventEngine: false }); - pubnub.setProxy(world.proxy.config); - const channel = world.n('outage_channel'); - const seen: string[] = []; - pubnub.addListener({ status: (s) => seen.push(String(s.category)) }); - const ready = connected(pubnub, { channels: [channel] }); - pubnub.subscribe({ channels: [channel] }); - await ready; - try { - const issues = nextStatus(pubnub, ['PNNetworkIssuesCategory'], 30_000); - world.proxy.outage(true); - await issues; - const recovered = nextStatus(pubnub, ['PNReconnectedCategory'], 60_000); - world.proxy.outage(false); - await recovered; - console.log(`legacy manager statuses: ${seen.join(', ')}`); - } finally { - world.proxy.outage(false); - pubnub.unsubscribeAll(); - } - }); - - // Access Manager: a 403 on the subscribe loop is a PNConnectionErrorCategory with PNAccessDeniedCategory in the nested error. - check('statusEventsAccessDeniedOnSubscribe', { keyset: 'pam', timeout: 45_000, proves: 'A 403 on the subscribe loop is PNConnectionErrorCategory with PNAccessDeniedCategory nested.' }, async () => { - const pubnub = world.client('secured', 'status-denied'); - const denied = nextStatus(pubnub, ['PNConnectionErrorCategory', 'PNAccessDeniedCategory']); - pubnub.channel(world.n('am_room')).subscription().subscribe(); - const status = await denied; - assert.equal(status.category, 'PNConnectionErrorCategory'); - console.log(`access denied status: ${JSON.stringify(status)}`); - // The nested error is the category name as a string. - assert.equal((status as { error?: unknown }).error, 'PNAccessDeniedCategory'); - pubnub.unsubscribeAll(); - }); -}); diff --git a/docs-snippets/getting-started/troubleshooting.ts b/docs-snippets/getting-started/troubleshooting.ts deleted file mode 100644 index 209f2ccfa..000000000 --- a/docs-snippets/getting-started/troubleshooting.ts +++ /dev/null @@ -1,67 +0,0 @@ -import assert from '../harness/assert.js'; -import PubNub from 'pubnub'; -import { page, check } from '../harness/step.js'; -import { rejectsValidation, rejectsWithCategory } from '../harness/assertions.js'; - -/** - * Troubleshooting examples. - */ -export default page('getting-started/troubleshooting', 'P0', (world) => { - // Each row is one documented cause of "the constructor throws". The SDK throws a PubNubError synchronously, before any request. - check('troubleshootingConstructorThrows', { proves: 'Each documented cause of a constructor failure throws a PubNubError before any request.' }, () => { - const keys = { subscribeKey: world.env.open!.subscribeKey }; - // "userId is omitted, is empty or only whitespace, or is set together with uuid." - for (const [config, pattern] of [ - [{ ...keys }, /'userId' not set/], - [{ ...keys, userId: '' }, /'userId' not set/], - [{ ...keys, userId: ' ' }, /'userId' is empty/], - [{ ...keys, userId: 'a', uuid: 'b' }, /use only 'userId'/], - ] as const) { - assert.throws( - () => new PubNub(config as unknown as PubNub.PubNubConfiguration), - (error: unknown) => (error as Error).name === 'PubNubError' && pattern.test((error as Error).message), - ); - } - }); - - check('troubleshootingPublishValidation', { proves: 'A publish without a publish key fails local validation, and the category differs from what the page states.' }, async () => { - const withoutPublishKey = world.track(new PubNub({ subscribeKey: world.env.open!.subscribeKey, userId: world.n('trouble-nokey') })); - await rejectsValidation(withoutPublishKey.publish({ channel: 'x', message: 'm' }), /Missing 'publishKey'/); - await rejectsValidation(world.shopper.publish({ message: 'm' } as unknown as PubNub.Publish.PublishParameters), /Missing 'channel'/); - await rejectsValidation(world.shopper.publish({ channel: 'x' } as unknown as PubNub.Publish.PublishParameters), /Missing 'message'/); - }); - - // "Publish fails with Message Too Large ... PubNub rejects the publish with HTTP 400." Record what the service does. - check('troubleshootingMessageTooLarge', { timeout: 60_000, proves: 'The service rejects an oversized publish and the step records the answer.' }, async () => { - const channel = world.n('my_channel'); - const outcome = async (size: number, sendByPost: boolean): Promise<{ statusCode?: number; category?: string; message?: string } | 'published'> => { - try { - await world.shopper.publish({ channel, message: 'a'.repeat(size), sendByPost, storeInHistory: false }); - return 'published'; - } catch (error) { - const status = (error as { status?: { statusCode?: number; category?: string; errorData?: { message?: string } } }).status; - return { statusCode: status?.statusCode, category: status?.category, message: status?.errorData?.message }; - } - }; - assert.equal(await outcome(30_000, true), 'published', 'a message below 32 KiB is published'); - const post = await outcome(33_000, true); - const get = await outcome(33_000, false); - console.log(`message above the limit: POST ${JSON.stringify(post)}, GET ${JSON.stringify(get)}`); - assert.notEqual(post, 'published'); - assert.notEqual(get, 'published'); - assert.equal((post as { statusCode?: number }).statusCode, 413); - assert.equal((get as { statusCode?: number }).statusCode, 414); - }); - - check('troubleshootingAccessDenied', { keyset: 'pam', timeout: 30_000, proves: 'A client without a token is denied on the secured keyset, and the error names Access Manager as the source.' }, async () => { - const pubnub = world.client('secured', 'trouble-denied'); - await assert.rejects(pubnub.publish({ channel: world.n('am_room'), message: 'm' }), (error: unknown) => { - const status = (error as { status?: { category?: string; statusCode?: number; errorData?: { service?: string } } }).status; - assert.equal(status?.category, 'PNAccessDeniedCategory'); - assert.equal(status?.statusCode, 403); - assert.equal(status?.errorData?.service, 'Access Manager'); - return true; - }); - await rejectsWithCategory(pubnub.fetchMessages({ channels: [world.n('am_room')] }), 'PNAccessDeniedCategory'); - }); -}); diff --git a/docs-snippets/harness/assert.ts b/docs-snippets/harness/assert.ts deleted file mode 100644 index 2ce9ea53b..000000000 --- a/docs-snippets/harness/assert.ts +++ /dev/null @@ -1,116 +0,0 @@ -/** - * Recorded drop-in for `node:assert/strict`. - * - * Page modules import this instead of Node's module (only the import line changed, the calls are the same). Every call - * behaves exactly like the Node original: same checks, same errors, same return values, and the same TypeScript types, - * so `assert.ok(x)` still narrows. On top of that each call is written to the recorder (recorder.ts), which is how the run - * report lists the assertions of every step with their messages. - * - * It wraps every function Node exports (`ok`, `equal`, `deepEqual`, `match`, `rejects`, `throws`, `fail`, ...), so a method - * a step starts using later is recorded without touching this file. - */ -import nodeAssert from 'node:assert/strict'; -import { appFrame, recorder, short } from './recorder.js'; - -type AnyFn = (...args: unknown[]) => unknown; - -/** Where the optional message sits for each assertion. The `throws` family takes it last, the others at a fixed position. */ -function messageOf(kind: string, args: unknown[]): string | undefined { - const lastIsMessage = (): string | undefined => { - const last = args[args.length - 1]; - return args.length > 1 && typeof last === 'string' ? last : undefined; - }; - switch (kind) { - case 'ok': - return typeof args[1] === 'string' ? args[1] : undefined; - case 'fail': - return typeof args[0] === 'string' ? args[0] : undefined; - case 'throws': - case 'rejects': - case 'doesNotThrow': - case 'doesNotReject': - return lastIsMessage(); - case 'ifError': - return undefined; - default: - // equal, notEqual, deepEqual, strictEqual, match, ... take (actual, expected, message). - return typeof args[2] === 'string' ? args[2] : undefined; - } -} - -/** The label of an assertion: its message, or a generated line that shows what was compared. */ -function labelOf(kind: string, args: unknown[]): string { - const message = messageOf(kind, args); - if (message) return message; - switch (kind) { - case 'ok': - return `ok: ${short(args[0])}`; - case 'match': - case 'doesNotMatch': - return `${kind}: ${short(args[0])} against ${short(args[1])}`; - case 'throws': - case 'rejects': - case 'doesNotThrow': - case 'doesNotReject': - return `${kind}: ${typeof args[1] === 'function' || args[1] instanceof RegExp || typeof args[1] === 'object' ? 'with the expected error' : 'any error'}`; - case 'fail': - return 'fail'; - default: - return `${kind}: ${short(args[0], 50)} and ${short(args[1], 50)}`; - } -} - -/** Records a failed assertion with the values and the first frame inside the suite. */ -function recordFailure(kind: string, args: unknown[], error: unknown, callSite: Error | undefined): void { - const e = error as { actual?: unknown; expected?: unknown; stack?: string }; - recorder.record({ - kind, - label: labelOf(kind, args), - ok: false, - actual: 'actual' in (e ?? {}) ? short(e.actual, 200) : undefined, - expected: 'expected' in (e ?? {}) ? short(e.expected, 200) : undefined, - frame: appFrame(e?.stack) ?? appFrame(callSite?.stack), - }); -} - -function wrap(kind: string, fn: AnyFn): AnyFn { - return function recordedAssertion(...args: unknown[]): unknown { - // A rejected async assertion loses the caller in its stack, so remember the call site up front (async kinds only). - const callSite = kind === 'rejects' || kind === 'doesNotReject' ? new Error() : undefined; - let result: unknown; - try { - result = fn.apply(nodeAssert, args); - } catch (error) { - recordFailure(kind, args, error, callSite); - throw error; - } - if (result instanceof Promise) { - return result.then( - (value) => { - recorder.record({ kind, label: labelOf(kind, args), ok: true }); - return value; - }, - (error) => { - recordFailure(kind, args, error, callSite); - throw error; - }, - ); - } - recorder.record({ kind, label: labelOf(kind, args), ok: true }); - return result; - }; -} - -// `assert(value, message)` is `assert.ok`. Every other export that is a function gets the same wrapper, except classes -// (AssertionError) and the `strict` self-reference. -const recorded = wrap('ok', nodeAssert as unknown as AnyFn) as AnyFn & Record; -for (const [name, member] of Object.entries(nodeAssert)) { - if (typeof member !== 'function' || name === 'strict' || /^[A-Z]/.test(name)) continue; - recorded[name] = wrap(name, member as AnyFn); -} -recorded.AssertionError = nodeAssert.AssertionError; -recorded.strict = recorded; - -// Typed as the Node module so every assertion signature (including `asserts value`) is kept. -export const assert: typeof nodeAssert = recorded as unknown as typeof nodeAssert; -export default assert; diff --git a/docs-snippets/harness/assertions.ts b/docs-snippets/harness/assertions.ts deleted file mode 100644 index 86fb643de..000000000 --- a/docs-snippets/harness/assertions.ts +++ /dev/null @@ -1,33 +0,0 @@ -/** - * Small assertion helpers that steps share. They exist because the SDK reports failures as a rejected promise - * carrying a `status` object (category, statusCode, errorData) rather than as a typed error class, so "this call must - * be refused for reason X" is the same few lines everywhere. - */ -import assert from './assert.js'; - -/** - * Adds `delta` to a PubNub timetoken (17-digit string) without losing precision. Timetokens are 10 ns ticks and exceed - * Number.MAX_SAFE_INTEGER, so plain arithmetic would round. BigInt keeps it exact. - */ -export function addToTimetoken(timetoken: string, delta: number): string { - return (BigInt(timetoken) + BigInt(delta)).toString(); -} - -/** Resolves when `promise` rejects with the SDK's validation error and the status message matches. */ -export async function rejectsValidation(promise: Promise, pattern: RegExp): Promise { - await assert.rejects(promise, (error: unknown) => { - const status = (error as { status?: { category?: string; message?: unknown } }).status; - assert.equal(String(status?.category), 'PNValidationErrorCategory', 'expected a validation error'); - assert.match(String(status?.message ?? ''), pattern); - return true; - }); -} - -/** Resolves when `promise` rejects with an SDK error whose category matches (for example access denied). */ -export async function rejectsWithCategory(promise: Promise, category: string): Promise { - await assert.rejects(promise, (error: unknown) => { - const status = (error as { status?: { category?: string } }).status; - assert.equal(String(status?.category), category); - return true; - }); -} diff --git a/docs-snippets/harness/cli.ts b/docs-snippets/harness/cli.ts deleted file mode 100644 index 6652194f7..000000000 --- a/docs-snippets/harness/cli.ts +++ /dev/null @@ -1,840 +0,0 @@ -import { spawnSync } from 'node:child_process'; -import fs from 'node:fs'; -import os from 'node:os'; -import path from 'node:path'; -import { pathToFileURL, fileURLToPath } from 'node:url'; -import Mocha from 'mocha'; -import { addSecrets, installRedaction, knownSecrets, redact } from './redact.cjs'; -import { - ciRunId, - describeRoles, - ENV_VARS, - loadEnv, - randomRunId, - ROLES, - runIdAgeMs, - type Env, - type Role, -} from './env.js'; -import { lintApp, printLint } from './lint-markers.js'; -import { stage } from './stage.js'; -import { runJanitor } from './janitor.js'; -import { launchProblems, outcomeProblems } from './policy.js'; -import { classifyPreflightError, preflightMessage } from './preflight-errors.js'; -import { buildManifest, docsIds, serialize } from './manifest.js'; -import { provisionDataSync } from '../provision/provision-datasync.js'; -import type { Registry, Selection } from './registry.js'; - -/** - * Commands (root package.json scripts): - * live npm run test:docs-snippets [-- --grep ] [-- --shard ] - * dry npm run test:docs-snippets:dry no network - * janitor npm run test:docs-snippets:janitor leftovers older than six hours, and the application-level grants - * manifest npm run docs-snippets:manifest [-- --check] - * stage only build docs-snippets/.run// - * lint marker lint only - * - * Run flow of `live`: - * 1. install output redaction, load env (prints "set" or "missing" per role, never a value), pick the run id - * 2. launch policy (CI refuses subset runs and debug logging), marker lint, stage into `.run//` - * 3. compute the selection from the staged plan. When it includes the datasync role, set up the DataSync classes first - * (provision/provision-datasync.ts), before preflight and before any step - * 4. run Mocha programmatically on `.run//harness/mocha-entry.ts`. Its root hooks prove Access Manager enforcement - * before and after, and run the janitor after. The reporter writes report/run.json - * 5. read that report back and apply the outcome policy (any skip fails the run, residue) - * 6. wait for the event loop to drain. If it is still busy 10 s later, print the open handles and exit 1 - * - * Mocha is driven from here and not through a config file alone because the file to run only exists after staging. - * `docs-snippets/.mocharc.cjs` holds the shared Mocha options (reporter, retries, timeout) and this file reads it. - */ - -const SUITE_DIR = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..'); -// The default local env file, passed on through the environment so the staged copy under .run// reads it too. -if (!process.env.DOCS_SNIPPETS_ENV_FILE && fs.existsSync(path.join(SUITE_DIR, '.env.local'))) { - process.env.DOCS_SNIPPETS_ENV_FILE = path.join(SUITE_DIR, '.env.local'); -} -const REPORT_DIR = path.join(SUITE_DIR, 'report'); -const args = process.argv.slice(2); -const command = args[0] && !args[0].startsWith('-') ? args[0] : 'live'; -const rest = command === args[0] ? args.slice(1) : args; - -/** `--name value` or `--name=value`. */ -function option(name: string): string | undefined { - for (let i = 0; i < rest.length; i += 1) { - if (rest[i] === name) return rest[i + 1]; - if (rest[i].startsWith(`${name}=`)) return rest[i].slice(name.length + 1); - } - return undefined; -} -const flags = new Set(rest.filter((a) => a.startsWith('--'))); - -async function mochaOptions(): Promise { - const config = (await import(pathToFileURL(path.join(SUITE_DIR, '.mocharc.cjs')).href)) as { - default: Mocha.MochaOptions; - }; - return { ...config.default, reporter: path.join(SUITE_DIR, 'harness', 'reporter.cjs') }; -} - -/** Imports the staged plan and registers it, without a network. Returns the staged registry. */ -async function stagedPlan( - runDir: string, - env: Env, - runId: string, -): Promise<{ - registry: Registry; - selection: (f: { shard?: Role; grep?: string; only?: string[] }) => Selection; - roles: (s: Selection) => Set; -}> { - const run = (await import(pathToFileURL(path.join(runDir, 'run.ts')).href)) as typeof import('../run.js'); - const worldModule = (await import( - pathToFileURL(path.join(runDir, 'harness', 'world.ts')).href - )) as typeof import('./world.js'); - const reg = (await import( - pathToFileURL(path.join(runDir, 'harness', 'registry.ts')).href - )) as typeof import('./registry.js'); - const world = new worldModule.World(env, runId); - run.registry.reset(); - run.registry.availability = { open: true, pam: true, datasync: true }; - run.registerAll(world); - return { - registry: run.registry, - selection: (f) => reg.selectSteps(run.registry, f), - roles: (s) => reg.rolesOf(run.registry, s), - }; -} - -/** Exits with `code` once the event loop is empty. If it is still busy after 10 s, prints the open handles and exits 1. */ -function exitWhenIdle(code: number): void { - const timer = setTimeout(() => { - const resources = process.getActiveResourcesInfo(); - // Real I/O: anything that could still talk to the service, a keyset or the machine. Timers alone cannot. - const io = resources.filter((r) => !['Timeout', 'Immediate', 'PipeWrap', 'TTYWrap'].includes(r)); - const lines = describeHandles(); - if (io.length > 0) { - console.error(`the event loop is still busy 10 s after the run finished (run exit code ${code}). Open handles:`); - console.error(` ${lines.join('\n ')}`); - process.exit(1); - } - // Only timers are left, for example a retry timer inside a destroyed SDK client. They cannot change any result, so the - // run keeps its exit code, but they are named so a real leak never goes unnoticed. - console.error( - `warning: only timers keep the event loop busy 10 s after the run (${resources.filter((r) => r === 'Timeout').length} timer(s)), exiting with the run's exit code ${code}`, - ); - process.exit(code); - }, 10_000); - timer.unref(); - process.once('beforeExit', () => { - clearTimeout(timer); - process.exit(code); - }); -} - -/** - * What keeps the event loop alive, one line per handle. Node's own lists, so nothing has to be loaded before the handles were - * created (why-is-node-running only sees handles created after it loads). Sockets show the remote host and port, timers - * their delay, child processes their pid. No key appears here, and stderr is redacted anyway. - */ -function describeHandles(): string[] { - const lines: string[] = []; - const handles = (process as unknown as { _getActiveHandles?: () => unknown[] })._getActiveHandles?.() ?? []; - for (const handle of handles) { - if (handle === process.stdout || handle === process.stderr || handle === process.stdin) continue; - const h = handle as { - constructor?: { name?: string }; - remoteAddress?: string; - remotePort?: number; - _host?: string; - servername?: string; - pid?: number; - spawnfile?: string; - _idleTimeout?: number; - address?: () => unknown; - hasRef?: () => boolean; - }; - const name = h.constructor?.name ?? typeof handle; - const detail: string[] = []; - if (h.servername || h._host) detail.push(`host ${h.servername ?? h._host}`); - if (h.remoteAddress) detail.push(`remote ${h.remoteAddress}:${h.remotePort ?? ''}`); - if (h.pid) detail.push(`pid ${h.pid}${h.spawnfile ? ` ${h.spawnfile}` : ''}`); - if (typeof h._idleTimeout === 'number' && h._idleTimeout > 0) detail.push(`timeout ${h._idleTimeout} ms`); - if (name === 'Server' && typeof h.address === 'function') detail.push(`listening ${JSON.stringify(h.address())}`); - if (typeof h.hasRef === 'function' && !h.hasRef()) detail.push('unref'); - lines.push(`${name}${detail.length ? ` (${detail.join(', ')})` : ''}`); - } - const resources = process.getActiveResourcesInfo().filter((r) => r !== 'TTYWrap'); - lines.push(`resource types: ${resources.join(', ') || 'none'}`); - return lines; -} - -async function live(): Promise { - installRedaction(); - const shardArg = option('--shard'); - if (shardArg) process.env[ENV_VARS.shard] = shardArg; - const env = loadEnv(); - console.log(describeRoles(env)); - const grep = option('--grep'); - if (!env.open && !env.pam && !env.datasync) { - console.log( - `skipped: no keyset is configured. Set the env names in docs-snippets/README.md or ${ENV_VARS.envFile}.`, - ); - return env.ci || flags.has('--require-env') ? 1 : 0; - } - const launch = launchProblems(env, process.env, { grep }); - if (launch.length > 0) { - launch.forEach((p) => console.error(p)); - return 1; - } - const runId = env.runId ?? (env.ci ? ciRunId(process.env, env.shard) : randomRunId()); - process.env[ENV_VARS.runId] = runId; - process.env.DOCS_SNIPPETS_REPORT_DIR = REPORT_DIR; - if (grep) process.env.DOCS_SNIPPETS_GREP = grep; - else delete process.env.DOCS_SNIPPETS_GREP; - console.log(`run id ${runId}${env.shard ? `, shard ${env.shard}` : ''} (staging in docs-snippets/.run/${runId}/)`); - - const phase = timer(); - const lint = lintApp(SUITE_DIR); - printLint(lint); - if (lint.errors.length > 0) return 1; - const staged = stage(SUITE_DIR, runId, { mutate: process.env.DOCS_SNIPPETS_MUTATE || undefined }); - console.log( - `staged ${staged.filesCopied} files, ${staged.rewrites.length} literal rewrites, ${staged.ranMarkers} snippet.end markers instrumented${staged.ranFallbacks.length ? ` (AST fallback: ${staged.ranFallbacks.map((f) => f.id).join(', ')})` : ''}${staged.mutated ? `, mutated ${staged.mutated}` : ''}`, - ); - if (staged.problems.length > 0) { - staged.problems.forEach((p) => console.error(p)); - return 1; - } - console.log(`phase stage: ${phase()}`); - - const only = process.env.DOCS_SNIPPETS_ONLY?.split(',') - .map((x) => x.trim()) - .filter(Boolean); - const plan = await stagedPlan(staged.runDir, env, runId); - const selection = plan.selection({ shard: env.shard, grep, only }); - const roles = plan.roles(selection); - console.log( - `selected ${selection.run.size - selection.prerequisites.size} step(s), ${selection.prerequisites.size} prerequisite(s), roles ${[...roles].join(', ')}`, - ); - if (selection.run.size === 0) { - console.error('no step matches the selection'); - return 1; - } - - if (roles.has('datasync')) { - const setup = timer(); - console.log('DataSync class setup (before preflight and before any step):'); - const result = await provisionDataSync(env, { check: false }); - result.log.forEach((line) => console.log(` ${line}`)); - if (!result.ok) { - result.problems.forEach((p) => console.error(`DataSync class setup failed: ${p}`)); - return 1; - } - console.log(`phase datasync setup: ${setup()}`); - } - - const mocha = new Mocha(await mochaOptions()); - mocha.addFile(path.join(staged.runDir, 'harness', 'mocha-entry.ts')); - await mocha.loadFilesAsync(); - const steps = timer(); - const failures = await new Promise((resolve) => { - mocha.run((count) => resolve(count)); - }); - console.log(`phase steps: ${steps()}`); - - const reportPath = path.join(REPORT_DIR, 'run.json'); - let problems: string[] = []; - let skips: Array<{ id: string; reason: string; note?: string }> = []; - if (fs.existsSync(reportPath)) { - const report = JSON.parse(fs.readFileSync(reportPath, 'utf8')) as { - runId: string; - skips: Array<{ id: string; reason: string; note?: string }>; - }; - if (report.runId !== runId) problems.push(`report/run.json belongs to run ${report.runId}, not ${runId}`); - skips = report.skips; - } else { - problems.push('no report was written to docs-snippets/report/run.json'); - } - problems.push(...outcomeProblems({ skips })); - for (const skip of skips) console.log(`skip: ${skip.id} (${skip.reason})${skip.note ? ` ${skip.note}` : ''}`); - problems.forEach((p) => console.error(p)); - return failures > 0 || problems.length > 0 ? 1 : 0; -} - -function timer(): () => string { - const started = Date.now(); - return () => `${((Date.now() - started) / 1000).toFixed(1)} s`; -} - -/** No network: everything that can be proven without keys. */ -async function dryRun(): Promise { - let failed = false; - const fail = (message: string): void => { - failed = true; - console.error(`FAIL ${message}`); - }; - const heading = (title: string): void => console.log(`\n== ${title}`); - - heading('1. Environment gate (no keys needed)'); - const emptyEnv = loadEnv({}); - console.log(` with no env names set: ${describeRoles(emptyEnv)}`); - if (emptyEnv.open || emptyEnv.pam || emptyEnv.datasync) fail('empty environment produced a keyset'); - { - // An env file with the older PN_DOCS_* names maps onto the suite's names, and real variables win. - const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'docs-snippets-env-')); - const file = path.join(dir, '.env.local'); - // The older names come from the mapping table itself, so this file holds no copy of them. - const { envSource, FILE_ALIASES } = await import('./env.js'); - const alias = (name: string): string => FILE_ALIASES[name][0]; - fs.writeFileSync( - file, - [ - `${alias('PUBLISH_KEY')}=pub-file`, - `${alias('SUBSCRIBE_KEY')}=sub-file`, - `${alias('PAM_PUBLISH_KEY')}=pub-sec`, - `${alias('PAM_SUBSCRIBE_KEY')}=sub-sec`, - `${alias('PAM_SECRET_KEY')}=sec-sec`, - ].join('\n'), - ); - const mapped = loadEnv(envSource({ [ENV_VARS.envFile]: file, PUBLISH_KEY: 'pub-real' })); - if (mapped.open?.publishKey !== 'pub-real' || mapped.open?.subscribeKey !== 'sub-file') - fail('env file: real variables must win over the file'); - if (mapped.pam?.secretKey !== 'sec-sec' || mapped.datasync?.secretKey !== 'sec-sec' || !mapped.sharedSecured) - fail('env file: the secured keyset must fill pam and datasync'); - const line = describeRoles(mapped); - if (/pub-file|sub-file|pub-sec|sub-sec|sec-sec|pub-real/.test(line)) fail('the loader output contains a key value'); - console.log(` env file mapping: ${line.replace(dir, '')}`); - fs.rmSync(dir, { recursive: true, force: true }); - } - - heading('2. Marker lint'); - const lint = lintApp(SUITE_DIR); - printLint(lint); - if (lint.errors.length > 0) failed = true; - - heading('3. Staging rewrite and honesty instrumentation'); - const runId = randomRunId(); - const staged = stage(SUITE_DIR, runId); - console.log(` copied ${staged.filesCopied} files, ${staged.rewrites.length} literal rewrites with run id ${runId}`); - for (const from of [ - 'my_channel', - 'chats.room1', - 'uuid', - 'replace_with_message_timetoken', - 'replace-with-end-timetoken', - ]) { - const hit = staged.rewrites.find((r) => r.from === from); - if (hit) console.log(` sample ${hit.file}:${hit.line} '${hit.from}' -> ${hit.to}`); - else fail(`no rewrite found for sample literal '${from}'`); - } - staged.problems.forEach((p) => fail(p)); - const leaked = staged.rewrites.filter( - (r) => r.kind === 'name' && fs.readFileSync(path.join(SUITE_DIR, r.file), 'utf8').includes(r.to), - ); - if (leaked.length > 0) fail(`source files contain rewritten names, e.g. ${leaked[0].file}: ${leaked[0].to}`); - const stepIds = lint.snippets.filter((s) => s.step).length; - console.log( - ` __snippetRan on ${staged.ranMarkers} of ${stepIds} snippet.end lines inside step bodies, AST fallback for: ${staged.ranFallbacks.map((f) => `${f.id} (${f.file}:${f.line})`).join(', ') || 'none'}`, - ); - if (staged.ranMarkers !== stepIds) - fail(`expected ${stepIds} instrumented snippet.end lines, got ${staged.ranMarkers}`); - if (!fs.realpathSync(path.join(staged.runDir, 'node_modules', 'pubnub')).startsWith(path.resolve(SUITE_DIR, '..'))) - fail('the staged node_modules/pubnub link does not point at this repository'); - { - let other = randomRunId(); - while (other === runId) other = randomRunId(); - const mutated = stage(SUITE_DIR, other, { mutate: 'publishBasicUsage' }); - const text = fs.readFileSync(path.join(mutated.runDir, 'api-reference', 'publish.ts'), 'utf8'); - if ( - mutated.mutated !== 'publishBasicUsage' || - !/throw new Error\('DOCS_SNIPPETS_MUTATE publishBasicUsage'\); __snippetStart\('publishBasicUsage'\); \/\/ snippet\.publishBasicUsage/.test( - text, - ) - ) - fail('DOCS_SNIPPETS_MUTATE did not insert the throw at the start marker'); - else - console.log( - ' DOCS_SNIPPETS_MUTATE=publishBasicUsage inserts the throw at the start marker of the staged copy only', - ); - if (!fs.existsSync(staged.runDir)) fail('a second stage removed the first one'); - fs.rmSync(mutated.runDir, { recursive: true, force: true }); - } - - heading('4. Step plan from the staged copy'); - const plan = await stagedPlan(staged.runDir, emptyEnv, runId); - const registry = plan.registry; - const wiring = registry.validate(); - wiring.forEach((p) => fail(p)); - const ordered = registry.ordered(); - const byRole = Object.fromEntries(ROLES.map((r) => [r, ordered.filter((s) => s.keyset === r).length])); - console.log( - ` ${ordered.length} steps registered (${ROLES.map((r) => `${r} ${byRole[r]}`).join(', ')}), ${wiring.length} wiring problems`, - ); - const cross = registry.crossRoleNeeds(); - console.log(` cross-role needs edges (${cross.length}):`); - for (const e of cross) console.log(` ${e.step} (${e.role}) needs ${e.needs} (${e.needsRole})`); - - heading('5. Static mapping: lint, registry and manifest agree'); - const lintIds = new Map( - lint.snippets.filter((s) => !s.importOnly && !s.file.startsWith('quickstarts')).map((s) => [s.id, s.step]), - ); - for (const s of ordered.filter((x) => x.kind === 'step')) { - for (const id of s.snippets) - if (lintIds.get(id) !== s.id) - fail(`snippet "${id}" is declared by step "${s.id}" but lint found it in step "${lintIds.get(id)}"`); - } - const claimed = new Set(ordered.flatMap((s) => s.snippets)); - for (const [id] of lintIds) if (!claimed.has(id)) fail(`snippet "${id}" is not run by any registered step`); - const built = buildManifest(lint, registry); - built.problems.forEach((p) => fail(`manifest: ${p}`)); - const manifestFile = path.join(SUITE_DIR, 'coverage.json'); - if (!fs.existsSync(manifestFile) || fs.readFileSync(manifestFile, 'utf8') !== serialize(built.manifest)) - fail('coverage.json is stale, run npm run docs-snippets:manifest'); - else console.log(` coverage.json is current: ${Object.keys(built.manifest.snippets).length} entries`); - console.log( - ` ${claimed.size} snippet ids run by steps, ${lint.snippets.filter((s) => s.importOnly).length} import-only snippets type checked only`, - ); - - heading('6. Selection and skip propagation (simulated, no network)'); - { - const one = plan.selection({ grep: 'publishBasicUsage' }); - const chosen = [...one.run].filter((id) => !one.prerequisites.has(id)); - console.log( - ` --grep publishBasicUsage selects ${chosen.join(', ')} and runs prerequisites ${[...one.prerequisites].join(', ') || 'none'}`, - ); - if (!chosen.includes('publishBasicUsage')) fail('--grep publishBasicUsage did not select the step'); - for (const need of registry.get('publishBasicUsage')?.needs ?? []) - if (!one.run.has(need)) fail(`prerequisite ${need} of publishBasicUsage would not run`); - for (const role of ROLES) { - const shard = plan.selection({ shard: role }); - const foreign = [...shard.prerequisites].filter((id) => registry.get(id)?.keyset !== role); - console.log( - ` shard ${role}: ${shard.run.size - shard.prerequisites.size} steps, ${foreign.length} prerequisite(s) of other roles`, - ); - } - const withDependents = ordered.find((s) => ordered.some((o) => o.needs.includes(s.id))); - if (!withDependents) fail('no step has dependents, nothing to simulate'); - else { - const result = registry.simulate([withDependents.id]); - const blocked = result.filter((r) => r.reason === `blocked-by:${withDependents.id}`); - console.log(` failing "${withDependents.id}" blocks ${blocked.length} step(s)`); - if (blocked.length === 0) fail(`failing "${withDependents.id}" blocked nothing`); - registry.availability = { open: true, pam: false, datasync: true }; - const noPam = registry.simulate([]).filter((r) => r.reason === 'missing-env:pam').length; - console.log(` without the pam keyset ${noPam} step(s) skip as missing-env:pam`); - registry.availability = { open: true, pam: true, datasync: true }; - } - } - - heading('7. Mocha binding and reporter (step bodies replaced by no-ops, no network)'); - { - const failId = ordered.find((s) => ordered.some((o) => o.needs.includes(s.id)))?.id; - const reportDir = path.join(staged.runDir, 'dry-report'); - process.env.DOCS_SNIPPETS_RUN_ID = runId; - process.env.DOCS_SNIPPETS_DRY = '1'; - if (failId) process.env.DOCS_SNIPPETS_DRY_FAIL = failId; - process.env.DOCS_SNIPPETS_REPORT_DIR = reportDir; - const savedSummary = process.env.GITHUB_STEP_SUMMARY; - delete process.env.GITHUB_STEP_SUMMARY; - const mocha = new Mocha({ ...(await mochaOptions()), timeout: 10_000 }); - mocha.addFile(path.join(staged.runDir, 'harness', 'mocha-entry.ts')); - await mocha.loadFilesAsync(); - const failures = await new Promise((resolve) => mocha.run((count) => resolve(count))); - for (const name of [ - 'DOCS_SNIPPETS_DRY', - 'DOCS_SNIPPETS_DRY_FAIL', - 'DOCS_SNIPPETS_RUN_ID', - 'DOCS_SNIPPETS_REPORT_DIR', - ]) - delete process.env[name]; - if (savedSummary) process.env.GITHUB_STEP_SUMMARY = savedSummary; - const report = JSON.parse(fs.readFileSync(path.join(reportDir, 'run.json'), 'utf8')) as { - runId: string; - summary: { passed: number; failed: number; skipped: number }; - steps: Array<{ - id: string; - status: string; - reason?: string; - snippets: string[]; - role: string; - assertions: number; - }>; - sdk: { version: string; commit: string }; - }; - console.log(` mocha reported ${failures} failure(s), run.json summary ${JSON.stringify(report.summary)}`); - if (failures !== 1) fail(`expected the one simulated failure, mocha reported ${failures}`); - if (!report.steps.some((s) => s.reason === `blocked-by:${failId}`)) - fail('run.json shows no step blocked by the simulated failure'); - if (report.runId !== runId) fail(`run.json carries run id ${report.runId}, expected ${runId}`); - // The commit is `unknown` in a copy without git history (a source download), the version always exists. - if (!report.sdk.version || !report.sdk.commit) fail('run.json lacks the SDK version or commit'); - const sample = report.steps.find((s) => s.status === 'passed' && s.snippets.length > 0); - if (!sample || !sample.role || !sample.snippets.includes(sample.id) || sample.assertions < 1) - fail('run.json step records lack role, snippet ids or assertion counts'); - const skippedBlocked = report.steps.filter((s) => s.reason?.startsWith('blocked-by:')).length; - console.log(` ${skippedBlocked} step(s) skipped as blocked-by the simulated failure of ${failId}, transitively`); - if (/0123abcd|AAAAAAAAAA/.test(fs.readFileSync(path.join(reportDir, 'run.json'), 'utf8'))) - fail('run.json leaks a key-shaped value'); - else console.log(' run.json has role, snippet ids, assertion counts, SDK version and commit, and is redacted'); - } - - heading('8. Run policy, preflight messages, redaction, run ids'); - { - const keys = { PUBLISH_KEY: 'pub-c-x', SUBSCRIBE_KEY: 'sub-c-x' }; - const base = loadEnv(keys); - const ci = loadEnv({ ...keys, CI: 'true' }); - const none = { skips: [] }; - if (outcomeProblems(none).length !== 0) fail('policy: a run without skips must pass'); - const reasons = ['blocked-by:preflightPresence', 'needs-feature:push-notifications', 'needs-feature:datasync-events', 'missing-env:pam']; - for (const reason of reasons) { - const result = outcomeProblems({ skips: [{ id: 'x', reason }] }); - if (result.length !== 1 || !result[0].includes(reason)) - fail(`policy: a ${reason} skip must fail the run and name its reason`); - } - // Every skip fails, locally and in CI alike. The loaded env has no part in it any more. - if (outcomeProblems({ skips: [{ id: 'a', reason: 'needs-feature:datasync-events' }, { id: 'b', reason: 'needs-feature:datasync-events' }] }).length !== 1) - fail('policy: skips must come out as one grouped message'); - if (launchProblems(ci, {}, { grep: 'x' }).length !== 1) fail('policy: --grep must fail in CI'); - for (const name of ['DOCS_SNIPPETS_ONLY', 'DOCS_SNIPPETS_MUTATE', 'DOCS_SNIPPETS_INJECT_FAILURE']) { - if (launchProblems(ci, { [name]: 'x' }).length !== 1) fail(`policy: ${name} must fail in CI`); - if (launchProblems(base, { [name]: 'x' }).length !== 0) fail(`policy: ${name} is fine locally`); - } - if (launchProblems(loadEnv({ CI: 'true', DOCS_SNIPPETS_LOG_LEVEL: 'debug' }), {}).length !== 1) - fail('policy: debug log level must be refused in CI'); - console.log(' run policy: any skip fails the run, subset options and debug logging behave as documented'); - - const cases: Array<[string, unknown, string]> = [ - [ - 'network', - { status: { category: 'PNNetworkIssuesCategory' }, message: 'REST API request processing error' }, - 'Network problem', - ], - [ - 'bad key', - { status: { statusCode: 400, errorData: { message: 'Invalid Subscribe Key' } } }, - 'keys were rejected', - ], - ['feature off', { status: { statusCode: 403, errorData: { message: 'Forbidden' } } }, 'does not have Presence'], - [ - 'bad request', - { status: { statusCode: 400, errorData: { message: 'Invalid arguments' } } }, - 'not a missing feature', - ], - ]; - for (const [label, error, expect] of cases) { - const message = preflightMessage(error, 'open', 'Presence', 'Presence'); - if (!message.includes(expect)) fail(`preflight message for ${label} should contain "${expect}", got: ${message}`); - } - if (classifyPreflightError({ code: 'ENOTFOUND' }).cause !== 'network') - fail('preflight: ENOTFOUND is a network problem'); - console.log(' preflight errors: network, bad key, feature off and bad request read differently'); - - const fake = 'pub-c-0123abcd-0123-4abc-8def-0123456789ab'; - const masked = redact( - `config { publishKey: '${fake}', secretKey: 'sec-c-QWERTYUIOPASDFGHJKL1234', token: 'p0F2AkF0Gl${'A'.repeat(30)}' }`, - {}, - ); - if (masked.includes('0123abcd') || masked.includes('QWERTYUIOP') || masked.includes('AAAAAAAAAA')) - fail('redaction left a secret'); - else console.log(' redaction masks keys and tokens'); - - const id = randomRunId(); - if ((runIdAgeMs(id) ?? Infinity) > 10_000) fail(`a fresh run id (${id}) should decode to a moment just now`); - const shardIds = new Set(ROLES.map((r) => ciRunId({ GITHUB_RUN_ID: '123456', GITHUB_RUN_ATTEMPT: '1' }, r))); - const rerunIds = new Set(ROLES.map((r) => ciRunId({ GITHUB_RUN_ID: '123456', GITHUB_RUN_ATTEMPT: '2' }, r))); - if (shardIds.size !== 3) fail('CI run ids of the three shards of one attempt must differ'); - if ([...shardIds].some((x) => !/^[0-9a-z]{6}$/.test(x))) fail('CI run ids must be six base36 characters'); - console.log( - ` run ids carry their start time, CI shards get distinct ids (${[...shardIds].join(', ')}; attempt 2: ${[...rerunIds].join(', ')})`, - ); - } - - heading('9. Reporter self-test (no key value in any written file)'); - { - // Loaded key values (if a local env file or real variables are set) plus fake ones, put into every free text field. - const env = loadEnv(); - addSecrets(['fake-secret-value-0001', 'sub-c-ffffffff-ffff-4fff-8fff-ffffffffffff']); - const secrets = knownSecrets(); - const { buildReport, writeReport } = (await import( - pathToFileURL(path.join(SUITE_DIR, 'harness', 'reporter.cjs')).href - )) as { buildReport: (s: unknown[], c: unknown) => unknown; writeReport: (r: unknown, dir: string) => string[] }; - const text = `config ${secrets.join(' ')} pub-c-0123abcd-0123-4abc-8def-0123456789ab`; - const steps = [ - { - id: 'leaky', - snippets: ['leaky'], - page: 'x', - phase: 'P4', - kind: 'step', - role: 'open', - prerequisite: false, - status: 'failed', - error: text, - note: text, - durationMs: 1, - assertions: 1, - output: [text], - failedAssertion: { label: text, expected: text, actual: text }, - }, - { - id: 'skipped', - snippets: [], - page: 'x', - phase: 'P4', - kind: 'check', - role: 'pam', - prerequisite: false, - status: 'skipped', - reason: 'needs-feature:x', - note: text, - durationMs: 0, - assertions: 0, - }, - ]; - const report = buildReport(steps, { - plan: [], - selection: { shard: null, grep: text }, - runId, - startedAt: '', - finishedAt: '', - durationMs: 1, - sdk: { version: '0', commit: '0' }, - }); - const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'docs-snippets-report-')); - const savedSummary = process.env.GITHUB_STEP_SUMMARY; - const summary = path.join(dir, 'step-summary.md'); - process.env.GITHUB_STEP_SUMMARY = summary; - const written = writeReport(report, dir); - if (savedSummary) process.env.GITHUB_STEP_SUMMARY = savedSummary; - else delete process.env.GITHUB_STEP_SUMMARY; - let leaks = 0; - for (const file of written) { - const content = fs.readFileSync(file, 'utf8'); - if (secrets.some((s) => content.includes(s)) || content.includes('0123abcd')) { - leaks += 1; - console.error(`leak in ${path.basename(file)}`); - } - } - if (leaks > 0) fail('the reporter leaks key values'); - else - console.log( - ` ${written.length} files written (run.json, summary.md, step summary), ${secrets.length} known secret value(s) checked, no leak (${env.open || env.pam || env.datasync ? 'loaded keys included' : 'no keys loaded, fake values only'})`, - ); - // The per snippet id summary (snippet-summary.cjs): right status per id, and the same leak rule as the other files. - { - const { renderSnippetSummary } = (await import(pathToFileURL(path.join(SUITE_DIR, 'harness', 'snippet-summary.cjs')).href)) as { - renderSnippetSummary: (r: unknown[], m: unknown) => string; - }; - const manifest = { - leaky: { file: 'x.ts', step: 'leaky', role: 'open', phase: 'P4', mode: 'live', reason: null }, - ghost: { file: 'x.ts', step: 'ghost', role: 'open', phase: 'P4', mode: 'live', reason: null }, - typed: { file: 'b.ts', step: null, role: null, phase: null, mode: 'compile-only', reason: 'browser-only' }, - }; - const md = redact(renderSnippetSummary([{ shard: 'open', report }], manifest)); - if (!/\| `leaky` \| FAILED \|/.test(md) || !/\| `ghost` \| NOT RUN \|/.test(md) || !/\| `typed` \| type checked only/.test(md)) - fail('snippet summary: a failed, a not-run and a compile-only id must each show their own result'); - if (!/\| 0 \| 1 \| 0 \| 1 \| 1 \| 0 \|/.test(md)) fail('snippet summary: counts row is wrong'); - if (secrets.some((x) => md.includes(x)) || md.includes('0123abcd')) fail('the snippet summary leaks key values'); - else console.log(' snippet summary: failed, not-run and type-checked ids shown, no leak'); - } - fs.rmSync(dir, { recursive: true, force: true }); - } - - heading('10. No key in any output (stdout, stderr, crashes, shortened values)'); - { - // A child process with a fake secret that has every character a URL encodes. It writes the secret in each way the - // security review named: split across writes, as a Uint8Array, URL-encoded, inside an object, in a URL-encoded token, - // and finally in an uncaught exception outside Mocha. Nothing of it may reach the child's stdout or stderr. - const secret = 'fake-Secret/Key+With=Specials0123456789'; - const token = `p0F2AkF0Gl${'A'.repeat(12)}%2B${'B'.repeat(20)}`; - // A token granted with authorizedUserId has one more CBOR field, so it starts with `qEF2` instead of `p0F2`. - const userToken = `qEF2AkF0GmkKbpZDdHRsAUNyZXOl${'C'.repeat(24)}`; - const script = ` - const { installRedaction } = require(${JSON.stringify(path.join(SUITE_DIR, 'harness', 'redact.cjs'))}); - installRedaction(); - const s = process.env.DS_SECRET_KEY; - process.stdout.write('split ' + s.slice(0, 9)); - process.stdout.write(s.slice(9) + '\\n'); - process.stdout.write(new TextEncoder().encode('bytes ' + s + '\\n')); - console.log('url https://x/?k=' + encodeURIComponent(s)); - console.log('object', { config: { secretKey: s } }); - console.error('token ${token}'); - console.error('user token ${userToken} encoded ' + encodeURIComponent('${userToken}/x+y')); - setTimeout(() => { throw new Error('boom ' + s); }, 10); - `; - const run = spawnSync(process.execPath, ['-e', script], { - env: { PATH: process.env.PATH, DS_SECRET_KEY: secret }, - encoding: 'utf8', - }); - const output = `${run.stdout}${run.stderr}`; - const forms = [ - secret, - encodeURIComponent(secret), - encodeURIComponent(secret).toLowerCase(), - secret.slice(9), - token.slice(3), - userToken.slice(4), - ]; - const leaked = forms.filter((f) => output.includes(f)).length; - if (leaked > 0) fail(`a child process printed ${leaked} form(s) of a secret`); - if (!/split \[redacted\]/.test(output) || !/bytes \[redacted\]/.test(output) || !/k=\[redacted\]/.test(output)) - fail('split, byte array or URL-encoded output was not redacted in place'); - if (!/token \[token\]/.test(output)) fail('a URL-encoded token was not masked'); - if (!/user token \[token\] encoded \[token\]/.test(output)) - fail('a token with an authorized user id (qEF2) was not masked'); - if (run.status !== 1 || !/boom \[redacted\]/.test(output)) - fail(`an uncaught exception must print redacted and exit 1, got exit ${run.status}`); - // Shortened values: the recorder cuts long values for the report. It must redact before it cuts. - addSecrets([secret]); - const { short } = await import('./recorder.js'); - const cut = short({ config: { secretKey: secret } }, 30); - if (cut.includes(secret.slice(0, 12))) fail('the recorder cut a secret before redacting it'); - if (leaked === 0) - console.log( - ' split writes, byte arrays, URL-encoded keys and tokens, objects, uncaught exceptions and shortened values are all redacted', - ); - } - - heading('11. The "keyset closed" probe counts only a 403 as refused'); - { - const { tokenlessAccess } = await import('./revoke.js'); - const denied = Object.assign(new Error('Forbidden'), { - status: { category: 'PNAccessDeniedCategory', statusCode: 403 }, - }); - const network = Object.assign(new Error('fetch failed'), { status: { category: 'PNNetworkIssuesCategory' } }); - const timeout = Object.assign(new Error('timed out'), { status: { category: 'PNTimeoutCategory' } }); - const server = Object.assign(new Error('Internal'), { status: { statusCode: 500 } }); - const bad = Object.assign(new Error('Bad request'), { status: { statusCode: 400 } }); - // A stand-in client: each call fails with the given error, or succeeds when it is undefined. - const fake = (publishError?: Error, readError?: Error) => - ({ - publish: async () => { - if (publishError) throw publishError; - return { timetoken: '1' }; - }, - fetchMessages: async () => { - if (readError) throw readError; - return { channels: {} }; - }, - }) as never; - const cases: Array<[string, Error | undefined, Error | undefined, 'closed' | 'open' | 'unknown']> = [ - ['both refused with 403', denied, denied, 'closed'], - ['read open (application-level read grant)', denied, undefined, 'open'], - ['read network error', denied, network, 'unknown'], - ['read timeout', denied, timeout, 'unknown'], - ['read 500', denied, server, 'unknown'], - ['read 400', denied, bad, 'unknown'], - ['publish network error', network, denied, 'unknown'], - ]; - for (const [label, publishError, readError, expected] of cases) { - const seen = await tokenlessAccess(fake(publishError, readError), 'probe'); - const verdict = seen.open.length > 0 ? 'open' : seen.unknown.length > 0 ? 'unknown' : 'closed'; - if (verdict !== expected) fail(`closure probe, ${label}: expected ${expected}, got ${verdict}`); - } - console.log( - ` ${cases.length} cases: only a 403 on both probes reads as closed, network errors, timeouts, 5xx and 400 read as unknown`, - ); - } - - heading('Result'); - console.log(failed ? 'dry run FAILED' : 'dry run ok'); - return failed ? 1 : 0; -} - -/** `docs-snippets:manifest`: writes coverage.json, or with --check verifies it. Reports docs ids either way. */ -async function manifest(): Promise { - const lint = lintApp(SUITE_DIR); - if (lint.errors.length > 0) { - printLint(lint); - return 1; - } - const runId = randomRunId(); - const staged = stage(SUITE_DIR, runId); - const plan = await stagedPlan(staged.runDir, loadEnv({}), runId); - fs.rmSync(staged.runDir, { recursive: true, force: true }); - const { manifest: built, problems } = buildManifest(lint, plan.registry); - const file = path.join(SUITE_DIR, 'coverage.json'); - const check = flags.has('--check'); - if (check) { - if (!fs.existsSync(file)) problems.push('coverage.json does not exist, run npm run docs-snippets:manifest'); - else if (fs.readFileSync(file, 'utf8') !== serialize(built)) - problems.push('coverage.json is stale, run npm run docs-snippets:manifest'); - } else if (problems.length === 0) { - fs.writeFileSync(file, serialize(built)); - } - const modes = Object.values(built.snippets).reduce>( - (acc, e) => ({ ...acc, [e.mode]: (acc[e.mode] ?? 0) + 1 }), - {}, - ); - console.log( - `manifest: ${Object.keys(built.snippets).length} entries (${Object.entries(modes) - .map(([m, n]) => `${m} ${n}`) - .join(', ')})${check ? '' : problems.length ? '' : ', written to docs-snippets/coverage.json'}`, - ); - - const docsDir = process.env.DOCS_SNIPPETS_DOCS_DIR || path.resolve(SUITE_DIR, '..', '..', 'documentation'); - const docs = docsIds(docsDir); - if (!docs) - console.log(`docs ids: skipped, ${docsDir}/docs/sdks/javascript does not exist (set DOCS_SNIPPETS_DOCS_DIR)`); - else { - const missing = [...docs.ids.keys()].filter((id) => !built.snippets[id]).sort(); - const unused = Object.keys(built.snippets) - .filter((id) => !docs.ids.has(id)) - .sort(); - console.log( - `docs ids: ${docs.ids.size} ids in ${docs.files} files of ${docsDir}/docs/sdks/javascript (report only)`, - ); - console.log( - ` docs ids missing from the manifest (${missing.length}): ${missing.map((id) => `${id} (${docs.ids.get(id)?.[0]})`).join(', ') || 'none'}`, - ); - console.log(` manifest ids no docs page uses (${unused.length}): ${unused.join(', ') || 'none'}`); - } - problems.forEach((p) => console.error(`manifest: ${p}`)); - return problems.length > 0 ? 1 : 0; -} - -async function main(): Promise { - // Every command masks keys in its output, also the ones that only load keys for a self-test. - installRedaction(); - switch (command) { - case 'live': - return live(); - case 'dry': - return dryRun(); - case 'manifest': - return manifest(); - case 'stage': { - const result = stage(SUITE_DIR, loadEnv().runId ?? randomRunId(), { - mutate: process.env.DOCS_SNIPPETS_MUTATE || undefined, - }); - console.log(`staged ${result.filesCopied} files, ${result.rewrites.length} rewrites into ${result.runDir}`); - result.problems.forEach((p) => console.error(p)); - return result.problems.length > 0 ? 1 : 0; - } - case 'lint': { - const result = lintApp(SUITE_DIR); - printLint(result); - return result.errors.length > 0 ? 1 : 0; - } - case 'janitor': { - installRedaction(); - const env = loadEnv(); - console.log(describeRoles(env)); - if (!env.open && !env.pam && !env.datasync) return env.ci ? 1 : 0; - // A fresh run id of its own: only leftovers of other runs older than six hours are swept, plus the grants. - const report = await runJanitor(env, randomRunId(), new Date(Date.now() - 60 * 60 * 1000)); - report.log.forEach((l) => console.log(l)); - report.critical.forEach((c) => console.error(`CRITICAL ${c}`)); - return report.critical.length > 0 ? 1 : 0; - } - default: - console.error(`unknown command "${command}". Use live, dry, manifest, janitor, stage or lint.`); - return 2; - } -} - -main().then( - (code) => exitWhenIdle(code), - (error) => { - console.error(error); - process.exit(1); - }, -); diff --git a/docs-snippets/harness/env.ts b/docs-snippets/harness/env.ts deleted file mode 100644 index a91f39505..000000000 --- a/docs-snippets/harness/env.ts +++ /dev/null @@ -1,294 +0,0 @@ -/** - * The environment contract of the suite: keysets per role, the admin key, run id, log level and the CI flag. - * - * One fixed set of names. The harness reads only these, wherever the value came from: - * - * role env names used for - * open SUBSCRIBE_KEY, PUBLISH_KEY most samples (no Access Manager) - * pam PAM_SUBSCRIBE_KEY, PAM_PUBLISH_KEY, PAM_SECRET_KEY Access Manager samples and 403 branches - * datasync DS_SUBSCRIBE_KEY, DS_PUBLISH_KEY, DS_SECRET_KEY DataSync samples - * admin PUBNUB_ADMIN_API_KEY DataSync class setup and keyset feature reads - * - * Two sources fill them. In CI the workflow sets them from CI secrets. Locally an env file fills them: `docs-snippets/.env.local` - * by default (copy `.env.example`), or the file `DOCS_SNIPPETS_ENV_FILE` names. It is read in memory and never copied or - * rewritten. It may use the names above or the older `PN_DOCS_*` names, which are mapped as `FILE_ALIASES` says. Real environment variables always win, so a - * CI run never reads a stray local file value. - * - * Output rule: nothing here prints a value or a prefix of one. `describeRoles` says "set" or "missing" per role. - */ -import fs from 'node:fs'; -import path from 'node:path'; -import { fileURLToPath } from 'node:url'; -import { addSecrets } from './redact.cjs'; - -/** - * The default local env file: `docs-snippets/.env.local`, ignored by git. Copy `docs-snippets/.env.example` there and fill in - * your own keys. `DOCS_SNIPPETS_ENV_FILE` points at another file instead. The CLI resolves this once and passes it on, because - * the staged copy under `.run//` sits in another directory. - */ -export const DEFAULT_ENV_FILE = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..', '.env.local'); - -/** The roles a step can run as. `pam` and `datasync` may point at the same keyset (they do locally). */ -export type Role = 'open' | 'pam' | 'datasync'; -export const ROLES: readonly Role[] = ['open', 'pam', 'datasync']; - -/** Key names per role, in the order publish, subscribe, secret. */ -export const KEY_NAMES = { - open: { publish: 'PUBLISH_KEY', subscribe: 'SUBSCRIBE_KEY' }, - pam: { publish: 'PAM_PUBLISH_KEY', subscribe: 'PAM_SUBSCRIBE_KEY', secret: 'PAM_SECRET_KEY' }, - datasync: { publish: 'DS_PUBLISH_KEY', subscribe: 'DS_SUBSCRIBE_KEY', secret: 'DS_SECRET_KEY' }, -} as const; -export const ADMIN_KEY = 'PUBNUB_ADMIN_API_KEY'; - -/** Every env name that holds a secret. The redaction (redact.cjs) masks their values. */ -export const SECRET_NAMES: readonly string[] = [ - KEY_NAMES.open.publish, - KEY_NAMES.open.subscribe, - KEY_NAMES.pam.publish, - KEY_NAMES.pam.subscribe, - KEY_NAMES.pam.secret, - KEY_NAMES.datasync.publish, - KEY_NAMES.datasync.subscribe, - KEY_NAMES.datasync.secret, - ADMIN_KEY, -]; - -/** Control variables. All start with `DOCS_SNIPPETS_`. */ -export const ENV_VARS = { - envFile: 'DOCS_SNIPPETS_ENV_FILE', - runId: 'DOCS_SNIPPETS_RUN_ID', - logLevel: 'DOCS_SNIPPETS_LOG_LEVEL', - shard: 'DOCS_SNIPPETS_SHARD', - openKeysetId: 'DOCS_SNIPPETS_OPEN_KEYSET_ID', - pamKeysetId: 'DOCS_SNIPPETS_PAM_KEYSET_ID', - datasyncKeysetId: 'DOCS_SNIPPETS_DS_KEYSET_ID', -} as const; - -/** - * The only place that knows the older `PN_DOCS_*` env names. A local env file that uses them works unchanged: - * its open keyset fills `open`, its secured keyset fills both `pam` and `datasync`. - */ -export const FILE_ALIASES: Readonly> = { - PUBLISH_KEY: ['PN_DOCS_OPEN_PUBLISH_KEY'], - SUBSCRIBE_KEY: ['PN_DOCS_OPEN_SUBSCRIBE_KEY'], - PAM_PUBLISH_KEY: ['PN_DOCS_SECURED_PUBLISH_KEY'], - PAM_SUBSCRIBE_KEY: ['PN_DOCS_SECURED_SUBSCRIBE_KEY'], - PAM_SECRET_KEY: ['PN_DOCS_SECURED_SECRET_KEY'], - DS_PUBLISH_KEY: ['PN_DOCS_SECURED_PUBLISH_KEY'], - DS_SUBSCRIBE_KEY: ['PN_DOCS_SECURED_SUBSCRIBE_KEY'], - DS_SECRET_KEY: ['PN_DOCS_SECURED_SECRET_KEY'], -}; - -export interface OpenKeyset { - publishKey: string; - subscribeKey: string; -} - -export interface SecuredKeyset extends OpenKeyset { - secretKey: string; -} - -export type LogLevelName = 'trace' | 'debug' | 'info' | 'warn' | 'error' | 'none'; - -export interface Env { - open?: OpenKeyset; - pam?: SecuredKeyset; - datasync?: SecuredKeyset; - /** Admin API key (Service Integration). Optional. Only provisioning and the read-only preflight use it. */ - adminKey?: string; - /** Missing env names per role. Empty when the role is usable. */ - missing: Record; - /** True when `pam` and `datasync` are the same keyset. Compared here, never printed. */ - sharedSecured: boolean; - runId?: string; - logLevel: LogLevelName; - ci: boolean; - /** `DOCS_SNIPPETS_SHARD=`: the role whose steps this run selects. Undefined means all roles. */ - shard?: Role; - shardProblem?: string; - /** Where the env file came from, for the "set or missing" line. Never its content. */ - envFile?: string; - /** - * Admin Portal keyset ids per role, for the read-only feature preflight (keyset-features.ts). Not secrets. The preflight - * confirms that each id's subscribe key equals the role's key before it relies on the id. - */ - keysetIds: Partial>; -} - -/** Minimal env file reader. Values stay in memory. Real environment variables win (see `envSource`). */ -export function readDotEnv(file: string): Record { - if (!file || !fs.existsSync(file)) return {}; - const out: Record = {}; - for (const raw of fs.readFileSync(file, 'utf8').split('\n')) { - const line = raw.trim(); - if (!line || line.startsWith('#')) continue; - const eq = line.indexOf('='); - if (eq === -1) continue; - const key = line - .slice(0, eq) - .trim() - .replace(/^export\s+/, ''); - let value = line.slice(eq + 1).trim(); - if ((value.startsWith('"') && value.endsWith('"')) || (value.startsWith("'") && value.endsWith("'"))) { - value = value.slice(1, -1); - } - out[key] = value; - } - return out; -} - -/** Maps a local env file onto the suite's names. A name the file sets directly wins over an alias. */ -function mapFile(file: Record): Record { - const out: Record = {}; - for (const [name, aliases] of Object.entries(FILE_ALIASES)) { - for (const alias of aliases) if (file[alias] && !out[name]) out[name] = file[alias]; - } - for (const name of [...SECRET_NAMES, ...Object.values(ENV_VARS)]) if (file[name]) out[name] = file[name]; - return out; -} - -/** File values (mapped) overlaid by the real environment. The env file path itself comes from the real environment only. */ -export function envSource(vars: Record = process.env): Record { - const file = vars[ENV_VARS.envFile]?.trim() || (fs.existsSync(DEFAULT_ENV_FILE) ? DEFAULT_ENV_FILE : undefined); - const mapped = file ? mapFile(readDotEnv(path.resolve(file))) : {}; - const out: Record = { ...mapped }; - for (const [key, value] of Object.entries(vars)) if (value !== undefined && value !== '') out[key] = value; - return out; -} - -function pick(source: Record, names: string[]): { values: string[]; missing: string[] } { - const values: string[] = []; - const missing: string[] = []; - for (const name of names) { - const value = source[name]?.trim(); - if (value) values.push(value); - else missing.push(name); - } - return { values, missing }; -} - -/** - * Reads and validates the environment. Pure with respect to `source`, which is how the dry run builds CI, budget and - * feature scenarios without touching the real environment. Registers every loaded secret with the redaction. - */ -export function loadEnv(source: Record = envSource()): Env { - const open = pick(source, [KEY_NAMES.open.publish, KEY_NAMES.open.subscribe]); - const pam = pick(source, [KEY_NAMES.pam.publish, KEY_NAMES.pam.subscribe, KEY_NAMES.pam.secret]); - const ds = pick(source, [KEY_NAMES.datasync.publish, KEY_NAMES.datasync.subscribe, KEY_NAMES.datasync.secret]); - const adminKey = source[ADMIN_KEY]?.trim() || undefined; - addSecrets([...open.values, ...pam.values, ...ds.values, adminKey]); - - const ci = Boolean(source.CI || source.GITHUB_ACTIONS); - // An unknown level falls back to `none`, so a typo cannot make the SDK log its configuration (keys included). - const rawLevel = (source[ENV_VARS.logLevel] ?? '').toLowerCase(); - const levels: LogLevelName[] = ['trace', 'debug', 'info', 'warn', 'error', 'none']; - const logLevel = (levels as string[]).includes(rawLevel) ? (rawLevel as LogLevelName) : 'none'; - - const rawShard = source[ENV_VARS.shard]?.trim().toLowerCase(); - let shard: Role | undefined; - let shardProblem: string | undefined; - if (rawShard) { - if ((ROLES as readonly string[]).includes(rawShard)) shard = rawShard as Role; - else shardProblem = `${ENV_VARS.shard}="${rawShard}" is not one of ${ROLES.join(', ')}`; - } - - let runId = source[ENV_VARS.runId]?.trim().toLowerCase(); - if (runId && !/^[0-9a-z]{6}$/.test(runId)) { - // Reduce any other value to six base36 characters, so names stay valid for DataSync. - runId = Number.parseInt(runId.replace(/\D/g, '') || '0', 10) - .toString(36) - .padStart(6, '0') - .slice(-6); - } - - const secured = (r: { values: string[]; missing: string[] }): SecuredKeyset | undefined => - r.missing.length === 0 ? { publishKey: r.values[0], subscribeKey: r.values[1], secretKey: r.values[2] } : undefined; - const pamKeys = secured(pam); - const dsKeys = secured(ds); - return { - open: open.missing.length === 0 ? { publishKey: open.values[0], subscribeKey: open.values[1] } : undefined, - pam: pamKeys, - datasync: dsKeys, - adminKey, - missing: { open: open.missing, pam: pam.missing, datasync: ds.missing }, - sharedSecured: Boolean( - pamKeys && dsKeys && pamKeys.subscribeKey === dsKeys.subscribeKey && pamKeys.publishKey === dsKeys.publishKey, - ), - runId, - logLevel, - ci, - shard, - shardProblem, - envFile: source[ENV_VARS.envFile]?.trim() || undefined, - keysetIds: { - open: source[ENV_VARS.openKeysetId]?.trim() || undefined, - pam: source[ENV_VARS.pamKeysetId]?.trim() || undefined, - datasync: source[ENV_VARS.datasyncKeysetId]?.trim() || undefined, - }, - }; -} - -/** Secured keysets to guard, without duplicates (pam and datasync may be one keyset). */ -export function securedKeysets(env: Env): Array<{ role: 'pam' | 'datasync'; keys: SecuredKeyset }> { - const out: Array<{ role: 'pam' | 'datasync'; keys: SecuredKeyset }> = []; - if (env.pam) out.push({ role: 'pam', keys: env.pam }); - if (env.datasync && !env.sharedSecured) out.push({ role: 'datasync', keys: env.datasync }); - return out; -} - -/** The loader's only output: "set" or "missing" per role. Never a value, never a prefix of one. */ -export function describeRoles(env: Env): string { - const parts = ROLES.map((role) => `${role} ${env[role] ? 'set' : 'missing'}`); - parts.push(`admin ${env.adminKey ? 'set' : 'missing'}`); - const from = env.envFile ? `, env file ${env.envFile}` : ''; - return `keysets: ${parts.join(', ')}${env.sharedSecured ? ' (pam and datasync are one keyset)' : ''}${from}`; -} - -/** - * Run ids carry their start time, so the janitor can tell a crashed run from a running one by pattern alone. - * Five base36 characters count 4 second ticks since 2026-01-01 and one random character keeps runs of the same tick apart. - * In CI a seventh character is not possible (DataSync and every name assume six), so CI runs mix the GitHub run id, attempt - * and shard into the random part (see `ciRunId`). - */ -const RUN_ID_EPOCH = Date.UTC(2026, 0, 1); -const RUN_ID_TICK_MS = 4_000; -const RUN_ID_MAX_AGE_MS = 365 * 24 * 60 * 60 * 1000; - -export function randomRunId(now: number = Date.now(), salt?: string): string { - const ticks = Math.max(0, Math.floor((now - RUN_ID_EPOCH) / RUN_ID_TICK_MS)) % 36 ** 5; - let last = Math.floor(Math.random() * 36); - if (salt) { - let hash = 0; - for (const ch of salt) hash = (hash * 31 + ch.charCodeAt(0)) >>> 0; - last = hash % 36; - } - return ticks.toString(36).padStart(5, '0') + last.toString(36); -} - -/** - * Run id for a CI job: the time tick plus one character derived from GITHUB_RUN_ID, GITHUB_RUN_ATTEMPT and the shard. Three - * shards start in the same tick, so the shard alone must separate them: the salt guarantees different characters for the - * three shard names of one run attempt (checked by the dry run). - */ -export function ciRunId( - vars: Record, - shard: string | undefined, - now: number = Date.now(), -): string { - const base = randomRunId(now).slice(0, 5); - const shardIndex = shard ? Math.max(0, (ROLES as readonly string[]).indexOf(shard)) : 3; - let hash = 0; - for (const ch of `${vars.GITHUB_RUN_ID ?? ''}.${vars.GITHUB_RUN_ATTEMPT ?? ''}`) - hash = (hash * 31 + ch.charCodeAt(0)) >>> 0; - // 9 slots of 4 characters: the run and attempt pick a slot, the shard picks a character inside it. - const slot = hash % 9; - return base + (slot * 4 + shardIndex).toString(36); -} - -/** Age of a run id in ms, or undefined when it does not decode to the last year. Hand-written ids are never swept. */ -export function runIdAgeMs(runId: string, now: number = Date.now()): number | undefined { - if (!/^[0-9a-z]{6}$/.test(runId)) return undefined; - const started = RUN_ID_EPOCH + Number.parseInt(runId.slice(0, 5), 36) * RUN_ID_TICK_MS; - const age = now - started; - return age >= -RUN_ID_TICK_MS && age <= RUN_ID_MAX_AGE_MS ? Math.max(0, age) : undefined; -} diff --git a/docs-snippets/harness/guard.ts b/docs-snippets/harness/guard.ts deleted file mode 100644 index 3fca0e6ec..000000000 --- a/docs-snippets/harness/guard.ts +++ /dev/null @@ -1,56 +0,0 @@ -/** - * Failure-on-unhandled-rejection guard. Snippets let errors throw, so a rejection nobody awaits means a step - * did something the harness did not assert. The guard records it against the running step and the step runner - * fails that step after the body returns. - */ - -// Module-level on purpose: the process-wide `unhandledRejection` event has no idea which step is running, so the step runner -// drains this list before and after each step to assign blame. -const recorded: Array<{ kind: string; error: unknown }> = []; -let installed = false; - -/** Idempotent. Without a handler Node would crash the whole run on an unhandled rejection. Here it fails one step. */ -export function installGuard(): void { - if (installed) return; - installed = true; - process.on('unhandledRejection', (error) => { - recorded.push({ kind: 'unhandledRejection', error }); - }); -} - -/** Returns and clears errors recorded since the last call. */ -export function drainGuard(): Array<{ kind: string; error: unknown }> { - return recorded.splice(0, recorded.length); -} - -/** One line for reports. Falls back to JSON for non-Error throws (the SDK sometimes rejects with a plain status object). */ -export function describeError(error: unknown): string { - if (error instanceof Error) return `${error.name}: ${error.message}`; - try { - return typeof error === 'string' ? error : JSON.stringify(error); - } catch { - return String(error); - } -} - -// The subset of the SDK's rejection shape that carries a reason. -interface SdkStatus { - category?: unknown; - statusCode?: unknown; - errorData?: { error_message?: unknown; message?: unknown; error?: { message?: unknown } } | unknown; - message?: unknown; -} - -/** - * The SDK reports most service failures as "REST API request processing error, check status for details". - * This appends the category, status code and the service's own message, so a report line explains itself. - */ -export function explainError(error: unknown): void { - if (!(error instanceof Error)) return; - const status = (error as Error & { status?: SdkStatus }).status; - if (!status || typeof status !== 'object') return; - const data = (status.errorData ?? {}) as { error_message?: unknown; message?: unknown; error?: { message?: unknown } }; - const detail = data.error_message ?? data.message ?? data.error?.message ?? status.message; - const parts = [String(status.category ?? ''), String(status.statusCode ?? '')].filter(Boolean).join(' '); - error.message = `${error.message} [${parts}${detail ? `: ${String(detail)}` : ''}]`; -} diff --git a/docs-snippets/harness/janitor-datasync.ts b/docs-snippets/harness/janitor-datasync.ts deleted file mode 100644 index 2c28e9b0c..000000000 --- a/docs-snippets/harness/janitor-datasync.ts +++ /dev/null @@ -1,134 +0,0 @@ -import PubNub from 'pubnub'; -import { hasRunSuffix } from '../fixtures/names.js'; -import type { Env } from './env.js'; - -/** - * Janitor for DataSync objects on the datasync keyset (called from janitor.ts). - * - * 1. remove this run's memberships, relationships, entities, users and channels (ids carry `-r`) - * 2. sweep objects with any run suffix that are older than six hours (crashed earlier runs) - * 3. report residue: objects created during the run window without a run suffix - * - * It uses the secret key client, lists every kind with paging and removes in dependency order. Classes that do - * not exist yet (`product`, `ProductOwner`) are skipped quietly. Objects that carry a run suffix but belong to the - * Access Manager token tests are cleaned the same way, because they live on the same keyset. - */ - -// Same six hour rule as janitor.ts. MAX_PAGES bounds a sweep if the service keeps returning cursors. -const SIX_HOURS = 6 * 60 * 60 * 1000; -const MAX_PAGES = 50; - -export interface DataSyncJanitorReport { - log: string[]; - residue: string[]; -} - -interface Listed { - id: string; - createdAt?: string; -} - -interface Page { - data: Listed[]; - meta?: { has_next?: boolean; next_cursor?: string | null }; -} - -async function listAll(fetchPage: (cursor?: string) => Promise): Promise { - const out: Listed[] = []; - let cursor: string | undefined; - for (let page = 0; page < MAX_PAGES; page += 1) { - const response = await fetchPage(cursor); - out.push(...response.data.map((o) => ({ id: o.id, createdAt: o.createdAt }))); - cursor = response.meta?.next_cursor ?? undefined; - if (!response.meta?.has_next || !cursor) break; - } - return out; -} - -// DS-0100 is the service code for an unknown class. `product` and `ProductOwner` are provisioned by hand (provision/), so a -// keyset without them is normal and not a janitor failure. -function missingClass(error: unknown): boolean { - const status = (error as { status?: { statusCode?: number; errorData?: unknown } } | undefined)?.status; - return status?.statusCode === 404 && /DS-0100/.test(JSON.stringify(status.errorData ?? {})); -} - -export async function sweepDataSync(env: Env, runId: string, since: Date): Promise { - const log: string[] = []; - const residue: string[] = []; - if (!env.datasync) { - log.push('datasync keyset not configured, no DataSync objects to clean'); - return { log, residue }; - } - const client = new PubNub({ ...env.datasync, userId: `janitor-ds-r${runId}`, logLevel: PubNub.LogLevel.None, heartbeatInterval: 0 }); - const ds = client.dataSync; - const now = Date.now(); - const mine = new RegExp(`-r${runId}(?=\\.|$)`); - - // Delete this run's objects and any old suffixed ones. Everything else is left alone (a human may own it). - const doomed = (o: Listed): boolean => { - if (mine.test(o.id)) return true; - const created = o.createdAt ? Date.parse(o.createdAt) : 0; - return hasRunSuffix(o.id) && created > 0 && now - created > SIX_HOURS; - }; - const note = (kind: string, o: Listed): void => { - const created = o.createdAt ? Date.parse(o.createdAt) : 0; - if (!hasRunSuffix(o.id) && created >= since.getTime()) residue.push(`dataSync ${kind}:${o.id}`); - }; - - // Dependency order: memberships and relationships reference users, channels and entities, so they go first. - const kinds: Array<{ - kind: string; - list: () => Promise; - remove: (id: string) => Promise; - optionalClass?: boolean; - }> = [ - { kind: 'membership', list: () => listAll((cursor) => ds.getMemberships({ limit: 100, cursor })), remove: (id) => ds.removeMembership({ id }) }, - { - kind: 'relationship', - list: () => listAll((cursor) => ds.getRelationships({ class: 'ProductOwner', limit: 100, cursor })), - remove: (id) => ds.removeRelationship({ id }), - optionalClass: true, - }, - { - kind: 'entity', - list: () => listAll((cursor) => ds.getEntities({ class: 'product', limit: 100, cursor })), - remove: (id) => ds.removeEntity({ id }), - optionalClass: true, - }, - { kind: 'user', list: () => listAll((cursor) => ds.getUsers({ limit: 100, cursor })), remove: (id) => ds.removeUser({ id }) }, - { kind: 'channel', list: () => listAll((cursor) => ds.getChannels({ limit: 100, cursor })), remove: (id) => ds.removeChannel({ id }) }, - ]; - - try { - for (const { kind, list, remove, optionalClass } of kinds) { - let objects: Listed[]; - try { - objects = await list(); - } catch (error) { - if (optionalClass && missingClass(error)) { - log.push(`DataSync ${kind}: class not provisioned, skipped`); - } else { - log.push(`DataSync ${kind}: listing failed, skipped (${(error as Error).message ?? error})`); - } - continue; - } - let removed = 0; - for (const object of objects) { - if (doomed(object)) { - try { - await remove(object.id); - removed += 1; - } catch { - // Already removed by a cascade (deleting a user removes its memberships). - } - } else { - note(kind, object); - } - } - log.push(`DataSync ${kind}s removed: ${removed}`); - } - } finally { - client.destroy(); - } - return { log, residue }; -} diff --git a/docs-snippets/harness/janitor.ts b/docs-snippets/harness/janitor.ts deleted file mode 100644 index f382b1fae..000000000 --- a/docs-snippets/harness/janitor.ts +++ /dev/null @@ -1,255 +0,0 @@ -import PubNub from 'pubnub'; -import { allGroupNames, hasRunSuffix, names, SUFFIX_PATTERN, suffixName } from '../fixtures/names.js'; -import { ROLES, runIdAgeMs, securedKeysets, type Env, type Role } from './env.js'; -import { sweepDataSync } from './janitor-datasync.js'; -import { revokeApplicationGrantOnKeyset } from './revoke.js'; - -/** - * Janitor. Runs in the Mocha root `after` hook and from `npm run test:docs-snippets:janitor`. - * - * Phase 1 scope: - * 1. delete this run's channel groups (registered names, 404 ignored) - * 2. remove this run's App Context users and channels on the open keyset - * 3. sweep App Context objects with a run suffix older than six hours (crashed earlier runs) - * 4. report residue: App Context objects created or updated during the run window without a run suffix - * - * Also removes files uploaded to the file sharing channels and push registrations of the registered device tokens, - * and revokes the application-level Access Manager grant on the pam and datasync keysets (fail-safe, see revoke.ts). - * - * Crashed-run sweep by pattern: run ids carry their start time (env.ts). Every run id found on a channel group or an - * App Context object that is older than six hours, and is not this run, gets the same per-run cleanup as this run: - * its channel groups, files on the file sharing channels and push registrations. The ids are printed first. - * DataSync objects on the datasync keyset are cleaned by janitor-datasync.ts. - * Messages cannot be listed by channel, so history residue relies on 1-day retention. - */ - -// Why six hours: longer than any single run (the full suite takes minutes), so a run id this old cannot belong to a live -// run, yet short enough that a crash is cleaned the same day. -const SIX_HOURS = 6 * 60 * 60 * 1000; -/** Channels the file sharing samples upload to. */ -const FILE_CHANNELS = ['my_channel', 'awesomeChannel', 'preflight_channel']; - -/** `log` lines are printed. `residue` and `critical` make the run fail (see mocha-entry.ts). */ -export interface JanitorReport { - log: string[]; - residue: string[]; - /** Cleanup that failed in a way that leaves the keysets in an unsafe state. The run must fail. */ - critical: string[]; -} - -interface ObjectLike { - id: string; - updated?: string; -} - -/** - * Cleans up after one run and after crashed ones. `since` is the run start, it bounds which unsuffixed objects count as - * residue. Never throws for ordinary cleanup failures (a missing object is fine), only reports `critical` for the revoke. - */ -export async function runJanitor(env: Env, runId: string, since: Date, roles: readonly Role[] = ROLES): Promise { - const log: string[] = []; - const residue: string[] = []; - const critical: string[] = []; - // Only the keysets of the roles this run used. CI runs the shards in parallel, and an open shard must never revoke the - // application-level grant a pam step is testing at that moment. - const uses = (role: Role): boolean => roles.includes(role) || (role === 'pam' && env.sharedSecured && roles.includes('datasync')); - // Safety first: never leave a secured keyset open to strangers. pam and datasync may be one keyset, it is revoked once. - for (const { role, keys } of securedKeysets(env).filter((k) => uses(k.role))) { - try { - await revokeApplicationGrantOnKeyset(keys, `r${runId}`); - // The revoke call succeeded, but Access Manager (v2) needs up to about 60 seconds to apply it. The root `after` hook waits - // for it (revoke.ts, closeAndVerifyApplicationGrant). The standalone janitor command only says so. - log.push(`application-level grant on the ${role} keyset revoked (it can take up to 60 s to take effect)`); - } catch (error) { - critical.push(`${role} keyset: ${(error as Error).message}`); - } - } - // DataSync objects live on the datasync keyset, see janitor-datasync.ts. A failure here must not skip the open keyset cleanup. - if (roles.includes('datasync')) { - try { - const dataSync = await sweepDataSync(env, runId, since); - log.push(...dataSync.log); - residue.push(...dataSync.residue); - } catch (error) { - log.push(`DataSync sweep failed (${(error as Error).message ?? error})`); - } - } - if (!env.open) { - log.push('open keyset not configured, nothing to clean'); - return { log, residue, critical }; - } - // A throwaway admin client of its own: the world's clients are destroyed before the janitor runs after a signal. - const admin = new PubNub({ - publishKey: env.open.publishKey, - subscribeKey: env.open.subscribeKey, - userId: `janitor-r${runId}`, - logLevel: PubNub.LogLevel.None, - }); - - try { - // Run ids of crashed runs, discovered by pattern before anything is deleted. - const stale = new Set(); - // Collects the run id from any resource name with a `-r` suffix, if that run is old enough to be dead. - const noteStale = (id: string): void => { - const match = SUFFIX_PATTERN.exec(id); - if (!match) return; - const other = match[0].slice(2); - if (other === runId) return; - const age = runIdAgeMs(other); - if (age !== undefined && age > SIX_HOURS) stale.add(other); - }; - let groupNames: string[] = []; - try { - groupNames = (await admin.channelGroups.listGroups()).groups; - groupNames.forEach(noteStale); - } catch (error) { - log.push(`channel group listing unavailable, group sweep skipped (${(error as Error).message ?? error})`); - } - - // 1. Everything this run created, then the same for each crashed run. - log.push(`run id ${runId}`); - await cleanupRun(admin, runId, log, residue); - const objects = new Map<'uuid' | 'channel', ObjectLike[]>(); - for (const kind of ['uuid', 'channel'] as const) { - try { - const listed = await listAll(admin, kind); - objects.set(kind, listed); - listed.forEach((o) => noteStale(o.id)); - } catch (error) { - log.push(`App Context ${kind} metadata unavailable, skipped (${(error as Error).message ?? error})`); - } - } - if (stale.size > 0) log.push(`crashed runs older than six hours found by pattern: ${[...stale].join(', ')}`); - for (const id of stale) { - log.push(`sweeping run id ${id}`); - await cleanupRun(admin, id, log, residue); - } - - // 2 to 4. App Context. `mine` = this run's suffix, `old` = any suffixed object not touched for six hours, - // anything else created during this run without a suffix is residue (the snippet used an unregistered literal). - // The lookahead accepts the suffix on the first dot segment (`chats-r.room1`) or at the end of the name. - const runSuffix = new RegExp(`-r${runId}(?=\\.|$)`); - for (const kind of ['uuid', 'channel'] as const) { - const listed = objects.get(kind); - if (!listed) continue; - const now = Date.now(); - let removed = 0; - for (const object of listed) { - const updated = object.updated ? Date.parse(object.updated) : 0; - const mine = runSuffix.test(object.id); - const old = hasRunSuffix(object.id) && updated > 0 && now - updated > SIX_HOURS; - if (mine || old) { - try { - if (kind === 'uuid') await admin.objects.removeUUIDMetadata({ uuid: object.id }); - else await admin.objects.removeChannelMetadata({ channel: object.id }); - removed += 1; - } catch { - // Already gone. - } - } else if (!hasRunSuffix(object.id) && updated >= since.getTime()) { - residue.push(`${kind}:${object.id}`); - } - } - log.push(`App Context ${kind} metadata removed: ${removed}`); - } - } finally { - admin.destroy(); - } - return { log, residue, critical }; -} - -/** Channel groups, files and push registrations of one run id on the open keyset. */ -async function cleanupRun(admin: PubNub, runId: string, log: string[], residue: string[]): Promise { - const label = `[${runId}]`; - let removedGroups = 0; - for (const name of allGroupNames()) { - try { - await admin.channelGroups.deleteGroup({ channelGroup: suffixName(name, runId) }); - removedGroups += 1; - } catch { - // Missing groups are fine. - } - } - log.push(`${label} channel groups deleted: ${removedGroups}`); - - let removedFiles = 0; - for (const base of FILE_CHANNELS) { - const channel = suffixName(base, runId); - try { - const listed = await admin.listFiles({ channel, limit: 100 }); - for (const file of listed.data) { - try { - await admin.deleteFile({ channel, id: file.id, name: file.name }); - removedFiles += 1; - } catch { - // Already gone. - } - } - } catch { - // Files not enabled or the channel never had a file. - } - } - log.push(`${label} files deleted: ${removedFiles}`); - - // Push registrations. Device tokens are derived from the run id (names.ts `deviceToken`), so a crashed run's - // tokens are recomputed from its run id alone. The registrations themselves cannot be listed without the token. - // Run ids of crashed push runs are found through the `push_marker` channel metadata the mobile push page sets - // (see api-reference/mobile-push.ts), which the App Context sweep above lists by pattern. - // `niceDevice` is the docs sample's token. The suite only registers it in the harness check that proves the - // apns/apns2 rejection claim, and that check removes it again, but a crash in between is covered here. - const devices = [...names.device.map((base) => suffixName(base, runId)), suffixName('niceDevice', runId)]; - // Gateway variants the suite can register on: fcm, apns (v1) and both apns2 environments with the docs topic. - const gateways: Array<{ - label: string; - remove: (device: string) => Promise; - list: (device: string) => Promise<{ channels: string[] }>; - }> = [ - { label: 'fcm', remove: (device) => admin.push.deleteDevice({ device, pushGateway: 'fcm' }), list: (device) => admin.push.listChannels({ device, pushGateway: 'fcm' }) }, - { label: 'apns', remove: (device) => admin.push.deleteDevice({ device, pushGateway: 'apns' }), list: (device) => admin.push.listChannels({ device, pushGateway: 'apns' }) }, - ...(['production', 'development'] as const).map((environment) => ({ - label: `apns2/${environment}`, - remove: (device: string) => admin.push.deleteDevice({ device, pushGateway: 'apns2', environment, topic: 'com.example.bundle_id' }), - list: (device: string) => admin.push.listChannels({ device, pushGateway: 'apns2', environment, topic: 'com.example.bundle_id' }), - })), - ]; - let removedDevices = 0; - for (const device of devices) { - for (const gateway of gateways) { - try { - await gateway.remove(device); - removedDevices += 1; - } catch { - // Mobile push off, a token the gateway rejects (niceDevice on apns), or nothing registered. - } - } - } - log.push(`${label} push device registrations deleted (calls that succeeded): ${removedDevices}`); - // Verify, per gateway: a list call for each token must come back empty. Errors mean push is off or the token - // never registered there, which leaves nothing behind. Only a non-empty answer is residue. - for (const device of devices) { - for (const gateway of gateways) { - try { - const left = await gateway.list(device); - if (left.channels.length > 0) residue.push(`push ${gateway.label} ${device.slice(0, 8)}...: ${left.channels.join(',')}`); - } catch { - // Nothing to list. - } - } - } -} - -// Pages through App Context (100 per page, at most 20 pages) so cleanup finds objects beyond the first page. -async function listAll(admin: PubNub, kind: 'uuid' | 'channel'): Promise { - const out: ObjectLike[] = []; - let next: string | undefined; - for (let page = 0; page < 20; page += 1) { - const response = - kind === 'uuid' - ? await admin.objects.getAllUUIDMetadata({ limit: 100, page: next ? { next } : undefined }) - : await admin.objects.getAllChannelMetadata({ limit: 100, page: next ? { next } : undefined }); - out.push(...response.data.map((d) => ({ id: d.id, updated: d.updated }))); - next = response.next; - if (!next || response.data.length === 0) break; - } - return out; -} diff --git a/docs-snippets/harness/keyset-features.ts b/docs-snippets/harness/keyset-features.ts deleted file mode 100644 index ffe722a4a..000000000 --- a/docs-snippets/harness/keyset-features.ts +++ /dev/null @@ -1,103 +0,0 @@ -/** - * Read-only keyset feature check through the Admin API. It never changes a keyset, an app or a key. - * - * With `PUBNUB_ADMIN_API_KEY` and a keyset id per role (`DOCS_SNIPPETS_OPEN_KEYSET_ID`, `DOCS_SNIPPETS_PAM_KEYSET_ID`, - * `DOCS_SNIPPETS_DS_KEYSET_ID`), the preflight reads `GET /v2/keysets/{id}` and `GET /v2/keysets/{id}/config` (documented in - * the docs repository, docs/admin-api/get-keyset-by-id.api.mdx and get-keyset-configuration.api.mdx). It first confirms - * that the keyset of that id has the role's subscribe key. The comparison happens here and only its result is reported, never - * a key or part of one. Then it names every feature the role needs and the keyset lacks. - * - * Without the key or the id, `readFeatures` returns `skipped` and the preflight probes with the SDK only. - */ -import type { Env, Role } from './env.js'; - -const BASE = 'https://admin-api.pubnub.com/v2/keysets'; -const API_VERSION = '2026-09-17'; - -/** The part of the keyset configuration the suite needs. Field names as the Admin API returns them. */ -export interface KeysetConfig { - accessManager?: { enabled?: boolean; tokenRevoke?: boolean }; - files?: { enabled?: boolean }; - streamController?: { enabled?: boolean; wildcardSubscribe?: boolean }; - appContext?: { - enabled?: boolean; - userMetadataEvents?: boolean; - channelMetadataEvents?: boolean; - membershipEvents?: boolean; - referentialIntegrity?: boolean; - disallowGetAllChannelMetadata?: boolean; - disallowGetAllUserMetadata?: boolean; - }; - dataSync?: { enabled?: boolean }; - messagePersistence?: { enabled?: boolean; deleteFromHistory?: boolean }; - presence?: { enabled?: boolean; rules?: Array<{ pattern?: string; tracking?: { enabled?: boolean } }> }; -} - -export type FeatureRead = - | { status: 'skipped'; why: string } - | { status: 'mismatch'; why: string } - | { status: 'error'; why: string } - | { status: 'read'; config: KeysetConfig }; - -async function get(url: string, apiKey: string): Promise<{ status: number; json: unknown }> { - const response = await fetch(url, { headers: { Authorization: apiKey, 'PubNub-Version': API_VERSION } }); - const text = await response.text(); - try { - return { status: response.status, json: text ? JSON.parse(text) : {} }; - } catch { - return { status: response.status, json: {} }; - } -} - -/** Reads a role's keyset configuration. Never returns or logs key material. */ -export async function readFeatures(env: Env, role: Role): Promise { - const keys = env[role]; - const id = env.keysetIds[role]; - if (!env.adminKey) return { status: 'skipped', why: 'PUBNUB_ADMIN_API_KEY is not set' }; - if (!id) return { status: 'skipped', why: `no keyset id for role ${role}` }; - if (!keys) return { status: 'skipped', why: `role ${role} has no keys` }; - const keyset = await get(`${BASE}/${encodeURIComponent(id)}`, env.adminKey); - if (keyset.status !== 200) return { status: 'error', why: `reading keyset ${id} answered HTTP ${keyset.status}` }; - const subscribeKey = (keyset.json as { subscribeKey?: unknown }).subscribeKey; - // The only comparison of key material in this module. Only the boolean leaves it. - if (subscribeKey !== keys.subscribeKey) - return { status: 'mismatch', why: `keyset ${id} does not hold the ${role} subscribe key` }; - const config = await get(`${BASE}/${encodeURIComponent(id)}/config`, env.adminKey); - if (config.status !== 200) - return { status: 'error', why: `reading the configuration of keyset ${id} answered HTTP ${config.status}` }; - return { status: 'read', config: config.json as KeysetConfig }; -} - -/** Features a role needs, as named in docs-snippets/README.md. Returns one sentence per missing feature. */ -export function missingFeatures(role: Role, c: KeysetConfig): string[] { - const out: string[] = []; - const on = (value: boolean | undefined, text: string): void => { - if (value !== true) out.push(`${text} is off on the ${role} keyset`); - }; - if (role === 'open') { - on(c.presence?.enabled, 'Presence'); - const wildcard = c.presence?.rules?.some((r) => r.pattern === '*' && r.tracking?.enabled === true); - on(wildcard, 'Presence tracking on all channels (rule pattern *)'); - on(c.messagePersistence?.enabled, 'Message Persistence'); - on(c.messagePersistence?.deleteFromHistory, 'Message Persistence Delete-From-History'); - on(c.appContext?.enabled, 'App Context'); - on(c.appContext?.userMetadataEvents, 'App Context user metadata events'); - on(c.appContext?.channelMetadataEvents, 'App Context channel metadata events'); - on(c.appContext?.membershipEvents, 'App Context membership events'); - if (c.appContext?.referentialIntegrity === true) - out.push('App Context referential integrity is on on the open keyset (it must be off)'); - if (c.appContext?.disallowGetAllUserMetadata === true || c.appContext?.disallowGetAllChannelMetadata === true) - out.push('App Context "disallow get all" is on on the open keyset (it must be off)'); - on(c.files?.enabled, 'Files'); - on(c.streamController?.enabled, 'Stream Controller'); - on(c.streamController?.wildcardSubscribe, 'Stream Controller wildcard subscribe'); - if (c.accessManager?.enabled === true) out.push('Access Manager is on on the open keyset (it must be off)'); - } else { - on(c.accessManager?.enabled, 'Access Manager'); - if (role === 'pam') on(c.accessManager?.tokenRevoke, 'Access Manager token revocation'); - on(c.messagePersistence?.enabled, 'Message Persistence'); - on(c.presence?.enabled, 'Presence'); - if (role === 'datasync') on(c.dataSync?.enabled, 'DataSync'); - } - return out; -} diff --git a/docs-snippets/harness/lint-markers.ts b/docs-snippets/harness/lint-markers.ts deleted file mode 100644 index 727660ac3..000000000 --- a/docs-snippets/harness/lint-markers.ts +++ /dev/null @@ -1,331 +0,0 @@ -import fs from 'node:fs'; -import path from 'node:path'; -import { fileURLToPath } from 'node:url'; -import ts from 'typescript'; -import { isPlaceholder, isRegisteredName, PLACEHOLDER_LOOKALIKE } from '../fixtures/names.js'; -import { isInsideRegion, isVisibleLine, scanMarkers, type SnippetRegion } from './markers.js'; - -/** - * Marker lint. Snippets let errors throw, so `try`, - * `catch` and `console.error` inside a snippet are errors too. - * - * Rules: - * 1. id matches ^[A-Za-z0-9_-]+$ and is unique across the suite - * 2. no nested start marker, every snippet has snippet.end - * 3. snippet.hide has a snippet.show, at most 3 lines per hidden block, and hidden lines only hold - * in-scope asserts or live configuration (they must hold `world.` or `assert(` as code, comments do not count, - * except one `@ts-expect-error` line that is listed in the output) - * 4. a snippet sits inside a step() callback, except import-only snippets at module top level - * 5. a step id is one of the snippet ids it contains, `snippets` equals the markers inside, - * check() contains none - * 6. resource-like string literals inside a snippet are in fixtures/names.ts, placeholder-like literals are - * registered placeholders - * 7. no try, catch or console.error inside a snippet (except the ids in ERROR_HANDLING_SNIPPETS) - */ - -// Role: the static gate. It runs without keys or network (step 2 of `npm run test:docs-snippets:dry` and a gate before every -// live run) and guarantees the invariants the docs and the stage rewrite depend on: a snippet is a clean, embeddable -// region inside a step, with registered literals. -// -// A slash is allowed because the docs embed `simpleNotificationPayloadFCMandAPNSH/2`. The docs extractor -// matches the requested id literally, so it works there. -const ID = /^[A-Za-z0-9_/-]+$/; -const MAX_HIDDEN_LINES = 3; -/** - * Snippets whose whole purpose is to show try/catch and console.error (the publish error handling sample). - * Rule 7 does not apply to these ids. Keep this list short: every other snippet lets errors throw. - */ -const ERROR_HANDLING_SNIPPETS = new Set([ - 'publishHandleErrors', - // The pages show these calls inside try and catch. A hidden world.keep() at the end of the try hands the result to the - // step, and world.take() fails the step when the catch ran (harness/world.ts), so a swallowed error cannot pass. - 'grantTokenSpacesBasicUsage', 'grantTokenSpacesVariousResources', 'grantTokenSpacesRegEx', 'grantTokenSpacesRegExAndResources', - 'historyBasicUsage', 'historyOldestThree', 'historyNewerThanTimetoken', 'historyUntilTimetoken', 'historyIncludeMeta', - 'quickstartNodePublish', -]); -/** Directory of standalone quickstart snippet sources, see the end of lintApp. */ -const STANDALONE_DIR = 'quickstarts'; -const SCAN_DIRS = ['api-reference', 'shared', 'getting-started', 'migration-guides']; -const RESOURCE_KEYS = new Set(['channel', 'channelGroup', 'uuid', 'userId', 'id', 'device', 'channelId', 'spaceId', 'entityAId', 'entityBId']); -const RESOURCE_ARRAY_KEYS = new Set(['channels', 'channelGroups', 'uuids', 'userIds']); -const RESOURCE_CALLS = new Set([ - 'channel', - 'channelGroup', - 'channelMetadata', - 'userMetadata', - 'setUserId', - 'dataSyncUser', - 'dataSyncChannel', - 'dataSyncEntity', - 'dataSyncRelationship', - 'dataSyncMembership', -]); - -/** One lint finding. `line` is 1-based in `file` (relative to the suite directory). */ -export interface LintIssue { - file: string; - line: number; - message: string; -} - -/** `snippets` lists every region found (its step, or `importOnly` for module-level imports), the dry run compares it with the registry. */ -export interface LintResult { - errors: LintIssue[]; - files: number; - snippets: Array<{ id: string; file: string; step?: string; importOnly: boolean }>; - steps: number; - /** Hidden type suppressions that the rule allows, listed so they stay visible. */ - suppressions: string[]; -} - -// A `step()` or `check()` call found in the AST, with the line span of its callback. Snippet regions are matched to -// the call whose span contains them. -interface StepCall { - kind: 'step' | 'check'; - id: string; - startLine: number; - endLine: number; - declared?: string[]; -} - -/** Scans every page directory and the standalone quickstart sources. Pure file reads, safe to call from the dry run and the CLI. */ -export function lintApp(appDir: string): LintResult { - const errors: LintIssue[] = []; - const snippets: LintResult['snippets'] = []; - const seen = new Map(); - const suppressions: string[] = []; - let files = 0; - let steps = 0; - - for (const dir of SCAN_DIRS) { - const root = path.join(appDir, dir); - if (!fs.existsSync(root)) continue; - for (const file of walk(root).filter((f) => /\.tsx?$/.test(f))) { - files += 1; - const rel = path.relative(appDir, file); - const text = fs.readFileSync(file, 'utf8'); - const add = (line: number, message: string): void => { - errors.push({ file: rel, line, message }); - }; - const source = ts.createSourceFile(file, text, ts.ScriptTarget.ES2022, true, ts.ScriptKind.TS); - const lineOf = (pos: number): number => source.getLineAndCharacterOfPosition(pos).line + 1; - const { regions, problems } = scanMarkers(text); - problems.forEach((p) => add(p.line, p.message)); - - // Collect step() and check() calls. The callback is the last function argument, so both the `(id, fn)` and - // `(id, options, fn)` overloads work. Only string-literal ids count: a computed id could not be matched to markers. - const calls: StepCall[] = []; - const collect = (node: ts.Node): void => { - if (ts.isCallExpression(node) && ts.isIdentifier(node.expression)) { - const name = node.expression.text; - if ((name === 'step' || name === 'check') && node.arguments.length > 0) { - const first = node.arguments[0]; - const fn = [...node.arguments].reverse().find((a) => ts.isArrowFunction(a) || ts.isFunctionExpression(a)); - if (ts.isStringLiteral(first)) { - let declared: string[] | undefined; - const options = node.arguments[1]; - if (options && ts.isObjectLiteralExpression(options)) { - for (const prop of options.properties) { - if (ts.isPropertyAssignment(prop) && ts.isIdentifier(prop.name) && prop.name.text === 'snippets' && ts.isArrayLiteralExpression(prop.initializer)) { - declared = prop.initializer.elements.filter(ts.isStringLiteral).map((e) => e.text); - } - } - } - calls.push({ - kind: name, - id: first.text, - startLine: fn ? lineOf(fn.getStart(source)) : lineOf(node.getStart(source)), - endLine: fn ? lineOf(fn.end) : lineOf(node.end), - declared, - }); - } - } - } - ts.forEachChild(node, collect); - }; - collect(source); - steps += calls.filter((c) => c.kind === 'step').length; - - // An import-only snippet (`importPubNub`) is allowed at module level because an import cannot sit inside a function. - const topLevelOnlyImports = (region: SnippetRegion): boolean => { - const inside = source.statements.filter((s) => { - const l = lineOf(s.getStart(source)); - return isInsideRegion(region, l); - }); - return inside.length > 0 && inside.every(ts.isImportDeclaration); - }; - - const containing = new Map(); - for (const region of regions) { - // Rule 1. - if (!ID.test(region.id)) add(region.startLine, `snippet id "${region.id}" does not match ${ID}`); - const previous = seen.get(region.id); - if (previous) add(region.startLine, `duplicate snippet id "${region.id}" (also in ${previous})`); - else seen.set(region.id, `${rel}:${region.startLine}`); - - // Rule 3. - for (const block of region.hidden) { - if (block.showLine === undefined) continue; - const lines = block.showLine - block.hideLine - 1; - if (lines > MAX_HIDDEN_LINES) { - add(block.hideLine, `hidden block in "${region.id}" has ${lines} lines, maximum is ${MAX_HIDDEN_LINES}`); - } - const body = text.split('\n').slice(block.hideLine, block.showLine - 1).join('\n'); - // Comment text does not count: a comment that merely contains the word "assert" used to pass this rule. - const code = stripComments(body); - if (isTypeSuppression(body)) { - // The one documented exception: a single `@ts-expect-error` line for a property the shipped types lack. - // `@ts-expect-error` (not `@ts-ignore`) stops compiling when the SDK types are fixed, which removes the hole. - suppressions.push(`${rel}:${block.hideLine} (${region.id})`); - } else if (!/\bworld\.|\bassert\s*[.(]/.test(code)) { - add(block.hideLine, `hidden block in "${region.id}" must hold an in-scope assert or live configuration as code (world.* or assert(...)), comments do not count`); - } - } - - // Rule 4. - const owner = calls.find((c) => region.startLine >= c.startLine && region.endLine <= c.endLine); - let importOnly = false; - if (!owner) { - if (topLevelOnlyImports(region)) importOnly = true; - else add(region.startLine, `snippet "${region.id}" is not inside a step() callback (only import-only snippets may sit at module level)`); - } else { - const list = containing.get(owner) ?? []; - list.push(region.id); - containing.set(owner, list); - } - snippets.push({ id: region.id, file: rel, step: owner?.id, importOnly }); - - // Rules 6 and 7 walk the AST inside the visible region. Hidden lines are excluded for rule 6, because they hold - // harness configuration and not docs text. - const visit = (node: ts.Node): void => { - const startLine = lineOf(node.getStart(source)); - const inRegion = isInsideRegion(region, startLine); - if (inRegion && !ERROR_HANDLING_SNIPPETS.has(region.id) && ts.isTryStatement(node)) add(startLine, `try/catch inside snippet "${region.id}" (snippets let errors throw)`); - if ( - inRegion && - !ERROR_HANDLING_SNIPPETS.has(region.id) && - ts.isCallExpression(node) && - ts.isPropertyAccessExpression(node.expression) && - node.expression.expression.getText(source) === 'console' && - node.expression.name.text === 'error' - ) { - add(startLine, `console.error inside snippet "${region.id}" (snippets let errors throw)`); - } - if (ts.isStringLiteral(node) && isVisibleLine(region, startLine)) { - const value = node.text; - if (PLACEHOLDER_LOOKALIKE.test(value) && !isPlaceholder(value)) { - add(startLine, `placeholder-like literal "${value}" in "${region.id}" is not in fixtures/names.ts placeholders`); - } else if (isResourcePosition(node) && !isRegisteredName(value) && !isPlaceholder(value)) { - add(startLine, `resource-like literal "${value}" in "${region.id}" is not in fixtures/names.ts`); - } - } - ts.forEachChild(node, visit); - }; - visit(source); - } - - // Rule 5. - for (const call of calls) { - const ids = containing.get(call) ?? []; - if (call.kind === 'step') { - if (ids.length === 0) add(call.startLine, `step "${call.id}" contains no snippet (use check() for harness-only steps)`); - else { - if (!ids.includes(call.id)) add(call.startLine, `step name "${call.id}" is not one of its snippet ids [${ids.join(', ')}]`); - if (ids[0] !== call.id && ids.includes(call.id)) add(call.startLine, `step "${call.id}" must be the first snippet id inside (found ${ids[0]} first)`); - const declared = call.declared ?? [call.id]; - if (declared.join('|') !== ids.join('|')) { - add(call.startLine, `step "${call.id}" declares snippets [${declared.join(', ')}] but contains [${ids.join(', ')}]`); - } - } - } else if (ids.length > 0) { - add(call.startLine, `${call.kind}() "${call.id}" contains snippet markers [${ids.join(', ')}], use step()`); - } - } - } - } - // Standalone sources (quickstarts/*.ts, *.tsx, *.jsx). They are the exact text of the Web, React and React Native - // quickstart pages and of the React, browser-only and React Native samples of the API reference. This repository cannot - // run them (no browser, no React, no React Native), so no step() callback runs them. Rules 1 and 2 apply (id shape, - // uniqueness, no nesting, snippet.end), hidden lines are not allowed except one `@ts-expect-error` line, and the - // step and literal rules (3 to 7) do not apply: the pages teach try/catch around publish and use their own channel - // literals. - const standalone = path.join(appDir, STANDALONE_DIR); - if (fs.existsSync(standalone)) { - for (const name of fs.readdirSync(standalone).filter((f) => /\.(tsx?|jsx)$/.test(f))) { - files += 1; - const rel = path.join(STANDALONE_DIR, name); - const source = fs.readFileSync(path.join(standalone, name), 'utf8'); - const { regions, problems } = scanMarkers(source); - problems.forEach((p) => errors.push({ file: rel, line: p.line, message: p.message })); - for (const region of regions) { - if (!ID.test(region.id)) errors.push({ file: rel, line: region.startLine, message: `snippet id "${region.id}" does not match ${ID}` }); - const previous = seen.get(region.id); - if (previous) errors.push({ file: rel, line: region.startLine, message: `duplicate snippet id "${region.id}" (also in ${previous})` }); - else seen.set(region.id, `${rel}:${region.startLine}`); - // The one exception, as in rule 3: a hidden block that is a single `@ts-expect-error` line for an SDK typing bug. - for (const block of region.hidden) { - const body = block.showLine === undefined ? '' : source.split('\n').slice(block.hideLine, block.showLine - 1).join('\n'); - if (isTypeSuppression(body)) suppressions.push(`${rel}:${block.hideLine} (${region.id})`); - else errors.push({ file: rel, line: block.hideLine, message: `snippet "${region.id}" hides lines, standalone snippets must show exactly what the page shows (only a single @ts-expect-error line may be hidden)` }); - } - snippets.push({ id: region.id, file: rel, importOnly: false }); - } - } - } - return { errors, files, snippets, steps, suppressions }; -} - -/** - * Removes `//` line comments and block comments, so rule 3 sees only code. The negative class before `//` keeps `https://` inside - * a string from being cut. A heuristic, adequate for hidden blocks of at most three lines. - */ -export function stripComments(body: string): string { - return body.replace(/\/\*[\s\S]*?\*\//g, '').replace(/(^|[^:'"`])\/\/.*$/gm, '$1'); -} - -/** A hidden block that is exactly one `// @ts-expect-error` comment line. */ -export function isTypeSuppression(body: string): boolean { - const lines = body.split('\n').filter((l) => l.trim() !== ''); - return lines.length === 1 && /^\s*\/\/\s*@ts-expect-error\b/.test(lines[0]); -} - -function isResourcePosition(node: ts.StringLiteral): boolean { - const parent = node.parent; - if (ts.isPropertyAssignment(parent) && parent.initializer === node) { - return ts.isIdentifier(parent.name) && RESOURCE_KEYS.has(parent.name.text); - } - if (ts.isArrayLiteralExpression(parent)) { - const holder = parent.parent; - return ts.isPropertyAssignment(holder) && ts.isIdentifier(holder.name) && RESOURCE_ARRAY_KEYS.has(holder.name.text); - } - if (ts.isCallExpression(parent) && parent.arguments[0] === node && ts.isPropertyAccessExpression(parent.expression)) { - return RESOURCE_CALLS.has(parent.expression.name.text); - } - return false; -} - -function walk(dir: string): string[] { - return fs.readdirSync(dir, { withFileTypes: true }).flatMap((entry) => { - const full = path.join(dir, entry.name); - return entry.isDirectory() ? walk(full) : [full]; - }); -} - -export function printLint(result: LintResult): void { - const ids = result.snippets.length; - const importOnly = result.snippets.filter((s) => s.importOnly).length; - if (result.errors.length === 0) { - console.log(`marker lint ok. ${result.files} files, ${result.steps} steps, ${ids} snippets (${importOnly} import-only).`); - for (const s of result.suppressions) console.log(` allowed hidden @ts-expect-error: ${s}`); - return; - } - for (const issue of result.errors) console.error(`${issue.file}:${issue.line} ${issue.message}`); - console.error(`marker lint failed with ${result.errors.length} problem(s) in ${result.files} files.`); -} - -const invokedDirectly = process.argv[1] && fileURLToPath(import.meta.url) === path.resolve(process.argv[1]); -if (invokedDirectly) { - const result = lintApp(path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..')); - printLint(result); - process.exit(result.errors.length === 0 ? 0 : 1); -} diff --git a/docs-snippets/harness/manifest.ts b/docs-snippets/harness/manifest.ts deleted file mode 100644 index 718bf79c5..000000000 --- a/docs-snippets/harness/manifest.ts +++ /dev/null @@ -1,145 +0,0 @@ -/** - * Coverage manifest: `docs-snippets/coverage.json`, one entry per snippet id, generated from the marker lint and the step - * registry. It says for every id how the suite covers it: - * - * live a step runs it against the service and asserts the effect - * compile-only type checked only: import-only snippets, and the browser sources of tsconfig.browser.json - * stub not run and not compiled here: React and React Native sources, which cannot run in this repository (the SDK has no - * React or React Native dependencies) - * - * `npm run docs-snippets:manifest -- --check` fails on a marker id without a step, a step id without a marker, a duplicate id - * or a stale coverage.json. It also reports, without failing, docs ids missing from the manifest and manifest ids that no - * docs page uses. Docs ids come from `` tags under docs/sdks/javascript of - * the docs checkout in DOCS_SNIPPETS_DOCS_DIR (default: a `documentation` checkout next to this repository). - */ -import fs from 'node:fs'; -import path from 'node:path'; -import type { LintResult } from './lint-markers.js'; -import type { Registry } from './registry.js'; - -export interface ManifestEntry { - file: string | null; - step: string | null; - role: string | null; - phase: string | null; - mode: 'live' | 'compile-only' | 'stub'; - reason: string | null; -} - -export interface Manifest { - $comment: string; - snippets: Record; -} - -/** Sources type checked by tsconfig.browser.json. Keep in sync with its `include`. */ -const BROWSER_CHECKED = [ - 'quickstarts/quickstart-web.ts', - 'quickstarts/browser-download-file.ts', - 'quickstarts/browser-shared-worker.ts', -]; - -export function buildManifest(lint: LintResult, reg: Registry): { manifest: Manifest; problems: string[] } { - const problems: string[] = []; - const entries: Record = {}; - const seen = new Map(); - for (const snippet of lint.snippets) { - const previous = seen.get(snippet.id); - if (previous) problems.push(`duplicate snippet id ${snippet.id} (${previous} and ${snippet.file})`); - seen.set(snippet.id, snippet.file); - if (snippet.step) { - const def = reg.get(snippet.step); - if (!def) { - problems.push(`marker ${snippet.id} sits in step ${snippet.step}, which the registry does not know`); - continue; - } - entries[snippet.id] = { - file: snippet.file, - step: def.id, - role: def.keyset, - phase: def.phase, - mode: 'live', - reason: null, - }; - } else if (snippet.importOnly) { - entries[snippet.id] = { - file: snippet.file, - step: null, - role: null, - phase: null, - mode: 'compile-only', - reason: 'import-only snippet, type checked by tsconfig.live.json', - }; - } else if (snippet.file.startsWith('quickstarts/')) { - const browser = BROWSER_CHECKED.includes(snippet.file); - entries[snippet.id] = { - file: snippet.file, - step: null, - role: null, - phase: null, - mode: browser ? 'compile-only' : 'stub', - reason: browser - ? 'browser-only, type checked by tsconfig.browser.json' - : 'react or react-native source, kept for the docs URLs, cannot be run in this repository', - }; - } else { - problems.push(`marker ${snippet.id} in ${snippet.file} belongs to no step`); - } - } - for (const def of reg.ordered()) { - for (const id of def.snippets) { - if (!seen.has(id)) problems.push(`step ${def.id} declares snippet ${id}, but no marker defines it`); - else if (entries[id]?.step !== def.id) - problems.push( - `snippet ${id} is declared by step ${def.id} but its marker sits in ${entries[id]?.step ?? 'no step'}`, - ); - } - } - const sorted = Object.fromEntries( - Object.keys(entries) - .sort() - .map((id) => [id, entries[id]]), - ); - return { - manifest: { - $comment: - 'Generated by npm run docs-snippets:manifest. Do not edit. One entry per snippet id: how the live suite covers it.', - snippets: sorted, - }, - problems, - }; -} - -export function serialize(manifest: Manifest): string { - return `${JSON.stringify(manifest, null, 2)}\n`; -} - -/** Snippet ids the JavaScript docs pages use, from their EmbeddedCode tags. Undefined when the dir is absent. */ -export function docsIds(docsDir: string): { ids: Map; files: number } | undefined { - const root = path.join(docsDir, 'docs', 'sdks', 'javascript'); - if (!fs.existsSync(root)) return undefined; - const ids = new Map(); - let files = 0; - const add = (id: string, where: string): void => { - ids.set(id, [...(ids.get(id) ?? []), where]); - }; - const walk = (dir: string): string[] => - fs - .readdirSync(dir, { withFileTypes: true }) - .flatMap((e) => - e.isDirectory() ? walk(path.join(dir, e.name)) : /\.mdx?$/.test(e.name) ? [path.join(dir, e.name)] : [], - ); - for (const file of walk(root)) { - files += 1; - const rel = path.relative(docsDir, file); - const text = fs.readFileSync(file, 'utf8'); - // Only EmbeddedCode tags with a JavaScript SDK repository URL count. - for (const tag of text.matchAll(//g)) { - if (!/pubnub\/javascript\//.test(tag[0])) continue; - const list = /snippets=\{\[([^\]]*)\]\}/.exec(tag[0]); - const single = /snippet=\{?['"]([^'"]+)['"]\}?/.exec(tag[0]); - const found = list ? [...list[1].matchAll(/['"]([^'"]+)['"]/g)].map((m) => m[1]) : single ? [single[1]] : []; - for (const id of found) add(id, rel); - } - } - return { ids, files }; -} diff --git a/docs-snippets/harness/markers.ts b/docs-snippets/harness/markers.ts deleted file mode 100644 index bcc165644..000000000 --- a/docs-snippets/harness/markers.ts +++ /dev/null @@ -1,117 +0,0 @@ -/** - * Snippet marker scanner. It mirrors the extractor the docs use (`src/helper/codeSnippetFetcher.ts` in the - * documentation repo): `// snippet.` starts a snippet, `// snippet.end` ends it, `// snippet.hide` and - * `// snippet.show` toggle visibility, and any other start marker silently ends the snippet before it. - * The scanner reports that last case as nesting. - */ - -// Matches the same marker syntax as the docs extractor. Ids may contain `/` (one docs id does). `LOOSE_MARKER` catches near -// misses such as `//snippet.foo bar` or `// end.snippet`, which the extractor would silently ignore. -const MARKER = /^\s*\/\/\s*snippet\.([A-Za-z0-9_/-]+)\s*$/; -const LOOSE_MARKER = /^\s*\/\/\s*(?:snippet\.|end\.snippet)/; - -export interface HiddenBlock { - /** 1-based line of the `snippet.hide` marker. */ - hideLine: number; - /** 1-based line of the `snippet.show` marker, or undefined when it is missing. */ - showLine?: number; -} - -export interface SnippetRegion { - id: string; - /** 1-based line of the start marker. */ - startLine: number; - /** 1-based line of the end marker, or the line that implicitly closed it. */ - endLine: number; - explicitEnd: boolean; - hidden: HiddenBlock[]; -} - -export interface MarkerProblem { - line: number; - message: string; -} - -export interface ScanResult { - regions: SnippetRegion[]; - problems: MarkerProblem[]; -} - -/** - * One pass over a file. Returns every closed region plus problems (nesting, missing end, hide without show). Line numbers - * are 1-based. Shared by the lint, the stage rewrite and the compare tool, so all three agree on what a snippet is. - */ -export function scanMarkers(text: string): ScanResult { - const lines = text.split('\n'); - const regions: SnippetRegion[] = []; - const problems: MarkerProblem[] = []; - let open: SnippetRegion | undefined; - let openHide: HiddenBlock | undefined; - - const close = (line: number, explicit: boolean): void => { - if (!open) return; - open.endLine = line; - open.explicitEnd = explicit; - if (openHide) { - problems.push({ line: openHide.hideLine, message: `snippet.hide in "${open.id}" has no snippet.show` }); - openHide = undefined; - } - regions.push(open); - open = undefined; - }; - - lines.forEach((raw, index) => { - const line = index + 1; - const match = raw.match(MARKER); - if (!match) { - if (LOOSE_MARKER.test(raw)) { - problems.push({ line, message: `marker-like comment is not a valid marker: ${raw.trim()}` }); - } - return; - } - const word = match[1]; - if (word === 'end') { - if (!open) problems.push({ line, message: 'snippet.end without an open snippet' }); - else close(line, true); - } else if (word === 'hide') { - if (!open) problems.push({ line, message: 'snippet.hide outside a snippet' }); - else if (openHide) problems.push({ line, message: 'snippet.hide inside a hidden block' }); - else { - openHide = { hideLine: line }; - open.hidden.push(openHide); - } - } else if (word === 'show') { - if (!open || !openHide) problems.push({ line, message: 'snippet.show without a matching snippet.hide' }); - else { - openHide.showLine = line; - openHide = undefined; - } - } else { - if (open) { - problems.push({ - line, - message: `snippet "${word}" starts inside snippet "${open.id}" (nesting silently truncates the outer snippet)`, - }); - close(line, false); - } - open = { id: word, startLine: line, endLine: line, explicitEnd: false, hidden: [] }; - } - }); - - if (open) { - const dangling: SnippetRegion = open; - problems.push({ line: dangling.startLine, message: `snippet "${dangling.id}" has no snippet.end` }); - close(lines.length + 1, false); - } - return { regions, problems }; -} - -/** True when `line` (1-based) is a visible line inside `region`, not a marker and not hidden. */ -export function isVisibleLine(region: SnippetRegion, line: number): boolean { - if (line <= region.startLine || line >= region.endLine) return false; - return !region.hidden.some((h) => line >= h.hideLine && line <= (h.showLine ?? region.endLine)); -} - -export function isInsideRegion(region: SnippetRegion, line: number): boolean { - return line > region.startLine && line < region.endLine; -} diff --git a/docs-snippets/harness/mocha-entry.ts b/docs-snippets/harness/mocha-entry.ts deleted file mode 100644 index de893dd3d..000000000 --- a/docs-snippets/harness/mocha-entry.ts +++ /dev/null @@ -1,351 +0,0 @@ -/** - * The Mocha spec file: where the registry becomes tests. It builds the `World`, registers every page, selects the steps of - * this run and creates one `it()` per step in phase order. It is loaded from the staged copy - * `docs-snippets/.run//harness/`, never from the source tree. It is not named `*.test.ts`, so the SDK's root - * `.mocharc.json` never picks it up, and it never loads `test/setup-why.ts` (that file restores nock). - * - * Per step it: sets the role, decides skip (registry.skipReason), runs the body, waits one tick for stray promise - * rejections (guard.ts), runs the honesty checks, records the result for later `needs`, and tags the Mocha test with - * `pnMeta`, `pnReason` and `pnNote` so reporter.cjs can write the JSON. - * - * Honesty checks after a passing body (`afterStep`): every snippet id of the step reached its `snippet.end` line (the - * staged copy calls `__snippetRan('')` there, see stage.ts), and the step recorded at least one assertion. They run - * inside the test and not in a Mocha `afterEach` hook on purpose: a failing hook makes Mocha abort every later test of the - * suite, a failing check here fails only this step. - * - * Selection: `DOCS_SNIPPETS_SHARD` (a role), `DOCS_SNIPPETS_GREP` (set by the CLI from `--grep`) and `DOCS_SNIPPETS_ONLY` - * (exact ids) choose steps. A chosen step's `needs` that the filter left out run on demand before it, with their own role's - * keys, and are reported as prerequisites, not as passed tests. - * - * Root hooks: `before` closes any leftover application-level grant on the secured keysets this run uses and proves that a - * stranger is refused. `after` runs the janitor, proves the same again, and fails the run on residue or a critical cleanup - * error. - */ -import fs from 'node:fs'; -import { fileURLToPath } from 'node:url'; -import { drainGuard, describeError, explainError, installGuard } from './guard.js'; -import { loadEnv, securedKeysets, type Role } from './env.js'; -import { registry, rolesOf, selectSteps, stepTitle, type StepDef } from './registry.js'; -import { World } from './world.js'; -import { runJanitor } from './janitor.js'; -import { closeAndVerifyApplicationGrant, revokeApplicationGrantOnKeyset } from './revoke.js'; -import { recorder } from './recorder.js'; -import assert from './assert.js'; -import { installQuiet, mute, unmute } from './quiet.js'; -import { installShutdown } from './signals.js'; -import { installRedaction } from './redact.cjs'; -import { SkipStep } from './step.js'; -import { registerAll } from '../run.js'; - -/** - * DOCS_SNIPPETS_DRY=1 swaps every step body for a no-op so the Mocha binding, the reporter and the skip logic can be - * exercised without a network. DOCS_SNIPPETS_DRY_FAIL names one step that throws. Used by `cli.ts dry` only. - */ -const dry = process.env.DOCS_SNIPPETS_DRY === '1'; -const dryFail = process.env.DOCS_SNIPPETS_DRY_FAIL; - -installRedaction(); -const env = loadEnv(); -const world = new World(env, env.runId); -installGuard(); -// SDK client logs printed by snippets are collected per step (quiet.ts). A user who set a log level or -// DOCS_SNIPPETS_VERBOSE=1 wants to see them, so nothing is muted then. -installQuiet({ - enabled: process.env.DOCS_SNIPPETS_LOG_LEVEL === undefined && process.env.DOCS_SNIPPETS_VERBOSE !== '1', -}); -registry.reset(); -registry.availability = dry - ? { open: true, pam: true, datasync: true } - : { open: Boolean(env.open), pam: Boolean(env.pam), datasync: Boolean(env.datasync) }; -registerAll(world); -const wiring = registry.validate(); -if (wiring.length > 0) throw new Error(`step wiring is broken:\n${wiring.join('\n')}`); - -const only = process.env.DOCS_SNIPPETS_ONLY?.split(',') - .map((id) => id.trim()) - .filter(Boolean); -const shard = env.shard; -const selection = selectSteps(registry, { shard, grep: process.env.DOCS_SNIPPETS_GREP || undefined, only }); -/** Roles that run in this process (selected steps and their prerequisites). Only their keysets are guarded and cleaned. */ -const usedRoles: Set = rolesOf(registry, selection); - -const startedAt = Date.now(); - -// Published for the reporter (same process): the full plan and the selection, so the report can list prerequisites and -// steps that a filter left out. -(globalThis as { __pnDocsPlan?: unknown }).__pnDocsPlan = registry.ordered().map((def) => ({ - id: def.id, - page: def.page, - phase: def.phase, - kind: def.kind, - snippets: def.snippets, - keyset: def.keyset, - proves: def.proves, - selected: selection.run.has(def.id) && !selection.prerequisites.has(def.id), - prerequisite: selection.prerequisites.has(def.id), -})); -(globalThis as { __pnDocsSelection?: unknown }).__pnDocsSelection = { - shard: shard ?? null, - grep: process.env.DOCS_SNIPPETS_GREP ?? null, - only: only ?? null, - mutate: process.env.DOCS_SNIPPETS_MUTATE ?? null, - roles: [...usedRoles], -}; - -// Snippet ids that reached their `snippet.end` line in the running step (the staged copy calls this, see stage.ts). -const ran = new Set(); -(globalThis as { __snippetRan?: (id: string) => void }).__snippetRan = (id: string): void => { - ran.add(id); -}; -// Assertions recorded before the first snippet of the step started (fixtures, `watch()` waits). They do not count for the -// zero-assertion check, which asks whether the step asserted anything about the docs code. -let beforeSnippet: number | undefined; -(globalThis as { __snippetStart?: (id: string) => void }).__snippetStart = (): void => { - if (beforeSnippet === undefined) beforeSnippet = recorder.total; -}; - -/** - * Debug aid: DOCS_SNIPPETS_INJECT_FAILURE=[,] runs the step normally, then makes it throw an SDK-style 403. - * It exists to look at the failure part of the report. The CLI refuses it in CI (policy.ts). - */ -const injected = new Set( - (process.env.DOCS_SNIPPETS_INJECT_FAILURE ?? '') - .split(',') - .map((id) => id.trim()) - .filter(Boolean), -); -function injectedError(id: string): Error { - const error = new Error('REST API request processing error, check status for details'); - error.name = 'PubNubError'; - (error as Error & { status?: unknown }).status = { - error: true, - category: 'PNAccessDeniedCategory', - operation: 'PNPublishOperation', - statusCode: 403, - errorData: { - message: `Forbidden (injected by DOCS_SNIPPETS_INJECT_FAILURE for step ${id})`, - status: 403, - service: 'Access Manager', - }, - }; - return error; -} - -/** Secured keysets this run touches, without duplicates. */ -const guarded = securedKeysets(env).filter( - (k) => usedRoles.has(k.role) || (env.sharedSecured && usedRoles.has('datasync')), -); - -/** Proves that a stranger (no auth key, no token) is refused on every guarded keyset. Used before and after the steps. */ -async function proveNoApplicationGrant(when: string): Promise { - for (const { role, keys } of guarded) { - const waited = await closeAndVerifyApplicationGrant(keys, `${when}-r${world.runId}`); - console.log( - `${env.sharedSecured ? 'pam and datasync' : role} keyset: a client without a token is refused (${when}${waited.attempts > 1 ? `, after ${Math.round(waited.ms / 1000)} s` : ''})`, - ); - } -} - -if (!dry) { - installShutdown( - // First and alone: close the application-level grant on every secured keyset this run uses. One quick call each, so it - // finishes before a CI cancel escalates to SIGKILL. - async () => { - // Stop collecting console output (quiet.ts): from here on every line of the cleanup must be visible. - unmute(); - // All keysets are tried, a failure on one never skips the others. The first failure is thrown afterwards. - const results = await Promise.allSettled( - guarded.map(async ({ role, keys }) => { - await revokeApplicationGrantOnKeyset(keys, `signal-r${world.runId}`, 1); - process.stderr.write(`${role} keyset: application-level grant revoked\n`); - }), - ); - const failed = results.find((r): r is PromiseRejectedResult => r.status === 'rejected'); - if (failed) throw failed.reason; - }, - async () => { - const report = await runJanitor(env, world.runId, new Date(startedAt), [...usedRoles]); - for (const line of report.log) console.log(`janitor: ${line}`); - await world.dispose(); - }, - ); - if (process.env.DOCS_SNIPPETS_PRINT_RESOLVE !== '0') { - // The suite tests this repository's build, never a package from npm. - console.log(`pubnub resolves to ${fs.realpathSync(fileURLToPath(import.meta.resolve('pubnub')))}`); - } -} - -/** Honesty checks of a passing step, see the header. */ -function afterStep(def: StepDef, assertions: number): void { - if (def.kind !== 'step' || dry) return; - const missing = def.snippets.filter((id) => !ran.has(id)); - if (missing.length > 0) { - throw new Error( - `honesty check: snippet ${missing.join(', ')} of step ${def.id} did not reach its snippet.end line, so the docs code did not run to the end`, - ); - } - if (assertions === 0) { - throw new Error( - `honesty check: step ${def.id} recorded zero assertions after its snippet started, so it proves nothing about what the docs page claims`, - ); - } -} - -describe('JavaScript SDK docs snippets', function () { - this.timeout(30_000); - - for (const def of registry.ordered()) { - if (!selection.run.has(def.id)) continue; - const prerequisite = selection.prerequisites.has(def.id); - const test = it(`${prerequisite ? '(prerequisite) ' : ''}${stepTitle(def)}`, async function () { - this.timeout(def.timeout); - world.useRole(def.keyset); - const reason = registry.skipReason(def); - if (reason) { - registry.record(def.id, { status: 'skipped', reason }); - (this.test as unknown as { pnReason?: string }).pnReason = reason; - this.skip(); - } - drainGuard(); - ran.clear(); - beforeSnippet = undefined; - const began = Date.now(); - const tagged = this.test as unknown as { - pnAssertions?: unknown; - pnDropped?: number; - pnSuppressed?: number; - pnOutput?: string[]; - }; - recorder.begin(); - mute(); - let collected: string[] = []; - let collectedTotal = 0; - let thrown: unknown; - let failedStep = false; - let assertionCount = 0; - try { - if (dry) { - await dryAssertions(def.id, def.id === dryFail); - if (injected.has(def.id)) throw injectedError(def.id); - } else { - await def.fn(); - if (injected.has(def.id)) throw injectedError(def.id); - } - // A snippet may leave a promise un-awaited. One macrotask later it has settled, and a rejection recorded by - // guard.ts fails this step instead of leaking into the next one. - await new Promise((resolve) => setImmediate(resolve)); - const stray = drainGuard(); - if (stray.length > 0) { - throw new Error( - `unhandled rejection during step ${def.id}: ${stray.map((s) => describeError(s.error)).join('; ')}`, - ); - } - } catch (error) { - thrown = error; - failedStep = true; - } finally { - const quietResult = unmute(); - collected = quietResult.lines; - collectedTotal = quietResult.total; - const afterStart = recorder.total - (beforeSnippet ?? 0); - const { assertions, dropped } = recorder.end(); - assertionCount = afterStart; - tagged.pnAssertions = assertions; - tagged.pnDropped = dropped; - tagged.pnSuppressed = collectedTotal; - if (failedStep && !(thrown instanceof SkipStep)) tagged.pnOutput = collected.slice(-60); - } - if (!failedStep) { - try { - afterStep(def, assertionCount); - } catch (error) { - thrown = error; - failedStep = true; - } - } - if (!failedStep) { - registry.record(def.id, { status: 'passed', durationMs: Date.now() - began }); - return; - } - const error = thrown; - if (error instanceof SkipStep) { - registry.record(def.id, { status: 'skipped', reason: error.reason }); - (this.test as unknown as { pnReason?: string; pnNote?: string }).pnReason = error.reason; - (this.test as unknown as { pnNote?: string }).pnNote = error.detail; - if (error.detail) console.log(`skipped ${def.id} (${error.reason}): ${error.detail}`); - this.skip(); - } - explainError(error); - registry.record(def.id, { status: 'failed', error: describeError(error), durationMs: Date.now() - began }); - if (collected.length > 0) - console.error( - `console output of ${def.id} before it failed (last ${Math.min(collected.length, 30)} of ${collectedTotal} lines):\n${collected.slice(-30).join('\n')}`, - ); - throw error; - }); - (test as unknown as { pnMeta: unknown }).pnMeta = { - id: def.id, - page: def.page, - phase: def.phase, - kind: def.kind, - snippets: def.snippets, - keyset: def.keyset, - needs: def.needs, - proves: def.proves, - prerequisite, - }; - } - - // A crashed earlier run, or a grant of another job on a shared keyset, may leave a secured keyset open. Steps such as - // `publishHandleErrors` need the keyset to really deny a client without a token, so prove it before the first step. - before(async function () { - this.timeout(150_000); - if (dry) return; - await proveNoApplicationGrant('before'); - }); - - after(async function () { - this.timeout(240_000); - try { - if (!dry && (env.open || env.pam || env.datasync)) { - const report = await runJanitor(env, world.runId, new Date(startedAt), [...usedRoles]); - for (const line of report.log) console.log(`janitor: ${line}`); - if (report.critical.length > 0) { - throw new Error(`janitor could not make the keysets safe: ${report.critical.join('; ')}`); - } - // Residue = objects created during the run whose id has no run suffix, so a snippet wrote a literal that is missing - // from fixtures/names.ts and the next run would collide with it. - if (report.residue.length > 0) { - throw new Error( - `residue: ${report.residue.length} object(s) created during the run have no run suffix, so a snippet uses ` + - `a literal missing from fixtures/names.ts: ${report.residue.join(', ')}`, - ); - } - // The janitor revoked the application-level grant. Wait until a stranger is refused again, so the next job on a - // shared keyset starts closed. - await proveNoApplicationGrant('after'); - } - } finally { - mute(); - await world.dispose(); - await new Promise((resolve) => setImmediate(resolve)); - unmute(); - } - }); -}); - -/** - * Dry run only: makes a no-op step look like a real one for the report, so the dry run proves that assertions, failures and - * descriptions reach report/run.json without a network. `fail` ends in a failing assertion. - */ -async function dryAssertions(id: string, fail: boolean): Promise { - assert.equal(1 + 1, 2, `dry run assertion of ${id}`); - assert.ok(true, 'dry run: ok assertion'); - if (fail) - assert.equal( - 1, - 1, - 'dry run: label with a key pub-c-0123abcd-0123-4abc-8def-0123456789ab and token p0F2AkF0Gl' + 'A'.repeat(30), - ); - if (fail) assert.deepEqual(['actual'], ['expected'], 'dry run: simulated failing assertion'); -} diff --git a/docs-snippets/harness/policy.ts b/docs-snippets/harness/policy.ts deleted file mode 100644 index e220bf90d..000000000 --- a/docs-snippets/harness/policy.ts +++ /dev/null @@ -1,63 +0,0 @@ -import type { Env } from './env.js'; - -/** - * Run policy: what turns a technically green Mocha run into a failed job. Pure functions, so the dry run tests every rule - * without keys. `cli.ts` calls `launchProblems` before the run and `outcomeProblems` after it. - * - * - A skip fails the run, always: locally, in CI, nightly. A run must run every valid test. A skip means a keyset lacks a - * feature or class, keys are missing, or an upstream step failed, and each of those is a defect to fix, not to tolerate. - * There is no budget, no allowed list and no opt-out. - * - In CI, every subset option is refused: `--grep`, `DOCS_SNIPPETS_ONLY`, `DOCS_SNIPPETS_MUTATE`, - * `DOCS_SNIPPETS_INJECT_FAILURE`. A subset run must never pass as the full run. A shard is not a subset: the workflow runs - * all three. - * - In CI, `trace` and `debug` log levels are refused because the SDK logs its configuration at those levels. - */ - -export interface PolicyReport { - skips: Array<{ id: string; reason: string; note?: string }>; -} - -/** Options that select a subset of the plan or break a step on purpose. Each is refused in CI. */ -export const SUBSET_VARS = ['DOCS_SNIPPETS_ONLY', 'DOCS_SNIPPETS_MUTATE', 'DOCS_SNIPPETS_INJECT_FAILURE'] as const; - -export function launchProblems( - env: Env, - vars: Record = process.env, - args: { grep?: string } = {}, -): string[] { - const problems: string[] = []; - if (env.shardProblem) problems.push(env.shardProblem); - if (env.ci) { - if (args.grep) problems.push('--grep is set in CI. A subset run must not pass as the full run. Remove it.'); - for (const name of SUBSET_VARS) { - if (vars[name]) - problems.push( - `${name} is set in CI. It selects a subset or breaks a step on purpose, which must never pass as the full run. Unset it.`, - ); - } - if (env.logLevel === 'trace' || env.logLevel === 'debug') { - problems.push( - `DOCS_SNIPPETS_LOG_LEVEL=${env.logLevel} is refused in CI. The SDK logs its configuration at that level, which includes keys. Use info or lower.`, - ); - } - } - return problems; -} - -export function outcomeProblems(report: PolicyReport): string[] { - if (report.skips.length === 0) return []; - const byReason = new Map(); - for (const skip of report.skips) byReason.set(skip.reason, [...(byReason.get(skip.reason) ?? []), skip.id]); - const lines = [...byReason].map(([reason, ids]) => ` ${reason}: ${ids.join(', ')}`); - const hints = new Set(); - for (const reason of byReason.keys()) { - if (reason.startsWith('missing-env:')) - hints.add(`set the ${reason.slice('missing-env:'.length)} keys (docs-snippets/.env.local locally, the SDK_DOCS_* secrets in CI)`); - else if (reason.startsWith('needs-feature:')) - hints.add(`turn ${reason.slice('needs-feature:'.length)} on for the keyset (see "Features each keyset needs" in docs-snippets/README.md)`); - else if (reason.startsWith('blocked-by:')) hints.add('fix the step a blocked-by reason names, the steps that need it skip until then'); - } - return [ - `${report.skips.length} step(s) were skipped. A run must run every valid test, so any skip fails it:\n${lines.join('\n')}\nTo fix: ${[...hints].join('. ')}.`, - ]; -} diff --git a/docs-snippets/harness/preflight-errors.ts b/docs-snippets/harness/preflight-errors.ts deleted file mode 100644 index 575e1d369..000000000 --- a/docs-snippets/harness/preflight-errors.ts +++ /dev/null @@ -1,75 +0,0 @@ -/** - * Role: used by preflight.ts to produce the failure message, and by the dry run to test every cause without a network. - * - * Turns an SDK error from a preflight call into one named cause. Before this, every failure read "the keyset does not - * have ", which was wrong for a mistyped key or an offline laptop. - * - * Causes, in the order they are tested: - * - network: no HTTP status, or a network and timeout category. Nothing is wrong with the keyset. - * - bad-key: HTTP 400 or 401 whose service message names the key, or 401. Check the variable that holds the key. - * - feature-off: HTTP 403. The key is fine and the portal feature is off (or Access Manager needs a token). - * - bad-request: any other 400. Not a missing feature. Look at the service message. - * - unknown: anything else. - */ - -export type PreflightCause = 'network' | 'bad-key' | 'feature-off' | 'bad-request' | 'unknown'; - -interface SdkLikeError { - message?: string; - code?: string; - status?: { category?: string; statusCode?: number; errorData?: unknown; message?: string }; -} - -const NETWORK_CATEGORIES = new Set(['PNNetworkIssuesCategory', 'PNNetworkDownCategory', 'PNTimeoutCategory', 'PNConnectionErrorCategory']); -const NETWORK_CODES = new Set(['ENOTFOUND', 'ECONNREFUSED', 'ECONNRESET', 'ETIMEDOUT', 'EAI_AGAIN', 'EHOSTUNREACH']); - -function serviceText(error: SdkLikeError): string { - const data = error.status?.errorData; - let text = ''; - try { - text = typeof data === 'string' ? data : JSON.stringify(data ?? ''); - } catch { - text = String(data); - } - return `${text} ${error.status?.message ?? ''} ${error.message ?? ''}`; -} - -export function classifyPreflightError(error: unknown): { cause: PreflightCause; detail: string } { - const e = (error ?? {}) as SdkLikeError; - const code = e.status?.statusCode; - const category = e.status?.category; - const text = serviceText(e); - const detail = [category, code, e.message].filter(Boolean).join(' '); - if ((category && NETWORK_CATEGORIES.has(category)) || (e.code && NETWORK_CODES.has(e.code)) || (code === undefined && /ENOTFOUND|ECONNREFUSED|ETIMEDOUT|network|fetch failed|timed out/i.test(text))) { - return { cause: 'network', detail }; - } - if (code === 401 || ((code === 400 || code === 403) && /invalid (subscribe |publish )?key|invalid subscribe|invalid publish|key.*not found|no such key|invalid.*subscribe.?key/i.test(text))) { - return { cause: 'bad-key', detail }; - } - if (code === 403) return { cause: 'feature-off', detail }; - if (code === 400) return { cause: 'bad-request', detail }; - return { cause: 'unknown', detail }; -} - -/** The message a failed preflight throws. `feature` and `setting` are used only when the cause is a missing feature. */ -export function preflightMessage( - error: unknown, - keyset: 'open' | 'pam' | 'datasync', - feature: string, - setting: string, -): string { - const { cause, detail } = classifyPreflightError(error); - const vars = keyset === 'open' ? 'PUBLISH_KEY and SUBSCRIBE_KEY' : keyset === 'pam' ? 'PAM_PUBLISH_KEY, PAM_SUBSCRIBE_KEY and PAM_SECRET_KEY' : 'DS_PUBLISH_KEY, DS_SUBSCRIBE_KEY and DS_SECRET_KEY'; - switch (cause) { - case 'network': - return `Network problem, not a keyset problem: PubNub could not be reached while checking ${feature} on the ${keyset} keyset. Check connectivity and proxy settings, then rerun. ${detail}`; - case 'bad-key': - return `The ${keyset} keyset keys were rejected, so the keys are wrong or belong to another keyset. Check ${vars}. ${detail}`; - case 'feature-off': - return `The ${keyset} keyset does not have ${feature}. The keys work. Enable "${setting}" for this keyset in the Admin Portal (see docs-snippets/README.md, "Keysets"). ${detail}`; - case 'bad-request': - return `The service answered 400 while checking ${feature} on the ${keyset} keyset. That is not a missing feature, so look at the service message first. ${detail}`; - default: - return `Could not verify ${feature} on the ${keyset} keyset (unclassified error). ${detail}`; - } -} diff --git a/docs-snippets/harness/preflight-summary.ts b/docs-snippets/harness/preflight-summary.ts deleted file mode 100644 index 2f28eb5a7..000000000 --- a/docs-snippets/harness/preflight-summary.ts +++ /dev/null @@ -1,190 +0,0 @@ -import { explainError } from './guard.js'; -import assert from './assert.js'; -import { readFeatures, missingFeatures } from './keyset-features.js'; -import { registry } from './registry.js'; -import { page, check } from './step.js'; -import { eventually, watch } from './wait.js'; -import type { Role } from './env.js'; - -/** - * One message per keyset that names every missing feature (design rule: "Presence is off on the open keyset", never a - * guess from a failed call). Registered last in run.ts, so these checks run after every other P0 preflight. - * - * Per role: - * 1. With PUBNUB_ADMIN_API_KEY and a keyset id, read the keyset configuration through the Admin API (read only, - * keyset-features.ts). Without them, this part is skipped and says so. - * 2. Confirm each feature with one SDK probe. Features with their own preflight step (Presence, Message Persistence, - * Delete-From-History, App Context, Files, Mobile Push) are not probed twice: their recorded results are collected. - * The features without one are probed here: Stream Controller wildcard subscribe and App Context events on the open - * keyset, Access Manager with token revocation on the pam keyset, Access Manager and the DataSync classes on the datasync - * keyset. - * 3. Fail once, with every missing feature in one message. Nothing is ever changed on a keyset. - * - * DataSync event rules have no API. They stay a probe in api-reference/datasync.ts (`datasyncEvents`, skips as - * needs-feature:datasync-events). - */ -const OTHER_PREFLIGHTS: Record = { - preflightKeyset: 'working keys (publish)', - preflightPersistence: 'Message Persistence', - preflightDeleteFromHistory: 'Message Persistence Delete-From-History', - preflightPresence: 'Presence with tracking on all channels', - preflightAppContext: 'App Context', - preflightFiles: 'Files', - preflightMobilePush: 'Mobile Push Notifications', -}; - -export default page('harness/preflight-summary', 'P0', (world) => { - /** Admin API part, shared by the three checks. Returns missing features and a note for the log. */ - const adminPart = async (role: Role): Promise<{ missing: string[]; note: string }> => { - try { - const read = await readFeatures(world.env, role); - if (read.status === 'read') - return { - missing: missingFeatures(role, read.config), - note: `${role} keyset configuration read through the Admin API`, - }; - if (read.status === 'mismatch') - return { missing: [`the Admin API keyset id for role ${role} is wrong: ${read.why}`], note: '' }; - return { missing: [], note: `Admin API read of the ${role} keyset skipped (${read.why}), SDK probes only` }; - } catch (error) { - return { - missing: [], - note: `Admin API read of the ${role} keyset failed (${(error as Error).message}), SDK probes only`, - }; - } - }; - - const finish = (role: Role, missing: string[], notes: string[]): void => { - for (const note of notes.filter(Boolean)) console.log(note); - // Each feature once, even when the Admin API and a probe both report it. - const unique = [...new Set(missing)]; - assert.deepEqual(unique, [], `features of the ${role} keyset`); - }; - - check( - 'preflightFeaturesOpen', - { - timeout: 90_000, - proves: 'Preflight: the open keyset has every feature the open samples need, named in one message when not.', - }, - async () => { - const { missing, note } = await adminPart('open'); - for (const [id, feature] of Object.entries(OTHER_PREFLIGHTS)) { - const result = registry.result(id); - if (result?.status === 'failed') - missing.push(`${feature} is not usable on the open keyset (preflight ${id} failed: ${result.error ?? ''})`); - if (result?.status === 'skipped' && result.reason?.startsWith('needs-feature:')) - missing.push(`${feature} is off on the open keyset (${result.reason})`); - } - // Stream Controller with wildcard subscribe: a wildcard subscription receives a publish to a matching channel. - try { - const root = world.n('preflight_wild'); - const listener = await watch(world.fresh('preflightwild'), [`${root}.*`]); - try { - await world.observer.publish({ channel: `${root}.probe`, message: { wildcard: true } }); - await listener.next('message', (m) => m.channel === `${root}.probe`, { timeout: 10_000 }); - } finally { - listener.stop(); - } - } catch (error) { - explainError(error); - missing.push( - `Stream Controller wildcard subscribe is not working on the open keyset (${(error as Error).message})`, - ); - } - // App Context events: a subscriber of a channel receives the objects event of its channel metadata. - try { - const channel = world.n('preflight_channel'); - const listener = await watch(world.fresh('preflightobjects'), [channel]); - try { - await world.observer.objects.setChannelMetadata({ channel, data: { name: 'preflight' } }); - await listener.next('objects', (e) => e.channel === channel, { timeout: 15_000 }); - } finally { - listener.stop(); - await world.observer.objects.removeChannelMetadata({ channel }).catch(() => undefined); - } - } catch (error) { - explainError(error); - missing.push( - `App Context channel metadata events are not delivered on the open keyset (${(error as Error).message})`, - ); - } - finish('open', missing, [note]); - }, - ); - - /** Access Manager probe for a secured role: grant a token, and with `revoke` also revoke it. */ - const accessManagerProbe = async (role: 'pam' | 'datasync', revoke: boolean, missing: string[]): Promise => { - let token: string | undefined; - try { - token = await world.server.grantToken({ - ttl: 1, - authorized_uuid: world.n('am-v2-user'), - resources: { channels: { [world.n('am_other')]: { read: true } } }, - }); - assert.ok(token, 'grantToken returned a token'); - } catch (error) { - explainError(error); - missing.push(`Access Manager is not usable on the ${role} keyset (${(error as Error).message})`); - return; - } - if (!revoke) return; - try { - await world.server.revokeToken(token); - } catch (error) { - explainError(error); - missing.push(`Access Manager token revocation is off on the ${role} keyset (${(error as Error).message})`); - } - }; - - check( - 'preflightFeaturesPam', - { - keyset: 'pam', - timeout: 60_000, - proves: 'Preflight: the pam keyset has Access Manager with token revocation, named in one message when not.', - }, - async () => { - const { missing, note } = await adminPart('pam'); - await accessManagerProbe('pam', true, missing); - finish('pam', missing, [note]); - }, - ); - - check( - 'preflightFeaturesDatasync', - { - keyset: 'datasync', - timeout: 60_000, - proves: - 'Preflight: the datasync keyset has Access Manager, DataSync and the product and ProductOwner classes, named in one message when not.', - }, - async () => { - const { missing, note } = await adminPart('datasync'); - await accessManagerProbe('datasync', false, missing); - const ds = world.server.dataSync; - try { - await ds.getUsers({ limit: 1 }); - } catch (error) { - explainError(error); - missing.push(`DataSync is not usable on the datasync keyset (${(error as Error).message})`); - } - for (const [label, read] of [ - ['class product v1', () => ds.getEntities({ class: 'product', classVersion: 1, limit: 1 })], - ['class ProductOwner v1', () => ds.getRelationships({ class: 'ProductOwner', classVersion: 1, limit: 1 })], - ] as const) { - try { - await eventually( - async () => { - await read(); - }, - { timeout: 5_000, label }, - ); - } catch (error) { - missing.push(`DataSync ${label} is missing on the datasync keyset (run npm run docs-snippets:provision)`); - } - } - finish('datasync', missing, [note, 'DataSync event rules have no API, the step datasyncEvents probes them']); - }, - ); -}); diff --git a/docs-snippets/harness/preflight.ts b/docs-snippets/harness/preflight.ts deleted file mode 100644 index ec2ab048c..000000000 --- a/docs-snippets/harness/preflight.ts +++ /dev/null @@ -1,64 +0,0 @@ -import assert from './assert.js'; -import { explainError } from './guard.js'; -import { preflightMessage } from './preflight-errors.js'; -import { page, check } from './step.js'; -import { eventually, PRESENCE_TIMEOUT, watch } from './wait.js'; - -/** - * Keyset capability checks. The first live run showed that a keyset without Presence, - * Storage Delete or App Context makes many snippets fail with the same opaque 403. A preflight turns that into - * one named failure, and steps that `needs` it are skipped as `blocked-by:`, so one root cause - * shows as one failure. - */ -export default page('harness/preflight', 'P0', (world) => { - // One throw per cause, with a message that names the Admin Portal setting. `explainError` first appends the service's - // own message to the SDK's generic one, so the report line explains itself. - const fail = (feature: string, setting: string, error: unknown): never => { - explainError(error); - throw new Error(preflightMessage(error, 'open', feature, setting)); - }; - - // Keys and network first, so a mistyped key or an offline machine reads as that and not as three missing features. - check('preflightKeyset', { proves: 'Preflight: the open keyset keys work and the network is reachable, shown by a publish.' }, async () => { - try { - await world.observer.publish({ channel: world.n('preflight_channel'), message: { preflight: true } }); - } catch (error) { - fail('working keys (publish and subscribe)', 'a publish and subscribe key pair', error); - } - }); - - // Each preflight below depends on the previous one, so one broken key shows one failure and the rest are blocked-by. - check('preflightPersistence', { needs: ['preflightKeyset'], proves: 'Preflight: Message Persistence is enabled on the open keyset.' }, async () => { - try { - await world.observer.fetchMessages({ channels: [world.n('preflight_channel')], count: 1 }); - } catch (error) { - fail('Message Persistence', 'Message Persistence', error); - } - }); - - check('preflightDeleteFromHistory', { needs: ['preflightPersistence'], proves: 'Preflight: Delete-From-History is enabled on the open keyset.' }, async () => { - // Deleting an empty range on an unused channel has no side effect. 403 means the setting is off. - const now = (await world.observer.time()).timetoken; - try { - await world.observer.deleteMessages({ channel: world.n('preflight_channel'), start: (BigInt(now) - 1n).toString(), end: now }); - } catch (error) { - fail('"delete from history"', 'Delete-From-History (Storage Delete)', error); - } - }); - - // Presence has no cheap read-only probe. Subscribe a fresh client and wait until hereNow lists it. That depends on - // heartbeats, hence the longer timeout and the short-heartbeat client options. - check('preflightPresence', { needs: ['preflightKeyset'], timeout: 60_000, proves: 'Preflight: Presence is enabled and tracks a subscribed client in hereNow.' }, async () => { - const channel = world.n('preflight_channel'); - const client = world.fresh('preflightpresence', world.presenceOptions); - await watch(client, [channel]); - try { - await eventually(async () => { - const here = await world.observer.hereNow({ channels: [channel] }); - assert.equal(here.channels[channel]?.occupancy, 1, 'hereNow does not list the subscribed client'); - }, { timeout: PRESENCE_TIMEOUT, label: 'client visible in hereNow' }); - } catch (error) { - fail('Presence', 'Presence', error); - } - }); -}); diff --git a/docs-snippets/harness/proxy.ts b/docs-snippets/harness/proxy.ts deleted file mode 100644 index 68ff68b63..000000000 --- a/docs-snippets/harness/proxy.ts +++ /dev/null @@ -1,75 +0,0 @@ -import http from 'node:http'; -import net from 'node:net'; -import type { AddressInfo } from 'node:net'; - -/** - * Local HTTP CONNECT proxy on 127.0.0.1 (`setProxy`). It tunnels to the real host and - * counts CONNECT requests, so a step can prove traffic went through the proxy and stopped after `setProxy()`. - * `outage(true)` refuses new tunnels and destroys open ones, for the later status-event steps. - */ -// An HTTP CONNECT proxy is enough because the SDK talks TLS to the service. The proxy never sees plaintext, it only pipes bytes. -// `http.createServer` rejects plain requests with 405 for the same reason. -export class LocalProxy { - private server?: http.Server; - private sockets = new Set(); - private blocked = false; - connects = 0; - port = 0; - readonly host = '127.0.0.1'; - - /** Listens on an ephemeral port (`listen(0)`), so parallel runs never collide. Idempotent. */ - async start(): Promise { - if (this.server) return; - this.server = http.createServer((_req, res) => { - res.writeHead(405).end('CONNECT only'); - }); - this.server.on('connect', (req, clientSocket: net.Socket, head) => { - this.connects += 1; - if (this.blocked) { - clientSocket.end('HTTP/1.1 503 Service Unavailable\r\n\r\n'); - return; - } - const [hostname, port] = (req.url ?? '').split(':'); - const upstream = net.connect(Number(port) || 443, hostname, () => { - clientSocket.write('HTTP/1.1 200 Connection Established\r\n\r\n'); - upstream.write(head); - upstream.pipe(clientSocket); - clientSocket.pipe(upstream); - }); - // Track both ends so `outage(true)` and `stop()` can cut established tunnels, which a refused new CONNECT would not. - this.sockets.add(clientSocket); - this.sockets.add(upstream); - const drop = (): void => { - this.sockets.delete(clientSocket); - this.sockets.delete(upstream); - clientSocket.destroy(); - upstream.destroy(); - }; - upstream.on('error', drop); - clientSocket.on('error', drop); - upstream.on('close', drop); - clientSocket.on('close', drop); - }); - await new Promise((resolve) => this.server!.listen(0, this.host, resolve)); - this.port = (this.server.address() as AddressInfo).port; - } - - /** Spread into `setProxy({...})` to point a snippet at this proxy. */ - get config(): Partial<{ hostname: string; port: number; protocol: string }> { - return { hostname: this.host, port: this.port, protocol: 'http' }; - } - - /** Simulates a network outage: refuses new tunnels with 503 and cuts the open ones, so the SDK reports an unexpected disconnect. */ - outage(on: boolean): void { - this.blocked = on; - if (on) for (const socket of this.sockets) socket.destroy(); - } - - async stop(): Promise { - for (const socket of this.sockets) socket.destroy(); - this.sockets.clear(); - const server = this.server; - this.server = undefined; - if (server) await new Promise((resolve) => server.close(() => resolve())); - } -} diff --git a/docs-snippets/harness/quiet.ts b/docs-snippets/harness/quiet.ts deleted file mode 100644 index 7cae6a417..000000000 --- a/docs-snippets/harness/quiet.ts +++ /dev/null @@ -1,64 +0,0 @@ -/** - * Keeps the console quiet while steps run, without losing anything. - * - * Why: docs snippets create clients with `logLevel: Debug` or `Trace` and the SDK's default console logger prints every - * entry (`PubNub-00ead8ce Trace EventEngine Leave all with parameters`), also later when `world.dispose()` destroys the - * client. A run with hundreds of such lines hides the Mocha result and is hard to redact by eye. The lines are not noise to - * the steps that assert them, those steps replace `console` themselves (`world.captureConsole`) or pass a `loggers` array. - * - * How: this module replaces the console methods ONCE per process with thin wrappers. While muted a wrapper stores the formatted - * line (a step can fetch them with `unmute()`), otherwise it calls the original. The wrappers are never removed, because - * `world.captureConsole` saves "the current console method" and restores it later, and that must stay valid. - * The stdout/stderr redaction (redact.cjs) is untouched: whatever finally prints is still masked, and the lines kept for the - * report are masked by the reporter. - * - * Off switch: set `DOCS_SNIPPETS_LOG_LEVEL` (you asked for logs) or `DOCS_SNIPPETS_VERBOSE=1` and nothing is muted. - */ -import { format } from 'node:util'; - -const METHODS = ['log', 'info', 'debug', 'warn', 'error', 'trace'] as const; -type Method = (typeof METHODS)[number]; - -let installed = false; -let enabled = true; -let muted = false; -let buffer: string[] = []; -// A Trace logger can print thousands of lines per step. Only the newest KEEP lines are stored, the total is still counted. -const KEEP = 500; -let total = 0; -const originals = {} as Record void>; - -/** Installs the wrappers. Idempotent. `enabled` false means "never mute" (you want to see the logs). */ -export function installQuiet(options: { enabled: boolean }): void { - enabled = options.enabled; - if (installed) return; - installed = true; - for (const method of METHODS) { - originals[method] = console[method].bind(console) as (...args: unknown[]) => void; - console[method] = (...args: unknown[]): void => { - if (muted && enabled) { - total += 1; - buffer.push(`[console.${method}] ${format(...args)}`); - if (buffer.length > KEEP) buffer.shift(); - } - else originals[method](...args); - }; - } -} - -/** Starts collecting console output instead of printing it. */ -export function mute(): void { - muted = true; - buffer = []; - total = 0; -} - -/** Stops collecting and returns the newest collected lines since `mute()` and how many lines there were in total. */ -export function unmute(): { lines: string[]; total: number } { - muted = false; - const lines = buffer; - const count = total; - buffer = []; - total = 0; - return { lines, total: count }; -} diff --git a/docs-snippets/harness/recorder.ts b/docs-snippets/harness/recorder.ts deleted file mode 100644 index e83b6b0be..000000000 --- a/docs-snippets/harness/recorder.ts +++ /dev/null @@ -1,123 +0,0 @@ -/** - * Assertion recorder: remembers every assertion a step executed, so the run report can show what a step really checked. - * - * Why it exists: the JSON report used to say "passed" or "failed" per step and nothing about what the step proved. A human - * reading a green run could not tell a step with ten assertions from a step with none. `harness/assert.ts` (the recorded - * drop-in for `node:assert/strict`) and the wait helpers (`eventually`, `expectEvent`, `connected`) call `record()`, the - * Mocha binding (mocha-entry.ts) brackets each step with `begin()` and `end()` and hands the result to reporter.cjs. - * - * Two details worth knowing: - * - Polling. `eventually` runs its callback many times and most attempts fail on purpose. Each attempt records into a private - * buffer (AsyncLocalStorage, so unrelated timers cannot leak into it). Only the buffer of the attempt that passed is kept. - * - Size. Repeated identical entries collapse into one entry with a count, and a step keeps at most MAX_ENTRIES entries. - * - * Nothing here changes whether an assertion passes. It only observes. - */ -import { AsyncLocalStorage } from 'node:async_hooks'; -import { inspect } from 'node:util'; -import { redact } from './redact.cjs'; - -/** One assertion (or one wait that behaves like one) as the report shows it. */ -export interface AssertionRecord { - /** `equal`, `deepEqual`, `ok`, `match`, `rejects`, `eventually`, `event`, `connected` and so on. */ - kind: string; - /** The message the step gave, or a generated one that shows the values. */ - label: string; - ok: boolean; - /** How many identical consecutive assertions this entry stands for. */ - count: number; - /** Failed assertions only: the values the assertion compared, already shortened. */ - expected?: string; - actual?: string; - /** Failed assertions only: the first stack frame inside the suite, as `file:line:column`. */ - frame?: string; -} - -const MAX_ENTRIES = 200; - -/** Shortens any value for a report line. Never throws, never prints more than `max` characters. */ -export function short(value: unknown, max = 80): string { - let text: string; - try { - // No maxStringLength: inspect would cut a long string inside an object, and a key cut in half escapes redaction. - text = typeof value === 'string' ? JSON.stringify(value) : inspect(value, { depth: 2, breakLength: Infinity, maxArrayLength: 8 }); - } catch { - text = String(value); - } - // Redact the whole text first, then shorten, so no key can be cut into a piece that matches neither its value nor its shape. - text = redact(text); - return text.length > max ? `${text.slice(0, max - 3)}...` : text; -} - -/** - * First stack frame that belongs to the suite (a page module, shared snippet or helper) and not to the recording layer, Node - * or a dependency. The staged copy lives in `.run//`, the prefix is cut so the frame names the source file, whose - * line numbers are the same (the stage rewrite changes string literals only, never lines). - */ -export function appFrame(stack: string | undefined): string | undefined { - if (!stack) return undefined; - for (const line of stack.split('\n').slice(1)) { - if (/node:internal|node_modules|harness\/(assert|recorder)\./.test(line)) continue; - const match = /\(?((?:file:\/\/)?[^()\s]+:\d+:\d+)\)?\s*$/.exec(line.trim()); - if (!match) continue; - return match[1].replace(/^file:\/\//, '').replace(/^.*?\.run\/[0-9a-z]+\//, '').replace(/^.*?\/docs-snippets\//, ''); - } - return undefined; -} - -class Recorder { - private entries: AssertionRecord[] | undefined; - private dropped = 0; - /** Assertions recorded into the running step so far, merged and dropped ones included. The honesty check reads it. */ - total = 0; - private readonly scope = new AsyncLocalStorage(); - - /** Starts a fresh list for the step that is about to run. */ - begin(): void { - this.entries = []; - this.dropped = 0; - this.total = 0; - } - - /** Returns what the step recorded and stops recording. `dropped` counts entries beyond the cap. */ - end(): { assertions: AssertionRecord[]; dropped: number } { - const result = { assertions: this.entries ?? [], dropped: this.dropped }; - this.entries = undefined; - this.dropped = 0; - return result; - } - - /** Adds an entry to the current polling attempt if there is one, otherwise to the running step. */ - record(entry: Omit): void { - this.push(this.scope.getStore() ?? this.entries, { ...entry, count: 1 }); - } - - /** Appends already finished entries (the passing attempt of a poll) to the current list. */ - commit(buffer: AssertionRecord[]): void { - const target = this.scope.getStore() ?? this.entries; - for (const entry of buffer) this.push(target, entry); - } - - /** Runs `fn` with a private buffer, so its failed attempts can be thrown away. */ - attempt(buffer: AssertionRecord[], fn: () => T): T { - return this.scope.run(buffer, fn); - } - - private push(target: AssertionRecord[] | undefined, entry: AssertionRecord): void { - if (!target) return; - if (target === this.entries) this.total += entry.count; - const last = target[target.length - 1]; - if (last && last.kind === entry.kind && last.label === entry.label && last.ok === entry.ok && entry.ok) { - last.count += entry.count; - return; - } - if (target.length >= MAX_ENTRIES && target === this.entries) { - this.dropped += entry.count; - return; - } - target.push(entry); - } -} - -/** One recorder per process. The staged mocha-entry and the staged page modules share this instance. */ -export const recorder = new Recorder(); diff --git a/docs-snippets/harness/redact.cjs b/docs-snippets/harness/redact.cjs deleted file mode 100644 index 0a6b13619..000000000 --- a/docs-snippets/harness/redact.cjs +++ /dev/null @@ -1,162 +0,0 @@ -'use strict'; -/** - * Secret masking. Nothing the suite prints or writes may contain a key, locally or in CI. GitHub masks secrets in job logs, - * but not in uploaded artifacts, and a local terminal has no masking at all, so this module is the protection everywhere. - * - * Masked: - * - the exact values of every configured key (the env names below, plus values registered with `addSecrets`, for example - * keys the loader read from a local env file), and their URL-encoded forms (secret keys can contain `+`, `/` and `=`, - * which a URL carries as `%2B`, `%2F` and `%3D`). A masked value keeps no characters, only `[redacted]`. - * - any key shaped like `pub-c-`, `sub-c-` or `sec-c-`, and Access Manager tokens, also when - * they are URL-encoded. - * - * Covered output paths: - * - `process.stdout` and `process.stderr` writes of any kind (strings, Buffers, any typed array). Writes are buffered up to - * the end of the line, so a key split across two writes is masked whole. A partial line is flushed after 100 ms, when the - * buffer is large, and at exit. - * - console output (it goes through those streams), Mocha output, SDK logger output. - * - an uncaught exception outside Mocha. Node would print it straight to the file descriptor and bypass the streams, so a - * handler prints the redacted stack instead. When another handler is installed (Mocha has one while tests run), that - * handler owns the error and this one stays out of the way. - * - files written by the reporter (they call `redact()` before writing). - * - * This is a CommonJS file because the Mocha reporter is CommonJS and the harness imports it too. - */ -// Keep in sync with SECRET_NAMES in env.ts (this file is CommonJS and cannot import it). -const KEY_VARS = [ - 'PUBLISH_KEY', - 'SUBSCRIBE_KEY', - 'PAM_PUBLISH_KEY', - 'PAM_SUBSCRIBE_KEY', - 'PAM_SECRET_KEY', - 'DS_PUBLISH_KEY', - 'DS_SUBSCRIBE_KEY', - 'DS_SECRET_KEY', - 'PUBNUB_ADMIN_API_KEY', -]; -// One character of a base64 or base64url value, plain or URL-encoded. -const B64 = '(?:[A-Za-z0-9_=+/-]|%2[BbFf]|%3[Dd])'; -const SHAPES = [ - [/\b(pub|sub)-c-[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}\b/gi, (m) => `${m.slice(0, 6)}***`], - [new RegExp(`\\bsec-c-${B64}{16,}`, 'g'), () => 'sec-c-***'], - // Access Manager tokens are base64 of a CBOR map whose first field is "v". The first four characters encode the map size, - // so they differ by shape: `p0F2` for a token without an authorized user id, `qEF2` with one, and so on. This matches every - // map size from 1 to 23 (`oUF2` to `t0F2`). - [new RegExp(`\\b[o-t][0EUk]F2${B64}{24,}`, 'g'), () => '[token]'], -]; - -// Process-wide state. The source tree and the staged copy under .run// each load their own copy of this module, so -// the secret list and the "installed" flag live on a global symbol that every copy shares. -const STATE = Symbol.for('docs-snippets.redaction'); -const state = globalThis[STATE] || (globalThis[STATE] = { extra: new Set(), installed: false }); -// Secrets that are not env vars (values the loader read from a local env file). -const extra = state.extra; - -/** Registers more literal secrets to mask, for example values read from a local env file. */ -function addSecrets(values) { - for (const value of values) if (typeof value === 'string' && value.trim().length >= 8) extra.add(value.trim()); -} - -/** Every literal form of a secret that can appear in output: the value and its URL-encoded form. */ -function forms(value) { - const out = [value]; - const encoded = encodeURIComponent(value); - if (encoded !== value) out.push(encoded, encoded.toLowerCase()); - return out; -} - -function literalSecrets(env) { - const out = new Set(); - for (const value of extra) for (const form of forms(value)) out.add(form); - for (const name of KEY_VARS) { - const value = env[name]; - if (value && value.trim().length >= 8) for (const form of forms(value.trim())) out.add(form); - } - // Longest first, so a secret that contains another one is masked whole and not left with a tail. - return [...out].sort((a, b) => b.length - a.length); -} - -/** Literal secrets become `[redacted]`. Shaped keys keep their kind (`pub-c-`, `sub-c-`) and nothing else. */ -function redact(text, env = process.env) { - let out = String(text); - for (const secret of literalSecrets(env)) out = out.split(secret).join('[redacted]'); - for (const [pattern, replace] of SHAPES) out = out.replace(pattern, replace); - return out; -} - -/** Text of any chunk a stream accepts. Undefined for chunks that carry no text (they pass through unchanged). */ -function chunkText(chunk) { - if (typeof chunk === 'string') return chunk; - if (Buffer.isBuffer(chunk)) return chunk.toString('utf8'); - if (ArrayBuffer.isView(chunk)) return Buffer.from(chunk.buffer, chunk.byteOffset, chunk.byteLength).toString('utf8'); - return undefined; -} - -const FLUSH_MS = 100; -const MAX_PENDING = 16 * 1024; - -/** Wraps one stream: buffers to the end of the line, redacts, then writes with the original `write`. */ -function wrapStream(stream) { - const write = stream.write.bind(stream); - let pending = ''; - let timer; - const flush = () => { - if (timer) clearTimeout(timer); - timer = undefined; - if (!pending) return; - const text = pending; - pending = ''; - write(redact(text)); - }; - stream.write = (chunk, encoding, callback) => { - const done = typeof encoding === 'function' ? encoding : callback; - const text = chunkText(chunk); - if (text === undefined) return write(chunk, encoding, callback); - const all = pending + text; - const end = all.lastIndexOf('\n'); - let result = true; - if (end === -1) { - pending = all; - } else { - pending = all.slice(end + 1); - result = write(redact(all.slice(0, end + 1))); - } - if (pending.length > MAX_PENDING) flush(); - else if (pending && !timer) { - timer = setTimeout(flush, FLUSH_MS); - timer.unref(); - } - if (done) process.nextTick(done); - return result; - }; - return flush; -} - -/** - * Installs the stream wrappers and the uncaught exception fallback. Process-wide and idempotent, which is why every entry - * point (cli, mocha-entry, provision) calls it first. - */ -function installRedaction() { - if (state.installed) return; - state.installed = true; - const flushers = [process.stdout, process.stderr].map(wrapStream); - const flushAll = () => flushers.forEach((flush) => flush()); - process.on('beforeExit', flushAll); - process.on('exit', flushAll); - process.on('uncaughtException', (error) => { - // Another handler (Mocha while it runs) owns the error. Only when this is the last one, act like Node, redacted. - if (process.listenerCount('uncaughtException') > 1) return; - flushAll(); - const text = error && error.stack ? error.stack : String(error); - process.stderr.write(`${redact(text)}\n`); - flushAll(); - process.exit(1); - }); -} - -/** The literal secrets currently known (for the reporter self-test only, never print them). */ -function knownSecrets(env = process.env) { - return literalSecrets(env); -} - -module.exports = { redact, addSecrets, installRedaction, knownSecrets }; diff --git a/docs-snippets/harness/redact.d.cts b/docs-snippets/harness/redact.d.cts deleted file mode 100644 index 55dc0ed7b..000000000 --- a/docs-snippets/harness/redact.d.cts +++ /dev/null @@ -1,5 +0,0 @@ -export function redact(text: string, env?: Record): string; -export function addSecrets(values: Array): void; -export function installRedaction(): void; -/** Literal secrets known to the redaction. Only for the reporter self-test, which checks output against them. Never print. */ -export function knownSecrets(env?: Record): string[]; diff --git a/docs-snippets/harness/registry.ts b/docs-snippets/harness/registry.ts deleted file mode 100644 index d31b22e98..000000000 --- a/docs-snippets/harness/registry.ts +++ /dev/null @@ -1,284 +0,0 @@ -/** - * Step registry: the single source of truth for what the suite will run and in which order. - * - * Page modules register steps through `step()` and `check()` (step.ts). The registry validates the wiring - * (ids unique, `needs` point at earlier steps), sorts by phase, and answers one question for the Mocha binding: - * "may this step run now?" (`skipReason`). It also remembers each step's result, because `needs` is evaluated against - * results at run time: a step runs only when every step it needs has passed. - * - * The dry run drives the very same code with fake bodies (`simulate`), so skip-on-upstream-failure is proven without - * a network. There is exactly one registry per process (the exported `registry`). The staged copy under - * `.run//` loads its own instance of this module, which is why `cli.ts dry-run` imports the registry back out - * of the staged `run.ts` before inspecting it. - */ -import type { Role } from './env.js'; - -/** Phases P0 to P9 run in this order. */ -export type Phase = 'P0' | 'P1' | 'P2' | 'P3' | 'P4' | 'P5' | 'P6' | 'P7' | 'P8' | 'P9'; -export const PHASES: readonly Phase[] = ['P0', 'P1', 'P2', 'P3', 'P4', 'P5', 'P6', 'P7', 'P8', 'P9']; -/** - * The role a step runs as, which decides its keyset and env names (env.ts): `open` has no Access Manager, `pam` has Access - * Manager (v2 and v3), `datasync` has DataSync classes plus Access Manager. An older layout had one `secured` keyset, which is - * split here into `pam` and `datasync`. - */ -export type Keyset = Role; -export type StepStatus = 'passed' | 'failed' | 'skipped' | 'pending'; - -export interface StepOptions { - /** Ids of steps that must have passed. Otherwise this step is skipped with `blocked-by:`. */ - needs?: string[]; - /** Overrides the page phase. */ - phase?: Phase; - /** Role (keyset) the step talks to. Defaults to `open`. */ - keyset?: Keyset; - /** Snippet ids the step runs. Defaults to `[id]`. Must equal the markers inside the step body (lint). */ - snippets?: string[]; - /** Mocha timeout in ms. Defaults to 30000. */ - timeout?: number; - /** - * One short sentence for the run report: what this step proves. Written from the comment block above the step. It is - * shown next to the step in report/.md and .html, so a reader sees intent without opening the code. - */ - proves?: string; -} - -export interface StepDef { - id: string; - /** Page path such as `api-reference/publish`. */ - page: string; - phase: Phase; - keyset: Keyset; - needs: string[]; - /** Snippet ids. Empty for `check()` steps. */ - snippets: string[]; - kind: 'step' | 'check'; - timeout: number; - /** One-line description for the report, see `StepOptions.proves`. */ - proves?: string; - fn: () => Promise | void; - /** Position in registration order. */ - order: number; -} - -export interface StepResult { - status: StepStatus; - reason?: string; - error?: string; - durationMs?: number; -} - -export type Availability = Record; - -/** What a filtered run executes: the selected steps plus the steps they need that the filter left out. */ -export interface Selection { - /** Step ids that run, in run order. */ - run: Set; - /** Ids that run only because a selected step needs them. Reported as prerequisites, not as passed tests. */ - prerequisites: Set; - /** Ids the filter left out and nothing needs. */ - notSelected: Set; -} - -/** Collects steps, validates their wiring, orders them by phase and tracks results for `needs`. */ -export class Registry { - private steps = new Map(); - private results = new Map(); - private currentPage = '(none)'; - private currentPhase: Phase = 'P4'; - /** Which keysets have keys. A step on a missing keyset skips as `missing-env:` instead of failing. */ - availability: Availability = { open: true, pam: true, datasync: true }; - - reset(): void { - this.steps.clear(); - this.results.clear(); - this.currentPage = '(none)'; - } - - /** Sets the page and default phase that `add` stamps on steps while `fn` runs (a page's `setup` callback). Restores them after. */ - withPage(page: string, phase: Phase, fn: () => T): T { - const previous = [this.currentPage, this.currentPhase] as const; - this.currentPage = page; - this.currentPhase = phase; - try { - return fn(); - } finally { - [this.currentPage, this.currentPhase] = previous; - } - } - - add( - kind: StepDef['kind'], - id: string, - options: StepOptions, - fn: StepDef['fn'], - ): StepDef { - // Same character set as snippet ids. The id doubles as the Mocha test title and the key in report JSON. - if (!/^[A-Za-z0-9_/-]+$/.test(id)) throw new Error(`step id "${id}" is not a valid snippet id`); - if (this.steps.has(id)) throw new Error(`duplicate step id "${id}" (already registered by ${this.steps.get(id)!.page})`); - const def: StepDef = { - id, - page: this.currentPage, - phase: options.phase ?? this.currentPhase, - keyset: options.keyset ?? 'open', - needs: options.needs ?? [], - // check() owns no snippets, so the lint can assert it contains no markers. - snippets: kind === 'check' ? [] : (options.snippets ?? [id]), - kind, - timeout: options.timeout ?? 30_000, - proves: options.proves, - fn, - order: this.steps.size, - }; - this.steps.set(id, def); - return def; - } - - get(id: string): StepDef | undefined { - return this.steps.get(id); - } - - /** Stable sort by phase, then registration order. This is the order Mocha runs the steps in. */ - ordered(): StepDef[] { - return [...this.steps.values()].sort( - (a, b) => PHASES.indexOf(a.phase) - PHASES.indexOf(b.phase) || a.order - b.order, - ); - } - - /** - * Selects steps by a predicate and adds, transitively, every step a selected step `needs`. Those run on demand as - * prerequisites with their own role's keys. With no predicate every step is selected. - */ - select(predicate?: (step: StepDef) => boolean): Selection { - const ordered = this.ordered(); - const chosen = new Set(ordered.filter((s) => !predicate || predicate(s)).map((s) => s.id)); - const needed = new Set(); - const visit = (id: string): void => { - for (const need of this.steps.get(id)?.needs ?? []) { - if (chosen.has(need) || needed.has(need)) continue; - needed.add(need); - visit(need); - } - }; - for (const id of chosen) visit(id); - const run = new Set(ordered.filter((s) => chosen.has(s.id) || needed.has(s.id)).map((s) => s.id)); - const notSelected = new Set(ordered.filter((s) => !run.has(s.id)).map((s) => s.id)); - return { run, prerequisites: needed, notSelected }; - } - - /** Every `needs` edge whose two ends run as different roles. */ - crossRoleNeeds(): Array<{ step: string; role: Role; needs: string; needsRole: Role }> { - const out: Array<{ step: string; role: Role; needs: string; needsRole: Role }> = []; - for (const step of this.ordered()) { - for (const need of step.needs) { - const target = this.steps.get(need); - if (target && target.keyset !== step.keyset) out.push({ step: step.id, role: step.keyset, needs: need, needsRole: target.keyset }); - } - } - return out; - } - - /** Wiring problems: unknown needs, needs that run later, cycles. Empty means the plan is sound. */ - validate(): string[] { - const problems: string[] = []; - const order = this.ordered(); - const position = new Map(order.map((s, i) => [s.id, i])); - for (const step of order) { - for (const need of step.needs) { - const target = this.steps.get(need); - if (!target) { - problems.push(`${step.id}: needs unknown step "${need}"`); - } else if (target.id === step.id) { - problems.push(`${step.id}: needs itself`); - } else if (position.get(need)! > position.get(step.id)!) { - problems.push(`${step.id} (${step.phase}) needs "${need}" (${target.phase}) which runs later`); - } - } - if (step.kind === 'step' && step.snippets.length === 0) problems.push(`${step.id}: step has no snippet ids`); - } - return problems; - } - - /** Why a step cannot run right now, or undefined when it can. Pure with respect to recorded results. */ - skipReason(step: StepDef): string | undefined { - // Precedence: missing keys first, then an upstream step that did not pass. - if (!this.availability[step.keyset]) return `missing-env:${step.keyset}`; - for (const need of step.needs) { - const result = this.results.get(need); - if (!result || result.status !== 'passed') return `blocked-by:${need}`; - } - return undefined; - } - - /** Called by the Mocha binding after each step. `needs` of later steps read these results. */ - record(id: string, result: StepResult): void { - this.results.set(id, result); - } - - result(id: string): StepResult | undefined { - return this.results.get(id); - } - - allResults(): Array<{ step: StepDef; result: StepResult }> { - return this.ordered().map((step) => ({ step, result: this.results.get(step.id) ?? { status: 'pending' } })); - } - - /** - * Runs the same skip logic the Mocha binding uses, with fake bodies. `failing` lists step ids that fail. - * Used by the dry run to prove skip-on-upstream-failure without a network. - */ - simulate(failing: string[]): Array<{ id: string; status: StepStatus; reason?: string }> { - const saved = new Map(this.results); - this.results.clear(); - const out: Array<{ id: string; status: StepStatus; reason?: string }> = []; - for (const step of this.ordered()) { - const reason = this.skipReason(step); - if (reason) { - this.results.set(step.id, { status: 'skipped', reason }); - out.push({ id: step.id, status: 'skipped', reason }); - } else if (failing.includes(step.id)) { - this.results.set(step.id, { status: 'failed' }); - out.push({ id: step.id, status: 'failed' }); - } else { - this.results.set(step.id, { status: 'passed' }); - out.push({ id: step.id, status: 'passed' }); - } - } - this.results = saved; - return out; - } -} - -export const registry = new Registry(); - -/** The Mocha title of a step. It carries the snippet ids, so `--grep ` finds the step. */ -export function stepTitle(def: StepDef): string { - const ids = def.snippets.length > 0 ? ` [${def.snippets.join(', ')}]` : ''; - return `${def.phase} ${def.page} :: ${def.id}${ids}`; -} - -export interface SelectionFilters { - shard?: Role; - grep?: string; - only?: string[]; -} - -/** The selection of a run, shared by the CLI (to decide setup work) and the Mocha binding (to create tests). */ -export function selectSteps(reg: Registry, filters: SelectionFilters): Selection { - const grep = filters.grep ? new RegExp(filters.grep) : undefined; - const filtered = Boolean(grep || filters.only || filters.shard); - return reg.select( - filtered - ? (def) => - (!filters.shard || def.keyset === filters.shard) && - (!grep || grep.test(stepTitle(def))) && - (!filters.only || filters.only.includes(def.id)) - : undefined, - ); -} - -/** Roles that run (selected steps and their prerequisites). `open` always counts: every role uses the open cast. */ -export function rolesOf(reg: Registry, selection: Selection): Set { - const roles = new Set(reg.ordered().filter((d) => selection.run.has(d.id)).map((d) => d.keyset)); - roles.add('open'); - return roles; -} diff --git a/docs-snippets/harness/reporter.cjs b/docs-snippets/harness/reporter.cjs deleted file mode 100644 index bd058dd70..000000000 --- a/docs-snippets/harness/reporter.cjs +++ /dev/null @@ -1,220 +0,0 @@ -'use strict'; -/** - * Mocha reporter of the docs snippet suite. Extends Mocha's Spec reporter (the console output stays the familiar one) and - * writes, when the run ends: - * - * docs-snippets/report/run.json per step: step id, snippet ids, page, role, status, skip reason, duration, assertion - * count, prerequisite flag, the error. Plus the SDK version and commit, the run id, the - * selection, per-phase durations and the skip list. Any skip makes the status `failed`. - * $GITHUB_STEP_SUMMARY a Markdown summary, when the variable is set (GitHub Actions). - * - * Everything written goes through `redact()` first. GitHub masks secrets in logs, not in uploaded artifacts, so this is the - * only protection of report/run.json. CommonJS because Mocha loads reporters with `require`. - */ -const fs = require('node:fs'); -const path = require('node:path'); -const { execFileSync } = require('node:child_process'); -const Mocha = require('mocha'); -const { redact } = require('./redact.cjs'); - -const { Spec } = Mocha.reporters; -const ANSI = /\u001b\[[0-9;]*m/g; -const PHASES = ['P0', 'P1', 'P2', 'P3', 'P4', 'P5', 'P6', 'P7', 'P8', 'P9']; - -/** SDK version and commit of the checkout under test. The suite lives in the SDK repository, two levels above harness/. */ -function sdkInfo() { - const root = path.resolve(__dirname, '..', '..'); - let version = null; - let commit = null; - try { - version = JSON.parse(fs.readFileSync(path.join(root, 'package.json'), 'utf8')).version; - } catch (e) { - version = null; - } - try { - commit = execFileSync('git', ['rev-parse', 'HEAD'], { - cwd: root, - encoding: 'utf8', - stdio: ['ignore', 'pipe', 'ignore'], - }).trim(); - } catch (e) { - commit = process.env.GITHUB_SHA || 'unknown'; - } - return { version, commit }; -} - -function message(error) { - return String((error && error.message) || error) - .replace(ANSI, '') - .split('\n') - .slice(0, 25) - .join('\n'); -} - -/** One step record from a Mocha test. `pnMeta`, `pnReason` and `pnNote` are tags that mocha-entry.ts put on the test. */ -function entry(test, status, error) { - const meta = test.pnMeta || {}; - const assertions = test.pnAssertions || []; - const count = assertions.reduce((sum, a) => sum + (a.count || 1), 0) + (test.pnDropped || 0); - const bad = assertions.find((a) => !a.ok); - const record = { - id: meta.id || test.title, - snippets: meta.snippets || [], - page: meta.page || null, - phase: meta.phase || null, - kind: meta.kind || null, - role: meta.keyset || null, - prerequisite: Boolean(meta.prerequisite), - status, - reason: test.pnReason || meta.skipIf || undefined, - note: test.pnNote || undefined, - durationMs: typeof test.duration === 'number' ? test.duration : 0, - assertions: count, - proves: meta.proves || undefined, - }; - if (error) { - record.error = message(error); - if (bad) - record.failedAssertion = { label: bad.label, expected: bad.expected, actual: bad.actual, frame: bad.frame }; - if (test.pnOutput && test.pnOutput.length) record.output = test.pnOutput.slice(-30); - } - return record; -} - -/** Builds the report object. Pure, the dry run self-test calls it with simulated steps. */ -function buildReport(steps, context) { - const plan = context.plan || []; - const counted = steps.filter((s) => !s.prerequisite); - const summary = { - passed: counted.filter((s) => s.status === 'passed').length, - failed: counted.filter((s) => s.status === 'failed').length, - skipped: counted.filter((s) => s.status === 'skipped').length, - prerequisites: steps.filter((s) => s.prerequisite).length, - notSelected: plan.filter((p) => !p.selected && !p.prerequisite).length, - }; - const skips = steps - .filter((s) => s.status === 'skipped') - .map((s) => ({ - id: s.id, - reason: s.reason || '(none)', - note: s.note, - })); - const phases = {}; - for (const phase of PHASES) { - const inPhase = steps.filter((s) => s.phase === phase); - if (inPhase.length) - phases[phase] = { steps: inPhase.length, ms: inPhase.reduce((sum, s) => sum + s.durationMs, 0) }; - } - const zero = steps.filter((s) => s.status === 'passed' && s.kind === 'step' && s.assertions === 0).map((s) => s.id); - return { - schema: 1, - status: summary.failed > 0 || summary.skipped > 0 ? 'failed' : 'passed', - runId: context.runId || null, - startedAt: context.startedAt, - finishedAt: context.finishedAt, - durationMs: context.durationMs, - sdk: context.sdk, - selection: context.selection || null, - summary, - phases, - skips, - zeroAssertionSteps: zero, - notSelected: plan.filter((p) => !p.selected && !p.prerequisite).map((p) => p.id), - steps, - }; -} - -function seconds(ms) { - return `${(ms / 1000).toFixed(1)} s`; -} - -/** The Markdown summary for GitHub Actions. Pure. */ -function renderMarkdown(report) { - const lines = []; - const shard = report.selection && report.selection.shard ? ` (shard ${report.selection.shard})` : ''; - lines.push(`## Docs snippets live run${shard}: ${report.status}`); - lines.push(''); - lines.push( - `Run \`${report.runId}\`, SDK ${report.sdk.version} at \`${String(report.sdk.commit || '').slice(0, 12)}\`, ${seconds(report.durationMs || 0)}.`, - ); - lines.push(''); - lines.push('| Passed | Failed | Skipped | Prerequisites | Not selected |'); - lines.push('| ---: | ---: | ---: | ---: | ---: |'); - const s = report.summary; - lines.push(`| ${s.passed} | ${s.failed} | ${s.skipped} | ${s.prerequisites} | ${s.notSelected} |`); - const failed = report.steps.filter((x) => x.status === 'failed'); - if (failed.length) { - lines.push('', `### Failures (${failed.length})`, ''); - for (const f of failed) - lines.push(`- \`${f.id}\` (${f.page}, role ${f.role}): ${String(f.error || '').split('\n')[0]}`); - } - if (report.skips.length) { - lines.push('', `### Skips (${report.skips.length})`, ''); - const byReason = new Map(); - for (const k of report.skips) byReason.set(k.reason, [...(byReason.get(k.reason) || []), k.id]); - for (const [reason, ids] of byReason) lines.push(`- \`${reason}\`: ${ids.map((id) => `\`${id}\``).join(', ')}`); - } - lines.push('', '### Time per phase', '', '| Phase | Steps | Time |', '| :-- | ---: | ---: |'); - for (const [phase, v] of Object.entries(report.phases)) lines.push(`| ${phase} | ${v.steps} | ${seconds(v.ms)} |`); - const slow = [...report.steps].sort((a, b) => b.durationMs - a.durationMs).slice(0, 10); - lines.push('', '### Slowest steps', '', '| Step | Time |', '| :-- | ---: |'); - for (const x of slow) lines.push(`| \`${x.id}\` | ${seconds(x.durationMs)} |`); - return `${lines.join('\n')}\n`; -} - -/** Writes run.json and the step summary, all redacted. Returns the paths written. */ -function writeReport(report, dir) { - fs.mkdirSync(dir, { recursive: true }); - const json = redact(JSON.stringify(report, null, 2)); - const file = path.join(dir, 'run.json'); - const tmp = `${file}.${process.pid}.tmp`; - fs.writeFileSync(tmp, json); - fs.renameSync(tmp, file); - const written = [file]; - const md = redact(renderMarkdown(report)); - fs.writeFileSync(path.join(dir, 'summary.md'), md); - written.push(path.join(dir, 'summary.md')); - if (process.env.GITHUB_STEP_SUMMARY) { - fs.appendFileSync(process.env.GITHUB_STEP_SUMMARY, md); - written.push(process.env.GITHUB_STEP_SUMMARY); - } - return written; -} - -class DocsSnippetsReporter extends Spec { - constructor(runner, options) { - super(runner, options); - const startedAt = new Date(); - const steps = []; - runner.on('pass', (test) => steps.push(entry(test, 'passed'))); - runner.on('fail', (test, error) => { - // Hook failures are reported too, under the hook title, so a broken root hook is never silent. - steps.push(entry(test, 'failed', error)); - }); - runner.on('pending', (test) => steps.push(entry(test, 'skipped'))); - runner.once('end', () => { - const finishedAt = new Date(); - const report = buildReport(steps, { - plan: globalThis.__pnDocsPlan, - selection: globalThis.__pnDocsSelection, - runId: process.env.DOCS_SNIPPETS_RUN_ID, - startedAt: startedAt.toISOString(), - finishedAt: finishedAt.toISOString(), - durationMs: finishedAt.getTime() - startedAt.getTime(), - sdk: sdkInfo(), - }); - const dir = process.env.DOCS_SNIPPETS_REPORT_DIR || path.resolve(__dirname, '..', 'report'); - try { - const written = writeReport(report, dir); - process.stdout.write(`report: ${written.join(', ')}\n`); - } catch (error) { - process.stderr.write(`report: could not write ${dir}: ${error && error.message}\n`); - } - }); - } -} - -module.exports = DocsSnippetsReporter; -module.exports.buildReport = buildReport; -module.exports.renderMarkdown = renderMarkdown; -module.exports.writeReport = writeReport; diff --git a/docs-snippets/harness/revoke.ts b/docs-snippets/harness/revoke.ts deleted file mode 100644 index e1f6ad1cc..000000000 --- a/docs-snippets/harness/revoke.ts +++ /dev/null @@ -1,152 +0,0 @@ -import PubNub from 'pubnub'; -import type { SecuredKeyset } from './env.js'; - -/** - * Fail-safe revoke of the application-level Access Manager (v2) grant. - * - * The `grantAllChannelsAllAuthKeys` sample grants read to everybody on the keyset. If a run dies between that grant - * and its revoke, the secured keyset stays open to strangers for the grant ttl (1440 minutes). So the revoke exists - * in four places: the step's own `finally`, the Mocha root `before` and `after` hooks, the janitor, and the signal handlers - * (they run the janitor). Every call retries and a final failure is loud. The `pam` and `datasync` keysets are shared with - * the SDK's other CI jobs, which is why the hooks also prove that a stranger is refused before and after the suite. - */ - -// Backoff is linear (1 s, 2 s, ...). The revoke is a single idempotent call, so retrying is safe, unlike a publish. -const SLEEP = (ms: number): Promise => new Promise((resolve) => setTimeout(resolve, ms)); - -/** - * `grant({ read: false, write: false, ttl: 1 })` on a secret-key client replaces the application-level Access Manager (v2) - * rights with none, which closes what `grantAllChannelsAllAuthKeys` opened. Throws after the last attempt, after a loud console warning - * that tells a human how to close the keyset by hand. - */ -export async function revokeApplicationGrant(client: PubNub, attempts = 3, delayMs = 1_000): Promise { - let last: unknown; - for (let attempt = 1; attempt <= attempts; attempt += 1) { - try { - await client.grant({ read: false, write: false, ttl: 1 }); - return; - } catch (error) { - last = error; - if (attempt < attempts) await SLEEP(delayMs * attempt); - } - } - const detail = last instanceof Error ? last.message : String(last); - console.error( - '\n!!! WARNING !!! The application-level grant on the secured keyset could NOT be revoked after ' + - `${attempts} attempts (${detail}). The keyset may be OPEN for reads until the grant ttl runs out (up to 1440 minutes). ` + - 'Run `npm run test:docs-snippets:janitor` or revoke it in the Admin Portal now.\n', - ); - throw new Error(`could not revoke the application-level grant on the secured keyset: ${detail}`); -} - -/** Revokes with a short-lived client of its own. Used by the root hook, the janitor and the signal handlers. */ -// Own client: the world's `server` client may already be destroyed (signal path) or never built (root `before` hook). -export async function revokeApplicationGrantOnKeyset(secured: SecuredKeyset, tag: string, attempts = 3): Promise { - const client = new PubNub({ ...secured, userId: `janitor-am-${tag}`, logLevel: PubNub.LogLevel.None, heartbeatInterval: 0 }); - try { - await revokeApplicationGrant(client, attempts); - } finally { - client.destroy(); - } -} - -/** True when the SDK error says the keyset refused the call for lack of permission (HTTP 403, access denied category). */ -function isAccessDenied(error: unknown): boolean { - const status = (error as { status?: { category?: string; statusCode?: number } } | undefined)?.status; - return status?.category === 'PNAccessDeniedCategory' || status?.statusCode === 403; -} - -/** What one probe saw. `unknown` means it got an answer that proves nothing (network error, timeout, 5xx, 400). */ -export interface ProbeResult { - /** What a client without a token could do. Empty when the keyset refused both. */ - open: string[]; - /** Answers that were neither a success nor a 403. */ - unknown: string[]; -} - -/** - * Asks the keyset whether a client with NO token and NO auth key can publish or read. Only two answers count: success (the - * keyset is open for that) and 403 / PNAccessDeniedCategory (closed). Anything else, a network error, a timeout, a 5xx or a - * 400, is `unknown`: it says nothing about enforcement, so it must never pass as "closed". - * - * Publish needs the write permission, read (`fetchMessages`) needs read. The application-level sample grants read only, so - * the read probe is the one that notices it. The keysets this runs against have Message Persistence on (README, "Features - * each keyset needs"), so a refused read is a 403 and not a 400. - */ -export async function tokenlessAccess(probe: PubNub, channel: string): Promise { - const result: ProbeResult = { open: [], unknown: [] }; - const attempt = async (label: string, call: () => Promise): Promise => { - try { - await call(); - result.open.push(label); - } catch (error) { - if (!isAccessDenied(error)) result.unknown.push(`${label}: ${describe(error)}`); - } - }; - await attempt('publish', () => probe.publish({ channel, message: { probe: true } })); - await attempt('read', () => probe.fetchMessages({ channels: [channel], count: 1 })); - return result; -} - -function describe(error: unknown): string { - const e = error as { message?: string; status?: { category?: string; statusCode?: number } }; - return `${e?.message ?? String(error)} [${e?.status?.category ?? ''} ${e?.status?.statusCode ?? ''}]`.trim(); -} - -/** - * Closes the application-level grant and then WAITS until the keyset really enforces Access Manager. - * - * Why waiting is needed: Access Manager (v2) grants and revokes take up to about 60 seconds to propagate. A previous run - * (or a run that was cancelled) may have left a grant, and the revoke only starts the clock. A step that expects a 403 for a - * tokenless client, such as `publishHandleErrors`, would otherwise fail on a keyset that still behaves as open. The probe - * is a tokenless client (no secret key, no auth key) on a channel no run ever grants, so a grant of the same run cannot hide it. - * - * Throws with a message that names the cause when the keyset is still open after `timeoutMs`. - * Returns how long it took (`attempts` 1 means it was closed right away). - */ -export async function closeAndVerifyApplicationGrant( - secured: SecuredKeyset, - tag: string, - options: { timeoutMs?: number; intervalMs?: number } = {}, -): Promise<{ attempts: number; ms: number }> { - const timeoutMs = options.timeoutMs ?? 120_000; - const intervalMs = options.intervalMs ?? 3_000; - await revokeApplicationGrantOnKeyset(secured, tag); - const started = Date.now(); - let attempts = 0; - let last = ''; - for (;;) { - attempts += 1; - // A fresh client (no secret key, no auth key, no token) on a fresh channel each attempt: only an application-level grant - // can let it through, because no run ever grants this channel name. - const probe = new PubNub({ - publishKey: secured.publishKey, - subscribeKey: secured.subscribeKey, - userId: `janitor-probe-${tag}-${attempts}`, - logLevel: PubNub.LogLevel.None, - heartbeatInterval: 0, - }); - let seen: ProbeResult; - try { - seen = await tokenlessAccess(probe, `docs-snippets-probe-${tag}-${Date.now().toString(36)}-${attempts}`); - } finally { - probe.destroy(); - } - // Closed only when both probes got a 403. An unknown answer is retried like an open one, never taken as closed. - if (seen.open.length === 0 && seen.unknown.length === 0) return { attempts, ms: Date.now() - started }; - last = [ - seen.open.length > 0 ? `a client without a token could ${seen.open.join(' and ')}` : '', - seen.unknown.length > 0 ? `no clear answer: ${seen.unknown.join('; ')}` : '', - ] - .filter(Boolean) - .join(', '); - if (Date.now() + intervalMs - started > timeoutMs) break; - await SLEEP(intervalMs); - } - throw new Error( - `could not prove that the secured keyset enforces Access Manager ${Math.round((Date.now() - started) / 1000)} s after the application-level grant was revoked (${last}). ` + - 'Steps that expect a 403 for a client without a token would fail. A grant from an earlier run is probably still effective: ' + - 'revoke and grant changes take up to about 60 seconds to propagate, but a grant whose revoke failed lasts up to 1440 minutes. ' + - 'Run `npm run test:docs-snippets:janitor`, check the keyset in the Admin Portal (Access Manager must be on), then run again.', - ); -} diff --git a/docs-snippets/harness/signals.ts b/docs-snippets/harness/signals.ts deleted file mode 100644 index a035bf1a0..000000000 --- a/docs-snippets/harness/signals.ts +++ /dev/null @@ -1,65 +0,0 @@ -/** - * SIGINT and SIGTERM handlers. A killed run must not leave a secured keyset open or leak resources, so the handler runs the - * cleanup, then exits with the conventional code. - * - * A cancelled GitHub Actions job gets SIGINT, then SIGTERM about 7.5 s later, then SIGKILL about 2.5 s after that. So: - * - `urgent` runs first and alone: it revokes the application-level Access Manager grants, a single quick call per keyset. - * That is what must finish before SIGKILL. - * - `cleanup` (janitor, client teardown) runs after it. - * - SIGTERM never skips the cleanup. Only a second SIGINT (a person pressing Ctrl-C twice) exits at once. - * - * Exit codes follow the shell convention 128 + signal number (SIGINT 2 -> 130, SIGTERM 15 -> 143), so CI sees a cancel as - * such. `budgetMs` (100 s) caps the whole cleanup so a hung janitor cannot keep a cancelled job alive. - */ -// Written straight to stderr, not through `console`: a cancel lands in the middle of a step, while quiet.ts collects console -// output, and these lines must be seen. stderr still goes through the redaction (redact.cjs). -const say = (line: string): void => { - process.stderr.write(`${line}\n`); -}; - -// One Ctrl-C or one CI cancel reaches this process more than once: the signal goes to the whole process group, and the -// launchers in between (npm, tsx) relay it to their child too. A repeat inside this window is the same signal, not a second -// Ctrl-C. -const SAME_SIGNAL_MS = 2_000; - -export function installShutdown(urgent: () => Promise, cleanup: () => Promise, budgetMs = 100_000): void { - let running = false; - let code = 0; - let firstAt = 0; - // In CI no signal may skip the cleanup: a cancel must never leave a keyset open. - const ci = Boolean(process.env.CI || process.env.GITHUB_ACTIONS); - const handle = (signal: NodeJS.Signals, signalCode: number): void => { - if (running) { - if (signal === 'SIGINT' && !ci && Date.now() - firstAt > SAME_SIGNAL_MS) { - say('SIGINT again, exiting without waiting for cleanup'); - process.exit(signalCode); - } - if (Date.now() - firstAt > SAME_SIGNAL_MS) say(`${signal} received while cleaning up, the cleanup continues`); - return; - } - running = true; - firstAt = Date.now(); - code = signalCode; - say(` -${signal} received: revoking Access Manager grants first, then cleaning up. Press Ctrl-C again to skip.`); - const timer = setTimeout(() => { - say('cleanup took too long, exiting'); - process.exit(code); - }, budgetMs); - const step = async (label: string, fn: () => Promise): Promise => { - try { - await fn(); - } catch (error) { - say(`${label} failed: ${(error as Error)?.message ?? error}`); - } - }; - void (async () => { - await step('revoke', urgent); - await step('cleanup', cleanup); - clearTimeout(timer); - process.exit(code); - })(); - }; - process.on('SIGINT', () => handle('SIGINT', 130)); - process.on('SIGTERM', () => handle('SIGTERM', 143)); -} diff --git a/docs-snippets/harness/snippet-summary.cjs b/docs-snippets/harness/snippet-summary.cjs deleted file mode 100644 index 4680a6564..000000000 --- a/docs-snippets/harness/snippet-summary.cjs +++ /dev/null @@ -1,144 +0,0 @@ -'use strict'; -/** - * Per snippet id results for the GitHub Actions run page. - * - * Why: the reporter (reporter.cjs) summarises one shard by step, and each shard writes its own block. A reader who wants to - * know "did the sample with id X pass" has to match step ids to snippet ids by hand across three blocks. This script reads the - * reports of every shard plus coverage.json and writes one Markdown block with one row per snippet id of the manifest: - * - * - live snippets: passed, failed, skipped (with the reason), or "not run" when no shard report has a step for the id - * - compile-only snippets: type checked in the dry job, never run - * - stubs: kept for the docs URLs, never run - * - * Usage (from the SDK repository root): - * node docs-snippets/harness/snippet-summary.cjs [reportDir ...] - * - * Each reportDir holds a run.json. The default is docs-snippets/report, the folder of a local run. CI passes the three shard - * folders (docs-snippets/report/open, pam, datasync). The Markdown goes to $GITHUB_STEP_SUMMARY when it is set, and always to - * stdout. Output passes through redact() like every other file the suite writes. Never fails the job, the exit code of the - * suite decides that. - */ -const fs = require('node:fs'); -const path = require('node:path'); -const { redact } = require('./redact.cjs'); - -const ICON = { - passed: 'passed', - failed: 'FAILED', - skipped: 'skipped', - 'not-run': 'NOT RUN', - 'compile-only': 'type checked only', - stub: 'stub, not run', -}; - -/** First line of an error, short enough for a table cell. */ -function firstLine(text) { - return String(text || '') - .split('\n')[0] - .replace(/\|/g, '\\|') - .replace(/`/g, "'") - .slice(0, 160); -} - -/** The result of one manifest entry, from the step records of all reports. Pure. */ -function resultOf(id, entry, records) { - const hit = records.filter((r) => (r.step.snippets || []).includes(id) || (entry.step && r.step.id === entry.step)); - if (hit.length === 0) { - if (entry.mode === 'compile-only') return { status: 'compile-only', reason: entry.reason }; - if (entry.mode === 'stub') return { status: 'stub', reason: entry.reason }; - return { status: 'not-run' }; - } - // A failure anywhere wins, then a skip, then pass. - const failed = hit.find((r) => r.step.status === 'failed'); - const skipped = hit.find((r) => r.step.status === 'skipped'); - const chosen = failed || skipped || hit[0]; - return { - status: chosen.step.status, - reason: chosen.step.status === 'skipped' ? chosen.step.reason : undefined, - error: chosen.step.status === 'failed' ? chosen.step.error : undefined, - stepId: chosen.step.id, - shard: chosen.shard, - page: chosen.step.page, - ms: hit.reduce((sum, r) => sum + (r.step.durationMs || 0), 0), - }; -} - -/** - * The Markdown block. Pure: `reports` is [{ shard, report }] (report = a parsed run.json), `manifest` is the `snippets` - * object of coverage.json. - */ -function renderSnippetSummary(reports, manifest) { - const records = []; - for (const { shard, report } of reports) for (const step of report.steps || []) records.push({ shard, step }); - const rows = Object.keys(manifest) - .sort((a, b) => a.localeCompare(b)) - .map((id) => ({ id, entry: manifest[id], result: resultOf(id, manifest[id], records) })); - - const count = (status) => rows.filter((r) => r.result.status === status).length; - const lines = []; - lines.push('## Docs snippets: result per snippet id', ''); - const shards = reports.map((r) => r.shard).join(', ') || 'none'; - lines.push(`Shards with a report: ${shards}. Snippet ids in the manifest: ${rows.length}.`, ''); - lines.push('| Passed | Failed | Skipped | Not run | Type checked only | Stubs |', '| ---: | ---: | ---: | ---: | ---: | ---: |'); - lines.push( - `| ${count('passed')} | ${count('failed')} | ${count('skipped')} | ${count('not-run')} | ${count('compile-only')} | ${count('stub')} |`, - ); - - const failed = rows.filter((r) => r.result.status === 'failed'); - if (failed.length) { - lines.push('', `### Failed (${failed.length})`, '', '| Snippet id | Page | Shard | Error |', '| :-- | :-- | :-- | :-- |'); - for (const r of failed) - lines.push(`| \`${r.id}\` | ${r.result.page || '-'} | ${r.result.shard} | ${firstLine(r.result.error)} |`); - } - const notRun = rows.filter((r) => r.result.status === 'not-run'); - if (notRun.length) { - lines.push( - '', - `### Not run (${notRun.length})`, - '', - 'Live snippets with no step in any shard report. Usually a shard that crashed or was cancelled before it got to them.', - '', - notRun.map((r) => `\`${r.id}\``).join(', '), - ); - } - const skipped = rows.filter((r) => r.result.status === 'skipped'); - if (skipped.length) { - lines.push('', `### Skipped (${skipped.length})`, '', '| Snippet id | Reason |', '| :-- | :-- |'); - for (const r of skipped) lines.push(`| \`${r.id}\` | \`${r.result.reason || '(none)'}\` |`); - } - - lines.push('', '### All snippet ids', '', '| Snippet id | Result | Where | Time |', '| :-- | :-- | :-- | ---: |'); - for (const r of rows) { - const { status, reason, page, shard, ms } = r.result; - const where = status === 'compile-only' || status === 'stub' ? r.entry.file || '-' : `${page || '-'} (${shard || '-'})`; - const note = status === 'compile-only' || status === 'stub' ? ` (${firstLine(reason)})` : ''; - lines.push(`| \`${r.id}\` | ${ICON[status]}${note} | ${where} | ${ms === undefined ? '-' : `${(ms / 1000).toFixed(1)} s`} |`); - } - return `${lines.join('\n')}\n`; -} - -/** Reads run.json from each directory. A missing file is skipped, so a cancelled shard still gets a summary. */ -function loadReports(dirs) { - const reports = []; - for (const dir of dirs) { - const file = path.join(dir, 'run.json'); - if (!fs.existsSync(file)) continue; - const report = JSON.parse(fs.readFileSync(file, 'utf8')); - const shard = (report.selection && report.selection.shard) || path.basename(dir); - reports.push({ shard, report }); - } - return reports; -} - -function main(argv) { - const root = path.resolve(__dirname, '..'); - const dirs = argv.length ? argv : [path.join(root, 'report')]; - const manifest = JSON.parse(fs.readFileSync(path.join(root, 'coverage.json'), 'utf8')).snippets; - const md = redact(renderSnippetSummary(loadReports(dirs), manifest)); - process.stdout.write(md); - if (process.env.GITHUB_STEP_SUMMARY) fs.appendFileSync(process.env.GITHUB_STEP_SUMMARY, md); -} - -if (require.main === module) main(process.argv.slice(2)); - -module.exports = { renderSnippetSummary, loadReports, resultOf }; diff --git a/docs-snippets/harness/stage.ts b/docs-snippets/harness/stage.ts deleted file mode 100644 index a7e06bdf0..000000000 --- a/docs-snippets/harness/stage.ts +++ /dev/null @@ -1,385 +0,0 @@ -import fs from 'node:fs'; -import path from 'node:path'; -import ts from 'typescript'; -import { allRewritableNames, isPlaceholder, isRegisteredName, placeholders, suffixName } from '../fixtures/names.js'; -import { isVisibleLine, scanMarkers, type SnippetRegion } from './markers.js'; - -/** - * Staging rewrite. - * - * Copies the suite into `docs-snippets/.run//` and rewrites registered literals inside the visible part of snippet - * regions. - * Rewrites stay on the same line, so stack traces still point at the right line of the source file. - * Hidden lines and everything outside the markers are left alone: authors use `world.n()` there. - * - * Why a staged copy at all. The docs embed the source files verbatim (EmbeddedCode fetches them from the repository), so - * the source must show `my_channel`, not `my_channel-rk3x9q`. A live run needs unique names though, because two people or - * two CI jobs share the keysets. The copy under `.run//` is what Mocha executes. The original is never modified. - * - * Why the TypeScript compiler and not a regex. Only string literals in value position may change. A property name - * (`{ channel: ... }`), an import path, a type literal or an option like `flags: 'a'` must stay. The AST tells them apart. - */ - -/** Directories whose files may contain snippets and are rewritten. */ -export const PAGE_DIRS = ['api-reference', 'shared', 'getting-started', 'migration-guides']; -// Everything the staged run imports. `quickstarts/` is absent on purpose: those sources are type checked only -// (tsconfig.browser.json) or not at all, because this repository cannot run a browser, React or React Native. -const COPY_DIRS = ['harness', 'fixtures', ...PAGE_DIRS]; -const COPY_FILES = ['run.ts', 'package.json', 'tsconfig.live.json', 'tsconfig.run.json']; - -/** Snippet ids whose `snippet.end` line is not at a statement position, so the ran marker went to the next statement. */ -export interface RanFallback { - id: string; - file: string; - line: number; -} - -export interface Rewrite { - file: string; - line: number; - from: string; - to: string; - kind: 'name' | 'placeholder'; -} - -export interface StageResult { - runDir: string; - runId: string; - rewrites: Rewrite[]; - filesCopied: number; - /** Registered names that no snippet region uses (they are legal, but worth a look). */ - unusedNames: string[]; - problems: string[]; - /** Ids whose `__snippetRan` call needed the AST fallback. */ - ranFallbacks: RanFallback[]; - /** Number of `snippet.end` lines that got a `__snippetRan` call. */ - ranMarkers: number; - /** The mutated id, when DOCS_SNIPPETS_MUTATE was applied. */ - mutated?: string; -} - -export interface StageOptions { - /** DOCS_SNIPPETS_MUTATE: insert a throw right after this snippet's start marker in the staged copy. */ - mutate?: string; -} - -/** Recursive copy that skips the directories a staged run must not contain: dependencies, other runs, reports. */ -function copyDir(from: string, to: string, onFile: () => void): void { - fs.mkdirSync(to, { recursive: true }); - for (const entry of fs.readdirSync(from, { withFileTypes: true })) { - if (entry.name === 'node_modules' || entry.name === '.run' || entry.name === 'report') continue; - const src = path.join(from, entry.name); - const dst = path.join(to, entry.name); - if (entry.isDirectory()) copyDir(src, dst, onFile); - else { - fs.copyFileSync(src, dst); - onFile(); - } - } -} - -/** Rewrites one source text. Exposed for tests and the dry run. */ -export function rewriteSource(text: string, file: string, runId: string): { text: string; rewrites: Rewrite[] } { - const { regions } = scanMarkers(text); - // No snippets, nothing to rewrite (shared helpers, fixtures). Skips parsing the file. - if (regions.length === 0) return { text, rewrites: [] }; - const source = ts.createSourceFile(file, text, ts.ScriptTarget.ES2022, true, ts.ScriptKind.TS); - const edits: Array<{ start: number; end: number; replacement: string; rewrite: Rewrite }> = []; - - const visit = (node: ts.Node): void => { - if (ts.isStringLiteral(node) && !isStructuralLiteral(node)) { - const start = node.getStart(source); - const { line: zeroLine } = source.getLineAndCharacterOfPosition(start); - const line = zeroLine + 1; - // Only literals on visible lines of a snippet. A multi-line template string is skipped because a rewrite must keep - // the line count identical (stack traces and the compare tool rely on that). - const region = regions.find((r) => isVisibleLine(r, line)); - if (region && !text.slice(start, node.end).includes('\n')) { - const quote = text[start]; - const value = node.text; - // Placeholder first: its literal is not a resource name, and it becomes a call, not a suffixed string. - if (isPlaceholder(value)) { - const replacement = `world.ph(${quote}${placeholders[value]}${quote})`; - edits.push({ start, end: node.end, replacement, rewrite: { file, line, from: value, to: replacement, kind: 'placeholder' } }); - } else if (isRegisteredName(value)) { - const suffixed = suffixName(value, runId); - edits.push({ - start, - end: node.end, - replacement: `${quote}${suffixed}${quote}`, - rewrite: { file, line, from: value, to: suffixed, kind: 'name' }, - }); - } - } - } - ts.forEachChild(node, visit); - }; - visit(source); - - // Apply edits from the end of the file backwards, so earlier offsets stay valid while the text changes length. - let out = text; - for (const edit of edits.sort((a, b) => b.start - a.start)) { - out = out.slice(0, edit.start) + edit.replacement + out.slice(edit.end); - } - return { text: out, rewrites: edits.map((e) => e.rewrite).sort((a, b) => a.line - b.line) }; -} - -// Option keys whose string values are never resource names, although the value may equal a registered name. -const NON_RESOURCE_KEYS = new Set(['flags', 'encoding']); - -/** Property names, import paths and similar positions are never resource values. */ -function isStructuralLiteral(node: ts.StringLiteral): boolean { - const parent = node.parent; - if (!parent) return false; - if (ts.isPropertyAssignment(parent) && parent.name === node) return true; - // Values of options that are never resource names, for example `fs.createWriteStream(path, { flags: 'a' })`. `a` is also a registered channel. - if (ts.isPropertyAssignment(parent) && parent.initializer === node && ts.isIdentifier(parent.name) && NON_RESOURCE_KEYS.has(parent.name.text)) return true; - if (ts.isImportDeclaration(parent) || ts.isExportDeclaration(parent)) return true; - if (ts.isLiteralTypeNode(parent)) return true; - // The key inside an already staged `world.ph('key')` call is not a snippet literal (a key may equal its literal). - if (ts.isCallExpression(parent) && parent.expression.getText().endsWith('world.ph')) return true; - if (ts.isElementAccessExpression(parent) && parent.argumentExpression === node) return false; - return false; -} - -/** Staging directories older than this are removed by the next stage. */ -const STALE_STAGE_MS = 24 * 60 * 60 * 1000; - -/** - * Stages into `.run//`, so two runs (a live run and a dry run, two agents, two CI jobs on one disk) never share - * or delete each other's copy. Only this run's directory is replaced. Directories untouched for a day are pruned. - */ -export function stage(appDir: string, runId: string, options: StageOptions = {}): StageResult { - const root = path.join(appDir, '.run'); - const runDir = path.join(root, runId); - fs.mkdirSync(root, { recursive: true }); - pruneStale(root, runId); - fs.rmSync(runDir, { recursive: true, force: true }); - fs.mkdirSync(runDir, { recursive: true }); - - let filesCopied = 0; - const bump = (): void => { - filesCopied += 1; - }; - for (const dir of COPY_DIRS) { - const from = path.join(appDir, dir); - if (fs.existsSync(from)) copyDir(from, path.join(runDir, dir), bump); - } - for (const file of COPY_FILES) { - const from = path.join(appDir, file); - if (fs.existsSync(from)) { - fs.copyFileSync(from, path.join(runDir, file)); - bump(); - } - } - - const rewrites: Rewrite[] = []; - const problems: string[] = []; - for (const dir of PAGE_DIRS) { - const root = path.join(runDir, dir); - if (!fs.existsSync(root)) continue; - for (const file of walk(root).filter((f) => f.endsWith('.ts') || f.endsWith('.tsx'))) { - const relative = path.relative(runDir, file); - const original = fs.readFileSync(file, 'utf8'); - const result = rewriteSource(original, relative, runId); - if (result.rewrites.length === 0) continue; - // Safety net for the rewrite: the staged file must still parse and keep the same number of lines, otherwise a - // failing step would point at a different line than the source file the author edits. - if (result.text.split('\n').length !== original.split('\n').length) { - problems.push(`${relative}: rewrite changed the line count`); - } - const diagnostics = ts.transpileModule(result.text, { - reportDiagnostics: true, - compilerOptions: { target: ts.ScriptTarget.ES2022, module: ts.ModuleKind.ESNext }, - }).diagnostics; - for (const d of diagnostics ?? []) { - problems.push(`${relative}: staged file does not parse: ${ts.flattenDiagnosticMessageText(d.messageText, ' ')}`); - } - fs.writeFileSync(file, result.text); - rewrites.push(...result.rewrites); - } - } - - // Second pass as a self-check. Running the rewrite on already staged text must find nothing: a hit means a registered - // literal escaped the first pass (for example inside a construct the visitor skipped). - // Verify: no registered literal is left unrewritten inside a visible snippet region of the staged tree. - for (const dir of PAGE_DIRS) { - const root = path.join(runDir, dir); - if (!fs.existsSync(root)) continue; - for (const file of walk(root).filter((f) => f.endsWith('.ts'))) { - const relative = path.relative(runDir, file); - const text = fs.readFileSync(file, 'utf8'); - const leftover = rewriteSource(text, relative, runId).rewrites; - // After one pass, rewriting again finds only literals that were not suffixed (the suffixed form is not registered). - for (const l of leftover) problems.push(`${relative}:${l.line}: "${l.from}" is still unrewritten after staging`); - } - } - - // Names registered in fixtures/names.ts that no snippet uses. Legal (harness steps may use them), shown so the registry stays tidy. - const used = new Set(rewrites.filter((r) => r.kind === 'name').map((r) => r.from)); - const unusedNames = allRewritableNames().filter((name) => !used.has(name)); - - // Honesty markers, after all marker parsing and rewriting: every snippet region inside a step() body records that its last - // line ran (`__snippetRan('')` on the `snippet.end` line), and DOCS_SNIPPETS_MUTATE breaks one snippet on purpose. - // Both edits stay on their line, so line numbers still match the source. - const ranFallbacks: RanFallback[] = []; - let ranMarkers = 0; - let mutated: string | undefined; - for (const dir of PAGE_DIRS) { - const root = path.join(runDir, dir); - if (!fs.existsSync(root)) continue; - for (const file of walk(root).filter((f) => f.endsWith('.ts'))) { - const relative = path.relative(runDir, file); - const text = fs.readFileSync(file, 'utf8'); - const result = instrumentSource(text, relative, options.mutate); - if (result.text === text) continue; - ranMarkers += result.marked; - ranFallbacks.push(...result.fallbacks.map((f) => ({ ...f, file: relative }))); - if (result.mutated) mutated = result.mutated; - problems.push(...result.problems.map((p) => `${relative}: ${p}`)); - fs.writeFileSync(file, result.text); - } - } - if (options.mutate && !mutated) problems.push(`DOCS_SNIPPETS_MUTATE=${options.mutate}: no snippet with that id inside a step() body`); - - // `pubnub` for child processes and scratch projects (the module format check, the Node quickstart, the TypeScript project - // checks). They run plain Node or tsc below this directory, so they resolve `pubnub` the way a user project does, through - // node_modules. The link points at this repository, whose package.json `main` is lib/node/index.js. Nothing is installed. - const modules = path.join(runDir, 'node_modules'); - fs.mkdirSync(modules, { recursive: true }); - fs.symlinkSync(path.resolve(appDir, '..'), path.join(modules, 'pubnub'), 'dir'); - - // A record of every rewrite for humans debugging a live failure (`.run//stage-manifest.json`). - fs.writeFileSync( - path.join(runDir, 'stage-manifest.json'), - JSON.stringify({ runId, generatedAt: new Date().toISOString(), rewrites }, null, 2), - ); - return { runDir, runId, rewrites, filesCopied, unusedNames, problems, ranFallbacks, ranMarkers, mutated }; -} - -/** True when the text parses as a module (the staged files must stay valid TypeScript). */ -function parses(text: string): string[] { - const diagnostics = ts.transpileModule(text, { - reportDiagnostics: true, - compilerOptions: { target: ts.ScriptTarget.ES2022, module: ts.ModuleKind.ESNext }, - }).diagnostics; - return (diagnostics ?? []).map((d) => ts.flattenDiagnosticMessageText(d.messageText, ' ')); -} - -/** Ids of regions that sit inside a `step()` callback (not check, not import-only). */ -function stepRegions(text: string, file: string): SnippetRegion[] { - const { regions } = scanMarkers(text); - if (regions.length === 0) return []; - const source = ts.createSourceFile(file, text, ts.ScriptTarget.ES2022, true, ts.ScriptKind.TS); - const spans: Array<[number, number]> = []; - const visit = (node: ts.Node): void => { - if (ts.isCallExpression(node) && ts.isIdentifier(node.expression) && node.expression.text === 'step') { - const fn = [...node.arguments].reverse().find((a) => ts.isArrowFunction(a) || ts.isFunctionExpression(a)); - if (fn) spans.push([source.getLineAndCharacterOfPosition(fn.getStart(source)).line + 1, source.getLineAndCharacterOfPosition(fn.end).line + 1]); - } - ts.forEachChild(node, visit); - }; - visit(source); - return regions.filter((r) => r.explicitEnd && spans.some(([a, b]) => r.startLine >= a && r.endLine <= b)); -} - -/** - * Finds where to put a statement for a marker line that is not at a statement position: right after the innermost statement - * that contains the line, so it runs once that statement finished. Returns an offset in `text`, or undefined. - */ -function afterEnclosingStatement(text: string, file: string, line: number): number | undefined { - const source = ts.createSourceFile(file, text, ts.ScriptTarget.ES2022, true, ts.ScriptKind.TS); - const position = source.getPositionOfLineAndCharacter(line - 1, 0); - let best: ts.Statement | undefined; - const visit = (node: ts.Node): void => { - if (node.getFullStart() > position || node.end < position) return; - const parent = node.parent; - const inList = parent && (ts.isBlock(parent) || ts.isSourceFile(parent) || ts.isCaseClause(parent) || ts.isDefaultClause(parent) || ts.isModuleBlock(parent)); - if (inList && node.getStart(source) <= position && node.end >= position) best = node as ts.Statement; - ts.forEachChild(node, visit); - }; - visit(source); - return best?.end; -} - -/** - * Adds `__snippetRan('')` to the `snippet.end` line of every region inside a step body, and with `mutate` a throw at - * the start marker of that id. Same line, so line numbers stay. When the plain edit does not parse (the marker sits inside an - * expression), the call goes after the enclosing statement instead, and the id is listed as a fallback. - */ -export function instrumentSource( - text: string, - file: string, - mutate?: string, -): { text: string; marked: number; fallbacks: Array<{ id: string; line: number }>; mutated?: string; problems: string[] } { - const regions = stepRegions(text, file); - const fallbacks: Array<{ id: string; line: number }> = []; - const problems: string[] = []; - let mutated: string | undefined; - let lines = text.split('\n'); - // From the bottom up, so earlier line numbers stay valid while lines change. - for (const region of [...regions].sort((a, b) => b.endLine - a.endLine)) { - const call = `__snippetRan('${region.id}');`; - const plain = [...lines]; - plain[region.endLine - 1] = plain[region.endLine - 1].replace(/^(\s*)/, `$1${call} `); - if (parses(plain.join('\n')).length === 0) { - lines = plain; - } else { - const current = lines.join('\n'); - const offset = afterEnclosingStatement(current, file, region.endLine); - if (offset === undefined) { - problems.push(`no statement position for __snippetRan('${region.id}') near line ${region.endLine}`); - continue; - } - const edited = `${current.slice(0, offset)} ${call}${current.slice(offset)}`; - const errors = parses(edited); - if (errors.length > 0) { - problems.push(`__snippetRan('${region.id}') does not parse after the fallback: ${errors[0]}`); - continue; - } - lines = edited.split('\n'); - fallbacks.push({ id: region.id, line: region.endLine }); - } - } - // `__snippetStart('')` on the start marker line: the honesty check counts only assertions made after the docs code began. - for (const region of [...regions].sort((x, y) => y.startLine - x.startLine)) { - const plain = [...lines]; - plain[region.startLine - 1] = plain[region.startLine - 1].replace(/^(\s*)/, `$1__snippetStart('${region.id}'); `); - if (parses(plain.join('\n')).length === 0) lines = plain; - else problems.push(`__snippetStart('${region.id}') is not at a statement position`); - } - if (mutate) { - const region = regions.find((r) => r.id === mutate); - if (region) { - const throwLine = `throw new Error('DOCS_SNIPPETS_MUTATE ${mutate}');`; - const plain = [...lines]; - plain[region.startLine - 1] = plain[region.startLine - 1].replace(/^(\s*)/, `$1${throwLine} `); - if (parses(plain.join('\n')).length === 0) { - lines = plain; - mutated = mutate; - } else { - problems.push(`DOCS_SNIPPETS_MUTATE=${mutate}: the start marker is not at a statement position`); - } - } - } - return { text: lines.join('\n'), marked: regions.length - problems.filter((p) => p.startsWith('no statement') || p.includes('after the fallback')).length, fallbacks, mutated, problems }; -} - -function pruneStale(root: string, keep: string): void { - for (const entry of fs.readdirSync(root, { withFileTypes: true })) { - if (!entry.isDirectory() || entry.name === keep) continue; - const full = path.join(root, entry.name); - try { - if (Date.now() - fs.statSync(full).mtimeMs > STALE_STAGE_MS) fs.rmSync(full, { recursive: true, force: true }); - } catch { - // Another run removed it first. - } - } -} - -function walk(dir: string): string[] { - return fs.readdirSync(dir, { withFileTypes: true }).flatMap((entry) => { - const full = path.join(dir, entry.name); - return entry.isDirectory() ? walk(full) : [full]; - }); -} diff --git a/docs-snippets/harness/step.ts b/docs-snippets/harness/step.ts deleted file mode 100644 index 397dc85bc..000000000 --- a/docs-snippets/harness/step.ts +++ /dev/null @@ -1,93 +0,0 @@ -/** - * The authoring API of the suite: what a page module imports to register its steps. - * - * page(path, defaultPhase, (world) => { step(...); check(...); }) - * - * - `step` runs one or more docs snippets (the code between `// snippet.` markers) and asserts around them. - * - `check` is a harness-only step with no markers: fixtures, effect checks, error tables, preflights. - * - `skipStep` lets a running step skip itself with a precise reason (see `SkipStep`). A skip fails the run (policy.ts). - * - * Nothing runs when these functions are called. They only record a definition in the registry (registry.ts). - * `harness/mocha-entry.ts` later turns each definition into a Mocha test, so the order of registration, the phase and - * the `needs` edges decide when and whether a step executes. - */ -import { registry, type Phase, type StepOptions } from './registry.js'; -import type { World } from './world.js'; - -export type { Phase, StepOptions } from './registry.js'; - -type Body = () => Promise | void; - -/** - * One docs snippet run (or a short sequence of snippets the docs show together). - * - * The id is the first snippet id inside the body. The marker lint checks that, checks that `snippets` equals - * the markers inside, and rejects nesting. Snippets let errors throw, so the body asserts around them. - */ -export function step(id: string, options: StepOptions, fn: Body): void; -export function step(id: string, fn: Body): void; -export function step(id: string, a: StepOptions | Body, b?: Body): void { - const options = typeof a === 'function' ? {} : a; - const fn = typeof a === 'function' ? a : (b as Body); - registry.add('step', id, options, fn); -} - -/** A harness-only step: fixtures, effect checks and error tables. It contains no snippet markers. */ -export function check(id: string, options: StepOptions, fn: Body): void; -export function check(id: string, fn: Body): void; -export function check(id: string, a: StepOptions | Body, b?: Body): void { - const options = typeof a === 'function' ? {} : a; - const fn = typeof a === 'function' ? a : (b as Body); - registry.add('check', id, options, fn); -} - -/** What a page file default exports. `run.ts` lists these, `setup` is called once per run with the shared `World`. */ -export interface PageModule { - path: string; - phase: Phase; - setup(world: World): void; -} - -/** A docs page module. `phase` is the default for its steps, a step may override it. */ -export function page(pagePath: string, phase: Phase, setup: (world: World) => void): PageModule { - return { - path: pagePath, - phase, - setup(world) { - registry.withPage(pagePath, phase, () => setup(world)); - }, - }; -} - -/** - * Thrown by a step or check to skip itself with a precise reason, for example a keyset that lacks DataSync - * classes. The runner records the step as skipped, and steps that `needs` it are skipped as `blocked-by:`. - * Every skip fails the run (policy.ts): a run must run every valid test, so a missing class or feature is a defect of - * the keyset to fix, never something to tolerate. - * - * Two reason families are accepted: - * - `blocked-by:`: a root cause (a missing class) hides this step. - * - `needs-feature:`: the keyset lacks a switch that cannot be set from code (DataSync event rules live in the - * Admin Portal). - * - * `detail` is free text for the human: it is printed and written to the report as `note`, so the reason itself can - * stay a short, matchable tag. - */ -export class SkipStep extends Error { - readonly reason: string; - readonly detail?: string; - constructor(reason: string, detail?: string) { - super(`skipped: ${reason}`); - this.name = 'SkipStep'; - this.reason = reason; - this.detail = detail; - if (!/^(blocked-by:.+|needs-feature:[a-z0-9-]+)$/.test(reason)) { - throw new Error(`SkipStep reason must be "blocked-by:" or "needs-feature:", got "${reason}"`); - } - } -} - -/** Skips the running step from inside its body. Typed `never`, so code after the call is known to be unreachable. */ -export function skipStep(reason: string, detail?: string): never { - throw new SkipStep(reason, detail); -} diff --git a/docs-snippets/harness/tsc-project.ts b/docs-snippets/harness/tsc-project.ts deleted file mode 100644 index 9d0611560..000000000 --- a/docs-snippets/harness/tsc-project.ts +++ /dev/null @@ -1,48 +0,0 @@ -import { spawnSync } from 'node:child_process'; -import fs from 'node:fs'; -import path from 'node:path'; -import { fileURLToPath } from 'node:url'; - -/** - * Compiles small TypeScript files with the real `tsc` against the installed SDK types and returns the diagnostics. - * Used by the type-check-only modules (migration guide imports). The scratch directory sits below the staged suite so - * that `pubnub` and the type packages resolve from the suite's `node_modules`. - */ -const TSC = path.resolve(path.dirname(fileURLToPath(import.meta.resolve('typescript'))), 'tsc.js'); - -export interface TscResult { - ok: boolean; - /** `tsc` output, one diagnostic per line. */ - output: string; -} - -export function tscProject(parent: string, files: Record, compilerOptions: Record = {}): TscResult { - const dir = fs.mkdtempSync(path.join(parent, '.tsc-')); - try { - for (const [name, content] of Object.entries(files)) { - fs.mkdirSync(path.dirname(path.join(dir, name)), { recursive: true }); - fs.writeFileSync(path.join(dir, name), content); - } - fs.writeFileSync( - path.join(dir, 'tsconfig.json'), - JSON.stringify({ - compilerOptions: { - target: 'ES2022', - module: 'ESNext', - moduleResolution: 'bundler', - strict: true, - noEmit: true, - esModuleInterop: true, - skipLibCheck: true, - types: [], - ...compilerOptions, - }, - include: Object.keys(files).filter((f) => /\.tsx?$/.test(f)), - }), - ); - const run = spawnSync(process.execPath, [TSC, '-p', dir], { encoding: 'utf8' }); - return { ok: run.status === 0, output: `${run.stdout}${run.stderr}`.trim() }; - } finally { - fs.rmSync(dir, { recursive: true, force: true }); - } -} diff --git a/docs-snippets/harness/wait.ts b/docs-snippets/harness/wait.ts deleted file mode 100644 index c86a74788..000000000 --- a/docs-snippets/harness/wait.ts +++ /dev/null @@ -1,397 +0,0 @@ -/** - * Waiting primitives. The rule of the suite is "no blind retries and no fixed sleeps": every wait either polls a read until - * an assertion holds (`eventually`) or listens for the exact event the action should cause. - * - * - `eventually` poll a read (hereNow, history, App Context filter) that is eventually consistent on the service. - * - `expectEvent` register a listener BEFORE the action, resolve on the first matching event. - * - `connected` resolve when a subscribe is really active. Subscribing is asynchronous, a publish sent before that - * point is not delivered to the new subscriber, which is the classic race in pub/sub tests. - * - `watch` subscribe once and record everything, so the assertion can come after the publish without racing. - * - `expectStatus` resolve on a status category such as `PNConnectedCategory`. - * - * Every helper rejects with a message that says what was awaited and what was seen, so a timeout reads as a diagnosis. - */ -import assert from './assert.js'; -import { recorder, type AssertionRecord } from './recorder.js'; -import type PubNub from 'pubnub'; -import { redact } from './redact.cjs'; - -export interface EventuallyOptions { - /** Milliseconds. Default 15000. Presence reads use 45000. */ - timeout?: number; - /** Milliseconds between attempts. Default 500. */ - interval?: number; - label?: string; -} - -// Presence is the slowest eventual state: join and timeout events lag the heartbeat (see World.presenceOptions). -export const PRESENCE_TIMEOUT = 45_000; - -/** - * Polls a read until the callback stops throwing, then returns its value. No blind retries: on timeout the - * error carries the last assertion message so the failure explains itself. - */ -export async function eventually(fn: () => Promise | T, options: EventuallyOptions = {}): Promise { - const timeout = options.timeout ?? 15_000; - const interval = options.interval ?? 500; - const deadline = Date.now() + timeout; - const name = options.label ? `eventually: ${options.label}` : 'eventually'; - let last: unknown; - let attempts = 0; - for (;;) { - attempts += 1; - // Assertions inside the callback record into a private buffer. A failed attempt is expected and must not show up in the - // report as a failed assertion, only the attempt that passed is kept (see recorder.ts). - const buffer: AssertionRecord[] = []; - try { - const value = await recorder.attempt(buffer, fn); - recorder.commit(buffer); - recorder.record({ kind: 'eventually', label: `${name} held${attempts > 1 ? ` after ${attempts} attempts` : ''}`, ok: true }); - return value; - } catch (error) { - last = error; - } - if (Date.now() + interval > deadline) break; - await sleep(interval); - } - const detail = last instanceof Error ? last.message : String(last); - recorder.record({ kind: 'eventually', label: `${name} did not hold within ${timeout}ms (${attempts} attempts)`, ok: false, actual: redact(detail).slice(0, 200) }); - const error = new Error(`eventually${options.label ? ` (${options.label})` : ''} timed out after ${timeout}ms. Last: ${detail}`); - (error as Error & { cause?: unknown }).cause = last; - throw error; -} - -/** - * Removes a listener after the current dispatch. Removing it inside its own callback mutates the SDK's listener - * array while it iterates, which makes the SDK skip the next listener (the first live rerun lost a snippet's - * status listener this way). - */ -function detach(target: { removeListener(listener: PubNub.Listener): void }, listener: PubNub.Listener): void { - setImmediate(() => target.removeListener(listener)); -} - -/** Anything with the listener API: the client, a subscription or a subscription set. */ -export interface ListenerTarget { - addListener(listener: PubNub.Listener): void; - removeListener(listener: PubNub.Listener): void; -} - -/** Listener callbacks the helpers can wait for. `status` has its own helpers (`connected`, `expectStatus`). */ -export type EventKind = 'message' | 'signal' | 'presence' | 'messageAction' | 'file' | 'objects'; - -type EventOf = Parameters>[0]; - -export interface ExpectOptions { - timeout?: number; -} - -/** - * Registers a listener now and resolves on the first event of `kind` that satisfies `predicate`. Call it - * before the action that causes the event. Rejects on timeout. - */ -export function expectEvent( - target: ListenerTarget, - kind: K, - predicate: (event: EventOf) => boolean = () => true, - options: ExpectOptions = {}, -): Promise> { - const timeout = options.timeout ?? 15_000; - return new Promise>((resolve, reject) => { - const listener: PubNub.Listener = {}; - const timer = setTimeout(() => { - target.removeListener(listener); - reject(new Error(`no "${kind}" event matched within ${timeout}ms`)); - }, timeout); - // The step awaits this promise, and Mocha's own test timer keeps the process alive meanwhile. unref() only means an - // abandoned wait cannot hold the process open after the run. - timer.unref(); - (listener as Record)[kind] = (event: EventOf) => { - // A predicate that throws (reading a field of an event of another shape) means "not this one", not a test failure. - let matched = false; - try { - matched = predicate(event); - } catch { - matched = false; - } - if (!matched) return; - clearTimeout(timer); - detach(target, listener); - recorder.record({ kind: 'event', label: `received a "${kind}" event that matched`, ok: true }); - resolve(event); - }; - target.addListener(listener); - }); -} - -// Statuses that mean a subscribe loop is up. A re-subscribe reports `SubscriptionChanged`, a recovered network `Reconnected`. -const CONNECT_CATEGORIES = new Set(['PNConnectedCategory', 'PNSubscriptionChangedCategory', 'PNReconnectedCategory']); - -/** - * Resolves when the client reports a connected or subscription-changed status that covers every channel and - * group. Register it BEFORE the call that subscribes (statuses are not replayed). Affected channels - * accumulate, so several subscribes in one tick that coalesce into one status still resolve. - */ -export function connected( - client: Pick, - targets: { channels?: string[]; channelGroups?: string[] }, - options: ExpectOptions = {}, -): Promise { - const timeout = options.timeout ?? 15_000; - const needChannels = new Set(targets.channels ?? []); - const needGroups = new Set(targets.channelGroups ?? []); - return new Promise((resolve, reject) => { - const seenChannels = new Set(); - const seenGroups = new Set(); - const listener: PubNub.Listener = { - status: (status) => { - if (!CONNECT_CATEGORIES.has(String(status.category))) return; - const event = status as PubNub.StatusEvent; - // A status without lists means the whole subscription is up. - event.affectedChannels?.forEach((c) => seenChannels.add(c)); - event.subscribedChannels?.forEach((c) => seenChannels.add(c)); - event.affectedChannelGroups?.forEach((g) => seenGroups.add(g)); - const done = [...needChannels].every((c) => seenChannels.has(c)) && [...needGroups].every((g) => seenGroups.has(g)); - if (done) { - clearTimeout(timer); - detach(client, listener); - recorder.record({ kind: 'connected', label: `subscription connected (${[...needChannels, ...needGroups].length} channel or group name(s))`, ok: true }); - resolve(); - } - }, - }; - const timer = setTimeout(() => { - client.removeListener(listener); - reject( - new Error( - `not connected within ${timeout}ms. Wanted channels [${[...needChannels]}] groups [${[...needGroups]}], ` + - `saw channels [${[...seenChannels]}] groups [${[...seenGroups]}]`, - ), - ); - }, timeout); - // The step awaits this promise, and Mocha's own test timer keeps the process alive meanwhile. unref() only means an - // abandoned wait cannot hold the process open after the run. - timer.unref(); - client.addListener(listener); - }); -} - -/** One event a `Watch` saw, tagged with the listener it arrived on. */ -export interface RecordedEvent { - kind: EventKind; - event: unknown; -} - -/** - * A subscription that records every event it receives, so a step can subscribe once and assert later without - * racing the publish. Created with `watch()`. - */ -export class Watch { - readonly events: RecordedEvent[] = []; - // Callers blocked in `next()`. Each is woken as soon as an event arrives, instead of polling on a timer only. - private waiters: Array<() => void> = []; - - constructor( - private readonly set: ReturnType, - private readonly client: PubNub, - ) { - const record = (kind: EventKind) => (event: unknown) => { - this.events.push({ kind, event }); - this.waiters.splice(0).forEach((wake) => wake()); - }; - set.addListener({ - message: record('message'), - signal: record('signal'), - presence: record('presence'), - messageAction: record('messageAction'), - file: record('file'), - objects: record('objects'), - }); - } - - /** Resolves with the first recorded event of `kind` matching `predicate`, waiting if needed. */ - async next( - kind: K, - predicate: (event: EventOf) => boolean = () => true, - options: ExpectOptions = {}, - ): Promise> { - const timeout = options.timeout ?? 15_000; - const deadline = Date.now() + timeout; - for (;;) { - for (const recorded of this.events) { - if (recorded.kind !== kind) continue; - let ok = false; - try { - ok = predicate(recorded.event as EventOf); - } catch { - ok = false; - } - if (ok) { - recorder.record({ kind: 'event', label: `watched client received a "${kind}" event that matched`, ok: true }); - return recorded.event as EventOf; - } - } - const remaining = deadline - Date.now(); - if (remaining <= 0) { - assert.fail(`no "${kind}" event matched within ${timeout}ms. Recorded: ${this.events.map((e) => e.kind).join(',') || 'none'}`); - } - // Sleep until the next event wakes us. The 250 ms cap is a safety net, not a poll interval. - await new Promise((resolve) => { - const timer = setTimeout(resolve, Math.min(remaining, 250)); - this.waiters.push(() => { - clearTimeout(timer); - resolve(); - }); - }); - } - } - - /** Unsubscribes and drops the listeners. Call it when the step is done so the observer stops receiving. */ - stop(): void { - this.set.unsubscribe(); - this.set.removeAllListeners(); - void this.client; - } -} - -export interface WatchOptions { - channelGroups?: string[]; - presence?: boolean; - timeout?: number; -} - -/** - * Subscribes `client` to channels (and groups), waits until connected and returns a recording `Watch`. - * The channels must already carry the run suffix (use `world.n()`). - * - * Pass a client that has not subscribed before, for example `world.fresh('observer')`. A reused client can - * still report the status of an earlier unsubscribe for the same channel, which would resolve `connected` - * before the new subscription is active. - */ -export async function watch(client: PubNub, channels: string[], options: WatchOptions = {}): Promise { - const set = client.subscriptionSet({ - channels, - channelGroups: options.channelGroups, - subscriptionOptions: options.presence ? { receivePresenceEvents: true } : undefined, - }); - // Order matters: register the connected listener first, then subscribe. Statuses are not replayed, so a listener added - // after `subscribe()` could miss the very status it waits for. - const ready = connected(client, { channels, channelGroups: options.channelGroups }, { timeout: options.timeout }); - const handle = new Watch(set, client); - set.subscribe(); - await ready; - return handle; -} - -/** - * Resolves on the first status event whose category is one of `categories`. Register it before the action. - */ -export function expectStatus( - client: Pick, - categories: string[], - options: ExpectOptions = {}, -): Promise { - const timeout = options.timeout ?? 15_000; - return new Promise((resolve, reject) => { - const listener: PubNub.Listener = { - status: (status) => { - const category = String(status.category); - if (!categories.includes(category)) return; - clearTimeout(timer); - detach(client, listener); - recorder.record({ kind: 'status', label: `status ${category} received`, ok: true }); - resolve(category); - }, - }; - const timer = setTimeout(() => { - client.removeListener(listener); - reject(new Error(`no status in [${categories.join(', ')}] within ${timeout}ms`)); - }, timeout); - // The step awaits this promise, and Mocha's own test timer keeps the process alive meanwhile. unref() only means an - // abandoned wait cannot hold the process open after the run. - timer.unref(); - client.addListener(listener); - }); -} - -export const sleep = (ms: number): Promise => new Promise((resolve) => setTimeout(resolve, ms)); - -/** - * Tallies the events a step's recording listener saw, by kind, with the time of the first one. A step that waits for - * "every event kind" uses it so a timeout names the kinds that arrived (with counts) and the kinds still missing, - * instead of reporting only the one assertion that failed last. Times are milliseconds since the log was created. - * - * `match` narrows a kind to the events the step actually triggered (for example the join of the second client, not any - * presence event on a busy channel). An event that does not match is kept in `of()` but does not count as arrived. - */ -export class KindLog { - private readonly events = new Map(); - private readonly first = new Map(); - private readonly started = Date.now(); - - constructor( - readonly kinds: readonly string[], - private readonly match: Record boolean> = {}, - ) { - kinds.forEach((kind) => this.events.set(kind, [])); - } - - /** Listener callback for one kind: `message: log.on('message')`. */ - on(kind: string): (event: E) => void { - return (event) => { - this.events.get(kind)!.push(event); - if (this.matches(kind, event) && !this.first.has(kind)) this.first.set(kind, Date.now() - this.started); - }; - } - - /** The events of one kind, in arrival order (all of them, matching or not). */ - of(kind: string): E[] { - return this.events.get(kind) ?? []; - } - - private matches(kind: string, event: E): boolean { - const predicate = this.match[kind]; - if (!predicate) return true; - try { - return predicate(event); - } catch { - return false; - } - } - - /** The events of one kind that count as the awaited one. */ - matching(kind: string): E[] { - return this.of(kind).filter((event) => this.matches(kind, event)); - } - - /** "arrived: message x1 (+120ms), presence x2 (+800ms). missing: signal, objects. other events: presence x3". */ - summary(required: readonly string[] = this.kinds): string { - const arrived = required.filter((k) => this.matching(k).length > 0).map((k) => `${k} x${this.matching(k).length} (+${this.first.get(k)}ms)`); - const missing = required.filter((k) => this.matching(k).length === 0); - const other = required.filter((k) => this.of(k).length > this.matching(k).length).map((k) => `${k} x${this.of(k).length - this.matching(k).length}`); - return `arrived: ${arrived.join(', ') || 'none'}. missing: ${missing.join(', ') || 'none'}${other.length ? `. other events: ${other.join(', ')}` : ''}`; - } - - /** Throws (for `eventually`) while any required kind has no matching event. The message carries the full summary. */ - assertArrived(required: readonly string[] = this.kinds): void { - if (required.some((k) => this.matching(k).length === 0)) throw new Error(this.summary(required)); - } -} - -/** - * For conditions that are not event kinds (a console line per handler, say). Throws one error that lists every - * condition that does not hold, so a timeout names all missing ones and not only the first. - */ -export function assertAll(conditions: Record): void { - const missing = Object.entries(conditions).filter(([, ok]) => !ok).map(([name]) => name); - if (missing.length > 0) { - const held = Object.entries(conditions).filter(([, ok]) => ok).map(([name]) => name); - throw new Error(`missing: ${missing.join(', ')}. held: ${held.join(', ') || 'none'}`); - } -} - -/** - * How long a step waits for a presence event it caused. The keyset announces a join or leave after the 2 s debounce when - * the channel holds at most 20 occupants (announce max), and falls back to one summary per 10 s interval above that. - * 30 s covers debounce, one interval and a long-poll cycle with margin. - */ -export const PRESENCE_EVENT_TIMEOUT = 30_000; diff --git a/docs-snippets/harness/world.ts b/docs-snippets/harness/world.ts deleted file mode 100644 index 425dce356..000000000 --- a/docs-snippets/harness/world.ts +++ /dev/null @@ -1,403 +0,0 @@ -/** - * The `world` object every page module receives. It is the harness side of the test: run id, name suffixing, - * placeholders, the cast of clients, cleanup tracking, spies and console capture. - * - * Architecture notes: - * - One `World` per run. `mocha-entry.ts` builds it from the environment, registers all pages with it, and disposes - * it in the root `after` hook (plus the signal handler for a killed run). - * - Clients are created lazily on first use and cached by role. A run that only has the open keyset never - * builds `server`, `alice` or `bob`, so it never needs those keys. - * - Every client is tracked and destroyed at the end. A leaked subscribed client can keep the Node process alive and - * leaves presence entries on the keyset. - * - Names go through `n()` (the run suffix, see fixtures/names.ts). Inside snippets the stage rewrite applies the suffix, - * outside snippets the step author calls `world.n()`. - */ -import type { ChildProcess } from 'node:child_process'; -import path from 'node:path'; -import { fileURLToPath } from 'node:url'; -import assert from './assert.js'; -import PubNub from 'pubnub'; -import { suffixName } from '../fixtures/names.js'; -import { randomRunId, type Env, type LogLevelName, type Role, type SecuredKeyset } from './env.js'; -import { LocalProxy } from './proxy.js'; -import { addSecrets } from './redact.cjs'; - -const HERE = path.dirname(fileURLToPath(import.meta.url)); - -/** - * The Sneaker Drop cast. `server` is the staff back office, `shopper`/`friend` are plain open-keyset users, `alice`/`bob` hold - * tokens. `server`, `alice` and `bob` live on a secured keyset: the one of the running step's role (`pam` or `datasync`). - */ -export type ClientRole = 'server' | 'alice' | 'bob' | 'shopper' | 'friend' | 'observer'; - -// The env var holds a name (`DOCS_SNIPPETS_LOG_LEVEL=debug`), the SDK wants its enum value. -const LOG_LEVELS: Record = { - trace: PubNub.LogLevel.Trace, - debug: PubNub.LogLevel.Debug, - info: PubNub.LogLevel.Info, - warn: PubNub.LogLevel.Warn, - error: PubNub.LogLevel.Error, - none: PubNub.LogLevel.None, -}; - -/** What `world.spy()` returns: every call's arguments and result, plus a way to undo the wrap. */ -export interface SpyHandle { - calls: unknown[][]; - results: R[]; - restore(): void; -} - -export interface ConsoleCapture { - lines: string[]; - restore(): void; -} - -/** - * Everything a step needs: run id, name suffixing, placeholders, a multi-client factory and tracked cleanup. - * - * Clients are created lazily, so a run that only touches the open keyset never needs the secured one. - * The secret key only ever reaches the `server` client and the snippet that shows server configuration. - */ -export class World { - readonly runId: string; - readonly env: Env; - /** Directory of fixture files. Steps that read relative paths change into it. */ - readonly fixtures = path.resolve(HERE, '..', 'fixtures'); - readonly proxy = new LocalProxy(); - private readonly tracked = new Set(); - private readonly children = new Set(); - private readonly placeholderValues = new Map(); - private readonly cache = new Map(); - /** - * Role of the step that is running, set by mocha-entry.ts before each step. Snippets name the secured keyset `secured` - * (hidden lines such as `...world.live.secured` are docs text and stay unchanged), and this decides which keyset that is. - */ - private role_: Role = 'open'; - private restorers: Array<() => void> = []; - private counter = 0; - - constructor(env: Env, runId?: string) { - this.env = env; - this.runId = runId ?? env.runId ?? randomRunId(); - } - - /** Sets the role of the running step. Called by the Mocha binding, never by a step. */ - useRole(role: Role): void { - this.role_ = role; - } - - get role(): Role { - return this.role_; - } - - /** The secured role of the running step. A step of role `open` that touches a secured keyset is a wiring error. */ - get securedRole(): 'pam' | 'datasync' { - if (this.role_ === 'open') { - throw new Error("this step runs as role open but used a secured keyset (world.server, alice, bob or live.secured). Give it keyset: 'pam' or 'datasync'."); - } - return this.role_; - } - - private securedKeys(): SecuredKeyset { - const role = this.securedRole; - const keys = this.env[role]; - if (!keys) throw new Error(`${role} keyset is not configured (${role === 'pam' ? 'PAM_*' : 'DS_*'})`); - return keys; - } - - /** - * Spread into snippet constructors through hidden lines: keys and log level only, never identity. - * The docs show `publishKey: 'YOUR_PUBLISH_KEY'`. The hidden `...world.live.open` after it overrides the placeholder with - * the real key, so the page text stays generic and the run is live. Each property is a getter so that reading a keyset - * that is not configured throws a clear message at the moment of use, not at startup. - * Typed as Partial so the spread after the visible placeholder keys compiles (TS2783 otherwise). */ - get live(): { - open: Partial<{ publishKey: string; subscribeKey: string; logLevel: PubNub.LogLevel }>; - /** Keys only. Use it where the snippet sets `logLevel` itself, because `open` carries the run's log level. */ - openKeys: Partial<{ publishKey: string; subscribeKey: string }>; - openSubscribeOnly: Partial<{ subscribeKey: string; logLevel: PubNub.LogLevel }>; - secured: Partial<{ publishKey: string; subscribeKey: string; secretKey: string; logLevel: PubNub.LogLevel }>; - } { - const logLevel = LOG_LEVELS[this.env.logLevel]; - const open = this.env.open; - const securedKeys = (): SecuredKeyset => this.securedKeys(); - return { - get openKeys() { - if (!open) throw new Error('open keyset is not configured (SUBSCRIBE_KEY, PUBLISH_KEY)'); - return { ...open }; - }, - get open() { - if (!open) throw new Error('open keyset is not configured (SUBSCRIBE_KEY, PUBLISH_KEY)'); - return { ...open, logLevel }; - }, - get openSubscribeOnly() { - if (!open) throw new Error('open keyset is not configured (SUBSCRIBE_KEY, PUBLISH_KEY)'); - return { subscribeKey: open.subscribeKey, logLevel }; - }, - get secured() { - return { ...securedKeys(), logLevel }; - }, - }; - } - - /** Spread into a configuration to point a client at a port where nothing listens (port 1 is reserved, connect is refused at once). Every request fails fast with a network error. */ - get unreachable(): { origin: string; ssl: false } { - return { origin: '127.0.0.1:1', ssl: false }; - } - - /** Applies the run suffix to a name. Use it outside snippet markers, where the stage rewrite does not apply. */ - n(name: string): string { - return suffixName(name, this.runId); - } - - // ---- placeholders --- - // A placeholder is a docs literal such as 'replace_with_message_timetoken' that only the run can fill. The stage rewrite - // turns it into world.ph('messageTimetoken'), and an earlier step supplies the value with world.setPh(). Reading a key - // nobody set throws, which points at the step order instead of sending the service a bogus timetoken.--------------------------------------------------------------------------------------- - - /** Live value for a placeholder literal. Staged snippets call this, see `fixtures/names.ts`. */ - ph(key: string): string { - const value = this.placeholderValues.get(key); - if (value === undefined) { - throw new Error(`placeholder "${key}" has no value yet. A step must call world.setPh('${key}', ...) first.`); - } - return value; - } - - setPh(key: string, value: string): void { - this.placeholderValues.set(key, value); - } - - // ---- values out of a try block -------------------------------------------------------------------------------- - // Some pages show a call inside `try { ... } catch (status) { console.log(status); }`. The `const` the call returns - // is block scoped, so a hidden `world.keep('name', value)` as the last line of the try hands it to the step's assertions, - // which read it with `world.take('name')`. If the try threw, the catch only logged and keep never ran: take then fails - // the step, so a swallowed error can never pass. - - private readonly keptValues = new Map(); - - /** Stores a value from inside a snippet's try block (hidden line). */ - keep(name: string, value: unknown): void { - this.keptValues.set(name, value); - } - - /** Reads and removes a value stored by `keep`. Throws when the snippet's try block did not reach `keep`. */ - take(name: string): T { - if (!this.keptValues.has(name)) { - throw new Error(`the snippet did not reach world.keep('${name}'): its try block threw and the catch only logged it (see the log above)`); - } - const value = this.keptValues.get(name) as T; - this.keptValues.delete(name); - return value; - } - - // ---- clients ------------------------------------------------------------------------------------------------ - - /** Registers a client for `dispose()`. Use it for clients a step builds by hand (for example the docs `new PubNub(...)`). */ - track(client: T): T { - this.tracked.add(client); - return client; - } - - /** Registers a child process. `dispose()` and the signal handlers kill it, so a killed run leaves no subscribed process behind. */ - trackChild(child: T): T { - this.children.add(child); - return child; - } - - /** - * Creates a tracked client on a keyset. `secured` means the secured keyset of the running step's role. `userId` gets the run - * suffix unless it already carries it. - */ - client(keyset: 'open' | 'secured' | Role, userId: string, extra: Partial = {}): PubNub { - const role: Role = keyset === 'secured' ? this.securedRole : keyset; - const keys = role === 'open' ? this.env.open : this.env[role]; - if (!keys) throw new Error(`${role} keyset is not configured`); - const config: PubNub.PubNubConfiguration = { - publishKey: keys.publishKey, - subscribeKey: keys.subscribeKey, - userId: userId.includes(`-r${this.runId}`) ? userId : this.n(userId), - logLevel: LOG_LEVELS[this.env.logLevel], - ...extra, - }; - return this.track(new PubNub(config)); - } - - /** A new open-keyset client with a unique user id, for steps that must not disturb shared clients. */ - fresh(label = 'fresh', extra: Partial = {}): PubNub { - this.counter += 1; - return this.client('open', `${label}-${this.counter}`, extra); - } - - /** - * Short heartbeat settings so presence waits stay small. The default presence timeout is 300 s, so a client that - * leaves without unsubscribing would linger in hereNow for five minutes. 20 s with an 8 s heartbeat keeps the waits in - * wait.ts (PRESENCE_TIMEOUT) realistic. - */ - get presenceOptions(): Partial { - return { presenceTimeout: 20, heartbeatInterval: 8 }; - } - - // Memoizes one client per role. Roles are shared across steps on purpose: tokens, listeners and subscriptions carry over, - // which mirrors an app that keeps one client. Steps that must not disturb that use `fresh()`. - // Secured roles are cached per keyset role, so `alice` of a pam step and `alice` of a datasync step are different clients. - private cast(role: ClientRole, build: () => PubNub): PubNub { - const key = role === 'server' || role === 'alice' || role === 'bob' ? `${this.securedRole}:${role}` : role; - let client = this.cache.get(key); - if (!client) { - client = build(); - this.cache.set(key, client); - } - return client; - } - - /** Open keyset, plain user. Most snippets run as this client. */ - get shopper(): PubNub { - return this.cast('shopper', () => this.client('open', 'shopper')); - } - get friend(): PubNub { - return this.cast('friend', () => this.client('open', 'friend')); - } - /** Never runs docs code. Subscribes and reads to verify effects. */ - get observer(): PubNub { - return this.cast('observer', () => this.client('open', 'observer')); - } - /** Secured keyset, secret key. Grants, DataSync writes, revokes. */ - get server(): PubNub { - return this.cast('server', () => { - const secured = this.securedKeys(); - // heartbeatInterval 0: a server that only signs grants has no use for presence heartbeats. - return registerGrantedTokens(this.client('secured', 'staff-server', { secretKey: secured.secretKey, heartbeatInterval: 0 })); - }); - } - /** Secured keyset, token holder. Call `setToken('alice', token)` after the grant step. */ - get alice(): PubNub { - return this.cast('alice', () => this.client('secured', 'user-alice')); - } - get bob(): PubNub { - return this.cast('bob', () => this.client('secured', 'user-bob')); - } - - setToken(role: 'alice' | 'bob', token: string): void { - (role === 'alice' ? this.alice : this.bob).setToken(token); - } - - // ---- helpers ------------------------------------------------------------------------------------------------ - - /** - * Wraps `obj[method]`, records calls and results, and calls through. Restored by `dispose()`. - * Used where the docs only show that something was called (a status listener, a transport) and the effect is internal. - */ - spy(obj: T, method: K): SpyHandle { - const original = (obj as Record)[method] as (...args: unknown[]) => unknown; - const handle: SpyHandle = { calls: [], results: [], restore: () => undefined }; - (obj as Record)[method] = function (this: unknown, ...args: unknown[]) { - const result = original.apply(obj, args); - handle.calls.push(args); - handle.results.push(result); - return result; - }; - handle.restore = () => { - (obj as Record)[method] = original; - }; - this.restorers.push(handle.restore); - return handle; - } - - /** - * Captures `console.log` (or another method) lines. The logging pages print through console, so asserting on the lines is - * the only way to prove the sample works. Call `restore()` in a `finally`. */ - captureConsole(method: 'log' | 'info' | 'debug' | 'warn' | 'error' = 'log'): ConsoleCapture { - const original = console[method]; - const lines: string[] = []; - console[method] = (...args: unknown[]) => { - lines.push(args.map((a) => (typeof a === 'string' ? a : safeJson(a))).join(' ')); - }; - const restore = (): void => { - console[method] = original; - }; - this.restorers.push(restore); - return { lines, restore }; - } - - /** Runs `fn` with `console.log` captured and restores it afterwards, also when `fn` throws. */ - async withConsole(fn: (logs: ConsoleCapture) => Promise | T): Promise { - const logs = this.captureConsole('log'); - try { - return await fn(logs); - } finally { - logs.restore(); - } - } - - /** - * Reusable shape checks for in-snippet or post-snippet assertions. A timetoken is checked the same way everywhere: a string - * of 17 digits (10 ns ticks since 1970), as the SDK types declare it. - */ - readonly check = { - timetoken(value: unknown): void { - assert.equal(typeof value, 'string', `expected a timetoken string, got ${typeof value} ${String(value)}`); - assert.match(String(value), /^\d{17}$/, `expected a 17 digit timetoken, got ${String(value)}`); - }, - /** - * The timetoken of `time()`. SDK finding: the types declare `timetoken: string` (src/core/endpoints/time.ts), but the - * SDK returns the parsed JSON number, which is above Number.MAX_SAFE_INTEGER and can lose its last digit. This check - * pins that behavior: when the SDK returns a string, it fails and the special case can go. - */ - serverTime(value: unknown): void { - assert.equal(typeof value, 'number', `time() timetoken: the SDK used to return a number although the types say string, got ${typeof value}`); - assert.match(String(value), /^\d{17}$/, `expected a 17 digit timetoken, got ${String(value)}`); - }, - }; - - /** Destroys every tracked client and restores spies and console. */ - async dispose(): Promise { - // Order: restore patched globals first so cleanup logging is not captured, then kill child processes (they hold - // subscriptions), then destroy clients, then stop the proxy their sockets might still use. - for (const restore of this.restorers.splice(0)) restore(); - for (const child of this.children) { - if (child.exitCode === null && !child.killed) child.kill('SIGKILL'); - } - this.children.clear(); - for (const client of this.tracked) { - try { - // Drop listeners first so the final unsubscribe status does not fire into a step that already finished. - client.removeAllListeners(); - client.destroy(); - } catch { - // A client that failed to build or was destroyed by its own step is fine. - } - } - this.tracked.clear(); - this.cache.clear(); - await this.proxy.stop(); - } -} - -/** - * Every token the server client grants becomes a known secret for the redaction (redact.cjs), so its exact value is masked - * in any output, on top of the token shape pattern. Samples print tokens with console.log, and a failing step shows its - * console output in the CI log and in report/run.json. - */ -function registerGrantedTokens(client: PubNub): PubNub { - const grant = client.grantToken.bind(client) as (...args: unknown[]) => unknown; - (client as unknown as { grantToken: (...args: unknown[]) => unknown }).grantToken = (...args: unknown[]) => { - const result = grant(...args); - if (result instanceof Promise) { - return result.then((token: unknown) => { - if (typeof token === 'string') addSecrets([token]); - return token; - }); - } - return result; - }; - return client; -} - -function safeJson(value: unknown): string { - try { - return JSON.stringify(value); - } catch { - return String(value); - } -} diff --git a/docs-snippets/import-pubnub.ts b/docs-snippets/import-pubnub.ts deleted file mode 100644 index d1ded746e..000000000 --- a/docs-snippets/import-pubnub.ts +++ /dev/null @@ -1,16 +0,0 @@ -// snippet.importPubNub -import PubNub from 'pubnub'; -// snippet.end - -// snippet.importFS -import fs from 'fs'; -// snippet.end - -// snippet.PubNubinitBasicUsage -// Initialize PubNub with your keys -const pubnub = new PubNub({ - publishKey: 'YOUR_PUBLISH_KEY', - subscribeKey: 'YOUR_SUBSCRIBE_KEY', - userId: 'YOUR_USER_ID', -}); -// snippet.end diff --git a/docs-snippets/message-persistence.ts b/docs-snippets/message-persistence.ts deleted file mode 100644 index 0d43d23d2..000000000 --- a/docs-snippets/message-persistence.ts +++ /dev/null @@ -1,51 +0,0 @@ -import PubNub from '../lib/types'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'myUniqueUserId', -}); - -// snippet.fetchMessagesWithMetaActions -try { - const response = await pubnub.fetchMessages({ - channels: ['my_channel'], - stringifiedTimeToken: true, - includeMeta: true, - includeMessageActions: true, - includeCustomMessageType: true, - }); - console.log('fetch messages response:', response); -} catch (error) { - console.error(`fetch messages failed with error: ${error}`); -} -// snippet.end - -// snippet.deleteSpecificMessages -try { - const response = await pubnub.deleteMessages({ - channel: 'ch1', - start: 'replace-with-start-timetoken', - end: 'replace-with-end-timetoken', - }); - console.log('delete messages response:', response); -} catch (error) { - console.error(`delete messages failed with error: ${error}`); -} -// snippet.end - -// snippet.messageCountTimetokensForChannels -try { - const response = await pubnub.messageCounts({ - channels: ['ch1', 'ch2', 'ch3'], - channelTimetokens: [ - 'replace-with-channel-timetoken-ch1', // timetoken for channel ch1 - 'replace-with-channel-timetoken-ch2', // timetoken for channel ch2 - 'replace-with-channel-timetoken-ch3', // timetoken for channel ch3 - ], - }); - console.log('message count response:', response); -} catch (error) { - console.error(`message count failed with error: ${error}`); -} -// snippet.end diff --git a/docs-snippets/migration-guides/v11.ts b/docs-snippets/migration-guides/v11.ts deleted file mode 100644 index dfbf4140a..000000000 --- a/docs-snippets/migration-guides/v11.ts +++ /dev/null @@ -1,88 +0,0 @@ -import assert from '../harness/assert.js'; -import PubNub from 'pubnub'; -import { page, step, check } from '../harness/step.js'; -import { connected, eventually } from '../harness/wait.js'; - -/** - * Examples for the version 11 migration guide. - */ -export default page('migration-guides/v11', 'P0', (world) => { - step('migrationV11LegacySubscriptionManager', { proves: 'The legacy manager fires PNConnectedCategory again on a second subscribe and never PNSubscriptionChangedCategory.' }, async () => { - // snippet.migrationV11LegacySubscriptionManager - const pubnub = new PubNub({ - subscribeKey: 'mySubscribeKey', - publishKey: 'myPublishKey', - userId: 'myUserId', - enableEventEngine: false, // set explicitly to keep legacy behavior - // snippet.hide - ...world.live.open, - // snippet.show - }); - // snippet.end - world.track(pubnub); - // Legacy behavior: PNConnectedCategory fires again when the channel mix changes, and there is no PNSubscriptionChangedCategory. - const categories: string[] = []; - pubnub.addListener({ status: (s) => categories.push(String(s.category)) }); - const first = connected(pubnub, { channels: [world.n('ch1')] }); - pubnub.subscribe({ channels: [world.n('ch1')] }); - await first; - pubnub.subscribe({ channels: [world.n('ch2')] }); - await eventually(() => assert.equal(categories.filter((c) => c === 'PNConnectedCategory').length, 2), { label: 'second PNConnectedCategory' }); - assert.ok(!categories.includes('PNSubscriptionChangedCategory')); - pubnub.unsubscribeAll(); - }); - - step('migrationV11RestoreFalse', { proves: 'A client built with restore false is valid and works.' }, async () => { - // snippet.migrationV11RestoreFalse - const pubnub = new PubNub({ - subscribeKey: 'mySubscribeKey', - publishKey: 'myPublishKey', - userId: 'myUserId', - restore: false, // set explicitly to keep legacy behavior - // snippet.hide - ...world.live.open, - // snippet.show - }); - // snippet.end - world.track(pubnub); - // The option is accepted and the client works. The network-loss behavior itself needs a browser. - world.check.serverTime((await pubnub.time()).timetoken); - }); - - // Offline: only the constructor is exercised. Version 11 lifted the upper bounds, so values the old SDK rejected must be accepted. - check('migrationV11RetryPolicyBounds', { proves: 'Retry policy values the old SDK rejected are accepted since version 11.' }, () => { - const open = world.live.open; - const config = (retryConfiguration: PubNub.RequestRetryPolicy): PubNub.PubNubConfiguration => ({ - subscribeKey: open.subscribeKey!, - userId: world.n('v11-bounds'), - retryConfiguration, - }); - // "Version 11.0.0 accepts" maximumRetry above 10 (linear), above 6 and maximumDelay above 150 (exponential). - world.track(new PubNub(config(PubNub.LinearRetryPolicy({ delay: 2, maximumRetry: 50 })))); - world.track(new PubNub(config(PubNub.ExponentialRetryPolicy({ minimumDelay: 2, maximumDelay: 600, maximumRetry: 20 })))); - // The minimum delay of the exponential policy is still checked. - assert.throws(() => new PubNub(config(PubNub.ExponentialRetryPolicy({ minimumDelay: 1, maximumDelay: 10, maximumRetry: 2 }))), /minimum|delay/i); - }); - - check('migrationV11MaintainPresenceState', { needs: ['preflightPresence'], timeout: 90_000, proves: 'With maintainPresenceState true the client re-sends its state after leaving and rejoining, and with false the state is gone.' }, async () => { - const channel = world.n('ch1'); - const kept = world.fresh('v11-state-kept', { ...world.presenceOptions }); - const dropped = world.fresh('v11-state-dropped', { ...world.presenceOptions, maintainPresenceState: false }); - const stateOn = async (client: PubNub): Promise => (await world.observer.getState({ uuid: client.getUserId(), channels: [channel] })).channels[channel]; - for (const client of [kept, dropped]) { - const first = connected(client, { channels: [channel] }); - client.subscribe({ channels: [channel], withPresence: true }); - await first; - await client.setState({ state: { mood: 'happy' }, channels: [channel] }); - await eventually(async () => assert.deepEqual(await stateOn(client), { mood: 'happy' }), { timeout: 30_000, label: 'state set' }); - client.disconnect(); - await eventually(async () => assert.deepEqual(await stateOn(client), {}), { timeout: 30_000, label: 'state cleared by the leave' }); - client.reconnect(); - } - await eventually(async () => assert.deepEqual(await stateOn(kept), { mood: 'happy' }), { timeout: 30_000, label: 'state re-sent when the client joins again' }); - // The other client joined at the same time, so its state would be back by now if it were sent. - assert.notDeepEqual(await stateOn(dropped), { mood: 'happy' }, 'with maintainPresenceState: false the state is gone after leaving'); - kept.unsubscribeAll(); - dropped.unsubscribeAll(); - }); -}); diff --git a/docs-snippets/migration-guides/v12.ts b/docs-snippets/migration-guides/v12.ts deleted file mode 100644 index 170424721..000000000 --- a/docs-snippets/migration-guides/v12.ts +++ /dev/null @@ -1,58 +0,0 @@ -import assert from '../harness/assert.js'; -import fs from 'node:fs'; -import path from 'node:path'; -import { fileURLToPath } from 'node:url'; -import PubNub from 'pubnub'; -import { page, check } from '../harness/step.js'; -import { LocalProxy } from '../harness/proxy.js'; - -/** - * Examples for the version 12 migration guide. - */ -const SDK_ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.resolve('pubnub'))), '..', '..'); - -export default page('migration-guides/v12', 'P7', (world) => { - check('migrationV12ProxyEnvironmentVariablesIgnored', { timeout: 45_000, proves: 'The SDK ignores proxy environment variables, shown by zero tunnel requests at the local proxy.' }, async () => { - const proxy = new LocalProxy(); - await proxy.start(); - const names = ['HTTP_PROXY', 'HTTPS_PROXY', 'ALL_PROXY', 'http_proxy', 'https_proxy', 'all_proxy']; - const saved = Object.fromEntries(names.map((n) => [n, process.env[n]])); - const uri = `http://${proxy.host}:${proxy.port}`; - for (const n of names) process.env[n] = uri; - try { - const pubnub = world.fresh('v12-env'); - const before = proxy.connects; - world.check.serverTime((await pubnub.time()).timetoken); - assert.equal(proxy.connects, before, 'proxy environment variables are not read'); - - // "read the variable in your own code and pass the proxy address to setProxy()": a URI string works. - pubnub.setProxy(process.env.HTTPS_PROXY!); - world.check.serverTime((await pubnub.time()).timetoken); - assert.ok(proxy.connects > before, 'setProxy(uri) routes requests through the proxy'); - pubnub.setProxy(); - } finally { - await proxy.stop(); - for (const n of names) { - if (saved[n] === undefined) delete process.env[n]; - else process.env[n] = saved[n]; - } - } - }); - - // Offline: the throw happens inside `setProxy()`, no request is sent. - check('migrationV12SetProxyKeepAlive', { proves: 'setProxy throws when the client was built with keepAlive true, before any request.' }, () => { - // "setProxy() ... throws if keepAlive is true." - const pubnub = world.fresh('v12-keepalive', { keepAlive: true }); - assert.throws(() => pubnub.setProxy({ hostname: '127.0.0.1', port: 8080, protocol: 'http' }), /keepAlive/); - }); - - check('migrationV12ReactNativePolyfill', { proves: 'The SDK manifest lists react-native-url-polyfill as an optional peer dependency and requires Node 22 or later.' }, () => { - // "Version 12.0.0 lists react-native-url-polyfill as an optional peer dependency, so installing pubnub no longer installs it." - const manifest = JSON.parse(fs.readFileSync(path.join(SDK_ROOT, 'package.json'), 'utf8')); - assert.ok(manifest.peerDependencies?.['react-native-url-polyfill']); - assert.equal(manifest.peerDependenciesMeta?.['react-native-url-polyfill']?.optional, true); - assert.ok(!manifest.dependencies?.['react-native-url-polyfill']); - assert.equal(manifest.engines.node, '>=22.0.0'); - void PubNub; - }); -}); diff --git a/docs-snippets/migration-guides/v8.ts b/docs-snippets/migration-guides/v8.ts deleted file mode 100644 index c92892653..000000000 --- a/docs-snippets/migration-guides/v8.ts +++ /dev/null @@ -1,40 +0,0 @@ -// snippet.migrationV8AfterImport -import PubNub, { AppContext } from 'pubnub' - -// snippet.end -import assert from '../harness/assert.js'; -import path from 'node:path'; -import { fileURLToPath } from 'node:url'; -import { page, step, check } from '../harness/step.js'; -import { tscProject } from '../harness/tsc-project.js'; - -/** - * Examples for the version 8 migration guide. - */ -const HERE = path.dirname(fileURLToPath(import.meta.url)); - -export default page('migration-guides/v8', 'P0', () => { - step('migrationV8AfterParams', { proves: 'The typed parameters constant of the v8 guide compiles against the SDK types.' }, () => { - void PubNub; - // snippet.migrationV8AfterParams - // other code - - const params: AppContext.GetMembershipsParameters = {}; - // snippet.end - assert.deepEqual(params, {}); - }); - - check('migrationV8TypeImports', { timeout: 90_000, proves: 'The before sample for type imports fails to compile and the after sample compiles.' }, () => { - const before = tscProject(HERE, { - 'before.ts': `import PubNub, { GetMembershipsParameters } from 'pubnub'\n\nconst params: GetMembershipsParameters = {};\n`, - }); - assert.equal(before.ok, false, 'importing GetMembershipsParameters directly fails in 8.x.x'); - assert.match(before.output, /GetMembershipsParameters/); - - const namespaces = ['Subscription', 'AppContext', 'ChannelGroups', 'Publish', 'Signal', 'Presence', 'History', 'MessageAction', 'FileSharing', 'PAM', 'Time', 'Push']; - const after = tscProject(HERE, { - 'after.ts': `import PubNub, { ${namespaces.join(', ')} } from 'pubnub'\n\nconst params: AppContext.GetMembershipsParameters = {};\nvoid PubNub; void params;\n`, - }); - assert.ok(after.ok, `the 8.x.x import and every namespace compile.\n${after.output}`); - }); -}); diff --git a/docs-snippets/mobile-push.ts b/docs-snippets/mobile-push.ts deleted file mode 100644 index d8046d126..000000000 --- a/docs-snippets/mobile-push.ts +++ /dev/null @@ -1,39 +0,0 @@ -import PubNub from '../lib/types'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'myUniqueUserId', -}); - -// snippet.simpleNotificationPayloadFCMandAPNS -const builder = PubNub.notificationPayload('Chat invitation', "You have been invited to 'quiz' chat"); -builder.sound = 'default'; - -console.log(JSON.stringify(builder.buildPayload(['apns', 'fcm']), null, 2)); -// snippet.end - -// snippet.simpleNotificationPayloadFCMandAPNSH/2 -const payloadBuilder = PubNub.notificationPayload('Chat invitation', "You have been invited to 'quiz' chat"); -payloadBuilder.apns.configurations = [{ targets: [{ topic: 'com.meetings.chat.app' }] }]; -payloadBuilder.sound = 'default'; - -console.log(JSON.stringify(payloadBuilder.buildPayload(['apns2', 'fcm']), null, 2)); -// snippet.end - -// snippet.simpleNotificationPayloadFCMandAPNSH/2CustomConfiguration -const configuration = [ - { - collapseId: 'invitations', - expirationDate: new Date(Date.now() + 10000), - targets: [{ topic: 'com.meetings.chat.app' }], - }, -]; -const customConfigurationBuilder = PubNub.notificationPayload( - 'Chat invitation', - "You have been invited to 'quiz' chat", -); -customConfigurationBuilder.apns.configurations = configuration; - -console.log(JSON.stringify(customConfigurationBuilder.buildPayload(['apns2', 'fcm']), null, 2)); -// snippet.end diff --git a/docs-snippets/package.json b/docs-snippets/package.json deleted file mode 100644 index 094aa8942..000000000 --- a/docs-snippets/package.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "type": "module", - "private": true -} diff --git a/docs-snippets/presence.ts b/docs-snippets/presence.ts deleted file mode 100644 index 821f00f1d..000000000 --- a/docs-snippets/presence.ts +++ /dev/null @@ -1,42 +0,0 @@ -import PubNub from '../lib/types'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'userId', -}); - -// snippet.hereNowWithState -try { - const response = await pubnub.hereNow({ - channels: ['my_channel'], - includeState: true, - }); - console.log('hereNow response:', response); -} catch (error) { - console.error(`hereNow failed with error: ${error}`); -} -// snippet.end - -// snippet.hereNowFetchOccupancyOnly -try { - const response = await pubnub.hereNow({ - channels: ['my_channel'], - includeUUIDs: false, - }); - console.log('hereNow response:', response); -} catch (error) { - console.error(`hereNow failed with error: ${error}`); -} -// snippet.end - -// snippet.hereNowChannelGroup -try { - const response = await pubnub.hereNow({ - channelGroups: ['my_channel_group'], - }); - console.log('hereNow response:', response); -} catch (error) { - console.error(`hereNow failed with error: ${error}`); -} -// snippet.end diff --git a/docs-snippets/provision/datasync-classes.json b/docs-snippets/provision/datasync-classes.json deleted file mode 100644 index 9cd93dc1f..000000000 --- a/docs-snippets/provision/datasync-classes.json +++ /dev/null @@ -1,63 +0,0 @@ -{ - "$comment": "Desired DataSync classes on the secured docs keyset. Applied by provision-datasync.ts (dry run by default). Only SubKey-level classes are managed here. The Global User, Channel and Membership classes are seeded by PubNub and cannot be changed. Event publishing per class version has no Admin API endpoint and is switched on in the Admin Portal (DataSync tab, Select entities to sync): User v1, Channel v1, Membership v1, product v1, ProductOwner v1.", - "entityClasses": [ - { - "name": "product", - "version": 1, - "description": "Sneaker product used by the JavaScript SDK docs snippet tests. The public projection exposes name and price, admin exposes everything.", - "config": { "ttlSec": 2678400 }, - "properties": [ - { - "name": "name", - "path": "/payload/name", - "valueKind": "string", - "filtering": "full", - "isNullable": true, - "projections": [{ "name": "__default__" }, { "name": "public" }, { "name": "admin" }] - }, - { - "name": "price", - "path": "/payload/price", - "valueKind": "number", - "filtering": "simple", - "isNullable": false, - "projections": [{ "name": "__default__" }, { "name": "public" }, { "name": "admin" }] - }, - { - "name": "stock", - "path": "/payload/stock", - "valueKind": "number", - "filtering": "simple", - "isNullable": true, - "projections": [{ "name": "__default__" }, { "name": "admin" }] - } - ] - } - ], - "relationshipClasses": [ - { - "name": "ProductOwner", - "version": 1, - "description": "Links a seller (entity A, any class) to a product (entity B) in the JavaScript SDK docs snippet tests.", - "cardinality": "many-to-many", - "entityBClass": "product", - "properties": [ - { - "name": "tier", - "path": "/payload/tier", - "valueKind": "string", - "filtering": "full", - "isNullable": true, - "projections": [{ "name": "__default__" }, { "name": "public" }, { "name": "admin" }] - } - ] - } - ], - "expectedBuiltIns": { - "entityClasses": [ - { "name": "User", "version": 1, "level": "Global" }, - { "name": "Channel", "version": 1, "level": "Global" } - ], - "relationshipClasses": [{ "name": "Membership", "version": 1 }] - } -} diff --git a/docs-snippets/provision/provision-datasync.ts b/docs-snippets/provision/provision-datasync.ts deleted file mode 100644 index 6e603eaf5..000000000 --- a/docs-snippets/provision/provision-datasync.ts +++ /dev/null @@ -1,363 +0,0 @@ -import fs from 'node:fs'; -import path from 'node:path'; -import { fileURLToPath } from 'node:url'; -import { describeRoles, loadEnv, type Env } from '../harness/env.js'; -import { installRedaction, redact } from '../harness/redact.cjs'; - -/** - * Sets up the DataSync classes of the datasync keyset from `datasync-classes.json`. - * - * npm run docs-snippets:provision creates missing classes, updates classes whose mutable fields drifted - * npm run docs-snippets:provision -- --check no-write diff, exit code 1 when anything differs - * - * The live suite calls `provisionDataSync` first in every run that includes the datasync role, before preflight and before any - * step (harness/cli.ts). - * - * Rules: - * - Never deletes a class. A class whose immutable fields differ (relationship cardinality or entity sides) fails with the - * diff, because only a person may recreate it. - * - Idempotent. A second run reports every class as unchanged and writes nothing. - * - Touches only the classes in datasync-classes.json. Classes other tests use on a shared keyset (for example the `JS*` - * classes of test/integration/endpoints/data_sync/helpers.ts) are never read for changes and never written. - * - Without PUBNUB_ADMIN_API_KEY it still checks, through the SDK with the datasync keys, and a missing class fails with its - * name. The DataSync steps are never skipped silently. - * - The per class event switch ("Select entities to sync") has no Admin API endpoint. It stays a feature probe - * (`needs-feature:datasync-events`, step datasyncEvents). - * - Output never contains a key or part of one. - * - * Admin API auth is the raw key in the `Authorization` header (no `Bearer`) plus `PubNub-Version`. - */ - -const HERE = path.dirname(fileURLToPath(import.meta.url)); -const BASE = 'https://admin-api.pubnub.com/v2/datasync/subkeys'; -const API_VERSION = '2026-09-17'; -const ENTITY_TYPE = 'application/vnd.pubnub.objects.entity-class+json;version=1'; -const RELATIONSHIP_TYPE = 'application/vnd.pubnub.objects.relationship-class+json;version=1'; - -type Json = null | boolean | number | string | Json[] | { [key: string]: Json }; -type Obj = { [key: string]: Json }; - -interface Desired { - entityClasses: Obj[]; - relationshipClasses: Obj[]; - expectedBuiltIns: { entityClasses: Obj[]; relationshipClasses: Obj[] }; -} - -interface Action { - kind: 'entity' | 'relationship'; - op: 'create' | 'replace' | 'unchanged' | 'blocked'; - name: string; - version: number; - diff: string[]; - body?: Obj; -} - -export interface ProvisionResult { - ok: boolean; - /** Lines for the console. No key material. */ - log: string[]; - /** What failed the setup, one line each. */ - problems: string[]; - /** Number of POST and PUT calls made. 0 on a second run. */ - writes: number; -} - -async function request( - method: string, - url: string, - apiKey: string, - contentType?: string, - body?: Obj, -): Promise<{ status: number; json: Obj }> { - const headers: Record = { Authorization: apiKey, 'PubNub-Version': API_VERSION }; - if (contentType) headers['Content-Type'] = contentType; - const response = await fetch(url, { method, headers, body: body ? JSON.stringify(body) : undefined }); - const text = await response.text(); - let json: Obj = {}; - try { - json = text ? (JSON.parse(text) as Obj) : {}; - } catch { - // Redact first, then cut: a key cut in half would match neither its literal value nor its shape. - json = { raw: redact(text).slice(0, 200) }; - } - return { status: response.status, json }; -} - -/** Subset comparison: every key in `desired` must match in `actual`. Arrays compare element by element. */ -function differences(actual: Json | undefined, desired: Json, where: string): string[] { - if (Array.isArray(desired)) { - if (!Array.isArray(actual)) return [`${where}: expected an array, found ${JSON.stringify(actual)}`]; - const out: string[] = []; - if (actual.length !== desired.length) - out.push(`${where}: expected ${desired.length} item(s), found ${actual.length}`); - desired.forEach((item, i) => { - const named = typeof item === 'object' && item !== null && 'name' in item; - const label = named ? `${where}[${(item as Obj).name}]` : `${where}[${i}]`; - const match = named ? actual.find((a) => (a as Obj)?.name === (item as Obj).name) : actual[i]; - out.push(...differences(match, item, label)); - }); - return out; - } - if (desired !== null && typeof desired === 'object') { - if (actual === null || typeof actual !== 'object' || Array.isArray(actual)) - return [`${where}: expected an object, found ${JSON.stringify(actual)}`]; - return Object.entries(desired).flatMap(([key, value]) => - differences((actual as Obj)[key], value, where ? `${where}.${key}` : key), - ); - } - return actual === desired ? [] : [`${where}: ${JSON.stringify(actual)} -> ${JSON.stringify(desired)}`]; -} - -function listOf(json: Obj): Obj[] { - return Array.isArray(json.data) ? (json.data as Obj[]) : []; -} - -function entityBody(d: Obj): Obj { - const data: Obj = { config: d.config as Json, properties: d.properties as Json }; - if (d.description) data.description = d.description; - return { data }; -} - -function relationshipCreateBody(d: Obj): Obj { - const data: Obj = { cardinality: d.cardinality as Json }; - for (const key of [ - 'description', - 'entityAClass', - 'entityAClassLevel', - 'entityBClass', - 'entityBClassLevel', - 'properties', - ]) { - if (d[key] !== undefined) data[key] = d[key] as Json; - } - return { data }; -} - -function relationshipReplaceBody(d: Obj): Obj { - const data: Obj = {}; - if (d.description) data.description = d.description; - if (d.properties) data.properties = d.properties; - return { data }; -} - -function desiredClasses(): Desired { - return JSON.parse(fs.readFileSync(path.join(HERE, 'datasync-classes.json'), 'utf8')) as Desired; -} - -/** Without an admin key: check through the SDK that each class exists. Never writes. */ -async function checkWithSdk(env: Env, desired: Desired): Promise { - const log: string[] = [ - 'PUBNUB_ADMIN_API_KEY is not set: checking the classes through the SDK with the datasync keys (no writes)', - ]; - const problems: string[] = []; - const keys = env.datasync; - if (!keys) - return { - ok: false, - log, - problems: ['the datasync keyset is not configured (DS_PUBLISH_KEY, DS_SUBSCRIBE_KEY, DS_SECRET_KEY)'], - writes: 0, - }; - const { default: PubNub } = await import('pubnub'); - const client = new PubNub({ - ...keys, - userId: 'docs-snippets-provision-check', - logLevel: PubNub.LogLevel.None, - heartbeatInterval: 0, - }); - try { - for (const d of desired.entityClasses) { - try { - await client.dataSync.getEntities({ class: String(d.name), classVersion: Number(d.version), limit: 1 }); - log.push(` = entity class ${d.name} v${d.version} exists`); - } catch (error) { - problems.push( - `DataSync entity class ${d.name} v${d.version} is missing on the datasync keyset (${(error as { status?: { statusCode?: number } }).status?.statusCode ?? (error as Error).message}). Set PUBNUB_ADMIN_API_KEY and run npm run docs-snippets:provision.`, - ); - } - } - for (const d of desired.relationshipClasses) { - try { - await client.dataSync.getRelationships({ class: String(d.name), classVersion: Number(d.version), limit: 1 }); - log.push(` = relationship class ${d.name} v${d.version} exists`); - } catch (error) { - problems.push( - `DataSync relationship class ${d.name} v${d.version} is missing on the datasync keyset (${(error as { status?: { statusCode?: number } }).status?.statusCode ?? (error as Error).message}). Set PUBNUB_ADMIN_API_KEY and run npm run docs-snippets:provision.`, - ); - } - } - } finally { - client.destroy(); - } - return { ok: problems.length === 0, log, problems, writes: 0 }; -} - -/** - * Reads the live classes, compares them with datasync-classes.json and, unless `check`, creates missing classes and updates - * drifted mutable fields. See the header for the rules. - */ -export async function provisionDataSync(env: Env, options: { check: boolean }): Promise { - const desired = desiredClasses(); - if (!env.adminKey) return checkWithSdk(env, desired); - const log: string[] = []; - const problems: string[] = []; - const subKey = env.datasync?.subscribeKey; - if (!subKey) - return { ok: false, log, problems: ['the datasync keyset is not configured (DS_SUBSCRIBE_KEY)'], writes: 0 }; - const apiKey = env.adminKey; - const entities = await request('GET', `${BASE}/${subKey}/entity-classes`, apiKey); - const relationships = await request('GET', `${BASE}/${subKey}/relationship-classes`, apiKey); - if (entities.status !== 200 || relationships.status !== 200) { - const hint = - entities.status === 403 || relationships.status === 403 - ? ' The Service Integration needs a DataSync permission row (Keyset level, Read & write) for the datasync keyset.' - : ''; - return { - ok: false, - log, - problems: [ - `could not read the live classes: entity-classes HTTP ${entities.status}, relationship-classes HTTP ${relationships.status}.${hint}`, - ], - writes: 0, - }; - } - const liveEntities = listOf(entities.json); - const liveRelationships = listOf(relationships.json); - log.push( - `datasync keyset: ${liveEntities.length} entity class(es), ${liveRelationships.length} relationship class(es) live`, - ); - - for (const builtIn of desired.expectedBuiltIns.entityClasses) { - if (!liveEntities.some((c) => c.name === builtIn.name && c.version === builtIn.version)) - problems.push( - `built-in entity class ${builtIn.name} v${builtIn.version} is missing, so DataSync is off on the datasync keyset`, - ); - } - for (const builtIn of desired.expectedBuiltIns.relationshipClasses) { - if (!liveRelationships.some((c) => c.name === builtIn.name && c.version === builtIn.version)) - problems.push( - `built-in relationship class ${builtIn.name} v${builtIn.version} is missing, so DataSync is off on the datasync keyset`, - ); - } - - const actions: Action[] = []; - for (const d of desired.entityClasses) { - const live = liveEntities.find((c) => c.name === d.name && c.version === d.version && c.level === 'SubKey'); - const { name, version, ...comparable } = d; - void name; - void version; - if (!live) - actions.push({ - kind: 'entity', - op: 'create', - name: String(d.name), - version: Number(d.version), - diff: ['(new class)'], - body: entityBody(d), - }); - else { - const diff = differences(live as Json, comparable as Json, ''); - actions.push({ - kind: 'entity', - op: diff.length ? 'replace' : 'unchanged', - name: String(d.name), - version: Number(d.version), - diff, - body: entityBody(d), - }); - } - } - for (const d of desired.relationshipClasses) { - const live = liveRelationships.find((c) => c.name === d.name && c.version === d.version); - const { name, version, ...comparable } = d; - void name; - void version; - if (!live) - actions.push({ - kind: 'relationship', - op: 'create', - name: String(d.name), - version: Number(d.version), - diff: ['(new class)'], - body: relationshipCreateBody(d), - }); - else { - const diff = differences(live as Json, comparable as Json, ''); - const immutable = diff.filter((line) => /^(cardinality|entityA|entityB)/.test(line)); - if (immutable.length) - actions.push({ - kind: 'relationship', - op: 'blocked', - name: String(d.name), - version: Number(d.version), - diff: immutable, - }); - else - actions.push({ - kind: 'relationship', - op: diff.length ? 'replace' : 'unchanged', - name: String(d.name), - version: Number(d.version), - diff, - body: relationshipReplaceBody(d), - }); - } - } - - for (const a of actions) { - const symbol = { create: '+', replace: '~', unchanged: '=', blocked: '!' }[a.op]; - log.push(` ${symbol} ${a.kind} class ${a.name} v${a.version} [${a.op}]`); - if (a.op !== 'unchanged') for (const line of a.diff) log.push(` ${line}`); - } - for (const a of actions.filter((x) => x.op === 'blocked')) { - problems.push( - `${a.kind} class ${a.name} v${a.version} differs in an immutable field (${a.diff.join('; ')}). Only a person may delete and recreate it.`, - ); - } - - const pending = actions.filter((a) => a.op === 'create' || a.op === 'replace'); - let writes = 0; - if (options.check) { - log.push( - pending.length - ? `check: ${pending.length} change(s) would be written, nothing was written` - : 'check: every class is unchanged', - ); - for (const a of pending) - problems.push(`${a.kind} class ${a.name} v${a.version} would be ${a.op === 'create' ? 'created' : 'updated'}`); - } else { - for (const a of pending) { - const dir = a.kind === 'entity' ? 'entity-classes' : 'relationship-classes'; - const url = `${BASE}/${subKey}/${dir}/${encodeURIComponent(a.name)}/versions/${a.version}`; - const type = a.kind === 'entity' ? ENTITY_TYPE : RELATIONSHIP_TYPE; - const result = await request(a.op === 'create' ? 'POST' : 'PUT', url, apiKey, type, a.body); - writes += 1; - const ok = result.status === 200 || result.status === 201; - log.push(` ${a.op === 'create' ? 'POST' : 'PUT'} ${a.kind} ${a.name} v${a.version}: HTTP ${result.status}`); - if (!ok) - problems.push( - `${a.op} of ${a.kind} class ${a.name} v${a.version} failed with HTTP ${result.status}: ${redact(JSON.stringify(result.json)).slice(0, 300)}`, - ); - } - log.push(`${writes} write(s), ${actions.filter((a) => a.op === 'unchanged').length} class(es) unchanged`); - } - return { ok: problems.length === 0, log, problems, writes }; -} - -const invokedDirectly = process.argv[1] && fileURLToPath(import.meta.url) === path.resolve(process.argv[1]); -if (invokedDirectly) { - installRedaction(); - const env = loadEnv(); - console.log(describeRoles(env)); - provisionDataSync(env, { check: process.argv.includes('--check') }).then( - (result) => { - result.log.forEach((line) => console.log(line)); - result.problems.forEach((p) => console.error(`problem: ${p}`)); - process.exitCode = result.ok ? 0 : 1; - }, - (error) => { - console.error(error instanceof Error ? error.message : error); - process.exitCode = 1; - }, - ); -} diff --git a/docs-snippets/publish-subscribe.ts b/docs-snippets/publish-subscribe.ts deleted file mode 100644 index 8a3893800..000000000 --- a/docs-snippets/publish-subscribe.ts +++ /dev/null @@ -1,263 +0,0 @@ -import PubNub, { PubNubError } from '../lib/types'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'userId', -}); - -// snippet.publishJsonSerialisedMessage -const newMessage = { - text: 'Hi There!', -}; - -try { - const response = await pubnub.publish({ - message: newMessage, - channel: 'my_channel', - customMessageType: 'text-message', - }); - - console.log('message published with server response:', response); -} catch (error) { - console.error( - `Publish Failed: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.publishStoreThePublishedMessagefor10Hours -try { - const response = await pubnub.publish({ - message: 'hello!', - channel: 'my_channel', - storeInHistory: true, - ttl: 10, - customMessageType: 'text-message', - }); - - console.log('message published with server response:', response); -} catch (error) { - console.error( - `Publish Failed: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.publishSuccessfull -const response = await pubnub.publish({ - message: 'hello world!', - channel: 'ch1', -}); - -console.log(response); // {timetoken: "14920301569575101"} -// end.snippet - -// snippet.publishUnsuccessfulByNetworkDown -try { - const response = await pubnub.publish({ - message: 'hello world!', - channel: 'ch1', - }); - console.log('message published with server response:', response); -} catch (error) { - console.error( - `Publish Failed: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.publishUnsuccessfulWithoutPublishKey -try { - const result = await pubnub.publish({ - message: 'hello world!', - channel: 'ch1', - }); - console.log('message published with server response:', response); -} catch (error) { - console.error( - `Publish Failed: ${error}.${ - (error as PubNubError).status ? ` Additional information: ${(error as PubNubError).status}` : '' - }`, - ); -} -// snippet.end - -// snippet.createSubscription -const channel = pubnub.channel('my_channel'); - -const subscriptionOptions = { receivePresenceEvents: true }; -channel.subscription(subscriptionOptions); -// snippet.end - -const subscription = pubnub.channel('channel_1').subscription(); -const subscriptionSet = pubnub.subscriptionSet({ channels: ['ch1', 'ch2'] }); -// snippet.ubsubscribe -// `subscription` is an active subscription object -subscription.unsubscribe(); - -// `subscriptionSet` is an active subscription set object -subscriptionSet.unsubscribe(); -// snippet.end - -// snippet.ubsubscribeAll -pubnub.unsubscribeAll(); -// snippet.end - -// *************** OLD SUBSCRIBE SYNTAX *************** -// snippet.OLDsubscribeMultipleChannels -pubnub.subscribe({ - channels: ['my_channel_1', 'my_channel_2', 'my_channel_3'], -}); -// snippet.end - -// snippet.OLDsubscribeWithPresence -pubnub.subscribe({ - channels: ['my_channel'], - withPresence: true, -}); -// snippet.end - -// snippet.OLDsubscribeWithWildCardChannels -pubnub.subscribe({ - channels: ['ab.*'], -}); -// snippet.end - -// snippet.OLDsubscribeWithState -pubnub.addListener({ - status: async (statusEvent) => { - if (statusEvent.category === 'PNConnectedCategory') { - try { - await pubnub.setState({ - state: { - some: 'state', - }, - }); - } catch (error) { - // handle setState error - } - } - }, - message: (messageEvent) => { - // handle message - }, - presence: (presenceEvent) => { - // handle presence - }, -}); - -pubnub.subscribe({ - channels: ['ch1', 'ch2', 'ch3'], -}); -// snippet.end - -// snippet.OLDsubscribeChannelGroup -pubnub.subscribe({ - channelGroups: ['my_channelGroup'], -}); -// snippet.end - -// snippet.OLDsubscribeChannelGroupWithPresence -pubnub.subscribe({ - channelGroups: ['family'], - withPresence: true, -}); -// snippet.end - -// snippet.OLDsubscribeMultipleChannelGroup -pubnub.subscribe({ - channelGroups: ['my_channelGroup1', 'my_channelGroup2', 'my_channelGroup3'], -}); -// snippet.end - -// snippet.OLDsubscribeChannelGroupAndChannels -pubnub.subscribe({ - channels: ['my_channel'], - channelGroups: ['my_channelGroup'], -}); -// snippet.end - -// snippet.OLDUnsubscribeMultipleChannels -pubnub.unsubscribe({ - channels: ['chan1', 'chan2', 'chan3'], -}); -// snippet.end - -// snippet.OLDUnsubscribeMultipleChannelGroup -pubnub.unsubscribe({ - channelGroups: ['demo_group1', 'demo_group2'], -}); -// snippet.end - -{ - // snippet.subscriptionSetFrom2IndividualSubscriptions - // create a subscription from a channel entity - const channel = pubnub.channel('channel_1'); - const subscription1 = channel.subscription({ receivePresenceEvents: true }); - - // create a subscription from a channel group entity - const channelGroup = pubnub.channelGroup('channelGroup_1'); - const subscription2 = channelGroup.subscription(); - - // add 2 subscriptions to create a subscription set - const subscriptionSet = subscription1.addSubscription(subscription2); - - // add another subscription to the set - const subscription3 = pubnub.channel('channel_3').subscription({ receivePresenceEvents: false }); - subscriptionSet.addSubscription(subscription3); - - // remove a subscription from a subscription set - subscriptionSet.removeSubscription(subscription3); - - subscriptionSet.subscribe(); - // snippet.end -} -{ - // snippet.SubscriptionSetFrom2Sets - // create a subscription set with multiple channels - const subscriptionSet1 = pubnub.subscriptionSet({ channels: ['ch1', 'ch2'] }); - - // create another subscription set with multiple channels and options - const subscriptionSet2 = pubnub.subscriptionSet({ - channels: ['ch3', 'ch4'], - subscriptionOptions: { receivePresenceEvents: true }, - }); - - // add a subscription set to another subscription set - subscriptionSet1.addSubscriptionSet(subscriptionSet2); - - // remove a subscription set from another subscription set - subscriptionSet1.removeSubscriptionSet(subscriptionSet2); - // snippet.end -} -// snippet.AddToExistingSubscriptionSet -// create a subscription set with multiple channels -const subscriptionSet1 = pubnub.subscriptionSet({ channels: ['ch1', 'ch2'] }); - -// subscribe to the set -// start receiving events from ch1 and ch2 -subscriptionSet1.subscribe(); - -// create another subscription set with multiple channels -const subscriptionSet2 = pubnub.subscriptionSet({ channels: ['ch3', 'ch4'] }); - -// add the new set to the initial set -subscriptionSet1.addSubscriptionSet(subscriptionSet2); - -// you're now receiving events from ch1, ch2, ch3, and ch4 -// because the set has been subscribed to previously - -// create and add another subscription to the set -const channelGroup = pubnub.channelGroup('channelGroup_1'); -const subscription2 = channelGroup.subscription(); -subscriptionSet1.addSubscription(subscription2); - -// you're now receiving events from ch1, ch2, ch3, and ch4 and channelGroup_1 -// snippet.end diff --git a/docs-snippets/run.ts b/docs-snippets/run.ts deleted file mode 100644 index 8efed9332..000000000 --- a/docs-snippets/run.ts +++ /dev/null @@ -1,107 +0,0 @@ -/** - * The plan: the one place that lists every page module of the suite. - * - * Role. `harness/mocha-entry.ts` imports this file from the staged copy (`docs-snippets/.run//run.ts`), calls - * `registerAll(world)` and turns each registered step into one Mocha test. A page module is a file that default - * exports `page(path, phase, setup)` (see `harness/step.ts`). It only registers steps when `setup(world)` runs, so - * importing the modules here has no side effects. - * - * Order. Steps sort by phase first (P0 to P9), then by the position of their page module in `plan` below, then by - * registration order inside the page. So this list decides the order inside one phase only. The phases are: P0 boot, P1 secure (Access Manager), P2 catalog (DataSync), P3 rooms (App Context, channel - * groups), P4 live (subscribe, publish, presence, files), P5 history, P6 push, P7 network (proxy, outage), P8 legacy, - * P9 teardown. Each page module sets its own default phase and a step may override it. - * - * Adding a page. Create `api-reference/.ts` (same base name as the docs page), import it here and add it to - * `plan`. `npm run test:docs-snippets:dry` then checks that every snippet the lint finds is claimed by exactly one step. - */ -import { registry } from './harness/registry.js'; -import type { PageModule } from './harness/step.js'; -import type { World } from './harness/world.js'; -import preflight from './harness/preflight.js'; -import preflightSummary from './harness/preflight-summary.js'; -import shared from './shared/import-pubnub.js'; -import configuration from './api-reference/configuration.js'; -import channelGroups from './api-reference/channel-groups.js'; -import publish from './api-reference/publish.js'; -import subscribe from './api-reference/subscribe.js'; -import presence from './api-reference/presence.js'; -import messageActions from './api-reference/message-actions.js'; -import messagePersistence from './api-reference/message-persistence.js'; -import misc from './api-reference/misc.js'; -import accessManager from './api-reference/access-manager.js'; -import accessManagerV2 from './api-reference/access-manager-v2.js'; -import accessManagerLegacy from './api-reference/access-manager-legacy.js'; -import encryption from './api-reference/encryption.js'; -import encryptionLegacy from './api-reference/encryption-legacy.js'; -import objects from './api-reference/objects.js'; -import datasync from './api-reference/datasync.js'; -import fileSharing from './api-reference/file-sharing.js'; -import fileSharingLegacy from './api-reference/file-sharing-legacy.js'; -import mobilePush from './api-reference/mobile-push.js'; -import subscribeLegacy from './api-reference/subscribe-legacy.js'; -import publishLegacy from './api-reference/publish-legacy.js'; -import messagePersistenceLegacy from './api-reference/message-persistence-legacy.js'; -import configurationLegacy from './api-reference/configuration-legacy.js'; -import quickstartNode from './getting-started/quickstart-node.js'; -import setUpYourEnvironment from './getting-started/set-up-your-environment.js'; -import logging from './getting-started/logging.js'; -import loggingFileLogger from './getting-started/logging-file-logger.js'; -import statusEvents from './getting-started/status-events.js'; -import troubleshooting from './getting-started/troubleshooting.js'; -import migrationV8 from './migration-guides/v8.js'; -import migrationV11 from './migration-guides/v11.js'; -import migrationV12 from './migration-guides/v12.js'; - -/** - * Page modules. Steps carry their own phase, so the order here only decides registration order inside a - * phase. The runner sorts by phase (P0 to P9), then by this order. - * - * Dependencies that must hold are written as `needs` on the step, which the registry validates (a need that runs - * later is a wiring error), so the order here is only a tie-breaker inside a phase. - */ -export const plan: PageModule[] = [ - preflight, - shared, - configuration, - channelGroups, - publish, - subscribe, - presence, - messageActions, - messagePersistence, - misc, - mobilePush, - fileSharingLegacy, - fileSharing, - objects, - datasync, - accessManager, - accessManagerV2, - accessManagerLegacy, - encryption, - encryptionLegacy, - // Legacy pages (P8). - subscribeLegacy, - publishLegacy, - messagePersistenceLegacy, - configurationLegacy, - // Getting started, logging, status events, troubleshooting, migration guides. - quickstartNode, - setUpYourEnvironment, - logging, - loggingFileLogger, - statusEvents, - troubleshooting, - migrationV8, - migrationV11, - migrationV12, - // Last, so its P0 checks run after every other P0 preflight and can name every missing feature at once. - preflightSummary, -]; - -/** Runs every page module's `setup`, which registers its steps in the global registry. Call `registry.reset()` first. */ -export function registerAll(world: World): void { - for (const module of plan) module.setup(world); -} - -export { registry }; diff --git a/docs-snippets/shared/import-pubnub.ts b/docs-snippets/shared/import-pubnub.ts deleted file mode 100644 index a0a181ff6..000000000 --- a/docs-snippets/shared/import-pubnub.ts +++ /dev/null @@ -1,32 +0,0 @@ -// snippet.importPubNub -import PubNub from 'pubnub'; -// snippet.end -// snippet.importFS -import fs from 'fs'; -// snippet.end -import assert from '../harness/assert.js'; -import { page, step } from '../harness/step.js'; - -/** - * Shared examples used next to the examples of many pages: the SDK import line, the `fs` import for Node.js file - * sharing and the generic client constructor. - */ -export default page('shared/import-pubnub', 'P0', (world) => { - step('PubNubinitBasicUsage', { proves: 'The generic init sample builds a working client and time() succeeds.' }, async () => { - // snippet.PubNubinitBasicUsage - // Initialize PubNub with your keys - const pubnub = new PubNub({ - publishKey: 'YOUR_PUBLISH_KEY', - subscribeKey: 'YOUR_SUBSCRIBE_KEY', - userId: 'YOUR_USER_ID', - // snippet.hide - ...world.live.open, - // snippet.show - }); - // snippet.end - - world.track(pubnub); - assert.equal(pubnub.getUserId(), world.n('YOUR_USER_ID')); - world.check.serverTime((await pubnub.time()).timetoken); - }); -}); diff --git a/docs-snippets/tsconfig.json b/docs-snippets/tsconfig.json deleted file mode 100644 index 0395b690b..000000000 --- a/docs-snippets/tsconfig.json +++ /dev/null @@ -1,43 +0,0 @@ -{ - "extends": "../tsconfig.json", - "compilerOptions": { - "module": "esnext", - "target": "es2017", - "outDir": "./docs", - "rootDir": "..", - "noEmit": true, - "skipLibCheck": true, - "resolveJsonModule": true, - "esModuleInterop": true, - "allowSyntheticDefaultImports": true, - "moduleResolution": "node", - "baseUrl": "..", - "paths": { - "pubnub": ["lib/node/index.js"] - } - }, - "include": [ - "**/*.ts" - ], - "exclude": [ - "node_modules", - ".run", - "report", - "api-reference", - "getting-started", - "migration-guides", - "shared", - "quickstarts", - "harness", - "fixtures", - "provision", - "run.ts", - ".mocharc.cjs", - "coverage.json", - "package.json", - "tsconfig.live.json", - "tsconfig.run.json", - "tsconfig.browser.json", - "README.md" - ] - } \ No newline at end of file diff --git a/docs-snippets/tsconfig.live.json b/docs-snippets/tsconfig.live.json deleted file mode 100644 index 6f982f27a..000000000 --- a/docs-snippets/tsconfig.live.json +++ /dev/null @@ -1,33 +0,0 @@ -{ - "//": "Live docs snippet suite. Used by tsx at run time (TSX_TSCONFIG_PATH) and by tsc (npm run test:docs-snippets:types). The first pubnub candidate is the built SDK that tsx loads, tsc skips it (no allowJs) and takes the shipped types.", - "compilerOptions": { - "target": "ES2022", - "module": "ESNext", - "moduleResolution": "Bundler", - "lib": ["ES2022"], - "types": ["node", "mocha"], - "strict": true, - "noEmit": true, - "skipLibCheck": true, - "esModuleInterop": true, - "allowSyntheticDefaultImports": true, - "resolveJsonModule": true, - "forceConsistentCasingInFileNames": true, - "allowJs": false, - "baseUrl": ".", - "paths": { - "pubnub": ["../lib/types/index.d.ts", "../lib/node/index.js"] - } - }, - "include": [ - "run.ts", - "harness/**/*.ts", - "fixtures/**/*.ts", - "provision/**/*.ts", - "shared/**/*.ts", - "api-reference/**/*.ts", - "getting-started/**/*.ts", - "migration-guides/**/*.ts" - ], - "exclude": ["node_modules", ".run", "report"] -} diff --git a/docs-snippets/tsconfig.run.json b/docs-snippets/tsconfig.run.json deleted file mode 100644 index 85fe23354..000000000 --- a/docs-snippets/tsconfig.run.json +++ /dev/null @@ -1,9 +0,0 @@ -{ - "//": "Run time only (tsx reads it through TSX_TSCONFIG_PATH). Same options as tsconfig.live.json, but pubnub resolves to the built SDK of this repository, lib/node/index.js. lib/node has no .d.ts, so type checking uses tsconfig.live.json, which maps pubnub to the shipped types.", - "extends": "./tsconfig.live.json", - "compilerOptions": { - "paths": { - "pubnub": ["../lib/node/index.js"] - } - } -} diff --git a/docs-snippets/use-cases/automated-polling.ts b/docs-snippets/use-cases/automated-polling.ts deleted file mode 100644 index c188efb59..000000000 --- a/docs-snippets/use-cases/automated-polling.ts +++ /dev/null @@ -1,102 +0,0 @@ -import PubNub from '../../lib/types'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'poll-service', -}); - -// snippet.automatedPollingPublishTriggeredPoll -const pollsByReaction = new Map([ - [ - '\u{1F621}', - { - title: 'Which team is playing dirtiest?', - options: [ - { id: 1, text: 'Home team' }, - { id: 2, text: 'Away team' }, - ], - }, - ], - [ - '\u{1F389}', - { - title: 'Whose fans are celebrating hardest?', - options: [ - { id: 1, text: 'Home team' }, - { id: 2, text: 'Away team' }, - ], - }, - ], -]); - -async function openPollForReaction(reaction = '') { - const template = pollsByReaction.get(reaction); - - if (!template) { - console.log('no poll is defined for', reaction); - return; - } - - try { - const response = await pubnub.publish({ - channel: 'game.new-poll', - message: { - id: `poll-${Date.now()}`, - title: template.title, - durationSeconds: 30, - options: template.options, - }, - customMessageType: 'poll-opened', - storeInHistory: true, - }); - console.log('triggered poll published at timetoken:', response.timetoken); - } catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error( - `Publishing the triggered poll failed: ${error}${status ? ` Additional information: ${status}` : ''}`, - ); - } -} -// snippet.end - -// snippet.automatedPollingThrottleTriggers -const minimumMillisecondsBetweenPolls = 60000; -let lastPollOpenedAt = 0; - -function shouldOpenPoll() { - const now = Date.now(); - - if (now - lastPollOpenedAt < minimumMillisecondsBetweenPolls) { - console.log('that trigger arrived inside the cooldown window, so no new poll opened'); - return false; - } - - lastPollOpenedAt = now; - return true; -} -// snippet.end - -// snippet.automatedPollingReceiveTrigger -const triggerSubscription = pubnub.channel('game.poll-triggers').subscription({ receivePresenceEvents: false }); - -triggerSubscription.onMessage = (event) => { - const trigger = event.message; - const rawReaction = - typeof trigger === 'object' && trigger !== null && !Array.isArray(trigger) && 'reaction' in trigger - ? trigger.reaction - : undefined; - - if (typeof rawReaction !== 'string') return; - - const reaction = rawReaction; - - console.log('open a poll because fans keep tapping', reaction); - - if (shouldOpenPoll()) { - void openPollForReaction(reaction); - } -}; - -triggerSubscription.subscribe(); -// snippet.end diff --git a/docs-snippets/use-cases/fan-behavior-management.ts b/docs-snippets/use-cases/fan-behavior-management.ts deleted file mode 100644 index f9656885e..000000000 --- a/docs-snippets/use-cases/fan-behavior-management.ts +++ /dev/null @@ -1,264 +0,0 @@ -import PubNub from '../../lib/types'; - -// Restricting what a fan may do is a server-side operation, so this client is -// configured with the keyset's secret key and runs on your own infrastructure. -const server = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - secretKey: 'demo', - userId: 'moderation-service', -}); - -// The fan's own client never holds the secret key. It only applies tokens. -const fanClient = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'fan-42', -}); - -// snippet.fanBehaviorStateStore -import fs from 'node:fs'; - -// A real token service tracks issued tokens and each fan's status in a database. -// This tutorial persists the same information in a JSON file next to the script, -// so `grant`, `mute`, `unmute`, and `ban` share state across separate `node server.js` -// invocations without needing a database just to run the tutorial. -const stateFilePath = new URL('./fan-state.json', import.meta.url); - -function loadState() { - try { - return JSON.parse(fs.readFileSync(stateFilePath, 'utf8')); - } catch { - return {}; - } -} - -function saveState(state = {}) { - fs.writeFileSync(stateFilePath, JSON.stringify(state, null, 2)); -} -// snippet.end - -// snippet.fanBehaviorGrantChatAccess -async function grantChatAccess(userId = '') { - const state = loadState(); - const entry = state[userId] ?? { status: 'active', tokens: [] }; - - if (entry.status === 'banned') { - console.log(`${userId} is banned, so no token was issued`); - return; - } - - const canWrite = entry.status !== 'muted'; - - try { - const token = await server.grantToken({ - ttl: 60, - authorizedUserId: userId, - resources: { - channels: { - 'game.chat': { read: true, write: canWrite }, - }, - }, - }); - - entry.status = entry.status ?? 'active'; - entry.tokens = [...entry.tokens, { token, write: canWrite }]; - state[userId] = entry; - saveState(state); - - console.log(`token that allows ${canWrite ? 'reading and writing' : 'reading'} chat:`, token); - } catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Granting chat access failed: ${error}${status ? ` Additional information: ${status}` : ''}`); - } -} -// snippet.end - -// snippet.fanBehaviorMuteFan -async function muteFan(userId = '') { - const state = loadState(); - const entry = state[userId] ?? { status: 'active', tokens: [] }; - - if (entry.status === 'banned') { - console.log(`${userId} is already banned, so there is nothing left to mute`); - return; - } - - const writableTokens = entry.tokens.filter((issued = { token: '', write: false }) => issued.write); - const stillValid = []; - - for (const issued of writableTokens) { - try { - await server.revokeToken(issued.token); - console.log('revoked an outstanding writable token'); - } catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Revoking a writable token failed: ${error}${status ? ` Additional information: ${status}` : ''}`); - // Keep tracking a token you couldn't revoke, so the next mute or ban retries it. - stillValid.push(issued); - } - } - - // Record the mute before issuing anything new, so a later grant can't hand out write access. - entry.status = 'muted'; - entry.tokens = [...entry.tokens.filter((issued = { token: '', write: false }) => !issued.write), ...stillValid]; - state[userId] = entry; - saveState(state); - - if (stillValid.length > 0) { - console.error(`${stillValid.length} writable token(s) are still valid. Run mute again to retry revoking them.`); - } - - try { - const token = await server.grantToken({ - ttl: 15, - authorizedUserId: userId, - resources: { - channels: { - 'game.chat': { read: true }, - }, - }, - }); - - entry.tokens = [...entry.tokens, { token, write: false }]; - state[userId] = entry; - saveState(state); - - console.log('token that allows reading chat but not writing to it:', token); - } catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Muting the fan failed: ${error}${status ? ` Additional information: ${status}` : ''}`); - } -} -// snippet.end - -// snippet.fanBehaviorUnmuteFan -async function unmuteFan(userId = '') { - const state = loadState(); - const entry = state[userId] ?? { status: 'active', tokens: [] }; - - // This is the one command that clears a ban as well as a mute. Calling it is always - // a deliberate decision by whoever operates server.js, never a side effect of anything - // else in this tutorial. - try { - const token = await server.grantToken({ - ttl: 60, - authorizedUserId: userId, - resources: { - channels: { - 'game.chat': { read: true, write: true }, - }, - }, - }); - - entry.status = 'active'; - entry.tokens = [...entry.tokens, { token, write: true }]; - state[userId] = entry; - saveState(state); - - console.log('token that allows reading and writing chat again:', token); - } catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Unmuting the fan failed: ${error}${status ? ` Additional information: ${status}` : ''}`); - } -} -// snippet.end - -// snippet.fanBehaviorBanFan -async function banFan(userId = '') { - const state = loadState(); - const entry = state[userId] ?? { status: 'active', tokens: [] }; - - const stillValid = []; - - for (const issued of entry.tokens) { - try { - await server.revokeToken(issued.token); - console.log('revoked an outstanding token'); - } catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Revoking a token failed: ${error}${status ? ` Additional information: ${status}` : ''}`); - // Keep tracking a token you couldn't revoke, so the next ban retries it. - stillValid.push(issued); - } - } - - entry.status = 'banned'; - entry.tokens = stillValid; - state[userId] = entry; - saveState(state); - - if (stillValid.length > 0) { - console.error( - `${userId} is banned, but ${stillValid.length} token(s) are still valid. Run ban again to retry revoking them.`, - ); - } else { - console.log(`${userId} is banned. Every outstanding token, read-only and writable, is now revoked.`); - } -} -// snippet.end - -// snippet.fanBehaviorServerDispatch -const [, , command, userIdArgument] = process.argv; -const targetUserId = userIdArgument ?? 'fan-42'; - -if (command === 'grant') { - await grantChatAccess(targetUserId); -} else if (command === 'mute') { - await muteFan(targetUserId); -} else if (command === 'unmute') { - await unmuteFan(targetUserId); -} else if (command === 'ban') { - await banFan(targetUserId); -} else { - console.log('usage: node server.js [userId]'); -} -// snippet.end - -// snippet.fanBehaviorHandleAccessDenied -fanClient.addListener({ - status: (event) => { - if (event.category === 'PNConnectedCategory') { - console.log('connected to game.chat'); - } else if (event.category === 'PNAccessDeniedCategory') { - console.log('this fan may no longer write to', event.affectedChannels); - } - }, -}); -// snippet.end - -// snippet.fanBehaviorApplyToken -const suppliedToken = process.argv[2]; - -if (!suppliedToken) { - console.error('usage: node fan.js '); - process.exit(1); -} - -fanClient.setToken(suppliedToken); -// snippet.end - -// snippet.fanBehaviorSubscribeAndPublish -const subscription = fanClient.channel('game.chat').subscription(); -subscription.subscribe(); - -try { - const response = await fanClient.publish({ - channel: 'game.chat', - message: { text: 'Come on!' }, - }); - console.log('chat message published at timetoken:', response.timetoken); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Publishing to game.chat failed: ${error}${status ? ` Additional information: ${status}` : ''}`); -} -// snippet.end - -// snippet.fanBehaviorInspectToken -const parsed = fanClient.parseToken('replace-with-the-token-to-inspect'); - -if (parsed) { - console.log('token expires in', parsed.ttl, 'minutes'); - console.log('channel permissions:', parsed.resources?.channels); -} -// snippet.end diff --git a/docs-snippets/use-cases/fan-re-engagement.ts b/docs-snippets/use-cases/fan-re-engagement.ts deleted file mode 100644 index 919f68bb5..000000000 --- a/docs-snippets/use-cases/fan-re-engagement.ts +++ /dev/null @@ -1,121 +0,0 @@ -import PubNub from '../../lib/types'; - -// Presence monitoring and the decision to alert a fan run under their own service -// identity, separate from any fan's own client, so this script's watcher is never -// the same connection whose absence it is trying to detect. -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'match-service', -}); - -// snippet.reEngagementCountFansWatching -try { - const response = await pubnub.hereNow({ - channels: ['game.stream'], - includeUUIDs: false, - }); - console.log('fans currently watching the stream:', response.totalOccupancy); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Counting the fans watching failed: ${error}${status ? ` Additional information: ${status}` : ''}`); -} -// snippet.end - -// snippet.reEngagementPublishMomentAlert -async function sendMomentAlert(userId = '') { - const alertChannel = `game.moment-alerts.${userId}`; - const moment = PubNub.notificationPayload('Injury time', 'Two minutes left, and it is still 2-2.'); - - moment.sound = 'default'; - moment.apns.configurations = [{ targets: [{ topic: 'com.example.matchday' }] }]; - - try { - const response = await pubnub.publish({ - channel: alertChannel, - message: { - ...moment.buildPayload(['apns2', 'fcm']), - moment: 'injury-time', - }, - customMessageType: 'moment-alert', - }); - console.log(`moment alert published to ${alertChannel} at timetoken:`, response.timetoken); - } catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Publishing the moment alert failed: ${error}${status ? ` Additional information: ${status}` : ''}`); - } -} -// snippet.end - -// snippet.reEngagementCheckOneFan -async function notifyIfAbsent(userId = '') { - try { - const response = await pubnub.whereNow({ uuid: userId }); - - if (response.channels.includes('game.stream')) { - console.log(`${userId} is still subscribed to game.stream, so no alert is needed`); - return; - } - } catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error( - `Checking where ${userId} is subscribed failed: ${error}${status ? ` Additional information: ${status}` : ''}`, - ); - return; - } - - console.log(`${userId} is not subscribed to game.stream, so sending a moment alert`); - await sendMomentAlert(userId); -} -// snippet.end - -// snippet.reEngagementWatchFansLeave -const streamSubscription = pubnub.channel('game.stream').subscription({ receivePresenceEvents: true }); - -streamSubscription.onPresence = (event) => { - if (event.action === 'leave' || event.action === 'timeout') { - console.log(`${event.uuid} stopped watching, and ${event.occupancy} fans remain`); - void notifyIfAbsent(event.uuid); - } -}; - -streamSubscription.subscribe(); -// snippet.end - -// snippet.reEngagementFanViewer -const viewerUserId = process.argv[2] ?? 'fan-a'; - -const viewerClient = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: viewerUserId, -}); - -const watchSubscription = viewerClient.channel('game.stream').subscription({ receivePresenceEvents: false }); -watchSubscription.subscribe(); - -try { - const response = await viewerClient.push.addChannels({ - channels: [`game.moment-alerts.${viewerUserId}`], - device: 'replace-with-the-fcm-registration-token', - pushGateway: 'fcm', - }); - console.log(`${viewerUserId} registered its device for game.moment-alerts.${viewerUserId}:`, response); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Registering the device failed: ${error}${status ? ` Additional information: ${status}` : ''}`); -} - -const alertSubscription = viewerClient.channel(`game.moment-alerts.${viewerUserId}`).subscription(); - -alertSubscription.onMessage = (event) => { - console.log(`${viewerUserId} received a moment alert:`, event.message); -}; - -alertSubscription.subscribe(); - -process.on('SIGINT', () => { - console.log(`${viewerUserId} left game.stream, but is still reachable for a moment alert`); - watchSubscription.unsubscribe(); -}); -// snippet.end diff --git a/docs-snippets/use-cases/game-chat-moderation.ts b/docs-snippets/use-cases/game-chat-moderation.ts deleted file mode 100644 index f8c708623..000000000 --- a/docs-snippets/use-cases/game-chat-moderation.ts +++ /dev/null @@ -1,139 +0,0 @@ -import PubNub from '../../lib/types'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'moderator-7', -}); - -// Removing a message from Message Persistence, and granting the token that lets -// moderator.js write hide decisions to the control channel, are both server-side -// operations, so this client is configured with the keyset's secret key and runs -// on your own infrastructure. -const server = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - secretKey: 'demo', - userId: 'moderation-service', -}); - -// snippet.chatModerationGrantControlChannelAccess -try { - const token = await server.grantToken({ - ttl: 60, - authorizedUserId: 'moderator-7', - resources: { - channels: { - 'game.chat': { read: true }, - 'game.chat.moderation': { read: true, write: true }, - }, - }, - }); - console.log('token that lets moderator-7 publish hide decisions:', token); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Granting moderator access failed: ${error}${status ? ` Additional information: ${status}` : ''}`); -} -// snippet.end - -// snippet.chatModerationApplyModeratorToken -pubnub.setToken('replace-with-the-token-server-js-printed'); -// snippet.end - -// snippet.chatModerationFlagMessage -try { - const response = await pubnub.publish({ - channel: 'game.chat.moderation', - message: { - action: 'hide', - messageTimetoken: 'replace-with-message-timetoken', - }, - customMessageType: 'moderation-hide', - storeInHistory: true, - }); - console.log('message flagged at timetoken:', response.timetoken); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Flagging the message failed: ${error}${status ? ` Additional information: ${status}` : ''}`); -} -// snippet.end - -// snippet.chatModerationReceiveModerationDecisions -const hiddenTimetokens = new Set(); - -const moderationSubscription = pubnub.channel('game.chat.moderation').subscription(); - -moderationSubscription.onMessage = (event) => { - const decision = event.message; - const action = - typeof decision === 'object' && decision !== null && !Array.isArray(decision) && 'action' in decision - ? decision.action - : undefined; - const messageTimetoken = - typeof decision === 'object' && decision !== null && !Array.isArray(decision) && 'messageTimetoken' in decision - ? decision.messageTimetoken - : undefined; - - if (action === 'hide' && typeof messageTimetoken === 'string') { - hiddenTimetokens.add(messageTimetoken); - console.log('hide the message published at', messageTimetoken); - } -}; - -moderationSubscription.subscribe(); -// snippet.end - -// snippet.chatModerationLoadHistoryWithFlags -try { - const response = await pubnub.fetchMessages({ - channels: ['game.chat', 'game.chat.moderation'], - count: 25, - }); - - const moderationEntries = response.channels['game.chat.moderation'] ?? []; - - moderationEntries.forEach((entry) => { - const decision = entry.message; - const action = - typeof decision === 'object' && decision !== null && !Array.isArray(decision) && 'action' in decision - ? decision.action - : undefined; - const messageTimetoken = - typeof decision === 'object' && decision !== null && !Array.isArray(decision) && 'messageTimetoken' in decision - ? decision.messageTimetoken - : undefined; - - if (action === 'hide' && typeof messageTimetoken === 'string') { - hiddenTimetokens.add(messageTimetoken); - } - }); - - const chatEntries = response.channels['game.chat'] ?? []; - - chatEntries.forEach((entry) => { - const hidden = hiddenTimetokens.has(entry.timetoken.toString()); - console.log(entry.timetoken, hidden ? '[hidden by a moderator]' : entry.message); - }); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Loading the moderated history failed: ${error}${status ? ` Additional information: ${status}` : ''}`); -} -// snippet.end - -// snippet.chatModerationDeleteMessage -try { - const messageTimetoken = 'replace-with-message-timetoken'; - const start = (BigInt(messageTimetoken) - BigInt(1)).toString(); - const end = messageTimetoken; - - const response = await server.deleteMessages({ - channel: 'game.chat', - start, - end, - }); - console.log('message deleted from Message Persistence:', response); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Deleting the message failed: ${error}${status ? ` Additional information: ${status}` : ''}`); -} -// snippet.end diff --git a/docs-snippets/use-cases/live-commentary.ts b/docs-snippets/use-cases/live-commentary.ts deleted file mode 100644 index 1fe2c7588..000000000 --- a/docs-snippets/use-cases/live-commentary.ts +++ /dev/null @@ -1,70 +0,0 @@ -import PubNub from '../../lib/types'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'fan-42', -}); - -// snippet.liveCommentaryPublishRemark -try { - const response = await pubnub.publish({ - channel: 'game.commentary', - message: { - text: 'Long ball over the top, and the striker is clean through.', - clock: '48:07', - }, - customMessageType: 'commentary', - storeInHistory: true, - }); - console.log('commentary published at timetoken:', response.timetoken); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Publishing the commentary failed: ${error}${status ? ` Additional information: ${status}` : ''}`); -} -// snippet.end - -// snippet.liveCommentarySubscribe -const commentaryChannel = pubnub.channel('game.commentary'); -const commentarySubscription = commentaryChannel.subscription({ receivePresenceEvents: false }); - -commentarySubscription.onMessage = (event) => { - console.log(`[${event.timetoken}] ${JSON.stringify(event.message)}`); -}; - -pubnub.addListener({ - status: (event) => { - if (event.category === 'PNConnectedCategory') { - console.log('connected and ready to receive commentary'); - } - }, -}); - -commentarySubscription.subscribe(); -// snippet.end - -// snippet.liveCommentaryLoadBacklog -try { - const response = await pubnub.fetchMessages({ - channels: ['game.commentary'], - count: 25, - }); - - const entries = response.channels['game.commentary'] ?? []; - - entries.forEach((entry) => { - console.log(entry.timetoken, entry.message); - }); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Loading the commentary backlog failed: ${error}${status ? ` Additional information: ${status}` : ''}`); -} -// snippet.end - -// snippet.liveCommentaryUnsubscribe -process.on('SIGINT', () => { - console.log('viewer shutting down, closing the commentary subscription'); - commentarySubscription.unsubscribe(); - process.exit(0); -}); -// snippet.end diff --git a/docs-snippets/use-cases/live-event-rate-limiting.ts b/docs-snippets/use-cases/live-event-rate-limiting.ts deleted file mode 100644 index 87da6f1f8..000000000 --- a/docs-snippets/use-cases/live-event-rate-limiting.ts +++ /dev/null @@ -1,67 +0,0 @@ -import PubNub from '../../lib/types'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'fan-42', -}); - -// snippet.rateLimitingReadShardOccupancy -try { - const response = await pubnub.hereNow({ - channels: ['game.chat.shard-0', 'game.chat.shard-1', 'game.chat.shard-2'], - includeUUIDs: false, - }); - - Object.entries(response.channels).forEach(([channel, data]) => { - console.log(`${channel} holds ${data.occupancy} fans`); - }); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Reading the shard occupancy failed: ${error}${status ? ` Additional information: ${status}` : ''}`); -} -// snippet.end - -// snippet.rateLimitingPickShardForFan -async function pickShardForFan(shardCount = 0, maxFansPerShard = 0) { - const channels = Array.from({ length: shardCount }, (_, index) => `game.chat.shard-${index}`); - - const response = await pubnub.hereNow({ channels, includeUUIDs: false }); - - const openShard = channels.find((channel) => (response.channels[channel]?.occupancy ?? 0) < maxFansPerShard); - - if (!openShard) { - throw new Error('every shard is full, so add more shards before the next match'); - } - - return openShard; -} - -const shard = await pickShardForFan(10, 1000); - -console.log('this fan joins', shard); -// snippet.end - -// snippet.rateLimitingThrottleFanPublishes -const minimumMillisecondsBetweenMessages = 2000; -let lastPublishedAt = 0; - -async function sendChatMessage(text = '') { - const now = Date.now(); - - if (now - lastPublishedAt < minimumMillisecondsBetweenMessages) { - console.log('too soon, so this message is dropped before it reaches the network'); - return; - } - - lastPublishedAt = now; - - await pubnub.publish({ - channel: shard, - message: { text }, - customMessageType: 'text-message', - }); -} - -await sendChatMessage('Offside, surely?'); -// snippet.end diff --git a/docs-snippets/use-cases/live-polls.ts b/docs-snippets/use-cases/live-polls.ts deleted file mode 100644 index aaeb050ae..000000000 --- a/docs-snippets/use-cases/live-polls.ts +++ /dev/null @@ -1,106 +0,0 @@ -import PubNub from '../../lib/types'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'fan-42', -}); - -// snippet.livePollsPublishPoll -const poll = { - id: 'poll-1', - title: 'Who will win the match?', - durationSeconds: 60, - options: [ - { id: 1, text: 'Home team' }, - { id: 2, text: 'Away team' }, - { id: 3, text: 'Draw' }, - ], -}; - -try { - const response = await pubnub.publish({ - channel: 'game.new-poll', - message: poll, - customMessageType: 'poll-opened', - storeInHistory: true, - }); - console.log('poll published at timetoken:', response.timetoken); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Publishing the poll failed: ${error}${status ? ` Additional information: ${status}` : ''}`); -} -// snippet.end - -// snippet.livePollsSubscribeToPolls -const pollChannel = pubnub.channel('game.new-poll'); -const pollSubscription = pollChannel.subscription({ receivePresenceEvents: false }); - -pollSubscription.onMessage = (event) => { - console.log('new poll:', event.message); -}; - -pollSubscription.subscribe(); -// snippet.end - -// snippet.livePollsFetchOpenPoll -try { - const response = await pubnub.fetchMessages({ - channels: ['game.new-poll'], - count: 1, - }); - - const entries = response.channels['game.new-poll']; - - if (entries && entries.length > 0) { - console.log('poll that is already open:', entries[0].message); - } -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Fetching the open poll failed: ${error}${status ? ` Additional information: ${status}` : ''}`); -} -// snippet.end - -// snippet.livePollsPublishVote -try { - const response = await pubnub.publish({ - channel: 'game.poll-votes', - message: { pollId: 'poll-1', optionId: 2 }, - customMessageType: 'poll-vote', - }); - console.log('vote published at timetoken:', response.timetoken); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Publishing the vote failed: ${error}${status ? ` Additional information: ${status}` : ''}`); -} -// snippet.end - -// snippet.livePollsPublishResults -try { - const response = await pubnub.publish({ - channel: 'game.poll-results', - message: { - pollId: 'poll-1', - totals: { 1: 812, 2: 1043, 3: 219 }, - closed: true, - }, - customMessageType: 'poll-results', - storeInHistory: true, - }); - console.log('results published at timetoken:', response.timetoken); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Publishing the results failed: ${error}${status ? ` Additional information: ${status}` : ''}`); -} -// snippet.end - -// snippet.livePollsSubscribeToResults -const resultsChannel = pubnub.channel('game.poll-results'); -const resultsSubscription = resultsChannel.subscription({ receivePresenceEvents: false }); - -resultsSubscription.onMessage = (event) => { - console.log('poll results:', event.message); -}; - -resultsSubscription.subscribe(); -// snippet.end diff --git a/docs-snippets/use-cases/match-stats.ts b/docs-snippets/use-cases/match-stats.ts deleted file mode 100644 index c3880d3f1..000000000 --- a/docs-snippets/use-cases/match-stats.ts +++ /dev/null @@ -1,58 +0,0 @@ -import PubNub from '../../lib/types'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'fan-42', -}); - -// snippet.matchStatsPublishOneStat -try { - const response = await pubnub.publish({ - channel: 'game.match-stats.score', - message: { value: '3-1' }, - customMessageType: 'match-stat', - storeInHistory: true, - }); - console.log('score published at timetoken:', response.timetoken); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Publishing the stat failed: ${error}${status ? ` Additional information: ${status}` : ''}`); -} -// snippet.end - -// snippet.matchStatsSubscribeToAllStats -const statsSubscription = pubnub.subscriptionSet({ - channels: ['game.match-stats.*'], - subscriptionOptions: { receivePresenceEvents: false }, -}); - -statsSubscription.onMessage = (event) => { - const statName = event.channel.split('.').pop(); - console.log(`${statName} is now`, event.message); -}; - -statsSubscription.subscribe(); -// snippet.end - -// snippet.matchStatsFetchCurrentValues -const statNames = ['score', 'possession', 'shots', 'cards']; - -try { - const response = await pubnub.fetchMessages({ - channels: statNames.map((statName) => `game.match-stats.${statName}`), - count: 1, - }); - - statNames.forEach((statName) => { - const entries = response.channels[`game.match-stats.${statName}`]; - - if (entries && entries.length > 0) { - console.log(`${statName} is currently`, entries[0].message); - } - }); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Fetching the current stats failed: ${error}${status ? ` Additional information: ${status}` : ''}`); -} -// snippet.end diff --git a/docs-snippets/use-cases/real-time-ads.ts b/docs-snippets/use-cases/real-time-ads.ts deleted file mode 100644 index 786a21e42..000000000 --- a/docs-snippets/use-cases/real-time-ads.ts +++ /dev/null @@ -1,65 +0,0 @@ -import PubNub from '../../lib/types'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'fan-42', -}); - -// snippet.realTimeAdsPublishReaction -try { - const response = await pubnub.publish({ - channel: 'game.stream-reactions', - message: { reaction: '\u{1F525}' }, - customMessageType: 'reaction', - }); - console.log('reaction published at timetoken:', response.timetoken); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Publishing the reaction failed: ${error}${status ? ` Additional information: ${status}` : ''}`); -} -// snippet.end - -// snippet.realTimeAdsReceiveAdDecision -const adSubscription = pubnub.channel('game.ad-decisions').subscription({ receivePresenceEvents: false }); - -adSubscription.onMessage = (event) => { - const decision = event.message; - const adId = - typeof decision === 'object' && decision !== null && !Array.isArray(decision) && 'adId' in decision - ? decision.adId - : undefined; - const clickPoints = - typeof decision === 'object' && decision !== null && !Array.isArray(decision) && 'clickPoints' in decision - ? decision.clickPoints - : undefined; - - if (adId) { - console.log(`show ad ${adId}, worth ${clickPoints} points`); - } else { - console.log('no ad to show, so clear the ad slot'); - } -}; - -adSubscription.subscribe(); -// snippet.end - -// snippet.realTimeAdsReceiveReactionUpgrade -const upgradeSubscription = pubnub.channel('game.reaction-upgrades').subscription({ receivePresenceEvents: false }); - -upgradeSubscription.onMessage = (event) => { - const upgrade = event.message; - const reaction = - typeof upgrade === 'object' && upgrade !== null && !Array.isArray(upgrade) && 'reaction' in upgrade - ? upgrade.reaction - : undefined; - const replacement = - typeof upgrade === 'object' && upgrade !== null && !Array.isArray(upgrade) && 'replacement' in upgrade - ? upgrade.replacement - : undefined; - - console.log(`render ${reaction} as ${replacement} from now on`); -}; - -upgradeSubscription.subscribe(); -// snippet.end diff --git a/docs-snippets/use-cases/real-time-chat.ts b/docs-snippets/use-cases/real-time-chat.ts deleted file mode 100644 index 7af2ed4d2..000000000 --- a/docs-snippets/use-cases/real-time-chat.ts +++ /dev/null @@ -1,131 +0,0 @@ -import PubNub from '../../lib/types'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'fan-42', -}); - -// snippet.fanChatCreateChannelMetadata -try { - const response = await pubnub.objects.setChannelMetadata({ - channel: 'game.chat', - data: { - name: 'Match chat', - description: 'Open chat for everyone watching the match', - }, - }); - console.log('channel metadata set:', response.data); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Setting the channel metadata failed: ${error}${status ? ` Additional information: ${status}` : ''}`); -} -// snippet.end - -// snippet.fanChatSetFanProfile -try { - const response = await pubnub.objects.setUUIDMetadata({ - uuid: 'fan-42', - data: { - name: 'Alex Moreau', - profileUrl: 'https://example.com/avatars/fan-42.png', - custom: { supports: 'home' }, - }, - }); - console.log('fan profile set:', response.data); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Setting the fan profile failed: ${error}${status ? ` Additional information: ${status}` : ''}`); -} -// snippet.end - -// snippet.fanChatSubscribeWithPresence -const chatChannel = pubnub.channel('game.chat'); -const chatSubscription = chatChannel.subscription({ receivePresenceEvents: true }); - -chatSubscription.onMessage = (event) => { - console.log(`${event.publisher}: ${JSON.stringify(event.message)}`); -}; - -chatSubscription.onPresence = (event) => { - if (event.action === 'interval' || event.action === 'join' || event.action === 'leave') { - console.log('fans in this chat:', event.occupancy); - } -}; - -chatSubscription.subscribe(); -// snippet.end - -// snippet.fanChatSendMessage -try { - const response = await pubnub.publish({ - channel: 'game.chat', - message: { text: 'What a save!' }, - customMessageType: 'text-message', - storeInHistory: true, - }); - console.log('chat message published at timetoken:', response.timetoken); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Publishing the chat message failed: ${error}${status ? ` Additional information: ${status}` : ''}`); -} -// snippet.end - -// snippet.fanChatLoadRecentMessages -try { - const response = await pubnub.fetchMessages({ - channels: ['game.chat'], - count: 25, - includeMessageActions: true, - }); - - const entries = response.channels['game.chat'] ?? []; - - entries.forEach((entry) => { - console.log(entry.timetoken, entry.message); - }); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Loading recent messages failed: ${error}${status ? ` Additional information: ${status}` : ''}`); -} -// snippet.end - -// snippet.fanChatCountFansOnline -try { - const response = await pubnub.hereNow({ - channels: ['game.chat'], - includeUUIDs: false, - }); - console.log('fans in the chat:', response.totalOccupancy); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Counting the fans online failed: ${error}${status ? ` Additional information: ${status}` : ''}`); -} -// snippet.end - -// snippet.fanChatAddReaction -try { - const response = await pubnub.addMessageAction({ - channel: 'game.chat', - messageTimetoken: 'replace-with-message-timetoken', - action: { - type: 'reaction', - value: '\u{1F44F}', - }, - }); - console.log('reaction added at timetoken:', response.data.actionTimetoken); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Adding the reaction failed: ${error}${status ? ` Additional information: ${status}` : ''}`); -} -// snippet.end - -// snippet.fanChatReceiveReactions -const reactionsSubscription = pubnub.channel('game.chat').subscription(); - -reactionsSubscription.onMessageAction = (event) => { - console.log(`${event.publisher} reacted with ${event.data.value} to the message at ${event.data.messageTimetoken}`); -}; - -reactionsSubscription.subscribe(); -// snippet.end diff --git a/docs-snippets/use-cases/score-alerts.ts b/docs-snippets/use-cases/score-alerts.ts deleted file mode 100644 index 98c0dd2eb..000000000 --- a/docs-snippets/use-cases/score-alerts.ts +++ /dev/null @@ -1,145 +0,0 @@ -import PubNub from '../../lib/types'; - -const pubnub = new PubNub({ - publishKey: 'demo', - subscribeKey: 'demo', - userId: 'fan-42', -}); - -// snippet.scoreAlertsEnvironmentConstant -// Every APNs call below, device registration, the notification payload's target, -// listing, and removal, reads this same value. An iOS device token only works in the -// APNs environment that issued it: a development (sandbox) token comes from a -// debug or development-signed build and only works with environment: 'development'; -// a production token comes from a TestFlight or App Store build and only works with -// environment: 'production'. Registering with one value and publishing toward the -// other is why a registration can succeed while the notification it's supposed to -// produce never arrives. Change this one constant when you move from a development -// build to a TestFlight or App Store build, rather than editing every call below. -const APNS_ENVIRONMENT = 'development'; -// snippet.end - -// snippet.scoreAlertsRegisterDeviceAPNs -try { - const response = await pubnub.push.addChannels({ - channels: ['game.score-alerts'], - device: 'replace-with-the-apns-device-token', - pushGateway: 'apns2', - environment: APNS_ENVIRONMENT, - topic: 'com.example.matchday', - }); - console.log('iOS device registered for score alerts:', response); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Registering the iOS device failed: ${error}${status ? ` Additional information: ${status}` : ''}`); -} -// snippet.end - -// snippet.scoreAlertsRegisterDeviceFCM -try { - const response = await pubnub.push.addChannels({ - channels: ['game.score-alerts'], - device: 'replace-with-the-fcm-registration-token', - pushGateway: 'fcm', - }); - console.log('Android device registered for score alerts:', response); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Registering the Android device failed: ${error}${status ? ` Additional information: ${status}` : ''}`); -} -// snippet.end - -// snippet.scoreAlertsBuildNotification -const goal = PubNub.notificationPayload('Leeds score!', 'Southampton 0 - 2 Leeds'); - -goal.sound = 'default'; -goal.apns.configurations = [{ targets: [{ topic: 'com.example.matchday', environment: APNS_ENVIRONMENT }] }]; - -const payload = goal.buildPayload(['apns2', 'fcm']); - -console.log(JSON.stringify(payload, null, 2)); -// snippet.end - -// snippet.scoreAlertsPublishNotification -try { - const response = await pubnub.publish({ - channel: 'game.score-alerts', - message: { - ...payload, - score: '0-2', - scorer: 'Leeds', - }, - customMessageType: 'score-alert', - }); - console.log('score alert published at timetoken:', response.timetoken); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error(`Publishing the score alert failed: ${error}${status ? ` Additional information: ${status}` : ''}`); -} -// snippet.end - -// snippet.scoreAlertsListDeviceRegistrationsAPNs -try { - const response = await pubnub.push.listChannels({ - device: 'replace-with-the-apns-device-token', - pushGateway: 'apns2', - environment: APNS_ENVIRONMENT, - topic: 'com.example.matchday', - }); - console.log('this device receives alerts on:', response.channels); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error( - `Listing the device registrations failed: ${error}${status ? ` Additional information: ${status}` : ''}`, - ); -} -// snippet.end - -// snippet.scoreAlertsListDeviceRegistrationsFCM -try { - const response = await pubnub.push.listChannels({ - device: 'replace-with-the-fcm-registration-token', - pushGateway: 'fcm', - }); - console.log('this device receives alerts on:', response.channels); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error( - `Listing the device registrations failed: ${error}${status ? ` Additional information: ${status}` : ''}`, - ); -} -// snippet.end - -// snippet.scoreAlertsRemoveDeviceRegistrationAPNs -try { - const response = await pubnub.push.removeChannels({ - channels: ['game.score-alerts'], - device: 'replace-with-the-apns-device-token', - pushGateway: 'apns2', - environment: APNS_ENVIRONMENT, - topic: 'com.example.matchday', - }); - console.log('device no longer receives score alerts:', response); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error( - `Removing the device registration failed: ${error}${status ? ` Additional information: ${status}` : ''}`, - ); -} -// snippet.end - -// snippet.scoreAlertsRemoveDeviceRegistrationFCM -try { - const response = await pubnub.push.removeChannels({ - channels: ['game.score-alerts'], - device: 'replace-with-the-fcm-registration-token', - pushGateway: 'fcm', - }); - console.log('device no longer receives score alerts:', response); -} catch (error) { - const status = error instanceof Error && 'status' in error ? error.status : undefined; - console.error( - `Removing the device registration failed: ${error}${status ? ` Additional information: ${status}` : ''}`, - ); -} -// snippet.end diff --git a/package.json b/package.json index dbf1fb95d..172a55c08 100644 --- a/package.json +++ b/package.json @@ -16,21 +16,13 @@ "lint": "eslint \"src/**/*\" --config .eslintrc.cjs", "test:integration:live": "TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json mocha --config test/integration/live/.mocharc.cjs", "test:integration:live:endpoints": "TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json mocha --config test/integration/live/.mocharc.endpoints.cjs", - "test:integration:live:types": "tsc -p test/integration/live/tsconfig.json", + "test:integration:live:types": "tsc -p test/integration/live/tsconfig.json && tsc -p test/integration/live/tsconfig.browser.json", "test:integration:live:janitor": "TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json tsx test/integration/live/support/janitor-cli.ts", "test:integration:live:provision": "TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json tsx test/integration/live/provision/provision-datasync.ts", "test:integration:live:provision:check": "TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json tsx test/integration/live/provision/provision-datasync.ts --check", - "test:integration:live:lint": "TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json tsx test/integration/live/support/lint-markers.ts && TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json tsx test/integration/live/support/manifest.ts --check && eslint \"test/integration/live/**/*.ts\"", + "test:integration:live:lint": "TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json tsx test/integration/live/support/lint-markers.ts && TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json tsx test/integration/live/support/manifest.ts --check && eslint \"test/integration/live/**/*.ts\" --ignore-pattern \"test/integration/live/quickstarts/**\"", "test:integration:live:manifest": "TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json tsx test/integration/live/support/manifest.ts", "test:integration:live:docs-check": "TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json tsx test/integration/live/support/docs-check.ts", - "test:snippets": "tsc --project docs-snippets/tsconfig.json --noEmit", - "test:docs-snippets": "TSX_TSCONFIG_PATH=docs-snippets/tsconfig.run.json tsx docs-snippets/harness/cli.ts live", - "test:docs-snippets:dry": "TSX_TSCONFIG_PATH=docs-snippets/tsconfig.run.json tsx docs-snippets/harness/cli.ts dry", - "test:docs-snippets:janitor": "TSX_TSCONFIG_PATH=docs-snippets/tsconfig.run.json tsx docs-snippets/harness/cli.ts janitor", - "test:docs-snippets:types": "tsc -p docs-snippets/tsconfig.live.json && tsc -p docs-snippets/tsconfig.browser.json", - "docs-snippets:summary": "node docs-snippets/harness/snippet-summary.cjs", - "docs-snippets:manifest": "TSX_TSCONFIG_PATH=docs-snippets/tsconfig.run.json tsx docs-snippets/harness/cli.ts manifest", - "docs-snippets:provision": "TSX_TSCONFIG_PATH=docs-snippets/tsconfig.run.json tsx docs-snippets/provision/provision-datasync.ts", "ci": "npm run clean && npm run build && npm run lint && npm run test", "ci:web": "npm run clean && npm run build:web && npm run lint && npm run test:web && npm run test:web:shared-worker", "ci:node": "npm run clean && npm run build:node && npm run lint && npm run test:node", diff --git a/test/integration/live/README.md b/test/integration/live/README.md index fe621f75f..54b99fdce 100644 --- a/test/integration/live/README.md +++ b/test/integration/live/README.md @@ -12,7 +12,7 @@ npm run test:integration:live:endpoints # only endpoints/**: real-n npm run test:integration:live -- --grep publishMinimal npm run test:integration:live -- --dry-run # lists the tests, no network, no keys needed npm run test:integration:live:types # type check (needs the build for lib/types) -npx eslint "test/integration/live/**/*.ts" # lint (npm run lint covers src/ only). Do not add --config: it drops the max-len override of live/.eslintrc.cjs +npx eslint "test/integration/live/**/*.ts" --ignore-pattern "test/integration/live/quickstarts/**" # lint (npm run lint covers src/ only). Do not add --config: it drops the max-len override of live/.eslintrc.cjs ``` Env names: `SUBSCRIBE_KEY`, `PUBLISH_KEY` (open keyset), `PAM_SUBSCRIBE_KEY`, `PAM_PUBLISH_KEY`, `PAM_SECRET_KEY` (Access Manager), `DS_*` (DataSync). Real environment variables win over the file. `LIVE_ENV_FILE=` points at another file. The run fails at once when the open keyset is missing. A docs-tier run without `--grep` (and without spec file arguments) needs every role: it fails at once, with the missing names, when `pam` or `datasync` is not set. A run with `--grep` or a file argument checks a role only when a test builds its client, so a page that needs only the open keyset runs with only that keyset. Run one live run per keyset at a time. @@ -30,10 +30,10 @@ Rebuild with `npm run build:node` after a change under `src/`, otherwise the tie ## Conventions -- One file per docs page: `.test.ts`, named like the old page module (`docs-snippets/api-reference/publish.ts` becomes `api-reference/publish.test.ts`; keep the folder (`api-reference/`, `getting-started/`, `migration-guides/`, `shared/`) so the docs URL changes only its prefix and suffix). One `describe` per page, one `it` per old step. Start the `it` title with the snippet id(s), then what it proves. +- One file per docs page: `.test.ts` under the folder that matches the docs section (`api-reference/`, `getting-started/`, `migration-guides/`, `shared/`, `use-cases/`), so the docs URL is `test/integration/live//.test.ts`. One `describe` per page, one `it` per sample. Start the `it` title with the snippet id(s), then what it proves. - Plain `describe` / `it`, `node:assert/strict`. No staging, no recorder, no `page` / `step` / `check`. -- Markers sit inside the `it` body: `// snippet.` ... `// snippet.end`. The code between them is copied character for character from the old page (indentation may differ, readers get it un-indented). Everything else (setup, listeners, assertions) stays outside. Use `// snippet.hide` / `// snippet.show` only where the sample cannot run without a line readers must not see. -- Check each page with the equivalence script (below). Every snippet id must stay and its text must not change. +- Markers sit inside the `it` body: `// snippet.` ... `// snippet.end`. The code between them is the published sample text (indentation may differ, readers get it un-indented). Never change it without the approval of the docs owner. Everything else (setup, listeners, assertions) stays outside. Use `// snippet.hide` / `// snippet.show` only where the sample cannot run without a line readers must not see. +- Run the marker lint (`npm run test:integration:live:lint`) after every change. A snippet id is part of the docs URL, so keep it stable. - **Literal names.** The code in the markers uses exactly the names the docs show (`'my_channel'`, `'foo'`, ...), and that is what runs. Isolation comes from one live run per keyset at a time and a cleanup pass before and after (the janitor). - **Assertions must not assume a private, clean resource.** Channels, users and groups are shared and persist across runs. Assert on this run's own data: the `timetoken` the sample returned, an ID, or a unique value your test generates outside the markers. Do not assert exact counts or "the latest N". When the sample does not keep its response, match as narrowly as the sample allows (publisher, payload) and say why in a comment. - **Reset, then assert exactly.** A test that works on a literal resource (a channel group, a user, a device token, a DataSync object) first resets it to a known state (remove, then seed what it needs), and then asserts the exact result. This is safe because the janitor cleans the listed names and one run owns a keyset at a time. Put the reset in the test or in a hook of the page, not in the sample. @@ -68,20 +68,12 @@ Rebuild with `npm run build:node` after a change under `src/`, otherwise the tie | `console.ts` | `captureConsole(method)`, `quietConsole()`, `restoreAllConsoles()` | | `resources.ts` | the janitor's resource list; `DATASYNC_PAGE_OBJECTS` is shared with `datasync-state.ts` | | `hooks.ts` | root hooks: role check before a full run, janitor, preflight, client cleanup after each test | -| `snippets.ts`, `compare-snippets.ts` | snippet extraction with the docs rules, and the equivalence check | +| `snippets.ts` | snippet extraction with the rules of the docs extractor | Other files in `support/` serve one page or one tool (`janitor*.ts`, `preflight.ts`, `datasync-*.ts`, `subscribe-groups.ts`, `proxy.ts`, `file-workdir.ts`, `access-manager.ts`). Every `observer()`, `shopper()` call builds a new client, so a test can have two clients with the same user id when it needs them. -## Equivalence check - -```bash -npx tsx test/integration/live/support/compare-snippets.ts docs-snippets/api-reference/publish.ts test/integration/live/api-reference/publish.test.ts -``` - -It extracts every snippet of both files with the rules of the docs extractor and compares them per id. It exits 1 on any difference, missing or extra id, or marker problem. - ## Cleanup (janitor), preflight, provisioning The tests use the literal names the docs show (`'my_channel'`, `'family'`, ...), so the state they leave behind is shared and persistent. Three pieces keep a run honest. @@ -109,7 +101,7 @@ A full pass walks every listed name, so it takes a while (see the measured time ### Preflight (`support/preflight.ts`) -After the janitor, the root hook fails fast, with a message that names each missing feature. With `PUBNUB_ADMIN_API_KEY` and the keyset ids (`DOCS_SNIPPETS_OPEN_KEYSET_ID`, `DOCS_SNIPPETS_PAM_KEYSET_ID`, `DOCS_SNIPPETS_DS_KEYSET_ID`) it reads each keyset's configuration through the Admin API (read only) and lists what the role needs and lacks (Presence, Message Persistence, Delete-From-History, App Context and its events, Files, Stream Controller, Access Manager, DataSync). Live probes on the open keyset always run: publish (keys and network), Message Persistence, Delete-From-History, and Presence when the Admin API did not vouch for it. The features each keyset needs are listed in `docs-snippets/README.md`. +After the janitor, the root hook fails fast, with a message that names each missing feature. With `PUBNUB_ADMIN_API_KEY` and the keyset ids (`DOCS_SNIPPETS_OPEN_KEYSET_ID`, `DOCS_SNIPPETS_PAM_KEYSET_ID`, `DOCS_SNIPPETS_DS_KEYSET_ID`) it reads each keyset's configuration through the Admin API (read only) and lists what the role needs and lacks (Presence, Message Persistence, Delete-From-History, App Context and its events, Files, Stream Controller, Access Manager, DataSync). Live probes on the open keyset always run: publish (keys and network), Message Persistence, Delete-From-History, and Presence when the Admin API did not vouch for it. The features each keyset needs are the ones the message names; `missingFeatures` in `support/preflight.ts` is the full list per role (open, pam, datasync). ### Provisioning (`provision/`) @@ -149,4 +141,4 @@ npm run test:integration:live:docs-check # docs cross-check (needs th **Manifest** (`support/manifest.ts`, output `coverage.json`, generated, do not edit). One entry per snippet id: file, the full Mocha title that runs it (describe + it), keyset role when the test names one (heuristic), and `mode`: `live` (inside an `it()`), `compile-only` (import-only samples: type-checked and loaded, not run as an API call), `mocked` (inside a nock-mocked test listed in `MOCKED_FILES` in `support/snippet-scan.ts`: `test/integration/endpoints/objects/channel.test.ts`, `test/integration/endpoints/time.test.ts`, `test/integration/components/subscription_manager.test.ts`, five samples; the same lint rules apply, except the title-names-the-id warning, which is the live tier's convention). `files` lists whole-file samples that a test runs (`getting-started/quickstart-node.mjs`). `-- --check` fails when `coverage.json` is stale. Things the docs embed that are deliberately not run (the browser, React and React Native quickstarts) are listed by hand, with a reason, in `snippet-exceptions.json`. -**Docs cross-check** (`support/docs-check.ts`). Reads every `` that points at `pubnub/javascript` in the documentation checkout (first argument, `DOCS_REPO`, or `../documentation`) and reports each embed as live, compile-only, mocked, exception or NOT COVERED (exit code 1 unless `--report-only`). It understands both URL styles. An old-style URL (`docs-snippets//.ts`) is mapped to `test/integration/live//.test.ts`. A new-style URL (a path under `test/integration/live/`, or one of the mocked test files) is checked as it stands: the file must exist and `coverage.json` must list each requested snippet id for that very file (a whole-file embed needs a file named by a test in `coverage.json`, or a listed exception); its `action` in the repoint list is `done`. Use `--out ` when you run it on a checkout so the default repoint list is not overwritten. For an embed that points at an old duplicate copy (`docs-snippets/basic-usage/*`, `docs-snippets/.ts`) it finds the same id in the tier and says whether the old copy and the tested twin are IDENTICAL or DIFFERENT (diffs in `/tmp/pn-live/dup-diffs/`). Such embeds stay listed as NOT COVERED until the docs are repointed. It writes the list for repointing the docs URLs to `/tmp/pn-live/docs-repoint.json` (`--out ` to change); its `action` is `repoint` (the mapped path), `repoint-to-twin` (the embed points at an old duplicate copy: change it to `newPath`, the tested twin), `keep` (documented exception), `done` (the URL already points at the tested tier) or `blocked` (not covered and no twin). The pages `users`, `spaces`, `memberships` and `access-manager-v2` of the API reference are out of scope. The documentation repository is only read. +**Docs cross-check** (`support/docs-check.ts`). Reads every `` that points at `pubnub/javascript` in the documentation checkout (first argument, `DOCS_REPO`, or `../documentation`) and reports each embed as live, compile-only, mocked, exception or NOT COVERED (exit code 1 unless `--report-only`). The URL must name a file under `test/integration/live/` or one of the mocked test files: the file must exist and `coverage.json` must list each requested snippet id for that very file (a whole-file embed needs a file named by a test in `coverage.json`, or a listed exception). Any other path, such as a leftover `docs-snippets/...` URL, is NOT COVERED. The pages `users`, `spaces`, `memberships` and `access-manager-v2` of the API reference are out of scope. The documentation repository is only read. diff --git a/test/integration/live/api-reference/access-manager-legacy.test.ts b/test/integration/live/api-reference/access-manager-legacy.test.ts index 820f5d504..cc3ee7ddb 100644 --- a/test/integration/live/api-reference/access-manager-legacy.test.ts +++ b/test/integration/live/api-reference/access-manager-legacy.test.ts @@ -7,7 +7,7 @@ import { granted } from '../support/access-manager'; /** * Access Manager token examples that use the deprecated `spaces`, `users` and `uuids` terms. The SDK maps them to the * current terms, so a `spaces` grant appears as `channels` in the parsed token. - * Port of docs-snippets/api-reference/access-manager-legacy.ts. Only `grantToken` is called (no application-level grant), + * Only `grantToken` is called (no application-level grant), * so no cleanup hook is needed. Tokens expire by themselves (ttl 15). */ describe('api-reference/access-manager-legacy', () => { diff --git a/test/integration/live/api-reference/access-manager.test.ts b/test/integration/live/api-reference/access-manager.test.ts index 897656ff9..9bc75c45b 100644 --- a/test/integration/live/api-reference/access-manager.test.ts +++ b/test/integration/live/api-reference/access-manager.test.ts @@ -9,7 +9,6 @@ import { connected, eventually } from '../support/wait'; /** * Access Manager v3 token examples. The `grantToken` calls run on a server that holds the secret key (pam keyset). - * Port of docs-snippets/api-reference/access-manager.ts. * * Tokens are bound to a user id and a resource, never to an application-wide grant, so nothing here opens the keyset for * strangers and no cleanup hook is needed. The only residue is tokens that expire by themselves (ttl 15 minutes) and the diff --git a/test/integration/live/api-reference/channel-groups.test.ts b/test/integration/live/api-reference/channel-groups.test.ts index 1eb5243de..19d25b51c 100644 --- a/test/integration/live/api-reference/channel-groups.test.ts +++ b/test/integration/live/api-reference/channel-groups.test.ts @@ -6,7 +6,7 @@ import { eventually, settled } from '../support/wait'; import { sorted } from '../support/util'; /** - * Channel group examples. Port of docs-snippets/api-reference/channel-groups.ts. + * Channel group examples. * The groups 'myChannelGroup' and 'family' are shared literal names, so every test deletes the group it works on first * (reset approach). After the reset the exact channel list assertions are legitimate. */ diff --git a/test/integration/live/api-reference/configuration-legacy.test.ts b/test/integration/live/api-reference/configuration-legacy.test.ts index 212dbe0d3..edab278a5 100644 --- a/test/integration/live/api-reference/configuration-legacy.test.ts +++ b/test/integration/live/api-reference/configuration-legacy.test.ts @@ -8,7 +8,7 @@ import { connected, eventually, settled } from '../support/wait'; /** * Examples for the deprecated configuration parameters `uuid`, `logVerbosity`, `cipherKey` and `useRandomIVs`, and for the - * deprecated `stop()` method. Port of docs-snippets/api-reference/configuration-legacy.ts. The page has no snippet markers. + * deprecated `stop()` method. The page has no snippet markers. * The cipherKey and useRandomIVs tests read stored messages back, so Message Persistence must be on for the open keyset. */ diff --git a/test/integration/live/api-reference/configuration.test.ts b/test/integration/live/api-reference/configuration.test.ts index 169dbffe1..2e3102c8e 100644 --- a/test/integration/live/api-reference/configuration.test.ts +++ b/test/integration/live/api-reference/configuration.test.ts @@ -7,7 +7,7 @@ import { keysOf, openKeys, securedKeys, subscribeKeyOnly } from '../support/env' import { eventually, watch } from '../support/wait'; /** - * Configuration examples. Port of docs-snippets/api-reference/configuration.ts. + * Configuration examples. * The samples show placeholder keys. The hidden spread after them puts the real keys of the keyset in, so the text readers see * stays generic. The spread is typed Partial, otherwise TypeScript reports the overwritten visible keys (TS2783). */ diff --git a/test/integration/live/api-reference/datasync.test.ts b/test/integration/live/api-reference/datasync.test.ts index 1faa2743c..75b0e0269 100644 --- a/test/integration/live/api-reference/datasync.test.ts +++ b/test/integration/live/api-reference/datasync.test.ts @@ -7,7 +7,7 @@ import { errorText, removePageObjects, untilPageObjectsGone } from '../support/d import { connected, eventually, settled } from '../support/wait'; /** - * DataSync examples, including the DataSync listener and subscription examples. Port of docs-snippets/api-reference/datasync.ts. + * DataSync examples, including the DataSync listener and subscription examples. * * Runs on the `datasync` keyset. The samples create DataSync objects with fixed literal ids, and the service answers a create of * an existing id with 409 (DS-0301), it does not overwrite. So a before hook removes every literal object the page creates @@ -81,7 +81,7 @@ describe('api-reference/datasync', () => { }); describe('preconditions', () => { - // The classes are provisioned separately (docs-snippets/provision/). This only reports a missing one. + // The classes are provisioned separately (test/integration/live/provision/). This only reports a missing one. it('the product entity class exists on the keyset', async () => { const klass = await server('datasync').dataSync.getEntities({ class: 'product', classVersion: 1, limit: 1 }); assert.equal(klass.status, 200); diff --git a/test/integration/live/api-reference/encryption-legacy.test.ts b/test/integration/live/api-reference/encryption-legacy.test.ts index 6ddbdf12c..1f2cfa5cb 100644 --- a/test/integration/live/api-reference/encryption-legacy.test.ts +++ b/test/integration/live/api-reference/encryption-legacy.test.ts @@ -4,7 +4,7 @@ import { createClient } from '../support/clients'; /** * Examples for the deprecated `encrypt()` and `decrypt()` methods and the cipher key arguments. Both methods log a - * deprecation warning on every call. Port of docs-snippets/api-reference/encryption-legacy.ts. Everything runs locally. + * deprecation warning on every call. Everything runs locally. */ describe('api-reference/encryption-legacy', () => { // Open keyset client with an AES-CBC crypto module, the setup the legacy methods fall back to. diff --git a/test/integration/live/api-reference/encryption.test.ts b/test/integration/live/api-reference/encryption.test.ts index 4036e8648..0f91524e4 100644 --- a/test/integration/live/api-reference/encryption.test.ts +++ b/test/integration/live/api-reference/encryption.test.ts @@ -7,7 +7,7 @@ import PubNub from 'pubnub'; import { createClient } from '../support/clients'; /** - * Encryption examples. Port of docs-snippets/api-reference/encryption.ts. Everything runs locally (no request is made), + * Encryption examples. Everything runs locally (no request is made), * the clients only carry a crypto module. Each file test works in its own temp directory. */ describe('api-reference/encryption', () => { diff --git a/test/integration/live/api-reference/file-sharing-legacy.test.ts b/test/integration/live/api-reference/file-sharing-legacy.test.ts index c409592b4..b97b83eea 100644 --- a/test/integration/live/api-reference/file-sharing-legacy.test.ts +++ b/test/integration/live/api-reference/file-sharing-legacy.test.ts @@ -5,8 +5,9 @@ import { withPlaceholders } from '../support/placeholders'; import { enterWorkdir, FIXTURE_FILE } from '../support/file-workdir'; /** - * Example for the deprecated per-call `cipherKey` parameter of file sharing. Port of - * docs-snippets/api-reference/file-sharing-legacy.ts. Files persist on the shared channel 'my_channel', so each test asserts + * Example for the deprecated per-call `cipherKey` parameter of file sharing. + * + * Files persist on the shared channel 'my_channel', so each test asserts * on the id it got back and an `afterEach` deletes the files the test uploaded. The download sample shows the id placeholder * '...': it runs unchanged through a client wrapper (`withPlaceholders`) that swaps it for the real id. */ diff --git a/test/integration/live/api-reference/file-sharing.test.ts b/test/integration/live/api-reference/file-sharing.test.ts index bd97a6bb0..8999945e6 100644 --- a/test/integration/live/api-reference/file-sharing.test.ts +++ b/test/integration/live/api-reference/file-sharing.test.ts @@ -10,7 +10,7 @@ import { enterWorkdir, FIXTURE_FILE } from '../support/file-workdir'; import { eventually, settled, watch } from '../support/wait'; /** - * File sharing examples for Node.js. Port of docs-snippets/api-reference/file-sharing.ts. + * File sharing examples for Node.js. * The open keyset needs File Sharing and Message Persistence enabled. * * Files persist on the shared channel 'my_channel', so every test asserts on the file id it got back and an `afterEach` diff --git a/test/integration/live/api-reference/message-actions.test.ts b/test/integration/live/api-reference/message-actions.test.ts index 8b0f03680..cc17fbdd3 100644 --- a/test/integration/live/api-reference/message-actions.test.ts +++ b/test/integration/live/api-reference/message-actions.test.ts @@ -6,7 +6,7 @@ import { withPlaceholders } from '../support/placeholders'; import { eventually, settled, watch } from '../support/wait'; /** - * Message action examples. Port of docs-snippets/api-reference/message-actions.ts. Message Persistence must be on for the open + * Message action examples. Message Persistence must be on for the open * keyset. The samples carry placeholder literals (`replace_with_message_timetoken` and so on): the client is wrapped with * `withPlaceholders`, so the sample text stays unchanged and the real timetokens reach the service. Every test publishes its * own message first, so the tests do not depend on each other. diff --git a/test/integration/live/api-reference/message-persistence-legacy.test.ts b/test/integration/live/api-reference/message-persistence-legacy.test.ts index 45d063ad6..1340592c0 100644 --- a/test/integration/live/api-reference/message-persistence-legacy.test.ts +++ b/test/integration/live/api-reference/message-persistence-legacy.test.ts @@ -10,7 +10,7 @@ import { eventually, settled, sleep } from '../support/wait'; /** * Examples for the deprecated `history()` method and the deprecated `data` property of `actions` on `fetchMessages()`. - * Port of docs-snippets/api-reference/message-persistence-legacy.ts. Message Persistence and Delete-From-History must be on + * Message Persistence and Delete-From-History must be on * for the open keyset. * * Channels are shared and persistent, and the samples read "the latest 100" of a channel. So every test reads its own diff --git a/test/integration/live/api-reference/message-persistence.test.ts b/test/integration/live/api-reference/message-persistence.test.ts index a28bb78e3..91f86c322 100644 --- a/test/integration/live/api-reference/message-persistence.test.ts +++ b/test/integration/live/api-reference/message-persistence.test.ts @@ -8,7 +8,7 @@ import { newSeedRun, seedPublish, settleSeeded, untilStored } from '../support/h import { eventually, settled } from '../support/wait'; /** - * Message persistence examples. Port of docs-snippets/api-reference/message-persistence.ts. Message Persistence and + * Message persistence examples. Message Persistence and * Delete-From-History must be on for the open keyset. * * The samples carry placeholder literals (`replace-with-start-timetoken` and so on). The client is wrapped with diff --git a/test/integration/live/api-reference/misc.test.ts b/test/integration/live/api-reference/misc.test.ts index 7dff1e118..c99b9dd94 100644 --- a/test/integration/live/api-reference/misc.test.ts +++ b/test/integration/live/api-reference/misc.test.ts @@ -6,7 +6,7 @@ import { LocalProxy } from '../support/proxy'; import { expectStatus, sleep, watch } from '../support/wait'; /** - * Network examples: `disconnect()`, `reconnect()` and `setProxy()`. Port of docs-snippets/api-reference/misc.ts. + * Network examples: `disconnect()`, `reconnect()` and `setProxy()`. * The samples name no channel, so each test listens on a channel of its own name. */ describe('api-reference/misc', () => { diff --git a/test/integration/live/api-reference/mobile-push.test.ts b/test/integration/live/api-reference/mobile-push.test.ts index 583cb845e..64b3f4306 100644 --- a/test/integration/live/api-reference/mobile-push.test.ts +++ b/test/integration/live/api-reference/mobile-push.test.ts @@ -7,7 +7,7 @@ import { eventually, settled, watch } from '../support/wait'; import { sorted } from '../support/util'; /** - * Mobile push notification examples. Port of docs-snippets/api-reference/mobile-push.ts. + * Mobile push notification examples. * Registering a device token does not send a push. Delivery needs a real device and APNs or FCM credentials. * Needs the Push Notifications add-on on the open keyset (no APNs or FCM credentials are needed). * diff --git a/test/integration/live/api-reference/objects.test.ts b/test/integration/live/api-reference/objects.test.ts index 674e1fdbc..4e775e9a5 100644 --- a/test/integration/live/api-reference/objects.test.ts +++ b/test/integration/live/api-reference/objects.test.ts @@ -7,7 +7,7 @@ import { eventually, settled, watch } from '../support/wait'; import { sorted } from '../support/util'; /** - * App Context examples. Port of docs-snippets/api-reference/objects.ts. + * App Context examples. * Needs App Context on the open keyset. Users, channels, memberships and members are shared literal names, so every test * resets the specific resources its sample works on first (reset approach) and then asserts exact lists. The `after` hook * removes what the samples leave behind. diff --git a/test/integration/live/api-reference/presence.test.ts b/test/integration/live/api-reference/presence.test.ts index a55aadabf..76a20d5dd 100644 --- a/test/integration/live/api-reference/presence.test.ts +++ b/test/integration/live/api-reference/presence.test.ts @@ -6,7 +6,7 @@ import { connected, eventually, PRESENCE_TIMEOUT, settled } from '../support/wai import { sorted } from '../support/util'; /** - * Presence examples. Port of docs-snippets/api-reference/presence.ts. + * Presence examples. * * The samples run as the user id `uuid` on the channels `ch1`, `my_channel` and the channel group `cg1`. One occupant client * joins them once in `before` and leaves in `after`, because joining is the slow part of presence. It is built by hand diff --git a/test/integration/live/api-reference/publish-legacy.test.ts b/test/integration/live/api-reference/publish-legacy.test.ts index 1f8c3c060..cda120fb1 100644 --- a/test/integration/live/api-reference/publish-legacy.test.ts +++ b/test/integration/live/api-reference/publish-legacy.test.ts @@ -7,7 +7,7 @@ import { observer, shopper, track } from '../support/clients'; import { sleep, watch } from '../support/wait'; /** - * Examples for the deprecated `fire()` method and its replacement. Port of docs-snippets/api-reference/publish-legacy.ts. + * Examples for the deprecated `fire()` method and its replacement. * The page has no snippet markers: every test is a behaviour check. Channels are unique per run (nothing is stored on them, * and a unique name keeps the "nothing arrived" checks free of other runs' traffic). */ diff --git a/test/integration/live/api-reference/publish.test.ts b/test/integration/live/api-reference/publish.test.ts index 1b5e64fa9..5ac9cc7aa 100644 --- a/test/integration/live/api-reference/publish.test.ts +++ b/test/integration/live/api-reference/publish.test.ts @@ -7,7 +7,7 @@ import { storedMessage } from '../support/history'; import { eventually, settled, sleep, watch } from '../support/wait'; /** - * Publish and signal examples, including `fire()`. Port of docs-snippets/api-reference/publish.ts. + * Publish and signal examples, including `fire()`. * Message Persistence must be on for the open keyset (publishBasicUsage and the ttl test read history). */ describe('api-reference/publish', () => { diff --git a/test/integration/live/api-reference/subscribe-legacy.test.ts b/test/integration/live/api-reference/subscribe-legacy.test.ts index b63fdcdbd..87a8aee96 100644 --- a/test/integration/live/api-reference/subscribe-legacy.test.ts +++ b/test/integration/live/api-reference/subscribe-legacy.test.ts @@ -7,7 +7,7 @@ import { sorted } from '../support/util'; /** * Examples for the older `pubnub.subscribe()`, `pubnub.unsubscribe()` and `pubnub.unsubscribeAll()` methods. - * Port of docs-snippets/api-reference/subscribe-legacy.ts. Presence must be on for the open keyset. + * Presence must be on for the open keyset. */ describe('api-reference/subscribe-legacy', () => { const tagOf = (m: { message: unknown }): string | undefined => (m.message as { run?: string }).run; diff --git a/test/integration/live/api-reference/subscribe.test.ts b/test/integration/live/api-reference/subscribe.test.ts index 25cf2cb62..42ece2fe3 100644 --- a/test/integration/live/api-reference/subscribe.test.ts +++ b/test/integration/live/api-reference/subscribe.test.ts @@ -7,7 +7,7 @@ import { ensureGroups, memberOf, removeGroupMembers } from '../support/subscribe import { sorted } from '../support/util'; /** - * Subscribe examples, without the DataSync examples. Port of docs-snippets/api-reference/subscribe.ts. + * Subscribe examples, without the DataSync examples. * Presence must be on for the open keyset (eventListenerBasicUsage reads join events). */ describe('api-reference/subscribe', () => { diff --git a/test/integration/live/endpoints/data_sync/events.channel.test.ts b/test/integration/live/endpoints/data_sync/events.channel.test.ts index 4ef0f286f..9045955b6 100644 --- a/test/integration/live/endpoints/data_sync/events.channel.test.ts +++ b/test/integration/live/endpoints/data_sync/events.channel.test.ts @@ -3,7 +3,6 @@ * * Channels ride the wire as `type: 'channel'` with `className: 'Channel'` / `classLevel: 'Global'`, * so `objectType` matches the wire type. Delivered on `channel === id`. - * */ import type PubNub from 'pubnub'; diff --git a/test/integration/live/endpoints/data_sync/events.entity.test.ts b/test/integration/live/endpoints/data_sync/events.entity.test.ts index f53a095f1..9a29bc323 100644 --- a/test/integration/live/endpoints/data_sync/events.entity.test.ts +++ b/test/integration/live/endpoints/data_sync/events.entity.test.ts @@ -4,7 +4,6 @@ * Entity-type events are delivered on `channel === object.id`, so we subscribe to the (client-supplied) * id before creating the entity. `JSCustomer` is a developer-defined class, so the wire carries * `className: 'JSCustomer'` / `classLevel: 'SubKey'` and `objectType` stays the wire `'entity'`. - * */ import type PubNub from 'pubnub'; diff --git a/test/integration/live/endpoints/data_sync/events.relationship.test.ts b/test/integration/live/endpoints/data_sync/events.relationship.test.ts index 9c2c6f2ef..24f8b3473 100644 --- a/test/integration/live/endpoints/data_sync/events.relationship.test.ts +++ b/test/integration/live/endpoints/data_sync/events.relationship.test.ts @@ -5,7 +5,6 @@ * (`entityAId` AND `entityBId`), NEVER on the relationship's own id. We therefore subscribe to both * endpoint ids and assert `channel ∈ {customerId, loanQuoteId}`; `data.id` still equals the rel id. * A delete keeps those endpoint ids on the body along with `id` and `deletedAt`. - * */ import assert from 'node:assert'; diff --git a/test/integration/live/endpoints/data_sync/projections.test.ts b/test/integration/live/endpoints/data_sync/projections.test.ts index e0c9df34f..365c5e682 100644 --- a/test/integration/live/endpoints/data_sync/projections.test.ts +++ b/test/integration/live/endpoints/data_sync/projections.test.ts @@ -13,7 +13,6 @@ * | `JSCustomer` | customerId, firstName, lastName, email, creditScore, city | same **minus** `email`, **plus** `private` | * | `JSLoanQuote` | quoteId, make, model, price | quoteId, `private` | * | `JSREQUESTED_BY` | linkedAt | linkedAt, `private` | - * */ import assert from 'node:assert'; diff --git a/test/integration/live/getting-started/logging-file-logger.test.ts b/test/integration/live/getting-started/logging-file-logger.test.ts index 4ddbca09d..e63e93990 100644 --- a/test/integration/live/getting-started/logging-file-logger.test.ts +++ b/test/integration/live/getting-started/logging-file-logger.test.ts @@ -14,7 +14,7 @@ import { openKeys } from '../support/env'; import { eventually } from '../support/wait'; /** - * File logger example for Node.js. Port of docs-snippets/getting-started/logging-file-logger.ts. + * File logger example for Node.js. * It reads the level name from `PubNub.LogLevel` because the SDK is a CommonJS package. */ describe('getting-started/logging-file-logger', () => { diff --git a/test/integration/live/getting-started/logging.test.ts b/test/integration/live/getting-started/logging.test.ts index 790525e0b..b94745c45 100644 --- a/test/integration/live/getting-started/logging.test.ts +++ b/test/integration/live/getting-started/logging.test.ts @@ -20,7 +20,7 @@ import { openKeys } from '../support/env'; import { eventually } from '../support/wait'; /** - * Logging examples. Port of docs-snippets/getting-started/logging.ts. + * Logging examples. * The samples show placeholder keys. Hidden lines spread the real keys of the open keyset over them. */ const LogLevel = PubNub.LogLevel; diff --git a/test/integration/live/getting-started/quickstart-node.test.ts b/test/integration/live/getting-started/quickstart-node.test.ts index 76616a7ff..2cbc0a02c 100644 --- a/test/integration/live/getting-started/quickstart-node.test.ts +++ b/test/integration/live/getting-started/quickstart-node.test.ts @@ -10,7 +10,7 @@ import { makeScratch, SDK_ROOT } from '../support/tsc-scratch'; import { eventually, watch } from '../support/wait'; /** - * Node.js quickstart examples. Port of docs-snippets/getting-started/quickstart-node.ts. + * Node.js quickstart examples. * The docs embed quickstart-node.mjs as a whole file, and the test runs that file in a child process. It sits next to this * test and is a byte for byte copy of the file of the docs. */ diff --git a/test/integration/live/getting-started/set-up-your-environment.test.ts b/test/integration/live/getting-started/set-up-your-environment.test.ts index a9c16b12a..aab42f76f 100644 --- a/test/integration/live/getting-started/set-up-your-environment.test.ts +++ b/test/integration/live/getting-started/set-up-your-environment.test.ts @@ -14,7 +14,7 @@ import { openKeys } from '../support/env'; import { makeScratch, SDK_ROOT, tscScratch } from '../support/tsc-scratch'; /** - * Examples for setting up your environment. Port of docs-snippets/getting-started/set-up-your-environment.ts. + * Examples for setting up your environment. * The import snippet sits at the top because an import cannot sit inside a test. It is type checked by the tier's tsconfig and * loaded when the file runs. */ diff --git a/test/integration/live/getting-started/status-events.test.ts b/test/integration/live/getting-started/status-events.test.ts index 2295fa8e6..9c2b17f67 100644 --- a/test/integration/live/getting-started/status-events.test.ts +++ b/test/integration/live/getting-started/status-events.test.ts @@ -9,7 +9,7 @@ import { LocalProxy } from '../support/proxy'; import { connected, eventually, expectEvent, sleep } from '../support/wait'; /** - * Status event examples. Port of docs-snippets/getting-started/status-events.ts. + * Status event examples. * Outages are simulated with a local CONNECT proxy. The fixed sleeps prove that a call is still pending or that nothing arrived. */ describe('getting-started/status-events', () => { diff --git a/test/integration/live/getting-started/troubleshooting.test.ts b/test/integration/live/getting-started/troubleshooting.test.ts index f7df7ba26..994bac6f7 100644 --- a/test/integration/live/getting-started/troubleshooting.test.ts +++ b/test/integration/live/getting-started/troubleshooting.test.ts @@ -5,7 +5,7 @@ import { createClient, shopper, track } from '../support/clients'; import { keysOf } from '../support/env'; /** - * Troubleshooting examples. Port of docs-snippets/getting-started/troubleshooting.ts. The page has no snippet markers. + * Troubleshooting examples. The page has no snippet markers. * The access denied test needs a secured (pam) keyset on which the secret-less client has no token. */ describe('getting-started/troubleshooting', () => { diff --git a/test/integration/live/migration-guides/v11.test.ts b/test/integration/live/migration-guides/v11.test.ts index 55e3e7fdc..70338b146 100644 --- a/test/integration/live/migration-guides/v11.test.ts +++ b/test/integration/live/migration-guides/v11.test.ts @@ -8,7 +8,7 @@ import { keysOf, openKeys } from '../support/env'; import { connected, eventually, settled } from '../support/wait'; /** - * Examples for the version 11 migration guide. Port of docs-snippets/migration-guides/v11.ts. + * Examples for the version 11 migration guide. * The presence test needs Presence on for the open keyset. */ diff --git a/test/integration/live/migration-guides/v12.test.ts b/test/integration/live/migration-guides/v12.test.ts index 80c181248..f5ee17730 100644 --- a/test/integration/live/migration-guides/v12.test.ts +++ b/test/integration/live/migration-guides/v12.test.ts @@ -9,7 +9,7 @@ import { SDK_ROOT } from '../support/tsc-scratch'; import { LocalProxy } from '../support/proxy'; /** - * Examples for the version 12 migration guide. Port of docs-snippets/migration-guides/v12.ts. The page has no snippet markers. + * Examples for the version 12 migration guide. The page has no snippet markers. */ describe('migration-guides/v12', () => { it('migrationV12ProxyEnvironmentVariablesIgnored: the SDK ignores proxy environment variables, shown by zero tunnel requests at the local proxy', async function () { diff --git a/test/integration/live/migration-guides/v8.test.ts b/test/integration/live/migration-guides/v8.test.ts index 3e7e1a347..55454fedc 100644 --- a/test/integration/live/migration-guides/v8.test.ts +++ b/test/integration/live/migration-guides/v8.test.ts @@ -8,7 +8,7 @@ import PubNub, { AppContext } from 'pubnub' import { tscScratch } from '../support/tsc-scratch'; /** - * Examples for the version 8 migration guide. Port of docs-snippets/migration-guides/v8.ts. + * Examples for the version 8 migration guide. * The import snippet sits at the top because an import cannot sit inside a test. It is type checked by the tier's tsconfig * (npm run test:integration:live:types) and loaded when the file runs. The scratch projects of the second test show what the * guide claims about the "before" import. diff --git a/test/integration/live/provision/provision-datasync.ts b/test/integration/live/provision/provision-datasync.ts index c52f4cff1..a0d6c193b 100644 --- a/test/integration/live/provision/provision-datasync.ts +++ b/test/integration/live/provision/provision-datasync.ts @@ -9,7 +9,7 @@ import { describeRoles, env as currentEnv, redact, type Env } from '../support/e * npm run test:integration:live:provision -- --check no-write diff, exit code 1 when anything differs * * The live tier does NOT run it: run it by hand when a keyset is new or the classes changed (the preflight cannot see - * classes). Working copy of docs-snippets/provision/provision-datasync.ts. + * classes). * * Rules: * - Never deletes a class. A class whose immutable fields differ (relationship cardinality or entity sides) fails with the diff --git a/docs-snippets/quickstarts/browser-download-file.ts b/test/integration/live/quickstarts/browser-download-file.ts similarity index 100% rename from docs-snippets/quickstarts/browser-download-file.ts rename to test/integration/live/quickstarts/browser-download-file.ts diff --git a/docs-snippets/quickstarts/browser-shared-worker.ts b/test/integration/live/quickstarts/browser-shared-worker.ts similarity index 100% rename from docs-snippets/quickstarts/browser-shared-worker.ts rename to test/integration/live/quickstarts/browser-shared-worker.ts diff --git a/docs-snippets/quickstarts/quickstart-react-app.tsx b/test/integration/live/quickstarts/quickstart-react-app.tsx similarity index 100% rename from docs-snippets/quickstarts/quickstart-react-app.tsx rename to test/integration/live/quickstarts/quickstart-react-app.tsx diff --git a/docs-snippets/quickstarts/quickstart-react-native-app.tsx b/test/integration/live/quickstarts/quickstart-react-native-app.tsx similarity index 100% rename from docs-snippets/quickstarts/quickstart-react-native-app.tsx rename to test/integration/live/quickstarts/quickstart-react-native-app.tsx diff --git a/docs-snippets/quickstarts/quickstart-react-native.tsx b/test/integration/live/quickstarts/quickstart-react-native.tsx similarity index 100% rename from docs-snippets/quickstarts/quickstart-react-native.tsx rename to test/integration/live/quickstarts/quickstart-react-native.tsx diff --git a/docs-snippets/quickstarts/quickstart-react.tsx b/test/integration/live/quickstarts/quickstart-react.tsx similarity index 100% rename from docs-snippets/quickstarts/quickstart-react.tsx rename to test/integration/live/quickstarts/quickstart-react.tsx diff --git a/docs-snippets/quickstarts/quickstart-web.html b/test/integration/live/quickstarts/quickstart-web.html similarity index 100% rename from docs-snippets/quickstarts/quickstart-web.html rename to test/integration/live/quickstarts/quickstart-web.html diff --git a/docs-snippets/quickstarts/quickstart-web.ts b/test/integration/live/quickstarts/quickstart-web.ts similarity index 100% rename from docs-snippets/quickstarts/quickstart-web.ts rename to test/integration/live/quickstarts/quickstart-web.ts diff --git a/docs-snippets/quickstarts/react-configuration.jsx b/test/integration/live/quickstarts/react-configuration.jsx similarity index 100% rename from docs-snippets/quickstarts/react-configuration.jsx rename to test/integration/live/quickstarts/react-configuration.jsx diff --git a/docs-snippets/quickstarts/react-fetch-messages.jsx b/test/integration/live/quickstarts/react-fetch-messages.jsx similarity index 100% rename from docs-snippets/quickstarts/react-fetch-messages.jsx rename to test/integration/live/quickstarts/react-fetch-messages.jsx diff --git a/docs-snippets/quickstarts/react-get-state.jsx b/test/integration/live/quickstarts/react-get-state.jsx similarity index 100% rename from docs-snippets/quickstarts/react-get-state.jsx rename to test/integration/live/quickstarts/react-get-state.jsx diff --git a/docs-snippets/quickstarts/react-here-now.jsx b/test/integration/live/quickstarts/react-here-now.jsx similarity index 100% rename from docs-snippets/quickstarts/react-here-now.jsx rename to test/integration/live/quickstarts/react-here-now.jsx diff --git a/docs-snippets/quickstarts/react-listeners.jsx b/test/integration/live/quickstarts/react-listeners.jsx similarity index 100% rename from docs-snippets/quickstarts/react-listeners.jsx rename to test/integration/live/quickstarts/react-listeners.jsx diff --git a/docs-snippets/quickstarts/react-native-download-file.ts b/test/integration/live/quickstarts/react-native-download-file.ts similarity index 100% rename from docs-snippets/quickstarts/react-native-download-file.ts rename to test/integration/live/quickstarts/react-native-download-file.ts diff --git a/docs-snippets/quickstarts/react-presence-events.jsx b/test/integration/live/quickstarts/react-presence-events.jsx similarity index 100% rename from docs-snippets/quickstarts/react-presence-events.jsx rename to test/integration/live/quickstarts/react-presence-events.jsx diff --git a/docs-snippets/quickstarts/react-set-state.jsx b/test/integration/live/quickstarts/react-set-state.jsx similarity index 100% rename from docs-snippets/quickstarts/react-set-state.jsx rename to test/integration/live/quickstarts/react-set-state.jsx diff --git a/docs-snippets/quickstarts/react-subscribe.jsx b/test/integration/live/quickstarts/react-subscribe.jsx similarity index 100% rename from docs-snippets/quickstarts/react-subscribe.jsx rename to test/integration/live/quickstarts/react-subscribe.jsx diff --git a/docs-snippets/quickstarts/react-where-now.jsx b/test/integration/live/quickstarts/react-where-now.jsx similarity index 100% rename from docs-snippets/quickstarts/react-where-now.jsx rename to test/integration/live/quickstarts/react-where-now.jsx diff --git a/test/integration/live/shared/import-pubnub.test.ts b/test/integration/live/shared/import-pubnub.test.ts index 02d019d24..840f94670 100644 --- a/test/integration/live/shared/import-pubnub.test.ts +++ b/test/integration/live/shared/import-pubnub.test.ts @@ -11,7 +11,7 @@ import { openKeys } from '../support/env'; /** * Shared examples used next to the examples of many pages: the SDK import line, the `fs` import for Node.js file sharing and - * the generic client constructor. Port of docs-snippets/shared/import-pubnub.ts. + * the generic client constructor. * The two import snippets sit at the top of the file because an import cannot sit inside a test. They run when the file loads * and the tests below prove that the imported names are the real thing. */ diff --git a/test/integration/live/snippet-exceptions.json b/test/integration/live/snippet-exceptions.json index cfeb16712..67d093410 100644 --- a/test/integration/live/snippet-exceptions.json +++ b/test/integration/live/snippet-exceptions.json @@ -1,22 +1,22 @@ { - "$comment": "Hand maintained. Things the public docs embed from this repository that are deliberately NOT run against the service, each with a reason. An entry is keyed by the docs-facing path (the path in the EmbeddedCode url, as the docs use it today) and the snippet ids; an entry without \"ids\" covers every embed of that path (whole-file embeds). Seeded from docs-snippets/coverage.json (modes stub and compile-only of the quickstarts). npm run test:integration:live:docs-check reads it.", + "$comment": "Hand maintained. Things the public docs embed from this repository that are deliberately NOT run against the service, each with a reason. An entry is keyed by the docs-facing path (the path in the EmbeddedCode url) and the snippet ids; an entry without \"ids\" covers every embed of that path (whole-file embeds). npm run test:integration:live:docs-check reads it.", "exceptions": [ { - "path": "docs-snippets/quickstarts/browser-download-file.ts", + "path": "test/integration/live/quickstarts/browser-download-file.ts", "ids": [ "downloadFileWebBasicUsage" ], - "reason": "Browser-only sample (needs window and a bundler or a script tag), so it cannot run in Node. Stays in docs-snippets/quickstarts/ and is type-checked by docs-snippets/tsconfig.browser.json (npm run test:docs-snippets:types); it is not run against the service. Old mode: compile-only." + "reason": "Browser-only sample (needs window and a bundler or a script tag), so it cannot run in Node. Stays in test/integration/live/quickstarts/ and is type-checked by test/integration/live/tsconfig.browser.json (npm run test:integration:live:types); it is not run against the service." }, { - "path": "docs-snippets/quickstarts/browser-shared-worker.ts", + "path": "test/integration/live/quickstarts/browser-shared-worker.ts", "ids": [ "configurationBasicUsageSubscriptionWorkerUrl" ], - "reason": "Browser-only sample (needs window and a bundler or a script tag), so it cannot run in Node. Stays in docs-snippets/quickstarts/ and is type-checked by docs-snippets/tsconfig.browser.json (npm run test:docs-snippets:types); it is not run against the service. Old mode: compile-only." + "reason": "Browser-only sample (needs window and a bundler or a script tag), so it cannot run in Node. Stays in test/integration/live/quickstarts/ and is type-checked by test/integration/live/tsconfig.browser.json (npm run test:integration:live:types); it is not run against the service." }, { - "path": "docs-snippets/quickstarts/quickstart-react-native.tsx", + "path": "test/integration/live/quickstarts/quickstart-react-native.tsx", "ids": [ "quickstartReactNativeCleanup", "quickstartReactNativeImport", @@ -24,109 +24,109 @@ "quickstartReactNativePublish", "quickstartReactNativeSubscription" ], - "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in docs-snippets/quickstarts/. Old mode: stub." + "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in test/integration/live/quickstarts/." }, { - "path": "docs-snippets/quickstarts/quickstart-react.tsx", + "path": "test/integration/live/quickstarts/quickstart-react.tsx", "ids": [ "quickstartReactImports", "quickstartReactInit", "quickstartReactPublish", "quickstartReactSubscribe" ], - "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in docs-snippets/quickstarts/. Old mode: stub." + "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in test/integration/live/quickstarts/." }, { - "path": "docs-snippets/quickstarts/quickstart-web.ts", + "path": "test/integration/live/quickstarts/quickstart-web.ts", "ids": [ "quickstartWebInit", "quickstartWebListeners", "quickstartWebPublish", "quickstartWebSubscribe" ], - "reason": "Browser-only sample (needs window and a bundler or a script tag), so it cannot run in Node. Stays in docs-snippets/quickstarts/ and is type-checked by docs-snippets/tsconfig.browser.json (npm run test:docs-snippets:types); it is not run against the service. Old mode: compile-only." + "reason": "Browser-only sample (needs window and a bundler or a script tag), so it cannot run in Node. Stays in test/integration/live/quickstarts/ and is type-checked by test/integration/live/tsconfig.browser.json (npm run test:integration:live:types); it is not run against the service." }, { - "path": "docs-snippets/quickstarts/react-configuration.jsx", + "path": "test/integration/live/quickstarts/react-configuration.jsx", "ids": [ "reactConfigurationLifecycle" ], - "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in docs-snippets/quickstarts/. Old mode: stub." + "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in test/integration/live/quickstarts/." }, { - "path": "docs-snippets/quickstarts/react-fetch-messages.jsx", + "path": "test/integration/live/quickstarts/react-fetch-messages.jsx", "ids": [ "reactFetchMessages" ], - "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in docs-snippets/quickstarts/. Old mode: stub." + "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in test/integration/live/quickstarts/." }, { - "path": "docs-snippets/quickstarts/react-get-state.jsx", + "path": "test/integration/live/quickstarts/react-get-state.jsx", "ids": [ "reactGetState" ], - "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in docs-snippets/quickstarts/. Old mode: stub." + "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in test/integration/live/quickstarts/." }, { - "path": "docs-snippets/quickstarts/react-here-now.jsx", + "path": "test/integration/live/quickstarts/react-here-now.jsx", "ids": [ "reactHereNow" ], - "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in docs-snippets/quickstarts/. Old mode: stub." + "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in test/integration/live/quickstarts/." }, { - "path": "docs-snippets/quickstarts/react-listeners.jsx", + "path": "test/integration/live/quickstarts/react-listeners.jsx", "ids": [ "reactListeners" ], - "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in docs-snippets/quickstarts/. Old mode: stub." + "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in test/integration/live/quickstarts/." }, { - "path": "docs-snippets/quickstarts/react-native-download-file.ts", + "path": "test/integration/live/quickstarts/react-native-download-file.ts", "ids": [ "downloadFileReactNativeBasicUsage" ], - "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in docs-snippets/quickstarts/. Old mode: stub." + "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in test/integration/live/quickstarts/." }, { - "path": "docs-snippets/quickstarts/react-presence-events.jsx", + "path": "test/integration/live/quickstarts/react-presence-events.jsx", "ids": [ "reactPresenceEvents" ], - "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in docs-snippets/quickstarts/. Old mode: stub." + "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in test/integration/live/quickstarts/." }, { - "path": "docs-snippets/quickstarts/react-set-state.jsx", + "path": "test/integration/live/quickstarts/react-set-state.jsx", "ids": [ "reactSetState" ], - "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in docs-snippets/quickstarts/. Old mode: stub." + "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in test/integration/live/quickstarts/." }, { - "path": "docs-snippets/quickstarts/react-subscribe.jsx", + "path": "test/integration/live/quickstarts/react-subscribe.jsx", "ids": [ "reactSubscribe" ], - "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in docs-snippets/quickstarts/. Old mode: stub." + "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in test/integration/live/quickstarts/." }, { - "path": "docs-snippets/quickstarts/react-where-now.jsx", + "path": "test/integration/live/quickstarts/react-where-now.jsx", "ids": [ "reactWhereNow" ], - "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in docs-snippets/quickstarts/. Old mode: stub." + "reason": "React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in test/integration/live/quickstarts/." }, { - "path": "docs-snippets/quickstarts/quickstart-web.html", - "reason": "Whole-file embed of the browser quickstart page (script tag, DOM, CDN build): needs a browser. Stays in docs-snippets/quickstarts/; not run, not type-checked. Its snippets are the same code as quickstart-web.ts." + "path": "test/integration/live/quickstarts/quickstart-web.html", + "reason": "Whole-file embed of the browser quickstart page (script tag, DOM, CDN build): needs a browser. Stays in test/integration/live/quickstarts/; not run, not type-checked. Its snippets are the same code as quickstart-web.ts." }, { - "path": "docs-snippets/quickstarts/quickstart-react-app.tsx", - "reason": "Whole-file embed of the React quickstart app: React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in docs-snippets/quickstarts/. Old mode: stub." + "path": "test/integration/live/quickstarts/quickstart-react-app.tsx", + "reason": "Whole-file embed of the React quickstart app: React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in test/integration/live/quickstarts/." }, { - "path": "docs-snippets/quickstarts/quickstart-react-native-app.tsx", - "reason": "Whole-file embed of the React Native quickstart app: React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in docs-snippets/quickstarts/. Old mode: stub." + "path": "test/integration/live/quickstarts/quickstart-react-native-app.tsx", + "reason": "Whole-file embed of the React Native quickstart app: React / React Native sample: this repository has no React or React Native dependency, so the sample can be shown but not run or type-checked here. Stays in test/integration/live/quickstarts/." } ] } diff --git a/test/integration/live/support/compare-snippets.ts b/test/integration/live/support/compare-snippets.ts deleted file mode 100644 index 689e8fb92..000000000 --- a/test/integration/live/support/compare-snippets.ts +++ /dev/null @@ -1,51 +0,0 @@ -/** - * Equivalence check of a ported page: extracts every snippet of the old file and of the new file and compares the text per id. - * Every id must exist in both and be byte-identical. Exit code 0 when all match, 1 otherwise. - * - * npx tsx test/integration/live/support/compare-snippets.ts - * - * It will grow into the marker lint. - */ -import fs from 'node:fs'; -import { extractSnippets } from './snippets'; - -function main(): number { - const [oldFile, newFile] = process.argv.slice(2); - if (!oldFile || !newFile) { - console.error('usage: compare-snippets.ts '); - return 2; - } - const before = extractSnippets(fs.readFileSync(oldFile, 'utf8')); - const after = extractSnippets(fs.readFileSync(newFile, 'utf8')); - let failures = 0; - - for (const [label, result] of [ - [oldFile, before], - [newFile, after], - ] as const) { - for (const problem of result.problems) { - failures += 1; - console.log(`PROBLEM ${label}: ${problem}`); - } - } - for (const id of new Set([...before.snippets.keys(), ...after.snippets.keys()])) { - const a = before.snippets.get(id); - const b = after.snippets.get(id); - if (a === undefined) { - failures += 1; - console.log(`EXTRA ${id}: only in the new file`); - } else if (b === undefined) { - failures += 1; - console.log(`MISSING ${id}: not in the new file`); - } else if (a !== b) { - failures += 1; - console.log(`DIFFERS ${id}\n--- old\n${a}\n--- new\n${b}`); - } else { - console.log(`SAME ${id} (${Buffer.byteLength(a)} bytes)`); - } - } - console.log(failures === 0 ? `OK: ${before.snippets.size} snippets identical` : `FAILED: ${failures} problem(s)`); - return failures === 0 ? 0 : 1; -} - -process.exitCode = main(); diff --git a/test/integration/live/support/docs-check.ts b/test/integration/live/support/docs-check.ts index 28c6faeb1..df659e660 100644 --- a/test/integration/live/support/docs-check.ts +++ b/test/integration/live/support/docs-check.ts @@ -1,18 +1,15 @@ /** * Docs cross-check: does every snippet the public docs embed from this repository map to a test that runs it? * - * npm run test:integration:live:docs-check -- [] [--report-only] [--out ] + * npm run test:integration:live:docs-check -- [] [--report-only] * * The checkout comes from the first non-flag argument, then $DOCS_REPO, then ../documentation next to this repository. * It parses every tag whose url points at pubnub/javascript and classifies each embedded id (or whole-file - * embed) as live / compile-only / mocked / exception (snippet-exceptions.json) / NOT COVERED. Two URL styles are understood: - * an old-style url (`docs-snippets//.ts`) is mapped to `test/integration/live//.test.ts`; a - * new-style url (a path under `test/integration/live/`, or one of the mocked test files) is checked as it stands: the file must - * exist and the coverage manifest must list each requested id for that very file. Exit code 1 when anything is NOT COVERED, - * unless --report-only. It writes the repoint list for a later mechanical edit of the docs URLs (default - * /tmp/pn-live/docs-repoint.json). The documentation checkout is only read. No network. + * embed) as live / compile-only / mocked / exception (snippet-exceptions.json) / NOT COVERED. The file named by the url must + * be a path under `test/integration/live/` or one of the mocked test files, it must exist, and the coverage manifest must list + * each requested id for that very file; any other path is NOT COVERED. Exit code 1 when anything is NOT COVERED, unless + * --report-only. The documentation checkout is only read. No network. */ -import { spawnSync } from 'node:child_process'; import fs from 'node:fs'; import path from 'node:path'; import { loadExceptions, type ExceptionEntry, type Manifest } from './manifest'; @@ -28,16 +25,14 @@ export const OUT_OF_SCOPE_PAGES = [ 'docs/sdks/javascript/api-reference/access-manager-v2.md', ]; -const TMP_DIR = '/tmp/pn-live'; - export interface Embed { docsFile: string; /** Line of `/.ts` becomes `test/integration/live//.test.ts`; other extensions keep their name. */ -export function mapPath(oldPath: string): string | undefined { - const m = /^docs-snippets\/(.+)$/.exec(oldPath); - if (!m) return undefined; - const rest = m[1]; - return `${LIVE_REL}/${/\.ts$/.test(rest) && !/\.d\.ts$/.test(rest) ? rest.replace(/\.ts$/, '.test.ts') : rest}`; -} - -function readIfExists(rel: string): string | undefined { - const abs = path.join(REPO_ROOT, rel); - return fs.existsSync(abs) ? fs.readFileSync(abs, 'utf8') : undefined; -} - -/** The old copies wrap the call in `try { ... } catch (e) { console.error(...) }`. Removes that wrapper (and two spaces of indent). */ -function unwrapTry(text: string): string | undefined { - const lines = text.split('\n'); - const open = lines.findIndex((l) => l === 'try {'); - const close = lines.findIndex((l, i) => i > open && /^\} catch \(\w+\) \{$/.test(l)); - if (open === -1 || close === -1) return undefined; - const body = lines.slice(open + 1, close).map((l) => (l.startsWith(' ') ? l.slice(2) : l)); - return [...lines.slice(0, open), ...body].join('\n'); -} - -function slug(s: string): string { - return s.replace(/[^A-Za-z0-9_-]+/g, '_'); -} - function exceptionFor( exceptions: ExceptionEntry[], - oldPath: string, + filePath: string, id: string | null, used: Set, ): ExceptionEntry | undefined { const hit = exceptions.find( - (e) => e.path === oldPath && (e.ids === undefined || (id !== null && e.ids.includes(id))), + (e) => e.path === filePath && (e.ids === undefined || (id !== null && e.ids.includes(id))), ); if (hit) used.add(hit); return hit; } -/** New-style url: the file is named by the url itself, so it is checked as it stands (no mapping, no old copy to compare). */ -function classifyNewStyle( +/** The file is named by the url itself, so it is checked as it stands. */ +function classifyTested( embed: Embed, manifest: Manifest, exceptions: ExceptionEntry[], used: Set, ): IdResult[] { - const file = embed.oldPath; + const file = embed.path; const exists = fs.existsSync(path.join(REPO_ROOT, file)); if (embed.wholeFile) { const sample = manifest.files[file]; @@ -200,7 +165,7 @@ function classifyNewStyle( return embed.ids.map((id): IdResult => { const entry = manifest.snippets[id]; if (exists && entry && entry.file === file) { - if (docsExtract(readIfExists(file) ?? '', id) === undefined) { + if (docsExtract(fs.readFileSync(path.join(REPO_ROOT, file), 'utf8'), id) === undefined) { return { id, status: 'NOT COVERED', @@ -230,98 +195,11 @@ export function classify( exceptions: ExceptionEntry[], used: Set, ): IdResult[] { - if (isNewStylePath(embed.oldPath)) return classifyNewStyle(embed, manifest, exceptions, used); - const target = mapPath(embed.oldPath); - const results: IdResult[] = []; - if (embed.wholeFile) { - const sample = target ? manifest.files[target] : undefined; - if (sample) { - const old = readIfExists(embed.oldPath); - const now = readIfExists(target!); - results.push({ - id: null, - status: 'live', - detail: `${sample.testFile} :: ${sample.test}`, - textMatch: old === undefined ? 'old-missing' : old === now ? 'identical' : 'different', - }); - return results; - } - const ex = exceptionFor(exceptions, embed.oldPath, null, used); - if (ex) return [{ id: null, status: 'exception', detail: ex.reason }]; - const reason = !target - ? `path is not under docs-snippets/ (${embed.oldPath})` - : fs.existsSync(path.join(REPO_ROOT, target)) - ? `whole-file embed of ${target}, which no test runs as a file (a whole-file embed needs a sample file such as quickstart-node.mjs that a test executes)` - : `whole-file embed: no sample file ${target}`; - return [{ id: null, status: 'NOT COVERED', reason }]; - } - for (const id of embed.ids) { - const entry = manifest.snippets[id]; - if (target && entry && entry.file === target) { - const old = readIfExists(embed.oldPath); - const oldText = old === undefined ? undefined : docsExtract(old, id); - const newText = docsExtract(readIfExists(target) ?? '', id); - if (newText === undefined) { - results.push({ - id, - status: 'NOT COVERED', - reason: `${target} has the marker but the docs extractor finds no text for "${id}" (empty or malformed)`, - }); - continue; - } - results.push({ - id, - status: entry.mode, - detail: entry.test ?? 'import-only, loaded with the file', - textMatch: oldText === undefined ? 'old-missing' : oldText === newText ? 'identical' : 'different', - }); - continue; - } - const ex = exceptionFor(exceptions, embed.oldPath, id, used); - if (ex) { - results.push({ id, status: 'exception', detail: ex.reason }); - continue; - } - let reason: string; - if (!target) reason = `path is not under docs-snippets/ (${embed.oldPath})`; - else if (!fs.existsSync(path.join(REPO_ROOT, target))) reason = `${target} does not exist`; - else reason = `id is not in ${target}`; - const result: IdResult = { id, status: 'NOT COVERED', reason }; - if (entry) { - // The same id is tested in another file: an old duplicate copy. - const oldText = (() => { - const old = readIfExists(embed.oldPath); - return old === undefined ? undefined : docsExtract(old, id); - })(); - const twinText = docsExtract(readIfExists(entry.file) ?? '', id); - const twin: NonNullable = { - file: entry.file, - identical: oldText === undefined || twinText === undefined ? null : oldText === twinText, - }; - if (twin.identical === false && oldText !== undefined) twin.tryOnly = unwrapTry(oldText) === twinText; - if (twin.identical === false) { - fs.mkdirSync(path.join(TMP_DIR, 'dup-diffs'), { recursive: true }); - const base = path.join( - TMP_DIR, - 'dup-diffs', - `${slug(embed.oldPath.replace(/^docs-snippets\//, ''))}__${slug(id)}`, - ); - fs.writeFileSync(`${base}.old.txt`, `${oldText}\n`); - fs.writeFileSync(`${base}.new.txt`, `${twinText}\n`); - const diff = spawnSync( - 'diff', - ['-u', `--label=old:${embed.oldPath}`, `--label=tested:${entry.file}`, `${base}.old.txt`, `${base}.new.txt`], - { encoding: 'utf8' }, - ); - fs.writeFileSync(`${base}.diff`, diff.stdout); - twin.diffFile = `${base}.diff`; - } - result.twin = twin; - result.reason = `${reason}; the same id is ${entry.mode} in ${entry.file}`; - } - results.push(result); - } - return results; + if (isNewStylePath(embed.path)) return classifyTested(embed, manifest, exceptions, used); + const reason = `path is not under ${LIVE_REL}/ (${embed.path})`; + return embed.wholeFile + ? [{ id: null, status: 'NOT COVERED', reason }] + : embed.ids.map((id): IdResult => ({ id, status: 'NOT COVERED', reason })); } function worst(results: IdResult[]): Status { @@ -332,9 +210,7 @@ function worst(results: IdResult[]): Status { export function main(argv: string[]): number { const reportOnly = argv.includes('--report-only'); - const outIndex = argv.indexOf('--out'); - const out = outIndex >= 0 ? argv[outIndex + 1] : path.join(TMP_DIR, 'docs-repoint.json'); - const positional = argv.filter((a, i) => !a.startsWith('--') && argv[i - 1] !== '--out'); + const positional = argv.filter((a) => !a.startsWith('--')); const defaultDir = path.resolve(REPO_ROOT, '../documentation'); const docsDir = path.resolve(positional[0] ?? process.env.DOCS_REPO ?? (fs.existsSync(defaultDir) ? defaultDir : '')); if (!docsDir || !fs.existsSync(docsDir)) { @@ -384,10 +260,6 @@ export function main(argv: string[]): number { console.log(`${pad('status', 14)}${pad('embeds', 8)}ids/whole files`); for (const s of STATUSES) console.log(`${pad(s, 14)}${pad(embedCount[s], 8)}${refCount[s]}`); console.log(`${pad('total', 14)}${pad(rows.length, 8)}${rows.reduce((n, r) => n + r.results.length, 0)}`); - const alreadyNew = rows.filter((r) => isNewStylePath(r.embed.oldPath)).length; - console.log( - `new-style urls (already point at ${LIVE_REL}/ or a mocked test file): ${alreadyNew} of ${rows.length} embeds`, - ); problems.forEach((p) => console.log(`note: ${p}`)); // NOT COVERED, grouped by docs file. @@ -400,145 +272,17 @@ export function main(argv: string[]): number { console.log(` ${file}`); for (const r of list) { for (const x of r.results.filter((y) => y.status === 'NOT COVERED')) { - console.log(` :${r.embed.line} ${r.embed.oldPath} ${x.id ?? '(whole file)'} ${x.reason ?? ''}`); + console.log(` :${r.embed.line} ${r.embed.path} ${x.id ?? '(whole file)'} ${x.reason ?? ''}`); } } } } - // Duplicate copies: NOT COVERED embeds whose id is tested in another file. - const twins = rows.flatMap((r) => r.results.filter((x) => x.twin).map((x) => ({ embed: r.embed, x }))); - if (twins.length > 0) { - const seen = new Set(); - console.log('\nDuplicate copies (old copy vs the tested twin, text extracted with the docs rules)'); - for (const { embed, x } of twins) { - const key = `${embed.oldPath}#${x.id}`; - if (seen.has(key)) continue; - seen.add(key); - const verdict = - x.twin!.identical === null - ? 'UNKNOWN (old copy missing)' - : x.twin!.identical - ? 'IDENTICAL' - : x.twin!.tryOnly - ? 'DIFFERENT (only the try/catch wrapper of the old copy)' - : 'DIFFERENT'; - console.log( - ` ${verdict} ${embed.oldPath} :: ${x.id} -> ${x.twin!.file}${x.twin!.diffFile ? ` (diff: ${x.twin!.diffFile})` : ''}`, - ); - } - } - - // Text of directly mapped embeds. - const differing = rows.flatMap((r) => - r.results.filter((x) => x.textMatch === 'different').map((x) => `${r.embed.oldPath} :: ${x.id ?? '(whole file)'}`), - ); - const compared = rows.flatMap((r) => - r.results.filter((x) => x.textMatch === 'identical' || x.textMatch === 'different'), - ); - console.log( - `\ntext of old-style embeds vs the file they map to (docs extractor rules; embeds that already use the new path are not compared): ${compared.length - differing.length} identical, ${differing.length} different`, - ); - [...new Set(differing)].forEach((d) => console.log(` DIFFERENT ${d}`)); - const unused = exceptions.filter((e) => !used.has(e)); unused.forEach((e) => console.log(`note: exception for ${e.path}${e.ids ? ` [${e.ids.join(', ')}]` : ''} matches no embed in the docs`), ); - // Repoint file. - const repoint = rows.map(({ embed, results, status }) => { - if (isNewStylePath(embed.oldPath)) { - // Already repointed: nothing to change (an exception still reads `keep`). - const done = status === 'exception' ? 'keep' : status === 'NOT COVERED' ? 'blocked' : 'done'; - return { - docsFile: embed.docsFile, - line: embed.line, - urlLine: embed.urlLine, - oldUrl: embed.oldUrl, - oldPath: embed.oldPath, - newPath: done === 'blocked' ? null : embed.oldPath, - newUrl: done === 'blocked' ? null : embed.oldUrl, - ids: embed.ids, - wholeFile: embed.wholeFile, - status, - action: done, - ...(done === 'blocked' ? { wouldBePath: null, twinFile: null } : {}), - perId: results.map((r) => ({ id: r.id, status: r.status })), - }; - } - const target = mapPath(embed.oldPath); - const twinFiles = [...new Set(results.filter((r) => r.status === 'NOT COVERED').map((r) => r.twin?.file))]; - // An embed that points at an old duplicate copy is repointed to the tested twin (the docs owner decided so), when every id - // that is not covered has a twin and all twins sit in one file. Anything else stays blocked. - const toTwin = status === 'NOT COVERED' && twinFiles.length === 1 && twinFiles[0] !== undefined; - const twinFile = toTwin ? twinFiles[0] : results.find((r) => r.twin)?.twin?.file; - const action = toTwin - ? 'repoint-to-twin' - : status === 'NOT COVERED' - ? 'blocked' - : status === 'exception' - ? 'keep' - : 'repoint'; - const newPath = - action === 'repoint' - ? (target ?? null) - : action === 'keep' - ? embed.oldPath - : action === 'repoint-to-twin' - ? twinFile! - : null; - return { - docsFile: embed.docsFile, - line: embed.line, - urlLine: embed.urlLine, - oldUrl: embed.oldUrl, - oldPath: embed.oldPath, - newPath, - newUrl: newPath ? embed.oldUrl.replace(embed.oldPath, newPath) : null, - ids: embed.ids, - wholeFile: embed.wholeFile, - status, - action, - ...(action === 'blocked' ? { wouldBePath: target ?? null, twinFile: twinFile ?? null } : {}), - perId: results.map((r) => ({ - id: r.id, - status: r.status, - ...(r.twin - ? { - twin: r.twin.file, - twinText: - r.twin.identical === null - ? 'unknown' - : r.twin.identical - ? 'IDENTICAL' - : r.twin.tryOnly - ? 'DIFFERENT-try-catch-wrapper-only' - : 'DIFFERENT', - } - : {}), - })), - }; - }); - fs.mkdirSync(path.dirname(out), { recursive: true }); - fs.writeFileSync( - out, - `${JSON.stringify( - { - $comment: - 'Generated by test/integration/live/support/docs-check.ts. action: repoint = change oldPath to newPath in the url; repoint-to-twin = the embed points at an old duplicate copy, change oldPath to newPath, the path of the tested twin file (decided by the docs owner); keep = documented exception, url stays; done = the url already points at the tested tier; blocked = not covered and no twin.', - docsDir, - embeds: repoint, - outOfScope: skipped.map((e) => ({ docsFile: e.docsFile, line: e.line, oldPath: e.oldPath, ids: e.ids })), - }, - null, - 2, - )}\n`, - ); - console.log( - `\nrepoint list: ${out} (${repoint.filter((r) => r.action === 'repoint').length} repoint, ${repoint.filter((r) => r.action === 'repoint-to-twin').length} repoint-to-twin, ${repoint.filter((r) => r.action === 'keep').length} keep, ${repoint.filter((r) => r.action === 'done').length} done, ${repoint.filter((r) => r.action === 'blocked').length} blocked)`, - ); - const failed = embedCount['NOT COVERED'] > 0; if (failed) console.log( diff --git a/test/integration/live/support/manifest.ts b/test/integration/live/support/manifest.ts index 3c2afd552..3fd83c1e0 100644 --- a/test/integration/live/support/manifest.ts +++ b/test/integration/live/support/manifest.ts @@ -41,7 +41,7 @@ export interface Manifest { } export interface ExceptionEntry { - /** Docs-facing path, as in the EmbeddedCode url (for example docs-snippets/quickstarts/quickstart-web.ts). */ + /** Docs-facing path, as in the EmbeddedCode url (for example test/integration/live/quickstarts/quickstart-web.ts). */ path: string; /** Snippet ids. Absent: every embed of the path, whole-file embeds included. */ ids?: string[]; @@ -60,8 +60,8 @@ export function checkExceptions(entries: ExceptionEntry[]): string[] { const seen = new Set(); entries.forEach((e, i) => { const where = `snippet-exceptions.json entry ${i + 1} (${e.path ?? '?'})`; - if (typeof e.path !== 'string' || !e.path.startsWith('docs-snippets/')) - problems.push(`${where}: "path" must be a docs-snippets/... path`); + if (typeof e.path !== 'string' || !e.path.startsWith('test/integration/live/')) + problems.push(`${where}: "path" must be a test/integration/live/... path`); if (typeof e.reason !== 'string' || e.reason.trim().length < 10) problems.push(`${where}: "reason" is required`); if ( e.ids !== undefined && diff --git a/test/integration/live/support/preflight.ts b/test/integration/live/support/preflight.ts index 4a9d89901..7dcd53d88 100644 --- a/test/integration/live/support/preflight.ts +++ b/test/integration/live/support/preflight.ts @@ -68,7 +68,7 @@ export async function readFeatures(environment: Env, role: Role): Promise { @@ -196,7 +196,7 @@ export async function runPreflight(environment: Env = env()): Promise<{ notes: s throw new Error( redact( `Keyset preflight failed. Fix these, then run again:\n${problems.map((p) => ` - ${p}`).join('\n')}\n` + - 'The features each keyset needs are listed in docs-snippets/README.md ("Features each keyset needs").', + 'The features each keyset needs are listed in test/integration/live/README.md ("Preflight").', ), ); } diff --git a/test/integration/live/support/resources.ts b/test/integration/live/support/resources.ts index d259086ce..b38023cfc 100644 --- a/test/integration/live/support/resources.ts +++ b/test/integration/live/support/resources.ts @@ -5,8 +5,8 @@ * what is listed here before and after every run, and nothing else. It never lists a keyset and deletes what it finds. * * To add a name: put it in the right list of the right role below (one string, no logic). A name that a test creates but that is - * not listed here is never cleaned and leaks from run to run. Seeded from docs-snippets/fixtures/names.ts and from the - * literals of the old page modules (snippet regions and `world.n(...)`). + * not listed here is never cleaned and leaks from run to run. The names are the literals of the docs samples + * (snippet regions) and of the setup around them. * * Which list does a name belong to? * channels every channel a test publishes to or reads history from (history is deleted on these, and on `fileChannels`) diff --git a/test/integration/live/support/snippet-scan.ts b/test/integration/live/support/snippet-scan.ts index fe59c4a07..a3b4178e1 100644 --- a/test/integration/live/support/snippet-scan.ts +++ b/test/integration/live/support/snippet-scan.ts @@ -23,8 +23,12 @@ export const MOCKED_FILES: readonly string[] = [ 'test/integration/components/subscription_manager.test.ts', ]; -/** Directories under live/ that never hold samples. */ -const SKIP_DIRS = new Set(['node_modules', 'support', 'provision', 'fixtures', '.run']); +/** + * Directories under live/ that no test runs. `quickstarts` holds the browser, React and React Native samples the docs embed: + * they cannot run in Node, so they are listed in snippet-exceptions.json (and the browser ones are type checked by + * tsconfig.browser.json) instead of being scanned for a test. + */ +const SKIP_DIRS = new Set(['node_modules', 'support', 'provision', 'fixtures', 'quickstarts', '.run']); const SAMPLE_EXT = /\.(mjs|cjs|js|jsx|tsx|ts|html)$/; export type Mode = 'live' | 'compile-only' | 'mocked'; diff --git a/docs-snippets/tsconfig.browser.json b/test/integration/live/tsconfig.browser.json similarity index 90% rename from docs-snippets/tsconfig.browser.json rename to test/integration/live/tsconfig.browser.json index 07b8b70b0..ec0cabc33 100644 --- a/docs-snippets/tsconfig.browser.json +++ b/test/integration/live/tsconfig.browser.json @@ -10,9 +10,9 @@ "skipLibCheck": true, "esModuleInterop": true, "types": [], - "baseUrl": ".", + "baseUrl": "../../..", "paths": { - "pubnub": ["../lib/types/index.d.ts"] + "pubnub": ["lib/types/index.d.ts"] } }, "include": ["quickstarts/quickstart-web.ts", "quickstarts/browser-download-file.ts", "quickstarts/browser-shared-worker.ts"] diff --git a/test/integration/live/tsconfig.json b/test/integration/live/tsconfig.json index 736cf8a6c..c3f243618 100644 --- a/test/integration/live/tsconfig.json +++ b/test/integration/live/tsconfig.json @@ -18,5 +18,6 @@ "pubnub": ["lib/types/index.d.ts", "lib/node/index.js"] } }, - "include": ["./**/*.ts"] + "include": ["./**/*.ts"], + "exclude": ["quickstarts"] } diff --git a/test/integration/live/use-cases/automated-polling.test.ts b/test/integration/live/use-cases/automated-polling.test.ts index 486894209..2645e7540 100644 --- a/test/integration/live/use-cases/automated-polling.test.ts +++ b/test/integration/live/use-cases/automated-polling.test.ts @@ -6,7 +6,7 @@ import { storedMessage } from '../support/history'; import { connected, eventually, sleep, watch } from '../support/wait'; /** - * Automated polling. Port of docs-snippets/use-cases/automated-polling.ts. + * Automated polling. * The trigger messages come from an Illuminate Business Rule on the real service. The keyset of the tier has no such rule, so * the test publishes the triggers itself with the payload the page describes and verifies the poll service side. * Needs Message Persistence on the open keyset (the poll is published with storeInHistory). diff --git a/test/integration/live/use-cases/fan-behavior-management.test.ts b/test/integration/live/use-cases/fan-behavior-management.test.ts index 2b7005f50..5a700bab5 100644 --- a/test/integration/live/use-cases/fan-behavior-management.test.ts +++ b/test/integration/live/use-cases/fan-behavior-management.test.ts @@ -10,7 +10,7 @@ import { withPlaceholders } from '../support/placeholders'; import { eventually } from '../support/wait'; /** - * Port of docs-snippets/use-cases/fan-behavior-management.ts (page use-cases/sports-media-entertainment/fan-behavior-management). + * * Runs on the secured keyset: the token service signs and revokes tokens, the fan client only applies them. * * The state-store snippet starts with an `import`, which only exists at module level, so it sits at file level, before the diff --git a/test/integration/live/use-cases/fan-re-engagement.test.ts b/test/integration/live/use-cases/fan-re-engagement.test.ts index 11cdaf796..939ae3d42 100644 --- a/test/integration/live/use-cases/fan-re-engagement.test.ts +++ b/test/integration/live/use-cases/fan-re-engagement.test.ts @@ -8,7 +8,7 @@ import { DEVICE_TOKEN_EXAMPLE } from '../support/resources'; import { connected, eventually, PRESENCE_TIMEOUT, settled, watch } from '../support/wait'; /** - * Fan re-engagement. Port of docs-snippets/use-cases/fan-re-engagement.ts. + * Fan re-engagement. * Needs Presence and Mobile Push Notifications on the open keyset. The test registers a device token for push but no real * device exists, so it cannot verify that a push notification is DELIVERED; it verifies the publish to the fan's own alert * channel, the payload, and the device registration. diff --git a/test/integration/live/use-cases/game-chat-moderation.test.ts b/test/integration/live/use-cases/game-chat-moderation.test.ts index fafd70569..89ce6b026 100644 --- a/test/integration/live/use-cases/game-chat-moderation.test.ts +++ b/test/integration/live/use-cases/game-chat-moderation.test.ts @@ -8,7 +8,7 @@ import { withPlaceholders } from '../support/placeholders'; import { connected, eventually, settled, sleep } from '../support/wait'; /** - * Hide and remove chat messages. Port of docs-snippets/use-cases/game-chat-moderation.ts. + * Hide and remove chat messages. * Runs on the Access Manager keyset (grantToken needs the secret key); Message Persistence and Delete-From-History must be on. */ describe('use-cases/game-chat-moderation', () => { diff --git a/test/integration/live/use-cases/live-commentary.test.ts b/test/integration/live/use-cases/live-commentary.test.ts index bb1448281..88d6355d2 100644 --- a/test/integration/live/use-cases/live-commentary.test.ts +++ b/test/integration/live/use-cases/live-commentary.test.ts @@ -7,7 +7,7 @@ import { storedMessage } from '../support/history'; import { connected, eventually, settled, sleep } from '../support/wait'; /** - * Live commentary tutorial. Port of docs-snippets/use-cases/live-commentary.ts. + * Live commentary tutorial. * Needs Message Persistence on the open keyset (the backlog step reads stored remarks). */ describe('use-cases/live-commentary', () => { diff --git a/test/integration/live/use-cases/live-event-rate-limiting.test.ts b/test/integration/live/use-cases/live-event-rate-limiting.test.ts index 1e14c53c6..34fc89799 100644 --- a/test/integration/live/use-cases/live-event-rate-limiting.test.ts +++ b/test/integration/live/use-cases/live-event-rate-limiting.test.ts @@ -5,7 +5,7 @@ import { captureConsole } from '../support/console'; import { settled, sleep, watch } from '../support/wait'; /** - * Live event rate limiting. Port of docs-snippets/use-cases/live-event-rate-limiting.ts. + * Live event rate limiting. * Needs Presence on the open keyset (the shard occupancy comes from Here Now). */ describe('use-cases/live-event-rate-limiting', () => { diff --git a/test/integration/live/use-cases/live-polls.test.ts b/test/integration/live/use-cases/live-polls.test.ts index b6c234ddd..5af6c1550 100644 --- a/test/integration/live/use-cases/live-polls.test.ts +++ b/test/integration/live/use-cases/live-polls.test.ts @@ -8,7 +8,7 @@ import { keysOf } from '../support/env'; import { connected, eventually, settled, sleep, watch } from '../support/wait'; /** - * Port of docs-snippets/use-cases/live-polls.ts (page use-cases/sports-media-entertainment/live-polls). + * * One flow over three channels: open a poll, a late fan fetches it, the fan votes, the service publishes the tally and the * fan receives it. Needs Message Persistence on the open keyset. */ diff --git a/test/integration/live/use-cases/match-stats.test.ts b/test/integration/live/use-cases/match-stats.test.ts index 8044681d9..7803e6df1 100644 --- a/test/integration/live/use-cases/match-stats.test.ts +++ b/test/integration/live/use-cases/match-stats.test.ts @@ -9,7 +9,7 @@ import { storedMessage } from '../support/history'; import { connected, eventually, settled, watch } from '../support/wait'; /** - * Port of docs-snippets/use-cases/match-stats.ts (page use-cases/sports-media-entertainment/match-stats). + * * One flow: a scoreboard subscribes to the wildcard, a feed publishes one stat, a late scoreboard fills in current values. * Needs wildcard subscribe (Stream Controller) and Message Persistence on the open keyset. */ diff --git a/test/integration/live/use-cases/real-time-ads.test.ts b/test/integration/live/use-cases/real-time-ads.test.ts index 64565ef8c..06c549185 100644 --- a/test/integration/live/use-cases/real-time-ads.test.ts +++ b/test/integration/live/use-cases/real-time-ads.test.ts @@ -6,7 +6,7 @@ import { captureConsole } from '../support/console'; import { connected, eventually, watch } from '../support/wait'; /** - * Real-time ads and reaction upgrades. Port of docs-snippets/use-cases/real-time-ads.ts. + * Real-time ads and reaction upgrades. * The ad decisions and reaction upgrades come from Illuminate Business Rules on the real service. The keyset used by the tier * has no such rule, so the test publishes the decisions itself, with the payload shape the page describes, and verifies the * client side: what the fan publishes and how the client reacts to a decision. diff --git a/test/integration/live/use-cases/real-time-chat.test.ts b/test/integration/live/use-cases/real-time-chat.test.ts index dc8986175..6ad0534a3 100644 --- a/test/integration/live/use-cases/real-time-chat.test.ts +++ b/test/integration/live/use-cases/real-time-chat.test.ts @@ -9,7 +9,7 @@ import { withPlaceholders } from '../support/placeholders'; import { connected, eventually, PRESENCE_TIMEOUT, settled } from '../support/wait'; /** - * Port of docs-snippets/use-cases/real-time-chat.ts (page use-cases/sports-media-entertainment/real-time-chat). + * * One flow, as the page runs it: set channel and fan metadata, join with presence, send, load history, count fans, react. * Needs Presence, Message Persistence and App Context on the open keyset. */ diff --git a/test/integration/live/use-cases/score-alerts.test.ts b/test/integration/live/use-cases/score-alerts.test.ts index 7f85aacba..8e73c69a1 100644 --- a/test/integration/live/use-cases/score-alerts.test.ts +++ b/test/integration/live/use-cases/score-alerts.test.ts @@ -8,7 +8,7 @@ import { keysOf } from '../support/env'; import { watch, eventually, settled } from '../support/wait'; /** - * Port of docs-snippets/use-cases/score-alerts.ts (page use-cases/sports-media-entertainment/real-time-score-alerts). + * * One flow: register a device, build and publish the alert, list the registration, remove it. Needs the Push Notifications * add-on on the open keyset. Delivery of the notification itself needs a real device and provider credentials: not verified. */ From 66a6a8bd1b90d16f0e887d84c4b4d083fa3c00b1 Mon Sep 17 00:00:00 2001 From: Mateusz Wiktor Date: Tue, 17 Nov 2026 13:19:10 +0100 Subject: [PATCH 08/13] refactor: remove docs snippets syntax test from CI workflow - Eliminated the test for documentation snippets from the CI workflow to streamline the testing process. - Updated the coverage.json file to clarify the description of the legacy decrypt functionality. --- .github/workflows/run-tests.yml | 3 --- test/integration/live/coverage.json | 2 +- 2 files changed, 1 insertion(+), 4 deletions(-) diff --git a/.github/workflows/run-tests.yml b/.github/workflows/run-tests.yml index 20c4465d0..af3ce18ff 100644 --- a/.github/workflows/run-tests.yml +++ b/.github/workflows/run-tests.yml @@ -46,9 +46,6 @@ jobs: run: | npm install npm run ${{ matrix.env }} - - name: Test docs snippets syntax - if: matrix.node == '22' && matrix.env == 'ci:node' - run: npm run test:snippets all-tests: name: Tests runs-on: diff --git a/test/integration/live/coverage.json b/test/integration/live/coverage.json index 9d03b95f5..6d82f69f6 100644 --- a/test/integration/live/coverage.json +++ b/test/integration/live/coverage.json @@ -303,7 +303,7 @@ }, "decryptBasicUsage": { "file": "test/integration/live/api-reference/encryption-legacy.test.ts", - "test": "api-reference/encryption-legacy decryptBasicUsage: the legacy decrypt returns the original JSON for text encrypted with the same cipher key", + "test": "api-reference/encryption-legacy decryptBasicUsage: the legacy decrypt resolves to the original JSON for text encrypted with the same cipher key", "role": "open", "mode": "live" }, From b35a3ba54a8e33b404fe99c859abbdf072f5fdf8 Mon Sep 17 00:00:00 2001 From: Mateusz Wiktor Date: Wed, 18 Nov 2026 07:44:56 +0100 Subject: [PATCH 09/13] run on non draft prs only --- .github/workflows/live-integration.yml | 20 ++-- test/integration/live/README.md | 133 +++++++++++++++++-------- 2 files changed, 104 insertions(+), 49 deletions(-) diff --git a/.github/workflows/live-integration.yml b/.github/workflows/live-integration.yml index 327df7ebf..eb26e74fd 100644 --- a/.github/workflows/live-integration.yml +++ b/.github/workflows/live-integration.yml @@ -1,12 +1,16 @@ name: Live integration # Live tier (test/integration/live/README.md): the public documentation's code samples run against the real PubNub service. -# Nightly and on demand only. The live job never runs on a pull request, so fork PRs never see secrets. Pull requests get the -# offline job only (lint, manifest freshness, types), and it is not a required check. +# Runs on non-draft pull requests into master, and on demand. No scheduled runs. Draft pull requests run nothing. Fork pull +# requests never get the live job (no secrets for fork code). on: - # Pull requests: offline job only, and only when something the tier or its mocked samples depend on changed. + # Pull requests into master, only when something the tier or its mocked samples depend on changed. Marking a draft as ready + # for review starts a run. pull_request: + branches: + - master + types: [opened, synchronize, reopened, ready_for_review] paths: - 'test/integration/live/**' - 'test/integration/endpoints/objects/channel.test.ts' @@ -14,8 +18,6 @@ on: - 'test/integration/components/subscription_manager.test.ts' - 'package.json' - '.github/workflows/live-integration.yml' - schedule: - - cron: '43 4 * * *' workflow_dispatch: inputs: grep: @@ -34,6 +36,7 @@ permissions: jobs: offline: name: Offline checks (lint, types) + if: ${{ github.event_name != 'pull_request' || !github.event.pull_request.draft }} runs-on: group: organization/Default timeout-minutes: 15 @@ -59,11 +62,8 @@ jobs: live: name: Live tier needs: [offline] - # TEMPORARY: also runs on pull requests from a branch of this repository, so the tier can be tested before it is merged - # (a workflow can only be run manually or on a schedule once its file is on the default branch). Never on a fork pull - # request: no secrets for fork code. Before or right after the merge, change this back to - # `github.event_name != 'pull_request'` so the tier runs nightly and on demand only. - if: ${{ github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository }} + # Pull requests: only non-draft ones from a branch of this repository (never a fork: no secrets for fork code). + if: ${{ github.event_name != 'pull_request' || (!github.event.pull_request.draft && github.event.pull_request.head.repo.full_name == github.repository) }} runs-on: group: organization/Default timeout-minutes: 40 diff --git a/test/integration/live/README.md b/test/integration/live/README.md index 54b99fdce..923109555 100644 --- a/test/integration/live/README.md +++ b/test/integration/live/README.md @@ -1,6 +1,6 @@ # Live integration tier -Mocha tests that run the code samples of the public docs against the real PubNub service. The nock-mocked tests in `test/integration/**` stay as they are, and `npm run test:node` never runs this folder (`.mocharc.json` excludes it). +Mocha (a JavaScript test runner) runs the code samples from the public documentation against the real PubNub service. The samples come from the PubNub Software Development Kit (SDK) docs. The mocked tests in `test/integration/**` stay as they are. They use nock, a library that fakes HTTP responses. The `npm run test:node` script never runs this folder, because `.mocharc.json` excludes it. ## Run @@ -15,41 +15,61 @@ npm run test:integration:live:types # type check (needs the bui npx eslint "test/integration/live/**/*.ts" --ignore-pattern "test/integration/live/quickstarts/**" # lint (npm run lint covers src/ only). Do not add --config: it drops the max-len override of live/.eslintrc.cjs ``` -Env names: `SUBSCRIBE_KEY`, `PUBLISH_KEY` (open keyset), `PAM_SUBSCRIBE_KEY`, `PAM_PUBLISH_KEY`, `PAM_SECRET_KEY` (Access Manager), `DS_*` (DataSync). Real environment variables win over the file. `LIVE_ENV_FILE=` points at another file. The run fails at once when the open keyset is missing. A docs-tier run without `--grep` (and without spec file arguments) needs every role: it fails at once, with the missing names, when `pam` or `datasync` is not set. A run with `--grep` or a file argument checks a role only when a test builds its client, so a page that needs only the open keyset runs with only that keyset. Run one live run per keyset at a time. +Environment variable names: `SUBSCRIBE_KEY` and `PUBLISH_KEY` (open keyset), `PAM_SUBSCRIBE_KEY`, `PAM_PUBLISH_KEY` and `PAM_SECRET_KEY` (Access Manager, PAM), and `DS_*` (DataSync, DS). Real environment variables take precedence over the file. `LIVE_ENV_FILE=` points to another file. The run stops at once if the open keyset is missing. -**Two runs, two keysets.** The tier has two Mocha configs. +A docs-tier run without `--grep` and without spec file arguments needs every role. It fails at once, and names the missing variables, when `pam` or `datasync` is not set. A run with `--grep` or a file argument checks a role only when a test builds its client. So a page that needs only the open keyset runs with only that keyset. Run one live run per keyset at a time. + +**Two runs, two keysets.** The tier has two Mocha configuration files. | Script | Config | Runs | Keys | Janitor / preflight | | :--- | :--- | :--- | :--- | :--- | | `test:integration:live` | `.mocharc.cjs` | every test except `endpoints/` (`api-reference/`, `getting-started/`, `migration-guides/`, `shared/`, `use-cases/`) | open, pam and datasync keysets of the docs | on (`LIVE_JANITOR=off`, `LIVE_PREFLIGHT=off` switch them off) | | `test:integration:live:endpoints` | `.mocharc.endpoints.cjs` | only `endpoints/**`: 6 DataSync event and projection files (`endpoints/data_sync/`) and 3 files with the real-network tests of the SDK endpoint suite (`endpoints/fetch_messages.test.ts`, `history.test.ts`, `message_actions.test.ts`; 14 tests) | no role is checked up front, each test checks its own: `datasync` (`DS_SUBSCRIBE_KEY`, `DS_PUBLISH_KEY`, `DS_SECRET_KEY`) for the DataSync files, `open` (`SUBSCRIBE_KEY`, `PUBLISH_KEY`, with Message Persistence and message actions) for the other three | off by default (`LIVE_JANITOR=on` / `LIVE_PREFLIGHT=on` turn them on) | -The DataSync endpoint tests need the DataSync classes `JSCustomer`, `JSLoanQuote` and `JSREQUESTED_BY`, which exist on a different keyset from the one the docs samples use. The other three files (`fetch_messages`, `history`, `message_actions`) are the tests that publish to the real service and read the result back; they were moved out of `test/integration/endpoints/` of the same names, whose remaining tests are nock-mocked, so that `npm run test:node` runs fully offline (a root hook in `test/setup-why.ts` refuses every request that no nock scope matches). They are not documentation samples and carry no `// snippet.` markers. CI runs the whole folder as one separate step with the SDK's own keysets (the ones `run-tests.yml` uses: `SDK_SUB_KEY`/`SDK_PUB_KEY` as `SUBSCRIBE_KEY`/`PUBLISH_KEY`, and `SDK_DS_*` as `DS_*`), not the docs keysets. Every test titles itself `endpoints/: ...`, so `--grep '^endpoints/(fetch_messages|history|message_actions)'` runs only the open-keyset files and `--grep '^endpoints/data_sync'` only the DataSync files; a test whose role is not set fails with `missing keys: the keyset is not configured, set `. These tests use channels named after a fresh UUID per test, so they are safe on a shared, persistent keyset, and they poll eventually consistent reads with `eventually` instead of a fixed wait. The janitor's resource list and the preflight's feature checks belong to the docs keysets, and the endpoint tests remove their own uniquely named objects. `.mocharc.endpoints.cjs` sets `LIVE_TIER=endpoints`, which `support/hooks.ts` reads: no role is required up front and the full-run rule below does not apply. Mocha adds a spec file on the command line to the config's `spec` (it does not replace it), so the docs-tier config excludes `endpoints/**` with `exclude`, and a file argument of the other tier is not added to a run. Check what a script runs with `-- --dry-run`. The root `npm run test:node` lists none of this folder. The workflow (`.github/workflows/live-integration.yml`) runs nightly and on demand; it does not alert anyone on failure yet (the notify job was removed, the GitHub run logs are the only signal). +The DataSync endpoint tests need the DataSync classes `JSCustomer`, `JSLoanQuote` and `JSREQUESTED_BY`. These classes exist on a different keyset from the one the docs samples use. + +The other three files (`fetch_messages`, `history`, `message_actions`) publish to the real service and read the result back. They moved out of `test/integration/endpoints/`. The files with the same names there are mocked. This keeps `npm run test:node` fully offline. A root hook in `test/setup-why.ts` refuses every request that no nock scope matches. These three files are not documentation samples, and they carry no `// snippet.` markers. + +Continuous Integration (CI) runs the whole folder as one separate step. It uses the SDK's own keysets, the ones `run-tests.yml` uses: `SDK_SUB_KEY` and `SDK_PUB_KEY` as `SUBSCRIBE_KEY` and `PUBLISH_KEY`, and `SDK_DS_*` as `DS_*`. It does not use the docs keysets. + +Each test titles itself `endpoints/: ...`. So `--grep '^endpoints/(fetch_messages|history|message_actions)'` runs only the open-keyset files, and `--grep '^endpoints/data_sync'` runs only the DataSync files. A test whose role is not set fails with `missing keys: the keyset is not configured, set `. + +These tests use a channel named after a fresh Universally Unique Identifier (UUID), one per test. So they are safe on a shared, persistent keyset. They poll eventually consistent reads with `eventually` instead of a fixed wait. + +The janitor's resource list and the preflight's feature checks belong to the docs keysets. The endpoint tests remove their own uniquely named objects. `.mocharc.endpoints.cjs` sets `LIVE_TIER=endpoints`, and `support/hooks.ts` reads that value. With it set, no role is required up front, and the full-run rule below does not apply. -Rebuild with `npm run build:node` after a change under `src/`, otherwise the tier tests the previous build. +Mocha adds a spec file from the command line to the config's `spec`. It does not replace it. So the docs-tier config excludes `endpoints/**` with `exclude`, and a file argument for the other tier is not added to a run. Check what a script runs with `-- --dry-run`. The root `npm run test:node` lists none of this folder. + +The workflow (`.github/workflows/live-integration.yml`) runs on non-draft pull requests into `master`, when the paths it watches change, and on demand. It has no scheduled runs. It does not alert anyone on failure yet. The GitHub run logs are the only signal. + +Rebuild with `npm run build:node` after a change under `src/`. Otherwise the tier tests the previous build. ## Conventions -- One file per docs page: `.test.ts` under the folder that matches the docs section (`api-reference/`, `getting-started/`, `migration-guides/`, `shared/`, `use-cases/`), so the docs URL is `test/integration/live//.test.ts`. One `describe` per page, one `it` per sample. Start the `it` title with the snippet id(s), then what it proves. -- Plain `describe` / `it`, `node:assert/strict`. No staging, no recorder, no `page` / `step` / `check`. -- Markers sit inside the `it` body: `// snippet.` ... `// snippet.end`. The code between them is the published sample text (indentation may differ, readers get it un-indented). Never change it without the approval of the docs owner. Everything else (setup, listeners, assertions) stays outside. Use `// snippet.hide` / `// snippet.show` only where the sample cannot run without a line readers must not see. -- Run the marker lint (`npm run test:integration:live:lint`) after every change. A snippet id is part of the docs URL, so keep it stable. -- **Literal names.** The code in the markers uses exactly the names the docs show (`'my_channel'`, `'foo'`, ...), and that is what runs. Isolation comes from one live run per keyset at a time and a cleanup pass before and after (the janitor). -- **Assertions must not assume a private, clean resource.** Channels, users and groups are shared and persist across runs. Assert on this run's own data: the `timetoken` the sample returned, an ID, or a unique value your test generates outside the markers. Do not assert exact counts or "the latest N". When the sample does not keep its response, match as narrowly as the sample allows (publisher, payload) and say why in a comment. -- **Reset, then assert exactly.** A test that works on a literal resource (a channel group, a user, a device token, a DataSync object) first resets it to a known state (remove, then seed what it needs), and then asserts the exact result. This is safe because the janitor cleans the listed names and one run owns a keyset at a time. Put the reset in the test or in a hook of the page, not in the sample. -- **Real keys.** A sample shows placeholder keys. Put the real ones in a `// snippet.hide` ... `// snippet.show` block inside the call: `...openKeys(),`, `...securedKeys(),` or `...subscribeKeyOnly(),` from `support/env.ts` (typed `Partial`, so no cast and no TS2783). Readers never see the block. -- **Placeholder literals.** When a sample shows a placeholder where a run needs a live value (a token, a file id `'...'`, a timetoken), the sample text stays and the test passes it a client wrapped by `withPlaceholders(client, { placeholder: liveValue })` (`support/placeholders.ts`). +- **One file per docs page.** Put each page in `.test.ts` under the folder that matches its docs section (`api-reference/`, `getting-started/`, `migration-guides/`, `shared/` or `use-cases/`). The docs URL then maps to `test/integration/live//.test.ts`. Use one `describe` per page and one `it` per sample. Start each `it` title with the snippet id or ids, then say what the test proves. +- Use plain Mocha `describe` and `it`, and `node:assert/strict`. Do not use staging, a recorder, or `page`, `step` or `check`. +- **Markers.** Put the markers inside the `it` body: `// snippet.` ... `// snippet.end`. The code between them is the published sample text. Indentation may differ, because readers see it unindented. Do not change the sample without approval from the docs owner. Setup, listeners and assertions stay outside the markers. Use `// snippet.hide` and `// snippet.show` only where the sample cannot run without a line that readers must not see. +- Run the marker lint (`npm run test:integration:live:lint`) after every change. A snippet id is part of the docs URL, so keep each id stable. +- **Literal names.** The code in the markers uses the exact names the docs show (`'my_channel'`, `'foo'`, and so on). Those names run as written. Isolation comes from two things: one live run per keyset at a time, and a cleanup pass (the janitor) before and after each run. +- **Assertions must not assume a private, clean resource.** Channels, users and groups are shared, and they persist across runs. Assert on data from this run: the `timetoken` the sample returned, an ID, or a unique value that the test generates outside the markers. Do not assert exact counts or "the latest N". If the sample does not keep its response, match as narrowly as the sample allows, for example by publisher or payload, and explain why in a comment. +- **Reset, then assert exactly.** A test that uses a literal resource (a channel group, a user, a device token, or a DataSync object) first resets it to a known state. It removes the resource, then seeds what it needs. Then it asserts the exact result. This is safe because the janitor cleans the listed names, and one run owns a keyset at a time. Put the reset in the test or in a hook of the page, not in the sample. +- **Real keys.** A sample shows placeholder keys. Put the real keys in a `// snippet.hide` ... `// snippet.show` block inside the call: `...openKeys(),`, `...securedKeys()` or `...subscribeKeyOnly(),` from `support/env.ts`. These helpers are typed `Partial`, so no cast is needed, and TypeScript does not raise error TS2783. Readers never see the block. +- **Placeholder literals.** When a sample shows a placeholder where a run needs a live value (a token, a file id `'...'`, or a timetoken), keep the sample text. Pass the test a client wrapped by `withPlaceholders(client, { placeholder: liveValue })` from `support/placeholders.ts`. - **Prettier.** Wrap a marker region in `/* eslint-disable prettier/prettier */` ... `/* eslint-enable prettier/prettier */`, placed outside the markers, when `eslint --fix` would reformat the sample. -- **Unique run ids.** Anything a test asserts on exactly and does not own (a channel it reads back, a payload marker) carries a per-run suffix: `unique('name')` from `support/util.ts`, or `randomUUID()` in the payload. Such names cannot be listed in `support/resources.ts`, so the janitor never clears them. Their messages expire with the retention of the keyset. -- **Console.** A client that logs at Debug or Trace level prints its configuration with the real keys. Wrap the test in `quietConsole()` (`support/console.ts`) and call `restore()` in a `finally`. The root hook also mutes the console while it destroys clients, so the destroy-time dumps stay out of the log. Sample `console.log` output is fine, but a sample that prints a granted token (`console.log('Granted Token:', token)`) runs under `captureConsole('log')`. The root hook restores any capture a failed test left behind (`restoreAllConsoles()`). -- **Transient service errors: retry setup, never the sample.** Setup and cleanup code outside the markers (seeding, resets) may wrap a call in `retryTransient` (`support/retry.ts`). It repeats only HTTP 5xx, 429 and network, timeout or unclassified transport failures, never a 4xx, a validation error or an assertion error, and it logs one `retryTransient:` line per retry. Code inside the markers, the assertions about a sample's effect, and calls whose failure a test asserts are never retried or wrapped. Mocha `retries` stays 0 and timeouts are not lengthened to hide flakiness. -- **Creates with a run-unique id, and seeding on a fresh channel (endpoint tests only).** A create is not idempotent, so `retryTransient` alone is wrong for it: a retry after a first attempt that the service stored answers 409. `createWithRunUniqueId(create, readBack, { label })` (`support/retry.ts`) is for ids no one else can hold (`freshId`, `projectionId`; both end in `uniqueSuffix()` of `support/util.ts`: the time in base 36 plus 6 random base-36 characters, so an id cannot meet an object that earlier runs left on a keyset that is years old): it retries transient failures, treats a 409 with DS-0301 on a RETRY as "the earlier attempt was stored" and returns `readBack()` (a get by id), and throws a 409 on the first attempt. `seedOnFreshChannel(newChannel, seed)` seeds a run-unique channel and, on a transient failure, starts the whole seeding again on a new channel (tests that assert exact counts must never reuse a half-seeded channel). Neither belongs in a documentation sample, which uses fixed literal ids and runs once. -- **Objects a test creates are removed whatever the outcome (endpoint tests).** Register the id before the create: `removeAfterTest(client, 'entity' | 'user' | 'channel' | 'relationship' | 'membership', freshId(...))` (`support/datasync-live-helpers.ts`, backed by `support/cleanup.ts`). The root `afterEach` removes the registered objects, newest first, before it destroys the clients, whether the test passed, failed on an assertion or failed in the create itself. A 404 is expected and silent. Nothing lists or sweeps the keyset: only ids this test generated are removed. A hook that creates objects (`before`) removes them in its own `after`. -- **Failures explain themselves.** `support/hooks.ts` wraps Mocha's `Runner#fail` once, so every failed test or hook of both npm scripts whose error is (or wraps, through `.cause`) a PubNub SDK error gets two lines appended to its message in the failure list: `category`, `statusCode`, `operation` and the service's error data (`support/explain-error.ts`). Keys, tokens and URL queries are masked. No test opts in, and nothing about pass or fail changes. -- **Retried publishes can be stored twice.** A failed attempt may have reached the service. A seeder whose test counts messages puts `run: seedRun.id` in the payload, publishes through `seedPublish` and calls `settleSeeded` (`support/history.ts`), which deletes stored copies the caller never received a timetoken for. -- **Waiting.** Poll with `eventually`, listen with `watch` / `expectEvent`. Use a fixed `sleep` only where the test proves that something did NOT arrive. `PRESENCE_EVENT_TIMEOUT` (30 s) is for a presence event the test caused; `PRESENCE_TIMEOUT` (45 s) is for polling `hereNow` and similar reads. -- **`settled` instead of `eventually`.** `eventually` returns on the first read that passes, which proves only that one replica caught up. Use `settled` (default: 3 passes in a row, 750 ms apart, so about 1.5 s once the state is visible) as the precondition of a sample that reads the same eventually consistent state ONCE and is asserted exactly: a filter or sort index, a list, a count, history, presence, message actions, file listing, push registrations, App Context members. Poll the same read the sample will make, from a helper client, and put it outside the markers. Keep `eventually` where the wait is itself the final assertion of the test, or where it waits for an event or local state. A fixed `sleep` or a longer timeout is never a substitute: they do not show that consecutive reads agree. Do not run a sample inside the wait to retry it. -- Clients come from `support/clients.ts` and are destroyed after every test by the root hook. Do not call `destroy()` for them yourself. A client built by hand (the sample's own `new PubNub(...)`) goes through `track()`. +- **Unique run ids.** Anything a test asserts on exactly, and does not own, gets a per-run suffix. Examples are a channel the test reads back, or a payload marker. Use `unique('name')` from `support/util.ts`, or `randomUUID()` in the payload. The janitor never clears such names, because they cannot be listed in `support/resources.ts`. Their messages expire with the retention period of the keyset. +- **Console.** A client that logs at Debug or Trace level prints its configuration, including the real keys. Wrap the test in `quietConsole()` from `support/console.ts`, and call `restore()` in a `finally` block. The root hook also mutes the console while it destroys clients, so the dumps printed at destroy time stay out of the log. Sample `console.log` output is fine. A sample that prints a granted token (`console.log('Granted Token:', token)`) runs under `captureConsole('log')`. The root hook restores any capture that a failed test left behind, with `restoreAllConsoles()`. +- **Retry setup, never the sample.** Setup and cleanup code outside the markers (seeding and resets) may wrap a call in `retryTransient` from `support/retry.ts`. It repeats only HTTP (Hypertext Transfer Protocol) 5xx and 429 responses, and network, timeout or unclassified transport failures. It never repeats a 4xx response, a validation error or an assertion error. It logs one `retryTransient:` line per retry. Code inside the markers, the assertions about a sample's effect, and calls whose failure a test asserts are never retried or wrapped. Mocha `retries` stays at 0, and timeouts are not lengthened to hide flakiness. +- **Creates with a run-unique id, and seeding on a fresh channel (endpoint tests only).** A create call is not idempotent, so `retryTransient` alone is wrong for it. If the service stored the first attempt, a retry gets a 409 response (conflict). Use `createWithRunUniqueId(create, readBack, { label })` from `support/retry.ts` for ids that no one else can hold. These are `freshId` and `projectionId`. Both end in `uniqueSuffix()` from `support/util.ts`. That suffix is the time in base 36 (digits and letters) plus six random base-36 characters. So an id cannot match an object that earlier runs left on a keyset that is years old. + + The helper retries transient failures. On a retry, it treats a 409 with DS-0301 as proof that the earlier attempt was stored, and it returns `readBack()` (a get by id). On the first attempt, it throws the 409. `seedOnFreshChannel(newChannel, seed)` seeds a run-unique channel. On a transient failure, it starts the whole seeding again on a new channel. Tests that assert exact counts must never reuse a half-seeded channel. Neither helper belongs in a documentation sample, which uses fixed literal ids and runs once. +- **Objects a test creates are removed whatever the outcome (endpoint tests).** Register the id before the create, with `removeAfterTest(client, 'entity' | 'user' | 'channel' | 'relationship' | 'membership', freshId(...))`. The helper lives in `support/datasync-live-helpers.ts` and uses `support/cleanup.ts`. The root `afterEach` removes the registered objects, newest first, before it destroys the clients. It does this whether the test passed, failed an assertion, or failed in the create itself. A 404 response (not found) is expected and silent. Nothing lists or sweeps the keyset. Only the ids that this test generated get removed. A hook that creates objects (`before`) removes them in its own `after`. +- **Failures explain themselves.** `support/hooks.ts` wraps Mocha's `Runner#fail` once. For each failed test or hook in either npm script, if its error is a PubNub SDK error, or wraps one through `.cause`, the failure list gets two extra lines. They show `category`, `statusCode`, `operation` and the service's error data (`support/explain-error.ts`). Keys, tokens and URL queries are masked. No test opts in, and pass or fail results do not change. +- **Retried publishes can be stored twice.** A failed attempt may still have reached the service. A seeder whose test counts messages adds `run: seedRun.id` to the payload. It publishes through `seedPublish`, then calls `settleSeeded` from `support/history.ts`. That call deletes the stored copies that the caller never received a timetoken for. +- **Waiting.** Poll with `eventually`. Listen with `watch` or `expectEvent`. Use a fixed `sleep` only where the test proves that something did NOT arrive. `PRESENCE_EVENT_TIMEOUT` (30 s) is for a presence event that the test caused. `PRESENCE_TIMEOUT` (45 s) is for polling `hereNow` and similar reads. +- **`settled` instead of `eventually`.** `eventually` returns on the first read that passes, and that proves only that one replica caught up. Use `settled` as the precondition of a sample that reads eventually consistent state once and asserts the result exactly. Examples of such state are a filter or sort index, a list, a count, history, presence, message actions, file listing, push registrations and App Context members. The default is 3 passes in a row, 750 ms apart, so about 1.5 s once the state is visible. Poll the same read the sample will make, from a helper client, and put that poll outside the markers. + + Keep `eventually` where the wait is the final assertion of the test, or where it waits for an event or local state. A fixed `sleep` or a longer timeout is never a substitute, because neither shows that consecutive reads agree. Do not run a sample inside the wait to retry it. +- Clients come from `support/clients.ts`. The root hook destroys them after every test, so do not call `destroy()` yourself. A client built by hand (the sample's own `new PubNub(...)`) goes through `track()`. ## Helpers (`support/`) @@ -57,7 +77,7 @@ Rebuild with `npm run build:node` after a change under `src/`, otherwise the tie | :--- | :--- | | `env.ts` | `env()`, `keysOf(role)`, `openKeys()`, `securedKeys(role?)`, `subscribeKeyOnly()`, `describeRoles()`, `loadEnv()`, `redact()` | | `clients.ts` | `createClient(role, userId, extra?)`, `shopper()`, `friend()`, `observer()`, `server(role?)`, `unreachable`, `track()`, `destroyClients()` | -| `wait.ts` | `eventually`, `settled`, `watch` (returns a `Watch` with `events`, `next`, `stop`), `connected`, `expectEvent`, `expectStatus`, `sleep`, `PRESENCE_TIMEOUT`, `PRESENCE_EVENT_TIMEOUT`. A timed-out `expectEvent`, `connected`, `expectStatus` or `Watch.next` says what the client saw meanwhile: statuses (category, operation, the cursor of a connected status), the count and kinds of events with their channels, and the channels it was subscribed to. Pass `{ describe: () => '...' }` to add what the test knows (timetokens). | +| `wait.ts` | `eventually`, `settled`, `watch` (returns a `Watch` with `events`, `next`, `stop`), `connected`, `expectEvent`, `expectStatus`, `sleep`, `PRESENCE_TIMEOUT`, `PRESENCE_EVENT_TIMEOUT`. A timed-out `expectEvent`, `connected`, `expectStatus` or `Watch.next` reports what the client saw. That includes the statuses (category, operation, and the cursor of a connected status), the count and kinds of events with their channels, and the channels it was subscribed to. Pass `{ describe: () => '...' }` to add what the test knows, such as timetokens. | | `assertions.ts` | `assertTimetoken` (a string, as `publish()` returns it), `assertServerTime` (a number, as `time()` returns it), `addToTimetoken`, `rejectsValidation`, `rejectsWithCategory` | | `placeholders.ts` | `withPlaceholders(client, values)` | | `util.ts` | `unique(base)`, `uniqueSuffix()`, `sorted(values)` | @@ -72,15 +92,15 @@ Rebuild with `npm run build:node` after a change under `src/`, otherwise the tie Other files in `support/` serve one page or one tool (`janitor*.ts`, `preflight.ts`, `datasync-*.ts`, `subscribe-groups.ts`, `proxy.ts`, `file-workdir.ts`, `access-manager.ts`). -Every `observer()`, `shopper()` call builds a new client, so a test can have two clients with the same user id when it needs them. +Every `observer()` or `shopper()` call builds a new client. So a test can hold two clients with the same user id when it needs them. ## Cleanup (janitor), preflight, provisioning -The tests use the literal names the docs show (`'my_channel'`, `'family'`, ...), so the state they leave behind is shared and persistent. Three pieces keep a run honest. +The tests use the literal names that the docs show (`'my_channel'`, `'family'`, and so on). So the state they leave behind is shared, and it persists. Three parts keep a run honest. ### Janitor (`support/janitor.ts`, `janitor-datasync.ts`, `resources.ts`) -`runJanitor()` cleans **exactly the resources listed in `support/resources.ts`**, per keyset role, and nothing else. It never lists a keyset to delete what it finds, never touches a DataSync class and never changes keyset configuration. A role whose keys are missing is skipped with a note. It is best effort and idempotent, and it reports what it removed, what it could not remove and what it left on purpose. +`runJanitor()` cleans **exactly the resources listed in `support/resources.ts`**, per keyset role, and nothing else. It never lists a keyset to find things to delete. It never touches a DataSync class, and it never changes keyset configuration. A role whose keys are missing is skipped, with a note. The janitor is best effort and idempotent. It reports what it removed, what it could not remove, and what it left on purpose. | Role | What it does | | :--- | :--- | @@ -88,31 +108,33 @@ The tests use the literal names the docs show (`'my_channel'`, `'family'`, ...), | pam | revokes the application-level Access Manager grant; the same lists as open when the role has any; history of its listed channels | | datasync | the same revoke (once when pam and datasync are one keyset); DataSync relationships and memberships, then entities, users and channels from the list; history | -It runs automatically from the root hooks: once before all tests (a failure fails the run with the list of what could not be removed; on a secured keyset it then waits until a client without a token is refused again, which can take up to about 60 s when the previous run just changed a grant; it gives up after 120 s), and once after all tests (a failure is printed in a loud banner and does not fail the run, except a failed grant revoke, which leaves a secured keyset open; the next run's cleanup before the run retries and fails when the keyset is still dirty). +The janitor runs automatically from the root hooks. It runs once before all tests. A failure fails the run, and the output lists what could not be removed. On a secured keyset, the janitor then waits until a client without a token is refused again. That can take up to about 60 s when the previous run just changed a grant. It gives up after 120 s. It also runs once after all tests. A failure there prints a loud banner and does not fail the run. The exception is a failed grant revoke, which leaves a secured keyset open. The next run's cleanup retries before the run, and it fails if the keyset is still dirty. ```bash npm run test:integration:live:janitor # the janitor alone, all roles; exit code 1 when something could not be removed npm run test:integration:live:janitor -- --role open ``` -A full pass walks every listed name, so it takes a while (see the measured time in the report of this slice). For a single page or a `--grep` run, switch it off: `LIVE_JANITOR=off npm run test:integration:live -- --grep publishMinimal`. Clean up with the janitor command before and after such runs when you want a clean slate. +A full pass walks every listed name, so it takes a while. For a single page or a `--grep` run, turn the janitor off: `LIVE_JANITOR=off npm run test:integration:live -- --grep publishMinimal`. Run the janitor command before and after such runs when you want a clean slate. -**Adding a name.** Add one string to the right list of the right role in `support/resources.ts` (`channels`, `fileChannels`, `channelGroups`, `appContext.users`, `appContext.channels`, `pushDevices`, `dataSync.*`). The file is data only. A resource a test creates that is not listed there is never cleaned and leaks from run to run. One channel is left out on purpose: `history_documented_behavior` holds more than 100 messages that its test writes once and reuses (see the comment in `resources.ts`). Names with a per-run suffix are not listed either; their messages expire with the keyset's retention. Channels that appear in a channel group must be listed under `channels` as well (deleting a group does not delete its member channels). +**Adding a name.** Add one string to the right list of the right role in `support/resources.ts`: `channels`, `fileChannels`, `channelGroups`, `appContext.users`, `appContext.channels`, `pushDevices`, or `dataSync.*`. The file holds data only. A resource that a test creates and does not list is never cleaned, so it stays on the keyset from run to run. One channel is left out on purpose: `history_documented_behavior`. It holds more than 100 messages, which its test writes once and reuses (see the comment in `resources.ts`). Names with a per-run suffix are not listed either. Their messages expire with the keyset's retention period. A channel in a channel group must also be listed under `channels`, because deleting a group does not delete its member channels. ### Preflight (`support/preflight.ts`) -After the janitor, the root hook fails fast, with a message that names each missing feature. With `PUBNUB_ADMIN_API_KEY` and the keyset ids (`DOCS_SNIPPETS_OPEN_KEYSET_ID`, `DOCS_SNIPPETS_PAM_KEYSET_ID`, `DOCS_SNIPPETS_DS_KEYSET_ID`) it reads each keyset's configuration through the Admin API (read only) and lists what the role needs and lacks (Presence, Message Persistence, Delete-From-History, App Context and its events, Files, Stream Controller, Access Manager, DataSync). Live probes on the open keyset always run: publish (keys and network), Message Persistence, Delete-From-History, and Presence when the Admin API did not vouch for it. The features each keyset needs are the ones the message names; `missingFeatures` in `support/preflight.ts` is the full list per role (open, pam, datasync). +After the janitor, the root hook fails fast. Its message names each missing feature. With `PUBNUB_ADMIN_API_KEY` and the keyset ids (`DOCS_SNIPPETS_OPEN_KEYSET_ID`, `DOCS_SNIPPETS_PAM_KEYSET_ID`, `DOCS_SNIPPETS_DS_KEYSET_ID`), the hook reads each keyset's configuration through the Admin API, read-only. It lists what the role needs and lacks: Presence, Message Persistence, Delete-From-History, App Context and its events, Files, Stream Controller, Access Manager and DataSync. + +Live probes on the open keyset always run. They check publish (keys and network), Message Persistence, Delete-From-History, and Presence when the Admin API did not confirm it. The features each keyset needs are the ones the message names. `missingFeatures` in `support/preflight.ts` is the full list per role (open, pam, datasync). ### Provisioning (`provision/`) -DataSync classes are created by `provision/provision-datasync.ts` from `provision/datasync-classes.json`. It is idempotent, never deletes a class and fails on a class whose immutable fields differ. The tier does not run it. Run it when a datasync keyset is new or the class file changes. +`provision/provision-datasync.ts` creates the DataSync classes from `provision/datasync-classes.json`. The script is idempotent. It never deletes a class, and it fails on a class whose immutable fields differ. The tier does not run it. Run it when a DataSync keyset is new or when the class file changes. ```bash npm run test:integration:live:provision:check # dry diff, writes nothing, exit code 1 when anything differs npm run test:integration:live:provision # creates missing classes, updates drifted mutable fields ``` -Without `PUBNUB_ADMIN_API_KEY` it only checks through the SDK that each class exists. The Admin API key needs a DataSync permission row for the datasync keyset. +Without `PUBNUB_ADMIN_API_KEY`, the script only checks through the SDK that each class exists. The Admin API key needs a DataSync permission row for the DataSync keyset. ### Environment variables @@ -125,11 +147,11 @@ Without `PUBNUB_ADMIN_API_KEY` it only checks through the SDK that each class ex | `LIVE_PREFLIGHT=off` | skip the preflight | | `LIVE_ENV_FILE` | another env file | -Everything the janitor, the preflight and the provisioning print goes through `redact()` (`support/env.ts`), which masks every key of the environment, the admin key, key-shaped strings and long token-like strings. +Everything the janitor, the preflight and the provisioning print goes through `redact()` (`support/env.ts`). It masks every key of the environment, the admin key, key-shaped strings and long token-like strings. ## Offline checks: marker lint, coverage manifest, docs cross-check -Three scripts that need no keys and no network. They answer one question: does every snippet the public docs embed from this repository run against the real service in a test here, or sit on a listed exception with a reason? +Three scripts need no keys and no network. They answer one question: does every snippet that the public docs embed from this repository run against the real service in a test here, or is it on a listed exception with a reason? ```bash npm run test:integration:live:lint # marker lint + manifest --check + eslint of the tier @@ -137,8 +159,41 @@ npm run test:integration:live:manifest # regenerate coverage.json npm run test:integration:live:docs-check # docs cross-check (needs the documentation checkout) ``` -**Lint** (`support/lint-markers.ts`; a self-test with deliberately broken files is `support/lint-selftest.ts`). It scans every `*.test.ts` and sample file under this folder and fails with `file:line` messages on: malformed, unbalanced or nested markers (the docs extractor would silently cut the outer snippet), `snippet.hide` without `snippet.show`, hidden blocks longer than 3 lines, an id outside `[a-zA-Z0-9_-]`, an id used twice anywhere in the tier, an empty snippet, test-only code in the visible text (`assert`, `expect`, `eventually`, `watch`, anything imported from `support/` or `node:assert`), and `try` / `catch` / `console.error` in a sample (allow-list `ERROR_HANDLING_SNIPPETS` and the `use-cases/` folder, `ERROR_HANDLING_FOLDERS`, both in the lint file). The `use-cases/` exemption is a decision of the docs owner: use-case samples are application code that shows its own error handling (try/catch around each call, `console.error`), so the protection against a swallowed error there is the assertion after the snippet. Instead of the old run-time proof it checks statically, with the TypeScript compiler API, that each snippet sits inside an `it()` (or a named helper an `it()` calls), that an assertion call follows the snippet's end line in that `it()` (`assert*`, `rejects*`, `expect*`, `eventually`, or an `await` of a promise armed earlier by such a call, like `const back = expectStatus(...)` ... `await back`), and that the snippet has no `return` of its own that could skip the rest. When the static check cannot see an assertion, put `// lint: asserted-via ` after the snippet in the `it()` body: the helper must be called after the snippet, and the lint prints every use. Only import-only snippets may sit at the top of a file (mode `compile-only`). One narrow escape exists for a file-level sample that starts with an `import` but also holds functions and other code (it cannot sit in an `it()`): put `// lint: asserted-by ` on the line right after its `// snippet.end`. The text must match exactly one `it()` title of the file, that `it()` must contain an assertion call, the manifest records mode `live` with that test, and the lint prints every use as a note (today: `fanBehaviorStateStore`, whose state file the test reads back). A hidden block stays at 3 lines or fewer: put setup in a helper defined before the snippet and call it from the hidden block. Known blind spots: the check proves an assertion call exists after the snippet, not that it checks the snippet's effect; a helper called through a variable or callback is not followed. +**Lint** (`support/lint-markers.ts`). A self-test with deliberately broken files is in `support/lint-selftest.ts`. The lint scans every `*.test.ts` file and every sample file in this folder. It fails with `file:line` messages on these problems: + +- Malformed, unbalanced or nested markers. The docs extractor would silently cut the outer snippet. +- `snippet.hide` without `snippet.show`. +- Hidden blocks longer than 3 lines. +- An id outside `[a-zA-Z0-9_-]`. +- An id used twice anywhere in the tier. +- An empty snippet. +- Test-only code in the visible text: `assert`, `expect`, `eventually`, `watch`, or anything imported from `support/` or `node:assert`. +- `try`, `catch` or `console.error` in a sample. The allow-list is `ERROR_HANDLING_SNIPPETS`, and the `use-cases/` folder is listed in `ERROR_HANDLING_FOLDERS`. Both live in the lint file. + +The `use-cases/` exemption is a decision of the docs owner. Use-case samples are application code that shows its own error handling: try/catch around each call, and `console.error`. So the protection against a swallowed error there is the assertion after the snippet. + +Instead of a run-time proof, the lint checks three things statically, with the TypeScript compiler API: + +- Each snippet sits inside an `it()`, or inside a named helper that an `it()` calls. +- An assertion call follows the snippet's end line in that `it()`. Assertion calls are `assert*`, `rejects*`, `expect*`, `eventually`, or an `await` of a promise that an earlier assertion call armed. For example: `const back = expectStatus(...)`, then `await back`. +- The snippet has no `return` of its own that could skip the rest. + +When the static check cannot see an assertion, add `// lint: asserted-via ` after the snippet, inside the `it()` body. The helper must be called after the snippet. The lint prints every use. + +Only import-only snippets may sit at the top of a file (mode `compile-only`). A file-level sample that starts with an `import` but also holds functions and other code cannot sit in an `it()`. For that case, one narrow escape exists: put `// lint: asserted-by ` on the line right after its `// snippet.end`. The text must match exactly one `it()` title in the file. That `it()` must contain an assertion call. The manifest records mode `live` for that test. The lint prints every use as a note. Today only `fanBehaviorStateStore` uses it, because its test reads back the state file. + +A hidden block stays at 3 lines or fewer. Put setup in a helper defined before the snippet, and call the helper from the hidden block. + +Known blind spots: the check proves that an assertion call exists after the snippet. It does not prove that the assertion checks the snippet's effect. A helper called through a variable or a callback is not followed. + +**Manifest** (`support/manifest.ts`). It writes `coverage.json`. That file is generated, so do not edit it. It has one entry per snippet id, with these fields: the file, the full Mocha title that runs the snippet (describe plus it), the keyset role when the test names one (a heuristic), and `mode`. The modes are: + +- `live`: inside an `it()`. +- `compile-only`: import-only samples. They are type-checked and loaded, but not run as an API call. +- `mocked`: inside a nock-mocked test listed in `MOCKED_FILES` in `support/snippet-scan.ts`: `test/integration/endpoints/objects/channel.test.ts`, `test/integration/endpoints/time.test.ts` and `test/integration/components/subscription_manager.test.ts`. These cover five samples. The same lint rules apply, except the title-names-the-id warning, which is the live tier's convention. + +`files` lists whole-file samples that a test runs (`getting-started/quickstart-node.mjs`). `-- --check` fails when `coverage.json` is stale. Docs embeds that are deliberately not run (the browser, React and React Native quickstarts) are listed by hand, with a reason, in `snippet-exceptions.json`. -**Manifest** (`support/manifest.ts`, output `coverage.json`, generated, do not edit). One entry per snippet id: file, the full Mocha title that runs it (describe + it), keyset role when the test names one (heuristic), and `mode`: `live` (inside an `it()`), `compile-only` (import-only samples: type-checked and loaded, not run as an API call), `mocked` (inside a nock-mocked test listed in `MOCKED_FILES` in `support/snippet-scan.ts`: `test/integration/endpoints/objects/channel.test.ts`, `test/integration/endpoints/time.test.ts`, `test/integration/components/subscription_manager.test.ts`, five samples; the same lint rules apply, except the title-names-the-id warning, which is the live tier's convention). `files` lists whole-file samples that a test runs (`getting-started/quickstart-node.mjs`). `-- --check` fails when `coverage.json` is stale. Things the docs embed that are deliberately not run (the browser, React and React Native quickstarts) are listed by hand, with a reason, in `snippet-exceptions.json`. +**Docs cross-check** (`support/docs-check.ts`). It reads every `` component that points at `pubnub/javascript` in the documentation checkout. The checkout path comes from the first argument, from `DOCS_REPO`, or from the default `../documentation`. Each embed is reported as live, compile-only, mocked, exception, or NOT COVERED. The command exits with code 1 when an embed is NOT COVERED, unless `--report-only` is passed. -**Docs cross-check** (`support/docs-check.ts`). Reads every `` that points at `pubnub/javascript` in the documentation checkout (first argument, `DOCS_REPO`, or `../documentation`) and reports each embed as live, compile-only, mocked, exception or NOT COVERED (exit code 1 unless `--report-only`). The URL must name a file under `test/integration/live/` or one of the mocked test files: the file must exist and `coverage.json` must list each requested snippet id for that very file (a whole-file embed needs a file named by a test in `coverage.json`, or a listed exception). Any other path, such as a leftover `docs-snippets/...` URL, is NOT COVERED. The pages `users`, `spaces`, `memberships` and `access-manager-v2` of the API reference are out of scope. The documentation repository is only read. +The URL must name a file under `test/integration/live/` or one of the mocked test files. The file must exist, and `coverage.json` must list each requested snippet id for that same file. A whole-file embed needs a file that a test in `coverage.json` names, or a listed exception. Any other path, such as a leftover `docs-snippets/...` URL, is NOT COVERED. The API reference pages `users`, `spaces`, `memberships` and `access-manager-v2` are out of scope. This script only reads the documentation repository. From 4d7908e52519e5de200f3357c5dcd5c548c1fb67 Mon Sep 17 00:00:00 2001 From: Mateusz Wiktor Date: Wed, 18 Nov 2026 08:31:56 +0100 Subject: [PATCH 10/13] feat: introduce API groups for live integration tests - Added support for grouping live integration tests into distinct API groups, enhancing organization and execution control. - Implemented `groups.cjs` to define groups, roles, and associated test files, ensuring each test file belongs to exactly one group. - Updated Mocha configurations to utilize group specifications, allowing targeted test execution via the `LIVE_GROUP` environment variable. - Enhanced the CI workflow to accommodate group-based testing, improving test management and reporting. - Refined README documentation to explain the new grouping structure and its implications for running tests. --- .github/workflows/live-integration.yml | 87 ++++++++--- package.json | 3 +- test/integration/live/.mocharc.cjs | 5 +- test/integration/live/.mocharc.endpoints.cjs | 5 +- test/integration/live/README.md | 57 +++++-- .../endpoints/data_sync/projections.test.ts | 32 ++-- test/integration/live/groups.cjs | 117 ++++++++++++++ .../integration/live/support/group-select.cjs | 26 ++++ test/integration/live/support/groups.ts | 146 ++++++++++++++++++ test/integration/live/support/hooks.ts | 28 +++- test/integration/live/support/preflight.ts | 12 +- 11 files changed, 461 insertions(+), 57 deletions(-) create mode 100644 test/integration/live/groups.cjs create mode 100644 test/integration/live/support/group-select.cjs create mode 100644 test/integration/live/support/groups.ts diff --git a/.github/workflows/live-integration.yml b/.github/workflows/live-integration.yml index eb26e74fd..a53528403 100644 --- a/.github/workflows/live-integration.yml +++ b/.github/workflows/live-integration.yml @@ -1,8 +1,14 @@ -name: Live integration +name: Live tests -# Live tier (test/integration/live/README.md): the public documentation's code samples run against the real PubNub service. -# Runs on non-draft pull requests into master, and on demand. No scheduled runs. Draft pull requests run nothing. Fork pull -# requests never get the live job (no secrets for fork code). +# Live tier (test/integration/live/README.md): the public documentation's code samples, and the real-network endpoint tests, run +# against the real PubNub service. Runs on non-draft pull requests into master, and on demand. No scheduled runs. Draft pull +# requests run nothing. Fork pull requests never get the live jobs (no secrets for fork code). +# +# One job per API group. The groups and their files are in test/integration/live/groups.cjs. The offline job turns that file into +# the job matrix, so a new group needs no change here except the `group` input options below (the lint checks that they match). +# The matrix runs one group at a time (max-parallel: 1) and does not stop at a failed group (fail-fast: false), so every group +# reports its own result and the run is red when any group failed. The groups do not depend on each other: each one checks out, +# builds, cleans the keysets before and after its tests, and can be re-run alone ("Re-run job" on that group). on: # Pull requests into master, only when something the tier or its mocked samples depend on changed. Marking a draft as ready @@ -20,12 +26,35 @@ on: - '.github/workflows/live-integration.yml' workflow_dispatch: inputs: + group: + description: 'Run only this group (test/integration/live/groups.cjs). "all" runs every group, one after another.' + required: false + default: 'all' + type: choice + # Must list 'all' and then every group name of groups.cjs, in that order (checked by npm run test:integration:live:lint). + options: + - all + - messaging + - getting-started + - use-cases + - access-manager + - app-context + - datasync + - endpoints-datasync + - endpoints-history grep: - description: 'Run only tests whose title matches this pattern (mocha --grep). Empty runs everything.' + description: 'Run only tests whose title matches this pattern (mocha --grep), inside the selected groups. Empty runs everything.' required: false default: '' type: string +# One live run at a time for the whole workflow: the tier uses fixed resource names on shared keysets, so the jobs of two runs must +# never interleave. A running run is never cancelled. Runs that get no live jobs (draft or fork pull requests) take a group of +# their own, so they never wait and never replace a pending live run. +concurrency: + group: ${{ (github.event_name == 'pull_request' && (github.event.pull_request.draft || github.event.pull_request.head.repo.full_name != github.repository)) && format('live-tests-no-live-{0}', github.run_id) || 'docs-snippets-live' }} + cancel-in-progress: false + defaults: run: shell: bash @@ -40,6 +69,8 @@ jobs: runs-on: group: organization/Default timeout-minutes: 15 + outputs: + matrix: ${{ steps.plan.outputs.matrix }} steps: - name: Checkout repository uses: actions/checkout@v7 @@ -54,24 +85,33 @@ jobs: run: npm ci - name: Build the SDK for Node run: npm run build:node - - name: Lint (markers, manifest, eslint) + - name: Lint (markers, manifest, groups, eslint) run: npm run test:integration:live:lint - name: Type check run: npm run test:integration:live:types + - name: Plan the group jobs + id: plan + env: + # Empty on pull requests: every group. Reaches the script only through the environment. + GROUP: ${{ inputs.group }} + run: | + matrix="$(npm run --silent test:integration:live:groups -- --matrix "${GROUP:-all}")" + echo "matrix=$matrix" >> "$GITHUB_OUTPUT" + echo "$matrix" live: - name: Live tier + name: ${{ matrix.group }} needs: [offline] # Pull requests: only non-draft ones from a branch of this repository (never a fork: no secrets for fork code). if: ${{ github.event_name != 'pull_request' || (!github.event.pull_request.draft && github.event.pull_request.head.repo.full_name == github.repository) }} runs-on: group: organization/Default - timeout-minutes: 40 - # One live run per keyset at a time: this tier uses fixed resource names, so two runs must never overlap. A running job is - # never cancelled. - concurrency: - group: docs-snippets-live - cancel-in-progress: false + timeout-minutes: 20 + strategy: + # One group at a time (the keysets are shared), and every group runs even after a failed one. + max-parallel: 1 + fail-fast: false + matrix: ${{ fromJSON(needs.offline.outputs.matrix) }} env: # Not a secret. Set on the job (not the steps) because it is only a keyset id for the Admin API feature check. DOCS_SNIPPETS_DS_KEYSET_ID: ${{ vars.SDK_DOCS_DS_KEYSET_ID }} @@ -91,17 +131,20 @@ jobs: - name: Build the SDK for Node id: build run: npm run build:node + # Only the groups that use the DataSync classes of the docs keyset (docs tier, role datasync). Read only. - name: Check the DataSync classes (read only) - if: ${{ steps.build.outcome == 'success' }} + if: ${{ steps.build.outcome == 'success' && matrix.provision }} env: DS_SUBSCRIBE_KEY: ${{ secrets.SDK_DOCS_DS_SUBSCRIBE_KEY }} DS_PUBLISH_KEY: ${{ secrets.SDK_DOCS_DS_PUBLISH_KEY }} DS_SECRET_KEY: ${{ secrets.SDK_DOCS_DS_SECRET_KEY }} PUBNUB_ADMIN_API_KEY: ${{ secrets.SDK_DOCS_ADMIN_API_KEY }} run: npm run test:integration:live:provision:check + # Every docs keyset, whatever the group: the janitor cleans every role before and after the group, so each job starts and + # ends clean even when the job before it failed. The role check and the preflight cover only the group's roles. - name: Docs samples (janitor and preflight on) id: samples - if: ${{ !cancelled() && steps.build.outcome == 'success' }} + if: ${{ !cancelled() && steps.build.outcome == 'success' && matrix.tier == 'docs' }} env: SUBSCRIBE_KEY: ${{ secrets.SDK_DOCS_OPEN_SUBSCRIBE_KEY }} PUBLISH_KEY: ${{ secrets.SDK_DOCS_OPEN_PUBLISH_KEY }} @@ -112,6 +155,7 @@ jobs: DS_PUBLISH_KEY: ${{ secrets.SDK_DOCS_DS_PUBLISH_KEY }} DS_SECRET_KEY: ${{ secrets.SDK_DOCS_DS_SECRET_KEY }} PUBNUB_ADMIN_API_KEY: ${{ secrets.SDK_DOCS_ADMIN_API_KEY }} + LIVE_GROUP: ${{ matrix.group }} # Untrusted input: reaches the script only through the environment. GREP: ${{ inputs.grep }} run: | @@ -121,19 +165,19 @@ jobs: npm run test:integration:live fi # The SDK's own keyset on purpose (the one run-tests.yml uses), not the docs keysets: the real-network endpoint tests are not - # documentation samples. The six DataSync files need the DataSync classes that exist only on the SDK's DataSync keyset + # documentation samples. The DataSync files need the DataSync classes that exist only on the SDK's DataSync keyset # (SDK_DS_*), and fetch_messages, history and message_actions need the SDK's open keyset with Message Persistence and - # message actions (SDK_SUB_KEY / SDK_PUB_KEY). Runs after a failed samples step too, so both results show. If either step - # fails the job fails (a failed step marks the job failed even when later steps pass). + # message actions (SDK_SUB_KEY / SDK_PUB_KEY). Janitor and preflight are off in this tier (.mocharc.endpoints.cjs). - name: Endpoint tests (SDK keysets) id: endpoints - if: ${{ !cancelled() && steps.build.outcome == 'success' }} + if: ${{ !cancelled() && steps.build.outcome == 'success' && matrix.tier == 'endpoints' }} env: SUBSCRIBE_KEY: ${{ secrets.SDK_SUB_KEY }} PUBLISH_KEY: ${{ secrets.SDK_PUB_KEY }} DS_SUBSCRIBE_KEY: ${{ secrets.SDK_DS_SUB_KEY }} DS_PUBLISH_KEY: ${{ secrets.SDK_DS_PUB_KEY }} DS_SECRET_KEY: ${{ secrets.SDK_DS_SEC_KEY }} + LIVE_GROUP: ${{ matrix.group }} GREP: ${{ inputs.grep }} run: | if [ -n "$GREP" ]; then @@ -142,9 +186,10 @@ jobs: npm run test:integration:live:endpoints fi # After a failure or a cancel the tier's own after-hook may not have run (killed process): revoke application-level - # Access Manager grants and remove leftovers, so a keyset is never left open until the next run. + # Access Manager grants and remove leftovers, so a keyset is never left open until the next group or run. Docs tier only: + # the endpoint tests never touch the docs keysets and remove their own uniquely named objects. - name: Revoke grants and clean up - if: ${{ (cancelled() || failure()) && steps.build.outcome == 'success' }} + if: ${{ (cancelled() || failure()) && steps.build.outcome == 'success' && matrix.tier == 'docs' }} env: SUBSCRIBE_KEY: ${{ secrets.SDK_DOCS_OPEN_SUBSCRIBE_KEY }} PUBLISH_KEY: ${{ secrets.SDK_DOCS_OPEN_PUBLISH_KEY }} diff --git a/package.json b/package.json index 077831388..29a1dcca3 100644 --- a/package.json +++ b/package.json @@ -20,7 +20,8 @@ "test:integration:live:janitor": "TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json tsx test/integration/live/support/janitor-cli.ts", "test:integration:live:provision": "TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json tsx test/integration/live/provision/provision-datasync.ts", "test:integration:live:provision:check": "TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json tsx test/integration/live/provision/provision-datasync.ts --check", - "test:integration:live:lint": "TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json tsx test/integration/live/support/lint-markers.ts && TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json tsx test/integration/live/support/manifest.ts --check && eslint \"test/integration/live/**/*.ts\" --ignore-pattern \"test/integration/live/quickstarts/**\"", + "test:integration:live:lint": "TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json tsx test/integration/live/support/lint-markers.ts && TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json tsx test/integration/live/support/manifest.ts --check && TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json tsx test/integration/live/support/groups.ts --check && eslint \"test/integration/live/**/*.ts\" --ignore-pattern \"test/integration/live/quickstarts/**\"", + "test:integration:live:groups": "TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json tsx test/integration/live/support/groups.ts", "test:integration:live:manifest": "TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json tsx test/integration/live/support/manifest.ts", "test:integration:live:docs-check": "TSX_TSCONFIG_PATH=test/integration/live/tsconfig.run.json tsx test/integration/live/support/docs-check.ts", "ci": "npm run clean && npm run build && npm run lint && npm run test", diff --git a/test/integration/live/.mocharc.cjs b/test/integration/live/.mocharc.cjs index 5d6adcc02..342494563 100644 --- a/test/integration/live/.mocharc.cjs +++ b/test/integration/live/.mocharc.cjs @@ -1,9 +1,12 @@ // Mocha config of the live tier's docs-sample tests (everything except endpoints/). Run it through `npm run test:integration:live`, // which also sets TSX_TSCONFIG_PATH so that `import PubNub from 'pubnub'` resolves to the built library (lib/node/index.js). // Paths are relative to the repository root. The tests of endpoints/ have their own config, .mocharc.endpoints.cjs. +// `LIVE_GROUP=` runs only the files of that docs-tier group of groups.cjs (support/group-select.cjs). +const { groupSpec } = require('./support/group-select.cjs'); + module.exports = { require: ['tsx', 'test/integration/live/support/hooks.ts'], - spec: ['test/integration/live/**/*.test.ts'], + spec: groupSpec('docs', ['test/integration/live/**/*.test.ts']), exclude: ['test/integration/live/endpoints/**'], timeout: 30000, retries: 0, diff --git a/test/integration/live/.mocharc.endpoints.cjs b/test/integration/live/.mocharc.endpoints.cjs index 7ac547531..04bec9298 100644 --- a/test/integration/live/.mocharc.endpoints.cjs +++ b/test/integration/live/.mocharc.endpoints.cjs @@ -4,13 +4,16 @@ // message naming the env variables when they are absent, so a --grep run needs only the keys of the selected tests. The janitor and the keyset preflight are off by default: their resource list and feature checks // belong to the docs keysets, and these tests clean up their own uniquely named objects. `LIVE_JANITOR=on` or `LIVE_PREFLIGHT=on` // in the environment turns either back on. A spec argument on the command line is added to the spec below (Mocha merges them). +// `LIVE_GROUP=` runs only the files of that endpoint-tier group of groups.cjs (support/group-select.cjs). +const { groupSpec } = require('./support/group-select.cjs'); + process.env.LIVE_TIER = 'endpoints'; process.env.LIVE_JANITOR ||= 'off'; process.env.LIVE_PREFLIGHT ||= 'off'; module.exports = { require: ['tsx', 'test/integration/live/support/hooks.ts'], - spec: ['test/integration/live/endpoints/**/*.test.ts'], + spec: groupSpec('endpoints', ['test/integration/live/endpoints/**/*.test.ts']), timeout: 30000, retries: 0, reporter: 'spec', diff --git a/test/integration/live/README.md b/test/integration/live/README.md index 923109555..515ecbe7d 100644 --- a/test/integration/live/README.md +++ b/test/integration/live/README.md @@ -11,13 +11,15 @@ npm run test:integration:live # the docs-sample tier: eve npm run test:integration:live:endpoints # only endpoints/**: real-network endpoint tests that are not doc samples (each test needs its own role: DS_* for DataSync, SUBSCRIBE_KEY and PUBLISH_KEY for the rest) npm run test:integration:live -- --grep publishMinimal npm run test:integration:live -- --dry-run # lists the tests, no network, no keys needed +LIVE_GROUP=publish npm run test:integration:live # one API group of groups.cjs (see "Groups and CI jobs") +LIVE_GROUP=endpoints-datasync npm run test:integration:live:endpoints # one group of the endpoint tier npm run test:integration:live:types # type check (needs the build for lib/types) npx eslint "test/integration/live/**/*.ts" --ignore-pattern "test/integration/live/quickstarts/**" # lint (npm run lint covers src/ only). Do not add --config: it drops the max-len override of live/.eslintrc.cjs ``` Environment variable names: `SUBSCRIBE_KEY` and `PUBLISH_KEY` (open keyset), `PAM_SUBSCRIBE_KEY`, `PAM_PUBLISH_KEY` and `PAM_SECRET_KEY` (Access Manager, PAM), and `DS_*` (DataSync, DS). Real environment variables take precedence over the file. `LIVE_ENV_FILE=` points to another file. The run stops at once if the open keyset is missing. -A docs-tier run without `--grep` and without spec file arguments needs every role. It fails at once, and names the missing variables, when `pam` or `datasync` is not set. A run with `--grep` or a file argument checks a role only when a test builds its client. So a page that needs only the open keyset runs with only that keyset. Run one live run per keyset at a time. +A docs-tier run without `--grep` and without spec file arguments needs every role. It fails at once, and names the missing variables, when `pam` or `datasync` is not set. A group run (`LIVE_GROUP`) without `--grep` needs exactly the `roles` of its group, in both tiers. A run with `--grep` or a file argument checks a role only when a test builds its client. So a page that needs only the open keyset runs with only that keyset. Run one live run per keyset at a time. **Two runs, two keysets.** The tier has two Mocha configuration files. @@ -30,7 +32,7 @@ The DataSync endpoint tests need the DataSync classes `JSCustomer`, `JSLoanQuote The other three files (`fetch_messages`, `history`, `message_actions`) publish to the real service and read the result back. They moved out of `test/integration/endpoints/`. The files with the same names there are mocked. This keeps `npm run test:node` fully offline. A root hook in `test/setup-why.ts` refuses every request that no nock scope matches. These three files are not documentation samples, and they carry no `// snippet.` markers. -Continuous Integration (CI) runs the whole folder as one separate step. It uses the SDK's own keysets, the ones `run-tests.yml` uses: `SDK_SUB_KEY` and `SDK_PUB_KEY` as `SUBSCRIBE_KEY` and `PUBLISH_KEY`, and `SDK_DS_*` as `DS_*`. It does not use the docs keysets. +Continuous Integration (CI) runs this folder in two jobs of its own, `endpoints-datasync` and `endpoints-history`. They use the SDK's own keysets, the ones `run-tests.yml` uses: `SDK_SUB_KEY` and `SDK_PUB_KEY` as `SUBSCRIBE_KEY` and `PUBLISH_KEY`, and `SDK_DS_*` as `DS_*`. It does not use the docs keysets. Each test titles itself `endpoints/: ...`. So `--grep '^endpoints/(fetch_messages|history|message_actions)'` runs only the open-keyset files, and `--grep '^endpoints/data_sync'` runs only the DataSync files. A test whose role is not set fails with `missing keys: the keyset is not configured, set `. @@ -40,7 +42,38 @@ The janitor's resource list and the preflight's feature checks belong to the doc Mocha adds a spec file from the command line to the config's `spec`. It does not replace it. So the docs-tier config excludes `endpoints/**` with `exclude`, and a file argument for the other tier is not added to a run. Check what a script runs with `-- --dry-run`. The root `npm run test:node` lists none of this folder. -The workflow (`.github/workflows/live-integration.yml`) runs on non-draft pull requests into `master`, when the paths it watches change, and on demand. It has no scheduled runs. It does not alert anyone on failure yet. The GitHub run logs are the only signal. +The workflow (`.github/workflows/live-integration.yml`, named "Live tests") runs on non-draft pull requests into `master`, when the paths it watches change, and on demand. It has no scheduled runs. It does not alert anyone on failure yet. The GitHub run logs are the only signal. + +## Groups and CI jobs + +`groups.cjs` splits the tier into API groups. Each group has a `name`, a `tier` (`docs` or `endpoints`), the keyset `roles` its tests use, and its `files`. `LIVE_GROUP=` makes `.mocharc.cjs` or `.mocharc.endpoints.cjs` run only the files of that group (`support/group-select.cjs`). The config's `spec` is replaced, so no other file runs. A name of the other tier stops the run and names the right script. `--grep` still works inside a group. + +| Group | Tier | Files | About | +| :--- | :--- | :--- | :--- | +| `messaging` | docs | `api-reference/publish*`, `subscribe*`, `presence`, `channel-groups`, `file-sharing*`, `mobile-push`, `message-persistence*`, `message-actions` | 130 s | +| `getting-started` | docs | `getting-started/*`, `shared/*`, `api-reference/configuration*`, `encryption*`, `misc`, `migration-guides/*` | 100 s | +| `use-cases` | docs | `use-cases/*` | 180 s | +| `access-manager` | docs | `api-reference/access-manager*` | 60 s | +| `app-context` | docs | `api-reference/objects` | 40 s | +| `datasync` | docs | `api-reference/datasync` | 50 s | +| `endpoints-datasync` | endpoints | `endpoints/data_sync/*` | 90 s | +| `endpoints-history` | endpoints | `endpoints/fetch_messages`, `history`, `message_actions` | 3 s | + +`groups.cjs` is the source of truth. The table above is a summary, and the times are rough runtimes of a green run. + +`datasync`, `endpoints-datasync`, `app-context` and `endpoints-history` stay separate because they failed most often in CI history, so a single re-run does not repeat stable tests. `access-manager` stays alone too, because it needs the DataSync role that the flaky groups share. The endpoint groups use other keysets and another Mocha config, so they never merge with docs groups. + +**Every test file is in exactly one group.** `npm run test:integration:live:lint` runs `support/groups.ts --check`. It fails when a test file is in no group or in two groups, when a listed file does not exist or sits in the other tier, when a file uses the `pam` or `datasync` keyset and its group does not list that role, and when the workflow's `group` input options differ from the group names. So a new test file fails the lint until you add it to a group. So the groups together run exactly the full run. Compare the `-- --dry-run` lists when in doubt. + +**One job per group.** The offline job checks the lint and types, then turns `groups.cjs` into the job matrix (`npm run test:integration:live:groups -- --matrix all`). The `live` job runs one matrix entry per group, one at a time (`max-parallel: 1`). A failed group does not stop the others (`fail-fast: false`), and the run is red when any group failed. Each job checks out, installs, builds and runs its group on its own. No job depends on another, so a single job can be re-run. + +- Re-run one failed group in Actions: open the run, select the group's job, and choose "Re-run job". "Re-run failed jobs" re-runs only the failed groups. +- Start one group by hand: run the workflow with the `group` input set to its name (`all` runs every group). The `grep` input narrows the tests inside the selected groups. +- Run one group locally: `LIVE_GROUP= npm run test:integration:live` (docs tier) or `LIVE_GROUP= npm run test:integration:live:endpoints`. + +The whole workflow has one concurrency group, `docs-snippets-live`, with `cancel-in-progress: false`. So the jobs of two runs never interleave on the keysets. A run that gets no live job (a draft or fork pull request) takes a group of its own. + +To add a group: add it to `groups.cjs`, then add its name to the `group` input options of the workflow, in the same order. The lint fails until both match. Rebuild with `npm run build:node` after a change under `src/`. Otherwise the tier tests the previous build. @@ -87,7 +120,8 @@ Rebuild with `npm run build:node` after a change under `src/`. Otherwise the tie | `explain-error.ts` | `explainSdkError(err)`: appends the SDK status summary to a failure's message; installed by `hooks.ts` | | `console.ts` | `captureConsole(method)`, `quietConsole()`, `restoreAllConsoles()` | | `resources.ts` | the janitor's resource list; `DATASYNC_PAGE_OBJECTS` is shared with `datasync-state.ts` | -| `hooks.ts` | root hooks: role check before a full run, janitor, preflight, client cleanup after each test | +| `hooks.ts` | root hooks: role check before a full run or a group run, janitor, preflight, client cleanup after each test | +| `groups.ts` | `GROUPS`, `selectedGroup()`, `checkGroups()`, `matrixOf()`; `--check` and `--matrix` on the command line | | `snippets.ts` | snippet extraction with the rules of the docs extractor | Other files in `support/` serve one page or one tool (`janitor*.ts`, `preflight.ts`, `datasync-*.ts`, `subscribe-groups.ts`, `proxy.ts`, `file-workdir.ts`, `access-manager.ts`). @@ -108,26 +142,30 @@ The tests use the literal names that the docs show (`'my_channel'`, `'family'`, | pam | revokes the application-level Access Manager grant; the same lists as open when the role has any; history of its listed channels | | datasync | the same revoke (once when pam and datasync are one keyset); DataSync relationships and memberships, then entities, users and channels from the list; history | -The janitor runs automatically from the root hooks. It runs once before all tests. A failure fails the run, and the output lists what could not be removed. On a secured keyset, the janitor then waits until a client without a token is refused again. That can take up to about 60 s when the previous run just changed a grant. It gives up after 120 s. It also runs once after all tests. A failure there prints a loud banner and does not fail the run. The exception is a failed grant revoke, which leaves a secured keyset open. The next run's cleanup retries before the run, and it fails if the keyset is still dirty. +The janitor runs automatically from the root hooks. It runs once before all tests of a run, so once before each group job in CI. A failure fails the run, and the output lists what could not be removed. On a secured keyset, the janitor then waits until a client without a token is refused again. That can take up to about 60 s when the previous run just changed a grant. It gives up after 120 s. This wait runs only when the run uses a secured keyset: always for a run of the whole tier, and for a group only when its `roles` list `pam` or `datasync`. It also runs once after all tests. A failure there prints a loud banner and does not fail the run. The exception is a failed grant revoke, which leaves a secured keyset open. The next run's cleanup retries before the run, and it fails if the keyset is still dirty. ```bash npm run test:integration:live:janitor # the janitor alone, all roles; exit code 1 when something could not be removed npm run test:integration:live:janitor -- --role open ``` -A full pass walks every listed name, so it takes a while. For a single page or a `--grep` run, turn the janitor off: `LIVE_JANITOR=off npm run test:integration:live -- --grep publishMinimal`. Run the janitor command before and after such runs when you want a clean slate. +A full pass walks every listed name. In CI it takes about 4 s, plus about 3 s for the wait on the secured keyset. The tests take the rest of the time. For a single page or a `--grep` run, you can turn the janitor off: `LIVE_JANITOR=off npm run test:integration:live -- --grep publishMinimal`. Run the janitor command before and after such runs when you want a clean slate. + +**Not scoped by group.** A group job cleans the whole list for every role that has keys, not only the names of its group. Most names (`my_channel`, `game.chat` and many others) are shared by several groups. A list per group could miss a name without any error. A full pass is cheap, and it starts each job from a clean keyset whatever the job before it left. After a failed or cancelled docs-tier job, the workflow runs `npm run test:integration:live:janitor` again. That revokes the application-level Access Manager grant and waits until the secured keyset refuses a client without a token. **Adding a name.** Add one string to the right list of the right role in `support/resources.ts`: `channels`, `fileChannels`, `channelGroups`, `appContext.users`, `appContext.channels`, `pushDevices`, or `dataSync.*`. The file holds data only. A resource that a test creates and does not list is never cleaned, so it stays on the keyset from run to run. One channel is left out on purpose: `history_documented_behavior`. It holds more than 100 messages, which its test writes once and reuses (see the comment in `resources.ts`). Names with a per-run suffix are not listed either. Their messages expire with the keyset's retention period. A channel in a channel group must also be listed under `channels`, because deleting a group does not delete its member channels. ### Preflight (`support/preflight.ts`) -After the janitor, the root hook fails fast. Its message names each missing feature. With `PUBNUB_ADMIN_API_KEY` and the keyset ids (`DOCS_SNIPPETS_OPEN_KEYSET_ID`, `DOCS_SNIPPETS_PAM_KEYSET_ID`, `DOCS_SNIPPETS_DS_KEYSET_ID`), the hook reads each keyset's configuration through the Admin API, read-only. It lists what the role needs and lacks: Presence, Message Persistence, Delete-From-History, App Context and its events, Files, Stream Controller, Access Manager and DataSync. +After the janitor, the root hook fails fast. Its message names each missing feature. A group run checks only the roles of its group. With `PUBNUB_ADMIN_API_KEY` and the keyset ids (`DOCS_SNIPPETS_OPEN_KEYSET_ID`, `DOCS_SNIPPETS_PAM_KEYSET_ID`, `DOCS_SNIPPETS_DS_KEYSET_ID`), the hook reads each keyset's configuration through the Admin API, read-only. It lists what the role needs and lacks: Presence, Message Persistence, Delete-From-History, App Context and its events, Files, Stream Controller, Access Manager and DataSync. Live probes on the open keyset always run. They check publish (keys and network), Message Persistence, Delete-From-History, and Presence when the Admin API did not confirm it. The features each keyset needs are the ones the message names. `missingFeatures` in `support/preflight.ts` is the full list per role (open, pam, datasync). ### Provisioning (`provision/`) -`provision/provision-datasync.ts` creates the DataSync classes from `provision/datasync-classes.json`. The script is idempotent. It never deletes a class, and it fails on a class whose immutable fields differ. The tier does not run it. Run it when a DataSync keyset is new or when the class file changes. +`provision/provision-datasync.ts` creates the DataSync classes from `provision/datasync-classes.json`. The script is idempotent. It never deletes a class, and it fails on a class whose immutable fields differ. The tests never run it. Run it when a DataSync keyset is new or when the class file changes. + +CI runs only the read-only check (`provision:check`), as a step of each docs-tier group whose `roles` list `datasync` (`access-manager` and `datasync`; the merged groups `messaging` and `getting-started` do not list it). Those are the groups that use the DataSync classes of the docs keyset. A failed check does not skip the tests of the job. The endpoint tier uses other classes on another keyset, so its jobs do not run the check. ```bash npm run test:integration:live:provision:check # dry diff, writes nothing, exit code 1 when anything differs @@ -143,6 +181,7 @@ Without `PUBNUB_ADMIN_API_KEY`, the script only checks through the SDK that each | `SUBSCRIBE_KEY`, `PUBLISH_KEY`, `PAM_*`, `DS_*` | keysets, as above | | `PUBNUB_ADMIN_API_KEY` | optional; Admin API feature check and provisioning | | `DOCS_SNIPPETS_OPEN_KEYSET_ID`, `DOCS_SNIPPETS_PAM_KEYSET_ID`, `DOCS_SNIPPETS_DS_KEYSET_ID` | optional; keyset ids for the feature check (not secrets) | +| `LIVE_GROUP` | run only one group of `groups.cjs` | | `LIVE_JANITOR=off` | skip the janitor before and after the run (any other value runs it) | | `LIVE_PREFLIGHT=off` | skip the preflight | | `LIVE_ENV_FILE` | another env file | @@ -154,7 +193,7 @@ Everything the janitor, the preflight and the provisioning print goes through `r Three scripts need no keys and no network. They answer one question: does every snippet that the public docs embed from this repository run against the real service in a test here, or is it on a listed exception with a reason? ```bash -npm run test:integration:live:lint # marker lint + manifest --check + eslint of the tier +npm run test:integration:live:lint # marker lint + manifest --check + groups --check + eslint of the tier npm run test:integration:live:manifest # regenerate coverage.json npm run test:integration:live:docs-check # docs cross-check (needs the documentation checkout) ``` diff --git a/test/integration/live/endpoints/data_sync/projections.test.ts b/test/integration/live/endpoints/data_sync/projections.test.ts index 365c5e682..53f97507c 100644 --- a/test/integration/live/endpoints/data_sync/projections.test.ts +++ b/test/integration/live/endpoints/data_sync/projections.test.ts @@ -477,17 +477,17 @@ describe('endpoints/data_sync/projections: DataSync projections admin vs __defau channelPatterns: ['customer.*'], projections: { patterns: { - entities: { 'customer.*': PROJECTION_ADMIN, 'loanquote.*': PROJECTION_ADMIN }, - relationships: { 'requestedby.*': PROJECTION_ADMIN }, + entities: { '^customer\\..*': PROJECTION_ADMIN, '^loanquote\\..*': PROJECTION_ADMIN }, + relationships: { '^requestedby\\..*': PROJECTION_ADMIN }, }, }, }); assert.deepStrictEqual(projectionsMeta(token), { pat: { - 'datasync:entities:customer.*': PROJECTION_ADMIN, - 'datasync:entities:loanquote.*': PROJECTION_ADMIN, - 'datasync:relationships:requestedby.*': PROJECTION_ADMIN, + 'datasync:entities:^customer\\..*': PROJECTION_ADMIN, + 'datasync:entities:^loanquote\\..*': PROJECTION_ADMIN, + 'datasync:relationships:^requestedby\\..*': PROJECTION_ADMIN, }, }); }); @@ -524,9 +524,10 @@ describe('endpoints/data_sync/projections: DataSync projections admin vs __defau memberships: { 'user.001:channel.001': PROJECTION_DEFAULT }, }, patterns: { - users: { 'user.*': PROJECTION_DEFAULT }, - channels: { 'channel.*': PROJECTION_DEFAULT }, - memberships: { 'user.*:channel.*': PROJECTION_DEFAULT }, + users: { '^user\\..*': PROJECTION_DEFAULT }, + channels: { '^channel\\..*': PROJECTION_DEFAULT }, + // A wildcard in the middle (`user.*:channel.*`) is not a supported grant pattern, so match the user prefix only. + memberships: { '^user\\..*': PROJECTION_DEFAULT }, }, }, }); @@ -538,9 +539,9 @@ describe('endpoints/data_sync/projections: DataSync projections admin vs __defau 'datasync:memberships:user.001:channel.001': PROJECTION_DEFAULT, }, pat: { - 'datasync:users:user.*': PROJECTION_DEFAULT, - 'datasync:channels:channel.*': PROJECTION_DEFAULT, - 'datasync:memberships:user.*:channel.*': PROJECTION_DEFAULT, + 'datasync:users:^user\\..*': PROJECTION_DEFAULT, + 'datasync:channels:^channel\\..*': PROJECTION_DEFAULT, + 'datasync:memberships:^user\\..*': PROJECTION_DEFAULT, }, }); }); @@ -577,11 +578,14 @@ describe('endpoints/data_sync/projections: DataSync projections admin vs __defau /** Readers come from the tier's `createClient`, so the root hook destroys them after each test. */ const track = (reader: PubNub): PubNub => reader; - /** Projection assignments covering every object this section seeded. */ + /** + * Projection assignments covering every object this section seeded. DataSync grant patterns accept only + * `.*`, `^lit$`, `^lit.*`, `.*lit$` and literal alternative groups, so the `.` of the id prefix is escaped. + */ const allAdmin = (): PubNub.PAM.DataSyncProjections => ({ patterns: { - entities: { 'customer.*': PROJECTION_ADMIN, 'loanquote.*': PROJECTION_ADMIN }, - relationships: { 'requestedby.*': PROJECTION_ADMIN }, + entities: { '^customer\\..*': PROJECTION_ADMIN, '^loanquote\\..*': PROJECTION_ADMIN }, + relationships: { '^requestedby\\..*': PROJECTION_ADMIN }, }, }); diff --git a/test/integration/live/groups.cjs b/test/integration/live/groups.cjs new file mode 100644 index 000000000..aef8f7707 --- /dev/null +++ b/test/integration/live/groups.cjs @@ -0,0 +1,117 @@ +// API groups of the live tier. DATA ONLY. One group is one job of the "Live tests" workflow (.github/workflows/live-integration.yml) +// and one `LIVE_GROUP=` run of the npm scripts: +// +// LIVE_GROUP=publish npm run test:integration:live a docs-tier group (tier 'docs') +// LIVE_GROUP=endpoints-datasync npm run test:integration:live:endpoints an endpoint-tier group (tier 'endpoints') +// +// .mocharc.cjs and .mocharc.endpoints.cjs read this file and replace their `spec` with the group's files. support/hooks.ts reads +// `roles`: a group run without --grep needs exactly these keyset roles up front, and the preflight and the secured-keyset wait +// cover only these roles. The workflow builds its job matrix from this list (support/groups.ts --matrix). +// +// Every *.test.ts file of the tier belongs to exactly one group. `npm run test:integration:live:lint` runs +// `support/groups.ts --check`, which fails on a test file in no group or in two groups, a listed file that does not exist, a file in +// the wrong tier, a role that a file uses but its group does not list, and a workflow `group` input whose options differ from these +// names. Add a new test file to a group here, or the lint fails. +// +// Paths are relative to test/integration/live/. Order is the order of the serial run in CI. +module.exports = [ + // Stable groups are merged to keep the job count low. The groups that failed most often in CI history (datasync, endpoints-datasync, + // app-context, endpoints-history) stay alone, so one re-run does not repeat stable tests. access-manager is alone too: it needs + // the DataSync role of the docs keyset, which the flaky groups share. + { + name: 'messaging', + tier: 'docs', + roles: ['open', 'pam'], + files: [ + 'api-reference/publish.test.ts', + 'api-reference/publish-legacy.test.ts', + 'api-reference/subscribe.test.ts', + 'api-reference/subscribe-legacy.test.ts', + 'api-reference/presence.test.ts', + 'api-reference/channel-groups.test.ts', + 'api-reference/file-sharing.test.ts', + 'api-reference/file-sharing-legacy.test.ts', + 'api-reference/mobile-push.test.ts', + 'api-reference/message-persistence.test.ts', + 'api-reference/message-persistence-legacy.test.ts', + 'api-reference/message-actions.test.ts', + ], + }, + { + name: 'getting-started', + tier: 'docs', + roles: ['open', 'pam'], + files: [ + 'getting-started/logging.test.ts', + 'getting-started/logging-file-logger.test.ts', + 'getting-started/quickstart-node.test.ts', + 'getting-started/set-up-your-environment.test.ts', + 'getting-started/status-events.test.ts', + 'getting-started/troubleshooting.test.ts', + 'shared/import-pubnub.test.ts', + 'api-reference/configuration.test.ts', + 'api-reference/configuration-legacy.test.ts', + 'api-reference/encryption.test.ts', + 'api-reference/encryption-legacy.test.ts', + 'api-reference/misc.test.ts', + 'migration-guides/v8.test.ts', + 'migration-guides/v11.test.ts', + 'migration-guides/v12.test.ts', + ], + }, + { + name: 'use-cases', + tier: 'docs', + roles: ['open', 'pam'], + files: [ + 'use-cases/automated-polling.test.ts', + 'use-cases/fan-behavior-management.test.ts', + 'use-cases/fan-re-engagement.test.ts', + 'use-cases/game-chat-moderation.test.ts', + 'use-cases/live-commentary.test.ts', + 'use-cases/live-event-rate-limiting.test.ts', + 'use-cases/live-polls.test.ts', + 'use-cases/match-stats.test.ts', + 'use-cases/real-time-ads.test.ts', + 'use-cases/real-time-chat.test.ts', + 'use-cases/score-alerts.test.ts', + ], + }, + { + name: 'access-manager', + tier: 'docs', + roles: ['open', 'pam', 'datasync'], + files: ['api-reference/access-manager.test.ts', 'api-reference/access-manager-legacy.test.ts'], + }, + { + name: 'app-context', + tier: 'docs', + roles: ['open', 'pam'], + files: ['api-reference/objects.test.ts'], + }, + { + name: 'datasync', + tier: 'docs', + roles: ['open', 'datasync'], + files: ['api-reference/datasync.test.ts'], + }, + { + name: 'endpoints-datasync', + tier: 'endpoints', + roles: ['datasync'], + files: [ + 'endpoints/data_sync/events.channel.test.ts', + 'endpoints/data_sync/events.entity.test.ts', + 'endpoints/data_sync/events.membership.test.ts', + 'endpoints/data_sync/events.relationship.test.ts', + 'endpoints/data_sync/events.user.test.ts', + 'endpoints/data_sync/projections.test.ts', + ], + }, + { + name: 'endpoints-history', + tier: 'endpoints', + roles: ['open'], + files: ['endpoints/fetch_messages.test.ts', 'endpoints/history.test.ts', 'endpoints/message_actions.test.ts'], + }, +]; diff --git a/test/integration/live/support/group-select.cjs b/test/integration/live/support/group-select.cjs new file mode 100644 index 000000000..0d5f5c982 --- /dev/null +++ b/test/integration/live/support/group-select.cjs @@ -0,0 +1,26 @@ +// Used by .mocharc.cjs and .mocharc.endpoints.cjs (plain CommonJS: Mocha reads its config before tsx is loaded). +// `LIVE_GROUP=` selects one group of ../groups.cjs: the config's `spec` becomes that group's files. Without LIVE_GROUP the +// config keeps its own `spec`. An unknown name, or a group of the other tier, stops the run with a message. +const path = require('node:path'); +const groups = require('../groups.cjs'); + +const LIVE_DIR_FROM_ROOT = 'test/integration/live'; + +/** The spec list of the selected group, or `fallback` when LIVE_GROUP is not set. `tier` is 'docs' or 'endpoints'. */ +function groupSpec(tier, fallback) { + const name = (process.env.LIVE_GROUP || '').trim(); + if (!name) return fallback; + const group = groups.find((g) => g.name === name); + if (!group) { + throw new Error( + `LIVE_GROUP=${name} is not a group of ${LIVE_DIR_FROM_ROOT}/groups.cjs. Groups: ${groups.map((g) => g.name).join(', ')}`, + ); + } + if (group.tier !== tier) { + const script = group.tier === 'endpoints' ? 'test:integration:live:endpoints' : 'test:integration:live'; + throw new Error(`LIVE_GROUP=${name} belongs to the ${group.tier} tier. Run it with npm run ${script}.`); + } + return group.files.map((file) => path.posix.join(LIVE_DIR_FROM_ROOT, file)); +} + +module.exports = { groupSpec }; diff --git a/test/integration/live/support/groups.ts b/test/integration/live/support/groups.ts new file mode 100644 index 000000000..4ff7f7251 --- /dev/null +++ b/test/integration/live/support/groups.ts @@ -0,0 +1,146 @@ +/** + * The API groups of the live tier (../groups.cjs), typed, plus two command line modes: + * + * tsx test/integration/live/support/groups.ts --check fails when a test file is in no group or in two groups, a listed + * file is missing, a file sits in the wrong tier, a file uses a + * role its group does not list, or the workflow's `group` input + * options differ from the group names (part of the lint script) + * tsx test/integration/live/support/groups.ts --matrix prints the job matrix of the "Live tests" workflow as JSON + * + * The check is what keeps a new test file from being dropped silently: CI runs only the files of the groups. + */ +import fs from 'node:fs'; +import path from 'node:path'; +import type { Role } from './env'; + +export type Tier = 'docs' | 'endpoints'; + +export interface LiveGroup { + name: string; + tier: Tier; + /** Keyset roles the group's tests use. A group run without --grep needs all of them up front. */ + roles: Role[]; + /** Paths relative to test/integration/live/. */ + files: string[]; +} + +export const LIVE_DIR = path.resolve(__dirname, '..'); +const WORKFLOW = path.resolve(LIVE_DIR, '../../../.github/workflows/live-integration.yml'); + +// eslint-disable-next-line @typescript-eslint/no-require-imports +export const GROUPS = require('../groups.cjs') as readonly LiveGroup[]; + +/** The group LIVE_GROUP names, or undefined when it is not set. The Mocha configs already reject an unknown name. */ +export function selectedGroup(): LiveGroup | undefined { + const name = process.env.LIVE_GROUP?.trim(); + return name ? GROUPS.find((g) => g.name === name) : undefined; +} + +/** Every *.test.ts file of the tier, relative to LIVE_DIR, with forward slashes. */ +function testFiles(dir = LIVE_DIR): string[] { + const out: string[] = []; + for (const entry of fs.readdirSync(dir, { withFileTypes: true })) { + const full = path.join(dir, entry.name); + if (entry.isDirectory()) out.push(...testFiles(full)); + else if (entry.name.endsWith('.test.ts')) out.push(path.relative(LIVE_DIR, full).split(path.sep).join('/')); + } + return out.sort(); +} + +/** + * Roles a file visibly uses (a lower bound: it can miss a role, it never invents one). `server()` without a role is the pam keyset. + */ +const ROLE_USE: Record, RegExp> = { + pam: /\bserver\(\s*(\)|'pam')|createClient\(\s*'pam'|keysOf\(\s*'pam'|securedKeys\(\s*(\)|'pam')/, + datasync: /\bserver\(\s*'datasync'|createClient\(\s*'datasync'|keysOf\(\s*'datasync'|securedKeys\(\s*'datasync'/, +}; + +/** The `group` options of the workflow's workflow_dispatch input, or undefined when the workflow has none. */ +function workflowGroupOptions(): string[] | undefined { + // eslint-disable-next-line @typescript-eslint/no-require-imports + const yaml = require('js-yaml') as { safeLoad(text: string): unknown }; + const doc = yaml.safeLoad(fs.readFileSync(WORKFLOW, 'utf8')) as { + on?: { workflow_dispatch?: { inputs?: { group?: { options?: string[] } } } }; + }; + return doc.on?.workflow_dispatch?.inputs?.group?.options; +} + +/** Problems, one sentence each. Empty when the groups are consistent. */ +export function checkGroups(groups: readonly LiveGroup[] = GROUPS): string[] { + const problems: string[] = []; + const owner = new Map(); + const names = new Set(); + for (const group of groups) { + if (!/^[a-z0-9-]+$/.test(group.name)) problems.push(`group "${group.name}": the name must match [a-z0-9-]+`); + if (names.has(group.name)) problems.push(`group "${group.name}" is defined twice`); + names.add(group.name); + if (group.tier !== 'docs' && group.tier !== 'endpoints') + problems.push(`group "${group.name}": tier must be 'docs' or 'endpoints'`); + if (group.tier === 'docs' && !group.roles.includes('open')) + problems.push(`group "${group.name}": a docs-tier run always needs the open role, list it`); + if (group.files.length === 0) problems.push(`group "${group.name}" has no files`); + for (const file of group.files) { + const previous = owner.get(file); + if (previous) problems.push(`${file} is in two groups: ${previous} and ${group.name}`); + owner.set(file, group.name); + const full = path.join(LIVE_DIR, file); + if (!fs.existsSync(full)) { + problems.push(`group "${group.name}": ${file} does not exist`); + continue; + } + if ((group.tier === 'endpoints') !== file.startsWith('endpoints/')) + problems.push(`group "${group.name}" (tier ${group.tier}): ${file} belongs to the other tier`); + const source = fs.readFileSync(full, 'utf8'); + for (const [role, pattern] of Object.entries(ROLE_USE) as Array<[Role, RegExp]>) + if (pattern.test(source) && !group.roles.includes(role)) + problems.push(`group "${group.name}": ${file} uses the ${role} keyset, add '${role}' to the group's roles`); + } + } + for (const file of testFiles()) + if (!owner.has(file)) + problems.push( + `${file} is in no group: add it to a group in test/integration/live/groups.cjs, or CI never runs it`, + ); + const options = workflowGroupOptions(); + const expected = ['all', ...groups.map((g) => g.name)]; + if (!options) problems.push('the workflow live-integration.yml has no workflow_dispatch input `group` with options'); + else if (options.join(',') !== expected.join(',')) + problems.push( + `the workflow input \`group\` options (${options.join(', ')}) must be, in this order: ${expected.join(', ')}`, + ); + return problems; +} + +/** The job matrix of the workflow: every group, or the one named. `provision` marks docs-tier groups that use DataSync classes. */ +export function matrixOf(selection: string, groups: readonly LiveGroup[] = GROUPS) { + const chosen = selection === 'all' || selection === '' ? groups : groups.filter((g) => g.name === selection); + if (chosen.length === 0) throw new Error(`no group named "${selection}" in groups.cjs`); + return { + include: chosen.map((g) => ({ + group: g.name, + tier: g.tier, + provision: g.tier === 'docs' && g.roles.includes('datasync'), + })), + }; +} + +function main(argv: string[]): number { + if (argv[0] === '--check') { + const problems = checkGroups(); + if (problems.length) { + problems.forEach((p) => console.error(`groups: ${p}`)); + return 1; + } + const files = GROUPS.reduce((n, g) => n + g.files.length, 0); + console.log(`groups: ${GROUPS.length} groups cover all ${files} test files, each file once`); + return 0; + } + if (argv[0] === '--matrix') { + console.log(JSON.stringify(matrixOf(argv[1] ?? 'all'))); + return 0; + } + console.error('usage: groups.ts --check | --matrix '); + return 2; +} + +if (require.main === module) process.exitCode = main(process.argv.slice(2)); diff --git a/test/integration/live/support/hooks.ts b/test/integration/live/support/hooks.ts index 5e8ececd0..0db2d5fd9 100644 --- a/test/integration/live/support/hooks.ts +++ b/test/integration/live/support/hooks.ts @@ -18,6 +18,11 @@ * The endpoint tests (`.mocharc.endpoints.cjs`, LIVE_TIER=endpoints) check no role up front: a test fails with a "missing keys" * message (env names) when the role it builds a client for is not set. Janitor and preflight default to off there. * + * Group runs (`LIVE_GROUP=`, groups.cjs, both tiers): without --grep the run needs exactly the group's `roles` up front and + * fails at once, naming the missing env variables, when one is not set. The janitor still cleans every role that has keys (a full + * pass takes a few seconds, and it gives every job a clean slate whatever ran before). The wait until a tokenless client is refused + * runs only when the group uses a secured role (pam or datasync), and the preflight checks only the group's roles. + * * `LIVE_JANITOR=off` skips both cleanups (use it for a single page with `--grep`: a full pass costs about the time * reported in the README). `LIVE_PREFLIGHT=off` skips the preflight. */ @@ -26,13 +31,14 @@ import { runCleanups } from './cleanup'; import { restoreAllConsoles } from './console'; import { describeRoles, env, janitorEnabled, preflightEnabled, redact, ROLES } from './env'; import { explainSdkError } from './explain-error'; +import { selectedGroup } from './groups'; import { formatReport, runJanitor, verifyApplicationGrantClosed } from './janitor'; import { runPreflight } from './preflight'; /** * True when the command line selects everything: no --grep / --fgrep and no spec file argument. A full run needs every - * role up front, so a missing keyset fails it at once and not after minutes in the first test that needs it. A partial run - * checks only the roles its tests ask for, when they build their client. + * role up front (a group run: every role of the group), so a missing keyset fails it at once and not after minutes in the first + * test that needs it. A partial run checks only the roles its tests ask for, when they build their client. */ function isFullRun(argv: readonly string[] = process.argv.slice(2)): boolean { const selects = (arg: string): boolean => /^(--grep|-g|--fgrep|-f)(=|$)/.test(arg) || /\.ts$/.test(arg); @@ -42,6 +48,9 @@ function isFullRun(argv: readonly string[] = process.argv.slice(2)): boolean { /** True under `.mocharc.endpoints.cjs` (it sets LIVE_TIER=endpoints): no role is checked up front (each test needs its own, see `keysOf`), janitor and preflight default to off. */ const endpointsTier = process.env.LIVE_TIER === 'endpoints'; +/** The group of LIVE_GROUP (groups.cjs), or undefined for a run of the whole tier. */ +const group = selectedGroup(); + /** * Every failure of the run, test or hook, passes through `Runner#fail` before the reporter sees it, so wrapping it once covers * all of them, including `before` and `beforeEach` hooks, which `afterEach` cannot reach (a failed hook is reported against the @@ -86,11 +95,14 @@ export const mochaHooks = { 'Set them in the environment or in test/integration/live/.env.local (see .env.example).', ); } - if (!endpointsTier && isFullRun()) { - const absent = ROLES.filter((role) => !env()[role]); + // A group run needs the group's roles, a full docs-tier run every role, a full endpoint-tier run none (see above). + const required = group ? group.roles : endpointsTier ? [] : ROLES; + if (isFullRun()) { + const absent = required.filter((role) => !env()[role]); if (absent.length) { throw new Error( - `A full run needs every keyset. Missing: ${absent.map((role) => `${role} (${env().missing[role].join(', ')})`).join('; ')}. ` + + `${group ? `The group ${group.name} needs` : 'A full run needs every keyset.'} ` + + `Missing: ${absent.map((role) => `${role} (${env().missing[role].join(', ')})`).join('; ')}. ` + 'Set them, or select tests with --grep (that run needs only the roles of those tests).', ); } @@ -105,7 +117,9 @@ export const mochaHooks = { 'Fix the cause, or run `npm run test:integration:live:janitor` to retry the cleanup alone.', ); } - const secured = env().pam ?? env().datasync; + // A group that uses no secured role does not wait for Access Manager (up to about 60 s after a grant change). + const usesSecured = !group || group.roles.some((role) => role !== 'open'); + const secured = usesSecured ? (env().pam ?? env().datasync) : undefined; if (secured) { const ms = await verifyApplicationGrantClosed(secured); console.log(`janitor: a client without a token is refused on the secured keyset (${ms} ms)`); @@ -114,7 +128,7 @@ export const mochaHooks = { console.log('janitor: skipped before the run (LIVE_JANITOR=off)'); } if (preflightEnabled()) { - const { notes } = await runPreflight(); + const { notes } = await runPreflight(env(), group?.roles); notes.forEach((note) => console.log(`preflight: ${note}`)); console.log('preflight: keyset features ok'); } diff --git a/test/integration/live/support/preflight.ts b/test/integration/live/support/preflight.ts index 7dcd53d88..d3cb00b2c 100644 --- a/test/integration/live/support/preflight.ts +++ b/test/integration/live/support/preflight.ts @@ -171,12 +171,18 @@ async function probeOpenKeyset(environment: Env, presenceVouched: boolean): Prom return problems; } -/** Throws one error that lists every problem. `notes` returns the roles the Admin API could not check. */ -export async function runPreflight(environment: Env = env()): Promise<{ notes: string[] }> { +/** + * Throws one error that lists every problem. `notes` returns the roles the Admin API could not check. `roles` limits the Admin API + * check (a group run passes its group's roles); the live probes of the open keyset run whenever the open keys are set. + */ +export async function runPreflight( + environment: Env = env(), + roles: readonly Role[] = ROLES, +): Promise<{ notes: string[] }> { const problems: string[] = []; const notes: string[] = []; let openVouched = false; - for (const role of ROLES) { + for (const role of roles) { if (!environment[role]) continue; const read = await readFeatures(environment, role).catch((error: unknown) => ({ status: 'error' as const, From 2b8eda2d299626b47bfcc6b4a28f4887085b620d Mon Sep 17 00:00:00 2001 From: Mateusz Wiktor Date: Thu, 19 Nov 2026 10:00:35 +0100 Subject: [PATCH 11/13] docs: update README and test files for clarity on keyset requirements - Revised the README to clarify the keyset and API key requirements, enhancing the understanding of necessary configurations for live integration tests. - Removed outdated comments in test files to streamline the code and improve readability. - Added a new section in the README detailing keyset roles and their associated requirements, ensuring comprehensive documentation for users. --- test/integration/live/README.md | 211 +++++++++++++++++- .../live/api-reference/publish.test.ts | 1 - .../live/api-reference/subscribe.test.ts | 5 - 3 files changed, 209 insertions(+), 8 deletions(-) diff --git a/test/integration/live/README.md b/test/integration/live/README.md index 515ecbe7d..c7536cd9f 100644 --- a/test/integration/live/README.md +++ b/test/integration/live/README.md @@ -159,7 +159,7 @@ A full pass walks every listed name. In CI it takes about 4 s, plus about 3 s fo After the janitor, the root hook fails fast. Its message names each missing feature. A group run checks only the roles of its group. With `PUBNUB_ADMIN_API_KEY` and the keyset ids (`DOCS_SNIPPETS_OPEN_KEYSET_ID`, `DOCS_SNIPPETS_PAM_KEYSET_ID`, `DOCS_SNIPPETS_DS_KEYSET_ID`), the hook reads each keyset's configuration through the Admin API, read-only. It lists what the role needs and lacks: Presence, Message Persistence, Delete-From-History, App Context and its events, Files, Stream Controller, Access Manager and DataSync. -Live probes on the open keyset always run. They check publish (keys and network), Message Persistence, Delete-From-History, and Presence when the Admin API did not confirm it. The features each keyset needs are the ones the message names. `missingFeatures` in `support/preflight.ts` is the full list per role (open, pam, datasync). +Live probes on the open keyset always run. They check publish (keys and network), Message Persistence, Delete-From-History, and Presence when the Admin API did not confirm it. The full list per role is in "Keyset and API key requirements" below. ### Provisioning (`provision/`) @@ -172,7 +172,7 @@ npm run test:integration:live:provision:check # dry diff, writes nothing, exi npm run test:integration:live:provision # creates missing classes, updates drifted mutable fields ``` -Without `PUBNUB_ADMIN_API_KEY`, the script only checks through the SDK that each class exists. The Admin API key needs a DataSync permission row for the DataSync keyset. +Without `PUBNUB_ADMIN_API_KEY`, the script only checks through the SDK that each class exists. The key permissions are in "Keyset and API key requirements" below. ### Environment variables @@ -188,6 +188,213 @@ Without `PUBNUB_ADMIN_API_KEY`, the script only checks through the SDK that each Everything the janitor, the preflight and the provisioning print goes through `redact()` (`support/env.ts`). It masks every key of the environment, the admin key, key-shaped strings and long token-like strings. +## Keyset and API key requirements + +This section lists what you must set up on fresh PubNub keysets and on the Admin API integration key so that all 8 groups pass. Every row comes from the code, and the last column names the file. A value marked "not verified from code" is not proven by any file in this folder. Check it in the Admin Portal. + +Shorthand: "preflight" is `missingFeatures` and the live probes in `support/preflight.ts`. "Test" means the header comment or the assertions of the named test file. The preflight reads the Admin API only when `PUBNUB_ADMIN_API_KEY` and the keyset id of the role are set (see "Admin API integration key"). Without them, a feature below is still required, but only a failing test shows it. + +### Keysets and roles + +| Role | Env names | Docs tier (CI secrets) | Endpoint tier (CI secrets) | +| :--- | :--- | :--- | :--- | +| open | `SUBSCRIBE_KEY`, `PUBLISH_KEY` | `SDK_DOCS_OPEN_SUBSCRIBE_KEY`, `SDK_DOCS_OPEN_PUBLISH_KEY` | `SDK_SUB_KEY`, `SDK_PUB_KEY` (the SDK open keyset) | +| pam | `PAM_SUBSCRIBE_KEY`, `PAM_PUBLISH_KEY`, `PAM_SECRET_KEY` | `SDK_DOCS_PAM_SUBSCRIBE_KEY`, `SDK_DOCS_PAM_PUBLISH_KEY`, `SDK_DOCS_PAM_SECRET_KEY` | not used | +| datasync | `DS_SUBSCRIBE_KEY`, `DS_PUBLISH_KEY`, `DS_SECRET_KEY` | `SDK_DOCS_DS_SUBSCRIBE_KEY`, `SDK_DOCS_DS_PUBLISH_KEY`, `SDK_DOCS_DS_SECRET_KEY` | `SDK_DS_SUB_KEY`, `SDK_DS_PUB_KEY`, `SDK_DS_SEC_KEY` (the SDK DataSync keyset) | + +Sources: `support/env.ts`, `.github/workflows/live-integration.yml`. + +- The docs tier needs the open role in every run, even for a group that lists no other role (`support/hooks.ts`, the check before the janitor). +- The docs tier has three role slots but may use two keysets. `pam` and `datasync` may be one keyset (`sharedSecured` in `support/env.ts`, `.env.example`). Then that one keyset needs every `pam` and every `datasync` feature below, and the janitor revokes the application-level grant once (`support/janitor.ts`). +- The open keyset must differ from the secured keyset: Access Manager is off on open and on for pam (preflight). +- The endpoint tier never uses the docs keysets and has no pam role (`.github/workflows/live-integration.yml`, step "Endpoint tests (SDK keysets)"). +- Run one live run per keyset at a time. The tests use fixed names (`support/resources.ts`, workflow `concurrency`). + +### Open keyset (docs tier) + +| Feature or setting | Required value | Groups | Proof | +| :--- | :--- | :--- | :--- | +| Access Manager | off | all docs groups | preflight (`missingFeatures`); the clients of `shopper()`, `friend()` and `observer()` never carry a token (`support/clients.ts`) | +| Presence | on | `messaging`, `getting-started`, `use-cases` | preflight; `api-reference/presence.test.ts`, `api-reference/subscribe.test.ts` and `subscribe-legacy.test.ts` (headers), `migration-guides/v11.test.ts` (header), `use-cases/real-time-chat.test.ts`, `fan-re-engagement.test.ts`, `live-event-rate-limiting.test.ts` (headers) | +| Presence tracking rule | a rule with pattern `*` and tracking on | same | preflight | +| Presence events to listeners | join events reach a subscriber; leave or timeout events too | `messaging`, `use-cases` | `subscribe.test.ts` (`eventListenerBasicUsage` reads the join event), `use-cases/fan-re-engagement.test.ts` (leave or timeout) | +| Message Persistence | on | `messaging`, `getting-started`, `use-cases`, `endpoints-history` | preflight (config and `fetchMessages` probe); `publish.test.ts`, `message-persistence*.test.ts`, `message-actions.test.ts`, `file-sharing.test.ts`, `configuration-legacy.test.ts`, `use-cases/live-commentary.test.ts`, `live-polls.test.ts`, `match-stats.test.ts`, `real-time-chat.test.ts`, `automated-polling.test.ts`, `endpoints/history.test.ts` (headers) | +| Delete-From-History | on | `messaging`, `use-cases` | preflight (config and an empty-range `deleteMessages` probe); `message-persistence.test.ts` and `message-persistence-legacy.test.ts` (headers); the janitor skips history cleanup with a note when it is off (`support/janitor.ts`, `cleanHistory`) | +| Message actions (add, remove, fetch, and the events they raise) | on | `messaging`, `endpoints-history` | `api-reference/message-actions.test.ts`, `endpoints/message_actions.test.ts`; no config field is read, so the preflight does not check it | +| Stream Controller | on, with wildcard subscribe | `messaging`, `use-cases` | preflight; `channel-groups.test.ts`, `presence.test.ts` (channel group `cg1`), `subscribe-legacy.test.ts` (`OLDsubscribeWithWildCardChannels`), `use-cases/match-stats.test.ts` (header) | +| App Context | on | `app-context`, `use-cases` | preflight; `objects.test.ts` (the `before` hook fails with "does not have App Context"), `use-cases/real-time-chat.test.ts` | +| App Context events | user metadata, channel metadata and membership events on | `app-context`, `use-cases` | preflight; `objects.test.ts` (`setChannelMetadataBasicUsage` reads the channel event). The user and membership events are proven only by the preflight | +| App Context referential integrity | off | `app-context`, `use-cases` | preflight; `support/janitor.ts` (`cleanAppContext` removes memberships first because deleting a user or channel leaves them) | +| App Context disallow get all (user and channel metadata) | off | `app-context` | preflight; `objects.test.ts` (`getAllUUIDMetadata`, `getAllChannelMetadata`) | +| Files | on | `messaging` | preflight; `file-sharing.test.ts` (header) | +| Push Notifications add-on | on. No APNs or FCM credentials: the tests only register device tokens and list them | `messaging`, `use-cases` | `api-reference/mobile-push.test.ts` (header and `PUSH_OFF`), `use-cases/fan-re-engagement.test.ts`, `use-cases/score-alerts.test.ts`. The preflight does not check it, and the janitor treats a push 400, 403 or 404 as "not applicable" | +| Subscribe filter expressions (Stream Filtering) | usable with `setFilterExpression` and message `meta` | `getting-started` | `api-reference/configuration.test.ts` (`filterExpressionDropsMessage`). No config field is read, so the preflight does not check it | +| Encryption | no keyset setting (client side only) | `getting-started` | `api-reference/encryption.test.ts` (header), `encryption-legacy.test.ts` | +| Illuminate Business Rules | none. The tests publish the trigger messages themselves | `use-cases` | `use-cases/automated-polling.test.ts`, `real-time-ads.test.ts` (headers) | + +### Pam keyset (docs tier) + +| Feature or setting | Required value | Groups | Proof | +| :--- | :--- | :--- | :--- | +| Access Manager | on, with the secret key | `messaging`, `getting-started`, `use-cases`, `access-manager` | preflight; `publish.test.ts` (a client without a token must get 403), `getting-started/status-events.test.ts`, `troubleshooting.test.ts` (header), `use-cases/game-chat-moderation.test.ts` | +| No application-level grant | none left over. The janitor revokes it and waits until a tokenless client is refused (3 refusals in a row, up to 120 s) | all groups that list `pam` or `datasync` | `support/janitor.ts` (`revokeApplicationGrant`, `verifyApplicationGrantClosed`), `support/hooks.ts` | +| Access Manager token revocation | on | `access-manager`, `use-cases` | preflight (role `pam` only); `access-manager.test.ts` (`revokeTokenBasicUsage`), `use-cases/fan-behavior-management.test.ts` (`revokeToken`) | +| Access Manager v3 tokens | `grantToken`, `parseToken`, `revokeToken`, `authorizedUserId`, patterns | `access-manager` and others | `access-manager.test.ts`, `access-manager-legacy.test.ts` (header: only `grantToken` is called) | +| Access Manager v2 grant | the janitor calls `grant({ read: false, write: false, ttl: 1 })`. The v2 grant call must stay allowed | all groups that list `pam` or `datasync` | `support/janitor.ts`. No test calls `grant` or uses an `authKey` for a live request | +| Message Persistence | on | `use-cases` | preflight; `use-cases/game-chat-moderation.test.ts` (header) | +| Delete-From-History | on | `use-cases` | `use-cases/game-chat-moderation.test.ts` (header). The preflight does not check it for this role | +| Presence | on | `getting-started`, `use-cases` | preflight only. No pam test reads presence | + +### Datasync keyset (docs tier) + +| Feature or setting | Required value | Groups | Proof | +| :--- | :--- | :--- | :--- | +| DataSync | on. The built-in classes `User` v1 (global), `Channel` v1 (global) and `Membership` v1 exist | `datasync`, `access-manager` | preflight; `provision/provision-datasync.ts` (`expectedBuiltIns`) | +| Access Manager | on, with the secret key | `datasync`, `access-manager` | preflight; `access-manager.test.ts` (`grantTokenDataSyncResources`, `grantTokenWithProjection` sign on `server('datasync')`) | +| Message Persistence | on | `datasync`, `access-manager` | preflight only | +| Presence | on | `datasync`, `access-manager` | preflight only | +| Classes, projections, event rules | see "DataSync classes, projections and event rules" | `datasync`, `access-manager` | `api-reference/datasync.test.ts` (header and `preconditions`) | + +### SDK open keyset (endpoint tier, `endpoints-history`) + +| Feature or setting | Required value | Proof | +| :--- | :--- | :--- | +| Message Persistence | on | `endpoints/history.test.ts`, `fetch_messages.test.ts`, `message_actions.test.ts` (headers) | +| Message actions | on, including the events of add and remove | `endpoints/message_actions.test.ts` (`should trigger event`) | +| Access Manager | off. The clients carry no token | `endpoints/*.test.ts` use `createClient('open', ...)` without a token | +| Everything else | not needed. Presence, Files, App Context, Stream Controller, Delete-From-History and Push are not used | no test in `endpoints/` calls them | + +The preflight and the janitor are off in this tier (`.mocharc.endpoints.cjs`), so nothing checks the keyset up front. A missing feature shows as a failing test. + +### SDK datasync keyset (endpoint tier, `endpoints-datasync`) + +| Feature or setting | Required value | Proof | +| :--- | :--- | :--- | +| DataSync | on, with the classes of the endpoint tests (see below) | `support/datasync-live-helpers.ts`, `endpoints/data_sync/*.test.ts` | +| Access Manager | on, with the secret key (the projection tests call `grantToken`) | `endpoints/data_sync/projections.test.ts` | +| Event rules | on for every class listed below | `endpoints/data_sync/events.*.test.ts`, `projections.test.ts` | +| Other features | none proven (Presence and Message Persistence are not used) | no test in `endpoints/data_sync/` calls them | + +### Groups, roles and the setting behind a failure + +| Group | Tier | Roles (`groups.cjs`) | Settings that make it fail when missing | +| :--- | :--- | :--- | :--- | +| `messaging` | docs | open, pam | open: Presence, Message Persistence, Delete-From-History, message actions, Stream Controller (channel groups), Files, Push add-on. pam: Access Manager on, no leftover grant | +| `getting-started` | docs | open, pam | open: Presence (v11 guide), Message Persistence (legacy configuration), filter expressions. pam: Access Manager on | +| `use-cases` | docs | open, pam | open: Presence, Message Persistence, App Context and its events, wildcard subscribe, Push add-on. pam: Access Manager with token revocation, Message Persistence, Delete-From-History | +| `access-manager` | docs | open, pam, datasync | pam: Access Manager with token revocation. datasync: Access Manager and DataSync (token grants only, no DataSync object is created). The group runs `provision:check` first | +| `app-context` | docs | open, pam | open: App Context, the three event switches, referential integrity off, disallow get all off. `objects.test.ts` builds no pam client, but `groups.cjs` lists the role, so the run needs the `PAM_*` keys | +| `datasync` | docs | open, datasync | datasync: DataSync, the classes `product` and `ProductOwner`, the event rules and the `admin` projection of `product`. `datasync.test.ts` builds no open client, but the docs tier needs the open keys anyway. The group runs `provision:check` first | +| `endpoints-datasync` | endpoints | datasync | SDK datasync keyset: classes `JSCustomer`, `JSLoanQuote`, `JSREQUESTED_BY`, event rules, `admin` projection | +| `endpoints-history` | endpoints | open | SDK open keyset: Message Persistence, message actions | + +### DataSync classes, projections and event rules + +Provisioning creates only what `provision/datasync-classes.json` lists, on the docs datasync keyset. The Admin Portal holds the rest. + +| Class | Kind | Version | Fields (property, type, filtering, nullable) | Projections | Created by | +| :--- | :--- | :--- | :--- | :--- | :--- | +| `User` | built-in entity, level `Global` | 1 | none declared by this tier | `__default__` | PubNub, when DataSync is on | +| `Channel` | built-in entity, level `Global` | 1 | none declared by this tier | `__default__` | PubNub, when DataSync is on | +| `Membership` | built-in relationship | 1 | none (the class cannot be extended, `datasync.test.ts`) | `__default__` | PubNub, when DataSync is on | +| `product` | entity | 1 | `name` string full nullable; `price` number simple not nullable; `stock` number simple nullable. Class `ttlSec` 2678400 | `name` and `price`: `__default__`, `public`, `admin`; `stock`: `__default__`, `admin` | `npm run test:integration:live:provision` | +| `ProductOwner` | relationship | 1 | many-to-many, entity B class `product`, entity A any class; `tier` string full nullable | `tier`: `__default__`, `public`, `admin` | `npm run test:integration:live:provision` | + +Facts for provisioning (`provision/provision-datasync.ts`): + +- It never deletes a class. It fails on a relationship class whose cardinality or entity sides differ, because those fields are immutable. Only a person may delete and recreate it. +- It replaces mutable fields of a class that drifted (PUT) and creates a missing class (POST). `--check` writes nothing and exits with 1 on any difference. +- It reads only classes it lists. It never touches the `JS*` classes. +- The per-class event switch ("Select entities to sync") has no Admin API endpoint. You set it by hand in the Admin Portal, DataSync tab. + +Event rules to switch on by hand in the Admin Portal: + +| Keyset | Event rule (class and version) | Needed by | +| :--- | :--- | :--- | +| docs datasync | `User` v1, `Channel` v1, `product` v1 | `api-reference/datasync.test.ts` (header and the `events and subscriptions` block). A kind without a rule fails the test that waits for its event | +| docs datasync | `ProductOwner` v1, `Membership` v1 | the comment in `provision/datasync-classes.json` lists them. No test in `datasync.test.ts` waits for their events | +| SDK datasync | `User` v1, `Channel` v1, `Membership` v1, `JSCustomer` v1, `JSREQUESTED_BY` v1 | `events.user`, `events.channel`, `events.membership`, `events.entity`, `events.relationship` tests | +| SDK datasync | `JSLoanQuote` v1 | `projections.test.ts`: the flag `LOAN_QUOTE_AND_RELATIONSHIP_EVENTS_ENABLED` is `true`, so the mirror channel events of `JSLoanQuote` and `JSREQUESTED_BY` are asserted | + +The classes of the endpoint tier are not in `datasync-classes.json`, and no script creates them. `support/datasync-live-helpers.ts` calls them "provisioned" (version 1). What the tests prove about them: + +| Class | Kind | Fields the tests send and read | Projections the tests rely on | +| :--- | :--- | :--- | :--- | +| `JSCustomer` v1 | entity | `customerId`, `firstName`, `lastName`, `email`, `creditScore`, `city` | `__default__`: all six fields. `admin`: the same without `email`, plus `private` | +| `JSLoanQuote` v1 | entity | `quoteId`, `make`, `model`, `price` | `__default__`: all four. `admin`: `quoteId` and `private` | +| `JSREQUESTED_BY` v1 | relationship, from `JSCustomer` (entity A) to `JSLoanQuote` (entity B) | `linkedAt` | `__default__`: `linkedAt`. `admin`: `linkedAt` and `private` | + +Source: `endpoints/data_sync/projections.test.ts` (header table and the field constants), `events.relationship.test.ts`, `support/datasync-live-helpers.ts`. The `private` field must exist in every class, and the `admin` projection must exist on all three. The tests assert that an unknown projection name is rejected, so `nosuchprojection` must not exist. Property types, filtering modes and cardinality of these three classes are not in the code (see "Not verified from code"). + +Projection names used by the docs tier (`api-reference/datasync.test.ts`, `access-manager.test.ts`): `admin` (the mirror channel `__admin__`, for example `__admin__product.*`) and `public` (in a token that assigns it to `product-sneaker-42` and `product-.*`). Both are declared on `product` in `datasync-classes.json`. + +### Admin API integration key + +Both Admin API calls send the raw key in the `Authorization` header (no `Bearer`) and the header `PubNub-Version: 2026-09-17`. The base is `https://admin-api.pubnub.com`. + +| Used by | Call | Permission it needs | Code | +| :--- | :--- | :--- | :--- | +| Preflight feature check (docs tier, role open, pam or datasync) | `GET /v2/keysets/{keysetId}` and `GET /v2/keysets/{keysetId}/config` | read access to the keyset configuration of each keyset you give an id for. The exact permission name is not verified from code | `support/preflight.ts` (`readFeatures`) | +| `provision:check` and `provision`, reads | `GET /v2/datasync/subkeys/{subscribeKey}/entity-classes` and `.../relationship-classes` | a DataSync permission row, keyset level, for the docs datasync keyset (the 403 hint says "Read & write") | `provision/provision-datasync.ts` | +| `provision` only, creates | `POST /v2/datasync/subkeys/{subscribeKey}/{entity-classes or relationship-classes}/{name}/versions/{version}` | DataSync, keyset level, write | same | +| `provision` only, updates | `PUT` on the same URL | DataSync, keyset level, write | same | + +What the key does in CI: + +- The step "Check the DataSync classes (read only)" and the step "Docs samples" get `PUBNUB_ADMIN_API_KEY` from the secret `SDK_DOCS_ADMIN_API_KEY`. The endpoint step and the cleanup step do not get it. +- `provision:check` runs only for groups with `provision: true`, which are the docs groups that list `datasync`: `access-manager` and `datasync`. It never writes. The CI key therefore needs only read access to DataSync. `provision` (writes) is a manual step and needs write access. +- The feature check needs a keyset id per role. The workflow sets only `DOCS_SNIPPETS_DS_KEYSET_ID`, from the repository variable `vars.SDK_DOCS_DS_KEYSET_ID`, on the job. `DOCS_SNIPPETS_OPEN_KEYSET_ID` and `DOCS_SNIPPETS_PAM_KEYSET_ID` are not wired in CI. So in CI the Admin API checks only the `datasync` role. The open and pam roles get no config check, and the open keyset gets only the live probes. +- The `datasync` role feature check also confirms that the keyset of that id holds the `DS_SUBSCRIBE_KEY` (`mismatch` otherwise). + +Without the key or without an id: + +- Preflight: the role is skipped with the note "Admin API feature check skipped". For the open role, the live probes still check publish, Message Persistence, Delete-From-History and Presence (`hereNow` must list a subscribed client). Nothing else is checked, so the run can fail later in a test. +- `provision:check` and `provision`: the script checks through the SDK, with the datasync keys, that `product` v1 and `ProductOwner` v1 exist, and it writes nothing. It does not diff fields and does not check the built-in classes. `provision` cannot create classes without the key. +- The tests themselves never use the Admin API key. The endpoint tier never uses it. + +CI secrets and variables that feed the tier (`.github/workflows/live-integration.yml`): + +| Where | Name | Becomes | +| :--- | :--- | :--- | +| secret | `SDK_DOCS_OPEN_SUBSCRIBE_KEY`, `SDK_DOCS_OPEN_PUBLISH_KEY` | `SUBSCRIBE_KEY`, `PUBLISH_KEY` | +| secret | `SDK_DOCS_PAM_SUBSCRIBE_KEY`, `SDK_DOCS_PAM_PUBLISH_KEY`, `SDK_DOCS_PAM_SECRET_KEY` | `PAM_SUBSCRIBE_KEY`, `PAM_PUBLISH_KEY`, `PAM_SECRET_KEY` | +| secret | `SDK_DOCS_DS_SUBSCRIBE_KEY`, `SDK_DOCS_DS_PUBLISH_KEY`, `SDK_DOCS_DS_SECRET_KEY` | `DS_SUBSCRIBE_KEY`, `DS_PUBLISH_KEY`, `DS_SECRET_KEY` (docs tier, janitor, `provision:check`) | +| secret | `SDK_DOCS_ADMIN_API_KEY` | `PUBNUB_ADMIN_API_KEY` (docs tier and `provision:check`) | +| variable | `SDK_DOCS_DS_KEYSET_ID` | `DOCS_SNIPPETS_DS_KEYSET_ID` (job level) | +| secret | `SDK_SUB_KEY`, `SDK_PUB_KEY` | `SUBSCRIBE_KEY`, `PUBLISH_KEY` (endpoint tier) | +| secret | `SDK_DS_SUB_KEY`, `SDK_DS_PUB_KEY`, `SDK_DS_SEC_KEY` | `DS_SUBSCRIBE_KEY`, `DS_PUBLISH_KEY`, `DS_SECRET_KEY` (endpoint tier) | + +### Setup checklist + +1. Create the keysets: one open keyset and one or two secured keysets for pam and datasync, plus the two SDK keysets (open and datasync) for the endpoint tier. Keep the open keysets free of Access Manager. +2. Open keysets: turn on Message Persistence and message actions on both (the SDK open keyset needs nothing else). For the docs open keyset, also turn on Delete-From-History, Presence with a tracking rule for `*`, Stream Controller with wildcard subscribe, App Context with the user, channel and membership events (referential integrity and "disallow get all" off), Files and the Push Notifications add-on. +3. Secured keysets: turn on Access Manager (with token revocation), Message Persistence, Delete-From-History (the pam role) and Presence. On the datasync keyset, turn on DataSync. +4. Create the Admin API integration key with the permissions in "Admin API integration key". Note the keyset id of the docs datasync keyset. +5. Copy `.env.example` to `.env.local` and fill the keys, `PUBNUB_ADMIN_API_KEY` and the `DOCS_SNIPPETS_*_KEYSET_ID` values (the file is read in memory and git ignores it). Add the CI secrets and the variable `SDK_DOCS_DS_KEYSET_ID`. +6. Run `npm run test:integration:live:provision` once on the docs datasync keyset to create `product` v1 and `ProductOwner` v1. +7. In the Admin Portal, DataSync tab, switch on the event rules in the table above. The three `JS*` classes must exist on the SDK datasync keyset with the `admin` projection and the `private` field. No script in this folder creates them. +8. Confirm the setup without changing state: + - `npm run test:integration:live -- --dry-run` and `npm run test:integration:live:endpoints -- --dry-run` list the tests and need no keys. + - `npm run test:integration:live:provision:check` diffs the classes and writes nothing. + - The preflight has no command of its own. It runs in the `before all` hook of every docs-tier run, after the janitor. It reads the Admin API (read only) and sends a few probe requests to the open keyset (a publish, a history read, an empty-range delete and a presence subscribe). Those requests are the only traffic it adds. The janitor sends requests too (it deletes the listed resources), so start a real run only against keysets meant for the tier. + +### Not verified from code + +- The exact Admin API permission name and level for `GET /v2/keysets/{id}` and `/config`. +- Whether the CI key needs write access for `provision:check` (the code only reads), and whether a Read permission row is enough. The 403 hint names "Read & write". +- Message Persistence retention: the tests need messages to stay only for the length of a run, and the channel `history_documented_behavior` is refilled when it holds fewer than 100 messages. No minimum retention is in the code. +- Message actions: no config field is read, so the name of the portal switch is not known. +- Stream Filtering: whether it needs its own portal switch. +- Presence: announce max, interval mode, and any minimum or maximum for `presenceTimeout` and `heartbeatInterval` on the server. The client uses `presenceTimeout: 20` and `heartbeatInterval: 8`. The Presence rule pattern is the only server setting the code reads. +- File retention, Files region, App Context region, data region of any keyset, and the plan tier or rate limits. No code reads them. +- Push Notifications as a keyset setting: the code proves only the add-on is needed and not APNs or FCM credentials. +- Whether the pam and datasync keysets are one keyset in the CI secrets. The code supports both. +- Whether Message Persistence, Presence or Delete-From-History is needed on the secured keysets beyond the preflight list and the `game-chat-moderation` header. +- Whether the Membership and `ProductOwner` event rules are needed in the docs tier, and whether `User` and `Channel` need a named projection other than `__default__`. +- The schema of `JSCustomer`, `JSLoanQuote` and `JSREQUESTED_BY` beyond the field lists above (types, filtering modes, cardinality, class `ttlSec`), and who creates them. +- Whether `grantToken` with `dataSyncProjections` is validated against the class at grant time (the `public` projection comment says the class must declare it). +- Whether the application-level v2 grant call (used only by the janitor) can be disabled on a keyset. + ## Offline checks: marker lint, coverage manifest, docs cross-check Three scripts need no keys and no network. They answer one question: does every snippet that the public docs embed from this repository run against the real service in a test here, or is it on a listed exception with a reason? diff --git a/test/integration/live/api-reference/publish.test.ts b/test/integration/live/api-reference/publish.test.ts index 5ac9cc7aa..922955889 100644 --- a/test/integration/live/api-reference/publish.test.ts +++ b/test/integration/live/api-reference/publish.test.ts @@ -144,7 +144,6 @@ describe('api-reference/publish', () => { const { timetoken: before } = await observer().time(); // snippet.fireBasicUsage - // Initialize PubNub with your keys await pubnub.fire({ message: { such: 'object', diff --git a/test/integration/live/api-reference/subscribe.test.ts b/test/integration/live/api-reference/subscribe.test.ts index 42ece2fe3..0af52acc0 100644 --- a/test/integration/live/api-reference/subscribe.test.ts +++ b/test/integration/live/api-reference/subscribe.test.ts @@ -31,7 +31,6 @@ describe('api-reference/subscribe', () => { it('createChannelGroupBasicUsage: the channel group entity carries the group name', () => { const pubnub = shopper(); // snippet.createChannelGroupBasicUsage - // Initialize PubNub with demo keys const channelGroup = pubnub.channelGroup('channelGroup_1'); // snippet.end assert.equal(channelGroup.name, 'channelGroup_1'); @@ -40,8 +39,6 @@ describe('api-reference/subscribe', () => { it('createChannelMetadataBasicUsage: the channel metadata entity carries the id and makes no network call', () => { const pubnub = shopper(); // snippet.createChannelMetadataBasicUsage - // Initialize PubNub with demo keys - const channelMetadata = pubnub.channelMetadata('channel_1'); // snippet.end assert.equal(channelMetadata.id, 'channel_1'); @@ -50,8 +47,6 @@ describe('api-reference/subscribe', () => { it('createUserMetadataBasicUsage: the user metadata entity carries the id and makes no network call', () => { const pubnub = shopper(); // snippet.createUserMetadataBasicUsage - // Initialize PubNub with demo keys - const userMetadata = pubnub.userMetadata('user_meta1'); // snippet.end assert.equal(userMetadata.id, 'user_meta1'); From 34420a0ae1d70af8715d2e0bbcf63f4a441dd1e4 Mon Sep 17 00:00:00 2001 From: Mateusz Wiktor Date: Thu, 19 Nov 2026 11:07:49 +0100 Subject: [PATCH 12/13] fix: update setTimeout type definitions and improve ESLint comments - Refined the type definition for the global setTimeout function in channel tests to enhance type safety. - Updated ESLint comments across multiple files to include both no-require-imports and no-var-requires rules, improving code quality and adherence to TypeScript best practices. --- test/integration/endpoints/objects/channel.test.ts | 2 +- test/integration/live/support/group-select.cjs | 2 ++ test/integration/live/support/groups.ts | 4 ++-- test/integration/live/support/hooks.ts | 2 +- 4 files changed, 6 insertions(+), 4 deletions(-) diff --git a/test/integration/endpoints/objects/channel.test.ts b/test/integration/endpoints/objects/channel.test.ts index 35c6ecdb3..4b660c32a 100644 --- a/test/integration/endpoints/objects/channel.test.ts +++ b/test/integration/endpoints/objects/channel.test.ts @@ -814,7 +814,7 @@ describe('objects channel', () => { // Retry delays are at least 2 seconds (policy validation), so record them and run the retries without waiting. const retryDelays: number[] = []; const originalSetTimeout = global.setTimeout; - global.setTimeout = ((handler: any, timeout?: number, ...args: any[]) => { + global.setTimeout = ((handler: (...handlerArgs: unknown[]) => void, timeout?: number, ...args: unknown[]) => { if (timeout !== undefined && timeout >= 1000) { retryDelays.push(timeout); return originalSetTimeout(handler, 1, ...args); diff --git a/test/integration/live/support/group-select.cjs b/test/integration/live/support/group-select.cjs index 0d5f5c982..af6a72bb9 100644 --- a/test/integration/live/support/group-select.cjs +++ b/test/integration/live/support/group-select.cjs @@ -1,7 +1,9 @@ // Used by .mocharc.cjs and .mocharc.endpoints.cjs (plain CommonJS: Mocha reads its config before tsx is loaded). // `LIVE_GROUP=` selects one group of ../groups.cjs: the config's `spec` becomes that group's files. Without LIVE_GROUP the // config keeps its own `spec`. An unknown name, or a group of the other tier, stops the run with a message. +// eslint-disable-next-line @typescript-eslint/no-require-imports, @typescript-eslint/no-var-requires const path = require('node:path'); +// eslint-disable-next-line @typescript-eslint/no-require-imports, @typescript-eslint/no-var-requires const groups = require('../groups.cjs'); const LIVE_DIR_FROM_ROOT = 'test/integration/live'; diff --git a/test/integration/live/support/groups.ts b/test/integration/live/support/groups.ts index 4ff7f7251..aa046d40f 100644 --- a/test/integration/live/support/groups.ts +++ b/test/integration/live/support/groups.ts @@ -27,7 +27,7 @@ export interface LiveGroup { export const LIVE_DIR = path.resolve(__dirname, '..'); const WORKFLOW = path.resolve(LIVE_DIR, '../../../.github/workflows/live-integration.yml'); -// eslint-disable-next-line @typescript-eslint/no-require-imports +// eslint-disable-next-line @typescript-eslint/no-require-imports, @typescript-eslint/no-var-requires export const GROUPS = require('../groups.cjs') as readonly LiveGroup[]; /** The group LIVE_GROUP names, or undefined when it is not set. The Mocha configs already reject an unknown name. */ @@ -57,7 +57,7 @@ const ROLE_USE: Record, RegExp> = { /** The `group` options of the workflow's workflow_dispatch input, or undefined when the workflow has none. */ function workflowGroupOptions(): string[] | undefined { - // eslint-disable-next-line @typescript-eslint/no-require-imports + // eslint-disable-next-line @typescript-eslint/no-require-imports, @typescript-eslint/no-var-requires const yaml = require('js-yaml') as { safeLoad(text: string): unknown }; const doc = yaml.safeLoad(fs.readFileSync(WORKFLOW, 'utf8')) as { on?: { workflow_dispatch?: { inputs?: { group?: { options?: string[] } } } }; diff --git a/test/integration/live/support/hooks.ts b/test/integration/live/support/hooks.ts index 0db2d5fd9..9b61227e4 100644 --- a/test/integration/live/support/hooks.ts +++ b/test/integration/live/support/hooks.ts @@ -60,7 +60,7 @@ const group = selectedGroup(); */ function installFailureExplainer(): boolean { try { - // eslint-disable-next-line @typescript-eslint/no-require-imports + // eslint-disable-next-line @typescript-eslint/no-require-imports, @typescript-eslint/no-var-requires const Runner = require('mocha/lib/runner') as { prototype: { fail?: (...args: unknown[]) => unknown } }; const original = Runner.prototype.fail; if (typeof original !== 'function') return false; From 68cb650f9d3b4715fbbe08442a16a68a336cb07f Mon Sep 17 00:00:00 2001 From: Mateusz Wiktor Date: Thu, 19 Nov 2026 11:28:20 +0100 Subject: [PATCH 13/13] feat: add workflow to cancel superseded live runs for pull requests - Introduced a new GitHub Actions workflow (`cancel-superseded-live-runs.yml`) that cancels older live test runs for the same pull request when a new push occurs, ensuring that only the most recent changes are tested. - Updated the `live-integration.yml` workflow to clarify its behavior regarding concurrency and cancellation of older runs. - Revised the README to explain the cancellation process and its implications for live test runs, enhancing documentation clarity. --- .../workflows/cancel-superseded-live-runs.yml | 65 +++++++++++++++++++ .github/workflows/live-integration.yml | 26 ++++---- test/integration/live/README.md | 6 +- 3 files changed, 80 insertions(+), 17 deletions(-) create mode 100644 .github/workflows/cancel-superseded-live-runs.yml diff --git a/.github/workflows/cancel-superseded-live-runs.yml b/.github/workflows/cancel-superseded-live-runs.yml new file mode 100644 index 000000000..1ea2001b2 --- /dev/null +++ b/.github/workflows/cancel-superseded-live-runs.yml @@ -0,0 +1,65 @@ +name: Cancel superseded live runs + +# Companion of live-integration.yml. When a pull request gets a new push, this workflow cancels the older "Live tests" runs of the +# same pull request, so a quick fix does not wait for a run that is already out of date. +# +# Why a separate workflow: live-integration.yml has one concurrency group for all pull requests (the jobs of two runs must never +# interleave on the shared keysets), and it never cancels in progress, because that would also cancel the run of another pull +# request. A run that waits for that group cannot run any step, so it cannot cancel anything itself. This workflow has no such +# lock and runs at once. +# +# A cancelled live run still runs its "Revoke grants and clean up" step, and the next run starts with its own janitor pass, so +# cancelling is safe (test/integration/live/README.md, "Cancelling a run"). Only runs of the same pull request with a different +# head commit are cancelled. Runs of other pull requests are never touched. Pull requests from forks have no live jobs and are +# skipped. + +on: + pull_request: + branches: + - master + types: [opened, synchronize, reopened, ready_for_review] + +permissions: + actions: write + contents: read + +jobs: + cancel: + name: Cancel older live runs of this pull request + if: ${{ github.event.pull_request.head.repo.full_name == github.repository }} + runs-on: + group: organization/Default + timeout-minutes: 5 + steps: + - name: Cancel runs for older commits + uses: actions/github-script@v9 + with: + script: | + const { owner, repo } = context.repo; + const pr = context.payload.pull_request; + const headSha = pr.head.sha; + let cancelled = 0; + for (const status of ['queued', 'in_progress', 'waiting', 'pending']) { + const runs = await github.paginate(github.rest.actions.listWorkflowRuns, { + owner, + repo, + workflow_id: 'live-integration.yml', + event: 'pull_request', + branch: pr.head.ref, + status, + per_page: 100, + }); + for (const run of runs) { + const samePullRequest = (run.pull_requests || []).some((item) => item.number === pr.number); + if (!samePullRequest || run.head_sha === headSha) continue; + core.info(`Cancelling run ${run.id} (${status}, commit ${run.head_sha.slice(0, 8)}) of pull request #${pr.number}`); + try { + await github.rest.actions.cancelWorkflowRun({ owner, repo, run_id: run.id }); + cancelled += 1; + } catch (error) { + // The run may have finished between the list and the cancel. + core.warning(`Could not cancel run ${run.id}: ${error.message}`); + } + } + } + core.info(`Cancelled ${cancelled} older run(s).`); diff --git a/.github/workflows/live-integration.yml b/.github/workflows/live-integration.yml index a53528403..44b1a32fc 100644 --- a/.github/workflows/live-integration.yml +++ b/.github/workflows/live-integration.yml @@ -11,19 +11,12 @@ name: Live tests # builds, cleans the keysets before and after its tests, and can be re-run alone ("Re-run job" on that group). on: - # Pull requests into master, only when something the tier or its mocked samples depend on changed. Marking a draft as ready - # for review starts a run. + # Every pull request into master, whatever files it changes, so SDK changes are tested too. A new push to the pull request starts + # a new run. Marking a draft as ready for review also starts a run. pull_request: branches: - master types: [opened, synchronize, reopened, ready_for_review] - paths: - - 'test/integration/live/**' - - 'test/integration/endpoints/objects/channel.test.ts' - - 'test/integration/endpoints/time.test.ts' - - 'test/integration/components/subscription_manager.test.ts' - - 'package.json' - - '.github/workflows/live-integration.yml' workflow_dispatch: inputs: group: @@ -49,11 +42,14 @@ on: type: string # One live run at a time for the whole workflow: the tier uses fixed resource names on shared keysets, so the jobs of two runs must -# never interleave. A running run is never cancelled. Runs that get no live jobs (draft or fork pull requests) take a group of -# their own, so they never wait and never replace a pending live run. +# never interleave. This group never cancels in progress, because that would also cancel the run of another pull request. A new +# push to a pull request cancels the older live runs of the same pull request through cancel-superseded-live-runs.yml (a run that +# waits for this group cannot cancel itself). Runs that get no live jobs (draft or fork pull requests) take one group per pull +# request, so they never wait for or replace a pending live run. A new push cancels the older run of that group, so repeated +# pushes keep at most one such run per pull request. concurrency: - group: ${{ (github.event_name == 'pull_request' && (github.event.pull_request.draft || github.event.pull_request.head.repo.full_name != github.repository)) && format('live-tests-no-live-{0}', github.run_id) || 'docs-snippets-live' }} - cancel-in-progress: false + group: ${{ (github.event_name == 'pull_request' && (github.event.pull_request.draft || github.event.pull_request.head.repo.full_name != github.repository)) && format('live-tests-no-live-{0}', github.ref) || 'docs-snippets-live' }} + cancel-in-progress: ${{ github.event_name == 'pull_request' && (github.event.pull_request.draft || github.event.pull_request.head.repo.full_name != github.repository) }} defaults: run: @@ -77,7 +73,7 @@ jobs: with: persist-credentials: false - name: Setup NodeJS 22 - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4 + uses: actions/setup-node@v4 with: node-version: 22 cache: 'npm' @@ -121,7 +117,7 @@ jobs: with: persist-credentials: false - name: Setup NodeJS 22 - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4 + uses: actions/setup-node@v4 with: node-version: 22 cache: 'npm' diff --git a/test/integration/live/README.md b/test/integration/live/README.md index c7536cd9f..278149c85 100644 --- a/test/integration/live/README.md +++ b/test/integration/live/README.md @@ -42,7 +42,7 @@ The janitor's resource list and the preflight's feature checks belong to the doc Mocha adds a spec file from the command line to the config's `spec`. It does not replace it. So the docs-tier config excludes `endpoints/**` with `exclude`, and a file argument for the other tier is not added to a run. Check what a script runs with `-- --dry-run`. The root `npm run test:node` lists none of this folder. -The workflow (`.github/workflows/live-integration.yml`, named "Live tests") runs on non-draft pull requests into `master`, when the paths it watches change, and on demand. It has no scheduled runs. It does not alert anyone on failure yet. The GitHub run logs are the only signal. +The workflow (`.github/workflows/live-integration.yml`, named "Live tests") runs on every non-draft pull request into `master` (whatever files it changes) and on demand. It has no scheduled runs. It does not alert anyone on failure yet. The GitHub run logs are the only signal. ## Groups and CI jobs @@ -71,7 +71,7 @@ The workflow (`.github/workflows/live-integration.yml`, named "Live tests") runs - Start one group by hand: run the workflow with the `group` input set to its name (`all` runs every group). The `grep` input narrows the tests inside the selected groups. - Run one group locally: `LIVE_GROUP= npm run test:integration:live` (docs tier) or `LIVE_GROUP= npm run test:integration:live:endpoints`. -The whole workflow has one concurrency group, `docs-snippets-live`, with `cancel-in-progress: false`. So the jobs of two runs never interleave on the keysets. A run that gets no live job (a draft or fork pull request) takes a group of its own. +The whole workflow has one concurrency group, `docs-snippets-live`, with `cancel-in-progress: false`. So the jobs of two runs never interleave on the keysets. A run that gets no live job (a draft or fork pull request) takes one group per pull request (`github.ref`), with `cancel-in-progress` on. So it never waits for a live run, and a new push cancels the older run of the same pull request. To add a group: add it to `groups.cjs`, then add its name to the `group` input options of the workflow, in the same order. The lint fails until both match. @@ -144,6 +144,8 @@ The tests use the literal names that the docs show (`'my_channel'`, `'family'`, The janitor runs automatically from the root hooks. It runs once before all tests of a run, so once before each group job in CI. A failure fails the run, and the output lists what could not be removed. On a secured keyset, the janitor then waits until a client without a token is refused again. That can take up to about 60 s when the previous run just changed a grant. It gives up after 120 s. This wait runs only when the run uses a secured keyset: always for a run of the whole tier, and for a group only when its `roles` list `pam` or `datasync`. It also runs once after all tests. A failure there prints a loud banner and does not fail the run. The exception is a failed grant revoke, which leaves a secured keyset open. The next run's cleanup retries before the run, and it fails if the keyset is still dirty. +**Cancelling a run.** A cancelled or failed docs-tier job runs the step "Revoke grants and clean up", which calls the janitor and revokes the Access Manager grant. The cancel signal gives that step a short window to run. If it does not run (a forced cancel, or a runner that dies), the next job starts with its own janitor pass before any test. That pass removes every listed name and revokes the grant, and it is safe to repeat. So a cancelled run does not break the next run. Two things stay behind. Names with a per-run suffix, such as the channels of `unique()`, are not listed, and their messages expire with the keyset's retention. The endpoint tier has no cleanup step, so a cancelled endpoint job can leave DataSync objects with run-unique ids. They cannot collide with a later run, and no script sweeps them. A new push to a pull request cancels the older live runs of the same pull request, through `.github/workflows/cancel-superseded-live-runs.yml`. That workflow runs at once, because a run that waits for the shared concurrency group cannot run any step. It cancels only runs of the same pull request with a different head commit, never a run of another pull request. A cancelled run cleans up as described here, and the new run waits until the cancelled one has finished. GitHub keeps only one waiting run per concurrency group. So if three pull requests push while one run is in progress, the middle waiting run is replaced. Push again or use "Re-run all jobs" to get it back. + ```bash npm run test:integration:live:janitor # the janitor alone, all roles; exit code 1 when something could not be removed npm run test:integration:live:janitor -- --role open