feat: monthly active users per platform, answered by /mau

README.md

@@ -10,11 +10,12 @@ share for translations. One package, `session_ops`, deployed to one self-hosted
 | --- | --- | --- |
 | `zendesk-digest` | Weekday Discord digest of the Zendesk tickets awaiting a reply, after closing positive app-store reviews | [zendesk-digest](docs/jobs/zendesk-digest.md) |
 | `zendesk-relay` | Drafts and sends Zendesk replies from `claude:` private notes | [zendesk-relay](docs/jobs/zendesk-relay.md) |
-| `session-ops-discord` | `/run` a job and see `/versions` from Discord | [session-ops-discord](docs/jobs/session-ops-discord.md) |
+| `session-ops-discord` | `/run` a job and see `/versions` from Discord; `/mau` and `/mau-upload` once mau is on | [session-ops-discord](docs/jobs/session-ops-discord.md) |
 | `github-prs-digest` | Weekday Discord digest of open pull requests from outside contributors | [github-prs-digest](docs/jobs/github-prs-digest.md) |
 | `crowdin-duplicates` | Crowdin string slots holding more than one translation, as they open and close | [crowdin-duplicates](docs/jobs/crowdin-duplicates.md) |
 | `crowdin-sync` | Weekday: Crowdin translations into iOS, Android and the localization module, and its submodule bumped in each client | [crowdin-sync](docs/jobs/crowdin-sync.md) |
 | `snode-list` | Weekday: the fallback service node list from the seed nodes into dynamic-assets, Desktop and iOS | [snode-list](docs/jobs/snode-list.md) |
+| `mau` | Off for now. `/mau` in Discord: last month's active users per platform and in total, from the store exports handed to `/mau-upload` | [mau](docs/jobs/mau.md) |
 | `session-ops-silence` | Discord alerts for a job that failed, or stopped running | [session-ops-silence](docs/jobs/session-ops-silence.md) |
 | `token-expiry` | Discord alerts 14 days, 7 days and 24 hours before a token expires | [token-expiry](docs/jobs/token-expiry.md) |
 | `store-versions` | Every 5 minutes: each store's Session release against GitHub's, posted as they diverge | [store-versions](docs/jobs/store-versions.md) |

deploy/README.md

@@ -7,10 +7,10 @@ does the work: accounts, venv, env files, units, timers, and migrating an older
 | Unit | What it is |
 | --- | --- |
 | `session-ops@<job>.timer` → `.service` | One per job with a `schedule`; a generated drop-in sets its account, env files and schedule. |
-| `session-ops@<job>.path` → `.service` | For a job with `discord = true`: starts it when `/run` requests it. |
+| `session-ops@<job>.path` → `.service` | For a job with `discord = true`: starts it when `/run`, or `/mau` for mau, requests it. |
 | `session-ops-queue.timer` → `.service` | Starts the jobs in `jobs.toml`'s `[queue]`, which then run one at a time in its order. |
 | `zendesk-relay.service` | Always on, `127.0.0.1:8080`: Zendesk's `claude:` note webhooks. |
-| `session-ops-discord.service` | Always on, `127.0.0.1:8081`: the `/run` and `/versions` slash commands. See [Discord commands](#discord-commands). |
+| `session-ops-discord.service` | Always on, `127.0.0.1:8081`: the `/run`, `/mau`, `/mau-upload` and `/versions` slash commands. See [Discord commands](#discord-commands). |
 | `session-ops-alert@.service` | Every unit's `OnFailure=` backstop; see [session-ops-silence](../docs/jobs/session-ops-silence.md). |
 
 `session-ops list` shows the jobs; `session-ops run <job> [--dry-run] [-- job arguments]`
@@ -64,7 +64,7 @@ files and state over. Remove `/opt/zendesk`, `/etc/zendesk` and `/var/lib/zendes
 
 ## Discord commands
 
-What `/run` and `/versions` do, and who may run them:
+What `/run`, `/mau`, `/mau-upload` and `/versions` do, and who may run them:
 [session-ops-discord](../docs/jobs/session-ops-discord.md). To set them up:
 
 1. In the Developer Portal, create an application. Put its public key, its application

deploy/env/discord.env.example

@@ -1,4 +1,4 @@
-# /etc/session-ops/discord.env: the Discord app behind /run and /versions.
+# /etc/session-ops/discord.env: the Discord app behind /run and /mau-upload.
 # A `#` starts a comment only as a line's first character. Restart session-ops-discord
 # after an edit: it reads this file at start.
 

deploy/env/mau.env.example

@@ -0,0 +1,6 @@
+# /etc/session-ops/mau.env: the monthly active users /mau answers with. Its failures go
+# to alerts.env's channel.
+# An App Store Connect team API key with the Sales and Reports role, for Apple's opt-in
+# rate: its issuer and key IDs here, its .p8 at /etc/session-ops/asc-key.p8 (mode 600).
+ASC_ISSUER_ID=
+ASC_KEY_ID=

deploy/install.sh

@@ -4,8 +4,9 @@
 #
 # It creates the accounts, builds the venv, creates any missing env file empty, moves
 # state and env files from the layout before session-ops@ units, installs the units
-# and each job's drop-ins, and enables every job whose env files have content, and no
-# other: on its own timer, on the queue's, or on its path unit. It never edits nginx, which certbot owns; see deploy/README.md for the route.
+# and each job's drop-ins, and enables every job that is on and whose env files have
+# content, and no other: on its own timer, on the queue's, or on its path unit. It
+# never edits nginx, which certbot owns; see deploy/README.md for the route.
 set -eu
 
 ROOT=$(cd "$(dirname "$0")/.." && pwd)
@@ -68,15 +69,16 @@ move /etc/github-prs/env "$ETC/github-prs.env" 600 root
 move "$ETC/env" "$ETC/alerts.env" 600 root
 # systemd reads EnvironmentFile= as root before dropping privileges, so these need
 # no group: a job's account cannot read another job's secrets.
-for name in zendesk github-prs crowdin publish alerts discord store-versions; do
+for name in zendesk github-prs crowdin publish alerts mau discord store-versions; do
     [ -e "$ETC/$name.env" ] || install -m 600 /dev/null "$ETC/$name.env"
 done
 # What each file takes, commented; the env files stay empty until filled, since a job
 # is enabled once its env files have content.
 install -m 644 "$ROOT"/deploy/env/*.example "$ETC/"
-# The publishing units load this as a credential, and a missing file would stop them
-# starting; left empty, their runs exit naming the key.
+# The publishing units and mau load these as credentials, and a missing file would stop
+# them starting; left empty, their runs fail naming the key.
 [ -e "$ETC/github-app.pem" ] || install -m 600 /dev/null "$ETC/github-app.pem"
+[ -e "$ETC/asc-key.p8" ] || install -m 600 /dev/null "$ETC/asc-key.p8"
 
 install -d -m 755 "$STATE"
 install -d -o ghdigest -g ghdigest "$STATE/github-prs-digest"
@@ -92,6 +94,11 @@ if [ -e "$NEW_HOUSE" ] && grep -qx "ZENDESK_HOUSE_ANSWERS=$OLD_HOUSE" "$ETC/zend
     echo "pointed ZENDESK_HOUSE_ANSWERS in $ETC/zendesk.env at $NEW_HOUSE"
 fi
 
+# The Discord relay's account puts /mau-upload's exports and /mau's request there, and
+# mau's account reads and moves them out.
+install -d -o sessionops -g sessionops -m 711 "$STATE/mau"
+install -d -o sessionops -g opsbot -m 770 "$STATE/mau/inbox"
+
 install -m 644 "$ROOT/deploy/session-ops.tmpfiles" /etc/tmpfiles.d/session-ops.conf
 systemd-tmpfiles --create session-ops.conf
 
@@ -170,6 +177,9 @@ fi
 for job in $("$OPS" list --not-ready); do
     echo "not enabled: session-ops@$job (its env file is empty)"
 done
+for job in $("$OPS" list --off); do
+    echo "not enabled: session-ops@$job (enabled = false in jobs.toml)"
+done
 # An always-on service, enabled once its env file has content.
 relay() {
     if [ -s "$ETC/$2.env" ]; then

deploy/session-ops-discord.service

@@ -34,6 +34,8 @@ MemoryDenyWriteExecute=yes
 SystemCallFilter=@system-service
 SystemCallErrorNumber=EPERM
 ReadWritePaths=/run/session-ops/requests
+# `-`: a host without the inbox yet still answers /run.
+ReadWritePaths=-/var/lib/session-ops/mau/inbox
 
 [Install]
 WantedBy=multi-user.target

docs/jobs/mau.md

@@ -0,0 +1,122 @@
+# Monthly Active Users
+
+`/mau` in Discord answers with last month's monthly active users, per platform and in
+total, or tells the person asking which exports it still lacks.
+
+mau is off: `enabled = false` in `jobs.toml`, so neither `/mau` nor `/mau-upload` is
+registered. To turn it on, remove that line, fill `mau.env` and `asc-key.p8`, then run
+`install.sh` and `session-ops discord-register` again.
+
+- Android: Play's MAU on the month's last day: users who opened Session in the 28 days
+  before.
+- Android outside Play: the latest stable release's GitHub APK downloads, and an F-Droid
+  estimate of 10% of every other figure, since F-Droid publishes no counts.
+- iOS: App Store Connect's Active in Last 30 Days on the month's last day, which counts
+  only devices sharing analytics with developers, divided by the month's opt-in rate
+  from Apple's App Opt In report, the share of first-time downloaders who opt in.
+- Neither store has an API for the active users, so they come from the exports handed
+  to `/mau-upload`. Each comes with the change on the month before.
+- Desktop, which has no active-user count: the latest stable release's downloads per
+  platform since its release, read from GitHub and Flathub when `/mau` runs.
+- The total adds them all. The answer names every estimate and how it was made.
+
+| | |
+| --- | --- |
+| Runs | `/mau [month:YYYY-MM]` in [Discord](session-ops-discord.md), and nothing else: no timer, so the silence checker does not watch it |
+| Answers | Only the person asking, through the interaction: the figures, a missing export or a failure. A failure also goes to `alerts.env`'s channel |
+| Secrets | `/etc/session-ops/mau.env`: `ASC_ISSUER_ID` and `ASC_KEY_ID`; `/etc/session-ops/asc-key.p8`, an App Store Connect team key with the Sales and Reports role |
+| Inbox | `/var/lib/session-ops/mau/inbox/` |
+| Dry run | `session-ops run mau --dry-run [-- --month YYYY-MM]` prints the answer, and moves and writes nothing; only once mau is on. On the host, see below |
+| Logs | `journalctl -u session-ops@mau -n 50 --no-pager` |
+
+## The exports
+
+One file per platform, each a daily series covering the month's last day, from a store
+set to English:
+
+| | Where | What |
+| --- | --- | --- |
+| Android | Play Console → Statistics, a saved report | Monthly active users (MAU), Unique users, Per interval, Daily; all countries, no breakdown; Export report → CSV |
+| iOS | App Store Connect → Analytics → Session → Metrics | Active in Last 30 Days, daily, no breakdown; Export |
+
+Hand each to `/mau-upload`, which says at once if it is not an export the job reads,
+and whether it has a final figure for last month's end. The job tells the two apart by
+their columns, so both go in the same inbox. From a shell instead:
+
+```sh
+rsync "All countries _ regions.csv" session_private_messenger-active_last_30_days-*.csv \
+    root@<host>:/var/lib/session-ops/mau/inbox/
+```
+
+`rsync`, not `scp`: it writes to a hidden temporary name and renames it once complete,
+and the job only reads `*.csv`. App Store Connect's Active Devices export is refused:
+it counts each day apart, and the App Store Connect API's App Sessions report is no
+substitute, since summing its rows counts a device once per app version, OS and
+territory it used in the month, 5% to 20% too many.
+
+## What `/mau` does
+
+The relay leaves the interaction to answer in `inbox/request.json` and starts the job,
+which removes that file before anything else.
+
+1. Merges every export in `inbox/` into `history.json`, one figure per platform and day,
+   then moves it to `done/`. An export may cover any range; where two give a day different
+   figures, the later one wins, and the change is kept until the figures are shown.
+2. Moves a file it cannot read to `rejected/` and fails the run naming it, once it has
+   answered.
+3. Answers with the figures once `history.json` holds a final figure for the month's last
+   day on both platforms. Play's figures trail by about eight days, so that is around the
+   9th. Apple gives a day it is still counting as a fraction, which does not count as
+   final. Until then, it tells the person asking which exports are missing.
+4. Lists in the answer any change to the month before's last day, the figure the
+   previous month's answer showed. A change is kept until no default `/mau` compares
+   against that day, so everyone asking that month sees it.
+
+A run that fails before step 1 leaves `inbox/` as it is; after it, the exports are in
+`history.json`. Either way, the next `/mau` picks up from there.
+
+The dry run on the host needs mau's account, environment and key, as its unit has them:
+
+```bash
+systemd-run --pipe --wait -p User=sessionops \
+  -p EnvironmentFile=/etc/session-ops/alerts.env -p EnvironmentFile=/etc/session-ops/mau.env \
+  -p LoadCredential=asc-key.p8:/etc/session-ops/asc-key.p8 \
+  -p StateDirectory=session-ops/mau -p StateDirectoryMode=0711 -p UMask=0077 \
+  /opt/session-ops/.venv/bin/session-ops run mau --dry-run -- --month 2026-09
+```
+
+Outside `--dry-run`, mau refuses to start as another account than the state's owner,
+and does nothing without a request from `/mau`.
+
+`history.json` cannot be rebuilt by a re-run: an unreadable one stops the job. If it is
+lost, hand over an export covering the last 365 days for each platform.
+
+## Desktop downloads
+
+The latest Desktop release that is neither a draft nor a pre-release. Updates count
+where the updater fetches an installer.
+
+| Platform | Counted |
+| --- | --- |
+| Linux | `.deb`, `.AppImage`, `.rpm`, `.freebsd`, and Flathub's installs of `network.loki.Session` since the release day |
+| macOS | `.dmg`, `.zip` |
+| Windows | `.exe` |
+
+`.blockmap`, `latest*.yml` and `signature.asc` are left out. Flathub builds from the
+GitHub `.deb` once, so its installs are not in GitHub's counts; Homebrew's `session`
+cask downloads the GitHub `.dmg`, so it already is. Flathub keeps 180 days of daily
+installs, so the run fails on a release older than that rather than undercount.
+
+## Apple's opt-in rate
+
+The job reads the App Opt In report from every analytics report request on Session's
+App Store Connect app: the one-time snapshot for history, and the ongoing one for each
+new day. A Sales and Reports key may read them but not create them; if Apple stops the
+ongoing request for inactivity, the run fails saying so, and an Admin key has to
+create a new one (`POST /v1/analyticsReportRequests`, `accessType: ONGOING`).
+
+## Android outside Play
+
+The `.apk` files of the latest Android release on GitHub that is neither a draft nor a
+pre-release, since its release, updates included. F-Droid builds its own APK and
+publishes no download counts, so its figure is 10% of every other figure in the answer.

docs/jobs/session-ops-discord.md

@@ -9,7 +9,9 @@
 
 | Command | Does |
 | --- | --- |
-| `/run job:<job>` | Starts `session-ops@<job>.service` for a job `jobs.toml` marks `discord = true` by writing `/run/session-ops/requests/<job>`, which the job's path unit watches, then waiting up to 1.5 seconds for the unit to start. If it has not, the reply says the job did not start. Refused while that job or any queued job is running or waiting to, or when the queue's next run is sooner than the job's timeout from now, or while another command is starting a job. The job posts its outcome in its own channel. |
+| `/run job:<job>` | Starts `session-ops@<job>.service` for a job `jobs.toml` marks `discord = true`, mau aside, by writing `/run/session-ops/requests/<job>`, which the job's path unit watches, then waiting up to 1.5 seconds for the unit to start. If it has not, the reply says the job did not start. Refused while that job or any queued job is running or waiting to, or when the queue's next run is sooner than the job's timeout from now, or while another command is starting a job. The job posts its outcome in its own channel. |
+| `/mau [month:YYYY-MM]` | Starts [mau](mau.md) as `/run` does, leaving it the interaction in its inbox to answer, to the person asking alone. A month that has not ended is refused first. |
+| `/mau-upload file:<csv>` | Downloads the attachment, checks it parses as a Play Console or App Store Connect export, and moves it into mau's inbox for the next `/mau`. Replies to the uploader alone, saying whether the export has last month's final figure. A file mau would reject is refused and never reaches the inbox. `/mau` and `/mau-upload` are registered and answered only while mau is on. |
 | `/versions` | One line per platform from [store-versions](store-versions.md)' last run: the store's version, GitHub's, ⚠️ while a warning is out, and since when a source has not read. Visible to the channel, and noting when that run is over 15 minutes old. |
 
 The app has no gateway connection and only the `applications.commands` scope, so Discord
@@ -18,17 +20,18 @@ channels. Each request carries who ran it, their roles, the server, the channel
 options chosen.
 
 A request is refused unless Discord signed it in the last five minutes, it comes from
-`DISCORD_GUILD_ID` and, when `ALLOWED_CHANNEL_IDS` is set, one of its channels, and its
-author is in `ALLOWED_USER_IDS` or holds a role in `ALLOWED_ROLE_IDS`. Both lists empty
-refuses everybody. `/run` is registered hidden from everyone but server administrators,
-and the server's Integrations settings show it to the right role; that only hides it, and
-these checks are what refuse everyone else, administrators included. `/versions` only reads, so it is shown to
+`DISCORD_GUILD_ID` and, when `ALLOWED_CHANNEL_IDS` is set, one of its channels, and its author is in `ALLOWED_USER_IDS` or holds a role in
+`ALLOWED_ROLE_IDS`. Both lists empty refuses everybody. `/run`, `/mau` and `/mau-upload` are
+registered hidden from everyone but server administrators, and the server's Integrations
+settings show them to the right role; that only hides them, and these checks are what
+refuse everyone else, administrators included. `/versions` only reads, so it is shown to
 everyone and skips the author check.
 
-The relay runs as `opsbot`, which can start no unit itself. It can write only to
+The relay runs as `opsbot`, which can start no unit itself. It can write only two places:
 `/run/session-ops/requests`, where `session-ops units` writes a `PathChanged=` watch for
-each `discord = true` job and nothing else, so a file of any other name starts nothing.
-It reads store-versions' `state.json`, which that job writes 0644. A
+each `discord = true` job and nothing else, so a file of any other name starts nothing;
+and the mau inbox. It reads store-versions' `state.json`, which that job writes 0644. A
 write to a request starts its job once, so nothing removes requests, and root never acts
 on a file `opsbot` controls. The job runs as its own account with its own unit's
-settings. The request carries nothing.
+settings. The request carries nothing; mau's `request.json` carries only the
+interaction to answer, which mau checks the shape of, and the month asked.

src/session_ops/jobs.toml

@@ -12,7 +12,10 @@
 #   repeat_alert_hours  report the same failure again only this often until a run
 #                  succeeds, for a schedule short enough to repeat it every few minutes
 #   unit           further [Service] lines, over the template's hardening
-#   discord        startable by the Discord relay's account: offered to `/run`
+#   discord        startable by the Discord relay's account: offered to `/run`, except
+#                  mau, which /mau starts
+#   enabled        false keeps a job off: install.sh enables none of its units, the
+#                  runner refuses it, and neither the silence checker nor Discord offers it
 #
 #   max_age_hours  for a queued job, its queue's: the weekend's 72 h (73 h across
 #                  April's DST change), plus every job ahead of it running to its timeout
@@ -137,3 +140,17 @@ channel_env = "CROWDIN_DISCORD_WEBHOOK_URL"
 max_age_hours = 80
 timeout = "10min"
 unit = ["LoadCredential=github-app.pem:/etc/session-ops/github-app.pem"]
+
+[[job]]
+name = "mau"
+description = "Monthly active users per platform, the answer to /mau"
+entry = "session_ops.platforms.mau:main"
+args = ["--state", "{state}"]
+user = "sessionops"
+env_files = ["/etc/session-ops/alerts.env", "/etc/session-ops/mau.env"]
+env = ["ASC_ISSUER_ID", "ASC_KEY_ID"]
+discord = true
+enabled = false
+timeout = "5min"
+# The App Store Connect API key, for Apple's opt-in rate.
+unit = ["LoadCredential=asc-key.p8:/etc/session-ops/asc-key.p8"]