Skip to content

feat(usage): prune token usage past a declared retention window - #276

Open
wolfy-j wants to merge 1 commit into
masterfrom
feat/usage-retention
Open

wolfy-j wants to merge 1 commit into
masterfrom
feat/usage-retention

Conversation

@wolfy-j

@wolfy-j wolfy-j commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

Token usage records are never deleted, so token_usage and token_usage_cache_writes grow without bound. This adds a declared retention policy, a bounded prune and a retention owner. Rollups are out of scope, and the module adds no scheduler of its own.

wippy.usage:policy declares meta.retention.token_usage (default "90d", 1d to 3650d) and batch_token_usage (default 500, at most 500). An invalid or missing value fails closed with a non-retryable INVALID error, and nothing is deleted. Override the entry to change the window.

token_usage_repo.prune(cutoff, limit) deletes at most limit records stamped at or before the cutoff, oldest first, through idx_token_usage_timestamp. It reports whether more expired records remain and the timestamp of the oldest remaining record. It deletes the token_usage_cache_writes rows of the batch in the same transaction. The ON DELETE CASCADE fires on PostgreSQL, but SQLite enforces it only on databases with foreign_keys: true, and that option is off by default.

wippy.usage:retention carries meta.type: kickside.retention.owner. Hosts that discover owners of that type call it on their own retention schedule. With no such host, nothing calls it. It returns {success, deleted, needs_continue, deadline_ms}. deadline_ms is the time the oldest remaining record expires, and the field is absent while expired records remain. The owner runs as system:wippy.usage.retention. Its two declared policies grant db.get on the target_db value, injected by the requirement, and registry.get on the requirement and the policy. These two grants are enough when the caller's scope has no grants at all.

Validation on Wippy v0.3.65a: 25 usage tests pass on SQLite and on PostgreSQL 17 with TZ=America/New_York. That includes 10 new retention cases: default and override, invalid policies, boundary, cache writes, batch continuation, oldest timestamp, argument checks, owner protocol, and an owner call from a scope with no grants. Before the change, the suite failed to load because wippy.usage:retention_policy was not found. With the explicit child delete removed, the SQLite cache-write cases fail (expected 0, got 2). With the declared policies removed, the no-grant owner case fails. Usage lint reports no issues, and the module manifest check passes.

@wolfy-j
wolfy-j requested a review from skhaz November 17, 2026 02:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants