Skip to content

About

AI gateway with model routing, MCP aggregation, redaction, and scalable relay-worker deployment.

Resources

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

prompt-ferry

An OpenAI-compatible AI gateway with peer-connected nodes. One node serves the browser, admin API, AI API and peer endpoint on one listener (default :8787). A node with PostgreSQL executes requests; a database-less edge forwards browser and AI traffic over its authenticated peer connection, without connecting to PostgreSQL or Valkey. Either side can list the other in PROMPT_FERRY__PEER_URLS; there is no multi-hop routing.

Quick start

For an existing installation, follow MIGRATING.md first. For a new Compose stack:

  1. Copy .env.example to .env. Generate PROMPT_FERRY__MASTER_SECRET once with openssl rand -base64 32, keep it private and back it up. Fill the database credentials. The host database URL and POSTGRES_PASSWORD contain separate copies of the password; change both together.
  2. Run docker compose up -d. Raw request and response bodies are stored in PostgreSQL, so no object-store bucket is needed. The Compose example binds the node and service ports to loopback and keeps both data and config volumes. Do not discard them: the data root contains the persistent node identity and old config volumes may contain backups.
  3. Visit http://127.0.0.1:8787. Complete first setup using the master secret over a trusted local connection or HTTPS, then log in normally. Create a managed AI client key in the UI and use that as the API bearer; the master secret is not an AI bearer token.

Managed client keys authenticate inference only: the applied route snapshot binds a key to its user and route, and the key credential cannot reach the admin control surface or user usage/billing data. Administrator actions require the existing session cookie and its live administrator role, while user-owned business routes keep requiring an authenticated session. Legacy static-token and database-less edge deployments keep their existing authentication path. Set PROMPT_FERRY__DATABASE_URL to a PostgreSQL URL for a database node or leave it empty for an edge. Set peers as a JSON array, for example ["wss://node.example:8787"], or []. All peers must run the same compiled version and protocol and share the same master secret. Nodes using the same database share state; multiple database nodes in a group require shared Valkey. A single database node without Valkey uses local sessions, which log out on restart; a configured but unavailable Valkey does not fall back. No cross-group in-memory takeover or transparent retry is promised once generation starts.

PostgreSQL must run TimescaleDB 2.32.0 or newer with the TSL license; the Compose example pins the non-OSS timescale/timescaledb:2.32.0-pg18 image. The request family is an eight-hypertable, one-day-chunk layout with the native columnstore. The runtime does not create the extension itself: install it once with CREATE EXTENSION IF NOT EXISTS timescaledb as a superuser if the image entrypoint did not. A database initialized with the legacy native-partition request family is refused before any gate or schema write; export its configuration and restore it into a fresh TimescaleDB database instead of reusing it. Closed day chunks are compressed by policy roughly 24 to 48 hours after their first row, so recent rows remain uncompressed. Retention is applied by the runtime maintenance window as whole-chunk drops under independent metadata, content and raw horizons, and the current UTC day chunk always survives; a full-history admin clear drops every older chunk and reports chunk drops separately from its bounded current-day row deletes. A bounded clear decompresses the chunks it touches inside one transaction before deleting rows, so it is not subject to the per-statement decompression guard and either commits all deletes or leaves the chunks as they were.

Request statistics (the summary, the series buckets and the overview) are served from a one-minute continuous aggregate over the request family. Per-request percentiles such as p95 always come from the raw rows, never from the rollup. A one-minute policy materializes and repairs the aggregate, and a clear or retention chunk drop refreshes the minutes it touched before it returns.

Use a canonical public HTTPS origin for MCP OAuth callbacks (PROMPT_FERRY__MCP_OAUTH_REDIRECT_BASE). Browser login is shared on the same origin; different domains require separate logins. Reverse proxies must preserve the original Host for setup Origin checks. See SECURITY.md for trust, TLS and proxy requirements.

Development

Configure a stable master in the environment or a separate dotenv selected by PROMPT_FERRY_DOTENV, then run bash scripts/dev.sh backend or bash scripts/dev.sh full. Full mode uses plaintext on the default 127.0.0.1:8787 node bind and Vite's API proxy; use it only on a trusted development host. The scripts do not generate a master. Run backend unit checks with bash scripts/ci/run-tests.sh; see CONTRIBUTING.md.

About

AI gateway with model routing, MCP aggregation, redaction, and scalable relay-worker deployment.

Resources

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages