Repository navigation
Workspace restructure and gatekeeper features - #2
Merged
Merged
Conversation
The engine (pattern-guard) moves to crates/engine and the gateway to crates/gateway. The root Cargo.toml becomes a virtual manifest that shares package metadata and dependency versions across crates (resolver 3, MSRV-aware). No code changes beyond path fixes in the benchmark and CLI integration test.
The payload-inspection engine moves to crates/waf so it can grow (word lists, parsers, libinjection) and be used on its own. The engine re-exports it as pattern_guard::waf, so existing users are unaffected.
Pure move: decoding, the indicator library and request walking get one module each, so WAF 2.0 work (word lists, parsers, scoring) has an obvious home. Behaviour is unchanged; the CRS benchmark result is identical (11.8 % of attack tests blocked).
ml/ holds the feature engine, model and built-in model; rules/ the optional threshold rules; util/ route templating and decayed counters. Public paths (pattern_guard::features, ::model, ::normalize) are kept through re-exports. No behaviour change.
The pattern-guard binary now lives in crates/cli, split into one module per subcommand (replay, train, wrap) plus shared argument/log helpers. The engine crate becomes a pure library. Same commands and flags.
- proxy/: request pipeline and decision fusion (mod), plus client IP resolution, upstream forwarding, generated responses, challenge verification and decision logging in their own files - intel/: reputation store, CrowdSec bouncer, Suricata/Wazuh feeds - observability/: metrics, JSON decision log, API inventory - challenge/: the proof-of-work page is now a real page.html No behaviour change; all gateway end-to-end tests pass unchanged.
The traffic simulator becomes a small library in an unpublished pattern-guard-eval crate, with realworld_eval, gen_traffic, waf_eval and crs_eval as its binaries (cargo run -p pattern-guard-eval --bin ...). Engine examples keep only the two demos. Results are unchanged (realworld_eval: 0.27 % false positives, 98.7 % detection).
guard.toml.example and gateway.toml.example move to config/, the two demo configs to config/demo/. Demos, the benchmark and the CLI tests locate them via CARGO_MANIFEST_DIR, so they run from any directory.
README and ARCHITECTURE gain the project tree; commands and file paths in the guides, feature tracker and research notes follow the new crates (-p pattern-guard-gateway / -cli / -eval, config/ examples); changelog notes the restructure.
[[rules]] in gateway.toml: conditions on path prefix/regex, methods, hosts (with *.wildcards), client IP in / not in CIDRs, user agent, query string and headers (present, matching or absent); actions allow (bypass all inspection), block, challenge, delay, log and skip_waf. Rules run in file order and the first terminal action wins; they join the decision fusion like the other layers and show up as layer "rule" in metrics and the decision log. Patterns are compiled at startup, so mistakes are reported by --check.
[[rate_limits]] with a route match (path prefix/regex, methods), a key (ip, ip_route or header:<name>) and a token bucket (requests per per_secs, bursts up to requests). Exceeding a limit blocks with 429 Too Many Requests and Retry-After (HTML or JSON), or challenges or delays. Buckets live in a bounded LRU; the decision is logged as layer "rate_limit".
- [bans]: threshold blocks within window_secs ban the IP for ban_secs, multiplied by escalation for every previous ban (capped). Bans are written to the intel store as source gateway:ban, so they are enforced, expire and are logged like other reputation entries. - [traps]: paths legitimate visitors never request; one hit bans immediately (layer "trap"). - Admin port: GET /bans lists active bans, POST /bans/unban?ip= lifts one.
[protocol]: allowed methods (405), request-target length (414), header count and total size (431), query argument count (400), NUL bytes in the target (400), required Host (400) and restricted file extensions such as .bak or .sql (403). Each refusal uses its proper status code and is logged as layer "protocol"; enforced refusals count towards automatic bans.
New hardening/ module. [response_headers] adds X-Content-Type-Options, X-Frame-Options and Referrer-Policy by default, and optionally HSTS, CSP, Permissions-Policy and COOP, only where the app hasn't set them; it strips server-fingerprinting headers (Server, X-Powered-By, ...). Invalid header values are rejected at startup.
[response_inspection] (opt-in) scans uncompressed text responses for database errors, PHP/Java/Python/.NET/Node stack traces, directory listings, private keys and AWS access keys. In prevent mode with action = "block" the response is replaced by a generic 500 page (layer "data_leak"); otherwise it is passed through and logged. Bodies longer than max_bytes are inspected at the start and the remainder is streamed unchanged.
[geo] loads an IP-to-country CSV (start_ip,end_ip,country_code, IPv4 and IPv6; the DB-IP Lite format) and applies deny_countries, challenge_countries and allow_countries (unknown or private addresses are never blocked). Custom rules gain a `countries` condition, and the decision log records each request's country. Configuring country rules without a database is rejected at startup.
Gateway guide sections for custom rules, rate limits, bans and traps, protocol enforcement, security headers, data-leak blocking and country access control, plus the layer precedence and new admin endpoints. Feature tracker rows updated, with a new parity table against Coraza/ModSecurity, nginx, fail2ban, CrowdSec, BunkerWeb, SafeLine, Anubis and open-appsec that also lists what is still missing. README status table, architecture layout and changelog updated.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two parts, one commit per step. Every commit builds and passes its tests.
1. Project restructure: one crate per product area, feature folders inside
Public Rust paths are unchanged. Benchmark results are unchanged: realworld_eval 0.27 % false positives / 98.7 % detection, and the WAF blocks 11.8 % of the CRS tests.
2. Gatekeeper features in the gateway
[[rules]]):[[rate_limits]]): a token bucket per route, keyed by IP, IP + route or API-key header. Over the limit gets 429 with Retry-After.[bans],[traps]), withGET /bansandPOST /bans/unbanon the admin port.[protocol]): methods 405, URL length 414, header count/size 431, query arguments 400, NUL bytes 400, missing Host 400, restricted extensions 403.[response_headers]), with fingerprinting headers removed.[response_inspection], opt-in): SQL errors, stack traces, directory listings and keys are replaced by a generic 500. Bodies larger than the inspection window stream through intact.[geo]): an IP-to-country CSV in DB-IP Lite format, a deny / challenge / allow policy, and acountriescondition for rules.Docs: gateway guide sections, a feature-tracker parity table against Coraza/ModSecurity, nginx, fail2ban, CrowdSec, BunkerWeb, SafeLine, Anubis and open-appsec (including what is still missing), README, architecture and changelog.
Checks
GitHub Actions has not been able to assign a runner on this repository (see PR #1). All CI steps were run locally on Rust 1.85, the version CI pins, and on stable:
cargo fmt --all --checkcargo clippy --workspace --all-targets -- -D warningscargo test --workspace: 112 tests, including 16 gateway end-to-end tests over real TCPcargo bench --workspace --no-runMerged by fast-forwarding
main, so the per-feature commits are kept unchanged.