diff --git a/README.md b/README.md index 7928569..af412bb 100644 --- a/README.md +++ b/README.md @@ -1,150 +1,175 @@ -# NetGuardian — Network Diagnostics & Evidence-Based Troubleshooting Platform +# NetGuardian — Hospitality IT Infrastructure Monitoring & Network Diagnostics Platform -Explore the fictional hotel dashboard on the web app home page after starting NetGuardian. It -requires no workspace token and uses only simulated observations. See -[Hospitality IT Operations](docs/product/HOSPITALITY_OPERATIONS.md). +[![Baseline checks](https://github.com/TYF009/NetGuardian/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/TYF009/NetGuardian/actions/workflows/ci.yml) [![MIT License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE) **English** · [简体中文](README.zh-CN.md) +[Demo quick start](#demo-quick-start) · [First diagnostic](#first-diagnostic-walkthrough) · [Full local deployment](#full-local-deployment) · [Documentation](docs/README.md) -NetGuardian helps IT teams investigate why a service is unreachable by checking -each layer of the connection and keeping a record of what each check found. +NetGuardian is an open-source network monitoring and evidence-based troubleshooting platform developed to support hotel IT operations. I designed and built it independently using FastAPI, Next.js, a background worker, SQLite/PostgreSQL and a Windows Agent. -[Quick start](#quick-start) · [Architecture](#system-architecture) · [Demo evidence](#demo-evidence) · [Verification](#test--verification-status) +- **Hotel operations:** Approved for use and deployed in IT operations at Hengze Hotel. +- **Portfolio demo:** A token-free walkthrough of a fictional property, served locally from the frontend. +- **Open-source implementation:** The authenticated workspace, scheduler, Agent and evidence pipeline are available under the MIT license. + +![Simulated hotel dashboard with endpoint trends and service health](docs/assets/screenshots/hotel-dashboard.jpg) + +*Aster Quay Hotel is fictional. Dashboard data is simulated.* ## Why I Built This -When a workstation can reach one service but a front-desk device cannot, -the support ticket rarely tells you whether the failure is in name resolution, -connectivity, TLS, or the application. I built NetGuardian to make that investigation -repeatable and to give the next technician the observations behind a finding. +Supporting hotel LAN/WLAN, Windows workstations, front desk systems and POS connectivity exposed a recurring problem: DNS, TCP, TLS and HTTP checks were spread across tools, repeated manually, and difficult to hand off with consistent evidence. I built NetGuardian to bring authorized checks, scheduled observations, incident history and evidence reports into one workflow. It grew from a Streamlit prototype into a FastAPI control plane, a Next.js workspace, a durable worker and an outbound Windows Agent. -NetGuardian grew from a Python and Streamlit toolkit into an independent portfolio -project informed by my work troubleshooting LAN/WLAN, TCP/IP, DNS, DHCP and -Windows endpoint connectivity. The scenarios here use controlled test systems. +## Hengze Hotel IT Operations -## Key Features +NetGuardian has been approved for use and deployed in IT operations at Hengze Hotel, where I worked as an IT Infrastructure & Network Engineer. My responsibilities included LAN/WLAN support, TCP/IP, DNS and DHCP troubleshooting, Windows support, front desk and POS connectivity, and Python automation. Production network data, deployment scope and the modules used at the hotel are private. Device counts, availability, MTTR and efficiency gains are not published. CI and integration tests do not validate hotel production performance. The public demo is isolated and uses Aster Quay Hotel, a fictional property. -| Capability | What it shows | -| --- | --- | -| Layered diagnostics | DNS records, TCP connectivity, TLS handshake and certificate details, and HTTP HEAD status and timing. Opt-in ICMP and traceroute provide additional reachability and path evidence. | -| Scheduled monitoring | Pause and resume checks, review collection freshness, and follow incident and recovery history. Missing observations are shown as gaps. | -| Two-location comparison | Pair a server-side check with one from an enrolled Windows Agent. Compare the recorded observations within the limits of their endpoints and timing. | -| Incident evidence | Open saved, redacted report snapshots, verify their SHA-256 digests, and trace findings back to the underlying observations. | -| Optional investigation | Use a configured Ollama model to summarize report evidence. Validated findings and unverified hypotheses are displayed separately. | +## Screenshots -## System Architecture +The images here and in the opening section are real browser captures of the simulated Aster Quay Hotel demo. -```mermaid -flowchart LR - Operator --> Web["Next.js dashboard"] - Web --> API["FastAPI control API"] - API --> DB[("SQLite / PostgreSQL")] - Worker["Background Worker"] --> DB - Worker --> Engine["Shared probe engine"] - Agent["Outbound Windows Agent"] -->|"Poll jobs and return observations over HTTPS"| API - Agent --> Engine - Engine --> Targets["Approved targets"] -``` +![Simulated topology with a selected POS node and incident timeline](docs/assets/screenshots/hotel-topology.jpg) -- [Web dashboard](web/) and [FastAPI API](service/api.py) handle authenticated requests and workspace views. -- The [Worker](service/worker.py) claims leased jobs from the database and persists observations. The [Agent](agent/) polls the API over HTTPS and applies its own local target policy. -- The [probe engine](probe_core/) runs bounded checks. [Alembic migrations](migrations/) track schema changes, and [report export](service/report_files.py) checks saved snapshot integrity. +![Simulated POS HTTP 503 in the DNS, TCP, TLS and HTTP diagnostic view](docs/assets/screenshots/hotel-diagnostics.jpg) -See the [system design](docs/architecture/SYSTEM_DESIGN.md) and [data model](docs/architecture/DATA_MODEL.md). +[More screenshots, including the authenticated workspace and evidence report](docs/product/DEMO_GUIDE.md) -## Engineering Highlights +## End-to-End IT Operations Workflow -- **Probe safety.** Server-side probes deny targets by default and require exact approved origins. The executor connects to a validated numeric address while preserving the hostname for HTTP and TLS. HTTP uses HEAD with bounded headers and no redirect following. -- **Measurement semantics.** DNS, TCP, TLS and HTTP timing are kept separate. HTTP duration describes an application-layer request, while ICMP round-trip time measures an echo exchange. -- **Asynchronous delivery.** API acceptance is separate from execution. Jobs have leases and expiry, and the Agent has a local result spool to retry delivery without repeating a completed probe. -- **Reproducible evidence.** Reports retain the observations used to explain an incident. The export preserves the stored snapshot for [offline checksum verification](scripts/verify_report.py). +```mermaid +flowchart LR + A["Configure and authorize · operator"] --> B["Monitor · scheduler and probe"] + B --> C["Detect · threshold and incident"] + C --> D["Investigate · operator reviews evidence"] + D --> E["Report · operator requests snapshot"] + E --> F["Review · operator verifies and hands off"] +``` -See [responsible probing and measurement details](docs/operations/LOCAL_DEVELOPMENT.md#target-policy-and-measurement-semantics) and the [threat model](docs/security/THREAT_MODEL.md). +In the demo, a POS endpoint returns HTTP 503 while the probe completes and collection remains current. The operator can inspect recorded layer evidence and review possible explanations. A separate local HTTPS integration scenario exercises incident creation, HTTP 204 recovery and an evidence snapshot. In the workspace the scheduler runs checks and applies incident thresholds automatically. An operator authorizes targets, assesses probable causes, requests reports and decides what action to take. Recovery records a measured change; it does not identify a root cause. -## Demo Evidence +## Core Features -| Scenario | Evidence | +| Capability | Implementation and verification | | --- | --- | -| Service returns HTTP 503, then recovers to HTTP 204 | A controlled loopback HTTPS integration exercised an Agent monitor. The local browser QA recorded both statuses, incident recovery and the checksum-verified report. [Browser QA record](docs/product/BROWSER_QA.md#populated-workflow-check-2026-11-05) | -| Operator exports a report | A local browser download was checked with the offline verifier. The test environment used isolated evidence and a test workspace. [Export QA record](docs/product/BROWSER_QA.md#real-model-investigation-and-report-export-2026-11-06) | -| An operator browses missing observations | A synthetic fixture of 101 failed jobs exercised pagination and the UI message for missing measurements. [Fixture QA record](docs/product/BROWSER_QA.md#retained-history-production-web-build-2026-11-05) | +| Network diagnostics | DNS, TCP, TLS and HTTP HEAD. ICMP and traceroute are opt-in system probes with bounded commands and explicit capability failures. HTTP duration and ICMP RTT are separate. | +| Scheduled monitoring and incidents | Durable schedules, freshness, failure and recovery thresholds, retained incident evidence and explicit gaps. Covered by worker and local integration checks. | +| Windows Agent and comparison | Enrollment and outbound HTTPS, independent local policy, and cloud-to-Agent comparison when the observations meet compatibility requirements. Verified in controlled integration. | +| Hospitality dashboard and topology | Live workspace trends use authorized API data. The interactive React Flow topology uses configured logical links in the simulated demo. | +| Evidence reports | Immutable snapshots and SHA-256 integrity verification. The digest can detect modified content; it does not authenticate the author or the observation. | +| Optional Ollama investigation | Evidence-checked recorded facts and separately labeled hypotheses. Experimental interpretation, with limited real-model validation. | +| Security | Default-deny origin policies, DNS/address validation and IP pinning, tenant-scoped data access and independent Agent policy. | -These are local QA records. The HTTPS results came from a controlled test server, and the history fixture contains synthetic job state. +## Architecture -## Technology Stack +```mermaid +flowchart LR + subgraph Browser + D["Public demo · synthetic fixture"] + U["Authenticated workspace"] + end + subgraph Control["Trusted control plane"] + W["Next.js authenticated proxy"] --> API["FastAPI · workspace and target policy"] + API --> DB[("SQLite / PostgreSQL · jobs and evidence")] + R["Worker · schedules and leases"] <--> DB + end + U -->|"Bearer token"| W + AG["Windows Agent · local policy"] -->|"Outbound HTTPS · poll and submit"| API + R -->|"Approved cloud targets"| T["Network targets"] + AG -->|"Locally approved targets"| T +``` -Python 3.11 · FastAPI · SQLAlchemy · Alembic · SQLite / PostgreSQL · Next.js · React · TypeScript · Node.js 24 · pnpm · pytest · Ruff · GitHub Actions +The API stores jobs; the worker or an enrolled Agent executes them asynchronously. Measurements share a versioned schema, and the database links incident evidence to source jobs. The authenticated workspace calls an allowlisted Next.js proxy. The demo has its own static fixture and no control API connection. [System design](docs/architecture/SYSTEM_DESIGN.md) · [Data model](docs/architecture/DATA_MODEL.md) -## Quick Start +## Demo Quick Start -**Windows local evaluation.** Install Python 3.11 and Node.js 24. From PowerShell: +Use Node.js 24 and pnpm 11.25.0. Run these commands in PowerShell from a directory where you can create a checkout: ```powershell git clone https://github.com/TYF009/NetGuardian.git Set-Location NetGuardian -python -m pip install uv==0.13.7 npm install -g pnpm@11.25.0 -uv sync --locked pnpm --dir web install --frozen-lockfile pnpm --dir web build +pnpm --dir web start +``` + +Open [the local demo](http://127.0.0.1:3000/). `/demo` is an alias. The demo requires no Python, database, Agent, backend or token, and sends no real probes. [Demo walkthrough](docs/product/DEMO_GUIDE.md) + +## Full Local Deployment +Use Windows, Python 3.11, uv 0.13.7, Node.js 24 and pnpm 11.25.0. Install and build the Web app as above, then stop the demo process with Ctrl+C so the full launcher can use port 3000. From the repository root, install Python dependencies and run the one-time setup. Copy the displayed local administrator token to a password manager. The ID file is under the ignored local runtime directory: + +```powershell +python -m pip install uv==0.13.7 +uv sync --locked $workspaceToken = (uv run --locked python -c "import secrets; print(secrets.token_urlsafe(32))").Trim() +$workspaceToken +New-Item -ItemType Directory -Path .netguardian-local -Force | Out-Null +$vantageFile = '.netguardian-local/workspace-vantage-id.txt' +if (-not (Test-Path -LiteralPath $vantageFile)) { + [guid]::NewGuid().ToString() | Set-Content -LiteralPath $vantageFile -Encoding ascii +} +``` + +On first startup and in each new PowerShell session, load the same local token and saved Worker vantage ID. Use an empty allowlist for a no-probe trial. Back up any existing database before using `--migrate`: + +```powershell +$workspaceToken = [pscredential]::new( + 'local', (Read-Host 'Local workspace token' -AsSecureString) +).GetNetworkCredential().Password $env:NETGUARDIAN_TOKENS = @{ $workspaceToken = 'local' } | ConvertTo-Json -Compress $env:NETGUARDIAN_ORIGINS = '{}' -$env:NETGUARDIAN_VANTAGE_ID = [guid]::NewGuid().ToString() -$workspaceToken +$env:NETGUARDIAN_VANTAGE_ID = (Get-Content -LiteralPath '.netguardian-local/workspace-vantage-id.txt' -Raw).Trim() uv run --locked python -m scripts.run_windows --migrate ``` -Open [the local workspace](http://127.0.0.1:3000/workspace) and enter the test workspace token printed -in PowerShell. The empty origin list lets you explore the UI -without enabling probes. Use a fresh checkout or back up an existing database -before migration. Keep a stable Worker location ID when retaining observations -across sessions. +Open [the authenticated workspace](http://127.0.0.1:3000/workspace) and use the generated local token. Keep the saved ID across restarts. The ID labels observations from this Worker location. It grants no access. Keep production location IDs out of demos and test environments. When started with `--migrate`, the launcher applies Alembic migrations before starting the services. Routine restarts can omit this flag when the database schema is current. The launcher starts the API, Worker and Web on loopback. [Separate service commands, database settings, Agent enrollment and troubleshooting](docs/operations/LOCAL_DEVELOPMENT.md) · [Operator runbook](docs/operations/RUNBOOK.md) + +## First Diagnostic Walkthrough + +### Public Demo + +1. Install and start the frontend using [Demo Quick Start](#demo-quick-start). Open the Aster Quay Hotel dashboard and confirm the **DEMO / SIMULATED** banner. +2. Open **Services** and compare the POS probe outcome, endpoint response and collection freshness. The simulated probe completed and recorded HTTP 503. +3. Open **Topology**, select **POS terminals**, then follow **DEMO-104 · Open** to the incident. Choose **Inspect evidence**. +4. Read the DNS, TCP, TLS and HTTP cards. Expand **Raw probe evidence (JSON)** to see the simulated record. Possible explanations and next checks are separate from observations. -For separate API, Worker and Web processes, authorized targets, Agent enrollment -and configuration, see the [local development guide](docs/operations/LOCAL_DEVELOPMENT.md). -See the [operations runbook](docs/operations/RUNBOOK.md) for readiness, backups and -troubleshooting. +### Authenticated Workspace -## Test & Verification Status +1. Use [Full Local Deployment](#full-local-deployment) to install the services and generate a local administrator token. Use a separate local or test workspace, away from production data. +2. Choose an exact HTTPS origin that you own or have explicit permission to monitor and that the cloud Worker can reach under its address policy. The value below is a placeholder. Replace it before running the example. Stop the launcher, set the allowlist in its PowerShell session, then restart without migration if the schema is current. -- **Automated checks.** GitHub Actions runs Python unit and selected runtime tests on Windows and Ubuntu, PostgreSQL integration tests, and Web tests, TypeScript checks, schema drift checks and a Next.js production build. See the [workflow](.github/workflows/ci.yml) and [main baseline run](https://github.com/TYF009/NetGuardian/actions/runs/42789779017). -- **Controlled local verification.** The [Windows delivery record](docs/product/WINDOWS_DELIVERY.md) documents loopback TLS, Agent workflows, database verification and browser checks with their scope. -- **Reproduce checks.** Run these from the repository root. The [local guide](docs/operations/LOCAL_DEVELOPMENT.md#checks-and-operations) describes optional integration prerequisites. + ```powershell + $approvedOrigin = 'https://authorized-host.example:443' + $env:NETGUARDIAN_ORIGINS = @{ local = @($approvedOrigin) } | ConvertTo-Json -Compress + uv run --locked python -m scripts.run_windows + ``` + +3. Open `/workspace`, enter the local token, and choose **Connect**. In **Diagnostics**, select the approved target, a probe such as **HTTP**, and **Cloud worker**. Click **Run diagnostic**. In **Diagnostic history**, wait for the job to finish and use **Inspect** to see its recorded result. If no measurement was returned, check the Worker and task state. An HTTP error response can still come from a completed probe. +4. In **Monitors**, select the approved target, probe and **Cloud worker**, give the monitor a name and an interval of at least 60 seconds, then choose **Create monitor**. Open **View history** after scheduled runs have been recorded. +5. Open **Incidents** if the monitor has enough qualifying observations. The UI uses the default thresholds of three consecutive failures to open an incident and two consecutive successes to recover it. If an incident exists, choose **Create evidence report** and find the saved snapshot in **Reports**. A healthy, new or missing-data monitor may have no report to create. + +## Testing & Verification ```sh uv run --locked ruff check . uv run --locked ruff format --check . uv run --locked pytest -q -uv run --locked pytest integration -q -rs +uv run --locked pytest integration/test_tls.py integration/test_dns_runtime.py integration/test_agent_workflow.py -q pnpm --dir web test pnpm --dir web typecheck pnpm --dir web build ``` -## Current Limitations - -- The documented delivery target is local Windows evaluation. Docker Compose deployment and broader Linux runtime qualification remain open. -- Loopback tests cover the job and Agent flows. Cross-machine rollout, signed Agent distribution and sustained multi-replica load remain unverified. -- Ollama is optional and experimental. A bounded local scenario demonstrates the evidence checks, with broader diagnosis quality still to evaluate. - -See the [delivery checklist](docs/product/DELIVERY_STATUS.md) for the current verification boundaries. +[GitHub Actions](https://github.com/TYF009/NetGuardian/actions/workflows/ci.yml) runs Windows and Ubuntu checks, PostgreSQL integration, OpenAPI drift checks and the Web build. Some integration cases require a disposable PostgreSQL database, real model or OS credential store. [Verification record](docs/product/VERIFICATION.md) · [Browser QA](docs/product/BROWSER_QA.md) · [Windows delivery and limits](docs/product/WINDOWS_DELIVERY.md) -## Roadmap +CI and test fixtures document behavior in controlled environments. The approved hotel use is a separate operational fact. Current local delivery targets Windows; the Compose configuration does not establish a supported public or cross-machine deployment. -- Publish a read-only demo of a controlled incident and recovery with clearly labeled data and a verifiable report. -- Verify Compose startup and migrations from a clean environment, then document the supported operating scope. -- Validate Agent enrollment, reconnect and two-location comparisons across separate test hosts. +## Security & Privacy -## Author +Probe only targets you are authorized to monitor. The control plane enforces exact origin allowlists and scope boundaries. Cloud address validation rejects private and special addresses; an Agent independently enforces its approved origins and explicit private CIDRs where supported. Workspace bearer tokens stay in browser memory and Agent credentials use the OS credential store. Keep production topology, addresses, logs and credentials private. The public demo uses only synthetic data. [Threat model](docs/security/THREAT_MODEL.md) · [Licensing and third-party assets](NOTICE.md) -I am a Communication Engineering graduate with Huawei HCIP Datacom certification, -experience in 400G optical transport operations, and hands-on IT infrastructure -and network support experience. NetGuardian brings that troubleshooting practice -together with Python automation and a web-based evidence workflow. +## Roadmap & Author -## License +Next priorities are private asset and zone configuration, device telemetry, monitor-to-topology mapping, stronger evidence analysis, and broader deployment and operational validation. -The project source and documentation are available under the [MIT License](LICENSE). -See [NOTICE.md](NOTICE.md) for third-party and legacy asset scope. +**TYF009** · Communication Engineering background · HCIP Datacom · Hotel IT infrastructure and technical support · Network automation. diff --git a/README.zh-CN.md b/README.zh-CN.md index 2413a3c..72b4e3c 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -1,127 +1,175 @@ -# NetGuardian · 网络诊断与循证故障排查平台 +# NetGuardian — 酒店 IT 基础设施监控与网络诊断平台 + +[![Baseline checks](https://github.com/TYF009/NetGuardian/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/TYF009/NetGuardian/actions/workflows/ci.yml) [![MIT License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE) [English](README.md) · **简体中文** +[演示快速开始](#演示快速开始) · [首次诊断演练](#首次诊断演练) · [完整本地部署](#完整本地部署) · [文档导航](docs/README.md) -NetGuardian 帮助 IT 支持人员排查服务为何无法访问。它逐层检查连接,并保留每次测量的结果,便于复盘故障和交接。 +NetGuardian 是我独立设计和开发的开源网络监控与证据化排障平台,面向酒店 IT 运维。平台由 FastAPI、Next.js、后台 Worker、SQLite/PostgreSQL 和 Windows Agent 组成。 -[快速开始](#快速开始) · [系统架构](#系统架构) · [演示证据](#演示证据) · [测试与验证](#测试与验证) +- **酒店运维。** 已获运城恒泽大酒店批准,并投入酒店 IT 运维使用。 +- **作品集演示。** 使用虚构酒店数据,在本地前端运行,无需 Token。 +- **开源实现。** 认证工作区、任务调度、Agent 和证据处理使用 MIT 许可证。 -## 项目缘起 +![模拟酒店看板,展示端点趋势和业务服务状态](docs/assets/screenshots/hotel-dashboard.jpg) -当一台办公电脑能访问服务而前台设备却连不上时,工单通常无法说明问题出在 DNS、TCP 连接、TLS,还是应用本身。我开发 NetGuardian,是为了让这些排查步骤可以重复执行,也让接手的人能看到判断依据。 +*Aster Quay Hotel 是虚构酒店,看板数据为模拟数据。* -项目从一个 Python 和 Streamlit 工具发展而来,由我独立开发,设计受到 LAN/WLAN、TCP/IP、DNS、DHCP 和 Windows 终端支持工作的启发。本文演示使用受控测试环境。 +## 为什么开发这个项目 -## 核心能力 +支持酒店 LAN/WLAN、Windows 工作站、前台系统和 POS 连接时,我发现 DNS、TCP、TLS 和 HTTP 检查分散在不同工具里,往往需要人工重复执行,结果也难以统一归档和交接。我用 NetGuardian 将授权探测、定时观测、事件记录和证据报告串在一起。项目从 Streamlit 原型逐步演进为由 FastAPI 控制面、Next.js 工作区、持久化 Worker 和出站 Windows Agent 组成的平台。 -| 能力 | 可以查看的内容 | -| --- | --- | -| 分层诊断 | DNS 记录、TCP 连通性、TLS 握手和证书详情,以及 HTTP HEAD 状态和耗时。可按需启用 ICMP 和 traceroute。 | -| 定时监控 | 暂停和恢复检查,查看最近采集时间、故障及恢复记录。缺少测量时,界面会显示数据缺口。 | -| 双位置对比 | 将服务端检查与已注册的 Windows Agent 检查配对,结合测量端点和时间比较记录。 | -| 事件证据 | 打开脱敏报告快照,验证 SHA-256 摘要,并追溯报告引用的原始观测。 | -| 可选辅助分析 | 配置 Ollama 后可整理报告证据。界面分别展示经过证据校验的结论和待验证的假设。 | +## 运城恒泽大酒店 IT 运维应用 -## 系统架构 +NetGuardian 已获酒店批准并投入运维使用。我担任 IT Infrastructure & Network Engineer 期间负责 LAN/WLAN 维护、TCP/IP、DNS 和 DHCP 故障排查、Windows 技术支持、前台及 POS 连通性支持,以及 Python 自动化。生产网络数据、实际部署范围及酒店使用的具体模块不公开。设备数量、可用率、平均恢复时间(MTTR)和效率提升数据均未公开。CI 和集成测试不验证酒店生产环境的性能。公开演示与真实环境隔离,使用虚构的 Aster Quay Hotel。 -```mermaid -flowchart LR - Operator["操作人员"] --> Web["Next.js 控制台"] - Web --> API["FastAPI 控制接口"] - API --> DB[("SQLite / PostgreSQL")] - Worker["后台 Worker"] --> DB - Worker --> Engine["共用探测引擎"] - Agent["仅主动出站的 Windows Agent"] -->|"通过 HTTPS 领取任务并上报结果"| API - Agent --> Engine - Engine --> Targets["已授权的目标"] -``` +## 截图 -- [Web 控制台](web/)与 [FastAPI 接口](service/api.py)负责身份验证、作业提交和工作区数据查询。 -- [Worker](service/worker.py)领取有租约的后台作业并保存观测结果。[Windows Agent](agent/)通过 HTTPS 轮询任务,并独立执行本地目标策略。 -- [探测引擎](probe_core/)在规定的资源和时间范围内执行检查。[Alembic 迁移](migrations/)管理数据库结构,[报告导出](service/report_files.py)校验已保存快照的完整性。 +首页和本节的截图来自实际运行的演示页面,数据均为模拟。 -[系统设计](docs/architecture/SYSTEM_DESIGN.md) · [数据模型](docs/architecture/DATA_MODEL.md) · [威胁模型](docs/security/THREAT_MODEL.md) +![模拟拓扑,展示选中的 POS 节点和事件时间线](docs/assets/screenshots/hotel-topology.jpg) -## 工程实现 +![模拟 POS HTTP 503 的 DNS、TCP、TLS 和 HTTP 分层诊断](docs/assets/screenshots/hotel-diagnostics.jpg) -- **默认拒绝探测。** 服务端只允许预先批准的精确源地址。连接使用校验后的 IP,同时保留 HTTP 和 TLS 所需的主机名。HTTP 探测使用 HEAD,限制响应头大小,并禁止跟随重定向。 -- **分开记录各层耗时。** DNS、TCP、TLS 和 HTTP 保留各自的时间字段。ICMP 往返时延单独记录。 -- **异步执行与重试。** API 接收作业后,由 Worker 或 Agent 执行。作业使用租约与过期机制,Agent 在本地暂存结果,可重新投递已完成的观测。 -- **保留故障依据。** 报告记录当时引用的观测快照,导出后可用 [离线校验脚本](scripts/verify_report.py)核验 SHA-256 摘要。 +[更多截图,包括认证工作区和证据报告](docs/product/DEMO_GUIDE.md) + +## 端到端运维流程 + +```mermaid +flowchart LR + A["配置与授权 · 运维人员"] --> B["监控 · 调度与探测"] + B --> C["检测 · 阈值与事件"] + C --> D["调查 · 运维人员查看证据"] + D --> E["报告 · 运维人员生成快照"] + E --> F["复盘 · 核验与交接"] +``` -[目标策略与测量说明](docs/operations/LOCAL_DEVELOPMENT.md#target-policy-and-measurement-semantics) +演示中的 POS 端点返回 HTTP 503,此时探测已完成,采集依然新鲜。运维人员可以逐层查看观测和可能原因。独立的本地 HTTPS 集成场景还验证了事件生成、HTTP 204 恢复及报告快照。真实工作区自动调度探测并按阈值更新事件。目标授权、原因判断、报告生成请求和后续处置由运维人员负责。恢复记录表明观测结果发生变化,根因仍需调查。 -## 演示证据 +## 核心功能 -| 场景 | 验证记录 | +| 功能 | 实现与验证 | | --- | --- | -| 服务从 HTTP 503 恢复为 HTTP 204 | 受控的回环 HTTPS 服务真实返回两个状态码。本地浏览器验证了监控事件恢复及报告校验。[浏览器记录](docs/product/BROWSER_QA.md#populated-workflow-check-2026-11-05) | -| 下载事件报告 | 使用隔离测试工作区下载报告,并执行离线摘要校验。[导出验证记录](docs/product/BROWSER_QA.md#real-model-investigation-and-report-export-2026-11-06) | -| 查看缺少测量的历史记录 | 使用 101 条合成失败作业测试分页及缺失测量提示。[夹具验证记录](docs/product/BROWSER_QA.md#retained-history-production-web-build-2026-11-05) | +| 网络诊断 | DNS、TCP、TLS 和 HTTP HEAD。ICMP、Traceroute 需显式启用,使用受限系统命令并显示能力错误。HTTP 耗时与 ICMP RTT 分开。 | +| 定时监控与事件 | 持久化调度、采集新鲜度、故障及恢复阈值、事件证据和缺测状态。Worker 与本地集成测试覆盖相关流程。 | +| Windows Agent 与对比 | 注册、出站 HTTPS、本地独立策略,以及满足兼容条件时的多位置观测对比。受控集成环境已验证。 | +| 酒店看板与拓扑 | 真实工作区趋势读取已认证 API。演示用 React Flow 展示预先配置的逻辑连接。 | +| 证据报告 | 保存不可变快照并校验 SHA-256。摘要可发现内容变动,但不认证作者身份或测量真实性。 | +| 可选 Ollama 调查 | 校验引用的已记录事实,并单独标记假设。解释能力属于实验功能,真实模型验证范围有限。 | +| 安全控制 | 默认拒绝的来源策略、DNS 与地址验证、IP 固定、工作区数据隔离及 Agent 本地策略。 | -## 技术栈 +## 架构 -Python 3.11 · FastAPI · SQLAlchemy · Alembic · SQLite / PostgreSQL · Next.js · React · TypeScript · Node.js 24 · pnpm · pytest · Ruff · GitHub Actions +```mermaid +flowchart LR + subgraph Browser + D["Public demo · synthetic fixture"] + U["Authenticated workspace"] + end + subgraph Control["Trusted control plane"] + W["Next.js authenticated proxy"] --> API["FastAPI · workspace and target policy"] + API --> DB[("SQLite / PostgreSQL · jobs and evidence")] + R["Worker · schedules and leases"] <--> DB + end + U -->|"Bearer token"| W + AG["Windows Agent · local policy"] -->|"Outbound HTTPS · poll and submit"| API + R -->|"Approved cloud targets"| T["Network targets"] + AG -->|"Locally approved targets"| T +``` -## 快速开始 +API 持久化任务,Worker 或已注册 Agent 异步执行。各探测位置使用版本化测量格式,数据库关联事件与源任务证据。认证工作区通过限定端点的 Next.js 代理访问 API,演示页只读取独立的模拟数据。[系统设计](docs/architecture/SYSTEM_DESIGN.md) · [数据模型](docs/architecture/DATA_MODEL.md) -在 Windows 上安装 Python 3.11 和 Node.js 24,然后在 PowerShell 中执行: +## 演示快速开始 + +需要 Node.js 24 与 pnpm 11.25.0。在可创建项目目录的位置打开 PowerShell: ```powershell git clone https://github.com/TYF009/NetGuardian.git Set-Location NetGuardian -python -m pip install uv==0.13.7 npm install -g pnpm@11.25.0 -uv sync --locked pnpm --dir web install --frozen-lockfile pnpm --dir web build +pnpm --dir web start +``` + +打开[本地演示](http://127.0.0.1:3000/),也可访问 `/demo`。演示无需 Python、数据库、Agent、后端或 Token,不发起真实网络探测。[演示指南](docs/product/DEMO_GUIDE.md) + +## 完整本地部署 +需要 Windows、Python 3.11、uv 0.13.7、Node.js 24 与 pnpm 11.25.0。按前述命令构建前端,再用 Ctrl+C 停止演示进程,释放端口 3000。回到仓库根目录安装 Python 依赖并完成一次性初始化。把显示的本地管理员 Token 保存到密码管理器。探测位置 ID 文件保存在 Git 忽略的本地运行目录: + +```powershell +python -m pip install uv==0.13.7 +uv sync --locked $workspaceToken = (uv run --locked python -c "import secrets; print(secrets.token_urlsafe(32))").Trim() +$workspaceToken +New-Item -ItemType Directory -Path .netguardian-local -Force | Out-Null +$vantageFile = '.netguardian-local/workspace-vantage-id.txt' +if (-not (Test-Path -LiteralPath $vantageFile)) { + [guid]::NewGuid().ToString() | Set-Content -LiteralPath $vantageFile -Encoding ascii +} +``` + +首次启动和以后每次打开新的 PowerShell 窗口时,加载同一个本地 Token 和已保存的 Worker 探测位置 ID。空白名单适合不发起探测的试运行。使用 `--migrate` 前请备份已有数据库: + +```powershell +$workspaceToken = [pscredential]::new( + 'local', (Read-Host 'Local workspace token' -AsSecureString) +).GetNetworkCredential().Password $env:NETGUARDIAN_TOKENS = @{ $workspaceToken = 'local' } | ConvertTo-Json -Compress $env:NETGUARDIAN_ORIGINS = '{}' -$env:NETGUARDIAN_VANTAGE_ID = [guid]::NewGuid().ToString() -$workspaceToken +$env:NETGUARDIAN_VANTAGE_ID = (Get-Content -LiteralPath '.netguardian-local/workspace-vantage-id.txt' -Raw).Trim() uv run --locked python -m scripts.run_windows --migrate ``` -打开[本地工作区](http://127.0.0.1:3000/workspace),输入 PowerShell 显示的测试工作区令牌。目标白名单为空时可以浏览界面。请使用全新检出目录,或在迁移前备份现有数据库。如果要跨会话保留观测并进行比较,请保持 Worker 位置标识稳定。 +打开[认证工作区](http://127.0.0.1:3000/workspace)并输入生成的本地 Token。重启时继续使用已保存的 ID。该 ID 只标识此 Worker 的观测位置,不授予访问权限。生产位置 ID 应与演示及测试环境分开。指定 `--migrate` 时,启动器会先执行 Alembic 迁移,再启动服务。数据库结构已是最新版本时,日常重启可省略此选项。API、Worker 和 Web 均在回环地址启动。[独立进程命令、数据库配置、Agent 注册及排障](docs/operations/LOCAL_DEVELOPMENT.md) · [运维手册](docs/operations/RUNBOOK.md) -[本地开发与 Agent 注册指南](docs/operations/LOCAL_DEVELOPMENT.md) · [就绪检查、备份与故障排查手册](docs/operations/RUNBOOK.md) +## 首次诊断演练 -## 测试与验证 +### 公开演示 + +1. 按[演示快速开始](#演示快速开始)安装并启动前端,打开虚构的 Aster Quay Hotel 看板,确认页面有 **DEMO / SIMULATED** 标识。 +2. 打开 **Services**,比较 POS 的探测结果、端点响应和采集新鲜度。模拟探测已完成,并记录到 HTTP 503。 +3. 打开 **Topology**,选中 **POS terminals**,沿 **DEMO-104 · Open** 进入事件,点击 **Inspect evidence**。 +4. 查看 DNS、TCP、TLS 和 HTTP 四层证据,展开 **Raw probe evidence (JSON)** 查看模拟记录。可能原因、下一步检查和已观测事实分别展示。 + +### 认证工作区 + +1. 按[完整本地部署](#完整本地部署)安装服务并生成本地管理员 Token。使用独立的本地或测试工作区,与生产数据隔离。 +2. 选择你拥有或明确获准监测、且云端 Worker 的地址策略允许访问的精确 HTTPS 来源。下面是占位值,运行前请替换。先停止启动器,在启动服务的 PowerShell 窗口设置白名单。数据库结构已更新时,无需迁移即可重启。 -GitHub Actions 在 Windows 和 Ubuntu 上运行 Python 检查及选定的运行时测试,并验证 PostgreSQL 集成、Web 测试、TypeScript、OpenAPI 一致性和 Next.js 生产构建。本地 Windows 验证范围涵盖回环 TLS、Agent 作业、数据库及浏览器工作流。[CI 工作流](.github/workflows/ci.yml) · [Windows 交付记录](docs/product/WINDOWS_DELIVERY.md) + ```powershell + $approvedOrigin = 'https://authorized-host.example:443' + $env:NETGUARDIAN_ORIGINS = @{ local = @($approvedOrigin) } | ConvertTo-Json -Compress + uv run --locked python -m scripts.run_windows + ``` -可在仓库根目录复现: +3. 打开 `/workspace`,输入本地 Token 并点击 **Connect**。在 **Diagnostics** 选择授权目标、探测类型(例如 **HTTP**)和 **Cloud worker**,点击 **Run diagnostic**。在 **Diagnostic history** 等任务结束后,点击 **Inspect** 查看记录。如果没有返回测量结果,检查 Worker 和任务状态。探测已完成也可能收到 HTTP 错误响应。 +4. 在 **Monitors** 选择授权目标、探测类型和 **Cloud worker**,输入名称,设置至少 60 秒的间隔,再点击 **Create monitor**。有定时运行记录后,点击 **View history**。 +5. 达到事件阈值后,可在 **Incidents** 查看事件。界面使用默认阈值:连续三次失败开启事件,连续两次成功恢复事件。已有事件时可点击 **Create evidence report**,并在 **Reports** 查看快照。健康、新建或缺少观测的监控可能没有事件可生成报告。 + +## 测试与验证 ```sh uv run --locked ruff check . uv run --locked ruff format --check . uv run --locked pytest -q -uv run --locked pytest integration -q -rs +uv run --locked pytest integration/test_tls.py integration/test_dns_runtime.py integration/test_agent_workflow.py -q pnpm --dir web test pnpm --dir web typecheck pnpm --dir web build ``` -[集成测试所需环境](docs/operations/LOCAL_DEVELOPMENT.md#checks-and-operations) - -## 当前限制 - -- 当前交付范围为 Windows 本地评估。Docker Compose 部署和更完整的 Linux 运行验证仍待完成。 -- 回环测试覆盖了 Worker 和 Agent 工作流。跨主机部署、Agent 签名分发和持续多副本负载仍待验证。 -- Ollama 辅助分析为可选实验功能,目前只验证过有限的本地场景。 +[GitHub Actions](https://github.com/TYF009/NetGuardian/actions/workflows/ci.yml) 覆盖 Windows、Ubuntu、PostgreSQL 集成测试、OpenAPI 漂移检查和 Web 构建。部分集成测试需单独配置可丢弃的 PostgreSQL 数据库、模型或系统凭据库。[验证记录](docs/product/VERIFICATION.md) · [浏览器验收](docs/product/BROWSER_QA.md) · [Windows 交付范围](docs/product/WINDOWS_DELIVERY.md) -[交付状态与验证边界](docs/product/DELIVERY_STATUS.md) +CI 与测试记录描述受控环境中的行为。酒店获批使用属于独立的运维事实。当前本地交付面向 Windows,Compose 配置仍需验证后才适用于跨主机或公开部署。 -## 路线图 +## 安全与隐私 -- 发布只读故障与恢复演示,注明数据来源并提供可校验报告。 -- 在干净环境中验证 Compose 启动和数据库迁移,记录支持范围。 -- 在独立测试主机之间验证 Agent 注册、重连和双位置测量。 +仅探测获授权的目标。控制面校验精确来源和工作区范围。云端探测拒绝私有与特殊地址,Agent 独立执行来源策略,并在支持时限制在明确允许的私网 CIDR 内。浏览器只在内存中保留 Token,Agent 使用系统凭据库。生产拓扑、地址、日志和凭据保持私有,公开演示只使用虚构数据。[威胁模型](docs/security/THREAT_MODEL.md) · [许可证与第三方资产](NOTICE.md) -## 作者与许可 +## 路线图与作者 -我拥有通信工程背景和华为 HCIP Datacom 认证,并有 400G 光传送网络以及 IT 基础设施运维经验。NetGuardian 将这些排查经验与 Python 自动化和 Web 证据工作流结合起来。 +后续优先完善私有资产及区域配置、设备运行指标、监控与拓扑关联、证据分析,以及更全面的部署和运维验证。 -项目代码和文档采用 [MIT 许可证](LICENSE)。第三方组件和历史素材的授权范围见 [NOTICE.md](NOTICE.md)。 +**TYF009** · 通信工程背景 · HCIP Datacom · 酒店 IT 基础设施与技术支持 · 网络自动化。 diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..e571b38 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,11 @@ +# NetGuardian documentation + +| Start here | Documentation | +| --- | --- | +| Project and quick start | [English README](../README.md) · [简体中文 README](../README.zh-CN.md) | +| Portfolio demo | [Walkthrough and screenshot gallery](product/DEMO_GUIDE.md) · [Hospitality workspace and data semantics](product/HOSPITALITY_OPERATIONS.md) | +| Deployment and operations | [Local setup, environment and Agent enrollment](operations/LOCAL_DEVELOPMENT.md) · [Runbook](operations/RUNBOOK.md) | +| Architecture | [System design](architecture/SYSTEM_DESIGN.md) · [Data model](architecture/DATA_MODEL.md) | +| Security | [Threat model](security/THREAT_MODEL.md) · [License and third-party assets](../NOTICE.md) | +| Verification | [Acceptance matrix](product/ACCEPTANCE_MATRIX.md) · [Verification](product/VERIFICATION.md) · [Browser QA](product/BROWSER_QA.md) · [Windows delivery](product/WINDOWS_DELIVERY.md) | +| Planning | [PRD](product/PRD.md) · [Implementation plan](product/IMPLEMENTATION_PLAN.md) · [Delivery status](product/DELIVERY_STATUS.md) | diff --git a/docs/assets/screenshots/hotel-dashboard.jpg b/docs/assets/screenshots/hotel-dashboard.jpg new file mode 100644 index 0000000..892dbd3 Binary files /dev/null and b/docs/assets/screenshots/hotel-dashboard.jpg differ diff --git a/docs/assets/screenshots/hotel-diagnostics.jpg b/docs/assets/screenshots/hotel-diagnostics.jpg new file mode 100644 index 0000000..3375141 Binary files /dev/null and b/docs/assets/screenshots/hotel-diagnostics.jpg differ diff --git a/docs/assets/screenshots/hotel-mobile.jpg b/docs/assets/screenshots/hotel-mobile.jpg new file mode 100644 index 0000000..c75ae29 Binary files /dev/null and b/docs/assets/screenshots/hotel-mobile.jpg differ diff --git a/docs/assets/screenshots/hotel-topology.jpg b/docs/assets/screenshots/hotel-topology.jpg new file mode 100644 index 0000000..0fab2f7 Binary files /dev/null and b/docs/assets/screenshots/hotel-topology.jpg differ diff --git a/docs/assets/screenshots/test-evidence-report.jpg b/docs/assets/screenshots/test-evidence-report.jpg new file mode 100644 index 0000000..4b951d2 Binary files /dev/null and b/docs/assets/screenshots/test-evidence-report.jpg differ diff --git a/docs/assets/screenshots/test-workspace.jpg b/docs/assets/screenshots/test-workspace.jpg new file mode 100644 index 0000000..775788a Binary files /dev/null and b/docs/assets/screenshots/test-workspace.jpg differ diff --git a/docs/product/DEMO_GUIDE.md b/docs/product/DEMO_GUIDE.md new file mode 100644 index 0000000..9b610f1 --- /dev/null +++ b/docs/product/DEMO_GUIDE.md @@ -0,0 +1,38 @@ +# Hospitality demo walkthrough + +Use the [frontend quick start](../../README.md#demo-quick-start) and open the +local home page. Aster Quay Hotel is fictional, and the page marks its +observations as **DEMO / SIMULATED**. The fixture never connects to the control +API or dispatches jobs. + +1. In **Overview**, inspect an endpoint trend bar to see its recorded status. + A gap means no data. An HTTP 503 is an observed response. +2. In **Services**, compare the POS probe outcome, HTTP endpoint health and + collection freshness. A HEAD result cannot measure an entire POS transaction. +3. In **Topology**, select **POS terminals** to view the simulated incident link. + Edges represent configured logical links. +4. Open **Incidents**, then **Inspect evidence**. The troubleshooting view shows + observed DNS, TCP, TLS and HTTP layers, possible explanations, suggested + checks, and raw simulated evidence. + +## Screenshot gallery + +The first four images were captured from the running frontend with the demo +fixture. The last two come from the authenticated frontend with a disposable +SQLite database from the loopback HTTPS integration test. The database +contains test-only records for `netguardian.test`. The report reflects actual +test fixture responses (HTTP 503 and 204). All captures exclude credentials +and hotel operational data. + +| Capture | Environment | +| --- | --- | +| [Dashboard and Business Service Health](../assets/screenshots/hotel-dashboard.jpg) | Simulated demo, desktop | +| [Topology and incident timeline](../assets/screenshots/hotel-topology.jpg) | Simulated demo, desktop | +| [POS troubleshooting](../assets/screenshots/hotel-diagnostics.jpg) | Simulated demo, desktop | +| [Mobile overview](../assets/screenshots/hotel-mobile.jpg) | Simulated demo, dark theme | +| [Authenticated engineer workspace](../assets/screenshots/test-workspace.jpg) | Local HTTPS test records | +| [Evidence report](../assets/screenshots/test-evidence-report.jpg) | Local HTTPS test records, verified snapshot checksum | + +The workspace requires the [full local services](../operations/LOCAL_DEVELOPMENT.md). +Generate a separate local token, start with no allowed origins, and access +`/workspace`. The empty state is expected for a new workspace.