Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .coderabbit.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,8 @@ knowledge_base:
code_guidelines:
enabled: true
filePatterns:
- AGENTS.md
- CLAUDE.md
- CONTRIBUTING.md
- .github/review-policy.md
chat:
auto_reply: false
art: false
8 changes: 4 additions & 4 deletions .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@ name: Bug report
description: A crash, a wrong result, or a build failure.
labels: [bug]
body:
- type: markdown
attributes:
value: |
Report security vulnerabilities through GitHub's private vulnerability reporting, as described in the [security policy](https://github.com/Reddimus/spc-cpp/security/policy), not here.
- type: textarea
id: what
attributes:
Expand Down Expand Up @@ -31,7 +35,3 @@ body:
- libcurl: 8.5.0
validations:
required: true
- type: markdown
attributes:
value: |
Report security vulnerabilities privately, as described in the [security policy](https://github.com/Reddimus/spc-cpp/security/policy).
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@ blank_issues_enabled: false
contact_links:
- name: Security vulnerability
url: https://github.com/Reddimus/spc-cpp/security/policy
about: Report it privately instead of opening an issue.
about: Use GitHub's private vulnerability reporting, as the security policy describes, not an issue.
3 changes: 2 additions & 1 deletion .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@

<!-- The commands you ran and anything you checked by hand. -->

- [ ] `make test` and `make lint` pass
- [ ] `make test`, `make lint`, `make lint-md`, `make fixtures-check`, and
`make test-consumers` pass
- [ ] `CHANGELOG.md` has an `[Unreleased]` entry for user-visible changes
- [ ] Public API changes are called out above
12 changes: 7 additions & 5 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ jobs:
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
build-essential clang-format libcurl4-openssl-dev locales
build-essential clang-format libcurl4-openssl-dev libssl-dev locales
# The locale tests skip without a comma-decimal locale.
sudo locale-gen de_DE.UTF-8
- name: Build
Expand Down Expand Up @@ -56,7 +56,7 @@ jobs:
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
build-essential clang-18 libc++-18-dev libc++abi-18-dev \
libcurl4-openssl-dev locales
libcurl4-openssl-dev libssl-dev locales
sudo locale-gen de_DE.UTF-8
- name: Build
run: >-
Expand All @@ -75,7 +75,7 @@ jobs:
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
build-essential clang-19 libcurl4-openssl-dev
build-essential clang-19 libcurl4-openssl-dev libssl-dev
- name: Build
run: >-
make build CMAKE_ARGS="-DCMAKE_CXX_COMPILER=clang++-19
Expand Down Expand Up @@ -197,6 +197,8 @@ jobs:
timeout-minutes: 10
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
# The Makefile pins the markdownlint-cli2 version this action bundles.
# Update MARKDOWNLINT there when this pin changes.
- uses: DavidAnson/markdownlint-cli2-action@696eb3f2509270d7b882d2aa68347bc676e7af34 # v25.0.0

sanitizer:
Expand All @@ -208,7 +210,7 @@ jobs:
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
build-essential libcurl4-openssl-dev locales
build-essential libcurl4-openssl-dev libssl-dev locales
sudo locale-gen de_DE.UTF-8
- name: Build with AddressSanitizer and UndefinedBehaviorSanitizer
run: >-
Expand All @@ -228,7 +230,7 @@ jobs:
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
build-essential libcurl4-openssl-dev locales
build-essential libcurl4-openssl-dev libssl-dev locales
sudo locale-gen de_DE.UTF-8
- name: Build with ThreadSanitizer
run: >-
Expand Down
5 changes: 3 additions & 2 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,11 +53,12 @@ jobs:

- name: Extract the release notes
# Write to a file: echoing the notes through the shell would run their
# `code` spans as command substitutions.
# `code` spans as command substitutions. index() matches the heading
# literally; a regex would read the version's "." and "+" as operators.
run: |
version="${GITHUB_REF_NAME#v}"
awk -v v="$version" '
$0 ~ "^## \\[" v "\\]" { capture = 1; next }
index($0, "## [" v "]") == 1 { capture = 1; next }
capture && /^## / { exit }
capture && /^\[[^]]+\]: / { exit }
capture { print }
Expand Down
14 changes: 14 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,24 @@ Thumbs.db

# Secrets
.env
.env.*
!.env.example
.envrc
.netrc
.npmrc
.pypirc
*.pem
*.key
*.p12
*.pfx
id_rsa
id_ecdsa
id_ecdsa_sk
id_ed25519
id_ed25519_sk
credentials*.json
service-account*.json
token.json
secrets/
**/.claude/settings.local.json
.claude/worktrees/
46 changes: 46 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,52 @@ uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Security

- An HTTPS request no longer follows a redirect to plain HTTP; it fails with
`InvalidRequest`.
- A one-slash URL such as `https:/host/x`, which libcurl fetches as absolute, is
no longer appended to `ClientConfig::base_url` and sent to that host instead.
- Bound memory across all pages of an ArcGIS call. One Web Mercator expansion
budget covers the whole call, both fire-weather layers included, and the pages
together may not exceed `ClientConfig::max_response_bytes` (`InvalidRequest`).
Before, a paging server could make one call hold gigabytes.

### Changed

- libcurl 7.86 is the minimum. 7.85's `curl/header.h` lacks `extern "C"`, so the
library never linked against it.

- Report a malformed URL or a scheme other than HTTP(S) as `InvalidRequest`, and a
redirect loop as `ServerError`. `with_retry` no longer repeats them.
- Map only HTTP 500-599 to `ServerError`; a status above 599 is `Unknown`.
- Retry ArcGIS 429 and 5xx errors sent in HTTP 200 bodies like the same HTTP
status, one page at a time.
- An ArcGIS error without a usable code reports `http_status` 0 and
`arcgisCode=0` instead of a made-up 400.

### Fixed

- Apply `ClientConfig::max_response_bytes` to the final response's decoded body. A
redirect hop's `Content-Length`, or a compressed length over the limit, no longer
fails a request whose body fits.
- Cap a `Retry-After` too large for int64 at the longest wait instead of ignoring it.
- Reset the daily quota when the clock steps back across midnight.
- Initialize `HttpResponse::status_code` to 0.
- A `std::exception` inside any client call, `std::bad_alloc` included, returns
`ParseError` instead of escaping from ArcGIS calls.
- Error bodies over 64 KiB are not parsed for their message; the message is the
start of the body.
- Skip features whose fields are not an object, conditional-intensity polygons
without a label, and day 1-2 fire-weather polygons with no known `dn` band.
- Read the numeric strings `"nan"` and `"inf"` as 0 instead of passing them on.
- Fall back to `wkid` when `latestWkid` is null or not a number, and ignore a null
geometry `spatialReference`.
- Link Glaze through its target, so a build that finds an installed Glaze
(`FETCHCONTENT_TRY_FIND_PACKAGE_MODE`) configures and installs, and MSVC gets the
options Glaze requires.
- The benchmarks' allocation-counter self-check no longer fails on MSVC.

## [0.4.7] - 2026-11-04

### Fixed
Expand Down
6 changes: 3 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@ when you touch ArcGIS layer ids; it needs the network.
- Public failures return `Result<T>` (`std::expected<T, Error>`). The standalone
`parse_*` functions throw `std::runtime_error` on malformed JSON; the clients
turn that into `ErrorCode::ParseError`.
- Glaze is private. It lives behind `src/models/json.hpp`, and no header under
`include/` may include it; `tools/test_consumers.sh` fails if an install
ships Glaze headers.
- Glaze is private: only `src/` includes it, mostly through
`src/models/json.hpp`. No header under `include/` may include it;
`tools/test_consumers.sh` fails if an install ships Glaze headers.
- `HttpClient` is thread-safe through a pool of libcurl handles. Custom
transports implement `HttpTransport::get`, which must be safe to call
concurrently.
Expand Down
32 changes: 21 additions & 11 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -99,8 +99,9 @@ if(SPC_ENABLE_COVERAGE)
add_link_options(--coverage)
endif()

# CURLOPT_PROTOCOLS_STR, which keeps the transport on http and https, needs 7.85.
find_package(CURL 7.85 REQUIRED)
# 7.85 added CURLOPT_PROTOCOLS_STR, which keeps the transport on http and https,
# and 7.86 made curl/header.h's functions extern "C", so C++ can link them.
find_package(CURL 7.86 REQUIRED)
find_package(Threads REQUIRED)

include(FetchContent)
Expand All @@ -114,15 +115,22 @@ FetchContent_Declare(
GIT_SHALLOW TRUE
)
FetchContent_MakeAvailable(glaze)
# Glaze's headers are system headers wherever it is linked, so its warnings
# never reach -Werror or clang-tidy. An installed (imported) Glaze already is.
get_target_property(SPC_GLAZE_TARGET glaze::glaze ALIASED_TARGET)
if(SPC_GLAZE_TARGET)
set_property(TARGET ${SPC_GLAZE_TARGET} PROPERTY SYSTEM TRUE)
endif()

add_subdirectory(src)

# The test and benchmark frameworks are pinned by commit, so a moved tag cannot
# change what runs. GIT_SHALLOW does not work with a hash; both clones are small.
if(SPC_BUILD_TESTS)
FetchContent_Declare(
googletest
GIT_REPOSITORY https://github.com/google/googletest.git
GIT_TAG v1.18.0
GIT_SHALLOW TRUE
GIT_TAG 063de7e9578f82b369302001269680b4b1553359 # v1.18.0
)
set(gtest_force_shared_crt ON CACHE BOOL "" FORCE)
set(INSTALL_GTEST OFF CACHE BOOL "" FORCE)
Expand All @@ -135,8 +143,7 @@ if(SPC_BUILD_BENCHMARKS)
FetchContent_Declare(
benchmark
GIT_REPOSITORY https://github.com/google/benchmark.git
GIT_TAG v1.9.5
GIT_SHALLOW TRUE
GIT_TAG 192ef10025eb2c4cdd392bc502f0c852196baa48 # v1.9.5
SYSTEM
EXCLUDE_FROM_ALL
)
Expand All @@ -162,12 +169,15 @@ if(SPC_INSTALL)
INCLUDES DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}
)
install(DIRECTORY include/spc DESTINATION ${CMAKE_INSTALL_INCLUDEDIR})
# The static libraries contain compiled Glaze code, so ship its license.
install(FILES LICENSE DESTINATION ${CMAKE_INSTALL_DATADIR}/licenses/spc-cpp)
install(FILES ${glaze_SOURCE_DIR}/LICENSE
DESTINATION ${CMAKE_INSTALL_DATADIR}/licenses/spc-cpp
RENAME glaze-LICENSE
)
# The static libraries contain compiled Glaze code, so ship its license.
# A packaged Glaze, found instead of fetched, ships its own.
if(glaze_SOURCE_DIR)
install(FILES ${glaze_SOURCE_DIR}/LICENSE
DESTINATION ${CMAKE_INSTALL_DATADIR}/licenses/spc-cpp
RENAME glaze-LICENSE
)
endif()

configure_package_config_file(
${PROJECT_SOURCE_DIR}/cmake/spcConfig.cmake.in
Expand Down
25 changes: 19 additions & 6 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# Contributing

Thanks for helping. Report security problems privately through
[SECURITY.md](SECURITY.md), not in a public issue.
Thanks for helping. Report security problems through GitHub's private
vulnerability reporting, as [SECURITY.md](SECURITY.md) describes, not in a
public issue.

## Set up

Expand Down Expand Up @@ -90,12 +91,13 @@ make tidy CMAKE_ARGS="-DCMAKE_CXX_COMPILER=$llvm/bin/clang++ \
## Fixtures

`tests/fixtures/` holds captured NOAA and IEM responses. After adding or
changing one, update the checksums:
changing one, update the checksums. Files in subdirectories are fixtures too;
dotfiles are not:

```bash
for f in $(ls tests/fixtures | grep -v -e README.md -e SHA256SUMS); do
shasum -a 256 "tests/fixtures/$f"
done > tests/fixtures/SHA256SUMS
find tests/fixtures -type f ! -path '*/.*' ! -path tests/fixtures/README.md \
! -path tests/fixtures/SHA256SUMS -exec shasum -a 256 {} + |
LC_ALL=C sort -k 2 > tests/fixtures/SHA256SUMS
make fixtures-check
```

Expand Down Expand Up @@ -162,6 +164,17 @@ with an error.
the tests and consumer checks, then publishes the release with the
changelog section as its notes.

## Security reports

SECURITY.md and the issue templates send reporters to GitHub's private
vulnerability reporting. It is a repository setting, and with it off reporters
have no private channel. Keep it on (Settings, Advanced Security) and check it
after changing repository settings:

```bash
gh api repos/Reddimus/spc-cpp/private-vulnerability-reporting # {"enabled":true}
```

## Repository tools

`.mcp.json` (Claude) and `.codex/config.toml` (Codex) configure
Expand Down
7 changes: 5 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ BUILD_DIR := build
CMAKE := cmake
NPROC := $(shell nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 4)
SOURCES = find src include tests examples benchmarks \( -name '*.cpp' -o -name '*.hpp' \) -print0
# The version bundled by the markdownlint-cli2-action pin in ci.yml, so local
# and CI lint agree. Change both together.
MARKDOWNLINT := npx --yes markdownlint-cli2@0.24.0

.PHONY: all build debug configure configure-debug test test-consumers fixtures-check lint \
lint-md format format-md tidy bench coverage pre-commit install-hooks clean help
Expand Down Expand Up @@ -39,14 +42,14 @@ lint:
@python3 tools/test_cpp_auto_audit.py

lint-md:
npx markdownlint-cli2 "**/*.md"
$(MARKDOWNLINT) "**/*.md"

format:
@command -v clang-format >/dev/null || { echo "clang-format is not installed"; exit 1; }
@$(SOURCES) | xargs -0 clang-format -i

format-md:
npx markdownlint-cli2 --fix "**/*.md"
$(MARKDOWNLINT) --fix "**/*.md"

# Compile the libraries with clang-tidy. On macOS, see CONTRIBUTING.md.
tidy:
Expand Down
17 changes: 12 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ int main() {
- CMake 3.31 or newer. Ubuntu 24.04 ships 3.28, so run
`sudo snap install cmake --classic`.
- GCC 13+, Clang 18+ with libc++ or 19+ with libstdc++, or Apple Clang 17+.
- libcurl 7.85 or newer with its headers: `libcurl4-openssl-dev` on Ubuntu,
- libcurl 7.86 or newer with its headers: `libcurl4-openssl-dev` on Ubuntu,
built into macOS.

CI builds on Linux, macOS for Apple silicon and Intel, and Windows with MSVC.
Expand Down Expand Up @@ -133,12 +133,16 @@ problems. `Error::code` says what went wrong:
| `NetworkError` | Connection, TLS, or timeout failure, after retries. |
| `RateLimited` | HTTP 429 or 503 after retries, or `ArchiveClient`'s own limit would wait more than 5 s. |
| `ServerError` | HTTP 5xx other than 503, after retries, or an ArcGIS query that never finished paging. |
| `InvalidRequest` | An unsupported day or hazard, HTTP 400, an ArcGIS error such as a bad layer ID, or a response over 64 MiB. |
| `InvalidRequest` | An unsupported day or hazard, HTTP 400, an ArcGIS error such as a bad layer ID, a response over 64 MiB, or an ArcGIS query whose pages total more than that. |
| `NotFound` | A URL or ArcGIS service that no longer exists. Report it as a bug. |
| `ParseError` | Malformed JSON or Esri geometry that cannot be converted. |
| `ParseError` | Malformed JSON, Esri geometry that cannot be converted, or not enough memory to build the result. |
| `FeedUnavailable` | Only `StaticFeedClient` returns it. SPC has not issued the product, as with day 1 probabilities overnight. Treat it as no data. `ArcGISClient` returns an empty result instead. |
| `Unknown` | Any other HTTP status, such as 422 for a malformed `ArchiveClient` timestamp. `Error::http_status` holds it. |

ArcGIS reports its own failures with HTTP 200 and an error code in the body.
`ArcGISClient` retries a 429 or 5xx code as it would the same HTTP status,
repeating only the page that failed.

The offline `parse_*` functions, such as `spc::parse_categorical(body, day)`,
throw `std::runtime_error` on malformed JSON or Esri geometry that cannot be converted.
Coordinates are longitude/latitude in degrees. Esri Web Mercator (3857, 102100,
Expand All @@ -148,12 +152,15 @@ Projected edges are subdivided to preserve their boundaries within one metre.
Malformed Esri points and invalid projected coordinates, including discarded
holes, raise a parse error.
A payload needing more than one million added points also raises a parse error.
An `ArcGISClient` call counts every page and layer it reads as one payload.

## Settings and threads

`spc::ClientConfig` sets the user agent, the 15 s timeout, and the 64 MiB
response limit. The default user agent carries spc-cpp's contact address, so
set one with yours:
response limit. The server decides how many pages an ArcGIS query has, so that
limit also caps the pages of one `ArcGISClient` call together; with a transport
of your own the cap is 64 MiB. The default user agent carries spc-cpp's contact
address, so set one with yours:

```cpp
const spc::ArcGISClient client{spc::ClientConfig{.user_agent = "myapp/1.0 (you@example.com)"}};
Expand Down
Loading
Loading