Skip to content

fix: harden parsers, pagination, and HTTP - #19

Merged
Reddimus merged 2 commits into
mainfrom
fix/sdk-hardening-20261025
Oct 26, 2026
Merged

Reddimus merged 2 commits into
mainfrom
fix/sdk-hardening-20261025

Conversation

@Reddimus

Copy link
Copy Markdown
Owner

Fix complete JSON reads, one-based CDO pagination, date/range validation, CSV and climate parsing, cache rollback, metadata field mapping, and retry quota accounting. Bound libcurl responses and callbacks, preserve parent compiler flags and custom hooks, and add portable HTTP and installed/FetchContent consumer regressions. Prepare v0.4.0 with CMake 3.31, libcurl 7.85+, and Glaze 9.

Validation: 199 native and 199 ASan/UBSan CTest entries, formatting/type audit, Markdown, hook cases, offline benchmark, installed/FetchContent consumers and both README examples passed. Live Access Service CSV/JSON data agreed and dataset metadata returned its field list. Authenticated CDO calls were not run without a token.

@greptile-apps

greptile-apps Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 4/5

[High impact] Pagination, HTTP bounds, and parser hardening across the SDK.

The PR appears safe to merge, with a non-blocking numeric parsing fix recommended.

Findings

  1. P2 Double signs become valid numbers ▶

Summary

This PR hardens parsers, CDO pagination, retry quotas, and HTTP requests for version 0.4.0.

  • CDO data requests start at the first record and account for each retry.
  • NCEI API requests keep parameter values and IDs inside their URL parts.
  • HTTP requests return bounded responses or clear transport errors.
  • NCEI parsers accept complete valid data and preserve output on failure.
  • Cache updates keep the LRU list consistent when an insertion fails.
  • Dataset metadata includes data types, units, and scale factors.
  • The v0.4.0 release builds cleanly for installed and embedded consumers.

Diagram

sequenceDiagram
    participant App
    participant CDOClient
    participant RateLimiter
    participant HttpClient
    App->>CDOClient: Request all records
    loop Each date chunk and page
        loop Each request attempt
            CDOClient->>RateLimiter: Acquire pacing token and daily quota
            RateLimiter-->>CDOClient: Allow or reject
            CDOClient->>HttpClient: Send encoded request
            HttpClient-->>CDOClient: Return bounded response
        end
        CDOClient->>CDOClient: Read JSON and advance one-based offset
    end
    CDOClient-->>App: Return records or error
Loading

Reviews (1) · Last reviewed commit: "fix: harden parsers, pagination, and HTT..." · Reviewed by Greptile

Comment on lines +164 to +165
if (text.starts_with('+')) {
text.remove_prefix(1);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Double signs become valid numbers

DataPoint::get_double removes a leading + before calling std::from_chars, so malformed text such as +-1.2 becomes a valid -1.2. The climate reader in src/models/climate_index.cpp does the same. Both readers can return a believable negative measurement instead of rejecting the cell. After removing +, reject another sign and add regression cases for both readers.

@Reddimus
Reddimus merged commit 1e70a54 into main Oct 26, 2026
4 checks passed
@Reddimus
Reddimus deleted the fix/sdk-hardening-20261025 branch October 26, 2026 02:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant