Repository navigation
fix: harden HTTP limits, redirects, ArcGIS paging, and parsers - #47
Merged
Merged
Conversation
…anent failures Fixes #45. CURLOPT_MAXFILESIZE_LARGE compared every Content-Length in the redirect chain, and encoded lengths, against the decoded-body limit. The write callback only sees the final body, so the early refusal moves there. Malformed URLs, disallowed schemes, and redirect loops were NetworkError, which with_retry repeats. Retry-After values past int64 were ignored, and statuses above 599 were ServerError.
…ferences Found in the audit: conditional-intensity and day 1-2 fire-weather layers kept features with null or missing fields, numeric strings "nan" and "inf" reached public doubles, and a null latestWkid or geometry spatialReference hid a valid reference. The locale test now drops dn so it can fail.
Glaze came in through glaze_SOURCE_DIR, which is empty when CMake finds an installed Glaze, so that configuration failed. Linking the target behind BUILD_LOCAL_INTERFACE keeps it out of the export set and brings Glaze's MSVC options; its headers stay system headers. Also test the MapServer and static-feed URL of each typed query, skip the benchmark self-check's library probe on MSVC, and fix stale docs.
Private vulnerability reporting was off while SECURITY.md and the issue templates sent reporters to it; it is now enabled and the docs say how it works. GoogleTest and Benchmark are pinned by commit, the Makefile's markdownlint-cli2 matches the CI action, and .gitignore covers more secret files.
…gs literally The explicit-type audit only saw auto at line start and treated any iterator-looking substring as an exemption; it now checks each auto token and exempts only an initializer that is one iterator call. The fixture checker recurses and rejects duplicate entries, release notes match the tag literally, and the tools run on Python 3.9.
An https request followed a redirect to plain http and returned that body as a normal response. Redirects from https now allow only https, and the refusal is InvalidRequest. A TLS loopback server (optional OpenSSL 3, test only) covers it; Linux CI installs libssl-dev so those tests run. libcurl 7.85's curl/header.h lacks extern "C", so curl_easy_nextheader never linked there; 7.86 is the real floor.
…0 bodies Each page got a fresh Web Mercator expansion budget and its own response cap, so a server that kept paging could make one call hold gigabytes. One budget now covers the call, both fire-weather layers included, and the pages together may not exceed max_response_bytes. ArcGIS 429 and 5xx errors sent with HTTP 200 are retried like the same status, one page at a time; a missing code is no longer reported as 400; and a std::exception such as bad_alloc in any client call is a ParseError instead of escaping.
From independent review. libcurl fetches https:/host as absolute HTTPS, but is_absolute_url wanted two slashes, so with a base_url the request went to the plaintext base. Error bodies over 64 KiB are no longer parsed into a JSON tree, and golden coverage now walks fixture subdirectories like the checksum tool does.
clang-tidy's bugprone-implicit-widening-of-multiplication-result flagged 64 * 1024 being computed in int.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes from a full audit of the repo, for v0.4.8. Fixes #45.
max_response_bytesapplies to the final response's decoded body, not to redirect hops or compressed lengths (HttpClientrejects a request when a redirect hop'sContent-Lengthexceedsmax_response_bytes, even though the final response body is tiny #45).https:/hostcount as absolute.NetworkError, so they are not retried.curl/header.hlacksextern "C", so we never linked against it.std::exceptionsuch asbad_allocin any client call isParseErrorinstead of escaping."nan"/"inf"strings read as 0.latestWkidfalls back towkid.Retry-Afterpast int64 is capped.ServerError.libssl-devso the new TLS redirect tests run.auto.Public API changes:
std::exception, which only a custom transport can throw, still propagate.Testing
make test,make lint,make lint-md,make fixtures-check, andmake test-consumerspassCHANGELOG.mdhas an[Unreleased]entry for user-visible changesWhere it ran:
#45: reproduced on libcurl 8.5.0, 8.7.1, 8.18 and 8.23 before the fix, with new loopback tests.
Live:
tools/verify_arcgis_metadata.py(39 layers) and every example pass on both machines.Review: an independent review found three issues, fixed in the last commit, and the re-review approved.