Skip to content

Read structs with defaults that hold a Validator without a valid default - #12

Open
clockblocker wants to merge 5 commits into
fix-700-prereqsfrom
fix-700-deferred-defaults
Open

clockblocker wants to merge 5 commits into
fix-700-prereqsfrom
fix-700-deferred-defaults

Conversation

@clockblocker

@clockblocker clockblocker commented Nov 11, 2026 •

Copy link
Copy Markdown
Owner

Fixes munich-data-engineering#700 at the cost of the library's first thread_local.

The only other acceptable solution I found was to document the suggested workaround in the code and docs and leave it be.

AI-authored

Before and after

struct Person {
  rfl::DefaultVal<std::string> name = "Homer Simpson";
  rfl::Validator<std::vector<std::string>, rfl::Size<rfl::Minimum<1>>> favorite_foods;
};

rfl::json::read<Person>(R"({"favorite_foods": ["donuts", "pizza"]})");
main:        error  Size validation failed: Value expected to be greater than or equal to 1, but got 0.
this branch: ok     {"name":"Homer Simpson","favorite_foods":["donuts","pizza"]}

On main no input can be read into this struct. The parser builds Person{} before it looks at the input, and the empty favorite_foods fails its own validator.

A field without a valid default still has to come from the input:

rfl::json::read<Person, rfl::DefaultIfMissing>(R"({})");
main:        error  Size validation failed: Value expected to be greater than or equal to 1, but got 0.
this branch: error  Field named 'favorite_foods' not found, and its default value is invalid: Size validation failed: Value expected to be greater than or equal to 1, but got 0.

Outside a read, Person{} and Validator() throw as before.

Approaches

Defaults of missing fields exist only as default member initializers, which C++ evaluates only while constructing the whole struct. So the parser has to build T{} before it reads, and Validator() has to return normally during that. It takes no arguments, so the parser cannot tell it who is calling.

Approach Why not, or what it costs
The user writes a valid dummy default; docs and a clearer error (fork PR #11) The dummy is real data under DefaultIfMissing and in Person{}.
Validator() never validates its default Person{} in user code holds an invalid value without an error.
Validator can hold "no value": a Result<T> (upstream PR #737) or a flag Every Validator grows and gets accessors that can fail.
Validator() detects the parser with std::source_location (upstream PR #707) Depends on the caller's function name. A missing field keeps the unvalidated default under DefaultIfMissing and DefaultVal<Validator>.
Thread-local scope, then a walk over the type to find invalid defaults (fork PR #8) +926 lines in include/, 647 of them the walk, which broke compilation for some rfl::Skip fields.
A new wrapper for required fields Users change their structs, and the wrapper has an empty state.
Defaults carried in the type instead of = value A second way to declare defaults, limited to compile-time constants.
Thread-local scope plus address records (this PR) The library's first thread_local. +265 −30 lines in include/.

Only the last one reads the issue's struct with no change to user code, leaves Validator's layout and its behaviour outside a read alone, and still rejects a missing field whose default is invalid. It needs no type walk, because each Validator that keeps an invalid default reports its own address.

How it works

  1. read_struct_with_default builds T{} in the result's storage, as on main. If that succeeds, nothing else changes.
  2. If it throws, T{} is built again while a DeferredDefaults scope is active on this thread (RAII, restores the previous scope).
  3. Inside the scope, a Validator() whose default fails validation and whose address lies inside T's storage keeps T() and records its address and error. Every other Validator() (temporaries, the heap, other threads) throws as before.
  4. The fields are read. A field found in the input is replaced as a whole, so a record inside it no longer matters.
  5. A record inside a missing field fails the read with the error shown above, with the path in details().
  6. A per-type relaxed std::atomic<bool> remembers that T{} throws, so later reads skip step 1.

Two related changes, each in its own commit:

  • rfl::Variant() constructs its first alternative in place, so rfl::Variant and rfl::TaggedUnion fields are covered.
  • to_schema<Person>() threw on main, and called std::terminate when Person was nested in another struct. It now returns a schema without default values.

Limits

One case succeeds with an invalid value. A default member initializer that copies or reads a sibling Validator with no valid default is not checked:

struct Copies {
  Positive a;                        // rfl::Validator<int, rfl::Minimum<1>>
  rfl::DefaultVal<Positive> b = a;
};
rfl::json::read<Copies>(R"({"a": 5})");  // ok, b == 0

Copies{} outside a read still throws. Closing this needs user-provided copy constructors on Validator, which would make Validator<int, ...> non-trivially copyable for everyone. A test pins the behaviour and the docs state it.

Everything else fails with an error:

  • An invalid default built outside the struct's storage: moved in (rfl::DefaultVal<Positive> p = Positive();, rfl::Tuple<Positive, int>, a small struct returned in registers, -fno-elide-constructors) or on the heap (std::vector<Positive>(1), rfl::Box).
  • A default that a helper builds invalid and then fixes in place counts as invalid if the field is missing.
  • A default that catches the exception of an invalid Validator and falls back to a valid value works with clang and GCC. MSVC builds that Validator directly inside the struct, so it counts as deferred, and the read fails if the field is missing.
  • rfl::Skip<Validator> without a valid default is never read, so the read always fails.
  • A struct whose T{} throws for another reason is constructed twice before the same error is returned.
  • Windows DLLs each have their own copy of the thread_local. A Validator() that runs in another DLL than the parser throws, and the read fails as on main.

Measurements

All against main (e91ea28), macOS arm64, Apple clang 17 and Homebrew GCC 16.2.

Run time at -O3, in ns, median of 5 interleaved runs:

clang main clang branch clang Δ GCC main GCC branch GCC Δ
read, the issue's Person 4585 (fails) 822 (ok) −3763 (−82%) 5096 (fails) 685 (ok) −4411 (−87%)
read, DefaultVal struct without a Validator 145.0 144.7 −0.3 (−0.2%) 139.1 144.7 +5.6 (+4.0%)
read, struct with valid-default Validator<int> fields 144.5 149.1 +4.6 (+3.2%) 141.3 149.2 +7.9 (+5.6%)
read, the issue's struct with a valid dummy default 283.8 282.8 −1.0 (−0.4%) 287.6 300.7 +13.1 (+4.6%)
Validator<int, Minimum<0>>() 0.04 0.04 0 0.32 0.32 0
Validator<std::string, Size<Maximum<10>>>() 2.07 2.61 +0.54 (+26%) 3.03 3.36 +0.33 (+11%)
{} of a struct with four of those and one int Validator 7.49 10.67 +3.18 (+42%) 10.54 11.80 +1.26 (+12%)

The read rows vary by up to 12% between runs, so only the first row's delta is outside the noise. The last two rows are the real cost: the failure path of Validator() can now return instead of only throwing, which costs 0.3 to 0.6 ns per default-constructed Validator of a non-trivial type.

Compile time and peak memory, clang -O2 -c, median of 5 interleaved runs:

main branch Δ time main branch Δ memory
self-recursive struct with one DefaultVal 1.68 s 1.73 s +0.05 s (+3.0%) 273 MiB 277 MiB +4 MiB (+1.5%)
4 mutually recursive structs, DefaultIfMissing 1.47 s 1.59 s +0.12 s (+8.2%) 254 MiB 264 MiB +10 MiB (+3.9%)
8 mutually recursive structs, DefaultIfMissing 1.86 s 2.06 s +0.20 s (+10.8%) 276 MiB 290 MiB +14 MiB (+5.1%)

That is 25 to 30 ms per struct type read with defaults. Of each time delta, 0.01 to 0.03 s comes from the base branch.

Tests

clang 17 GCC 16.2
reflect-cpp-json-tests 310 pass 310 pass
reflect-cpp-generic-tests 102 pass 102 pass
reflect-cpp-cli-tests 75 pass not built (fails to build on main too: strtod_l)

The new tests also pass with -std=c++23 -DREFLECTCPP_USE_STD_EXPECTED (both compilers), with -fsanitize=address,undefined and with -fsanitize=thread (clang).

The fork has no CI. On the same commit (4c04f88), all 176 checks of the upstream CI pass. That covers Linux with GCC 11 to 14 and clang 16 to 18 (C++20 and C++23), MSVC with static and shared builds, macOS arm64 and Intel, and the test targets of all formats. Not tested: a Validator() that runs in another DLL than the parser (see Limits).

Stacking

Based on fix-700-prereqs: fork PRs #9, #10 and #11, plus one commit that adapts a test of #10 to the message of #11. On that base the "before" message carries the prefix from #11 (Could not default-construct Person, ...).

🤖 Generated with Claude Code

clockblocker and others added 4 commits November 11, 2026 09:58
rfl::Variant() default-constructed its first alternative as a temporary
and moved it into its storage. Construct it in place instead, which
saves the move.

This matters for the next commit: when a struct is read, the parser can
defer the check of a Validator whose default value is invalid, but only
if the Validator is constructed inside the struct. A temporary is not,
so a struct with an rfl::Variant or rfl::TaggedUnion field whose first
alternative holds such a Validator could not be read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A struct that contains an rfl::DefaultVal, or that is read with
rfl::DefaultIfMissing, is default-constructed before it is read, so that
missing fields keep their default values. If a Validator in it has no
valid default value, as in

    struct Person {
      rfl::DefaultVal<std::string> name = "Homer Simpson";
      rfl::Validator<std::vector<std::string>,
                     rfl::Size<rfl::Minimum<1>>> favorite_foods;
    };

T{} throws, and the struct could not be read at all, even when the
input contains the field (upstream issue `munich-data-engineering#700`).

C++ has no way to get the default member initializer of one field
without constructing the whole aggregate, and Validator() can only tell
that it runs inside a read from state outside of it. So:

- read_struct_with_default still tries T{} first, exactly as before.
  Only if that throws does it construct T{} again, in the same storage,
  while an internal::DeferredDefaults is active on this thread.
- Validator() validates as before. Only if validation fails does it look
  for an active DeferredDefaults. If the Validator lies inside the
  storage of the struct being constructed, it keeps T() and records its
  address and error instead of throwing. Validators anywhere else, such
  as temporaries, on the heap or in a nested read, throw as before.
- After reading, a record inside a field that the input contains is
  harmless, because the reader replaces such a field as a whole. A
  record inside a missing field is the error "Field named 'x' not found,
  and its default value is invalid: ...". A record outside any field
  (not expected to happen) is the default-construction error.
- Once T{} needed this, later reads of T skip the first, throwing try.

T is constructed with placement new rather than returned from a helper,
because a trivially copyable T may be copied on return, which would
move the recorded Validators.

Outside of read, Validator() and T{} behave as before. A struct whose
T{} does not throw is read by the same steps as before.

The test of fork PR 11 used structs that can now be read. They now get
invalid default values that are moved into the struct, which cannot be
deferred.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
To put the default values of rfl::DefaultVal fields into a JSON schema,
to_schema default-constructs the struct. If that throws, for instance
because a Validator has no valid default value, the exception escaped
rfl::json::to_schema, and for a nested struct it called std::terminate,
because NamedTupleParser::to_schema is noexcept.

Leave out the default values of such a struct instead. An invalid
default value would not belong in the schema anyway, and the struct can
be read since the previous commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace the section that said every field of a struct with defaults
needs a valid default value. Such a field can now be read; it must not
be missing. List the cases that still fail with the default-construction
error and the copy that is not checked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_default_val_with_validator_limits_fallback failed with MSVC. Its
default member initializer catches the exception of a Validator and falls
back to a valid value. While a struct is constructed with deferred
validation, whether that exception is thrown depends on where the compiler
constructs the returned Validator: outside of the struct (returned in
registers under the Itanium ABI), where it throws as usual, or directly
inside the struct (MSVC), where its validation is deferred like that of any
other field. In the second case the read fails with an error if the field
is missing from the input.

The test now accepts both outcomes and checks that the read succeeds with
every compiler if the input contains the field. The docs mention the case.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant