Skip to content

Read structs with defaults that hold a Validator without a valid default - #780

Open
clockblocker wants to merge 13 commits into
munich-data-engineering:mainfrom
clockblocker:fix-700-deferred-defaults
Open

clockblocker wants to merge 13 commits into
munich-data-engineering:mainfrom
clockblocker:fix-700-deferred-defaults

Conversation

@clockblocker

@clockblocker clockblocker commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #700 at the cost of the library's first thread_local.

The only other acceptable solution I found was to document the suggested workaround in the code and docs and leave it be.

AI-authored

Stacked on three open PRs

This branch sits on #776, #778 and #779, so until they merge the diff here includes them. Only the last five commits belong to this PR: compare view, +974 −47 lines in 14 files. The branch also holds one commit that adapts a test of #778 to the message of #779. Compared with #779 instead of main, the "before" message below carries its prefix (Could not default-construct Person, ...).

Before and after

struct Person {
  rfl::DefaultVal<std::string> name = "Homer Simpson";
  rfl::Validator<std::vector<std::string>, rfl::Size<rfl::Minimum<1>>> favorite_foods;
};

rfl::json::read<Person>(R"({"favorite_foods": ["donuts", "pizza"]})");
main:        error  Size validation failed: Value expected to be greater than or equal to 1, but got 0.
this branch: ok     {"name":"Homer Simpson","favorite_foods":["donuts","pizza"]}

On main no input can be read into this struct. The parser builds Person{} before it looks at the input, and the empty favorite_foods fails its own validator.

A field without a valid default still has to come from the input:

rfl::json::read<Person, rfl::DefaultIfMissing>(R"({})");
main:        error  Size validation failed: Value expected to be greater than or equal to 1, but got 0.
this branch: error  Field named 'favorite_foods' not found, and its default value is invalid: Size validation failed: Value expected to be greater than or equal to 1, but got 0.

Outside a read, Person{} and Validator() throw as before.

Approaches

Defaults of missing fields exist only as default member initializers, which C++ evaluates only while constructing the whole struct. So the parser has to build T{} before it reads, and Validator() has to return normally during that. It takes no arguments, so the parser cannot tell it who is calling.

Approach Why not, or what it costs
The user writes a valid dummy default; docs and a clearer error (#779) The dummy is real data under DefaultIfMissing and in Person{}.
Validator() never validates its default Person{} in user code holds an invalid value without an error.
Validator can hold "no value": a Result<T> (#737) or a flag Every Validator grows and gets accessors that can fail.
Validator() detects the parser with std::source_location (#707) Depends on the caller's function name. A missing field keeps the unvalidated default under DefaultIfMissing and DefaultVal<Validator>.
Thread-local scope, then a walk over the type to find invalid defaults (clockblocker#8) +926 lines in include/, 647 of them the walk, which broke compilation for some rfl::Skip fields.
A new wrapper for required fields Users change their structs, and the wrapper has an empty state.
Defaults carried in the type instead of = value A second way to declare defaults, limited to compile-time constants.
Thread-local scope plus address records (this PR) The library's first thread_local. +265 −30 lines in include/.

Only the last one reads the issue's struct with no change to user code, leaves Validator's layout and its behaviour outside a read alone, and still rejects a missing field whose default is invalid. It needs no type walk, because each Validator that keeps an invalid default reports its own address.

How it works

  1. read_struct_with_default builds T{} in the result's storage, as on main. If that succeeds, nothing else changes.
  2. If it throws, T{} is built again while a DeferredDefaults scope is active on this thread (RAII, restores the previous scope).
  3. Inside the scope, a Validator() whose default fails validation and whose address lies inside T's storage keeps T() and records its address and error. Every other Validator() (temporaries, the heap, other threads) throws as before.
  4. The fields are read. A field found in the input is replaced as a whole, so a record inside it no longer matters.
  5. A record inside a missing field fails the read with the error shown above, with the path in details().
  6. A per-type relaxed std::atomic<bool> remembers that T{} throws, so later reads skip step 1.

Two related changes, each in its own commit:

  • rfl::Variant() constructs its first alternative in place, so rfl::Variant and rfl::TaggedUnion fields are covered.
  • to_schema<Person>() threw on main, and called std::terminate when Person was nested in another struct. It now returns a schema without default values.

Limits

One case succeeds with an invalid value. A default member initializer that copies or reads a sibling Validator with no valid default is not checked:

struct Copies {
  Positive a;                        // rfl::Validator<int, rfl::Minimum<1>>
  rfl::DefaultVal<Positive> b = a;
};
rfl::json::read<Copies>(R"({"a": 5})");  // ok, b == 0

Copies{} outside a read still throws. Closing this needs user-provided copy constructors on Validator, which would make Validator<int, ...> non-trivially copyable for everyone. A test pins the behaviour and the docs state it.

Everything else fails with an error:

  • An invalid default built outside the struct's storage: moved in (rfl::DefaultVal<Positive> p = Positive();, rfl::Tuple<Positive, int>, a small struct returned in registers, -fno-elide-constructors) or on the heap (std::vector<Positive>(1), rfl::Box).
  • A default that a helper builds invalid and then fixes in place counts as invalid if the field is missing.
  • A default that catches the exception of an invalid Validator and falls back to a valid value works with clang and GCC. MSVC builds that Validator directly inside the struct, so it counts as deferred, and the read fails if the field is missing.
  • rfl::Skip<Validator> without a valid default is never read, so the read always fails.
  • A struct whose T{} throws for another reason is constructed twice before the same error is returned.
  • Windows DLLs each have their own copy of the thread_local. A Validator() that runs in another DLL than the parser throws, and the read fails as on main.

Measurements

All against main (e91ea28), macOS arm64, Apple clang 17 and Homebrew GCC 16.2.

Run time at -O3, in ns, median of 5 interleaved runs:

clang main clang branch clang Δ GCC main GCC branch GCC Δ
read, the issue's Person 4585 (fails) 822 (ok) −3763 (−82%) 5096 (fails) 685 (ok) −4411 (−87%)
read, DefaultVal struct without a Validator 145.0 144.7 −0.3 (−0.2%) 139.1 144.7 +5.6 (+4.0%)
read, struct with valid-default Validator<int> fields 144.5 149.1 +4.6 (+3.2%) 141.3 149.2 +7.9 (+5.6%)
read, the issue's struct with a valid dummy default 283.8 282.8 −1.0 (−0.4%) 287.6 300.7 +13.1 (+4.6%)
Validator<int, Minimum<0>>() 0.04 0.04 0 0.32 0.32 0
Validator<std::string, Size<Maximum<10>>>() 2.07 2.61 +0.54 (+26%) 3.03 3.36 +0.33 (+11%)
{} of a struct with four of those and one int Validator 7.49 10.67 +3.18 (+42%) 10.54 11.80 +1.26 (+12%)

The read rows vary by up to 12% between runs, so only the first row's delta is outside the noise. The last two rows are the real cost: the failure path of Validator() can now return instead of only throwing, which costs 0.3 to 0.6 ns per default-constructed Validator of a non-trivial type.

Compile time and peak memory, clang -O2 -c, median of 5 interleaved runs:

main branch Δ time main branch Δ memory
self-recursive struct with one DefaultVal 1.68 s 1.73 s +0.05 s (+3.0%) 273 MiB 277 MiB +4 MiB (+1.5%)
4 mutually recursive structs, DefaultIfMissing 1.47 s 1.59 s +0.12 s (+8.2%) 254 MiB 264 MiB +10 MiB (+3.9%)
8 mutually recursive structs, DefaultIfMissing 1.86 s 2.06 s +0.20 s (+10.8%) 276 MiB 290 MiB +14 MiB (+5.1%)

That is 25 to 30 ms per struct type read with defaults. Of each time delta, 0.01 to 0.03 s comes from the three PRs this one sits on.

Tests

clang 17 GCC 16.2
reflect-cpp-json-tests 310 pass 310 pass
reflect-cpp-generic-tests 102 pass 102 pass
reflect-cpp-cli-tests 75 pass not built (fails to build on main too: strtod_l)

The new tests also pass with -std=c++23 -DREFLECTCPP_USE_STD_EXPECTED (both compilers), with -fsanitize=address,undefined and with -fsanitize=thread (clang).

CI: all 176 checks pass on 4c04f88. That covers Linux with GCC 11 to 14 and clang 16 to 18 (C++20 and C++23), MSVC with static and shared builds, macOS arm64 and Intel, and the test targets of all formats. Not tested: a Validator() that runs in another DLL than the parser (see Limits).

🤖 Generated with Claude Code

clockblocker and others added 12 commits November 11, 2026 07:35
DefaultVal(U&&) was unconditionally noexcept, so an exception from
constructing the stored value called std::terminate. That is the case
for an initializer that a validator rejects:

    rfl::DefaultVal<rfl::Validator<int, rfl::Minimum<1>>> retries = 0;

Default-constructing such a struct aborted instead of throwing, and so
did every rfl::json::read of it, although read_struct_with_default
catches the exception to return it as an error.

Make the constructor noexcept only if constructing Type from U is.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
ParserSkip::read is noexcept. For a field that is skipped when reading,
it constructs the default value of the wrapped type, which throws for a
Validator whose default value is invalid, such as
rfl::Skip<rfl::Validator<int, rfl::Minimum<1>>>. Reading a struct with
such a field in the input called std::terminate.

Catch the exception and return it as the field's error.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A struct that contains rfl::DefaultVal or is read with
rfl::DefaultIfMissing is default-constructed before it is read. If T{}
throws, for instance because a validated field has no valid default
value, the message of the exception was returned as it is:

  Size validation failed: Value expected to be greater than or equal
  to 1, but got 0.

This reads as if the input was wrong, even when the field is in the
input and valid, and it names neither the struct nor the cause.

What is read and what fails stays the same. But the error now names the
struct, says why it was default-constructed and what to do about it,
followed by the original message:

  Could not default-construct Person, which is required because it
  contains rfl::DefaultVal. Give every field a valid default value.
  Original error: Size validation failed: Value expected to be greater
  than or equal to 1, but got 0.

Only a failure of T{} gets this text. An exception thrown later in
read_struct_with_default keeps its own message.

Also document the restriction and the workaround in default_val.md and
processors.md: a field that is given an initializer for this reason
stays required, unless the struct is read with rfl::DefaultIfMissing.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
NamedTupleParser::handle_one_missing_field and
TupleReader::handle_missing_fields_impl are noexcept. For a field that
is not required and missing from the input, they construct the default
value in place, which throws for a type such as
rfl::Skip<rfl::Validator<int, rfl::Minimum<1>>>. Reading a struct, a
named tuple or a tuple without that field called std::terminate.

Catch the exception and report it as the field's error. The field is
not marked as set, so its destructor is not run. Types that are nothrow
default constructible are constructed as before.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
test_default_val_throwing_initializer (PR 10) expects the bare validator
message when T{} throws, while PR 11 prefixes it with "Could not
default-construct ...". Both are merged here, so the test failed. Expect
the message that the combination produces.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rfl::Variant() default-constructed its first alternative as a temporary
and moved it into its storage. Construct it in place instead, which
saves the move.

This matters for the next commit: when a struct is read, the parser can
defer the check of a Validator whose default value is invalid, but only
if the Validator is constructed inside the struct. A temporary is not,
so a struct with an rfl::Variant or rfl::TaggedUnion field whose first
alternative holds such a Validator could not be read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A struct that contains an rfl::DefaultVal, or that is read with
rfl::DefaultIfMissing, is default-constructed before it is read, so that
missing fields keep their default values. If a Validator in it has no
valid default value, as in

    struct Person {
      rfl::DefaultVal<std::string> name = "Homer Simpson";
      rfl::Validator<std::vector<std::string>,
                     rfl::Size<rfl::Minimum<1>>> favorite_foods;
    };

T{} throws, and the struct could not be read at all, even when the
input contains the field (upstream issue `munich-data-engineering#700`).

C++ has no way to get the default member initializer of one field
without constructing the whole aggregate, and Validator() can only tell
that it runs inside a read from state outside of it. So:

- read_struct_with_default still tries T{} first, exactly as before.
  Only if that throws does it construct T{} again, in the same storage,
  while an internal::DeferredDefaults is active on this thread.
- Validator() validates as before. Only if validation fails does it look
  for an active DeferredDefaults. If the Validator lies inside the
  storage of the struct being constructed, it keeps T() and records its
  address and error instead of throwing. Validators anywhere else, such
  as temporaries, on the heap or in a nested read, throw as before.
- After reading, a record inside a field that the input contains is
  harmless, because the reader replaces such a field as a whole. A
  record inside a missing field is the error "Field named 'x' not found,
  and its default value is invalid: ...". A record outside any field
  (not expected to happen) is the default-construction error.
- Once T{} needed this, later reads of T skip the first, throwing try.

T is constructed with placement new rather than returned from a helper,
because a trivially copyable T may be copied on return, which would
move the recorded Validators.

Outside of read, Validator() and T{} behave as before. A struct whose
T{} does not throw is read by the same steps as before.

The test of fork PR 11 used structs that can now be read. They now get
invalid default values that are moved into the struct, which cannot be
deferred.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
To put the default values of rfl::DefaultVal fields into a JSON schema,
to_schema default-constructs the struct. If that throws, for instance
because a Validator has no valid default value, the exception escaped
rfl::json::to_schema, and for a nested struct it called std::terminate,
because NamedTupleParser::to_schema is noexcept.

Leave out the default values of such a struct instead. An invalid
default value would not belong in the schema anyway, and the struct can
be read since the previous commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace the section that said every field of a struct with defaults
needs a valid default value. Such a field can now be read; it must not
be missing. List the cases that still fail with the default-construction
error and the copy that is not checked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_default_val_with_validator_limits_fallback failed with MSVC. Its
default member initializer catches the exception of a Validator and falls
back to a valid value. While a struct is constructed with deferred
validation, whether that exception is thrown depends on where the compiler
constructs the returned Validator: outside of the struct (returned in
registers under the Itanium ABI), where it throws as usual, or directly
inside the struct (MSVC), where its validation is deferred like that of any
other field. In the second case the read fails with an error if the field
is missing from the input.

The test now accepts both outcomes and checks that the read succeeds with
every compiler if the input contains the field. The docs mention the case.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reading JSON into a struct with a DefaultVal field and a size validator field always raises a size validation error

1 participant