Skip to content

fix(gateway): a dotted secret module setting gets a nested env name - #5241

Merged
paul-nechifor merged 5 commits into
mainfrom
aa-monts/fix/dim-510-nested-secret-env-names
Nov 15, 2026
Merged

paul-nechifor merged 5 commits into
mainfrom
aa-monts/fix/dim-510-nested-secret-env-names

Conversation

@aa-MONTS

Copy link
Copy Markdown
Contributor

To be reviewed by @paul-nechifor.

Why do we need this?

Launch a blueprint through the gateway with a secret module setting given as a dotted path, for example config.modules.cam = {"auth.token": "..."}. The gateway keeps secrets off the command line and passes them in the environment, but module_env (experimental/gateway/launches.py:153-160) builds the name as CAM__AUTH.TOKEN. dimos run splits environment names on __ only (dimos/core/coordination/blueprint_config/merging.py:124), so it reads one unknown field auth.token and refuses the launch with Extra inputs are not permitted.

What is this feature?

module_env now turns each dot of a field name into __, so the name is CAM__AUTH__TOKEN, the form dimos run already reads for a nested setting (GO2CONNECTION__MUJOCO__STEPS_PER_FRAME in docs/usage/configuration.md). One production line; the existing secrets test gets a dotted secret.

This is only the secret half of DIM-510. The public half is not fixed here and needs a decision: the gateway folds {"mujoco.steps_per_frame": 3} (or the nested object) into --go2connection.mujoco={"steps_per_frame":3}, which the CLI rejects as an unknown option, while the same folding is what a dict-valued field such as controlcoordinator.addresses needs. The gateway cannot tell the two apart without the blueprint's parser schema. Options:

  • The CLI accepts a JSON object at a model-typed parent, as the environment already does (CAM__PROFILE='{"exposure":{"gain":3}}' parses today). Roughly 25 to 55 lines in merging.py, and a new CLI form.
  • The gateway expands to leaf flags from the parser schema. It never loads a blueprint in its own process today, so this needs the introspection result passed into launches.start.
  • The gateway sends dotted public settings through the environment too. The environment already accepts leaf paths for nested models and for dict-valued fields, but those values would then rank below CLI flags.

How to manually run this

uv run python - <<'EOF'
from experimental.gateway import launches
from experimental.gateway.launches import LaunchConfig

launch = LaunchConfig({}, {"cam": {"auth.token": "s3cret"}})
print(launches.secret_env(launch.global_, launch.modules, launch.secret_paths()))
EOF

main prints {'CAM__AUTH.TOKEN': 's3cret'}, this branch {'CAM__AUTH__TOKEN': 's3cret'}.

Checked parser-only, with a synthetic module whose config nests a pydantic model (no shipped module has a nested secret field): fed to BlueprintConfigParser, main's name is rejected and the new name sets the nested field. No gateway was started and nothing was launched. A launch that also carries a dotted public nested setting is still rejected, for the reason above.

Which issue(s) does this PR close?

Part of DIM-510: the secret half. The public half stays open.

Checklist

  • I have read and approved the CLA.

A secret module setting sent with a dotted path (`auth.token`) went to
dimos as MODULE__AUTH.TOKEN. dimos splits environment names on `__`, so
it read that as one unknown field and refused the launch. The name is now
MODULE__AUTH__TOKEN, which dimos reads as the nested setting.
@aa-MONTS aa-MONTS added the backport:skip Skip creating a backport to any release branches label Nov 12, 2026
@aa-MONTS aa-MONTS added the backport:skip Skip creating a backport to any release branches label Nov 12, 2026
@greptile-apps

greptile-apps Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] This change appears safe to merge.

Summary

The gateway now turns dots in secret module field paths into __, matching the names dimos run reads.

  • Dotted module secrets now use nested environment names.

Reviews (1) · Last reviewed commit: "fix(gateway): a dotted secret module set..." · Reviewed by Greptile

@github-actions github-actions Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

go2 replay realtime (arm64)

Details
Benchmark suite Current: da9117f Previous: 0ac6c52 Ratio
peak memory 1769.812 MB 1748.082 MB 1.01
peak threads 378 threads 391 threads 0.97
network (transport) 2729.302 MB 2732.054 MB 1.00
disk write 1.641 MB 1.613 MB 1.02
instructions 138.09 G 137.927 G 1.00

This comment was automatically generated by workflow using github-action-benchmark.

@codecov

codecov Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

@@           Coverage Diff           @@
##             main    #5241   +/-   ##
=======================================
  Coverage   84.58%   84.59%           
=======================================
  Files        1925     1925           
  Lines      189595   189595           
  Branches    15503    15503           
=======================================
+ Hits       160376   160382    +6     
+ Misses      25164    25160    -4     
+ Partials     4055     4053    -2     
Components Coverage Δ
Tests 96.71% <ø> (+<0.01%) ⬆️
Flag Coverage Δ
OS-macos-latest 78.99% <ø> (-0.01%) ⬇️
OS-ubuntu-24.04-arm 78.57% <ø> (-0.01%) ⬇️
OS-ubuntu-latest 79.04% <ø> (+<0.01%) ⬆️
Py-3.10 79.03% <ø> (-0.01%) ⬇️
Py-3.11 79.01% <ø> (-0.01%) ⬇️
Py-3.12 79.02% <ø> (-0.01%) ⬇️
SelfHosted-Linux 40.67% <ø> (+<0.01%) ⬆️
SelfHosted-macOS 40.40% <ø> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 8 files with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@paul-nechifor
paul-nechifor enabled auto-merge (squash) November 15, 2026 04:21
Recursively identify and redact secrets, emit nested model leaf flags, and keep secret-bearing collections intact in environment values. Resolve parser targets in the launch checkout with its global overrides and preserve configuration aliases and dictionary keys.

Add parser round-trip, catalog introspection, and launch/relaunch regression coverage.
…uments

The fake run_child in the gateway test fixture only took dimos_dir and args
positionally. State.introspected passes stdin as a third positional argument,
so any test that introspects a real blueprint through the client failed with
a TypeError. The multi-Go2 catalog tests added on main in #5227 do that, which
broke CI on the merge of this branch with main.

The fake now forwards any extra positional arguments to the real run_child.
…3.10

On Python 3.10 pydantic leaves a string inside a builtin generic such as
list["RecursiveOptions"] unresolved in the field annotation. The blueprint
config parser then builds a TypeAdapter for it outside the test module, where
the name is unknown, and raises PydanticUserError. Two introspect tests failed
that way on the 3.10 CI job only.

Quoting the whole annotation makes pydantic resolve it on every version.
Conflicts with #5242 (global flags spelled from validated GlobalConfig values):
- experimental/gateway/launches.py: keep LaunchConfig.normalized() from main
  next to this branch's secret_paths(targets).
- experimental/gateway/tests/test_launch_config.py: union of both import lists.
@github-actions github-actions Bot added the ready-to-merge Required CI checks have passed on this PR label Nov 15, 2026
@paul-nechifor
paul-nechifor merged commit ba82b7f into main Nov 15, 2026
42 checks passed
@paul-nechifor
paul-nechifor deleted the aa-monts/fix/dim-510-nested-secret-env-names branch November 15, 2026 08:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:skip Skip creating a backport to any release branches ready-to-merge Required CI checks have passed on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants