Repository navigation
Conversation
9202c85 to
1a34f1f
Compare
There was a problem hiding this comment.
Note
Copilot was unable to complete this review at the selected tier due to model policy or plan restrictions.
🟡 Changes recommended
The change still has an unresolved mismatch between the stated behavior and the implementation in internal/fs/vault.go (line 642), where the code returns after the first forbidden key instead of deterministically reporting all rejected names as described, indicating remaining active issues to address before approval.
1 open issue
What changed in this PR
Hardens ti against environment-based subprocess attacks by blocking process-configuration vault field names for run-with-secret and ensuring sensitive credentials are scrubbed from selected subprocess environments.
Changes:
- Reject vault fields whose normalized names match a process-configuration env var denylist before spawning the child.
- Scrub credential-bearing environment variables from
gitandti updatevalidation/help probe subprocesses. - Add tests validating denylist behavior, pre-spawn refusal, and subprocess env scrubbing; update docs to describe the new rules.
| File | Description |
|---|---|
| README.md | Documents the new run-with-secret vault field-name restrictions and their security rationale. |
| internal/update/update.go | Updates ti update validation and helper-probe subprocesses to run under a credential-scrubbed environment while preserving required overrides. |
| internal/update/update_test.go | Adds tests ensuring ti update validation/help-probe subprocesses do not inherit credential environment variables. |
| internal/fs/vault.go | Introduces and enforces a normalized environment-key denylist for vault-derived child process environments in run-with-secret. |
| internal/fs/vault_secret_env_test.go | Adds unit tests for forbidden/allowed vault field names, deterministic error messaging, and refusal before spawning a child process. |
| internal/fs/git.go | Routes all git subprocess invocations through a scrubbed subprocess environment to prevent credential leakage. |
| internal/fs/git_subprocess_env_test.go | Adds tests that shim git to verify credential variables are not passed into git subprocess environments. |
| internal/credentialenv/credentialenv.go | Adds/standardizes a ScrubbedEnviron() helper for constructing subprocess environments without credential-bearing variables. |
| AGENTS.md | Updates contributor/agent guidance to reflect the new subprocess environment-scrubbing and vault field-name security rules. |
Review effort: Balanced
1a34f1f to
a144c8e
Compare
qiffang
left a comment
There was a problem hiding this comment.
Reviewed the vault read → normalization → validation → child spawn path, credential scrubber and platform matching, changed git/update subprocess sites, and regression tests. One blocking gap remains in the process-configuration filter, detailed inline.
Validation was static/source-based, including the official Lua startup contract; no local tests, child-process reproductions, or live service requests were executed. Current CI test and CLA checks pass. No sensitive information found in the submitted diff.
| "_JAVA_OPTIONS": "Java startup", | ||
| "JAVA_OPTIONS": "Java startup", | ||
| "CLASSPATH": "Java class loading", | ||
| "LUA_INIT": "Lua startup", |
There was a problem hiding this comment.
[P1] Reject versioned Lua startup variables as executable configuration
LUA_INIT_5_4 is not in this exact-name table, survives normalizeVaultEnvKey unchanged, and its printable value passes control-byte validation. vaultSecretChildEnv then injects it into the child. When a consumer runs ti fs-vault run-with-secret ... -- lua script.lua with Lua 5.4 (without -E), Lua checks LUA_INIT_5_4 before LUA_INIT and executes its value before the requested script; this is the documented Lua standalone startup behavior. Thus a secret writer can still supply startup code through a field named LUA_INIT_5_4; even the harmless value print("vault-init") would execute despite LUA_INIT being forbidden. This leaves the exact code-execution boundary this PR intends to close open for a runtime already covered by the denylist. The versioned module-path variables LUA_PATH_5_4 and LUA_CPATH_5_4 likewise bypass the corresponding base-name entries. Please enforce the same startup/module-loading restriction for supported versioned names and cover them with regression tests, including normalization/case variants.
There was a problem hiding this comment.
Fixed in 0c4a3cf, and generalized so the class cannot come back through another versioned spelling.
Denylist lookup now goes through vaultEnvKeyDenied: it checks the exact normalized name, then strips trailing all-digit segments (trimVaultEnvVersionSuffix, so LUA_INIT_5_4 -> LUA_INIT, LUA_PATH_5_4 -> LUA_PATH, LUA_CPATH_5_4 -> LUA_CPATH) and checks again. That covers every Lua release spelling instead of enumerating 5_1..5_4, and the suffix rule only applies to a denied base, so DB_URL_2 or LUA_VERSION stay injectable.
I also added the same-class runtime names I had missed: JDK_JAVA_OPTIONS (Java 9+ startup, next to JAVA_TOOL_OPTIONS), PHPRC / PHP_INI_SCAN_DIR (php.ini and scan-dir loading, e.g. auto_prepend_file), and RUBYGEMS_GEMDEPS (RubyGems dependency autoloading).
Tests: TestVaultSecretChildEnvRejectsProcessConfigurationKeys now includes LUA_INIT_5_4, lua_init_5_4, LUA_PATH_5_4, LUA_CPATH_5_1; TestVaultSecretChildEnvAllowsSecretLikeKeys pins LUA_VERSION, DB_URL_2, PGPASSWORD_5 as still allowed; TestTrimVaultEnvVersionSuffix pins the stripping edges (LUA_INIT_, _9LIVES, no-suffix names). The reject test fails if the version-suffix rule is disabled (mutation-checked). README and AGENTS document the version-segment rule. go vet ./... + go test ./... pass locally; CI is running on the new head.
mornyx
left a comment
There was a problem hiding this comment.
Review summary
This PR hardens ti fs-vault run-with-secret against process-configuration environment variable injection and routes git and ti update validation subprocesses through a shared credentialenv.ScrubbedEnviron() helper. Multi-key deterministic error reporting from the earlier review round is cleanly resolved, and the credential-prefix scrubbing design is solid. Under a strict maintainability and security bar, one merge-blocking gap remains: modeling vaultEnvKeyDenylist purely as an exact-name lookup leaves pre-script and helper code-execution bypasses open in runtimes and tools the denylist explicitly covers (including the open thread item on versioned Lua startup variables, plus OpenJDK JDK_JAVA_OPTIONS, Git config/directory/diff variables, and CPython PYTHONUSERBASE). Separately, git subprocess construction in internal/fs can be simplified to eliminate open-coded env wiring, cover a missed dormant git call in fuse_git.go, and remove dead ti self-spawn code.
What looks good
- Centralizing
credentialenv.ScrubbedEnviron()with case-insensitive prefix matching (TIDB_CLOUD_*,TDC_*,DRIVE9_*,TI_FS_*,TI_VAULT_*) prevents scrub-list drift across Unix and Windows while keepingTI_PROFILEand test controls intact. - Enforcing
vault.env_key_forbidden(exit 2) on normalized keys duringrun-with-secret—while keepingread-secret --format envreadable—and reporting every rejected field in sorted order with per-variable explanations gives users a deterministic, actionable error without locking them out of existing secrets. - Regression tests use executable shims to assert the actual environment received by child processes and verify that
run-with-secretrefuses protected keys before spawning the child.
Blocking (please address before merge)
1. Exact-name-only vaultEnvKeyDenylist leaves code-execution bypasses in covered runtimes and tools (internal/fs/vault.go:550-653)
validateVaultEnvFields (internal/fs/vault.go:650) checks only vaultEnvKeyDenylist[key], which leaves direct startup and helper code-execution bypasses in families already inside this boundary:
- Lua (
internal/fs/vault.go:597-599): Standalone Lua 5.2+ (without-E) checks versionedLUA_INIT_<maj>_<min>(e.g.LUA_INIT_5_4) beforeLUA_INITand executes inline Lua code or@filebefore the target script runs;LUA_PATH_5_4andLUA_CPATH_5_4likewise supersedeLUA_PATHandLUA_CPATH. Fields such asLUA_INIT_5_4orlua-init-5.4survivenormalizeVaultEnvKeyand bypass the exact-name map. - Java (
internal/fs/vault.go:593-596):JAVA_TOOL_OPTIONSand_JAVA_OPTIONSare blocked, butJDK_JAVA_OPTIONS—the standard OpenJDK 9+ launcher variable that prepends JVM options such as-javaagent:/path/evil.jaror@argfilebeforemain(along withIBM_JAVA_OPTIONS/OPENJ9_JAVA_OPTIONS)—passes through. - Git (
internal/fs/vault.go:605-617):GIT_CONFIG_GLOBAL,GIT_CONFIG_SYSTEM,GIT_CONFIG_COUNT,GIT_TEMPLATE_DIR, andGIT_SSH_COMMANDare blocked, yetGIT_CONFIG_PARAMETERS(passes arbitrary config such ascore.fsmonitororcore.sshCommandwithout needingGIT_CONFIG_COUNT),GIT_CONFIG_KEY_<N>/GIT_CONFIG_VALUE_<N>(effective wheneverGIT_CONFIG_COUNTis set in the parent environment),GIT_EXTERNAL_DIFF(executed directly bygit diff/git show/git log -p), andGIT_DIR/GIT_COMMON_DIR/GIT_WORK_TREE(pointing Git at an attacker-chosen.git/configand.git/hooks/) are allowed. - Python (
internal/fs/vault.go:583-585):PYTHONPATHandPYTHONSTARTUPare blocked, butPYTHONUSERBASEis allowed; on default CPython startup (without-s/-S/-I),sitescans<PYTHONUSERBASE>/lib/pythonX.Y/site-packages/*.pthandexec()s any line starting withimportbefore the script runs (PYTHONBREAKPOINTsimilarly imports and calls an arbitrary callable onbreakpoint()).
Suggested fix: Keep exact matching for broad prefixes so secret-like names (SSH_PRIVATE_KEY, GIT_TOKEN, PATH_EXTRA, LD_PRELOAD_BACKUP) remain usable, and replace the raw map lookup with a small forbiddenVaultEnvKey(key string) (controls string, blocked bool) helper that combines:
- Targeted pattern/prefix rules for versioned, indexed, or loader-only namespaces (
LUA_INIT_<digits...>,LUA_PATH_<digits...>,LUA_CPATH_<digits...>,GIT_CONFIG_KEY_<digits>,GIT_CONFIG_VALUE_<digits>,DYLD_*, and_32/_64orLD_32_*loader variants). - Exact-name entries for
JDK_JAVA_OPTIONS,IBM_JAVA_OPTIONS,OPENJ9_JAVA_OPTIONS,GIT_CONFIG_PARAMETERS,GIT_EXTERNAL_DIFF,GIT_DIR,GIT_COMMON_DIR,GIT_WORK_TREE,PYTHONUSERBASE, andPYTHONBREAKPOINT.
Please also cover versioned/indexed and normalized variants with regression tests and update the "exact name match" wording in README.md:53 and AGENTS.md:1164-1166 accordingly.
Should fix (non-blocking, strong)
2. Replace gitSubprocessEnv() with a single newGitCommand helper, cover fuse_git.go, and delete dead startGitHydrateBackground (internal/fs/git.go:1602-1623, internal/fs/fuse_git.go:196)
gitSubprocessEnv()(internal/fs/git.go:1602-1604) is a zero-argument one-line wrapper aroundcredentialenv.ScrubbedEnviron()applied manually at individualexec.CommandContextsites, which missed(r *remoteFuseRuntime).gitReadFileininternal/fs/fuse_git.go:196(exec.CommandContext(ctx, "git", "-C", repoRoot, "cat-file", "-p", objectSHA)still inheritsos.Environ()verbatim; currently dormant because public mounts delegate toti-drive9, but inconsistent with theAGENTS.md:1248-1252rule).startGitHydrateBackground(internal/fs/git.go:1606-1623) has zero callers anywhere in the repo and spawnsos.Executable()(ti fs-git hydrate-git-workspace), notgit—scrubbingTI_FS_TOKENandTIDB_CLOUD_*there would strip env-only credentials if it were ever invoked.- Replacing
gitSubprocessEnv()with a sharednewGitCommand(ctx context.Context, args ...string) *exec.Cmdhelper inpackage fsthat setscmd.Env = credentialenv.ScrubbedEnviron()in one place (used acrossinternal/fs/git.goandinternal/fs/fuse_git.go:196) makes unscrubbedgitspawns impossible by construction and lets you delete the unreferencedstartGitHydrateBackgroundfrom the 1,639-lineinternal/fs/git.go(updatingAGENTS.md:1248to match).
3. Cover adjacent native loader, OpenSSL, and CLR startup hooks in vaultEnvKeyDenylist (internal/fs/vault.go:568-602)
Several environment variables in the same threat class load attacker-chosen shared objects or startup assemblies into ordinary binaries before user code runs: OpenSSL config/engine/provider hooks read by TLS-linked tools (OPENSSL_CONF, OPENSSL_ENGINES, OPENSSL_MODULES), .NET startup/profiler hooks (DOTNET_STARTUP_HOOKS, CORECLR_PROFILER, CORECLR_PROFILER_PATH, COR_PROFILER, COR_PROFILER_PATH), ELF loader peers (LD_ORIGIN_PATH, GLIBC_TUNABLES), Windows PERL5SHELL, zsh/ksh FPATH, and SUDO_ASKPASS. Adding them alongside the item 1 updates closes obvious lateral bypasses in the same table.
Nits
internal/fs/vault.go:500-697:internal/fs/vault.gogrew from 725 to 848 lines in this PR and will approach ~900 lines once pattern rules and extra entries are added. Moving the self-contained child-env normalization, denylist/pattern check, and env-merge helpers intointernal/fs/vault_secret_env.go(alongsideinternal/fs/vault_secret_env_test.go) would keepvault.gofocused on vault control/mount operations and comfortably below 1,000 lines.
Recommendation
Request changes. One verified blocking gap remains in the run-with-secret environment-key filter (LUA_INIT_5_4 and sibling versioned/indexed/runtime hooks bypass the exact-name denylist); please address before merge.
a144c8e to
0c4a3cf
Compare
…dentials Two findings from the security review of main 182fdc8. 1. `ti fs-vault run-with-secret` converted stored vault field names into child environment variables without any name restriction, so a field named `LD_PRELOAD`, `PATH`, `BASH_ENV`, or `NODE_OPTIONS` let whoever can write the secret choose what the child loads or executes. Such names now fail with `vault.env_key_forbidden` before the child is spawned, and the error lists every rejected field in sorted order with the behavior it would configure. Matching runs on the normalized key, so `path` and `PaTh` hit `PATH`; it is exact-name, so plausible secret names such as `SSH_PRIVATE_KEY`, `GIT_TOKEN`, or `LD_PRELOAD_BACKUP` stay injectable; it strips trailing all-digit version segments, so `LUA_INIT_5_4`, `LUA_PATH_5_4`, `LUA_CPATH_5_4`, and `LD_PRELOAD_32` resolve to their base entries; and it matches the families whose suffix the field name chooses by prefix (`DYLD_*`, `LD_32_*`, `LD_64_*`, `GIT_CONFIG_KEY_<n>`, `GIT_CONFIG_VALUE_<n>`). `read-secret --format env` still converts those names because it only prints values and never spawns a child. The denylist and the child-env construction moved to `internal/fs/vault_secret_env.go`. 2. `internal/credentialenv` existed but was not applied to every subprocess that inherits it: git invocations handed the credentials to git and to the hooks, clean/smudge filters, pagers, and helpers git runs, and `ti update` handed the TiDB Cloud keys and the filesystem bearer token to the staged ti binary and the downloaded companion it validates. Every git spawn now goes through `newGitCommand` (including the dormant native-FUSE reader) and `ti update` uses `credentialenv.ScrubbedEnviron()`; the unreferenced `startGitHydrateBackground` self-spawn is deleted rather than scrubbed, since it spawns ti itself and would lose env-only credentials if it ever ran. Verification: gofmt, `go vet ./...` on linux/darwin/windows, `go test -count=1 ./...`, `make e2e`. Each new test was mutation-checked: it fails on the pre-fix code and passes with the fix.
0c4a3cf to
a413bcf
Compare
|
Addressed both Blocking item — startup/helper bypasses (
Tests: Should-fix 2. Should-fix 3. All the loader/OpenSSL/CLR/shell entries you listed are in the table. Nit. The env cluster ( Verification at |
mornyx
left a comment
There was a problem hiding this comment.
Review summary
All blocking, structural, and maintainability findings from the previous review round are cleanly resolved on a413bcf. vaultEnvKeyDenied in internal/fs/vault_secret_env.go now pairs the expanded normalized denylist with trailing all-digit version-segment stripping (trimVaultEnvVersionSuffix) and targeted prefix families (DYLD_, LD_32_, LD_64_, GIT_CONFIG_KEY_, GIT_CONFIG_VALUE_), closing the versioned/indexed startup and helper bypasses while keeping legitimate secret names injectable. Every git subprocess now goes through newGitCommand (including internal/fs/fuse_git.go), the unreferenced startGitHydrateBackground helper is gone, and extracting internal/fs/vault_secret_env.go leaves internal/fs/vault.go compact at 519 lines under a strict maintainability bar.
What looks good
vaultEnvKeyDenied+trimVaultEnvVersionSuffix+vaultEnvKeyFamilieshandles versioned (LUA_INIT_5_4,LD_PRELOAD_32) and indexed/namespace (DYLD_*,LD_32_*,GIT_CONFIG_KEY_*) process-configuration keys cleanly without rejecting numbered secret names such asDB_URL_2orPGPASSWORD_5, while still reporting the caller's normalized keys in deterministic sorted order.- Centralizing
gitsubprocess construction innewGitCommandacrossinternal/fs/git.goandinternal/fs/fuse_git.goeliminates per-callsitecmd.Envdrift by construction, andcredentialenv.ScrubbedEnviron()remains the single case-insensitive credential denylist shared withti updateandrun-with-secret. - Splitting the self-contained child-environment normalization, denylist/family matching, and execution logic into
internal/fs/vault_secret_env.gopairs directly withinternal/fs/vault_secret_env_test.goand keepsinternal/fs/vault.gofocused on vault control and mount operations.
Nits
internal/fs/vault_secret_env.go:152-157:vaultEnvKeyDenyliststill enumerates six individualDYLD_*keys (DYLD_INSERT_LIBRARIESthroughDYLD_IMAGE_SUFFIX) with"the dynamic loader", which are completely shadowed by{"DYLD_", "the dynamic loader"}invaultEnvKeyFamilies(internal/fs/vault_secret_env.go:304). Dropping those six entries would keepvaultEnvKeyFamiliesas the single source of truth for theDYLD_namespace.
Recommendation
Approve. All blocking and should-fix items are verified resolved with comprehensive regression tests and updated documentation; only an optional deduplication nit remains.

Two findings from the security review of
main(182fdc8).1.
ti fs-vault run-with-secretinjected process configuration into the childrun-with-secretturned every stored vault field name into a child environment variable afternormalizeVaultEnvKey, but validated only the values (control bytes). A field namedLD_PRELOAD,DYLD_INSERT_LIBRARIES,PATH,BASH_ENV,NODE_OPTIONS,PYTHONPATH,SSH_AUTH_SOCK, orGIT_SSH_COMMANDtherefore let whoever can write the secret's fields choose which code the child loads or which helper it executes — code execution inside the child even though the attacker never sees ti's own credentials.Field names that normalize to a process-configuration variable now fail with
vault.env_key_forbidden(usage, exit 2) before the child is spawned. The rejection set is an exact-name table (vaultEnvKeyDenylistininternal/fs/vault.go) grouped by what the name controls: program lookup, shell startup/evaluation, dynamic loader, language-runtime startup, helper selection, and Windows shell/loader names.pathandPaThhitPATH; case cannot bypass it.SSH_PRIVATE_KEY,GIT_TOKEN, orLD_PRELOAD_BACKUPstay injectable.LUA_INIT_5_4,LUA_PATH_5_4,LUA_CPATH_5_4, andLD_PRELOAD_32resolve to their base entries. The rule only applies to a denied base name, soDB_URL_2stays injectable.DYLD_*,LD_32_*,LD_64_*,GIT_CONFIG_KEY_<n>,GIT_CONFIG_VALUE_<n>.read-secret --format envstill converts those names: it only prints values and never spawns a child.2.
internal/credentialenvexisted but several subprocesses inherited credentials anywayinternal/fs/git.go,internal/fs/fuse_git.go): every invocation inherited the full environment. git runs hooks, clean/smudge filters, pagers, and helpers configured by the repository or the user, soTIDB_CLOUD_PRIVATE_KEY,TI_FS_TOKEN, andTI_VAULT_TOKENwere readable by that code. All git spawns are now built bynewGitCommand, which applies the scrubbed environment at construction time, including the dormant native-FUSE reader. The unreferencedstartGitHydrateBackgroundself-spawn is deleted: it spawns ti itself, so scrubbing it would have broken env-only credentials rather than hardened anything.ti update(internal/update/update.go):validateInstalledBinaryand the downloaded-companion help probe inheritedos.Environ(), handing the TiDB Cloud keys and the filesystem bearer token to a freshly unpacked binary. Both now run withcredentialenv.ScrubbedEnviron();validateInstalledBinarykeeps itsTI_LOGGING=offoverride on top.The credential-bearing spawn paths are unchanged: the companion environment still receives the
DRIVE9_*values it needs, and therun-with-secretchild still gets its existing scrub.Verification
gofmt -lclean;go vet ./...clean on linux,GOOS=windowsandGOOS=darwinclean.go test -count=1 ./...passes;make e2epasses.TestVaultSecretChildEnvRejectsProcessConfigurationKeys(versioned, prefixed, and runtime-hook spellings),TestVaultSecretChildEnvAllowsSecretLikeKeys,TestVaultSecretChildEnvForbiddenKeyMessageIsStable,TestTrimVaultEnvVersionSuffix,TestVaultSecretChildEnvCannotExportShellFunctions,TestRunVaultSecretCommandRefusesProtectedFieldBeforeSpawn,TestGitSubprocessesDoNotInheritCredentials(shimgitrecords the environment it receives),TestUpdateValidationSubprocessesDoNotInheritCredentials.README.mdandAGENTS.mddocument both rules.