[varunj-msft]

Description

Part 3 of saving Registry entries across restarts. Stacked on part 2 (#3892); part 4 (#3909) builds on it. This PR's own changes: varunj-msft/PyRIT@varunj-msft/13178-Read-Registry-Recipes...varunj-msft/13178-Rebuild-Saved-Entries

Design: 4.5 Entries that use other entries, 4.3 What a recipe holds. Where every part fits: PR map.

This is one of the changes for saving Registry entries across restarts. It builds on #3892. Until #3891 and #3892 merge, the diff here also shows their changes; this PR's own code is pyrit/registry/registry_recipe_restore.py and RegistryRestoreReport, with their tests.

Problem. Nothing turns saved recipes back into registry entries.

Change. RegistryRecipeRestorer: read() reads every folder without building anything, so a caller on an event loop can run it in a worker thread. restore_async(build_async=...) then works out the order and builds; called without what read() returned, it reads in a worker thread itself:

• If a folder can't be listed, it builds nothing, since other entries may use the ones it couldn't read.
• Classes are looked up among the registered ones by name, so a class that moved within PyRIT still restores, and a technique's attack type is looked up in the attack registry. It never imports a module a file names.
• It plans one order across the four kinds from Parameter.reference and the technique fields. A name resolves to a saved entry first (even a broken one), then to a startup entry. Missing and failed dependencies and loops are reported with the entry each one waits on.
• Each entry is built, one at a time on the calling loop, by the builder its caller passes in. The backend's builder (FEAT: Restore saved registry entries at startup and stay not ready if any fail (part 4) #3909) calls the API's own create for each kind, so a saved entry passes the same checks as a new one, and the library keeps no copy of those checks.
• It refuses a recipe that stores a value for a parameter its class marks as a credential, or that sets a server file or folder.
• Each failure gets a reason: a name another entry holds is name_clash; a reserved name, or a technique name a scenario can't use, is invalid_recipe; a request the create refuses keeps the create's reason; anything else is build_error. Taken names raise RegistryNameTakenError and technique admission problems raise TechniqueAdmissionError, so nothing parses error messages.
• Each failure is logged once with its message and traceback, with URL credentials removed. In the report, the message has URL credentials removed, control characters escaped and its length capped.
• It returns a RegistryRestoreReport: restored entries, failures (kind, name, reason, message, version, dependency) and ignored files.

Not in this PR.

• Calling it. FEAT: Restore saved registry entries at startup and stay not ready if any fail (part 4) #3909 (part 4) runs it at startup and on apply, with the builder.
• Credential rules that need each class to declare its fields: credentials a class would read from server settings, and fields that can carry a credential without being marked as one (headers, raw HTTP requests, URLs with tokens). Those come with saving, and restore applies the same rules. Until PyRIT writes recipes, a stored file is trusted the way configuration is.

Tests and Documentation

• New tests/unit/registry/test_registry_recipe_restore.py (39 tests), building through a small test builder: order across the four kinds (including converters in either of a technique's lists), loops, missing and failed dependencies, clashes with startup entries, read() building nothing and restore_async using what it returned without reading again, or reading in a worker thread and building on the calling one when it wasn't given it, a startup entry found by its exact name, stored credentials, server paths, a class found by name after it moved within PyRIT, a real module under tmp_path that restore never imports, unknown classes and attack types, technique recipes matching the create request, the reason each kind of failure gets, one build error that doesn't stop the rest, failure text (URL credentials removed, control characters escaped, long text capped), and logging each failure once without URL credentials.
• tests/unit/common/test_azure_storage.py: URL credentials are removed from any text.
• pytest tests/unit/registry tests/unit/models tests/unit/common -n 6: 4,663 passed, 5 skipped, 4 xfailed
• ruff, ruff format and ty on the changed files: passed (the pre-commit hooks can't install in my environment, so CI runs them)
• No doc change: nothing calls this yet. JupyText: not run (no notebooks changed).

[Richard Lundeen (richlundeen)]

Approving. To finish this without another review round, I pushed two small commits:

• f59306e:
  • When restore_async isn't given saved, it reads storage in a worker thread (the open P2).
  • Each restore failure is logged as an error.
  • redact_urls now takes time in proportion to the text's length. Before, text like "a." * 80_000 took 21 s, and a long value in a saved file can reach it through a build error.
  • It also finds a URL right after _, so endpoint_https://user:secret@host no longer leaks.
  • Tests: two cases now go through public paths, and a duplicate technique-admission test and an assertion that couldn't fail are gone.
• f1ff143: keeps an @ in a URL's path. The first commit had treated everything before it as a user name and password, which rewrote such URLs to name the wrong host (#4175: don't redact what operators need).

I'm withdrawing the main point of my Nov 9 review. When a saved entry can't be restored, the backend should not become ready. #4175 now states this as "Fail loudly rather than run partially." This PR already leaves that decision to its caller and reports every failure in one pass, so nothing changes here. The change is in #3909.

Non-blocking, for a later part: the server-path check here duplicates the registry's own check on API input, and it is stricter for converters: it refuses the data-URI and blob-URL uploads that the converter API accepts. When converter saving lands (#4051), it should use the same check as restore so the two can't disagree.

When you merge this into #3909, registry_recipe_restore.py may conflict with #3909's held-name code. Failing closed removes that code anyway.