Skip to content

feat(workflows): expose workflow catalog sources through MCP - #5239

Open
franciscogdeveloper wants to merge 2 commits into
github:mainfrom
franciscogdeveloper:feat/4913-workflow-catalog-list-mcp
Open

franciscogdeveloper wants to merge 2 commits into
github:mainfrom
franciscogdeveloper:feat/4913-workflow-catalog-list-mcp

Conversation

@franciscogdeveloper

Copy link
Copy Markdown

Description

Adds specify_workflow_catalog_list for #4913, as proposed in the issue. specify workflow catalog list and the new MCP tool now share a typed, read-only workflow.catalog.list operation, following the structure of #5115. Only this command changes to available in the inventory.

Refs #4913

Credentials in catalog URLs

_validate_catalog_url accepts URLs such as https://user:token@host/catalog.json. The MCP result removes userinfo, query strings and fragments, and MCP errors use fixed messages, because config errors can quote a configured URL. The CLI still prints URLs as configured. A secret placed in the URL path isn't detected, which the docs state.

Please check: differences from my proposal

  1. The CLI rejects more unsafe paths. I wrote "CLI output unchanged", and that holds for normal listings and config errors. However, the shared operation now applies validate_workflow_storage and the project file policy, so the CLI rejects unsafe workflow storage and a symlinked .specify/workflow-catalogs.yml unless allow_symlinks is set. It used to follow them. I chose this for adapter parity and because workflow list behaves this way. If you'd rather keep the old CLI behavior, it's a small change.
  2. Fragments are removed too, in addition to userinfo and the query string, because they can also carry secrets.
  3. The tool takes limit (1-1000, default 100) and reports truncated, matching specify_workflow_list, to keep responses bounded. The CLI still lists everything.

The environment override and user-level config come from the server process rather than the selected project, the same resolution order as the CLI. The docs state this.

Testing

  • Tested locally with specify --help
  • Ran existing tests with uv sync && uv run pytest (partial on Windows, see below)
  • Tested with a sample project

All results are from Windows 11 with Python 3.13, using the project's .venv.

Check Without this change With this change
tests/specify_cli/workflows, tests/specify_cli/mcp_server, tests/test_workflows.py 1,701 passed, 35 failed, 73 skipped 1,783 passed, 35 failed, 74 skipped
  • The same 35 tests fail in both runs. I also compared 65 other tests that failed in a full run, and they fail in both too.
  • My full run stopped early at a failure limit I had set: 9,142 passed, 95 failed, 602 skipped. All 95 failures came with WinError 1314, because this host can't create symlinks. So the full suite hasn't completed on my machine, and I'm relying on CI for the rest.
  • test_symlinked_project_config_is_not_followed is skipped here for the same reason, so a real symlink hasn't been exercised locally. test_project_config_read_goes_through_the_file_policy covers the error mapping by simulating the denial.
  • CLI output is byte-identical to main in 6 cases: defaults, a configured stack with brackets and credentials, and 4 invalid configs, including exit codes.
  • Disabling the URL stripping makes 10 tests fail, which confirms they protect it.
  • uvx ruff@0.15.0 check src tests and markdownlint-cli2 on the changed docs pass.

No slash command templates or scripts changed, so I didn't run manual slash-command tests.

AI Disclosure

  • I did not use AI assistance for this contribution
  • I did use AI assistance (fill in the disclosure below)

AI disclosure: Posted by @franciscogdeveloper. Claude Code (model: Claude Opus 5.5, auto mode, autonomous) inspected the source and wrote the code, tests, docs and this description. I reviewed the change before submitting it.

Add a typed, read-only workflow.catalog.list operation shared by the CLI
and a new specify_workflow_catalog_list MCP tool, and mark only this
command as available in the workflow inventory.

The MCP result removes userinfo, query strings and fragments from catalog
URLs, because they can carry credentials, and its errors never quote
configured values. The CLI keeps printing URLs as configured.

Reading through the shared operation means the CLI now also rejects unsafe
workflow storage and a symlinked project catalog config unless the project
policy allows symlinks, matching workflow list.

Refs github#4913

Assisted-by: Claude Code (model: Claude Opus 5.5, autonomous)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI auto review requested due to automatic review settings November 4, 2026 20:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Nested YAML aliases in catalog text fields are stringified before request or response limits apply, allowing a small configuration to exhaust MCP server memory.

5 open issues
What changed in this PR

Extends Specify’s workflow MCP support with catalog-source discovery, using the shared-operation pattern established for installed workflow listing.

Changes:

  • Share typed catalog resolution between CLI and MCP, including project file-policy checks.
  • Add a first-class MCP tool with strict schemas, bounded responses, URL credential stripping, and sanitized errors.
  • Extend inventory, regression and protocol coverage, and user documentation.

AI disclosure: GitHub Copilot (model: unknown, autonomous) generated this review for the requester; no human review is claimed.

File Description
tests/​specify_cli/​workflows/​test_mcp.py Extend inventory and contract parity checks.
tests/​specify_cli/​workflows/​test_mcp_list.py Update workflow tool discovery expectations.
tests/​specify_cli/​workflows/​catalog/​test_operation_list.py Cover shared resolution, limits, and file policies.
tests/​specify_cli/​workflows/​catalog/​test_mcp_list.py Test schemas, dispatch, redaction, and response bounds.
tests/​specify_cli/​workflows/​catalog/​test_command_list.py Check CLI output and configuration errors.
tests/​specify_cli/​mcp_server/​test_stdio.py Exercise catalog listing over stdio.
tests/​specify_cli/​mcp_server/​test_server.py Include the new tool in discovery.
src/​specify_cli/​workflows/​catalog/​mcp_list.py Implement the strict, bounded MCP adapter.
src/​specify_cli/​workflows/​catalog/​command_list.py Delegate CLI listing to the shared operation.
src/​specify_cli/​workflows/​catalog/​_operation_list.py Define typed, read-only catalog resolution.
src/​specify_cli/​workflows/​catalog/​_domain.py Support policy-controlled project config reads.
src/​specify_cli/​workflows/​_mcp.py Mark catalog listing available and register its adapter.
docs/​overview/​mcp.md Document inputs, results, redaction, and errors.
docs/​overview/​core.md Add catalog discovery to the MCP overview.

Review effort: Max (auto)

)
files = project_files(root)
validate_workflow_storage(root)
configs = WorkflowCatalog(root, read_files=files).get_catalog_configs()
)
files = project_files(root)
validate_workflow_storage(root)
configs = WorkflowCatalog(root, read_files=files).get_catalog_configs()

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines +101 to +103
files = project_files(root)
validate_workflow_storage(root)
configs = WorkflowCatalog(root, read_files=files).get_catalog_configs()
Comment thread docs/overview/mcp.md Resolved
@mnriem

mnriem commented Nov 4, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback

- Measure catalog configs with aliases unfolded before constructing them,
  covering nested aliases, merge keys, recursive aliases and deep nesting
- Keep the shared invalid_catalog_config message free of configured values;
  the CLI keeps its local diagnostics through the exception cause
- Load the project policy before classifying .specify, matching workflow.list
- Declare warning_types on the workflow.catalog.list descriptor
- Document the symlink policy difference between catalog list and other
  catalog readers, and qualify the MCP error wording
- Cover contained config symlinks under both policies and tighten the
  file-policy and default-read tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI auto review requested due to automatic review settings November 4, 2026 22:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Empty-scalar aliases bypass the expansion limit, allowing a small catalog configuration to cause excessive memory allocation before MCP response-size checks.

6 open issues
1 previously missed issue

In code that hasn't changed since last review

Medium severity Preserve errors for dangling project config symlinks

src/​specify_cli/​workflows/​catalog/​_domain.py:436

With allow_symlinks: true, a project catalog config linked to a missing file inside the project raises FileNotFoundError for the target. This handler treats it as an absent configuration, so get_active_catalogs() silently returns user or built-in sources instead. The FileHelper contract (design/file-helper.md, “Boundary and policy”) requires existing regular-file targets. Please preserve the failure for a present dangling link and add a regression test; only a genuinely absent config should select the next layer.

AI-generated review for the requester by GitHub Copilot (model unavailable, autonomous); no human validation claimed.

7 resolved since last review

Review effort: Max (auto)

def _expanded_size(node: yaml.Node, sizes: dict[int, int], active: set[int]) -> int:
"""Measure a composed node with aliases unfolded, visiting shared nodes once."""
if isinstance(node, yaml.ScalarNode):
return len(node.value)
except FileNotFoundError:
return None
data = _safe_load_catalog_config(text)
except (yaml.YAMLError, OSError, UnicodeError) as exc:
Comment on lines +848 to +850
raw = _safe_load_catalog_config(
config_path.read_text(encoding="utf-8")
)
@mnriem

mnriem commented Nov 4, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback

@mnriem mnriem added the triage-nice-to-have Verdict: evidence-backed fix or greenlit feature — land after review label Nov 4, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

triage-nice-to-have Verdict: evidence-backed fix or greenlit feature — land after review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants