Skip to content

fix(search-events): List tags, not feature flags, for error and issue event searches - #2172

Open
sentry-junior[bot] wants to merge 1 commit into
mainfrom
fix/issue-event-fields-query-tags
Open

sentry-junior[bot] wants to merge 1 commit into
mainfrom
fix/issue-event-fields-query-tags

Conversation

@sentry-junior

@sentry-junior sentry-junior Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

The embedded agents behind search_issue_events and search_errors check which tags exist before they build a query. Both lookups call /organizations/{org}/tags/ with useFlagsBackend=1. With that flag, Sentry reads the flags backend and returns feature flag keys instead of tags. See organization_tags.py: the param is documented as "query feature flags instead of tags", and the code uses backend = tagstore.flag_backend. In the traces, the Snuba query groups BY flags_key.

So the agent never sees a project's custom tags. Its prompt says that unlisted tags make queries fail, so it removes valid filters. The user feedback MCP-SERVER-GNJ shows this. search_issue_events got environment:PROD sessionId:<id>, and the agent's explanation said: "Removed sessionId because it is not an available tag for this issue's project". executedSearch ran environment:PROD only and returned unrelated events, and those events still carried sessionId tags. MCP-SERVER-GJX (an explicit custom tag filter dropped by search_issue_events) has the same symptom.

How widespread: over the last 30 days, about 490k of roughly 500k sampled issueEventFields executions returned exactly 34 entries. 34 is the size of the built-in ISSUE_EVENT_TAGS list, so these runs had zero project-specific tags. Counts above 34 show up only for orgs that have feature flags.

Changes

  • Drop useFlagsBackend: true from createIssueEventFieldsTool (search-issue-events) and from the errors branch of fetchCustomAttributes (search-events).
  • Remove the now-unused useFlagsBackend option from SentryApiService.listTags.
  • Fix docs/contributing/search-events-api-patterns.md, which recommended the flag "for latest features".

Tests

New regression tests in both places mock the tags endpoint. With useFlagsBackend=1 the mock returns [], and without it the mock returns sessionId. Both tests failed before this change and pass after it. pnpm run tsc passes for mcp-core, and so does the full mcp-core vitest suite (2247 passed).

Not in this PR

via Sentry.

--

View Session [Sentry]

… event searches

The issue event and errors-dataset field lookups called the org tags
endpoint with useFlagsBackend=1. Sentry then reads the flags backend and
returns feature flag keys instead of tags (organization_tags.py), so custom
tags such as sessionId never reached the embedded agent. The agent, told
that unlisted tags make queries fail, removed valid filters from the query.

Drop the flag from both lookups, remove the now-unused listTags option,
and fix the docs that recommended it.
@vercel

vercel Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
sentry-local Ready Ready Preview Nov 11, 2026 12:34pm UTC

Request Review

@sentry-junior

sentry-junior Bot commented Nov 16, 2026

Copy link
Copy Markdown
Contributor Author

Another report of this pattern: user feedback MCP-SERVER-GQM (2026-11-16, search_errors, server 0.42.0). That makes 3 reports: GNJ, GJX, GQM.

This time the agent didn't drop the valid built-in tags os and device. It renamed them to look-alike fields from the static errors list, and both searches came back empty.

Evidence (sentry/mcp-server spans; there was no mcp.session.id, so I matched by user and client id)

  1. Trace 1128adb0, span 971369a6. The caller sent environment:production device.family:iOS os:"iOS 27.2".
    • The tag lookup (span 903a4153) used GET …/tags/?dataset=events&…&useFlagsBackend=1.
    • datasetAttributes (span b2a0a216) returned exactly 30 fields. That is only the static errors list in config.ts:410-436, with no project tags.
    • The first validateSearch passed with os:"iOS 27.2" (span aaac4e75), but the agent still replaced it with os.name:"iOS 27.2". Its explanation: "corrected the OS field to the verified os.name".
    • The executed search returned 0 rows.
  2. Trace 228356d7, span 8ee8cde1. The caller asked in natural language for "production error events on iPad15,8 devices". The agent wrote device.family:"iPad15,8", also with useFlagsBackend=1 lookups, and the search returned 0 rows.

Why those filters can't match: in getsentry/sentry, src/sentry/interfaces/contexts.py:186-219 sets the device tag to {model} (e.g. iPad15,8) and device.family to {family}. The os tag is the combined name and version (iOS 27.2), and os.name is the name only (iOS).

Does this PR cover it? Mostly. With real tags in the list, os and device count as verified, so the rename in case 1 shouldn't happen. Case 2 may still go wrong, because the static errors list (search-events/config.ts:424-426) describes os.name as "Operating system name" and device.family as "Device family". It has no entries for os (name and version) or device (model). Adding those two with short descriptions would help the agent translate natural language correctly. That could go in this PR or a follow-up.

@sentry-junior

sentry-junior Bot commented Nov 17, 2026

Copy link
Copy Markdown
Contributor Author

Another report of this pattern: user feedback MCP-SERVER-GR2 (2026-11-17, search_errors, server 0.42.0). Reports: 4 (GNJ, GJX, GQM, GR2).

This time the agent couldn't map natural language to a project's custom tag. The project tags errors with http.status_code, which isn't in the static errors field list (search-events/config.ts:409-436; http.status_code there is only under spans). Since the tag lookup goes to the flags backend, the agent never saw it and guessed look-alike keys instead.

Evidence (sentry/mcp-server spans, no mcp.session.id, matched by user and client id)

  • Every errors tag lookup in the session was GET …/tags/?dataset=events&…&useCache=1&useFlagsBackend=1 (search-events/utils.ts:853 on main). In getsentry/sentry, organization_tags.py:124-126 switches to tagstore.flag_backend for that param.
  • Trace 2b18e31a: query HTTP status 503 errors during <start> through <end>; return count()… → executed query=http.status:503 → 0 rows.
  • Trace ec87fd85: same request for a different window → executed query=http.response.status_code:503 → 0 rows.
  • Traces c4006e69, 2c048275, f9885451, 37adf6ab, 99e81f96: when the caller wrote http.status_code:503 explicitly, the executed query kept it and returned events. So explicit tags were preserved; only the NL translation was wrong.

Does this PR cover it? Yes, as far as I can tell: with useFlagsBackend dropped, datasetAttributes lists the project's real tags, including http.status_code. I couldn't replay the lookup against the reporter's org, so whether http.status_code shows up in that org's top tags is inferred, not verified.

Required checks pass and the pattern now has 4 reports, so I'm marking this ready for review.

@sentry-junior

sentry-junior Bot commented Nov 18, 2026

Copy link
Copy Markdown
Contributor Author

Another report of this pattern: user feedback MCP-SERVER-GS4 (2026-11-18, search_issue_events, server 0.42.0). Reports: 5 (GNJ, GJX, GQM, GR2, GS4).

Same symptom as GNJ: a valid custom tag filter was dropped silently, and the tool returned unfiltered events.

Evidence (sentry/mcp-server spans; no mcp.session.id, matched by user and client id)

  • Trace 6ae498fe, span af29b75e: the caller sent search_issue_events with query=school_id:<value> on one issue, includeExplanation=false.
  • The embedded agent called issueEventFields (span 88ad0484), which hit GET …/tags/?dataset=events&statsPeriod=14d&useCache=1&useFlagsBackend=1 (span b87e8390). The Snuba query grouped BY flags_key (span a9dc8a69). The tool returned 34 fields, which is just the static ISSUE_EVENT_TAGS list, so school_id wasn't in it.
  • The result had executedSearch.query: "" and notes: []. 30 events came back with no filter and no warning that the filter was removed.
  • Workaround in the same session: search_errors with the same explicit school_id:<value> (traces 0ce5cd9c, cf74eb51) kept the filter in executedSearch.query and returned matching events.

Does this PR cover it? Yes. Without useFlagsBackend, issueEventFields lists the project's tags, so the agent has no reason to drop school_id. The silent drop (notes: []) is still the separate theme in the "Not in this PR" section: a warning when the agent removes a filter would make this visible even if the tag list is incomplete.

Required checks pass and the pattern has 5 reports, so I'm marking this ready for review and requesting the code owners (@getsentry/OCPO).

@sentry-junior
sentry-junior Bot marked this pull request as ready for review November 18, 2026 16:06
@sentry-junior
sentry-junior Bot requested a review from a team as a code owner November 18, 2026 16:06
@github-actions github-actions Bot added the risk: medium PR risk score: medium label Nov 18, 2026
@sentry-junior

sentry-junior Bot commented Nov 20, 2026

Copy link
Copy Markdown
Contributor Author

Another report of this pattern: user feedback MCP-SERVER-GT9 (2026-11-20, search_issue_events, server 0.42.0). Reports: 6 (GNJ, GJX, GQM, GR2, GS4, GT9).

The agent dropped a built-in error tag this time, server_name, because the flags-backend lookup didn't list it.

Evidence (sentry/mcp-server spans; no mcp.session.id, matched by user and client id)

  • Trace 68e11988, span 952ff89b (16:07:37 UTC): the args had environment:production (server_name:<prefix-a>-* OR server_name:<prefix-b>-*) timestamp:>=… timestamp:<…. The result had executedSearch.query: "environment:production" and notes: []. 4 of the 6 server_name values in the returned events matched neither prefix.
  • Span 98f0eb6c in the same trace: the issueEventFields lookup called tags/?dataset=events&statsPeriod=14d&useCache=1&useFlagsBackend=1. The org had feature flags, so it returned 39 entries: the 34 static ISSUE_EVENT_TAGS plus 5 flag keys. server_name isn't in the static list (search-issue-events/config.ts:166), so the agent never saw it.
  • Span ac5ef4dc (invoke_agent) gives the agent's reason: "The verified issue-event tags did not include server_name, so the server-name filters were removed to avoid a query failure."
  • search_errors kept the same predicates in that session. Its choosePreservingRepairedQuery guard (search-events/search.ts:406) falls back to the caller's query when a repair drops tokens. search_issue_events runs parsed.query as is (catalog/search-issue-events.ts:190), so there's no second line of defense there.

This PR's removal of useFlagsBackend covers this case. One possible follow-up, not for this PR: port the token-preserving guard to search_issue_events so that any tag the lookup misses fails loudly instead of being dropped.

@sentry-junior

sentry-junior Bot commented Nov 21, 2026

Copy link
Copy Markdown
Contributor Author

Another report of this pattern: user feedback MCP-SERVER-GTH (2026-11-21, search_issue_events, server 0.42.0, kind=broken, impact=worked_around). Reports: 8 (GNJ, GJX, GQM, GR2, GS4, GT9, GTA, GTH). MCP-SERVER-GTJ came in 15s later from the same session with the same complaint.

The caller filtered by a project's custom tag context. It says context:<value> was rewritten to transaction:<value> once and to a message search another time, and that !has:context was dropped. It saw this in executedSearch and got around it with tag-value inspection.

Evidence (sentry/mcp-server, no mcp.session.id; matched by user and client.id, client sample rate 0.3):

  • Every sampled search_issue_events call in the session ran issueEventFields → GET /organizations/<org>/tags/?dataset=events&…&useFlagsBackend=1. The response body was 2 bytes ([]), and the tool listed "Available Tags and Fields (34 total)" with an empty "Project-Specific Tags" section, so context was missing (trace b4c37a92, spans b00b239d and a32a18c7).
  • The repair prompt tells the agent that "Using non-existent tags will cause query failures", so an unlisted custom tag can be rewritten or removed.
  • The agent is nondeterministic here. In the 3 sampled calls with context:<value> or !has:context (traces e27f0627, d7eaf152, b4c37a92), it kept the filter and the query returned 3 events. The calls that were rewritten weren't sampled, so I couldn't capture the agent's output for them.
  • search_issue_events runs the agent's query with no check (query = parsed.query || "", catalog/search-issue-events.ts:190 @ e692073), and explanation came back null, so nothing told the caller.

This PR fixes the tag lookup, so context would be listed. The reporter also asks for the rewrite to be reported or prevented. That's still the "Not in this PR" item: there's no guard against dropped or rewritten filters for search_issue_events.

This branch was successfully deployed

2 active deployments
Preview — 16abca90 Deployed Nov 11, 2026 by vercel[bot]
Actions — 16abca90 Deployed Nov 11, 2026 by sentry-junior[bot] via eval #1794
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk: medium PR risk score: medium

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants