Skip to content

fix: keep turn spans parented correctly under event-ordering races - #147

Open
enternal111 wants to merge 9 commits into
DEVtheOPS:mainfrom
enternal111:fix/turn-trace-parenting
Open

enternal111 wants to merge 9 commits into
DEVtheOPS:mainfrom
enternal111:fix/turn-trace-parenting

Conversation

@enternal111

@enternal111 enternal111 commented Nov 14, 2026 •

Copy link
Copy Markdown

Which problem is this PR solving?

OpenCode may deliver session.step.started and fire the model.request hook before the queued session.execution.started event is consumed, and it can start the next step's request while the previous step's span is still finishing. Both event-ordering races produce broken traces:

  1. LLM spans started before the run span exists become roots of their own traces, splitting one turn into several traces (the run span exports alone and the gateway spans land in a separate trace).
  2. The model.request hook then propagates the still-active previous step's span context to the next request, so gateway spans are parented to a span that already ended (children outliving their parent).
  3. Auxiliary (non-primary) model requests such as title generation receive no explicit context, so the HTTP layer stamps its own native traceparent and each such request forks a gateway-only trace.

Short description of the changes

  • ensureRunSpan() creates the run span from the model.request hook and from step.started when execution.started has not been consumed yet, anchored to the enqueued prompt time so performance views keep the pre-request lead time; execution.started now enriches an eagerly created run span instead of starting a duplicate.
  • resolveRunContext() falls back to the parent session's run span so early subagent spans join the parent turn's trace.
  • The model.request hook refuses to reuse an active entry whose span belongs to an already-ended or previous step (tracked via isRecording() and a fromStep marker), and injects the run span's context into auxiliary requests so they join the turn instead of forking.
  • finalizeSession() ends a dangling run span when an execution ends.

How Has This Been Tested?

  • bun run typecheck
  • bun run lint
  • bun test (unit suite green; the one adjusted test now asserts auxiliary requests anchor to the run span rather than receiving no context at all)

Depends on #146 (branch is stacked; only the last commit is new here).

Fixes #152

…URE_MESSAGE_CONTENT

Span attributes that embed user or model content (input.value,
output.value, llm.input_messages, llm.output_messages, session run
span prompt text, and tool parameters/inputs/outputs) are now
emitted only when capture is explicitly enabled, either via the new
captureContent plugin option or the standard
OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT environment
variable (true/1). The default is privacy-first: content is not
captured, matching the OpenTelemetry GenAI semantic-conventions
guidance for message content.

Fixes DEVtheOPS#144
@enternal111
enternal111 force-pushed the fix/turn-trace-parenting branch from 77deb2c to ac7c647 Compare November 14, 2026 11:50
@enternal111

Copy link
Copy Markdown
Author

@codex review

@enternal111

Copy link
Copy Markdown
Author

@codex security review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ❌ Failed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ❌ Failed 2026-11-16T11:58:08.436949Z
We couldn’t complete this review. Comment @codex review to try again.
db85949 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ac7c64720e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/handlers/chat-headers.ts Resolved
Comment thread src/config.ts Resolved
Comment thread src/handlers/chat-headers.ts Resolved
…option

Follow the repository convention that plugin settings use the OPENCODE_
namespace. Document captureContent in the README environment variable and
plugin option tables, and update the privacy note to cover span content.
@enternal111
enternal111 force-pushed the fix/turn-trace-parenting branch from ac7c647 to d40b519 Compare November 15, 2026 06:58
@enternal111

Copy link
Copy Markdown
Author

@codex review

@enternal111

Copy link
Copy Markdown
Author

@codex security review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d40b51903f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/handlers/tool.ts Resolved
Cap the JSON.stringify of tool input to 64 KB, matching the bound the
step handler already applies to streamed text.
@enternal111
enternal111 force-pushed the fix/turn-trace-parenting branch from d40b519 to b0b15c8 Compare November 15, 2026 07:09
@enternal111

Copy link
Copy Markdown
Author

@codex review

@enternal111

Copy link
Copy Markdown
Author

@codex security review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b0b15c8b31

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/handlers/chat-headers.ts Resolved
Comment thread src/handlers/session.ts Resolved
Comment thread src/util.ts Resolved
Comment thread src/handlers/chat-headers.ts Resolved
@enternal111

Copy link
Copy Markdown
Author

@codex review

@enternal111

Copy link
Copy Markdown
Author

@codex security review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Something went wrong. Try again later by commenting “@codex review”.

Setup repeatedly failed. Update the setup script or commit, then try again later.
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Re-encode the truncated text so span attributes always carry valid JSON,
and remove the remaining test comment.
@enternal111
enternal111 force-pushed the fix/turn-trace-parenting branch from b0b15c8 to 427da59 Compare November 15, 2026 07:29
@enternal111

Copy link
Copy Markdown
Author

@codex review

@enternal111

Copy link
Copy Markdown
Author

@codex security review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 427da59bec

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/handlers/session.ts Resolved
Comment thread src/handlers/session.ts Resolved
@enternal111

Copy link
Copy Markdown
Author

@codex review

@enternal111

Copy link
Copy Markdown
Author

@codex security review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c922bc83be

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/handlers/chat-headers.ts Resolved
@enternal111

Copy link
Copy Markdown
Author

@codex review

@enternal111

Copy link
Copy Markdown
Author

@codex security review

@enternal111

Copy link
Copy Markdown
Author

@codex security review

@enternal111
enternal111 force-pushed the fix/turn-trace-parenting branch from c9ffe09 to 99debb5 Compare November 16, 2026 02:37
@enternal111

Copy link
Copy Markdown
Author

@codex review

@enternal111

Copy link
Copy Markdown
Author

@codex security review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 99debb50ae

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/handlers/session.ts Resolved
@enternal111

Copy link
Copy Markdown
Author

@codex review

@enternal111

Copy link
Copy Markdown
Author

@codex security review

@enternal111

Copy link
Copy Markdown
Author

@codex review

@enternal111

Copy link
Copy Markdown
Author

@codex security review

@enternal111

Copy link
Copy Markdown
Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: 99debb50ae

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@enternal111
enternal111 force-pushed the fix/turn-trace-parenting branch from 99debb5 to 1781a6e Compare November 16, 2026 08:03
@enternal111

Copy link
Copy Markdown
Author

@codex review

@enternal111

Copy link
Copy Markdown
Author

@codex security review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1781a6e84b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/util.ts
Comment on lines +39 to +40
.replace(/\b(?:for|with|query|search(?:ing)?\s+for)\s+.+$/i, " for [redacted]")
.slice(0, 128)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Omit error text that this regex cannot redact

When captureContent is false and a provider error echoes user text without one of these markers (for example, ProviderError: private prompt), this replacement does nothing and the following 128-character slice still exports the echoed text in span error status or attributes. Provider error formats are arbitrary, so omit the message or export only an allowlisted summary when capture is disabled.

Useful? React with 👍 / 👎.

@enternal111

Copy link
Copy Markdown
Author

@codex review

@enternal111

Copy link
Copy Markdown
Author

@codex security review

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@enternal111

Copy link
Copy Markdown
Author

@codex review

@enternal111

Copy link
Copy Markdown
Author

@codex security review

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@enternal111

Copy link
Copy Markdown
Author

@codex review

@enternal111

Copy link
Copy Markdown
Author

@codex security review

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: db85949721

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +108 to +112
while (Date.now() < deadline) {
await new Promise((resolve) => setTimeout(resolve, ACTIVE_LLM_SETTLE_TICK_MS))
await ctx.tracing.eventQueue
const current = ctx.tracing.activeLlm.get(event.sessionID)
if (!current || current.spanContext.spanId !== active.spanContext.spanId) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Skip the settle delay for an already-current step span

When session.step.started for the same call has already been consumed before model.request, active is already that call's span and there is no queued transition to change its ID. This loop still sleeps in 15 ms increments until the 200 ms deadline, delaying every request in this ordering before it reaches the provider, including retries. Only wait when a previous-step transition is actually pending instead of polling an unchanged current span.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: model.request can propagate the previous step's LLM span context to the next step's gateway request

1 participant