[kaihackney-scale]

Why

When a multi-turn conversation uses a user-sim, only the agent's trajectory is kept. The user-sim's own run isn't. So when a user-sim goes wrong, nobody can see what it did on that turn: what it read, what it reasoned, or which tools it called. Any fix to user-sim behavior has to be checked against exactly that.

What

When the user-sim's card serves urn:agentenv:trajectory/v1, each of its answers' trajectories is stored under <trajectory prefix>/usersim/. Each one is recorded in metadata.usersim_trajectories with:

• the turn;
• the message id the conversation sent it under;
• where it was stored.

It's best effort:

• a failed fetch is logged and the conversation goes on;
• a fetch that stores nothing is logged and not recorded.

Human peers and user-sims without the extension are unchanged. The only cost is one trajectory fetch per user-sim answer, and only when its card offers one.

An earlier revision of this PR also added user_message_template (wrapping the agent's reply) and user_sim_guard (a length/markdown retry). Both are removed:

• across many tasks, a length/markdown check flags ordinary user messages: detailed instructions, pasted emails, lists of files;
• the framing's effect isn't established.

They'll come back, if at all, as part of a measured design.

Tests

• tst/unit/task_step/test_prompt_agent_user_side.py covers three cases:
  • each answer's trajectory is stored under the step's prefix and recorded per turn, under the id its message was sent with;
  • a fetch that fails is tolerated;
  • a fetch that stores nothing is tolerated.
• pytest tst/unit/task_step -k prompt_agent: 232 passed. ruff check . is clean.

End to end

I ran a clone of a real multi-turn prod task with a user-sim on this branch's step code, using agent-env task run --k 1. All 11 steps completed.

• metadata.usersim_trajectories lists all three user-sim turns, each pointing under <trajectory prefix>/usersim/.
• All three objects are in S3 (82.6, 35.7 and 32.2 KB) and parse as the user-sim's Claude Code stream.
• That run also had the since-removed template and guard settings set; the screenshot shows them. The trajectory path doesn't depend on either.

The sandboxes were terminated after the run.

🤖 Generated with Claude Code

Confidence Score: 5/5

The current diff appears safe to merge.

Summary

The current diff adds user-sim trajectory capture to prompt_agent.

• Prompt-agent runs now save user-sim trajectories when available.

_{Reviews (4) · Last reviewed commit: "Merge remote-tracking branch 'framework/..." · Reviewed by Greptile}