[sarah-chitty-1]

Additions to the built-in user sim (deploy_user_sim_agent):

• the user's first message can attach files (FD-5494);
• a new step, rate_user_satisfaction, asks the user sim to rate the outcome once the run is over (FD-5491, Anisha's rubric A5);
• two optional persona sections, already_told and should_already_know (from Anisha's task ideas);
• a tagged_lines map in the record, so every tag in the user sim's used resolves to its line.

Tasks that use none of these are unchanged, byte for byte: the golden tests that pin the user sim's prompt and reads pass untouched.

Files in the first message (FD-5494)

• opening_prompt takes a string, as before, or a list of text and file parts, the same shape as prompt_agent's parts. Any file type. A file part may carry a one-line metadata.description for the user sim.
• always_on sends the files in place on the first wake. They're part of the user's first turn in the graded conversation; the description is removed before the agent sees it.
• The user sim can't open files. Its prompt and its first read say what it attached, e.g. (you attached itinerary.pdf: Mom and Dad's flight itinerary), or just the name. One more line says it knows only that description, not what's in the file; asked about the contents, it says it doesn't remember or points the assistant to the file.
• always_on's timeline wake event lists the attached files (files: [{name, uri}]).
• prompt_agent with the built-in user sim: the check that its prompt matches the opening now compares the files too (same files, same order, after seeding).
• Seed placeholders are filled in a file's uri and name. Sharing goes through the usual path.
• The record keeps opening as the joined text and adds opening_parts; training's _first_prompt uses the joined text.

The rating step (FD-5491)

rate_user_satisfaction runs after an always_on or prompt_agent run and sends the user sim one private message in the same session, so it remembers the run. The message has:

• any agent messages it hadn't read yet;

• the task's satisfaction lines ("HOW YOU'D RATE THE JOB", tagged S1…);

• a 1–5 ask: 5 perfect, 3 fine, 1 "you'd have done it yourself". It adds "Something you were never told was done doesn't count as done."

• The satisfaction lines live on the rating step, never in the persona, so the user sim doesn't see them during the run and the run behaves as tested.

• The rating goes to metadata["user_sim_satisfaction"][<run step id>], with the full ask in the object store. It never reaches the agent, the graded conversation or verifications.

• A run that ended in an error isn't rated; skipped records why. A failed rating never fails the task (fail_task_on_error false).

• The step is refused at load if it has no run step, several and no run_step_id, a user that isn't a built-in user sim, or bad lines.

• One always_on change: its record now keeps the user sim's session id (user_sim.context_id), so the step can continue that session.

Two optional persona sections

Both go in description, after what_you_know, each line tagged like the others:

field	tag	what the user sim does
already_told	T	Facts the user gave the assistant before this task. Asked for one again, it reminds the assistant, a little annoyed, that it already said, and gives it again.
should_already_know	C	What the assistant should know without being told. It never brings these up; if the assistant gets one wrong, it corrects it briefly, a little irritated, and adds nothing else.

Each should_already_know fact must also be in the assistant's own instructions or data, or the task is unfair.

tagged_lines

The record at metadata["user_sims"][name] now has tagged_lines: each tag the prompt shows with its line as written ({"W1": "...", "D9": "..."}). An answer's used tags resolve through it, so they stay readable when sections are added or reordered. The rating step records its own S tags the same way.

Tests

test_user_sim_opening_files.py (26) and test_user_sim_satisfaction.py (50), plus one in harness_test.py. tst/unit/task_step, task/episode, training and cli: 3,824 passed; the two test_mcp_cli_builder tests that need click were deselected. No paid runs yet. The rating's wording gets a small replay test (good / fine / bad outcomes against Sarah's own ratings) before tasks use it.

🤖 Generated with Claude Code

Confidence Score: 5/5

The PR appears safe to merge; both earlier findings are fixed and no new blocking issue remains.

Summary

Adds opening-message files, a private rating step after a run, optional persona sections, and recorded tag-to-line maps.

• A user's first message can include files for the agent.
• A new step asks the user sim to rate the finished run.
• The user sim can remember what it already told the assistant.
• Each recorded user-sim tag keeps the line it referred to.

_{Reviews (10) · Last reviewed commit: "feat(user_sim): two optional description..." · Reviewed by Greptile}