|
|
def _build_hint_judge_messages(response_text: str, next_state_text: str, next_state_role: str = "user") -> list[dict]: system = ( "You are a process reward model used for hindsight hint extraction.\n" "You are given:\n" "1) The assistant response at turn t.\n" "2) The next state at turn t+1, along with its **role**.\n\n" "## Understanding the next state's role\n" "- role='user': A reply from the user (follow-up, correction, new request, etc.).\n" "- role='tool': The return value of a tool the assistant invoked. " "This content was NOT available before the assistant's action — " "it exists BECAUSE the assistant called the tool. " "A successful, non-error tool output generally means the assistant's " "action was appropriate; do NOT treat it as information the assistant " "should have already known.\n\n" "Your goal is to decide whether the next state reveals useful hindsight information\n" "that could have helped improve the assistant response at turn t.\n\n" "Output format rules (strict):\n" "- You MUST include exactly one final decision token: \\boxed{1} or \\boxed{-1}.\n" "- If and only if decision is \\boxed{1}, provide a concise, information-dense hint in 1-3 sentences,\n" " wrapped between [HINT_START] and [HINT_END].\n" "- If decision is \\boxed{-1}, do not provide a hint block.\n" "- Hint must be concrete and actionable for improving the previous response." ) user = ( f"## Assistant response (turn t)\n{response_text}\n\n" f"## Next state (turn t+1) [role: {next_state_role}]\n{next_state_text}\n\n" "Now output your decision and (if positive) the hint in the required format." ) return [{"role": "system", "content": system}, {"role": "user", "content": user}]
|
|
发表于