Skip to content

fix: validate RFT rewards before reporting rollout success - #6572

Open
luyuzhe111 wants to merge 1 commit into
aws:masterfrom
luyuzhe111:codex/rft-reward-validation
Open

luyuzhe111 wants to merge 1 commit into
aws:masterfrom
luyuzhe111:codex/rft-reward-validation

Conversation

@luyuzhe111

Copy link
Copy Markdown

Summary

sagemaker_rft_handler currently reports dictionary results without a reward as ready without submitting a reward. Invalid reward values can also trigger ready before reward submission fails.

Validate rewards before reporting success. Accept finite integers and floats, or non-empty lists of those values; reject missing values, None, booleans, non-numeric values, and non-finite floats. Invalid rewards use the existing report_error path to report failed with reward 0. The handler's return value, explicit error/skipped handling, public signatures, and feedback exception policy remain unchanged.

This tightens automatic reporting: a decorated handler that manually reports feedback but returns a dictionary without a valid reward will now also attempt to report failed.

Validation

  • RFT unit tests: 36 parameterized cases passed across synchronous and asynchronous handlers. Tests run the real decorator and feedback client, mocking only HTTP submission to verify the complete feedback request sequence and unchanged return values.
  • Black and flake8 passed for both changed Python files.

@luyuzhe111

luyuzhe111 commented Nov 7, 2026 •

Copy link
Copy Markdown
Author

cc @992X @SifeiLi for review. issue identified by @qingliz in awslabs/agentcore-rl-toolkit#209

This branch is waiting to be deployed

1 waiting deployment
manual-approval — da6d95a8 Waiting Nov 7, 2026 by luyuzhe111 via wait-for-approval #1917
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants