Skip to content

fix(idempotency): add deterministic support for set, frozenset, and bytes types - #8741

Open
Neeraj111010 wants to merge 1 commit into
aws-powertools:developfrom
Neeraj111010:fix/8725-idempotency-set-bytes-support
Open

Neeraj111010 wants to merge 1 commit into
aws-powertools:developfrom
Neeraj111010:fix/8725-idempotency-set-bytes-support

Conversation

@Neeraj111010

@Neeraj111010 Neeraj111010 commented Nov 1, 2026 •

Copy link
Copy Markdown

Issue number: closes #8725

Summary

This PR adds native support for set, frozenset, and bytes types to the AWS Lambda Powertools _IdempotencyKeyEncoder. It ensures sets and frozensets are sorted deterministically regardless of insertion order, and encodes bytes using Base64 so they can be securely and reliably hashed and validated in the idempotency utility

Changes

  • Encoder Logic (aws_lambda_powertools/utilities/idempotency/persistence/base.py):
    • Added support for set and frozenset by converting them into sorted lists to guarantee identical hashes across varying element insertion orders, with safe fallback sorting (key=repr) for non-comparable items.
    • Added Base64 encoding support for bytes (base64.b64encode(obj).decode("utf-8")).
  • Test Suite (tests/functional/idempotency/_boto3/test_idempotency_hash_serialization.py):
    • Removed set and bytes from unsupported error test cases.
    • Added test_set_frozenset_and_bytes_support to validate persistent replay, payload validation, and insertion order determinism.

User experience

  • Before: Passing a payload containing set, frozenset, or bytes objects to an idempotent function would raise a TypeError because they were not JSON serializable or produced non-deterministic hashes due to unordered iteration.
  • After: These types are now natively and deterministically serialized, allowing seamless caching and payload validation without requiring custom serializations.

Verification

  • Linting & Formatting: Ran make lint / ruff, passing all style and formatting checks with zero errors or warnings.
  • Test Suite: Ran uv run pytest tests/functional/idempotency/, with 33 functional test cases passing successfully.
  • AI Assistance Note: An AI assistant was used to help structure the implementation logic and write test cases, which were fully verified and tested locally by the author.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

@Neeraj111010
Neeraj111010 requested a review from a team as a code owner November 1, 2026 22:57
@powertools-for-aws-oss-automation powertools-for-aws-oss-automation Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Nov 1, 2026
@boring-cyborg boring-cyborg Bot added the tests label Nov 1, 2026
@boring-cyborg

boring-cyborg Bot commented Nov 1, 2026

Copy link
Copy Markdown

Thanks a lot for your first contribution! Please check out our contributing guidelines and don't hesitate to ask whatever you need.
In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link

@sonarqubecloud

sonarqubecloud Bot commented Nov 1, 2026

Copy link
Copy Markdown

@blacksoxx blacksoxx left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment. The encoder change does what #8725 asks and the two insertion orders hashing to one key is the test that matters; one small gap in the docs.

Reviewed head fc3b4a5a against develop at 3b261550; both unchanged just before posting, PR open. Validation: diff read against the current default(); the key=repr fallback is deterministic for equal contents and an element that is not serialisable still fails when the sorted list is encoded, so no silent hash. Not run here: the idempotency suites. History: #8725 is the maintainers' own issue, assigned to you; no earlier PR for these types (searched "idempotency set frozenset bytes", "idempotency set bytes hash", 100 results per query, capped).

[P3] docs/utilities/idempotency.md line 195 lists which values the hash encoder converts (UUID, paths, dates, enums); sets, frozensets (as sorted lists) and bytes (base64) belong in that sentence now, otherwise the docs still say they are unsupported. No unrelated scope changes.

I would merge this with that line added; the decision is with the maintainers.

@leandrodamascena leandrodamascena added this to the 3.41.0 milestone Nov 4, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/L Denotes a PR that changes 100-499 lines, ignoring generated files. tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: idempotency payloads with set, frozenset or bytes fields cannot be hashed

3 participants