Skip to content

Fix/fix service object handling - #632

Merged
semx merged 10 commits into
bb-Ricardo:developmentfrom
ekrichbaum:fix/fix-service-object-handling
Nov 15, 2026
Merged

semx merged 10 commits into
bb-Ricardo:developmentfrom
ekrichbaum:fix/fix-service-object-handling

Conversation

@ekrichbaum

Copy link
Copy Markdown

This adjusts a resource where it can't find an existing object by aws_resource_id. Adds a fallback if there is exactly one object with the same name and parent, it adopts that object only if it has no AWS resource id, its service is missing or matches, and is already owned by this AWS source (through source reference). the handler then updates it with the AWS data instead of treating it as a name collision and skipping the resource.

The intent was to recover existing AWS tagged obects missing the identity fields without accidentally taking over another source's object.

Removed json_safe function to simplify AWSHandler class.
Removed test for EKS timestamps handling and related assertions.
Added information about S3 bucket tagging and its implications for inventory roles.
Add json_safe function to convert AWS timestamps to UTC ISO 8601 strings for NetBox JSON fields.
Add tests for AWS service metadata handling and legacy resource adoption.
Clarified the purpose of AWS metadata and timestamp handling.
@ekrichbaum

Copy link
Copy Markdown
Author

had to add back in a fix for the timestamping. It got lost in the shuffle of revamping back to multiple PRs

@semx

semx commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator

Thanks for splitting this out. The adoption is much narrower now, and the S3 collision cases are back with objects that don't carry the source tag, which is what I was after.

Two of the three conditions have no test yet. If I drop identity_missing or service_matches from the if, all 200 AWS tests still pass. Could you add the cases where an object with the same name, parent and source tag is not adopted: one that already has a different aws_resource_id, and one with a different aws_service? Both should still end in the name collision.

Smaller things:

  • docs: "The policy includes s3:GetBucketTagging for optional S3 tag synchronization" and the next paragraph say opposite things. Since the tags are always requested, I'd keep just the second paragraph.
  • please keep the old json_safe() docstring. The reason for UTC (the local timezone of the host must not change the stored value) is the part worth keeping.
  • what setup left you with source-tagged objects without aws_resource_id? A sentence in the description is enough. It's what someone reading the adoption branch later will want to know.

@ekrichbaum

ekrichbaum commented Nov 13, 2026 •

Copy link
Copy Markdown
Author

very difficult to recall the way these were adjusted and pulled apart because that's why the patch in the first place. I'll go back through and look at the items, but they'll be in context with the other changes more than likely. I don't think I'll get to it in the next few hours so will probably be tomorrow.

The the missing resource id, it came apart from my original code. I couldn't pinpoint which situation created the issue but it did occur in multiple passes. One note is that it takes several hours to run each of my tests due to the volume of data but a small change like a doc update should be easy.

ekrichbaum and others added 4 commits November 14, 2026 14:06
Removed optional S3 tag synchronization details and clarified the behavior of current S3 discovery regarding bucket tags.
Refactor AWS service tests to improve timestamp handling and add new test for JSON safe normalization.
Updated json_safe function docstring to clarify its purpose and added comments for better understanding.
@semx
semx merged commit 9e63205 into bb-Ricardo:development Nov 15, 2026
1 check passed
@semx

semx commented Nov 15, 2026

Copy link
Copy Markdown
Collaborator

Thanks. I pushed one commit on top before merging. It puts back test_eks_timestamps_are_stored_as_json, which checks the metadata read back from NetBox; the new json_safe() unit test doesn't cover that. It also adds back a test where the adoption actually happens: without it all tests passed with the adoption switched off. And it rewrites the service_object() docstring. Merged as a squash.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants