Repository navigation
feat(s3): add CORS, web console, and SigV4 verification to the standalone image - #19
Conversation
…image CORS: PutBucketCors/GetBucketCors/DeleteBucketCors, preflight, and response headers, plus DEVCLOUD_S3_CORS_ALLOW_ORIGINS as a default for buckets without their own rules. Nothing is allowed unless configured. Web console: embedded React app and JSON API on DEVCLOUD_S3_CONSOLE_ADDR (9001 in the image). Mutations go through the S3 provider protocol. Strict mode adds HTTP Basic; a Host allowlist stops DNS rebinding. SigV4 in strict mode: verify aws-chunked chunk and trailer signatures, enforce a 15-minute clock skew and credential scope date, require host and behavioural x-amz-* headers to be signed, and compare presigned signatures in constant time. The binary refuses an unknown auth mode or strict mode without credentials. Fixes: - Keys with spaces, non-ASCII, or reserved characters failed strict auth because the canonical URI was double-encoded. - aws-chunked framing was stored as part of the object in relaxed mode. - A non-ASCII x-amz-date panicked while holding the store lock. - Transfer-Encoding: chunked bodies were read as empty; conflicting Content-Length values are now refused.
Builds the standalone image and exercises it through real sockets, Chromium, and the AWS SDK for JavaScript: CORS from a browser, signed requests, the web console, image startup, and the existing S3 acceptance gate.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 51fdbf484a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const parts = prefix.split('/').filter(Boolean) | ||
| return parts.map((part, index) => ({ label: part, prefix: `${parts.slice(0, index + 1).join('/')}/` })) |
There was a problem hiding this comment.
For valid S3 keys containing consecutive or leading slashes, filtering out empty segments changes the prefix that breadcrumb buttons navigate to. For example, browsing a//b/ and clicking its b breadcrumb navigates to a/b/, a different object namespace; subsequent uploads then target that different prefix and can overwrite objects there. Build breadcrumb destinations from slices of the original prefix rather than reconstructing them from nonempty segments.
Useful? React with 👍 / 👎.
Shares the accept retry policy main added to the S3 listener.
a//b/ and a/b/ are different locations, so a breadcrumb rebuilt from non-empty segments could lead an upload to the wrong prefix.
Summary
Adds CORS, a web console, and complete SigV4 verification to the standalone S3 image (
services/s3/Dockerfile). Usage and settings are inservices/s3/README.md.PutBucketCors/GetBucketCors/DeleteBucketCors, preflight, and response headers.DEVCLOUD_S3_CORS_ALLOW_ORIGINSsets a default for buckets without their own rules. Nothing is allowed unless configured.DEVCLOUD_S3_CONSOLE_ADDR(9001 in the image). Mutations go through the S3 provider protocol. Strict mode adds HTTP Basic, and a Host allowlist stops DNS rebinding.aws-chunkedchunk and trailer signatures, enforces a 15-minute clock skew and the credential scope date, and requireshostand behaviouralx-amz-*headers to be signed. The binary refuses an unknown auth mode, or strict mode without credentials.npm run test:e2einweb/s3-consolebuilds the image and tests it through real sockets, Chromium, and the AWS SDK for JavaScript.Bugs fixed
aws-chunkedframing was stored as part of the object in relaxed mode, corrupting uploads that use trailing checksums.x-amz-datepanicked while holding the store lock.Transfer-Encoding: chunkedbodies were read as empty, and repeatedContent-Lengthvalues could overwrite an object with the wrong bytes. Conflicting values are now refused.Behaviour changes to review
route_with_corsanswersOPTIONSbefore signature checks, because browsers never sign a preflight.OPTIONSused to return 405 on buckets and objects.services/s3/tests/http_parity.rssigns with the current time instead of a fixed date.127.0.0.1unless strict mode is on.Test results
cargo test --workspaceVERIFY_STAGE=full bash scripts/s3-autoloop/verify.shnpm run test:e2euse casesThe base commit's end-to-end results matched the values predicted before running them, except the two predictions that were corrected and rerun: the unreachable console renders a blank page, and the existing gate stops early when the new tests are overlaid on the base commit because they do not compile there.
Known limits
x-amz-checksum-crc32are accepted but not validated.DeleteObjectsandPostObjectstill return 405, as before. SigV2 and SigV4a are not implemented.display:flex, so its bottom border sits slightly lower than the rest of the row. It is cosmetic and not fixed here.