perf(upload): Preserve compressed TUS uploads when forwarding by vjeranc · Pull Request #6572 · getsentry/relay

vjeranc · 2026-10-27T12:42:45Z

Preserve compressed TUS upload bytes and Content-Encoding when forwarding. Decode and enforce decoded-size limits in the Upload service’s objectstore branch, reusing the existing decompressor and byte counters.

Bound incoming request bytes at every hop. Record the decompression ratio when the input body finishes or is dropped.

Refs INGEST-1222

Follow-up: INGEST-1348 removes the stream-validation workaround after upgrading to a fixed Objectstore client release.

Compressed TUS uploads currently get decompressed and recompressed at forwarding hops. Preserve their bytes and Content-Encoding until they reach the Relay writing to objectstore. Select decompression during route initialization based on objectstore availability. Keep decoded-size validation, upload offsets, signed attachment lengths, and attachment-byte accounting at the storage Relay. Other routes retain decompression. Refs INGEST-1222 Co-authored-by: Codex <noreply@openai.com>

linear-code · 2026-10-27T12:42:53Z

INGEST-1222

Remove the duplicated test router and codec helper. Cover routing, encoding rejection, normalization, and malformed passthrough through actual Relay instances; retain direct classifier and Tower response parity tests. Refs INGEST-1222 Co-authored-by: Codex <noreply@openai.com>

Preserve both changelog entries and add the upload PR reference. Co-authored-by: Codex <noreply@openai.com>

Keep HEAD, PATCH, and the decompressed method fallback together. Register the complete upload route after the other public routes receive decompression, removing the trailing router merge. Refs INGEST-1222 Co-authored-by: Codex <noreply@openai.com>

Select passthrough or configured encoding before constructing the upstream request. Read the encoding config only for decoded bodies, and pass the prepared stream and encoding to the request constructor. Refs INGEST-1222 Co-authored-by: Codex <noreply@openai.com>

Supply the decoded-body marker directly at route initialization, matching the forwarding adapter and removing the redundant handler wrapper. Refs INGEST-1222 Co-authored-by: Codex <noreply@openai.com>

Co-authored-by: Codex <noreply@openai.com>

Clarify that compression overhead can increase payload size and that the decoded-size limit is enforced upstream, with no separate compressed-byte cap. Co-authored-by: Codex <noreply@openai.com>

Keep passthrough bytes, headers, and retry-state assertions local. Leave multi-hop forwarding and final decompression to the existing integration tests. Refs INGEST-1222 Co-authored-by: Codex <noreply@openai.com>

Co-authored-by: Codex <noreply@openai.com>

Convert delivery and upload failures in a small helper using await??. Keep the PATCH handler focused on the operation and preserve logging for both error paths. Refs INGEST-1222 Co-authored-by: Codex <noreply@openai.com>

Apply max_upload_size to incoming bytes at forwarding hops and retain decoded-size checks at the objectstore Relay. Remove the unlimited compressed-body branch and cover wire and decoded rejection for fixed-length and chunked requests. BREAKING CHANGE: Compressed TUS requests exceeding max_upload_size are rejected by forwarding Relays even when their decoded bodies fit. Refs INGEST-1222 Co-authored-by: Codex <noreply@openai.com>

Use an explicit match for strict Content-Encoding parsing and explain whether upload limits count compressed or decoded bytes. Co-authored-by: Codex <noreply@openai.com>

Co-authored-by: Codex <noreply@openai.com>

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit 0d7c0e5. Configure here.}

Co-authored-by: Codex <noreply@openai.com>

jjbayer

Thanks for making the change, this looks like it's almost good to go!

Co-authored-by: Codex <noreply@openai.com>

jjbayer · 2026-10-30T08:00:36Z

+                let stream = BoundedStream::new(
+                    MeteredStream::new(stream, referrer),
+                    0,
+                    config.max_upload_size(),


If the mode has a length / chunk_length, we could still use that as an upper bound instead of max_upload_size, wdyt? In practice a compressed stream should always be smaller than its decoded length.

Not a blocker because the objectstore relay enforces the bounds in the end.

yes, should be smaller, not sure if that's always the case, maybe someone compresses twice or sends an encrypted blob ? even a short string due to overhead compresses to more bytes...

Fair enough, double compression is a realistic scenario 👍

Let's keep max_upload_size then. Might be worth a code comment explaining why we cannot apply the declared length as an upper bound.

jjbayer · 2026-10-30T08:03:04Z

+    let compression = match HttpCompression::from_encoding(encoding) {
+        None => return Either::Right(stream),
+        Some(compression) => compression,
+    };


nit

Suggested change

let compression = match HttpCompression::from_encoding(encoding) {

None => return Either::Right(stream),

Some(compression) => compression,

};

let Some(compression) = HttpCompression::from_encoding(encoding) else {

return Either::Right(stream);

};

jjbayer · 2026-10-30T08:14:58Z

+    use http_body_util::BodyExt;
+
+    let body =
+        http_body_util::StreamBody::new(stream.map(|result| result.map(hyper::body::Frame::data)));
+    let mut request = http::Request::new(body);
+    if let Some(codec) = compression {
+        request.headers_mut().insert(
+            http::header::CONTENT_ENCODING,
+            HeaderValue::from_static(codec.encoding().name().unwrap()),
+        );
    }
+    let request = crate::middlewares::decompress_request(request, UPLOAD_PATCH_PATH.into())
+        .await
+        .expect("the upload encoding has already been validated");
+    request
+        .into_body()
+        .into_data_stream()
+        .map(|result| result.map_err(std::io::Error::other))
+        .boxed()


Creating an artificial request here seems a bit awkward. I get the idea of using the same decompression mechanism as for other endpoints, but I think it would be better here to do the reverse of encode_body, i.e. match on compression and call a DeflateDecoder, GzipDecoder, etc.

That would simplify the code and get rid of the .expect().

yes, i agree, i stopped thinking after we stopped using tower when everything moved to upload.rs

will make sure to support the same behavior.

Select codecs directly in the objectstore upload path and share byte counters between HTTP bodies and streams. Preserve zlib-wrapped deflate, concatenated zstd frames, and reporting on EOF, error, or drop. Co-authored-by: Codex <noreply@openai.com>

Keep the reporter alive with the decoded body or stream instead of emitting at EOF or error. Replace optional reporters with typed guards and verify reporting when retained, partial, and unread streams drop. Co-authored-by: Codex <noreply@openai.com>

HTTP deflate requires a zlib wrapper. Replace the raw encoder and add a regression that decodes the emitted bytes with ZlibDecoder. Co-authored-by: Codex <noreply@openai.com>

Co-authored-by: Codex <noreply@openai.com>

Move the upload changelog entry back to Unreleased Internal after the 26.10.1 release. Co-authored-by: Codex <noreply@openai.com>

jjbayer · 2026-10-30T10:50:24Z

+                let stream = BoundedStream::new(
+                    MeteredStream::new(stream, referrer),
+                    0,
+                    config.max_upload_size(),


Fair enough, double compression is a realistic scenario 👍

Let's keep max_upload_size then. Might be worth a code comment explaining why we cannot apply the declared length as an upper bound.

jjbayer · 2026-10-30T10:51:35Z

+    use http_body_util::BodyExt;
+
+    let body =
+        http_body_util::StreamBody::new(stream.map(|result| result.map(hyper::body::Frame::data)));
+    let mut request = http::Request::new(body);
+    if let Some(codec) = compression {
+        request.headers_mut().insert(
+            http::header::CONTENT_ENCODING,
+            HeaderValue::from_static(codec.encoding().name().unwrap()),
+        );
    }
+    let request = crate::middlewares::decompress_request(request, UPLOAD_PATCH_PATH.into())
+        .await
+        .expect("the upload encoding has already been validated");
+    request
+        .into_body()
+        .into_data_stream()
+        .map(|result| result.map_err(std::io::Error::other))
+        .boxed()


Co-authored-by: Codex <noreply@openai.com>

Keep both upload forwarding and Unreal integration entries under Unreleased Internal. Co-authored-by: Codex <noreply@openai.com>

Co-authored-by: Codex <noreply@openai.com>

Apply CORS to the retained method fallback so OPTIONS requests receive CORS responses. Add coverage for preflight requests without a content encoding. Co-authored-by: Codex <noreply@openai.com>

Co-authored-by: Codex <noreply@openai.com>

Hold back the final decoded chunk until the input stream reaches EOF, so checksum and length errors cannot be skipped by the Objectstore client. Validate zero-length bodies before sending them. Co-authored-by: Codex <noreply@openai.com>

Compile the stream-completion helper only for processing builds and tests. Track its removal after the fixed Objectstore client release with INGEST-1348. Co-authored-by: Codex <noreply@openai.com>

Co-authored-by: Codex <noreply@openai.com>

A codec can finish before its encoded source. Check the underlying reader before reporting EOF to reject trailing input and propagate late source errors. Co-authored-by: Codex <noreply@openai.com>

vjeranc requested a review from a team as a code owner October 27, 2026 12:42

vjeranc marked this pull request as draft October 27, 2026 12:44

vjeranc and others added 9 commits October 27, 2026 13:52

chore(upload): Merge master into upload passthrough branch

1465ba0

Preserve both changelog entries and add the upload PR reference. Co-authored-by: Codex <noreply@openai.com>

ref(upload): Inline the decoded PATCH adapter

7b11f6a

Supply the decoded-body marker directly at route initialization, matching the forwarding adapter and removing the redundant handler wrapper. Refs INGEST-1222 Co-authored-by: Codex <noreply@openai.com>

docs(upload): Clarify the unsupported encoding response

87b3045

Co-authored-by: Codex <noreply@openai.com>

docs(upload): Explain the uncapped compressed forwarding body

8bab534

Clarify that compression overhead can increase payload size and that the decoded-size limit is enforced upstream, with no separate compressed-byte cap. Co-authored-by: Codex <noreply@openai.com>

test(upload): Replace the simulated chain with a request test

4a4da9e

Keep passthrough bytes, headers, and retry-state assertions local. Leave multi-hop forwarding and final decompression to the existing integration tests. Refs INGEST-1222 Co-authored-by: Codex <noreply@openai.com>

ref(upload): Restore descriptive stream bound names

9e4d53f

Co-authored-by: Codex <noreply@openai.com>

vjeranc marked this pull request as ready for review October 27, 2026 13:58

sentry-warden Bot reviewed Oct 27, 2026

View reviewed changes

Comment thread relay-server/src/endpoints/upload.rs Resolved

ref(upload): Restore the flat upload result boundary

3bf2c39

Convert delivery and upload failures in a small helper using await??. Keep the PATCH handler focused on the operation and preserve logging for both error paths. Refs INGEST-1222 Co-authored-by: Codex <noreply@openai.com>

sentry-warden Bot reviewed Oct 27, 2026

View reviewed changes

Comment thread relay-server/src/endpoints/upload.rs Resolved

tobias-wilfert reviewed Oct 28, 2026

View reviewed changes

Comment thread relay-config/src/config.rs Resolved

tobias-wilfert reviewed Oct 28, 2026

View reviewed changes

Comment thread relay-server/src/services/upload.rs Resolved

jjbayer requested changes Oct 28, 2026

View reviewed changes

ref(upload): Clarify encoding parsing and stream bounds

189a714

Use an explicit match for strict Content-Encoding parsing and explain whether upload limits count compressed or decoded bytes. Co-authored-by: Codex <noreply@openai.com>

tobias-wilfert reviewed Oct 28, 2026

View reviewed changes

Comment thread relay-server/src/endpoints/upload.rs Resolved

tobias-wilfert reviewed Oct 28, 2026

View reviewed changes

Comment thread relay-server/src/endpoints/upload.rs Resolved

tobias-wilfert reviewed Oct 28, 2026

View reviewed changes

Comment thread relay-server/src/endpoints/upload.rs Resolved

ref(upload): Centralize decoding

0d7c0e5

Co-authored-by: Codex <noreply@openai.com>

cursor Bot reviewed Oct 28, 2026

View reviewed changes

Comment thread relay-server/src/services/upload.rs Resolved

vjeranc and others added 3 commits October 28, 2026 14:18

docs(upload): Clarify comments

997fa2e

Co-authored-by: Codex <noreply@openai.com>

ref(server): Narrow route builder inputs

da08a79

Co-authored-by: Codex <noreply@openai.com>

ref(upload): Move compression type into server

b77de64

Co-authored-by: Codex <noreply@openai.com>

vjeranc and others added 2 commits October 28, 2026 14:29

ref(upload): Share backend config

2a07c6a

Co-authored-by: Codex <noreply@openai.com>

ref(upload): Rename project context

5fc94bc

Co-authored-by: Codex <noreply@openai.com>

jjbayer reviewed Oct 28, 2026

View reviewed changes

Comment thread tests/integration/test_upload.py Resolved

Comment thread relay-server/src/services/upload.rs Resolved

vjeranc and others added 2 commits October 28, 2026 16:03

ref(upload): Clarify stream alias names

fdc9613

Co-authored-by: Codex <noreply@openai.com>

test(upload): Trim encoding matrix

d8220b1

Co-authored-by: Codex <noreply@openai.com>

vjeranc force-pushed the perf/ingest-1222-upload-passthrough branch from 8bc8011 to d8220b1 Compare October 28, 2026 15:03

jjbayer reviewed Oct 30, 2026

View reviewed changes

vjeranc and others added 6 commits October 30, 2026 11:04

fix(upload): Use zlib framing for HTTP deflate

030d40a

HTTP deflate requires a zlib wrapper. Replace the raw encoder and add a regression that decodes the emitted bytes with ZlibDecoder. Co-authored-by: Codex <noreply@openai.com>

docs(upload): Clarify upload forwarding behavior

f478214

Co-authored-by: Codex <noreply@openai.com>

ref(upload): Use let-else for upload encoding

3c45190

Co-authored-by: Codex <noreply@openai.com>

chore(upload): Merge master into upload passthrough branch

41d99f0

Move the upload changelog entry back to Unreleased Internal after the 26.10.1 release. Co-authored-by: Codex <noreply@openai.com>

jjbayer approved these changes Oct 30, 2026

View reviewed changes

vjeranc and others added 7 commits October 30, 2026 12:49

docs(upload): Explain compressed upload size bounds

beb7070

Co-authored-by: Codex <noreply@openai.com>

chore(upload): Resolve changelog conflict with master

950683c

Keep both upload forwarding and Unreal integration entries under Unreleased Internal. Co-authored-by: Codex <noreply@openai.com>

test(upload): Trim test matrices

010c9dd

Co-authored-by: Codex <noreply@openai.com>

test(upload): Keep complete upload scenarios in cases

263afa6

Co-authored-by: Codex <noreply@openai.com>

fix(upload): Restore CORS handling for preflight requests

8e8332b

Apply CORS to the retained method fallback so OPTIONS requests receive CORS responses. Add coverage for preflight requests without a content encoding. Co-authored-by: Codex <noreply@openai.com>

chore(upload): Merge master into upload passthrough branch

5afe5d1

Co-authored-by: Codex <noreply@openai.com>

vjeranc mentioned this pull request Nov 3, 2026

fix(rs-client): Validate resumable upload streams through EOF getsentry/objectstore#754

Open

vjeranc and others added 3 commits November 3, 2026 14:52

fix(upload): Gate the completion workaround on processing

f46fc4e

Compile the stream-completion helper only for processing builds and tests. Track its removal after the fixed Objectstore client release with INGEST-1348. Co-authored-by: Codex <noreply@openai.com>

test(upload): Reproduce premature decoded stream completion

71bf9c0

Co-authored-by: Codex <noreply@openai.com>

fix(upload): Validate input before completing decompression

163dc94

A codec can finish before its encoded source. Check the underlying reader before reporting EOF to reject trailing input and propagate late source errors. Co-authored-by: Codex <noreply@openai.com>

Conversation

vjeranc commented Oct 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

linear-code Bot commented Oct 27, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jjbayer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

vjeranc commented Oct 27, 2026 •

edited

Loading