[ghazariann]

Opening this as a proposal rather than a merge request.

Replaces the log_fn added in #197 with a single observation callback on the transparent proxy. It receives the request method, host, path, headers and the ACL verdict, and can opt in to the request body as it streams. A response slot does the same for the upstream response, once its status and headers are known.

Updated per the review: the body is teed chunk by chunk instead of buffered, and the two callbacks are collapsed into one. The response direction uses the same tee, so it is covered here rather than left as a follow-up.

New API

Builder method	Effect
http_observer(Arc<dyn Fn(&HttpRequestEvent) -> HttpSinks + Send + Sync>)	Optional. Called once per request that reaches the proxy, after the ACL has decided it. The returned HttpSinks carries an optional request sink and an optional response hook; empty observes metadata only. Spawns the proxy even without ACL rules, the same way http_log_fn did.

pub struct HttpRequestEvent<'a> {
    pub method: &'a str,
    pub host: &'a str,
    pub path: &'a str,
    pub headers: &'a http::HeaderMap,
    pub allowed: bool,
}

pub type HttpBodySink = Box<dyn FnMut(&[u8]) + Send + Sync>;

pub struct HttpResponseEvent<'a> {
    pub status: u16,
    pub headers: &'a http::HeaderMap,
}

pub type HttpResponseHook = Box<dyn FnOnce(&HttpResponseEvent) -> Option<HttpBodySink> + Send>;

#[derive(Default)]
pub struct HttpSinks {
    pub request: Option<HttpBodySink>,
    pub response: Option<HttpResponseHook>,
}

No new CLI flags. Off by default (None).

The observer runs after the ACL check so it can report the verdict, and every request that reaches the proxy is observed, including ones the ACL goes on to block.

Rather than carrying Option<&[u8]> in the event, the callback returns a sink. That gives correlation and end of stream for free: the sink belongs to one request, and it is dropped when that request's body ends, so a consumer can finalize from Drop without a second callback.

Why this is useful

One possible usage is in downstream crates: the HTTP request body of the child application can be intercepted to give more observability, for example access to each prompt an agent makes.

let policy = Sandbox::builder()
    .http_allow("POST api.openai.com/*")
    .http_inject_ca("/etc/ssl/certs/ca-certificates.crt")
    .http_observer(Arc::new(|_event| HttpSinks {
        request: Some(Box::new(|chunk: &[u8]| { /* append to a file */ })),
        response: None,
    }))
    .build()?;

Discussion

Does this belong in core? Honestly, this is not needed for learn as a test-before-run tool. Where it is useful is learn used purely as an observability tool, to see what resources a child process accessed and what syscalls and HTTP calls it made. It has to live in core either way, since the body only exists inside AclService::handle and a downstream crate has no way to reach it. Is this too divergent from agent sandboxing, and does it make the codebase less focused?

Response bodies:: covered here, using the same tee. The response slot holds a hook rather than a sink because a response's status and headers do not exist when the observer runs, so that metadata has to arrive later: the hook is called once, when it does, and may then return a sink or decline. A consumer needs that before the first chunk, not with it, to build a streaming decoder from Content-Encoding, to route or skip by Content-Type, or to keep failures and drop large successful bodies. A request the ACL denied has no upstream response, so the hook is never called for one, though the child still receives sandlock's 403.

I know this complicates the API. The observer now returns a struct whose response slot is a closure that returns another closure. Is there a better shape for the response direction?

Tests

• test_http_observer_records_requests: the observer records method, host and path for a forwarded request
• test_http_observer_sink_receives_body: the sink receives the exact bytes the child sent, and the upstream server receives the same unmodified bytes
• test_http_observer_reports_denied_request: a request the ACL blocks is reported with allowed false and still returns 403
• test_http_observer_sink_receives_streamed_chunks: a 1 MB body reaches the sink as several frames, summing to the full body, and the upstream receives all of it
• test_http_observer_response_hook_tees_body: the hook sees the upstream status and its sink receives the response body

🤖 Generated with Claude Code

[congwang-mk]

Thanks for the proposal. The use case makes sense and the hook does have to live in core, but I'd like a different shape:

1. Don't buffer. collect() reads the whole body into supervisor memory before the cap applies, so a sandboxed child can OOM the supervisor. It also turns every upload into a buffered one and makes denied requests read their body before the 403. Tee chunks to the observer as they stream through instead; no cap needed.
2. One observer, not two. Replace http_log_fn with a single callback taking an event struct (method, host, path, headers, ACL verdict, optional body chunks). Pre-1.0, so no need to keep both. Headers matter: without Content-Encoding a gzipped body is unreadable.

Response bodies can follow the same tee approach later without breaking SSE.

[congwang-mk]

@ghazariann any update? thanks.

[ghazariann]

The Rust musl (x86_64) failure seems to be unrelated to this change: apt-get could not resolve the Ubuntu mirrors, so it just needs a re-run.