Skip to content

fix(divide): enforce requestMaxSize on chunked request bodies - #7820

Open
Sean-Walker0 wants to merge 1 commit into
apache:masterfrom
Sean-Walker0:fix/7816-divide-request-max-size-chunked
Open

Sean-Walker0 wants to merge 1 commit into
apache:masterfrom
Sean-Walker0:fix/7816-divide-request-max-size-chunked

Conversation

@Sean-Walker0

Copy link
Copy Markdown
Contributor

Fixes #7816

Modifications

A positive divide rule requestMaxSize was only compared against the Content-Length header. Chunked or unknown-length requests report -1 for that header, so the comparison succeeded without examining a single body byte — oversized streaming bodies were forwarded, and the limit only protected known-length input (the issue's probe: knownLength=false limit=1 forwardedBytes=2).

For unknown-length requests the limit is now enforced on the actual bytes: the request is decorated with a counting body that aborts the stream once the forwarded size exceeds the configured maximum:

return exchange.mutate().request(new ServerHttpRequestDecorator(exchange.getRequest()) {
    private long forwarded;
    @Override
    public Flux<DataBuffer> getBody() {
        return super.getBody().doOnNext(buffer -> {
            forwarded += buffer.readableByteCount();
            if (forwarded > maxSize) {
                throw new ShenyuException("request entity is too large");
            }
        });
    }
}).build();

The known-length fast path (immediate 413 before forwarding) and all other rule handling are unchanged.

Verifying this change

  • New DividePluginTest#doExecuteRejectsChunkedBodyExceedingRequestMaxSize — posts a 2-byte chunked body (Transfer-Encoding: chunked, no Content-Length) under a requestMaxSize=1 rule; the downstream drains the body and the stream must abort. Red on the pre-fix code (the body was forwarded completely), green after.
  • ./mvnw -pl shenyu-plugin/shenyu-plugin-proxy/shenyu-plugin-divide -am test -B — full module 25/25 green (12 pre-existing DividePluginTest cases + circuit-breaker suite unchanged), checkstyle clean.

Notes

  • Behavior change: chunked/unknown-length requests that exceed a configured requestMaxSize now abort mid-stream instead of being forwarded in full — matching what the same rule already does for known-length bodies.
  • The abort surfaces as an error to the WebFlux pipeline (response typically not yet committed for streaming bodies, so the global error handler can still answer); once the body has partially streamed, connection-level termination is the only correct signal.
  • Codec-level maxInMemorySize aggregation limits are a separate mechanism (returning 413 pre-forwarding) and remain untouched.
  • Orthogonality: no open PR modifies DividePlugin (verified against the file lists of all 120 open PRs).

Make sure that:

  • You have read the contribution guidelines.
  • You submit test cases (unit or integration tests) that back your changes.
  • Your local test passed ./mvnw clean install -Dmaven.javadoc.skip=true (module-scoped: shenyu-plugin-divide with -am, 25 tests + checkstyle green).

A positive divide rule requestMaxSize was only compared against the
Content-Length header. Chunked or unknown-length requests report -1
for that header, so the comparison succeeded without examining a
single body byte and oversized streaming bodies were forwarded
(413 only for known-length input).

For unknown-length requests the limit is now enforced on the actual
bytes: the request is decorated with a counting body that aborts the
stream once the forwarded size exceeds the configured maximum. The
known-length fast path and all other rule handling are unchanged.
Regression test posts a 2-byte chunked body under a 1-byte limit and
asserts the stream is aborted (red on the previous code, which
forwarded it).

@Aias00 Aias00 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes for one leftover artifact and one client-visible inconsistency. The core mechanism is right: enforcing the limit while the body streams is the only way to cover Transfer-Encoding: chunked, and skipping the decorator when getContentLength() >= 0 keeps the existing known-length path untouched.

1. Leftover javadoc fragment (must fix). DividePlugin.java lines 204-206 contain an abandoned comment immediately before the real method javadoc:

    /**
     * forward with circuit b

    /**
     * Forward the request through the plugin chain and record its outcome into

The first /** is never closed on its own, so it swallows into the following block and the rendered javadoc for forwardWithCircuitBreaker now starts with "forward with circuit b". It compiles, but it is clearly a leftover from editing and should be deleted.

2. Chunked rejection surfaces as HTTP 500, not 413. The known-length branch answers with a proper client error:

Object error = ShenyuResultWrap.error(exchange, ShenyuResultEnum.REQUEST_ENTITY_TOO_LARGE);
return WebFluxResultUtils.result(exchange, error);

but the streaming branch throws ShenyuException. GlobalErrorHandler only special-cases IllegalArgumentException and ResponseStatusException; everything else falls through to the default status, so an oversized chunked request reaches the client as a server error. Same condition, two different answers, and the wrong one for a limit the client can fix.

Suggestion: throw new ResponseStatusException(HttpStatus.PAYLOAD_TOO_LARGE, "request entity is too large") from doOnNext — GlobalErrorHandler already honours that exception and returns its status. Alternatively keep ShenyuException but route through ShenyuResultWrap.error(..., REQUEST_ENTITY_TOO_LARGE) on the error path.

3. The counter is decorator-scoped, not subscription-scoped (robustness). forwarded is a field on the decorator instance while getBody() returns a fresh Flux on every call, so a second subscription continues from the previous count instead of restarting. Today this is safe because AbstractHttpClientPlugin calls getCachedRequestBody(exchange) once when retryTimes > 0 and replays the cached buffers (which bypass the decorator), so the body is counted exactly once. But that correctness depends on a consumer two modules away; a per-subscription counter (reset in doOnSubscribe) would make it local and keep working if retry ever re-reads the source body.

4. Test asserts nothing about the failure mode. verifyError() passes for any exception, which is why issue 2 is invisible to the suite. Please assert the type, and ideally the status, e.g. .verifyError(ResponseStatusException.class) plus a check that getStatusCode() is 413.

Everything else looks good — the effectiveExchange threading through the load balancer, attributes and forwardWithCircuitBreaker is consistent, and the test correctly forces the chain to drain the body so the streaming limit actually fires.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Divide requestMaxSize is not enforced for chunked or unknown-length request bodies

2 participants