How to send big headers

[marcstern]

I'm trying to send a request with a header containing 90.000 bytes and I receive the error

java.lang.IllegalArgumentException: Request header too large
	at org.eclipse.jetty.client.http.HttpSenderOverHTTP$HeadersCallback.process(HttpSenderOverHTTP.java:182)
	[...]

I launch the JVM with -Dserver.maxHttpHeaderSize=100000 and the same request works in HTTP 1.1 (HttpClient4).
Is there an inherent limit or can I adapt the behaviour?

Thanks a lot

[marcstern]

Some more info: if I send in HTTPS 4 headers of 8 K each, I get the following exception

org.eclipse.jetty.http2.hpack.HpackException$SessionException: Could not hpack encode GET{u=https://vh1.waf-test.approach.be:443/TestSecHeadersSizeMax100k,HTTP/2.0,h=15,cl=-1,p=null}
	at org.eclipse.jetty.http2.hpack.HpackEncoder.encode(HpackEncoder.java:278)
	at org.eclipse.jetty.http2.generator.FrameGenerator.encode(FrameGenerator.java:56)
	at org.eclipse.jetty.http2.generator.HeadersGenerator.generateHeaders(HeadersGenerator.java:65)
	at org.eclipse.jetty.http2.generator.HeadersGenerator.generate(HeadersGenerator.java:52)
	at org.eclipse.jetty.http2.generator.Generator.control(Generator.java:86)
	at org.eclipse.jetty.http2.HTTP2Session$ControlEntry.generate(HTTP2Session.java:1245)
	at org.eclipse.jetty.http2.HTTP2Flusher.process(HTTP2Flusher.java:215)
	at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:232)
	at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:214)
	at org.eclipse.jetty.http2.HTTP2Session.frame(HTTP2Session.java:777)
	at org.eclipse.jetty.http2.HTTP2Session.frames(HTTP2Session.java:749)
	at org.eclipse.jetty.http2.client.HTTP2ClientConnectionFactory$HTTP2ClientConnection.onOpen(HTTP2ClientConnectionFactory.java:116)
	at org.eclipse.jetty.io.AbstractEndPoint.upgrade(AbstractEndPoint.java:451)
	at org.eclipse.jetty.io.NegotiatingClientConnection.replaceConnection(NegotiatingClientConnection.java:117)
	at org.eclipse.jetty.io.NegotiatingClientConnection.onFillable(NegotiatingClientConnection.java:87)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:319)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)
	at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:558)
	at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:379)
	at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:146)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)
	at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:412)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:381)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:268)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:190)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:894)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1038)
	... 1 more
Caused by: java.nio.BufferOverflowException
	at java.base/java.nio.Buffer.nextPutIndex(Buffer.java:725)
	at java.base/java.nio.DirectByteBuffer.put(DirectByteBuffer.java:355)
	at org.eclipse.jetty.http2.hpack.Huffman.encode(Huffman.java:504)
	at org.eclipse.jetty.http2.hpack.Huffman.encode(Huffman.java:434)
	at org.eclipse.jetty.http2.hpack.HpackEncoder.encodeValue(HpackEncoder.java:462)
	at org.eclipse.jetty.http2.hpack.HpackEncoder.encode(HpackEncoder.java:363)
	at org.eclipse.jetty.http2.hpack.HpackEncoder.encode(HpackEncoder.java:249)
	... 28 more

The same request in HTTP (h2c):

 java.io.IOException: frame_size_error/invalid_frame_length
	at org.eclipse.jetty.http2.HTTP2Session.toFailure(HTTP2Session.java:591)
	at org.eclipse.jetty.http2.HTTP2Session$StreamsState.onSessionFailure(HTTP2Session.java:1948)
	at org.eclipse.jetty.http2.HTTP2Session.onSessionFailure(HTTP2Session.java:536)
	at org.eclipse.jetty.http2.HTTP2Session.onConnectionFailure(HTTP2Session.java:531)
	at org.eclipse.jetty.http2.parser.Parser$Listener$Wrapper.onConnectionFailure(Parser.java:409)
	at org.eclipse.jetty.http2.HTTP2Connection$ParserListener.onConnectionFailure(HTTP2Connection.java:414)
	at org.eclipse.jetty.http2.parser.BodyParser.notifyConnectionFailure(BodyParser.java:218)
	at org.eclipse.jetty.http2.parser.BodyParser.connectionFailure(BodyParser.java:210)
	at org.eclipse.jetty.http2.parser.Parser.connectionFailure(Parser.java:204)
	at org.eclipse.jetty.http2.parser.Parser.parseHeader(Parser.java:150)
	at org.eclipse.jetty.http2.parser.Parser.parse(Parser.java:116)
	at org.eclipse.jetty.http2.HTTP2Connection$HTTP2Producer.produce(HTTP2Connection.java:278)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:446)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:239)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:190)
	at org.eclipse.jetty.http2.HTTP2Connection.produce(HTTP2Connection.java:208)
	at org.eclipse.jetty.http2.HTTP2Connection.onFillable(HTTP2Connection.java:155)
	at org.eclipse.jetty.http2.HTTP2Connection$FillableCallback.succeeded(HTTP2Connection.java:378)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)
	at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53)
	at org.eclipse.jetty.util.thread.Invocable.invokeNonBlocking(Invocable.java:151)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.invokeAsNonBlocking(AdaptiveExecutionStrategy.java:429)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:371)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:268)
	at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:190)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:894)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1038)
	... 1 more

[3dgiordano]

Hi @marcstern

We would need to know a bit more about which plugin version you are using, jmeter version and java version.

It also asks you if you configured the maximum buffer size allocated for requests as indicated in the manual.
https://github.com/Blazemeter/jmeter-http2-plugin#buffer-capacity
Try a value that is greater than 90MB to see if it resolves your issue.

We are waiting for the version information.

[marcstern]

Same with -DhttpJettyClient.maxBufferSize=120000000

JMeter 5.5
HTTP/2 Plugin 2.0.2
java 19.0.1 2022-10-18
Java(TM) SE Runtime Environment (build 19.0.1+10-21)
Java HotSpot(TM) 64-Bit Server VM (build 19.0.1+10-21, mixed mode, sharing)
Windows 10 Pro

[3dgiordano]

Hi @marcstern

the use of -D is for "java system property" you need to use the -J for "jmeter property".
https://jmeter.apache.org/usermanual/get-started.html#override
Try with -J or in jmeter.properties file.

[marcstern]

Sorry, I didn't know that difference.
Unfortunately, same with -JhttpJettyClient.maxBufferSize=120000000

[marcstern]

Any other test I could try?

[3dgiordano]

Hi @marcstern

Sorry but looking at the implementation of the used http client, Jetty, the property we are using applies only to the response buffer, and not to the send buffer with headers included.

Currently the plugin does not have a method that allows you to assign that property in the Jetty client.

Thanks for reporting the issue, we're sorry we don't have a solution right now.

[kykyis]

has there been a solution?