Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EagerContentHandler. #9051 #12077

Merged
merged 84 commits into from
Nov 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
84 commits
Select commit Hold shift + click to select a range
5d1a9f6
Revert delayed dispatch handling to the connections.
gregw Jul 23, 2024
8c07278
Revert delayed dispatch handling to the connections.
gregw Jul 23, 2024
548770c
Cleanup of HttpConnection
gregw Aug 6, 2024
ec0d325
Delay until MultiPartFormData
gregw Aug 7, 2024
fb85057
Merge remote-tracking branch 'origin/jetty-12.1.x' into experiment/je…
gregw Aug 7, 2024
da5f09f
Wait for release
gregw Aug 7, 2024
5993364
fixed removal of deprecated methods
gregw Aug 7, 2024
3ef3d6e
fixed no mimeType
gregw Aug 7, 2024
5ede88a
PR #12077 - add test for DelayedHandler with multipart
lachlan-roberts Aug 8, 2024
e8ab483
Merge remote-tracking branch 'origin/experiment/jetty-12.1.x/delayedD…
lachlan-roberts Aug 8, 2024
9e1f431
updates from review
gregw Aug 22, 2024
e0f5b7e
fixed test
gregw Aug 22, 2024
b7c8bcb
Merge remote-tracking branch 'origin/jetty-12.1.x' into experiment/je…
gregw Sep 12, 2024
d4dcd6a
WIP updates from review
gregw Sep 17, 2024
9eb9e7d
WIP updates from review
gregw Sep 18, 2024
72ab339
WIP updates from review
gregw Sep 18, 2024
e45ecad
WIP updates from review
gregw Sep 18, 2024
cac0d62
Merge branch 'jetty-12.1.x' into experiment/jetty-12.1.x/delayedDispatch
gregw Sep 18, 2024
a585cfc
WIP updates from review
gregw Sep 18, 2024
8c33aa4
Use lowercase for charsets #11741
gregw Oct 3, 2024
ccec3e4
Use lowercase for charsets #11741
gregw Oct 4, 2024
1001c31
Use lowercase for charsets #11741
gregw Oct 4, 2024
8c25183
Use lowercase for charsets #11741
gregw Oct 4, 2024
3d7f5da
Use lowercase for charsets #11741
gregw Oct 6, 2024
6c0b6f9
Use lowercase for charsets #11741
gregw Oct 6, 2024
c87adb6
javadoc
gregw Oct 9, 2024
1172d59
updates from review
gregw Oct 14, 2024
7913cbb
updates from review
gregw Oct 14, 2024
33aaadb
Merge branch 'jetty-12.1.x' into experiment/jetty-12.1.x/delayedDispatch
gregw Oct 14, 2024
2cb7da6
Merge branch 'fix/jetty-12.1.x/11741/mimetypes' into experiment/jetty…
gregw Oct 14, 2024
4c5be88
WIP
gregw Oct 15, 2024
bb37636
Experiment to reuse buffer in HttpConnection to make retaining chunks…
gregw Oct 15, 2024
af81974
Experiment to reuse buffer in HttpConnection to make retaining chunks…
gregw Oct 15, 2024
907da62
Merge branch 'jetty-12.1.x' into experiment/jetty-12.1.x/delayedDispatch
gregw Oct 16, 2024
b48cfda
fixed mimetype lookup
gregw Oct 16, 2024
039e1c9
Merge remote-tracking branch 'origin/jetty-12.1.x' into experiment/je…
gregw Oct 21, 2024
ba98543
Delay content until 75% of an input buffer is read.
gregw Oct 22, 2024
472233b
Delay content until 75% of an input buffer is read.
gregw Oct 22, 2024
b32294e
Merge remote-tracking branch 'origin/jetty-12.1.x' into experiment/je…
gregw Oct 22, 2024
0dd3279
updates from review
gregw Oct 23, 2024
346adea
updates from review
gregw Oct 23, 2024
da8e70f
updates from review
gregw Oct 23, 2024
8ded76f
Merge remote-tracking branch 'origin/jetty-12.1.x' into experiment/je…
gregw Oct 24, 2024
847f06e
Merge branch 'jetty-12.1.x' into experiment/jetty-12.1.x/delayedDispatch
gregw Oct 24, 2024
a9b1578
updates from review
gregw Oct 24, 2024
353a350
Merge branch 'jetty-12.1.x' into experiment/jetty-12.1.x/delayedDispatch
gregw Oct 29, 2024
2fc4aa1
configurable space
gregw Oct 29, 2024
f400278
update javadoc and updates from review
gregw Oct 30, 2024
4692460
Merge remote-tracking branch 'origin/jetty-12.1.x' into experiment/je…
gregw Oct 30, 2024
775bef8
Merge branch 'jetty-12.1.x' into experiment/jetty-12.1.x/delayedDispatch
gregw Oct 31, 2024
ecbf249
Merge branch 'jetty-12.1.x' into experiment/jetty-12.1.x/delayedDispatch
gregw Oct 31, 2024
8fe31fe
Merge remote-tracking branch 'origin/jetty-12.1.x' into experiment/je…
gregw Nov 4, 2024
0543ae0
Implement non-compact algorithm in HTTP and HTTP/2
gregw Nov 4, 2024
687a7bc
Implement non-compact algorithm in HTTP and HTTP/2
gregw Nov 4, 2024
7cae30e
Merge remote-tracking branch 'origin/jetty-12.1.x' into experiment/je…
gregw Nov 4, 2024
1657145
fixed comment
gregw Nov 5, 2024
a9c709b
Merge remote-tracking branch 'origin/jetty-12.1.x' into experiment/je…
gregw Nov 5, 2024
acfba41
improved comments
gregw Nov 5, 2024
1e838f7
improved comments
gregw Nov 6, 2024
a23c8f4
updates from review
gregw Nov 7, 2024
0e9c9ec
Merge remote-tracking branch 'origin/jetty-12.1.x' into experiment/je…
gregw Nov 7, 2024
1f89ff5
Fix test leaks
gregw Nov 7, 2024
18f655c
Fix test leaks
gregw Nov 7, 2024
73e21e7
updates from review
gregw Nov 8, 2024
278e925
Implemented for H3
gregw Nov 9, 2024
4685621
Updates from review
gregw Nov 9, 2024
4a0e1c1
Configurable chunk overhead
gregw Nov 11, 2024
9a23ecc
Updates after review
gregw Nov 12, 2024
2f06d2f
deprecated DelayedHandler
gregw Nov 12, 2024
db9f4d6
Merge remote-tracking branch 'origin/jetty-12.1.x' into experiment/je…
gregw Nov 12, 2024
50265b4
Fixed several XmlConfiguration issues so that a Builder pattern can b…
gregw Nov 12, 2024
2b0fe1b
Fixed several XmlConfiguration issues so that a Builder pattern can b…
gregw Nov 13, 2024
e391c0b
Updates from review
gregw Nov 13, 2024
da36c86
Updates from review
gregw Nov 13, 2024
6bfc885
Added distribution test cases for EagerContentHandler modules.
sbordet Nov 13, 2024
4997eed
Merged eager modules to one module
gregw Nov 14, 2024
a68e9b3
Removed the check on inputState.
gregw Nov 14, 2024
494fe3a
Merge remote-tracking branch 'origin/jetty-12.1.x' into experiment/je…
gregw Nov 14, 2024
e8ce082
Fixed bad content type test
gregw Nov 14, 2024
fbd09fb
Added documentation about `EagerContentHandler`.
sbordet Nov 14, 2024
6fc59b4
Merged branch 'jetty-12.1.x' into 'experiment/jetty-12.1.x/delayedDis…
sbordet Nov 14, 2024
8dd0dfe
Renamed properties "jetty.eager.retained" to "jetty.eager.content".
sbordet Nov 14, 2024
ef2aae0
Clarified documentation about `EagerContentHandler`.
sbordet Nov 14, 2024
187d57b
Merge remote-tracking branch 'origin/jetty-12.1.x' into experiment/je…
lorban Nov 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,21 @@ The module properties are:
include::{jetty-home}/modules/debuglog.mod[tags=documentation]
----

[[eager-content]]
== Module `eager-content`

The `eager-content` module installs the `org.eclipse.jetty.server.handler.EagerContentHandler` at the root of the `Handler` tree.

The `EagerContentHandler` can eagerly load request content, asynchronously, before calling the next `Handler`.
For more information see xref:programming-guide:server/http.adoc#handler-use-eager[this section].

`EagerContentHandler` can eagerly load content for form uploads, multipart uploads and any request content, and you can configure it with different properties for these three cases.

The module properties are:

----
include::{jetty-home}/modules/eager-content.mod[tags=documentation]
----

[[eeN-deploy]]
== Module `{ee-all}-deploy`
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1198,6 +1198,44 @@ In the example above, `ContextHandlerCollection` will try to match a request to
NOTE: `DefaultHandler` just sends a nicer HTTP `404` response in case of wrong requests from clients.
Jetty will send an HTTP `404` response anyway if `DefaultHandler` has not been set.

[[handler-use-eager]]
==== EagerContentHandler

`EagerContentHandler` reads eagerly the HTTP request content, and invokes the next `Handler` in the `Handler` tree when the request content has been read.

`EagerContentHandler` should be installed when web applications use blocking I/O to read the request content, which is the typical case for Servlet or RESTful (JAX-RS) web applications.

Because the request content is read eagerly and asynchronously, the web application will never (or rarely) block while reading the request content.
In this way, the application obtains the benefits of asynchronous I/O without forcing web application developers to use more complicated asynchronous I/O APIs.

The `Handler` tree structure looks like the following:

[,screen]
----
Server
└── (GzipHandler) // optional
└── EagerContentHandler
└── ContextHandler /app
└── AppHandler
----

`EagerContentHandler` should be installed in the `Handler` tree _after_ other ``Handler``s that may modify or transform the request content, like for example the `GzipHandler`.

`EagerContentHandler` eagerly reads request content in the following cases:

* Form request content.
* MultiPart request content.
* Any other type of request content.

For Form request content, `EagerContentHandler` reads the whole request content, parses it into a `Fields` object, and then invokes the next `Handler`.
This allows web applications that use blocking API calls such as `HttpServletRequest.getParameterMap()` to avoid blocking, since they can directly use the already created `Fields` object.

Similarly, for MultiPart request content, `EagerContentHandler` reads the whole request content, parses it into `MultiPartFormData.Parts`, and then invokes the next `Handler`.
This allows web applications that use blocking API calls such as `HttpServletRequest.getParts()` to avoid blocking, since the can directly use the already created `MultiPartFormData.Parts` object.

For other types of request content, `EagerContentHandler` reads and retains request content bytes up to a configurable amount, and then invokes the next `Handler`, without any further processing of the request content bytes.
This allows web applications that use blocking API calls such as `HttpServletRequest.getInputStream()` to avoid blocking in most cases (if the request is smaller than what has been configured in `EagerContentHandler`).

[[handler-use-servlet]]
=== Servlet API Handlers

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;

import org.eclipse.jetty.http.MimeTypes;
import org.eclipse.jetty.util.Fields;

/**
Expand All @@ -32,7 +33,10 @@ public FormRequestContent(Fields fields)

public FormRequestContent(Fields fields, Charset charset)
{
super("application/x-www-form-urlencoded", convert(fields, charset), charset);
super(charset == StandardCharsets.UTF_8
? MimeTypes.Type.FORM_ENCODED_UTF_8.asString()
: MimeTypes.Type.FORM_ENCODED.asString() + ";charset=" + charset.name().toLowerCase(),
convert(fields, charset), charset);
}

public static String convert(Fields fields)
Expand All @@ -48,7 +52,7 @@ public static String convert(Fields fields, Charset charset)
{
for (String value : field.getValues())
{
if (builder.length() > 0)
if (!builder.isEmpty())
builder.append("&");
builder.append(encode(field.getName(), charset)).append("=").append(encode(value, charset));
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@

import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.containsInAnyOrder;
import static org.hamcrest.Matchers.equalToIgnoringCase;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;

Expand All @@ -57,14 +58,12 @@ public void testFormContentProvider(Scenario scenario) throws Exception
protected void service(Request request, Response response)
{
assertEquals("POST", request.getMethod());
assertEquals(MimeTypes.Type.FORM_ENCODED.asString(), request.getHeaders().get(HttpHeader.CONTENT_TYPE));
FormFields.from(request).whenComplete((fields, failure) ->
{
assertEquals(value1, fields.get(name1).getValue());
List<String> values = fields.get(name2).getValues();
assertEquals(2, values.size());
assertThat(values, containsInAnyOrder(value2, value3));
});
assertThat(request.getHeaders().get(HttpHeader.CONTENT_TYPE), equalToIgnoringCase(MimeTypes.Type.FORM_ENCODED_UTF_8.asString()));
Fields fields = FormFields.getFields(request);
assertEquals(value1, fields.get(name1).getValue());
List<String> values = fields.get(name2).getValues();
assertEquals(2, values.size());
assertThat(values, containsInAnyOrder(value2, value3));
}
});

Expand All @@ -89,7 +88,7 @@ public void testFormContentProviderWithDifferentContentType(Scenario scenario) t
fields.put(name1, value1);
fields.add(name2, value2);
final String content = FormRequestContent.convert(fields);
final String contentType = "text/plain;charset=UTF-8";
final String contentType = "text/plain;charset=utf-8";

start(scenario, new EmptyServerHandler()
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Comparator;
import java.util.HashMap;
import java.util.List;
import java.util.Locale;
Expand Down Expand Up @@ -305,10 +304,7 @@ else if (Supplier.class.isAssignableFrom(context.getClass()))
initializeContextPath(contextHandler, path);

if (Files.isDirectory(path))
{
contextHandler.setBaseResource(ResourceFactory.of(this).newResource(path));
System.err.println("SET BASE RESOURCE to " + path);
}

//TODO think of better way of doing this
//pass through properties as attributes directly
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,11 @@ public class MimeTypes
public enum Type
{
FORM_ENCODED("application/x-www-form-urlencoded"),
FORM_ENCODED_UTF_8("application/x-www-form-urlencoded;charset=utf-8", FORM_ENCODED),
FORM_ENCODED_8859_1("application/x-www-form-urlencoded;charset=iso-8859-1", FORM_ENCODED),

MESSAGE_HTTP("message/http"),

MULTIPART_BYTERANGES("multipart/byteranges"),
MULTIPART_FORM_DATA("multipart/form-data"),

Expand All @@ -77,6 +81,10 @@ public HttpField getContentTypeField(Charset charset)
return super.getContentTypeField(charset);
}
},

TEXT_HTML_8859_1("text/html;charset=iso-8859-1", TEXT_HTML),
TEXT_HTML_UTF_8("text/html;charset=utf-8", TEXT_HTML),

TEXT_PLAIN("text/plain")
{
@Override
Expand All @@ -89,6 +97,9 @@ public HttpField getContentTypeField(Charset charset)
return super.getContentTypeField(charset);
}
},
TEXT_PLAIN_8859_1("text/plain;charset=iso-8859-1", TEXT_PLAIN),
TEXT_PLAIN_UTF_8("text/plain;charset=utf-8", TEXT_PLAIN),

TEXT_XML("text/xml")
{
@Override
Expand All @@ -101,21 +112,14 @@ public HttpField getContentTypeField(Charset charset)
return super.getContentTypeField(charset);
}
},
TEXT_JSON("text/json", StandardCharsets.UTF_8),
APPLICATION_JSON("application/json", StandardCharsets.UTF_8),

TEXT_HTML_8859_1("text/html;charset=iso-8859-1", TEXT_HTML),
TEXT_HTML_UTF_8("text/html;charset=utf-8", TEXT_HTML),

TEXT_PLAIN_8859_1("text/plain;charset=iso-8859-1", TEXT_PLAIN),
TEXT_PLAIN_UTF_8("text/plain;charset=utf-8", TEXT_PLAIN),

TEXT_XML_8859_1("text/xml;charset=iso-8859-1", TEXT_XML),
TEXT_XML_UTF_8("text/xml;charset=utf-8", TEXT_XML),

TEXT_JSON("text/json", StandardCharsets.UTF_8),
TEXT_JSON_8859_1("text/json;charset=iso-8859-1", TEXT_JSON),
TEXT_JSON_UTF_8("text/json;charset=utf-8", TEXT_JSON),

APPLICATION_JSON("application/json", StandardCharsets.UTF_8),
APPLICATION_JSON_8859_1("application/json;charset=iso-8859-1", APPLICATION_JSON),
APPLICATION_JSON_UTF_8("application/json;charset=utf-8", APPLICATION_JSON);

Expand Down Expand Up @@ -691,7 +695,25 @@ public static MimeTypes.Type getMimeTypeFromContentType(HttpField field)
if (field instanceof MimeTypes.ContentTypeField contentTypeField)
return contentTypeField.getMimeType();

return MimeTypes.CACHE.get(field.getValue());
String contentType = field.getValue();
int semicolon = contentType.indexOf(';');
if (semicolon >= 0)
contentType = contentType.substring(0, semicolon).trim();

return MimeTypes.CACHE.get(contentType);
}

public static String getMimeTypeAsStringFromContentType(HttpField field)
{
if (field == null)
return null;

assert field.getHeader() == HttpHeader.CONTENT_TYPE;

if (field instanceof MimeTypes.ContentTypeField contentTypeField)
return contentTypeField.getMimeType().asString();

return getBase(field.getValue());
}

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
import org.eclipse.jetty.io.Content;
import org.eclipse.jetty.util.Attributes;
import org.eclipse.jetty.util.Promise;
import org.eclipse.jetty.util.resource.ResourceFactory;

import static org.eclipse.jetty.http.ComplianceViolation.Listener.NOOP;

Expand Down Expand Up @@ -51,6 +52,15 @@ public Builder()
{
}

/**
* @param location the directory where parts will be saved as files.
*/
public Builder location(String location)
{
location(ResourceFactory.root().newResource(location).getPath());
gregw marked this conversation as resolved.
Show resolved Hide resolved
return this;
}

/**
* @param location the directory where parts will be saved as files.
*/
Expand All @@ -70,7 +80,7 @@ public Builder maxParts(int maxParts)
}

/**
* @return the maximum size in bytes of the whole multipart content, or -1 for unlimited.
* @param maxSize the maximum size in bytes of the whole multipart content, or -1 for unlimited.
*/
public Builder maxSize(long maxSize)
{
Expand All @@ -79,7 +89,7 @@ public Builder maxSize(long maxSize)
}

/**
* @return the maximum part size in bytes, or -1 for unlimited.
* @param maxPartSize the maximum part size in bytes, or -1 for unlimited.
*/
public Builder maxPartSize(long maxPartSize)
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ private static class HTTP2ClientConnection extends HTTP2Connection implements Ca

private HTTP2ClientConnection(HTTP2Client client, EndPoint endpoint, HTTP2ClientSession session, Promise<Session> sessionPromise, Session.Listener listener)
{
super(client.getByteBufferPool(), client.getExecutor(), endpoint, session, client.getInputBufferSize());
super(client.getByteBufferPool(), client.getExecutor(), endpoint, session, client.getInputBufferSize(), -1);
this.client = client;
this.promise = sessionPromise;
this.listener = listener;
Expand Down
Loading
Loading