Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Server function streaming with serializable types #2623

Merged
merged 1 commit into from
Jun 28, 2024

Conversation

ealmloff
Copy link
Contributor

This PR handles serialization and deserialization for custom types with streaming server functions (#1284)

It adds a new StreamingJson encoding which serializes each item in the stream with serde_json. The implementation is a fairly straight forward combination of the streaming bytes and json codecs with two exceptions:

  1. The content type is the default streaming type instead of JSON because if the chunks are read as merged chunks they cannot be parsed as valid json
  2. When using a custom stream type with StreamingJson, this PR requires the bound Stream<Item=T> in order to make the T type used in the trait implementation. We don't use the bound, but it makes the T type constrained. Alternatively, I could add an unused generic to StreamingJson or FromReq, but I would imagine both of those would effect user code more

@gbj
Copy link
Collaborator

gbj commented Jun 28, 2024

This looks good to me. Thanks very much for the addition!

@gbj gbj merged commit 551f9b0 into leptos-rs:main Jun 28, 2024
55 of 59 checks passed
@ealmloff ealmloff deleted the server-fn-streaming-json branch June 28, 2024 15:59
@boblehest
Copy link

I don't think this is correct; it decodes each chunk read from the stream using serde_json::from_slice, but doesn't seem to do any framing to keep track of message boundaries. E.g. sending 1 immediately followed by 2 will usually be read as the integer 12, and sending 1.0 immediately followed by 2.0 will usually fail when it tries to decode it as 1.02.0.

@ealmloff
Copy link
Contributor Author

I don't think this is correct; it decodes each chunk read from the stream using serde_json::from_slice, but doesn't seem to do any framing to keep track of message boundaries. E.g. sending 1 immediately followed by 2 will usually be read as the integer 12, and sending 1.0 immediately followed by 2.0 will usually fail when it tries to decode it as 1.02.0.

StreamingJson uses the http chunked encoding. HTTP streaming is chunked with a built in delimiter instead of a single uninterrupted stream of binary data. More info here

@boblehest
Copy link

I don't think this is correct; it decodes each chunk read from the stream using serde_json::from_slice, but doesn't seem to do any framing to keep track of message boundaries. E.g. sending 1 immediately followed by 2 will usually be read as the integer 12, and sending 1.0 immediately followed by 2.0 will usually fail when it tries to decode it as 1.02.0.

StreamingJson uses the http chunked encoding. HTTP streaming is chunked with a built in delimiter instead of a single uninterrupted stream of binary data. More info here

Thank you for the explanation. Which part of the code is responsible for adding the Transfer-Encoding header? I tried using StreamingJson in my app but it fails in the way I described in my previous comment -- I might get around to providing a test case later this week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants