Clear Payload buffer after receiving #32
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Wrench\Client keeps all received Payload objects in perpetuity. This results in a script eventually running out of memory if it's reading a large stream of data. Instead, after receiving any Payload objects buffered since the last call to receive(), clear the buffer of received Payloads. This is safe because they're never accessed again. Clear the buffer in ChromeDevtoolsProtocol\WebSocket\WebSocketClient to avoid upstream changes to Wrench\Client. Additional info: Wrench\Client stores all received Payloads in Client::$received. In fact, if Payload::getPayload() is called (as it is in DevtoolsClient), memory usage will double because the received data is copied from the Payload's comprising Frames (Payload::$frames) into Payload::$buffer. The data is then stored a third time in the ReadResponse and returned to the original caller. The caller only has the ability to delete the third copy of the data in the ReadResponse. This results in memory usage growing quickly and perpetually as a long-lasting stream is read until the script eventually runs out of available memory. For example, if a user generates a large PDF (printToPDF) using a transferMode of ReturnAsStream, they may call $session->io()->read() many times to fetch the generated PDF. As they do so, they'll likely write each block of data to a file. If the buffer of received Payloads is never cleared to free memory, the user is limited to receiving a PDF approximately 1/2 the size of the memory_limit. If the buffer of received Payloads is cleared regularly, then there is theoretically no limit to the size of PDF they can read.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Wrench\Client keeps all received Payload objects in perpetuity. This results in a script eventually running out of memory if it's reading a large stream of data. Instead, after receiving any Payload objects buffered since the last call to receive(), clear the buffer of received Payloads. This is safe because they're never accessed again. Clear the buffer in ChromeDevtoolsProtocol\WebSocket\WebSocketClient to avoid upstream changes to Wrench\Client.
Additional info: Wrench\Client stores all received Payloads in Client::$received. In fact, if Payload::getPayload() is called (as it is in DevtoolsClient), memory usage will double because the received data is copied from the Payload's comprising Frames (Payload::$frames) into Payload::$buffer. The data is then stored a third time in the ReadResponse and returned to the original caller. The caller only has the ability to delete the third copy of the data in the ReadResponse. This results in memory usage growing quickly and perpetually as a long-lasting stream is read until the script eventually runs out of available memory.
For example, if a user generates a large PDF (printToPDF) using a transferMode of ReturnAsStream, they may call $session->io()->read() many times to fetch the generated PDF. As they do so, they'll likely write each block of data to a file. If the buffer of received Payloads is never cleared to free memory, the user is limited to receiving a PDF approximately 1/2 the size of the memory_limit. If the buffer of received Payloads is cleared regularly, then there is theoretically no limit to the size of PDF they can read.