Skip to content

Commit

Permalink
lnrpc: increase max message size for ws proxy
Browse files Browse the repository at this point in the history
In this commit, we increase the max message size for the ws proxy. We
have a similar setting for the normal gRPC server which was tuned to be
able to support decoding `GetNetworkInfo` as the channel graph got
larger. We keep the default buffer size of 64 KB, but allow that to be
expanded to up to 4 MB (current value) to decode larger messages.

One alternative would be to modify the `Split` function to break up
larger lines into smaller ones. We'd need to double check that the
libraries at a higher level of abstraction can handle the chunks. The
scan function would look something like:
```go
splitFunc := func(data []byte, eof bool) (int, []byte, error) {
        if len(data) >= chunkSize {
                return chunkSize, data[:chunkSize], nil
        }

        return bufio.ScanLines(data, eof))
}
scanner.Split(splitFunc)
```
  • Loading branch information
Roasbeef committed Sep 18, 2023
1 parent 2d5c0c9 commit 247c1d8
Show file tree
Hide file tree
Showing 2 changed files with 20 additions and 1 deletion.
5 changes: 5 additions & 0 deletions docs/release-notes/release-notes-0.17.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,11 @@ None

* The [WalletBalance](https://github.com/lightningnetwork/lnd/pull/7857) RPC
(lncli walletbalance) now supports showing the balance for a specific account.

* The [websockets proxy now uses a larger default max
message](https://github.com/lightningnetwork/lnd/pull/7991) size to support
proxying larger messages.


## lncli Updates
* Added ability to use [environment variables to override `lncli` global
Expand Down
16 changes: 15 additions & 1 deletion lnrpc/websocket_proxy.go
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,11 @@ const (
// an arbitrary non-empty message that has no deeper meaning but should
// be sent back by the client in the pong message.
PingContent = "are you there?"

// MaxWsMsgSize is the largest websockets message we'll attempt to
// decode in the gRPC <-> WS proxy. gRPC has a similar setting used
// elsewhere.
MaxWsMsgSize = 4 * 1024 * 1024
)

var (
Expand Down Expand Up @@ -413,9 +418,18 @@ func (r *requestForwardingReader) CloseWriter() {
// what's written to it and presents it through a bufio.Scanner interface.
func newResponseForwardingWriter() *responseForwardingWriter {
r, w := io.Pipe()

scanner := bufio.NewScanner(r)

// We pass in a custom buffer for the bufio scanner to use. We'll keep
// with a normal 64KB buffer, but allow a larger max message size,
// which may cause buffer expansion when needed.
buf := make([]byte, 0, bufio.MaxScanTokenSize)
scanner.Buffer(buf, MaxWsMsgSize)

return &responseForwardingWriter{
Writer: w,
Scanner: bufio.NewScanner(r),
Scanner: scanner,
pipeR: r,
pipeW: w,
header: http.Header{},
Expand Down

0 comments on commit 247c1d8

Please sign in to comment.