Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support request size negotiation and increased throughput on high-latency connections #267

Open
wants to merge 14 commits into
base: master
Choose a base branch
from

Conversation

ccope
Copy link
Contributor

@ccope ccope commented Oct 1, 2021

I have found that a combination of increasing readahead and read limits to ~4MB has increased throughput from 1.1MB/s to 7.5MB/s on a connection with a 160ms round-trip time.

I need to submit another patch to the Linux kernel to allow configuring the readahead size. libfuse already seems to accept it as a parameter but the kernel side doesn't utilize it. Here's a patch that I used for testing which overrode the hard-coded limits (based on Ubuntu-hwe-5.4-5.4.0-79.88_18.04.1): https://gist.github.com/ccope/0ff1cac336fceb827696c40715a4a828

The OpenSSH SFTP server recently added request limit negotiation, but it also has a fairly restrictive hard-coded cap. I need to file a patch to make that limit more configurable as well.

I reworked how the sftp init process is invoked to ensure limits get updated before forking more threads. In the process I created some high-level helper functions for sending requests and receiving responses without threads, and used them for all of the sftp init requests. I'd be happy to move them to a separate PR but I just kind of got tired of reworking my git history.

@h4sh5
Copy link
Collaborator

h4sh5 commented Sep 23, 2023

There seem to be some issues with this that causes processes to hang; it might be caused by deadlocks from the new sftp_request_process_sync and related functions. I am not sure exactly.

See https://bugzilla.suse.com/show_bug.cgi?id=1215574 and discussions here deadbeefsociety#1 (I assume it is WIP and hasn't been thoroughly tested)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants