Are there optimizations for proxying larger file sizes?

[deanebarker]

Years ago, I wrote a proxy in C#.

I have recently re-written this in Node (v2.0.6), using this library on Express.

I've been doing some testing. I have both proxies running locally on the same hardware with the same network connection, pointed at the same endpoint.

Across 5,000 requests to each proxy (actual requests to URLs on the endpoint), the performance of both are very similar (number is the average time of request completion).

• C#: 102ms
• Node: 120ms

This difference (about 15%) could be caused by any number of things, so I don't consider it material.

However, notably, the difference is far more pronounced when proxying files, rather than simple HTML. Example:

• C#: 106ms
• Node: 160ms

This is a 34% difference, up from 15%. The difference is very consistent across test runs (the highest I recorded was 45%).

(I realize that "files" and "HTML" are both just bytes, but the average file is far larger than the average HTML payload, so perhaps it's more accurate to say this difference occurs with larger proxy payloads.)

Is there any particular optimization for handling larger payloads? Anything I should be doing that might account for the increasing performance difference as the payload sizes go up?

Here's the code for the middleware, which you can see is pretty simple.

// Makes the proxy call
app.use('/', createProxyMiddleware({
    target: '[endpoint domain]'
    pathRewrite: rewritePath,
    changeOrigin: true,
    selfHandleResponse: true,
    onProxyRes: templateResponse
}));

(Note: the templateResponse does nothing of note for files -- it exits at the top of the method for content-type headers that start with image. I did remove that completely for one test run, so that it was never even called, and that dropped the difference by a bit -- down to 32%.)

Is there a configuration setting or optimization I should be using?

[acidbotmaker]

I'm also interested in this topic. I need to check a few keys in redis before forwarding the s3 bucket URL to the client.
I'm not sure if proxying files will be a bottleneck at scale or it won't.