Response Data is Showing as Buffer

[timchang87]

Hi,

I am currently working on a proxy server that can capture request and response details. I am using another project that I've completed to test this. The proxy server should log the request url, http method of the request, response status code, processing time (total round trip time of the request/response cycle), and any data sent with the response. In this project, I am using webpack dev server, and proxying the traffic to my proxy server.

I am able to capture the url, method, status code, and processing time. However, the expected data from the response is intermittently logged. Most of the time, the response data is logged as <Buffer > (see below for examples). In the example below, you'll see 4 GET requests. The last 2 are requests to the /api/v1/auth/logout endpoint. However, the first has logged the expected response body. The latter, produces a response body of <Buffer >. They should both be logging {"message":"Logged out"}.

I have tested this against other projects that I have and I get the same result. I have also console logged the data event listener. When the response body logs <Buffer >, I've noticed that the data event listener on proxyRes, never fires. It's like the data event is never emitted.

Any idea what could be causing this?

proxy.js

const { createProxyMiddleware } = require('http-proxy-middleware');

const options = {
  target: 'http://localhost:5002', // Your target URL here
  selfHandleResponse: true,
  onProxyRes: (proxyRes, req, res) => {
    let body = Buffer.from('');
    proxyRes.on('data', (data) => {
      body = Buffer.concat([body, data]).toString();
    });
    proxyRes.on('end', () => {
      const response = body.toString('utf8');
      console.log('Request URL:', req.url);
      console.log('Request method:', req.method);
      console.log('Response body:', body);
      console.log('Response status code:', proxyRes.statusCode);
      res.writeHead(proxyRes.statusCode, proxyRes.headers);
      res.end(body);
    });
  },
};

const proxy = createProxyMiddleware(options);

// Start server
const express = require('express');
const app = express();
app.use('/', proxy);
app.listen(9000, () => {
  console.log('Proxy server listening on port 9000');
});

webpack config

devServer: {
    port: 8080,
    host: 'localhost',
    hot: true,
    liveReload: false,
    open: true,
    compress: true,
    historyApiFallback: true,
    headers: {
      'Access-Control-Allow-Origin': '*',
      'Access-Control-Allow-Headers': '*',
    },
    static: {
      directory: path.resolve(__dirname, 'static/'),
      publicPath: '/',
    },
    proxy: {
      context: ['/api/**', 'static/**'],
      target: 'http://localhost:9000/',
      secure: false,
    },
  },

Console Log

Request URL: /api/v1/auth/getCurrentUser
Request method: GET
Response body: {"message":"Cannot destructure property 'id' of 'req.user' as it is undefined."}
Response status code: 500
Request URL: /api/v1/auth/getCurrentUser
Request method: GET
Response body: {"message":"Cannot destructure property 'id' of 'req.user' as it is undefined."}
Response status code: 500
Request URL: /api/v1/auth/logout
Request method: GET
Response body: {"message":"Logged out"}
Response status code: 200
Request URL: /api/v1/auth/logout
Request method: GET
Response body: <Buffer >
Response status code: 304

[chimurai]

Might be the compression.

Did you try the responseInterceptor?
It automatically decompresses the response and returns it as a Buffer, which can inspect or manipulate.

https://github.com/chimurai/http-proxy-middleware/blob/v2.0.6/recipes/response-interceptor.md