Reducing maxLen for ChunkedResponse still needed?

[tueddy]

Hi,

while using the former ESPAsyncWebserver together with ChunkedResponse it was necessary to reduce maxLen for stability. This code was recommended (use only half of buffer):

maxLen = maxLen >> 1; // some sort of bug with actual size available, reduce the len

This now causes problems and e.g. longer JSON payload is being truncated & invalid afterwards.

We use the web server in ESPuino, an RFID music player for children. In this more complex code, 2 ChunkedResponse are used, one to display the cover image from a MP3 audio file in the web interface and the other to transfer the stored RFID assignments from the NVS as JSON to the web interface.

My question, is it still necessary with the current version? Or even counterproductive? Best would be to use the full buffer available?

Thank's
Dirk

[mathieucarbou]

I am not aware of such issue.

I suspect that maxLength is based on the client available space given by AsyncTCP, and which should not be more than CONFIG_TCP_MSS. But in any case, you should be able to fill maxLen bytes in the buffer.

Also note that this is a buffer of bytes that you write into, not a char buffer.

I suspect the use of snprintf is wrong because the length passed includes the added null terminator that snprintf will put and that you specifically do not want in a response. Besides, the retuned len will be len == maxLen - len - 1 because snprintf returns the number of characters written excluding the null terminator written.

len += snprintf(((char *) buffer + len), maxLen - len, ",%s", json.c_str());

Just use a copy instead, something like this pseudo-code (not tested)

// position is the request temporary object, which can be reused when the callback is recalled. Initially 0.
  size_t len = strlen(json) - position;
  if(len <= maxLength) {
    memcpy(buffer, ((uint8_t*) json.c_str()) + position, len);
    return len;
  } else {
    // your json is too big to fit, you need to split - something like that maybe:
    memcpy(buffer + position, (uint8_t*) json.c_str(), maxLength);
    position += maxLength;
    return maxLength;
  }

[tueddy]

Hi @mathieucarbou , thanks for your fast response!

I will discuss it in the ESPuino forum (german only )& come back, it is very likely that the problem is in the ESPuino client code!

This was the original post of reducing maxLen for ESPAsyncWebserver:
https://stackoverflow.com/questions/61559745/espasyncwebserver-serve-large-array-from-ram

I finally found a solution now which uses the chunked response. Also ArduinoJson is not used any more. I had to shrink the buffer to get it stable (maxLen = maxLen >> 1;). The library gives me a wrong max. buffer length with maxLen. This could be bug.

Thanks and best regards
Dirk

[mathieucarbou]

The SO answer does not make any sense to me: they forget also that the destination buffer is a continuous serie of bytes that are written into the network and the callback is just a way to ask to fill this buffer of a maximum of maxLength bytes

If the content type is not octet-stream (content type received by browser will be treated as a String) then there should not be any null character inside.