[sda-download] Performance/"latency"

[pontus]

Currently, operations such as listing dataset contents or fetching a file (or even tiny parts of it) typically take more than one second, which is not great.

There are many things that can contribute to that:

• unoptimised database queries
• very little internal caching
• possibly unoptimised pathways (e.g. the proxy acting as ingress not passing on the response until it have received it all)

while we should probably strive to be our best in all of those, the lowest hanging fruit is probably caching things in sda-download.