-(coffee cup for scale)With the amount of data that we are passing around across services, it is often beneficial to use compression on the data to reduce the transmission time. Modern platforms and algorithms are now very efficient at compressing regular data, particularly if that data is text or json data.
+If the developer of the endpoint has prepared their service for compression, the client must still indicate that they are ready to receive the compressed data. Luckily, most implementations of modern http clients in R, Python, JavaScript and Dotnet support compression / decompression and are seamless for the client. This means that you can set the compression headers on and simply benefit from compressed data being received.
+We can also check in the Content-Encoding header which compression was used. I've found that example.com is sending responses compressed with gzip.
+Python
+import requests
+
+url = "http://example.com" # Replace with the actual URL you want to request
+
+# Specify the accepted encoding methods in the headers
+headers = {
+ 'Accept-Encoding': 'gzip, br',
+}
+
+response = requests.get(url, headers=headers)
+print(response.text)
+
+# In case you want to see if it was compressed, you can check via the headers
+#if 'Content-Encoding' in response.headers:
+# print(f"Response was compressed using: {response.headers['Content-Encoding']}")
+#else:
+# print("Response was not compressed.")
+R
+library(httr)
+
+# The URL to which you're sending the request
+url <- "http://example.com"
+
+# Setting the Accept-Encoding header
+response <- GET(url, add_headers(`Accept-Encoding` = 'gzip, br'))
+
+# The content of the response will be automatically decompressed by httr, so you can access it directly.
+content(response, "text")
+C#
+In C#, for some ungodly strange reason, the standard HTTP endpoint doesn't decompress for you automatically unless you add a decompression handler - see handler HttpClientHandler.AutomaticDecompression
+using System;
+using System.Net.Http;
+using System.Net.Http.Headers;
+using System.Threading.Tasks;
+using System.Text;
+
+class Program
+{
+ static async Task Main(string[] args)
+ {
+ HttpClientHandler handler = new HttpClientHandler();
+ handler.AutomaticDecompression = System.Net.DecompressionMethods.GZip; //Adding automatic Decompression means that the accept headers are added automatically
+
+ using (var client = new HttpClient(handler))
+ {
+ string url = "http://example.com";
+ HttpResponseMessage response = await client.GetAsync(url);
+ response.EnsureSuccessStatusCode();
+
+ Console.WriteLine(await response.Content.ReadAsStringAsync());
+ }
+ }
+}
+JavaScript
+it is so easy that you don't even need to do anything else than setting the gzip: true for the support
+const request = require('request');
+const requestOptions = {
+ url: 'http://example.com',
+ gzip: true, // This is all that is required
+};
+request(requestOptions, (error, response, body) => {
+ // Handle the response here
+});
++