feat(redirects): handle grapher redirects in grapher worker

[marcelgerber]

Work around Cloudflare's 2000 static redirects limit by handling redirects directly in the worker.

This works by baking a grapher/_grapherRedirects.json file as part of the baking process, where all grapher redirects always go.
The original _redirects file will then only contain WP redirects, plus some hardcoded ones.
I checked and we're currently down to just 506 redirects in the file now, so quite some room to spare.

I tried out the following variations and they all work:

• /grapher/life-expectancy goes through
• /grapher/life-EXPECTANCY redirects to lowercase
• /grapher/life-expectancy/ removes trailing slash
• /grapher/life-expectancy-globally-since-1770 redirects
  • ^ this one also works with uppercase and a trailing slash mixed in
• /grapher/number-affected-by-natural-disasters redirects to an explorer using WP redirect

Caveats:

• The thumbnails worker doesn't really take redirects into account, so if you went to /grapher/thumbnail/life-expectancy-globally-since-1770 for example, it would just fail instead of resolving the redirect itself.
• In the very rare case where we have a grapher redirect and a WP redirect for the same URL, the grapher redirect now takes precedence. This is not a problem at all.

The /grapher/_grapherRedirects.json file looks like this:

{
  "correlation-between-child-mortality-and-mean-years-of-schooling-for-those-aged-15-and-older-2000": "correlation-between-child-mortality-and-mean-years-of-schooling-for-those-aged-15-and-older",
  "us-education-expenditure-as-share-of-gdp-public-and-private": "us-education-expenditure-as-share-of-gdp-public-and-private-institutions",
  "revenues-for-public-schools-by-source-us": "revenues-for-public-schools-by-source-gdp-us",
  // ...
}

[marcelgerber]

This is really my biggest question in working on this, what is the best way to efficiently fetch the redirects file, which we're gonna have to await on every request now.
Other options include:

• Storing redirects in Workers KV, somehow
• Uploading the redirects.json as part of the worker (ugh, but that's probably actually the fastest way)

I'm also not entirely sure whether the CF caching header will actually do something meaningful here, but it probably can't hurt also, and maybe it can actually speed things up?

[Marigold]

I was surprised by how fast (uncached) fetch from R2 is, so I'd say that fetching redirects from assets explicitly should be light speed. It could be worth logging latency just to be sure.

[Marigold]

(I'm still getting 404 from http://staging-site-master/grapher/life-expectancy-of-women-vs-life-expectancy-of-women that is shown in life expectancy. It could be just that workers don't work on staging?)

[marcelgerber]

(I'm still getting 404 from http://staging-site-master/grapher/life-expectancy-of-women-vs-life-expectancy-of-women that is shown in life expectancy. It could be just that workers don't work on staging?)

Wrong server, http://staging-site-redirects-handling-worker/grapher/life-expectancy-of-women-vs-life-expectancy-of-women works fine.

[Marigold]

Wrong server, http://staging-site-redirects-handling-worker/grapher/life-expectancy-of-women-vs-life-expectancy-of-women works fine.

You're right; it was a staging server URL issue (fixed by now).