Add LegacyNearest handler for legacy support of mlab-ns resources

[stephen-soltesz]

This change adds legacy handlers for /ndt and /ndt_ssl to the Locate service so that the python2.7 version of mlab-ns can be decommissioned.

This change will help M-Lab eliminate operational dependency on mlab-ns without entirely removing support from integrations that have not yet migrated to Locate v2. This also provides us the ability to slowly disable support for these resources before removing them entirely (e.g. by tiered probability between now and future sunset date).

Merging this PR is conditional on choosing an absolute sunset date for this functionality.

This change is 

[coveralls]

Pull Request Test Coverage Report for Build 1283

Details

• 0 of 121 (100.0%) changed or added relevant lines in 3 files are covered.
• 4 unchanged lines in 1 file lost coverage.
• Overall coverage decreased (-0.01%) to 96.846%

---

Files with Coverage Reduction	New Missed Lines	%
cmd/heartbeat/main.go	4	82.26%

Totals
Change from base Build 1262:	-0.01%
Covered Lines:	1873
Relevant Lines:	1934

---

💛 - Coveralls

[stephen-soltesz]

Example invocations:

$ curl --dump-header - 'https://locate-dot-mlab-sandbox.appspot.com/ndt?policy=geo_options'
[{"city":"New York","country":"US","fqdn":"ndt-mlab4-lga0t.mlab-sandbox.measurement-lab.org","site":"lga0t"},{"city":"New York","country":"US","fqdn":"ndt-mlab1-lga1t.mlab-sandbox.measurement-lab.org","site":"lga1t"},{"city":"San Francisco Bay Area","country":"US","fqdn":"ndt-mlab1-nuq0t.mlab-sandbox.measurement-lab.org","site":"nuq0t"}]

$ curl --dump-header - 'https://locate-dot-mlab-sandbox.appspot.com/ndt'
{"city":"New York","country":"US","fqdn":"ndt-mlab2-lga0t.mlab-sandbox.measurement-lab.org","site":"lga0t"}

$ curl --dump-header - 'https://locate-dot-mlab-sandbox.appspot.com/ndt_ssl'
{"city":"New York","country":"US","fqdn":"ndt-mlab1-lga0t.mlab-sandbox.measurement-lab.org","site":"lga0t"}

$ curl --dump-header - 'https://locate-dot-mlab-sandbox.appspot.com/ndt_ssl?format=bt'
New York, US|ndt-mlab2-lga0t.mlab-sandbox.measurement-lab.org

$ curl --dump-header - 'https://locate-dot-mlab-sandbox.appspot.com/ndt_ssl?format=bt&policy=geo_options'
New York, US|ndt-mlab4-lga0t.mlab-sandbox.measurement-lab.org
San Francisco Bay Area, US|ndt-mlab1-nuq0t.mlab-sandbox.measurement-lab.org

[cristinaleonr]

The changes look fine, but there is a non-trivial amount of complexity/translation in handler_legacy.go. Why are we adding mlab-ns resources to the Locate? Do we know what percentage of users target mlab-ns? I thought we were going to stop supporting it (blog).

Reviewable status: 0 of 1 approvals obtained

[stephen-soltesz]

Your question is good. I'm trying to do the right thing. The following are all true:

• App Engine has ended support and deployment for python2.7 apps, though they can continue running (like mlab-ns)
• mlab-ns still depends on old naming conventions for ndt servers, i.e. ndt-iupui, stalling changes to siteinfo & prometheus-support
• ~90% of requests for ndt5/raw go to mlab-ns (/ndt or /ndt_ssl) vs Locate v2 (/v2/nearest/ndt/ndt5)
• It looks like ndt5+raw is the second most common protocol in the data after ndt7+wss (admittedly a distant second, 91% vs ~3.5%)
• Our own tooling (ndt5-client-go) still use mlab-ns
• While ~50% of mlab-ns requests are from identifiable client integrations, M-Lab has not had time yet to reach out to any about migration.

So, rather than rip the bandaid off, I'm trying to provide an intermediate path that allows us to disable the old App Engine code base in exchange for a much smaller handler which we can control how we want (e.g. to reduce availability gradually to encourage migration). Also, the total code under maintenance reduces ~20x from ~10141 loc vs ~500 loc with tests, so this would be a net reduction.

In summary: net code reduction, more control for us operationally, preserve some backward compatibility for existing users until M-Lab can coordinate the migration and registration process.

This was the original evaluation, the queries dates are a little dated now, but the same trends seem to be there today: https://docs.google.com/document/d/1Tr5ImBK86_GWNFLlzuVvbG6EegodLAiCXMxQlmBU6vA/edit

Reviewable status: 0 of 1 approvals obtained

[stephen-soltesz]

PTAL?

[cristinaleonr]

It might be worth creating a github issue that you can reference in the removal TODOs.

Reviewable status: complete! 1 of 1 approvals obtained

[stephen-soltesz]

Thank you. Updated TODOs to reference #185

Reviewable status: complete! 1 of 1 approvals obtained