Handling dataset redirects

[PeterAJansen]

Currently datasets-server doesn't appear to handle dataset redirects, and gives an error.

For example:

• Querying datasets endpoints with the dataset squad works

• Querying datasets-server endpoints with the dataset squad gives an "unknown error"

• Querying datasets-server endpoints with a random dataset name (e.g. xyz2309348) gives an expected response (e.g. dataset unknown/doesn't exist)

• Querying datasets-server endpoints with the redirected name (rajpurkar/squad) works.

[severo]

The dataset viewer only knows the last repository name if it has been renamed once or more. For example, squad was renamed to rajpurkar/squad. The previous entries for squad in the database were deleted, and new ones were created for rajpurkar/squad. As we don't check if the repo was renamed, asking for dataset=squad returns an error.

To support this, if the dataset has not been found in the database, we should request the Hub to get the current name of the repo (in the example: rajpurkar/squad) and look again if the dataset exists in the database.

We can get this info by looking at the id field in hfh.dataset_info() (see https://huggingface.co/api/datasets/squad)

[severo]

also:

Querying datasets-server endpoints with the dataset squad gives an "unknown error"

^ this is a bug

[severo]

With #3035, it now returns 404 with {"error":"The dataset has been renamed. Please use the current dataset name."}. At least, this bug is fixed.

https://datasets-server.huggingface.co/splits?dataset=mnist or https://datasets-server.huggingface.co/splits?dataset=squad

[severo]

Let's close. We don't plan to support repositories redirection in this API.