Sync with Github error (Request failed with status code: 504)

[joshmoore]

This is a re-opening of #957. Please see the discussion there.

Issue is (additionally) related to the number of organizations and repositories a user is a member of.

The only workaround currently is to disconnect repositories and reconnect them, which for users with many repositories is very sub-optimal.

Related to Zenodo Support email [Ticket#332699]

[joshmoore]

On the Zenodo Support email thread:

We are aware of an issue with the Github integration for users that have a large amount of repositories, such as in your case. Unfortunately, we haven't fixed this yet and we still need to investigate the issue further to solve it. What happens is that we iterate over the list of repositories that we fetch from the user's Github account, which causes the request from the client to eventually timeout and return an HTTP 504 Gateway Timeout.
Sometimes the sync process manages to complete, even if the request timed out, but it is unclear to us how to force this. For instance, I tried to sync your repositories and couldn't make it work.

In the future, we might have a workaround for this, but we can't commit to a date/timeframe since our team is fully committed to working on other features.

We are sorry for the inconvenience this is causing you.

[krokicki]

I'm also consistently getting this error. It is a 504 timeout after 30 seconds. I only have one organization, but it's a large one: https://github.com/JaneliaSciComp

This effectively means that we can no longer use Zenodo for new repositories, which is really a shame.

To anyone from Zenodo reading this: this type of error is usually easy to mitigate by increasing the read timeout of your reverse proxy. For example, with Nginx the relevant configuration option is proxy_read_timeout.

[bethac07]

+1 for this; I'm a member of a number of large orgs and this would really help!

[caufieldjh]

I am also experiencing this error and I'm in ~16 different orgs.

[lldelisle]

I am also experiencing this error...