Mopidy fails to download rss: 403

[vasyugan]

For some reason, the RSS http://www.freie-radios.net/portal/podcast.php?rss fails in Mopidy-Podcast. I can view the RSS in a browser just fine and it also works in BeyondPod:

ERROR Error loading podcast http://www.freie-radios.net/portal/podcast.php?rss: HTTP Error 403: Forbidden

I also get a timeout for http://downloads.bbc.co.uk/podcasts/worldservice/docarchive/rss.xml, which when loaded in a browser redirects to
http://www.bbc.co.uk/programmes/p02nq0lx/episodes/downloads.rss

[tkem]

First issue: Bad luck. If the server for whatever reason denies access to Mopidy-Podcast, there's little we can do.
Second issue what happens when you try to access the podcast using the "final" URL?

[vasyugan]

tkem, I don't think that any server especially denies Mopidy-Podcasts access while allowing everyone else. Well how can I test this? What is the user-agent string mopidy-podcast uses? If they really deny access to that particular user agent, it should be reproducible.
Second issue: When I use the final URL it is ok.

[tkem]

@vasyugan: Of course not, but if servers deny access to certain clients, e.g. based on user-agent, then they (hopefully) have valid reasons and surely the right to do so. IIRC, #31 was a similar issue; don't know what happened with that, though.

Starting with Mopidy-Podcast v2.0, the User-Agent is as provided by mopidy.httpclient: http://mopidy.readthedocs.io/en/latest/api/httpclient/
And no, I don't plan to add a config setting to overwrite this, so please don't ask ;-)

I'll investigate the second issue ASAP; this may actually be a bug with handling redirects. Thanks for reporting!

[vasyugan]

tkem, I just asked you for the user agent string to test it. wget allows to specify a custom user agent with the -u switch. If the server really locks out that agent, this should be reproducible. From the link you have provided I have unfortunately been unable to guess how the user agent string looks like. Anyway, I'll write to the webmaster of freie-radios.net and see if they respond.

[vasyugan]

I should note, that this is not a cloudflare server, according to the HTTP headers, it's an Apache 2.2.22. Well, anyway, I hope they will respond

[tkem]

@vasyugan: as the docs say, the actual user-agent string depends on the respective versions of Mopidy and Mopidy-Podcast, as well as your Python interpreter. For e.g. Mopidy-Podcast v2.0.0, it can be retrieved like this:

$ python
>>> from mopidy import httpclient
>>> httpclient.format_user_agent('Mopidy-Podcast/2.0.0')
'Mopidy-Podcast/2.0.0 Mopidy/1.1.2 CPython/2.7.3'

[vasyugan]

I have contacted the hosters and apparently they had blacklisted python-urllib as a bad user agent. They removed it from the list, now it works. Which indicates that the HTTP client does not send the mopidy user agent but the standard one from python urllib. This is what they found in the server log:

[my IP Address] 9 - - [19/May/2016:14:11:55 +0200] "GET /portal/podcast.php?rss HTTP/1.1" 403 491 "-" "Python-urllib/2.7

[tkem]

@vasyugan: Which Mopidy-Podcast version are you using? Versions <2.0 send the standard urllib header.

[vasyugan]

the version, which is in the AUR of Arch Linux: 1.1.2

https://aur.archlinux.org/packages/mopidy-podcast

[tkem]

Regarding issue #2: I tried the BBC redirect URL, and it seems to be really slow... I had to set the timeout config value to 30 in Mopidy-Podcast v2.0 to get it working. For Mopidy-Podcast v1.1.2, setting feeds_timeout should have a similar effect, but I haven't tested this.

[vasyugan]

flagged mopidy-podcast in the AUR out-of-date