Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stop iterating on the content that is 404'd or DMCA'd #93

Open
fl0werpowers opened this issue Nov 21, 2022 · 2 comments
Open

Stop iterating on the content that is 404'd or DMCA'd #93

fl0werpowers opened this issue Nov 21, 2022 · 2 comments

Comments

@fl0werpowers
Copy link

fl0werpowers commented Nov 21, 2022

Some content that is present in the archives either does not exist anymore due to it being deleted by the original uploader, or it is taken down via DMCA claims. The tool clearly emits the exceptions (as 'Download failed with status "404 Not Found"' and 'Download failed with status "403 Forbidden"' respectively), with the 403 one clearly specifying that the content in question has been struck by DMCA. Iterating through such content multiple times is a waste of time, and such media can be skipped to save time.

@fl0werpowers
Copy link
Author

these are the exceptions in question

FAIL. Media couldn't be retrieved from https://pbs.twimg.com/media/EbH_bxcUYAgxbki.png:orig because of exception: Download failed with status "404 Not Found". Response content: ""

FAIL. Media couldn't be retrieved from https://video.twimg.com/ext_tw_video/1560406436982804480/pu/vid/1280x720/m7-vUTLunERc4auB.mp4?tag=12 because of exception: Download failed with status "403 Forbidden". Response content: "{"error_code":2,"error_response":"Dmcaed"}"

@chibiconsulting
Copy link

Agree. Was going to raise this issue myself. Thanks for the well written issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants