Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: Display the list of failed downloads as links and their location #251

Open
Acters opened this issue Feb 28, 2024 · 6 comments

Comments

@Acters
Copy link

Acters commented Feb 28, 2024

I tried searching and I don't see this mentioned anywhere.

Please include the option to export the list of failed downloads, and the location it is found at.

Another thought, include in advanced options or tracking, to allow these failed downloads to appear in the viewer/added to database. May want to include special formatting that denotes it is a failed download.

Thank you,
Acters

@chylex
Copy link
Owner

chylex commented Feb 28, 2024

It's a good idea, but any major UI additions like this are going to be low priority for a long time.

Listing the failed downloads and manually adding/editing downloaded files can be done with any decent database tool that supports SQLite, such as https://sqlitebrowser.org.

@Acters
Copy link
Author

Acters commented Feb 28, 2024

It's a good idea, but any major UI additions like this are going to be low priority for a long time.

Listing the failed downloads and manually adding/editing downloaded files can be done with any decent database tool that supports SQLite, such as https://sqlitebrowser.org.

Thank you for all your efforts, the viewer is a nice and you did a wonderful job.

From investigation on why there was a failed download. The image in question is only accessible when this is applied at the end of the url:
?size=96

I know this because this is a user's profile picture has been changed but my client still had cached the old profile picture in the chat. I will need to reload discord or cause discord to free up this cache in some way.(Desktop Client, likely differentl, or smaller cache, in the browser)

My guess is Discord seems to allow these kinds of cache shenanigans for less strain on their servers. So the image is only accessible as a cached small chat profile picture and the full resolution is removed. Hence the requirement to include ?size=96 at the end. However, while appending ?size=96 works for the browser, updating the database to change the url with this appended at the end will cause the Discord History Tracker server to be perpetually listing the download as "pending" after clicking retry failed download.

Should I create a separate issue for this strange behavior?

@chylex
Copy link
Owner

chylex commented Feb 28, 2024

DHT performs URL normalization to avoid downloading duplicate files and try to use highest quality variants, I'm guessing you edited the normalized URL instead of the download URL? If you do that, DHT won't be able to find it anymore because it will remove the ?size parameter when looking up the downloaded file in the database. The download_metadata table maps normalized URLs to download URLs, you'll have to change the download URL there.

@Acters
Copy link
Author

Acters commented Feb 29, 2024

DHT performs URL normalization to avoid downloading duplicate files and try to use highest quality variants, I'm guessing you edited the normalized URL instead of the download URL? If you do that, DHT won't be able to find it anymore because it will remove the ?size parameter when looking up the downloaded file in the database. The download_metadata table maps normalized URLs to download URLs, you'll have to change the download URL there.

Thank you for the succinctly written answer. I have edited only the download URL, not the Normalized URL in the download_metadata table. However, Discord History Tracker will automatically remove ?size=96 once it loads the modified database. The download will sit in pending for eternity. I don't want to clog up this feature request with this issue, I will create another issue for this.

@espressoelf
Copy link

It's a good idea, but any major UI additions like this are going to be low priority for a long time.

Listing the failed downloads and manually adding/editing downloaded files can be done with any decent database tool that supports SQLite, such as https://sqlitebrowser.org.

I was looking for a way to find urls that didn't download, too. Is there a better way than just

SELECT * FROM attachments 
LEFT JOIN download_blobs USING(normalized_url)
WHERE blob IS NULL

?

@chylex
Copy link
Owner

chylex commented Apr 7, 2024

It's a good idea, but any major UI additions like this are going to be low priority for a long time.
Listing the failed downloads and manually adding/editing downloaded files can be done with any decent database tool that supports SQLite, such as https://sqlitebrowser.org.

I was looking for a way to find urls that didn't download, too. Is there a better way than just

SELECT * FROM attachments 
LEFT JOIN download_blobs USING(normalized_url)
WHERE blob IS NULL

?

Try:

SELECT * FROM download_metadata WHERE status != 200

Status codes: https://github.com/chylex/Discord-History-Tracker/blob/master/app/Server/Data/DownloadStatus.cs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants