Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Alpine wget without --delete-after option #4

Open
vergissberlin opened this issue Mar 26, 2021 · 3 comments · May be fixed by #10
Open

Alpine wget without --delete-after option #4

vergissberlin opened this issue Mar 26, 2021 · 3 comments · May be fixed by #10

Comments

@vergissberlin
Copy link

vergissberlin commented Mar 26, 2021 •
edited
Loading

Hi!

I'm using Alpine as a base for my cron container. And wget in Alpine has no option --delete-after which causes an exception on execution.

wget: unrecognized option: delete-after
BusyBox v1.31.1 () multi-call binary.

Usage: wget [-c|--continue] [--spider] [-q|--quiet] [-O|--output-document FILE]
        [-o|--output-file FILE] [--header 'header: value'] [-Y|--proxy on/off]
        [-P DIR] [-S|--server-response] [-U|--user-agent AGENT] [-T SEC] URL...

What I can see is, that you delete the files afterwords anyway.
So it should be no problem to remove that flag.

kind regards
André
vergissberlin added a commit to vergissberlin/t3fetch that referenced this issue Mar 26, 2021
@vergissberlin
Remove delete-after flag
a7266b1 
Resolves visuellverstehen#4
@vergissberlin vergissberlin mentioned this issue Mar 26, 2021
Closed
@malteriechmann
Copy link
Member

Hi there, thanks for reaching out.

We added this, because otherwise t3fetch/wget will take a lot of disk space while running. Sometimes websites have a lot of resources. If we download and store all of these, a lot of disk space would be needed.

From the documentation:

This option tells Wget to delete every single file it downloads, after having done so.

Well, I see your point. But just removing this flag will not really work.

Any other ideas?

vergissberlin pushed a commit to vergissberlin/t3fetch that referenced this issue Apr 19, 2021
visuellverstehen#4 Remove files after download
b0b367f
@vergissberlin
Copy link
Author

vergissberlin commented Apr 19, 2021 •
edited
Loading

Hi @malteriechmann,

with the filter on images and video files it shouldn't be too big.
Maybe use quota to limit the download size like so:

wget -r -q -Q 144k https://example.com -R 'mp4,mov,pdf,png'

An extension setting could be a compromise?

kind regards,
André

vergissberlin pushed a commit to vergissberlin/t3fetch that referenced this issue Apr 19, 2021
visuellverstehen#4 Avoid error on create directories
4203161
@malteriechmann
Copy link
Member

A filter on images and video files sound nice. Maybe we should add a setting to enable/disable the filter.

Do you want to create a pull request?

@runepiper runepiper linked a pull request Feb 20, 2024 that will close this issue
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
2 participants
@vergissberlin @malteriechmann