Skip to content

Commit

Permalink
unshit my script
Browse files Browse the repository at this point in the history
  • Loading branch information
NotaInutilis committed Oct 23, 2023
1 parent 65ef30c commit bfa5b52
Show file tree
Hide file tree
Showing 5 changed files with 5,056 additions and 2 deletions.
2 changes: 1 addition & 1 deletion scripts/update.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

# Cleanup sources:
## Special cleanup for imported sources of other formats (AdBlock, hosts, etc.).
find ./sources/_imported -type f -name "*.txt" -exec sed -ri '/^[^#[:punct:]]/s/^/#/; s/^0.0.0.0[[:space:]]*\.//i' {} \;
find ./sources/_imported -type f -name "*.txt" -exec sed -ri 's/^[^#[:alnum:]]/#&/; s/^0\.0\.0\.0[[:space:]]*//i' {} \;
## Normalizes URLs into domains: lowercases, remove leading spaces, protocol (`x://`) `www.` subdomains, everything after `/`, only one space before `#`. Keeps comments intact.
find ./sources -type f -name "*.txt" -exec sed -ri 'h; s/[^#]*//1; x; s/#.*//; s/.*/\L&/; s/^[[:space:]]*//i; s/^.*:\/\///i; s/^[.*]*//i; s/^www\.//i; s/\/[^[:space:]]*//i; s/[[:space:]].*$/ /i; G; s/(.*)\n/\1/' {} \;
## Remove duplicate domains from each source file (keeps repeated comments and empty lines for organization).
Expand Down
3 changes: 2 additions & 1 deletion sources/_imported/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@ https://peertube_isolation.frama.io/
https://github.com/DandelionSprout/adfilt/blob/master/Sensitive%20lists/TabloidRemover.txt
https://github.com/MassMove/AttackVectors/blob/master/LocalJournals/fake-local-journals-list.txt
https://assets.windscribe.com/custom_blocklists/clickbait.txt
https://github.com/StevenBlack/hosts/blob/master/alternates/fakenews-only/hosts
https://github.com/StevenBlack/hosts/blob/master/alternates/fakenews-only/hosts
https://github.com/antifa-n/pihole
Loading

0 comments on commit bfa5b52

Please sign in to comment.