Geoffrey Frogeye
7d01d016a5
It's not very performant by itself, especially since pyre2 isn't maintained nor really compilableinstallable anymore. The performance seems to have decreased from 200 req/s to 0.2 req/s when using 512 threads, and to 80 req/s using 64 req/s. This might or might not be related,as the CPU doesn't seem to be the bottleneck. I will probably add support for host-based rules, matching the subdomains of such hosts (as for now there doesn't seem to be any other pattern for first-party trackers than subdomains, and this would be a very broad performace / compatibility with existing lists improvement), and convert the AdBlock lists to this format, only keeping domains-only rules.
20 lines
540 B
Bash
Executable file
20 lines
540 B
Bash
Executable file
#!/usr/bin/env bash
|
|
|
|
# Get rules
|
|
curl https://easylist.to/easylist/easyprivacy.txt > rules/easyprivacy.cache.txt
|
|
|
|
# Get a list of nameservers
|
|
|
|
rm -f nameservers
|
|
touch nameservers
|
|
[ -f nameservers.head ] && cat nameservers.head >> nameservers
|
|
curl https://public-dns.info/nameservers.txt | sort -R >> nameservers
|
|
|
|
# Get top 1M subdomains
|
|
|
|
wget http://s3-us-west-1.amazonaws.com/umbrella-static/top-1m.csv.zip
|
|
unzip top-1m.csv.zip
|
|
sed 's|^[0-9]\+,||' top-1m.csv > subdomains/cisco-umbrella_popularity.cache.list
|
|
rm top-1m.csv top-1m.csv.zip
|
|
|