Commit graph

11 commits

Author SHA1 Message Date
Geoffrey Frogeye 3b6f7a58b3
Remove support for Rapid7
They changed their privacy / pricing model and as such I don't have
access to their massive DNS dataset anymore,
even after asking.

Since 2022-01-02, I put the list on freeze while looking for an alternative,
but couldn't find any.
To make the list update again with the remaining DNS sources I have,
I put the last version of the list generated with the Rapid7 dataset
as an input for subdomains, that will now get resolved with MassDNS.
2022-11-13 20:10:27 +01:00
Geoffrey Frogeye a4a908955a
Added index webpage 2019-12-27 15:21:33 +01:00
Geoffrey Frogeye 018f6548ea
Fixed feed_dns not saving in single-threaded mode
Would you believe it, seven hours of processing for nothing
2019-12-26 00:02:01 +01:00
Geoffrey Frogeye b310ca2fc2
Clever pruning mechanism 2019-12-25 14:54:57 +01:00
Geoffrey Frogeye c65ae94892
Added ability to use Rapid7 API
Closes #11
2019-12-24 15:08:18 +01:00
Geoffrey Frogeye 69b82d29fd
Improved rules handling
Rules can now come in 3 different formats:
- AdBlock rules
- Host lists
- Domains lists
All will be converted into domain lists and aggregated
(only AdBlock rules matching a whole domain will be kept).

Subdomains will now be matched if it is a subdomain of any domain of the
rule.
It is way faster (seconds rather than hours!) but less flexible
(although it shouldn't be a problem).
2019-12-03 08:48:12 +01:00
Geoffrey Frogeye 04fe454d99 Automatically get top 1M subdomains 2019-11-14 11:23:59 +01:00
Geoffrey Frogeye a0a2af281f Added possibility to add personal sources 2019-11-11 11:19:46 +01:00
Geoffrey Frogeye 2f1af3c850 Added progressbar and ETA 2019-11-10 21:59:06 +01:00
Geoffrey Frogeye d49a7803e9 Fixed typos 2019-11-10 18:29:16 +01:00
Geoffrey Frogeye 80b23e2d5c Initial commit 2019-11-10 18:14:25 +01:00