Commit graph

173 commits

Author SHA1 Message Date
Geoffrey Frogeye eabe4b84ae
massdns data import better error handling 2024-12-12 01:28:09 +01:00
Geoffrey Frogeye d8a3e2ac9c
Allow selecting region for DNS resolvers 2024-12-09 19:38:14 +01:00
Geoffrey Frogeye 3acffde75a
massdns data import should work with incomplete files 2024-12-09 19:36:52 +01:00
Geoffrey Frogeye 53049d4781
Fix support for DOS newlines in list files 2024-12-07 16:12:20 +01:00
Geoffrey Frogeye 3b6f7a58b3
Remove support for Rapid7
They changed their privacy / pricing model and as such I don't have
access to their massive DNS dataset anymore,
even after asking.

Since 2022-01-02, I put the list on freeze while looking for an alternative,
but couldn't find any.
To make the list update again with the remaining DNS sources I have,
I put the last version of the list generated with the Rapid7 dataset
as an input for subdomains, that will now get resolved with MassDNS.
2022-11-13 20:10:27 +01:00
Geoffrey Frogeye 49a36f32f2
Add requirements.txt file 2022-02-26 13:01:11 +01:00
Geoffrey Frogeye 29cf72ae92 Fix most of the README being bold
Why did I go with this Markdown generator again?
2021-08-28 20:58:34 +02:00
Geoffrey Frogeye 998c3faf8f
Add SAS.com 2021-08-22 18:02:37 +02:00
Geoffrey Frogeye c8a14a4e21
Add DataUnlocker 2021-08-22 17:07:25 +02:00
Geoffrey Frogeye 1ec26e7f96
Add Plausible.io 2021-08-22 16:53:58 +02:00
Geoffrey Frogeye 5b49441bc0 Add Branch.io tracker 2021-08-22 16:37:31 +02:00
Geoffrey Frogeye afd122f2ab
Update usage recommendations 2021-08-15 13:04:55 +02:00
Geoffrey Frogeye 6ae3d5fb55
Add Lead Forensics tracker 2021-08-15 11:39:37 +02:00
Geoffrey Frogeye 10a505d84f
Add Fathom 2021-08-15 11:18:35 +02:00
Geoffrey Frogeye c06648da53
Added Pardot tracker 2021-08-15 11:06:53 +02:00
Geoffrey Frogeye f165e5a094
Fix (most) mypy / flake8 errors 2021-08-14 23:35:51 +02:00
Geoffrey Frogeye 3dcccad39a
Black pass 2021-08-14 23:27:28 +02:00
Geoffrey Frogeye a023dc8322
Fix deprecated np.bool 2021-08-14 23:21:03 +02:00
Geoffrey Frogeye 389e83d492
Fix database maximum cache size cap 2021-08-14 23:19:12 +02:00
Geoffrey Frogeye edf444cc28
Add ad-cloud.jp and improve names of Japanese trackers
Closes #19

Names from https://github.com/AdguardTeam/cname-trackers/issues/1
2021-08-14 22:55:58 +02:00
Geoffrey Frogeye fa23d466d2
Actually remove ThreatMetrix
Forgot -i when grepping
2021-08-14 21:55:44 +02:00
Geoffrey Frogeye f5f9f88c42
Remove ThreatMetrix
I received a lot of false positives for this one,
and while I wasn't able to reproduce the issue in most of the cases,
I trust the community.
It's also not in any other CNAME tracker list, probably for the same reason.
Plus, it's apparently not very nasty.
So I'll let it go.

Closes #17
2021-08-14 21:24:48 +02:00
Geoffrey Frogeye 2997e41f98
Investigated >0.5% trackers from Fukuda paper 2020-12-19 13:41:07 +01:00
Geoffrey Frogeye 6cf1028174
Added other tracking source for Adobe
Found on the Adobe documentation and in the wild
https://experienceleague.adobe.com/docs/analytics/implementation/vars/config-vars/trackingserversecure.html?lang=en#s.trackingserversecure-in-appmeasurement-and-launch-custom-code-editor
2020-12-19 13:15:38 +01:00
Geoffrey Frogeye b98a37f9da
Add 1st chain Act-On
To unclobber -only lists
2020-12-07 08:27:20 +01:00
Geoffrey Frogeye 8828d4cf24
Investigated >1% trackers from Fukuda paper 2020-12-07 00:03:58 +01:00
Geoffrey Frogeye 04205dd9fc
Add AdGuard in the distribution README 2020-12-06 23:18:27 +01:00
Geoffrey Frogeye cec96b7e50
Add Fukuda & co research paper to test suite 2020-12-06 22:13:05 +01:00
Geoffrey Frogeye eb1fcefd49
Use more correct terms 2020-12-06 21:29:48 +01:00
Geoffrey Frogeye 0ecb431728 Add AdGuard for multiparty 2020-12-06 21:01:24 +01:00
Geoffrey Frogeye c1619b3cff Add more sources and acknowledgement 2020-12-06 21:01:20 +01:00
Geoffrey Frogeye 2c0286e36b
Add genieesspv.jp CNAME tracker
Closes #18
2020-08-22 10:46:43 +02:00
Geoffrey Frogeye 954bc86eaa
More Tracedock domains
From https://gist.github.com/pietvanzoen/ed7b8322a552542bc00a83ced7332d33
2020-08-08 09:14:09 +02:00
Geoffrey Frogeye b09f861c27
README: Added more reasons the browsers trust first party 2020-01-11 13:01:51 +01:00
Geoffrey Frogeye 9326dc6aca
Added similar projects 2020-01-11 11:43:14 +01:00
Geoffrey Frogeye c803a714fa
I don't know how to write the word “explanation“... 2020-01-11 11:31:16 +01:00
Geoffrey Frogeye b3a3219f93
Improved usage scenarios for different lists 2020-01-11 11:26:54 +01:00
Geoffrey Frogeye fbc06f71bb
Added symlink to latest explaination 2020-01-07 14:37:01 +01:00
Geoffrey Frogeye 63ab7651fc
Disabled RDNS import due to #15 2020-01-07 14:17:38 +01:00
Geoffrey Frogeye 0724feed26
README: Removed help message and fixed category for finder 2020-01-06 16:44:45 +01:00
Geoffrey Frogeye adb07417f5
Fixed import_rapid7 script typo 2020-01-05 22:35:12 +01:00
Geoffrey Frogeye 0cc18303fd
Re-import Rapid7 datasets when rules have been updated 2020-01-04 10:54:46 +01:00
Geoffrey Frogeye 708c53041e
Added two japanese trackers 2020-01-03 22:09:16 +01:00
Geoffrey Frogeye 808e36dde3
Improvements to subdomain collection
I use this for tracker identification so it's not perfect but still it's
a bit better.
2020-01-03 22:08:06 +01:00
Geoffrey Frogeye 2b97ee4cb9
Better list output 2019-12-27 21:46:57 +01:00
Geoffrey Frogeye fd8bfee088
Improved -only variants descriptions 2019-12-27 15:58:20 +01:00
Geoffrey Frogeye e93807142c
Explanations folder 2019-12-27 15:35:30 +01:00
Geoffrey Frogeye a4a908955a
Added index webpage 2019-12-27 15:21:33 +01:00
Geoffrey Frogeye 7e06e98808
Added TraceDock FP tracker
Thought they did change the URL of their load balancers,
guess I was wrong.
2019-12-27 13:43:38 +01:00
Geoffrey Frogeye 4fca68c6f0
Fixed handling of unknown field error 2019-12-27 01:10:21 +01:00