Commit graph

170 commits

Author SHA1 Message Date
Geoffrey Frogeye 53049d4781
Fix support for DOS newlines in list files 2024-12-07 16:12:20 +01:00
Geoffrey Frogeye 3b6f7a58b3
Remove support for Rapid7
They changed their privacy / pricing model and as such I don't have
access to their massive DNS dataset anymore,
even after asking.

Since 2022-01-02, I put the list on freeze while looking for an alternative,
but couldn't find any.
To make the list update again with the remaining DNS sources I have,
I put the last version of the list generated with the Rapid7 dataset
as an input for subdomains, that will now get resolved with MassDNS.
2022-11-13 20:10:27 +01:00
Geoffrey Frogeye 49a36f32f2
Add requirements.txt file 2022-02-26 13:01:11 +01:00
Geoffrey Frogeye 29cf72ae92 Fix most of the README being bold
Why did I go with this Markdown generator again?
2021-08-28 20:58:34 +02:00
Geoffrey Frogeye 998c3faf8f
Add SAS.com 2021-08-22 18:02:37 +02:00
Geoffrey Frogeye c8a14a4e21
Add DataUnlocker 2021-08-22 17:07:25 +02:00
Geoffrey Frogeye 1ec26e7f96
Add Plausible.io 2021-08-22 16:53:58 +02:00
Geoffrey Frogeye 5b49441bc0 Add Branch.io tracker 2021-08-22 16:37:31 +02:00
Geoffrey Frogeye afd122f2ab
Update usage recommendations 2021-08-15 13:04:55 +02:00
Geoffrey Frogeye 6ae3d5fb55
Add Lead Forensics tracker 2021-08-15 11:39:37 +02:00
Geoffrey Frogeye 10a505d84f
Add Fathom 2021-08-15 11:18:35 +02:00
Geoffrey Frogeye c06648da53
Added Pardot tracker 2021-08-15 11:06:53 +02:00
Geoffrey Frogeye f165e5a094
Fix (most) mypy / flake8 errors 2021-08-14 23:35:51 +02:00
Geoffrey Frogeye 3dcccad39a
Black pass 2021-08-14 23:27:28 +02:00
Geoffrey Frogeye a023dc8322
Fix deprecated np.bool 2021-08-14 23:21:03 +02:00
Geoffrey Frogeye 389e83d492
Fix database maximum cache size cap 2021-08-14 23:19:12 +02:00
Geoffrey Frogeye edf444cc28
Add ad-cloud.jp and improve names of Japanese trackers
Closes #19

Names from https://github.com/AdguardTeam/cname-trackers/issues/1
2021-08-14 22:55:58 +02:00
Geoffrey Frogeye fa23d466d2
Actually remove ThreatMetrix
Forgot -i when grepping
2021-08-14 21:55:44 +02:00
Geoffrey Frogeye f5f9f88c42
Remove ThreatMetrix
I received a lot of false positives for this one,
and while I wasn't able to reproduce the issue in most of the cases,
I trust the community.
It's also not in any other CNAME tracker list, probably for the same reason.
Plus, it's apparently not very nasty.
So I'll let it go.

Closes #17
2021-08-14 21:24:48 +02:00
Geoffrey Frogeye 2997e41f98
Investigated >0.5% trackers from Fukuda paper 2020-12-19 13:41:07 +01:00
Geoffrey Frogeye 6cf1028174
Added other tracking source for Adobe
Found on the Adobe documentation and in the wild
https://experienceleague.adobe.com/docs/analytics/implementation/vars/config-vars/trackingserversecure.html?lang=en#s.trackingserversecure-in-appmeasurement-and-launch-custom-code-editor
2020-12-19 13:15:38 +01:00
Geoffrey Frogeye b98a37f9da
Add 1st chain Act-On
To unclobber -only lists
2020-12-07 08:27:20 +01:00
Geoffrey Frogeye 8828d4cf24
Investigated >1% trackers from Fukuda paper 2020-12-07 00:03:58 +01:00
Geoffrey Frogeye 04205dd9fc
Add AdGuard in the distribution README 2020-12-06 23:18:27 +01:00
Geoffrey Frogeye cec96b7e50
Add Fukuda & co research paper to test suite 2020-12-06 22:13:05 +01:00
Geoffrey Frogeye eb1fcefd49
Use more correct terms 2020-12-06 21:29:48 +01:00
Geoffrey Frogeye 0ecb431728 Add AdGuard for multiparty 2020-12-06 21:01:24 +01:00
Geoffrey Frogeye c1619b3cff Add more sources and acknowledgement 2020-12-06 21:01:20 +01:00
Geoffrey Frogeye 2c0286e36b
Add genieesspv.jp CNAME tracker
Closes #18
2020-08-22 10:46:43 +02:00
Geoffrey Frogeye 954bc86eaa
More Tracedock domains
From https://gist.github.com/pietvanzoen/ed7b8322a552542bc00a83ced7332d33
2020-08-08 09:14:09 +02:00
Geoffrey Frogeye b09f861c27
README: Added more reasons the browsers trust first party 2020-01-11 13:01:51 +01:00
Geoffrey Frogeye 9326dc6aca
Added similar projects 2020-01-11 11:43:14 +01:00
Geoffrey Frogeye c803a714fa
I don't know how to write the word “explanation“... 2020-01-11 11:31:16 +01:00
Geoffrey Frogeye b3a3219f93
Improved usage scenarios for different lists 2020-01-11 11:26:54 +01:00
Geoffrey Frogeye fbc06f71bb
Added symlink to latest explaination 2020-01-07 14:37:01 +01:00
Geoffrey Frogeye 63ab7651fc
Disabled RDNS import due to #15 2020-01-07 14:17:38 +01:00
Geoffrey Frogeye 0724feed26
README: Removed help message and fixed category for finder 2020-01-06 16:44:45 +01:00
Geoffrey Frogeye adb07417f5
Fixed import_rapid7 script typo 2020-01-05 22:35:12 +01:00
Geoffrey Frogeye 0cc18303fd
Re-import Rapid7 datasets when rules have been updated 2020-01-04 10:54:46 +01:00
Geoffrey Frogeye 708c53041e
Added two japanese trackers 2020-01-03 22:09:16 +01:00
Geoffrey Frogeye 808e36dde3
Improvements to subdomain collection
I use this for tracker identification so it's not perfect but still it's
a bit better.
2020-01-03 22:08:06 +01:00
Geoffrey Frogeye 2b97ee4cb9
Better list output 2019-12-27 21:46:57 +01:00
Geoffrey Frogeye fd8bfee088
Improved -only variants descriptions 2019-12-27 15:58:20 +01:00
Geoffrey Frogeye e93807142c
Explanations folder 2019-12-27 15:35:30 +01:00
Geoffrey Frogeye a4a908955a
Added index webpage 2019-12-27 15:21:33 +01:00
Geoffrey Frogeye 7e06e98808
Added TraceDock FP tracker
Thought they did change the URL of their load balancers,
guess I was wrong.
2019-12-27 13:43:38 +01:00
Geoffrey Frogeye 4fca68c6f0
Fixed handling of unknown field error 2019-12-27 01:10:21 +01:00
Geoffrey Frogeye 54a9c78534
Handled another error 2019-12-26 20:38:35 +01:00
Geoffrey Frogeye 171fa93873
Force pv output
Even if redirected to a file
Allow to see progress when ran in a cron or something
2019-12-26 15:38:56 +01:00
Geoffrey Frogeye 095e51fad9
Ensure massdns output is lower case
For some reason some server output part of their response as upper case.
This fails the reading process as it's designed to only work on lower
case for performance reasons.
2019-12-26 15:32:24 +01:00