eulaurarien

geoffrey/eulaurarien

Fork 0

Commit graph

Author	SHA1	Message	Date
Geoffrey Frogeye	d7c239a6f6	Workflow: Some modifications	2019-12-14 16:04:19 +01:00
Geoffrey Frogeye	5023b85d7c	Added intermediate representation for DNS datasets It's just CSV. The DNS from the datasets are not ordered consistently, so we need to parse it completly. It seems that converting to an IR before sending data to ./feed_dns.py through a pipe is faster than decoding the JSON in ./feed_dns.py. This will also reduce the storage of the resolved subdomains by about 15% (compressed).	2019-12-13 21:59:35 +01:00

Author

SHA1

Message

Date

Geoffrey Frogeye

d7c239a6f6

Workflow: Some modifications

2019-12-14 16:04:19 +01:00

Geoffrey Frogeye

5023b85d7c

Added intermediate representation for DNS datasets

It's just CSV.
The DNS from the datasets are not ordered consistently,
so we need to parse it completly.
It seems that converting to an IR before sending data to ./feed_dns.py
through a pipe is faster than decoding the JSON in ./feed_dns.py.
This will also reduce the storage of the resolved subdomains by
about 15% (compressed).

2019-12-13 21:59:35 +01:00

2 commits