This is a quick post to demonstrate how you can utilise the bulk whois tool we have just pushed to our github. It is a script that was built a while back during an investigation; however the previous version wasn’t brilliant, it lacked error handling and often encountered connection resets, so this version has been rewritten to counter those issues!
The tool bulk_whois.py takes a list of domains and extracts the following information at the moment:
- Creation Date
- Updated Date
- Expiration Date
- Organisation
- Name
- Phone
- Street
- City
- Postcode
- Country
The information that is retrieved can either be pushed to a csv file or printed to stdout in the terminal window.
Lets do a quick demonstration into how you can utilise this tool to aid you from both a curiosity perspective, but more importantly from an investigative perspective.
In this example I loaded wireshark and captured traffic for around 2 minutes whilst accessing several sites in my browser (making dns requests). Once I had saved the output I then needed to extract the DNS requests that were made within that time frame, which would provide us with a list of domains to use in conjunction with the bulk whois tool. To do this I ran the following command (also shown in the screenshot below): tshark -r [pcap_file] -T fields -e dns.qry.name -Y “dns.flags.response eq 0 and dns.qry.name” > domains.txt
Awesome I now have a list of DNS hostnames that have been extracted from the pcap to a file named domains.txt.
Lets quickly breakdown the command I ran using tshark. The ‘-r’ switch is used to specify the input file we wish to parse, in this instance our pcap was labelled pcap01.pcap. After this I then used the ‘-T’ switch to set the format of the output followed by ‘-e’. The ‘-e’ switch allows us to tell tshark what fields we want it to display in the output. Lastly I used the ‘-Y’ switch to specify the filter to be applied when parsing the pcap file.
The filter that was applied in this instance was: dns.flags.response eq 0 and dns.qry.name.
dns.flags.response eq 0 – This filter tells tshark that we only want to see packets that have the dns flag response of 0. A DNS response flag of 0 is a query. The image below shows the type of data we are applying our filter (taken within wireshark):
dns.qry.name – This filter tells tshark to filter and only use packets that contain the dns.qry.name field.
Using bulk_whois.py
Now that a list of domains have been extracted from the pcap (note other methods other than tshark can be used) we can now utilise the bulk whois script to retrieve the information mentioned at the start of this post from the domains.
When using bulk_whois.py you can specify the -o switch and enter a filename of your choice. This will ensure that the whois information that is retrieved is pushed to a csv file for later review. Alternatively you can exclude the -o switch and the output will be printed to stdout within the terminal window.
By specifying the -i switch it tells the script which file contains the list of domains to parse. Shown below is two examples of the script printing to stdout, and exporting the information to a csv file named whois.csv.
stdout
CSV File
That’s almost it! Now you understand how you can utilise the script to extract whois information for a list of domains. We added a sleep of two seconds between each lookup; this prevents you receiving connection resets from the lookup server, however whilst it solves one problem it makes the script a little slow when performing lookups against a large list of domains. We are working on this issue so bare with us, we should have a solution pushed out soon!
Also if you have read the SNI sniffing blog post we also mentioned another tool we developed for extracting SNIs from SSL/TLS traffic. This can be used in conjuction with this tool!!
Once you have your SNIs logged to a file using the script which can be downloaded here or on from our github, run one of the following commands to make a list of domains to parse to this script from the output:
- cat [output_file] | awk -F”,” ‘{print $4}’
- cat sni_sniffed.csv | cut -d, -f4
Whois information is extremely handy information to retrieve during investigations, especially data such as creation/update date when attempting to identify fast flux domains.
