If you’re a regular reader of our Diaries, you may remember that over the last year and a half, a not insignificant portion of my posts has been devoted to discussing some of the trends in internet-connected systems. We looked at changes in the number of internet-facing machines affected by BlueKeep[1], SMBGhost[2], Shitrix[3] and several other vulnerabilities [4] as well as at the changes in TLS 1.3 support over time[5] and several other areas [6,7].  Today, we’re going to take a look at the tool, that I’ve used to gather data, on which the Diaries were based, from Shodan.io.

In keeping with the Shodan theme, I’ve called the tool TriOp. It is a Python script, which enables you to quickly build a list (or multiple lists) of Shodan search terms and then repeatedly (e.g. once every day) get the count of public IPs, which satisfy each of them, from the Shodan API.

The basic use of the tool is quite straightforward – in addition to creating a Shodan account and getting its API key, one only needs to create a list of Shodan queries one wishes to monitor over time, input this list into TriOp (as a CSV file where the first row specifies the queries or through a command line as a comma-separated list) and the tool will do the rest. It can output the results of the queries to command line, but its more useful feature is the option to save the results in a CSV, which can later be used as an input for TriOp.

 

The basic search can be done using the -s/--search option in the following way:

triop.py -s "port:80,port:443"

In such a case, the output might look like this:

TriOp 1.0
Current IP count for query port:80 is 72010982
Current IP count for query port:443 is 59072465

Where things get interesting is the output to a file using the -O/--output_file option:

triop.py -s "port:80,port:443" -O http_ports.csv

The resultant CSV file would have the following structure:

Date,2021-01-26
port:80, 72036704
port:443, 59145503

You can probably see why using the same file as input on another day might be useful. If one were to use this file as an input today using the -S/--search_file option and add the -a/--append option, TriOp would add another row to the file with today’s date.

triop.py -S http_ports.csv -a

The updated CSV file would then look like this:

Date,2021-01-26,2021-01-27
port:80,72036704,72010982
port:443,59145503,59072465

If one wanted to monitor the situation on a day to day basis, one would only need to run the same command each day (preferably using some automatic scheduling mechanism).

Although gathering data about the number of public IPs with different ports open to the internet may be interesting, as it gives us some idea about how the global network changes over time[8], we are certainly not limited to just the “ports:” filter.

Since TriOp only gets a "count" for each of the queries and not the related list of IP addresses, which satisfy the queries, one may use any combination of Shodan search filters with it (even those, which are normally accessible only to enterprise or researches-level accounts), even with a free account.

This means that one may use TriOp to monitor the changes in different open ports in specific IP ranges (filter “net:”), ASNs (filter “asn:”) or countries (filter “country:”), but also to monitor changes in the number of IPs affected by specific vulnerabilities (filter “vuln:”), systems with specific JARM[9] fingerprints (filter “ssl.jarm:”), etc.

Since we’ve mentioned vulnerabilities, if this is area that interests you, you may also use TriOp as a high-level “passive” vulnerability scanner. Shodan itself detects machines affected by some vulnerabilities – currently it seems to be able to identify about 2246 of the approximately 190k CVEs published so far, according to the results of my tests[10] – and nothing is stopping us from getting the “count” for these. List of the CVEs “supported” by Shodan, which I’ve been able to identify, are included in TriOp and one may search for them simply using any query in combination with the --vuln_search_level option:

triop.py -s "country:US" --vuln_search_level 3

The previous command would result in a very long output giving us the number of public IPs in the US, on which systems vulnerable to specific CVEs might be found:

TriOp 1.0
Checking whether Shodan detects any vulnerabilities for search country:US.
Current IP count for query country:US has_vuln:true is 10398899
Current IP count for query country:US is 160792718
Current IP count for query country:US vuln:CVE-1999-0045 is 1
...

The tool has additional features as well (adding new queries to existing search files, exporting data related to similar searches from multiple search files, etc.) and you may find some of them demonstrated in the tutorial video bellow.

The one last feature I will mention here is the ability to load multiple input files based on a specified “mask”. I’ve originally created TriOp with the intention to monitor changes in the number of vulnerable systems, ports and services both globally as well as in different countries and I’ve created over a hundred different search files by now (one per each country I was interested in as well, several for different sets of vulnerabilities, etc.). In order for me to be able to “update” each of them on a daily basis, TriOp supports the --filename_load option, which enables one to specify a string, which is then used to select files which should be used as inputs. If one were to use the following mask for example, all CSV files in the current folder would be used as inputs and consequently updated.

triop.py --filename_load .csv -a

As you may see, although TriOp is a fairly simple tool, which only gathers “counts” for each of the submitted queries, its outputs can be quite useful. This goes especially for any (national) CSIRT, which wants to monitor public IPs of its constituency, but lacks a capability to scan it on a daily basis, or for any security researcher who wants to, for example, compare the number of devices affected by specific vulnerabilities in different countries.

In any case, if you’d like to try TriOp yourself, you may download it from my GitHub page.

[1] https://isc.sans.edu/diary/25506
[2] https://isc.sans.edu/diary/26732
[3] https://isc.sans.edu/diary/26900
[4] https://isc.sans.edu/diary/26798
[5] https://isc.sans.edu/diary/26936
[6] https://isc.sans.edu/diary/25854
[7] https://isc.sans.edu/diary/26374
[8] https://untrustednetwork.net/en/2021/01/01/open-ports-statistics-for-2020/
[9] https://engineering.salesforce.com/easily-identify-malicious-servers-on-the-internet-with-jarm-e095edac525a
[10] https://untrustednetwork.net/en/2020/11/18/most-common-vulnerabilities-based-on-shodan/

-----------
Jan Kopriva
@jk0pr
Alef Nula