ISC StormCast for Tuesday, September 8th 2015 http://isc.sans.edu/podcastdetail.html?id=4645

Hunting for IOC's with ioc-parser

Published: 2015-09-07. Last Updated: 2015-09-07 13:50:36 UTC
by Xavier Mertens (Version: 1)

Threat intelligence became a hot topic for a while. The food of threat intelligence is based on IOC's (Indicators of Compromise) which contains technical information like:

Files, path
Hashes
IP addresses
Domains
Users

Mixed with other sources of information or tools, they help in detecting malicious behaviors of programs or networks. They are plenty of sources to collect IOC's. Some are publicly available while others are compiled and maintained by organizations for their customers or restricted users. DShield is of course a good source of IP addresses but Lenny (another ISC handler) is maintaining a nice list of resource on his website(1). Usually, free services offer lists of IOC's in common format that are reusable in your own environment. But sometimes, you will find interesting information published online. Many security researchers analyze pieces of malware and publish the results on their blog. Big organizations like to publish nice PDF reports containing juicy information. In both case, IOC's can be present but how to extract them automatically?

ioc-parser(2) is a nice Python script which might be very helpful in this case. It parses an input file and generates a list of IOC's in another format. It supports the following input formats: Text files, PDF files or HTML (URLs). Results can be generated in CSV, JSON, YARA or NetFlow. The idea is simple, it searches for patterns based on regular expressions. Everything is configurable and your own regexp can be added.

Here is the list of IOC's extracted from an old PDF report about Duqu 2.0 written by Kasperky Lab:

$ ./iocp.py -p patterns.ini -i pdf -l pypdf2 The_Mystery_of_Duqu_2_0_a_sophisticated_cyberespionage_actor_returns.pdf
The_Mystery_of_Duqu_2_0_a_sophisticated_cyberespionage_actor_returns.pdf   2   Filename   msi.dll
The_Mystery_of_Duqu_2_0_a_sophisticated_cyberespionage_actor_returns.pdf   2   Filename   klif.dll
The_Mystery_of_Duqu_2_0_a_sophisticated_cyberespionage_actor_returns.pdf   2   Filename   12CTwoPENC.dll
The_Mystery_of_Duqu_2_0_a_sophisticated_cyberespionage_actor_returns.pdf   2   Filename   KMART.dll
The_Mystery_of_Duqu_2_0_a_sophisticated_cyberespionage_actor_returns.pdf   2   Filename   portserv.sys
The_Mystery_of_Duqu_2_0_a_sophisticated_cyberespionage_actor_returns.pdf   3   URL   https://en.wikipedia.org/wiki/Duqu
The_Mystery_of_Duqu_2_0_a_sophisticated_cyberespionage_actor_returns.pdf   3   URL   http://www.kaspersky.com/about/news/virus/2011/Duqu_The_Step_Brother_of_Stuxnet
The_Mystery_of_Duqu_2_0_a_sophisticated_cyberespionage_actor_returns.pdf   3   URL   http://70.auschwitz.org/index.php?lang=en
The_Mystery_of_Duqu_2_0_a_sophisticated_cyberespionage_actor_returns.pdf   3   Host   70.auschwitz.org
The_Mystery_of_Duqu_2_0_a_sophisticated_cyberespionage_actor_returns.pdf   3   CVE   CVE-2015-2360

But you can access URLs directly and extract IOC's present in the HTML code of the latest MalwareMustDie blog article:

$ ./iocp.py -p patterns.ini -i html -l requests -d http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   URL   http://www.blogger.com/go/cookiechoices
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   210.92.18.118
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.120.167.25
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.120.167.13
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.120.167.15
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.120.167.10
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.120.162.175
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.120.167.14
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.38.187.100
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.38.187.103
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.120.167.100
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.38.187.106
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.120.167.102
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.38.187.113
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.38.187.105
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.38.187.118
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   101.199.109.151
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.120.162.174
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.120.162.178
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   15.167.120.106
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.120.160.0
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.120.167.8
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   8.167.120.106
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.120.162.176
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   176.162.120.106
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   14.167.120.106
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.38.187.101
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.38.176.0
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.38.187.102
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.38.187.104
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.120.167.9
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   9.167.120.106
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   174.162.120.106
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.38.187.115
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.38.187.116
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   101.199.109.144
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   101.199.108.0
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.120.167.29
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   29.167.120.106
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   178.162.120.106
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.120.167.92
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   92.167.120.106
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.120.167.90
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   90.167.120.106
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   106.120.167.86
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   86.167.120.106
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   210.92.0.0
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   IP   222.186.34.220
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   Host   360.cn
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   Host   qi89.f3322.org
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   Host   qurl.qh-lb.com
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   Host   qup.qh-lb.com
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   Host   sdupm.360.cn
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   Host   sdup.360.cn
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   Host   sdup.qh-lb.com
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   Host   flux.sh
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   Host   15.167.120.106.static.bjtelecom.net
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   Host   chinatelecom.com.cn
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   Host   167.120.106.static.bjtelecom.net
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   Host   176.162.120.106.static.bjtelecom.net
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   Host   14.167.120.106.static.bjtelecom.net
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   Host   174.162.120.106.static.bjtelecom.net
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   Host   29.167.120.106.static.bjtelecom.net
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   Host   178.162.120.106.static.bjtelecom.net
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   Host   92.167.120.106.static.bjtelecom.net
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   Host   90.167.120.106.static.bjtelecom.net
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   Host   86.167.120.106.static.bjtelecom.net
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   Host   dshw.co.kr
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   Host   f3322.org
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   Host   astpbx.com
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   Host   libworker.so
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   Host   www.blogger.com
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html   1   Email   ppyy@astpbx.com

And the same results generated in YARA format:

$ ./iocp.py -p patterns.ini -i html -l requests -d -o yara http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html
rule mmd_0042_2015_hunting_mr_black_ids_via
{
   strings:
       $URL1 = "http://www.blogger.com/go/cookiechoices"
       $IP1 = "210.92.18.118"
       $IP2 = "106.120.167.25"
       $IP3 = "106.120.167.13"
       $IP4 = "106.120.167.15"
       $IP5 = "106.120.167.10"
       $IP6 = "106.120.162.175"
       $IP7 = "106.120.167.14"
       $IP8 = "106.38.187.100"
       $IP9 = "106.38.187.103"
       $IP10 = "106.120.167.100"
       $IP11 = "106.38.187.106"
       $IP12 = "106.120.167.102"
       $IP13 = "106.38.187.113"
       $IP14 = "106.38.187.105"
       $IP15 = "106.38.187.118"
       $IP16 = "101.199.109.151"
       $IP17 = "106.120.162.174"
       $IP18 = "106.120.162.178"
       $IP19 = "15.167.120.106"
       $IP20 = "106.120.160.0"
       $IP21 = "106.120.167.8"
       $IP22 = "8.167.120.106"
       $IP23 = "106.120.162.176"
       $IP24 = "176.162.120.106"
       $IP25 = "14.167.120.106"
       $IP26 = "106.38.187.101"
       $IP27 = "106.38.176.0"
       $IP28 = "106.38.187.102"
       $IP29 = "106.38.187.104"
       $IP30 = "106.120.167.9"
       $IP31 = "9.167.120.106"
       $IP32 = "174.162.120.106"
       $IP33 = "106.38.187.115"
       $IP34 = "106.38.187.116"
       $IP35 = "101.199.109.144"
       $IP36 = "101.199.108.0"
       $IP37 = "106.120.167.29"
       $IP38 = "29.167.120.106"
       $IP39 = "178.162.120.106"
       $IP40 = "106.120.167.92"
       $IP41 = "92.167.120.106"
       $IP42 = "106.120.167.90"
       $IP43 = "90.167.120.106"
       $IP44 = "106.120.167.86"
       $IP45 = "86.167.120.106"
       $IP46 = "210.92.0.0"
       $IP47 = "222.186.34.220"
       $Host1 = "360.cn"
       $Host2 = "qi89.f3322.org"
       $Host3 = "qurl.qh-lb.com"
       $Host4 = "qup.qh-lb.com"
       $Host5 = "sdupm.360.cn"
       $Host6 = "sdup.360.cn"
       $Host7 = "sdup.qh-lb.com"
       $Host8 = "flux.sh"
       $Host9 = "15.167.120.106.static.bjtelecom.net"
       $Host10 = "chinatelecom.com.cn"
       $Host11 = "167.120.106.static.bjtelecom.net"
       $Host12 = "176.162.120.106.static.bjtelecom.net"
       $Host13 = "14.167.120.106.static.bjtelecom.net"
       $Host14 = "174.162.120.106.static.bjtelecom.net"
       $Host15 = "29.167.120.106.static.bjtelecom.net"
       $Host16 = "178.162.120.106.static.bjtelecom.net"
       $Host17 = "92.167.120.106.static.bjtelecom.net"
       $Host18 = "90.167.120.106.static.bjtelecom.net"
       $Host19 = "86.167.120.106.static.bjtelecom.net"
       $Host20 = "dshw.co.kr"
       $Host21 = "f3322.org"
       $Host22 = "astpbx.com"
       $Host23 = "libworker.so"
       $Host24 = "www.blogger.com"
       $Email1 = "ppyy@astpbx.com"
   condition:
       $URL1 or $IP1 or $IP2 or $IP3 or $IP4 or $IP5 or $IP6 or $IP7 or $IP8 or $IP9 or $IP10 or $IP11 or $IP12 or $IP13 or $IP14 or $IP15 or $IP16 or $IP17 or $IP18 or $IP19 or $IP20 or $IP21 or $IP22 or $IP23 or $IP24 or $IP25 or $IP26 or $IP27 or $IP28 or $IP29 or $IP30 or $IP31 or $IP32 or $IP33 or $IP34 or $IP35 or $IP36 or $IP37 or $IP38 or $IP39 or $IP40 or $IP41 or $IP42 or $IP43 or $IP44 or $IP45 or $IP46 or $IP47 or $Host1 or $Host2 or $Host3 or $Host4 or $Host5 or $Host6 or $Host7 or $Host8 or $Host9 or $Host10 or $Host11 or $Host12 or $Host13 or $Host14 or $Host15 or $Host16 or $Host17 or $Host18 or $Host19 or $Host20 or $Host21 or $Host22 or $Host23 or $Host24 or $Email1
}

This is a nice script to keep in your personal toolbox. Of course, be careful to not re-use the generated data "as is", there could be false positives or bad regular expression matches.

Happy IOC's hunting!

Xavier Mertens
ISC Handler - Freelance Security Consultant
rootshell.be
truesec.be

(1) https://zeltser.com/malicious-ip-blocklists/
(2) https://github.com/armbues/ioc_parser

Keywords:

10 comment(s)

Internet Storm Center

Hunting for IOC's with ioc-parser

Comments