One practice I strongly suggest you undertake, for both yourselves and your organizations, is Open Source Intelligence (OSINT) gathering. OSINT is actively utilized as part of the reconnaissance phase for penetration testing. Assume the bad guys are doing it too, so add it to the list of tactics in which you need to be proficient so as to better defend yourselves. Per Sec. 931 of Public Law 109-163, or the National Defense Authorization Act for Fiscal Year 2006, OSINT is “intelligence that is produced from publicly available information and is collected, exploited, and disseminated in a timely manner to an appropriate audience for the purpose of addressing a specific intelligence requirement.” As a tool aficionado and regular author on tool topics I am occasionally faced with a scenario we in the information security practices struggle with at times. Some of the tools I discuss and espouse are as easily used for evil as they are for good.  I’ve seen very recent evidence of this as it pertains to articles I’ve written about OSINT tools FOCA and Maltego.  Quoting the Roman philosopher Seneca the Younger, “A sword never kills anybody; it’s a tool in the killer’s hand.” As one who would rather see the sword in the good guy’s hand, I was recently asked by a fellow white hat if there is an elegant way to export FOCA results for direct import into Maltego. The short answer is no, so I endeavored to solve the problem. FOCA includes the option to Save log to File resulting in a tab separated values text file inclusive of Time, Source, Severity, and Message. As an example, from a FOCA analysis of my domain, log file output resembles:

11:25:23 PM   ShodanSearch   medium   Found IP Information 70.40.197.37

8:50:18 AM   DNSCommonNames   medium   [74.220.195.31] Found subdomain blog.holisticinfosec.org

The fundamental challenge with taking this output from FOCA and importing it into Maltego is that there are no readily consumable entities. Maltego now imports CSV files quite nicely but again the CSV must be formatted accordingly where column content matches an entity type in order to conduct transforms To solve this little problem I wrote two down and dirty little PowerShell scripts using regex to parse unique IP addresses (ipAddress.ps1) or unique domain names (URI.ps1) from a FOCA log and write results to a CSV for import into Maltego. Grab them from the foca2maltego CodePlex project site. Yes, PowerShell favors Windows users; feel free to rip and replace for use in Python or your preferred scripting venue.

1. From a PowerShell prompt run .\ipAddress.ps1 or .\URI.ps1. You’ll be asked to provide the path to your FOCA log file then the script will write a CSV file to your working directory (IP_parsed.csv or domain_parsed.csv).
2. In Maltego, click the Maltego menu icon then Import | Import graph from table… | select CSV from the file type drop down menu | select CSV created by script | click on unmapped column to highlight it & select appropriate entity type (Domain,  DNS Name, IPv4 Address, etc.) then Next | keep default in Sampling window & click Next | click Finish.
3. Once all the entities populate the Maltego UI (there may be many if your FOCA output was extensive), you can select them in groups or as individuals to conduct further analysis and establish possible relationships.

The regex is really lean and probably overly simple. As an example \b(?:[a-z0-9]+(?:-[a-z0-9]+)*\.)+[a-z]{2,}\b for domain name parsing misinterprets .txt or .pdf in a full URI for a file as a TLD. As such, I built a not match exclusion for PDF, as an example. Feel free to expand on it as you see fit. I also dropped the ShodanHQ name space and URLs from the FOCA output as there is an entire Shodan toolkit for Maltego. Given my poor excuse for scripting and development skills you may likely find shortcomings or better ways to solve the task. Do feel free to enlighten me and our readers via the comments form. I’ll keep tuning, look for other entity opportunities, and create similar Python options to add to the project site; you’re welcome to contribute.