When analyzing malware, it is often convenient to infect an isolated laboratory system with the sample to observe how it behaves. Behavioral analysis often involves performing experiments iteratively, slightly varying the lab environment to evoke new behavior and learn about the sample's capabilities. To accomplish this, we need the ability to quickly revert to a known state of the laboratory system.

Restoring state using VMware

Malware analysis like using virtualization software--usually VMware--for setting up the lab. VMware offers the convenience of taking a snapshot of the virtual machine with a click of a button. Reverting to a known state after that is just another button-click away.

VMware Server, which is available for free, supports a single snapshot of the virtual machine. VMware Workstation,a commercial product, supports multiple snapshots in a highly flexible manner. It costs $189.  (Microsoft Virtual PC seems to some snapshot capabilities, too, but I am not very familiar with it.)

Malware authors often check whether their programs are running within a virtual machine.Techniques for concealing the use of virtualization involve patching the executable to deactivate the virtualization-checking code, or using a debugger to return spoofed results to virtualization checks. (If this is interesting, check out the recent additions to my malware analysis course.)

Sometimes it is easier to move away from a virtual to a physical system, rather than to locate and manipulate the virtualization-checking code.

Disk cloning via dd (software)

Disk cloning software, such as Ghost or dd allows the analyst to save the laboratory system's hard disk image, and then reapply it after completing the analysis. (dd is available for free for pretty much all Unix-flavored operating systems.)

Cloning large disks via this method may be time-consuming. However, while not as convenient as clicking a button to revert the system's state, it is a time-tested  and reliable method. We received the following details of a lab setup from ISC reader Tyler Hudak:

"After initially installing the OS we are going to test malware on, we save it off while in Linux using dd and gzip (like you would when imaging a drive for forensics). Whenever we want to run a test in that OS, we just re-apply the image and reboot. The partition is small so it takes less than 2-3 minutes (on slow hardware) to perform. This method is advantageous in that its OS-independent - we could test malware on Windows or *nux/*BSD w/o worrying about if the tool is compatible."

Tyler further described a free tool, that helps apply changes to a dd disk image:

"Since we test malware in different OS configurations (e.g. different patch levels, different AV installed) one of my fellow workers wrote a program called ddp (dd-delta-patch). We use this to create a patch from an existing dd image and then re-apply it when we want to run that specific configuration. We've released ddp and it can be downloaded from http://www.korelogic.com/tools.html."

If disk cloning is not convenient or fast enough for you, several tools are available for quickly rolling back the system to a pristine state.

Deep Freeze (software)

Once installed on the physical system, Deep Freeze lets you "freeze" the system's configuration in its pristine state, automatically reverting to that configuration when necessary after a reboot.

DeepFreeze is available for Windows, OS X, and Linux operating systems. It's sold in 10-packs and is priced from $13.55 per system. The price depends on your industry.

Windows SteadyState (software)

Windows SteadyState is a free product from Microsoft, and is available for Windows XP. Like Deep Freeze, SteadyState is positioned to help lock-down public systems, such as Internet kiosks and library computers. It has the ability to restore the system to a known state via its Disk Protection feature.

Returnil

Another product in this category is Returnil. It is marketed as a tool for combating malware infections by resetting the system to a trusted state. By enabling its System Protection feature, you can make use of this functionality for rolling back system-level changes in your lab.

Returnil runs on Windows. The company offers a free version for personal use. A commercial license for the product's Premium edition starts at $24.95.

CoreRestore (hardware)

CoreRestore differs from the tools listed above in that it is a hardware component, not a software product. It is a card that you need to install between the system's motherboard and the disk drive IDE controller.

The card redirects system changes to a "temporary working area," allowing the administrator to revert to a pristine state via a reboot. Each card costs $149.97. [Update: Chris Sia pointed out that the drawback of using this tool, in comparison to VMware Workstation snapshots is that CoreRestore only supports a single state to revert to.]

Updates to the original posting

Update 1: Tyler Hudak discussed the effectiveness of "dd" and pointed us to the "ddp" tool (see above).

Update 2: An ISC reader told us about Centurion Technology's CompuGuard products that help lock down a system and offer a mechanism for automatically restoring its state.

Update 3: An ISC reader described an experience with one of the software snapshot products, during which the tool did not restore the master boot record (MBR) of the disk. As a result, Mebroot malware was able to infect the MBR and survived reboots. "When running malware on a physical host, make sure that your software / hardware solution also protects against this kind of threats."

Update 4: Brian Miller suggested using hardware RAID-1 to restore a system. "Before installing the malware, pull one disk of the mirror out.  Once you are ready to revert, insert the good disk, and then rebuild the mirror.  Obviously, you would do this on server class hardware, but if you have spare systems sitting around, why not?"

Update 5: Daniel warned us that "dd" may fail of when an OS "write" call results in a partial buffer write. "In that case, tail bytes of this buffer are lost! This can occur, e. g., when writing on a network filesystem (nfs, sshfs, ...)."

Have you had positive or negative experiences with the products mentioned above? Can you recommend other tools for restoring a system's state during malware analysis? Let us know.

-- Lenny

Lenny Zeltser
Security Consulting - SAVVIS, Inc.
www.zeltser.com