An Overview of Malware Self-Defense and Protection

Many malware authors spend a great deal of time and effort to develop complex code. Their success depends on a threat’s remaining undetected and avoiding sandbox analysis, antivirus efforts, or malware analysts. This post offers an overview of the mechanisms used by malware to evade detection.

If malware is detected quickly, it has little time to steal data or to maximize its impact. The IT security market has matured and security tools and applications are today more efficient. However, attackers understand and monitor the operations of security tools. In addition, organizations do not always follow best practices. Antimalware tools are sometimes outdated, and sandboxes can easily be detected due to misconfiguration.

Malware self-defense

Malware can use several mechanisms to avoid detection and analysis. We can classify these techniques into three categories:

• Anti-security tools: Used to avoid detection by antivirus, firewall, and other tools that protect the environment.
• Anti-sandbox: Used to detect automatic analysis and avoid engines that report on the behavior of malware.
• Anti-analyst: Used to detect and fool malware analysts. For example, spotting monitoring tools such as Process Explorer or Wireshark, as well as some process-monitoring tricks or packers, to avoid reverse engineering.

Some malware techniques are common to these three categories. Malware can use a technique like RunPE (which runs another process of itself in memory), to evade antivirus software, a sandbox or an analyst.

Sandbox evasion

Sandboxes are an effective tool to quickly detect and understand malware; however, it is relatively trivial for malware to detect a sandbox if it is not hardened. Malware can perform several basic checks:

• MAC address detection: Virtual environments such as VMware or VirtualBox use known MAC addresses. This address is often stored in the registry at (HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Class\{4D36E972-E325-11CE-BFC1-08002BE10318}\0000\NetworkAddress). The malware can detect this two ways, either by requesting the Hive key or using the GetAdapterInfo API.

  DWORD GetAdaptersInfo(
            _Out_   PIP_ADAPTER_INFO pAdapterInfo,
            _Inout_ PULONG                   pOutBufLen
  );

• Process discovery: Malware may be able to detect whether there are any running processes related to a sandbox. For example, processes such as VmwareService.exe can be easily detected with the CreateToolHelp32Snapshot API, to get a snapshot of the current running processes, and then list each process of the snapshot with the APIs Process32First and Process32Next.

  HANDLE WINAPI CreateToolhelp32Snapshot(  
            _In_ DWORD dwFlags,  
            _In_ DWORD th32ProcessID
  );
  
  BOOL WINAPI Process32First(  
            _In_    HANDLE           hSnapshot,  
            _Inout_ LPPROCESSENTRY32 lppe
  );
  
  BOOL WINAPI Process32Next(  
            _In_  HANDLE           hSnapshot,  
            _Out_ LPPROCESSENTRY32 lppe
  );

• Registry detection: Virtual environments create registry keys on the system that can be detected by malware. Following we see an incomplete list of the registry keys that malware can check:
  “HARDWARE\\DEVICEMAP\\Scsi\\Scsi Port 0\\Scsi Bus 0\\Target Id 0\\Logical Unit Id 0”
  “SOFTWARE\\VMware, Inc.\\VMware Tools”
  “HARDWARE\\Description\\System”
  “SOFTWARE\\Oracle\\VirtualBox Guest Additions”
  “SYSTEM\\ControlSet001\\Services\\Disk\\Enum”
  “HARDWARE\\ACPI\\DSDT\\VBOX__”
  “HARDWARE\\ACPI\\FADT\\VBOX__”
  “HARDWARE\\ACPI\\RSDT\\VBOX__”
  “SYSTEM\\ControlSet001\\Services\\VBoxGuest”
  “SYSTEM\\ControlSet001\\Services\\VBoxMouse”
  “SYSTEM\\ControlSet001\\Services\\VBoxService”
  “SYSTEM\\ControlSet001\\Services\\VBoxSF”
  “SYSTEM\\ControlSet001\\Services\\VBoxVideo”

Malware can also perform some advanced techniques to detect a sandbox:

• Checking hook function: A hook is basically a technique to alter the behavior of an internal function of an operating system or an application. Sandboxes use hook techniques to alter the behavior of a sample; for example, by hooking the DeleteFile function, malware will try to delete a file that will be intercepted by the sandbox. These kinds of function are located in a specific place in memory (kernel land).

Malware can detect a hook by checking the address of the calling function. For example, if the returned address is not located in the kernel, the function is currently hooked.

Malware can use other techniques such as checking the size of the hard drive or using special instructions to detect specific registers (“Red Pill” or “No Pill” techniques). These techniques are based on the fact that registers are unique in a machine and have to be relocated on a virtual environment.

Antivirus evasion

Antivirus tools use three basic functions: signatures, scanner, heuristics.

• Evading signatures can be performed by changing the hash of the sample, which is as easy as changing only one byte in the executable.
• Evading a scanner can be performed by creating a big file to confuse the emulator.
• Evading heuristic analysis is more complex, but can be performed by hooking back functions.

Another way to evade antivirus tools is for the malware to disable the tool or add an exception. Polymorphic codes are particularly difficult to detect.

Antidebugging

Malware analysts often have to dig deep during code analysis. Antidebugging is another malware technique for avoiding reverse engineering by a debugger. It is relatively trivial to detect the presence of a debugger using the Windows API.

• IsDebuggerPresent: This function checks a specific flag in the process environment block for the field IsDebugged, which will return zero if the process is not running in a debugger or a nonzero if a debugger is attached.

  BOOL WINAPI IsDebuggerPresent(void);

• FindWindow: This function can search for windows by name or class (for example, OllyDbg). This function can also detect tools such as Wireshark or Process Explorer.

  HWND WINAPI FindWindow(  
            _In_opt_ LPCTSTR lpClassName,  
            _In_opt_ LPCTSTR lpWindowName
  );

• CsrGetProcessId: This function can find the process ID of csrss.exe, a system process. By default, a process has the SeDebugPrivilege privilege in the access token disabled. However, when the process is loaded by a debugger such as OllyDbg or WinDbg, the SeDebugPrivilege privilege is enabled. If a process can open csrss.exe, it means that the process has the privilege SeDebugPrivilege enabled in the access token, thus suggesting that the process is being debugged.

Anti-Disassembly

Anti-disassembly is another technique to avoid analysis through reverse engineering. There are many ways to hinder a disassembler:

• API obfuscation can hide a call to a special function. The result could be a call without the name of the API function, for example. The analyst has to reverse it to understand which function was used. This takes time.
• Inserting junk code: Junk code can be inserted into the malware to fool analysts into wasting time trying to reverse unusable code. The junk code does not change the behavior of the sample because this code is never executed.

The “Unprotect Project”

There are many ways to avoid malware analysis. Some open projects list these techniques. The Unprotect Project is an open wiki that collects and lists malware protection and self-defense techniques. The project includes a mind map that lists techniques for a better understanding of malware protection capabilities.

The goal of this project is to help the community better understand techniques used by malware to stay undetected, bypass security protection, and avoid analysis. The following categories appear on the website:

• Sandbox evasion techniques: To evade sandboxes analysis.
• Antivirus evasion techniques: To evade detection by antivirus.
• Anti-debugging techniques: To fool debuggers and avoid analysis.
• Anti-disassembly: To avoid reverse engineering and understand the behavior of malware with a disassembling tool.
• Process tricks: To hide the malware processes on the system and stay undetected.
• Obfuscation and data encoding: To hide data or part of code in the malware.
• Packers: To protect malware code and add other evasion capabilities.

The wiki is updated continuously.

Conclusion

Malware is constantly growing smarter and evolving techniques to stay undetected. Understanding these techniques and sharing the experiences of the information security community are effective ways to fight malware.

Introducing McAfee+

Identity theft protection and privacy for your digital life

Download McAfee+ Now