Detection Example: DSE vs Code Modifications & Inline Hooks
Description: Code modifications (including inline hooks), also known as patches, are a very powerful way to directly modify the behaviour of a program (or indeed the operating system itself). Software engineers use code modifications to fix bugs, crackers use code modifications to defeat software protection schemes, and malware such as rootkits use code modifications to attack other software as well as protect themselves, for example by hiding themselves.
Inline hooks are a common form of code modification. They are generally used by security software to intercept and prevent malicious calls to important security-related system functions. However, inline hooks are also used by malware such as rootkits to also tap into important security-related system functions, but for malicious reasons. An inline hook essentially re-routes the flow of code from the hooked function to the address of the malware's choosing (usually to itself).
However, inline hooks aren't the only type of code modification that can cause security issues. In the demonstration below we will show you how a simple 1 byte patch can defeat a system function.
Detection: For this example we will use a debugger to apply the code modifications as this makes it easiest to see the modifications themselves, although for all intents and purposes we could have used a live rootkit (many of the rootkit detection examples reveal code modifications).
Here we can see the original unaltered code for the TerminateProcess function in kernel32.dll:
Now lets patch some code! The program that is running in the debugger has a DLL that has been injected into the process called C:\MYHOOK.DLL, and it has an exported function called MyHookFunction. So we will patch the TerminateProcess function so that the very first thing it does is jump to my hook function.
Here is the applied patch:
As you can see by comparing the two images we have modified 5 bytes, overwriting 3 instructions with 1 ...
Before
8BFF mov edi, edi 55 push ebp 88EC mov ebp, espAfter
E9 BDF30484 jmp myhook.MyHookFunctionNow whenever the TerminateProcess function is called the MyHookFunction will be immediately activated. The function can then analyse the parameters sent to the function, and either disregard the function call, or jump back to the rest of the TerminateProcess function to allow it to continue.
As you can see you can do a lot of damage with just a 5 byte patch, but sometimes all that's needed is a 1 byte patch (and sometimes it may even simply be 1 bit in the byte that gets patched).
Here we've gone to the entrypoint of MyHookFunction in c:\myhook.dll, and patched its first byte, changing it from 55 (push ebp) to C3 (retn):
The retn instruction returns to the code that called it. Normally the retn instruction can be found at the very end of functions (as you can see in the above function), so this patch effectively renders the MyHookFunction null & void, even though all we've done is patch 1 byte. (Actually, we've only had to change 4 of the 8 bits in that byte, changing 01010101 to 11000011).
Detecting with DSE ...
The Code Modifications feature of DSE flies through all processes and their modules (DLLs etc), as well as all kernel modules (drivers), hunting for modified code. For each file it encounters DSE builds an image of how the code sections in the file should look, then compares that image to the live image in memory. Some programs that detect function hooks simply look at the start of exported functions due to the difficulty in determining which parts of a file are code as opposed to data, but DSE attempts to intelligently determine which areas of a program contain code, and then examines ALL of those code regions. In other words, an inline hook could be patched into the very middle or even the end of a function, or even in a so-called "code cave", and DSE will still detect the code modification.
Here we can see the modifications we made:
We can see the process involved (c:\codetest.exe), along with its PEB and KPEB addresses for analysts. Below that we see that there are modifications in two modules within that process - c:\myhook.dll and kernel32.dll (as well as their base addresses), and below each of those we see the patches that have been made.
For each patch displayed we can see the exact address of the start of the patch, the size of the patch (1 and 5 in this case), and then the before & after code bytes themselves.
DSE also does an analysis of the patch in order to determine what it might be doing, and in the case of calls/jumps it will try to determine what it is trying to jump to, and where. For example here we can see the TerminateProcess has been modified with code that jumps to MyHookFunction in c:\myhook.dll.
Copyright © 2008, Diamond Computer Systems Pty. Ltd. All rights reserved.