courses

Advanced Static Analysis: x86 Assembly, Ghidra, and C Code Constructs

This page summarizes the core concepts from Practical Malware Analysis (Sikorski & Honig), Chapters 4–6, which form the foundation for static disassembly-based malware analysis.


Chapter 4: A Crash Course in x86 Disassembly

Levels of Abstraction

Programs exist at six levels of abstraction: hardware → microcode → machine code → assembly → high-level language → interpreted language. Malware authors write at the high level; analysts work at the assembly level, using a disassembler to convert binary machine code back into human-readable mnemonics.

x86 Architecture and Memory

The x86 processor follows the Von Neumann model: CPU (registers, ALU, control unit), RAM, and I/O interconnected by a bus. The instruction pointer (EIP) fetches the next instruction from RAM.

Each running program sees four logical memory regions:

Region Contents Direction
Data segment Global and static variables, initialized at load time Fixed
Code segment Executable instructions Fixed
Heap Dynamically allocated memory (malloc, new) Grows up
Stack Local variables, function call management Grows down (toward lower addresses)

x86 is little-endian: the least significant byte occupies the lowest memory address. Network data is big-endian, so IP addresses appear byte-reversed when viewed in memory — important when analyzing network-related malware.

Registers

General-purpose registers (32-bit; 16-bit AX/BX/CX/DX and 8-bit AH/AL sub-registers accessible):

Register Primary convention
EAX Accumulator; holds function return values
EBX Base register; general use
ECX Counter for loops and rep instructions
EDX Data; used in multiplication/division overflow
EBP Base pointer — anchors the current stack frame
ESP Stack pointer — always points to the top of the stack
ESI Source index for string/memory operations
EDI Destination index for string/memory operations

EFLAGS (status register) — individual bits set by arithmetic and logic instructions:

Flag Meaning
ZF Zero Flag — set when result is zero
CF Carry Flag — set on unsigned overflow
SF Sign Flag — set when result is negative
TF Trap Flag — enables single-step debugging

Segment registers (CS, SS, DS, ES, FS, GS) — mostly irrelevant to user-mode analysis, except FS, which points to the Thread Information Block (TIB) in Windows.

EIP — the instruction pointer; cannot be directly modified except by control-flow instructions (call, ret, jmp, jcc).

Instruction Set Overview

Intel syntax: mnemonic destination, source. Three operand types:

Key instructions:

Instruction Description
mov dst, src Copy data; does not modify flags
lea dst, [expr] Load the address computed by expression into dst (no dereference); used for pointer arithmetic and fast multiplication
add, sub Addition, subtraction
inc, dec Increment, decrement by 1
mul/imul Unsigned/signed multiply; result in EDX:EAX
div/idiv Unsigned/signed divide; dividend is EDX:EAX; quotient → EAX, remainder → EDX
or, and, xor Bitwise logical operations
shl/shr Logical shift left/right
rol/ror Rotate left/right
nop No operation (opcode 0x90; alias xchg eax, eax); used in NOP sleds

xor eax, eax is the canonical zero-register idiom — smaller encoding than mov eax, 0 and commonly seen in compiler output.

The Stack and Function Calls

The stack is LIFO and grows downward (toward lower addresses). ESP always points to the current top. EBP anchors the frame for the current function, remaining fixed while the function executes.

Function call sequence

  1. Caller pushes arguments onto the stack (right to left in cdecl/stdcall)
  2. call target — pushes the return address (next EIP) and jumps to the function
  3. Prologue: push ebp / mov ebp, esp / sub esp, N — saves caller’s EBP, establishes new frame, allocates space for locals
  4. Function body; local variables at negative EBP offsets [ebp-N], arguments at positive offsets [ebp+N]
  5. Epilogue: mov esp, ebp / pop ebp / ret (equivalently, leave / ret) — restores caller’s frame and jumps to the return address

Return values are placed in EAX before ret.

Stack frame layout (highest address at the top — the stack grows down the page):

[ebp+8]   arg_0 (first argument)
[ebp+4]   return address
[ebp+0]   saved EBP  ← EBP points here
[ebp-4]   var_4 (first local variable)
[ebp-8]   var_8
...
          ← ESP points here (top of stack)

Conditionals and Branching

test op1, op2 — computes op1 AND op2, sets flags, discards result. test eax, eax is the standard NULL/zero check (sets ZF if EAX = 0).

cmp op1, op2 — computes op1 − op2, sets flags, discards result.

jmp — unconditional jump. Over 30 conditional jump variants examine EFLAGS after cmp/test:

Instruction Condition
jz / je ZF=1 (zero / equal)
jnz / jne ZF=0 (not zero / not equal)
jg / jge Greater / greater-or-equal (signed)
ja / jae Above / above-or-equal (unsigned)
jl / jle Less / less-or-equal (signed)
jb / jbe Below / below-or-equal (unsigned)
js SF=1 (sign flag set; result negative)
jecxz ECX = 0

Rep Instructions

The rep prefix repeats the following string instruction, decrementing ECX each iteration until ECX = 0 (or a flag condition halts it). ESI = source, EDI = destination.

Instruction C equivalent Description
rep movsb memcpy Copy ECX bytes from ESI to EDI
rep stosb memset Fill ECX bytes at EDI with value in AL
repe cmpsb memcmp Compare bytes at ESI and EDI until mismatch or ECX=0
repne scasb strchr/search Scan EDI buffer for byte in AL until found or ECX=0

Variant prefixes: repe/repz also stop when ZF=0; repne/repnz also stop when ZF=1.

Direction flag (DF): when DF=0, ESI/EDI increment after each step; when DF=1, they decrement (used for reverse-direction copies in shellcode).

C main and Pointer Offsets

int main(int argc, char **argv) — argc is the argument count (including program name); argv is an array of string pointers. On 32-bit systems each pointer is 4 bytes, so argv[1] is at [argv + 4], argv[2] at [argv + 8]. In assembly:

mov eax, [ebp+argv]   ; load argv base pointer
mov ecx, [eax+4]      ; load argv[1] (offset 4 = second pointer)
mov ecx, [eax+8]      ; load argv[2] (offset 8 = third pointer)

Chapter 5: Interactive Disassembly (Ghidra)

Note on the textbook. Practical Malware Analysis chapter 5 is written around IDA Pro, which was the only realistic option when the book was published. This course uses Ghidra, so the material below covers the same ground with Ghidra’s terminology and keys. The concepts — xrefs, stack frame recovery, renaming, redefining code and data — transfer directly. The Ghidra cheat sheet has an IDA → Ghidra Rosetta table if you are reading the book or an IDA-based write-up alongside.

Ghidra is an interactive disassembler: it performs automatic function discovery, stack frame analysis, local variable identification, and cross-referencing, then lets you annotate, rename, and extend the disassembly. Work is saved in a project, not in the binary, and resumes across sessions. Unlike a one-file-per-database model, one Ghidra project can hold a whole malware family and cross-reference between its components.

Ghidra’s equivalent of IDA’s FLIRT signatures is FunctionID (.fidb databases), which labels statically-linked library functions so you can skip them.

Loading an Executable

At import time Ghidra detects the file format (PE, ELF, Mach-O, COFF) and processor. Options worth knowing:

After import, double-click the file and accept the auto-analysis prompt. Let it finish before working.

The Interface

The CodeBrowser shows the Listing (linear disassembly) and the Decompiler side by side, linked bidirectionally. There is no separate “graph mode” toggle; open Window → Function Graph for the flowchart view, where edge colors carry branch semantics:

Listing fields (address, bytes, mnemonic, comments) are configurable via the Browser Field Formatter (the arrow icon at the top right of the listing).

Key Windows

Window Purpose
Symbol Tree → Functions All functions; FunctionID-matched library code is labeled
Symbol Tree → Imports Imported APIs; expand and use xrefs to reach call sites
Symbol Tree → Exports Exported functions; critical for DLL analysis
Window → Defined Strings All recovered strings; a primary analysis starting point
Window → Data Type Manager Type and struct layouts, including Windows API archives
Window → Memory Map Section/segment layout and permissions
Window → Symbol Table Every named address
Action Method
Jump to any address or symbol G
Follow a link Double-click any FUN_, LAB_, DAT_, or string reference
Navigate history Alt+Left / Alt+Right
Jump between listing and decompiler Click — the two panes stay in sync

Searching

Option Use
Search → Program Text Text search across the disassembly and/or decompiler
Search → Memory (S) Hex, string, or regex byte-pattern search — useful for encryption constants and signatures
Search → For Instruction Patterns Find instruction sequences rather than literal bytes
Search → For Scalars Locate numeric constants

Cross-References (Xrefs)

An xref records every location where a function is called or a data item is accessed. Right-click a name → References → Show References To.

Window → Call Trees shows incoming and outgoing calls as an expandable tree, which is the closest equivalent to IDA’s xref-graph buttons.

Analyzing Functions

Ghidra recovers stack frames and names variables automatically:

These dummy names should be renamed as understanding develops (press L). Ghidra propagates the new name to every reference.

If Ghidra missed a function boundary, press F on the first instruction to create a function at the cursor. If the recovered signature is wrong, right-click the function name → Edit Function Signature — correcting parameter counts and types there propagates to every caller and usually fixes more decompiler noise than retyping individual variables.

Enhancing the Disassembly

Renaming: replace FUN_00401200 with DNSrequest (L); the name updates everywhere.

Comments (all via the ; dialog):

Type Appears
EOL End of the instruction line
Pre / Post Above / below the instruction
Plate Banner above a function — good for section headers
Repeatable At the instruction and at every xref to it

Formatting operands: right-click an immediate value to reformat as decimal, hex, octal, binary, or character.

Named constants: right-click an operand → Set Equate to replace a raw integer with a symbolic name (e.g. 80000000h → GENERIC_READ). Windows API types and constants come from the data type archives in the Data Type Manager.

Redefining code and data

This matters when malware appends shellcode after the PE structure: it arrives as undefined bytes until you disassemble it manually.

Extending Ghidra with Scripts and Plug-ins

Script Manager (Window → Script Manager) ships hundreds of example scripts and runs your own in Java or Python.

PyGhidra — CPython 3 integration exposing Ghidra’s API. As of Ghidra 12, Python scripts default to PyGhidra; Jython is an optional extension requiring a # @runtime Jython header.

Headless — analyzeHeadless in the install’s support/ directory runs analysis and scripts with no GUI and no license, which is what makes batch triage practical. See Static Analysis for a worked example.

Comparable tooling


Chapter 6: Recognizing C Code Constructs in Assembly

Effective reverse engineering requires recognizing groups of instructions as high-level constructs — loops, conditionals, function calls — rather than reading every instruction in isolation. Since most malware is written in C, the following patterns appear constantly.

Global vs. Local Variables

Variable type Assembly signature Scope
Global Fixed memory address (e.g., dword_40CF60) Accessible by all functions; writes persist
Local Stack offset relative to EBP (e.g., [ebp-4], labeled local_4 by Ghidra) Exists only within its stack frame

The distinction matters because a write to a global modifies persistent state, while a local disappears when the function returns.

Arithmetic Operations

Most arithmetic maps directly. Notable cases:

Recognizing if Statements

Simple if-else assembly pattern:

cmp   eax, [ebp+var_4]     ; compare
jnz   short loc_40102B     ; if NOT equal, jump to else-body
push  offset aXEqualsY     ; true-body
call  printf
jmp   short loc_401038     ; skip else-body
loc_40102B:
push  offset aXIsNotEqual  ; else-body
call  printf
loc_401038:
...                        ; code after the if/else

Key: a conditional jump skips the true body, followed by an unconditional jump skipping the else body. In Ghidra’s Function Graph, the branch node has false (red) and true (green) edges.

Nested if statements add additional cmp+jcc pairs within a branch. Three nested levels produce three conditional jumps in sequence.

Recognizing Loops

for loop — four identifiable components:

mov   [ebp+var_4], 0    ; (1) initialization
jmp   short loc_401016  ; (2) jump to comparison
loc_40100D:
mov   eax, [ebp+var_4]  ; (4) body
add   eax, 1            ;     increment
mov   [ebp+var_4], eax
loc_401016:
cmp   [ebp+var_4], 64h  ; (3) comparison
jge   short loc_40102F  ;     exit if done
... body ...
jmp   short loc_40100D  ; back to increment

In Ghidra’s Function Graph, the upward-pointing edge after the increment block is the definitive loop indicator.

while loop — similar structure but without a separate initialization or explicit increment section. A conditional jump at the top exits when the condition fails; an unconditional jmp at the bottom returns to the top. Frequently seen as an infinite receive-and-process loop:

while(status == 0) {
    result = performAction();
    status = checkResult(result);
}

Function Call Conventions

Calling conventions govern argument order, stack cleanup responsibility, and register use:

Convention Arg order Stack cleanup Notes
cdecl Right to left Caller (add esp, N after call) Most common for C programs
stdcall Right to left Callee (retn N inside function) The Win32 API convention (WINAPI). The exception is variadic functions such as wsprintf, which must be cdecl — a callee cannot clean up a stack whose size it does not know
fastcall First 2 args in ECX, EDX; rest on stack Callee (usually) Most efficient; avoids stack push for first two args

Push vs. Move: Visual Studio typically uses push to pass arguments. GCC may instead use sub esp, N followed by mov [esp+offset], value. The Visual Studio version contains a stack-pointer restore instruction that has no GCC equivalent. An analyst must recognize both patterns.

Analyzing switch Statements

If-style (few or non-contiguous cases): a chain of cmp+jz pairs, each case body ending with an unconditional jmp to the function end. Visually identical to nested if statements.

Jump table (many contiguous cases): the compiler generates a single indirect jump through a lookup table:

sub   ecx, 1                    ; adjust to 0-based index
mov   [ebp+var_8], ecx
cmp   [ebp+var_8], 3            ; bounds check
ja    short loc_401082          ; default case if out of range
mov   edx, [ebp+var_8]
jmp   ds:off_401088[edx*4]      ; indirect jump through table

off_401088  dd offset loc_40102C  ; case 1
            dd offset loc_401042  ; case 2
            dd offset loc_401058  ; case 3
            dd offset loc_40106E  ; case 4

Each entry in the table is a 4-byte address, so the index is multiplied by 4. This is significantly more efficient than chained comparisons and is a distinctive pattern in disassembly. The table itself appears as dd offset entries in the data section immediately after the function.

Adding even one case to a dense switch can trigger the compiler to switch from if-style to jump table — the two look entirely different in assembly.

Disassembling Arrays

Arrays are accessed as base address + (index × element size):

; Local array a[i] where elements are 4-byte ints
mov   ecx, [ebp+var_18]          ; load index i
mov   [ebp+ecx*4+var_14], edx    ; a[i] = edx

; Global array b[i]
mov   ecx, [ebp+var_18]          ; load index i
mov   dword_40A000[ecx*4], eax   ; b[i] = eax

The multiplier on the index register reveals the element size: ×1 = byte, ×2 = short, ×4 = int/pointer, ×8 = double. When no multiplier is present, the element is 1 byte or the index has been prescaled.

Identifying Structs

Structures are accessed via a base pointer + field offset:

mov   eax, [ebp+arg_0]       ; load struct pointer (q)
mov   byte ptr [eax+14h], 61h; q->y = 'a' (field at offset 0x14)
fld   ds:dbl_40B120          ; load double constant
fstp  qword ptr [ecx+18h]    ; q->z = 15.6 (field at offset 0x18)

Floating-point fields are identifiable by fld/fstp instructions at a particular offset. The sequence of offsets accessed reveals the field layout. In Ghidra, press T on a memory reference to apply a structure definition, transforming [eax+14h] into a named field access. Define the structure first in the Data Type Manager (right-click an archive → New → Structure).

Windows API functions frequently require passing populated structs; identifying the base pointer and field offsets is essential for understanding API call parameters.

Analyzing Linked List Traversal

A linked list consists of structs where each struct contains a pointer to the next struct of the same type (self-referential). The key assembly signature:

mov   eax, [ebp+var_4]    ; load current node pointer (curr)
mov   eax, [eax+4]        ; load next field (curr->next, 4 bytes in)
mov   [ebp+var_4], eax    ; curr = curr->next

The self-referential assignment — var_4 is assigned from [eax+4] where EAX previously held var_4 — is the definitive indicator. Whatever struct var_4 points to must contain a pointer 4 bytes into it, which points to another struct of the same type. Malware uses linked lists for managing active connections, task queues, and plugin registrations.


Summary: Construct Recognition Quick Reference

Construct Key assembly indicators
if/else cmp/test → conditional jump → body → unconditional jump → else body
Nested if Multiple sequential cmp+jcc pairs
for loop init → jmp to comparison → body → increment → jmp back (upward edge in the Function Graph)
while loop Conditional jump at top → body → unconditional jmp back
cdecl call push args right-to-left → call → add esp, N (caller cleans)
stdcall call push args right-to-left → call → no caller cleanup (retn N inside callee)
fastcall First args in ECX, EDX → call → callee cleans
switch (if-style) Chain of cmp+jz → independent bodies each ending with jmp to end
switch (jump table) sub index → bounds ja → jmp ds:table[reg*4] → dd offset entries
array access base + index*size (multiplier reveals element size)
struct access Fixed pointer + field offsets; FP fields via fld/fstp
linked list Self-referential pointer: var = [var + offset] in a loop
global variable Fixed memory address (dword_40CF60)
local variable Negative EBP offset ([ebp-4], labeled local_4 by Ghidra)