Malware Analysis Lab Setup

• Malware Analysis Lab Setup
  • Note for ARM-based Mac Users
  • Important: Safe Analysis Environment
  • System Configuration
  • Install Tools
  • AFL++ (Fuzzing)
  • angr (Symbolic Execution)
  • IDA Classroom
  • Rizin and radare2
  • Cutter
  • Directory Structure

At this point, you should have a running REMnux VM — a purpose-built Ubuntu-based Linux distribution for malware analysis, maintained by Lenny Zeltser. It ships with a large collection of analysis tools pre-installed. If you don’t have it yet, see the REMnux configuration page.

Note for ARM-based Mac Users

This course requires Windows at certain points (for running Windows malware samples and tools such as IDA). ARM-based Macs (Apple Silicon) cannot run x86 Windows VMs natively. To accommodate this, you can use the CAT RemoteLab machines, which provide remote access to x86 Windows systems:

• CAT RemoteLab

Log in with your MCECS credentials. RemoteLab machines have the necessary software available or can be configured as needed. Contact the CAT if you have trouble accessing the service.

Important: Safe Analysis Environment

Never analyze malware on your host machine or any system you care about. All analysis must be performed inside your isolated REMnux VM with network access disabled or heavily filtered. Treat every sample as hostile.

Before analyzing a sample:

1. Take a VM snapshot so you can revert to a clean state
2. Disable or isolate the VM’s network adapter
3. Work in a dedicated analysis directory (e.g., ~/malware/samples/)

System Configuration

First, make sure REMnux is up to date:

$ sudo apt update
$ sudo apt upgrade -y

Install Tools

REMnux already includes many malware analysis tools. The script installs supplemental tools needed for this course that are not in the base image. Run it with:

$ curl -LO https://web.cecs.pdx.edu/~dmcgrath/courses/malware/setup.sh
$ chmod +x setup.sh
$ # edit the script to fill in your git name and email near the bottom
$ ./setup.sh

This installs:

• Triage tools: file, strings, binwalk, exiftool, ssdeep, yara, FLOSS
• Static analysis: radare2, rizin, Cutter, objdump, readelf, pev, oletools
• Dynamic analysis: strace, ltrace, Wireshark
• Debuggers: gdb, gdb-multiarch, lldb, gdbserver, pwndbg
• Fuzzing: AFL++, clang/LLVM (for sanitizer builds), Z3 (SMT solver)
• Python libraries: pefile, capstone, yara-python, pyelftools, angr

IDA Classroom must be installed separately — see IDA Classroom below.

Reboot after the script completes, then log back in. Launch a terminal to verify everything looks correct.

AFL++ (Fuzzing)

AFL++ is installed by the setup script via apt. Verify it works:

$ afl-fuzz --help
$ afl-cc --version

To fuzz a target, compile with the AFL++ wrapper and run the fuzzer:

$ afl-cc -o target target.c
$ mkdir inputs && echo "test" > inputs/seed
$ afl-fuzz -i inputs -o out ./target

For maximum bug-finding, combine with AddressSanitizer:

$ AFL_USE_ASAN=1 afl-cc -o target_asan target.c

angr (Symbolic Execution)

angr is installed via pip as part of setup. Verify:

$ python3 -c "import angr; print(angr.__version__)"

A minimal solve script:

import angr, claripy
proj = angr.Project('./target', auto_load_libs=False)
flag = claripy.BVS('flag', 8 * 20)
state = proj.factory.entry_state(stdin=angr.SimFile(name='stdin', content=flag))
simgr = proj.factory.simulation_manager(state)
simgr.explore(find=0x401234, avoid=0x401250)
if simgr.found:
    print(simgr.found[0].posix.dumps(0))

IDA Classroom

IDA Classroom is the free educational edition of IDA Pro — the industry-standard disassembler and decompiler. It requires a free account at Hex-Rays.

1. Register at https://hex-rays.com/ida-free/ using your PDX email
2. Download the Linux installer (.run file)
3. Install it:

$ chmod +x idafree_linux.run
$ ./idafree_linux.run

The installer places IDA in ~/idafree-9.x/ by default. Launch with:

$ ~/idafree-9.x/ida64     # for 64-bit binaries
$ ~/idafree-9.x/ida       # for 32-bit binaries

Add to your PATH for convenience:

$ echo 'export PATH="$HOME/idafree-9.x:$PATH"' >> ~/.zshrc.local

On first open of a binary, IDA will run auto-analysis. Accept the defaults. The Hex-Rays decompiler (pseudocode view, F5) is included in the Classroom edition.

Rizin and radare2

Rizin is a community fork of radare2 with a cleaner API, more consistent command syntax, and active development. Both are installed by the setup script. Their command sets are nearly identical; prefer rizin for new work.

$ rizin -A sample     # open and analyze (rizin)
$ r2 -A sample        # open and analyze (radare2)
[0x00401000]> afl     # list all functions (works in both)
[0x00401000]> pdf @main  # disassemble main

Verify both are installed:

$ rizin -v
$ r2 -v

Cutter

Cutter is the GUI frontend for Rizin. It is installed by the setup script via apt. Launch it with:

$ cutter

Cutter is the primary tool for ELF/Linux analysis. Open a binary via File → Open. The dashboard shows an overview of sections, imports, and strings. The Disassembly, Graph, and Decompiler panes are dockable. The Console pane accepts Rizin commands directly.

Directory Structure

The setup script creates the following layout for your analysis work:

~/malware/
├── samples/    ← place malware samples here (NEVER execute outside the VM)
└── analysis/   ← analysis notes, output files, screenshots

Your GitLab repo will hold your documented analysis in Markdown. Keep the actual malware samples out of your repo.