courses

Malware Analysis Lab Setup

At this point, you should have a running REMnux VM — a purpose-built Ubuntu-based Linux distribution for malware analysis, maintained by Lenny Zeltser. It ships with a large collection of analysis tools pre-installed. If you don’t have it yet, see the REMnux configuration page.

Building the environment

The course environment is built by an Ansible playbook in the malware-env repository. It replaces the old setup.sh, which no longer works on current REMnux — two of the package names it installed were removed in Ubuntu 24.04, and because apt fails an entire transaction on one unknown name, the script installed nothing at all.

Run it on the VM itself, against the machine it is running from. First install Ansible there — it is in the Ubuntu archive, so one apt line does it:

$ sudo apt update
$ sudo apt install -y ansible
$ ansible --version
ansible [core 2.16.3]

Then clone the repository and run the playbook against localhost:

$ git clone https://gitlab.cecs.pdx.edu/dmcgrath/malware-env.git
$ cd malware-env
$ ansible-playbook -i localhost, -c local setup.yml -K

Three parts of that command are easy to get wrong:

Flag Why
-i localhost, The trailing comma is not a typo. It is how Ansible takes an inline host list instead of an inventory file; without it, localhost is read as the filename of an inventory.
-c local Run the tasks directly rather than over SSH. Without it Ansible tries to ssh to localhost, which needs a key and a running sshd for no reason.
-K Prompt for the sudo password (Ansible calls it the BECOME password). The playbook installs packages, so it escalates; the remnux account’s sudo wants a password, and without -K the run dies on the very first task with sudo: a password is required.

Run it as your normal account — not as root, and not with sudo ansible-playbook. The playbook configures the account that runs it: login shell, group membership, ~/.zsh, ~/malware. Under sudo all of that would land in /root, so the playbook checks and refuses to start:

fatal: [localhost]: FAILED! => {"changed": false, "msg": "The connecting user's
home resolved to '/root'. This playbook installs a student environment and must
not run as root. Connect as the student account instead."}

You will also be prompted for your git name and email. Both may be left blank, and the run ends by reminding you to set them before your first commit.

It targets Ubuntu 24.04, which is what current REMnux is built on, and detects REMnux so it does not disturb the pinned tool versions the distribution manages itself. To install only parts of it, see the tag table in the repository’s README — --tags shell-deps,shell,tmux,fonts brings up the shell alone.

The playbook is idempotent: re-run it after a git pull to pick up changes.

Note for ARM-based Mac Users (and most other ARM platforms)

There are some significant limitations if you aren’t running an x86-64-based system. Especially when it comes to running 32-bit binaries.

For Linux-native tooling, you have several options. REMnux is x86-64-only and publishes no ARM build, but you can run it on an M-series Mac either by emulating a full x86-64 guest or by translating x86-64 binaries with Rosetta — the latter runs the REMnux container at roughly native speed. See Running x86-64 Linux on Apple Silicon for both routes, measured performance figures, and the limits of each. You can also manually configure most of the tooling as ARM-native builds.

For detonating Windows malware, either emulation or obtaining another machine are your only real options. Contact me if you need access to a Windows environment.

Important: Safe Analysis Environment

Never analyze malware on your host machine or any system you care about. All analysis must be performed inside your isolated REMnux VM with network access disabled or heavily filtered. Treat every sample as hostile.

Before analyzing a sample:

  1. Take a VM snapshot so you can revert to a clean state
  2. Disable or isolate the VM’s network adapter
  3. Work in a dedicated analysis directory (e.g., ~/malware/samples/)

Online Account Registration

Several online services are used throughout the course for sample lookup, submission, and threat intelligence. Register for accounts before the first homework is due.

VirusTotal

VirusTotal scans files and URLs against dozens of antivirus engines and provides community threat intelligence. A free account is required to access full scan reports and to submit files.

  1. Go to the VirusTotal website
  2. Click Join us in the top right
  3. Register using your PDX email address
  4. Verify your email

With a free account you can submit files for scanning, search by hash, and view detailed reports. Keep your API key (found in your profile) — it is used by command-line tools such as vt-cli.

MalwareBazaar

MalwareBazaar is a free malware sample repository run by abuse.ch. It is used for downloading known samples for analysis and for submitting new samples to the community.

  1. Go to the MalwareBazaar website
  2. Click Login and select one of the options to create a profile
  3. Verify your email

Your API key is available on your account page and is used to query and download samples programmatically:

# Query by hash
$ curl -s -X POST https://mb-api.abuse.ch/api/v1/ \
    -d 'query=get_info&hash=<sha256>' \
    -H 'API-KEY: <your_key>'

AFL++ (Fuzzing)

AFL++ is installed by the playbook via apt. Verify it works:

$ afl-fuzz --help
$ afl-cc --version

To fuzz a target, compile with the AFL++ wrapper and run the fuzzer:

$ afl-cc -o target target.c
$ mkdir inputs && echo "test" > inputs/seed
$ afl-fuzz -i inputs -o out ./target

For maximum bug-finding, combine with AddressSanitizer:

$ AFL_USE_ASAN=1 afl-cc -o target_asan target.c

angr (Symbolic Execution)

angr, along with the other analysis Python packages (pefile, capstone, yara-python, pyelftools), is installed into a virtualenv at ~/.venvs/malware. It is not on the system Python: Ubuntu 24.04 marks that interpreter as externally managed, and on REMnux it is what the distribution’s own tooling runs on, so nothing is installed over the top of it.

Activate the virtualenv before using them:

$ source ~/.venvs/malware/bin/activate
$ python3 -c "import angr; print(angr.__version__)"

Or call the interpreter directly without activating:

$ ~/.venvs/malware/bin/python -c "import angr; print(angr.__version__)"

A minimal solve script:

import angr, claripy
proj = angr.Project('./target', auto_load_libs=False)
flag = claripy.BVS('flag', 8 * 20)
state = proj.factory.entry_state(stdin=angr.SimFile(name='stdin', content=flag))
simgr = proj.factory.simulation_manager(state)
simgr.explore(find=0x401234, avoid=0x401250)
if simgr.found:
    print(simgr.found[0].posix.dumps(0))

Ghidra

Ghidra is the NSA’s open-source reverse engineering suite, and the disassembler and decompiler this course uses. There is no registration, no license file, and no per-student key — install it and go.

It requires a 64-bit JDK 21. Check what you have first:

$ java -version

The playbook installs Ghidra and then checks it: every run ends by confirming a launcher is on your PATH, printing the version it found and the path to analyzeHeadless, and failing if Ghidra is not usable. So in most cases the sections below are background rather than something you need to do.

REMnux

Ghidra is pre-installed, and the playbook leaves it alone:

$ ghidra

Arch / CachyOS

$ sudo pacman -S ghidra

This puts ghidra on your PATH and installs the suite under /opt/ghidra.

Anywhere else

Ghidra is not in any Ubuntu release, so on a plain Ubuntu box the playbook fetches it from the GitHub release, checks it against the SHA-256 upstream publishes, and unpacks it to /opt/ghidra with ghidra on your PATH.

To do it by hand, download a release from the Ghidra releases page, extract it, and run the launcher:

$ ./ghidraRun      # Linux / macOS
> ghidraRun.bat    # Windows

First run

Create a project with File → New Project → Non-Shared Project, import a sample with File → Import File, then double-click it and accept the auto-analysis prompt. The decompiler is built in and appears beside the listing — there is no separate pseudocode license to buy.

For batch work, analyzeHeadless lives in the installation’s support/ directory (/opt/ghidra/support/analyzeHeadless on a package install). Static Analysis has a worked example, and the Ghidra cheat sheet has the keymap.

Rizin and radare2

Rizin is a community fork of radare2 with a cleaner API, more consistent command syntax, and active development. Both come from REMnux rather than the playbook — neither is in the Ubuntu archive (rizin is absent entirely, and radare2 only at an old version), so on REMnux you already have them and elsewhere you install them yourself. Their command sets are nearly identical; prefer rizin for new work.

$ rizin -A sample     # open and analyze (rizin)
$ r2 -A sample        # open and analyze (radare2)
[0x00401000]> afl     # list all functions (works in both)
[0x00401000]> pdf @main  # disassemble main

Verify both are installed:

$ rizin -v
$ r2 -v

Cutter

Cutter is the GUI frontend for Rizin, and is part of the REMnux tool set — not something the playbook installs, and not a package in the Ubuntu archive. Launch it with:

$ cutter

Cutter is the primary tool for ELF/Linux analysis. Open a binary via File → Open. The dashboard shows an overview of sections, imports, and strings. The Disassembly, Graph, and Decompiler panes are dockable. The Console pane accepts Rizin commands directly.

Directory Structure

The playbook’s dirs tag creates the following layout for your analysis work:

~/malware/
├── samples/    ← place malware samples here (NEVER execute outside the VM)
└── analysis/   ← analysis notes, output files, screenshots

Your GitLab repo will hold your documented analysis in Markdown. Keep the actual malware samples out of your repo.