Malware Reverse Engineering

Instructor: Wu-chang Feng
Class: (Zoom link)
Contact and discussion:
  • DM @wuchang pdx-cs.slack.com
  • Class channel #cs492_592_malware on pdx-cs.slack.com
  • In-class questions/feedback anonymously sayat.me
Office hours: here (10am on Mondays) on the course Zoom link.
TA: Allison Naaktgeboren (naak at pdx)
TA Office hours: Mondays 12:30pm-2pm on Zoom | In-person by arrangement
Required textbook
Resources

Schedule

Week Topic Slides Labs Homework
1 Introduction
Motivation
Basic Analysis
Chapter 1: Basic Static Techniques
Chapter 2: Malware Analysis in VMs
Chapter 3: Basic Dynamic Analysis
00_Intro

01_BasicTechniquesTools

Install VM (video)
1-1, 1-2
3-2, 3-4
Cyberwar talk (edited)
Ch01, Ch03
2
Chapter 3: Basic Dynamic Analysis
Advanced Static Analysis
Chapter 4: x86 Assembly
Chapter 5: IDA Pro
Chapter 6: C code in Assembly
02_C_x86_Windows
5-1, 6-1, 6-2 Ch04, Ch06
3
Chapter 7: Malicious Windows Programs
Advanced Dynamic Analysis
Chapter 8: Debugging
Chapter 9: OllyDbg
03_Debugging
7-2
9-2
Ch08
4 Malware Functionality
Chapter 11: Malware Behavior
Chapter 12: Covert Launching
04_Functionality
11-1
12-1, 12-3
Ch11, Ch12
5
Chapter 13: Data Encoding
Chapter 14: Network Signatures
Anti-Reverse-Engineering
Chapter 15: Anti-Disassembly
  Software Armoring


05_AntiReverse
talk
13-1
14-1
15-1, 15-2
Ch13
Ch15
6
Chapter 16: Anti-Debugging
Chapter 17: Anti-VM Techniques
  BluePill
Chapter 18: Packers and Unpacking
Chapter 21: 64-bit Malware
Talks #1,#2 | Slides #1,#2
16-1
17-1
18-1
Ch16
Ch18
Ch21
7
Symbolic execution/analysis
07_Fuzzing_SymbolicExecution
angr setup
angr CTF 00-12
8
Fuzzing
Symbolic execution (cont.)
AFL labs
angr CTF 13-17
Mon. 8/15
 (11:59pm)
Deadline to submit all coursework      

Assignments

Labs and notebook
Lab assignments will be given each class covering the course material. You will perform each one, while maintaining a lab notebook in a Google Doc that documents your progress via screenshots with your OdinID in them. The notebook should also include answers to any questions in the labs.  Notebooks should be exported as a PDF file and include a table of contents generated by Google Docs. Submission will be done via adding, committing and pushing the file to your private git repository. Use the following naming convention to submit your notebooks.
  • notebooks/<week_number>.pdf e.g. notebooks/Week1.pdf
The notebook will be graded based upon the following rubric:
  • Neatness and organization
  • Completeness
  • Inclusion of OdinID or project identifier in screenshots
Homework (MetaCTF)
For homework, we will be applying the concepts learned in the labs to Linux binaries. Assignments are to be done individually at the following sites: cs492.oregonctf.org and angr.oregonctf.org. Binaries are unique to each student and no collaboration is allowed. The binaries implement a set of capture-the-flag challenges that require you to reverse engineer a set of binary executables. Each binary asks for a password that will unlock it and print "Good Job". While you will be running binaries on your own machine, answers to each should be submitted at the above site.

To download binaries on a linuxlab machine, do the following:
mkdir metactf; cd metactf; virtualenv -p python3 env
source env/bin/activate
pip install requests bs4
wget http://thefengs.com/wuchang/courses/cs492/meta_dl.py
python meta_dl.py cs492.oregonctf.org username password
If you wish, instead, to run the binaries on a Ubuntu VM of your own, directions for doing so are here. Note that you must ensure that radare2 and the 32-bit libraries are installed on the VM by performing
sudo apt-get install gcc-multilib radare2

Course objectives

  • Understand the underlying mechanisms used by malware on compromised systems.
  • Understand counter-measures that detect malware.
  • Understand techniques malware uses to circumvent and evade detection and analysis
  • Develop skills to monitor the behavior of malware safely.
  • Develop skills to reverse-engineer malware

Policies

Grading
Attendance 5%
Homeworks 50%
Lab notebook 45%
Academic misconduct
  • Includes allowing another student to copy your work unless specifically allowed by the instructor.
  • Results in a grade of 0 for the assignment or exam.
  • Results in the initiation of disciplinary action at the university level.