courses

Homework 4

Due Date: 2026-11-20 23:59:59 (Friday of week 8)

Introduction

This homework covers the general topic of network security. You will be looking at packet capture filters, packet traces in wireshark, and some basic network identification tasks.

What you must do

  1. Complete the following module on HackTheBox: Introduction to Networking
  2. Complete the following module on HackTheBox: Threat Hunting with IPS/IDS
  3. Complete the following tcpdump/BPF practice exercises.
    1. Perform a tcpdump capture where you only capture DNS packets.
    2. Perform a tcpdump capture where you capture TCP packets that are destined for either port 443 or 8080, and originate from your computer.
    3. Perform a tcpdump capture where traffic is either UDP or TCP, is inbound to your computer, and destined for a port between 20000 and 35000.

    All dumps should have full data dumps without link layer headers, in hex and ascii, with Unix epoch style timestamps. Something like this:

    1585339575.038549 IP (tos 0x0, ttl 64, id 37629, offset 0, flags [DF], proto UDP (17), length 60)
         kali478-0.38378 > _gateway.domain: [bad udp cksum 0x6ede -> 0x96ae!] 54659+ A? www.google.com. (32)
       	0x0000:  4500 003c 92fd 4000 4011 390f ac10 8b81  E..<..@.@.9.....
       	0x0010:  ac10 8b02 95ea 0035 0028 6ede d583 0100  .......5.(n.....
       	0x0020:  0001 0000 0000 0000 0377 7777 0667 6f6f  .........www.goo
       	0x0030:  676c 6503 636f 6d00 0001 0001            gle.com.....
    

    Captures can be included in your markdown file by enclosing them in triple backticks, like so:

    ```
    1585339575.038549 IP (tos 0x0, ttl 64, id 37629, offset 0, flags [DF], proto UDP (17), length 60)
         kali478-0.38378 > _gateway.domain: [bad udp cksum 0x6ede -> 0x96ae!] 54659+ A? www.google.com. (32)
       	0x0000:  4500 003c 92fd 4000 4011 390f ac10 8b81  E..<..@.@.9.....
       	0x0010:  ac10 8b02 95ea 0035 0028 6ede d583 0100  .......5.(n.....
       	0x0020:  0001 0000 0000 0000 0377 7777 0667 6f6f  .........www.goo
       	0x0030:  676c 6503 636f 6d00 0001 0001            gle.com.....
    ```
    

    A minimum of 10 packets, as requested, should be included for each capture. You can use the -c flag to limit the number of packets captured. You will likely need to trigger traffic for some of these captures. That is part of the exercise.

  4. We often wish to examine network traffic to analyze whether a given network is running correctly, is under attack, has secrets in it we would like to reveal or protect, and so on. There is an engineering trade-off here. We can’t store and analyze all the data that flies by in the network. So we would like to use a portion of the total data to decide where to look deeper.

    Often the initial security analysis uses protocol headers or security and application logs.

    • This information is small, so it can be stored for more traffic over a longer period of time
    • This information can be determined from packets flying by with a minimum of processing, so it is possible to get the info without either spending a fortune in monitoring hardware or slowing down the network traffic

    There is a perception that the content data is “sensitive” and that header data is “anonymized” or information that people don’t care as much about. If you have time to read Kieran Healy’s amusing article on “Using Metadata to Find Paul Revere”, you can see that this is not always a valid perception. Side note: there’s a deeper article linked from the bottom of that page that is also worth reading if you want a more in-depth analysis.

    Let’s see what we can find from the most basic packet header data. The trick here is to rearrange the data in various ways and look for ‘spikes’ in frequency. Then, look at what all the frequent items have in common, such as the same protocol, similar IP addresses, etc. To do this well, you need to know about common TCP and UDP services. Fortunately, we have Google to help us with that!

    You are provided with two samples of packet data, R and O. The packet data is in order, and covers a brief period of time. The same traffic is published in two formats, and which one you use depends on which course you are enrolled in:

    Enrolled in Use Parse it with
    CS 491 (undergraduate) R.csv, O.csv Python’s built-in csv module, or pandas
    CS 591 (graduate) R.pcap.gz, O.pcap.gz scapy

    No parsing code is supplied, and you should not go looking for a bespoke library to do it for you. Getting data out of a file and into a structure you can count is the skill the rest of this assignment is built on, and both of the libraries above are a handful of lines from what you need. 591 students who would rather use tshark, pyshark, or dpkt than scapy may do so — just say what you used.

    For each packet, you get:

    • Length, IP type, IP Source Address, IP Destination Address
    • TCP: Flags, Source port, Destination Port
    • UDP: Source port, Destination port
    • ICMP: type, code

    The CSV files hold exactly those fields and nothing else, one packet per row, so you can look at one in a terminal or read it into a spreadsheet to see what you are dealing with. The captures hold the same packets as real Ethernet frames, taken with a 96-byte snaplen — headers only, no payload. That is deliberate, and it is the trade-off described above: header data is small enough to keep for a lot of traffic over a long period. Wireshark will note that the packets were “limited during capture”; that is expected, not damage.

    The R data (100k packets) is a lot smaller than the O data (1M packets). Prototype your scripts using R data, then switch to O data to apply the concepts on a larger data set.

    Both formats describe the same traffic, so they must produce the same answers. If your numbers disagree with those of a classmate who used the other format, one of you has a bug.

    The ultimate goal of this exercise is to understand the function of the networks monitored in the R and O data sets. What is their function? We will choose among: work, home, data center, ISP. On the way, we’ll learn as much as we can about these networks from the scanty data that we have.

    Start by collecting statistics based on IP protocol numbers. Run against the R data, your first script should report something like this:

    packets 99142, bytes 71683046
      1:         7
      2:         2
      6:     39138
     17:     59995
    

    You can look the IP numbers up in /etc/protocols to find out that IP protocol 1=icmp; 2=igmp; 6=tcp; 17=udp. For example, ICMP (IP Protocol 1) occurs in 7 packets. You will have learned a little of these protocols from the advance reading. If not, google them now!

    Each protocol has its own uses. For example, IGMP (proto=2) is used for router-to-router communications. So we could find the addresses of routers by looking at flows that use this protocol. Without writing a script at all, we could just filter from the command line:

    ❯ grep -e len -e '^[0-9][0-9]*,2,' R.csv
    len,proto,ipsrc,ipdst,tcpflags,tcpsport,tcpdport,udpsport,udpdport,icmpcode,icmptype
    28,2,10.5.63.36,234.42.42.42,,,,,,,
    28,2,10.5.63.36,234.142.142.142,,,,,,,
    

    The same question asked of the capture:

    ❯ tshark -r R.pcap.gz -Y 'ip.proto == 2' -T fields -e ip.src -e ip.dst
    10.5.63.36	234.42.42.42
    10.5.63.36	234.142.142.142
    

    We can now infer that 10.5.63.36 is a router IP address. The other addresses that start with 234...* are multicast addresses (put them into google to see this).

    Checking your work with tshark’s built-in statistics

    tshark ships a set of statistics taps, reached with -z, that compute several of the things this assignment asks you to compute. They do not replace the script — you are asked below for specific options and output formats, and one part (question 10) is not something any tap produces. Use them to check your script’s numbers, which is exactly how you would sanity-check a real analysis:

    Tap What it reports Relevant to
    -z io,phs protocol hierarchy, with frame and byte counts per layer the protocol histogram above
    -z endpoints,ip every IP address with packet and byte counts, sent and received question 3, --countip
    -z dests,tree destination address → protocol → port, with counts and percentages question 9, --connto
    -z endpoints,tcp / -z endpoints,udp each address/port pair with its packet counts question 1
    -z conv,tcp / -z conv,udp conversations, as address/port pairs questions 9–10
    -z io,stat,0,<filters> a count for each display filter you supply anything you can write a filter for

    For example, counting two services and two protocols in one pass:

    ❯ tshark -r R.pcap.gz -q -z 'io,stat,0,FRAMES()ip.proto#1==6,FRAMES()ip.proto#1==17,COUNT(tcp.dstport)tcp.dstport==139,COUNT(udp.dstport)udp.dstport==53'
    ==================================================
    | IO Statistics                                  |
    |                                                |
    | Duration: 75.0 secs                            |
    | Interval: 75.0 secs                            |
    |                                                |
    | Col  1: FRAMES()ip.proto#1==6                  |
    |      2: FRAMES()ip.proto#1==17                 |
    |      3: COUNT(tcp.dstport)tcp.dstport==139     |
    |      4: COUNT(udp.dstport)udp.dstport==53      |
    |------------------------------------------------|
    |              |1       |2       |3      |4      |
    | Interval     | FRAMES | FRAMES | COUNT | COUNT |
    |------------------------------------------------|
    |  0.0 <> 75.0 |  39138 |  59995 |  9455 |   428 |
    ==================================================
    

    Two traps in that command line, both of which will silently give you wrong answers:

    • ip.proto can appear more than once in a packet. An ICMP error quotes the header of the datagram that provoked it, and a GRE packet contains the header of the packet it is tunnelling, so those packets have an inner ip.proto as well as an outer one. Filtering on ip.proto == 17 matches a packet if either layer is UDP, which over-counts. ip.proto#1 means “the first (outermost) occurrence”, which is the one you want. On the O data the difference is over 4,700 packets.
    • COUNT() counts how many times a field occurs, not how many packets matched. Use FRAMES() when you want a packet count. For the same reason as above, COUNT(ip.proto) on ICMP errors returns two per packet.

    The second trap has an analogue in the CSV: grep -c ,17, R.csv will match the string anywhere in the row, including inside an address. Count the field, not the line.

    1. Find statistics on TCP and UDP services.

      Most interesting user protocols run on TCP, although a few run on UDP. This usually works as follows:

      • A server process listens on a given TCP or UDP port number. By convention, these “well known ports” are allocated from 1-1024 (some are outside that range by now). A well-known port is often called a service.
      • Clients connect in to the server using this as the TCP or UDP destination port.

      The file /etc/services contains a listing of well-known port numbers for TCP and UDP.

      Extend your script’s statistics gathering to count the use of all well-known destination port numbers for TCP and UDP (ports 1-1024). For example, you should be able to look up in your output how many TCP packets have destination port 80 and how many UDP packets have destination port 53. Run your new script on the R and O data. Enable this function using a --stats flag — the script should produce no output unless --stats is given on the command line.

    2. Characterize the main function of each network. Based on the statistics from (1), what kind of network is each one: work, home, data center, or ISP? Commit to an answer for both R and O; you will revisit it as you gather more evidence.

    3. Add a --countip option which creates a list of distinct IP addresses with their usage counts. Sort the list by the usage count, not by the IP address.

    4. Run --countip on the R and O data. Does this change or reinforce your answer to (2)?

    5. Attempt to determine the network number (network prefix) that seems to dominate the traffic.

    6. Some IP protocols are used only between routers or other special networking devices, so traffic in those protocols identifies the infrastructure of the network under observation. Generate sorted --countip output restricted to each of the following, and identify the IP addresses that use them:

      • GRE (Generic Routing Encapsulation) — used to create tunnels between networks with overlapping address spaces. It is also the base protocol for PPTP, a remote access mechanism.
      • IPsec — the protocol suite that creates virtual private networks, an overlay network structure on top of the Internet. Most IPsec is router-to-router these days.
      • OSPF (Open Shortest Path First) — the “standard” routing protocol for Internet routers, allowing them to discover the topology and choose the best paths as connections appear and disappear.

      Hint: add a protocol argument that filters the data going into --countip to just these protocols. Alternately, use the grep pipeline shown above for IGMP traffic and change the 2 to the right protocol number, or the tshark filter and change ip.proto == 2 to the right one. You can find the numbers in /etc/protocols.

    7. Find another network prefix that also seems to be associated with this traffic.

    8. Does the OSPF information change or reinforce your answer to (2)?

      The server machines are the main assets of each site. Can we find them?

      Imagine drawing a diagram for each service, with a line from each client machine that uses the service to the server that provides it. Then the servers look like stars, with clients around each server. Important servers will stick out as stars with lots of spokes. To find the “stars” in the diagram, we want to find IP addresses that are the destination of many transactions.

    9. Add a --connto option which counts the number of packets sent to each service (ports 1-1024) on the network. For example, a dictionary maps each destination address to the tuple <proto, dport>, where proto is tcp or udp based on the IP protocol (6 or 17) and dport is that packet’s TCP or UDP destination port.

    10. Sort the --connto output by the number of distinct source IP address / source port combinations, so that servers which serve a lot of different connections all cluster at one end of the output. Generate a summary line that shows, for each destination IP address, how many distinct source IP addresses accessed it and what ports were referenced:

      ipdst 1.2.3.4 has 334 distinct ipsrc on ports: udp/53, tcp/80, tcp/443
      ipdst 5.6.7.8 has 335 distinct ipsrc on ports: tcp/22, tcp/25
      

    Include your script, the output of your script, and your analysis of the output in your hw4.md file. Include any graphs, images, or other artifacts you generate as part of this analysis. Runnable files and visual artifacts should also live in your repo, not just in your document.

Submission

Once you have completed the above, you should have a markdown file in your repo called hw4/hw4.md that contains all the requested information. Commit and push this to your repo. Also commit and include any requested screenshots. Once you have done this, you can consider the assignment submitted.

In order to include output from your shell, see the technical writing page for some basic instructions. For instance:

❯ ps -efH --no-header | awk '{print $1}' | grep -Ev $(python3 -c 'import sys; print("|".join(sys.argv[1:]))' $(cut -f1 -d':' /etc/passwd)) | sort | uniq -c | sort -rn | head -n 11
     30 USER01
     27 USER02
     24 USER03
     23 USER04
     23 USER05
     22 USER06
     22 USER07
     19 USER08
     19 USER09
     19 USER10
     18 USER11

Could be created with the following markdown:

```sh
❯ ps -efH --no-header | awk '{print $1}' | grep -Ev $(python3 -c 'import sys; print("|".join(sys.argv[1:]))' $(cut -f1 -d':' /etc/passwd)) | sort | uniq -c | sort -rn | head -n 11
     30 USER01
     27 USER02
     24 USER03
     23 USER04
     23 USER05
     22 USER06
     22 USER07
     19 USER08
     19 USER09
     19 USER10
     18 USER11
```