Homework 4
Due Date: 2026-11-20 23:59:59 (Friday of week 8)
Introduction
This homework covers the general topic of network security. You will be looking at packet capture filters, packet traces in wireshark, and some basic network identification tasks.
What you must do
- Complete the following module on HackTheBox: Introduction to Networking
- Complete the following module on HackTheBox: Threat Hunting with IPS/IDS
- Complete the following
tcpdump/BPF practice exercises.- Perform a tcpdump capture where you only capture DNS packets.
- Perform a tcpdump capture where you capture TCP packets that are destined for either port 443 or 8080, and originate from your computer.
- Perform a tcpdump capture where traffic is either UDP or TCP, is inbound to your computer, and destined for a port between 20000 and 35000.
All dumps should have full data dumps without link layer headers, in hex and ascii, with Unix epoch style timestamps. Something like this:
1585339575.038549 IP (tos 0x0, ttl 64, id 37629, offset 0, flags [DF], proto UDP (17), length 60) kali478-0.38378 > _gateway.domain: [bad udp cksum 0x6ede -> 0x96ae!] 54659+ A? www.google.com. (32) 0x0000: 4500 003c 92fd 4000 4011 390f ac10 8b81 E..<..@.@.9..... 0x0010: ac10 8b02 95ea 0035 0028 6ede d583 0100 .......5.(n..... 0x0020: 0001 0000 0000 0000 0377 7777 0667 6f6f .........www.goo 0x0030: 676c 6503 636f 6d00 0001 0001 gle.com.....Captures can be included in your markdown file by enclosing them in triple backticks, like so:
``` 1585339575.038549 IP (tos 0x0, ttl 64, id 37629, offset 0, flags [DF], proto UDP (17), length 60) kali478-0.38378 > _gateway.domain: [bad udp cksum 0x6ede -> 0x96ae!] 54659+ A? www.google.com. (32) 0x0000: 4500 003c 92fd 4000 4011 390f ac10 8b81 E..<..@.@.9..... 0x0010: ac10 8b02 95ea 0035 0028 6ede d583 0100 .......5.(n..... 0x0020: 0001 0000 0000 0000 0377 7777 0667 6f6f .........www.goo 0x0030: 676c 6503 636f 6d00 0001 0001 gle.com..... ```A minimum of 10 packets, as requested, should be included for each capture. You can use the
-cflag to limit the number of packets captured. You will likely need to trigger traffic for some of these captures. That is part of the exercise. -
We often wish to examine network traffic to analyze whether a given network is running correctly, is under attack, has secrets in it we would like to reveal or protect, and so on. There is an engineering trade-off here. We can’t store and analyze all the data that flies by in the network. So we would like to use a portion of the total data to decide where to look deeper.
Often the initial security analysis uses protocol headers or security and application logs.
- This information is small, so it can be stored for more traffic over a longer period of time
- This information can be determined from packets flying by with a minimum of processing, so it is possible to get the info without either spending a fortune in monitoring hardware or slowing down the network traffic
There is a perception that the content data is “sensitive” and that header data is “anonymized” or information that people don’t care as much about. If you have time to read Kieran Healy’s amusing article on “Using Metadata to Find Paul Revere”, you can see that this is not always a valid perception. Side note: there’s a deeper article linked from the bottom of that page that is also worth reading if you want a more in-depth analysis.
Let’s see what we can find from the most basic packet header data. The trick here is to rearrange the data in various ways and look for ‘spikes’ in frequency. Then, look at what all the frequent items have in common, such as the same protocol, similar IP addresses, etc. To do this well, you need to know about common TCP and UDP services. Fortunately, we have Google to help us with that!
You are provided with two samples of packet data, R and O. The packet data is in order, and covers a brief period of time. The same traffic is published in two formats, and which one you use depends on which course you are enrolled in:
Enrolled in Use Parse it with CS 491 (undergraduate) R.csv,O.csvPython’s built-in csvmodule, orpandasCS 591 (graduate) R.pcap.gz,O.pcap.gzscapyNo parsing code is supplied, and you should not go looking for a bespoke library to do it for you. Getting data out of a file and into a structure you can count is the skill the rest of this assignment is built on, and both of the libraries above are a handful of lines from what you need. 591 students who would rather use
tshark,pyshark, ordpktthanscapymay do so — just say what you used.For each packet, you get:
- Length, IP type, IP Source Address, IP Destination Address
- TCP: Flags, Source port, Destination Port
- UDP: Source port, Destination port
- ICMP: type, code
The CSV files hold exactly those fields and nothing else, one packet per row, so you can look at one in a terminal or read it into a spreadsheet to see what you are dealing with. The captures hold the same packets as real Ethernet frames, taken with a 96-byte snaplen — headers only, no payload. That is deliberate, and it is the trade-off described above: header data is small enough to keep for a lot of traffic over a long period. Wireshark will note that the packets were “limited during capture”; that is expected, not damage.
The R data (100k packets) is a lot smaller than the O data (1M packets). Prototype your scripts using R data, then switch to O data to apply the concepts on a larger data set.
Both formats describe the same traffic, so they must produce the same answers. If your numbers disagree with those of a classmate who used the other format, one of you has a bug.
The ultimate goal of this exercise is to understand the function of the networks monitored in the R and O data sets. What is their function? We will choose among: work, home, data center, ISP. On the way, we’ll learn as much as we can about these networks from the scanty data that we have.
Start by collecting statistics based on IP protocol numbers. Run against the R data, your first script should report something like this:
packets 99142, bytes 71683046 1: 7 2: 2 6: 39138 17: 59995You can look the IP numbers up in
/etc/protocolsto find out that IP protocol1=icmp;2=igmp;6=tcp;17=udp. For example, ICMP (IP Protocol 1) occurs in 7 packets. You will have learned a little of these protocols from the advance reading. If not, google them now!Each protocol has its own uses. For example, IGMP (proto=2) is used for router-to-router communications. So we could find the addresses of routers by looking at flows that use this protocol. Without writing a script at all, we could just filter from the command line:
❯ grep -e len -e '^[0-9][0-9]*,2,' R.csv len,proto,ipsrc,ipdst,tcpflags,tcpsport,tcpdport,udpsport,udpdport,icmpcode,icmptype 28,2,10.5.63.36,234.42.42.42,,,,,,, 28,2,10.5.63.36,234.142.142.142,,,,,,,The same question asked of the capture:
❯ tshark -r R.pcap.gz -Y 'ip.proto == 2' -T fields -e ip.src -e ip.dst 10.5.63.36 234.42.42.42 10.5.63.36 234.142.142.142We can now infer that 10.5.63.36 is a router IP address. The other addresses that start with 234...* are multicast addresses (put them into google to see this).
Checking your work with tshark’s built-in statistics
tsharkships a set of statistics taps, reached with-z, that compute several of the things this assignment asks you to compute. They do not replace the script — you are asked below for specific options and output formats, and one part (question 10) is not something any tap produces. Use them to check your script’s numbers, which is exactly how you would sanity-check a real analysis:Tap What it reports Relevant to -z io,phsprotocol hierarchy, with frame and byte counts per layer the protocol histogram above -z endpoints,ipevery IP address with packet and byte counts, sent and received question 3, --countip-z dests,treedestination address → protocol → port, with counts and percentages question 9, --connto-z endpoints,tcp/-z endpoints,udpeach address/port pair with its packet counts question 1 -z conv,tcp/-z conv,udpconversations, as address/port pairs questions 9–10 -z io,stat,0,<filters>a count for each display filter you supply anything you can write a filter for For example, counting two services and two protocols in one pass:
❯ tshark -r R.pcap.gz -q -z 'io,stat,0,FRAMES()ip.proto#1==6,FRAMES()ip.proto#1==17,COUNT(tcp.dstport)tcp.dstport==139,COUNT(udp.dstport)udp.dstport==53' ================================================== | IO Statistics | | | | Duration: 75.0 secs | | Interval: 75.0 secs | | | | Col 1: FRAMES()ip.proto#1==6 | | 2: FRAMES()ip.proto#1==17 | | 3: COUNT(tcp.dstport)tcp.dstport==139 | | 4: COUNT(udp.dstport)udp.dstport==53 | |------------------------------------------------| | |1 |2 |3 |4 | | Interval | FRAMES | FRAMES | COUNT | COUNT | |------------------------------------------------| | 0.0 <> 75.0 | 39138 | 59995 | 9455 | 428 | ==================================================Two traps in that command line, both of which will silently give you wrong answers:
ip.protocan appear more than once in a packet. An ICMP error quotes the header of the datagram that provoked it, and a GRE packet contains the header of the packet it is tunnelling, so those packets have an innerip.protoas well as an outer one. Filtering onip.proto == 17matches a packet if either layer is UDP, which over-counts.ip.proto#1means “the first (outermost) occurrence”, which is the one you want. On the O data the difference is over 4,700 packets.COUNT()counts how many times a field occurs, not how many packets matched. UseFRAMES()when you want a packet count. For the same reason as above,COUNT(ip.proto)on ICMP errors returns two per packet.
The second trap has an analogue in the CSV:
grep -c ,17, R.csvwill match the string anywhere in the row, including inside an address. Count the field, not the line.-
Find statistics on TCP and UDP services.
Most interesting user protocols run on TCP, although a few run on UDP. This usually works as follows:
- A server process listens on a given TCP or UDP port number. By convention, these “well known ports” are allocated from 1-1024 (some are outside that range by now). A well-known port is often called a service.
- Clients connect in to the server using this as the TCP or UDP destination port.
The file
/etc/servicescontains a listing of well-known port numbers for TCP and UDP.Extend your script’s statistics gathering to count the use of all well-known destination port numbers for TCP and UDP (ports 1-1024). For example, you should be able to look up in your output how many TCP packets have destination port 80 and how many UDP packets have destination port 53. Run your new script on the R and O data. Enable this function using a
--statsflag — the script should produce no output unless--statsis given on the command line. -
Characterize the main function of each network. Based on the statistics from (1), what kind of network is each one: work, home, data center, or ISP? Commit to an answer for both R and O; you will revisit it as you gather more evidence.
-
Add a
--countipoption which creates a list of distinct IP addresses with their usage counts. Sort the list by the usage count, not by the IP address. -
Run
--countipon the R and O data. Does this change or reinforce your answer to (2)? -
Attempt to determine the network number (network prefix) that seems to dominate the traffic.
-
Some IP protocols are used only between routers or other special networking devices, so traffic in those protocols identifies the infrastructure of the network under observation. Generate sorted
--countipoutput restricted to each of the following, and identify the IP addresses that use them:- GRE (Generic Routing Encapsulation) — used to create tunnels between networks with overlapping address spaces. It is also the base protocol for PPTP, a remote access mechanism.
- IPsec — the protocol suite that creates virtual private networks, an overlay network structure on top of the Internet. Most IPsec is router-to-router these days.
- OSPF (Open Shortest Path First) — the “standard” routing protocol for Internet routers, allowing them to discover the topology and choose the best paths as connections appear and disappear.
Hint: add a protocol argument that filters the data going into
--countipto just these protocols. Alternately, use thegreppipeline shown above for IGMP traffic and change the2to the right protocol number, or thetsharkfilter and changeip.proto == 2to the right one. You can find the numbers in/etc/protocols. -
Find another network prefix that also seems to be associated with this traffic.
-
Does the OSPF information change or reinforce your answer to (2)?
The server machines are the main assets of each site. Can we find them?
Imagine drawing a diagram for each service, with a line from each client machine that uses the service to the server that provides it. Then the servers look like stars, with clients around each server. Important servers will stick out as stars with lots of spokes. To find the “stars” in the diagram, we want to find IP addresses that are the destination of many transactions.
-
Add a
--conntooption which counts the number of packets sent to each service (ports 1-1024) on the network. For example, a dictionary maps each destination address to the tuple<proto, dport>, whereprotoistcporudpbased on the IP protocol (6 or 17) anddportis that packet’s TCP or UDP destination port. -
Sort the
--conntooutput by the number of distinct source IP address / source port combinations, so that servers which serve a lot of different connections all cluster at one end of the output. Generate a summary line that shows, for each destination IP address, how many distinct source IP addresses accessed it and what ports were referenced:ipdst 1.2.3.4 has 334 distinct ipsrc on ports: udp/53, tcp/80, tcp/443 ipdst 5.6.7.8 has 335 distinct ipsrc on ports: tcp/22, tcp/25
Include your script, the output of your script, and your analysis of the output in your
hw4.mdfile. Include any graphs, images, or other artifacts you generate as part of this analysis. Runnable files and visual artifacts should also live in your repo, not just in your document.
Submission
Once you have completed the above, you should have a markdown file in your repo called hw4/hw4.md that contains all the requested information. Commit and push this to your repo. Also commit and include any requested screenshots. Once you have done this, you can consider the assignment submitted.
In order to include output from your shell, see the technical writing page for some basic instructions. For instance:
❯ ps -efH --no-header | awk '{print $1}' | grep -Ev $(python3 -c 'import sys; print("|".join(sys.argv[1:]))' $(cut -f1 -d':' /etc/passwd)) | sort | uniq -c | sort -rn | head -n 11
30 USER01
27 USER02
24 USER03
23 USER04
23 USER05
22 USER06
22 USER07
19 USER08
19 USER09
19 USER10
18 USER11
Could be created with the following markdown:
```sh
❯ ps -efH --no-header | awk '{print $1}' | grep -Ev $(python3 -c 'import sys; print("|".join(sys.argv[1:]))' $(cut -f1 -d':' /etc/passwd)) | sort | uniq -c | sort -rn | head -n 11
30 USER01
27 USER02
24 USER03
23 USER04
23 USER05
22 USER06
22 USER07
19 USER08
19 USER09
19 USER10
18 USER11
```