courses

Packet Mangling with NFQUEUE

Why this matters

Everything you have done so far with firewalls has been a decision: accept, drop, reject, redirect. The rule looks at the packet and votes. What it never does is change the packet’s contents — and a great deal of interesting network security lives exactly there. Transparent proxies rewrite headers. Protocol downgrade attacks strip a STARTTLS capability out of an SMTP banner. Response injection races a legitimate DNS server. Traffic normalizers rewrite ambiguous fields so that an IDS and an endpoint cannot be made to disagree about what a packet means.

nftables has a mechanism for this, and it is not a special “mangle” feature — it is an escape hatch. The queue statement hands the packet to a program you write, in userspace, which can inspect it, rewrite any byte of it, and then tell the kernel what to do with the result. From man nft(8):

This statement passes the packet to userspace using the nfnetlink_queue handler. The packet is put into the queue identified by its 16-bit queue number. Userspace can inspect and optionally modify the packet if desired. Userspace must provide a drop or accept verdict.

That one sentence is the whole feature. Everything below is the consequences of it — and the consequences are sharper than they look, because a packet mangler that is almost right does not fail loudly. It corrupts data silently, and we will demonstrate that happening.

This page assumes you are comfortable with firewalls and with reading packets in scapy.

How it works

The path a packet takes

flowchart LR
    A[NIC] --> B[nftables hook]
    B -->|rule matches<br/>queue to 0| C[nfnetlink_queue]
    C --> D[your program<br/>userspace]
    D -->|verdict + optional<br/>rewritten payload| C
    C --> E[next base chain hook]
    B -->|no match| E
    E --> F[delivery / forwarding]

In words, for anyone whose reader does not render the diagram: a packet arrives at a NIC and reaches an nftables hook. If no rule matches, it continues to the next base chain hook and on to delivery or forwarding. If a rule with queue to 0 matches, the packet is handed to nfnetlink_queue, which passes it to your userspace program; that program returns a verdict plus an optionally rewritten payload, and the packet re-enters the stack at the next base chain hook.

Three things about that path are worth pinning down, because each of them surprises people:

  1. The packet is copied to userspace and back. This is a context switch and a memory copy per packet. It is not a fast path.
  2. accept resumes at the next base chain hook, not at the rule after queue. The remaining rules in the chain you queued from are skipped. If you were expecting queue to behave like a counter — do the thing, then keep evaluating — you will misread your own ruleset.
  3. If nothing is listening on that queue number, the packets are dropped. By default. Silently. We will prove this in a moment because it is the single most common way students lock themselves out of a remote machine.

The rule syntax

table inet mangleq {
    chain output {
        type filter hook output priority filter; policy accept;
        udp dport 9999 queue flags bypass to 0
    }
}

man nft(8) gives the full grammar:

queue [flags QUEUE_FLAGS] [to queue_number]
queue [flags QUEUE_FLAGS] [to queue_number_from - queue_number_to]
queue [flags QUEUE_FLAGS] [to QUEUE_EXPRESSION ]

QUEUE_FLAGS      := bypass | fanout
QUEUE_EXPRESSION := numgen | hash | symhash | MAP STATEMENT
Form Meaning
queue to 0 Send to queue 0. Fail closed — no listener means the packet dies.
queue flags bypass to 0 Fail open — no listener means the rule acts like accept.
queue flags bypass,fanout to 0-3 Spread across queues 0–3 by CPU ID, one worker per core.
queue to numgen inc mod 4 Round-robin the queue number at runtime.
queue to jhash ip saddr mod 4 Pin each source address to a consistent queue — keeps a flow with one worker.

Every one of those forms was syntax-checked with nft -c against nftables v1.1.6 before it was written down here. Up to 65,535 queues are available, and the mechanism needs Linux 3.14 or later.

⚠️ bypass is a security trade-off, not a convenience flag. With it, an attacker who can crash or merely stall your handler has turned your inspection off while traffic keeps flowing. Without it, that same attacker has caused a denial of service. There is no third option: you are choosing which failure mode you prefer, and you should choose it deliberately rather than by forgetting to type a flag. For an IPS, fail-closed is usually right. For a rule on the box you SSH into, fail-open will save your afternoon.

Building a lab you cannot break

Do not experiment with this on a machine you care about, and especially not on the interface carrying your SSH session. Use a pair of network namespaces: they are free, they are completely isolated from the host’s networking, and ip netns del erases every trace.

#!/bin/bash
set -e
ip netns add nfqa
ip netns add nfqb
ip link add va type veth peer name vb
ip link set va netns nfqa
ip link set vb netns nfqb
ip -n nfqa addr add 10.99.0.1/24 dev va
ip -n nfqb addr add 10.99.0.2/24 dev vb
ip -n nfqa link set va up
ip -n nfqb link set vb up
ip -n nfqa link set lo up
ip -n nfqb link set lo up
# Turn off offloads so we see real, individual packets with real checksums.
ip netns exec nfqa ethtool -K va tx off rx off gso off tso off gro off
ip netns exec nfqb ethtool -K vb tx off rx off gso off tso off gro off
ip netns exec nfqa ping -c1 10.99.0.2

Tear it down with ip netns del nfqa; ip netns del nfqb.

That ethtool line matters more than it looks. With offloads enabled, the kernel hands your hook a synthetic 64 KB “super packet” that never existed on the wire, and it also lets checksums be computed by the NIC after your hook runs — which means a checksum bug in your handler will not show up in the lab and will show up in production. Turn the offloads off while you are learning so the lab tells you the truth.

On Kali, the userspace library binding is one package:

$ sudo apt-get install -y python3-netfilterqueue
Setting up python3-netfilterqueue (1.1.0-5)…

Worked example 1: observing before you modify

Always start read-only. This handler prints every queued packet and accepts it unchanged:

#!/usr/bin/env python3
"""Observe-only NFQUEUE handler: print each packet, then accept it unchanged."""
from netfilterqueue import NetfilterQueue
from scapy.layers.inet import IP

def handle(pkt):
    p = IP(pkt.get_payload())
    print(f"{p.src} -> {p.dst}  proto={p.proto}  len={len(p)}  payload={bytes(p.payload.payload)!r}",
          flush=True)
    pkt.accept()

nfq = NetfilterQueue()
nfq.bind(0, handle)
try:
    nfq.run()
except KeyboardInterrupt:
    pass
finally:
    nfq.unbind()

pkt.get_payload() returns the raw bytes starting at the IP header, which is exactly what scapy’s IP() wants. With the rule from earlier loaded in nfqa and three UDP datagrams sent to nfqb:

=== ruleset in nfqa ===
table inet nfqlab {
	chain output {
		type filter hook output priority filter; policy accept;
		udp dport 9999 queue flags bypass to 0
	}
}

=== handler saw ===
10.99.0.1 -> 10.99.0.2  proto=17  len=34  payload=b'alpha\n'
10.99.0.1 -> 10.99.0.2  proto=17  len=34  payload=b'bravo\n'
10.99.0.1 -> 10.99.0.2  proto=17  len=36  payload=b'charlie\n'

The plumbing works. Now we can start changing things.

Worked example 2: rewriting a payload

The rule stays the same; only the handler changes. This one substitutes foo → bar in UDP payloads:

#!/usr/bin/env python3
"""Rewrite UDP payloads on the fly: foo -> bar (equal length)."""
from netfilterqueue import NetfilterQueue
from scapy.layers.inet import IP, UDP

def handle(pkt):
    p = IP(pkt.get_payload())
    if UDP in p and b"foo" in bytes(p[UDP].payload):
        old = bytes(p[UDP].payload)
        new = old.replace(b"foo", b"bar")
        p[UDP].remove_payload()
        p[UDP].add_payload(new)
        del p[IP].chksum          # force scapy to recompute both checksums
        del p[UDP].chksum
        print(f"rewrote {old!r} -> {new!r}", flush=True)
        pkt.set_payload(bytes(IP(bytes(p))))
    pkt.accept()

nfq = NetfilterQueue()
nfq.bind(0, handle)
try:
    nfq.run()
except KeyboardInterrupt:
    pass
finally:
    nfq.unbind()

Three messages sent from nfqa, with a plain UDP receiver in nfqb:

=== handler log ===
rewrote b'the foo is loose' -> b'the bar is loose'
rewrote b'foo foo foo' -> b'bar bar bar'
=== what nfqb actually received ===
the bar is loose
no match here
bar bar bar

The receiving process has no way to know this happened. It made an ordinary recvfrom() call and got bytes that the sender never sent.

The del p[IP].chksum / del p[UDP].chksum idiom is scapy-specific: deleting a field marks it unset, so scapy recomputes it when the packet is serialised. Leaving the old value in place is the single most common NFQUEUE bug, so let us look at exactly what it costs.

Worked example 3: the checksum failure, deliberately

Same rewrite, one difference — the two del lines are gone:

        new = bytes(p[UDP].payload).replace(b"foo", b"bar")
        p[UDP].remove_payload()
        p[UDP].add_payload(new)
        # NOTE: no `del p[IP].chksum` / `del p[UDP].chksum` here.
        print(f"rewrote, stale udp chksum=0x{p[UDP].chksum:04x}", flush=True)
        pkt.set_payload(bytes(p))

=== nfqb UDP counters BEFORE ===
UdpInDatagrams                  4                  0.0
UdpInErrors                     0                  0.0
UdpInCsumErrors                 0                  0.0
=== handler log ===
rewrote, stale udp chksum=0xdadb
rewrote, stale udp chksum=0xbad9
=== nfqb received ===
(end)
=== nfqb UDP counters AFTER ===
UdpInErrors                     2                  0.0
UdpInCsumErrors                 2                  0.0

The handler is convinced it is working. It logged two successful rewrites. The receiving application received nothing at all, and produced no error, because the kernel discarded both datagrams before they ever reached a socket. The only evidence anywhere on the system is a counter in nstat.

This is the shape of nearly every packet-mangling bug you will hit: the sending side reports success and the receiving side reports silence. Learn to check nstat on the receiver. UdpInCsumErrors and TcpInCsumErrors are the first things to look at when a rewrite “does nothing”.

Note also which checksum broke. The substitution was equal-length, so the IP total length never changed and the IP header checksum stayed valid — only the UDP checksum, which covers the payload, went stale. A handler that recomputes only the IP checksum will pass a naive test and fail a real one.

Worked example 4: why you cannot change the length of a TCP payload

TCP sequence numbers count bytes, and they are agreed end-to-end. If your mangler removes three bytes from a segment, the receiver’s idea of “how many bytes have arrived” permanently diverges from the sender’s by three — and neither endpoint has any idea why.

The intuitive prediction is that the connection stalls. The reality is worse. Here is the foo is loose\n (17 bytes) sent over TCP through a handler that replaces foo with the empty string:

=== sender wrote this many bytes ===
17
=== handler ===
shrink: b'the foo is loose\n' -> b'the  is loose\n'
=== receiver, byte for byte ===
00000000: 7468 6520 2069 7320 6c6f 6f73 650a 7365  the  is loose.se
00000010: 0a                                       .

Seventeen bytes in, seventeen bytes out — but not the same seventeen. The receiver got the is loose\n (the 14 bytes we sent it), then TCP noticed a three-byte hole, the sender retransmitted, and the retransmitted segment’s tail landed at the wrong offset and appended se\n. The stream is now corrupted, not truncated, and every layer above TCP believes it is reading clean data.

Compare the equal-length run through the same code path:

--- MODE=equal ---
handler: equal: b'the foo is loose\n' -> b'the bar is loose\n'
receiver got: the bar is loose

So the rule is:

⚠️ Equal-length substitution in a TCP payload is safe. Changing the length is not. If you must add or remove bytes from a TCP stream, you need a proxy — something like TPROXY plus a userspace listener that terminates the client connection and originates a fresh one to the server, so that each side gets its own consistent sequence space. A packet mangler cannot do it, and the way it fails is silent corruption rather than an error.

UDP has no such constraint: each datagram is independent, so you can resize freely as long as you fix the IP total length and the UDP length and checksum.

Deployment models: where you put the mangler

Everything so far has assumed the packets are already passing through the machine running the rules. That assumption is where most real designs fall over, because the obvious way to get at “all the traffic” — a mirror port and a promiscuous interface — turns out not to work at all.

A promiscuous tap cannot be mangled

Consider the appliance everyone reaches for first: a box with a NIC in promiscuous mode on a SPAN/mirror port, seeing everything, rewriting what it dislikes. Build it and measure what each observation point actually sees. The topology is c (10.50.0.1) ── bridge ── s (10.50.0.2), with a third namespace w receiving mirrored copies via tc mirred. w has no IP address and is in PROMISC mode. Five pings from c to s:

Observation point on the tap Packets seen
AF_PACKET (what tcpdump uses) 32
nftables netdev / ingress 5
nftables ip / prerouting 0

The IP hooks see nothing at all. A frame whose destination MAC is not yours is marked PACKET_OTHERHOST and discarded inside ip_rcv_core() before NF_INET_PRE_ROUTING is ever reached. Promiscuous mode delivers frames to AF_PACKET sockets; it does not deliver them to netfilter. Every rule you write in the ip, ip6, or inet families is dead code on tapped traffic.

That leaves the netdev family’s ingress hook, which does see the mirrored packets — it runs at the tc layer, ahead of the pkt_type check. So can you queue from there?

$ nft -f - <<< 'table netdev q { chain ing {
      type filter hook ingress device vw priority 0; queue flags bypass to 0 } }'
/dev/stdin:4:9-31: Error: Could not process rule: Operation not supported
        queue flags bypass to 0
        ^^^^^^^^^^^^^^^^^^^^^^^

No. The kernel rejects queue in the ingress hook, because a verdict there has no reinjection path. The only hook that can see tapped traffic is the only hook that cannot queue it.

⚠️ Even if the plumbing allowed it, the design is self-defeating: a tap gives you a copy. By the time your handler is looking at the packet, the original has already been delivered. Modifying a copy modifies nothing. “Transparent tap with inline modification” is a contradiction in terms — to change traffic you must be in the path, not beside it. This distinction between a tap (out-of-band, observe only, fails open by construction) and an inline device (in-band, can modify, is now a single point of failure) is worth being precise about, because vendors are frequently not.

Bump-in-the-wire: the model that works

An inline mangler is a transparent bridge: two NICs joined by a Linux bridge, no IP addresses anywhere, and rules in the bridge family’s forward hook.

table bridge b {
    chain f {
        type filter hook forward priority 0; policy accept;
        udp dport 9999 queue flags bypass to 0
    }
}

Running the foo → bar handler from Worked example 2 unchanged on that bridge, between two hosts that know nothing about it:

=== c sent ===
  the foo is loose / clean traffic / foo again
=== mangler (running on the bridge, which has NO IP address) ===
rewrote b'the foo is loose' -> b'the bar is loose'
rewrote b'foo again' -> b'bar again'
=== s received ===
  the bar is loose / clean traffic / bar again
=== confirm the bridge really is transparent (no L3 identity) ===
  IPv4 addresses on br0: 0
  ip_forward: 0

No address, no routing, and — because bridging is a layer-2 operation — no TTL decrement, so the device does not appear in a traceroute. It is invisible to both endpoints and rewrites their traffic.

Two details you need before building one:

The payload starts at the IP header, not the Ethernet header. Printing the raw bytes the bridge-family queue hands over:

[1] raw[0:20] = 450000548749400040019ef90a3200010a320002
[1] as IP()  = IP / ICMP 10.50.0.1 > 10.50.0.2 echo-request 0 / Raw

0x45 is IPv4 with a 5-word header — so scapy’s IP() parses it directly and every handler on this page works unmodified in the bridge family. The corollary is that the L2 header is not in the buffer you replace, so you cannot rewrite MAC addresses this way.

Use the native bridge family, not br_netfilter. If you would rather write inet-family rules, the br_netfilter module can route bridged traffic through them — but it is off by default:

net.bridge.bridge-nf-call-iptables inet/forward sees bridge/forward sees
0 (default) 0 packets 3 packets
1 3 packets 3 packets

It works, but it is a global toggle that changes behaviour for every bridge on the host — this is the setting notorious for breaking Docker and Kubernetes networking — and it adds per-packet cost. For a purpose-built appliance, write bridge-family rules and leave br_netfilter alone.

If you really want promiscuous mode

Skip netfilter entirely. Open an AF_PACKET socket on each NIC in promiscuous mode, read a frame from one, modify it, write it to the other. You are implementing the bridge yourself, which means you get the complete L2 frame including MAC addresses and you are not subject to any of the hook restrictions above — at the cost of losing MAC learning, STP, and everything else the kernel bridge provides. This is broadly how commercial inline appliances are built, and it scales with AF_XDP or DPDK when a per-packet syscall becomes the bottleneck.

Operational consequences of being inline

When you should not use NFQUEUE at all

NFQUEUE is the right tool when you need to parse — when the decision depends on application-layer structure that nftables cannot express. It is the wrong tool when a fixed-offset header edit would do, because nftables can do those in-kernel at a small fraction of the cost:

ip saddr set 10.0.0.1              # source NAT by hand
ip dscp set af41                   # remark for QoS
ip ttl set 64                      # normalise TTL (defeats some OS fingerprinting)
tcp option maxseg size set 1400    # MSS clamping
@th,16,16 set 0x0000               # raw write, 16 bits at transport-header offset 16
meta mark set 0x1                  # tag for policy routing

That last raw-payload form (@base,offset,length) is worth knowing: @th is the transport header, @nh the network header, @ll the link layer, with offset and length in bits. The kernel fixes the checksums for you on these. It is limited to fixed offsets — it cannot search — but when it fits, it is orders of magnitude faster than a trip to userspace.

Roughly:

Need Use
Rewrite a field at a known offset Native nftables mangling
Decide based on parsed application data NFQUEUE
Change the byte-length of a TCP stream A proxy (TPROXY)
Line-rate processing of all traffic eBPF/XDP or tc
Detect and alert, not modify Suricata

Performance and the queue itself

Every queued packet is a round trip to userspace. Some practical consequences:

Security implications worth thinking about

The reason this belongs in a network security course rather than a systems course is that NFQUEUE is a capability, and capabilities cut both ways:

Key takeaways

References

Related course pages: Defensive Measures · Introduction to scapy · Suricata IDS/IPS · Network Traffic Capture · FreeBSD Netgraph

🛠️ Maintenance note: all rule forms on this page were syntax-checked against nftables v1.1.6 and the transcripts produced on kernel 6.18 with python3-netfilterqueue 1.1.0-5 and scapy 2.7.01; re-verify each term, as nft occasionally tightens its parser and scapy’s checksum-recompute idiom (del p[IP].chksum) is the kind of API detail that shifts between major versions. The offload names accepted by ethtool -K also drift with kernel releases — if a name is rejected, check ethtool -k <dev> for the current spelling. If the course VM image changes distributions, confirm the package name (python3-netfilterqueue) still exists. The “queue in netdev/ingress is unsupported” result is a kernel limitation rather than an nftables one, so it is the claim on this page most likely to change without warning — re-run that one-line nft -f test each term rather than trusting the transcript.