Lua in Network Security
- Lua in Network Security
Lua is a lightweight, embeddable scripting language that has become a standard extension point for network security tools. Its small runtime, clean C API, and easy-to-learn syntax make it the natural choice when tool authors want to give users programmable customization without depending on a heavy scripting runtime. In network security you encounter Lua in three major contexts:
- Wireshark — custom protocol dissectors, traffic analysis listeners, and capture automation
- Nmap — the Nmap Scripting Engine (NSE), which drives nearly all of Nmap’s intelligent scanning
- Suricata — Lua-scripted detection rules for pattern matching beyond what signature syntax can express
Understanding Lua in these tools lets you move from running pre-built scripts to writing your own, which is essential for analyzing non-standard protocols, detecting novel threats, or automating repetitive tasks.
Lua Basics
Lua is dynamically typed, garbage-collected, and uses 1-based array indexing. A few things to know before reading the examples below:
-- This is a comment
-- Variables are global by default; use local to scope them
local x = 42
local s = "hello"
-- Tables are the universal data structure (array, map, object)
local t = { "a", "b", "c" } -- array-style (index 1, 2, 3)
local m = { host = "10.0.0.1", port = 80 } -- map-style
print(m.host) -- "10.0.0.1"
print(m["port"]) -- 80
-- Functions are first-class values
local function greet(name)
return "hello, " .. name -- .. is string concatenation
end
-- String methods
local line = "GET /index.html HTTP/1.1"
if line:find("^GET") then -- Lua pattern, not regex
print("it's a GET")
end
-- String patterns use % as the escape character (not \)
-- %d = digit, %s = whitespace, %a = letter, %p = punctuation
-- . * + ? work as in regex; ^ and $ anchor to start/end
Lua patterns are similar to regular expressions but not identical. The most common difference: use %d instead of \d, %s instead of \s, and %. to match a literal dot.
Nmap Scripting Engine (NSE)
Overview
NSE is what makes Nmap more than a port scanner. When you run nmap -sC or nmap --script=vuln, you are executing Lua scripts from /usr/share/nmap/scripts/. NSE scripts can:
- Banner-grab and fingerprint services beyond what version detection offers
- Test for specific vulnerabilities
- Brute-force credentials
- Enumerate users, shares, routes, DNS records
- Detect malware and backdoors
There are roughly 600 scripts in the default Nmap installation, organized into categories:
| Category | Description |
|---|---|
default |
Run with -sC; safe and generally useful |
safe |
Non-intrusive, unlikely to harm the target |
discovery |
Enumerate information about hosts and services |
version |
Enhance version detection |
auth |
Authentication credential testing |
brute |
Brute-force login attacks |
vuln |
Vulnerability checks |
exploit |
Actively exploit vulnerabilities (use with authorization) |
intrusive |
Likely to affect the target’s state or logs |
malware |
Detect signs of malware or backdoors |
fuzzer |
Send unexpected data to find crashes |
dos |
Denial of service tests (destructive; use with care) |
external |
Consult external resources (DNS, whois, etc.) |
broadcast |
Discover hosts via broadcast/multicast |
Running Scripts
# Run default scripts
nmap -sC 10.0.0.1
# Run a specific script
nmap --script=http-title 10.0.0.1
# Run all scripts in a category
nmap --script=vuln 10.0.0.1
# Run multiple scripts
nmap --script=http-title,http-headers 10.0.0.1 -p 80,443
# Run scripts matching a pattern
nmap --script="http-*" 10.0.0.1 -p 80
# Pass arguments to a script
nmap --script=http-brute --script-args http-brute.path=/login 10.0.0.1
# List all scripts in a category
nmap --script-help vuln
# Update the script database after adding new scripts
nmap --script-updatedb
Script Structure
Every NSE script is a Lua file with three sections:
-- ============================================================
-- HEAD: metadata
-- ============================================================
description = [[
Checks whether a web server is running behind a known CDN provider
by examining the Server header and IP geolocation.
]]
categories = {"discovery", "safe"}
author = "Your Name"
license = "Same as Nmap -- See https://nmap.org/book/man-legal.html"
-- NSE library imports
local http = require "http"
local stdnse = require "stdnse"
local shortport = require "shortport"
-- ============================================================
-- RULE: when should this script run?
-- ============================================================
-- portrule: run once per open port
portrule = shortport.http -- shorthand: run on any open HTTP port
-- hostrule: run once per host (use instead of portrule for host-level checks)
-- prerule: run once before scanning begins
-- postrule: run once after all hosts scanned
-- ============================================================
-- ACTION: the script's logic
-- ============================================================
action = function(host, port)
local response = http.get(host, port, "/")
if not response or not response.header then
return nil
end
local server = response.header["server"] or "unknown"
return stdnse.format_output(true, "Server: " .. server)
end
The action function receives host and port table objects with fields like host.ip, host.name, port.number, port.protocol, port.state. It returns a string (shown in nmap output) or nil to produce no output.
Key NSE Libraries
nmap — core API
local nmap = require "nmap"
-- Create a socket
local socket = nmap.new_socket()
socket:set_timeout(5000) -- milliseconds
-- Exception-safe wrapper: on error, calls the cleanup function and re-raises
local try = nmap.new_try(function() socket:close() end)
try(socket:connect(host.ip, port.number))
try(socket:send("HELLO\r\n"))
local response = try(socket:receive())
socket:close()
-- Check port state
local p = nmap.get_port_state(host, {number=22, protocol="tcp"})
if p and p.state == "open" then ... end
-- Set a port's state/version info
nmap.set_port_version(host, port)
shortport — concise portrule helpers
local shortport = require "shortport"
-- True if the port is in a given list (open or open|filtered)
portrule = shortport.port_or_service(80, "http")
-- True for any HTTP-like port
portrule = shortport.http
-- True for SSL/TLS ports
portrule = shortport.ssl
-- True for a port number
portrule = shortport.portnumber(22)
-- True for a named service
portrule = shortport.service("ftp")
stdnse — standard Nmap script utilities
local stdnse = require "stdnse"
-- Formatted output (shown indented under host in nmap output)
return stdnse.format_output(true, {"line one", "line two"})
-- Debug printing (shown with -d)
stdnse.debug(1, "Got response: %s", response)
-- Get a script argument (from --script-args)
local path = stdnse.get_script_args("myscript.path") or "/"
-- Sleep (seconds as float)
stdnse.sleep(0.5)
http — HTTP requests
local http = require "http"
local response = http.get(host, port, "/admin/")
-- response.status (integer, e.g. 200)
-- response.body (string)
-- response.header (table, lowercased keys)
-- response.rawheader (table)
local resp2 = http.post(host, port, "/login", nil, nil,
"username=admin&password=admin")
-- Pipeline multiple requests
local pipeline = http.pipeline_new()
http.pipeline_add("/", nil, pipeline)
http.pipeline_add("/robots.txt", nil, pipeline)
local results = http.pipeline_go(host, port, pipeline)
vulns — standardized vulnerability reporting
local vulns = require "vulns"
local vuln_db = vulns.DB:new()
local vuln = {
title = "CVE-2021-12345: Remote Code Execution in Foo Service",
state = vulns.STATE.VULN, -- or NOT_VULN, LIKELY_VULN, UNKNOWN
IDS = { CVE = "CVE-2021-12345" },
risk_factor = "High",
description = "An unauthenticated attacker can execute arbitrary code.",
dates = { disclosure = { year='2021', month='03', day='15' } },
references = { "https://nvd.nist.gov/vuln/detail/CVE-2021-12345" }
}
return vuln_db:add(vuln)
Writing a Simple Script from Scratch
A script that checks for an open Telnet banner and flags it as insecure:
description = [[
Detects open Telnet services and retrieves the login banner.
Telnet transmits credentials in plaintext — flag for remediation.
]]
categories = {"discovery", "safe"}
author = "Your Name"
license = "Same as Nmap -- See https://nmap.org/book/man-legal.html"
local nmap = require "nmap"
local shortport = require "shortport"
local stdnse = require "stdnse"
portrule = shortport.port_or_service(23, "telnet")
action = function(host, port)
local socket = nmap.new_socket()
socket:set_timeout(3000)
local status, err = socket:connect(host.ip, port.number)
if not status then
return nil
end
-- Read the banner (Telnet sends it immediately)
local ok, banner = socket:receive()
socket:close()
if not ok then
return "Open Telnet port (no banner received)"
end
-- Strip non-printable bytes from the banner
banner = banner:gsub("[%c]", ".")
return stdnse.format_output(true, {
"WARNING: Telnet transmits credentials in plaintext",
"Banner: " .. banner
})
end
Save to /usr/share/nmap/scripts/telnet-insecure.nse, run nmap --script-updatedb, then:
nmap --script=telnet-insecure -p 23 10.0.0.0/24
Wireshark Lua Scripting
Overview
Wireshark’s Lua API lets you extend the application without recompiling it. The primary uses in network security are:
- Custom dissectors — parse proprietary or undocumented protocols and display fields in the packet tree
- Tap listeners — collect statistics across a capture (e.g., count C2 beacon intervals, extract all DNS queries)
- Post-dissectors — run after all other dissectors and annotate packets with derived fields
Scripts are loaded from:
- The global plugins directory (
/usr/lib/x86_64-linux-gnu/wireshark/plugins/on Linux) - The user’s personal plugins directory (
~/.local/lib/wireshark/plugins/) - The command line:
wireshark -X lua_script:myscript.luaortshark -X lua_script:myscript.lua
Core Objects
| Object | Description |
|---|---|
Proto |
Defines a new protocol; has a name, description, and fields |
ProtoField |
A typed field that appears in the packet detail tree |
Tvb |
A “TV Buffer” — a reference to a packet’s raw bytes; supports slicing |
Pinfo |
Packet information: source/destination addresses, ports, timestamps |
TreeItem |
A node in the packet detail tree; used to add children and labels |
Dissector |
Reference to an existing dissector; used to hand off to sub-dissectors |
DissectorTable |
A table mapping protocol values to dissectors (e.g., udp.port) |
Listener |
A tap listener invoked for each packet matching a filter |
Writing a Protocol Dissector
A dissector lets Wireshark understand a custom protocol and display its fields. The example below parses a simple 4-byte header: 1 byte type, 1 byte flags, 2 bytes length.
-- myproto.lua — simple 4-byte header dissector
-- Load with: tshark -X lua_script:myproto.lua -r capture.pcap
-- 1. Create the protocol object
local myproto = Proto("myproto", "My Custom Protocol")
-- 2. Define fields (name, display label, type, optional base/values)
local f_type = ProtoField.uint8 ("myproto.type", "Message Type", base.HEX,
{ [0x01]="HELLO", [0x02]="DATA", [0x03]="BYE" })
local f_flags = ProtoField.uint8 ("myproto.flags", "Flags", base.HEX)
local f_len = ProtoField.uint16("myproto.length","Payload Length", base.DEC)
local f_data = ProtoField.bytes ("myproto.data", "Payload")
myproto.fields = { f_type, f_flags, f_len, f_data }
-- 3. Implement the dissector function
function myproto.dissector(buf, pkt, tree)
-- Minimum packet length check
if buf:len() < 4 then return end
-- Set the Info column
pkt.cols.protocol = "MYPROTO"
-- Add protocol subtree to the packet detail pane
local subtree = tree:add(myproto, buf(0, 4))
-- Add each field: buf(offset, length) slices the buffer
subtree:add(f_type, buf(0, 1))
subtree:add(f_flags, buf(1, 1))
subtree:add(f_len, buf(2, 2))
local payload_len = buf(2, 2):uint()
if buf:len() >= 4 + payload_len and payload_len > 0 then
subtree:add(f_data, buf(4, payload_len))
end
end
-- 4. Register the dissector for UDP port 9999
local udp_table = DissectorTable.get("udp.port")
udp_table:add(9999, myproto)
Run it against a capture:
tshark -X lua_script:myproto.lua -r capture.pcap -Y myproto
Or in Wireshark, reload plugins via Tools → Lua → Reload Lua Plugins, or place the script in the plugins directory and restart Wireshark.
ProtoField Types
| Method | C equivalent | Common options |
|---|---|---|
ProtoField.uint8(name, label, base) |
uint8_t |
base.DEC, base.HEX, base.OCT |
ProtoField.uint16(name, label, base) |
uint16_t |
— |
ProtoField.uint32(name, label, base) |
uint32_t |
— |
ProtoField.int8/16/32(...) |
signed int | — |
ProtoField.bool(name, label, size) |
boolean | size in bits |
ProtoField.ipv4(name, label) |
IPv4 address | — |
ProtoField.ipv6(name, label) |
IPv6 address | — |
ProtoField.ether(name, label) |
MAC address | — |
ProtoField.string(name, label) |
C string | — |
ProtoField.bytes(name, label) |
byte array | — |
Tap Listeners
A Listener (tap) is invoked for every packet matching a display filter. It is useful for extracting data across a whole capture — generating reports, counting events, or building tables of activity.
-- dns_tap.lua — print all DNS query names to stdout
-- Usage: tshark -X lua_script:dns_tap.lua -r capture.pcap -q
local tap = Listener.new("dns", "dns.flags.response == 0")
-- Called once per matching packet
function tap.packet(pinfo, tvb, tapdata)
local qname = tostring(pinfo.private["dns.qry.name"] or "")
if qname ~= "" then
print(tostring(pinfo.number) .. "\t" .. qname)
end
end
-- Called after all packets are processed
function tap.draw()
print("--- DNS query extraction complete ---")
end
function tap.reset()
-- Called when the capture is restarted; reset any counters here
end
A more useful pattern collects data and prints a summary:
-- beacon_counter.lua — count packets per destination IP
-- Useful for spotting beaconing malware (regular-interval connections)
-- Usage: tshark -X lua_script:beacon_counter.lua -r capture.pcap -q
local counts = {}
local tap = Listener.new("ip")
function tap.packet(pinfo, tvb)
local dst = tostring(pinfo.dst)
counts[dst] = (counts[dst] or 0) + 1
end
function tap.draw()
print("\nDestination IP packet counts:")
print(string.format("%-20s %s", "IP Address", "Packets"))
print(string.rep("-", 35))
-- Collect and sort
local sorted = {}
for ip, count in pairs(counts) do
table.insert(sorted, {ip=ip, count=count})
end
table.sort(sorted, function(a,b) return a.count > b.count end)
for _, entry in ipairs(sorted) do
print(string.format("%-20s %d", entry.ip, entry.count))
end
end
Run it in tshark with -q to suppress per-packet output:
tshark -X lua_script:beacon_counter.lua -r capture.pcap -q
Suricata Lua Detection Scripts
Overview
Suricata’s signature language is powerful but has limits: it cannot express logic like “alert if the same host makes more than 10 requests in 60 seconds” or “alert only if the User-Agent matches one of 200 known-bad strings.” Lua scripts fill this gap.
The lua keyword in a Suricata rule invokes a Lua script to decide whether the rule matches. The rest of the rule (source/destination IPs, ports, protocol, other keywords) still applies as normal — the Lua script is an additional condition.
The lua Keyword
alert http any any -> any any (
msg:"Suspicious PHP POST via HTTP/1.0";
flow:established,to_server;
lua:detect_php_post.lua;
classtype:web-application-attack;
sid:9000001; rev:1;
)
The script file is resolved relative to the Suricata rules directory. If Lua compilation is disabled at build time, the lua keyword silently fails to match — check suricata --build-info | grep lua.
Script Structure
Every Lua detection script must define exactly two functions: init and match.
-- detect_php_post.lua
-- Matches HTTP/1.0 POST requests to .php files
-- init: declare which buffer(s) to inspect
function init(args)
local needs = {}
needs["http.request_line"] = tostring(true)
return needs
end
-- match: inspect the buffer and return 1 (match) or 0 (no match)
function match(args)
local request_line = tostring(args["http.request_line"])
-- Lua pattern: matches "POST /anything.php HTTP/1.0"
if request_line:find("^POST%s+.+%.php%s+HTTP/1%.0$") then
return 1
end
return 0
end
The init function registers buffers by name in the returned table. Only one HTTP buffer can be registered per script. The match function receives those buffers as string values in args, inspects them, and returns 1 to signal a match.
Available Inspection Buffers
| Buffer | Contents |
|---|---|
packet |
Full packet including headers (non-stream) |
payload |
Packet payload only (non-stream) |
http.uri |
Normalized HTTP URI |
http.uri.raw |
Raw (un-normalized) HTTP URI |
http.request_line |
Full HTTP request line (METHOD URI VERSION) |
http.request_headers |
HTTP request headers, normalized |
http.request_headers.raw |
HTTP request headers, raw |
http.request_body |
HTTP request body (POST data, etc.) |
http.request_cookie |
Cookie header value |
http.request_user_agent |
User-Agent header value |
http.response_headers |
HTTP response headers |
http.response_body |
HTTP response body |
http.response_cookie |
Set-Cookie value |
dns.query |
DNS query name |
tls.sni |
TLS SNI (Server Name Indication) |
Multi-Pattern Matching Example
Lua detection becomes most powerful when matching against a large set of patterns that would be impractical as multiple Suricata signatures:
-- detect_malicious_ua.lua
-- Alert when User-Agent matches any known-malicious string
local bad_agents = {
"python%-requests", -- common in automated attacks
"zgrab", -- internet scanner
"masscan", -- mass scanner
"libwww%-perl", -- old automated client often abused
"Go%-http%-client/1%.1",-- common in Go-based scanners
}
function init(args)
local needs = {}
needs["http.request_user_agent"] = tostring(true)
return needs
end
function match(args)
local ua = tostring(args["http.request_user_agent"])
if #ua == 0 then return 0 end
ua = ua:lower()
for _, pattern in ipairs(bad_agents) do
if ua:find(pattern) then
return 1
end
end
return 0
end
The corresponding rule:
alert http any any -> $HTTP_SERVERS any (
msg:"Known malicious scanner User-Agent";
flow:established,to_server;
lua:detect_malicious_ua.lua;
classtype:web-application-attack;
sid:9000002; rev:1;
)
DNS Exfiltration Detection Example
Detecting DNS-based data exfiltration using Lua — high-entropy subdomain labels are a hallmark of DNS tunneling:
-- detect_dns_tunnel.lua
-- Flag DNS queries with high-entropy subdomains (likely base32/base64 encoded data)
-- Shannon entropy calculation
local function entropy(s)
local freq = {}
for i = 1, #s do
local c = s:sub(i,i)
freq[c] = (freq[c] or 0) + 1
end
local h = 0
for _, v in pairs(freq) do
local p = v / #s
h = h - p * math.log(p, 2)
end
return h
end
function init(args)
local needs = {}
needs["dns.query"] = tostring(true)
return needs
end
function match(args)
local query = tostring(args["dns.query"])
if #query == 0 then return 0 end
-- Extract the leftmost label (before first dot)
local label = query:match("^([^%.]+)")
if not label or #label < 12 then return 0 end
-- Entropy above ~3.8 bits/character is suspicious for a hostname label
if entropy(label) > 3.8 then
return 1
end
return 0
end
Deploying Lua Scripts in Suricata
- Place the
.luafile in your rules directory (e.g.,/etc/suricata/rules/). - Reference it in the rule with
lua:filename.lua;. - Add the rule to your
.rulesfile. - Reload rules without restarting:
suricatasc -c reload-rulesorkill -USR2 $(pidof suricata). - Test in offline mode first:
suricata -r capture.pcap -S /etc/suricata/rules/custom.rules -l /tmp/suricata-test/
Comparison and When to Use Each
| Nmap NSE | Wireshark Lua | Suricata Lua | |
|---|---|---|---|
| Primary purpose | Active scanning and enumeration | Packet analysis and visualization | Inline detection and alerting |
| Runs on | Live scan targets | Captured traffic (pcap) or live capture | Live traffic (inline or IDS) |
| Script trigger | Port/host rule; before/after scan | Every packet, or tap filter | Matching flow + other rule conditions |
| Network access | Full (sockets, raw packets) | Read-only (packet bytes via Tvb) | Read-only (buffer strings from dissectors) |
| Typical use | Vuln checks, banner grabs, brute force | Custom protocol display, statistics | Complex pattern matching, entropy checks |
| Script location | /usr/share/nmap/scripts/ |
Plugins dir or -X lua_script: |
Rules dir, referenced in rule |
| Key API | nmap, http, shortport, vulns |
Proto, ProtoField, Listener, Tvb |
init(args) / match(args) |
Things to Try Yourself
-
NSE: Write a script that connects to a service on port 80 and checks whether the
Server:response header reveals a version number. Output a warning if it does. -
NSE: Write a
postrulescript (runs after all hosts are scanned) that prints a summary table of all hosts where port 22 was open and the service banner contained “OpenSSH”. -
Wireshark: Write a tap listener that extracts every unique TLS SNI from a capture file and prints a sorted list. Compare the output to
tshark -T fields -e tls.handshake.extensions_server_name. -
Wireshark: Write a dissector for a simple 8-byte header protocol you define yourself (e.g., 2-byte magic, 2-byte type, 4-byte sequence number). Generate test traffic with
scapyand confirm your dissector parses it correctly. -
Suricata: Write a Lua detection script that flags HTTP requests where the URI contains a base64-encoded string (look for sequences of
[A-Za-z0-9+/=]longer than 20 characters in the path). Test it against a pcap containing benign and suspicious traffic.