Network Debugging: Ports to Packet Traces
By Xavier Collantes
9/24/2025
In July 2024, 8.5 million systems crashed due to a faulty update to widely used
software. Since the internet is a collection of machines, even a small set of
machines failing can bring down the entire internet.
The internet was brought to its knees when CrowdStrike distributed a faulty
update to its Falcon Sensor security software that caused widespread problems
with Microsoft Windows computers, resulting in roughly 8.5 million systems
crashing in what has been called the largest outage in the history of
information technology.
- Financial damage estimated at least $10 billion worldwide from 2024 CrowdStrike-related IT outages
- Manual fixes required on millions of machines, causing extended downtime
Network-level debugging is the art of peering into the digital highways where
your data travels. It is like being a detective with X-ray vision, able to see
exactly what happens when bits and bytes move between servers, containers,
and clients.
Why Network-Level Debugging Matters
In the case of the CrowdStrike outage, the issue was with other machines. But at
its heart, that is what the network is—it is a collection of machines.
Think of your application like a restaurant. You can have the best chefs,
perfect recipes, and efficient kitchen staff, but if the supply trucks cannot
deliver ingredients on time due to traffic jams, road closures, or wrong
addresses, your restaurant fails.
Essential Network Debugging Tools
tcpdump: The Network Microscope
tcpdump is your Swiss Army knife for packet capture. It shows you exactly
what is crossing the wire.Bash
1# Capture HTTP traffic on port 80
2sudo tcpdump -i any -n port 80
3
4# Capture traffic to/from specific host
5sudo tcpdump -i any host api.example.com
6
7# Capture and save to file for analysis
8sudo tcpdump -i eth0 -s 65535 -w capture.pcap
9
10# Monitor DNS queries
11sudo tcpdump -i any port 53
12snippet hosted withby Xavier
Security-Focused Traffic Analysis
Monitor for suspicious patterns and potential security threats:
Bash
1# Detect port scanning attempts (many connections to different ports)
2sudo tcpdump -i any -n 'tcp[tcpflags] & (tcp-syn) != 0 and tcp[tcpflags] & (tcp-ack) = 0'
3
4# Monitor for unusual outbound connections (potential data exfiltration)
5sudo tcpdump -i any -n 'src host YOUR_SERVER_IP and not dst net 192.168.0.0/16 and not dst net 10.0.0.0/8'
6
7# Detect potential DDoS or brute force attacks (high connection volume)
8sudo tcpdump -i any -n 'tcp[tcpflags] & (tcp-syn) != 0' | head -100
9
10# Monitor for suspicious DNS queries (potential DNS tunneling)
11sudo tcpdump -i any -n 'port 53 and greater 512'
12
13# Capture failed connection attempts (potential reconnaissance)
14sudo tcpdump -i any -n 'tcp[tcpflags] & (tcp-rst) != 0'
15
16# Monitor for large data transfers (potential exfiltration)
17sudo tcpdump -i any -n 'greater 1500 and not port 22'
18
19# Detect connections to uncommon ports
20sudo tcpdump -i any -n 'not port 22 and not port 80 and not port 443 and not port 53'
21snippet hosted withby Xavier
Red flags to watch for:
- Large DNS queries: Possible DNS tunneling
- Connections to unusual high ports: Backdoor or malware communication
Wireshark: A Hacker's Tool For Good
While tcpdump captures packets, Wireshark makes sense of them with a powerful
GUI and protocol analysis.
netstat: Connection Archaeology
Bash
1# Show all listening ports
2ss -tuln
3
4# Show processes using network connections
5ss -tulnp
6snippet hosted withby Xavier
ping and traceroute: Basic Network Path Analysis
Bash
1# Basic connectivity test
2ping -c 4 google.com
3
4# Trace the network path
5traceroute google.com
6snippet hosted withby Xavier
Advanced Debugging Scenarios
The Mysterious Timeout
Problem: Your API calls randomly time out after exactly 30 seconds.
Investigation:
Bash
1# Monitor TCP connections during the issue
2ss -i | grep :8080
3
4# Look for connection state changes
5tcpdump -i any host api-server and port 8080
6snippet hosted withby Xavier
Common findings:
- Connection established but no data flowing (application hanging)
Network Metrics That Matter
Monitor these key indicators:
- Connection establishment time: How long TCP handshakes take
- Retransmission rate: Percentage of packets being resent
- Connection drops: Connections reset or timed out
- DNS resolution time: Time to resolve hostnames
- Bandwidth utilization: Network capacity usage
When capturing packets in production, ensure you comply with privacy regulations. Consider capturing only packet headers or using sampling techniques to reduce data volume.
Container Network Debugging
Bash
1# Debug container networking
2docker exec -it container_name ss -tuln
3docker exec -it container_name tcpdump -i eth0
4
5# Check container network namespaces
6docker exec -it container_name ip addr show
7
8# Monitor inter-container communication
9tcpdump -i docker0
10snippet hosted withby Xavier
Protocol-Specific Debugging
HTTP/HTTPS Issues
Bash
1# Monitor HTTP headers and responses on port 80
2tcpdump port 80
3
4# SSL/TLS handshake analysis
5openssl s_client -connect xaviercollantes.dev:443 -debug
6
7# Test HTTP/2 connections
8curl -v --http2 https://xaviercollantes.dev
9snippet hosted withby Xavier
Building a Network Debugging Toolkit
Create a debugging script that automates common tasks:
Bash
1#!/bin/bash
2
3# Xavier script for network debugging.
4
5HOST=${1:-"xaviercollantes.dev"} # host can be changed to a target site like `xaviercollantes.dev`
6PORT=${2:-"80"}
7
8echo "=== target site ==="
9ping -c 3 $HOST
10echo "Date: $(date)"
11
12echo -e "\n=== DNS info ==="
13# For DNS of a target site. Shows if site has propagated to DNS servers like
14# Cloudflare, etc.
15dig $HOST
16
17echo -e "\n=== port ==="
18nmap -p $PORT $HOST
19
20echo -e "\n=== Route Tracing ==="
21traceroute $HOST
22
23echo -e "\n=== Active Connections ==="
24ss -tuln | grep :$PORT
25
26echo -e "\n=== Network stats ==="
27ss -s
28snippet hosted withby Xavier
Example output:
Bash
1=== Network Debug Report for localhost:80 ===
2Date: Wed Sep 24 16:15:56 PDT 2025
3
4=== basics ===
5PING localhost (127.0.0.1): 56 data bytes
664 bytes from 127.0.0.1: icmp_seq=0 ttl=64 time=0.041 ms
764 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.063 ms
864 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.085 ms
9
10--- localhost ping statistics ---
113 packets transmitted, 3 packets received, 0.0% packet loss
12round-trip min/avg/max/stddev = 0.041/0.063/0.085/0.018 ms
13-e
14=== DNS info ===
15
16; <<>> DiG 9.10.6 <<>> localhost
17;; global options: +cmd
18;; Got answer:
19;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: XXXX
20;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
21
22;; OPT PSEUDOSECTION:
23; EDNS: version: 0, flags:; udp: XXXX
24;; QUESTION SECTION:
25;localhost. IN A
26
27;; AUTHORITY SECTION:
28. XXXX IN SOA a.root-servers.net. nstld.verisign-grs.com. XXXXXXX
29
30;; Query time: 17 msec
31;; SERVER: 1.1.1.1#53(1.1.1.1)
32;; WHEN: Wed Sep 24 16:15:58 PDT 2025
33;; MSG SIZE rcvd: XXX
34
35-e
36=== port ===
37Starting Nmap 7.95 ( https://nmap.org ) at 2025-09-24 16:15 PDT
38Nmap scan report for localhost (127.0.0.1)
39Host is up (0.00012s latency).
40Other addresses for localhost (not scanned): ::1
41
42PORT STATE SERVICE
4380/tcp closed http
44
45Nmap done: 1 IP address (1 host up) scanned in 0.02 seconds
46-e
47=== Route Tracing ===
48traceroute to localhost (127.0.0.1), 64 hops max, 40 byte packets
49 1 localhost (127.0.0.1) 0.192 ms 0.061 ms 0.049 ms
50-e
51=== Active Connections ===
52temp.sh: line 25: ss: command not found
53-e
54=== Network stats ===
55temp.sh: line 28: ss: command not found
56snippet hosted withby Xavier
Common Network Anti-Patterns (Avoid These)
The Connection Pool Disaster
Problem: At the application development layer, creating new database
connections for every request instead of reusing them.
Network signature: High volume of TCP SYN packets to database servers,
short-lived connections.
The Chatty Application
Problem: Making hundreds of individual API calls instead of batching
requests.
Network signature: High packet count but low data volume, excessive
connection establishment.
The DNS Lookup Antipattern
Problem: Resolving the same hostnames repeatedly instead of caching results.
Network signature: Constant DNS queries for the same domains.
Putting It All Together
Network debugging is both an art and a science. The key is building intuition
about how network protocols behave and having the right tools ready when issues
arise.
Remember: networks are eventually consistent, packets can arrive out of order,
connections can be reset at any time, and DNS can lie. Embrace this chaos, and
you will become significantly better at diagnosing and fixing the networking
issues that inevitably plague production systems.
Related Article
Network debugging pairs perfectly with async Python patterns for building resilient, high-performance applications that gracefully handle network instability.
The next time your application mysteriously slows down or fails, remember: the
network knows exactly what happened. You just need to know how to ask the right
questions.
Further Reading
Related Articles
Related by topics:
devops
infrastructure
cloud