Network Debugging Tools: mitmproxy, Charles, Fiddler, and Wireshark

Network Debugging Tools: mitmproxy, Charles, Fiddler, and Wireshark

Every developer eventually hits a bug where the problem is "somewhere in the network." The API returns unexpected data, but only in production. The mobile app works on WiFi but fails on cellular. The webhook fires but the receiver claims it never arrived. CORS errors block requests that work fine in curl.

Browser DevTools handle simple cases, but they cannot inspect traffic from mobile apps, backend services, or anything outside the browser. For those scenarios, you need a dedicated network debugging tool.

This guide covers the four tools worth knowing, with practical debugging workflows for common development scenarios.

The Tools at a Glance

mitmproxy: The Developer's Proxy

mitmproxy is an open source interactive HTTPS proxy. It intercepts HTTP/HTTPS traffic, displays it in a terminal UI or web interface, and lets you modify requests and responses on the fly. It is scriptable in Python, which makes it extraordinarily powerful for automated debugging and testing.

Installation and Basic Usage

# Install
brew install mitmproxy       # macOS
pip install mitmproxy         # Any platform
sudo apt install mitmproxy    # Debian/Ubuntu

# Start the proxy (terminal UI)
mitmproxy

# Start the web interface
mitmweb

# Start in dump mode (stdout, good for scripting)
mitmdump

By default, mitmproxy listens on port 8080. Configure your system or application to use it as an HTTP proxy:

# Set proxy for curl
curl --proxy http://localhost:8080 https://api.example.com/users

# Set proxy via environment variables (works with most HTTP clients)
export HTTP_PROXY=http://localhost:8080
export HTTPS_PROXY=http://localhost:8080

# Python requests
import requests
proxies = {"http": "http://localhost:8080", "https": "http://localhost:8080"}
requests.get("https://api.example.com/users", proxies=proxies)

HTTPS Interception

To inspect HTTPS traffic, you need to install mitmproxy's CA certificate on the client device:

# Start mitmproxy
mitmproxy

# Navigate to http://mitm.it in the proxied browser/device
# Download and install the certificate for your platform

On macOS, add the certificate to Keychain and trust it. On iOS/Android, install the profile. On Linux, add it to the system certificate store or your application's trust store.

Scripting: The Killer Feature

mitmproxy's Python scripting API is what sets it apart. You can write scripts that modify, filter, or log traffic programmatically.

# log_api_calls.py -- Log all API calls with timing
from mitmproxy import http
import time
import json

request_times = {}

def request(flow: http.HTTPFlow):
    request_times[flow.id] = time.time()

def response(flow: http.HTTPFlow):
    duration = time.time() - request_times.get(flow.id, time.time())

    if "/api/" in flow.request.pretty_url:
        status = flow.response.status_code
        method = flow.request.method
        url = flow.request.pretty_url
        size = len(flow.response.content)

        print(f"{method} {url} -> {status} ({size} bytes, {duration:.2f}s)")

        # Log slow responses
        if duration > 2.0:
            print(f"  WARNING: Slow response ({duration:.2f}s)")

        # Log error responses with body
        if status >= 400:
            try:
                body = json.loads(flow.response.content)
                print(f"  Error body: {json.dumps(body, indent=2)}")
            except (json.JSONDecodeError, UnicodeDecodeError):
                pass

# Run with script
mitmproxy -s log_api_calls.py

More useful scripts:

# mock_endpoint.py -- Return a mock response for a specific endpoint
from mitmproxy import http
import json

def request(flow: http.HTTPFlow):
    if flow.request.path == "/api/feature-flags":
        flow.response = http.Response.make(
            200,
            json.dumps({"dark_mode": True, "new_checkout": True}),
            {"Content-Type": "application/json"}
        )

# add_auth.py -- Inject auth headers into all requests
from mitmproxy import http

AUTH_TOKEN = "your-dev-token-here"

def request(flow: http.HTTPFlow):
    if "api.example.com" in flow.request.host:
        flow.request.headers["Authorization"] = f"Bearer {AUTH_TOKEN}"

# Chain multiple scripts
mitmproxy -s add_auth.py -s log_api_calls.py

Strengths: Free and open source, scriptable in Python (enormously flexible), terminal UI and web UI options, supports HTTP/2 and WebSockets, excellent for automation and CI integration.

Weaknesses: HTTPS certificate installation is a manual step per device, terminal UI has a learning curve, no native GUI on par with Charles for visual inspection.

Charles Proxy: The Visual HTTP Debugger

Charles Proxy is a commercial HTTP debugging proxy with a polished GUI. It is particularly popular among mobile developers because it makes inspecting traffic from iOS and Android devices straightforward.

Key Features

Structure view: Charles organizes requests by host in a tree structure. This makes it easy to focus on traffic to a specific API while ignoring noise from analytics, CDNs, and third-party scripts.

Breakpoints: Set breakpoints on specific URLs to pause and edit requests before they are sent or responses before they are received. This is invaluable for testing how your app handles unexpected API responses:

1. Right-click a request -> Breakpoints
2. When the request fires, Charles pauses it
3. Edit the request body, headers, or URL
4. Or edit the response status code, headers, or body
5. Click Execute to send the modified version

Map Remote: Redirect requests from one URL to another without changing your application code. Useful for testing against a staging API while running the production app:

• Map api.production.com/v2/* to api.staging.com/v2/*
• Map a specific failing endpoint to a local mock server

Map Local: Return a local file instead of making a network request. Save a problematic API response to a file, modify it, and serve the modified version to test your app's behavior.

Repeat and Repeat Advanced: Replay a captured request, optionally with modifications. Repeat Advanced lets you send multiple concurrent requests to test rate limiting or race conditions.

Throttling: Simulate slow or unreliable network conditions. Presets for 3G, Edge, and custom bandwidth/latency settings. Essential for testing loading states, timeouts, and offline behavior.

Mobile Device Setup

Charles is especially useful for mobile debugging:

1. Start Charles on your computer
2. On the mobile device, set the WiFi proxy to your computer's IP on port 8888
3. Navigate to chls.pro/ssl on the device to install the Charles CA certificate
4. On iOS: Settings -> General -> About -> Certificate Trust Settings -> Enable the Charles certificate
5. On Android: Settings -> Security -> Install certificates

Strengths: Polished GUI, excellent for mobile development, breakpoints for request/response editing, Map Remote/Local for traffic redirection, built-in throttling, visual traffic organization.

Weaknesses: $50 license (reasonable, but mitmproxy is free), no scripting API (manual-only workflows), Java-based (resource usage can be high), less suitable for automated or CI workflows.

Fiddler: The Windows Developer's Choice

Fiddler has a long history in the Windows and .NET ecosystem. Fiddler Classic (Windows-only, free) is the original tool. Fiddler Everywhere (cross-platform, subscription) is the modern rewrite with collaboration features.

Fiddler Classic (Windows)

Fiddler Classic automatically configures itself as the system proxy on Windows, so it captures traffic from all applications without manual proxy configuration. This is both its strength (zero setup) and its risk (it captures everything, including sensitive traffic).

Key features:

• AutoResponder: Map URL patterns to local files or custom responses (similar to Charles Map Local)
• FiddlerScript: Write scripts in JScript.NET to modify requests and responses
• Composer: Build and send HTTP requests manually (like a basic Postman)
• Filters: Show/hide traffic based on host, process, status code, etc.

Fiddler Everywhere (Cross-Platform)

Fiddler Everywhere is a paid product ($12/month or $10/month annual) with a modern UI and team collaboration features like shared sessions and team rules.

Best for: Windows developers who want zero-configuration system-wide traffic capture, .NET developers debugging IIS/ASP.NET applications, teams that want collaborative debugging features.

Weaknesses: Fiddler Classic is Windows-only, Fiddler Everywhere requires a subscription, the product direction has split the user base between Classic and Everywhere.

Wireshark: When HTTP Is Not Enough

Wireshark is a packet analyzer, not an HTTP proxy. It captures all network traffic at the packet level -- TCP, UDP, DNS, TLS, HTTP, WebSocket, and hundreds of other protocols. It is far more powerful and far more complex than an HTTP proxy.

When You Need Wireshark

You probably do not need Wireshark for day-to-day API debugging. Use it when:

• DNS resolution is failing or returning unexpected results
• TLS handshakes are failing (certificate errors, cipher mismatches)
• TCP connections are timing out or resetting
• You need to debug non-HTTP protocols (MQTT, gRPC at the wire level, custom TCP protocols)
• You are diagnosing network-level issues (packet loss, retransmissions, MTU problems)

Practical Filters

Wireshark captures everything on the network interface. Without filters, it is overwhelming. Learn these display filters:

# HTTP traffic only
http

# Traffic to/from a specific host
ip.addr == 10.0.0.1

# DNS queries
dns

# HTTP requests to a specific domain
http.host contains "api.example.com"

# Failed TCP connections (RST packets)
tcp.flags.reset == 1

# TLS handshake failures
tls.alert_message

# Slow TCP retransmissions
tcp.analysis.retransmission

# HTTP response codes >= 400
http.response.code >= 400

# WebSocket traffic
websocket

# Combine filters
http.host contains "api.example.com" && http.response.code >= 400

Debugging TLS Issues

When HTTPS connections fail before any HTTP traffic appears, Wireshark shows you the TLS handshake:

# Show TLS handshake details
tls.handshake

# Show certificate information
tls.handshake.type == 11

# Show cipher suite negotiation
tls.handshake.type == 1 || tls.handshake.type == 2

Common findings:

• tls.alert_message.desc == 40: Handshake failure (often a cipher mismatch)
• tls.alert_message.desc == 42: Bad certificate (expired, wrong hostname, untrusted CA)
• tls.alert_message.desc == 48: Unknown CA (missing intermediate certificate)

Command-Line Alternative: tshark

For scripted or headless environments, tshark (Wireshark's CLI) captures and filters traffic:

# Capture HTTP traffic to a specific host
tshark -i eth0 -f "host api.example.com" -Y "http"

# Capture DNS queries
tshark -i eth0 -f "port 53" -Y "dns.qry.name"

# Save capture for later analysis
tshark -i eth0 -f "host 10.0.0.1" -w capture.pcap

# Read and filter a saved capture
tshark -r capture.pcap -Y "http.response.code >= 400"

Strengths: Sees everything (every packet, every protocol), essential for non-HTTP debugging, powerful display and capture filters, free and open source, tshark for scripting.

Weaknesses: Steep learning curve, captures far more than you need for HTTP debugging, cannot decrypt HTTPS without the server's private key or an SSLKEYLOGFILE, overwhelming without filters.

Debugging Workflows

Workflow 1: "The API Returns Wrong Data"

The API works in curl but returns unexpected data in your application.

1. Start mitmproxy or Charles
2. Configure your app to use the proxy
3. Reproduce the issue
4. Compare the captured request with your working curl command
5. Common culprits: missing headers (Accept, Content-Type, Authorization), wrong HTTP method, URL encoding differences, different request body format

Workflow 2: "The Mobile App Cannot Reach the API"

1. Start Charles on your computer
2. Configure the mobile device's WiFi proxy to point at Charles
3. Install the Charles CA certificate on the device
4. Reproduce the issue
5. Check: Is the request leaving the device? Is DNS resolving correctly? Is the server responding? What is the response status and body?

Workflow 3: "Intermittent Timeout Errors"

1. Start mitmproxy with a logging script that records response times
2. Run your application under load
3. Look for patterns: Do timeouts correlate with specific endpoints? Specific times? Specific response sizes?
4. If the proxy shows the server responding slowly, the problem is server-side
5. If the proxy shows no response at all, use Wireshark to check for TCP retransmissions or connection resets

Workflow 4: "CORS Errors in the Browser"

1. Open browser DevTools Network tab (this is sufficient for most CORS issues)
2. Look at the preflight OPTIONS request -- is it returning the right Access-Control-Allow-Origin header?
3. If you need more detail, use mitmproxy to intercept and modify CORS headers:

# fix_cors.py -- Add permissive CORS headers during development
from mitmproxy import http

def response(flow: http.HTTPFlow):
    if "api.example.com" in flow.request.host:
        flow.response.headers["Access-Control-Allow-Origin"] = "*"
        flow.response.headers["Access-Control-Allow-Methods"] = "GET, POST, PUT, DELETE, OPTIONS"
        flow.response.headers["Access-Control-Allow-Headers"] = "Content-Type, Authorization"

Workflow 5: "Works Locally, Fails in Production"

1. Use mitmproxy's record feature to capture traffic from the working local environment:

mitmdump -w local-traffic.flow

2. Replay the recorded traffic against the production API:

mitmdump -r local-traffic.flow --set upstream_cert=false --mode upstream:https://api.production.com

3. Compare responses between local and production to find the divergence.

Recommendations

For most API development: Use mitmproxy. It is free, scriptable, and handles 90% of debugging scenarios. The Python scripting API is genuinely transformative -- once you write your first custom logging script, you will wonder how you debugged without it.

For mobile development: Use Charles Proxy. The $50 is well spent. Its mobile device setup workflow, breakpoints, and Map Remote/Local features are tailored for the "debug traffic from a physical device" use case.

For Windows/.NET development: Use Fiddler Classic. Its system-wide proxy auto-configuration and AutoResponder rules work well in the Windows ecosystem.

For non-HTTP issues: Use Wireshark. When DNS, TLS, TCP, or other protocols are involved, no HTTP proxy can help. Learn the basic display filters and keep Wireshark installed for when you need it. You will not need it often, but when you do, nothing else will do.

General advice: Start with the simplest tool that can answer your question. Browser DevTools for browser HTTP issues, mitmproxy for application HTTP issues, Wireshark only when you have exhausted HTTP-level debugging. Going straight to Wireshark for an HTTP problem is like using a microscope to read a book -- technically possible, but unnecessarily difficult.