Mitmproxy in UI autotests: traffic management for QA

Integrating mitmproxy into UI Autotests: Traffic Control for QA Engineers

Proxying network traffic is a game-changer for boosting UI autotests efficiency. Unlike manual testing with tools like Charles Proxy or Fiddler, automation demands programmable solutions. Mitmproxy, with its Python API and headless mode, lets you embed a controllable network layer right into your test infrastructure.

Why Proxies Matter in Automated Testing

UI autotests often suffer from "network blindness"—they can't spot issues with APIs, WebSockets, or backend services. This leads to flaky tests, false failures, and tough debugging. Adding a proxy gives your tests full network visibility. Key use cases:

• Intercepting and analyzing HTTP requests/responses to verify app behavior.
• Tweaking status codes (e.g., 200 → 404) to test client-side error handling.
• Swapping data in JSON/XML responses to simulate test scenarios without dev environments.
• Logging traffic and tracking request sequences.
• Handling WebSocket messages, including live injection and modification.

Why mitmproxy is the Top Choice

Tools like Charles, Proxyman, and Fiddler fall short for automation due to no external control APIs. Open-source mitmproxy shines with:

• Python API for scripting traffic interception and modification.
• Headless mode for CI/CD pipelines without a GUI.
• WebSocket support, essential for modern web and mobile apps.
• Cross-platform (Windows, Linux, macOS).

Mitmproxy runs in three modes: mitmproxy (interactive CLI), mitmweb (web UI), and mitmdump (non-interactive dump). For autotests, use mitmdump or mitmweb with custom scripts.

Integration Architecture with Test Frameworks

Run mitmproxy as a standalone process, independent of your test framework. They communicate via a config file (e.g., config.json) and a Python module (proxy_handler.py). Basic workflow:

• Install mitmproxy (e.g., brew install mitmproxy).
• Set up certificates on test devices for HTTPS interception.
• Launch with mitmweb -s proxy_handler.py.
• Dynamically control proxy behavior by updating config.json from tests.

Status Code Mocking: Hands-On Example

Let's mock HTTP status codes for specific APIs. Define in config.json:

{
    "status": {"api/v1/user": 404, "api/v2/settings": 500}
}

Use a number for one-time mocks, or a list (e.g., [404]) for ongoing. The proxy_handler.py module handles it:

import re
import mitmproxy.ctx as ctx
from mitmproxy import http
from file_worker import FileWorker

file_worker = FileWorker()

def response(flow: http.HTTPFlow) -> None:
    url = flow.request.url
    if not flow.response.content:
        return
    cfg = file_worker.get_proxy_params()
    cfg_status = cfg.get("status")
    for api, sc in list(cfg_status.items()):
        if bool(re.compile(api).search(url)):
            flow.response.status_code = int(sc[0] if isinstance(sc, list) else sc)
            ctx.log.info(f"Status code was mocked '{api}' -> {sc}")
            if not isinstance(sc, list):
                del cfg_status[api]
                file_worker.set_proxy_param("status", cfg_status)
                cfg["status"] = cfg_status
            break

Key implementation notes:

• Regex (re module) for flexible URI matching.
• Support for one-time and persistent mocks.
• Logging for easy debugging.
• File ops in a dedicated FileWorker class for clean code.

Expanding Capabilities: Data Mocking and Logging

Similarly, mock response bodies. Add a "mock" section in config.json:

{
    "mock": {"api/v1/data": {"field": "new_value"}}
}

In proxy_handler.py:

cfg_mock: dict = cfg.get("mock")
if cfg_mock:
    for mock_api, params in list(cfg_mock.items()):
        if bool(re.compile(mock_api).search(url)):
            data = flow.response.content.decode()
            modified = file_worker.mock(params[0] if isinstance(params, list) else params, data)
            if modified:
                flow.response.content = modified.encode()
                ctx.log.info(f"Param {params} were mocked for '{mock_api}'")
            if not isinstance(params, list):
                del cfg_mock[mock_api]
                file_worker.set_proxy_param("mock", cfg_mock)
                cfg["mock"] = cfg_mock
            break

For saving server responses to files (great for analysis), use a "get_response" section in config.json. Perfect for:

• Validating data structures in automated checks.
• Building golden responses for regression tests.
• Debugging tricky backend interactions.

Key Takeaways

• Network Visibility: Proxies fix autotests' blind spots to network issues, boosting diagnostics and reliability.
• mitmproxy Flexibility: Python API and WebSocket support enable complex traffic scenarios without GUI dependencies.
• Config-Driven Control: JSON configs make dynamic proxy tweaks seamless for test frameworks.
• Performance: Optimize handlers to avoid slowing down proxies or tests.
• Extensibility: Modular design (proxy_handler.py, FileWorker) makes adding features like throttling or WebSocket analysis a breeze.

Conclusion

Integrating mitmproxy into UI autotests turns proxies from manual tools into automated powerhouses. It supercharges client-side testing, especially for backend simulations and deep network monitoring. Balance config flexibility with handler performance for success.

— Editorial Team