New tools for detecting HTTPS interception

As the use of HTTPS grows, the desire of unauthorized persons to infiltrate protected traffic increases. A 2017 study by The Security Impact of HTTPS Interception found this to be an increasingly common practice. Analysis of traffic on Firefox update servers showed that in some countries the percentage of introducing foreign agents into HTTPS reaches 15%.

Since the study, the situation is unlikely to have improved. Now, even the latest Sennheiser wireless headphone model requires installing a root certificate (with insecure parameters) in the system .

Most often, antivirus or corporate middlebox compromises protection (see the list below), but it can be worse. In any case, it’s best to know for sure when your HTTPS channel is actually not secure end-to-end. Even when a third-party system intercepts well-intentioned traffic, it often does not support modern ciphers or does not validate certificates, thus reducing the overall protection of the connection. But you can intercept SSL traffic not only for good purposes, but also for malicious purposes: for example, for censorship at the state level.

HTTPS monitoring programs act as transparent proxies that break off a TLS session, inspect content, and then establish a new session with the destination server. They use different versions of TLS libraries than popular browsers, which allows them to be detected on the server side by the mismatch between the HTTP User-Agent and TLS Client Hello of the real browser and proxy.

To get started, here are some practical insights from a 2017 study that included Mozilla, Google, GlobalSign, as well as academic researchers from the University of Michigan, the University of Illinois at Urbana-Champaign, and the University of California at Berkeley.

Practical harm from HTTPS interceptionlies in the degradation of encryption and additional vulnerabilities of a transparent proxy. The authors of the study evaluated a number of popular middleboxes by these parameters. As you can see, in 2017, almost half of them did not support modern cipher suites, and five registered serious vulnerabilities. Only 1 out of 12 was able to completely mirror all custom cipher suites. Antivirus activity on users' computers is also usually detected on the server side. The study showed that only two antiviruses completely mirror user ciphers, in other cases encryption degrades.







Protecting against some threats, antiviruses reduce encryption and add their own vulnerabilities. In general, installing a third-party antivirus often significantly degrades the overall security of the system. Some experts advise users to always remove third-party antiviruses from the system , because their invasive and poorly written code makes it difficult for browsers and other programs to provide their own security.

On the way between the client and the server, other transparent proxies can also work. To better detect these agents, Cloudflare recently released two new tools:

1. MITMEngine , a free library for detecting HTTPS interception.
2. MALCOLM , a dashboard for displaying metrics about HTTPS interception on the Cloudflare network.

The main interest is the MITMEngine library. The developers write that they took the popular Caddy Server MITM Detection tool as a sample . It supports a set of popular browsers and recognizes HTTPS interception by the specific fingerprints of TLS Client Hello and User Agent messages, as described above.

The developers tried to provide extensibility by simplifying the future addition of new versions of browsers to the detector, as well as improving performance and expanding the functionality of the detector compared to the Caddy Server MITM Detection. It analyzes the following TLS Client Hello parameters, comparing real data with fingerprints of known browsers:

• TLS version
• cipher suites;
• extensions and their meanings;
• supported groups of elliptic curves;
• formats of points of an elliptic curve.

In fact, MITMEngine performs partial fingerprinting of a user without deanonymizing it, but with a reliable determination that the connection is established by a real browser, not an intermediary.

Work example

Suppose MITMEngine sees the following User Agent from the user: This User Agent corresponds to Chrome 47 under Windows 7. It is accompanied by the TLS Client Hello message, which indicates the following cipher suites, in hex: This message corresponds to the following cipher suites : Default cipher suites for Chrome 47 the following: If you look closely, you can see: in real traffic, the list of ciphers is slightly shorter than it should be in a real browser. Although the rest are arranged in the same order, two sets are missing: We can assume that traffic from the user's browser goes through a transparent proxy that does not support ChaCha ciphers. This reduces user protection because the AES-CBC cipher suites that are next to ChaCha are vulnerable.

Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/47.0.2526.111 Safari/537.36



0000 c0 2b c0 2f 00 9e c0 0a c0 14 00 39 c0 09 c0 13 .+./.... ...9....
0010 00 33 00 9c 00 35 00 2f 00 0a .3...5./ ..



TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02b)
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)
TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (0x009e)
TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (0xc00a)
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)
TLS_DHE_RSA_WITH_AES_256_CBC_SHA (0x0039)
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA (0xc009)
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)
TLS_DHE_RSA_WITH_AES_128_CBC_SHA (0x0033)
TLS_RSA_WITH_AES_128_GCM_SHA256 (0x009c)
TLS_RSA_WITH_AES_256_CBC_SHA (0x0035)
TLS_RSA_WITH_AES_128_CBC_SHA (0x002f)
TLS_RSA_WITH_3DES_EDE_CBC_SHA (0x000a)



0000 c0 2b c0 2f 00 9e cc 14 cc 13 c0 0a c0 14 00 39 .+./.... .......9
0010 c0 09 c0 13 00 33 00 9c 00 35 00 2f 00 0a .....3.. .5./..



TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 (0xcc14)
TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (0xcc13)

for attacks like padding oracle.

In some cases, MITMEngine even allows you to determine which specific program causes the degradation of the HTTPS connection (in the example above, it is Sophos antivirus). The server can warn the user about this threat.

---