Censors Ignore Unencrypted HTTP/2 Traffic

Using Unencrypted HTTP/2 to Circumvent Censorship

*Authors in alphabetical order – all contributed equally

Abstract

Censors worldwide have long censored unencrypted HTTP traffic. In this blog post, we show that a specific HTTP version—unencrypted HTTP/2—is unaffected by censorship in China and Iran. We access otherwise censored websites in both countries over unencrypted HTTP/2. Despite no web browser implementing unencrypted HTTP/2, we detect that up to 6.28% of websites support unencrypted HTTP/2 traffic. To aid the community and ease future research, we provide a tool that evaluates the unencrypted HTTP support of a website. Finally, we discuss the limitations and potential of unencrypted HTTP/2 for censorship circumvention. We consider our finding an interesting addition to current censorship circumvention techniques.

Note: Do not send sensitive data over unencrypted HTTP/2, it can be eavesdropped!

Introduction and Background

In this section, we provide background information on HTTP and its censorship. We place special emphasis on HTTP/2 and its comparison to previous HTTP versions.

HTTP (Censorship)

HTTP can be considered the protocol for accessing websites on the Internet. While HTTP is usually used in conjunction with TLS, censorship of the plain HTTP protocol is still present and prior research analyzed the most common version HTTP/1.1  extensively . These works discovered that censors such as the ones in China and Iran use information in the Host header and request path to determine whether an HTTP request should be censored. The HTTP/1.1 GET request below shows that the requested path and domain name can be easily extracted by a censor:

GET /<request path> HTTP/1.1
Host: <censored_domain>

To censor a request, censors inject TCP RST packets, HTTP block pages, or null-route the whole connection .

HTTP/2

As outlined above, HTTP/1.1 censorship has been widely analyzed. Nevertheless, there is a lack of research for the newer version: HTTP/2 . HTTP/2 maintains the same semantics as its predecessor: it uses the same port numbers, runs over TCP, and most importantly it conveys the request path and accessed domain name unencrypted as depicted in the exemplary HTTP/2 request below.

:method              GET
:scheme              HTTP
:host                <censored_domain>
:path                <request_path>

Despite its similarities to HTTP/1.1, HTTP/2 introduced a major change by switching from a text-based format to a binary format. While HTTP/2 is still easily censorable due to missing encryption, HTTP/2’s binary format complicates censors’ efforts as it requires different parsing mechanisms.

To facilitate the upgrade from HTTP/1.1 to HTTP/2, clients can use either of two mechanisms: connect with HTTP/2 (Prior Knowledge) or upgrade an existing HTTP/1.1 connection (Upgrade Mechanism). Below, we describe both mechanisms.

Prior Knowledge

Using prior knowledge, the client directly attempts to speak HTTP/2 with the server. As the initial step, the client establishes a TCP connection with the server. Then, the client sends the so-called connection preface, followed by a SETTINGS frame. This connection preface was specifically defined to trigger errors on HTTP servers without HTTP/2 support. The SETTINGS frame contains additional configuration parameters for the connection. After sending the connection preface and SETTINGS frame, the client sends its HTTP/2 request. If the server supports HTTP/2, it responds with its own connection preface, containing a SETTINGS frame, and answers the client’s HTTP/2 request with an HTTP/2 response.

Upgrade Mechanism

During the upgrade mechanism, the client establishes a TCP connection with the server and then sends an HTTP/1.1 request, including the Upgrade and HTTP2-Settings headers. An example HTTP/1.1 Upgrade request can look like this:

GET / HTTP/1.1
Host: server.example.com
Connection: Upgrade, HTTP2-Settings
Upgrade: h2c
HTTP2-Settings: <base64url encoding of HTTP/2 SETTINGS payload>

If the server supports the upgrade mechanism, it responds with a 101 status code, indicating Switching Protocols, and then transitions to an HTTP/2 connection. An example of a successful server answer can look like this:

HTTP/1.1 101 Switching Protocols
Connection: Upgrade
Upgrade: h2c
[ HTTP/2 answer for "GET /" ... ]

Initially defined by RFC 7540 , the upgrade mechanism has been deprecated since RFC 9113  as it was not widely adopted. It is also less fit for censorship circumvention than prior knowledge as it involves an HTTP/1.1 request that is caught by HTTP censors that do not support HTTP/2.

Research Gap

In the context of censorship, HTTP/2 has not been analyzed extensively. In this blogpost, we analyze HTTP/2’s viability as a censorship circumvention technique and show the following.

1. HTTP/2 is uncensored in China and Iran, two countries that widely employ HTTP/1.1 censorship.
2. Up to 6.28% of servers support unencrypted HTTP/2, with smaller servers and censored domains showing higher support than large websites.
3. Unencrypted HTTP/1.1 is still supported by around 20% of servers.
4. To easily analyze a server for its unencrypted HTTP support, we provide the tool Does-It-Support-Unencrypted-Http.

Results

In the following, we show that unencrypted HTTP/2 is not censored in China and Iran. Then, we outline existing server support for unencrypted HTTP/2.

Censorship

In China, HTTP/1.1 is censored with a combination of different TCP RST packets with the ACK flag set or not, depending on the specific domain. In Iran, HTTP/1.1 requests are censored with a block page, a TCP RST, or null routing, depending on the specific domain. HTTP/1.1 censorship in China and Iran can be triggered with the curl command curl http://nsfwyoutube.com --resolve nsfwyoutube.com:80:208.78.226.162. The --resolve flag prevents possible DNS censorship from interfering.

While HTTP/1.1 is censored in China and Iran, we detected that HTTP/2 with prior knowledge is not censored at all in both countries. For any domain censored over HTTP/1.1, the equivalent HTTP/2 requests were not censored in our evaluations. For instance, curl --http2-prior-knowledge http://nsfwyoutube.com --resolve nsfwyoutube.com:80:208.78.226.162 accesses a website censored over HTTP/1.1 in both countries. In China and Iran, any server that supports unencrypted HTTP/2 can be accessed without interference from existing HTTP censorship.

Server Support

To determine servers’ support for unencrypted HTTP, we tested domains from three lists: We selected domains from Tranco top one million, the CitizenLab test list for China, and the CitizenLab test list for Iran. The Tranco top one million list contains domains popular worldwide; the Citizenlab lists contain domains censored in the respective country. Some domains were unresolvable or did not open a TCP socket; we excluded them from our evaluations.

To each reachable website, we sent a GET request on the path / with either HTTP/1.1, HTTP/2 prior knowledge, or the HTTP/2 upgrade mechanism. We followed all redirects to other HTTP websites and considered a server to support the used HTTP version if it responded with a 200 OK using the desired HTTP version. The following table shows the number of domains that support HTTP/1.1, HTTP/2 prior knowledge, and the HTTP/2 upgrade mechanism.

List	HTTP/1.1	HTTP/2 Prior Knowledge	HTTP/2 Upgrade Mechanism	Total
Tranco Top 1M	156 316 (18.62%)	20 973 (2.50%)	5227 (0.62%)	839 393
CitizenLab China	96 (19.35%)	24 (4.84%)	3 (0.60%)	496
CitizenLab Iran	161 (21.52%)	47 (6.28%)	13 (1.74%)	749

Below, we detail our test vectors and servers’ support for each HTTP version we evaluated.

Unencrypted HTTP/1.1 Support

During the unencrypted HTTP/1.1 support scan, we sent the following test vector for all domains:

GET / HTTP/1.1
Host: <domain_name>
User-Agent: Mozilla/5.0 (...) Gecko/20100101 Firefox/127.0
Connection: close

Across the three used lists, the HTTP/1.1 support varies between 18.62% and 21.52%. These results show that - surprisingly - unencrypted HTTP/1.1 is still supported by many domains.

Unencrypted HTTP/2 with Prior Knowledge Support

During the unencrypted HTTP/2 with prior knowledge scan, we sent the connection preface first and then the following test vector for all domains:

GET / HTTP/2
Host: <domain_name>
User-Agent: Mozilla/5.0 (...) Gecko/20100101 Firefox/127.0

The HTTP/2 with prior knowledge support varies between 2.5% and 6.28%, depending on the domain list. Importantly, censored domains from the CitizenLab lists show higher support for HTTP/2 prior knowledge than popular domains from the Tranco top one million list. This shows that while servers’ support for HTTP/2 prior knowledge is lower than for HTTP/1.1 across all lists, a considerable number of censored domains support unencrypted HTTP/2.

Unencrypted HTTP/2 with Upgrade Mechanism Support

During the unencrypted HTTP/2 with upgrade mechanism scan, we sent the following HTTP/1.1 request as a test vector for all domains:

GET / HTTP/1.1
Host: <example.com>
User-Agent: Mozilla/5.0 (...) Gecko/20100101 Firefox/127.0
Connection: Upgrade, HTTP2-Settings
Upgrade: h2c
HTTP2-Settings: AAEAABAAAAIAAAABAAQAAP__AAUAAEAAAAgAAAA...

The upgrade mechanism has the lowest support ranging from 0.6% to 1.74%, depending on the used domain list. This is unsurprising as RFC 9113  deprecated this method due to its low adoption rate.

Tranco Relation

Across all scanned HTTP versions, support for unencrypted HTTP varied widely across the Tranco top one million list. Below, we depict servers’s support for the different HTTP versions across the Tranco top 1 million list.

Unencrypted HTTP support by servers from the Tranco Top 1 Million list. Unencrypted HTTP/2 support increases for less accessed websites, contrasting unencrypted HTTP/1.

Interestingly, unencrypted HTTP/2 support increases for lower Tranco ranks while unencrypted HTTP/1 support increases until about 400,000 and then decreases again. This aligns with support by websites from Citizenlab list which are usually smaller than Tranco websites. While overall support for HTTP/2 is low, we argue that it can still be used to access certain blocked websites successfully and adds to the arsenal of censorship circumvention techniques.

Tool for Detecting HTTP Support

During our analyses, we developed a tool that evaluates a website’s support for unencrypted HTTP support. Below, we provide an exemplary output for lgbtchinatour.com

lgbtchinatour.com analysis started.
Server online. Scanning!

#####################

HTTP/1.0: REDIRECT(www.lgbtchinatour.com/) -> SUCCESS
HTTP/1.1: SUCCESS
HTTP/2 (Prior Knowledge): FAILURE
HTTP/2 (Upgrade): FAILURE

This indicates that lgbtchinatour.com supports unencrypted HTTP/1.0 after a redirect and unencrypted HTTP/1.1 but no unencrypted HTTP/2. We refer to the GitHub project for a detailed overview of the tool’s functionality. We hope that our tool aids the community and fellow researchers in their evaluation of HTTP censorship and its circumvention.

Discussion

We have successfully circumvented the HTTP censors in China and Iran and showed that up to 6.28% of censored websites are accessible via unencrypted HTTP/2. This still leaves over 93% of censored websites inaccessible with unencrypted HTTP/2 as a circumvention method. Despite this, we consider unencrypted HTTP/2 a valuable addition to current censorship circumvention techniques in the cat-and-mouse game between censors and affected people. Below, we discuss the limitations and potential of unencrypted HTTP/2 as a circumvention technique.

What makes unencrypted HTTP/2 special?

HTTP/2 differs vastly from previous HTTP versions such as HTTP/1.1. While all HTTP versions up to and including HTTP/1.1 consist of human-readable ASCII letters, HTTP/2 is a byte-based protocol. This makes HTTP/2 parsers incompatible with previous HTTP parsers. Furthermore, HTTP/2 has been designed for encrypted usage in HTTPS. Unencrypted usage is allowed but not advertised. Accordingly, browsers do not implement unencrypted HTTP/2. We suspect that this is the reason censors do not analyze it and why the censorship circumvention community has ignored unencrypted HTTP/2.

Difficulty of blocking unencrypted HTTP/2

Censors can block HTTP/2 but face additional challenges compared to previous HTTP versions. For previous HTTP versions, censors could analyze the first message sent by the client to extract the connection’s destination from human-readable ASCII bytes. In HTTP/2 the first message must not necessarily contain the connection destination forcing the censor to hold additional state and parse additional messages. HTTP/2 also utilizes a new form of header compression which censors would have to accommodate. We suspect that the complexity added in HTTP/2 contributed to the censors’ decision not to analyze it. We emphasize that censors could still implement HTTP/2 censorship despite its complexity.

Circumventing Censors with unencrypted HTTP/2

Practically applying unencrypted HTTP/2 as a censorship circumvention technique is possible but comes with challenges. In this blog post, we accessed otherwise blocked resources with unencrypted HTTP/2 using curl. Curl’s built-in support for unencrypted HTTP/2 can be used to access specific HTML sites but its usability is limited in comparison to a full-fletched web browser. Unfortunately, we are not aware of any web browser that supports unencrypted HTTP/2. To utilize unencrypted HTTP/2 as a censorship circumvention technique from a browser, we propose an HTTP update proxy that translates unencrypted HTTP/1.1 spoken by the browser into unencrypted HTTP/2 and vice-versa. A similar proxy could be installed on the other side of the firewall, downgrading unencrypted HTTP/2 to unencrypted HTTP/1.1, allowing unencrypted HTTP/2 traffic to pass the censor and connect to a server that only supports unencrypted HTTP/1.1. Overall, we consider the practical deployment of HTTP/2 as a censorship circumvention method difficult and are interested in possible approaches.

Conclusion

In summary, we introduced unencrypted HTTP/2 as a censorship circumvention method by accessing blocked resources in China and Iran. While server support for unencrypted HTTP/2 is low, we showed that a non-negligible number of censored websites supports it. To aid future evaluations of servers’ support for unencrypted HTTP/2, we developed a tool and made it accessible on GitHub. Feel free to use it or contact us for further discussion and future work. We hope that our contributions aid affected people and the censorship circumvention community. Note that unencrypted HTTP/2 does not protect your traffic; do not use it as a censorship circumvention technique when transmitting sensitive data.