Cloudflare Reverse Proxies are Dumping Uninitialized Memory

Between 22 Sep 2016 – 18 Feb 2017 passwords, private messages, API keys, and other sensitive data were leaked by Cloudflare to random requesters.
On this page

Cloudflare Reverse Proxies are Dumping Uninitialized Memory

Between 22 Sep 2016 – 18 Feb 2017 passwords, private messages, API keys, and other sensitive data were leaked by Cloudflare to random requesters. Data was cached by search engines, and may have been collected by random adversaries over the past few months. Many people call this issue Cloudbleed.

This bug reported by Tavis Ormandy from Google’s Project Zero. He was seeing corrupted web pages being returned by some HTTP requests run through Cloudflare. Once they understood what they see and the implications, they stopped and contacted cloudflare security.

Cloudbleed Impact

Requests to sites with the HTML rewrite features enabled triggered a pointer math bug. Once the bug was triggered the response would include data from ANY other Cloudflare proxy customer that happened to be in memory at the time.

Meaning a request for a page with one of those features could include data from Uber or one of the many other customers that didn’t use those features. So the potential impact is every single one of the sites using Cloudflare’s proxy services (including HTTP & HTTPS proxy).

Here’s a random example of the data Google’s Project Zero Member are finding and purging.

Cloudbleed Dump

Fitbit

a private message from a popular dating site, <strong>okcupid</strong>

The greatest period of impact was from February 13 and February 18 with around 1 in every 3,300,000 HTTP requests through Cloudflare potentially resulting in memory leakage (that’s about 0.00003% of requests), potential of 100k-200k paged with private data leaked every day — source

Confirmed affected domains found in the wild: http://doma.io/2017/02/24/list-of-affected-cloudbleed-domains.html.

Detailed Timeline

* Time in UTC

  • 2017-02-18 0011 Tweet from Tavis Ormandy asking for Cloudflare contact information.
  • 2017-02-18 0032 Cloudflare receives details of bug from Google
  • 2017-02-18 0040 Cross functional team assembles in San Francisco
  • 2017-02-18 0119 Email Obfuscation disabled worldwide
  • 2017-02-18 0122 London team joins
  • 2017-02-18 0424 Automatic HTTPS Rewrites disabled worldwide
  • 2017-02-18 0722 Patch implementing kill switch for cf-html parser deployed worldwide
  • 2017-02-20 2159 SAFE_CHAR fix deployed globally
  • 2017-02-21 1803 Automatic HTTPS Rewrites, Server-Side Excludes and Email Obfuscation re-enabled worldwide

Source : Cloudflare

What should I do?

Check your password managers and change all your passwords, especially those on these affected sites (below). Rotate API keys & secrets, and confirm you have 2FA set up for important accounts. This might sound like fear-mongering, but the scope of this leak is truly massive, and due to the fact that all Cloudflare proxy customers were vulnerable to having data leaked, it’s better to be safe than sorry.

Theoretically sites not in this list can also be affected (because an affected site could have made an API request to a non-affected one), you should probably change all your important passwords.

Full List Sites

Download the full list.zip (22mb). 4.287.625 possibly affected domains. Download this file, unzip it, then run :

1grep -x domaintocheck.com sorted_unique_cf.txt

to see if a domain is present.

Also, a list of some iOS apps that may have been affected.

Methodology

According to Nick Sweeting, this list was compiled from 3 large dumps of all Cloudflare customers provided by crimeflare.com/cfs.html, and several manually copy-pasted lists from stackshare.io and wappalyzer.com. Crimeflare collected their lists by doing NS DNS lookups on a large number of domains, and checking SSL certificate ownership.

He scraped the Alexa top 10,000 by using a simple loop over the list:

1for domain in (cat ~/Desktop/alexa_10000.csv)
2    if dig $domain NS | grep cloudflare
3        echo $domain >> affected.txt
4    end
5end

The Alexa scrape, and the Crimeflare dumps were then combined in a single text file, and passed through uniq | sort.

Data sources

You can also contact Nick Sweeting on twitter @theSquashSH if you believe your site is not affected, submit a PR on github.

Sources