Analysis of the Alexa Top 1M sites (October 2016)

, in Security

Last April, I ran a scan of the Alexa Top 1M websites using the Mozilla Observatory. The results were dire, indicating a broad lack of awareness around modern security technologies such as Content Security Policy, Strict Transport Security, Subresource Integrity, and others.

But that was six months ago. With the Mozilla Observatory being publicly released almost two months ago, I was curious as to whether significant improvement had been made around the internet. After all, in those two months, the Observatory has scanned approximately 1.3M sites, totalling over 2.5M scans.

With that in mind, I ran a new scan of the Alexa Top 1M at the end of October, and here is what I found:

Technology April 2016 October 2016 % Change
Content Security Policy (CSP) .005%1 / .012%2 .008%1 / .021%2 +60%
Cookies (Secure/HttpOnly)3 1.88% 2.44% +30%
Cross-origin Resource Sharing (CORS)4 93.78% 96.21% +3%
HTTPS 29.64% 33.57% +13%
HTTP → HTTPS Redirection 5.06%5 / 8.91%6 7.94%5 / 13.29%6 +57%
Public Key Pinning (HPKP) 0.43% 0.50% +16%
  — HPKP Preloaded7 0.41% 0.47% +15%
Strict Transport Security (HSTS)8 1.75% 2.59% +48%
  — HSTS Preloaded7 .158% .231% +46%
Subresource Integrity (SRI) 0.015%9 0.052%10 +247%
X-Content-Type-Options (XCTO) 6.19% 7.22% +17%
X-Frame-Options (XFO)11 6.83% 8.78% +29%
X-XSS-Protection (XXSSP)12 5.03% 6.33% +26%

I'll admit, I was a bit taken aback by the overall improvement across the top million sites, especially as some of these security technologies are almost a decade old.

When we did our initial scan of the top million six months ago, a stunning 97.6% of websites were given a failing grade from the Observatory. Have those results changed since then, given the improvements above?

Grade April 2016 October 2016 % Change
  A+ .003% .008% +167%
A .006% .012% +100%
B .202% .347% +72%
C .321% .727% +126%
D 1.87% 2.82% +51%
F 97.60% 96.09% -1.5%

While a decrease of 1.5% in failing grades might seem like only a small improvement, the latest Observatory scan contained 962,011 bg-successful scans. With each percentage point representing nearly ten thousand sites, a drop from 97.6% to 96.09% represents approximately fifteen thousand top websites making significant improvements in their security.

I'm excited for the possibility of seeing further improvements as additional surveys are completed. Please share the Mozilla Observatory and help to make the internet a safer and more secure place for everyone!



Footnotes:

  1. Allows 'unsafe-inline' in neither script-src nor style-src
  2. Allows 'unsafe-inline' in style-src only
  3. Amongst sites that set cookies
  4. Disallows foreign origins from reading the domain's contents within user's context
  5. Redirects from HTTP to HTTPS on the same domain, which allows HSTS to be set
  6. Redirects from HTTP to HTTPS, regardless of the final domain
  7. As listed in the Chromium preload list
  8. max-age set to at least six months
  9. Percentage is of sites that load scripts from a foreign origin
  10. Percentage is of sites that load scripts
  11. CSP frame-ancestors directive is allowed in lieu of an XFO header
  12. Strong CSP policy forbidding 'unsafe-inline' is allowed in lieu of an XXSSP header

[Category: Security] [Tags: Alexa, Cookies, CORS, CSP, HPKP, HSTS, Observatory, HTTPS, Redirection, SRI, XCTO, XFO, XXSSP]