Thread by @Scott_Helme

It's been a while since I've had chance to sit down and produce a report on the security of the Top 1 Million sites, but thanks to @Venafi's support, the crawler project lives on and a brand new report is out! venafi.com/blog/crawler-r…

It takes a lot of resources to gather this data and a lot of time to analyse it all and write the report, so genuinely, it wouldn't have happened without them. There hasn't been a report for 18+ months so let's take a look at what changed! 😎

HTTPS adoption continues to surge 🔐📈

72% of sites in the Top 1M are now actively redirecting HTTP --> HTTPS 🤩

We're using more HTTPS right now than at any point in history... 😮

Redirecting to HTTPS isn't quite enough though, sites also need to use HSTS and we've seen strong growth there too! ✔

scotthelme.co.uk/hsts-the-missi…
scotthelme.co.uk/hsts-preloadin…

Without any doubt, credit has to be given to @letsencrypt as the largest issuing CA in the Top 1M sites by quite some margin! 💪

Almost 25% of the Top 1M sites are using a @letsencrypt certificate, 240,461 sites!

The presence of @letsencrypt is smaller in the largest sites as they are still using more traditional CAs.

@letsencrypt isn't the only big change in the CAs though, we can see a large shift towards automated or service provided certificates!

As HTTPS surges, it seems sites are choosing DV over EV when it comes to certificates. EV is now at the lowest usage levels I've recorded. 📉

My suspicions:
1. EV certs are 💰💰💰
2. EV certs are harder to automate 🔄
3. EV UI removed in the browser ❌🔒

Another very notable trend in the Top 1M sites is the use of @Cloudflare! ⛅

You can see their presence in the certificate stats above, but it really shows when you look at the server headers of sites and see they're the clear choice 🥇

I suspect this large presence of @Cloudflare is largely responsible for some other positive trends we've seen. ⛅📈🔒

For example, the new TLSv1.3 protocol has seen faster adoption than I expected since it's standardisation in 2018.

Adoption of TLSv1.3 changes the preferred cipher suites for the Top 1M sites. 🧾

The clear winner is a TLSv1.3 suite, whilst TLSv1.2 suites still hold firm in 2nd and 3rd place.

🥇TLS_AES_256_GCM_SHA384
🥈ECDHE-RSA-AES256-GCM-SHA384
🥉ECDHE-RSA-AES128-GCM-SHA256

An interesting trend in cipher suite data; sites at the top of the ranking tend to prefer AES128 more than AES256. 🔑 <-> 🗝️

This is particularly noticeable in the TLSv1.2 data and still a clear trend in the TLSv1.3 data! Are they worried about performance? 🤔

Talking of performance, looking at the auth keys that sites are using does raise some questions! 🔏

RSA is still the clear winner, beating ECDSA by quite some margin. Weirdly, though, there's still a lot of use of RSA3072 and RSA4096(!). 😲

Another important performance metric is the adoption of HTTP/2.0 which has taken a big leap! 🔥

Enabled by the rise in HTTPS support (prerequisite for h2), we see a sharp rise in adoption in the higher ranked sites looking to go faster. 🏎

It's nice to see sites deploying a security.txt file too! 📞

If you don't have one, you should check out this blog post: scotthelme.co.uk/say-hello-to-s…