Thread by @dinodaizovi

1/n: There is a lot of risk in layering disparate security models because they often leave exploitable gaps at the seams. In the cloud, when you run k8s in the cloud, you are layering many security models: IAM, k8s RBAC, k8s Pod "sandbox", Linux containers, Unix user/groups, etc.

2/n: When I kick the tires on k8s clusters, I go straight for the seam between k8s and the cloud IAM permissions for maximal privilege escalation. This usually gives you access to powerful IAM roles in large, shared cloud accounts. It's a high-risk, large blast radius design.

3/n: Compare to the approach of baking app+OS into a single immutable AMI and embracing IAM security model with roles, sub-accounts/projects. App and OS vulns are roughly equivalent because they can only escalate to app's IAM role. If you use sub-accounts per app, even better.

4/n: Cloud sub-accounts/projects are strong security boundaries, they are also what protect customer A from customer B. You can still put shared resources in shared sub-account/projects and get more security with AWS SCPs / GCP OPCs and more development velocity at the same time

5/n: When you put everyone's resources cloud stuff in big shared accounts, you effectively have to build your own security model to keep using one big security domain safe when there are fewer and weaker security boundaries. That's a lot of risky undifferentiated heavy lifting.

Addendum: I think of this like Active Directory cross domain and forest trusts. It's a lot of risk to put your entire organization in one AD Domain and I'd bet that Microsoft is the only org on the planet who can do it safely.

Don't run your clouds like one huge AD domain!