Thread by @Nick_Craver

Whenever someone tells me we should use containers for everything, my reaction is: "meh"

For some things they are great. For some things they are not. If it helps your use case: great! If not: don't be trendy to be trendy. Do what makes sense. Don't blindly follow any crowd.

We will be using containers for some scenarios here at Stack. Not all, at least not for quite some time, but *where it makes sense*.

Deploying to Azure? Client-hosted? Appliance? All great cases. Our production web tier? Not so much. Because it doesn't make sense there for now.

For instance, we're in a situation where the infrastructure is mature and optimized for the purpose. We're utilizing the most of the hardware it's on, and we don't need to touch it that much. Should we containerize Redis? Elastic? HAProxy? Maybe, maybe not. Lots of factors.

Breaking here to say: this is me, speaking from my view. Of course we have many opinions here and this is a topic we discuss every so often. Different groups have different needs and tradeoffs, e.g. devs, architecture, and SRE have different cost/benefit ratios on most decisions.

Now, should we containerize a service we touch 1-2 times a year, and has a tremendous amount of in-memory state?

What are the wins? Deployment? For something we don't deploy that often? Gut check there, is it worth it? *Will it save you more human time than current maintenance?*

The answer is: "maybe". In our case, the pain is not in deploying or upgrading the service (Puppet does that with a version number). It's the failover of huge in-memory instances at scale...that's an app/state/failover issue. How an offline instance is upgraded is the easy part.

What about Elastic? Well, the hardware it's on is fully utilized at times (we're bare metal), the indexes need to be local, etc. - so effectively you'd be deploying to a 1-container machine. (same for HAProxy & Redis) Does that make sense? Is the orchestration overheard worth it?

Put another way: what do containers get you *above and beyond Puppet*? That's the delta that matters.

There are wins, not debating that. But there are also complications, human work, and tradeoffs to get those wins. These must be weighed to see if something makes sense.

Compare this with our cloud-hosted deploys for Enterprise:
Elastic in containers? Hell yeah. Not much state, easy to rebuild that scale - just swap a cluster on upgrade if we need a new version and rebuild in 2 minutes.
Redis? Same story, it's an ephemeral cache that'll rebuild.

Now *if we wanted to be portable to the cloud*, that's a payoff for being in containers. For us though, that's not really the hardest problem after .NET Core (which we're wrapping up now for Q&A). It's latency...a problem sometimes tied to containers in the cloud.

Latency to your wherever your container service is (milliseconds will likely add up to big $ here) or the hosted services (e.g. managed Elastic, Redis Cache, etc.) is *huge*. That determines how much horsepower is needed on the front-end for concurrency due to slower pipelines.

Anyway, I don't want to ramble here. I just wanted to illustrate a tiny bit of why: "you should <x>" is almost never good universal advice.

We're hopefully going to be digging on some different scenarios for deployment this year and learn a lot. Stay tuned, planning ahead.

Again, not everyone at Stack shares all my views here (and that's awesome) - different teams have different tradeoffs. For example, the Enterprise SREs are using containers and love it - I totally agree that case makes the most sense.

If other Stackers chime in, I'll retweet :)