How can you develop against prod, doing it safely and securely?
And how can you do the reverse: route prod traffic to a dev environment? Run the app store app on your phone, and have it trigger a breakpoint on your laptop?
Here's a working solution. Thread with some boxes.
And how can you do the reverse: route prod traffic to a dev environment? Run the app store app on your phone, and have it trigger a breakpoint on your laptop?
Here's a working solution. Thread with some boxes.
1. We want our dev service, running on our laptop to talk to a production service. For example, we're modifying the onboarding service in dev that needs to access the login service in prod. Or something like that.
Point is, we can't just talk to it directly.
Point is, we can't just talk to it directly.
2. Let's add a proxy client that will forward messages. And let's box all of this up into a VM that can run locally on our dev machine (we'll see why we'll use a VM in a minute).
3. We *really* don't want to access production unbounded. This could lead to security, data integrity and many other issues. So let's build a data access gateway that will both log all traffic access (so we can later audit) and have rules to ensure data is accessed securely.
4. The access control gateway is built/maintained by the infra team, not the local developer. And it's this gateway that can talk to prod. Tada!
Oh, wait, how are we going to get the response back?
Oh, wait, how are we going to get the response back?
5. This is where the VM comes into play. Because the response goes to the Access Control Gateway, who then sends this back to the VM, which forwards to the proxy client (the proxy client listens on the right port) - and the proxy onto the dev service.
Boom! 💥
Boom! 💥
6. Okay, so we have dev services talk to prod. Now what about routing production traffic to dev? Think: I run my local service, open the production app, tap a button, and get a breakpoint in my service? We want to do exactly the reverse, in this case:
7. So this will be more work. First, we need to install a "Route from prod" middleware on all our prod instances. I know, I know, lots of work t write this, then install it, but it will be worth it.
8. This "route from prod middleware" will talk to a forwarding service. Seeing a pattern here?
9. Okay, let me help. So we want the forwarding service to talk to the dev service.
This looks... awfully similar to what we just did, when we had dev traffic route to prod, but dev and prod are swapped!
This looks... awfully similar to what we just did, when we had dev traffic route to prod, but dev and prod are swapped!
10. Could we just reuse our ProdAccessGateway we built last time, that allowed secure routing? You bet we could!
11. Okay, let's draw in the boxes to remind what ProdAccessGateway actually does.
12. And this, is a possible way to safely develop against prod, and route prod traffic to dev.
The devil lies in the details, and there's some work to do: building the Access Control Gateway and the Route From Prod Middleware. But do this, and it will be (safe) magic.
The devil lies in the details, and there's some work to do: building the Access Control Gateway and the Route From Prod Middleware. But do this, and it will be (safe) magic.
13. The end. And this is something that actually works in production.
