Thread by @jthomerson

It's time! @ben11kehoe is talking about AWS Step Functions. A thread with some notes. #reinvent #serverless

First, general comments from @ben11kehoe:

"Don't complain about AWS where AWS can't hear you. Things won't get better if they don't know". Use #awswishlist or tag Ben himself.

Also: "Serverless is about not owning technology that doesn't differentiate you as a business"

"Everything that says IoT, ignore that it says IoT and evaluate it for your own purposes". The AWS IoT offerings are super powerful for all kinds of other things. IoT Events can work for state machines where Step Functions wouldn't be a good fit.

Using callback URLs for approval emails with AWS Step Functions:

Blog post: tinyurl.com/sfn-callback-u…
Example SAR app: tinyurl.com/sfn-callback-u…

Avoid containers and long-running things whenever you can. In other words, don't start an ECS task that works as a poller. Build an event-driven workflow. (see next tweet about the callback pattern as a replacement for pollers)

Callback pattern:

Your SF invokes a Lambda with a token and URLs, one for each outcome you want (e.g. approve, deny). Lambda generates the email with those URLs in it. The user clicks on the URL, which hits the APIGW, which uses the token to call back to SF to update the state.

The token can be encrypted, encoded, and put in the URL. The URL then has all the state that's needed. The APIGW will decrypt the token and then use it to call the SendTaskSuccess API on the Step Functions SDK.

docs.aws.amazon.com/step-functions…

Side note: the Step Functions web console is actually really useful. Really nice to see the state machine execution history. iRobot requested the feature where a caught failure (which was green) is now orange to indicate that there was a failure, even though it was caught.

Never-ending state machine pattern:

iRobot deploys firmware updates with a state machine to progressively roll out updates incrementally. Their state machine queries for the robots that need the update, tells the robots the update is ready for them, and then watches what happens

Then there's a Lambda that listens to the "execution completed" CloudWatch event on the state machine and starts a new execution of the state machine.

b/c it's possible the CW event/Lambda fails, there's also a scheduled func that makes sure there's always one machine running

Side note: I had a similar usecase and just had a Lambda function that runs at the end of the state machine that starts a new/separate (not nested) execution of the same machine. Your call on which way to do it. Depends on how you want to separate concerns.

Distributed transactions pattern:

You might have used SQS -> Lambda for this before. The Lambda would do a checklist of things, but if one of those things failed, the entire function failed, and the entire list had to be redone (because the message would be redriven).

Step Functions are better for managing this checklist of things to do. Once you get the syntax correct for your state machine, it's more likely to "just work" and keep working than your Lambda that's managing the checklist of things.

tl;dr: Write less code. Use services like SF

Note: Step Functions (standard workflows) guarantee exactly-once execution. The new express workflows guarantee at-least-once execution. Keep this in mind when designing your distributed transaction.

See aws.amazon.com/step-functions…

Not many people in this session were excited about express workflows. @ben11kehoe said look into your Lambdas for any place where you have a list of steps that's being done. You get better visibility into those steps if you break them out and use an express workflow for it.

Execution workflows should be part of your _architecture_, not your _code_.

Step Functions is the new business process engine.

EventBridge is the new event messaging bus.

They're really-well integrated into the AWS ecosystem and fit well with modern architecture patterns.

Similarly, @ben11kehoe prefers making each endpoint on an API a separate function. It makes it easier to see which function is failing when there's errors (e.g. read vs create), and you can tune memory allocations based on the needs of each function. (sorry about the blurry arm)

Thanks @ben11kehoe! Great session, as usual. I appreciate your clear and simple explanations, and your pragmatism when responding to questions.

@ben11kehoe I didn't have her tag at the time, but @beccaodelay gave a great intro to the session and showed how much she values customer feedback. That's the thing that constantly impresses me about the AWS folks ... they genuinely listen, even if you're a "little guy". Thanks @beccaodelay!