@fitzsimons_dev @dynamodb Oh! You're one of today's lucky 10,000!

DynamoDB streams are GREAT! I use them in almost every place I use a DDB table with #Serverless. I suggest doing non-essential work via stream vs in one long function call, so the end user gets the fastest experience possible.

Here's a 🧵
@fitzsimons_dev @dynamodb 1. Sending a confirmation email after a user registers.

This is an important function, but users shouldn't have to wait for my API to call an email provider before getting a response from the backend. Stream it to another function!
@fitzsimons_dev @dynamodb 2. Stateful/synchronizing needs! Lambda functions are a distributed system, no way around that (unless you set function concurrency to 1, but I digress). So you need to build idempotency into your app.

Example 👇
@fitzsimons_dev @dynamodb 2.cont Say you receive a Github PR webhook and want to kick off a job, like a CI task. Use the first invocation to ack the webhook, and load a new task in your DDB table.
Then use the stream-triggered function and a conditional update to guarantee your job runs only one time!
@fitzsimons_dev @dynamodb 3. A user swipes their card for your paid service, but you don't want to update multiple records in the callback from your payment service, because you can't risk dual charging.

Use a DDB stream to update the necessary record(s) once the initial invoice is processed!
@fitzsimons_dev @dynamodb 4. You can subscribe MANY functions to ONE stream. Consider Twitter for Pets. When someone important like @QuinnyPig's dog tweets, you will probably want to promote that in other TLs. Do that in a function triggered by DDB streams, don't hold up Corey's dog!
@fitzsimons_dev @dynamodb @QuinnyPig 5. DDB Streams work on a batch, and you can set a timeout window. Imagining Twitter for Pets again - you can use a stream with a batch of events to produce/collect trending topics. 5,000 dog park tweets in the last 20 minutes? Something is happening!
@fitzsimons_dev @dynamodb @QuinnyPig 6. Flaky APIs are a bummer. You can use of course use SNS and SQS to retry a flaky API with a backoff, but that's just as easy to do with a DDB stream, and DDB streams work off of shards (like kinesis, but DDB records are durable) - this can help prevent a noisy-neighbor issue
@fitzsimons_dev @dynamodb @QuinnyPig 7. Say you've got a busy app and are writing to your table frequently, but want to report aggregated metrics somewhere. You can use a DDB stream to read a batch and emit aggregated metrics, in a way that won't slow down your writing functions/code.
@fitzsimons_dev @dynamodb @QuinnyPig 8. Another concrete fan-out example. Your app denormalizes data to Elasticsearch for app search, but your hotshot new CIO wants it in redshift.

You can add a specific function (with its own error handling, biz logic, etc) for ES as well as Redshift - on the same stream
@fitzsimons_dev @dynamodb @QuinnyPig 9. Say I'm creating a slackbot for my custom deploy script for my custom k8s cluster (because I've entirely lost my mind. If this ever happens - send help).

Store the request & ack the webhook from Slack, then use DDB streams to trigger a function and run the deploy
@fitzsimons_dev @dynamodb @QuinnyPig 9.cont You can keep using conditional updates and treat your DDB record as a state machine - conditionally updating the record to update progress & ensure idempotency.
@fitzsimons_dev @dynamodb @QuinnyPig 10. Let's end with something more concrete - removing related data when a user deletes their account. (Not single table design in this case)

You won't want to open a transaction and remove 100s of records - that could time out or fail partway through.

Use a DDB stream!
@fitzsimons_dev @dynamodb @QuinnyPig 10.cont when the user record is deleted, you can trigger a function to size up a batch delete. It can perform it (and retry on failure), or fan out to other systems that need to clean up data. GDPR and DDB streams are meant to be
@fitzsimons_dev @dynamodb @QuinnyPig To sum it up - end your function as quickly as possible, and do the rest of the work asynchronously with streams!

The more you practice, the more opportunities you'll see to perform background/async work with DDB streams

• • •

Missing some Tweet in this thread? You can try to force a refresh

Keep Current with AJ Stuyvenberg

AJ Stuyvenberg Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!


Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!