Jay Kreps Profile picture
CEO of @confluentinc (https://t.co/fKQHUe4nIJ). Co-creator of @apachekafka. Sí se puede.
Aug 18, 2023 13 tweets 3 min read
1. The two-phase commit proposal for @apachekafka (KIP-939) is pretty interesting. Quick thread on why it matters. cwiki.apache.org/confluence/dis… 2. The first use case was actually to make @apachekafka and @ApacheFlink
work better together. That was one driver for @confluentinc to work on it right now (we have a Flink service coming!). But the applicability is much broader.
May 16, 2023 23 tweets 7 min read
1. Here’s a summary of my #kafkasummit talk for those who missed the live stream with links and pics of all the things we announced! Image 2. This was probably the most new open source stuff, cool at-scale cloud details, and new Confluent features I’ve ever had in a keynote. But it was too exciting to leave anything out.
Oct 29, 2021 16 tweets 3 min read
1. Thoughtworks notes that "Kafka continues toward its status as a de facto standard." noting that Kubernetes, Kafka, and the CSPs are becoming stable layers in the next gen stack and churn around alternative platforms seems to have waned. thoughtworks.com/content/dam/th… 2. This matches our internal data as well. Measuring open source usage is pretty hard, but our best data is that Kafka adoption is growing 7x faster than the fastest growing alternative off a base that is more than 15x the scale.
Jun 24, 2021 16 tweets 3 min read
1. A quick reflection on Confluent's IPO today and the journey so far (a thread!). 2. We wrote the initial Kafka code base at LinkedIn in 2009-2010. In 2011 we released the initial Kafka code as open source to...resounding silence. No one cared!
Aug 25, 2020 34 tweets 6 min read
1/ In April we at @confluentinc kicked off what we call Project Metamorphosis, which is all about building a real cloud-native service around Kafka and it's ecosystem. I talked about why I think this is a big deal in my Kafka Summit Keynote today. Here's a twitter summary: 2/ My talk's central thesis - There are two major trends that so far have been largely disjoint: cloud-native data systems and event streaming, and these need to converge. What do I mean by that?
Aug 3, 2018 7 tweets 2 min read
1/ Faust is a python library from for stream processing with @apachekafka from @RobinhoodApp. I think it's really cool. It highlights one of the things I think we got right with Kafka Streams: supporting stream processing in Kafka at the protocol level. github.com/robinhood/faust 2/ This means having a model in Kafka's core protocol for elastic scalability, partitioning, stateful processing, and transactionally correct processing that covers both input, output, and state changes but is decoupled from any implementation of code that does this processing.
Nov 1, 2017 36 tweets 4 min read
1/ People often ask why it took so long for Kafka to go 1.0. 2/ The answer isn't about stability: Kafka's been in production at thousands of companies for years.
Jun 30, 2017 6 tweets 1 min read
If you saw Segment's homegrown exactly once support, the Kafka feature is built into the protocol and vastly faster segment.com/blog/exactly-o… Why? Cause it doesn't build a random access DB of all keys, instead it gives sequential sequence numbers dramatically reducing the state.