the convergence of batch and streaming worlds #current22
Being able to write the same SQL without needing to code for time windows etc is more accessible and makes it feel much more like a regular database #current22
working with streaming data in dbt and snowflake. Streaming and batch nodes in the same lineage chart #current22
Who owns all of this? The analytics engineer.
🎯 Shouldn’t be talking about batch vs streaming, but what your company needs #Current22
Spicy take, but important one. *why* does it need to be real-time? What are they doing with that data? What is the business impact if we *don’t* have real-time #Current22
Sometimes though you *do* need real-time. @notamyfromdbt’s example was an airline that had a screen showing when to close the gate for a flight. A five minute SLA was ok and only what was possible with the tooling at the time - but real-time *would* have been better #Current22
Sometimes it’s not either/or - it’s both #Current22
The world is shifting. It should be about the use cases, not batch vs streaming. Analytics engineer is well placed to own this intersection. #current22
• • •
Missing some Tweet in this thread? You can try to
force a refresh
Gosh, I'd forgotten the pleasures and pains of abstract review for a conference. Pet peeves this morning:
☠️ Vendors submitting piss-poor product pitch abstracts. It doesn't take a genius to see through your abstract and look you up on LinkedIn to put 2 and 2 together. (1/n)
If you're a vendor, be open about your talk and its content. Pitching per se is not evil, but trying to hide it is. If you have useful things to say then perhaps your talk will be useful for the confernce. But be up front and honest about it. (2/n)
☠️ DAs with piss-poor abstracts. This is *literally your job*. Either bring it, or GTFO.
There are so many resources out there to help with writing abstracts (and hey, DM me and *I'll* help too, srsly), but you cannot just phone it in. It looks bad on you, and your company (3/n)
So does @MySQL Heatwave "Lakehouse" actually act as a lakehouse as defined elsewhere and write *back* to object storage through a table format? Or it's just MySQL that can also query data that's on object storage? The latter is cool of course, but the naming is puzzling me.
The press release is unclear, other than in the fact that OMG OUR BENCHMARK SHOWED WE ARE FASTER, WHO'DA THUNK IT?!! oracle.com/news/announcem…
The technical brief (oracle.com/a/ocom/docs/my…) makes note of "the HeatWave internal format" for working with external data. There's lots of mention of CSV and Parquet and magic fairies^H^H^H^H^H^H^Hmachine learning to guess at schemas.
Audience feedback at conference talks is *really* useful for speakers and organisers. It lets speakers understand what they're doing well (and perhaps what they're not). It helps organisers gauge the direction of content (more of this, less of that).
Reading these this morning makes me very proud of all the speakers at #Current22 😁
There's also some fair criticism in there that's great feedback to work with speakers and the program committee on
Looks like a fascinating set of talks at @coalesceconf#dbtcoalesce next week. I'll be firing up my 56k modem and dialling in for several of them including:
Keynote: The End of the Road for The Modern Data Stack You Know, from @jthandy and @margaretfrancis