Looks like a fascinating set of talks at @coalesceconf#dbtcoalesce next week. I'll be firing up my 56k modem and dialling in for several of them including:
Keynote: The End of the Road for The Modern Data Stack You Know, from @jthandy and @margaretfrancis
Money, Python, and the Holy Grail: Designing Operational Data Models, from @bennstancil
I realise I'm at risk of being a total fanboi, but Benn's talk at #Current22 was great and with a punny title like this how can I not put this on my agenda :)
Particularly looking forward to this talk from @JayPeeDevlin and Sydney Burns, since a few years ago this wouldn't even have been a discussion so see how things have changed
Are we going to have batch and streaming forever, or will they converge? @esammer says at the heart of systems lambda arch will go away and kappa will eventually win out. Once in DW perhaps batch will remain for its familiarity to analytics engineers.
@notamyfromdbt - Microbatching gets used to simulate streaming but with same toolset for familiarity, but it doesn’t scale
Dan Sotolongo at #current22: RDBMS and SQL have stood the test of time. Sets the scene for stream processing by covering core concepts of tables and steams
#current22 handling event time joins in SQL using functions.
The next problem is making sure we have all the data. It’s watermarks, but not really