How is #DuckDb going to be used inside a company? My head can't think beyond it being used for local development. Can it really replace a DWH without being distributed?
If it does go beyond local, I do have a few things to ponder upon, 🧵
1. Search and Discovery: Where are users going to find artifacts inside the DuckDB cluster (eventually)? We'll need a catalog interface to surface this portion. The metadata of the artifacts in the data model (database, table) needs to be consistent across the cluster.
2. How is privacy handled? What if someone downloads sensitive data onto their laptop and it is stolen thus opening up a data breach? My paranoia comes from working on Data infra for GDPR, and CCPA. Access control should be implemented and enforced.
) earlier this week. Let's do @Twitter's tech and Data
This company might be popular for its platform at large but there were a lot of data industry pieces that are worth calling out. Here is a 🧵
Streaming Processing: Storm! This was streaming before streaming. It paved the way for a lot of streaming processing systems. Event processing at the earliest. blog.twitter.com/engineering/en…