Prukalpa Profile picture
28 Mar, 9 tweets, 13 min read
Modern #data platforms are emerging as the answer to the holy grail: creating a truly data-driven organization. With this, “how” we use data has changed.

Many vendors today brand themselves as the be-all and end-all... but this isn’t true. 🙅
It’s impossible to work today with a single modern data platform from one vendor! A modern #data platform is a collection of tools and processes. 🧰

In this thread, I’ll break down what a modern data platform means in practice today. 2/n
Today, data platforms have basic building blocks that look something like this. 👇 3/n…
The center stone is a modern #data storage and processing layer. This can be 👇

Data Warehouse 👉 @SnowflakeDB, #Redshift, #BigQuery

Data Lake 👉 Object storage (#AWS S3/ #GCS) + Query Layer (@trinodb, @starburstdata, @dremio) + Metadatastore (Glue, @HiveData) or @databricks
The #Data Ingestion Layer: Getting data into the data warehouse/ lake from dozens of sources and ingest it into the core data storage layer.

#SaaS tools 👉 @fivetran, @HevoData, and @stitch_data and also the good old custom #Python jobs and Airflow DAGs.
Data Transformation: Taking raw data and converting it into the final “golden layer”.

With data warehouses 👉 @getdbt (SQL first), @matillion (no code)

With an orchestration engine 👉 @ApacheAirflow + #Python, R, or #SQL
While BI tools have been in the picture for a while, the tools today are more “self-served” to fit within a larger modern data platform.

Visualization/ BI 👉 @tableau, @LookerData, @Sisense, @metabase, @apachesuperset

#BusinessIntelligence #dataviz
As #metadata itself becomes big data, there are Metadata Management/ Data Catalogs built on the premise of embedded collaboration (think @github, @figmadesign, @SlackHQ, @NotionHQ, @Superhuman) 🤝

#SaaS 👉 @AtlanHQ
#OpenSource 👉 @amundsenio, #Datahub

#BigData #datacatalog
Data Access Governance: Tools that can act as an entitlement engine to apply #privacy and security policies across the #data stack. 🔐

#SaaS 👉 @ImmutaData, @okerainc, @privacera,

#OpenSource 👉 @ApacheRanger

#datagovernance #dataprotection

• • •

Missing some Tweet in this thread? You can try to force a refresh

Keep Current with Prukalpa

Prukalpa Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!


Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @prukalpa

4 Jun 20
Incredibly candid chat with @gauravmunjal in @_surgeahead about THE most important thing: How to grow yourself 10x-100x as a Founder.

Some learnings that are applicable to every single ambitious person out there👇
#1 Move the needle: Do something "needle moving" everyday. Relook at your task list everyday, every week and measure if you've only been occupied with 'base tasks or have done something needle moving
#2 Note taking: Create a personal learning dashboard filled with notes. Take notes during every conversation, every book you read, every video you watch. Single greatest personal hack.
Read 9 tweets
1 Jan 20
1/ It has been over 18 months since we started building @AtlanHQ in stealth – going back to day zero, iterating and finding Product Market Fit.

Last quarter, our usage graph started going 📈🚀!

Here are our learnings from 2019 on the road to PMF 👇
2/ A great framework to break PMF journey is into 3 stages:
1⃣Problem- Value Fit: Are the problems you're solving important?
2⃣Problem - Solution Fit: Can your envisaged solution solve the problem?
3⃣ Product - Market Fit: Find a set of similar buyers - "beach-head" market
3/ We tried many experiments to find problem-value-solution fit quickly (launch landing pages, A/B test ads, secondary research etc) and finally found directly reaching out to target customers on LinkedIn (at least for B2B) as the best way!
Read 17 tweets
2 Jul 19
So excited to share the launch of @atlanHQ today after 18 months of working under the hood. #HumansofData, Welcome Home!
@AtlanHQ 2/ Also excited to announce our pre-series A fundraise of INR 18 crores and welcome @WaterbridgeVentures! They've been working with us in our stealth journey for the past 15 months and been great partners! cc @mkheterpal…
3/ We are living in the world of the "consumerisation of enterprise".. In 2019, Slack, Github, Zoom, Quip, Superhuman are all amazing examples of why human experience matters more than ever.
Read 13 tweets
15 Jun 18
1/ Recently been speaking to some early stage founders and realised that most folks go through same hiring & talent challenges. Here are some reflections and learnings:
2/ Before we started hiring at SocialCops, every book we read said "hire people better than you" - and we took it at face value. So we called all the smart ppl we know and convinced them to join us. ALL of them said NO!
3/ This is when it got tempting to compromise and easy to become desperate. We thought - we have so much work to do & this person interviewing with us might not be perfect but isn't THAT bad!
Read 18 tweets
1 Jan 18
My reflections and learnings from 2017: Surround yourself with people smarter than you- I only really understood the meaning of this in 2017. Reach out to people whom you respect, and have a list of questions for them. It will give you perspectives you never dreamt of. (1/n)
Don't undervalue the time you need to think, learn and grow - it is probably the most imp time you'll spend. As a founder/leader - it is easy to book your day in meetings and feel “busy” - frankly, if someone else can handle that important meeting, let them! (2/n)
You don’t need to have a “brand” of leadership - Lead the way someone wants to be led, not the way you want to lead. Your only role as a leader is to bring out the best in the other person. (3/n)
Read 17 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!