Thread by @jillzzy

WAP
a collabo feat @spbail and @tayloramurphy via @coalesceconf
because....of course rap and data pipelines intersect 🎵🎶

Data in this warehouse
there's some data in this warehouse
there's some data in this warehouse
there's some data in this warehouse

I said, certified geek
Seven days a week
Write-audit-publish
Make that cron-job game weak, woo

Yeah, yeah, yeah, yeah
Yeah, you messing with some write-audit-publish
Bring some SQL and a job for this write-audit-publish
Query everything you’ve got with this write-audit-publish

Beat it up, an’lysts, need a large
Extra charge and extra shards
Put this query in the database
Stack them bars, make a pivot chart

Hop to Mode, I wanna try
To produce this viz I found online
Pull master, hundo commits behind
Data is dirty, uuuh, code is DRY
Transform it up, find a surprise
Let's change roles, materialize

I want you to shove that JSON blob
Right in this variant column

Apache beam? In your dreams
Drop that schema, it ain’t clean
I don’t batch, I don’t stream
But I got a regex to match that string (ayyy, ayyy)

Extract me, batch-load me, use dbt transform me
Quick, run it ‘fore this lake makes a fool of me
I know where to put it, never know where it will be queried
Run a CTE ‘fore I have sub-queries running me

Test your shit, check-in git
Ask for a ship while you push that fix (while you push that fix)
You really ain’t never gotta pay ‘em for a thang
Open sourced it from beginning

Now get your maps and your keys
For this write-audit-publish
Bought Census just for syncing
Of this write-audit-publish
Pay my subscription just to query
On this write-audit-publish
Now make it stream if you wanna
See some write-audit-publish

Look, I need a good linter, I need a sound broker
I need to upgrade Sinter, this UX is mediocre
Not a row store, I need columnar
With an optimizer, MPP game over

Sequoia money, that’s where the market headed
Talkin A rounds to B, no debt
Will they go for C, hold on a minute
Build community, now time to expand it

I dont wanna quit, I want some help
I wanna drill, I wanna poke
I want you to trim() that runaway string
That thing gon make the database choke

My merge game is fire, improve me review me
My coding is dry, but not without coffee
I filter that thang where the join is behind me
I split_part and ilike and now you tryna find me, woo

Pair program, I’m a geek trick, do not repeat this
Switch my branch, you sure you want to see this
Stash it in the logs, cuz its troubleshoot’n’ season
Catch this epic fanout, aww, brutal -- its cartesian

In the pipeline, we can’t transform nada
If it flip my bool, might need to refactor
Big E stand for big extractor
I could make it load before we see some drama

If queries hang, we can’t explain
Leading comma only, legible, sane
If you trail and ask me to review
When I commit it, I won’t refrain

Ah (data in this warehouse)
Yeah, yeah, yeah
Yeah you messin’ with some write-audit-publish
Bring some SQL and a job for this write-audit-publish
Query everything you got for with write-audit-publish

Now from the top, make it drop
That’s some write-audit-publish
Now get some SQL and a job
That’s some write-audit-publish

I’m talking WAP, WAP. WAP
Write-audit-publish
Curly brackets in the clause
That’s some write-audit-publish

sooo, serious question

should @spbail and @tayloramurphy actually lay this track down?

#wap #wap #wap #analyticsengineering #writeauditpublish #pipelines

@threadreaderapp unroll please

WAP write-audit-publish now has it's very own git file, in a lyrical repository 🤩

github.com/voiceoftheanal…
🔥

magical things happening at @coalesceconf ✨
It's not too late to JOIN! getdbt.com/coalesce

#dbt #analyticsengineering and beyond 🚀