Your company and your projects can get considerable value out of traditional machine learning methods, *without* giant language models or massively-scaled architectures —
just like you can still exercise, or hike, or rock climb, even if you aren't prepping to conquer Everest.
Also: traditional models are generally cheaper to implement; more straightforward to explain and understand; and easier to maintain.
Giant models are exciting, from a research perspective; & can unlock capabilities. But you don't *have* to use them, if a smaller hammer works. 🤷♀️
It is shocking to me that there is still such a wide gap between the deep learning community and the traditional data analysis / PyData community. 😞
We are all just nerds trying to use data to understand the world, and build software systems to answer questions/ solve problems.
PS: if you're part of the PyData contingent, and you're on the fence about trying machine learning because you think you might not have the background, or because it seems too challenging:
We all struggle. You are *more than capable* of building interesting and useful projects.
My biggest 💔 is that all the bright, curious humans who *could* be doing machine learning don't feel empowered, or don't have resources.
"Software experts are—of necessity— comfortable with high cognitive friction. They pride themselves on their ability to work in spite of its adversity.
Normal humans, who are new users of these products, lack the expertise
to judge whether the cognitive friction is avoidable."
"High cognitive friction polarizes people: it either makes them
feel frustrated & stupid for failing, or giddy with power at overcoming the extreme difficulty.
These people either adopt cognitive friction as a lifestyle, or
they go underground and accept it as a necessary evil."
"You can predict which features in any new technology will be used & which won’t. The use of a feature is inversely proportional to the amount of interaction
needed to control it.
In other words: if a feature is useful & requires no work to employ, it is used 100% of the time."
TIL that Claude Shannon's revelation that 0s and 1s could convey information was a *master's thesis*, and that he was only twenty-one. 😲
"He gave engineers the conceptual tools to digitize information and send it flawlessly (or, to be precise, with an arbitrarily small amount of error) -- a result considered hopelessly utopian up until the moment Shannon proved it was not."
"Wizard-bearded physicist" is a profession and a vibe I never knew I needed until this moment. 🔮💫
"I get taught life lessons all the time. Probably the most important one is that people will tell you [that] you can’t do something, and you have to ignore them because you can."
"Are you working to become a good ancestor -- to make something truly interesting -- or are you just working to make money?
Because you can make money by burning villages down and rebuilding them; that’s a good way to make money, but it’s not a good way to be a good ancestor."
"If you don’t contribute, don’t pay attention and you don’t give back then it can go away in the blink of an eye...
And so I’m rededicated to this notion of no: if you want a democracy it doesn’t mean you live in a democracy.
College had always been very grad-school-focused, for me: do the NSF REUs, do the industry internships, take the GRE, get a PhD, be lady Carl Sagan.
But halfway through senior year, my mom got very sick (heart issues); and I needed to get a big-girl-job to support us.
Role: developing geophysics plug-ins & geoprocessing scripts at Chevron.
That job isn't something that I would have selected, by a long mile—but it let me experiment with machine learning, with data science, and with tools like Spark, on massive amounts of accelerators/compute.
Chevron also paid for me to go to grad school, at night (carbonate geology, computer science), and offered a flexible every-other-Friday-off schedule...
...which meant that I had time to go to conferences, contribute to open-source, found a PyLadies chapter in Houston, more.
Hey, internet: just in case you're not one of those engineers who hears "Hey, compilers!" and comes running delightedly, here's a thread on why I think #MLIR is going to transform the machine learning industry:
We're seeing an explosion of domain-specific accelerated hardware (yay!).
@GoogleAI's TPUs are an example of this, sure; but you also have @Apple's specialized chips for iPhones, @BaiduResearch's Kunlun, and literally *thousands* of others.
Developer-facing tooling is essentially non-existent (or quite bad, even for TPUs); many models are built, and then found to be impossible to deploy to edge hardware; etc.
You also have very rad, but specialized types (e.g., bfloat16).
had a really odd dream that I co-founded a startup
pitch: applying transfer learning concepts to education
(ex: if you're an engineer who is adept at natural language processing, what additional machine learning concepts would you need to master to solve computer vision tasks?)
🧠 we used @KhanAcademy data to train the models (zero idea how imaginary-startup got access to it, but 🤷♀️)
the dream ended with our acquisition by @Coursera, who eventually seized complete control of the undergraduate (101- and 201-level) and enterprise education markets 💪
learning paths were also personalized for each user
so if Billy learned best when reading, then trying a hands-on activity, then a video + another hands-on activity, that was his path
but if Maria could grok concepts with only a hands-on activity, that was her curriculum design