Thread by @charles_irl

the final video for the @weights_biases Math4ML series, on probability, is now up on YouTube!

@_ScottCondron and I talk entropies, divergence, and loss functions

🔗:

this is the final video in a four-part series of "exercise" videos, where Scott and I work through a collection of Jupyter notebooks with automatically-graded Python coding exercises on math concepts

read more in this 🧵

https://twitter.com/charles_irl/status/1457840021772259332?s=20

each exercise notebook has a corresponding lecture video.

the focus of the lectures is on intuition, and in particular on intuition that i think programmers trying to get better at ML will grok

for linear algebra, we take a "programmer's view": arrays are functions that operate on arrays -- loops, parallelization, and higher-order functions all appear

https://twitter.com/charles_irl/status/1348720290570792960

for calculus, the focus is on approximation -- on using gradients as a "good enough" answer -- rather than on dynamics (as in physics) or limits (as in analysis)

https://twitter.com/charles_irl/status/1349093279111868416

for probability, we cover the importance of unlikely events and the utility of log-probabilities, which can be understood as a quantification of the commonsense notion of "surprise"

https://twitter.com/charles_irl/status/1349456179483668480

these are my own, hard-won intuitions for these topics, honed by trying to program learning machines for a decade with more chutzpah than formal mathematical training

i hope they are as useful for others as they have been for me!