Albert Rapp Profile picture
May 19, 2023 26 tweets 10 min read Read on X
Ever heard of logistic regression? Or Poisson regression? Both are generalized linear models (GLMs).

They're versatile statistical models. And by now, they've probably been reframed as super hot #MachineLearning.

Brush up on their math with this thread. #rstats
Let's start with logistic regression. Assume you want to classify a penguin as male or female based on its

* weight,
* species and
* bill length

Better yet, let's make this specific. Here's a dataviz for this exact scenario. It is based on the {palmerpenguins} data set. Image
As you can see, the male and female penguins form clusters that do not overlap too much.

However, regular linear regression (LR) won't help us to distinguish them. Think about it. Its output is something numerical. Here, we want to find classes.
How about trying to predict a related numerical quantity then? Like a probability that a penguin is male. Could we convert the classes to 0 and 1 and then run a LR?

Well, we could. But this won't give us probabilities either. Why? Because predictions are not restricted to [0, 1] Image
I suspect you're REALLY determined to use LR. After all, what have you learned ordinary least squares (OLS) for if not for using it everywhere?

So, what saves you from huge predictions? That's the glorious logistic function (applied to LR's predictions). Image
I've applied this strategy to our data to "predict" probabilities. Then, I used a 50% threshold for classification (Note that 50% is in general not a good threshold but that's beside the point here.)

So, does this yield good results? Have a look for yourself. ImageImage
The predictions for male and female penguins overlap quite much. This leads to many incorrect classifications.

Not bueno. At this point, you may as well have trained a model that answers "Is this a male penguin?" with "Nope, just Chuck Testa".
Our classification is bad and I have hopefully convinced you that OLS isn't the way to go here. What now?

Well, it wasn't all bad. The idea of linking a desired quantity (like a probability) to a linear predictor is actually what GLMs do.

To make it work, take a step back.
Usually, we model our response variable Y by decomposing it into

1️⃣ deterministic function f(X_1,..., X_n) dependent on predictors
2️⃣ random error term

Thus, regression is nothing but finding a fct describing the AVERAGE outcome. A change in notation makes this clearer. Image
In linear regression, this deterministic function is given by a linear predictor. It depends on a parameter beta (that's a vector here). Image
Alright, we've emphasized that we're really trying to model an expectation. Now, think about what we're trying to predict. We're looking for probabilities, are we not?

And do we know a distribution whose expectation is a probability? Bingo! We're thinking about Bernoulli.
Therefore, let us assume that our response variable Y is Bernoulli-distributed (given our predictors).

And now we're back with our idea to link the average outcome to a linear predictor via a suitable transformation (the same one as before). This sets up our model. Image
You're thinking we've tried this already, aren't you? How will we get different results?

Isn't this new setup just semantics? Theoretic background is useless in practice, right?

(I've actually heard someone say that to a speaker at a scientific workshop. A shitshow ensued 😆)
Previously, we used the OLS estimator to find the linear predictor's parameter beta. But with our new model setup comes a new way of estimating beta.

Take a look. Compare the results of using the OLS estimator with what we get when we "maximize the likelihood". ImageImage
Much better results, right? And that's despite having used the same 50% threshold once I predicted probabilites.

This means that maximizing the likelihood delivers a way better estimator. Let's see how that works.
The likelihood function is the product of the densities of the assumed distribution of Y given the predictors (here Bernoulli) . This makes it the joint probability of the observed data.

We find beta by maximizing this function or equivalently (but easier) its logarithm. Image
How do we find this maximum? By using the same strategy as for any other function that we want to maximize:

Compute the first derivative and find its root.

In this context, this derivative is also known as score fct. Image
Here, finding a root is not easy as no analytical solutions exist. Thus, we'll rely on numerical methods.

A well-known procedure is Newton's method. In each iteration it tries to get closer to a function's root by moving along its gradient. Have a look at this GIF from Wikipedia
Congrats! You've brushed up on ONE example of GLMs, namely logistic regression. But GLMs wouldn't be general if that were all.

Depending on the assumed distribution and the function that links linear predictor and expectation, GLMs have many names. Here's one more.
Poisson regression is a GLM which assumes that Y follows a Poisson distribution (who would have seen that coming?). A suitable link function is the exponential fct.

This model is used when you try to estimate count data and the formulas looks very similar to logistic regression. ImageImageImage
This begs two questions 🤔

1️⃣ Does this work with any distribution?
2️⃣ How in the world do we choose the link function?

The secret ingredient that has been missing is a concept known as exponential families. It can answer BOTH questions. Isn't that just peachy?
Exponential families are distributions whose density can be rewritten in a *very* special form.

Honestly, this curious form is anything but intuitive. Yet, it is surprisingly versatile and the math just works. If you ask me, that's quite mathemagical. ✨ Image
It probably doesn't come as a surprise that Bernoulli and Poisson distributions are exp. families (see table).
But what may surprise you is this:

The function b plays an extraordinary role: Its derivative can be used as link function! In fact, that's the canonical choice. Image
Now you know GLMs' ingredients: Exponential families and link functions.

You can find more example distributions on Wikipedia en.wikipedia.org/wiki/Exponenti…

Also, an interesting perspective on what makes exponential families so magical can be found on SE: stats.stackexchange.com/questions/4118… Image
Bam! We made it. And we've covered

* two popular GLMs and
* mathematical foundations of GLMs

Let me know if you liked this thread by liking the start of the thread below ☺️

And if you want to see more of my content, feel free to follow @rappa753.

See you next time 👋
If you liked this post, you may enjoy my 3-minute newsletter too.

Every week, my newsletter shares insights on
- R & dataviz,
- Shiny and web dev

Reading time: 3 minutes or less

You can join at
alberts-newsletter.beehiiv.com/subscribe

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Albert Rapp

Albert Rapp Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @rappa753

Sep 9, 2023
Three steps to use color in your title instead of wasting space on a huge legend. Image
1 // Wrap your subtitle into <span> tags

These span-tags are HTML notation for inline text. So in principle, adding them should change nothing.

But as you can see, it does have an impact.
Image
Image
2 // Enable HTML notation

The problem is that ggplot does not know that you want to use HTML notation.

So, enable that with element_markdown() from the {ggtext} package in theme.

This will render the span-tags instead of displaying them as text:
Image
Image
Read 7 tweets
Aug 26, 2023
Paired bar charts suck at comparing values. The only reason they're used all the time is because they are easy to create.

But there are better alternatives that are just as easy.

Here's how to create 4 better alternatives with #rstats. Image
0 // Where's the code?

The code for all plots can be found at

This thread walks you through the code quickly.albert-rapp.de/posts/ggplot2-…
1 // Dot plot

Instead of using bars next to each other, why not points on the same line?

Makes comparison suuper easy.

And it takes only a geom_point() layer. Dead-simple, right?

I think it's even easier to create than a paired bar chart.
Image
Image
Read 15 tweets
Aug 19, 2023
R makes it dead-simple to use some of the most effective dataviz principles.

Here are six principles that are so easy that any ggplot beginner’s course should teach them.
1 // Make sure your labels are legible

Too many plots use waaaay too small texts.
With ggplot, it just takes one line to fix this.

Img 1: Way too small fonts & unclear labels
Img 2: Fixed with labs() and theme_gray(base_size = 20)
Img 3: Full code

Image
Image
Image
2 // Use a minimal theme

As a rule of thumb, you should minimize everything that could potentially distract your audience.

That’s why I usually recommend to use a minimal theme: Just use `theme_minimal()` instead of `theme_gray()`. Image
Read 9 tweets
Jun 17, 2023
Need to extract days, months, years or more from time data?

Don't compute them all manually with {lubridate}. That's way too tedious.

The {timetk} package has a nice function that does all the heavy lifting for you.

LEFT: {lubridate} workflow
RIGHT: {timetk} workflow
#rstats ImageImage
BONUS: Maybe you don't want use all of the stuff that {timetk} computes for you.

Here's a simple function that extracts only the parts you want.

All of the code can be found on GitHub at gist.github.com/AlbertRapp/2c9… Image
Also, shoutout to @EatsleepfitJeff for teaching me about this function from {timetk} ☺️
Read 4 tweets
Jun 10, 2023
Everybody loves colors but only few know how to use them well.

With the right guidelines, using colors becomes super easy.

Let me show you how to implement these guidelines with ggplot 🧵
#rstats
Anyone can create a stacked bar chart with ggplot.

But that can end up in a colorful & messy plot.

Let's implement a couple of guidelines from this datawrapper blog post to level up our color game blog.datawrapper.de/10-ways-to-use… Image
The key is to reduce the amount of colors and leverage the `alpha` aesthetic as well. Image
Read 11 tweets
Jun 7, 2023
Sometimes people ask me if I can do one-on-one R tutoring.

Sure I can. But then my hourly rate applies. And there are many amazing *free* resources. Want to try them first?

Here are a few that I recommend. #rstats
1 // Yet Again: R + Data Science

Find it at yards.albert-rapp.de

I'll start with one of my own bc I assume that you like my style (otherwise why ask me?)

Beware though: YARDS is a graduate-level course that I taught for math students w/ a bit of programming experience. Image
2 // R for Data Science

To me this book is like the R bible. It introduced me to the so-called tidyverse and taught me much of what I know.

This one starts out slow and is really beginner-friendly

r4ds.had.co.nz
Read 11 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

:(