🧹Tidier.jl 0.6.0 is available on the #JuliaLang registry.
What’s new?
- New logo!
- distinct()
- n(), row_number() work *everywhere*
- `!` for negative selection
- pivoting functions are better
- bug fixes to mutate() and slice()
If you use distinct() without any arguments, it behaves just like the #rstats {tidyverse} distinct().
It checks if rows are unique, and returns all columns just as you would expect.
If you use distinct() with arguments as shown here, then it returns all columns for unique values of the supplied column.
This is slightly different behavior than {tidyverse} distinct(), but I kind of like it. Can easily pair this with select() to mimic dplyr behavior.
Another thing that’s new is the helper function n().
While seemingly simple, implementing this was fairly difficult. When used inside of summarize/summarise, it behaves just like DataFrames.jl’s nrow() function.
So far, so good.
However, n() — and its counterpart row_number() — also work inside of mutate(), where nrow() from DataFrames.jl isn’t as straightforward to use, particularly inside of an expression (e.g. n() + 1).
n() provides a standard interface from all functions.
You can even use n() inside of slice().
To select the last 2 rows of a dataframe?
slice(n() - 1 : n()) — notice the order of operations is slightly different from R bc the `-` takes precedence over the `:` so no need for extra `()`.
Otherwise, this is exactly like R tidyverse.
And when the update says that n() and row_number() work *everywhere*, it’s really true.
You can even use row_number() inside of filter() to mimic the functionality provided by slice(), just like in R.
When an outcome influences a predictor, it’s “outcome leakage.” But what about when a predictor influences an outcome?
With @AkhilVaidMD @girish_nadkarni et al, we simulated what happens when a model predicts a bad outcome, but then you intervene to prevent that outcome.
If you evaluate such a model *after* it has been linked to a clinical workflow, the model’s “apparent” performance will look worse.
People who were supposed to experience the outcome didn’t experience it (bc you prevented it!)
This matters most when interventions are effective.
But what if you update a model? Or if you build a 2nd related model after the first one is implemented? Or if you simultaneously implement 2 models where the intervention for one affects the predictors of the other?
You end up with scenarios where the performance looks worse.
🧹Amazing progress on TidierPlots.jl by @randyboyes.
What’s new?
1. It looks *just* like ggplot2 now - nearly all macros converted to functions.
2. Thanks to #JuliaLang’s multiple dispatch, you can add plots together using `+` OR use pipes.
3. ggsave()
4. Works with Pluto.jl
TidierPlots.jl is getting to be crazily feature-complete, even supporting `geom_text()`, `geom_label()`, and faceting.
And if you’re a #JuliaLang user of AlgebraOfGraphics, what’s super cool is that you can literally mix and match TidierPlots code with AlgebraOfGraphics code because TidierPlots uses AoG under the hood.
In earlier single-center study @umichmedicine, our paper and accompanying editorial framed our AUC 0.63 as a failure of “external” validity. The result was somewhat surprising bc other studies reported higher AUCs/sens/spec.
One interesting thing is that lag() and lead() take in a vector and return a vector (similar to ntile).
This means that these functions *should not* be auto-vectorized. So in addition to re-exporting, they are included on the package’s do-not-vectorize list.
In the near future, we will provide a mechanism to edit the package’s do-not-vectorize list of functions.
For frequently used functions, this means you won’t have to use the tilde-prefix notation to call them.
For years, I’ve been trying out different non-tidyverse implementations of tidyverse. It’s fun seeing folks mold languages to run analysis code inspired by it.
If you like screenshots of code, come along for a visual tour.
Let’s start w/ R.
If you thought that one tidyverse was enough for R, you would be wrong.
There are 2 fully independent re-implementations: {poorman} and {tidytable}.
{poorman} is powered by base R only - no dependencies! It’s a great pkg to use with binder/CI workflows.
{tidytable} has a similar premise, except it relies primarily on {data.table} and {tidyselect}.
While it’s similar to {dtplyr} in some ways, the syntax is even cleaner bc you don’t need to declare your data.table or use collect() to get the results.
When we link an intervention to a model threshold (eg alerts), we often worry about overalerting.
Overalerting can take on multiple forms. Either there are too many alerts bc many alerts are wrong. Or, there are too many alerts bc we lack capacity to act even if they are right.
Consider this: a model scoring 10 patients. Using a threshold of 20%, you identify 4 out of 5 patients needing ICU-level care.