A new working paper for holiday reading! @peder_isager and I provide an introduction to three-sided testing, a framework for testing an estimate's practical significance. We offer a tutorial, Shiny app, + commands/code in #Rstats, Jamovi, and #Stata (🔗 below!) 1/9
#EconTwitter
Equivalence testing lets us test whether estimates are stat. sig. bounded beneath practically negligible effect size Δ (e.g., pink estimate). But estimates can be both stat. sig. diff. from zero and stat. sig. bounded beneath Δ. 2/9
Estimates can also be stat. sig. bounded outside of Δ (e.g., blue estimate). What should we conclude about estimates like these blue/orange estimates? Standard equivalence testing frameworks don't give us clear answers. We introduce researchers to a framework that does. 3/9
The three-sided testing (TST) framework combines two-sided minimum effects tests for inferiority/superiority with the two one-sided tests (TOST) equivalence testing procedure. TST can provide stat. sig. evidence that estimates are practically significant, or practically = 0. 4/9
This procedure was developed by Goeman, @aldosolari, & Stijnen (2010), who show that by partitioning estimates' parameter spaces into disjoint regions, TST controls error rates over all three of its tests w/ no penalty to power. 5/9
onlinelibrary.wiley.com/doi/10.1002/si…
@aldosolari Practical significance conclusions about an estimate can be easily inferred from double-banded confidence intervals that combine the estimate’s (1 - α) CI (e.g., its 95% CI) with its (1 - 2α) CI (e.g., its 90% CI). 6/9
@aldosolari To make things easy, we offer the ShinyTST app, a point-and-click Shiny app that tells you which test/confidence interval is relevant, provides p-values, and visualizes test results given an estimate, standard error, and SESOI. 7/9
jack-fitzgerald.shinyapps.io/shinyTST/
@aldosolari We also offer the tst() command in the eqtesting R package, the tsti command in Stata, and Jamovi code. You can visit the paper to find download instructions for all, + guidelines for implementation. We hope you find it useful! (8/9)
osf.io/preprints/psya…
@aldosolari For those on #EconTwitter, in addition to our PsyArXiv paper, we’ve also deposited a version into the Tinbergen Institute Discussion Paper Series: (9/9)
papers.tinbergen.nl/24077.pdf
@aldosolari @threadreaderapp unroll this
Share this Scrolly Tale with your friends.
A Scrolly Tale is a new way to read Twitter threads with a more visually immersive experience.
Discover more beautiful Scrolly Tales like this.