Thread by @FitzgeraldJack_

A new working paper for holiday reading! @peder_isager and I provide an introduction to three-sided testing, a framework for testing an estimate's practical significance. We offer a tutorial, Shiny app, + commands/code in #Rstats, Jamovi, and #Stata (🔗 below!) 1/9

#EconTwitter

Equivalence testing lets us test whether estimates are stat. sig. bounded beneath practically negligible effect size Δ (e.g., pink estimate). But estimates can be both stat. sig. diff. from zero and stat. sig. bounded beneath Δ. 2/9

Estimates can also be stat. sig. bounded outside of Δ (e.g., blue estimate). What should we conclude about estimates like these blue/orange estimates? Standard equivalence testing frameworks don't give us clear answers. We introduce researchers to a framework that does. 3/9

The three-sided testing (TST) framework combines two-sided minimum effects tests for inferiority/superiority with the two one-sided tests (TOST) equivalence testing procedure. TST can provide stat. sig. evidence that estimates are practically significant, or practically = 0. 4/9

This procedure was developed by Goeman, @aldosolari, & Stijnen (2010), who show that by partitioning estimates' parameter spaces into disjoint regions, TST controls error rates over all three of its tests w/ no penalty to power. 5/9

onlinelibrary.wiley.com/doi/10.1002/si…

@aldosolari Practical significance conclusions about an estimate can be easily inferred from double-banded confidence intervals that combine the estimate’s (1 - α) CI (e.g., its 95% CI) with its (1 - 2α) CI (e.g., its 90% CI). 6/9

@aldosolari To make things easy, we offer the ShinyTST app, a point-and-click Shiny app that tells you which test/confidence interval is relevant, provides p-values, and visualizes test results given an estimate, standard error, and SESOI. 7/9

jack-fitzgerald.shinyapps.io/shinyTST/

@aldosolari We also offer the tst() command in the eqtesting R package, the tsti command in Stata, and Jamovi code. You can visit the paper to find download instructions for all, + guidelines for implementation. We hope you find it useful! (8/9)

osf.io/preprints/psya…

@aldosolari For those on #EconTwitter, in addition to our PsyArXiv paper, we’ve also deposited a version into the Tinbergen Institute Discussion Paper Series: (9/9)

papers.tinbergen.nl/24077.pdf

@aldosolari @threadreaderapp unroll this