Thread by @MedEdModels

#MedTwitter – we’re excited to share with you a new #MedEd Model! This #Tweetorial is on validity.

Validity describes how justifiable the decisions are that result from a test, whether we are referring to an exam, a rating scale, or a clinical diagnostic.

If you prefer to review this material in a text-only format or if you will be using a screen reader, you can use the following link to access a text document: bit.ly/validity-mem.

Let’s get started!

Here are the objectives of this #Tweetorial.

We’ll focus on describing current validity discourses, then list the components of a validity argument, and wrap up by applying validity to #MedEd.

Let’s review 3 validity discourses in #MedEd:

1. Validity is a property of a given test
2. Validity is an argument supporting a proposed interpretation & use of assessment data
3. Validity is a social imperative that considers individual & societal consequences of assessment

Let’s dive deeper into the second discourse: validity as an argument. First, we need to understand what we mean by “argument.”

The core components of an argument have been described in the argumentation literature by Toulmin in a 3-part framework:
1. Claim
2. Warrant
3. Grounds

Now, let’s consider what it would mean for validity to be presented as an argument.

Check out the image to see how the claim, warrant, and grounds connect to validity in #MedEd.

In other words, whenever we use an assessment to make a decision in #MedEd, we need to 1) articulate the inferences that justify using that assessment for the proposed decision and 2) generate evidence supporting those inferences.

This matters, because if we use an assessment to make a decision *without* ensuring we have evidence supporting our inferences, we could end up making misguided (or even harmful) decisions in #MedEd.

Making a plan to gather evidence for a validity argument can be daunting. Luckily, Kane’s Framework can help us!

Kane’s framework identifies four inferences in using a test for decision-making:
- Scoring
- Generalization
- Extrapolation
- Implications

By applying Kane’s Framework, we can uncover the weakest assumptions in a validity argument, and then gather evidence to explicitly/robustly address those assumptions.

If the evidence is refuting, we revise our plan.

What constitutes evidence for each of the four inferences?

We’re glad you asked, #MedTwitter! Check out the table for some ideas/examples of quant & qual validity evidence!

Now let’s practice applying the concept of validity to #MedEd.

Check out the prompt below and share your answers by replying to this tweet. We can’t wait to hear your thoughts!

We’re super lucky for this #MEM to have support and a comment from @Midwest_MedPeds!

Thank you for suggesting this topic (mededmodels.com/submit-ideas) & contributing, Dr. Kinnear!

Here are the take homes from today’s #Tweetorial.

What did you learn from this?

We’d love to hear what concepts stood out and what you’d like to learn more about!

#MEM is just a primer! To Learn More (#2LM), check out these scholars:
@CookMedEd
@chsto8
@meredithyoung1
@rbrydges
@sginsburg1
@HatalaRose
@Midwest_MedPeds
@DrDanSchumacher
@erikwdriessen
@LaraVarpio

Particularly in:
@MedEd_Journal
@advhlthsciedu