Thread by @RachelSolnickMD

1/ Our new study @JAMANetworkOpen: Do patients evaluate black/female docs lower simply bc of race/gender? Results from vignette experiments suggest not. @basmahsaf @peyton_k @gordonkrafttodd
doi:10.1001/jamanetworkopen.2019.20511
#tweetorial #diversityinmedicine #womeninmedicine

2/ Inspiration for the study came from personal experiences as a young female ER doc frequently restating my credential to patients, increased reports of #patientbias, and a headline of a young black physician who was rejected from assisting a passenger on a plane

3/ We have seen in prior observational work that many docs report patient-initiated harassment. But in the age of patient satisfaction surveys, no clear causal estimates abt. whether physician race/gender affects these ratings.

4/ In 2 preregistered experiments, we recruited 1 online convenience sample from MTurk & 1 quota sample matched to US census demographics from @lucid_hq (total N > 3k participants). Participants played the role of a patient reporting to ER w/ belly pain, vomiting, diarrhea.

5/ Participants randomly assigned Docs of different race/gender in 2x2 factorial (Black Female, Black Male, White Female, White Male). Docs provided same diagnosis of gastro using script *written by real ED Docs*. Doc diagnoses were *contradicted* by online symptom checker.

6/ After simulated encounter, participants rated Docs using patient satisfaction scales (adopted from @PressGaney + @CMSgov HCAHPS) and confidence in Doc’s diagnosis (e.g. whether they would ask for more tests).

7/ Across both studies, we found statistically equivalent evaluation scores across ALL combinations of Doc’s race/gender, and could not reject null of no differences across treatment arms, e.g. that White Male Docs (the “control” group) received same scores as Black Female docs

8/ We use Bayesian Additive Regression Trees (BART), a #Machinelearning tool to automate the search for effect heterogeneity (e.g. did older white men rate Black Female Docs lower than White Male Docs?)academic.oup.com/poq/article/76…).

9/ BART found constant null effects (all 95% credible intervals include 0) of Doc race/gender across various subgroups of participants

10/ In short: across two pre-registered experiments that randomized physician race/gender, we find null results that appear fairly constant as a function of participant background characteristics (cc: @JohnHolbein1). #NullEffectsMatter #TeamPreciseNulls

11/ Limitations: There are over 145 million(!) ED visits/ year. Our study populations were different from ED patients in terms of less minorities, less Medicaid. But there’s no prior research to suggest these groups would be more likely to discriminate against minorities

11/Limitation: the patients most biased against Docs may fit a different covariate profile than those we sampled (mostly whites who are willing to take online surveys for $). Important Q for theory building: what does patient pop. biased in favor of white male Docs look like?

12/ Limitations: This is a simulation study, could real-life interactions/ communication impact patient confidence + satisfaction? Perhaps, but we wanted to isolate just race/ gender in a controlled environment.

13/Limitation: This was in an ER, could an office setting create a different patient evaluation? Perhaps, but by getting to an office a patient has already expressed preference by *selecting* particular doc, choosing to schedule repeated appts, etc.

14/Limitation: This was just Black versus white & male v. female physicians, could we see a different effect with other groups? Perhaps, but for this study we prioritized Black/White contrast in particular based on prior theory & qualitative/survey work.

15/Why does this matter now? CMS is in the feasibility testing stage of the ED Patient Experience of Care Survey, which will expand the reimbursement-tied HCAPS to the ED. Our findings suggest this rollout may not give white male docs an advantage in patient satisfaction scores

@jaykaplanmd 16/ Our study does NOT mean race/gender concordance is unimportant. Increasing # of Black physicians, in particular, is especially likely to have big public health benefits in US. See recent field experiment in Oakland, CA: nber.org/papers/w24787.

17/We don't find systematic evidence of #patientbias in evals, on average. This does NOT mean docs don’t experience serious, harmful bias/discrimination. This DOES happen, causes burnout, turnover. Health systems, docs, allies MUST work to create inclusive workplaces

18/Our study DOES point the research spotlight back towards investigating other sources of workplace discrimination that contribute to attrition, decreased job satisfaction in underrepresented docs such as from institutional culture/ policies, colleagues, and superiors.
/end