Thread by @rnsoicher

Next up for me at #SIRC2019 - Implementation Outcomes and Measures breakout session.

1st up: Aaron Tierney speaking on his work from Stanford - Advancing evidence synthesis from effectiveness to implementation - recs for integrating implementation measures into evidence review. #SIRC2019

Project trying to - (1) enhance usefulness of evidence synthesis projects and (2) start conversation about standardizing reporting of implementation factors in future intervenion & evidence synthesis projects. #SIRC2019

Operationalized Terms: (10 of them!) acceptability, adoption, appropriateness, feasibility, fidelity, implementation cost, intervention complexity...from Proctor (2011) and the REAIM framework. #SIRC2019 (I got 7, not bad, hahaha)

29 studies of physician communication skills were chosen for a feasibility review - RCT or controlled observational design. 2 reviewers + consensus reviewer to determine if the measure was present, where it was described, quite/supporting evidence. #SIRC2019

Ex: for acceptability, looked for patient satisfaction and provider satisfaction with the intervention. Excerpt from a paper - "Some team members did not agree to participate in the training (as a result of a lack of time or interest" -- #SIRC2019

Suggestions for future projects - include abstraction for implementation factors in future evidence synthesis projects. Hope for standardized structure for reporting implementation factors in intervention trials w/stricter defs of measures.

Also - only 24% of the studies had info on sustainability so future studies need to address this more. #SIRC2019

Up next: Taren Swindle on what fidelity data DON'T say - types of adopters and resisters. @taren_swindle #SIRC2019

@taren_swindle The intervention is WISE - a nutrition intervention used in early education. Improves educator knowledge and improves children's dietary habits. Components are: use of mascot, role modeling, hands-on exposure (educators involve children in lesson) #SIRC2019

@taren_swindle But even after training educators, still not full implementation. Identified these barriers/facilitators: leadership support, capacity & climate for change, curring board reminder, educational resources and incentives. These are stakeholder selected. #SIRC2019

@taren_swindle *Cutting board reminder is an implementation strategy (not a barrier, sorry). #SIRC2019

@taren_swindle From a hybrid type 3 trial testing stakeholder strategies, sites are using either enhanced or basic implementation. About 300 children in each condition. Quantitative fidelity assessments quarterly. Observers are 85% reliable. #SIRC2019

@taren_swindle Formative evaluations help even more. Targeting lagging implementers early and randomly selected those struggling for qualitative interviews. Response types classified as attitude, fidelity, and adaptations. #SIRC2019

@taren_swindle Attitude of educator toward intervention matters. Either active or passive in holding that attitude. Levels of fidelity (low, moderate, high). Adaptations are fidelity consistent or inconsistent. #SIRC2019

@taren_swindle Used these characteristics to classify educators as they implement the intervention. About 35% of sample is enthusiastic. Hard resisters are about 24% of educators. Soft resisters have passive attitude, low to moderate fidelity (17%) #SIRC2019

@taren_swindle Educators with active positive attitude, low fidelity, fidelity inconsistent adaptations. "Over-Adapting Adopters" #SIRC2019

@taren_swindle Enthusiastic adopters get it and want to help the kids. Resisters aren't shy about not wanting it and not having time. Soft resisters aren't against the intervention, but they're not ready to engage. These educators feel overwhelmed. #SIRC2019

@taren_swindle Over-adapting adopters used puppet to shame the kids into eating new foods. Obviously fidelity-inconsistent. #SIRC2019

@taren_swindle High agreement on these characterizations again in May (initial assessment the previous Nov). And as people shifted categories, it was usually in the positive direction. This also led to fidelity increases as researchers understood the educators better. #SIRC2019

@taren_swindle Products for Future: checklist of attitude cues, adaptation examples (consistent and inconsistent). Developing front-line responses tailored to type of educator. Encourage the hard resisters and build relationship with them. For soft resister, work on buy-in, increase motivation

@taren_swindle With over-adapting adopters - use their enthusiasm as leverage to do goal-setting. Enthusiastic adopters - don't let them go unrecognized. Encourage them, reward them, connect them with struggling classrooms. #SIRC2019

@taren_swindle Conclusions: Avoid forming impression from quantitative data alone. Construct more informative with qualitative data. In a low-resource setting, standardize tailoring of facilitation. Compare class-informed facilitation to standard. [e-mail] if you have questions!

@taren_swindle Next up: Dr. Zarnie Khadjesari discusses systematic review on implementation outcome instruments used in physical healthcare settings and their measurement properties. #SIRC2019

@taren_swindle Review started as a complement to Proctor et al. (2011) that looked at outcomes for imp res in mental health care settings. Used same definition - "effects of deliberate and purposive actions to implement new treatments, practices, and services"

@taren_swindle Aim of this systematic review: identify, appraise, and determine quality & usability of quantitative implementation outcome instruments in physical healthcare settings. Protocol is published! bmjopen.bmj.com/content/7/10/e…

@taren_swindle Inclusion criteria: psychometric studies in physical healthcare settings with the 7 implementation outcomes (not fidelity). Fidelity measures tend to focus on a specific treatment and aren't generalizable. #SIRC2019

@taren_swindle To assess methodological quality of each study, COSMIN guidelines (cosmin.nl). Psychometric quality: Contemporary Psychometrics checklist (new criteria designed by King's College London's Psychometrics Lab). Usability - # of items. #SIRC2019

@taren_swindle For more info on ConPsy measure, visit bit.ly/2kNaVl9 #SIRC2019

@taren_swindle Final review included 65 studies that met criteria. Started with 11,277 records retrieved through database search; screened 8,731 records; 372 full-text articles retrieved. #SIRC2019

@taren_swindle 37/65 looked at acceptability, 11 adoption, 5 appropriateness, 2 feasibility, 4 penetration, 6 sustainability Most studies conducted in high income countries. #SIRC2019

@taren_swindle Settings incl. digital health, oncology, diabetes, cardiology, sexual health, arthritis, trauma, smoking cessation, accident and emergency, anesthetics. #SIRC2019

@taren_swindle Level of analysis: most focused at provider level, then less at consumer/patient, even fewer at org level. Most focused on pre-imp only OR post-imp only. Fewer than 5 looked at pre- and post-imp. #SIRC2019

@taren_swindle Average length - 22 items. Methodological quality - most studies looked at only a few properties. Of those that looked at structural or content validity were excellent, good, or fair. Least measured - reliability. And those that did rated as poor. #SIRC2019

@taren_swindle Research contributes to a repository of appraised implementation outcome instruments that complements Lewis et al. 2015 - Very rigorous approach but incredibly time-consuming. Proctor's taxonomy helpful. #SIRC2019

@taren_swindle Future considerations: further psychometric testing of existing measures. Need more development to measure appropriateness, feasibility, and sustainability. We also need briefer measures. #SIRC2019

@taren_swindle Next up: Helena Fridberg discusses developing an instrument to evaluate implementation of person-centered care. #SIRC2019

@taren_swindle ImProve - Implementing Person-Centered Care: process evaluation of strategies, leadership and health economics. Policy-makers in Sweden are pushing for person-centered care to promote high-quality healthcare. #SIRC2019

@taren_swindle People at local development department had to come up with how to implement person-centered care because politicians were wanting it implemented. Opportunity to observe this process from beginning to end across system levels. #SIRC2019

@taren_swindle Collecting data from 6 healthcare units, 2 inpatient psychiatric wards, 2 geriatric wards, 1 large primary care, and 1 renal outpatient care (dialysis). Very different contexts across these settings. Using case study approach. #SIRC2019

@taren_swindle Collaborate with an organization across 21 regions in the country (sorry, missed the name!). Since 2009, have conducted the National Patient Survey. Measures patients perceptions around quality of healthcare. #SIRC2019

@taren_swindle Innovation: Person-centered care; ethical approach to establish collaboration between healthcare staff, patients, and families. Mutual respect, information at patient cognitive level, etc. although operationalization is unclear in the literature. #SIRC2019

@taren_swindle 3 key elements: (1) Narrative - important to listen to patients, (2) Partnership - everyone brings expertise (of diff kinds) to the table, and (3) Documentation - what are the goals and have we met them? #SIRC2019

@taren_swindle The problem is measuring before and after implementation of person-centered care. How can we measure it? How can we measure change over time? Solution: Measure patient perception of person-centered care. #SIRC2019

@taren_swindle Measure shouldn't be tied to profession or context of care or diagnosis. Wanted to have short instrument (< 15 items). Looked at many options: Herbert 2002 (?), Moullin et al., 2018, Tennant & Conaghan, 2007 #SIRC2019

@taren_swindle For this project, based on Rasch Measurement. rasch.org/rmt/rmt213d.htm ncbi.nlm.nih.gov/pmc/articles/P…
doi.org/10.1177/105381…

@taren_swindle Constructed questionnaire, conducted content validation with experts, revised and then had patients (n=553) to questionnaire. Conducted Rasch analysis and cognitive interviews (n=10) to better understand items and readability. Revised questionnaire (2.0) - 24 items

@taren_swindle Tested 2.0 questionnaire w/ patients again (n=563). Next steps: Rasch analysis w/ new data & final revisions for measurement with robust qualities. Then SALAR will translate into seven languages, design web and paper surveys, design procedures for recruitment and dissemination.

@taren_swindle Collaborate to - identify potential stakeholders and initiate collaborations to expand survey opps and dissemination. Develop questionnaires based on a rigorous measurement approach. Enable benchmarking and eval of clinical implementation efforts for regions across a country.

@threadreaderapp unroll