Welcome to the first ISPAH Epidemiology Council online journal club. This journal club is an initiative of Council co-chairs @drbrigidmlynch and @drterryboyle, and will be facilitated by Suzanne @DixonSuen, the Council secretary. #ISPAH_Epi #epitwitter
The @ISPAH Epidemiology Council journal club will run over 48 hrs to allow participation across time-zones, wrapping at 29 April 10pm UTC. #ISPAH_Epi #epitwitter
The session topic is improving observational epidemiology, focusing on an @AmJEpi paper by Dr Sam Harper: A Future for Observational Epidemiology: Clarity, Credibility, Transparency. Am J Epi: 2019;188(5): 840-845. doi.org/10.1093/aje/kw…
@sbh4th #ISPAH_Epi #epitwitter
@sbh4th #ISPAH_Epi #epitwitter
Journal club questions will be posted in one thread. Participate in the discussion by including #ISPAH_Epi and #epitwitter in all journal club-related tweets. To make the conversation easier to follow, if responding to a posted Q you could reply to the specific question tweet.
We will post new questions at regular intervals over the initial portion of the journal club to keep the event rolling, but feel free to keep the conversation going on earlier topics – you can revisit earlier questions anytime over the course of the event. #ISPAH_Epi #epitwitter
Let’s get started! #ISPAH_Epi #epitwitter
Q1: Sam Harper writes “Presently, there are concerns that the evidence produced by most observational studies in epidemiology is not credible and contributes to research waste.” Do you agree with Harper’s view of the current state of epidemiology? #ISPAH_Epi #epitwitter
Q2: How does this apply to physical activity/sedentary behaviour research? If we accept there are some problems in applying observational epidemiology generally, are they more or less pronounced in physical activity/sedentary behaviour research? #ISPAH_Epi #epitwitter
Q3: Have you used any of the analytic methods Harper suggests (section ‘Credibility’, pages 841-2) to improve inference in observational epidemiology? What was their reception by the physical activity/sedentary behaviour research community? #ISPAH_Epi #epitwitter
(Q3 continued) E.g., quantitative bias analysis or quasi-experimental designs such as interrupted time-series, difference-in-differences, synthetic controls, instrumental variables, regression discontinuity #ISPAH_Epi #epitwitter
Are these methods gaining acceptance? How could their use be increased?
Q4 (1/3): Harper calls for increased transparency in multiple aspects of epidemiological research to improve quality + credibility, including in deposition of study materials (datasets) so others can reproduce our results. #ISPAH_Epi #epitwitter
Q4 (2/3): Do you think that increased data sharing would accelerate the pace of knowledge gain and ultimately improvement in population activity levels? #ISPAH_Epi #epitwitter
Q4 (3/3): What are the barriers to achieving data sharing? Are any of these barriers specific to physical activity research? What can we do to remove these barriers? #ISPAH_Epi #epitwitter
Addendum: study materials to help others reproduce our results include datasets AND code #ISPAH_Epi #epitwitter
Q5: Has anyone conducted what Harper calls a “pure replication study”? Could you share your experiences of conducting and publishing these – in terms of barriers, enablers, and advantages for the field? #ISPAH_Epi #epitwitter
Q6: Do you integrate reproducible research strategies into your own work? How can we as a community achieve more widespread use of these strategies? #ISPAH_Epi #epitwitter
For any Twitter newbies following this journal club: click on each tweet to see any replies (these will appear at the end of the entire thread, so keep scrolling down!), then select each reply in turn to keep unrolling the conversation. #ISPAH_Epi
Q7: Earlier in today’s discussion, @drbrigidmlynch commented on the importance of statistical analysis plans, a point also taken up by @sbh4th in his article. Should we move towards more pre-registration (specification) of our analysis plans? #ISPAH_Epi #epitwitter
And for those of you who publicly post your analysis plans (for non-randomized studies) before starting analyses, perhaps you'd like to comment on your preferred platforms for this? #ISPAH_Epi #epitwitter
Q8 (1/3): In his @AmJEpi article, @sbh4th calls for epidemiology "to embrace more vigorous challenges of the evidence it produces" (p 842), contrasting with the response of other disciplines to the 'replication crisis'. #ISPAH_Epi #epitwitter
Q8 (2/3): This relates to both transparency and credibility, and emphasises the importance of replication studies and reproducible research practices (drawing together some of the earlier Qs in this journal club) #ISPAH_Epi #epitwitter
Q8 (3/3): Would anyone like to comment on whether your institutions have discussed how the replication ‘crisis’ relates to your field, or perhaps share any protocols your institution have in place to ensure quality and improve study reproducibility? #ISPAH_Epi #epitwitter
Q5 expanded/clarified: @sbh4th defines a "pure replication" study as QC: others using your materials (data+code) to replicate findings, "seem[ingly] the bare minimum for most published work" (p842) #ISPAH_Epi #epitwitter
Q5 POLL: #ISPAH_Epi #epitwitter In your experience, does pure replication (an attempt by another researcher to replicate your results using your data and code) happen:
Q5 extension (1/2): Yesterday, we asked about experiences of publishing "pure replication" studies, which may be rare as these QC efforts may not often be made public. #ISPAH_Epi #epitwitter
Q5 extension (2/2): You may also/instead like to share experiences of attempts to publish other types of replication studies (e.g. replicating previously-published associations in a different population) – were your target journals supportive? #ISPAH_Epi #epitwitter
Thanks for voting! Three of the four respondents chose "rarely or never" and the fourth respondent voted "sometimes".
And that brings the first @ISPAH Epidemiology Council twitter journal club to a close! This session was based around @sbh4th's paper"A Future for Observational Epidemiology: Clarity, Credibility, Transparency" and its implications for our research. #ISPAH_epi #epitwitter
We touched on multiple topics, including defining research questions, pre-specifying analysis plans, maintaining quality in secondary analyses of large datasets, publication practices and use of pre-print servers, academic incentive structures... #ISPAH_Epi #epitwitter
... as well as: issues of measurement error (especially with self-reported exposures) and misconceptions about its impact; the importance of (but low uptake of) quantitative bias analysis; single-timepoint vs repeated measures... #ISPAH_Epi #epitwitter
... and touched on some quasi-experimental designs, such as recent studies using Mendelian randomization to estimate the causal effect of physical activity on disease risk... #ISPAH_Epi #epitwitter
... we also covered reproducible research practices including registering analysis plans, data/code sharing, and the value of independent replication… #ISPAH_Epi #epitwitter
… and we took a poll which found 4/5 respondents had rarely/never knowingly experienced other researchers attempting to replicate their findings using their data and code. #ISPAH_Epi #epitwitter
Thanks everyone for reading along and contributing to this first ISPAH Epidemiology Council journal club - we hope you found these ideas and questions useful and that you will keep thinking about how these topics relate to your work. See you next time! #ISPAH_Epi #epitwitter
