Thread by @ShawnMcGuirk

My #CSPC2019 live tweeting #THREAD starts here!

I'll reply to this tweet w/ each panel I attend, creating a thread under each w/ live notes

My schedule: 🔗cspc2019.sched.com/shawnmcguirk

And, ICYMI, see the new CSPC magazine for my op-ed about @DSP_SPE & #CapacityBuilding in #scipol!

I wasn't intending on tweeting the symposium, but 2 really good points on skills:

🔹 For communications, the ability to be brief (tweet length summaries!) is key

🔹 Universities should promote organic mentorship by leveraging alumni office, get alums to meet students over ☕!

🔹 Skills development has strong socio-economic dimension. Accessibility to good schooling even as early as Kindergarten can have large impacts later in life. The foundation of transferable (read: critical) skills stays with you as you later develop expertise.

#CSPC2019 #skills

Interesting point by @CukierWendy, peer review committee members for grant programs are often not all on the same page on the importance of capacity building 4 collaboration & skillset development. She observed that those from smaller universities tend to value this more than U15

Q on how to better measure transferable skills?

🔹 Standardized testing measuring skills have shown many grads don't meet the benchmark.

🔹 Would be useful to regularly assess the programs that are meant to provide these skills, to see if they achieve their goals.

. @ChiefSciCan leads the opening plenary on leveraging data for #scienceadvice in response for health emergencies (e.g. Zika, SARS, Ebola)

w/ Rolf Heur @EU_Commission group of scientific advisors, @uksciencechief, @YasminKhan5 @UofT, @AajKamranKhan

Nearly every emergency has some kind of scientific dimension, that benefits from rapid and sound #scienceadvice. In the UK, @uksciencechief is responsible for giving integrated scientific advice (obtained by meeting with science advisory group, SAGE) to the Cabinet.

The @EU_Commission Group of Scientific Advisors are critical to supporting decisions during periods of crisis. Clear communication across borders and developing trust through mutual learning w/ stakeholders and by engaging the public.

Where does the data come from? Publications? Reports?

UK has a standing Joint Modelling Group, who work well together and are pulled together within 4h of an emergency. They pull data from public and proprietary resources and write clear summaries for SAGE.

What could help front-line health providers? @YasminKhan5 says Public Health does background work to ensure preparedness, e.g. investigations, regulatory or policy measures. System based approach is important to address knowledge gaps before emergencies arise.

Kamran Khan @hellobluedot adds that data are just raw signals, not insights or evidence that can be readily used by decision-makers.

Natural language processing helps BlueDot run through big data in every language to monitor time and location data for infectious disease.

... This enables tracking disease progress, geographically and through different vectors. Working in this innovative area enabled connections with academics and with government organizations. These relationships are key for coordinating btw science and policy under emergency

Data management, making sense of data, is also critical. Rolf Heuer says @CERN has to do lots of data mgmt in order to ensure smooth operation of such a large initiative.

This also enables good open communication, in academic publication, to policymakers, to the public.

Next panel at #CSPC2019, on developing a Quantum strategy for Canada

w/ @TanguayGen, David Cory @QuantumIQC, @AimeeKGunther, Gail Murphy @UBC, @zaribafiyan

China has a 10B quantum strategy, Japan, Netherlands, UK, so many countries have quantum strategies. Companies received 450M USD in private sector funding for quantum.

Canada is developing a strategy.

What is the potential impact of quantum? Is it a fad, or is it the future?

Tech like quantum computing can have a huge impact on Canada, and this is only one application of quantum.

Quantum overall is a transformative technology ➡️ can affect Canada not just for making cool toys, it can be used against us (national security issue, eg. cybersecurity)

David Cory's #CFREF team @UWaterloo has applied quantum sensing for medical diagnosis at @Sunnybrook, in the lab to look at dark matter, and in developing a new device to get 10,000x higher resolution on biomedical imaging ➡️ imaging proteins in a cell membrane is now possible

. @AimeeKGunther used to work on lasers in a dark lab, now's is applying her critical thinking to how a Quantum strategy should be developed. Overarching theme is a high need for training.

G Murphy says developing a strategy enables the acceleration of growth of tech & capacity

. @zaribafiyan says Canada is punching above its weight in #Quantum & now is the time to strategize to continue building that capacity.

David Cory adds that great individuals are working on quantum, but individuals are not enough. Collective approach & interdisciplinarity is 💪

And the strategies from other countries won't work for Canada. We have strengths in many areas that overlap with #Quantum, and a strategy should address this. Need to work across disciplines, across academia and industry, and address shortage of talent.

#CSPC2019

Talent retention is also key! Need to create jobs, startups, opportunities to make sure those that are trained in great Canadian labs on quantum, and labs on quantum-related disciplines, have the chance to keep advancing their careers in 🇨🇦.

... 3 CFREF & 5 CREATE from @NSERC_CRSNG help develop best practises for talent development and retention by creating platforms for mentorship programs, to develop project management skills, to facilitate paths for trainees to create startups.

There is a valley of death for startups. @NRC_CNRC and other government departments can be key stakeholders to be first adopters of made-in-Canada innovation from 🇨🇦 startups and SMEs

How should policymakers be engaged on #quantum?

We need their creativity! Shout out to SIF program, @NSERC_CRSNG Alliance and CREATE, scholarships and fellowships. But how do we ensure that research makes it through the innovation pipeline? How can we coordinate govt programs?

... Department of defense is building their own quantum strategy, which requires consideration of current state of technology and research, of where there are friction points, and where key actions can help advance quantum tech faster.

Quantum is advancing quickly worldwide, Canada is doing well but has to keep up. We don't have multinationals investing, but we have many smaller scale efforts that could be leveraged more effectively by creating a more coordinated Team Canada approach

Focusing on building hammers w/o thinking of how to use them is short-sighted. Need to develop the tools for quantum while looking at the bigger picture of how Canada can lead in adopting them.

Researchers shouldn't charge ahead without engage nationally, incl with policymakers

Also need to integrate fundamental and applied research on quantum. These drive each other, symbiotically. One can hardly be done without considering the other

It would also be a risky bet to invest in one single challenge, we need to strategize in a way that remains diversified and ensures that investments are made with knowledge of risk relative to benefit.

#CSPC2019 /fin

Final panel of #CSPC2019 day 1, Not a palaver! How can interdisciplinary, intersectoral and international collaboration be successful?

How do we fund interdisciplinary research internationally? Can grand challenges be a viable approach?

Yes but collaboration shouldn't be forced by an overly complex call. Should give researchers freedom to address the issue w/ creativity. Challenges also don't have to be grand

Rather than create a box for researchers to fit themselves into, #NFRF was designed to be open, so researchers define the challenges they want to address.

Issue w/ grand challenges is often that social sciences and humanities researchers aren't as well represented in the teams

The science of team science is still out - how can you be sure that a team applying to a grand challenge is ideal to address the challenge?

UKRI has a Global Challenges Research Fund of 1.5B£ over 4 years, for the benefit of the global South.

Grand challenges do promote interdisciplinarity, but also lots of interdisc happens organically (bottom-up). Cannot ignore this.

How can we reconcile national interests, with a growing desire to support international research, and how do we resolve the role of industry in this?

@drjoyjohnson notes that collaboration and competition are both important to drive innovative ideas and research

Ted Hewitt @SSHRC_CRSH notes that national interests have to understand the importance of international collaboration. We can't do it all, and working with others who can bring their strengths to the project should be supported.

. @MelanieWelham @UKRI_News says there must be a sweet spot of national and international approaches that maximizes benefits to all. There are ways to make this work, though development of solid relationships and a shared goal (e.g. International project to increase wheat yield)

John Laughlin adds that international collaborations can capitalize on and advance Canada's strengths (e.g. in AI, Quantum, Cybersecurity) and also elevate areas in which we are weaker. Not capitalizing on these opportunities will slow global growth. And for what?

Q from Jeff Crelinsten @research_money, do only older researchers with less to lose participate in international collab / grand challenges? Or do younger researchers apply? If we only award researchers on publication metrics, this might be a risky approach.

A - it's important that there are diverse programs that support the research community broadly. There is a balance of ensuring we seize opportunities, while maintaining core support for the research community

Q from @OlgaPena, who is from Columbia and now a @MitacsCanada fellow, asking about how to support more fundamental research via international researchers, through normal funding calls. Many international researchers aren't aware of opportunities.

A from @drjoyjohnson - need to build bridges relationship by relationship. Exchanges between universities, and even with international industry, are good options. It's not just about funding.

UKRI also has a program for making these kinds of connections, which is well received

For this morning's panel at #CSPC2019, check out @this_is_farah's tweets!

https://twitter.com/this_is_farah/status/1194960724919046144?s=19

Referring to @AimeeKGunther for live tweets for the international S&T panel this morning ##CSPC2019
⬇️

https://twitter.com/AimeeKGunther/status/1194975301639557122?s=19

#CSPC2019 Research without borders: funding agency case studies on international collaboration ⬇️

Panel w/ @MonicaGattinger @ISSP_uOttawa, Claire Hemingway @NSF, Rainer Grulich @dfg_public, @jclemay4 @SciChefQC #scidip fellow, Chloe Summers @UKRI_NorthAmer & @decker_je @NRC_CNRC

There is excellence around the globe, and @NSF supports a wide variety of inter/transdisciplinary & convergence research, engages in bilateral / multilateral calls with many int'l funders. There is also an embassy fellow program, which Claire Hemingway participated in last year.

. @UKRI_News / @UKRI_NorthAmer supports leading talent for a "global Britain" to push the frontiers of human knowledge and understanding. There is a new international strategy in development, aimed at building capacity worldwide ⬇️

. @jclemay4 describes Quebec's leadership in research via @FRQ_NT @FRQS1 @FRQSC which mirror the federal funders @NSERC_CRSNG @CIHR_IRSC @SSHRC_CRSH respectively, and it's international activities ⬇️

. @decker_je describes @NRC_CNRC research and business innovation programs, which includes working closely with @ISED_CA and other science based departments. They also promote international collaboration for R&D in Canada and abroad ⬇️

Rainer Grulich @dfg_public describes increasing levels of funding support for research in Germany (3-5% increase per year) which supports great programs at DFG ➡️ supporting all branches of science and humanities, in basic research and international collaboration ⬇️

. @dfg_public is also greatly involved in advocacy for international research policy (e.g. for #EDI) and works with organizations like @NSERC_CRSNG, @univcan, @NSF and many more to unlock global potential for science & policy cooperation

. @ChiefSciCan giving a status update at #CSPC2019, outlining the importance of the position of Chief Science Advisor to Canada, and the importance of having young people #NextGenCanScience engaged in #scipol & with society 🙌🙌🙌

Last year was very busy, publicly and privately in the Office of @ChiefSciCan. Priorities are #OpenScience, #scienceadvice, science coordination, science promotion, and #sciencediplomacy

And, of course, ensuring that the office can be a permanent fixture of the Government of 🇨🇦

There are now science advisors in multiple departments and agencies (@GovCanHealth @environmentca @NRC_CNRC @NRCan @csa_asc) and more upcoming (@FishOceansCAN)

The appetite for #scienceadvice in emergency response has also increased significantly, in part due to engaging models that exist in UK and other partner countries.

The dream of @ChiefSciCan is to have us stop talking about where science is going in Canada, instead to discuss how we are advancing together to advance knowledge and socio-economic conditions of all Canadians and across the globe

Canada also now has an advisory committee on #AI, with diverse expertise, who can help 🇨🇦 retain and further our leadership in this key area of research

Starting January 1st, all government departments have adopted a science integrity policy, which means respect for the responsibilities and privilege of scientists and also of the employer

Canadian universities are climbing in international rankings, which outlines our strengths and also a capacity to attract top talent from around the world.

An important role of @ChiefSciCan is also to recognize and have recognized the great science being done in federal labs, from fisheries to space, from agriculture to national defense.

+ if we don't ensure that we have capacity to adopt the breakthroughs that our intramural and extramural researchers are making, we will lose out on opportunities for 🇨🇦 to shine and advance on the world stage.

The friendship between these two is so clear in their fireside chats!

#CSPC2019 @sciencepolicy

Q - how will the new (minority) government engage #scienceadvice?

A - same as before, hopefully with renewed appetite. Time will tell!

@ChiefSciCan got a call today from @JustinTrudeau, asking to say hi to @MehrdadHariri & everyone at #CSPC2019 @sciencepolicy

Q - you mention open science, what is the future of this?

A - Canada is behind on #openscience and #opendata, though its notable the granting agencies have good policies on open access and research data management. Compliance remains an issue.

... There are still misconceptions about #OpenScience, not everyone understands it or appreciates it importance. More discussions are necessary to advance open science in Canada, especially to ensure the public is engaged in the science that is publicly funded.

Q from Matt McCloud, federal scientist - are there areas where the @ChiefSciCan mandate should expand? Is it well defined and resourced?

A - it took some time to determine what needs to be done - it's more clear now & more resources may come in the future. Form follows function!

Comment from Elissa Strome @CIFAR_News - to remain an AI leader, we need to invest in fundamental research, talent development and applications of #AI. #EDI is also especially needed in this field.

@ChiefSciCan agrees on need for #EDI, should not perpetuate bias thru algorithms

Q from researcher from University of Lethbridge about how can pay for open access? There isn't enough money in the system to afford article processing charges.

A - completely understand, we need a strategy and resources, but money isn't the only solution.

... Many countries have approached this in thoughtful ways. Deals with publishers, support for different models of open access, etc.

It's important to engage with leaders in this and be part of the international movement for open science. Let's be on the right side of history.

. @ChiefSciCan asks scientists #cdnsci to keep engaging policymakers! The Science Meets Parliament pilot will be repeated, it's important to keep improving connections between science and policy!

Andrew Harris (@sciencepolicy volunteer) asks how we can incentivize the writing of lay summaries for research papers? This would help with accessibility.

@ChiefSciCan says a "readers' digest" of science may be a good idea!

Q on insufficient #scidip in 🇨🇦 & no science advisor at GAC. How do we move forward?

@ChiefSciCan says we have support including from @PrivyCouncilCA Clerk to engage in #scidip. Developing an international science policy was recommended in last annual report.

. @dom_roche notes govt invested highly in digital research strategy, but psychology / training is also important to ensure data sharing practise is adopted by researchers.

@ChiefSciCan agrees that we need more conversations (and champions) for sharing and openness in research

And with that, performance review success for @ChiefSciCan!

Too many great concurrent panels at #CSPC2019!! It's a Sophie's Choice every time.

This time I picked Convergence Science & Tackling Grand Challenges, w/ Julie Greene @PrivyCouncilCA, @ImpactCanada Roman Szumski @NRC_CNRC, Julie Angus @OceanRobotics and Shahab Shahnazari @MaRSDD

Impact Canada initiative at @PrivyCouncilCA has 120M$ portfolio to experiment with methods of addressing grand challenges, including through behavioural insights.

Priority areas include food waste, opioid crisis, numeracy, mining, indigenous housing, +++

impact.canada.ca

. @NRC_CNRC has convening power for R&D, has brought together workshops on experts e.g. on cannabis to inform science based policy that has been introduced by Minister Blair. The Minister found it refreshing not to be lobbied, but to be engaged by top experts!

Much of our oceans are poorly explored and mapped. More data is needed to make informed decisions about oceans, whether that is #climatechange policy or decisions about fisheries and vessel traffic. @OceanRobotics is attempting to address this.

Opioid data challenge @MaRSDD was designed to inform policies and solutions downstream. This Grand challenge enabled access to a more diverse crowdsourcing of data. Shahab Shahnazari says this kind of reach is something that traditional grants do not readily achieve.

Challenges have a time and place, it's a mechanism that can solve issues but that shouldn't be used for "innovation theatre". If a solution exists, you don't need a challenge, you need to resource that solution.

Seems @this_is_farah and I finally managed to attend the same panel 😀

She's already got a head start on livetweeting the session by @E4Dca
w/ Preston Manning, @kimberlygirling, @tedhsu, @KarenAkerlof

https://twitter.com/this_is_farah/status/1195076498920792065?s=19

Results from @E4Dca study show most MPs start their evidence search w/ Google, and lots comes from the network their already have. But how to they deal with conflicting evidence? Most prefer talking to ppl directly (constituents, experts, peers in caucus, external orgs)

Library of Parliament is a trusted source of information ➡️ confidential and unbiased information relevant to their work.

For external orgs, trust is hard to achieve and is not universal. Better links btw MPs and scientists /experts / advisors who can provide evidence is key

. @KarenAkerlof studies the use of science in the US congress. They use it all the time! There are barriers however - social / bias, institutional access, and inability to deal with uncertainty that is inherent to research

So they ran a study w/ staffers, with a good distribution of liberal & conservative.

8% don't use science at all, 19% use it substantively. Most use of science is strategic and rhetorical ➡️ finding evidence that supports their viewpoint / eventual decision.

Issues include bias (obv) but also time constraints and complexity of the science.

Bias doesn't disqualify information, particularly if the information bias aligns with their strategic priorities.

Report:
researchgate.net/publication/32…

Briony M Lalor ran a similar study during her work @mcgillu, on environmental policy-making specifically.

Evidence was useful mostly when it could be contextualized for what policymakers need. Credibility, legitimacy, and salience were the 3 main criteria.

The perception of a scientifically illiterate public (ministers' words) and a real lack of accountability for using science in decision-making also played a role in reticence to adopt scientific evidence

⬇️

Sensational stories win over scientific evidence

Solutions:
🔹 Senior academics should step up & get involved, to increase public scientific literacy & increase the accountability for governments to legitimately use scientific evidence

🔹 Innovate institutional structures to improve connection & access to experts and evidence

. @tedhsu contributes his perspective as a prev MP. One of the last times he was approached on a science issue, a neighbour came running w/ a flyer warning of the dangers of 5G on human health.

How is an MP supposed to be ready to respond when quick responses are necessary?

Well, it's important to maintain connections with top experts and with groups like @cca_reports who have a high reputation for trustworthy assessment.

Knowing where to go on short notice is critically important for ensuring evidence is used in policy decisions & communications

Having many universities in his riding, @tedhsu was kept on his toes by academics who aren't interested in party lines and want direct answers to their questions. There's also a great benefit, since he can engage his constituents to gather good evidence.

In the end, TRUST is the 🗝 for legitimacy in the policy & political machine.

⬇️⬇️⬇️

To make the machine accountable to evidence, we must maintain the integrity of scientific institutions and increase dialogue btw (trusted) academics and policymakers / politicians.

#CSPC2019

Important note by @kimberlygirling that evidence is vast and goes beyond fundamental research ➡️ includes economic, social, & political considerations

The word "evidence" is broad, and can be incredibly subjective. Scientific information is a form of evidence, one that would hopefully not be used in a biased way to support rhetoric ➡️ but it is.

What can be done to make sure scientific information actually makes it to MPs? To the PM?

A 50 volume report quickly becomes a 50 page summary, then a 2 pager, and down to a bullet point that gets communicated verbally in a 5 min meeting.

How much science is left?

Scientific information needs to be communicated in a way that is succinct (60-90 sec max) and publicly understandable.

Academics can seek #scicomm training to learn how to do this. Or, reach out to #scicomm experts who can help frame the message in a way that resonates.

. @kimberlygirling @E4Dca adds another 💯💯💯 point, many MPs who wish there was a study on issue X, don't think to ask universities if a study is underway or if a researcher would be interested in running this study!

Many researchers would love this kind of interaction

Our great hosts @CStachulak & @MadameCosinus are starting up the #CSPC2019 gala with style!

The cross town youth chorus did a great job with the national anthem 🇨🇦😀

Félicitations @MehrdadHariri for 10 years of CSPC!!

A great panel moderated by Janet Haliwell @sciencepolicy w/ @GGJuliePayette, #DonnaStrickland @UWaterloo and Avery Broderick @Perimeter at the #CSPC2019 gala tonight ⬇️

Isn't it more important to contribute to the economy than to win a @NobelPrize? - #DonnaStrickland

Fundamental and applied science go hand in hand; Donna needed to build a laser to be able to make further discoveries

Despite being 0.5% of the world population, 🇨🇦 is a beacon for science worldwide - that doesn't happen by accident!

The Event Horizon telescope 🔭 could only happen w/ a diverse group of people working together towards one extraordinary goal - #AveryBroderick @Perimeter

It's very important to celebrate not only music, actors, artists ➡️ but we shouldn't forget to celebrate #cdnsci - @GGJuliePayette

+ gives a shout out to @NSERC_CRSNG for funding Art McDonald's research at @SNOLABscience, that 🇨🇦 research funding is what leads to 🇨🇦 Nobel Prize

The international space station doesn't make the front page news very often ➡️ that's because up there, working on #science together, astronauts / researchers from each country all get along 🤝 - @GGJuliePayette

#scidip #sciencediplomacy #CSPC2019

... There are benefits to science beyond winning prizes, and we need to communicate this better. The role of science in society is much greater than most realize. Let's promote our science more, and stick a red maple leaf on it to show 🇨🇦 pride ✌️- #AveryBroderick
#cspc2019

J Halliwell asks, how can we better promote science and the excitement of discovery?

#DonnaStrickland says we need to ensure science is seen as the frontier of knowledge, not a belief system. But we also need to respect other ways of knowing.

#AveryBroderick warns that society can walk back from science, from technology.

Science isn't considered a "book of facts", rather it is a complex (frustrating) process. Almost every idea we have is wrong ➡️ try to prove yourself wrong, at one point you'll find something right.

Leave a baby in a room with an open door, they'll walk out that door 🚪 (discovery research!)

Leave them a toy and they'll destroy it 💣 (Engineering!)

They are also avid dreamers 💭 (creativity! philosophy!)

We all start here, and we can learn from this - @GGJuliePayette

Innovation starts at the foundation - from early education, to research, to support from the granting councils, to policy and applications for society. We have to support the innovation continuum, starting from the foundation - @GGJuliePayette #CSPC2019

Don't support and elevate individuals that only elevate themselves. We advance together by supporting each other and the next generation - #AveryBroderick #CSPC2019

If you want to attract top talent, you need high level projects and high levels of funding to support that talent - #DonnaStrickland #CSPC2019

Congrats to #scipol Youth Award winner @airplaneavocado

40% of seafood sold in Canada is mislabelled ➡️ the industry provides 6B$ to the economy. Seafood fraud puts us all at risk. Her proposal incl better labeling standards & DNA barcoding to safeguard the supply chain 🔗 🎣

A new #CSPC2019 award announced by @MehrdadHariri for exceptional contributions in science policy! In the future, #scicomm and #scidip prizes to come!

Award winner spent his life in science policy, is a mentor, and a champion for the next gen in #scipol. It's Paul Dufour 🎉🎊!!

Very early morning at #CSPC2019, discussing collaboration w/ Federal Science!

Panelists: Janet King @PSPC_SPAC, @katemoran and Kim Juniper of Ocean Networks Canada, and Iain Stewart @NRC_CNRC

#Thread ⬇️

@PSPC_SPAC @katemoran @NRC_CNRC Collaborations include infrastructure ➡️ the Atlantic Enterprise Centre in Moncton is a new center that has been launched in collab with many government departments, to created a space to foster S&T in support of the coastal ecosystem.

This incl support for IT and cybersecurity

@PSPC_SPAC @katemoran @NRC_CNRC The new Research Facilities Navigator in partnership btw @PSPC_SPAC and @InnovationCA has so far tracked 51 facilities and 348 individual machines, to promote sharing resources for federal science

Our panel @NSERC_CRSNG @SSHRC_CRSH @CIHR_IRSC @InnovationCA #CRCC starts now!

w/ Liette Vasseur, @alejandroadem
@TammyJClifford, Roseanne O'Reilly Runte, and Ted Hewitt

#Thread ⬇️

@NSERC_CRSNG @SSHRC_CRSH @CIHR_IRSC @InnovationCA @alejandroadem @TammyJClifford Liette Vasseur @BrockUniversity @CCUNESCO starts the panel as moderator, mentioning the breadth of activities that make research excellent. It is a profound cultural activity.

Dr. Vasseur notes that by signing #DORA, the agencies are committing to evaluating research on its own merits, that it is more important than research metrics.

Dr. Hewitt @SSHRC_CRSH reflects on the importance of research excellence, that if the agencies cannot assure that we are in big trouble! The concept of research excellence is changing, there are new considerations of what this constitutes. Canada is leading in this with #CRCC

@SSHRC_CRSH #NFRF will grow significantly by 2022-2023, and this program was designed to take us to places we haven't been -- it encourages researchers to take risks & work with partners. To innovate, by engaging across disciplines and by seriously considering #EDI in research design.

@SSHRC_CRSH Dr Hewitt followed the pilot of #NFRF very closely, asked peer reviewers for their insights, and notes the program will keep improving via incorporating their recommendations.

@SSHRC_CRSH . @TammyJClifford notes that research excellence is top of mind @CIHR_IRSC, since the research funded by CIHR is what drives the future of healthcare in Canada, this is the kind of research that saves lives.

Ensuring to get #EDI right, that we value great contributions, is key

@SSHRC_CRSH @TammyJClifford @CIHR_IRSC There is a growing understanding that research excellence is not just H-index and the number of publications. More and more, researchers are showing their leadership & excellence in new ways, w/ multi-dimensional teams

@SSHRC_CRSH @TammyJClifford @CIHR_IRSC U of Glasgow has included collegiality as a measure of promotion! Not sure how this will be measured, but it's certainly something that has value in the overall excellence of a researcher.

By promoting exceptional training & training environments, we help trainees to be active contributors in academia in beyond. The CIHR training strategy includes enabling trainees to gain professional skills & get experience with interdisciplinary teams

... research findings that lead to stronger policies, strategies, and other advances in society should be celebrated.

Notable that the agencies already align with many DORA recommendations, by promoting #EDI, through research data management, through open access, and many other key priorities

As a member of the mathematics community, @alejandroadem has never subscribed to impact factors. Read the paper in front of you, and evaluate it on its own merits!

.@alejandroadem @NSERC_CRSNG notes that there is increasing consideration of the relationship between research excellence and equity, diversity and inclusion, that established scientific research impact metrics cannot capture all the forms in which research contributes to society

@alejandroadem @NSERC_CRSNG He further notes the success of the #Dimensions #EDI program, which is now signed by 1 in 3 post-secondary institutions, the EDI capacity building grants allocated by @NSERC_CRSNG, and the #Alliance program which supports research with private, public, nonprofit partners

@alejandroadem @NSERC_CRSNG . @alejandroadem @NSERC_CRSNG looks forward to working with colleagues at the #CRCC, across the agencies, and with the community to continue to support excellent research in Canada and internationally, and to ensure #EDI and representation of all groups in research.

@alejandroadem @NSERC_CRSNG Roseann O'Reilly Runte @InnovationCA notes the changing nature of research. What was once a great innovation, plastics, is now our greatest source of waste. Different cultures also bring their own perspectives to the meaning of excellence in research.

Research is a systematic reflection, it must have evolved, and be evolving. Excellent researchers are honest, responsible to society and the ecosystem, will manage and share data and results, will mentor, will review, and will contribute to moving forward together.

.@InnovationCA signed on to DORA a year ago, because it aligns with their mandate to promote interdisciplinarity, and with their assessment of excellence. They do not ask for publication metrics, they ask researchers to show their ability to complete their project

... it's not perfect, it continues to evolve. Our reflection on the nature of research and research excellence is an ongoing discussion - Roseann Runte @InnovationCA

@InnovationCA First Q. How will the agencies promote and raise awareness of the changing research system, to researchers and institution? How will this be communicated? How will the public be impacted by open access and a more comprehensive assessment of research excellence? #cspc2019

@InnovationCA Ted Hewitt @SSHRC_CRSH says the launch of #NFRF included a significant comms strategy on how this will support interdisciplinarity and new frontiers. The question that kept coming back was: "this sounds great, but how are you going to do that?"

@InnovationCA @SSHRC_CRSH It is important to make sure this is communicated, and it's an uphill climb. For NFRF, some research was clearly excellent, but not interdisciplinary, and it was a teaching moment to have to turn down great research that does not align with the transformative program design.

@InnovationCA @SSHRC_CRSH Dr Runte adds that researchers want to be working across borders with international partners, and this has to be reflected in how we as funders align with worldwide culture change on research excellence. Canada cannot lag behind on this

@InnovationCA @SSHRC_CRSH .@TammyJClifford adds that the entire community needs to be engaged and active in this culture change. Some researchers think there is opposition between excellence and #EDI -- we need to reinforce the notion that these go hand in hand, and to be able to show why this is required

@InnovationCA @SSHRC_CRSH @TammyJClifford In the CIHR Foundation Grants, the CV was being evaluated before the research, and the realization that this creates bias was a strong reasoning for sunsetting that program, to better support excellence in other programs.

.@alejandroadem approaches research excellence and culutre change scientifically. We need to design this appropriately, ensure we can measure the impact of changes introduced. And these changes are good for science! #EDI creates better science. It's the right thing to do.

@alejandroadem .@arthi_ramac (@MitacsCanada fellow, @DSP_SPE and @CCUNESCO member) asks what are the consequences for not following #EDI guidelines? @alejandroadem @NSERC_CRSNG says there is strong consideration of EDI during grant assessment.

Dr Runte @InnovationCA says evaluation happen in an ongoing manner for infrastructure, #EDI considerations have increased here. Dr Hewitt @SSHRC_CRSH adds that it's on us to self-monitor this progress. Educating peer reviewers in a large part of this work.

.@TammyJClifford adds, on consequences, that formal policies can be required for this, and we do have a Responsible Conduct of Research Secretariat that can be engaged when research integrity has been breached. A community based approach is necessary

.@dom_roche @ucarleton is glad to see that research excellence is being discussed. But DORA came out in 2013, why only sign now? Dr Hewitt @SSHRC_CRSH says we can't lead on everything, sometime we recognize that we need to catch up.

@dom_roche @ucarleton @SSHRC_CRSH Knowing a tomato is a fruit is knowledge, knowing not to put it in a fruit salad is wisdom. How can we assess that we are evolving our wisdom, not just our knowledge?

Dr Runte asks back, how do we know when we have the answer? How do we know we are wise?

We may never know, but we can work towards it by getting a broader perspective on what is excellent. We have to look not just at what is published, but what is important. We will gain collectively by growing this understanding together. And yes, that includes #EDI! - Dr Runte

Jeff Crelinsten @research_money doubles down on @arthi_ramac's question on consequences for breaching #EDI. If researchers say in an application that they will do it, great! But what if they don't? Who is responsible for compliance? What are the carrots and sticks?

Dr Hewitt @SSHRC_CRSH says there is a system of carrots & sticks, there is a funding agreement that should keep researchers accountable. Likely more needs to be done to ensure that #EDI keeps moving forward. It's an uphill battle to which the agencies & community are committed

Dr Runte @InnovationCA adds that carrots are better than sticks, and it's important to ensure young researchers have the opportunity and accessibility to launch their careers.

Crelinsten - and what about the lone researcher, in this new research environment? Some don't want to work in teams, what is the channel for them?

Dr Runte says that without forcing interdisciplinarity, we can still encourage convergence research.

Comment from the crowd that this conversation on research excellence is highly appreciated, and it needs to continue! We owe it to the next generation of researchers to support a more comprehensive view of excellence.

What we see now vs 5 years from now will be very different. Where do you see the assessment of research & the peer review process going in 5 years? What will this mean for researchers?

@alejandroadem @NSERC_CRSNG says all forms of research can lead to great discovery,

... it's about the quality of the research more than anything. Dr Runte @InnovationCA adds that researchers have so many incredible ideas about the future (labs! in! space!), what we need to do is ensure there is a way for them to get there. Enabling more people to join us, ...

... if we are open and diverse, in Canada and to international partners, then we can get there more quickly and more effectively.

Q from a grad student @UAlberta - any consideration for those who don't fit the typical researcher box? How should they fill out a CCV?

@TammyJClifford says using Biosketch rather than CCV especially for patient participants in research design may be considered.

.@annie_montpetit #CIE @SciChefQC asks how the criteria through which grad students and postdocs will be affected by changing nature of research excellence?

(not answered)

Additional closing comments:

@TammyJClifford - we have to continue to prioritize knowledge translation efforts (bedside & beyond) and to focus on training stategy. ECRs will be most critical to make sure this happens, so they can thrive rather than survive. @CIHR_IRSC #cspc2019

... if you want to go fast, go alone. If you want to go far, go together. This is why the agencies are working in coordination, with each other and with the community. Dr Hewitt @SSHRC_CRSH agrees with all colleagues, appreciates the openness of this discussion

@SSHRC_CRSH Dr Vasseur closes by saying it's also on universities and other institutions to engage their researchers, especially ECR and research trainees who are the future of research in Canada, to help drive this change

We are just starting this discussion, let's keep this going 🤝🙌

Thanks all for this great panel! Staff and speakers @NSERC_CRSNG @SSHRC_CRSH @CIHR_IRSC @InnovationCA #CRCC put together an excellent plenary #CSPC2019!

Our next @NSERC_CRSNG panel is starting at #CSPC2019, on Mapping Dynamic Research Ecosystems!

w/ @emmeslin @cca_reports, Liz Boston @NSERC_CRSNG, @stefhaustein @uOttawa, @ajrbradley @ontariotech_u, @XiaodanZhu2048 @queensu

Thanks for the shout out, Eric!

. @emmeslin @cca_reports the bigger challenges to research data is not the amount, it's how we should make use of it, how to understand it, especially as we look across disciplines to understand and support interdisciplinary research.

... 30 yrs ago, research across disciplines was v difficult to get funded

So how do we use data to understand a research ecosystem? Some things we measure aren't important, not everything that is important can be measured. Stop measuring the same thing, looking for diff answers.

This panel was designed to be interactive, but make sure to pose your questions in question form! Just ask it, don't give us the long background. The skill to ask a good question is the foundation of good science - @emmeslin @cca_reports #CSPC2019

How do granting agencies identify priority areas of research? How do you identify gaps?

Dr Boston @NSERC_CRSNG is not a data expert, but is a funding agency expert, and she starts us off w/ terms used to describe how we view the research ecosystem. Often interdisc, not always ⬇️

All emerging areas may not be priorities, not all priorities are emerging areas. Dr Boston shows us examples of best practises across agency programs ⬇️ #CSPC2019

For E.g., the Discovery Frontiers program at @NSERC_CRSNG had calls on biodiversity, #AMR, Arctic, biomaterials for clean energy. Areas were determined via large consultation exercise + expert committees.

Notable that new #Alliance program doesn't have targeted areas

. @SSHRC_CRSH has a Imagining Canada's Future exercise to look at emerging priorities and challenge areas. The #FSR #Naylor report identified key emerging areas, the CRCC has a mandate to do so as well, @cca_reports has also done this. Many methodologies.

Key questions incl what makes us competitive? What are international leaders prioritizing? Which areas are falling between the cracks?

How do we address this? Additional funding to key areas (given available funds!), special calls, etc. There are also many policy approaches ⬇️

How can data help? We need to know where to look and how to validate it, and reconcile what the role of expert advice is in coordination with a data-driven approach #CSPC2019

. @ajrbradley @ontariotech_u continues by presenting how their lab approaches data analytics & bibliometrics. We rely on network analysis (how connections occur over time) because it is measurable, but it's still hard to understand unless you're an expert! Need accessibility.

. @lariviev contributed to this research, asking how we can contextualize data. If a paper is cited 500 times because it's wrong, it shouldn't be counted as impactful.

@ChrisNF @ajrbradley lab is developing tools to determine not just what is there, but what is interesting?

They look at a dataset w/ 60M documents, to determine where 🇨🇦 leads and lags on key research areas. They engage natural language processing #AI to translate a real question, into what can be measured.

Are these data the same? The answer is, what do we mean by the same. The data looks different, but ALL of the statistics that are measurable from this are identical. The different data can convey the same message, and data can be used to say anything. It's very subjective.

... And if this doesn't make you scared about how data might be used for policy-making, you need to look again #CSPC2019

. @stefhaustein now presents how current bibliometrics ignores so many other important and valuable considerations.

Looking at graphs below, we can see the younger researcher (@lariviev) can be undervalued because his shorter career means citation score is much lower..

... Need to correct for what is not captured by biased quantitative measures

There are also MANY other ways to show and drive impact of your research. Writing op-eds, post on social media, make your data and code open, publish open access.

Researchers from diff fields don't use alternative communication tools though, so case by case basic necessary.

Social sciences and professional fields use social media quite a bit, vs so little in math and engineering. @altmetric can be a helpful measure but it's not equally applicable. Need to define what we mean by impact, to measure it properly.

What role can #AI play in this, how can bias be avoided? @XiaodanZhu2048 presents some challenges ➡️ data is not knowledge, but it is a language to itself. Enabling computers to understand unstructured data languages, it can help us answer real questions. #CSPC2019

Key idea is that deep learning can represent words as vectors, which can be linked to each other in vectorspaces. The meaning of words needs to be well represented, as well as associations between words. This can help understand sentences.

This research area has advanced rapidly

If a computer doesn't have common sense, it can't meaningfully help us to model data. @XiaodanZhu2048 's lab works on training #NLP, from simple associations to complex reasoning for why these associations exist. If training is biased, however, the answers are biased.

Through #NLP, you can analyze trends and sentiments towards research topics, you can map topics, detect influential research being cited, and determine interaction and inference between disciplines.

To avoid bias, to find research areas falling between the cracks, data needs to be prepared very carefully. Biased research data will lead to more biased research decisions - @XiaodanZhu2048 #CSPC2019

Q from @emmeslin - what is the future of data analytics, in 15-20yrs? In research question forming? Policy-making? Funding decisions?

@ajrbradley says there are critical ethical issues in data analytics. We know data is biased! Train a chatbot on the Internet, it'll be racist!

... There is a necessary realignment btw data research and social sciences and humanities. Even researchers working on this every day have trouble addressing ethical issues ➡️ you can manipulate data to get to the top of the pack by taking advantage of biases.

. @stefhaustein says librarians have a large role to play, they need to understand the state of the field and the role of AI. AI experts need to talk to librarians. We keep reinventing the wheel to get new tools, but we have so many that we can already learn from.

Liz Boston @NSERC_CRSNG says we need experts to help us contextualize the data, to help us understand it and ask better questions. It's a real partnership.

Q from @ucarleton student, what about other sources of data showing economic or policy impacts?

@stefhaustein says there are ways to measure it and it should be!

Kevin Fitzgibbons @NSERC_CRSNG asks, since the agencies signed #DORA, we need to reflect on what we mean by research excellence, and what data shows this. How do we adapt research analysis to this?

. @XiaodanZhu2048 says this is a challenge currently, #NLP may be able to help as it develops further. This is possible.

Liz Boston @NSERC_CRSNG adds that it would be v useful to have aggregated / useful data on how researchers demonstrate excellence broadly, incl on mentorship

But there is a literacy component to this. @stefhaustein says H index was developed and quickly debunked, yet some agencies are still using it because they don't know what else to use. Maybe #DORA can guide this, but lots of work ahead.

Any metric becomes a bad metric once it's used for goal setting - @stefhaustein

She is now developing #scicomm tools to help teach people why the H Index is biased and how to be more equitable in measurements.

. @Patjulchem asks how we can find meaning in a giant landscape of types of metrics? @ajrbradley says interpretation depends on the question being asked, and so we need to use deduction to make sure we are addressing that question properly.

Knowing HOW tools are answering questions is key. If an AI is trying to address questions in a way humans don't, then the answer won't reflect the question asked by humans.

Is it time to start using big data to make funding decisions / measure research excellence? Liz Boston @NSERC_CRSNG says no, we need to be supremely careful in how we use data. In short term, maybe it can help us understand the system at large, but not to make decisions.

To consider :

@emmeslin published a paper in a small journal, which cost 25$ to buy access thru a paywall, whereas a sci-fi book on the same topic is 3$ at Wal Mart. This speaks to importance of open access.

Jeff Crelinsten @research_money asks, what is the motivation to use data in research analytics? To replace peer review? Or complement it?

@stefhaustein says metrics should never replace peer review, but if data is well organized it can help drive evidence informed decisions.

... You can't apply peer review to analyze 10,000 papers, data analytics can help

Liz Boston @NSERC_CRSNG adds that peer review is a must! This is the foundation of our research ecosystem. But through data perhaps we can understand if research areas are falling between the cracks, and try to address these issues in an evidence-driven way.

. @ajrbradley adds that we need good tools to understand and use data properly; you can't look at the stars without a telescope 🔭, and you won't reach the stars without seeing them

/fin #CSPC2019

Thanks all for this amazing panel! Stellar moderating from @emmeslin as always, and great contributions from all the panelists. Looking forward to more conversations about this!

The #OpenScience Fishbowl from @NSERC_CRSNG @SSHRC_CRSH @CIHR_IRSC is starting now @sciencepolicy #CSPC2019

w/ @HolzerAngela @dfg_public, @dsweeneyUK @ResEngland, @Monsauce @environmentca, @dom_roche @ucarleton, @skettley @cdnsciencepub, Guy Rouleau @TheNeuro_MNI

And of course, our moderator extraordinaire, Kevin Fitzgibbons @NSERC_CRSNG!

PS - not live tweeting this one, since I'm leaving in 15 min for a flight to another conference!

It is phenomenal so far though, and would be very happy if someone can tweet and/or share their notes 🙏

@threadreaderapp unroll please!

@threadreaderapp unroll please

@threadreaderapp unroll please

@threadreaderapp unroll please

@threadreaderapp unroll please

@threadreaderapp unroll please

@threadreaderapp unroll please