#COVID19 and '111'
Serious questions about the '111' Triage Service.
Is it fit for purpose?
@NHSDigital @DrGregorSmith @CMO_England @doclourda @CMOWales @FatmaMansab
dx.doi.org/10.1136/bmjhci…
Serious questions about the '111' Triage Service.
Is it fit for purpose?
@NHSDigital @DrGregorSmith @CMO_England @doclourda @CMOWales @FatmaMansab
dx.doi.org/10.1136/bmjhci…
Our study published today in the BMJ HCI examined the NHS '111' online triage tool.
Results include:
It could not reliably differentiate between mild and severe Covid-19.
It missed severe cases, advising such cases to say at home.
Results include:
It could not reliably differentiate between mild and severe Covid-19.
It missed severe cases, advising such cases to say at home.
The study used 52 cases simulating various presentations of Covid-19 from seven national online triage tools.
Reliability was poorest for the '111' version.
Reliability was poorest for the '111' version.
The conclusion:
"..our results necessitate a recommendation for the NHS 111 symptom checker and CDC coronavirus symptom checker to be subject to further analysis prior to their ongoing use in COVID-19 clinical care pathways."
"..our results necessitate a recommendation for the NHS 111 symptom checker and CDC coronavirus symptom checker to be subject to further analysis prior to their ongoing use in COVID-19 clinical care pathways."
Because:
"The stakes of patient triage are simply too high, and the reliability of symptom checkers is simply too poor, to justify their ongoing use."
"The stakes of patient triage are simply too high, and the reliability of symptom checkers is simply too poor, to justify their ongoing use."
There were notable differences between nations, even across the UK. 
What vulnerabilities were considered serious also varied between UK nations...
N. Ireland was least;
Scotland depended only on shielding categories;
Wales a bit more than Scotland, and
England more extensive than others.
N. Ireland was least;
Scotland depended only on shielding categories;
Wales a bit more than Scotland, and
England more extensive than others.
So for example, if you are a 65 yr old diabetic with a cough and fever for 7 days (confirmed COVID +ve), you would be advised to stay home in N. Ireland, Scotland, but NHS Wales '111' online would advise to call 111.
Beyond not being able to determine severity of COVID, the most concerning aspect of the '111' online covid triage is that only NHS Wales considered age as a factor in triage decisions.
So an 80 yr old with a cough and fever for 7 days is advised by NHS Scotland, NHS Northern Ireland, and NHS England to remain at home, and would not receive a single call from a nurse or doctor or anyone checking to see how they are doing.
The CDC Coronavirus Symptom Checker also performed poorly. Despite an easy to use interface, it still classed mild to moderate shortness of breath in under 65s as not requiring any clinical input.
[To be clear, any shortness of breath with COVID-19 requires urgent care]
[To be clear, any shortness of breath with COVID-19 requires urgent care]
For those that don't know:
Triage is where an initial assessment is undertaken to decide how serious a condition might be and how URGENT medical care is required.
It is one of the most important parts of the care pathway.
The '111' online tool tries to automate the process
Triage is where an initial assessment is undertaken to decide how serious a condition might be and how URGENT medical care is required.
It is one of the most important parts of the care pathway.
The '111' online tool tries to automate the process
Getting triage wrong costs lives...,
but also the delay in presentation to medical care also delays recovery, and increases the burden on the health service (late presentation leading to longer stays and higher care needs e.g. ICU).
but also the delay in presentation to medical care also delays recovery, and increases the burden on the health service (late presentation leading to longer stays and higher care needs e.g. ICU).
None of the other countries analysed depended on the Symptom Checkers, except the UK.
The UK is the only country we know of that has replaced clinical triage with online, automated triage (i.e. the computer decides if you need medical help).
The UK is the only country we know of that has replaced clinical triage with online, automated triage (i.e. the computer decides if you need medical help).
There is a whole series of further questions here...
why is age not accounted for...why are the thresholds set so high to speak to someone...why the disparity within the UK...where is the pilot, safety data...And does the telephone triage use the same algorithm
why is age not accounted for...why are the thresholds set so high to speak to someone...why the disparity within the UK...where is the pilot, safety data...And does the telephone triage use the same algorithm
We do need these answered URGENTLY...
...over 6,000 members of the public over the age of 70 used the online COVID symptom checker last month alone. Likely, 150 older people will use it today to help them decide if they need medical help.
...over 6,000 members of the public over the age of 70 used the online COVID symptom checker last month alone. Likely, 150 older people will use it today to help them decide if they need medical help.
And as for the question, why was it used as the UK's COVID-19 Triage Service in the first place...
https://twitter.com/danielgoyal/status/1448971087581224985?s=20
There are no studies or discoverable internal quality control studies we could find @NHSDigital ...
When questioned about the first part of this study (completed April 2020), NHS digital stated the study was out of date:
bbc.co.uk/news/health-56…
When questioned about the first part of this study (completed April 2020), NHS digital stated the study was out of date:
bbc.co.uk/news/health-56…
Together with our most recent findings...
"...considerable improvements are needed to render the current NHS 111 COVID-19 symptom checkers fit for purpose."
Mansab et al., BMJ Health & Care Informatics 2021;28:e100448. doi: 10.1136/bmjhci-2021-100448
"...considerable improvements are needed to render the current NHS 111 COVID-19 symptom checkers fit for purpose."
Mansab et al., BMJ Health & Care Informatics 2021;28:e100448. doi: 10.1136/bmjhci-2021-100448
• • •
Missing some Tweet in this thread? You can try to
force a refresh
