Read on Twitter
The medical industry will never be the same again.
Google has released Med-PALM2, their LLM specifically for medical purposes.
It scored higher than humans on a medical exam.
Heres the breakdown, and a glimpse of future of healthcare:
Google has released Med-PALM2, their LLM specifically for medical purposes.
It scored higher than humans on a medical exam.
Heres the breakdown, and a glimpse of future of healthcare:
Alright so what' is med-palm? It's google's version of palm-2, but fine tuned and trained specifically for medical purposes.
That means its much MUCH better than a generic LLM at
- understanding users questions
- understanding various health scenarios
- synthesizing answers
That means its much MUCH better than a generic LLM at
- understanding users questions
- understanding various health scenarios
- synthesizing answers
And here is the crazy part. It scored 86.5% on the MedQA dataset, the highest out any LLM.
The dataset contains general USMLE style questions. These are the same questions a person is applying for a medical license would answer in an exam beforehand.
The dataset contains general USMLE style questions. These are the same questions a person is applying for a medical license would answer in an exam beforehand.
When they did an independent long-form evaluation with physician rater, they founded that the answers were comparable to a physician answer! (basically doctors rated the LLM's answers)
Yes you heard that right, Med-PaLM2 was able to answer the questions as well as doctors.
Yes you heard that right, Med-PaLM2 was able to answer the questions as well as doctors.
It actually gets even better..
They asked a group of of doctors to look at answers given by Med-PaLM 2 and other doctors for over 1k health related questions.
The doctors actually preferred the ones from the LLM over the ones given by their colleagues in most cases
They asked a group of of doctors to look at answers given by Med-PaLM 2 and other doctors for over 1k health related questions.
The doctors actually preferred the ones from the LLM over the ones given by their colleagues in most cases
They also looked at how people with no medical backgrounds liked the questions, and it was a similar story.
The evaluations were very very close for Med-PaLM 2 vs physicians
The evaluations were very very close for Med-PaLM 2 vs physicians
Can you imagine a world where we have no more mis diagnosis dude to negligent doctors?
There should be no reason why you'd get the wrong advice from a doctor thats using some AI assistant.
There should be no reason why you'd get the wrong advice from a doctor thats using some AI assistant.
The report also talks about how there are still quite a lot of limitations, namely that:
- it hasn't been extensively tested in a real-world clinic setting
- evaluations were performed on specific datasets ( might not be applicable to all medical scenarios)
- it hasn't been extensively tested in a real-world clinic setting
- evaluations were performed on specific datasets ( might not be applicable to all medical scenarios)
- The outcome of these models in practice might vary depending on features like answer length and the level of detail provided, which can change the perceived quality of responses.
At the end of the day, I think medical focused LLM's have the power to turn healthcare on its head.
Doctors + AI = Better diagnosis, quicker turn around, more time for doctors to spend time with patients (instead of grunt work)
Better healthcare ftw
Doctors + AI = Better diagnosis, quicker turn around, more time for doctors to spend time with patients (instead of grunt work)
Better healthcare ftw
That's a it!
If you enjoyed this thread:
Give me follow me @SullyOmarr for more of these :)
If you enjoyed this thread:
Give me follow me @SullyOmarr for more of these :)
https://twitter.com/1328913688892346370/status/1660318717174071297
oh yah link to paper:
arxiv.org/pdf/2305.09617…
arxiv.org/pdf/2305.09617…
• • •
Missing some Tweet in this thread? You can try to
force a refresh
