I've had the real fortune of working at scripts research for the last 17 years it's the the largest nonprofit biomedical institution in the country and I've watched some of my colleagues who have spent two to three years to define the crystal 3D structure of a protein well now that can be done or two or three minutes and that because of the work of alphafold which is a derivative of Deep Mind Demis aabus and John jumper recognized by the American Nobel Prize in September what's interesting this work which is taking the amino acid sequence in one dimension and predicting the three-dimensional protein at atom Atomic level is now inspired many other of these protein structure prediction models as well as RNA and antibodies and even being able to pick up all the msense mutations in the genome and even being able to come up with proteins that have never been invented before that don't exist in nature now the only thing I think about this is it was a Transformer model we'll talk about that in a moment in this award since Demus uh and John and their team of 30 scientists don't understand how the Transformer model Works shouldn't the AI get an asterisk as part of that award I'm going to switch from life science which has been the singular biggest contribution just reviewed to medicine and in the medical community the thing that we don't talk much about are diagnostic medical errors and according to the National Academy of Medicine all of us will experience at least at least one in our lifetime and we know from a recent John's Hopkins study that these errors have led to 800,000 Americans uh dead or seriously disabled uh each year so this is a big problem and the question is can AI help us and you keep hearing about the term Precision medicine well if you keep making the same mistake over and over again that's very precise yeah we don't need that we need accuracy and precision medicine so can we get there well this is a picture of the retina and this was the first major hint training 100,000 images with supervised learning could the machine see things that people couldn't see and so the question was to the retinal experts is this from a man or a woman and the chance of of getting it accurate was 50% but the AI got it right 97% so that training the features are not even fully defined of how that was possible well that gets then to all of medical images this is just representative the chess X-ray and in fact with the chest x-ray the ability here for the AI to pick up the radiologist expert radiologist missing the nodule which turned out to be picked up by the AI as cancerous and uh this is of course representative of all of medical scans whether it's CT scans MRI ultrasound that through supervised learning of large labeled annotated data sets we can see AI do at least as well if not better than expert Physicians and 21 randomized Trials of picking up pop Machine Vision during colonoscopy have all shown that pops are picked up better with the aid of Machine Vision than by the gastroenterologist alone especially as the day goes on later in the day interestingly we don't know whether picking up all these all these additional popups changes the natural history of cancer but it tells you about machine eyes the power of machine eyes now that was interesting but now still with deep learning models not Transformer models we've seen and learned that the ability for computer vision to pick up things that human eyes can't see is quite remarkable here's the retina picking up the control of diabetes and blood pressure kidney disease liver and gallbladder disease the heart calcium score which you would normally get through a scan of the heart uh Alzheimer's disease before any clinical symptoms have been manifest predicting heart attacks and strokes hyper lipidemia and seven years before any symptoms of Parkinson's disease to pick that up now this is interesting because in the future we'll be taking pictures of our retina as checkups this is the gateway to almost every system in the body it's really striking and just we'll come back to this because each one of these studies LED was done with tens or hundred thousands of images with supervised learning and they're all separate Studies by different uh investigators now as a cardiologist I love to read cardiograms I've been doing it for over 30 years but I couldn't see these things uh like the age and the sex of the patient or the ejection fraction of the heart making difficult diagnoses that are frequently missed the anemia of the patient that is a hemoglobin to the desmal point predicting whether a person who's never had atrial fibrillation or stroke from the ECG whether that's going to likely occur diabetes the diagnosis of diabetes and pre-diabetes from the cardiogram uh the feeling pressure of the heart hyperthyroid ISM and kidney disease imagine getting an electroc cardigan to tell you about all these other things not really so much about the heart then there's the chest x-ray who would have guessed that we could accurately determine the race of the patient no less the ethical implications of that from a chest x-ray through machine eyes and interestingly picking up the diagnosis of diabetes as well as how well the DI the diabetes is controlled through the chest x-ray and of course so many different parameters about the heart which we could never Radiologists or cardiologists could never be able to uh come up with uh but Machine Vision can do it the Pathologists often argue about a slide about what does it really show but with this ability of machine eyes the driver genomic ations of the cancer can be defined no less the structural copy number variants that are accounting or present in that tumor also where is that tumor coming from for many patients we don't know but it can be determined uh through uh Ai and also the prognosis of the patient just from the slide by all of the training again this is all just convolutional neural networks not transform models so when we go from the deep neural networks to Transformer models this classic pre-print one of the most cited pre-prints ever attention is all you need the ability to now be able to look at many more items whether it be language or or images and be able to put this in context setting up a transformational uh progress in many fields the Prototype is the outgrowth of this is gp4 with over a trillion connections our human brain has a 100 trillion connections or parameters but one trillion just think of all the information knowledge that's packed into those one trillion and interestingly this is now multimodal with language with images with speech and it involves a massive amount of graphic processing units and it's with so self-supervised learning which is a big bottleneck in medicine because we can't get experts to label images this can be done with self-supervised learning so what does this set up in medicine it sets up for example keyboard Liberation the one thing that both doctors clinicians and patients would like to see everyone hates being data clerks as clinicians and patients would like to see their doctor when they finally have the visit they've waited for a long time so the ability to change the uh face-to-face contact as just one step along the way uh by having the Liberation from keyboards with synthetic notes that are driven derived from the conversation and then all the downstream normal data clerk functions that are done often off hours now we're seeing in Health Systems across the United States where people uh Physicians are saving many hours of time and heading towards ultimately keyboard Liberation we recently published with the group at morefields ey Institute led by Pierce Keane the First Foundation model in medicine from the retina and remember those eight different things that were all done by separate studies this was all done with one model this is with 1.6 million retinal images predicting uh all these different uh outcome likelihoods and this is all open source which is of course really important that others can build on these models now I just want to review a couple of really interesting patients Andrew who is now 6 years old he had three years of relentlessly increasing pain AR rested growth his gate suffered with a dragging of his left foot he had severe headache he went to 17 doctors over 3 years his mother then entered all his symptoms into chat GPT it made the diagnosis of occult spinabifida which many he had a tethered spinal cord that was missed by all 17 doctors over three years he had surgery to release the cord he's now perfectly healthy this is a patient that was sent to me who was suffering with what she was told long covid she saw many different Physicians neurologists and her sister entered all her symptoms after getting nowhere no treatment for long Co there is no treatment validated and her sister uh put this all her symptoms into CH GPT and found out it actually was not long Co she had limic en sephtis which is treatable she was treated and now she's doing extremely well but these are not just anecdotes anymore 70 very difficult cases that are the clinical pathologic conferences at the New England Journal of Medicine were compared to gp4 and the chatbot did as well or better than the expert Master clinicians in making the diagnosis so I just want to close with a recent conversation with my fellow uh medicine is still an apprenticeship and Andrew Cho uh is 30 years old in his second year of Cardiology Fellowship we see all patients together in the clinic at the end of Clinic the other day I sat down and said to him Andrew you are so lucky you're going to be practicing medicine in an era of keyboard Liberation you're going to be connecting with patients the way we haven't done done for decades that is the ability to have the note and the work from the conversation to drive things like pre-authorization uh billing prescriptions F future appointments all the things that we do including nudges to the patient for example did you get your blood pressure checks and what did they show and all that coming back to you but much more than that to help with making diagnosis and the gift of of time that having all the data of a patient that's all teed up before even seeing the patient and all this support changes the future of the patient doctor relationship bringing in the gift of time so this is really exciting I said to Andrew everything has to be validated of course that the benefit greatly out at weighs any risk but it is really a remarkable time and for the future of healthcare is so damn exciting thank you
No comments yet. Be the first to comment!