---
title: 'Artificial Intelligence: Complications Detection During Endovascular Procedures'
type: 'lesson'
url: 'https://doc.social/courses/neurosurgery-and-artificial-intelligence/artificial-intelligence-complications-detection-during-endovascular-procedures'
course: 'Neurosurgery and Artificial Intelligence'
course_url: 'https://doc.social/courses/neurosurgery-and-artificial-intelligence'
position: '2 of 3'
author: 'John Bennett MD'
transcript: 'yes'
---

# Artificial Intelligence: Complications Detection During Endovascular Procedures

## Summary

_This lesson is a recording without a written summary._

## Transcript

9,540 words · 69 min · 5 speakers · automatically transcribed (aws-transcribe) and not checked by a person; it may contain errors, particularly in names, drugs and dosages. The recording is the source.

[0:09] Speaker 1: Good morning from Miami Beach, home of Neurosurgical TV. It gives me a great pleasure to introduce, uh, Cyprian Yonita, a PhD from University of Buffalo, Jacobs Med, who gave a presentation last week on Artificial intelligence and Neurosurgery, and that's going to, uh, do a presentation tonight on complication detection during endovascular procedures.
[0:35] Speaker 1: And we're tremendously honored to have uh a couple of, um, very noted uh Chinese neurosurgeons with us. Uh hello, Ben.
[0:46] Speaker 1: Hello.
[0:47] Speaker 1: Yeah, Ben's at, uh, can you just say a few words about yourself, Ben?
[0:53] Speaker 2: Yeah, I'm very interested in this topic, and, uh, I, I think it means some future for the neurosurge neurosurgery.
[1:01] Speaker 1: Yeah, OK, very good. Can you introduce your associate then uh that you told me about, so, uh.
[1:10] Speaker 2: So I'm just listening because this is my first time.
[1:13] Speaker 1: Oh, I know, I know, but you're there's an associate on the panel that you said is from Beijing,
[1:19] Speaker 2: yeah, yeah, yeah, uh, Professor Zhao Yuanli from Beijing, uh, Tiananang Hospital, and, uh, their hospital is, uh, so big, and, uh, the department, they have, uh, around 170.
[1:33] Speaker 2: And neurosurgeons.
[1:35] Speaker 1: Well, Cyprian, I would say, I would say you hit a home run. Thank you. OK, Cyprian, welcome, and it's all yours.
[1:44] Speaker 3: Thank you.
[1:53] Speaker 3: OK, so can you see this? Yeah, perfect.
[1:56] Speaker 3: All right. Thank you. So, um, uh, we started the lecture of a series of 3 lectures on uh use of AI for neurosurgery. Last week, we focused on the use of stroke for
[2:09] Speaker 3: Of the use of AI for stroke, and we dealt mostly with CT data and how we streamlined the patients by detecting fast intracranial hemorrhages, large vessel occlusions, aspect scoring, and then better ways of estimating the infarct core versus penumbra for selection of the patients.
[2:30] Speaker 3: For mechanical thrombectomies. Uh, today we're going to go one step farther, and it's what's happening after the patient shows up and uh it is uh selected for treatment. I have some disclosures. I'm supported by National Institute of Health and National Science Foundations, and I have some patents relevant to this work. So, um, last time we talked about the AI medical applications, and we thought that, um, uh.
[3:01] Speaker 3: For stroke, we will be dealing with demonstrate equivalence with human performance, optimized workflow, faster diagnostic and patient training. Today I'm going to focus on complication detections during the procedure, and next week I'll show treatment planning and treatment guidance with a prognosis of the surgical outcome in real time.
[3:24] Speaker 3: So a little bit the, the, the work that I'm going to be talking today, it is what's happening during image guided endovascular interventions where you take a catheter under fluoroscopy and you treat some sort of lesion into the brain.
[3:44] Speaker 3: Using different devices and mostly I'm going to be focusing on the aspects that potential complications and detections of those complications in in real time.
[3:56] Speaker 3: So during this procedure, the only way we can look at the the vasculature is with a process like shown in here which is called digital subtraction and geography. I apologize if this is too basic for some of you. I just, but I will, I'll go.
[4:12] Speaker 3: Fairly fairly fast through through some of these procedures and until now a lot of these procedures have been used to just look at structure like stenosis, arteriovenous malformations, and intracranial aneurysms, etc. Very little is paid or it's used.
[4:32] Speaker 3: During this procedure to look at the quantitative aspects of the flow, and there are some challenges that I will go through in these presentations and how we can bypass them using some AI aspects. So bottom line, at the end of the day, these things are used mostly for structural and not enough for functional and information. So.
[4:58] Speaker 3: Last time we we were in this domain, what's happening before we got to the angiographic room and during the procedures. Now we are in the angio room and during these procedures we have systems like this biplane systems where an X-ray detectors on two directions, and these are the kind of the data that we that we acquire. The key.
[5:24] Speaker 3: One aspect is that if we are here in this domain at the city, we take the city data we can upload it into a cloud we can upload it into a different kind of a program and analyze and do some sort of prognosis and predictions, but when we are in the angio room, everything is contained. We do not have access to this data unless we upload them into the packs of the hospital.
[5:49] Speaker 3: And then we, we analyze and we come back to the doctors and tell them what's going on that can add easily between 5 to 10 minutes. That is a big no no in the neurosurgery field, especially when you have catheters inside in the arteries of a of a patient, and so, so.
[6:11] Speaker 3: The general Approach it is we do the procedures, we do the treatment, treatments, and then we wait and see, take them to the to the CT scan and we diagnose, and it's sort of a
[6:23] Speaker 3: A diagnosis amaurosis during this during this time. So what we can do with this nowadays, it is we can add some quantitative aspects and no worries, I'm going to go in full detail about this quantitative analysis. We can process some of these images and then using AI we might be able in real time, in a fraction of a second, to tell the doctor if something went wrong.
[6:53] Speaker 3: And a lot of the things that I'm going to be focusing today, it is in deficit in flow, and this can be a cure that you dislodge a plaque, you.
[7:06] Speaker 3: inject an air bubble or place a device that blocks a small perforator or a bunch over a smaller artery that you shouldn't be blocked like a flow diverter placed over the ophthalmic artery or so. So the idea is that can we detect this in real time and tell the doctors, hey, you should check this area because the flow has been decreased in here or something. It's fundamentally not, not right, and.
[7:36] Speaker 3: They can look at it, say it's fine, or can take the patient to observation immediately after the procedure. And again, this is for an AI system that resides on the C arms. It is fundamentally different.
[7:53] Speaker 3: From something like uh I shown last time where anyone can develop this kind of a pro uh uh uh uh software have them on the cloud and uh and do this kind of a processing and diagnosis, etc.
[8:07] Speaker 3: So I will just go with a really quick 101 quantification of the of the flow in planar angiography, and the idea is that even when we look at this kind of of angiograms, as the contrast goes through this system, it follows the laws of physics and convection, and as the curve goes as the contrast goes through this system, it flattens and it widens.
[8:36] Speaker 3: This is exaggerated but it really in reality this is what happens uh and uh if we have fine analysis kind of a uh um.
[8:50] Speaker 3: Systems or data or software we might be able to pick up on the subtle differences which sometimes they are not quite obvious for for the human eye and what we do we parameterize these curves we take all sorts of parameters and I I would just
[9:09] Speaker 3: Choose today a few of them in transit time, time to peak, time to arrival area end of the curve, etc. but you can parameterize these curves in about 2 dozen ways. So and everyone has has its own Approach in in doing this this kind of analysis.
[9:32] Speaker 3: John, can you, can you smooth, OK, thank you. So, um, this is kind of the, the way they work, the software works. Takes the angio data. It does some sort of a processing to make them smaller to increase the, uh, the, um.
[9:48] Speaker 3: The processing ways, it looks at every single pixel into the image like we look into the CT and CT perfusion, then we discriminate to eliminate some of the background and make sure that we count only structures that matter and again this is to to to speed it up and then.
[10:06] Speaker 3: Sometimes we use fitting data because it is it takes some, it smooths it out and it takes care of some of the noise and then we calculate for every single pixel into that image this kind of maps and you get pictures like this. And what is this thing? What are these pictures in here? It is a
[10:30] Speaker 3: Like you're looking at an object with different filters.
[10:36] Speaker 3: You are looking at the vast glacier with different filters, and every single one gives you one piece of information about the hemodynamics and the flow into that area.
[10:47] Speaker 3: Peak height tells you how much of the contrast is going in. Time to peak, it tells you what is the delay. Mean transit time, it tells you things like how long a particle of contrast goes into an area and stays there.
[11:02] Speaker 3: That can be related to the stasis if you're thinking of the aneurysm. So every single picture that is in here, it tells you one story about the hemodynamics, but not only it tells you things about the physiology in an area, it tells you also where it's located. So this kind of maps gives you information about location and the function. There is a big problem with them.
[11:33] Speaker 3: The problem is that these are planar, so you're looking through the entire skull, through the entire brain when you're looking at this. Everything is overlaid and the signal of the lesion of the of the disease, it is very small.
[11:50] Speaker 3: Which makes every single of the parameters weakly correlated with the underpinning disease or with the physiology, but it is a lot of data. So what we are going to do now, since we cannot really
[12:05] Speaker 3: Correlate one parameters with outcomes or with the severity of the disease, we are going to go into data-driven models in order to use all this information in context of a large amount of data that these hospitals are generating and see if we can solve the problem. So this kind of angiograms, I would say per patient you can take about a dozen of them per procedure. So and
[12:33] Speaker 3: A lot of them are getting discarded and
[12:37] Speaker 3: The idea is that we sit on a humongous
[12:41] Speaker 3: Probe of data set, it's golden. Most probably it is more data than any Other imaging modality in in the in the neurosurgery department or in the cardiovascular departments. And yet we're not using this. We're just pass it along. Some of it gets deleted, deleted, and we just save snapshots of of normal DSA, so.
[13:06] Speaker 3: And that being said, first question that we're posing as, as we said before, AI, it is new to the medical field, and if we do not take baby steps into it, it will be not, it will be.
[13:21] Speaker 3: A potentially discarded. So there are certain steps. First you show that it's possible. Then you show that it matches the human performance, and then we move and say we can do better.
[13:35] Speaker 3: So in this study what we looked at is we posed the following questions can the AI
[13:43] Speaker 3: Grade, um, uh, uh, grade, um.
[13:47] Speaker 3: Um,
[13:49] Speaker 3: Uh, the perfusion following mechanical thrombectomy. In this case, I, we're using the modified Tiki scores following mechanical thrombectomy. There are different grades ranging from no perfusions to grade 3 of complete perfusion. So a patient.
[14:07] Speaker 3: That comes with a large vessel occlusion. They place a clot retrievers and then try to pull the clot out and after each one of them, after the first pass or second pass, sometimes they do another angiogram and grade that patient based on this, this kind of semi qualitative quantitative assessments, and we pose the question whether the AI may be able to do something similar.
[14:36] Speaker 3: And in the same way we take every, we took every single angiograms and several and generated several maps for every single one of them. So if you look in this case, these are the map of peak height area under the curve, time to arrival, time to peak, MDT, and so forth. So based on these maps, if we look at them in general, we can't quite say what is the status is the status of a
[15:03] Speaker 3: Of the of the reperfusion. So for us.
[15:10] Speaker 3: What we are going to do is have all this angiographic parameter or individualism and combine them in order to have an improved predictive strengths on these cases, so.
[15:26] Speaker 3: If you remember from last time, we are in this context, we want to use the AI and ask the question, what is this?
[15:33] Speaker 3: And before we had cat, cat, dog, and so forth and so on. Now we have tiki 0, tiki 2, tiki 3, tiki 2B, and so forth and so on. And we go through an iteration where we try to find the function that takes this image into a label preset and where which has been kind of provided.
[15:53] Speaker 3: By the neurosurgeons and we keep going through this iteration over and over again until the AI is going to be able to identify this is a dog and for us it's going to be this is an empty key of 2B or so so this is kind of the iteration that we showed last time.
[16:14] Speaker 3: And I'm not going to go into too much of a detail breaking this down on how the AI is taught and so, but that's just, just showing it to give a perspective of how the process goes. Now the problem is we have a whole bunch of maps or we have in this case two C arms.
[16:40] Speaker 3: And the idea is that we have different options in here. We can train a neural network or a a program like I showed you in there per map.
[16:52] Speaker 3: And then we have all these AI structures or all these AI parameters combined, or we can have multiple maps for each of these sources and then combine them per view. We take lateral versus AP view, and these kind of games can be played not just with the imaging.
[17:15] Speaker 3: A source like this, it can be, for example, you're looking into the EMR and you want to bring some patient characteristics. You might want to have complications based on age, smoking, drinking, etc. and so on. BMI. Every single one of these networks can be different, and this, in this case that I'm going to show in here is just based on the hemodynamics and and complimentary information that comes from the from the biplane.
[17:45] Speaker 3: A now for these weights in here there's different ways you can Approach them. You can count them equally. You can count them that they are all normalized. They add to one contribution, or you can do again a retraining. So you have all this individual Artificial intelligence.
[18:08] Speaker 3: Maps and this can be done for each one separately and then you go again through a final training where you recalculate these contributions from every single of the maps. So we have done a study using 383 DSAs.
[18:25] Speaker 3: And uh we are looking at patients with uh um occlusions uh in the from ranging from uh M1, M2, and ICA also tandems and a few others. And these are the distributions of the MTT scores that uh that we have uh um.
[18:43] Speaker 3: That we have used, uh, our neuro interventions were involved into scoring every single, uh, every single DSA, and, uh, if we had cases with disagreements, then we had them, uh, uh, convene and decide, uh, decide, um, what's going on with, uh, uh, what is the correct uh uh MTT score.
[19:06] Speaker 3: So the biplane data has been, has been data has been acquired from every single one, from every single image plane, and then I'm not describing in this presentation, but we do a lot of image processing and we do a lot of Other things to account for surgeon's variability because two surgeons are going to inject differently.
[19:35] Speaker 3: And we're also going to account for different imaging views or what we call for shortening because those C arms can be pointing in all directions, but I'm not going to insist on, on, on those technical aspects. I'm just going to focus on the implementation clinical and clinical applications in here, so.
[19:59] Speaker 3: You have to look at it and be careful about because these DSAs can be very long. You can do some image processing to use only data.
[20:11] Speaker 3: Enhancing structures that you are interested in.
[20:15] Speaker 3: Because if you keep the entire sequence sometimes you might run into troubles because you add structures that do not matter and it's just if you add structures that do not matter with regard to the task you want to have, then you need a lot more data.
[20:34] Speaker 3: So
[20:35] Speaker 3: As a rule of thumb for the AI, whatever you think and what your gut feeling, not your gut, your experience, your, your reasoning actually, based on years and years of doing this kind of cases, whatever it doesn't make sense, take it out.
[20:52] Speaker 3: Just keep the things that matter in order to train the AI, and then and then things are going to become a little bit easier.
[21:02] Speaker 3: And we, we, we went through the same process and we get these maps for every single one for lateral, for I'm sorry, for the AP, anterior posterior, and for lateral, and then every single one goes into this training algorithms where that paradigm that I showed with the dog is done for every single map where we train versus the labels that the doctors have provided.
[21:31] Speaker 3: And then we assemble them and we run it again.
[21:35] Speaker 3: And here and we write it again in here in order to recalculate those weights.
[21:40] Speaker 3: It's to decide how much weight is this map of course going to contribute to my decision, how much this map, and how much is lateral, and the and so forth and so on. And then we are going to do a classification and in here I'm going to do it in just for to showing for just the two groups.
[22:01] Speaker 3: So here is kind of the architecture and it's just going into an input. We do some convolutions and and these are in general some sort of a
[22:13] Speaker 3: Convolution simple convolutional networks, uh, we are usually training the data set on about 70% and then we do a testing set 20% that is never used for the training. Again, we never use this this kind of things for, for the training in order not to bias the um.
[22:37] Speaker 3: not to bias the network. And then there are Other things like details of of the network that you use training optimizer atom, loss function binary cross entropy and this is important because next week I'm going to talk about how we can add some physics into this more than just binary cross entropy and
[23:01] Speaker 3: Uh, and, um, um, and, and, um.
[23:06] Speaker 3: Once it's is it trained, we are actually for every single training, I apologize, we do what is called the Monte Carlo Carlo cross validation, and this is a requirement from I'm sorry, Radiology Society of North America.
[23:22] Speaker 3: The RSNA has has published a set of about 10 rules that should be used every single time you're going to do a an AI training or validation, and among some of that is that
[23:38] Speaker 3: You need to make sure that your data set.
[23:42] Speaker 3: Or your AI is not trained using just a lucky sort of splitting of the cases and in this case we are using we split it 20 times and we run every single time the training and we test it.
[24:00] Speaker 3: And then for every single one we go and evaluate it using all sorts of things like area under the curve for ROC sensitivity, specificity. Matthew correlation coefficient is very important for AI training and so, so I highly recommend people that do this kind of analysis to go and use this uh this Approach and here are the results.
[24:25] Speaker 3: The uh just a simple kind of like feasibility study onto this, it just shows that this kind of AI.
[24:35] Speaker 3: A techniques can come and predict really well this kind of outcomes. You look at an accuracy and you have an 82.7. You have an area under the ROC. It is about 0.86. The key is if I would take two interventionalists and I would ask them what is this, the MTQ2B or MTK2C.
[24:58] Speaker 3: Most probably they will kind of also diverge in there so uh I think as we are looking at things to kind of like unify on these labels.
[25:10] Speaker 3: We might be actually able to um.
[25:13] Speaker 3: To get even better, better prognosis because at the end of the day, the AI, it is trained on these images and what we tell the AI that these images represent.
[25:27] Speaker 3: If the doctor says it's to be MTK2B is going to say all the next one is MTK-2B, so you see there needs to be some sort of a consensus into the labels in order to kind of like have this AI is properly trained.
[25:42] Speaker 3: And here is an example where we take the images and we look at them and we predict predict in this case, hey, uh, we are confident with a 61% that this is an MTK2B2C or 3, so.
[25:59] Speaker 3: Um, what is key about this is that all this, it is stealth, is supposed to happen on the CRM itself onto the software. The doctor does not need to go and click this, click that, or do any kind of adjustments. It is right there happening in the background, and in a fraction of a second it just comes back and tells them this kind of scores and evaluations.
[26:28] Speaker 3: And again this is just the initial part when we start to say, hey, is this possible and indeed this kind of analysis in here kind of like showed without doubt that we can evaluate this kind of really, really fine structures and so on. But the next thing it is, well, can we do it even better? Can we go to the next step and
[26:56] Speaker 3: and do something like, can we do better than humans or can, can we match the human performance and in order to do that we need to do two things. We need to actually not only to be able to say, hey, there is a deficit deficit into this area, but where is it?
[27:15] Speaker 3: And that is definitely if we look at the CTs we have these kind of things pre and post diagnosis for infarct cores and IC ages where you can kind of look and say, hey, here it is a problem. He is an IC age and here it is the infarc core and here is a penumbra and so can we do kind of the same thing again into the angio suite? And the problem with that in here, if we look at this kind of a
[27:44] Speaker 3: Uh, injections and we have going from arrival all the way through the end and we pose the question, where is the problem in here? A very experienced neurosurgeon probably will be able to say it, an entry level or a fellow might be struggling with it. Nonetheless, can we have the AI?
[28:07] Speaker 3: Capable of doing this and the way we approached this is that we went and
[28:14] Speaker 3: automatically place these kind of regions of interest into the onto the skull or onto the AP and lateral view of the C arms. There is 4 by 5 into the AP directions and there is again 4 by 5 into lateral, which kind of like if you think about in 3 dimension is 20 by 20, it's about 400, this kind of a.
[28:42] Speaker 3: Um, this kind of regions of interest in there, and we have done a study where we used about, I think in this one we, it wasn't too, too much, too big of a study we used 25 subjects in here and we have looked mostly at patients that got TK2B and TK33 after, after the mechanical thrombectomy.
[29:11] Speaker 3: And um the study goes like here. So these patients went for mechanical thrombectomy and immediately post procedure, we had them have an MRI, an MRI.
[29:23] Speaker 3: We went and took flair data and went and segmented.
[29:29] Speaker 3: this kind of
[29:32] Speaker 3: This kind of images and then we have recreated.
[29:37] Speaker 3: Views, 2D views or a um lateral and AP.
[29:45] Speaker 3: Of this inf cores to match the DSA data.
[29:50] Speaker 3: As you see it in here, so we, we took this kind of now projected location of the infar cores. We overlaid over the DSAs like you see in here, and then we looked at exactly the same parameters. What you see in here is time to peak, mean transit time, bolus arrival time, area under the curve, and, and we have looked at.
[30:17] Speaker 3: Patients or areas that had deficits in flow versus the ones that didn't, and when you start looking at this, you start seeing that there is already a correlation in between this, which means that you don't need to go all the way through like very complicated Artificial intelligence kind of a software or the one that requires a lot of training. You can go to something simpler like something maybe like a
[30:45] Speaker 3: SVMs support vector machine.
[30:49] Speaker 3: So we, we take this all the data, we find this what is referred as hyperplanes to separate the regions of interest that have deficit and flow versus the ones that don't.
[31:04] Speaker 3: And then we have trained it.
[31:08] Speaker 3: And applied for new cases.
[31:12] Speaker 3: Which obviously again had MR immediately post procedure and what we have figured out, we found out that this kind of a procedure, this kind of Approach, not only can tell you that there is a problem with this kind of uh uh with the flow, but it tells you the areas affected by this uh this um this flow deficit.
[31:37] Speaker 3: And when we look at again what is the ability of the AI kind of a software to not only tell you there is something wrong but also locate it, we start seeing accuracy of an 80% area under the ROC 86 0.0.86, which is even better if you remember than some of, some of the things when we looked at the global aspect of the flow, so.
[32:07] Speaker 3: And it's better because I think we had very clear labels, also simpler labels, you know, in our core or not in our core. Well there we had about 5 or 6 grades in the modified TT scores. The more labels you have, the more difficulties you have. The more data you need to train the AI in general.
[32:33] Speaker 3: So the um the idea is that now we have this kind of an issue, this problem, but uh this kind of a solution, sorry, but what we try to do next, it is actually in the real time again in the angioro suite, not only tell them about the, the doctors and the neurosurgeon interventionalists about, hey, there is something wrong in here.
[32:59] Speaker 3: But have a neurologist label every single one of these ROIs. Uh, neuro neurosurgeons, uh, especially some of ours, they told me, I do not need you to tell me this. I know exactly what this patient is going to happen to them when this is at the end of the day. Um, Others, they said, Hey, this is really great, uh, you know, I would like to know it.
[33:23] Speaker 3: Because if I see a clot in the M2 and you kind of like give me an idea about what deficits these patients might have when they come out, um, I might be actually.
[33:37] Speaker 3: Weight the benefit of these patients versus the risk of putting a clot retriever into the M2.
[33:44] Speaker 3: and decide to go for it.
[33:47] Speaker 3: So this is indeed very, it is debatable, and I think you can already look at it and see the problems that might raise within the context of the regulatory office like Food and Drug Administration and so.
[34:07] Speaker 3: The Other ways to look at it, we might actually not look at this kind of deficits into the uh uh uh potential deficits into the patient, but actually do an MRI score um.
[34:24] Speaker 3: Uh, and, um, uh, a modified ranking score, and that's a, that's uh for uh um.
[34:32] Speaker 3: Uh, that's another outcome for this kind of treatments. So the next thing is a little bit different, um, and now we're going to move even farther. Now, placing the ROIs is cool and, and so, but can we bring
[34:51] Speaker 3: The kind of like uh uh the performance of the CT or MRI.
[34:57] Speaker 3: Into the angio suite.
[35:00] Speaker 3: And that is a different kind of a problem, and you remember last time I showed you this where we tried to look at an image and then we segment it and we say, hey, ones are a person and 3 are plants, 2 is a purse, building structures are on the side. What if we could do something like this also for this kind of images into the into the angiographic suite.
[35:27] Speaker 3: And just to remind you what, how this is done, it is so done, it is used so-called convolutional neural networks where we have some filters or it's some mathematical filter like this and we pass it across.
[35:44] Speaker 3: The entire image we do all sorts of calculation to that. Then we do something recalled as pulling or an upsampling pre layers where we take pieces of the images and we compact them and we make them smaller and smaller and smaller.
[36:01] Speaker 3: And you see structures like this as it goes, and what's happening, you take the image, you apply those mathematical filters, and you make them smaller and you apply again those mathematical filters and you make it smaller and so forth and so on. You get in here at this bottom layer where you kind of like
[36:19] Speaker 3: encode the information in the image and then you go back up again and you output an image that identified every single kind of a structure in there so we try to do the same thing.
[36:34] Speaker 3: And this is actually how we started. We were very ambitious in the beginning. We want, we thought we're going to go, we're going to take these images and we are going to output stuff like this where the infar core is. So we took again parametric imaging maps. We took ground level label in our cores, and we wanted to have the network or the AI predict the location of the infarct.
[37:01] Speaker 3: And in order to do that again, we went to the patients that we had the data and geographic data and immediate after the procedure we had the flare MRIs, DWIs, etc. and what we did, we take this, this data from the MRI, we rearrange it to recreate like a view, a projection, and you kind of like see the, the data you see here is quite a
[37:29] Speaker 3: Quite degraded, but it doesn't bother us since we are kind of like see pretty clear where the infarct cores are and so.
[37:37] Speaker 3: And what we do, we take that data and we co-register it with the AI with the uh with the data from the angiography, and then we train the AI, we take those those labels.
[37:53] Speaker 3: And when this kind of in this case you look at the mean transit time, which is the average time it takes a particle to reside into a certain area. So we take those kind of a we take this kind of maps and those MRI co registered data.
[38:15] Speaker 3: As you see in here, these are SMSsks, and now we train the Artificial intelligence, in this case something like a unit you've seen over there, in order to recognize that this pattern in here corresponds to an infarct core, and this is what the AI, for example, gives us.
[38:34] Speaker 3: It gives us a probability field where it tells you that, hey, here I think it is Zein Farquor.
[38:44] Speaker 3: And you can see it, you actually, we might be able to do this in real time. That's the idea is to do it in real time during the procedure. As you take your catheter out, you take your, uh, you take your clot retriever out.
[39:00] Speaker 3: You can navigate it to the area.
[39:03] Speaker 3: Do the changes, do the uh the procedure, and it will tell you in real time what kind of a flow changes has occurred into this area. And when we compare, for example, with the projected views of the MRI, these are the kind of uh uh the uh the correlation.
[39:21] Speaker 3: I have to say this is, this was our greatest. It was about 2 years ago. This, this thought was our thought initially and we thought it's going to be a walk, not a walk in the park, but quite straightforward, and it wasn't. And then that's where we decided, hey, let's step back and we go through what I showed you in the first application, second application, and now.
[39:51] Speaker 3: We're, I think, ready to go into uh back to this problem and solve it, because it's not always working as well, but I think now, two years later, we have a lot better understanding on how to do this, and I have to say that it is extremely important that you understand that just taking DSA data.
[40:16] Speaker 3: From uh from the CRM without.
[40:19] Speaker 3: Doing accounting for all the things for differences in imaging, differences in hand injections. If everybody in this world will inject with an automatic injector, I think the problem will be straightforward, but it's not. And they, and I highly doubt that a lot of the neurosurgeons are going to do for every single angiogram they are going to take, they are going to actually connect the automatic injector.
[40:49] Speaker 3: So you just need to take uh an image, a snapshot or a, or a sequence, see what's going on, and keep uh keep moving.
[40:58] Speaker 3: So as I said, the, the issue with the data that I show or with the analysis that I've shown you is that
[41:10] Speaker 3: We are constricted to the 2D world.
[41:15] Speaker 3: And we are constricted to just plan our imaging because we want to save those.
[41:21] Speaker 3: Uh, we cannot do a CT scan to get every single time to do a CT fluoroscopy to guide catheters and so because the dose is a problem.
[41:32] Speaker 3: But there are ways to go around this issue and I want to show you where we're going with this to even improve the AI performance and bypass this to delimitation.
[41:46] Speaker 3: And get into the AI driven reconstructions to help us with this kind of processes.
[41:53] Speaker 3: So if you look at the normal Chromebeam CT data, this is the reconstruction, but when we acquire views of this, just one slice, when we look at the projections, this every single line, it is almost like a like a DSA or an angiogram in here.
[42:17] Speaker 3: This is kind of like what the data looks like. Uh and um the idea is that in order to get this, you need about 180 degrees of projections or uh that's what the CT scans do around.
[42:31] Speaker 3: One of the questions that we posed is, hey, what if I have only a little bit of the data? Can we actually get back this reconstruction? And it is from the point of the AI is that I show you only the nose of the person, and somehow the AI reconstructs the entire face.
[42:52] Speaker 3: Uh, it's a, it's a little bit, uh, uh, in a way in that form, so.
[42:58] Speaker 3: A go from full fledged data to fragmented data and be able to reconstruct the the entire information and this is done with what is referred as a generated adversarial networks and in here it is exactly the same as we've shown you before that data in.
[43:21] Speaker 3: Something give me back the actual good image and it's again one of those units that I've shown, but unlike this one that we showed you before, there is another.
[43:33] Speaker 3: Network in here that analyzes every single image that this guy generates. It is like this. It's a thief or a money counterfeiter that tries to make fake money.
[43:47] Speaker 3: And this in here is the police trying to figure out
[43:52] Speaker 3: Which one is fake or which one is uh is is uh is a it's a legit uh um.
[44:00] Speaker 3: Uh,
[44:02] Speaker 3: Uh legit money. So, um,
[44:07] Speaker 3: And we keep this training where this one tries to fake this image and make it good and this guy tries to discriminate until we reach a peak and when we reach the peak,
[44:21] Speaker 3: We stop the training and we take this part from here and we use it to reconstruct the data and in here you can see.
[44:30] Speaker 3: But for example, data that is ground truth, this is what you can get without with just fragmented data. It's bogus. You pretty much see nothing in there.
[44:42] Speaker 3: But when you apply this kind of trained networks, you get back the entire, the entire information, and you get it with about a third of the data that you have, or in this case maybe 45 degrees.
[44:59] Speaker 3: And the idea is to go even smaller and smaller in such a way that even when we have just maybe just two views or a little bit of a previous information about the 3D data, we may be able to bring the 3D information back into the engine room not only for diagnosis purposes purposes but also for guidance.
[45:22] Speaker 3: And if you're wondering, well,
[45:25] Speaker 3: Is this guy telling us the truth? We, we took that CPD trained or that GEN network, that AI that was trained on sparse data. We applied it to phantoms that we know precisely.
[45:42] Speaker 3: What it is and everything that that works in there and we lost something in here like this fine structure but then when we look at the gray values and it will come back to the Huntsville units, you know, degraded after using just a little bit of the data and classic reconstructions, but once you apply the AI, it's coming back and what you have to realize this is again, it's about 150 milliseconds per slice.
[46:11] Speaker 3: This is really, really fast happening again.
[46:14] Speaker 3: And look at the linearity. This is the linearity before taking with normal scanners, and this is the one that we take after using the reconstruction algorithms. So how does it, how can we use this into the engine room? Well, here is the results for using it on the actual reconstructed data.
[46:39] Speaker 3: On a sea arm, if you're going to take a full scan around, this is a ground truth.
[46:45] Speaker 3: When you have things like this, this is the vasculature that you have, but then.
[46:51] Speaker 3: The idea is just use the 45 degrees. It's like you do the arm, you do 20 degrees into this direction, 20 degrees into this direction, and in a system like Siemens.
[47:03] Speaker 3: Can do this kind of things.
[47:05] Speaker 3: You just swing the arm a little bit. You take your angles. If you try to reconstruct that, forget it. But if you try to to apply a uh artificial intelligent reconstruction, it brings you back.
[47:19] Speaker 3: Almost the entire 3D and again we think that taking this into the next step of reconstructions and combining it with AI for prediction of infar core where we try to move into um uh we are going to try to move into predicting um delayed cerebral ischemia following subarachnoid hemorrhage.
[47:46] Speaker 3: And uh uh maybe looking into the AVMs and predict uh you know, which uh which uh for which artery you should go first to block it and and so forth and so on. So this altogether and for, for the AVM in particular, 3D, it is extremely important and and so.
[48:08] Speaker 3: So to conclude, what we are trying to do in here, it is create or have an AI that is not in the cloud, that is not somewhere on your laptop. It is on this system. It is right here, communicates directly and analyzes every single image that it goes, goes through.
[48:30] Speaker 3: It does the magic, its own magic in the background.
[48:35] Speaker 3: Doesn't bother anyone.
[48:37] Speaker 3: And then only on the same screen that you have in the engine room comes and makes recommendations. It's not taking over the diagnosis capabilities or over the ability or over the practice of a neurosurgeon. It's just enhancing their practice by saying, hey, with a probability of 60%, I think something has occurred into this area.
[49:04] Speaker 3: Uh, please check it.
[49:06] Speaker 3: You can check it in the angio room or you can immediately say, follow this patient into the ICU. Give me an MRI. We showed that how you can use this into the internet on the C arms. The idea is to use it during the procedures, kind of like make the recommendations without uh without bothering the surgeons. They can accept them or can choose to follow them up.
[49:34] Speaker 3: Um, they are really fast. Everything happens in less than 1 2nd.
[49:40] Speaker 3: And this is, I would say, the biggest key. We cannot work this by ourselves.
[49:47] Speaker 3: We need to work this with the people that make the sea arms with the Siemens, GEs, Phillipses, and Cannons of the world, because then it's losing its value overall.
[50:01] Speaker 3: So these are the uh faculty that uh make this happen and help me with this. The Dr. Adnan Siddiqui is our director of Research. Doctor Ead Levi, it is our uh chairman. And actually, the all the software that I've shown you today and all the work, um, it's done by these uh people. These are the PhD students in my group, um.
[50:28] Speaker 3: And um a lot really, uh, the more senior ones, uh, I have to keep up with them. They are, they become so good at programming and uh and uh and and making these networks and analyzing them that it's a, it's a sometimes a challenge to keep up with them. We are also looking really eager to, to collaborate with others. This is my email, so email over there, uh, you cannot build.
[50:57] Speaker 3: Um,
[50:58] Speaker 3: Good AIs or are relevant AIs with data from only one center. So, as a matter of fact, we cannot do a proper AI or that applies to a large population with data from only one country. So the idea is that without a
[51:19] Speaker 3: Um
[51:20] Speaker 3: Large collaborations, these, these things will not go very, very far. We're not as fortunate as Other AI applications where people post billions of pictures of dogs and cats and Other things and tables and things on the internet. Our data is actually quite restricted, and I understand that there is a huge problem with patients' confidentiality, and I think only by
[51:50] Speaker 3: Kind of like bringing it up over and over again to these people, regulatory people, and so maybe they will start understanding that, you know, we can uh we can work around it without actually revealing anything from the patients, uh, any information. We just need the images and we need to know.
[52:12] Speaker 3: What happened or what every image represents and uh hopefully this uh this will will move forward. I'm done and I'm open for questions.
[52:24] Speaker 1: OK, fantastic, Cyprian. It's great to bring your world to our world of neurosurgery and, and you have a really good audience here. I mean, a fantastic audience. Uh, Louis, are you there? You just, you, I know you're a little late. Are you there, Louis?
[52:42] Speaker 1: Uh, Louis Barber, yes, I'm here. Hey, Louis, good to see you. Good to see you.
[52:48] Speaker 1: Yeah, I, I, I know you came late, but I just want to introduce you. Uh, uh, he's reaching a lot of people, and you're, you're a very important person to reach. It's good to have you here. I'm
[53:00] Speaker 4: humble here. I'm here to learn. Very impressive work, very impressive work.
[53:05] Speaker 3: Thank you very much. As I said, these are, these are the folks that make it happen.
[53:11] Speaker 1: OK, it's open now for comments or questions.
[53:16] Speaker 1: Yeah
[53:19] Speaker 4: Oh, Professor Xu Minghu, hi, hi, how are you? good to
[53:23] Speaker 2: see you, Professor Luis. How are you? Fine, uh, thank you for your, uh, fantastic presentation. So, uh
[53:35] Speaker 2: Did you ever uh analyze the chronicle ischemic conditions?
[53:42] Speaker 2: Like Moya Moya.
[53:44] Speaker 2: Because most of your work was, uh, uh, analyze, uh, uh, emergency, uh, ischemic uh patients like, uh, stroke.
[53:56] Speaker 2: The MCA or ICA occlusion cases.
[54:02] Speaker 3: The, the ICAD in Moa Moya, it is actually Ongoing. We have one of the of the students looking into getting, uh, gathering the data, and I think the bottleneck as usual for these things, it's always a database.
[54:21] Speaker 3: It will take maybe like half a year to go into the packs of the hospitals, download these images, and label them.
[54:29] Speaker 3: And then start thinking about looking at them, try to make sense, and if you see things visually, then you go to the AI. So ICAD and Moya Moya, it is right in the beginning, and we have the questions, especially for, for ICADD is what to do.
[54:50] Speaker 3: Uh, should we go and, uh, I do the angioplasty? Should we just observe it? Should we put a medication and those kind of things. But, uh, yes, it is, uh, it is on one of the things that we would like to tackle.
[55:05] Speaker 2: Mhm
[55:08] Speaker 2: Yeah, I think it's also helpful in analyzing the effect of the Surgery.
[55:16] Speaker 2: You can see the reperfusion and the, and the improvement of the perfusion after the Surgery.
[55:24] Speaker 2: Yeah, use this AI, I think it's also very helpful.
[55:29] Speaker 3: Yeah
[55:32] Speaker 3: So thank you. Yes, uh, uh, and as I said, the data set, it's uh everything, and especially labeled data set with non outcomes.
[55:42] Speaker 1: I'm just gonna bump you off the screen share there a second. There we go. OK, there you go.
[55:52] Speaker 1: OK, more comments, questions for uh Cyprian?
[55:57] Speaker 3: Mm
[56:00] Speaker 4: Nick, Nick, when.
[56:02] Speaker 4: Supposing here we are using this uh for endovascular procedure, but
[56:11] Speaker 4: Use the ICG from the microscope.
[56:16] Speaker 4: See the intraoperative angiogram with ICG.
[56:21] Speaker 4: Do you think it is possible to predict something?
[56:25] Speaker 3: Um I have.
[56:27] Speaker 3: I know the folks at the Johns Hopkins are looking into something like this. And uh using, using just a microscope. Um, I am not sure uh that uh
[56:41] Speaker 3: Uh, I will jump into, uh, as, as, uh, let's put it this theme. I'm not sure that we are going to jump into the, uh, uh, I'm not analyzing the optical aspects. I know that there are Other folks that are doing it. It's, uh, um, Professor Dunn's, uh, at, uh, in Texas that's uh doing speckled imaging, uh, in order to, uh, to analyze the flow through vasculature and to the seas issue.
[57:09] Speaker 3: Uh, there is a, a professor Jordi Kita at um at Johns Hopkins University that's, uh, developing AI's for microscope and robotic surgeries.
[57:21] Speaker 3: Uh, so there are folks that are doing this kind of things. However, for us, the only thing that we were asked for is guidance before you even actually go and open the patient to tell them uh optimal placement of the burr of the where for access and to place a clip over an aneurysm neck with the least.
[57:50] Speaker 3: Uh, maneuvers and so. Um, also, uh, placement of the, uh, deep brain stimulation electrodes in order to make sure that you're not touching structures or areas of, uh, of interest. But Other than that, uh, um, I think, uh, there are, there are a lot of folks that are looking into this. Um, and yes, to answer your question.
[58:16] Speaker 3: It is possible to do AI for uh for a mic for a trial periphery Surgery microscope and there are people that are doing it.
[58:24] Speaker 3: For sure.
[58:26] Speaker 1: Yeah, yeah, Louis, we have to try to get a hold of John Hopkins. Yeah,
[58:32] Speaker 1: so maybe,
[58:34] Speaker 1: go ahead, go ahead.
[58:36] Speaker 5: Uh, yeah, I'm Doctor Sarah Van. I'm actually a pediatrician.
[58:40] Speaker 5: I actually know Doctor Boorta. I'm like on the genetic side, but the question today is, how long have you been doing this? Because I kind of have a approximate timeline, but my main question is how, uh, how can this be utilized? Has it been utilized in scanning the brain? I know it has been to some extent, um, in patients who have.
[59:05] Speaker 5: Uh, degenerative, I mean, neurodegenerative diseases, and can it be used in pediatric patients yet? Has it been analyzed, uh, to help in seizure control or in, uh, Parkinson's or dementia, or how about, uh, also, one main thing is schizophrenia.
[59:25] Speaker 5: Because you talk about a lot about neurosurgery, but how about Other neurological diseases? Has it been utilized? Has it been useful, found useful, because I'm, I'm pretty sure because it's regarding the blood supply in the brain. So can't we, can't we expand it much further? Can't we start Research in that angle?
[59:47] Speaker 5: Please, uh, please educate me because I'm no neurosurgeon, but I have an uncle who was a very prominent neurosurgeon whom I have not even talked to because he was so damn busy.
[59:59] Speaker 5: And uh his son is a neurosurgeon, but I've never even watched him perform neurosurgery because I am in the US since 1994.
[1:00:10] Speaker 1: OK, ma'am, ma'am, thanks for your question. Uh, can you, can you answer that, Cyprian, Cyprian, can you?
[1:00:16] Speaker 3: So, um, let's, it's OK, uh, from, from the point of view of the, um, AI applications, um, as I said, the, it needs, uh, a data set. It needs some images or some data and then some labels.
[1:00:35] Speaker 3: Uh, those labels in general can be a, uh, patient outcome. Let's say we, we would like to know what is the likelihood of these patients to, uh, to return to normal. Should I open it or not? OK. It gives you, uh, for example, it can give you those kind of things. Now, with regards to uh dementia and all the Other things, um, it is the, um, that you mentioned, it is the, uh, the playground, I would say, of the neurologists.
[1:01:05] Speaker 3: And they are so much involved into this, uh, I think.
[1:01:13] Speaker 3: If I would have to choose somebody that does this the best, I think neurologists are out there. Uh, especially, um, uh, I know folks from, uh, let's say like Doctor Zaatnov, but uh, uh, uh, in our neurology. Uh, there is, um, there are again, uh, folks at the UCSF, uh, that do a lot of the AI.
[1:01:38] Speaker 3: For, uh, for, uh, the afflictions that you had just mentioned, uh, the issue, uh, I think.
[1:01:47] Speaker 3: It is for us to move one step farther.
[1:01:52] Speaker 3: And bring things like, uh, you know, how you have a Tesla driver and drives your car? We don't want that. We want something like to tell you, hey, don't change lanes, you have somebody in the black in your, in your blind spot, or it is risky to do it. That's the kind of assistive driving that for neurosurgeries that we want. And from that point of view, I think this is kind of the first.
[1:02:19] Speaker 3: I'm not sure that I've seen it out there. And that's why I'm, uh, we're kind of like conservative conservative when it comes about it because it will, from the point of view of the regulatory office.
[1:02:36] Speaker 3: It's going to be very, very dangerous in a way.
[1:02:41] Speaker 3: We are in the engine room telling you that something is wrong. What if it is not wrong and you go and screw up or something like that? So it is a software as a medical device.
[1:02:53] Speaker 3: And uh applying this into the angio rooms or into the Surgery rooms to guide your electrodes or to go for deep brain stimulation, it will not be accepted this year and probably not even next year unless we show that this works.
[1:03:11] Speaker 3: Uh, at a, at a, at a big scale, um.
[1:03:16] Speaker 3: I, I would say that every single one, the, uh, deep brain stimulation folks have asked us for this. The neurosurgeons that do Spine, uh, for cervical loidosis, they want to know, um, they want to know when they are done with uh adjusting the Spine in real time using your O arms. Uh, others, they want to know, uh um uh how to uh uh uh about the tumors.
[1:03:45] Speaker 3: Uh, and, uh, access points to, uh, to avoid like speech areas and so far to reach them. So there is a lot, um, there is a lot of uh um of requirements, but as far as concerned, as AI assisted Surgery in real time, I don't think there is much out there.
[1:04:10] Speaker 3: Uh, if anybody knows something, please, uh, let me know. I would like to know.
[1:04:17] Speaker 2: So, you, you need the Dcom files of the DSA.
[1:04:23] Speaker 3: No, I actually
[1:04:24] Speaker 2: just analyze,
[1:04:25] Speaker 3: I just need a cable into their data stream.
[1:04:28] Speaker 3: A splitter
[1:04:30] Speaker 3: What comes from this, from the detector itself?
[1:04:34] Speaker 3: That's the only thing that we need. Otherwise, any kind of, if we put it on the packs, it will not work because from there it goes into, into your console and not until you finish your case you upload the things on the packs. And your, your red pack is going to be there and say uh load these images only.
[1:04:55] Speaker 3: Nobody, as far as I know loads all their data into the packs from an angiogram run or just snapshots or
[1:05:04] Speaker 3: Uh, or acquisitions that make a diagnosis or is very important. Most of it is discarded.
[1:05:12] Speaker 3: So that's, that's the key for, for this one. For us to communicate directly with the CRM and the uh the manufacturers have to be uh involved into this.
[1:05:26] Speaker 2: You mean the DSM machine manufacturers, OK.
[1:05:31] Speaker 5: Can I ask the cost for this procedure? Is it, uh, is the cost during the uh Surgery different from the cost before the Surgery and after the Surgery?
[1:05:41] Speaker 3: This, uh, will technically if it's integrating into their CRM it will be at no cost for the users. The, um, uh, the.
[1:05:53] Speaker 3: Manufacturers.
[1:05:55] Speaker 3: Could offer this with their, with their um with their equipment. Uh, I think that's uh that's one of the keys.
[1:06:05] Speaker 5: So, would then, let's see, because I'm, uh I want to know because medical insurance, will it cover it 100% then?
[1:06:14] Speaker 3: Um, from what we talked with, uh, with medical insurance folks, uh, they will not want to pass to the patient any additional costs. Everything, yeah. So, it will have to be something that resides and the manufacturer present their C arm or their equipment and say, hey, we have these extra features.
[1:06:38] Speaker 1: Very good. You know, you wanna establish as many collaborations as possible, right, Cyprian, uh diversely,
[1:06:46] Speaker 3: optimally,
[1:06:46] Speaker 1: optimally.
[1:06:48] Speaker 3: Uh, that's, um, that's kind of like, uh, the idea, and, uh, also, uh, like, uh, all, uh, what Doctor P uh Purnima has, uh, she said, uh, we would like to know Other applications.
[1:07:02] Speaker 3: I mean, we, we live in this place called Gates Vascular Institute, which is a comprehensive, um, uh, stroke center in the Western New York, and the only thing that we see, it is stroke, stroke, stroke or, uh, you know, that's a, that's a, that's kind of like uh a lot of our focus and uh as I showed you before, we're, uh, uh, I would say 6 people.
[1:07:27] Speaker 3: Uh, 4 of them are very good programmers, but we cannot tackle all the problems for sure.
[1:07:34] Speaker 1: OK. Very good. Any more comments, questions from the panel?
[1:07:41] Speaker 1: OK, Cyprian, uh, it's great, it's great. I mean, you hit a home run. I think you made some great connections and, uh, things will go forward. Thank you, Louis, for coming. Thanks, Ben.
[1:07:55] Speaker 5: Dr. Bennett, just one question.
[1:07:57] Speaker 1: No, no, ma'am, we're, we're closing up, OK? No,
[1:08:00] Speaker 5: it's not regarding the subject. Yeah,
[1:08:02] Speaker 1: hold on. What do I do in a situation like this?
[1:08:07] Speaker 5: It's not regarding the subject. Uh, are doctors trained? Are, are computer programmers trained to do this? That's my last question.
[1:08:16] Speaker 3: No, this is all engineers, really, uh, yeah, so, uh, but we do, we do only what they tell us.
[1:08:24] Speaker 3: We never go, yeah, doctors.
[1:08:27] Speaker 3: Decide what we do pretty much.
[1:08:29] Speaker 1: OK, OK, very good, Cyprian. We'll edit this and send it to everybody. Thanks everybody for coming and stick around. Uh, we're going to go off camera. So thank
[1:08:39] Speaker 2: you.

## In this course

- [Previous: Emerging Artificial Intelligence Technologies for Neurovascular Disease Diagnosis and Treatment Guide](https://doc.social/llms/courses/neurosurgery-and-artificial-intelligence/emerging-artificial-intelligence-technologies-for-neurovascular-disease-diagnosis-and-trea.md)
- [Next: Real Time Surgical Guidance using AI](https://doc.social/llms/courses/neurosurgery-and-artificial-intelligence/real-time-surgical-guidance-using-ai.md)
- [All lessons: Neurosurgery and Artificial Intelligence](https://doc.social/llms/courses/neurosurgery-and-artificial-intelligence.md)