---
title: 'Real Time Surgical Guidance using AI'
type: 'lesson'
url: 'https://doc.social/courses/neurosurgery-and-artificial-intelligence/real-time-surgical-guidance-using-ai'
course: 'Neurosurgery and Artificial Intelligence'
course_url: 'https://doc.social/courses/neurosurgery-and-artificial-intelligence'
position: '3 of 3'
author: 'John Bennett MD'
transcript: 'yes'
---

# Real Time Surgical Guidance using AI

## Summary

_This lesson is a recording without a written summary._

## Transcript

8,201 words · 57 min · 3 speakers · automatically transcribed (aws-transcribe) and not checked by a person; it may contain errors, particularly in names, drugs and dosages. The recording is the source.

[0:10] Speaker 1: Good morning from Miami Beach. It's Doctor John Bennett broadcasting for Neurosurgical TV. We have part three of a series of webcasts done by Cyprian Yonita, a PhD from University of Buffalo, Jacob Smith. Uh, he's gonna be talking about generally the interface between artificial, uh, intelligence and neurosurgery, an exciting topic, uh, nowadays. Uh, so welcome, uh, Cyprian, and thank you.
[0:41] Speaker 2: Thank you for the introduction, John. So let me share my screen here.
[0:48] Speaker 2: Right.
[0:51] Speaker 2: Uh, just a second. Now this one here.
[1:00] Speaker 2: OK, can you, can you see, can you, can you see this?
[1:04] Speaker 1: Yes, perfect.
[1:06] Speaker 2: So, uh, it's just a full slide, real-time surgical guidance to using AI?
[1:11] Speaker 1: Yes, full slide.
[1:12] Speaker 2: Thank you. OK, thank you. So, um, today I'll, it's the 3rd part and the final of use of AI for, um, neurovascular interventions, and, uh, uh, I started with, uh, um, with the predictions or uh triaging of the patients that with the complications, and today I'm going to
[1:35] Speaker 2: Kind of like go over potential use of the AI by the endovascular Surgery. uh, again, my uh uh my conflicts of interest are here. I, I have some uh NIH and NSF funds, and I have some patents relevant to this, uh.
[1:53] Speaker 2: Uh, to these presentations. So the AI medical applications, as we said, then at least into the endovascular procedures, uh, we start first with simple things to help triaging patients and to help with stroke diagnosis, especially in the areas where
[2:11] Speaker 2: There are a lot of patients where they have a lot of cases and uh this in general is just to, for early identification of the patients and really uh uh for patients to, to identify those patients that are uh are immediate risk and prioritize them. Uh, the next thing that we have talked about it is to uh detect complications during the procedures.
[2:36] Speaker 2: Uh, with many cases and subtle, uh, changes into the flow, into the, uh, angiograms or images, um, there might be a chance sometimes that
[2:50] Speaker 2: You know, complications or small or small adverse events not to be noticed, and we think that AI in this case may enhance the neurosurgery, neurosurgeon's practice or procedure by
[3:07] Speaker 2: Identifying this potential complications and flagging them again, this is not something that, let's say the AI will fly solo and will take over as a procedure. It's just to bring to the attention of the of the interventionists or as a neurosurgeon that something might be wrong in a certain area. And today I'm just going to go over treatment planning and treatment guidance using the AI predictions.
[3:36] Speaker 2: So again, this is a repeat of a few slides or a few a repeat of the previous one of my previous lecture, part two. I'm not, I wasn't sure how many people have seen it, so I'll just quickly go over some of this, so.
[3:53] Speaker 2: The procedures that I'm focusing on are exclusively.
[3:58] Speaker 2: A catheter-based procedures using X-ray image guidance where the neurosurgeon takes the catheter under X-ray guidance, takes it to the area where a lesion is located into the brain, and using different devices, tries to modify, usually tries to modify or the flow into the lesion or re-establish flow, reduce flow, etc.
[4:24] Speaker 2: So why do we want to do this? Well, it minimizes in general the trauma, the Pain, the blood loss, and reduced wound complication. It's not very uncommon to you to see a patient released within 48 hours from the time of acceptance for some of these elective elective procedures. So, um, bottom line, it is added safety to to the procedures. Well, the issue with this.
[4:54] Speaker 2: Uh, is that, uh, some of the procedures that we do, uh, the devices that we use are limited. Uh, we do not have ways to measure directly the effect of the treatment that we're doing. And, uh, a lot of these procedures may require additional, uh, or followups. Uh, it's not uncommon for aneurysm treated with, uh, with coils to see 2 to 3, uh, followups where coils are being added or due to incomplete occlusion.
[5:23] Speaker 2: Uh, etc. or coil compaction.
[5:27] Speaker 2: So I'm not sure how many in my audience, how many were trainees or residents, so and I will go over this a little bit because they are the main focus of the presentation for aneurysm treated with coils. A device like this is placed into the aneurysm, and the idea in here it is to use the structure of the device to
[5:51] Speaker 2: Reduce the risk of rupture by modifying the hemodynamics into the aneurysm dome. Uh, the, the need for this one is to have a narrow neck, uh, and, uh, uh, and, um.
[6:04] Speaker 2: Uh, in general, um, not.
[6:09] Speaker 2: Uh, not to have the neck is actually in here is the most, the most important part to, to decide whether to treat or not. But the Other treatment, it is a simulation or the simulating the concept of the flow diverter. This is not a flow diverter, but uh the idea, it is represented in here that you divide a much denser device over here.
[6:31] Speaker 2: That will divert the flow away from the aneurysm neck, and there's different sizes and if you're thinking about how the device works, it's like more like a finger trap. If you, if you elongate it, the the the um.
[6:51] Speaker 2: Spaces in between the wire becomes wider.
[6:54] Speaker 2: And the flow is less.
[6:56] Speaker 2: And then if you're compacting it.
[7:00] Speaker 2: And bring it together, then the, the spaces in between the devices, the strats, the metal struts becomes smaller.
[7:09] Speaker 2: And diverts the flow a lot. So there are neurosurgeons that are looking at this. And if the device, let's say you have a 3.5 millimeter device, they might oversize this or undersizing it. If you're oversizing and let's say for a 3.5 millimeter you go over to the 3.75 device, you elongate it and the pores in between become larger, so the flow into the aneurysm is less.
[7:38] Speaker 2: If you go with a 3 millimeter device for a 3.5 millimeter diameter in here, then you have to crunch it together, kind of push into the device, and what's going to happen, you're going to have a higher density in here over the of metal over the aneurysm neck and divert more. The issue with oversized with undersizing and crunching it together, if you're not careful when you push onto the device, you will unbuckling it into the aneurysm neck. So it requires
[8:07] Speaker 2: There's a lot of finesse onto the neurosurgeons to kind of like feel the pressures into the catheters and the backflow into the uh to avoid this kind of uh of uh of complications. So others, they want to be safer and they're saying I don't want to, I don't want to, let's say um uh really fully occlude this device. I just want to make sure it's not going to grow.
[8:32] Speaker 2: They, they oversize in Other main, in, in Other words, they are taking a stent that is slightly larger than the device, and it's safer for them to to deploy without worrying about the device buckling into the aneurysm neck, so.
[8:50] Speaker 2: Once this procedure is done, the only way that we can look at these devices and figure out that something is happening, it is using digital subtraction and geography, and we can look and measure all sorts of things in here stenosis, intracryan aneurysm, so dome sizes, aneurysm, neck, etc. to decide on the device that we can use, but not as much function.
[9:17] Speaker 2: So here I'm showing you an example of an angiogram run. Here is a device. It is an aneurysm, and it was treated with that device with a pipeline. These are the pre and these are the post uh uh treatment uh pictures. In general, the surgeon is looking at them and qualitatively saying that I think I'm done.
[9:42] Speaker 2: Um, there is not much I can do in here. If we dec if besides that it's not done, some of them like Doctor Siddiqui sometimes places a microcaheter into this uh aneurysm dome over the, uh,
[9:57] Speaker 2: the pipeline before deployment and decides to drop a coil in.
[10:03] Speaker 2: Or decides to put a second coil in or
[10:07] Speaker 2: He's wondering about it. You know what, I think this device, it doesn't have quite well wall opposition and goes with a balloon and inflates it slightly to make sure that the pipeline device is fully has fully positioned with the with the aneurysm neck, and based on these images they have to decide. And as you can see in here at 6 months follow up this is gone.
[10:32] Speaker 2: So what kind the idea is, what kind of the flow information we can extract from these angiograms in order to decide whether we're done or you should do something more about this treatment, and this is more so about coiling because coils you can deploy them in one at a time.
[10:55] Speaker 2: And decide whether you're done or not. And the idea is that if you're over, if you're really deployed too many or you're pushing too much, you might rupture the aneurysm or you might end with a coil coming into the, into the.
[11:11] Speaker 2: Artery and then you might have to deploy deploy a stent assisted coil embolization that will put the patient again onto the antiplatelet therapies. So there is a lot of of implications with regard to the deployments. So what we want to do it is shown in here.
[11:31] Speaker 2: I showed this picture with the bottom part where we're deciding ischemia. In here we actually want to do actually real treatment and what I'm going to go today, I'm going to go through this upper branch in here. So normally you treat, you wait 6 months, and you see if the aneurysm occluded or not, and then you, you decide again to retreat. What we want to do, we want to deploy a software residing on the CR.
[12:00] Speaker 2: That looks at this angiograms and then without any user interface or any kind of application form from the user finds the aneurysm right there it goes, looks where where the aneurysm is located.
[12:17] Speaker 2: And
[12:18] Speaker 2: Um,
[12:20] Speaker 2: And then uh I think uh we have a little bit of a background, John all right. I got it, I got it.
[12:28] Speaker 2: OK, so, and then, uh, uh, and then.
[12:32] Speaker 2: Not only finds where it is, it segments it out, which means it takes only the area of the aneurysm and extracts those parameters out.
[12:42] Speaker 2: And then it makes a prediction. And today my talk, it is about how we do this pretty much with the AI, but I also want to show you evidence that all this data is supported. You cannot go to the AI in a way expect for it to do its magic and do a prediction that doesn't match.
[13:04] Speaker 2: The our gut feeling or not necessarily gut feeling, but our understanding, scientific understanding of the processes associated with this disease with a particular disease. So the Other thing that I'm going to insist at the end is that
[13:21] Speaker 2: Never ever use the AI to tell a surgeon that a definite answer and say,
[13:29] Speaker 2: This is going to occlude. No, just give likelihoods of things to occur because let's say if the AI decides with 52% that this aneurysm is going to occrue, it's going to give a definitive answer. Then a 52% confidence into a prediction, it is not the same with a 90% confidence. So it is good for us to break the AI down and
[13:56] Speaker 2: Show the surgeons how how confident is the software and that and displayed into our interpretation or predictions of the data.
[14:07] Speaker 2: So, um, let's go over the physics of it. So, as I said, in the aneurysm, what we are trying to do it is to use a contrast to decide where, whether we're done or not.
[14:20] Speaker 2: And in here I'm just showing you some data, scientific experimental data, flow data that backs that up.
[14:27] Speaker 2: So we took phantoms like this you see in here a long time ago. We, we've been at this for more than 20 years, uh, uh, and uh I would say that the flow diverter concept also emerged partially also in Buffalo, um, uh, in, in the early 2000s. So, um,
[14:51] Speaker 2: The device is placed across the aneurysm neck and diverts the flow. Here you see an untreated one, and you can appreciate here 31314 millimeters per second, about 31 centimeters per second inflow impinging into the aneurysm neck. And then we use different flow diverters again. We can modify by how much it changes the flow in, and here is one that reduces to 5 millimeters per second and another.
[15:17] Speaker 2: That reduces to about 2.9 millimeters per second. Also, the vorticity inside dropped from 100 seconds to -1 to 1 2nd to -1, and so forth to magnitudes. And now when we look at the X-rays, you see these are the velocities and how do we know that this is right? It is because we measure them, and this is kind of showed the concept we pass these particles through those phantoms that we showed you.
[15:44] Speaker 2: With a laser, we shine the laser and we track small particles all around. All these particles are being tracked and then gives us measurements like this, so we know exactly the flow, how it goes in. And when we look at the contrast, how it goes in, it exactly follows to a certain degree.
[16:03] Speaker 2: the flow is from down up. You see the vortex, and then it goes from here, and again you see like it comes at the neck, goes out, comes as the neck and goes out, not quite like here, and here you see it goes right at the neck in, sips in and goes out exactly the same. So the key point that I want to make is that yes, the contrast follows the hemodynamics up to a certain degree.
[16:31] Speaker 2: So we have to go and when we look at the contrast flow, we can't trust it. We can trust that the patterns that we are actually see are are related to the hemodynamics and the hemodynamics changes that come into the flow. The problem is if I'm asking a person or a surgeon right now and tell them, is this going to occlude or this one is going to occlude.
[16:56] Speaker 2: because this is not available. These velocities, forget it, they are not going to be able to for us to measure them into the aneurysm when we use the devices because the the there is a bone in the way. We cannot use ultrasounds. We cannot have any means to measure this kind of velocities, and this is where the bottleneck where the problem is. If we would measure these velocities, we don't need AI any longer.
[17:20] Speaker 2: But the situation is in such a way that we can't, and the only way we can look at these processes, it is through the digital subtraction and geography.
[17:32] Speaker 2: So how we do that, I told you before, I'm just going to go really briefly. We look at things like di dilution curves and as a contrast propagates with the system, they tend to spread really, and this is exaggerated. They are the differences are slighter. I would say something maybe between this and that occurs, but we can look at these parameters and and figure out what are any fear, kind of like.
[18:01] Speaker 2: estimate what hemodynamics are going in there, and we measure these quantities which are very common for the people that do CT perfusion to evaluate stroke, mean transit time, time to peak, peak, height, etc. mathematical formulas are given here. We use.
[18:18] Speaker 2: Uh, we're kind of old school and in this one we're using the gamma variate functions. A lot of the uh in the CT perfusion now, the more advanced techniques like CT, like SVD plus and
[18:32] Speaker 2: Bias in algorithms are used to evaluate those flows, and it just goes exactly like NCT. It takes the data, you look, uh, you do some processing to make it faster. You look at every single voxel, in this case pixel, derive this kind of curves, you speed it up, do the analytical function function feeding, and derive the parameters, and then do this for every single one and to the image and you get pictures like this.
[19:02] Speaker 2: So, um,
[19:03] Speaker 2: Again, these pictures are as you're looking at the flow with different filters, and every single filter gives you a different kind of information. Whereas the mean transit time, for example, is the average time a particle spans into a certain area. Time to arrival it is how fast it goes there. Peak height is almost like cerebral blood volume, and for the archery equivalent, how much contrast gets into a certain area.
[19:28] Speaker 2: In this case, so an area under the curve, it is a combination of the mean transit time and peak peak height.
[19:36] Speaker 2: So the question is,
[19:38] Speaker 2: Are these parameters related to the hemodynamics? I show you that the patterns are, but the values are. The question is, are they related?
[19:49] Speaker 2: And here we do, we did, we did again lots of experiments where we really confirmed that mean transit time, bolus arrival time are related with the velocities because we have pumps that can control this. We have Ultrasound we can go into that our region of interest area, measure it. This is all benched up kind of experiments, and we know that it.
[20:11] Speaker 2: Go and look at this for different flow rates in milliliters per minute or in centimeters per second measured that area we see that there is direct correlation between Bolus arrival time and mean transit time in there, and these are very repeatable. So what is the problem? Why we cannot have just the mean transit time at some point.
[20:34] Speaker 2: Really tell us what's going on. You take the main transit time, correlate it with the disease with the outcome, and things will go easy for you. The problem is this.
[20:45] Speaker 2: Nobody is going to inject who's an automatic injected injector and
[20:50] Speaker 2: In the same center and and across different centers it is even more different. And if you look at, let's say I have in here an automated injector and I do 5 mL volume and I do a 10 mL per second, I'm sorry, a different, a different Injection rates, 5, 1015, and 20 mL per second and we're looking at this situation in here.
[21:19] Speaker 2: Um, this aneurysm, what are the mean transit times? And there is a huge variation in here because now if I have a center and say this, this surgeon in here is injecting 20 mL per second at a at a 10 mL volume and another one somewhere else is injecting something something less than 5 mL and so they're going to get.
[21:47] Speaker 2: The same in transit times.
[21:50] Speaker 2: And
[21:51] Speaker 2: Here what we did, we placed a stenosis and then he's going to conclude that the patient with the stenosis has the same kind of hemodynamics as a patient without stenosis, which is entirely different kind of hemodynamics in there again, Injection parameters.
[22:12] Speaker 2: Or the hand Injection, it is extremely important and it will skew the data to the point that you cannot relate an angiogram from one center to another.
[22:24] Speaker 2: Hence, we need to take care of this, and we have developed something that is called an SVD, which is single value decomposition correction, where you have to go for the ICA. You take really the arterial input function like you do it in CT perfusion, and you fix for it.
[22:41] Speaker 2: And not only that, that we flatten it and we say this, what it actually this process, what it does.
[22:49] Speaker 2: It is simulating or bringing the data as if you do a hand Injection with a delta function.
[22:57] Speaker 2: Um, that's a, that's what it is, which means a very short Injection with very in a lot of contrast and as if, as if everything happens in a fraction of a second. So it brings variability to that type of Injection, stimulated Injection and the parameters, and what I want to point out is that stenosis versus in this case versus non-stenosis.
[23:26] Speaker 2: A, a big difference in between them, and it's depend on the milliliters per second and in the same way it's independent of of the volume. The volumes are over, over, over, they are overlapped again. This is stenosis versus non-stenosis. Now with a mathematical process like this, we can start seeing we reduce these variabilities that doesn't make any sense.
[23:56] Speaker 2: We want to just take
[23:58] Speaker 2: Hemodynamics of the lesion and at the same time takes away some of the corrections and I will show you why I'm going through this because it's extremely important. This adds a variability which limits our ability.
[24:12] Speaker 2: To predict what's going on.
[24:15] Speaker 2: And then if we add the C arm changes into the play, then it's going to be even a bigger problem.
[24:23] Speaker 2: And when we have these variabilities or these inconsistencies in the acquisition,
[24:30] Speaker 2: We are not able to to do our job properly, so what we're going to do, we are going to go back, we're going to start looking into things like AI or data driven models to help us overcome this variability and patient variability by looking at lots and lots of data in order to solve the problem.
[24:55] Speaker 2: So the current status that I will show you today uh of the our, our AI Technology is is as follows. It detects the lesion in this case the aneurysm. It just, it goes and extracts the parameters, then it predicts outcomes, and I again here I have to make sure that I make myself clear.
[25:16] Speaker 2: This is not outcomes like yes or no.
[25:20] Speaker 2: We are thinking as engineers when we started this I mean
[25:23] Speaker 1: like
[25:26] Speaker 2: uh can you
[25:29] Speaker 1: I'm sorry, I'll get that. I'm sorry. OK.
[25:35] Speaker 1: OK, OK,
[25:36] Speaker 1: that's fine.
[25:38] Speaker 2: And again it's not about prediction of the outcomes. It is actually likelihood. When we started in the beginning, where as engineers we wanted to predict everything and tell everyone that this is the ground truth and so far, and as we talked to more and more of you guys.
[25:54] Speaker 2: We realize that we have to step back and step back and step back and actually behave like physicians, give you likelihoods, and so forth. And then also to protect against complications, let's say a wire might go through um.
[26:12] Speaker 2: Uh, through the aneurysm neck and and creates a subarachnid hemorrrrhage and uh uh uh and predictor, let's say, of something like a de delayed cerebral ischemia.
[26:25] Speaker 2: So, um, first of all, how we, uh, tackle the aneurysm identification, uh, or how do you do to, to, for lesion identification. And this you can apply again for, uh, for any kind of lesions. You can do for AVMs, you can do for intracranial disease, stenosis, uh, and, uh, and um
[26:48] Speaker 2: And hemorrhages. So we took 350 DSAs, and again, this is our limitations. We're working on one center. It is only Canon, uh, it's Canon equipment at Buffalo Buffalo Gates Vascular Institute Centers. We use only adult patients treated for saccular aneurysm with with coil or a pipeline.
[27:11] Speaker 2: And is uh uh and, and we're using, as I said, uh, Canon in Phoenix uh pipelines for everything that we do. If you're wondering about uh the distribution 55 prior for treatment, 45 following coil procedures or pipelines.
[27:27] Speaker 2: And these are the distributions. We went really from mostly in the internal carotid and we're very limited like in our studies on the basilar and posterior cerebral circulations. We also use 44, 452 negatives. So poor image quality, it is a big problem. Things like this immediately throws the AI off. So um.
[27:53] Speaker 2: Even an aneurysm like this, if you're not careful with the AI uh might be actually, uh, be, be uh thrown away. So, uh, one Other thing that AI could be used in this case is it is actually fixed for this kind of uh artifacts.
[28:09] Speaker 2: And then what we did actually we processed every single sequence to increase the signal of the aneurysms neck, and then we had the the neurosurgeons or fellows, not the actual attendings, fellows and non-residents and scientists go through every single one and hand contour the aneurysm.
[28:33] Speaker 2: Uh, we can have an automatic feature that can segment all the arteries in here, but for lesion itself, it is very good if you have somebody go and hand contours it in order to make sure that and that the data is right, and it better be an expert in the area.
[28:53] Speaker 2: And um now, as I said, uh, since we are 11 center, this kind of data is not quite good for us, uh, for, for the training of the AI and we need into the thousands, and that's why where collaborations come very, very important into this. Uh, I would say if you want to do an AI that is relevant to many centers and many, uh, uh, a large populations, uh, uh, segment.
[29:21] Speaker 2: You need collaborations with different centers. So in order to bypass this, we really did, we cheated in a way, and we did rotations, we did zooms, we, we did a little bit of data augmentation, what it's called, by messing up or changing the the images, and we made sure that when we crop, the aneurysm is not out like in this case, so it becomes a true negative in this case.
[29:50] Speaker 2: Uh, and then we, we got 1500 examples and 600 we kept for testing. In Other words, these guys in here, you never touch during the, uh, the training of the of the of the network. For, uh, for those interested into the uh details of the AI structure, we use uh Keras and we use what is called a VGG 16 architecture, and this one, was developed uh.
[30:17] Speaker 2: Really for um.
[30:20] Speaker 2: Depicting or type of objects onto the internet pictures, uh, labeling pictures onto the internet, and what we do, we kind of like, look, you train the networks, and as they are being trained, this parameters that you see here in convolution 122, and so far all the segments are the weights or the experience of the AI is being set.
[30:46] Speaker 2: In Other words, we're going to go and we take those pre-trained weights on existing data that is out there on the internet, and we start with those and apply them to our images. That saves us a lot of time in order to do things like segmentations and so on.
[31:06] Speaker 2: Uh, this is about 3 years old, I think, no, 2 years old, and it was awarded at the SBIE Medical Imaging in 2019. It received an award. So since then we, we improved this, this method. Again for people that are interested into details. This is what we used at that time. We use an Ata Delta that adapts learning rate over time.
[31:35] Speaker 2: Uh, we, we did transfer learning, as I said, from ImageNet and, uh, in order to to calculate this, we use, uh, to calculate the, um, uh, Efficiency or the accuracy, use what is called this uh uh dice, uh, to optimize, I'm sorry, we use a dice loss. For quantitative analysis, we're using the normal area under the curve, but also JCR index. And how do you read those, this for those people that are not familiar with it.
[32:04] Speaker 2: It is you have two shapes. In this case, this green stuff, this green square, it is what let's say the expert labeler has done, and the red one, it is what the AI comes up with. If you get something like a 0.4, it is a poor match between what AI predicts.
[32:26] Speaker 2: And what the uh the deliverer has done. This is a good agreement, and excellent agreement is usually you're looking somewhere into 0.92. For clinical applications, you must be somewhere in this area and above, um, in order to do the segmentation. This is another coefficient that we have used and um uh uh we used uh also um this network next to
[32:56] Speaker 2: Determine where it is able to extract those parameters, those flows parameters, those angiographic parametric imaging parameters to see if it is able to extract them, and we compare them again with some with a
[33:11] Speaker 2: Export user, we had for every single case we had an expert go and extract the parameters like mean transit time, time to peak, and we compared the AI with this expert user to see how it does using PRA correlation coefficients.
[33:26] Speaker 2: Here are some examples of the of the results in here. You see, we start with a with an image like this. We are a sequence of data like this. These are hand contours and you see the user usually plays a circular ROI and he places 11 here to take into account those variabilities because of the hand Injection.
[33:50] Speaker 2: And as you see, this is done, we don't need this to be done uh with the AI. You, we can do this automatically just by image analysis. So I will focus only on the aneurysms. And you see here how the AI has done pretty good job at the, at the, uh the aneurysm, but uh the distal part into the arteries, it's uh it's so so. So, in general, it's good agreement between this and if we use coils, again, there is no problem.
[34:20] Speaker 2: Very good agreement between uh what the user will do and what the AI will cement. I would say the AI actually is better because it doesn't do this kind of uh circular placement. It's goes and identifies the edges. And the thing, it is slight motion artifact, it is OK. It, it's not being thrown away. If the patient has, in this case, this patient had an intracranial hemorrhage and he had a uh a craniotomy as you've seen here to
[34:50] Speaker 2: Um, uh, in a shunt placed. Um, again, no problem when you have hardware like this in here. Uh, high, uh, highly attenuated structure is not throwing it off because it has been trained only to look at this kind of data and it can easily ignore those. Other things like, uh, cath you see again another example of a, uh, stent, uh, a stents assisted, uh, uh, aneurysm. The stent, you cannot quite see it well in there.
[35:17] Speaker 2: Um, catheter presence, no problem. It can ignore it very well. Area under the curve for aneurysm neck detection or the aneurysm sac detection is 0.798.
[35:29] Speaker 2: When you look at the Jacquard indexes, it is into the area again, this is overlap dissimilarity coefficients area under the curves. They are into the clinical area acceptable and confidence interval pretty tight nonetheless for vasculature in here it is, it is on the bottom.
[35:50] Speaker 2: Uh, what we're focusing in general, it is over these values dissimilar coefficient, similarity coefficient, which is really good. We like it. However, for again, for the air and of the curve for the vessels, it is a problem because it's trying to segment this slow signal data in here and it's, it's poorly enhanced. So, um, but again, we do not care when we look for lesions, we do not care about this uh this uh this artery in there.
[36:19] Speaker 2: So the uh the uh peripheral vascular, which is very small, uh, is not uh is not a problem that really uh
[36:29] Speaker 2: Um
[36:30] Speaker 2: It is not, does not segment it very well. I, I care about what is here and in the area where the aneurysm, it's, it's a segmented, it does very well. Now the question is how does the AI do in extracting those parameters bolus arrival time, time to peak mean transit area under the curve, peak height, and the air, it is 0.2% between the AI and the human.
[36:59] Speaker 2: All right, so, uh, correlation again, superb. It's here's some coefficients almost except for Bola's arrival time. It is pretty, pretty good. It's a very strong association with the with the API values. Now the thing is, uh, let's say.
[37:19] Speaker 2: What we're saying in here, it is that the AI performs noninferiorly to an expert user in in in taking these parameters. However, unlike manual input where you have to load the data, look through them, do that, which can take anywhere for somebody that is not quite a quite an expert, can do 20 minutes to extract and save that data.
[37:45] Speaker 2: At the time when this was done, it was 5 seconds. Right now it is 0.5 seconds in extracting the data by this by this AI, which means it happens almost instantaneously after every single angiogram. This software or this Approach, this AI Approach can go and extract these things.
[38:09] Speaker 2: And really what does is open for us, it is to be used in the clinical setting, extract those parameters, and actually perform some sort of prediction of the outcomes or the uh uh uh um prognosis of the outcome. And right there in the jury room, instead of waiting at the six month follow up, to allow the surgeons to do uh to do uh.
[38:33] Speaker 2: Um, to do the, uh, the adjustments. So here are some situations. Case A, case B, case 3, all treated with coils.
[38:43] Speaker 2: And um what we see, we see the cases before, and then we see what's happening after.
[38:50] Speaker 2: And when we ask them, well, is this going to occlude or not?
[38:56] Speaker 2: And the idea is that at the end of the day you see in here.
[39:01] Speaker 2: One of them is totally gone.
[39:04] Speaker 2: One of them is partially, and this one, it's pretty much there, not only that you have the neck and regrowth, but you have also into the dome. The same thing applies for uh for the uh for the flow diverters. Here is the aneurysm. Here is a flow diverter you kind of like see it in there depending on the screen on, on your the resolution on your screens, and then the aneurysm is gone at follow up.
[39:30] Speaker 2: Case B, 5 aneurysms on this patient. 1 big pipeline that went from there all the way down, blocking every single one. All of these guys gone. This one in there, depending if you look very careful, it is some sort of a seepage in there. But in this guy in here, here is the aneurysm. Here is the after the treatment.
[39:53] Speaker 2: 6 months later, nothing happened.
[39:56] Speaker 2: So the question is, can we tell them of the treatment?
[40:01] Speaker 2: Whether they're the surgeon or what is the likelihood, so to allow of occlusion.
[40:09] Speaker 2: So the surgeon can do an adjustment of the treatment or at least put that patient on observation because something like this, you might end with it rupturing.
[40:21] Speaker 2: So the way we are doing it is we are taking these parameters and we're taking parameters before and after the treatment. We always do a baseline comparison and then we are looking at this 4 maps time to peak, mean transit time, peak height, and area and the curve. We have done this over a few 100 cases to train our networks.
[40:45] Speaker 2: Uh, the, the first module of the API or the AI takes the data, passes through deep neural networks, makes a prediction, and then here it says with an 81% probability that this aneurysm is going to occlude. And in this case in here it said with a 99.8% probability or confidence this aneurysm is not going to occlude. And indeed when we looked at the six month follow up, the aneurysm was there intact.
[41:13] Speaker 2: So what you need to know is that you need to to address this, and this is where we spend a lot of the time, 2 to 4 years we spent on how we make sure we do, we account for everything. So because into the AI junk in junk out is going to be if you're not and this is where is our, our proprietary method and
[41:37] Speaker 2: You have to kind of like figure make sure that look what is happening as the prediction or the accuracy if you use raw data without any processing, without accounting for hand Injection, without accounting for CRM orientation, and so you're getting a flip of a coin.
[41:54] Speaker 2: Worse than a flip of a coin.
[41:56] Speaker 2: Now, when you correct for Injection.
[41:59] Speaker 2: You get into 6 0.67 area under the curve. When you do also account for CRM orientation and and and for shortening, you get into the 77 0.77 area under the curve, and our interface as it is, the one that we have developed right now, what it does.
[42:22] Speaker 2: It's still manual. We hope this will be on the, on a CHMm. It just goes, loads the data, then automatically, without, without the surgeon interfering, finds the aneurysm, extracts the arterial input function, and then it predicts what's coming on, and it gives in here, right there, displays the maps, says probability of unsuccessful treatment in this case is going to be 79.9%.
[42:49] Speaker 2: It gives you how the values tries to do some interpretation and say, hey, before the treatment you had a mean transit time of 1.08. After you have a 1.04. Your mean transit time went down for the area where the particles went in. That should be immediately a sign for you that.
[43:10] Speaker 2: Particles spent less time into the aneurysm dome than before, even though the peak went from, let's say, from a 0.819 to a 0.78, so the peak has decreased by 10%. You see less contrast goes in, it gets out faster.
[43:27] Speaker 2: And the signal should be, hey, this has a problem with wall opposition or something, so the way I diverted the flow needs to be reassessed. I will end with with where we're going next with this, and we want to add even more physics into it. So you remember that I said that the AI just tries to find this function that takes the input to the output by comparing it with the growth label, and it's just comparing them.
[43:55] Speaker 2: Over many, many, many iterations until you get for you show apples and it says it's apples.
[44:02] Speaker 2: But now in here we are thinking to go one step farther and it says hey for this kind of flow.
[44:11] Speaker 2: Add some physics into it.
[44:14] Speaker 2: Let's put in there things like mass conservation, conservation of momentum, conservation of energy into those decisions when we optimize this, and it's, do not worry about the equation, it's just examples. It doesn't mean absolutely nothing, but we can take stuff like this and say these are the rules that in this gray area must be followed.
[44:39] Speaker 2: These are the rules that the physics or the contrast that you see must follow.
[44:45] Speaker 2: And we take that and we
[44:47] Speaker 2: Make like a lost function. It's what we minimize the error that we try to minimize, and then we tell them again boundary condition outside the gray area everything is zero.
[45:00] Speaker 2: On these areas and when it comes in here you have to have these velocities you kind of like go and you add every single physics phenomena that you have into this huge term in here.
[45:13] Speaker 2: And then you put it to go to 0.
[45:17] Speaker 2: So when you train the AI, you tell the AI, hey, minimize this function instead of minimizing the error between the values that you have and the things and and the and the expert expert users. And the thing is, it doesn't matter what you're doing because even with the simplest of the AI, you always calculate these values. You always calculate derivatives in order to minimize your function.
[45:45] Speaker 2: And here what you see it's almost like a real time.
[45:49] Speaker 2: Predictions of in this case the velocity into this direction, velocity V is into the Y direction, and the pressures.
[45:59] Speaker 2: And imagine until now, these are simulations by the way, but imagine until now in order to extract this kind of velocities what you did, you put into something called computational fluid dynamics which can take 48 hours to calculate. What about having something like this? As soon as you upload from your data into the Surgery, you upload a picture like this, it starts calculating those values and tells you what is happening.
[46:29] Speaker 2: So we're moving even one step further when we can do physics informed Artificial intelligence.
[46:37] Speaker 2: So I'm going to end this. This is a big, big, uh back to the big picture. We are into this domain. We are looking at stroke Protocols to triage the patients. We're looking at actually detecting in real time if complication has happened, and we're trying to predict as well.
[46:58] Speaker 2: If a Surgery is going to be successful. This is in a way the big picture is in here, in order uh schematics, there's two AI units residing on the CR.
[47:10] Speaker 2: The one of them is looking at identifying the lesion.
[47:15] Speaker 2: And predicting occlusion. The Other one tries to detect area of, of delayed ischemia or acute ischemia, telling the surgeons that something is going on. And we want to do this without any kind of disturbance into your workflow. The idea is this has to be as stealth as possible. We need to work with Cannons and Siemens and Phillips and Gees, and I'm sorry if I forgot any of those uh others, um.
[47:42] Speaker 2: We need to have this software working on their CRM to actually help you in real time. Without them, forget it. This has absolutely like uh doesn't have a value because you're not going to step out of the Surgery room, really look at the, at the, at the software on a on a laptop and go back in.
[48:02] Speaker 2: It, it is just not going to work. So uh with that in mind, these are my uh um collaborators. Every single idea that we presented that in here, it is done by the engineers, but it has been initiated by this gentleman.
[48:17] Speaker 2: Uh, Doctor Siddiqui, Doctor Levi, who's the chairman of the Neurosurgery, Doctor Ken Snyder and Jason Davis. These are their patients, and they come with the problems to us and then we try to solve. And these are actually the folks that, uh, write the code.
[48:33] Speaker 2: They go stay in the M&M meetings. They go to endovascular planning meetings and discuss, uh, listen, just listen to the neurosurgeons teaching their fellows and the residents, and then they go back and write those codes uh that I've shown you today. With that, uh, I think this uh, this is where I end the discussion and thank you, John, for allowing me to give these 3 presentations over the last 3 weeks.
[48:58] Speaker 1: Yeah, thank you very much, Cyprian. uh, and it's great to see those engineers interacting with neurosurgeons, uh, to make the project go forward. OK, we're open for questions or comments.
[49:14] Speaker 1: Oh,
[49:16] Speaker 3: thank you for your nice presentation and uh it's really fascinating uh but you know the the uh aroysm rupture rate uh is uh correlate with uh
[49:31] Speaker 3: Energy loss according to the uh
[49:36] Speaker 3: Bernoulli's uh principle. So can you, uh, calculate the
[49:43] Speaker 3: Uh, energy loss when you do the angiogram.
[49:47] Speaker 3: Uh, in the special segment of the, uh,
[49:51] Speaker 3: to calculate
[49:52] Speaker 2: the what loss
[49:54] Speaker 3: energy loss,
[49:56] Speaker 2: energy
[49:56] Speaker 3: loss, yeah,
[49:57] Speaker 3: yeah, like, like, uh, uh, dynamic energy,
[50:03] Speaker 2: yeah, yeah, so, um,
[50:04] Speaker 3: because
[50:04] Speaker 3: you know the local energy loss may be the cause of the rupture of theroot androgen.
[50:10] Speaker 2: As, uh, uh, this is, um, the work of my colleague, uh, uh, Doctor Hu Mong.
[50:18] Speaker 2: She has put a lot of time into this, and she's calculating this uh risk rupture scores.
[50:26] Speaker 3: Um,
[50:27] Speaker 2: and, uh, that's something they're looking into it and they also went to the AI because, uh, to account for.
[50:37] Speaker 2: Huge variability into the data sets that they are seeing, um, until now, the, a lot of the computational fluid dynamics have relied on uh idealized, uh, boundary conditions, those conditions that I said what's happening at the input and the output, which is not the case.
[50:58] Speaker 2: Every patient is different. Every patient has different hemodynamic conditions, and they are looking to use this AI to improve their CFD. Uh, and since it is about aneurysm rupture prediction, it is not.
[51:14] Speaker 2: Really imminent to happen in real time.
[51:17] Speaker 2: We can take a CTA, preferably see a cone beam CT on a, on an angiogram, and then, uh, use that, and that's like really, really very hot topic for them. Use the AI to extract boundary conditions so we can make our CFD better.
[51:36] Speaker 3: Yes, thank you so it is, it is, uh,
[51:41] Speaker 2: You just nailed it right there. We spent a lot of time into that.
[51:46] Speaker 3: Yeah, just as, as you mentioned, the idea.
[51:50] Speaker 3: Uh, stent stent should be partially
[51:54] Speaker 3: Uh, high density, uh.
[51:59] Speaker 3: Meter rate to cover the neck of the androgen. Yes,
[52:05] Speaker 2: uh, hopefully somebody from Medtronic will see this and listen to what happens, yeah.
[52:16] Speaker 3: Oh, OK, thank you.
[52:19] Speaker 1: OK, thank you, Ben. Uh, we have a question from Mohammed Helmi, a neurosurgeon from Egypt. Uh, greetings from Egypt, exciting lecture by Professor AE Iyunita. I wonder why didn't you apply this Technology more to the poster posterior circulation aneurysms. As you know, the post-circulation aneurysms are less debated to undergo coiling versus clipping.
[52:46] Speaker 2: Yeah. So, um, first, uh, the, um, the pipelines that we really, really focused a lot because it looks like that the, the field is going to that direction.
[52:58] Speaker 2: They're, uh, they're mostly into anterior circulation.
[53:01] Speaker 2: Now, uh, every now and then, if you go for a pipeline into, uh, let's say, into the basilar artery, it is like asking for trouble because of all those perforators in there.
[53:13] Speaker 2: Uh, Other, uh, as far as concerned coiling, uh, into the posterior circulation, we are actually doing that. Some, um, some, uh, some of those uh results for segmentations are for, for posterior circulation and so, but, um,
[53:30] Speaker 2: Um, as I said, we, we really use one single center.
[53:36] Speaker 2: And we see that whenever we are wrong, it is very likely to be wrong about posterior circulation.
[53:44] Speaker 2: So more data into posterior circulation and collaboration with others.
[53:49] Speaker 2: To make sure that we have a uniform data set will be really awesome. We, we don't have just enough in our centers, uh, center enough uh for posterior circulation.
[54:01] Speaker 2: So, um, and the
[54:04] Speaker 2: I, uh, I'm not answering your, your answer really, but, uh,
[54:10] Speaker 2: We do not spend that much time because we really don't have that much data into that space also.
[54:19] Speaker 2: So anyone that wants to work on this, we are more than
[54:23] Speaker 1: open to, so you're looking for data centers, correct, to supply that data
[54:31] Speaker 2: uh good collaborations and I understand, I understand that everyone is tied by these IRBs and the rules which are really draconic everywhere. And, um, I'm, I, I have like uh really no easy solution Other than have uh
[54:47] Speaker 2: Uh, you know, bigger, uh, organizations like National Institutes of Health.
[54:53] Speaker 2: Putting out these challenges where different centers can contribute to create these data sets or so, or, you know, um.
[55:03] Speaker 2: Otherwise, having just uh us collaborating is going to take a much longer time. They can snap their fingers and all of a sudden, a 60 to 100,000 cases.
[55:16] Speaker 2: Uh, materializes like uh RSNA does it all the time. In the neurosurgery, we don't have that.
[55:24] Speaker 1: OK.
[55:26] Speaker 1: OK, we'll, we'll work on that. Um, OK, more comments or questions from the panelists?
[55:35] Speaker 3: Yes, I, I think the, the aneurysm rupture, uh, prediction is uh is a very hard work because it's not only correlated with the, uh, hemodynamic uh parameters, but also the surrounding structures like uh CSF, like uh the cortex, the nerve, the duos, and uh everything surrounding the, uh.
[56:05] Speaker 3: Anderson do
[56:07] Speaker 2: Yeah, the, uh,
[56:09] Speaker 3: yeah,
[56:09] Speaker 2: it's the wall imaging, it is like, uh, the really hot topic now with the MRI is looking at the wall imaging and bringing that into the model. It's also very important for here.
[56:22] Speaker 3: Yeah, so you are also doing that work?
[56:25] Speaker 2: Uh, yes, we are collecting wall imaging data for uh as, as much as we can right now.
[56:31] Speaker 3: OK, great.
[56:33] Speaker 1: Mhm. Very good. More comments or uh
[56:38] Speaker 1: More questions? OK, uh, great, uh, Ciprian, I appreciate it, uh, the time you took for the free, and we'll have it edited and sent to you and be archived, and thank you, Ben, for coming. Uh, and Rosana Romani, if you wanna say hi from Argentina, I don't know if you wanna say hi. Maybe I'm embarrassing you. I'm sorry. OK, anyways, uh, thanks everybody for coming, and we'll end this now.
[57:07] Speaker 1: Thank you. See you later. Thank you.

## In this course

- [Previous: Artificial Intelligence: Complications Detection During Endovascular Procedures](https://doc.social/llms/courses/neurosurgery-and-artificial-intelligence/artificial-intelligence-complications-detection-during-endovascular-procedures.md)
- [All lessons: Neurosurgery and Artificial Intelligence](https://doc.social/llms/courses/neurosurgery-and-artificial-intelligence.md)