Nipy on functional brain MRI
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Part Number | 158 | |
| Number of Parts | 169 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/21090 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
EuroPython 2016158 / 169
1
5
6
7
10
11
12
13
18
20
24
26
29
30
31
32
33
36
39
41
44
48
51
52
53
59
60
62
68
69
71
79
82
83
84
85
90
91
98
99
101
102
106
110
113
114
115
118
122
123
124
125
132
133
135
136
137
140
143
144
145
147
148
149
151
153
154
155
156
158
162
163
166
167
169
00:00
Red HatComputer fontMoment (mathematics)Multiplication signLecture/Conference
00:30
Mathematical singularityComputer fontModul <Datentyp>Host Identity ProtocolDatabaseComputer fontDatabaseBitFunktionalanalysisLecture/ConferenceComputer animation
01:14
Inclusion mapFile formatMeasurementDecision tree learning1 (number)Electronic program guideDemonInstallation artPlanningMassMaxima and minimaInterface (computing)Thermische ZustandsgleichungRight angleModule (mathematics)Electronic mailing listComputer fileType theoryProcess (computing)FunktionalanalysisVideo gamePoint (geometry)File formatSet (mathematics)Endliche ModelltheorieInterface (computing)Lecture/ConferenceComputer animation
02:45
Interface (computing)Maxima and minimaMetropolitan area networkMathematical analysisEmailUniform resource nameWide area networkInterface (computing)Multiplication signAlgorithmProcess (computing)Hybrid computerTime seriesFiber (mathematics)Lecture/ConferenceComputer animationXML
03:26
ArmExecution unitMaxima and minimaGamma functionModule (mathematics)Library (computing)Endliche ModelltheorieStatisticsDataflowDifferent (Kate Ryan album)FunktionalanalysisWebsiteMeasurementLevel (video gaming)Lecture/ConferenceComputer animation
04:42
Level (video gaming)AverageInteractive televisionMeasurementVolume (thermodynamics)Computer animation
05:25
Multiplication signVolume (thermodynamics)Time seriesWord2 (number)Lecture/Conference
05:59
Sample (statistics)ArmMetropolitan area network3 (number)Interior (topology)EmulationMUDLattice (order)Uniform resource nameLetterpress printingMaß <Mathematik>Mathematical singularityStructural loadDifferent (Kate Ryan album)Goodness of fitRight angleDatabaseState of matterLimit (category theory)Point (geometry)Type theoryDimensional analysisBitEmailPhysical systemMetreSet (mathematics)FunktionalanalysisVolume (thermodynamics)VoxelSpeech synthesisElectronic mailing listExtension (kinesiology)Computer fileComputer animation
08:04
Graphical user interfaceData typeoutputPointer (computer programming)Metropolitan area networkTerm (mathematics)Uniform resource nameMathematical singularityObject (grammar)Volume (thermodynamics)Data storage deviceFunktionalanalysisTime seriesMereologyCopyright infringementSubject indexingComputer fileEndliche ModelltheorieStatisticsPlotterShape (magazine)EmailLaptopLecture/ConferenceComputer animation
09:24
FingerprintExecution unitProgram slicingBitMultiplication signStatisticsNoise (electronics)Loop (music)SmoothingComputer animation
10:02
Computer multitaskingProgram slicingMultiplication signOrder (biology)Type theoryVolume (thermodynamics)Moment (mathematics)Computer animationLecture/Conference
10:56
AlgorithmEmailVolume (thermodynamics)2 (number)Order (biology)Social classElectronic mailing listMultiplication signInstance (computer science)FunktionalanalysisProgram slicingSpacetimeReal numberLine (geometry)Open setParameter (computer programming)Optical disc driveVideo gameComputer animation
12:27
Pointer (computer programming)Radio-frequency identificationElectronic mailing listCASE <Informatik>ResultantTransformation (genetics)Process (computing)Dimensional analysisMultiplication signShape (magazine)Volume (thermodynamics)Term (mathematics)MereologyTranslation (relic)FunktionalanalysisDistanceAreaMatrix (mathematics)Social classLecture/ConferenceComputer animation
14:39
Wide area networkRight angleDifferent (Kate Ryan album)Programming paradigmInformation privacySet (mathematics)Computer animation
15:12
Task (computing)Task (computing)Right angleStatisticsAreaProgramming paradigmLecture/ConferencePanel painting
15:49
MeasurementBitRight angleState of matterModal logicOrder (biology)Connected spaceMultilaterationLecture/Conference
16:53
outputGlass floatMountain passDigital filterMaxima and minimaComputer fileSupremumFunktionalanalysisCodeInformationFrequencyComputer fileObject (grammar)Message passingComputer animation
18:04
Auditory maskingAreaProcess (computing)Noise (electronics)VoxelComputer animation
18:42
Auditory maskingFrame problemCalculationProcess (computing)NeuroinformatikLecture/Conference
19:14
Scalar fieldElectronic visual displayContrast (vision)Image resolutionComputer-generated imageryProcess (computing)Volume (thermodynamics)Auditory maskingComputer animation
19:55
Image resolutionContrast (vision)Computer-generated imageryElectronic visual displayRing (mathematics)Chi-squared distributionPointer (computer programming)Maxima and minimaModal logicImage resolutionContrast (vision)InformationImage registrationSet (mathematics)AlgorithmCore dumpLecture/ConferenceComputer animation
20:59
MathematicsPointer (computer programming)Uniform resource nameFunktionalanalysisComputer fileKernel (computing)Clique-widthLimit (category theory)SmoothingMetreLecture/ConferenceComputer animation
21:53
State of matterContrast (vision)MereologyBitDialectMathematical analysisAutocovarianceConnected spaceComputer animationLecture/Conference
23:39
Visualization (computer graphics)Data dictionaryStructural loadPointer (computer programming)PlotterMultiplicationInformation managementSoftware development kitExecution unitSpecial unitary groupPhysical lawTunisArtificial neural networkWide area networkHand fanMetropolitan area networkElectronic visual displayCache (computing)Maxima and minimaSummierbarkeitMoving averageRotationLibrary (computing)AutocovarianceVarianceValue-added networkSound effectMatrix (mathematics)DialectFunktionalanalysisOpen sourceAutocovarianceMeasurementPixelConnected spaceElectronic mailing listSet (mathematics)Computer hardwareInheritance (object-oriented programming)Module (mathematics)Adjacency matrixMathematical analysisComputer animation
25:27
Different (Kate Ryan album)State of matterType theoryDialect1 (number)AreaLecture/Conference
26:26
Conditional-access moduleMetropolitan area networkEmailWordComponent-based software engineeringPrincipal component analysisState of matterIndependence (probability theory)Lecture/ConferenceSource code
27:04
Wide area networkState of matterSystem callInterior (topology)StatisticsComputer networkError messageComponent-based software engineeringNumberSource codeComputer animation
27:45
Raw image formatPort scannerWave packetComponent-based software engineeringVolume (thermodynamics)State of matterMultiplication signTemplate (C++)SoftwareAreaEndliche ModelltheorieType theorySet (mathematics)Different (Kate Ryan album)Module (mathematics)Lecture/ConferenceComputer animation
29:05
Online helpMathematical analysisMultiplication signType theoryLibrary (computing)Process (computing)Latent heatWave packetField (computer science)Different (Kate Ryan album)Measurement1 (number)BitGreatest elementImage processingGeneric programmingAlgorithmSmoothingSoftware developerTheory of relativityInterface (computing)Port scannerState of matterComputer scienceConnected spaceEndliche ModelltheorieSoftwareDialectAreaUniverse (mathematics)Group actionVoxelLine (geometry)ChainInstallation artPhysical lawMereologyObject (grammar)Natural numberPosition operatorInstance (computer science)Cartesian coordinate systemQuicksortLevel (video gaming)NeuroinformatikOctaveMoment (mathematics)MathematicianWeightModal logicInsertion lossDampingArithmetic meanView (database)Video gameFunktionalanalysisSystem callParameter (computer programming)Lecture/ConferenceComputer animation
39:05
MUDRed HatDemonHash functionComputer animationLecture/Conference
Transcript: English(auto-generated)
00:00
Welcome everybody. We'll have a talk by Alexander Savio about brain imaging in Python. Please welcome Alexander. Thank you everyone for coming. Good morning. I'm Alexander Savio.
00:22
I work in the clinic called Hestiaiza in Munich. So, the outline of the talk is going to be, first I'm going to introduce the NiPi tools for neuroimaging.
00:42
I'm going to give a brief introduction to most of them. Then I'm going to explain a bit of what is functional MRI. I'm going to explain a bit of public database
01:01
that's available to do this kind of experiments and I'm going to show a bit how you can pre-process fMRI with the NiPi tools. So, NiPi is the name of a community.
01:22
It's a community of people that do neuroimaging and neuroscience with Python. It's also a module, so there's also a module called NiPi. So, if you go to NiPi.org, you have a nice list
01:40
of the modules available for neuroimaging. So, the first one I would say is NiBabble. Which allows you to open different types of neuroimaging file formats. So, you have IO functions for analyze,
02:08
gifty, nifty, so these are different types of neuroimaging files. Then you have NiPi, which is a set of tools
02:22
to process anatomical and functional MRI images. Then there's NiPipe, which is a tool that allows you to define pipelines
02:43
to process neuroimages. It provides a common interface to many other tools available elsewhere. So, usually people do neuroimaging through command line or Matlab, and NiPipe allows you
03:01
to put all these tools together, so it's very handy. Then there's NiTime, which has many algorithms for time series analysis. And DiPi, which is to process diffusion imaging
03:23
so when you see these images of fiber tracts of the brain, it's with diffusion. So, this is a nice module to process this kind of data. And then you have NeLearn, which is I think
03:42
the newest module, which is done by the keynote speaker of this morning. So, it provides fast and easy methods to do statistics with neuroimaging, and it uses Scikit-Learn.
04:04
So, what is functional MRI? So, functional MRI uses the Bolt signal, which is blood oxygenation level dependent.
04:23
So, it uses the difference in magnetization of the blood flow when you have oxygen and not oxygen in the blood.
04:41
So, this allows you to more or less measure where the activity in the brain is going on, because neuron activity needs oxygen to work. So, yeah, this has some problems, but it works.
05:04
So, the idea is that you acquire one whole volume.
05:25
So, the idea is that you acquire one whole volume each time, and you have, usually you take, every two seconds, you take one whole volume of the brain, and the functional image
05:41
is a time series of volumes, of brain volumes. And so, well, there is an international neuroimaging data sharing initiative, which has many interesting databases with different sets of patients.
06:07
So, you have patients with schizophrenia, Alzheimer's, autism, and not all of them have fMRI,
06:20
but most of them, and you can download them and start playing with this kind of images. So, I'm going to show a bit of how resting the state of MRI is pre-processed.
06:42
So, here I have a nifty file. This is the extension of a zipped nifty file. So, I use a knee bubble to load this image.
07:01
So, I use the load function, and this is the image data with some headers. So, the header is necessary to put the brain
07:21
in the correct coordinate system. Well, here's a more clean list of the header. This is a long list. You have, for example, the type of each voxel, the first three dimensions, the temporal dimensions,
07:44
so this is 240 volumes of this size. And here it says that each voxel has three millimeters, so it's three by three by 3.6, and so on.
08:06
So, this image object has a get data function which stores this volume from the image.
08:21
From the file. So, this is a NumPy array, and it has the shape that we saw before in the header file. So, Nylon, apart from the statistics part, it has some very nice plotting tools
08:43
that they work very well with the IPython notebook. So, I import the plotting sub-module. I use the index image function to pick
09:00
the first volume of the resting image, and I use this plot inet function just to show you how is the first volume of the functional image. I tried to plot the whole time series in a loop, but I couldn't do it with Matplotlib.
09:24
I heard I could use Bokeh, but, well, I'm using a tool that I daily work with, and here, this is the loop of the functional image. As you can see, there's a bit of movement,
09:43
so there's a bit of noise, so this image, before doing any statistics with it, I need to correct for motion, for slice timing, and smooth it, so I'm going to show a little bit about these steps.
10:06
So, slice timing and motion correction is the first step for this type of images. Usually, you need to know what was the slice order
10:21
of the acquisition, so for each volume, the scanner acquired, it acquires one slice per moment, so the slices have different times, and since this is a time-related data, you need to shift these slices in the correct order
10:44
to make this image look better, and so this is dependent on the scanner. So, for example, for this type of image, the slice order is like this, so it's alternating.
11:03
So first, it does the even, the odd slice, and then the even ones, and it alternates.
11:20
So, to do this motion correction, I'm using here NiPi, it has, in this long sub-module, you have the space-time real line class
11:40
that allows you to do motion correction and slice timing correction. So here, I create an instance of this object, of this class, I give my functional image, I say the repetition time, which is also a scanning parameter,
12:02
so this means that it's two seconds for each volume. Slice time is the name of a function to say what is the slice order to acquire each slice. So this class is going to search for the function
12:21
and apply to the image in order to build the slice order list to perform the correction. So I run this, and in this case, it takes around 10 minutes.
12:43
I could use also JobLib to cache the result of this. This is a very nice tool, JobLib is something very used in neuroscience to store the steps
13:00
that are performed in an experiment. And so when this process finishes, I get a result which has the same shape as before, but now it looks better. So I save it, I use a NiPi save image
13:22
to save this in this path. And well, I could, well, so first, I'm going to check the transformation that was performed.
13:41
So realigner is the fit class that I used to correct the image, and I pick the transformations, and for each transformation, I'm only picking the translation part of the transformation.
14:00
So it's an affine transformation. Yeah, so here I'm only using, I'm going to plot the translations. So I have for each dimension X, Y, Z, I have the translation that was applied to each volume of the resting state image.
14:28
So this is the affine matrix of the first transformation. Okay, and let's check how it looks like now.
14:44
So well, it's not very easy to see the differences, but if you check on the top right there, you'll see that there's much less movement than before. Well, you have to trust me.
15:07
So well, with fMRI, there are many paradigms. In fact, in the public data sets I showed you before, you have task fMRI, where you put the subject
15:22
in the scanner and you ask the subject to perform different tasks, and you get different signals depending on the task, and you can do statistics to see what areas are activating according to these tasks.
15:43
But also a very, well, so this is another paradigm is resting state of fMRI, which you just leave the patient inside the scanner,
16:00
you ask them to not think in anything, and you filter the image to only measure a low frequency signal in order to see how the brain is working in rest.
16:21
So with this kind of measures, you can do some diagnosis. So it's a very interesting modality to work with. I'm going to talk a bit about connectivity later.
16:41
So the first thing you have to do with this kind of, with this kind of images, you have to do a bandpass filter. You have to filter it with a low pass filter of 0.08, hertz, and a high pass frequency of 0.01.
17:05
I have this function here, I think I imported it from, well, I copied it from NiPipe, which is something I'm not showing how to use here, so I just copied the code of the function to show you,
17:22
but too complex for now. So I applied this function to my realigned, motion corrected function image, I save it using the two file name of a function
17:42
which comes with the NiBabel objects. So this function is returning me a NiBabel object, a nifty image, and I use the two file name function to save it, it's quite handy.
18:01
And now let's look how it, let's see how it looks now. So it's much messier, but you see some low, well, it's low frequency noise there. No, not noise, it's activity.
18:22
It's true that I could mask the brain area to get only the voxels that are from the brain. And well, this is, so you can see, well, you cannot see something happening here, but this is how these images look like
18:41
when in the middle of the process. So NiLearn allows you to calculate brain masks and it has this compute EPI mask
19:04
to calculate brain mask from FMRI. And well, it does quite a good job if you have a clean image.
19:21
You can also, so here I use, just to check, I use, so the plotting NiLearn to add the brain mask as edge here in the, over the first volume of the resting state image.
19:47
So another thing that we can do is to use the anatomical image. So usually when you do functional MRI, you also acquire an anatomical image. It's also called T1-weighted or MP-Rage
20:02
and it's the MRI modality that gives you the best resolution and tissue contrast, so it allows you to do anatomical segmentation from the brain and you can use that information over the functional image. So for example, I'm showing here
20:23
an anatomical image from the COBRA data set and I would have to register this image to the functional image to work
20:41
with the segmentations that I do here in the functional image. There are some registration algorithms with NiPi tools but I use other tools from, well they run in Matlab.
21:08
Another step, important step is smoothing since these images are so noisy, it's important to do, to smooth them and NiLearn gives you a very nice function,
21:23
it's called smooth image and you can, so this, you give the file, so this is the motion corrected, filtered image and you give the full width half maximum,
21:42
so it's the size of the kernel to do the smoothing in millimeters. And now let's see how the smooth image looks like. So it's black and well, sometimes it happens.
22:04
I'm going to change the contrast and now it looks a little bit better and you can see the function, so it's the resting state part of the functional image.
22:30
So connectivity analysis, so connectivity analysis is when you pick
23:06
this filtered smooth functional image and you select different regions, usually different anatomical images, you extract the signal of each of these regions and you calculate the covariance between these regions,
23:26
so a covariance of the resting state data between these regions. And for that, you usually need a brain atlas, so an atlas gives you regions in the brain
23:41
that are anatomical or functional relevant. So my learn in the data sets super module has, for example, the fetch atlas hardware docs for function which allows you to get a nice atlas.
24:07
So down there I'm showing the regions of these atlas and you can use each of these regions to perform connectivity analysis.
24:21
So my learn is a very nice, it's very well documented tool. So it uses scikit-learn to perform
24:48
the covariance measure and here's the connectivity matrix. So in each pixel of the image,
25:03
you have a covariance measure between the different regions, every other region of the brain, of the atlas that you chose. And it also has a tool to plot the connections between the regions.
25:26
So this is very used to compare different types of patients, subjects in different states. So there's another very interesting thing
25:41
about resting state is that it has been discovered that the brain during rest, it has connected regions that work when you're resting. So these are the most,
26:01
famous ones. People are still doing research over this kind of, this kind of knowledge. And so these are different areas of the brain that are known to work together when you are resting.
26:24
And you can use this, for example, as well to check how well it works in patients with Alzheimer's and other neurodegenerative diseases. So with an iLearn decomposition,
26:41
you can perform ICA, which is independent component analysis. They have, it's called canonical ICA and also DICT learning. And you can use this to extract these resting state components of the functional image.
27:12
So yeah, this didn't work, but, so how I do it, I import from iLearn decomposition,
27:23
DICT learning, I have to set the number of components I want to extract from the resting state image. So basically this, well, so these are the most significant
27:40
components of the whole volume, of the whole resting state volume. And you can also, so apply this to your data and you get the components.
28:01
So here, there's a very nice example from iLearn. So this is, once I extract these components, I plot them over a template,
28:21
and these are the areas where this component is present. So I get different, these different resting state networks. So well, I'm using NiPipe a lot.
28:45
I'm building a module that's called Pipes, which has many of these tools that I showed you that are easy to use with a big data set. So you can configure pipes to run over your data set.
29:02
And it has workflows to different types of images. And yeah, that's all. If you have any question, you need any help with this kind of analysis, please come to me. I'm happy to help. Thank you.
29:26
Thank you, Exxon. Do you have any questions? Thank you for your talk. I have a couple of questions. You showed at the beginning of the talk some algorithms to smooth images and stuff like that.
29:41
I wanted to know if there's something specific about brain images that, for instance, can go to scikit-image or other more generic image processing libraries. And the other one is that I've heard that MATLAB, for instance, is very used to do image processing. And I would like to know how does all these tools
30:00
compare to commercial alternatives? Okay, so there's some of the specifics of this type of images regarding other, so for example, doing brain image with scikit-image.
30:22
So for example, for diffusion image, you need very specific models to fit to your data. For functional image, you could,
30:41
so scikit-image is not really prepared for the, I mean, you can't do it, but for example, you are working with chunks. So for example, to do a smoothing, if you run smoothing with scikit-image, for example, a smoothing you need to know the voxel size,
31:02
because I'm working in millimeters, for example. Yeah, there's some specificities that with scikit-image is not, wouldn't be practical. And so yeah, MATLAB is very used.
31:21
There are other tools that are command line based from different universities. The thing is that if you want to work with this, I mean, you are doing research, basically, and you need to publish. So if you want to publish,
31:41
you have to use a tool that everybody knows, otherwise you are going to get a lot of critics from the reviewers, yeah. So for example, there's a very nice tool from University College London, which is called Statistical Parametric Mapping, which works totally on MATLAB. It has a GUI, so it's easy to use.
32:05
The good thing is that, for example, NiPipe has an interface for most of the useful things that SPM offers. And yeah, there's many, it's another world. There's a lot of tools available to do this.
32:22
Most of them are public, yeah. Other questions? I was wondering how medical research,
32:42
software development, and data algorithmic development go on kind of, I mean, who goes first? What's the relation of, I mean, I guess there are different, of course, different professionalities and teams, but how do they interact, who drive the things forward,
33:05
how everyone can do his own thing and push it to the other? How does it work? Does it work? Does it work? So yeah, it's a multidisciplinary field
33:24
where you have physicists, mathematicians, you have psychologists, you have medical doctors. You have computer scientists. So it depends on the application, I would say.
33:42
If you want to do, if you are in a lab where they do psychology tests, psychologist experiments, mostly it's a, could be a physiologist who is leading the ideas or a psychologist. Medical doctors are there to check
34:03
whether your data is saying something to them because they are the ones that know about the diseases, the brain. Of course, in the end, I have to, I'm a computer scientist and I have to learn a bit of brain anatomy,
34:24
understand a bit the diseases as well, but most, I would say computer scientists are at the bottom of it.
34:43
Thank you for the talk. Actually, I was wondering why in this particular field you need so many tools. In the beginning of your talk, you showed us different tools. All of them are in Python, that's gorgeous, but why so many?
35:03
So yeah, so the MRI scanner is a very flexible tool. You can set, you can configure it to run different types of measures from the brain and each of these measures, they are,
35:21
they are one whole field of research on its own. So you have diffusion, you have functional, you have anatomical imaging. So all these fields, they are, so all these modalities, they are, they have different types of analysis processes.
35:49
Thanks for your talk. Difficult to talk about computer science and neuroscience in one slot. I'm wondering if you guys have any training and if there is training, if you guys have developed any training
36:01
specifically for scientists to get involved in this rather than computer scientists who are trying to learn the science, so to speak. Well, I have been, well I tried in May to, I'm doing software Carpentries and I included a NiPipe, a NiPipe tutorial on,
36:23
so half a day of NiPipe and people enjoyed it. So because, mostly because they know the tools that are under it, so the MATLAB tools, the common line tools, so I wanted to show them how to use NiPipe because it's very useful.
36:41
For now, that's it. I mean, I'm not paid to run a software Carpentry or in fact, I had to get some holidays, but yeah, it's something I enjoy doing. If there are no more questions, I have one. You mentioned an atlas of connectivity, of brain activity.
37:05
What does, what do the entries in this atlas represent exactly? Is it a special action that the body does? Sorry, can you repeat? So there is an atlas of brain connectivity, brain activity?
37:21
Yes. If we take an entry in this atlas, like is it, it would be, if I understand well, some regions of the brain which would be active at the same moment? Yes, so it's a, yeah, so it's how the different areas
37:42
work together when the, so for example, in resting state connectivity, you are measuring how synchronized are these regions. Okay, and then what does that represent? Can you ensure that the body is doing
38:00
a special movement or? No, it's resting, so the subject is quiet in the scanner, but you can see, for example, how in Alzheimer's disease patients, some areas, they are not, they are disconnected,
38:23
or the connection is different or lower. Yeah, this is. Okay, thank you. Thank you. Still one question? So the last one.
38:42
Have you had your brain scanned? Many times. Did it look familiar or? It looks fine. It's fine. So I do have, I do have it here. So that is one of my brain scans.
39:04
So this is me inside and this is me outside. Thank you very much, Alexander.