The Brain’s Codes for Space and Time
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 340 | |
| Author | ||
| License | CC Attribution - NonCommercial - NoDerivatives 4.0 International: You are free to use, copy, distribute and transmit the work or content in unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/45069 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
Lindau Nobel Laureate Meetings302 / 340
3
5
15
16
18
22
23
27
32
33
34
35
47
54
58
67
69
70
72
73
78
79
83
84
85
86
87
90
92
95
100
102
103
104
105
106
114
115
116
118
119
120
122
126
128
129
130
131
133
137
142
143
145
147
148
149
153
155
156
159
162
163
165
168
169
174
176
178
181
182
189
194
198
201
202
203
206
209
213
214
217
218
219
220
225
227
228
237
240
241
244
245
250
254
257
260
261
266
273
278
284
285
287
291
293
297
302
308
310
317
318
319
321
325
327
328
333
00:00
Lecture/Conference
00:35
Lecture/Conference
01:16
Lecture/Conference
09:14
Lecture/Conference
10:09
Lecture/ConferenceComputer animation
18:09
Lecture/Conference
23:45
Lecture/Conference
24:46
Lecture/Conference
31:00
Lecture/Conference
31:24
Computer animation
Transcript: English(auto-generated)
00:14
So, good morning or afternoon. I would like to thank the organizers for inviting me to spend half an hour
00:24
on the work that we are doing in Trondheim in Norway. I want to begin by introducing you to the field and some of the basic work we have done over the last 10, 15 years, and then spend the second part on new, unpublished material
00:41
that will give you an impression of where we are today and maybe where you in the audience may take the field in the future. So, let me begin then with the cortex. The cortex is the sheet of our brain which has most developed
01:01
late in evolution and which is responsible for much of our intellectual activity. So, I'm a psychologist by training, and then, naturally, I'm particularly interested in that. But understanding the cortex of the brain has been very challenging and, until quite recently, almost considered inaccessible.
01:23
So, there have been some advantages, especially in the sensory cortices, where people like Torsten Wiesel and David Jubel, actually almost 60 years ago, started to investigate the sensory cortices and found many of the building blocks of cortical computation.
01:45
But what has been much more inaccessible is the stations of the brain that are further in, which we often refer to as association cortices. They are really hard to understand because you don't find activity there
02:01
that can easily be referred to something in the outside world. So, decomposing this has been really, really difficult. But then, there are maybe a few areas where things are a little bit easier, and one is the area of space, our spatial mapping system,
02:21
which I illustrate here by showing some examples. One is from the Garden of Assay, a difficult task to find your way here, and equally difficult also if you are in the desert, and landscapes are very monotonic. But in most cases, it's not like that, and we find our way without even thinking about it.
02:41
It's a task that the brain solves without much conscious activity. So, how does the brain do that? That's a question that has interested me a lot. And the reason for being interested in this is not only because I wonder about space, but it is because it is one of the areas
03:00
where much understanding has been obtained and where there are correlates of cells that actually can be referred to the outer world. So, I'll show you that now quite soon. But let's begin first with the positioning systems of the brain. So, knowing where you are and knowing where you should go
03:20
involves a lot of brain, not only two or three areas. It's all over. But there are some areas that still are particularly important, and those include the hippocampus, the red area here, and, more recently, the entorhinal cortex, which is the blue area. So, both of those are in the medial temporal lobe on the side of the brain, somewhat embedded under,
03:42
so you can't see it from the surface. They are old parts of the brain and haven't really changed very much in evolution. And that's actually a big advantage for us, because studying the human brain, if you really want to understand navigation and how you know where you are, it's not easy to do that with a cellular resolution
04:03
in the human brain. So then, it comes as an advantage that the hippocampus and entorhinal cortex have a pretty much similar structure in all mammals, more or less. So, I show three examples here. You see to the left, rat, and then middle, monkey,
04:22
and then humans, and then in the middle, you can see here, first again, to the right, you see the human brain, red and blue, and then monkey brain, and then to the left, it's a rat brain, where you see that the structures are positioned somewhat differently,
04:43
but still, they look similar. They consist of similar circuits, neural circuits, similar neurons that are wired together in similar ways, and also have similar physiological properties. And in addition, then, we know that navigation and our ability to know where we are
05:03
and go from one place to the other hasn't really evolved very much. It's a fundamental ability that all mammals share and need to solve in order to survive, actually. So, we can actually expect to learn a lot from,
05:20
for example, rodents, about how navigation is accomplished in the human brain. And with that as a starting point, then a lot has been done, and one of the major discoveries in the history of this field is John O'Keefe's finding in 1971
05:40
that there are neurons in the hippocampus of rats. We now know they are in humans and other mammals, too. Neurons that signal the position of an animal or a rat as it moves around in space. So here, you see a rat in a box. A box is usually typically in the lab.
06:02
It's a one-by-one-meter-large box, and then the rat walks around. It has a cable connected to electrodes, electrodes in the brain, in the hippocampus. Those are very, very thin, so typically, they have been around 15 to 20 micrometers wide,
06:22
so they sort of squeeze in between neurons and then pick up electrical activity, after a solution of single cells. And then with this approach, O'Keefe then found the following, which I'm going to show in a movie. So this is a movie of a rat walking around in a box,
06:43
chasing for chocolate sprinkles, and that keeps it going all the time. And at the same time, we listen to the activity of one single cell in the hippocampus as the rat is walking around. And each time that cell is active or fires an action potential,
07:00
you will hear a pop sound, and you'll also see a red dot appearing on the screen. So let's start the movie. You now hear the action potentials or the electrical discharges of that particular cell, and you can already see and hear that these discharges occur
07:21
when the rat is at a particular place in the box, only when it's in the upper left part of the box. Otherwise, the cell is silent, does not fire action potentials. So because this cell is active at a single place, some of them are active at more places, but anyway, then O'Keefe called them place cells,
07:43
cells that fire at certain places. And then he found that different neurons in the hippocampus fire at different places. So I'll illustrate it here with a color-coded map or heat map showing for that cell you see, again, red is high activity and blue is low activity.
08:01
And then different cells in the hippocampus have different activity fields, so fire or are active in different places in the box, so that if you have 100 cells, you can be pretty sure that any possible place in the box is covered by some neuron. So the combination of activity then tells,
08:20
whatever structure reads out this activity, tells exactly where the rat is at any given moment. And because you can read out the position of the rat from the combined activity of the cells, then O'Keefe and his colleague, Leonard Del, suggested a few years later, in 1978,
08:41
that the hippocampus is the basis of an internal map of space. They call it a cognitive map. Then borrowing the term from Edward Tolman, who already in the 1940s suggested that we must have such a map, but then he didn't have the tools to find them in the brain. So this was 1978.
09:02
And then lots of things happened, which I'll skip, but over the next 20, 30 years, properties of place cells were better understood. But there were some questions that still remained unresolved, and one of them was, where does this place cell signal come from? Because this is, in a way, in the middle of the cortex.
09:24
It's far away from the sensory cortex. It doesn't belong to any sensory cortex. The cortex is really at the convergence from many sensory inputs, and it's really quite surprising that there is such a sharp signal about the position of the rat, because there is no place signal in the outer world.
09:43
You don't sense place through your fingers or your ears or your eyes. So how is this created? It was a big mystery still, and that's one of the questions that, when Mairet Moser and I started a lab in 1996 in Trondheim, in Norway, then this is one of the things that we really wanted to understand.
10:05
So what we began with was to try to find out what could be the input to the place cells. So what is it that comes into the hippocampus? And the natural choice was to record also from the entorhinal cortex,
10:22
you remember, the blue area in the human brain. So to the right, you see a rat brain from the behind, and the colored area shows the entorhinal cortex. And we chose to record from the upper region, which is called dorsal, because that's what really feeds into the dorsal hippocampus,
10:44
where most of the place cells had been recorded at that time. No one had ever recorded there, so it was totally unknown what was going on in that area. So what we discovered was, first of all, that there are spatial cells there, too,
11:02
cells that are active at certain places. But we also found that those cells are different from the ones in the hippocampus, and I'll show you that in the next slide. So what you see here, in the bottom right, is an overview picture of a similar box, as you saw,
11:20
but now the box is much bigger, it's 220 by 220 centimeters, and the gray trace is where the rat was walking over a period of 30 minutes, and then each black dot is the location where the cell was active, like the red dots in the movie. And what you see is that now the cell doesn't only have one area where it's active,
11:42
it's actually many areas, and, as you see, those areas form a particular pattern, a very regular pattern that spreads out all over the available space. And I tried to help you seeing the pattern by putting these red lines on top of the other image, and you can see it's repeating equilateral triangles,
12:03
or actually a hexagonal pattern, a regular hexagon that repeats itself all over the space. So it's like a grid, and for that reason we call these cells grid cells. So these cells contain information about both distances and directions, and if you use combined input from them, feed that to computer,
12:24
you can very easily, just like for place cells, but actually more accurately tell where the animal is. So we observed that grid cells come in different varieties. Some have a small scale, some have a large scale. So what you see here to the left is an image of a rat brain seen from the side.
12:46
The red area is the entorhinal cortex, which then feeds into the hippocampus, which is this air-like structure to the left. And if you start from the top, or the dorsal area of the entorhinal cortex, and then go down gradually,
13:02
what you will see is that the scale of the grid increases. So at the top you have cells that have very high frequency, or very close activity peaks, and then the peaks get bigger and more distant as you go further down, and at the bottom, of course, it's more difficult to measure,
13:22
because you need big boxes, but at least several meters between each field. And the top is a distance about maybe 25, 30 centimeters at the smallest in a rat. So, of course, now we know that these maps of grid cells consist of different scales,
13:42
but we still wonder, is this a continuous scale, or is it discrete, other maps of different scales that are actually quite non-overlapping? It turned out to be the latter, because what you see here on the x-axis, this is a recording from one single animal.
14:02
On the x-axis you see a position along the entorhinal cortex, so from top to bottom, and on the y-axis you see the size or the scale of the grid, the distance between the peaks, and then each dot is one cell, and you can then see that, yes, the scale increases as you go from left to right or from top to bottom,
14:21
but you will also see that there are actually discrete levels, so in this case four levels, we call them modules, and we call them module one, M1, M2, M3, M4, and this is very, very consistent. It happens in every single animal, so we find usually four or five, but there may be a few more.
14:41
What we also find is that there is a distinct scale relationship between them, so that means to get from module one to module two, you multiply by a constant factor, a factor of about 1.4. To get from module two to module three, you again multiply by 1.4. So let's say that module one has a scale of 30 centimeters,
15:05
then module two has a scale of 30 times 1.4, and then module three has 30 times 1.4 times 1.4, and so on. So it's like a geometric progression where you keep a constant scale relationship,
15:22
and we still don't understand why that is an advantage, but it has been one of the ideas is that this is the most efficient way of organizing representation of space if you want to use as few neurons as possible. So how does this arise?
15:40
Well, a natural way to investigate that is actually try to get rid of as many sensory inputs as possible, because naturally, in most cases, we have recorded these grid cells when the rats walk around in an open box, as you saw, and then they have proprioceptive motor inputs from the legs as they move.
16:02
They have visual inputs and maybe some odor inputs, so you want to get rid of that, and then that can be done, for example, by recording while they're sleeping. And of course, you don't get movement when they sleep, but what we find is that there's still a kind of hidden grid structure
16:21
because those cells that are active at the same time when the rat is awake and moves around, they are also active at the same time when the rat sleeps. And those that are not active together when the rat walks around are also not active together when the rat sleeps. So you see that here, just two examples. So if you take first those that are active together,
16:43
the in-phase, as we call them, you see two grid cells that have dots on the same places in the box, and then to the right, you see so-called cross-correlation diagrams. That means that you see the probability of cell number two firing when cell number one is active,
17:00
and you see the peak is at zero when the rat is running, so that's our starting point. Of course, when they have peaks at the same places, the probability of cell two being active when cell one is active, that's very high at zero. But then during sleep, both the slowest sleep and REM sleep,
17:21
then you can see you have similar peaks. So it actually means that the cells that are active together in the active state are also active together in the sleep state. And the bottom row shows then cells that are out of phase, and you can again, now you see a dip in the cross-correligrams, and that's also maintained in the sleep state. So it means that actually you have the grid pattern maintained
17:42
even when the rat is sleeping, which points to an intrinsic structure creating this grid. It's not something that really is made from the sensor input, but it's something that probably this area of the brain creates by itself. So just to say that this is not only for rats and mice,
18:03
it also exists in bats and in monkeys and in humans, but much more difficult, of course, to study in those species. So now then, a few words about other types of cells. So it turned out that while grid cells are very predominant
18:23
in the entorhinal cortex, they are not alone. So first of all, there are cells that signal the direction of the animal, regardless of its absolute location in space. So these are cells that actually were found earlier by Jim Rank already in 1985 in a neighboring region.
18:41
So they are cells that, if you just focus on the plots to the right, which are polar plots, then show that the cell is active when the rat is going in the left direction, regardless of where it is in space. And the other cell is active when the rat is going in the up, in the towards the northwest direction in the box.
19:02
So direction cells, or compass cells, are cells that are active exclusively when the rat is at borders of the environment. So this is now a heat map, and you see, the first map shows one cell that is active only in the right part of the box, seen from above.
19:20
And then if you stretch the box in various directions, it's still active on the right side. And below, this is when the rat is taken to a different room. So now it remaps, but chooses the left side instead. And then if you insert a wall, it even fires along that wall insert as well. So these are, we call them border cells, so it's a completely different type of cell,
19:41
but it's intermingled among the grid cells. So somehow these are very tightly linked, and it may be that these cells actually provide some sort of visual reference frame for the grid cells to become stable in the space where the rat is. Then there are speed cells, which we found in 2015,
20:03
which are cells that, as you see in the left diagram, then you see one cell per line, so two minutes. Color shows the firing rate or activity of each cell. And in gray, you see the speed of the rat as it's walking around. And for example, if you focus on the yellow one, you can see how closely the firing rate actually follows the speed of the rat.
20:24
So it's like a speedometer. These cells really read out the speed of the animal, and then, as you can imagine, this must be an essential part of the network that creates a grid cell, too. I mean, you can't make it out of speed alone, but you need to know the speed for the cell actually to fire at the right spaces.
20:45
Otherwise, you can't keep the firing at the same place each time. And then, finally, I want to share with you some new type of cell that we just have found, still not published.
21:01
But these are cells that encode some sort of vector relationship to significant objects in the space. So, as you realized, until now, I just showed you empty boxes. And, of course, we try to strip off everything, make it as simple as possible in order to find patterns. But the real world still is more complex than that,
21:22
and, among other things, contains objects. So if you now put an object into space, like the circular one there to the left, as a mouse, and then a tower made of Lego, then what we found, and we mean Even Heudel, who is a PhD student in our lab, was that cells that don't really choose any particular region to fire
21:44
when there is no object, when it's an empty space, as you can see to the left, then when you introduce an object, an object is shown as a white circle, again, it is mostly a Lego tower of some sort, then the cell suddenly gets a very discrete firing field,
22:01
and that firing field is not at the object, but is displaced somehow in some direction, can be any direction. In this case it's to the north of the object, and at some distance, and that distance can be anything between zero and up to at least 50 centimeters. And then if you move the object, as in the right panel there,
22:20
you can see that it still maintains its north vector relationship to the object. And you can choose different objects, like here, different prism-like objects, different cylinder-like objects, put them in at the same time, as you can see for these five cells here. So the circles show the objects, and then you can see, for example, for cell 8,
22:42
then you can see this is a cell that has fields on the southeast side of the object, or if you look at cell 3, you can see one that fires on the west side of each of the objects. Regardless of which object it is, they share some similarities, but they're also quite different. So what these cells do is that they signal direction and distance
23:03
from prominent landmarks in the open space, which then tells us that this system that contains grid cells as a kind of coordinate system also has another population which is equally large, about 15 percent of the cells in the superficial layers,
23:23
that encodes a vector relationship to significant objects in the space. So there are two ways to compute position. You can use a grid-like pattern, like a coordinate system, or like a graphics paper that we had in the old days, or you can use a vector system that's based on these new types of cells.
23:45
And then finally, during my last six minutes, I want to talk about another aspect of computation in hippocampal and entorhinal cortices, which is time. So there has been a lot of progress in our understanding of space during the past decades.
24:05
And we know a lot of the components and have some ideas about the computations that are going on. But still, any experience happens not only in space, it always happens also in time. And one may wonder how much does space and time actually come together.
24:20
And it turns out that there is actually a quite significant representation also of time in the entorhinal cortices. But that's not in the medial entorhinal cortex, which I talked about until now. So it is in the sister region of that region, which is called lateral entorhinal cortex. So it's next to it.
24:41
And I'll show you this work, which is also a former PhD student of us, now a postdoc at Stanford, whose name is Albert Tsao, who has definitely really made huge progress on our understanding of time in hippocampal systems.
25:02
So what you see here, first, again, a recording box. A rat is walking around. It could either be a white one, as in this case, or it could be a black one. There are two versions. And then at the bottom, you see a row of trials where the rat alternates between trials in black and white
25:20
semi-randomly between black and white through a sequence. And then in between, there are pauses where it just rests on a box. So he recorded throughout the sequence. And this was then to get a sequence of activity that extended over a time of approximately one and a half hours.
25:43
So first of all, in the lateral entorhinal cortex, he found cells that have firing rates that follow the progression of time, either as you see in the top here, which shows blocks of black and white boxes. So you see black, white, white, black, black, white, and so on.
26:01
And you can see that within each box trial, you see the cell is ramping up in activity. Or there are others that increase, whose activity decreases not only within the trials but also across the whole course of time, or others at the same time increase. It's not due to instability. This is a real change in the firing rates of the cells.
26:23
Or there could be more complicated schemes like combinations where it only ramps up or down depending on the color of the box. So what he did, first analyzing still a single cell level, was to break it down and use a GLM analysis, a general linear model to find out
26:41
what was the contribution of different factors to the firing rate of all cells. And then he compared three brain areas, the lateral entorhinal cortex, LEC, CA3 of the hippocampus, and the medial entorhinal cortex, where you have the grid cells and all the object vector cells and all these other spatial cells. And what you can see, the bottom line here is time.
27:02
Time is one of the factors in the analysis. And you can see it's very high compared to all other factors in the lateral entorhinal cortex, whereas in the two other areas, time is very, very low. There's not much representation of time. Although it's not zero, that's very important, but I have no time to say much about that.
27:22
So this is at the single cell level, but it actually turns out that most of the cells are not like this. So time on Lake Space really seems to be distributed among many, many neurons. So it's actually quite hard to capture because each neuron may just have a little bit or trace of it, but it's not like the grid cells where it's all or none,
27:42
and you can tell immediately that this cell is firing for some aspect of space. So he used a population approach. Instead, he used a machine learning decoding approach using a leave one out procedure, where he trains a network based on activity of many, many cells. All the cells are recorded.
28:01
And then leaves one trial or one block of time out, and then uses the trained network to predict what was actually the time when the left out part was taken. And then this shows number of hits, so decoding accuracy. And then, again, compare lateral entorhinal cortex CA3 and MEC,
28:23
and you can see that almost every time the network or the computer hits, it says exactly the right time when this happened out of the combined activity of all of these cells. And the right diagram just shows the same. But then the final data slide here shows
28:42
that this is actually dependent on the nature of the task. So if you now do not let the rat walk in an open box, but use a structured task where it walks in a figure eight pattern, over and over again the same task, then what happens is that the decoding accuracy
29:01
for which trial this was taken from is actually going down. It's the one to the left compared to BW12 is the box. So that's mysterious. But if you now instead, so the same is shown once again to the left, but if you now look at the right, it shows decoding of time relative to some temporal landmark.
29:24
And that temporal landmark is when the rat passes a certain point where it goes into the central arm. Because then it can actually decode time very precisely and more precisely than when the rat walks in the box. And what that tells us is that once you start training the rat on a task
29:44
and doing the same over and over again, you get coding of time that's no longer just free-floating randomly, but it's actually relative to temporal landmarks. And that means that this is a time code that is not really absolute time. It's totally different from the circadian reasons that we heard about this morning.
30:02
But this is time relative to events. So this is order or sequences or events, and it's a different kind of timing that goes into the hippocampal system. So my final slide then is what does this tell us about neural computation. I think we're still at the beginning.
30:20
So we are at the nuts and bolts level. We know many of the elements of the space system and also beginning now with time. But how they work together is what we really want to understand and what we need to understand to know how we get the sense of space and time. And as it's obvious, here you don't only have to do experiments.
30:42
There are now new techniques which allow you to get activity from many cells. But what you really also need is quantitative skills. So then invite all of you who have some interest in computation, in mathematics, in physics, come with it, because that's really needed now during the next decade. And with that, just mention the people who have been involved,
31:02
especially Margaret Moser, many collaborators. I want to mention again Albert Sow for the time coding and Even Heudel for the object vector cells. And then many people have funded this. And with that, I want to thank you for your attention.
Recommendations
Series of 13 media