The future of nueroimaging
Psychologist Russ Poldrack is a mind reader of sorts, but not in the traditional sense of the word.
Instead, he makes movies of blood flow in the brain using functional MRI to draw insights into the function of specific regions of the brain to decipher how they control behavior. In one example, Poldrack has shown how people with obsessive-compulsive disorder have enhanced “salience networks” that turn on when surprising things happen. This finding could have “very profound” outcomes in diagnosis and treatment, Poldrack tells host Russ Altman in this wide-ranging exploration of the implications and the ethics of functional neuroimaging on Stanford Engineering’s The Future of Everything podcast.
Transcript
[00:00:00] Russ Altman: This is Stanford Engineering's The Future of Everything, and I'm your host, Russ Altman. Since we started this show eight years ago, it's become an archive of amazing and impactful work by my Stanford colleagues. Research is not something that just happens in the lab, and as you'll hear on this show, the research at Stanford can impact areas like health, technology, law, and business, and many other topics that can affect everyday life.
[00:00:23] We hope you'll tune in to learn more about how research has the potential to help your life and to help the lives of people you care about in your family and your community.
[00:00:32] Russ Poldrack: I've written a lot about how we don't really believe you can, like, read the contents off of a person's mind right now with the methods that we have, right?
[00:00:38] You can, you can get some information. You can certainly read off, like, what kinds of things they're thinking about. But in terms of, like, the, the the train of thought from inside your head kind of thing.
[00:00:46] Russ Altman: Right or lie detection.
[00:00:46] Russ Poldrack: Right, right, exactly. That kind of stuff just doesn't work right now.
[00:00:56] Russ Altman: This is Stanford Engineering's The Future of Everything, and I'm your host Russ Altman. Thanks for listening, and if you have a moment, please rate and review the podcast. We love to get a 5.0 if we deserve it. Your comments, we read every one of them, and it helps us grow the show.
[00:01:09] Today, Russ Poldrack will tell us that functional MRI imaging of the brain can do a lot. It can tell us the similarities between humans and also the differences in their brains, and it can potentially detect brain disorders early and help us more quickly identify diagnoses that need to be treated. It's 'The Future of Neuroimaging.'
[00:01:33] Today, we're continuing our feature, which we call The Future in a Minute. At the end of our discussion, I will ask Russ a few rapid-fire questions, and he will give us some rapid-fire answers. And before we get started, another reminder to rate and review the show. It's hugely important for growing it, for giving feedback to us, and so we can see how well we're doing
[00:01:59] So I think we all know that brains are complicated. And we also know that we can measure our brain with CT scans of the brain and with MRIs of the brain. That often happens if you bonk your head or there's a problem, a seizure, migraine headaches, lots of reasons why you might get an MRI of the brain.
[00:02:15] But one of the things that researchers can do is not just take single pictures of the brain, but they can take a picture of the brain like a movie while the brain is working, while it's thinking about things, or while you're performing specific tasks. This is called functional MRI, and it's based on looking where the blood flows because the parts of your brain that are working the hardest are where the blood flows the most.
[00:02:38] Well, you can use that then to understand which parts of the brain are involved with which kinds of activities and how the brain coordinates and communicates all the things that are going on in your body at the single moment in time and over time during the trajectory of your activities.
[00:02:55] Well, Russ Poldrack is a professor of psychology and of psychiatry and behavioral science at Stanford University. He's an expert at neuroimaging and the use of functional MRI to dissect the similarities and differences between different people's brains and how this may all be correlated with disease.
[00:03:15] Russ, how did you decide to focus your research on neuro and brain imaging?
[00:03:20] Russ Poldrack: You know, I, I started out in cognitive psychology in graduate school, just studying, you know, how people's minds work by measuring how long it takes them to do things, how quickly they learn, stuff like that.
[00:03:31] And then I came to Stanford actually to do a postdoc and wasn't planning on doing brain imaging. I was planning on studying, you know, people with brain disorders. And this was right when functional MRI was, you know, kind of up and coming. Stanford was one of the few places that had been doing it in the early 1990s. And I kind of got sucked into it. And in part, it, it played to my love of playing around with data. and you know, because the data you get from brain imaging are just so much richer and bigger than what we got from behavioral studies.
[00:04:02] And so, that's how I got pulled in back in the mid-1990s. and then it's, you know, since then, functional MRI has become the primary tool that my lab uses to try to understand sort of, you know, how it is that the brain makes the mind work.
[00:04:18] Russ Altman: Great. Great. So, you you mentioned a couple of things, and I did wanna get to kind of a little basic tutorial for people who don't think about neuroimaging every day. What is functional MRI? Can you give us a brief thumbnail about how it works? And then, and then we can get into how you're using it to ask the research questions that you ask.
[00:04:36] Russ Poldrack: Sure, yeah. So, we, you know, when, when the brain does stuff, neurons fire action potentials. They do a lot of metabolic activity. And what, you know, we would love to be able to directly measure that, but without going into a person's skull, we can't do that noninvasively, or at least we couldn't until people figured out that, something interesting happens when neurons fire, which is that the brain sends extra blood there.
[00:05:02] It actually sends more than it needs to send to make up for the oxygen that's used by the brain. And so, you can use the fact that, basically oxygenated and deoxygenated hemoglobin have slightly different magnetic characteristics. This was discovered by Linus Pauling back in the 1930s. You can build MRI, what we call sequences, basically like programs for the MRI scanner, that are sensitive to this slight difference in the magnetic characteristics, and you can use that to basically indirectly image where activity changes in the brain.
[00:05:38] Russ Altman: Wow. Okay. And then this is for, at what resolution? Are we looking at centimeters or inches or, millimeters? Yeah. What, what, how, how fine grain can your measurements get?
[00:05:51] Russ Poldrack: So, the standard measurements that we do, it depends on the the strength of the magnetic field. So, the, the standard scanners that we use, which are kind of like the high-end scanners that would be used in a hospital today, are, the, the magnet is 3 Tesla. That's the measurement of the magnetic field.
[00:06:06] At 3 Tesla, we can generally measure about two millimeters.
[00:06:11] Russ Altman: Okay.
[00:06:12] Russ Poldrack: And we're, we're constrained not so much by... We could make the pixels smaller, what we call voxels, 'cause they're 3D pixels. We could make them smaller, but the problem is that the, the mechanism that we're measuring, this blood flow mechanism, the blood actually has to kind of go downstream a bit of the actual capillary bed before there's enough for us to see it. And so, at 3T, that is a, a, a, a, a millimeter or two.
[00:06:40] At higher field strength, you can actually do better because you can see things happening closer to the capillaries. And so, we're starting to collaborate now with people at Stanford. There's a new 7 Tesla MRI system that we're hoping to do, and then you can get down to about a millimeter, depending on, on how you wanna do it.
[00:06:59] Russ Altman: Okay. So, if, to, to summarize what I think we've learned is that you're measuring what part of the brain is kind of working because it's using up, its more blood, and that is the whole brain. You're looking at the whole brain, and so I guess, and, and you'll tell me if I'm wrong, and I, and I have heard you speak a couple of times, that what we're doing then is we ask a, a study participant to do some sort of cognitive test.
[00:07:21] Think about this, do this activity, or are you sad or are you not sad? And then we see what parts of the brain are, are, are active, and that allows you to create these associations between parts of the brain and types of activity. So, is that generally the idea?
[00:07:38] Russ Poldrack: That's, that's one
[00:07:39] Russ Altman: Okay
[00:07:39] Russ Poldrack: ... that we do things.
[00:07:40] Yeah, so that's the classic way. When I started doing functional MRI, you know, 30 years ago almost, we would give people tasks to do, and we still do that. You know, they sit in the MRI scanner, they're lying there with buttons and, you know, visual presentation.
[00:07:53] Russ Altman: Yeah.
[00:07:54] Russ Poldrack: We, we do things like having them, you know, make actions and then occasionally stop themself or change what they're doing. We're interested in how people kind of control their behavior.
[00:08:02] There's another one thing that you can do, which is simply have them lay there and do nothing and collect data, and then look at kind of what's correlated with what over time, 'cause you're taking a picture every roughly, you know, one and a half to two seconds.
[00:08:16] Russ Altman: Okay.
[00:08:17] Russ Poldrack: And so, this is what we call resting functional MRI, and this actually has turned out to be super useful in understanding structure in the brain that we didn't really know about until the early 2000s when people started doing this. It turns out that there's this set of, we now call them kind of like, you know, large scale networks in the brain, depending on who you ask, maybe 13, 15 of them, maybe 20, that engage at the same time, and that engage in relation to the same types of things.
[00:08:48] So for example, there's one network we call the salience network that, which involves some places in the parietal lobe and some places in the frontal lobe, that anytime something surprising happens, that whole network kind of turns on.
[00:09:00] Russ Altman: Okay. So, good. So that is actually super exciting. and I wanted to ask, now tell me what are some of the exciting things that you're doing? I know that you, you even mentioned in the introduction that you had an interest in, what went wrong in the brain and mental illness and, and other diseases of the brain. So how are you approaching it? What are the questions that you're asking?
[00:09:22] Russ Poldrack: Yeah, the, I mean the, the big questions that we're asking right now in the lab are, there- there's a couple of, of, of directions I'll talk about.
[00:09:29] One is sort of how, it goes back to this idea of, you know, how we control our behavior. We think that, you know, a number of different, mental health disorders and neurological disorders involve deficits in the ability to, for example, stop yourself from thinking something or stop yourself from doing something.
[00:09:48] So we've, one of the, the things that my, my lab has studied the most is how we stop ourselves. So, we, and we do this in a really simple way. We have you doing a task where you usually press a button, but occasionally you, you get a signal that says, "Oh, don't press the button." We call it a stop signal task.
[00:10:02] And we, we, you know, a long time ago first kind of worked out the circuitry in the brain, in the human brain that allows you to stop. And what we're, a lot of the work we're doing now is trying to better understand how that relates to other aspects of control. For example, if you are doing one task and then you have to change to doing another task, or if you have a habit that you're trying to override, you know, those different aspects of control.
[00:10:28] So we're trying to understand to what degree they involve sort of, you know, overlapping versus separate circuitry in the human brain.
[00:10:34] Russ Altman: Yeah. So, you mentioned, tying that back to your previous comment, you described that there are these networks in the brain. You, you, I think you called it organization. You said the brain is organized and I, and it's organized into these, I guess, sets of cells that communicate with each other a lot. What are the kinds of...
[00:10:49] So I, I assume that you're looking at those networks when you're asking these questions. And so, is it that all 15 of them are, or 20, or whatever it is, are interacting, or do you find that certain of them are quiet during some tasks, but they get more active at others?
[00:11:07] And it's, it just sounds very complicated because, you know, we all, even we know from, from intuition that our brain has a lot of things going on at any given moment. So how do you figure out what you can, ascribe to the task that you've asked them to do versus they're just trying to keep the, the body moving and, you know, maybe their mind is wandering to other things because they're sitting in a big tube, it's making a lot of noise? How, how do you do all that?
[00:11:32] Russ Poldrack: Yeah. Well, one, you know, we, I, I think you're, you're exactly right that the brain is a complicated thing. So, there's not just 15 things going on at once in the brain. There's, you know, probably billions of things. And in part, you know, we have to take kind of a hierarchical approach, right?
[00:11:45] We, you know, we look at the brain at a relatively, I, I would say mesoscopic, kind of middle scale view, right down to maybe a millimeter up to, like, these networks are often, you know, centimeters. Obviously, there are people, you know, in neuroscience who look at things down at the molecular level, the cellular level.
[00:12:02] So, we're taking a particular view where there are going to be fewer things going on. And one of the interesting things that we know is that within those, say, 15 networks, there's substructure as well. There are individual areas, there may be hundreds of them in the brain, that, that we know make up those, those larger scale areas, and they will also show differences. Even though they, they might together move around, say, when you get some, when you see something surprising, different ones are gonna react in slightly different ways.
[00:12:33] And so, we can use... we don't just analyze things at the level of those large-scale networks, we analyze them at many different levels, down to the, to the, the millimeter level. And we do, you know, there's, there's various ways that we can ask these questions. One is, you know, how much activity is there in, you know, in those particular things? Another interesting question that, or an interesting direction that's come about is asking for a for a particular individual, how much of their brain is sort of, assigned to each of those networks?
[00:13:04] Russ Altman: Yeah. 'Cause, you know, before you go on, I wanted to ask, are those networks the same for everybody, and are they in the same spots?
[00:13:12] Russ Poldrack: So, sort of. Yeah. So, everybody, pretty much everybody has all the networks. Okay. Right? And until about 15 years ago, we didn't really know. You know, we thought, we knew that, that everybody kind of had all the networks, but we never had enough data from any individual to really map out the kind of, you know, what's the, what's the fine grain structure there.
[00:13:34] And so, I actually did a study on myself back
[00:13:38] Russ Altman: Yes, I wanted to ask you about that.
[00:13:39] Russ Poldrack: ... 2012.
[00:13:40] Russ Altman: That's very exciting.
[00:13:41] Russ Poldrack: Yeah, 2012, 2013, where, you know, I was, I was interested in this kind of resting state functional MRI approach, but we hadn't really done it in my lab. We had always taken that other kind of task-based approach. And I didn't really trust the data because we didn't have very much data from any individual. So, I basically said, one, I wanted to know just when you have enough data, what does it look like, and also how does it change over time?
[00:14:03] So, and, you know, I was director of an imaging center in Austin. I had a scanner that I could easily get into regularly, so I basically started scanning myself, generally twice a week when I was, you know, not traveling or something, every Tuesday and Thursday morning at 7:30 AM. I would get into the MRI scanner for 30 minutes, and we'd collect some data on me.
[00:14:21] Russ Altman: And what instructions did you give yourself about what to think about or what to do?
[00:14:26] Russ Poldrack: For the... we collected 10 minutes of data with me just sitting there resting and, you know, for the first few times it was, like, just me trying not to have a panic attack because I don't, I didn't really like getting into an MRI scanner, right? But then by, you know, by the end, I was, like, almost falling asleep.
[00:14:41] Russ Altman: Yeah.
[00:14:41] Russ Poldrack: It became almost like my daily meditation. But for those 10 minutes, I basically just sat there with my eyes closed and just thought about whatever I want to thought, think about. Then I usually also did some kind of other measurements, either task measurements or other types of MRI measurements. And then actually every Tuesday I would also go get a blood draw, to do some various, like, you know, biological analyses.
[00:15:04] Russ Altman: Well, we salute you for your dedication to the cause.
[00:15:07] Russ Poldrack: Thanks.
[00:15:08] Russ Altman: So, what did you, so what do you learn from these, from these experiments? It's, it's a, a hugely... Yeah, right, I mean, you are able, and I, I assume that this is all ethical and that somebody approved this even doing to yourself. What kind of things do you learn about the resting state, and how does that relate
[00:15:23] Russ Poldrack: Yeah
[00:15:24] Russ Altman: ... what somebody else might yield if you had me do it twice a week for, for, for half an hour?
[00:15:29] Russ Poldrack: Yeah. I'll just mention there's a bunch of interesting ethics questions about self-experimentation. At, at University of Texas, they basically said, "We don't consider this research. You don't... We will not even consider it as a, as a, as an IRB protocol. You just go do it yourself."
[00:15:44] Russ Altman: Interesting.
[00:15:45] Russ Poldrack: Other people, differ.
[00:15:47] Russ Altman: Yes.
[00:15:47] Russ Poldrack: But, okay, so what did we learn? the first thing we learned is that when you have enough data, you can very reliably characterize not just these large scale networks, but, you can do what we call parcellation, where you find like small patches of the, of the cerebral cortex that are kind of talking to each other more intensely, by, by which I mean they're correlated with each other, more than they are to other parts of the brain.
[00:16:13] And so. When we when we took those existing methods, and I ended up collaborating with a group at, at Washington University in St. Louis who had been developing these methods. When we took their methods, we found about 600 of those like little maybe centimeter to two-centimeter patches across my brain. And when we took about, a few hours of data and did it, and then took a different few hours of data and did it again, we saw very similar results. And so, that told us that these things are reliable.
[00:16:41] Russ Altman: Now, are these patches in the same network that you told us about before, or are they talking to other networks?
[00:16:47] Russ Poldrack: They're, they're mostly... So, you can think about it like each network is kind of broken into these little parcels.
[00:16:53] Russ Altman: Yeah.
[00:16:53] Russ Poldrack: But different ones of them have different degrees of communication, and there's all these sort of like network theory models about hubs and participation and all this sort of stuff you can use to, to try to understand that. And we, we did some of that work.
[00:17:06] The really interesting thing was that, so, you know, we got all this data on me. We, we did the parcellation, and for each of the individual parcels you can ask, like what, which network is it talking to most strongly?
[00:17:19] Russ Altman: Yes. Yes.
[00:17:20] Russ Poldrack: And most of them kind of looked like what you would expect, but some of them were in weird places. So, like in, you know, we, we traditionally expected the, the middle of the prefrontal cortex is general, was generally thought of as, being part of this thing we call the default mode network. Which is a network that's mostly active when you're just sitting there not doing anything. It actually turns down when you start doing a cognitive task.
[00:17:43] Russ Altman: It's like an idle.
[00:17:44] Russ Poldrack: Exactly. Right. Yeah. And within that part of the brain, there was a little spot that was connected to the salience network that I mentioned. And nobody had seen that before because they were taking a small amount of data from a bunch of people and averaging them together. And, so we, we actually didn't know is, do I, does Russ Poldrack just have a weird brain or is everyone like this?
[00:18:03] So a group at Wash U led by Niko Dosenbach went and did a study they called the Midnight Scan Club 'cause they were getting in the scanner at midnight because the time was cheap. So, 10 of them scanned themselves a bunch and found that it turns out that everybody has these little function- we call them functional variants now.
[00:18:20] And but they're in different places. So, when you average a bunch of people together, they kind of wash out.
[00:18:24] Russ Altman: Okay, so that does answer my question about whether everybody's the same. The answer is that maybe at a high level, as you said, the network functionality is there, but at a lower level, exactly how your brain implements that communication can vary. And do we know, is that a function of genetics or how you were raised, or who knows?
[00:18:44] Russ Poldrack: We, we don't know. And we, that's a really important question that I think we'd love to answer. I mean, there's some data showing that these functional variants are heritable, so there's presumably there's some part of the, of the genetic plan that relates to, to wiring.
[00:18:55] Russ Altman: See if you can get your father and mother into an MRI for a couple times a week, right?
[00:18:59] Russ Poldrack: Right, I know, yeah. But the, the one really interesting thing is that, and we just learned this in the last couple of years, there was a, a paper a couple years ago that showed, that basically did the same thing on people who'd been diagnosed with depression. And imaged them a bunch and saw that they had more of the brain associated with the salience network.
[00:19:27] Like, substantially more.
[00:19:28] Russ Altman: Oh.
[00:19:29] Russ Poldrack: It's a small sample, but they used some other data sets to try to, to try to validate it. and you know, I was a little bit skeptical about it. But we, working with Carolyn Rodriguez in the psychiatry department at Stanford, did a study where we brought in people, who have obsessive compulsive disorder symptoms, scanned them 10 times over a few months, and saw that they show the same phenomena. They show this expand- like, pretty marked expansion of the salience network.
[00:19:58] Russ Altman: So, this is potentially very profound because now you're hitting upon, people who have perhaps even diagnosed, diagnosed, psychiatric or other brain diseases, and now you can maybe. Of course, the implication is maybe this is a diagnostic tool, but it also could be used to track the efficiency of treatments. Like, are the treatments working? Is the network behaving more the way we, we might expect? Is that, is that a direction? have I imputed too much, or is that a direction that we're going in?
[00:20:27] Russ Poldrack: Well, it's interesting because it's actually, like, the, the, the, that Nature paper from 2024 showed that it, that the size of the salience network didn't actually track with symptoms. Or with, with, like, you know, with remission. So, it almost seems more like a trait-
[00:20:42] Russ Altman: Okay
[00:20:43] Russ Poldrack: ... that might, like, you know, kind of, give one a, a higher, probability of having depression. But what, the, the, I think the greater excitement is about, you know, we know that most people who come in and get treated for depression, the treatments work about a third of the time, right?
[00:20:59] Russ Altman: Yeah.
[00:20:59] Russ Poldrack: The first time. They have to try a bunch of different treatments, and some people, you know, really never find a treatment that works for them. The question is, could we use something, like these data to, to do a better job of predicting what particular treatment is going to be useful, and then potentially, if it's like a brain stimulation treatment, to target that?
[00:21:17] Russ Altman: This is Russ Altman from The Future of Everything. I'm speaking with Russ Poldrack from Stanford University. We've been talking about the basics of functional MRI and brain imaging, how it works and what you can learn.
[00:21:27] And, and I wanted to move to a discussion of reproducibility and open science, but before that, I wanna remind you that at the end of my conversation with Russ, we'll pause and we'll do Future in a Minute, where I'll ask him some rapid-fire questions and he'll give me some rapid-fire answers.
[00:21:42] So Russ, in the area of brain imaging, there's been a lot of discussion in, even in the lay press, and certainly in the scientific press, about reproducibility of the results. And you've just described what sounded like very careful studies that you did both on yourself and on participants. What is the reproducibility issue and, and how is the field responding to it?
[00:22:02] Russ Poldrack: Yeah. So, you know, and brain imaging isn't unique-
[00:22:04] Russ Altman: Right
[00:22:04] Russ Poldrack: ... in this fact. I think a lot of scientific fields have been struggling with this. You know, what we noticed over the last, couple of decades was that, you know, it wasn't always the case that when somebody reported a result, that that result could be reproduced by someone else.
[00:22:19] And this was especially true in studies looking at, you know, differences between groups like, you know, depressed versus healthy individuals, or at kind of correlations with, you know personality features, those sorts of things.
[00:22:32] Russ Altman: Although you did tell me in the first segment about you did your study on yourself, and then you had your 10 colleagues who were able to reproduce it. So, I just wanted to throw in that it's not always the case that you have trouble.
[00:22:43] Russ Poldrack: That's true.
[00:22:43] Russ Altman: Go ahead. Please continue.
[00:22:44] Russ Poldrack: Yeah. Yeah, and I'll, and I'll just point out that that the move to collecting so much more data, I think is what got us there. Yeah. So, so I think, you know, we, a lot of us, you know, I was writing papers, you know, 10 years ago about the, the challenges of, you know, what we needed to do to make the result, the research more reproducible.
[00:23:04] And we, we felt like there were two moves that one could make. One is, you know, much larger samples, just as in, for example, genetics. You know, people used to look at genetic-
[00:23:13] Russ Altman: Right
[00:23:13] Russ Poldrack: ... variants and their effects in 20 people. Now they look at it in 100,000 people, right? Because they knew that the small studies were not reproducible. Similarly, there's been a move in brain imaging to work with much larger data sets that have helped try to, to make the results, more robust and more reproducible.
[00:23:30] Russ Altman: So just to clarify, is that more participants or more data on each participant or both?
[00:23:36] Russ Poldrack: Yeah. So, it, it so in, we would love it to be both. It's hard to do both.
[00:23:42] So, for example, there's a study called the Adolescent Brain Cognitive Development Study that's imaging about 10,000 kids repeatedly over time. And they collect, you know, about an hour of brain imaging data per kid per session, and that's just because you, you couldn't get kids to come back that often. But you have thousands of kids, so you can at least start to pull out some, especially, you know, things that are looking at correlations with, you know, various cognitive features across people.
[00:24:12] So there, you can either kind of go broad like that and just have large samples and, and kind of use the, the power of large numbers to wash out, you know, the noise of the tiny samples, or you can go deep. And that's, you know, there's a, in, since my study, basically, there's been a move within the brain imaging literature to, to go much deeper, and it's, it's really led to a number of like, you know, really incredible new findings. I mentioned the one about depression. There's others about, like, kind of new aspects of brain structure that we didn't know about before. But those are the two directions that the field has gone to try to address this.
[00:24:51] There's still another issue, which is, you know, when people get data, they often analyze it in very different ways.
[00:24:57] Russ Altman: Right, right.
[00:24:58] Russ Poldrack: And we did a study, published back in 2020 that tried to look at how much this matters. So, we collected a data set, a brain imaging data set, gave it out to 70 different groups of researchers, and basically had them each answer a set of questions, like a hypothesis test, is there brain activity in area X for this aspect of the task that the person was doing? And then we looked at how consistent their results were, and it turned out that they were not very consistent at all.
[00:25:22] For a lot of the hypotheses, about a third of the teams said yes, the rest of them said no.
[00:25:27] Russ Altman: And these were all pretty-
[00:25:28] Russ Poldrack: There were very few in
[00:25:28] Russ Altman: You know, these were all credentialed colleagues.
[00:25:31] Russ Poldrack: These were all credentialed people who had
[00:25:33] Russ Altman: Right
[00:25:33] Russ Poldrack: ... brain research before. They were, they worked in that particular domain.
[00:25:35] Russ Altman: And they were trying hard. They were trying hard.
[00:25:38] Russ Poldrack: And were trying hard, yeah, yeah, yeah.
[00:25:40] And so, this highlighted the degree to which, you know, analytic variability, what we, that's what we call it, has a big impact. And there's now work in other areas as well that have shown this.
[00:25:51] So, one of the things that we've done is, is kind of try to lean into that and not say we need to figure out the one best way to analyze the data.
[00:25:57] Russ Altman: Yeah.
[00:25:58] Russ Poldrack: But rather we need to look at the data across a, a range of plausible analyses and say how which results are consistent, which aren't, and what are the features of the analysis that might drive that variability? And so that's a direction we've been going.
[00:26:12] Russ Altman: That's really interesting because I really can see the challenges there. You know, there might be an urge to say everybody should do this the same, so that we're all kind of in line. But that could introduce a huge systematic bias in in that you always miss certain things that that particular analytic method might not be good at picking up. So, so you're describing a world where you, you don't tell your colleagues that we all have to do it the same.
[00:26:35] So just take me through what you tell them instead.
[00:26:39] Russ Poldrack: I tell them, you know, you, you could do the way, the analysis any way you want, but then you need to look at how sensitive the results are to changing various features of that analysis.
[00:26:48] Russ Altman: Okay. Okay. And what are we finding? Is that leading to...
[00:26:51] So we, we, we started the conversation with reproducibility, and then you gave this kind of sad story where it didn't look like it was reproducible because very credentialed workers in the field came up with diametrically opposed answers. So, how has this now played out in terms of, I guess, more reproducibility?
[00:27:10] Russ Poldrack: Well, it's, it has certainly shown us that in, that, you know, so for example, we have a paper we published a couple years ago now where we, we had looked across the literature at how people had built their statistical analyses of a particular task that's used in functional MRI, and we came up with 240 plausible analyses.
[00:27:28] Russ Altman: Wow.
[00:27:28] Russ Poldrack: All of which had been, at least different parts of which had been used by somebody in the literature. And if you looked across those in the brain area that people care about, you got anything from a huge positive effect to a medium-sized negative effect from the same data. And we saw this across multiple data sets.
[00:27:44] So it just highlights the fact that, you know, that we need to, this is an emerging thing that. We call it multiverse analysis 'cause you're looking across the analytic multiverse.
[00:27:52] Russ Altman: Yes.
[00:27:53] Russ Poldrack: And I think groups are just starting to grapple with this.
[00:27:55] Russ Altman: And I know that you've also been active. This is part of, this, part of this movement, and we haven't talked about it explicitly, is this idea of open science.
[00:28:02] So I, if I understand correctly, with this, realization and challenge of reproducibility, not only have people, of course been very transparent about their methods, but also, I, it, I, my understanding is that data sharing has kind of gone up as a, as a way to kind of, increase the transparency of what the methods can and cannot do.
[00:28:26] Russ Poldrack: Yeah, that's right. I think our field has been actually really good in both doing the hard work of coming up with the infrastructure to share data well, and also the kind of the social engineering-
[00:28:37] Russ Altman: Right
[00:28:37] Russ Poldrack: ... Of convincing, of people deciding that they really do wanna share their data. Now it, you know, I think it's, it's not everybody shares their data, but it's become kind of a standard expectation in brain imaging that you will share your data.
[00:28:48] Russ Altman: I saw that you wrote
[00:28:49] Russ Poldrack: and so, you
[00:28:50] Russ Altman: I'm sorry to interrupt. I saw that you wrote a paper on how to respond to a request for data sharing. 'Cause I can imagine you're thinking, "Well, I still wanna analyze this data, and I still think there's some gold in there, and so I might not wanna quite share with you yet, although I do, I am committed to data sharing." so that was a fun, it was a fun article.
[00:29:08] Russ Poldrack: Yeah, yeah, cool. Yeah, there's, there's various models for data sharing. Like we run a project called OpenNeuro, which now has, almost 90,000 subjects worth of data, most of them humans, a few rats and mice and stuff. but those data are shared completely openly.
[00:29:24] They've been, you know, we remove-
[00:29:26] Russ Altman: Yeah
[00:29:26] Russ Poldrack: ... facial features so they're de-identified. But but those data are all available basically to anybody, and, and they've been used in a bunch of different ways. Now, that required a lot of infrastructure work.
[00:29:40] In particular, we, we started back in 2015, this thing that is called the Brain Imaging Data Structure, which it, it sounds like the most boring thing on earth because it's basically telling people how to name their files and how to organize their folders in the dataset. But it's turned out to be really amazing because, one, you know, people can upload their data to OpenNeuro, and it looks to see whether their data meet the standard.
[00:30:02] Russ Altman: Right.
[00:30:02] Russ Poldrack: If it does, there's no, no more curation is necessary. The data can go in and be shared, and then people can download the data and immediately know how to work with it.
[00:30:10] Russ Altman: Yes. I mean, in my field, we had a time when people had to share data, but they didn't want to, so what they would send is an Excel file where none of the rows or columns were labeled. So, it was the data, but you had no idea what, what was what because they left out the columns, and it was almost like an act of, of insubordination.
[00:30:29] So in the... We only have about a minute left, but I I did wanna ask about this removing facial features. So, when you do a study, you create a certain trust with the participants, and you, they, they sign waivers, but also, they look into your eyes in many cases, and they trust that you're gonna do good science and you're not gonna put their personal data at risk.
[00:30:47] In the setting of data sharing, how do you manage the, the risk of, private information, especially when it's about somebody's brain and what they're thinking? How do, how do you manage that, I guess, in a nutshell?
[00:31:00] Russ Poldrack: Yeah. I mean, well, the first thing we do is we are very open in the consent with people about the way in which we're gonna share the data.
[00:31:06] We say that, you know, and, and open about the fact that we can't guarantee that they won't be re-identified, right? Now most of the studies that people are participating in, you know, just standard cognitive study, a healthy person off the street, even if their data were re-identified, the risk to them is not that high.
[00:31:21] Now, if it's a person with a mental disorder, a diagnosis, something like that, or, you know, if you have so much data on a person where you start to think, you know, we might be able to see. You know, I, I don't, you know, I've written a lot about how we don't really believe you can, like, read the contents off of a person's mind right now with the methods that we have, right?
[00:31:38] Russ Altman: Right.
[00:31:38] Russ Poldrack: You can, you can get some information. You can certainly read off, like, what kinds of things they're thinking about. But in terms of, like, the, the the train of thought inside their
[00:31:44] Russ Altman: Right,
[00:31:44] or lie detection
[00:31:46] Russ Poldrack: .... you can't get. Right, right, exactly. That kind of stuff just doesn't work right now.
[00:31:51] But we try to be very clear to them that, you know, that we're gonna, we're gonna do everything we can to protect them, but we can't guarantee it.
[00:31:56] And then when we start to think about these data sets where there's more risk to the subject, that's where we start thinking about the need for additional protections beyond just removing the face and the name and all that sort of stuff.
[00:32:06] Russ Altman: Great. Well, well, that's great, and that, that is a great place to stop because we want people to participate in these studies, and it's incredibly important for them to know that their data is protected and they really are making a huge contribution to science.
[00:32:17] Before we finish up, I wanted to ask if you're ready for our feature we call The Future in a Minute.
[00:32:24] Russ Poldrack: I think I'm as ready as I can be.
[00:32:25] Russ Altman: Okay, here we go. What is one thing that gives you the most hope about the future?
[00:32:31] Russ Poldrack: I think it’s; it's the fact that despite all the headwinds right now, there are these amazing students who really wanna come dedicate their lives to science.
[00:32:38] Russ Altman: What's one thing you want people to walk away from this episode remembering?
[00:32:43] Russ Poldrack: That as scientists, we take criticism seriously, and we actually try to use it to make our work better, and that's what we've tried to do in making, you know, neuroimaging research more reproducible.
[00:32:51] Russ Altman: Aside from money, what is one thing you need to succeed in your research?
[00:32:56] Russ Poldrack: Time to think, which is increasingly hard given the AI acceleration of everything.
[00:33:00] Russ Altman: We gotta get you back in that scanner. Okay. If all goes well, what does the future look like?
[00:33:08] Russ Poldrack: Scientists figure out how to use AI rigorously to make science better while still retaining the human element of discovery.
[00:33:14] Russ Altman: And if you were starting over again and you had to get your degree or certification in a different discipline, what would it be?
[00:33:20] Russ Poldrack: I think it'd be applied math.
[00:33:22] Russ Altman: Thanks to Russ Poldrack. That was 'The Future of Neuroimaging'. Thank you for listening to this episode. Don't forget we have a back catalog with more than 300 episodes, so you can listen to the future of just about anything for as long as you want.
[00:33:35] If you like what you hear, please follow the show and press subscribe. That'll ensure that you never miss an episode and you're always clued in to the future of everything.
[00:33:44] You can connect with me on many social media outlets such as LinkedIn, Threads, Bluesky, and Mastodon, where I'm @RBAltman or @RussBAltman. Also, you can follow the Stanford School of Engineering @StanfordSchoolOfEngineering, or more simply, @StanfordENG.
