The future of networking technology
Computer scientist Keith Winstein is an expert in how computers communicate.
Computer networks create what he calls shared fictions – abstract realities, like a website or a Zoom call, that exist only because the computers on either end agree to act as if they are real. Unfortunately, today’s networks lack a shared notion of a “computation,” which hurts market efficiency in cloud computing and frustrates efforts to hold tech companies accountable for the results of their algorithms. As computational power becomes concentrated in a smaller number of companies, Winstein advocates for a shared language of “computational truths,” defining computations precisely so results are reproducible and auditable. His research group hopes this will lead to greater transparency and accountability in the cloud and, ultimately, to greater confidence in the computations that companies do every day on our behalf. The truth matters, Winstein tells host Russ Altman on this episode of Stanford Engineering’s The Future of Everything podcast.
Transcript
[00:00:00] Russ Altman: This is Stanford's The Future of Everything, and I'm your host Russ Altman. Since we started this podcast eight years ago, it's become an archive of the amazing and impactful work done by my colleagues at Stanford University. In a time when the sheer volume of information available to us can make our head spin and make it hard to determine what's accurate, I'm proud to be able to bring you experts in law, medicine engineering, technology, and much more.
[00:00:27] Keith Winstein: The part that I think of as like so central to computer science is the playfulness of it. I mean, computer science is about reasoning about the consequences of procedural reasoning and computation, and the fact that you can design an automaton in your head and then see it play out at, you know, billions of times per second. I mean, that's fascinating.
[00:00:49] Russ Altman: This is Stanford Engineering's The Future of Everything, and I'm your host, Russ Altman. If you're enjoying the show, please tell your friends and family about it. Word of mouth is a great way to spread news about the podcast. Today, Keith Winstein will tell us that there's a great opportunity in computer networking to create systems that can track computational truth. It's the future of networking. Today we're continuing our segment called the Future In a Minute, I'll ask Keith a few questions, rapid-fire style, and he'll give us some rapid-fire answers. Also, remember to tell your friends and family about the show to spread news about The Future of Everything.
[00:01:35] Networks and networking are key to our life and are key to the information technology revolution. We use networks to watch our favorite videos, our television shows, our movies. We communicate with our friends, family, and coworkers. And upon the network, are built stacks and stacks and stacks of applications meant to facilitate our modern life. However, that networking technology is not perfect. It can have many, many middle men which lead to delays and suboptimal performance. These delays sometimes are not a problem, but sometimes they make things just impossible to do.
[00:02:13] Have you ever tried to sing a song with somebody on a video chat? It just doesn't work. Well, for the most part. Keith Winstein is a professor of computer science and electrical engineering at Stanford University. He's a networking expert and he's built systems where you can perform music together. And he's now teaching a class where music and the production of music is a core feature of the entire class. He'll also tell us about his ideas about what is needed in networking next. He's developed this idea of computational truth, and he thinks that our network systems of the future can embody and embrace an idea of provenance, truth, and auditability of all computation.
[00:02:56] Keith, to start off with, why have you focused your research on networking technology?
[00:03:01] Keith Winstein: It's fun. I mean, I think, um, you know, there was a period of computing in the 1800s when it was about data tabulation and the census and these kinds of things. And in, you know, World War II when it was about breaking codes and maybe there's a theory about, you know, statistics distinguishing Nazi aircraft from not Nazi aircraft. And then in the 50s and 60s, you know, sort of the mainframe era. Um, and then the personal computer era. But you know, this ability to connect my thinking machine to your thinking machine and have them talk to each other and invent a world that only exists in the mind of the computer is a, is a amazing thing.
[00:03:42] Um, you know, the things that we create via networking, via computers, talking to other computers are fictions. Um, whether it's a, you know, a, a meeting like we're having now, like this Zoom meeting, or I guess it's a, this, this podcast, it only exists in, in my computer and your computer. Um, it's a fiction created by the fact that our two computers are communicating and we, the humans buy into that. And, you know, games work this way and almost everything on the internet works this way, what we call TCP connections. The very fact that there's a stream of bytes coming from one computer to another, you know, that's not real. That's a fiction in, in the minds of the computers. And the fact that you can synthesize this sort of shared fiction out of multiple thinking automata talking to each other is, is a beautiful thing.
[00:04:22] Russ Altman: Great. Thank you. That actually, uh, is a wonderful answer and leads me to kind of the next, I think, natural question is we are so dependent on this technology. What are the main challenges that, that folks like you who are at the frontier, like what, what do you, it's fun and what do you worry about in terms of its capabilities and where it needs to go?
[00:04:44] Keith Winstein: Well, I think any serious answer to that would focus on the meaning of what we communicate over the network, which is not exactly my department. Uh, maybe it should be, but you know, I mean the, the networking technology, you know, you know, created among other things, you know, the internet and WIFI and cellular networks and, and these kinds of things. Um, and you know, on top of that, someone else built the worldwide web, you know, someone from a research program from Switzerland. And on top of that, people built things like Facebook, and on top of that, people built, you know, possibly the end of human self-governance. Um, I think if you're gonna say What worries me, you know, it would be that. But, you know, I think we have to be humble that, um, you know, anyone is qualified to opine on these things, but I'm not, I'm not an expert on those things. But, you know, if that's what troubles me or what I think needs to change, you know, that, that would be the top.
[00:05:35] Russ Altman: So how about from a technical perspective? 'Cause that, what, your answer is very fair. Like we, the, the use of networking raises issues and what, I hope we'll get to them later on, that are quite profound. But in terms of you waking up and saying, okay, I'm a professor of computer science and I, I need to do some research, how do you pick the problems and why have you picked the problems that you're currently working on, as the ones that are like best matched to what you can make a contribution?
[00:05:59] Keith Winstein: I'll give you two that you might find interesting. We can go down the, the road. I mean, one that we're spending a lot of time on is trying to formalize and communicate something that we have sort of implicitly now, which is this idea of computational truth. If you think about what your computer does and what we communicate over the network, we are very good at expressing compositional relationships between our data. You know, you have on your hard drive, I don't know, you use a Mac or a PC or something else. What do you use?
[00:06:26] Russ Altman: I use a Mac.
[00:06:27] Keith Winstein: Okay, so you have folders, you know, and inside those folders are other folders, and inside that you have files. And you know, this is a model popularized, I think, I think here at Stanford, actually, I think in the mother of all demos they had that, I think they may have had a hierarchical file system in the 60s. Um, I don't know the history on that, but you know, we have this notion of the compositional relationships. And I don't know about you, but you know, when I'm writing a paper, something like that, I have all these hierarchies and you know. I have a graph and, you know, the graph came from some Python script that read some data, and the data came from another Python script in another folder. I mean, you probably have the same thing.
[00:06:58] Russ Altman: Yes.
[00:06:59] Keith Winstein: And we are very good at communicating these compositional relationships among our data over the network. There's a thing called Git that everyone now uses that was created by the same guy that made, you know, Linux, the operating system that took over. This is a Finnish college student. He made his own, you know, operating system Kernel. And it's now, you know, taken over all the, the commercial, taken away all the commercial, um, competition. And then the same guy, you know, 10 years later made this thing called Git, which is a very, you know, a formal way of representing these compositional relationships. And now there's GitHub and we can share these arrangements of our data.
[00:07:30] But we, and our computers and our networks are terrible at remembering the computational relationships between our data. That's all just implicit. And so, I dunno if you've had this experience, but like someone comes to you 6 months later and says, Hey, there's a graph in your paper, you know, I'd like to reproduce it. Or, you know, I think this data point is wrong. I'd like to recompute the graph with this one thing changed. This is really harder than it needs to be because when I go back and I look at that directory or that folder, you know, I see the graph, but then I don't know which Python file reading, which data file made that thing, that computational relationship which was manifest at the time it was made is just forgotten.
[00:08:06] Russ Altman: Yes.
[00:08:06] Keith Winstein: And so it would be really nice if those kinds of relationships were something the computers kept track of and communicated and guaranteed and could communicate to other people over the network. So I would love if you clone my Git repository, do you guys do that in your?
[00:08:20] Russ Altman: Yes, absolutely. It's, it is actually a requirement of most of our funding agencies that we share the code that upon which our findings are based.
[00:08:28] Keith Winstein: Great. So if you clone my Git repository, I don't want you to just see the sort of files and directories sitting in folders with these composition relationships. I want you to see the computational truths. I want you to see that this graph is what happens when you run this Python script in this environment, and then you, you click on the data that it processes. Where did this come from? Well, this is what happens when you run this other script in this environment. Maybe click on Python. Where did that come from? Well, that is what happens when you run the C compiler on the Python source code.
[00:08:54] That's interesting. Maybe you click on the C compiler. Where did that come from? Well, that's the most interesting. That's what happens when you run the C compiler on the C compiler source code. It's kind of like a circular relationship. And the fact that we can't speak about these computational relationships formally or communicate them over the network really, really hampers the way that we use the network and the way that we outsource computing across the network.
[00:09:18] Russ Altman: Is it fair to say this is a discussion of provenance data, and not just the, the provenance of any computational artifact? Like where did it come from and can you give me kind of a guaranteed, um, ancestry of where this computational artifact, uh, sprung from, uh, both for the reasons of replicability, which you just referred to, but, and I think, I think you're being more profound than, than that. It's not just for replicating, it's for, um, making sure that we're not building a house of cards of like, uh, fragile systems.
[00:09:50] Keith Winstein: Well, I think it's definitely both of those things, but I'll give you a third one, which it, I think, impoverishes the language of what we do when we compute, and especially when we ask other people to compute. So if you think about systems like, you know, Amazon Web Services or Google Cloud, or, uh, Microsoft Azure or Alibaba, you know, these companies are in the business of renting out computers by the millisecond. And so the way it works, like let's say on Amazon, you know, you upload a function for them and that's like a zip file of, of some, some code. And later that function gets invoked and someone says, I'd like to run this function with this arguments, you know, maybe a URL of a file it should process. So Amazon's job is to find some computer and their vast, they have millions of computers, find some computer or some slice of a computer that's free.
[00:10:33] Start running your code and then all they start doing is start running like a meter, like how many milliseconds was the code there until it quit? And so probably your code, the first thing it does probably is fetch the data. Like you, you know, you generally need data to do something. So probably it looks at that invocation arguments and goes back to Amazon's, you know that they have a storage system called S3. They say, well, I would like this file. So now your code is stuck there waiting for Amazon. They've got this S3 node that says, well, maybe I have it locally. That's called cached. You know, maybe I have it locally, in which case, here's your file. And then, you know, you do the computation and you finish quickly.
[00:11:07] But maybe Amazon's local S3 node doesn't have the file. Maybe they don't have a very good caching system, or maybe they just don't have it, or you know, so now you're stuck there waiting, or your code is stuck there waiting, while the meter's running. But they are the ones, you know, the balls in their court. They have to go search for that file somewhere in S3 in that region, eventually give it to you. Then you do the processing, then you're off the machine, and the bill is higher. So this is a weird system where Amazon makes more money if your code takes longer and the code takes longer because they had a cache miss, because they didn't give you the file quickly.
[00:11:40] This is what we call pay for effort. It's kind of like a, I don't know, you and I both live in the Bay Area, so maybe this is a fantasy for both of us, but if we had a big house with a big lawn, hypothetically, and we hired someone to mow that lawn, imagine if you did a deal where you paid them per hour. You know, say, yeah, please mow my lawn and I'll pay you for every hour you work. This is not a good way to run an economy. Because, uh, first, you know, you're, you're encouraging them to take longer, but also there, there's no innovation surplus that you, you want the person mowing your lawn to be incentivized to invent a better lawnmower, you know?
[00:12:15] But to do that, you need to have a different way of thinking about the service being provided. You need to have maybe something close to what we call pay for results. You know, maybe with a lawnmower you kind of have this, you have some understanding what does it mean to have my lawn mowed? And then however you want to do it, that's great. If you can invent a better lawnmower that mows it, you know, 40% faster than maybe, you know, maybe you give me a 20% discount, you keep the other 20% for yourself. Everyone wins. This kind of the theory of innovation, the theory of markets. Uh, but to do that, you have to be able to define the result of a computation or the result of the job.
[00:12:47] And so we have to be able to talk about a, a different abstraction. Computation can't just be renting time on someone else's computer. That's like not a very interesting transformation. You know, Amazon and these companies, they buy the computer and they rent it out per minute. I mean, that's what landlords do. This is not a economically productive transformation. But if we could agree on what it means to have the result of a computation, the actual computational truth, to have a claim where, you know, Amazon or Google or wherever could say, Hey, you asked me to do computation. You know, F applied to data X and we did it, and we're willing to tell you that.
[00:13:20] Like, yes, F of X produces Y, and uh, we'll stand by this result. Like, that's the thing you're paying us for is this object, the computational truth. We'll stand behind this. In fact, if it turns out we made a mistake, you know, you can. Um, take it to State Farm Insurance and they'll, they'll check to see if we're right or you're right. And if it turns out we made a mistake, you know, State Farm will pay you $10 million in sort of errors in emissions insurance, then you could, you would commoditize this thing of computing and describing a computation over the network. And I think we would live in a very different world.
[00:13:51] Russ Altman: Okay. So that's very interesting. I think I do understand what you're saying. Um, but at the same time, you also, a few minutes ago, described this incredibly deep stack, right? You said we start with a computation, and then there's a code and there's a compiler, and there's a, there's a, there's a, you know, code that's reading the compiler, the compiler's executing, and then it, it spits out. So there, there's a gonna be a, very deep network, as you know very well, I used the word, uh, of, of relationships. Um, and so obviously what I think people listening to your example said, well, I, I totally see how you, you said the clock is ticking and in some cases you're gonna get a bargain because the, the, the data's available, the, the CPU is available or whatever, they do the calculation and it's a super short time and you get a good deal. And other times, just because of the way they've architected their system, you're paying a lot more for that same exact result. And that does seem unfair, and I would love to right that wrong, so to speak. Uh, do we have the technical capability to, to do what you're describing and I'm, I'm sure this is what you're thinking about. So what's the first step in realizing such a system?
[00:14:58] Keith Winstein: Uh, the first step is probably coming up with an unambiguous definition of a computational truth. To say, we ran this program or this piece of code in this environment, and the result has to be a deterministic function of that environment and that piece of code. And anyone who looks at that and does it will get the same answer. So, you know, in mathematics we sort of have this like in, you know, you can say one plus three, like, everyone who's been educated the same way will agree on what that means. It's not one plus three. Oh, it's one thing because of the weather. You know, if it's sunny, it's one thing, if it's rainy, another thing. So just defining that abstraction.
[00:15:30] What does it mean? What is the meaning of a computation in a way that is a shared language? That's the probably the first step. And I would say even being able to name that computation is incredibly powerful. We are moving towards a world where the vast majority of the computational power is owned by other people. Uh, you, you've been around, you know, you remember AI, you know, 60 and 50 and 40 and 30 years ago. It, we had it, right? It ran, we had it.
[00:15:59] Russ Altman: Yes, it was in my computer.
[00:16:01] Keith Winstein: And now, you know, I have never used a top quality LLM, right? And neither have you. We have used what we call an RPC. A service over the internet where you communicate a query.
[00:16:13] Russ Altman: And what does RPC stand for?
[00:16:15] Keith Winstein: Oh, sorry. A remote procedure call or a, a network service. You know, they are running the LLM and they're controlling the random seed. They're controlling all kinds of things. You know, one of these companies. We just get to use a service that they provide. Uh, you know, Google search was like this, you know, what, 20, gosh, almost 30 years ago. Uh, we become dependent on these network services. And so even being able to name the computation that someone else is doing for me is incredibly powerful.
[00:16:44] I'll give you some examples like, um, you know, Google runs this ad auction, you know, that's how they make almost all their money. Um, and for a long time they were very proud of saying it was an incentive compatible second price auction. So, you know, if I bid a hundred dollars for the keyword, you know, Beanie Babies, and you bid $80 and someone else bids $60, you know, I win the auction, but I only pay $80 because that's the, you know, in a single shot model that's the way that we're all encouraged to give our true values.
[00:17:12] Russ Altman: Right, right.
[00:17:13] Keith Winstein: Now, everyone knows it's possible to cheat if you're the auctioneer, you know, you can always pretend, oh, sorry, sir. Yes, you bid a hundred dollars and we had someone else, you know, who bid 99. Of course, I can't tell you who it was, but yeah, so you're gonna have to pay 99. Or if you're the auctioneer, you can always cheat. And allegedly, according to a whole bunch of lawsuits, Google did at some point succumb to that temptation and they did start cheating. And this is, you know, this is a spinoff from our department, Google, like it's a little bit painful when like your own, your own kin start, uh, doing the thing that it's not even clever. Everyone knows you can cheat if you're the auctioneer, but then to actually do it, it's kind of like, come on.
[00:17:46] Anyway, so if there were a way to name that piece of code that does the auction, then you could have something to refer to. You, you know, maybe when they give you the result, they could say, of course they're not gonna release the code for the auction. That's highly proprietary. But at least they could say, well, here it is under this opaque name, and you'll know when it changes. Or if you wanna sue later, you could say like, well, I think you fiddled with it. You know, I think these results were not honest, and at least you've given me the name for the thing, the computational truth that you did at the time. I want you to unwrap it now with my lawyers or everything. Just being able to name it is very powerful.
[00:18:19] Russ Altman: Right. So if I can make sure I understand, when you say a name is, it means there's a piece of code that's running, that's producing output. We're gonna give it, it's probably gonna be a long string of letters and numbers. It's gonna not be a name that any of us would think of as a name. But the key thing is it will be controlled, I don't know if it'll be blockchain or it'll be some very secure bookkeeping mechanism, uh, that will then allow you to say, uh, that's code that was run to produce my result.
[00:18:46] And now you say, and now there's an allegation that that code was not so, uh, honest. And if it, and then through the process of law and stuff, you could uncover that. You could know that you're looking at the actual code that was actually run and say, hey, you made up that 99% bid. The highest bid you got was really only 80%. You did something bad and now social mechanisms are gonna be applied to kind of right this wrong. That is not possible right now, um, is my presumption.
[00:19:13] Keith Winstein: Yes, you're right. I mean, there are all these lawsuits against Google and all the other companies for all their various alleged sins. And you know, I have a colleague at Harvard who's become like one of the leading expert witnesses against Silicon Valley. Um, and you know, he gets to see the super secret auction code in a highly locked down room in Washington DC. But, you know, you know, you have to trust in the legal system enforces.
[00:19:33] Russ Altman: And it's not routine.
[00:19:34] Keith Winstein: No, it's definitely not.
[00:19:34] Russ Altman: It's not routine and it's very expensive.
[00:19:36] Keith Winstein: But how do you know that that is what produced, you know, that that is the computation, right? We don't have a way to, you know, if we just talk about computation as like time on a computer, we're, we're lacking that language. So that's a thing that, you know, my group and I are spending a lot of time on.
[00:19:48] Russ Altman: This is The Future of Everything with Russ Altman. We'll have more with Keith Winstein next. Welcome back to The Future of Everything. This is Russ Altman. I'm speaking with Keith Winstein from Stanford University. In the last segment, Keith told us about his work and his recent focus on computational truth, the ability to track computations in great detail, to understand what they do, how they do it, and to have an audit trail that is pretty much perfect. In the next segment, we're gonna move to a different topic, which is how Keith and his colleagues have created ways for live performances, especially of music, and how that works. He'll also tell us about a new class where he is trying to introduce playfulness by having a xylophone, a child's xylophone, be the object of an entire class on computer science and electrical engineering. Don't forget, at the end of this segment, I will have the Future In a Minute segment with Keith where I'll ask him some quick questions and he'll give us, to the best of his ability, some quick answers.
[00:20:56] Keith, I know you've done some really interesting thing on performance over networks, and when I say performance, I mean things like music. And during the pandemic I tried to sing a song with somebody on Zoom. What a simple idea. It does not work or it did not work. And I, and, and it seemed to be a network problem. So can you tell me what have you done in this area?
[00:21:17] Keith Winstein: You know, a lot of the work that we do now is kind of, we've all become biologists. Um, you know, I used to be a medical reporter. I was at the Wall Street Journal and I, I, for, for three years, I wrote about science and medicine and I read every issue of the New England Journal of Medicine and I wrote about a lot of these medical studies and I don't envy that world. It's really hard, you know, to, to, to run an experiment like a, especially a medical experiment, you know, you have to collect the patients and it takes like years to, you know, get the effect size that you're expecting and everything.
[00:21:51] And at the end you're like using these extraordinarily sophisticated statistics to try and see like, you know, does Lipitor help or not help? And even those are the big studies. And somehow that, you know, the, the thing about computer science is that we are fueled by, uh, a motivation that, that comes from a confidence that the mystery is always solvable. You know, we're like Colombo, we, we're always gonna find it. Because we have incredible tools in computer science on the problems that we choose to address because, you know, for example, in computer science, any mystery you have, we can inspect any part of a computational system without screwing it up. You know, this is not a privilege the biologists have.
[00:22:37] You can't dissect the frog and like keep the frog alive. Um, and it's, you know, it's not a privilege that physicists have. You can't observe something, you know, and that, that it's quantum mechanics, but it is a privilege we have. Um, because of something called Turing completeness. You know, you can always emulate this as, this is a very deep result, you know, of Alan Turing, the founder of, of both computer science and arguably of AI, you know, that you can always inspect the system without screwing it up. That gives incredible tools and then incredible confidence, and we can always rerun the system as many times as we want while investigating it. You know, this is, again, not a privilege the astronomers have. You can't see an interesting supernova and say, oh, let's, you know, run it again, God.
[00:23:15] You know, that you don't get to reproduce it. So we have the most fun, I think, in this university, the computer scientists, because we have this incredible privilege of these, you know, tools on the problems that we address. And that gives incredible motivation. And that's like, I think why most of us got into this. And you all the statistics people are like, we're getting this virus. We're like, suddenly we're having to become, you know, biologists and astronomers where a lot of our work becomes more in essence statistical and slow, where you do something for a long time. You know, I did this, um, work with my students on video streaming where, you know, this is like you're watching Netflix and maybe it starts to rebuffer for a while and, uh, you know, can we make better algorithms for that?
[00:23:59] And, you know, so we have our own video streaming website here at Stanford, puffer.stanford.edu, where people watch television, and then we run various algorithms. And, you know, but to do that kinda work, you have to run it for a year and collect a lot of data and then analyze it and maybe it doesn't, and this is just not, it's not that fun. You know, it's not the playfulness and the interactivity and the confidence of being able to, to play with it and see the result that I think made me happy to be a computer scientist. And so, you know, we had done this work about sort of video and audio over the internet. And then during the pandemic I got contacted by this, uh, you know, connected with this colleague Michael Rau, former guest on this podcast, who's a genius theater person. I mean,
[00:24:38] Russ Altman: Yes. Super fun to talk to.
[00:24:40] Keith Winstein: I mean, the humanities faculty at, at, at any academic institution have to, there's like a million people applying for one job. So the privilege of being here is like your, you know, your colleagues are amazing and this guy is, is incredible. And, uh, so, you know, he's like, look, I'm trying to teach theater during the pandemic to these students who are trying to do a play over, over Zoom. And it's like horrible. You know, like the, there's the delay. They can't hear each other very well. They can't do overlapping dialogue. They can't sing, you know, in unison, as I guess as you discovered, you know? And he says, well, I've seen you've had research on this. Can we use your research? And I had to explain, it was sort of embarrassing that, you know, there's a thing we call research code, which is like code that is not really, it's only written so that you can write the paper.
[00:25:19] It's not like a code that really even works per se, you know, and or that anyone else would ever want to use. Uh, but you know, we decided, uh, with my doctoral students, Sadjad, uh, then doctoral student, Sadjad Fouladi, we decided to try and really do it with him and try and build like a bespoke system for actors and musicians to collaborate during the pandemic. So we taught this class in the fall of 2020 for undergrads and grad students about building new kinds of video conferencing systems. And then we actually did it, uh, Michael and I, we, it was the main stage production in I think March of 2021. We had five actors, Stanford students, and then we have these three, uh, New York musicians who were very famous people, but you know, during the pandemic they weren't working.
[00:25:59] So it was an incredible privilege that they were like willing to work with this academic research group. Um, so, you know, they did songs over our system and they, you know, it was a lot of fun to be back in this world where you're, you're, the mystery is inside a box and you know that you can solve it. And you know, a lot of the problems with the Zoom and that kind of thing are not for any interesting academic reason. It's just kind of like they have a business and they're trying to be on millions of people's computers. And you know, the Zoom guy, um, you know, he's a friend of our, of us. I mean, I, I've never met him, but he's, I think he's come to our networking class before I used to teach it. And you know, he's been very candid. He's like, look, we just didn't prioritize kind of software quality. You know, you may remember like Zoom. They were so bad.
[00:26:42] Russ Altman: Yeah. They had a moment. They had a moment. And it was an unexpected moment.
[00:26:45] Keith Winstein: Do you remember when they, they screwed up so badly that Apple had to push an emergency update to Mac OS to delete zoom from everyone's computer. Do you remember that?
[00:26:53] Russ Altman: Yeah. This, this rings a bell. Yes.
[00:26:55] Keith Winstein: So, you know, there are a lot of sort of plumbing reasons why the delay in these systems is much, much larger than it needs to be. And, and you know, to fix that, you really don't have to be a genius. You just kind of have to, um, be a plumber and like really get your hands dirty and like get, you know, find wherever the delays are coming from and squeeze that out. And so we did that in this stage cast system. And it was really nice. I mean, that the, you know, the musicians were telling us, you know, like, oh my God, like this is my craft, is, you know, performing with people and I haven't done it for, you know, at, by that point a year.
[00:27:23] And now I finally feel like I'm in the, I'm actually part of an ensemble. And the students, they asked us to turn on the system so they could have their party, you know, that was very rewarding. So that was a lot of fun to do and, and I learned that we're not really teaching our students how to do this kind of engineering. You know, a lot of it is kind of like you write some code, maybe it's gonna train some machine learning classifier and it runs for a while, and then eventually you look at it and then, I mean, I, you know, I have this in my own work. You look at it and you sort of squint and like, well, can we say that this worked? Or, you know. And that I think is just less fun. And so I've been trying to, um, oh, please,
[00:28:00] Russ Altman: Before you go on, I just, this is so interesting. I just wanna ask a couple of things. So what somebody might think, okay, if you wanted to have a live performance with musicians in New York, especially famous ones, it's a matter of getting a super duper fast fat wire that connects and it's very expensive, but you do it because like, it's this unbelievable demonstration. But my understanding is you didn't get a super duper fat dedicated wire. You were just using the same old internet that all of us use. And, and can you give us just a little bit more on like what was the magic idea that takes away what, what we refer to as latency? The, the delay from when I, when I sing a C and when somebody in New York sings a G so that we can make part of a C chord.
[00:28:44] Keith Winstein: Yeah. So, you know what you're talking about is the delay between when I sing something and you hear it and when you sing something and I hear it. And, and that is unrelated to the fatness of the wire. The fatness of the wire controls what we call throughput or data rate, or what people sometimes call bandwidth. I mean, everyone calls it bandwidth. That's not, anyway, throughput or data rate. So that's kind of like, if I'm at home plate, you know, and I'm throwing something to you, you know, if I throw a bowling ball, that's gonna, you know, that's a, that's a lot. And if I throw like a, you know, a baseball, yeah, they're, they're, you know, the bowling ball is like fatter, but you know, they're probably gonna get there at the same speed.
[00:29:18] Certainly if you like drop them, they're gonna get there at the same speed. So having a fatter pipe, more megabits per second does not change the delay. But the other thing you have to understand is that the network is not the problem here. Everyone benefits when you blame the network. If you are having a bad experience on YouTube or Zoom or whatever. If you say, oh man, you know, Verizon is terrible, or Xfinity is terrible, you have to understand, everyone wants you to think that way. Xfinity would love for you to think that way if it means you're gonna upgrade your Xfinity if you blame them. And Zoom or, or Chrome, any of these companies would be very happy for you not to blame them.
[00:29:57] Um, but in practice it's not really Xfinity's fault. Like the delay is probably the same. You know, the delay is mostly, for something like Xfinity, like a wired connection, you know, the delay is mostly just about the speed of light through the wires and that kind of thing. And so it doesn't, you know, and, and the speed of light through the wires, it's very fast. You know, like from New York to someone else in New York, you know, we're talking like, you know, it's like the same distance of if you and I work 15 feet apart in a room and just talking to each other. You know, that's the, that's the actual communications latency, but the, what you hear on Zoom is more like 10 times that. It's more like being like a, you know, almost half of a football field away. And the reason for that is, I don't know what you wanna say. It's, it's bureaucracy, you know? I mean,
[00:30:43] Russ Altman: So you, did you cut through, so did you cut through bureaucracy? I think there was elements, if I, I, 'cause I did read a little bit about it, a little bit of element of kind of predictiveness where you said I know enough about music and performance that I can kind of guess what the next notes may be. Was there an element of that?
[00:31:01] Keith Winstein: That's a follow on. We didn't do that with the musicians. I mean, the truth is that if you look at the way a system like Chrome and Google Meet or, or Zoom work, you know there's a Zoom piece of software and it's getting packets in from over the network and they contain audio. And then it puts them into what we call a buffer, because it, it doesn't wanna run out of audio to play. If it, if it hasn't, if it starts playing the audio as soon as the packet arrives, then that audio's gonna finish maybe before the next packet arrives. And that's gonna sound like the robot voice, the, you've heard that on Zoom.
[00:31:29] Russ Altman: Right.
[00:31:30] Keith Winstein: So they don't want that to happen, so they don't play the audio as soon as it arrives. They put it in a receiver side buffer. That adds a lot of delay. And then when it gets outta that buffer, it goes into something called the rubber band buffer, the thing that adjusts the time. 'Cause if they have too much data in the buffer, you know they wanna play it faster so that they can catch up. That's when you hear,
[00:31:49] Russ Altman: And we've all heard that too. Yes.
[00:31:51] Keith Winstein: So that's a, but, but that's a separate software component that they license from a different company and so it needs its own buffer. Then from there it goes into the operating system's play out buffer in the sound vice, you know, the operating system. And then from there it goes into the hardware device buffer. So you end up with everyone, you know, it's different companies making every part of this, and they all add delay and that's how you end up with 200 milliseconds. You know, that's like, it's like, you know, a significant fraction of a second. And so if you do it with a bunch of Stanford students and you just build it, you know, to, to do one thing and, and you know, not, we're not trying to have it work on Mac and Windows and, you know, we're trying to do one thing.
[00:32:29] Russ Altman: Right. Right, right,
[00:32:30] Keith Winstein: Um, then you just have one buffer and every piece of software works on that one buffer. So you can get, you know, you can get literally quarter millisecond, um, intrinsic latency if you just do it in a good engineering sense.
[00:32:41] Russ Altman: Thank you. That, that really does answer my question. So you, you literally, and you said it before, but I didn't know what you meant. You said we, we avoided all the middlemen. I think you said something like that and you, so you just built up a bespoke piece of software that went to the lowest level of the network or some very low level of the network where you had control and you could make a lot of these buffers unnecessary.
[00:33:00] Keith Winstein: Yeah, and we had one, one trick, which is maybe interesting, which is that, you know, Zoom is trying to pick one trade off between delay and quality. So, you know, if you watch something on Netflix, the delay is huge. You know, they filmed that, that TV show six months ago or whatever, and it's spent a minute in your television before you're actually playing it. But the audio is perfect, and if you're on Zoom, you know, it's a much smaller delay. The audio is a little bit glitchy, and on a cell phone, you know, it's maybe even more glitchy. So there is a trade off between how glitchy it's gonna be and what the delay is gonna be. And so we were able to set that trade off differently in different situations without sending more packets.
[00:33:36] So the musicians, for example, are incredibly sensitive to delay. Like we would come on, I mean these are professional musicians, but we would come on like, and we'd say, try out the new version of the software. It's gonna be great. And they'd play for a little bit and they'd say, no, no, no, no, no. Go back to the Git commit you had two weeks ago. It's like, how do you even remember that? It's like, it was better then, right? So for them, we gave them the glitchiest audio, but it was absolutely real time. And then for the actors, we gave them, you know, less glitchy audio, but still good enough for them to say that overlapping dialogue. And then the audience, of course, got the perfect audio. And then, you know, in the middle we had people that were adjusting the levels and the behind the scenes people and this kind of thing. So with the same stream of packets, we could pick different trade offs between glitchiness and delay. And so that was the one academic trick.
[00:34:20] Russ Altman: Thank you. That vignette and, and, and the idea that the three principles are having very different actual experiences, uh, but all of them well matched to what they need to do their job of producing or listening, very helpful. I wanna make sure that we, I give you a chance to talk about this, this new class that you're teaching.
[00:34:38] Keith Winstein: Yes. Yeah. I mean, what I learned from this and then from some follow on work, um, is that, you know, we're not teaching our students this joyful part. The part that I think of is like, so central to computer science is the playfulness of it. I mean, computer science is about reasoning about the consequences of procedural reasoning and computation. And the fact that you can design an automaton in your head and then see it play out at, you know, billions of times per second. I mean, that's fascinating to see so quickly the consequences of a, of a thought. And I don't think we're teaching, I mean, you, you teach here. I think that's not the kind of playfulness that we, that we are teaching and it's much more about like, I'm gonna train this thing and it's gonna run for a while, then I'm gonna test it.
[00:35:16] You know, it's not so interactive. And so I would like, so anyway, I, I'm with some, um, you know, a bunch of, uh, people, I'm starting a new, trying to start a new freshman introduction to computer science and electrical engineering that emphasizes the playfulness and the joy and the interactivity of computation. And so the idea of the class, it's a CSE class, the idea of the class is about real time interaction with the real world. And everyone's gonna get one of these, like toddler xylophones. You've seen these things.
[00:35:43] Russ Altman: I know it so well.
[00:35:44] Keith Winstein: Okay. You got kids?
[00:35:46] Russ Altman: Yes. I have grandkids. I've had two or three generations, I've had three generations. 'Cause I used one. My kids used one, my grandchildren use one.
[00:35:54] Keith Winstein: Well, no, no judgment. Maybe just you like with, you know, it could be anyone. They're fun. So, you know, we're gonna give everyone one of these xylophones. And we're gonna talk about, you know, what is sound, you know, it's a continuous pressure wave. It's a continuous function of time. And you know, why is it these fundamental frequency of these notes? I mean, isn't it amazing that the ancient Greeks and the ancient Chinese came up with basically the same scale? You know, how do these two ancient societies, thousands of miles apart, derive the same relationships? And then, you know, how would you reason about this information, this audio in a computer? You know, you might think that, well, it's a continuous wave. It has an infinite amount of information. It's a, there's a point at every, every time it has a different amplitude, different air pressure.
[00:36:33] But it was discovered only a hundred years ago, you know, after relativity, after quantum mechanics, they discovered that no, it's possible to take a finite number of samples per second and perfectly capture all the information in this continuous wave. That's pretty mind blowing. Um, that's called, you know, the Nyquist-Hartley theorem, I think. And so, um, when you have that information in a computer, you know, I want the students to write programs very live, very interactively that understand the sound and manipulate it. So can you write a program that understands, and these are for freshmen, these are people with, you know, no background. Can you write a program that understands what note of the xylophone did you just hit in real time?
[00:37:12] And can we come up with a code that maps kind of the letters of the alphabet to pairs of xylophone notes, you know, A, B, C, et cetera. Can I give you a sentence like, you know, the sky is blue outside and you play it, and then your phone or your computer in real time prints out the sky is blue outside, in real time, not sort of run it for a day and, and check and that kind of thing. You know, this idea that human language can be expressed in bits and talked about, you know, that was quite controversial. You know, that, you know Claude Shannon, who's the founder of Information Theory and you know, hung out with all these Turing and those kinds of people, I dunno about Turing, but those people, you know, he wrote this famous paper in the 40s about information theory, and one of his chapters was about, you know, human language and the information content, the perplexity of human language.
[00:37:54] And when the famous Soviet mathematician Kolmogorov read that paper, he didn't get that chapter 'cause it was censored by the Soviet translators. You know, the Russian translators believed that the idea that, you know, that even the human language is quantifiable or that you can talk about the information content of human language quantitatively, that was thought to be contrary to Marxist doctrine. So these are, these are controversial topics. The idea that you can have computation about language and about thought. And so I, I would love to have this much more playful introduction, joyful introduction to computer science and electrical engineering through this sort of real time interaction with the real world. And that came from this collaboration with Michael Rau that we did, you know, five years ago, at this point, the theater.
[00:38:34] Russ Altman: It's a fantastic way for us to end with this hopeful note, uh, no pun intended, that this is the way we're gonna introduce the next generation of computer science to their craft. Uh, and it's real time, it's playful, and it may be not what people are expecting at this time, uh, in, in, in the future of computer science. So it's great to hear. Uh, before we end, I'm wondering if, um, we can do the Future In a Minute segment where I ask you a few rapid-fire questions and you give me, uh, as, as is possible, uh, a short, sweet answer.
[00:39:04] Keith Winstein: I'll, I'll do my best.
[00:39:05] Russ Altman: Here we go. What is one thing that gives you the most hope about the future?
[00:39:10] Keith Winstein: I, I mean, I think many of our students really do have an optimism about their ability to make a positive difference. And, and, and many of our students are just incredible, incredibly impressive in both their ability to get things done, but also the, the, the dreams that they have and the hope they have about the future. And I think that's, that's very fueling.
[00:39:31] Russ Altman: What's one thing you want people to walk away from this episode remembering?
[00:39:35] Keith Winstein: I mean, I think that truth matters and that the computer scientists are trying at least to find a way to capture that.
[00:39:42] Russ Altman: Aside from money, what is the one thing you need to succeed in your research?
[00:39:47] Keith Winstein: Oh, I mean, fantastic, you know, doctoral students.
[00:39:50] Russ Altman: If all goes well, what does the future look like?
[00:39:53] Keith Winstein: You know, I mean, I think the things that I prize, and I think things that are core to computer science is this idea of independence of thought. So I, I, you know, the idea that a college student in Finland can make the operating system that turns out to, you know, take over the world and a guy in, in Switzerland can make the thing that, you know, we all communicate over the worldwide web and, you know, the, the two bike mechanics in Ohio can make the airplane. This independence of thought and distribution of where innovation comes from. I mean, most innovative, I know this podcast is just Stanford people, but most innovation, if you look at it, it does not come from Stanford and MIT and Berkeley, right? Most of it comes from nobody's. Uh, most of the most significant innovations in the world come from not where it was supposed to come from. And for that to remain possible, there needs to be this distribution of people near the frontiers of innovation, the bike mechanics or the research programmers, you know, all over the place so that some people can take the step, um, you know, and, and it's never who you think it's gonna be to innovate.
[00:40:47] Russ Altman: And finally, if you were starting over again and you needed to get your certification or your degree in a different discipline, what would it be?
[00:40:54] Keith Winstein: I always wanted to be one of those people that was fluent in Latin, very, you know, if you read like The Secret History, I was very jealous of that world.
[00:41:01] Russ Altman: Thanks to Keith Winstein, that was the future of network systems. Thank you for listening to The Future of Everything. Don't forget, we have a huge catalog of back episodes and you can listen to the future of just about anything. Also, please rate and review the podcast, your ratings and reviews, we read them, we respond to them, and they help us grow the community. So please do it. If we deserve a 5, give it to us. Otherwise, give us whatever we deserve. Thanks very much. You can follow me on many social media apps such as LinkedIn, Threads, Bluesky, and Mastodon, where I'm @RussBAltman, or @RBAltman you can also follow the Stanford School of Engineering @StanfordSchoolOfEngineering, or @StanfordENG.
