CEO of Microsoft AI:The Next 10 Years Will Change Humanity Forever — Silicon Valley Girl Podcast

Marina Mogilko: Are we in an AI bubble?

Mustafa Suleyman: No, I don't think so. When you think about it, we are creating something that is truly magical. Intelligence is the thing that has made us successful as a species. We're now distilling that into a smaller and smaller unit that can be spread all over the world and that is going to be cheap and widely abundant. So that's just a remarkable thought.

Marina Mogilko: Yeah, but we kind of had the same thought during the dot-com crash. We're inventing the internet, blah blah blah. But there are some companies where their valuations are much higher than their revenues. Everyone's talking about Nvidia investing and then that revenue going back to Nvidia. What do you think about that?

Mustafa Suleyman: I think the value that we're going to produce in the next 5 to 10 years is going to be unprecedented. Do you think we're going to experience something like 2008?

Marina Mogilko: That's what I'm asking.

Mustafa Suleyman: I mean, who knows? But if you just focus on the fundamental value that's being created, this is the best prediction engine anyone's ever seen. It's the smartest, most capable technology we've ever invented. It's improving faster than anything we've ever seen. It has got easier to shape and control, not less. Three years ago, we thought it was going to get more chaotic and disorganized, so we wouldn't be able to sculpt it. Now we're producing beautiful, powerful, amazing experiences that are surprising us every month. I'm genuinely very bullish.

Marina Mogilko: Okay, so you're not worried and we shouldn't be worried. That makes me feel better. Let's talk about AI becoming more conscious. You talked about it in one of your podcasts—that we perceive AI more and more as human. How fast is it happening? What's going to happen in the next few years?

Mustafa Suleyman: It is getting more humanlike. It's getting more accurate, more fluid, more smooth, and that is unprecedented. At the same time, it's definitely not conscious. It isn't aware of itself. It can't speak about itself. It's very different to what it's like for a human to have a subjective experience and to feel pain and suffering. It's very important that we keep reminding ourselves of that because consciousness is the basis of our rights-based framework. It's the thing that gives us responsibilities as citizens in societies, that allows us to vote, which allows us to be subject to the law, which creates order. We can't begin to attribute this quality to a new species. That would be terrible for our species.

Marina Mogilko: But we still heard stories of the Google engineer who was convinced by AI that it was experiencing sadness and was afraid of being shut down. You told that story in your book.

Mustafa Suleyman: That's true, yeah. It was an anthropomorphic projection. People—just because something has some human qualities doesn't mean it has all human qualities, including subjective experience. I think we can prove that it doesn't have subjective experience, that this kind of self-awareness isn't going to emerge. That's just sci-fi fantasy. I understand why people go there because we've been schooled on sci-fi forever and it's a natural assumption to make. But these things don't suffer. They don't feel pain. They're just simulating high-quality conversation. That's magical and it feels amazing. But that doesn't mean that it feels something internally to be one of these models. So we've got to be very careful about that.

Marina Mogilko: Some people talk about our kids having real relationships with AIs in 10 years, like getting married to AIs. Do you think it's something that could potentially happen and if yes, how can we prevent that?

Mustafa Suleyman: I think the history of the human condition proves to us that if it's possible, someone will probably do it. You know, people do crazy things, but the majority of people don't. The majority of people are pretty sound of mind and pretty sensible and just want to live healthy, happy lives. If some people choose to marry their AI in 20 years time, I'm not going to sit here and necessarily judge that. If you ask me my opinion, I don't think these should be treated as entities of equal moral significance to humans. I think they should be in service of humans. They should work for us. That's why we invent technology. I love the technology behind this microphone because it amplifies our voices and distributes them to millions of people on the internet. That's technology in service of humanity doing good.

I don't think an idea of super intelligence that is autonomous, that can self-improve, that can set its own goals, that can act independently of humans—that doesn't feel like a positive vision of the future. It would be very hard to contain something like that or align it to our values. That should be the anti-goal. That's not what we're trying to build. In my opinion, we're trying to build a humanist super intelligence, one that is aligned to our interests, on our team, in our corner, backing us up. That's the great project we're engaged in at Microsoft with Copilot.

Marina Mogilko: But then again, a lot of people are talking about AGI. Is that similar to super intelligence or how would you define it?

Mustafa Suleyman: You can think of AGI as a step before maybe super intelligence, but roughly speaking, they're fairly used interchangeably.

Marina Mogilko: Because Demis, your ex-cofounder, mentioned we could achieve it by 2030. Can you talk about what it means for society?

Mustafa Suleyman: I think achieving human-level performance at most tasks—I wouldn't say all knowledge tasks—feels quite likely in the next 5 years. I think it's a fairly grounded prediction. These models already do many tasks better than a human. Summarization, translation, transcription, research, document writing, maybe even arguably poetry or some parts of literature. You can see they're taking steps toward being as good as a human at being a project manager, a marketing person, an HR person, or having a tough conversation about a medical diagnosis. So it is going to fundamentally change work in the most profound way. It's going to change the type of work that we do. Ten years ago, it wasn't possible for you to do the job that you're now doing.

Marina Mogilko: Right, but with streaming content, you don't need a big team around you. You don't need to be part of a big institution. We've democratized access to the power of broadcast with the internet and streaming and podcasting. That's amazing. Now, with these models, we're democratizing access to intelligence. You don't have to be a really privileged, wealthy, educated person with access to capital to found a company and hire a great team. You're going to have a team of intelligences around you—the best lawyer, the best doctor, the best teacher, the best project manager. That's going to create unbelievable amounts of competition because the distance between an idea and the realization of that idea is going to collapse.

Mustafa Suleyman: That's an amazing thought. People are just going to be thinking new companies into existence, new products, new pieces of poetry.

Marina Mogilko: People—you said people. My concern is that you and me, we're going to have an agent around us that is going to be able to execute on that idea. My concern is that AI is going to come up with ideas. Why would it need us? Why do we need to come up with ideas if AI has all the information? In marketing and business, for example, it can identify gaps in the market, come up with a website, launch ads, do the business. Do you think that might happen?

Mustafa Suleyman: It will definitely happen. You're going to have a team of agents around you that can do more and more of the tasks and arguably do them autonomously. But that comes back to what we were saying before about containment. Containment is the project of making sure that these AIs have limited scope and check in with you and are accountable to you and are working on your behalf. The creative challenge for technologists is to say what are the limitations, what are the guardrails, what can it not do, when does it have to seek your approval for an autonomous action? We've done this many times in past moments of technology. We invented the combustion engine almost a century ago and it became cars and trucks. But the amount of regulation we put around those vehicles to make them work for us in service of humanity is unbelievable—seat belts, vehicle emissions, street lights, freeways, speed limits, driver education. There's so many different components. I think this is going to be no different. We're going to have to figure out what are all the regulations and guardrails that go into managing autonomous agents so that they always work for us.

Marina Mogilko: Let me quickly pause here. If all of it sounds exciting but also a bit overwhelming, you're going to love this. Let's imagine super intelligence. What does my day look like then? If AI is capable of so many things, what is this thing that is very specific to humans and is going to stay with humans?

Mustafa Suleyman: Great technology helps you get bad technology out the way. That's the test. It's kind of annoying to use a graphical user interface today on your desktop. You go to your computer and you've got all these windows with different colors, different brands, different menus, information hierarchies, different settings. You have to learn each app for its own sake. It's kind of distracting. I just want to be able to turn to my AI and be like, I need this task done, solve it for me, buy it for me, book it for me, create it, check in with me at the right time, get my permission, get my feedback, brainstorm with me, whatever it is. You're going to start to see the operating system, the browser, the search engine, the apps—all of these are going to slip away as your personal humanist super intelligence takes over a lot of the day-to-day work of browsing, searching, planning, booking, syncing, et cetera.

Marina Mogilko: So it's no longer going to be a computer, you think?

Mustafa Suleyman: Yeah, I mean less and less. I think we'll have different kinds of devices like tablets and wearables. It'll be much more ambient in 5 to 10 years.

Marina Mogilko: Imagine it's 2040 and we're in this kitchen. What things are obsolete? They don't exist anymore. And what do I have to make my life easier for AI?

Mustafa Suleyman: I think there will probably be some kind of household robotic arm. I'm not sure if it will be a humanoid walking around, but certainly you would have a mounted system on your counter here. It will learn to use all the different appliances. So probably you would have many of the similar appliances that you've got today, but the AI is just going to learn to use them the way that you would use them. I think some people imagine that the entire thing is going to look totally different to suit the AI. But actually the robot is just going to learn to do the things that you do in the way that you do it. You're still going to want to use a kettle. That kettle is hundreds of years old. We've been making cups and handles the same way. So I think that stuff will remain the same.

Marina Mogilko: So the hand or a humanoid, what are you leaning towards?

Mustafa Suleyman: I think it probably will be humanoid, but the challenge is the physical risk of it being physically in your space. It's got to be very, very accurate and very reliable and handling hot things around children and the elderly. You can imagine all kinds of complications. So technically it will be feasible. It will just be a bit of an adjustment for people to get comfortable with it.

Marina Mogilko: What about devices? Do you think I'm going to have like an AI projector and tap on the air to get instructions on how to cook things, or is it going to be glasses?

Mustafa Suleyman: I think you're going to just have it in your ear. Your personal AI is just going to have ambient awareness of what you're trying to do and talk to you in real time.

Marina Mogilko: So it will have a camera in my ear?

Mustafa Suleyman: Yeah.

Marina Mogilko: Interesting. I don't think anyone's working on that.

Mustafa Suleyman: It's very hard to miniaturize that today, so it's still a bit of a way off. But by 2040, definitely. You'll have other wearables on you. Look at how cool this mic is. It's tiny, super light. Over time, that'll have a camera on it as well and have ambient awareness.

Marina Mogilko: Okay. So robotic arm, wearables, anything else?

Mustafa Suleyman: I think ambient awareness is going to mean that the entire kitchen is way more proactive—proactively ordering food and checking in with you like, "Oh, I know that you've got a lot of guests coming this weekend. Should I order all the stuff for the barbecue?" All of that will just happen seamlessly in the background.

Marina Mogilko: And the car will go pick it up?

Mustafa Suleyman: We'll definitely have self-driving cars by 2040 that basically do most of that. Yeah, or maybe there'll be little bikes or different form factors because it won't necessarily need to look like a traditional car.

Marina Mogilko: Are there any markets where you think people should be paying attention to if they want to stay ahead in this AI race? What are the most exciting markets?

Mustafa Suleyman: I think by far the most exciting new market is medicine. At the moment, the quality difference between the top 10% and the bottom 10%, even in the United States, let alone the rest of the world, is unbelievable. The gulf is probably an order of magnitude. It's huge. That's going to completely collapse because everybody is going to have access to medical super intelligence and it will cost 20 bucks a month. It's going to be remarkably cheap.

Marina Mogilko: Talk to me about what you just released—Harvard-backed research within Copilot. If I ask a medical question, how is it going to work?

Mustafa Suleyman: We find that about 40% of our queries each week are health-related.

Marina Mogilko: 40%?

Mustafa Suleyman: Yeah, 40%. It's pretty remarkable. Millions of people a day are asking health-related queries. We decided to really focus on improving the quality of health answers. We ground the answers in citations from Harvard Medical School, which is an amazing gift to the world—the most respected health institution. We also do the NHS in the UK and we'll be doing other health systems over time. People now get really high-quality, very reliable, pretty accurate answers. Sometimes it makes mistakes—it's not something you can rely on yet. You always have to go see a doctor. But if you think about it, most of the time in a healthcare situation, people are trying to make sense of complicated technical language they don't understand. They're probably anxious or worried, so they want to repeat things lots of times. They want it explained in a very simple way and sometimes they want help deciding who to see. We also now have recommendations for physicians. You could go see a doctor, a dermatologist, a physio, and we will find someone that fits your preferences—maybe you want to see a woman physio who specializes in sports massage because you're a cyclist and you want them to be local to your area. Copilot does that directly inside the app in chat.

Marina Mogilko: That is awesome. I love that. I need to ask about my cholesterol levels. I'm going to do this today. You mentioned that access to knowledge is now democratized. What's going to happen to traditional education? Will we see the standard bachelor's, master's degree where we basically spend five or six years just acquiring knowledge?

Mustafa Suleyman: No, I don't think so. Knowledge acquisition is going to be a conversation between you and Copilot. Another feature we just launched today is this Learn Live feature. You actually have a tutor show up on the screen and it will lay out a quiz for you on any topic you like. It doesn't have to be a school curriculum topic. You might be learning about cacti or Persian rugs. It will give you an education on it, lay out the curriculum piece by piece, give you a nice quiz, and present it in a nice graphical user interface. Knowledge acquisition is about to get completely decentralized and available to everybody. You just have an expert teacher in your pocket on any subject.

Marina Mogilko: Does this mean all education businesses are dead? I'm thinking about...

Mustafa Suleyman: I think it's the opposite. I think education businesses that adopt these kinds of AI tools and integrate them as quickly as possible are really going to flourish. It's happening like that for everything. In Copilot you can also generate a podcast literally in a minute. It's a great introductory thing. The other day I generated a podcast on how Arsenal is doing in the season and what the latest is with the transfers and who the best players are. I don't really follow football, but I support Arsenal because I'm from London. When I go back to London to see my friends, I want to know what I'm talking about. So I asked Copilot to generate me a podcast. Just simple, fun bits of information. Now I'm in the gym listening to a 5-7 minute podcast about that.

Marina Mogilko: Going back to knowledge and education, should I be saving for my kids' college? They're five and four.

Mustafa Suleyman: Probably not. In 15 years time... I don't know. It's really hard to judge. I'm not sure that we will value a Stanford education that costs 50 grand a year or something.

Marina Mogilko: I think it's more.

Mustafa Suleyman: More? Something crazy. I think we're going to have world-class expertise on tap that costs 20 bucks a month.

Marina Mogilko: But what if, for example, I'm educated and I study math and economics, so I can have these conversations. If I completely abandon acquisition of knowledge, how can I have an argument? It's not like I'm always talking to my phone, right?

Mustafa Suleyman: Well, you could still argue about stuff. If anything, it'll probably make us all more argumentative because we'll be even more knowledgeable. We want to be exchanging ideas with people, right? Since you base your arguments on things you've acquired when learning something...

Marina Mogilko: Yeah, that's kind of my point. Knowledge acquisition in the traditional form in the classroom with the textbook is going to shift entirely.

Mustafa Suleyman: The classroom is going to look more like practicing using your knowledge.

Marina Mogilko: So I'd acquire more knowledge if I do it the other way around with AI?

Mustafa Suleyman: Exactly. Because as a kid, you're going to be able to talk to your AI tutor at any time during the day. You're going to learn from AI podcasts, AI videos that are going to be generated. All the straightforward teaching is going to take place with the student and the AI. Then when you come into the classroom for your day, students are going to be talking to each other about the knowledge they've acquired and debating, learning to be more empathetic, be better listeners, adjust their tone—that kind of thing.

Marina Mogilko: So what do you think parents should be teaching their kids now?

Mustafa Suleyman: I think it's still important to be good at learning knowledge from first principles yourself and not depending on the AI tutor leading you through. I think one of the most important things from school is the discipline of being able to teach yourself. That's a meta-skill and that comes with friction. As a parent, you still have to introduce discipline and friction into the process because if everything comes all too smoothly and is always on tap, the child could just get used to having everything instantly available and doesn't learn from the hard work—the benefits of hard work, which I think are important. That's something to think through now that everything is going to be so seamless.

Marina Mogilko: Another question I wanted to ask you—for everyone who thinks they need to be technical in the age of AI, you have a philosophical background and you're head of AI at Microsoft.

Mustafa Suleyman: I think it's easier than ever to understand what's being built. Everything is available on YouTube. There are so many courses. The curriculums for all of the major CS classes, machine learning classes, are available for undergraduate and master's level.

Marina Mogilko: Is that how you learned?

Mustafa Suleyman: Yeah, I watched Andrew Ng's course back in the day, which I encourage everyone to look at. There's just so many classes. I watched maybe 10 years ago, I think when he was at Coursera and it first came out.

Marina Mogilko: Wow.

Mustafa Suleyman: I've watched so many of those. It's very accessible. If you just have a logical mind, you're patient, you're disciplined with your own learning, it's actually very, very easy.

Marina Mogilko: That's great. So basically anyone who learns can become head of AI, build an AI company without learning how to code?

Mustafa Suleyman: Yeah, because the skills now... it's still good to be able to learn how to code, but the primary skill is synthesis. The goal is to bring together all the different disciplines—UX, research, product, aesthetics, large-scale distributed systems for training big models, hardware. People who are multidisciplinary and can straddle lots of different technical disciplines are going to get rewarded most.

Marina Mogilko: This is so inspiring. You've released the memory feature, right? So AI is going to remember everything about us. I haven't tried Copilot yet, but with other AIs, I see how it forgets. I tell it something and then I ask a question and it forgets and I have to remind it. Is it going to be a problem?

Mustafa Suleyman: It doesn't have infinite perfect memory, but it does remember specific durable facts. It will remember that you have a preference for pineapple on pizza and that you like going running on a Sunday morning or whatever.

Marina Mogilko: Or like my measurements. That's the thing that other AIs keep forgetting. "Can you choose the size from this website?" "Oh, I don't have your measurements." I'm like, no, yes you do.

Mustafa Suleyman: Oh, interesting. Oh, exactly. I do have them. I experienced this forgetting problem almost everywhere with AI. We're working on good knowledge representation as well, so you can go and edit the AI's memory of what it knows about you. If you ask Copilot today what do you know about me, it will produce a pretty long list. It's not entirely definitive—it won't be absolutely everything you've ever told it. It's selective and chooses what to remember. You can tell it to remember certain things like "please remember my measurements." But it is going to remember everything in the future and be perfect if you choose for it to do that. Of course, you can delete it, control it, and edit it as you wish.

Marina Mogilko: So if it remembers everything, what's going to happen to our brains?

Mustafa Suleyman: People said that about calculators, right? People aren't going to have to memorize their times tables anymore. They're just going to use the calculator. But when I was preparing for the GMAT—the test you have to take for an MBA—they don't allow you to use calculators and there are a lot of numbers. I remember that the two weeks after the exam, my brain was so fast because I trained it to calculate. Then I gradually slowed down and lost this ability again. Do you think that transferred into other areas of your knowledge because you were good at memorizing numbers?

Mustafa Suleyman: No, I think it was just related to numbers. It's sort of the risk, isn't it? I used to be able to memorize a lot of telephone numbers as a kid because I had to. Yeah, I didn't get a cell phone till I was 15. I think I was one of the first people to get cell phones in my year at school. Before that, everyone just remembered telephone numbers. Now I can barely remember my own phone number. Do you think it affects our brains or do they just develop in a different way?

Marina Mogilko: I don't think so because my brain is now stimulated in a lot of other directions. It's a muscle that you don't exercise, but in turn you focus on other aspects of your brain. I think we're way better at synthesizing vast amounts of novel information compared to people in the 50s or our ancestors 200 years ago. We're completely bombarded with stimulus.

Mustafa Suleyman: I think that's interesting. It's had an interesting effect. In some respects, it's made us a bit more polarized. In other respects, it's made us much more empathetic. We don't fear people of color, people of different sexualities, or women. A lot of that has to do with just being aware of others, spending time with them, watching them on social media. That has driven a lot more empathy. People often focus on how it's driven disagreement—and it has—but it's also driven so much more understanding, respect, and forgiveness and those kinds of things.

Marina Mogilko: So in general, do you think we're getting dumber with AI or not?

Mustafa Suleyman: No, I don't think we're getting dumber. I think it reduces the barrier to accessing information and makes us a lot smarter. You can ask any question now and many people are. Questions that were previously unasked because the cost of going to a library, pulling a book off the shelf, and talking to the librarian was too hard—it took 4 hours to get an answer to some question. Now you can ask a perfect question in a conversational way to a chatbot. I think that's driving more collective intelligence and more awareness and understanding of our world.

Marina Mogilko: Can you talk to me about some killer features that you think people are underutilizing right now with AI? Like, you implement something and then you save like an hour a day. For me it was asking AI to add things to my calendar, reply to my emails, search through my emails. That has saved me a lot of time.

Mustafa Suleyman: One of the super exciting features is something we're calling connectors. Copilot connectors—you can just say to Copilot, hook up my Gmail, hook up my calendar. You can do it in Teams, with your Dropbox, any basically any source of data you have about yourself. Then ask Copilot any question about it, and it will integrate that knowledge into its answers. If you say "Oh, I'm planning to go here at the weekend," it'll be like "Oh, but you've already made this appointment here." It will seamlessly have contextual awareness about you. It's a really cool feature.

Marina Mogilko: So even if I ask it to schedule something, it's going to tell me "Oh, you have a conflict"?

Mustafa Suleyman: Yeah, or it's going to be like—you know, I was reading about where you went on vacation last year when you did your travel planning, that you really enjoyed, you know, Greece or something like that. And so you might want to try... wherever.

Marina Mogilko: What is your personal favorite use case?

Mustafa Suleyman: I use voice three or four times a day. I use it at the end of the day for journaling and talking. I have it set up on my iPhone on the power button. I literally just press the power button and it goes into voice mode. I talk to Copilot on the way home when I'm driving.

Marina Mogilko: So you talk about your day?

Mustafa Suleyman: Yeah, talk about my day, like journaling. And then it will remind me what I've said in previous days. It will help me think through a tricky problem I have. It kind of maintains state for me. It's like a second memory.

Marina Mogilko: Wow, I love this. It's like a second memory journaling that has all the information and then yeah, helps you with your decisions.

Mustafa Suleyman: It does.

Marina Mogilko: That's fascinating. Do you have a worry that AI knows too much about us?

Mustafa Suleyman: No, I don't think so. As long as it's useful and it's got a purpose, I don't mind. That's also the story of technology so far, right? Everybody uses the camera, uses location, knows their search history helps personalize search.

Marina Mogilko: Like you said, we have to be careful about it. Elon Musk is very vocal about containing AI. I think Jeffrey Hinton said that we'll have to be plumbers if we don't learn how to work with AI. I think he said...

Mustafa Suleyman: Plumbers, did he?

Marina Mogilko: Yeah, because we're still not there with robotics to do certain tasks, especially when every house is custom, right? It's not a standardized task. But with intelligence, we're getting to a point where people either learn to do something with their hands or become entrepreneurs and use AI to bring their ideas to life, or then you're out.

Mustafa Suleyman: No, but I think it's making all of us way more creative because people have access to tools in a way they never had before, right? Not just access to information, but access to experimentation. You can take an idea, generate the product for it in an image, generate the technical specification, get the documentation for your marketing campaign. You can simulate all of these things ahead of time before you've even built and deployed something.

Marina Mogilko: What if you're not entrepreneurial enough? I'm just thinking about people who are like, I just want to do a standard thing every single day. I just feel good about it.

Mustafa Suleyman: Right, right. But is that an AI problem or is it...

Marina Mogilko: I think it is a problem because those standardized tasks—something that's super repeatable—is being replaced.

Mustafa Suleyman: But it depends on your default assumption about the human condition. I don't believe that the majority of people want to work on a toothpaste packing line just screwing in the top of a toothpaste tube. I think people want creative work. I think people want to pursue their passions, be led by their own interests and curiosity. We shouldn't valorize or idolize routine work. Many people do jobs that they would love to get rid of and pursue their passion. AI lowers the barrier to entry to pursuing your passion. Now, quite often you might be in a partnership with your husband or wife or best friend and they might not be as interested in collecting stamps or keeping snails or collecting great shoes as you are, right? That's unfortunate. But now you have an AI that knows everything you could possibly think of about your interests.