Today, I’m talking with Matt Garman, the CEO of Amazon Web Services, or AWS. Matt took over as CEO last June — you might callback that we had his predecessor, Adam Selipsky, on the show just over a year ago. That makes this episode terrific Decoder bait, since I love proceeding how fresh CEOs decide what to change and what to keep erstwhile they’ve settled into their role.
Matt has a truly interesting position for that kind of conversation since he’s been at AWS for 20 years — he started at Amazon as an intern and was AWS’s first product manager. He’s now the 3rd CEO in just 5 years, and I truly wanted to realize his broad view of both AWS and where it sits inside an manufacture that he had a pivotal function in creating.
Listen to Decoder, a show hosted by The Verge’s Nilay Patel about large ideas — and another problems. Subscribe here!
You’ll hear Matt say that most companies are inactive barely in the cloud, and that chance remains massive for AWS, even though it’s been the marketplace leader for years. If you’re a product manager or an aspiring product manager, you’ll catch Matt talking about these things precisely like the product manager he was from the start, only now with a broad view from the CEO chair.
But just acquiring fresh customers isn’t the game any longer: like all cloud provider, Amazon is reorienting its full computing infrastructure for a planet of generative AI. That includes more than $8 billion in backing for Anthropic, a immense push to build its own AI chips to compete with Nvidia, and even atomic power investments as the energy request for AI continues to grow. After Matt and I talked before the holidays, AWS announced an $11 billion investment to grow its data center operations in Georgia.
Matt’s position on AI as a technology and a business is refreshingly distinct from his peers, including those more incentivized to hype up the capabilities of AI models and chatbots. I truly pushed Matt about Sam Altman’s claim that we’re close to AGI and on the precipice of machines that can do tasks any human could do. I besides wanted to know erstwhile any of this is going to start returning — or even justifying — the tens of billions of dollars of investments going into it.
His answers on both subjects were beautiful candid, and it’s clear Matt and Amazon are far more focused on how AI technology turns into real products and services that customers want to usage and little about what Matt calls “puffery in the press.”
One note before we start — we recorded this episode just before the holidays, so I asked Matt about Netflix, 1 of AWS’s biggest customers, and whether it would hold up while streaming live events, especially the NFL games it streamed on Christmas. Turns out, Netflix did just fine with those, but the answers here were beautiful interesting. Matt inactive checks in on his large customers, even as CEO.
Okay, AWS CEO Matt Garman. Here we go.
This transcript has been lightly edited for dimension and clarity.
Matt Garman, you’re the CEO of Amazon Web Services (AWS). Welcome to Decoder.
I am very excited to talk to you. You’re like a perfect Decoder guest. You are, I believe, the first product manager at AWS, you started as an intern and now you’re the CEO. We have quite a few listeners who want to be on that journey, so there’s lots to talk to you about just in that.
You’re besides the fresh CEO. We had your predecessor, Adam Selipsky, on the show just a small over a year ago. You’re about six months on the occupation now. So, there’s quite a few Decoder stuff in there — how you’re changing the organization and how you’re reasoning about it. And then, obviously, we’re going to talk about AI. It’s going to happen. I hope you’re ready for it.
I’m ready for it. Shoot, fire away. I’m happy to go wherever you want.
All right. But I actually want to start with a very hot-button, profoundly controversial topic. Are you ready?
Okay, it’s Jake Paul. I want to start with Jake Paul. My knowing is Netflix is the prototypical AWS customer, right? They started on AWS, they made a large bet on AWS. They’re inactive the customer, right? They haven’t left AWS?
Yeah, Netflix is simply a large client of ours. Absolutely.
They just had the live stream of Jake Paul fighting Mike Tyson. You can think anything you want about those 2 men fighting each other.
I was hoping Mike would win, honestly.
I think most were, but that’s okay. It was fun to see him out there.
You’ve just set off a million more conspiracy theories about this fight. Anyhow, I told you it was controversial. All right, but the stream was beautiful glitchy. I think everybody agrees on that. erstwhile I watched it, it degraded to 360p at any point for me. Netflix CEO Ted Sarandos was just on phase at a conference. Netflix said the request is 108 million people globally, and here’s what Ted said about that stream: “We were stressing the limits of the net itself that night. We had a control area up in Silicon Valley that was re-engineering the full net to keep it up during this fight due to the unprecedented request that was happening.”
You’re the CEO of AWS, you’re the internet. Did they gotta re-engineer the net for the Jake Paul fight?
You’ve got to ask Ted about that. I think where they were stressed about the [content transportation network] they run, and you can ask Ted about that too. Netflix has its own homegrown CDN that it uses, and that’s the part that I think was stressed. I don’t know the details of precisely where they were moving into barriers, but it wasn’t in the AWS infrastructure, it was in the Netflix-controlled part of their structure.
Yeah, their CDN is truly fancy, right? They’ve got boxes and ISPs and everything. I was just curious due to the fact that what we’re about to talk about, in a immense way, is how providers like AWS can meet the increasing request for compute everywhere and then get it to the people who request it. And it feels like most people in 2024 take video streaming for granted, but it’s inactive beautiful hard.
It is. And I think in particular, there are a couple of things around that that are challenging, right? By the way, it’s a super hard thing that they did. Number one, it’s their first time doing a big, scaled live stream like that. The first time is actually what’s hard. another people have done that before. We’ll stream Thursday Night Football and another places like that that have figured out how to do things at that scale, but it’s not the first time. So, I’m certain that the next time — I think they have a Christmas day game — they’ll most likely work out any of those kinks and figure that part out.
The first time you do it you’ll find those bottlenecks. And it’s actual about any compute strategy where you have an order of magnitude more [to figure out]. They evidently have shows that have streamed more, but they’re spread across more time. So it’s this single spike up where everybody comes in a 30-minute window, and if it’s outside of what you planned for … If they planned for — I don’t know what their numbers were — 150 million and they got 180 million, it was outside of what they thought their advanced limit was. We’ve seen this before in AWS and we’ve seen this in Amazon. The first time we did Prime Day we most likely had issues across that too, of just people hitting the website and another things. So the first time you do events like this, it’s a learning process.
I think it’s most likely overstating it to say that they had to re-architect the full internet, but it is that key spike where quite a few applications are just not … peculiarly erstwhile you own the infrastructure, and this is 1 of the benefits of the cloud, by the way, is you get to ride on the law of large numbers where any 1 spike doesn’t overwhelm everything else. Netflix evidently has a immense number of customers, and I guess that they’ll be much more prepared for next time. But it’s a good learning experience for anybody even at a much smaller scale. erstwhile you’re planning an event that has the possible to be materially more than your average baseline, there are always risks that there are any scaling factors you don’t anticipate.
So it’s not a amazing problem to me. We’ve seen it over and over again and it’s 1 of those problems that the cloud helps to solve. But even in the cloud, planning is required and you gotta think about how you scale ahead of it, and things like that.
When you were at home watching the fight, did your pager go off?
I was texting back and distant to our support squad to make certain we were supporting the Netflix squad as much as possible, yes.
How frequently does that happen to you as you usage the net and you think, “Boy, this is most likely moving on AWS. I had better make certain it’s going fast?”
More back in the day erstwhile we were scaling and learning — back in 2007 and 2008 where we were learning how to scale there. Today, we’re frequently at a broad scale and so everything, lots of things on the net and around the world, run on AWS. And we usually run beautiful reliably, so it comes up little than it utilized to, for sure.
Do you have Down Detector bookmarked on your laptop?
We’ve got to get the CEO of Down Detector on the show. That is simply a fascinating service across the board.
Let me ask the Decoder questions due to the fact that I think this subject of “we are going to be more reliant on cloud infrastructure for compute in the planet of AI,” and that’s got to scope all the people and hopefully make everybody any money and make any useful products and services — that’s the theme. And I think whether or not we can stream people punching each other, and whether or not we can stream AI, the problems there are the same in the general sense.
But I want to ask the Decoder questions first so I can realize how you are solving those problems, having been at AWS for so long. So you are taking over for Adam who was on about just a small over a year ago. He stepped down about six months ago, you took over. You’ve been there a long time. You started as the first product manager of AWS, which is simply a beautiful chaotic place to begin a career and end up as a CEO. How are you reasoning about AWS, the organization, right now?
There are a couple of things that I’m reasoning about. One, I have been here for 18 years, so I’ve been fortunate to learn quite a few the different parts of the business and have seen it from the early days until where we are now. Over 18 years we’ve grown to be a $110 billion business increasing at 19 percent, so that’s great, and we’re just at the early stages of what that business can be. I’m pushing the teams to consistently think about how we innovate faster. How do we think bigger? And how do we support our customers?
As we think about the possible of AWS being a $200 billion, $300 billion, $500 billion business, or whatever size it gets to, we want to continuously think: What are the organizational structures? What are the mechanisms we use? What are the ways that we supported customers, which worked to get us to $100 billion, and may not work at $200 or $300 billion?
Some of that is just reasoning about how we scale those aspects. And how do we think about supporting customers in a large way? How do we think about scaling our services in a large way? How do we think about continuously innovating across many different paths? And as you think about it, we gotta truly innovate along our core — the thing that got us here around compute, databases, storage, and networking. But we besides gotta innovate around AI, around any higher-level capabilities, and analytics.
We besides gotta innovate around helping customers who might be little technically savvy, so they can take advantage of the cloud. They may not be at Netflix-level sophistication, which is evidently a very sophisticated technology team, but want to take advantage of any of the cloud capabilities. I think we’re continuing to think about how we keep pushing that envelope to aid more and more customers take advantage of what we have.
One of the things that I spend quite a few time reasoning about is: how we organize so that our teams don’t lose agility and velocity as we get bigger. That’s any of what I’m reasoning about, and it’s nothing that’s broken today. Instead, it’s kind of like looking around corners to see erstwhile the business is twice as large as it is today, how do we make certain that we proceed to execute and run as fast as possible?
Can I ask about that part of the puzzle? Where does the next fresh client come from?
When you started at AWS they were all fresh customers. Now, most immense companies at least have an thought of what they might do with the cloud, whether they’re utilizing AWS or something else. We have quite a few CEOs who come on here and say, “Look, I request to have multiple clouds so that I can go do rate negotiations with all of them.” Fine.
There is simply a fresh class of companies that assumes they don’t request any software support. They’re just going to hire a bunch of software as a service (SaaS) vendors, and they’ll run their business and usage the SaaS products nevertheless they want to usage them. And it seems very improbable that they will become AWS customers themselves due to the fact that they’ve outsourced a bunch of business functionality to a bunch of another software vendors. I’m just wondering if that’s a fresh class of possible customer, right? That kind of business didn’t be until recently.
It’s true, and I think that there’s most likely subtlety there. So I’ll take a couple of those, 1 at a time. Number one, we do have quite a few large customers that are moving in AWS in the cloud today, and a immense number of them inactive have massive amounts of their property on-premise. And so there’s a immense amount of growth available there. You can even take our largest customers, many of them only have 10, 20, 30, or 40 percent of their workloads in the cloud. There’s a massive amount of growth just helping them get to 70 or 80 percent, or whatever that number is going to be, and don’t even presume you get to a hundred. There’s a immense amount of business there.
I besides think there’s a immense amount of business available with customers that only have 1 percent, or rounding to zero, of their property in the cloud due to the fact that they’re inactive moving on-premise workloads, whether it’s IT or core business pieces. any of it is moving in data centers. any of that is workloads that haven’t moved to a cloud planet yet. Think telco networks, broadly. Most telco networks inactive run in conventional telco networks. There are a fistful of customers, like the Dish networks of the world, who have thought about and have moved to building in the cloud. Since they got to start from zero, and have built it in the cloud, they get the benefits of that agility — but most haven’t.
Think about all of the compute that happens in a infirmary today. It’s mostly in the hospital. And they’re just examples of where there’s an tremendous amount of compute that could take advantage of these broad-scale cloud systems that haven’t yet moved there. So there’s a immense amount of possible in those additional businesses. There’s besides just, as you think about fresh customers, all single year there are a immense number of startups that are created from scratch and they all start in the cloud too. There’s inactive lots of greenfield chance for us.
I think your reflection about companies leaning more into SaaS is super interesting and it’s why they’re specified a focus for us. It’s why we focus on deep partnerships. How do we make certain that AWS is the best place to run SAP, it’s the best place to run Workday, it’s the best place to run ServiceNow, it’s the best place to run … Keep going down the list. And so, those SaaS independent software vendors (ISVs) have always been a truly crucial client base for us.
And increasingly, you see us build capabilities that make AWS even more powerful for SaaS vendors. At re:Invent, we announced a capability called Q Business Index where you can have all of your SaaS data pulled together into a single index that’s owned and controlled by the enterprise, but you can share across SaaS products. I think you’ll see more things like that where we can aid customers not just say, “Okay, my data’s in a bunch of these SaaS islands and I can’t get benefits across them.”
I don’t think customers won’t be an AWS customer, due to the fact that they’re inactive going to have a data lake of their own data, they’re inactive going to have their own applications, they’re inactive going to run their own websites. There are also things that customers are inactive going to want to do. And so I think more of their applications will be in SaaS as opposed to self-managed software, for sure. It’s hard to imagine many customers that won’t have their own compute retention database needs also.
When Adam was on the show, I asked him, “What’s the point of the airport ads? Who doesn’t know about AWS?” And his answer fundamentally tracked with what you’re saying. There are inactive quite a few customers who we request to get reasoning about moving to the cloud, and that’s why there are Thursday Night Football ads.
Is that your answer? erstwhile you get off the plane and you see the AWS logo, you’re like, “I’m going to get that guy?”
I mean, look, you can make that argument for lots of ads. Like, who doesn’t know that Coca-Cola exists? But you inactive see Coca-Cola ads. And so any of it is keeping it top of mind. any of it is besides … If you think about the advertising that we do together with any of the sports networks — whether it’s NFL, F1, or others — quite a few what that does is to aid connect the dots. You may know that AWS exists, but helping see that in a context that you understand, which is football, F1, Bundesliga, or whatever the athletics is, and how we’re helping do analytics for that sport, is 1 of those things that helps customers connect the dots.
And so, it’s not just an ad that says, “Hey, AWS exists,” but it is connecting those dots that says, “Okay, if we’re able to do analytics that can see how fast a football player can run, or see what the chance is that an F1 car can pass,” it helps customers just connect the dots as to where we might be able to aid their business too. It besides opens the door for us to do that next deep dive where we can dive in and realize that. And we find that that connection point is rather valuable even if people know that AWS exists already.
I do love the thought of any CEO coming to you and saying, “I request a win probability metre for my squad all minute of the day in real time.”
Let me ask you about telco for 1 second. Just due to the fact that telecommunications has long been a peculiar fascination of mine. Dish started from scratch. They announced loudly that they were going to usage AWS as their cloud provider, that they wanted to do all the compute they needed for 5G and all that stuff to run that network in the cloud. Compare and contrast that to the another telcos.
When Verizon was launching 5G, for example, they told me that they were going to build a competitor to AWS due to the fact that they needed the compute at the edge to run the network anyway. And they said they might as well just sale the excess capacity in their data centers to customers and say it would have a lower latency, or whatever you get from being very much at the edge. Did that pan out? Or are you saying, “Okay, that didn’t work, and I can go conquer those customers now. I can go get Verizon or AT&T or whoever else on the network?”
Well, Verizon was a small bit different. It was a partnership with us where we were talking about possibly selling any of that compute space together at the edge. I think that technology is most likely a small bit ahead, and I inactive think that there’s an interesting eventual win there. But I think that the thought was a small bit ahead of the technology of truly low-latency compute at the edge, mostly due to the fact that quite a few that latency was taken up in the network, and so it’s hard to get that benefit of a tiny latency gap.
Look, if you go back 15 years, many companies were reasoning that they would just go offer the cloud. It looked like it was easy. And then they said, “Oh, it’s just a hosting thing. I have a data center. I can sale that.” I think most companies today, outside of the fistful of 3 or 4 companies that are truly in the space, don’t think that they can supply a real cloud offering. It’s hard.
There are niche offerings in peculiar slices, but I think increasingly we view this as a partnership chance where we can add value together. So, I think our partnership with Verizon is great. We look at how we can add value together, and over time we’d love for more of the broader network. due to the fact that if you look globally, you’re starting to see another telcos start to thin into this model of, “Okay, possibly more of the core can be run in AWS” … Then possibly that part is, “Okay, that can be run in central data centers,” and so we’re starting to see more core. And then you think about, “Can the radio access network (RAN) be run in AWS? Maybe. Yeah, it can.” And they’re starting to see that part in there.
I think it will be a transition over time. But I do think that as we add more value and show that we can give programmability to their networks, scale to the networks, and show benefits on patching and another things like that where there’s a lot more flexibility there — I think you’ll see more and more telcos leaning into to cloud-based place deployments.
I’m certain your partners at the conventional telco companies appreciate your support in the retconning of their promises around 5G. You’re doing great.
There’s a real divided here. I hope people can hear it. We’re talking about inactive trying to get customers to come usage cloud services. Step one: decision any of your compute out of the basement of the infirmary and into the cloud. And quite a few companies aren’t there yet, and it seems like you perceive that there’s inactive chance there.
Then we’re going to, in a minute, we’re going to talk about AI, which is the absolute cutting edge of, “How do we even run these companies? What do these computers even do? How does the cost work out?” How are you structuring the organization to deal with that split? “Don’t have your own servers in the basement?” versus, “Turn your decision-making over to any agentic AI strategy that we’re going to run for you.”
Well, in any ways it’s a much stronger carrot. If the pitch is, “Hey, run the exact same thing that you’re doing, but do it a small bit more efficiently and a small bit little expensively,” that is little of a value proposition than if you can do something that hasn’t been possible before. And so, I think that’s why many of the workloads that you’ve seen decision to the cloud already are the super scalable ones, or the ones where they request lots of compute, or the ones where they have a truly large footprint due to the fact that they see the wins are tremendous for those types of customers. For a server moving in the basement of a hospital, possibly they can save a small bit of money, or possibly they can save a small bit of IT work or whatever, but the value proposition may not be there unless we can truly deliver quite a few value.
You’re not going to be able to get quite a few the value that’s promised from AI from a server moving in your basement, it’s just not possible. The technology won’t be there, the hardware won’t be there, the models won’t live there, et cetera. And so, in many ways, I think it’s a tailwind to that cloud migration due to the fact that we see with customers, forget proof of concepts … You can run a proof of concept anywhere. I think the planet has proven over the last couple of years you can run lots and lots and lots of proof of concepts, but as shortly as you start to think about production, and integrating into your production data, you request that data in the cloud so the models can interact with it and you can have it as part of your system.
And I do think that that is going to be a tailwind over the next couple of years as people want to have these agentic systems. They want to have their data in a safe environment but integrated into an AI workflow. You can’t orchestrate an AI workflow pointing it on a mainframe. It’s not going to be possible. If you have the data going back and distant to any model, the safety and control of making certain that that intellectual property (IP) stays with you is risky too.
But if you decision the full data into a safe cloud environment, you’ll have a modern data lake that has all your data. Your application will work there, you’ll be colocated with where the model, all the controls, and guardrails can run, and you can have a retrieval augmented generation (RAG) index that’s close to take advantage of all that data — that’s erstwhile you can truly start integrating it into your production applications. And that’s where you’re going to see quite a few the truly meaningful wins, not just kind of a cool, “Hey, that’s neat that I can have a chatbot,” but truly integrate it into how your workflows change and how you can do business changes.
I have seen early signs that, to your question about organization, they’re very complementary. It’s not A or B, it is all pushing in the same place. So we’ll gotta have different capabilities, we’ll gotta have different motions to aid all of that. But I do think that that decision of getting your data into a cloud planet is kind of a essential condition to have a really, truly successful, profoundly integrated AI, I think, into your business processes.
So this leads right into the classical Decoder question: How is AWS structured now? What’s the org chart?
What do you mean? So say more about that. Just what is our org structure?
Yeah. How have you structured AWS? I mean you’re new, so I imagine you might change it, but how is it structured right now, and how are you reasoning about changing it?
Well, I will say that an org structure, number one, is simply a surviving thing. So whatever I tell you present may not be actual tomorrow, and I think you should be agile there. But broadly, how we think about structuring our teams, I think, is beautiful well documented in the manufacture around Amazon. We want single-threaded teams that can focus on a peculiar problem and decision fast. And so what that means is you truly want a squad who can own a problem and not be matrixed across 10 different things where they gotta coordinate a bunch.
In any ways, I think about it like a large monolithic computer program — it’s very efficient as long as that monolithic computer program is small. And as it gets bigger and you have multiple people working on that program, then you get a mainframe, and it’s very slow and you can’t iterate on it or decision fast.
So what you do is decouple and build services that talk to each another through well-defined APIs. And then you proceed to decouple those programs, you proceed to refactor. That’s how to build modern technology systems. And you can think about containers as the current way of doing that, which are small, independently moving systems that can talk to each another through APIs.
Now, if you think about org structure, it’s not that dissimilar from that. If you think about how do you have teams that can run truly fast? There is going to be coordination, but what you want to do is minimize that coordination taxation as much as possible. And so, if you have a well-defined API between them, which is like, “I build a service over here, you build a service over here,” we can innovate. Occasionally our teams will get together and make certain that we broadly know what our imagination is. We want to know what the thing is that we’re moving towards. But then I can go and my service, my organization, or my feature, can run independently and not gotta have coordination.
High level, if the Amazon Elastic Compute Cloud (EC2) squad and the Amazon Simple retention Service (S3) squad had to talk all time they were going to launch a feature to make certain it worked together, we would decision really, truly slow. But we don’t, and so the teams can decision truly fast.
Then we make certain we have … It’s kind of part of the leadership and the product leadership squad to get together and say, “Okay, we think going after this space is super important. And any of that is customers are going onto this usage case, and so broadly we’re going to gotta go after this thing,” but we can inactive then have the teams go out and run fast. That is an organizing rule that … And then there are also parts of the organization where we have teams that run kind of the data centers and another global, and any of those are our separate teams. But if you think about the product and organizing around the product and technology, that’s how we think about it.
This question is always bait for Amazon executives in peculiar due to the fact that Amazon executives are raised in a culture to think precisely in this way and describe the company as a series of microservices. But how is AWS structured?
Just like that. I mean, even more so than Amazon.
Go through it, what are the services? What do you think about allocating the squad for those services?
There are 200 different services, so I’m not going to go through all of them, but that is it. And we’ll continually refactor and re-think about them. From a technology point of view, we think about a compute service. You can think about EC2, and then you can think about EC2 networking, and then you can think about, “How do we make certain that it’s optimized around containers?” And then down at the bottom, you think about, “How do we have teams of 10 to 20 people that are focused on a subcomponent of that, that are full separable?”
We have thousands of developers that are all organized on that principle. Sometimes we’ll decision them around organizationally, but it’s not truly the org structure. The key part is truly ownership at the bottom. The top part is just how efficient you are at management, and how do you make certain that you’re managing the teams well, and doing that high-level coordination bit. That’s actually where you decision around. But at the core, those teams are beautiful solid. As you find a fresh opportunity, you spin up a fresh squad that goes after it and figure out where it makes the most sense in the org structure. But at the core, that is the organizing principle. We have those tiny teams and we proceed to drive them. So that’s it.
And then we organize our sales, go-to-market, and marketing teams separate from that. But from the core product side, that’s how we think about it and it works well for us. I think the positives are … Look, there are pros and cons to any organizational structure from our side. The pros importantly outweigh the cons. From the cons side, sometimes, and I’m certain you’ve heard this criticism or feedback of AWS, which is that sometimes it seems like it’s not perfectly consistent or this XYZ feature is not supported across all single service yet. And that is the downside of that organizational structure — your fit and finish across all single service is not always perfect, and sometimes it takes a small while to catch up to all of those things, which is expected as you have 1,000 different teams run at different paces on different things.
But the trade-off is we get to decision truly fast, we’re super agile, and we can respond to client feedback truly quickly. And I think that is the another secret — that it’s not just an organizing principle, but it is besides that you teach those teams to truly perceive to the customer. I’m certain all leader you have on here says they perceive to their customers, and I don’t believe that they … Amazon does a truly good occupation of actually internalizing that down to all individual contributor, and we think about how we go solve client problems. And erstwhile you’re small, agile, and can make decisions, you can actually go solve client problems truly fast in your area. Those things play on each another and are helpful.
You did start as a product manager. As a product manager-
Technically an intern before AWS launched in 2005.
That’s true. But as a PM, you’re moving any product and you’re most likely reasoning about the client a lot. What were the frustrations you had as a p.m. that you think you can now reduce as the CEO?
Well, it was a very different business back in the day. I was the product manager for all of AWS, so …
And so you inactive are is what you’re saying?
Yeah, exactly. I have the same occupation now. No, and I kid, there were a couple of another product managers at the time too. But the frustrations then and now are besides similar, but different. It’s evidently a different scale that we’re operating at. But 1 of the things I was frustrated at back in 2006 was that I knew a ton of things that we just needed to go deliver for our customers. I just had a immense list and it was all about prioritizing that list, but I want that we could deliver them faster and do more, and even at the scale that AWS is present that’s inactive true. I want we could do more and do it faster, and that’s part of why we focus on that organizing rule of making certain that you can get out of the way of the teams to decision fast. And so, my occupation present is simply a small bit more of, “How do I remove those barriers and aid teams decision fast?” But that’s it.
I think it’s quite a few we want to make certain that we’re innovating, we want to make certain that we’re leaning ahead. any of the challenges we have present are different than we had in 2006. In 2006, we had to answer the question, “Why would a bookseller always run my computers?” And that question, we get little and little today, actually. I don’t think I’ve gotten that 1 for a while.
But now we gotta deal with scale, think about enterprise requirements, and about: How do I meet audit requirements? How do we support governments? How do we think about scale? And how do we make certain that we have adequate electricity in the world? And all of those kinds of questions. But all good problems for us to solve so that we can take them on so the customers don’t have to.
This is the another large Decoder question and it’s going to lead us right into AI due to the fact that I think you have quite a few decisions to make here. Amazon famously has the one-way door versus two-way door decision-making framework. Everyone applies it differently. all Amazon executive I’ve always talked to holds onto that thought and they apply it differently. What’s your decision-making framework? How do you make decisions?
Well, part of my occupation is to make the one-way door decisions. So I think that framework is, it’s a useful 1 to think about. And just to clarify, in case you’re not aware of it, mostly that’s how you go fast. You effort to specify what those decisions are. They can be crucial decisions by the way. I think sometimes it’s misunderstood what are the crucial decisions and not crucial decisions. It’s not that.
You want the people that are owning those teams at the edges of the organization that truly own those products to make crucial decisions due to the fact that they know best about their product. But they’re besides decisions that could be undone if we decide that it wasn’t the right thing to do. And then the bigger kind of, I’m going to go invest $1 billion, or any decision, or I’m going to launch a fresh service that is hard to pull back or is painful to pull back, those are the one-way door decisions that I think we want to have a small bit more inspection on. And even those, though, I think we are trying to figure out how do we make those faster too, and enable a broader swath of people to make those?
But you asked how I make decisions? I think for better or worse, my take is I am rarely, if ever, the expert on any peculiar subject that we’re working on. And whether we’re working on compute or on storage, talking about hypervisors, sales compensation, power contracts that we’re signing, go-to-market efforts, or marketing, I am seldom the expert in the area on those. And so I make certain that I perceive and leave space for those experts who spend all of their days reasoning about that to weigh in as to how they’ve come up with their recommendation, how they think about what we should do.
And then the part that I bring to that is to one, take a view of a non-expert and ask any questions and realize how they’re reasoning about the problem. Then two, aid connect the dots to the another part of the organization that they may not have visibility into and realize if there are trade-offs that they may not have thought about due to the fact that they’re making a marketing decision and didn’t know about a fresh product that we were delivering over there. I effort to make certain that, as an organization, we’ve connected those dots and then ask the right sets of questions. And then if there’s a tiebreaker decision I’ll gotta do it so that we can decision fast. I think the place we don’t want to be in is to sit there and just debate forever. At any point, you request a tiebreaker decision, and that’s what I view my occupation as doing as well.
All right, so I think this does bring us consecutive into AI due to the fact that this is simply a bunch of decisions that everyone has to make and the outcomes are, I would say, inactive uncertain. As an industry, everyone is telling me this is the core enabling technology of the next generation of computing. This is simply a platform shift is the phrase that a bunch of CEOs have utilized with me. Do you think AI is simply a platform shift? Do you think it’s that large of a deal? Or is it just another suite of capabilities that AWS will offer people?
It’s a good question. I’ll start with how I believe that AI is incredibly transformational, whether you call it platform shift or not I can get to that in a second, but I think it’s an incredibly transformational technology that more than kind of … Look, these things come around all decade or so. I think it is 1 of the technologies that can be completely transformational. Whether it’s transforming industries, companies, jobs, workloads, or workflows, I think it has a real possible to have a material impact on all single part of how we think about work, life, user experiences, and the like. I’m a full believer, that that is true. And I think there’s a timeline question: is that going to be in the next 12 months, 24 months, or the next 5 years? But I do think it is going to happen and it’s going to have a real change on quite a few pieces of business.
Platform shift is an interesting question due to the fact that “platform” assumes that AI is not yet a platform and I think that that is simply a more open question. It’s a immense enabling technology. And whether you build on that AI or that AI is embedded in everything that you build with and is simply a core component of what you build with and how you think about … It’s a tool that is truly meaningful and impactful. I think it remains to be seen as precisely what that means, but it is simply a transformational technology that-
Wait, can I make that simpler?
Can I put that on a spectrum for you, just to make this more concrete for the listener?
Do you think AI is more like multi-touch? Or do you think it’s more like the iPhone?
I don’t know if it’s truly like either of those. I would bet that it-
Well, due to the fact that multi-touch is like … You can’t make an iPhone without multi-touch, but that doesn’t imply that we’re all going to start utilizing touchscreens all of the time.
Yeah. It’s not like multi-touch. It’s not like that. I don’t know if it’s an iPhone either, though. It may be more akin to the net disruption. That’s what I’m saying. I don’t know if the net is simply a platform, per se, it’s a shift in how you would deliver an application. So possibly it’s a platform. But I think it’s more akin to where there will be fundamental shifts in how you deliver products, offerings, and services, and how you do your work daily.
So the net has been hugely transformational with how you do your work daily. You utilized to sit there on a typewriter or, I don’t know, compose memos, or do whatever, and now you’re on a computer all day. You’re interacting on SaaS applications, emailing people, or there’s just fundamental connectivity. And I do think that AI is more akin to something like that, where it has that fundamental shift into how you’re going to get work done.
Yeah, I think you and I are both about the same age and you described the typewriter workforce with the same kind of, “I think that’s what it was like.”
Yeah. I don’t know. I never had a occupation like that.
It’s the same for me. I think, “Typewriters… people had them.” The timeline thing you brought up is truly interesting: what is the timeline for this? It’s peculiarly interesting to me due to the fact that I get a bunch of AI CEOs coming on the show telling me what their timeline for artificial general intelligence (AGI) is.
So Sam Altman late said AGI would be possible on current hardware, and OpenAI is making quite a few sound about AGI for a variety of reasons that we can unpack at a later time. Mustafa Suleyman, who is the Microsoft AI CEO, was just on Decoder, and he said, “I don’t think we’re going to get to AGI on current hardware, but possibly within 2 to 10 years.” And he said we’re definitely not going to get there on Nvidia GB-200s.
You run data centers, you have a bunch of Nvidia chips in those data centers, and you are developing your own chips which I want to talk about. Where do you see yourself playing in that debate? Is it, “One of these vendors is going to light up AGI on someone’s data center, and I hope it’s AWS?” Is it, “I’m building this hardware to enable that to happen?” Is it, “This is what everyone’s talking about to goose their stock prices and I just request to sale more capabilities to more customers?”
Well, number one, it’s an impossible question to ask due to the fact that there’s no definition of what AGI is. So erstwhile you scope is besides an impossible definition due to the fact that I don’t know. You can’t specify erstwhile you scope an undefined thing.
What I would say is that I think that it’s just a continuum and I think that AI — we’ll call it AI inference, the ability to go do work — is going to proceed to get more capable over time, and I think that there is simply a long road of this to get much, much, much more capable over time. And it’s going to get much little costly to run over time, which I think then explodes the number of ways in which people will make it useful. Whether it’s moving agents, doing another workflows, or performing long-running reasoning tasks, I think there’s a full host of things that you can imagine. And so, there’s just a continuum of where the things yet land and where you’re able to ask the computers to do more for you at lower costs.
I think hardware platforms are going to play a large part in that. I think software algorithms are going to play a large part in that and you’re going to request both of those. I don’t know erstwhile you scope AGI, I don’t know what that means, but I do think that the next generation of compute will be … it’s going to deliver somewhere between. And whatever the current generation is that we just announced with Trainium 2, and yet with Blackwells and GB-200s, I think we’ll give customers a 2–4x boost in compute capability per dollar. We announced Trainium 3, which will give another 2x boost to compute by the end of 2025.
That is going to aid that goal. You will proceed to get more and more, and you’re going to be able to do bigger and bigger things, and you’re going to request algorithmic improvements as well, which many of the teams, ours included, are very focused on doing.
But just straightforwardly, if OpenAI declares that it has achieved AGI, which it seems very much poised to do, it will have done that on a bunch of Azure data centers. Do you think AWS needs to credibly claim, “Oh, we can do that too,” to compete with Azure? I mean, they’ve defined AGI down, to be clear. But they’re going to say it beautiful soon.
Yeah, I realize there are contractual terms that they’re working through. But they have any motivation for reasons to do that, from my understanding. But it’s not about declaring anything. It is just, “Let’s figure out what you are as a customer.” I am little curious in puffery in the press and more curious in how I can aid customers accomplish actual outcomes. And so it’s fine, there can be marketing statements. They can be like, “I have the biggest compute cluster in the world,” or, “I have AGI.”
Okay, but at any point I want to aid a bank figure out how they can reduce the amount of fraud that they’re seeing, or improve the velocity at which they can approve loans, or whatever the thing is that actually goes and helps the business. I want to aid a biotech find cancer cures faster and better and figure out how they can importantly shrink and or improve the efficacy of what they find.
So those to me are interesting and useful outcomes. And so if you tell me, “Hey, can you aid a client find cures for cancer faster?” Awesome. That is simply a thing that I’m focused on. Was that AGI that did it or not? I don’t know. I’m not curious in that, per se. I’m more curious in, “Can I actually aid our customers deliver value to their businesses?” And a small bit little on, “Can I have a stake in the ground around marketing?” due to the fact that I think, at the end of the day, customers actually care about that first one, not that second one.
I think this leads right into the next part of the AI puzzle that I’m seeing unfold. It’s where should the investment go? Is it training fresh models which might be hitting a kind of scaling law problem, and getting little capable at a slower rate than they were before with all successive model? Or is it in inference, which is what you’re describing? “Hey, we can bring the cost and velocity of inference down on the existing models and make cheaper, better, more cost-effective products.” Where’s your emphasis right now?
I don’t think you can choice 1 or the other. You absolutely … The planet is going to deliver more capable models and they are expensive. They require quite a few compute, and it’s an area of investment for us, and it’s an area of investment for many of our customers. And I think it’s the right area of investment for quite a few those due to the fact that I do think … You don’t get more capable, smaller models if you don’t have the large model to start with. That is just how it works. You can’t come out with something that’s a really, truly powerful tiny model if you didn’t besides build a frontier model, or start with a frontier model. So you gotta have those large frontier models and I think we’re going to request those to be more capable.
There’s quite a few innovation and inference in how you can drive costs down. any of that is simply a systems problem, any of that is simply a hardware problem, and any of that is an algorithmic problem. You can think about model distillation. There’s a full bunch of techniques that you can do to get these smaller, faster inference models, which I think are going to be hugely impactful and crucial to delivering real value to enterprises.
I think you go talk to customers now and they are no longer curious in bright, shiny AI proof of concepts. They want something with a real return on investment (ROI) associated with it. And the ways you deliver large ROI are that you either have more value and/or little cost. I think both of those are going to be crucial to keep raising the level of ROI that you can deliver. So, if we think there is this massive ability to transform organizations, we gotta keep expanding what models can do and decreasing how much they can cost. I don’t see how you choice 1 of those. I think you gotta do both.
If you had to choice one, it sounds like you would choice inference, right? due to the fact that that’s where the products are getting built.
Yeah. Well, what I’ll tell you is, in my keynote at re:Invent, I talked about another thing that I like to do in Amazon, and we do here, which is that we refuse a thing we call the “tyranny of the or,” which is forcing individual to choice A or B stifles innovation. It means that you don’t go out and invent how to do A and B. And so you can’t pick. I’m telling you, it is not an A or a B chance, it’s an A and B, and we gotta push our teams to figure out how to do both, which includes bigger training — and we gotta lower the cost of that, by the way. It can’t just keep scaling linearly, which is all part of the silicon investments that we’re making and networking, and things like that. How do you make the cost to train these truly large models lower, so that you can train bigger models?
And I think we gotta make that investment. We are making that investment and it’s a immense area of chance for us due to the fact that present it’s besides costly to proceed to ramp at the rates of the cost of the infrastructure. That’s a large part of Trainium, investing in how to get the cost down for training. I think the inference side has to drive costs down too, which is incredibly crucial for the adoption side of it. So you gotta do both. It won’t work if you just do 1 side.
I did watch your keynote and you are welcome for that alley-oop on the “tyranny of ‘or.’” I knew it was coming due to the fact that I wanted to ask about Trainium. This is simply a immense investment. You’ve been at it for respective years, you announced Trainium 2 at re:Invent, it has additional capabilities in training and inference. It’s designed to be good at inference, so you can usage the same chip everywhere.
Building these chips is simply a immense investment, and you are up against dedicated chip companies. You’re up against AMD, which is besides making a immense investment. You’re up against Microsoft, which is making its own investments. You’re up against Nvidia, which is the leader and has a immense head start, not only in the chips but besides in the software ecosystem around the chips. What do you think about that competition and that investment?
It’s little a competition and more an addition of choice. I don’t think it is GPUs or-
Oh, by the way, I forgot Google. I should most likely point out that Google has an advanced data center and AI capabilities.
Yeah, Google does, that’s right. And so it turns out we’ve been making chips now for over a decade. So we’ve been making silicon chips, our own customized silicon for more than a decade. We’re actually … we have 1 of the most experienced teams in the manufacture doing this, and so it’s not a fresh thing. It’s not like we dove in here and said, “We have no thought what we’re doing,” By the way, any of those others are learning it for the first time. Not Nvidia of course, or AMD, and Google’s been making chips for a small while too. I think Microsoft is beautiful fresh to this space. But we think that that is simply a large advantage for us as we realize how to do this at scale, and we realize how to do it in the cloud.
I think we have any advantages in that we don’t gotta do it for a broad set of customers. We gotta deploy our chips in precisely 1 environment. We gotta deploy them in an AWS data center. We gotta deploy them in precisely 1 server, or we don’t gotta support a full OEM infrastructure, a set of different drivers, or a bunch of different things. It’s just in our environment and we know precisely what that’s going to look like. And we think it’s a choice. We don’t think that it has to meet all single usage case for all single customer.
We think that Nvidia GPUs, AMD GPUs, and others are going to be super interesting. They have good platforms. Both of them have very good teams that are executing really, truly well, and I think they will proceed to do that. I don’t see any reason why they wouldn’t. We plan to be a large partner of theirs for a truly long time and support that and offer it to customers erstwhile it’s the right technology choice for their usage case.
We think that we can offer interesting choices, and we’ve done it with Graviton. We’ve proven that we can launch a processor at a broad scale that is very useful for a set of workloads, a broad set of workloads for our customers. And in Graviton’s case, it doesn’t mean we don’t buy a ton of Intel and AMD chips and offer those to customers. We of course do, and those are increasing businesses for us as well. It’s just more choice. And we think that choice makes AWS a more attractive platform for customers due to the fact that they have more choices than they do another places. That additional choice is nice, and part of that choice is we want to truly thin in and make certain it’s the best place to run Nvidia GPUs, AMD, Intel, and others.
But it’s a large chance for us. And if you do think, which we do, that AI is going to disrupt all of those different industries, it’s a massive chance where it’s not 1 player that is going to be the only compute platform that all of those things run in over time. We think that we have an chance to build any of that and supply differentiated choices for customers who choose to run AWS.
Chips and chip investment is simply a long-term decision. You’re making decisions now and allocating capital that might not pay off for a decade or more. Do you think that model training is hitting a scaling limit? That it’s going to plateau the way that any people are saying it’s plateauing?
I think people like to talk about scaling laws due to the fact that again, it sounds fun to talk about. But I think that it most likely just means there should be more levels of invention. I think if you look over any technology ramp, you see 1 peculiar method ramping up like this and then it slows down, and then individual says, “Oh, how about you effort this?” And then it goes back up again, and then you effort something else. And so there’s going to should be software and algorithmic changes. I think it’s not a blind dump of more data, add more compute, close your eyes, and then you get a bigger model next year. You’re going to request smart people looking at it, driving it, and figuring out fresh ways to aid that. But that doesn’t mean that you’ve hit a limit. I think it’s just that you’re going to gotta keep innovating in different ways.
Think about, number one, how long, and it was longer than a decade, that people were saying that we were hitting Moore’s Law of scaling limits. That was, “Can you take 17 nanometers and make it 15 nanometers and 13 nanometers?” And you’re saying, “Okay, there’s going to be a limit.” They had to figure out the technology to get past a couple of those. I remember somewhere around 10 nanometers, people were like, “I don’t think you can get past this,” and now we’re building three-nanometer chips. And so you keep getting smaller due to the fact that there are fresh technologies in there.
You had to figure out how you deal with interference, and you had to think about actually stacking the memory, different structures of the chips, and another things like that — but you work through those. In the meantime, you kind of figured out how to do more compute on an accelerator like a GPU, which then gave you a immense step change in compute. And so, no longer are people worried about whether we are hitting the limits of what a 17-nanometer Intel chip from 10 years ago is doing, right? Now we’re orders of magnitude more compute than that.
Well, hold on, hold on. I mean, this is the real limit. 1 company figured that out. Taiwan Semiconductor Manufacturing Company (TSMC) figured that out utilizing an EV device from 1 company in the Netherlands. And they’re the supplier for everyone, which means you are now asking TSMC for capacity in competition with Nvidia, Apple, Qualcomm, AMD, and even, to any extent, in competition with Intel, right?
They figured out parts of that. I mean, they figured out the layout chip. And by the way, [TSMC CEO] C.C. Lei and the squad did a fantastic occupation of figuring it out. So yes, but the planet figures it out, right?
But Intel famously did not figure this out.
I mean, that’s where they are right now.
I’m saying right now the bottleneck in the chip industry, in the investment, is 1 company can supply this product. Is that something that you actively think about? Like, “Do they have the capacity to let us compete?”
I mean, they’re making lots of investments and I think they’re scaling. I think others are looking to catch up in that space too. They have a large lead, and this is besides actual in technology and has been for a long time. individual jumps ahead and figures it out, gets a lead, and it’s a benefit for them for a while and others catch up. I think you can look at any of the advanced Bandwidth Memory (HBM), and any of those another fabrications that are coming up, and they’re catching up and uncovering another fresh ways to do that. There will be another inventions that leapfrog over time. But obviously, fabs are hugely capital-intensive investments. And so, I am certain that others will yet find fresh and different ways to innovate around that too. It has always been actual in technology.
Are you making any bets on any non-TSMC fabs?
I wouldn’t have anything to announce there, but we partner with lots of folks. We partner with Samsung, Intel, and others that have their own fabs as well, and buy lots of another stuff from them. From memory to CPUs, we buy parts from lots of different fabs around the world.
The another large constraint is power. You have said 2 to 3 generations from where we are in AI we’re going to request 1 to 5 gigawatts of power, about a average city. This led you to talk about atomic power and how we’re going to request that. That’s a large deal to say, “Okay, we’re going to request so much AI capacity that we’re going to build atomic power plants.” Microsoft and another companies have said the same thing. Is that inactive where your head is? This is going to be so successful that Amazon is going to effort to build any power plants?
Yes. It is. We’ve made crucial investments there. And that’s a scope of things, by the way. It’s a portfolio. This is not a fresh plan for us. Over the last 5 years, we have commissioned more renewable power projects than … Each year for the last 5 years we’ve commissioned more than any company in the world. And that’s bringing on fresh power into the grids, and whether they’re fresh solar farms or the fresh wind farms, and now we’re adding atomic to that. So it’s just a portfolio of that. I think the planet is going to request more carbon-free energy, and compute and data centers are a large condition of that. We are pushing hard to make certain that the planet has adequate sources of that. I do think that atomic power will be an crucial component of that plan over the next couple of decades.
And so, we are excited about tiny modular reactors. I think that it’s a technology that’s a small ways away. By the way, it’s not a solve for the next couple of years, but past 2030 and beyond, I think it could be a very crucial component. One, you can actually put it close where you request the power to be.
Another of the bottlenecks that we run into is around transmission. It’s not just power generation, but it’s transmission. So you can have a solar farm out in the desert, but if you don’t have transmission to get it to where your data centers are, then it doesn’t do quite a few good. Those are both problems that request to be solved. And it’s not just data centers, it’s electrical cars, it’s electrification of all of our businesses. There’s a bunch of these things that are going to request to happen, and so I think atomic power is going to be an crucial part of that, and tiny modular reactors.
I think the world’s going to gotta build more of these large industrial-scale atomic plants as well. I think quite a few people’s heads are in the “That was scary back in the ‘50s erstwhile the technology wasn’t as safe.” Today, it’s a very safe, scalable technology, but it’s something that we gotta keep spending on and scaling.
We’re going to have you back for another full hr on atomic power plants. That’s a full rabbit gap that I want to talk about at any point in the future. But we’re moving out of time here. And I just want to ask the biggest question of all. This is simply quite a few immense forward investment. You’re designing chips, we’re investing in TSMC’s capacity. We’re talking about atomic power plants, we’re building bigger data centers. There’s an $8 billion investment in Anthropic to aid build a data center and then run Anthropic and Claude.
When is any of this going to make a dollar? You request a product in the consumer or enterprise marketplace that throws off adequate margin at adequate scale to fund all of this investment and inactive make money for the people making the product. And ideally, the people paying for the product are utilizing it to make more money on the another side. The economics of this are inactive very unclear to me unless you are Nvidia. erstwhile does all of this make a dollar for you?
Yeah. Well, AWS is simply a nice, profitable business for Amazon.
Right, you’ve got the margin to spend on it, but at any point, it has to return.
I think, look, and for customers, they’re increasingly looking at it this way. It’s not just us. And I said this a small bit ago. If you talk to customers they are very focused on how they can have ROI-positive AI projects. I think the cloud has already proven to be ROI affirmative across a broad swath of industries. We’re moving your data to the cloud, your compute to the cloud, and you gain agility. And so I think we’ve proven that we can deliver large ROI for customers in moving to the cloud broadly and taking AI aside.
And so, what we’re increasingly seeing customers say is, “I want to see the ROI of these AI projects.” And I do think that that is an crucial shift where it is not just the cool, it’s not just the shiny object factor, it is a, “How do I make certain this makes sense?” And we are spending time with customers reasoning about that. How do you work through the usage cases that are enabled present that can deliver real value? any of those are broadly reported around things like modernizing your contact center, and we think Connect is simply a large offering for customers to do that. We’re actually seeing a immense number of customers decision to Connect in a cloud contact center to take advantage of many of those AI capabilities. You see any of that in optimizing your back-office projects.
And I think increasingly, as the agentic workflows truly get much more powerful, and as we think about collaborative agentic workflows and longer moving agentic workflows, you’re going to see more and more value come up through these. As the models get more capable you’re going to see more value coming up through those. And so I think it’s on us. It’s incumbent on us to make certain that these are very profitable for end customers to go and implement.
But let me just put that in a framework that makes it possibly a small bit sharper.
You’ve been at AWS since the beginning. AWS started, and I’m going to flatten this narrative, you can correct me for it being a small besides flat, but just in the flattest possible way: Amazon is building a bunch of these services. “Hey, we have excess capacity. Hey, we want to build microservices for our own components. We can resell those.”
So you get a bunch of benefits along the way of just building Amazon, and then you can turn that into a business. AI, right now, feels like there are a bunch of ideas for products that might be useful. Inside Amazon, outside of Amazon, for AWS’s customers, whoever, but it requires a massive amount of forward investment.
It’s not just, “We’re kind of doing it anyway.” It’s much more, “Hey, there’s a immense chance here. We request to leapfrog ahead and possibly get any more customers.” Or possibly there’s a platform shift or whatever it is. We all see the immense promise that is happening at a subsidy, and that subsidy seems dangerous.
It’s not the right characterization of it. So there are a couple of things I would say. Number 1 is that AWS was never about excess capacity of Amazon. Just like math doesn’t work. You can imagine that I’ve heard that narrative, it sounds nice. And as shortly as Christmastime comes around, if I gotta take Netflix’s servers distant so that we can support retail traffic, that doesn’t truly work as a business. So that was never the idea, intent, or goal of AWS.
And we built the businesses from scratch. They weren’t reusing Amazon components. We learned from that. They’re an incredible early client to learn from the components that they would need. But we built them from the ground up to support a broad scope of customers. AWS itself was a large investment by Amazon to go after a broad fresh business. As you think about it now, we had Amazon as a large client of ours, for sure, and they were a super helpful client for us to learn about what large enterprises would request from services like AWS and they proceed to be.
I think AI is not that dissimilar. Amazon needs AI. You mentioned that you watched my re:Invent keynote, Andy was up there for 25 minutes talking about all of the cool things that the remainder of Amazon is doing with regards to AI. And you’re talking about Rufus, you’re talking about how we’re reasoning about our supply chain and fulfillment centers, and across the full scope of … And Alexa. That business desperately needs AI capabilities to, again, reimagine our business, get more efficiencies, and deliver fresh experiences for customers. Amazon is client number 1 for a bunch of these capabilities. So if AWS can build them and Amazon can take advantage of them, that’s fantastic and both of those things are true.
So yes, it’s a large forward investment, but we besides have Amazon inactive utilizing them, and we are in a different place now. erstwhile we started in 2006, we had zero external customers, and we now have a million external customers or multiple millions of external customers. That is simply a immense client base that is ready, willing, and excited to buy and usage the products that we have. So that investment is simply a forward investment, but you besides have a truly large base that you can amortize it across and go offer it to, which makes that investment thesis a small bit easier to get over.
All right. So I’m going to ask you the same question again to wrap up with all this context. erstwhile do you think all this investment will become ROI positive?
I think it’s a affirmative ROI. Well, it depends on what you mean by ROI positive. I think there’s quite a few investment in the world.
Right. But this is simply quite a few investment in AI across the industry. erstwhile do you think it’s going to start returning?
I mean, if you think globally, I think it’s ROI affirmative now. I think the question is erstwhile does it become more evenly distributed? Look, I think the hardest question of that, honestly, is for the model producers. I think that’s the single hardest question. I actually think today, or if not today, very soon, it is going to be ROI affirmative for the broad swath of customers utilizing AI and building it in, like banks, insurance companies, pharmaceuticals, and others. You can make that ROI-positive communicative today, and I think it will proceed to get better. And I think for infrastructure providers like Nvidia, of course, it’s very …
I think the question is erstwhile does … The folks who are making the immense investments are the ones who are building foundational models from a software position and then reselling those foundational models. It’s a good question. I don’t know the answer to erstwhile that investment kind of full pays off for an OpenAI or an Anthropic. I think Amazon and Google most likely have a different math of erstwhile we can make those pay off due to the fact that you get interior usage of them from your own use. I don’t know that. But there’s quite a few smart people investing in, continuing to put investment in a broad swath of AI companies. And you gotta believe, which we do, that there is simply a massive economical benefit from many of these AI capabilities that are orders of magnitude bigger.
I do think it truly plays into that math equation. As inference gets cheaper and more capable there are multiple orders of magnitude more inference to be done. And that is erstwhile it yet starts to pay off, I think, for quite a few those model providers, and in a huge, massive way.
All right, you are clearly in the weeds of all these products, which is fun to hear. Let’s end here. Last question. erstwhile you’re trying out all these AI products, which is the 1 that you usage that makes you think, “Okay, is this investment worth it”?
That’s a good question. I don’t know if there was any 1 product that I got excited about. The first product that I always utilized that I said, “Hey, I think this is real,” is just like everybody else. I think ChatGPT was just a transformational product. It was a large UI and it truly unlocked for everyone what was possible. So the first time that I truly realized that this was going to take off. We were making investments internally, but I think we were hopeful that they would get there. I think that’s the first 1 that I utilized that I truly understood.
Now it’s hard due to the fact that I usage thousands of them and I think all of them are truly cool. And I think there are quite a few startups from people that are building AI products. People who are making fresh proteins — which is incredible — folks like Perplexity who are making search engines that are much more interesting, contact centers, and banking applications. There’s a full host of them now that are incredible. I think Amazon makes some, and many of our partners make many, so those are all incredible. But it truly was, just like the remainder of the world, I think ChatGPT was the first 1 that truly helped solidify it.
Got it. Very diplomatic answer. Matt, this was great. You’ve got to come back. I truly enjoyed this conversation.
Great. Thanks for having me.
Decoder with Nilay Patel /
A podcast from The Verge about large ideas and another problems.
