The text below is a transcript of the audio from Episode 49 of Onward, "Data centers will be the largest investment in US history, with Kervin Pillay, former CTO of Cisco".
Disclaimer: This transcript has been automatically generated and may not be 100% accurate. While we have worked to ensure the accuracy of the transcript, it is possible that errors or omissions may occur. This transcript is provided for informational purposes only and should not be relied upon as a substitute for the original audio content. Any discrepancies or errors in the transcript should be brought to our attention so that we can make corrections as necessary.
---
Ben: Hello and welcome to Onward.
My guest today is Kervin Pillay, who was the CTO of Cisco’s Automation Group. Kervin has been building data centers for over 25 years on nearly every continent in the world. He has a rare combination of technical and operational knowledge.
Before we get started, I want to remind you that this podcast is not investment advice, it is intended for informational and entertainment purposes only.
Ben: Kervin Pallay, welcome to Onward.
Kervin Pillay: Thank you. Thanks for having me.
Benjamin Miller: So I'm excited to have you because, uh, data centers are a con combination of the two things we focus on, which is tech and real estate.
Kervin Pillay: Mm-hmm.
Benjamin Miller: And here you are an expert on, on, um, data centers. So let's, let's just briefly before we get into it, like why are data centers a big deal? Like how, how much demand, how big could they get?
Kervin Pillay: Well, I mean, if you look at just the rise of AI and where it's gone and the, the new use cases that are coming out for the use of ai, of those depend on a data center somewhere. So it's almost impossible for us to get to the, call it the promise of ai. having these data centers to be able to, uh, consume, produce, um, and modify all of the data that's coming out of AI so that it's consumable by end users, by businesses, by, you know, everybody that needs to get access to it. So what that means is, like, like the internet, um, you know, it started small and it got to the point where it was ubiquitous. And ubiquity means that it has to be everywhere. It has to be accessible. And at some point it becomes almost like a utility, right? And that's where I think data centers are going.
And if you consider that as the, as the paradigm of where we are gonna end up, then it's inevitable for data centers to be everywhere and there to be many, many, many data centers, much more than we have today. I guess that's the overview of why I think, uh, in whichever way that you look at this, data centers are gonna be a super critical part of the infrastructure going forward.
Benjamin Miller: Well, let's put some, uh, numbers on it. So, uh, I saw numbers saying something like a trillion dollars of data centers per year starting by 2028. Uh, do you have any stats on this?
Kervin Pillay: Yeah. So, so what do we know? We know what's been, I guess, publicly announced and what's been, uh, forecasted by all of the, the, the really big companies that are, that are building. know that, for example, the Stargate Project is, you know, gigawatts, probably 10 to 20 gigawatts worth of data centers. Um, we know that in the Middle East, they are trying to build AI hubs to encourage more AI in the region. know that there's going to be the rise of sovereign ai. Um, why? Because everybody is scared of what AI will do with their data. So sovereign AI is a way for, uh, governments to protect their data, but still have access to ai. So if you consider that, you know, 200 plus countries are building data centers of their own and you're getting hyperscalers building data centers for everybody to use a trillion dollars is a, is a small amount of where this thing could be. So, you know, a trillion dollars sounds like a lot. Um, the biggest impediment is gonna be the, the time to get to that, uh, to that outcome. You know, depending on how you build data centers, it could take anything from six months to five years. So saying that there's gonna be a trillion dollars worth of data centers in 2028 means that maybe the capital is committed, but not all of that will be built, and not all of that will be ready to be consumed in 2028.
Benjamin Miller: Yeah, I, I did a little bit of prep and I, uh, found that the interstate highway system built under Eisenhower cost in today's dollars, a half a trillion. So you're talking about building like two or two to five interstate highway systems just to build the underlying, you know, highway system for, for ai.
Kervin Pillay: Yeah. And, and I think you gotta look beyond, if you look at the effect of the highways,
Benjamin Miller: Mm-hmm.
Kervin Pillay: gave rise to, uh, automobiles and motels, which didn't exist before. And that's not counting in the half a trillion dollars that you're talking about. So
Benjamin Miller: Mm-hmm.
Kervin Pillay: same with data centers. There's gonna be a demand for power, which is going to be probably another trillion dollars by itself. of the equipment that goes in the data center itself, which is going to be refreshed. And if you look at the current pace of things, somewhere between three to five years, you know that, that explodes as well, where you're getting the infrastructure piece. And you're getting all of these ancillary investments that need to happen in order to make this data center useful.
So imagine that you build all of these data centers and there's hundreds of gigawatts worth of data centers, and there's no connectivity to the people that are using it. Completely useless. So fiber rollout, broadband rollout, um, faster wifi, um, you know, better chips on end devices, all of those things are gonna require investment in order to actually get the, the, the promise of this, uh, of what we think this is gonna be.
Benjamin Miller: Okay. So lots of questions now. Um, I do want to, before I ask you about your background, so people understand why you're a great person to speak to. It just, can you just say what a gigawatt is? Like some measure of the, because people say gigawatt and it's like, I don't think most people know what a gigawatt really means in terms of like how much power is that.
Kervin Pillay: Yeah, I mean, you know, you could, you could say that a gigawatt, um, would power maybe California for, you know, less than a year.
Benjamin Miller: Okay. So, and that's one data center,
Kervin Pillay: That's one data center,
Benjamin Miller: or actually Stargate, you said it was 10 to 20 of those.
Kervin Pillay: 10 to 20. So,
Benjamin Miller: I.
Kervin Pillay: so if you think about, uh, you know, the average consumption, you know, using your, your, your tv, uh, you know, doing stuff at, at, at home, you know, that doesn't equate to anywhere near what the, the, these data centers consume. So typically you're gonna see, and when you look at where these data centers are built, they built close to power stations. And, uh, in rural areas, typically they, you know, somewhere between 10 and a thousand times more power than that community consumes or that that state even consumes. So if you look at the really big data centers, they are the biggest consumers of power in that state or even in that country.
Benjamin Miller: Mm-hmm.
Kervin Pillay: that's, that's a, you know, if, if people had to picture what you would do with a gigawatt and we thinking we, we talking hundreds of gigawatts, uh, you know, in, in a, in the next couple years, it could power the earth for the next, I don't know, 5,200 years.
Benjamin Miller: Okay, well, we'll get to the double-edged sword of that. 'cause I think there, there's some questions about the environment, uh, that I want to follow up on. But let's, let's go back to, so, so why are you an expert on this? Have you been, have you ever built data center Kervin?
Kervin Pillay: So, you know, I've been, I've been building data centers for about 25 years now before data centers were actually cool. To build. And the very first data center I built was in a container without access to power, and it had to be run by a diesel generator 24 by seven in order to keep it running. you know, that was in the nineties. I then worked for company that had the largest data center footprint by square foot, through, you know, through the 2010s. Uh, that's not a good measure, by the way, anymore of, of who has the biggest data center,
Benjamin Miller: Mm-hmm.
Kervin Pillay: not by square foot because that's, that's not how people consume it anymore. It's by, you know, the actual usable it load, which is another dynamic that we should, we should talk about. Um, because there's a lot of wastage when, when we take the power from the power, um, uh, station and consume it. In the data center, there's a, there's a bunch of, um, wastage that happens,
Benjamin Miller: Hmm.
Kervin Pillay: in order to power the, the. The servers that are inside the data centers. So, you know, I've been building these data centers from, uh, before they were called data centers.
I used to build them for telephony exchanges back in the nineties and two thousands, uh, before the rise of, you know, mobility and cloud and where data center overtook telephony. At one stage, you know, data centers were built for telephony. So if you look at the DA largest data centers in the nineties, you know, as at and t and Verizon, and they had the biggest data centers because they had to support all of the, these, uh, voice calls far overshadowed now, um, by data usage and video usage.
So, um, you know, the transition, I've been through the transition of telephony, mobile, uh, data, um, video, uh, and now ai. all of those things required a slightly different way of thinking about data centers, building data centers, financing data centers, uh, and imagining how these data centers are gonna be in the future. So in a nutshell, you know, I, I've, I've watched the rise of data centers over the past 25 years.
Benjamin Miller: And you've, your last job was at Cisco, right?
Kervin Pillay: That's right. I was, I was CTO at Cisco and I was responsible for, um, automation for service providers and service providers, uh, you know, exist in every country. But essentially bringing automation and AI into what was a legacy environment of service providers in, in the, um, in the world that they used to to live in.
Where now they've been, uh, relegated to providing bit bugs.
Benjamin Miller: So what does it take to build a data center? Maybe you could, you could take us through the process, um, in the United States if you were to say, okay, if you're gonna build, I don't know if a watch the right number, but you pick a large number and say, okay, um, you and I are gonna fund it and we will, we'll put aside the financing for a minute.
You and I are gonna build data center. How do we do it? Kervin?
Kervin Pillay: yeah. So, you know, there's, there's a couple of steps that you need to think about as you are approaching this data center Bill. The, probably the most important thing is you cannot have a data center without power, as you mentioned. And, you know, for simplicity, let's take a a hundred megawatt data center. I know you said financing aside, but you know, that runs into few hundreds of millions of dollars just for, for perspective to build a hundred megawatt data center. so what will you need? First, you will need access to the power. first hurdle that you're gonna come up with is. you go to a power producer and say, I want to consume a hundred megawatts from your, uh, from your, uh, power grid, they're gonna say, well, number one, we don't have that much spare capacity lying around, so we're gonna have to build it for you.
And that may take a couple years. Number two, depending on where your data center is, we're gonna have to transmit that power through the grid, and the grid cannot support that type of power. And as you know, in the United States, you know, the grids have been aging for quite a while and they've been multiple different, uh, I guess, ambitions to try and modernize the grid. And, you know, you still see, um, uh, overland wires. You still see, uh, forest fires as a result of the aging infrastructure.
Benjamin Miller: Mm-hmm.
Kervin Pillay: it's, it's important that when you get this power, you also need to be able to get it to where you want it to be. it means that you, you are, you are relying on the power producer to. Uh, transmit that power to you via a grid of some sort to a location that you decide. Then you've got the next problem, which is, is the location that you've chosen suitable for a data center? Why the environmentals around a data center are anything we've ever seen before. It generates an immense amount of heat. It, the, uh, floor loading is really, really high, which means that you need to worry about, um, what's happening underground in order to support this immense weight that you want to put on the ground at that place. And as data centers become more compact, which means that they consume more power in a smaller space, it actually makes the problem worse. So you can get smaller data centers consuming a high amount of power, but the land can't support it. So then you have to, um, um, figure out how to essentially fortify the land in order to build this data center that you want to build. Okay, so now you're at the point where you've got the power. You've, um, you know, you've spent probably, I don't know, six months to a year getting the environmentals approved by, uh, local authorities or potentially national authorities in order to build this.
And you're ready to, um, put a spade into the ground. Essentially, at that point, you have a couple choices to make, which is what type of data center are you wanting to build. Now, you know, let's, let's have a little bit of a history lesson here. When you were building a data center in the early two thousands, a 10 megawatt data center was huge. It was unheard of to have a 10 megawatt data center. Like nobody thought that that would. Ever we would ever need more than a 10 megawatt data center in a, in, in one location. Um, they obviously didn't plan for, uh, the explosion of mobile, the explosion of cloud and explosion of ai. So we get to this point where you've got, um, designs that were good for the nineties and good for the two thousands, those companies are trying to essentially leverage their r and d that they had, which is 20 years old in order to deliver something that you need today. And what that hap what happens then is you, you end up with this huge sprawl of data center that takes up, uh, you know, way more space than you think it does. And, and, and there's, there's quite a few Google maps, pictures of data centers, uh, in the Midwest. And if you look at the size of those things, it's, you know, larger than farms, right. Um. So now you've got essentially old technology trying to fit into a new paradigm. and then you run into the, the manufacturing concern, which is, how do I manufacture that amount of stuff, which requires steel, it requires aluminum, it requires, uh, cabling, it requires cooling, it requires, um, you know, plumbing of the scale at, at which these manufacturers have never seen before. And you see, um, that some of these manufacturers are, you know, in, at the point where they are trying to launch new ways of building in order to become more efficient. But essentially if you walk into a data center today, is, you know, 15 to 20 years old. So your challenge with this money that we have is trying to figure out how to make the most use of this money, how to be most efficient about it. And the second problem you're gonna have is, um. you try to do many, many things with the data center, you end up with having to make really tough decisions of segmenting the data center to support these different workloads. So, lemme give you an example. Um, a data center built for, um, mobile data very, very different in design, in format, in consumption, from something that's a hyperscale cloud data center. Um, you know, a, a mobile data center, uh, is not at peak utilization all the time. So you get peaks and troughs and at some point you can turn off parts of the data center, which is really, really efficient. But if you're running a hyperscale data center, these things are running at, you know, 40 to 70% utilization all the time. Okay. Which is a very different. and cooling problem that you have. So you need to extract all the heat from it, which means that the design of the data center needs to change based on the fact that you're building a hyperscale data center and then you transition to something like an AI data center. And when you're doing training and you, you, and these are publicly available, uh, stats where, uh, meta was training LAMA for somewhere between three to six months. That's a hundred percent utilization for six months. And that's a huge difference to, you know, a mobile data center or a mobile data data center that's using, you know, 20 to 30%, uh, utilization. So these things end up with problems that you never had before. How do we extract the heat? How to build racks so that they can consume the power that you need. So, um, you know, do the cables overheat. What is the, uh, fire suppression that you need? It's very different to what you would need in a, uh, a, a traditional data center. So these are, these are design decisions that you need to make way before you even think about putting a spade in the ground. And we're not even 50% of the way there. So you have the data center design, then you go to the power producer and say, this is how I'd like to consume the data. typically what's gonna happen is they're gonna have to build an entire microgrid for you, multiple new substations in order to, to deliver the power from where the power, uh, is produced, to where you're going to consume the power, means that you are running, you know, a completely separate grid or you are overloading an existing grid. you're building substations in order to get that power and consume that power. Um, and at that point you've got, um, you know, the manufacturing that's starting. You've got the construction that's happening, and if you, you're talking about traditional data centers, it's brick and mortar. So this is, this looks more like a real estate, uh, deal at this point where, you know, you building buildings, it may be, you know, 3, 2, 3, 4 stories high. Um, each of those floors though, need to be fortified because you've got, you know, tons and tons of, um, of equipment that need to be housed on these different floors, which means that it's not a typical, even a commercial data center more, it looks more like a, like a factory floor a, than a, a commercial real estate build.
Benjamin Miller: Mm-hmm.
Kervin Pillay: and as you can imagine from your background, uh, it gets exponentially more expensive when you start to, uh, put more and more steel into the, into the building in order to make sure that it, it, it can carry the weight that it needs to. Um, so that's happening in parallel while the data center, um, racks and the equipment are being built. And at some point, um, you know, the shell is built, the power is available, and then you start to, to bring in all of the internal data center components that you need. Typically the cooling and the power distribution first, and then the, um, and then the rack, and then the equipment. And lastly, you get the network equipment that comes in to be able to connect this
Benjamin Miller: Mm-hmm.
Kervin Pillay: into, into the internet. And that's where you, you end up with the final piece of this, which is, how do I connect this data center to the internet and who's going to use it? at some point, um, all of the data center providers are thinking about making this data center available to everybody on earth. order for that to happen, the data center actually needs to be able to connect to the undersea cable network. In order to be most effective. So somebody in Europe could use a data center in the US for example, and vice versa. Um, so, you know, typically this is a somewhere between a six month to three year project, um, before you can even get any revenue in the door. And this is part of the problem of building data centers, right? It's um, in order to get revenue, you have to have a phenomenal amount of investment first you can start to realize any revenue coming in the door. And at that point, you know, negotiations for the optic could take anywhere from six months to a year. the, the integration of whoever's using your data center could take another three to six months. we are talking about, you know, somewhere between four and five years from when you think about building this, this project to when you actually start to see revenue. And I know that one of your, your further questions was, well, why aren't people making money now? And, and, and that gives some sort of clue as to why.
Benjamin Miller: Mm-hmm. Mm-hmm. Um, how is an AI data center? Different. I mean, I know you're saying it's denser, more power, more expensive. I mean, it's a GPU versus CPU. What, what does that mean in practice?
Kervin Pillay: yeah. So, so let's talk about the, call it the latest generation. And, and, and for simplicity we'll talk about the, uh, Nvidia Rx, um, 'cause it's something that everybody knows. So they produce something called an, uh, NVL 72. Uh, and the NV L 72, uh, has two unique constraints. Number one, it, it, requires somewhere around 150 to 200 kilowatts per rack. Which is if you look at, uh, existing data centers, they were maximum 20 kilowatts per rack. So we are talking about, you know, five to eight times more power than they need. And secondly, they need to be water cooled. And previously, um, these data center racks were air cooled, which means that the air blue through the server and you had, uh, things that look like household, um, air conditioners, extracting air from the data center and cooling it and, and, and pumping cool air back in. this has to be completely water cooled, which means that you need to have, uh, essentially water plumbing to every single rack. that all the way back to the outside heat exchange. It so cool the water back down and then pump the water back in through the server so that it can cool the server. this is a fundamental shift in how data centers are built.
So, uh, where previous data centers were built for air cooling only, a water cool data center is a fundamentally different design. And you see quite a few different approaches as to how to approach that problem
Benjamin Miller: Mm-hmm.
Kervin Pillay: water cooling. This, this AI data center. Um, and once you have that, you, you've got the, the, the, the startup problem, right? So these things consume, uh, you know, call it 150 kilowatts per rack. If you try to start all of these things up at once, you literally will overload the, the power producer. So you cannot turn every single machine on at once, which means that if you have a brownout, um, you have to have a controlled shutdown and restart of these data centers so as not to cause a brownout essentially for the entire county, city, state. um, whatever may be affected by these things. Um, and you know, tho that's, that's where the problem starts, right? Where, um, the utilization of this, the power utilization, the cooling utilization, um, the designs that you need to, uh, now implement for liquid cooling looks completely different to what hyperscalers have been building before.
Hence, um, when you see all of these new data centers being built, they specifically say that they're building an AI data center because it is a different design.
Benjamin Miller: So this must mean a lot more, uh, wastewater. And you said wastage was the big, big challenge. Um, can you talk a little bit about the, not just the environmental impact, but just the practical constraints of needing that much water? Having that much heat to dump into the area noise. I mean, it's, it's, I often describe these investments like we've invested in a few as like investing in like an aircraft carrier.
Kervin Pillay: Yep.
Benjamin Miller: They're, as, you know, $2 billion. They're $3 billion. They're as big as an aircraft carrier. There's complicated and um, and so their impact to the region is just enormous.
Kervin Pillay: Yeah. So, you know, to, to go back to a previous point, that's why environmentals are so important. Um, so what do we have to worry about? We have to worry about the amount of heat and cooling that you need out of this data center. And of course, the noise and the noise is twofold. Um, first dimension of the noise is, uh, the machines actually running themselves creates a huge amount of noise. That means that they cannot be near residential areas, or at least they cannot be close to residential areas because you'd hear this hum of the data center all the time. The second thing is centers generally have a very, very high uptime. So we talk about uptime in, uh, in terms of percentage uptime, uh, and it's, you know, four nines, five nines,
Benjamin Miller: Mm-hmm.
Kervin Pillay: That means that it, and if you talk about a five nines data center, it cannot be down for more than 36 minutes in a year. Okay?
Benjamin Miller: Mm-hmm.
Kervin Pillay: What that means is that you need to have backup power, and that backup power is, uh, supplied by generators. And if you're talking about a gigawatt data center, the noise that those diesel generators make and the amount of diesel generators that you'll need, it literally feels ike a thousand aircraft taking off at the same time. That's the, the, the, call it, the air pollution part of it. Um, then it's access to water, right? So you need, um, relatively speaking, clean water. Which means it has to be filtered. why do you need it filtered? 'cause it's going through all of these compo, uh, these cooling components. And if the cooling components, uh, have, um, impurities in it, know, it starts to build up, uh, it stops cooling and you
Benjamin Miller: Mm-hmm. Mm-hmm.
Kervin Pillay: servers will
Benjamin Miller: buildup.
Kervin Pillay: Absolutely. Servers will blow up. you need to filter this water. You need to have access to clean water, which means that you can't just, you know, take any recycled water and pump it through the, through the, um, through the servers in order to cool it. Okay? and then there's the air cooling piece. Um, you know, these typically you are running, uh, inside of a data center in degree Celsius, somewhere between 18 and 22 degrees Celsius. the equipment runs at 55 degrees Celsius, which means that that delta of 30 degrees Celsius needs to be, um, extricated from the data center and released into the atmosphere. Essentially, so you are increasing the microclimate temperature around a data center, whether you like it or not. Whether you are using, uh, liquid cooling or air cooling, you increasing the temperature of the microclimate around that data center. And these have environmental concerns on, you know, uh, wildlife. It has, uh, on, you know, flora and Florida. All of these things need to be considered when you, when you thinking about building a data center.
Benjamin Miller: Yeah, I, I did read that some data centers are using that extra heat to heat, uh, households.
Kervin Pillay: Yep.
Benjamin Miller: know if how common that is.
Kervin Pillay: Yeah, it's, it's not as common as you think, um, because realistically the households can't consume all of the heat that comes out of the data center. So you're still gonna
Benjamin Miller: Mm-hmm.
Kervin Pillay: heat coming out of the data center. And also, very realistically, you're not gonna heat a home in, in spring and summer. So, you know, you've got maybe six months at best of being able to use that heat. Um, so in most cases, given that, you know, from our previous constraint that you can't be close to a residential area, it makes it really difficult to extract that heat, um, and, um, pipe it to a residential area. There's one really cool project that I, that I did work on with,
Benjamin Miller: Mm-hmm.
Kervin Pillay: in, in Japan. in Tokyo, um, obviously the, the. The population density of Tokyo is really, really high. And we were able to build a data center in a residential area, it, we had to fortify it to reduce the noise and reduce the heat that came out of it. We did some really innovative things there, like, uh, like using plasma cooling instead of uh, uh, air conditioners and liquid cooling. Um, so essentially it was a free cooling solution, which means that it doesn't generate any noise from cooling, which significantly reduced the, the, the noise threshold that we had
Benjamin Miller: How much more expensive did that make it?
Kervin Pillay: probably twice as more, twice as
Benjamin Miller: Twice. Twice as expensive. Okay.
Kervin Pillay: Yeah. But, um, and you know this from, from your own, you know, personal experience. You want things to be really, really quick when you, when you access services that are hosted by a data center, which means that the closer you can bring a data center to the user. The better experience they're gonna have. And that's the dichotomy that, that we are, we live in, right? We want to, we want the user to have a really quick experience, which means that we have to bring the data center as close as possible to them. we need these immense [00:30:00] power requirements and immense cooling requirements, which means that it cannot be close to, uh, uh, the user. and you need to balance those things.
Benjamin Miller: Just to compare it to who I think of as the, the, the biggest builder of, um, power in the world. And just to think of how they would be doing it. So China, I think China builds the equivalent of the amount of power in all of the United States, like every, every three years or something like that. Like, there's just, I think they, they announce or build a coal fire, fire plant every week one to two times a week.
They build. So it's just, you know, they're the best at building, um, at scale. And, and so I compare that of power that you need. We need the United States needs versus China. Do you have a sense of how China's gonna do this? Um, how much more they might build, I know they're, you're chip constrained currently 'cause of the export controls, but how would you compare the scale that we need to build versus what China's likely to build?
Kervin Pillay: Yeah. So, uh, and you know, most recently, um, Elon Musk has been tweeting about, um, solar
Benjamin Miller: Mm-hmm.
Kervin Pillay: much more solar China has than the United States. Um, and I think that's the way that they meet the demand.
Benjamin Miller: Hmm.
Kervin Pillay: There's a, there's a couple of dimensions to consider in this. Number one, they obviously have a lot of land, um, that they can build solar
Benjamin Miller: Mm-hmm.
Kervin Pillay: on. Number two, solar technology is gonna get better and better every year. means that they don't really have to build new plants. In order to get more power, need to upgrade the, the, the solar panels
Benjamin Miller: Hmm.
Kervin Pillay: and the battery technology. Okay. So the advances in battery technology means that you can capture more of the power and store it for a longer time, which immensely improves their access to power, to, uh, to power. Um, it also means that you can build it in any region that you want in the most remote of places as long as you've got access to solar. So as much as they are building coal stations, I think they, their competitive advantage is actually gonna be solar. Um, and you know, again, paraphrasing Elon here, if we don't in the United States, um, back solar. one of the sources that we use as power, we are gonna be way behind China and it's gonna be extremely difficult for us to catch up.
Benjamin Miller: But I don't hear anybody building data centers with solar at scale. Is that. I mean, Stargate is using natural gas.
Kervin Pillay: Yep, yep. So the, the two options that we have now are natural gas and nuclear. So we've heard some announcements last week about, um, regulation around nuclear
Benjamin Miller: I,
Kervin Pillay: allows smaller nuclear reactors to be built. And I think that's gonna be huge in order for us to supply power in general, but supply power to data centers, uh, specifically in, in, in terms of this conversation. So the, um, the advancements in nuclear are gonna be big. For us, and if we can leapfrog, I think that's the way that we stay at least on par. I don't think it's a, it's a, you know, one size fits all. I think it's a combination of solar, natural gas,
Benjamin Miller: mm-hmm.
Kervin Pillay: and potentially, you know, other data, data sources or other power sources that are gonna gonna come online that help us to augment this, uh, this
Benjamin Miller: Mm-hmm.
Kervin Pillay: that we're gonna have.
Benjamin Miller: Let me, let me go back to the, the REITs. You mentioned that, but there are two major public REITs that are data center REITs, and, um, they seem like they have underperformed Equinix last week announced that it was, um, gonna take a hundred percent of its cash flow and spend it on CapEx 4.5 billion.
They have, they don't seem to have much revenue growth, like a few percent a year. So why, why are the public REITs um, underperforming so much? And at the same time, data centers are the most exciting real estate investment.
Kervin Pillay: Yeah, so great question. If you look at Equinix and Digital Realty and, you know, some of the data center focused, uh, companies out there, or data center, real estate companies out there, they were building the, uh, data centers of the past where the utilization was really low. if you've got low utilization of your data center, you don't make money essentially. So getting your utilization up from 20% to 40% to 60% is huge. for those companies. Unfortunately, they have really old estates cannot be used for AI unless they completely revamp those, uh, those data centers, which means they need to redesign it. And what does that mean? They need to take it offline.
And what does that mean? That means it doesn't produce revenue. So they're in a very difficult position. They need to, uh, essentially rebuild their existing estates to support the new AI uh, designs. But if they choose to do that, they end up with a, with a situation where they have to actually turn off data centers for, you know, anywhere up to three years that, that can start to, to, to produce data or produce revenue. Um, contrast that with somebody building an AI data center today where it's completely oversubscribed, and we know this from,
Benjamin Miller: Mm-hmm.
Kervin Pillay: weaves. Public filings a couple weeks ago. they've got, $27 billion worth of backlog, um, for data centers. Um, and when those datas come online, they are going to be 80 to a hundred percent utilized, which means that it's far easier for them to recoup that investment
Benjamin Miller: Mm-hmm.
Kervin Pillay: a traditional data center that's running at 20, 30, 40, 50% utilization.
Benjamin Miller: Mm-hmm. Mm-hmm.
Kervin Pillay: So that's, in my opinion, the primary reason why this is hard. But, um, you cannot do it as an independent operator. So, you know, trying to build one data center, gonna be really difficult to recoup your money, right? It, it has to be in combination with, um, a company that has a lot of, uh, other estates so that users can move those workloads around and use your data center for. You know, very specific workloads if they need to. Um, you know, those are things that make the offering, uh, more compelling. So you, so unfortunately you need these, uh, older data centers that are able to support the old workloads, the, uh, hyperscale data centers that are supporting current cloud workloads and the new AI data centers that are gonna be supporting AI workloads so that you can essentially to market a complete offering.
Benjamin Miller: So one, you're saying one company needs all three of those things. And other than the hyperscalers, which is to be clear, are Amazon, Google, I mean arguably meta, uh, yeah, Microsoft and then, and then Oracle. Those are the five I think of.
Kervin Pillay: yep.
Benjamin Miller: um, does anybody else have that scale?
Kervin Pillay: So. terms of footprint, um, certainly digital, really, Equinix and NTT have the scale
Benjamin Miller: Mm-hmm.
Kervin Pillay: to do that. Um, they've got the footprint in multiple countries, in multiple regions in those countries to be able to do that. Um, but again, they, they had to build data centers. And I say they, I was, I was there at the time. Um, we had to build data centers to support many, many, many different types of workloads
Benjamin Miller: Mm-hmm.
Kervin Pillay: versus a, a hyperscaler that supports a far fewer, uh, subset of workloads.
Benjamin Miller: Mm-hmm.
Kervin Pillay: So Google, when they initially built their data centers, they just had such, they could hyper optimize their data centers for search,
Benjamin Miller: Mm-hmm.
Kervin Pillay: right? Um, if you look at, uh, digital reality, Equinix, NTT. They had to build data centers to support very, very different types of workloads. So, you know, a small enterprise that's hosting a single server to back up their user's laptops versus, uh, you know, running a, an entire SAP instance versus, um, you know, a company like Adobe running, uh, all of their cloud applications, uh, in some data center. So every one of those workloads are in an Equinix data center, and they have to be able to support this. It's difficult to optimize for any one type of workload,
Benjamin Miller: Mm-hmm.
Kervin Pillay: you know, further complicates the ability to extract revenue out of this, um,
Benjamin Miller: Do they have the power also at these estates? You, you're talking about so much more power.
Kervin Pillay: Yeah, typically they don't. So, you know, when you, when you want to consume, uh, a hundred megawatts plus these existing data centers were not built for that.
Benjamin Miller: Mm-hmm.
Kervin Pillay: So you're gonna have to get more power, build more infrastructure, more grid infrastructure, more substation infrastructure, uh, then build essentially all new [00:40:00] wings of data centers in order to be able to support these new workloads, um, while trying to not disrupt your existing revenue stream.
Benjamin Miller: Mm-hmm. Because I was with, uh, one, one of the big private equity shops. Uh, you know, right now, you know, Blackstone, Starwood, a few others are, are putting out like, um, I mean they're building tens of billions of dollars of data centers.
Kervin Pillay: Yep.
Benjamin Miller: And he, he was worried that they were, um, overbuilding. This is a real, and I find the real estate industry is, is so the tech industry, what I, what I'm, 'cause I sort of sit across both
Kervin Pillay: Mm-hmm.
Benjamin Miller: feels like there's infinite demand
Kervin Pillay: Yeah.
Benjamin Miller: real estate industry is worried about it being overbuilt.
There's a new, you know, one gigawatt announced every week. And so, uh, what would you say to him?
Kervin Pillay: So, yeah, I mean the, um, the underlying infrastructure that you're gonna build is always gonna be used. Okay, so what does that mean? It means the brick and mortar, the power, the cooling. There's always going to be a use for it. The problem is that the equipment inside the data center, um, at the rate that, technology is evolving right now, it means that every three years or less, the equipment that you have in it becomes obsolete. So, NVIDIA's, um, last evolution, uh, you know, the H one hundreds.
Benjamin Miller: Mm-hmm.
Kervin Pillay: Basically, nobody wants to buy them anymore.
Benjamin Miller: Mm-hmm.
Kervin Pillay: Right. They all want Grace Backwell 200 and the upcoming Grace Backwell 300, which means that all of the, the other stuff becomes obsolete. So if you are a full stack provider, in other words, you own the, the, the GPUs and the equipment inside the data center, that you, you gotta get your, your utilization up really, really high to stay ahead of that curve. If you fall behind the curve of utilization, it's, it's a very difficult business to make money in. Very difficult business.
Benjamin Miller: Because I was looking at that idea. Um, and if you, let's say you were to list as round numbers, a hundred million dollars of, uh, of GPUs and infrastructure to support that, um, with three year amortization and, you know, you wanting at least, let's say a 20% annual return, um, you're talking about you spent a hundred, a hundred million needing.
35, $40 million of current cash flow just to cover amortization plus your IRR plus your return on investment. I can't believe you can, you can put out a hundred million dollars and get a 35% return on cost, but they can't exist on does it.
Kervin Pillay: in five years. So when we model data centers, you model it for 20 to 30 years. You don't model for five years. Nobody models a data center for five years.
Benjamin Miller: but you just said that the, the equipment inside's obsolete within
Kervin Pillay: Absolutely.
Benjamin Miller: three years.
Kervin Pillay: so, so, so, so this is a great segue to talk about. The different types of models that exist out there for, for data center providers, are data center providers that, um, essentially lease out powered shelves.
Benjamin Miller: Mm-hmm.
Kervin Pillay: So, so the brick and mortar, the power and the cooling, and they say, bring whatever you want. Uh, you take the risk of, of the stuff inside it going obsolete. I will sell you, uh, essentially power on a per kilowatt hour basis. So you can do whatever you want. You can put whatever stuff you want in it. Um, uh, you are responsible for all this stuff inside. So if it goes obsolete, that's your problem, not mine. Okay. And then you get, and, and that's primarily the, that was the, the business of a digital realty and, and
Benjamin Miller: Mm-hmm. Mm-hmm.
Kervin Pillay: what they did. then you get, um. Cloud providers that are moving up the stack to pass SaaS and ias, they're saying, we will take responsibility for this infrastructure inside the data center.
We also take responsibility for keeping it current. And the way they made money was to be able to, uh, charge a really high margin on the services and the software.
Benjamin Miller: Mm-hmm.
Kervin Pillay: So in other words, if you are not providing the software top of the infrastructure, you are not making money. Hence, the hyperscalers make money because they sell software,
Benjamin Miller: Mm-hmm.
Kervin Pillay: selling the server, uh, or renting the server out and trying to amortize that over, over the, the, the, the lifetime of it. Very difficult business, is why you see very few providers actually doing that. Uh, a lot of the. The neo data centers, which are, you know, the name given to all the, the, the, the AI data centers.
Now offering a full stack service, which means that you can rent a piece of A GPU for a fixed amount of time, whether that's minutes, hours, days, months, whatever it is. that's where the margin comes in. Okay. Um, what does that mean though? It means that you need people to be able to manage that data center, manage the hardware, manage the software, do the maintenance. It becomes more like a, and translating this into real estate terms, more like a managed apartment where you are responsible for everything in that apartment. Uh, and the tenant just moves in and, and uses the, the, the infrastructure. And when they're done, they move out. it's more akin to that. so you don't make money off the, off the, brick and mortar itself.
You're making money off the services that you're providing.
Benjamin Miller: So we have a, um, AI application we've been working on at, at our company, and we we're out there trying to get, uh, tokens we want. And so we we're, we have, we're working with a, a bunch of the big model providers working with Amazon's bedrock. We're working with, uh, Google's Vertex and who you have to shop across all of them to get the tokens you need,
Kervin Pillay: Yeah.
Benjamin Miller: but I never see Core Weave or anybody else who's sort of providing GPUs.
So. Is Core Weave, are these guys wrapping Core Weave where if they're, if you're selling software and I'm, 'cause I'm an end consumer of tokens, right? Which are, what is, uh, is how people meter ai, right? Where are these GPUs? Like where, where is the intermediary that I'm missing if I'm the end consumer too?
Kervin Pillay: So, uh, let's take a, Google is probably a bad example because they own everything.
Benjamin Miller: Mm-hmm.
Kervin Pillay: let's take a company like Perplexity
Benjamin Miller: Mm-hmm.
Kervin Pillay: They provide you the service and the service is AI search or it's, um, uh, AI code assistance. Okay. Behind that, they are consuming Core Weave.
Benjamin Miller: They are buying the tokens from Core Weave.
Kervin Pillay: They are buying, no, they are buying the utilization per hour of A GPU from Core Weave. They're not buying tokens, they're buying the, the
Benjamin Miller: You think Perplexity is buying GPU time from Perplex, from Coral Weave. They're not buying tokens from a open AI or philanthropic,
Kervin Pillay: No. They, they, they do a combination of everything, right? So they need to run their own models, they need to consume Andro, they need to consume open ai. So they do a hybrid approach of, of all of those.
Benjamin Miller: but there are not that many companies at scale that would need their own models plus open ai, plus philanthropic. You know, I feel like my, maybe there's 10, you know, cursor, um, bolt. I mean there's maybe not even Bolt, right? Maybe there's just literally a wrap of open ai. So I. So how does Core Weave get to 35 billion or 27 billion of backlog when, when there's so few AI applications that are really are consuming the scale you're describing
Kervin Pillay: Yeah. So you know, a company like OpenAI will consume from Cove, right? They will consume any data center that they can get their hands on that's not owned by the, uh, by the hyperscalers.
Benjamin Miller: and Okay. That's what, that makes more sense to me. So an an philanthropic two or is that mostly open ai?
Kervin Pillay: anthropic
Benjamin Miller: of them.
Kervin Pillay: Yeah.
Benjamin Miller: And what about this question of decentralized for centralized 'cause so many of the problems you described is because the centralization makes the data centers massive, massively expensive, massively power generating power, consuming heat, water, et cetera.
Can you decentralize it or does that really break down the efficiencies?
Kervin Pillay: Yeah. So I think there's a, there's of generations of technology and enhancement that we need to go through in order to get efficient decentralization. so let's describe the, the outcome first, and then we, we'll come back to the problems.
Benjamin Miller: Mm-hmm.
Kervin Pillay: The outcome is that everybody on earth in the fullness of time has access to their own model, trained on their own data that only they have access to and nobody else can, can use.
Benjamin Miller: Mm-hmm.
Kervin Pillay: That means that there's 8 billion models out there. Okay. Um, and many of those the most part, are exact copies of other models. There's, there's a few pieces of the, of the model that are trained on your data that look different to my data. Okay.
Benjamin Miller: Mm-hmm.
Kervin Pillay: like copies of each other. Um, so if you, if you take this to the logical extreme, it makes sense that your model can be smaller and it can run closer to you on a smaller data center. But the problem with that is, um, the way that the, the models are built today, it requires a huge amount of memory and it requires the ability to move that memory between GPUs, uh, in order to, to, to make the tokens come out quick enough.
Benjamin Miller: Mm-hmm.
Kervin Pillay: if you don't have that, and that's, you know, where, or at least one of NVIDIA's modes, if you don't have that, it makes, it makes the user experience really bad.
Benjamin Miller: Mm-hmm.
Kervin Pillay: but it's really bad. And, you know, for the, for the listeners out there who have tried to run an AI model on their laptop, it works. It's just really slow. And over time it becomes, uh, it, it's not something that you wanna do on a daily basis. You'll do it for a once off, but you don't wanna do it for every task because it takes so long for even your Mac, uh, and four to, to produce these tokens on the newest models that it just doesn't make sense.
It makes it's cheaper to, to buy the tokens for a hundred bucks and
Benjamin Miller: Mm-hmm.
Kervin Pillay: you know, for a month, two months, three months. Um, so. one dimension is that models need to change. And I think fundamentally where we are now is not the end point of where we need to be for AI to be hallucinating and efficient. So that's number one. Um, number two, needs to be able to run on less memory, which means smaller models that can retain more information, which necessarily requires a different architecture of the AI model. And number three, it. GPUs today consume an immense amount of power, and there has to be a few more steps of how do we consume less power, have the same outcome. Okay. If those things are realized, then you get to a point where you can efficiently decentralize these models so that everybody on earth, 8 billion of us can each have our own model running on our own data center. As close to as close as possible to ourselves. What does that mean? It doesn't necessarily mean that you're gonna run it on your laptop, but it may mean that at the end of your street a, call it a mini data center that's hosting the models for everybody in your, in your postcode or your zip code.
Benjamin Miller: How many years away are you talking about here? Like, that seems like pretty radical, uh, re-architecture of how AI works today.
Kervin Pillay: Yeah, you would. You would think that, and I think, you know. The future is, is here. It's just not evenly distributed. So there's a couple of companies that are developing Asics that allow you to do that today. low power utilization, uh, run your own model in a, uh, relatively small form factor device. but there's a couple of problems with them, right?
Which means that when you develop an asic, um, you freeze the technology in it for, for the,
Benjamin Miller: Mm-hmm.
Kervin Pillay: of time
Benjamin Miller: Mm-hmm.
Kervin Pillay: the way that technology's evolving right now, which means those things could become obsolete in six
Benjamin Miller: Mm-hmm.
Kervin Pillay: year, in two years. So the economics don't work out. that. When we get to a point where we are would, like, for example, cloud computing everybody knows everything almost for the most part, then it's easy to do that.
And that's where cloud computing is going. Now with edge data centers. They're moving cloud workloads as close to the, to the, to the user as possible.
Benjamin Miller: Mm-hmm.
Kervin Pillay: these edge data centers that consume, you know, 10 kilowatts to 20 kilowatts, but are able to support, uh, you know, thousands if not tens of thousands of people.
Benjamin Miller: Mm-hmm. Mm-hmm.
Kervin Pillay: already with, with, uh, cloud data centers. That will happen with AI data centers. My prediction is somewhere more than five years, less than 10 years,
Benjamin Miller: Mm-hmm. Mm-hmm.
I want, I just, let me just go international for a minute and I'm gonna come back to home. So the Middle East has been announcing these huge, uh, data centers. We're gonna ship a lot of, I don't know how many millions GPUs there.
Kervin Pillay: Yeah.
Benjamin Miller: Uh, what's their strategy and what do you think about this strategy of trying to build data centers in the Middle East?
Kervin Pillay: Well I think, you know, every country and every region should be thinking about building their own data centers for AI anyway. Why? 'cause several sovereign AI is actually a, an opportunity and a, uh, in a way to control your future, right? Think about this. Um, and we've experienced this literally in the last, I don't know, a year or so. Uh, open AI comes out with a new model and it turns out the new model doesn't work as well as the old model. If you are tied into that ecosystem, going backwards relative to everybody else. But if you control your own destiny, which means that you have your own data in your own data center and you can choose what to run and when to run it, gives you the control that you need. So I think that, um, in order to foster, uh, I guess innovation, you need to have these regional data centers that are specific to a region or a country or a, you know, uh, some geograph, geographically separated, uh, uh, entity. Uh, so that you can foster innovation in that, uh, location. 'cause the other problem is brain drain.
Everybody wants to go where the GPUs are and you need to keep them, uh, uh, local. And then the second thing is, from a government perspective, is why sovereign AI is so important, um, there's regulations and laws that don't allow you to. Move your data or the citizen's data out of the your borders, which means that if you want to use ai, you have to build an AI data center.
Benjamin Miller: So I can see that, how, I can see how that plays out for, um, in a way, you said 200 countries in the world, but it seems like the Middle East strategy goes beyond that. They're trying to become a, a hub for world, uh, uh, AI in terms of the scale of what they're, what they've been announcing
Kervin Pillay: Yeah.
Benjamin Miller: is that. I know, I've heard a lot of people think that's smart, that they're gonna, they're trying to, you know, their oil doesn't go forever.
So if they can become this, the source of, uh, of where people build ai, um, 'cause they have a lot of power and they don't have the same regulatory challenges the US has. Uh, does that actually, do you think that, 'cause I, I think last time we spoke you were, you had some skepticism about it. Wondering if, if that's, um, something you could expand on.
Kervin Pillay: Yeah. So, you know, it's, it's, it's all to do with data sovereignty, right? You don't want your data as a individual user or as a country be hosted in a region that you have no control over. Okay? So us, in the us, 99.9% probability that we won't run any government workloads in the Middle East AI data center, right?
Maybe even more than that, we don't want a, uh, security issue to cause the leakage of data to, to users that shouldn't have them. And if we thinking about that, then every other country in the world is thinking about them as well. So what does that leave? It leaves public common open services. Can be consumed from a essentially worldwide AI data center. And if you think about, uh, you know, back to my previous con uh, comment on having their own AI with their own data, you wanna know where that is. So I'm not convinced that building a 10 gigawatt data center means that the whole world is gonna use it.
Benjamin Miller: Mm-hmm.
Kervin Pillay: it's just, it's not just about the power and the, and the compute. It's about what the AI does and what data goes into the ai and who has access to that, to that
Benjamin Miller: Mm-hmm.
Kervin Pillay: essentially.
Benjamin Miller: So you could, you could imagine any private company in the United States saying it's, we better host it here. So our customers, our US customers, our us, uh, consumers, you know, don't have risk, uh, beyond us Sovereignty.
Kervin Pillay: A hundred percent. And, and it may be even more local than that, right? It may be even, um, you know, and I'm in California. California may say, well, we want our own ai, uh, cluster because we don't wanna share California data with, with any of the other states.
Benjamin Miller: God.
Kervin Pillay: That could happen.
Benjamin Miller: Sounds like an interstate commerce, uh, litigation to me. Um, I've also heard you say, um, some things about semi analysis, which is a, you know, leading. Um, thinker in this area, but when it comes to hardware and, and, and so I was curious what you think they often get wrong as a 'cause they're an fabulous analyst, but you're an operator, right?
So,
Kervin Pillay: So,
Benjamin Miller: what does an operator know That an analyst doesn't?
Kervin Pillay: Yeah. And, and I think fundamentally the, the experience of operating a data center, is very different to the analysis of why you should build a data center and how to make money off a data center.
Benjamin Miller: Mm-hmm.
Kervin Pillay: Um, so on the face of it, you could build a data center with, uh, AI servers that has exactly the same capacity as another AI data center.
Benjamin Miller: Mm-hmm.
Kervin Pillay: analysis still holds true, but the type of equipment that you're using, the failure rates of the equipment, the combination of the failure rates of the individual components is not something you can model in an analysis, and that only comes from operational experience. in other words, um, you know, do the aircons fail more than once a month? What is the effect of that on your design of power? Therefore, what is the effect of the design on power, on your cost? what is the design on your backup and redundancy strategy? How many more generators do will you need?
Benjamin Miller: Mm-hmm.
Kervin Pillay: have more generators and more air conditioning and more downtime, how many more people will you need? How many more people, um, are you gonna need to be able to operate a data center? How much more is that gonna cost you? That all factors into the real operational cost of a data center. So the build of a data center, they get really, really good information and there's plenty of information out there. The operations of a data center, the, the data center operator is not exposing those costs.
Benjamin Miller: Mm-hmm.
Kervin Pillay: many times the, the servers are failing, how many, uh, times they have to replace equipment, uh, how many times the, the equipment goes down, uh, how many more shifts you need to have. And, and again, for 36 minutes of downtime, you, you're probably gonna have three shifts of people. So you think you need a hundred people, you actually need 300 people.
Benjamin Miller: Yeah. When I'm invested in data centers, it's like signing, uh. It's like going into the F-B-I-C-I-C-I-A you sign and confidentiality. They, you know, you're not supposed to disclose anything. Uh, I mean, there's, it's high security,
Kervin Pillay: yeah.
Benjamin Miller: imagine that there's not a lot of data out there.
Kervin Pillay: No, and, and you don't. And that's part of your moat of if you are an efficient operator of data centers. You can make money.
Benjamin Miller: Mm-hmm.
Kervin Pillay: If you're an inefficient operator of data centers, you do not make money. So let's relate this to, uh, an investment strategy. I could say, well, I've got a hundred million dollars.
Let's, let's give it to someone to build a data center, only to find out that it costs a billion dollars to operate that data center because they're being super inefficient about how they, they operate it. So my actual, and the problem is once I've invested the a hundred million and I have this asset, I have to operate it. So now I'm committed for the, for the billion, even though I didn't know that that was gonna be the case, and I have no option at this point because I've invested a hundred million dollars. What do I do? Do I write off that a hundred million dollars or do I just keep operating this right? Versus, um, that has, um, figured out operations of a data center has chosen the right equipment so that the meantime between the failures is minimized. Based on the equipment that they've chosen, and maybe it's $150 million data center, not a hundred million dollars data center, but the uptime is higher, the utilization is higher, the maintenance costs are lower, and now you need 50 million to, to operate that data center. So these operational costs could really kill you.
Benjamin Miller: Mm-hmm.
Kervin Pillay: really kills an investment. And sometimes you don't know what you don't know, right? You, you, you start, and certainly, um, you can count on probably, uh, you know, one hand the number of people that have built more than a gigawatt worth of data center. Alright? So, uh, if you think somebody that knows how to build a 10 megawatt data center knows how to run a one gigawatt data center, fundamentally different skill sets
Benjamin Miller: Mm-hmm. Is that the biggest, mis, biggest misconception in the data center business? Or is there something else?
Kervin Pillay: No, that, that's probably one of the biggest, uh, I guess, unknowns of. How much effort it actually takes to run a data center. It's not like a, like a commercial building where, you know, you turn it on and the, and the, the, the tenant does basically most of the work. Um, it's, it's a 24 by 7, 365 operation with three shifts of people the time.
People on call. Uh, nobody gets to go on holiday. It's, it's a, it's a tough business run a data center. Um, and if you're not prepared for what it takes to run a data center, um, you come short, right? then you get, uh, consolidation, which is what happened.
Benjamin Miller: Mm-hmm.
Kervin Pillay: you know, the reason why Digital Realty and Equinix are the size that they are now is because they've been acquiring, they've been
acquiring these smaller data centers that, couldn't operate as efficiently at the small scale
Benjamin Miller: Mm-hmm.
Kervin Pillay: where Digital Realty and Equinix could at a larger scale.
Benjamin Miller: Well, this was fabulous. I, I, um, I feel like I learned a lot. Do you have any, uh, parting, uh, predictions over the next few years in terms of how AI plays out?
Kervin Pillay: Yeah. So, so right now we are in the, in the, in the stage of, um, everybody's training a model, or at least, you know, everybody that has the capital to do it is training a model. Um, we are getting to the point where it's going to be a, um, a flip toward more inferencing than training.
Benjamin Miller: Mm-hmm. Mm-hmm.
Kervin Pillay: I think we, we are at the stage where, uh, you know, there's a lot of training happening, uh, and probably less inferencing, we're gonna get to a stage where the, the models are more stable. The technology has moved on, it, it changes to inferencing. I mean, literally, I think it was three days ago. Uh, Google launched a project on Kago that gives you $150,000 if you could figure out a way to run an AI model on your phone for something useful for an end user. So if that happens and you know, we get to the point where the AI model runs on your, on your mobile phone, then it's all inferencing from that point on, right?
The inferencing is gonna dwarf training, by orders of magnitude. That's both good and bad, right? It's a different way of consuming GPUs.
Benjamin Miller: Mm-hmm.
Kervin Pillay: opens up the, um, the lane for people that are developing asics Benjamin Miller: Mm-hmm.
Kervin Pillay: in and develop these asics, Google specifically have been developing the small model to run on their, their devices and, and other devices that have the same hardware changes the dynamics for all of the GPU providers and all of the data center providers, right? So running it on your mobile phone means that you don't need this data center for. Call it 80% of the, of, of the things that you
Benjamin Miller: Mm-hmm.
Kervin Pillay: a daily basis. Like, my next meeting? Uh, you know, uh, where did I, where did I have dinner three days ago? Like, those are things that can be answered on your mobile phone versus do me some deep research on, you know, all of the semi providers out there, and tell me the pros and cons of why they are leading the market.
That's not something you do on your mobile phone. but that's still required. And Benjamin Miller: Mm-hmm.
Kervin Pillay: be, um, you know, those use cases that require these really complex models that process for hours, if not days, to get you a, to get you an answer.
Benjamin Miller: Mm-hmm.
Kervin Pillay: So to summarize, the, my predictions far more inferencing than training and smaller models running closer to the user.
Benjamin Miller: Is that Apple's like actual game plan here because they seem to have been out of the game so far?
Kervin Pillay: Yeah, I mean, think Apple at the, at the stage where they can, um, choose what they do and they can run a hybrid model with their secure AI cloud that they've got. So it makes it feel like it's running on your phone, but it's not,
Benjamin Miller: Mm-hmm.
Kervin Pillay: which is, you know, great because they own the ecosystem,
Benjamin Miller: Mm-hmm.
Kervin Pillay: So when you type in something to your phone, you may think it's running on your phone, it could be running in the secure cloud, but it's not running in a, in a, in a public AI data center somewhere. Uh, you still get the speedy response. Uh, and then for things that, you know, need to be done in your phone, like email summarization, that can be done on the phone. So I
Benjamin Miller: Hmm.
Kervin Pillay: a, it's a hybrid. Um, apple and Google are probably the best placed to take advantage of those, uh, of those advances that are
Benjamin Miller: Mm-hmm.
Kervin Pillay: Um, but again, gonna be so many more use cases. And I mean, the, the big thing is. If you look at where how quickly video has evolved,
Benjamin Miller: Mm-hmm.
Kervin Pillay: AI video, it, it's clear to me that AI video is gonna dominate this, this space in the next months, if not years.
Benjamin Miller: Mm-hmm.
Kervin Pillay: gonna absolutely dominate the space. It's gonna dominate the usage of, uh, models, of GPUs, of devices creating video.
Benjamin Miller: Yeah, my brother's in the entertainment business and he just like, is, he hates it. I, I think it's great, but it's, it's not so good for you're, if you're a, uh, writer in, in, uh, Los Angeles.
Kervin Pillay: to, uh, to wrap, wrap that up. I think the way that, um, we're gonna see AI really being used is as a superpower to people that have the knowledge, right? There's, there's a lot of, um, I. I guess rhetoric out there that says, you know, anybody can build anything with AI even if you have no experience. But the reality is that those people are gonna get a 20 to 30%, uh, improvement on what they were doing because they don't understand the, the, the intricate details of, of how to make a movie. But people like your brother that are in the entertainment industry, gonna give them a 300% boost.
Benjamin Miller: I keep telling him that. Uh, so we'll see.
Kervin Pillay: Yeah. So, yeah. I, I, I think that's how we see this thing play out is, is the people that have the knowledge and know how to direct AI are really gonna make, uh,
Benjamin Miller: mm-hmm. Mm-hmm. That's already how technology has been, is asymmetrics people. It makes the top 10%, a hundred times more powerful.
Kervin Pillay: Yeah.
Benjamin Miller: and, and that's, that has its own issue. So,
Kervin Pillay: Yeah.
Benjamin Miller: but Vin, this has been fabulous. I really appreciate you taking the time.
Kervin Pillay: Thank you for having me, and I, I, I hope we are able to cover enough ground so that we, uh, you know, we have people thinking about how to invest, um, what to look for in the market and, and, and a where to not go in the market. I guess
Benjamin Miller: I, I certainly learned a lot, so onward.
Kervin Pillay: Great. Ben.
Ben: You have been listening to Onward, featuring Kervin Pillay, former CTO of Cisco’s Automation Group . My name is Ben Miller, CEO of Fundrise. We invite you again to please send your comments and questions to onward@fundrise.com.
And if you like what you heard, rate and review us on Apple Podcast and be sure to follow us wherever you listen to podcasts.
Finally, for more information on Fundrise sponsored investment products, including relevant legal disclaimers, check out our show notes. Thanks so much for listening, and we'll see you next episode.
