Viktor/Whitney
00:00:00.000
I hope that agents are not really replacing the tools that are already working and mostly autonomous. Mm-hmm. They're probably replacing Our part of that work. Mm-hmm. And whenever I hear it's not deterministic, my answer is neither You are, neither are you?
Viktor/Whitney
00:00:18.919
Yeah. Because everybody somehow thinks, oh yeah, you are right. Actually people also make mistakes, but somehow I will not tell you that because I will, I will sound like an idiot, but that does not apply to me.
Darin
00:01:32.699
CubeCon EU 2026 might be the conference where Kubernetes officially became an AI platform, not a container orchestrator that can also do ai, but an AI platform that happens to run containers too. Viktor and Whitney. Does that seem to be true? Is that what's really happening there?
Viktor/Whitney
00:01:51.832
I feel it is, yeah. Especially for, I mean, I think that it's still more about inference than, uh, anything else. but all the pieces are coming back together, I feel, and, uh, CNCF is pushing hard on that being ai, everything. Whatever you can image. Oh yeah, the keynote's all all about ai. There's a new sandbox project, LLMD. Um, they talked about Nvidia donating a driver to CNCF, so that's happening. it's a lot. The conformance group. Yeah.
Darin
00:02:25.466
All of that happened. Uh, LLM dash D came from Red Hat. Uh, Nvidia actually made quite a few contributions. DRA driver for GPUs.
Darin
00:02:35.906
See, this is the easy thing. You're on the ground, you can't see everything. I'm just sitting away watching all the news flow through. Um, the other thing that Nvidia did, which is probably bigger, is the Kai Scheduler, KAI scheduler. It's a high performance AI workload scheduler. So, and GPU support for confidential computing,
Darin
00:02:56.006
which is becoming a big deal, especially in eu and a couple of new projects, uh, specifically from Nvidia, Google Open Source, GKE cluster autoscaler, which I guess is no different than what AWS did with carpenter,
Darin
00:03:11.726
Effect effectively carpenter's, autoscaler, but Google also did a DRA driver for TPUs, so. Nvidia did the DRA driver for GPUs, DRA driver from Google for TPUs? Does containers even matter anymore? I, I don't think it does. I mean, it does, but it doesn't.
Viktor/Whitney
00:03:31.126
Y Yeah, it's not interesting, that's for sure because I've, I heard during the conference this week that you can't even run inference workloads in containers. They can escape, so you need to use micro VMs.
Viktor/Whitney
00:03:47.081
and you know, the Boot up time and the size of VMs, whether it's micro VM or something else, or container, does not really matter. At least not for inference. Right, because those are so massive and so big that if you tell me, Hey, I'm going to speed up, uh, the startup time by, uh, one second, say, uh, why do I care? You know, kind like, what's five minutes? Uh, what's, what's one second then if it takes one minute in total, or five minutes in total. Right? And same thing for size. If there are terabytes in size, uh, at least for inference, right? Do you care that much? What is the base size? You probably don't.
Darin
00:04:28.661
I don't think we do care. The other thing that I saw come through was the gateway, API inference extension that's in beta right now. To where Did you guys see that at all?
Viktor/Whitney
00:04:39.161
Yeah. Yeah. I heard about it through the grapevine, but I don't explain it to me, Darren.
Darin
00:04:44.501
Well, I'll, I'll try to explain it 'cause I'm, I'm still trying to get my head around it too. So, based on what I'm reading, it's AI model routing baked directly into the Kubernetes networking layer.
Darin
00:04:55.751
So Viktor, what was the, what was that? Router and not open router. Right? Open route. This sounds like open router on steroids. And specifically Istio announced support via agent gateway. Uh, the other thing I'm seeing is inference routing is becoming a first class networking primitive.
Viktor/Whitney
00:05:14.055
because it's a different type of routing, right? It's, uh, now we are talking about large scale where, uh, inference is running on multiple Node.js and very big ones and what not, and you need routing. And that routing needs to be somehow related to this user sessions and what not. And if you put multi cluster setups, which are becoming a thing, especially for inference, then that becomes even more complicated, and that's probably going to come into the next phase, almost certainly. Right? That's, that's one of the problems everybody's trying to solve right now. During the keynotes, Lynn Sun did a live demo, a very brave live demo of, these projects of, rooting.
Darin
00:05:57.131
to me, it's interesting. I'll add, add in. One more I saw was the Microsoft AI runway, which is an. An API for inference workloads with hugging face model discovery, GPU, memory indicators and cost estimates, supporting Nvidia, dynamo, QRE, LLM dash D already, and Chito runtimes. again, it sounds like they just built a layer on top of everything else but Viktor, what we were saying, multiple clusters. Is this really where. I'm going to want, not only multiple clusters, but multiple clusters from different providers based on the TPUs, GPUs, and whatever's available from each of those places.
Viktor/Whitney
00:06:37.622
Yeah, that's exactly, that's something you will not be able to afford as a Darin, but that's, that's some that's already happening, right? It's just, uh, more a question of, getting away from custom tooling created by com Individual companies, you know, and tropic, just as an example, are open ai. They all need those things and they're all using sticks and stones to set it up somehow, right. and it's not only reserved for those, if you think of large bank or you know, any large, larger company, is likely going to start moving from I pay you for API access to, I run inference myself. It's going to cost hell of a money, but they're all gonna go into that direction. And, uh, then you really need scale, right? Because everybody hopes that, um. The whole planet will be using that inference one way or another. Right?
Darin
00:07:30.105
Well, I think that's where the LLM dash D project is gonna kind of come into role play. 'cause now instead of being that monolith, we can split it apart. They're calling it disaggregated architecture. That just feels really strange. Why not call it distributed? That's. Because they need a new word to buzzword with. I don't
Viktor/Whitney
00:07:50.640
If things seem too easy, they might not stop paying us as much money as they do. Exactly. Kind of. Once, once the word becomes known to everybody, kind of like repeating it, that doesn't hold much value. We come up with a new word that's, that's normal. That's, that's to be expected.
Darin
00:08:05.175
know, but that's silly too. Uh, did either of you go to the NGINX Day was one of the early days.
Viktor/Whitney
00:08:13.650
I haven't been, uh, no. Uh, is it one of the day zeros? I might have spoken at it. I think I spoke at it,
Darin
00:08:21.525
I spoke at it. Uh, yeah, I didn't look at your schedule, so I'm not sure what it is. Uh, but the, the keynote topic was supposedly agents as first class users in production. Not agents as agents, but agents as users.
Darin
00:08:38.216
I mean, really the humans more than ever before are going to be barely managers in, probably in this case, right,
Viktor/Whitney
00:08:48.206
I think that that's already decided. We are all managers. Just not everybody got the memo yet. We're all PMs now. It's a dream come true man. Without pm, without people, pm without. Per.
Darin
00:09:05.861
Right. But we're also realizing that we used to as, as the Lord grunts, we used to say, well, what do the managers do? All they do is just go to meetings and talk to people. And now that I'm sitting in front of three, four, or five different windows of Claude coming back to me, it's like, Hey, how about, how about now? How about now? What about this? It's like, how did they managers even do their jobs?
Viktor/Whitney
00:09:26.711
That's easy, man. They will create their agents and you will create their, your agents and. Their agents will, whenever their agents ask your agents, your agents will respond to their agents and everybody will be happy and nobody will know what happened.
Darin
00:09:41.966
If everybody just followed that circular talk that Viktor said, you're doing better than I am right now. Wow.
Viktor/Whitney
00:09:49.181
it's like we all have marionettes and our, our marionettes talk to each other. We never do. Yeah. I mean, it, it, it's similar to, so I'm, I'm getting pull requests on some of the projects now. Right. And I can clearly see that the, pull request was created by an agent. It's so easy to read it. Right. But the interesting part comes in a conversation in the PR. When you see my Claude asking the user and then his agent answering to me, kind of like, I'm never going to see anything written by a human or read it myself. Have you seen that? Reddit? That's for agents. Okay. You mean like multiple or like Yeah, the, yeah. Malt bot. Yeah, but there's like a Reddit for all the different mold bots.
Darin
00:10:37.699
Well, speaking of which Reddit said yesterday or today, I'm not sure the exact timing of they're gonna start authenticating, humans and only certain bots will be allowed to be on Reddit. After they've gone through a verification process, now how they go through that verification process? Who knows? I, I'm still looking through a lot of the announcements from what happened there, and again, I'm looking for things that aren't AI and not finding anything. The only thing I can
Viktor/Whitney
00:11:16.745
Um, Abby Banger did a keynote this morning. Mm-hmm. And she was talking about, APIs for the producers and, and platform engineering. We've been talking about making APIs to make the, the user experience really smooth, but now we're talking about making APIs to make the, the capability producers really smooth and the platform engineering being more like a marketplace level. I thought that evolution was very interesting.
Darin
00:11:44.273
Well, that actually brings up a, a question I had written down earlier was if you're a platform engineer now running Kubernetes, like, so that's, that's your day job. did this CubeCon really give you a big new to-do list if you haven't dealt with ingress yet? It's, it's done. Done now,
Viktor/Whitney
00:12:04.963
Uh, yeah, if you haven't done the with Ingris, I don't know what you're doing in coupon.
Darin
00:12:09.968
Yeah. Different conversation I guess. But people are gonna start if, if they're not running inference workloads yet in Kubernetes, if you're running Kubernetes, more than likely you'll be running inference workloads,
Darin
00:12:25.628
right? That's versus if you're still a monolithic VM shop, you may not be running as many inference workloads.
Viktor/Whitney
00:12:32.573
I'm not sure about that to be honest. Running influence is extremely expensive. Right. And uh, it's a long-term commitment and it's uncertain commitment and we don't know what to, so I, while I do think that many companies will be running inference, I'm not sure that many are ready to do it today or that they should. Right. and probably inference will not be run by platform engineers or that will be run by somebody else for platform engineers, I think it's more about, okay, so how do I. Throw to trash half of what I was doing before. Uh, because people don't want to click on any ui and they get very annoyed when they have to instruct their cloud to click on UI that I built for them. and really it's, it's more about how do we extend people's agents, let's call them genetic agents, to do something specific to basically how do I provide services that I was providing to humans. Now to agents operated by humans, agents as first class users of a platform, basically what you're saying. Exactly. Exactly.
Darin
00:13:40.926
Well, this is, we've talked about this before. I can't remember who we talked with, but, uh, maybe it was, uh, Matt Bergos from Apollo to where their documentation is written for agents first. Now in human second, it's like everything becomes agent first.
Viktor/Whitney
00:13:56.791
I think we are gonna see that even on, on, on public facing, documents or sites and stuff like that. Like one of the sites I'm involved in, I'm actually urging the author. To kind of, okay, let, let's start with markdown because by the time this is released, I dunno whether people will watch it, read it.
Darin
00:14:13.893
But if you know if it's marked down, again for people that haven't gone out to the DevOps paradox.com website, it's fully AI friendly. It has LLMs text, it has index dot MDs for every page. I mean, I'm hoping to get as much stuff into. The next agent models this, this is the difference, I guess, really from the standard web crawlers versus the agent Crawlers is web crawlers. It can be there in minutes to hours to days. Whereas with a model we've, you know, a foundation model, we gotta wait for the next big release of the model to see if it, if we hit or not. Typically, I wanna go back though, to confidential computing. Big deal in the eu, but Whitney, you brought up that everybody's thinking that these agents that are gonna be running in need to be in micro VMs instead of just containers. Right? how much of a pain is that gonna be for people actually be operating a Kubernetes cluster? Is it, is it a big deal?
Viktor/Whitney
00:15:13.745
I don't know. Yeah, you, you, I don't know what's going on here. I know there's a Cuber project. I know that there are, so I don't know how difficult it's going to be to abandon one to pick up the other. I mean, I, I can only suspect that it'll be difficult because really, uh. The ecosystem is made for containers. Yeah. Right. And one thing is that you come up and what was the project? Secure container? Something like that. Confidential containers, Coco, confidential containers. Yeah. Yeah. I, I dunno, the current state. But the pain was not really to run that container. The pain was everything else. And everything else is usually massive, right? Oh. So, okay, we have, uh, this is our networking. Oh, this is how we do volumes. This is how we do this. This is how we do, and this does not work. All of a sudden. And once we make it work, then we realize that actually it cannot be in a confidential stuff because it needs to redesigned in the first place. And, and I wanna say about confidential containers, it involves specialized hardware. It's not just a matter of running it. Okay? Then, then, then let's, let's just forget about it. the challenge is always how to bring something new into the ecosystem, right? it's easy to run it in containers. Uh, sorry to run it in con, in Kubernetes. Okay. Maybe in this case, no, because Specialist Berg, but in general, that's the easy part. Everything else is hard. and that's one of the reasons why VAM is not picking up really. Because if you go back to vam, you can say the same thing for vam. Is vam cool? Yes. Can you run it? Yes. Does it work? No. What I mean, it doesn't work within the ecosystem. Oh. Because it's like, it would involve a lot of shifting of your doing. Yeah. All of a sudden you kind of like, you cannot connect it to other, uh, apps in a cluster. All of a sudden, uh, network policies are screwed. It becomes complicated. Right. Can, can a was a module hold a inference workload? Uh, I'm, I'm surprised that they're not yelling about it yet because Yeah. Wasm is mostly about. Figuring out kind of the, the real use case people wanna pay for and all the previous use cases were not very successful. So one be the nice one. One. This is the one. Maybe this, maybe this is their day. This is the one. There we go.
Darin
00:17:19.891
So SUSE either announced or reiterated their rancher AI crew, which is agents managing your infrastructure. Is there a greater than 0% chance you would allow agents to manage all of your infrastructure, like to do full fleet management
Viktor/Whitney
00:17:36.146
Uh, depends whether you're asking today or tomorrow, right? Today I'm very skeptical.
Viktor/Whitney
00:17:45.151
Uh, I mean, today I'm, I'm skeptical that I think that infra, uh, in general is not very, very yet into. You know, and, and those are usually different people than people building those tools. First of all, we need to distinguish those, right? Mm-hmm. It's usually developers building tools, and they're all on board, kind of like we are all agent. That's, that's kind of great. That's fine, but, uh, operations are not yet there, right? And you have those steps of evolution that kind of, oh, you cannot do anything but write, uh, write a document for me, right? And now you can write some YAML for me, but I need to verify every single line of that YAML, right? And so on and so forth. And it takes steps of maturity in a way until you say, okay, now go kind of like do stuff. So full agent management feels like too big of a joke to too big of a jump for majority. Yes. Now, that does not mean that they shouldn't be doing that. I think that they should because projects are not built overnight. And then by the time kind of that matures and what's not, the world might be ready for it. I wanna say, show me the deterministic guardrails. Show me the literal code that keeps it inside, keeps, keeps each agent in line that can't possibly step out of its lane. Show me a person that can do that. I mean, that's why we have like policies and stuff so people don't jump out of their lane. Same thing like, yeah, I need determinism. It can't just be like really well-written prompts. We, we still have determinism right. In, in terms of policies or this or that. Mm-hmm. Uh, I, I feel that agents are, I hope that agents are not really replacing the tools that are already working and mostly autonomous. Mm-hmm. They're probably replacing Our part of that work. Mm-hmm. And whenever I hear it's not deterministic, my answer is neither You are, neither are you?
Viktor/Whitney
00:19:43.849
Yeah. Because everybody somehow thinks, oh yeah, you are right. Actually people also make mistakes, but somehow I will not tell you that because I will, I will sound like an idiot, but that does not apply to me.
Darin
00:19:56.294
Well, we'll move on. We've talked about ingress and X already being dead. In fact, it will be gone. Gone with the release of one point 36 in theory on April 22nd, meaning the in theory meaning releasing on April 22nd still should be gateway. API has been there. It's been out for how long. And it's a while, right? A couple
Darin
00:20:15.719
and a half at least. there was another new project that came out called gras. It's a new sandbox project. It's an AI Knative gateway with an in ingress migration path. tia's mesh list approach was the other way to do it. Gateway, API without side cars, is it, just say, thanks ingress for all the help for getting us here and. Have a nice life.
Viktor/Whitney
00:20:40.536
I mean, this is the part where I have called grudge against Kubernetes. Community maintainers. I think that Kubernetes should do the first step and say deprecated will be removed from, I don't know, what's the rule? I think eight versions from now. Mm-hmm. Uh, is the general rule, so deprecated and that means eight versions in the future, if I'm correct, I might be wrong, it'll be removed completely and start planning, uh, migration to gateway, API. Mm-hmm. That's what should be done. Right gateway. API is mature enough, it's production ready enough as a replacement for ingress. We can discuss how gateway API likes certain things that are being developed right now, but as a replacement for ingress, it should be there. And I mean that in your work with enterprises as well. Kind of until you say this will not, you will not be able to use this. People are gonna going to continue using it. It's similar like what we had with Docker, right? Kind. Oh, we are going to remove Docker from Kubernetes. Uh, okay. Uh, I'm not doing anything then. Oh, Docker is removed. Uh, uh, es can you please make it work somehow in my cluster? Right. So, uh, are you saying you're disappointed in Kubernetes because they didn't give eight releases worth of, of No. I'm disappointed that Kubernetes for not deprecating ing Englishs. Oh, and it's like this is deprecated and will be removed in this version of Kubernetes Uhhuh start preparing to move away. Gateway. API everybody.
Darin
00:22:15.929
So what you're just saying is it just came too quick. They didn't follow their own path of rules.
Viktor/Whitney
00:22:21.479
No, it didn't came too quick. It's been, it's been in the making. It's just that, uh, it, it like engineering English. So everybody knew that it's going to disappear years ago. Right. And everybody knows that more or less that gateway APIs should replace ingress. Mm-hmm. It's just that companies, once they implement something, they are not willing to migrate unless they're forced. And then once they're forced they say, oh, this is so good. How it's, it was such a good decision to, to force me to move to this because this is a better option. Definitely. Right.
Darin
00:22:57.229
One thing that's coming in 1 36 that we didn't talk about on the Friday shows, uh, was OCI artifacts as volume source. That's gonna be stable. So it allows you to mount model weights in large files as OCI artifacts, decoupled from the container images themselves.
Darin
00:23:13.423
think that will fix, like, one thing I'm thinking, again, being an old Java guy, uh, being able to mount. A Maven local cache or mounting any kind of maven or a NPM cache as a file system would be great, versus having to deal with container images. It's, that's gonna be able to be reused for a lot of different things. But that means you gotta get to 1 36. I mean, I don't know when it was introduced, but man, that might be a,
Viktor/Whitney
00:23:47.053
I suspect that the main driver behind that is must be models because it can take you anything between five minutes and the infinity to just download the image of a model,
Viktor/Whitney
00:23:59.413
right? And then you say, oh, okay, now, uh, how about you run on a different node, or let's scale up, and then, then, then you go and watch something on Netflix. Um, it's important. It's Netflix, not YouTube, because Netflix has longer shows than YouTube, right? YouTube is, And, and then you come back and you check, right? Uh, mounting those things or distributing them somehow making it faster is, is imperative right now. And especially given the cost of GPUs and everything like that, right? Because, those things cost a lot of money and, uh, it costs even more if you, if you're waiting for hours for something to happen, right?
Darin
00:24:41.370
I wanna talk about the new Sandbox projects. We've mentioned a couple of the names already. LLM dash D from Red Hat for, I'll have to say the phrase again because it's just fun to say. Disaggregated. LLM Inference on Kubernetes. That's what it's for. Kai scheduled from Nvidia for a I AI workload scheduling. This one caught me off guard. Uh, Valero. From Broadcom for Kubernetes. Backup is now in sandbox
Viktor/Whitney
00:25:11.570
Wait. Valero was not in C. CI was surprised to hear that too. Actually when I came. Yeah. Oh, I did not. I, if you ask me, I would say with a hundred percent certainty that it's C. C, same. Maybe proud come. I took it back and then decided, nah, I'll give it CNZF. Again, we cannot turn
Darin
00:25:29.645
about that in a minute. Yeah, I'm gonna, I'm gonna ask a question about that in just a minute. Holmes, GPT from Microsoft, it's AG agentic troubleshooting for Kubernetes. Dlec
Viktor/Whitney
00:25:38.825
How's that different from Kate's? We, wait, wait, wait. GGPT, uh, I need to backtrack. Did you say home GPT from Microsoft.
Viktor/Whitney
00:25:53.225
Okay. And how's that different from Kate's, GPT? Oh, no home's. GPT is mostly about alerting and stuff like that, or at least it was before, the last time I checked it. Uh huh uh, kind of, you know, I'm going to watch what's happening in your cholesterol alert you when something is wrong and things like that. And Kate's just a bit like a chatbot available. Yeah. PT is basically, an agent that talks Kubernetes like any other agent. Okay.
Darin
00:26:17.572
So it looks like it was actually Holmes GPT was accepted October of 25. So it wasn't during this one. So my, my dates are not, I should have done some, basically I had Claude help me write the notes and I didn't go through every line item here today. and it said from Microsoft, but who knows? But okay, again, I could be wrong. Uh, so I won't say names anymore, but I will say project names Dlec, D-A-L-E-C. It's a declarative system. Package specs with SOM. To me, s words, ress, we mentioned it's the AI Knative gateway. I will say it's from Alibaba. So, uh, metal three is moving to incubation for bare metals, bare metal Kubernetes provisioning. but donating Valero to CNCF is a big deal. Uh, but the question behind that is, uh, is CNCF now becoming a dumping ground for projects that companies don't wanna maintain? Or is this a healthy thing to be happening?
Viktor/Whitney
00:27:14.367
I think it's both, right? Uh, like those, uh, that you AI related influence this or that. I'm pretty sure that that's what, companies are actively working on, And, uh, they will continue working on it for a while. And then there are others like, okay, I dunno what to do with this thing. Let's, let's go to C ncf.
Viktor/Whitney
00:27:38.440
What, what's CCFs incentive to, to accept projects that are being dumped? I dunno. More sponsorships. Yeah. If you're a big enough company and support, I mean, it doesn't cost them, doesn't cost since you have to accept projects. Okay. Yeah. Accept volunteer time to review them. Mm-hmm.
Darin
00:27:59.892
I wanna review a couple of the things from the big hyperscalers AWS school and Microsoft, uh, EKS can scale to a hundred thousand worker Node.js in a single cluster. Uh, do you have the money to run a hundred thousand worker Node.js? That's different conversation. Uh, four nines s yeah, I guess that's the only thing I can think of. Four nines. SLA for provision control plane clusters. Uh, neuron, DDRA driver. and the other thing they mentioned was just AWS backup for EKS. backup even matter at scale? how do you do a, uh, rehydrate if everything's down? I know.
Darin
00:28:48.931
Uh, Microsoft talking about their Kubernetes application Network Mesh List. Istio Fleet Manager, a KS desktop is now GA for local Kubernetes development. Have you heard about that? So
Darin
00:29:06.271
Well, no, no, no. I'm comparing it to Rancher desktop basically for me, but they now have a KS desktop as ga.
Viktor/Whitney
00:29:16.836
There are probably people who like desktop. I dunno. It seems, we were just saying everything's about to be a agentic and we don't need UIs anymore, but Cool. Yeah. Go, go, go for it.
Darin
00:29:31.041
Red hat. We've already talked about LMD a few times. and they're getting into the, the whole sovereign ai. We haven't mentioned that phrase yet. There's another one. I guess to go along with the, what was the c words that we were saying earlier before,
Darin
00:29:48.079
are people really gonna be able to run their own inference? Let's get real. Do, people really want to do that?
Viktor/Whitney
00:29:57.368
Yeah. What level depends on their scale, first of all. Right? Kind of. If you're talking about something really massive, like, I don't know, the HSBC one of the biggest banks in the world, they probably do, right? Certain Definitely does. No, for, vast majority of the rest of us, I feel it's too expensive, to run it and too big of a commitment and all that dress. and people then say, start saying, yeah, I can run a tiny model and I'm happy with it. And they don't know. I think that's, that's insanity. so yes and no, but the prices hopefully will go down and the market will stabilize and give it a couple of years and then yes. Right. especially in EU because there is that kind of, Hey, maybe we should also not depend on others, like others don't depend on us, type of situation, right? so here will definitely move in that direction, not necessarily on a company level, even on a company level. Uh, companies in EU are very much pressured to be, EU centric or something like that for good or bad. I'm not entering into that debate now.
Darin
00:31:06.164
that's, I guess, what we'll just have to expect out of the EU for a while, if not forever. already talked about Qver. Uh, there was a new release, 1.8, Lima 2.1, which is for Mac OS guest support, uh, added, whatever you wanna call it. AI Agent Safety Sandbox. Uh, with that, cube Scape four oh came out,
Viktor/Whitney
00:31:27.812
The trivia thing happened this week. Did y'all, that that was during Coon, didn't it?
Darin
00:31:40.397
LA last week late, or, and then there was another one that happened within the past couple days of, a Python package for ll. M light, LLM, light, something like that. these, all these keys are now compromised
Viktor/Whitney
00:32:04.022
but here's my question. That borders conspiracy theory, right? the, the the Tri afa, right. Aqua, they, they, they did not have a, this is the first cucu. They don't have a stand in Cucu. Oh yeah. Did they, did they always choose not to have a stand or do they No, no. They always had a stand. Always. Well, I know that. And a big one, like, do they have plans to have a stand at this coupon? That's question. And then like, that's my question of embarrassment. Yeah. That's, that's the conspiracy part. Or do they know? It cannot be answered from with certainty, but, uh, can be speculated, but no, no one, no Boot. No, no. And usually it's, it's not just a Boot. It's, it's a massive one.
Darin
00:32:47.170
so let's not throw Trivi completely under the bus. It was the trivi GitHub action that was compromised, not the Trivi standalone CLI, which I do use locally, and I did check and make sure, okay, I'm, I'm still okay, but now it's making me question. Okay, well then what do I need to move to now? Because do I trust the company? I'm not throwing 'em under the bus, but I'm just saying do I now trust the company enough to keep using their projects?
Viktor/Whitney
00:33:19.305
You know. that can happen to any company. Kind of the, the, the, the speed of reaction and transparency in that reaction is what is being questioned. Rather than that simply happening. What, tell me the story of that. What, what the, I think it took them five days or something like that to actually, uh, say what happened and why it happened, and complete lack of transparency and all that stuff. Right.
Darin
00:33:43.744
But I also have to say on the other side of that, it's like, look, you've gotta get your ducks in the row before you start saying anything. So is it just timing? We, we don't know. We don't know. I wanna ask something about platform engineering. have we moved on from, Hey, we've built an IDP now to, is what we have to do. This is it.
Darin
00:34:06.166
Well just, okay, hey, we built an IDP, right? That would be all, you know, end of the day, Hey, look, we built an IDP, we're done. Versus now it's like, okay, well you forgot. Governance, guardrails, standardizations, self-service at scale, all those kinds of things that people just, versus just building an IDP,
Viktor/Whitney
00:34:24.701
Uh, I heard this week that, that the bottleneck with platform engineering is officially culture. That the problem is getting the teams to work together in a way. The problem is people. Yes. That the technology and the strategy is sufficiently worked out, but that, that's the core of the problem that on my interpretation of problem that I interpreting is many people complain about the agents and how much I trust them and stuff like that. How, how can a company that doesn't try trust their employee actually ever go into agent something? If you talk about talking, agents are doing this, agents are doing that. And you have a co bunch of companies that realistically know you as a human, you as intelligence in any form of way, carbon or silicone. You are not allowed to do anything but the bare minimum required for whatever you're doing, You can write code, but you cannot review code. You can review code, but you cannot deploy code. You can deploy code, but you cannot scale code and so on and so forth. Right?
Darin
00:35:29.284
it seems like as maybe what we need to do with platform engineering is just apply the SUSE model to it and just let the agents take care of everything for us. That way we eliminate the, the humans.
Viktor/Whitney
00:35:39.379
I'm sure that that's coming. I'm, I'm not, uh, that's coming. Variation that's coming. I don't think that we are going to eliminate humans anytime so. But that we will be concentrate. I think that the real que the art that companies will have to master is to figure out what matters, what is important and what isn't. Most companies are, are having, uh, separating those things in a as blank statements, kind of like it's important to review every pool request, or it is not important to review any pool request. There is no. This pool request should be reviewed, And until you do that, you cannot really effectively advance to, to agent something because it doesn't matter how much you speed up any part of the processes. If everything else is equally slow or bad, you'll start to make, start making decisions on per case basis, right? Kind of Okay, agents, deploy this, deploy this, deploy this. John needs to review this one.
Darin
00:36:46.018
Well, we've talked about this is just because we speed up one part of the pipe or a couple parts of the pipe, unless the whole pipe is sped up. We're hosed.
Viktor/Whitney
00:36:53.638
Yeah. And, and the, the big part of that speeding up will have to be the decision of when person or people in plural need to be involved in something. And that cannot be a generic statement anymore, right? It cannot be always at this point of the, of the life cycle.
Darin
00:37:13.220
Let me ask this question. you could argue, some people might say, AI on Kubernetes is over engineered. Okay, fine. Let's assume that's true for a moment. If your team doesn't need inference, then okay, we buy that. Uh, the only people are gonna be able to afford this now is true enterprise, like stupid money enterprise to where they lay off, you know, 3000 people at a time, and then the. At the same time, the CEO is getting a new jet, right? That's the kind of enterprise that's gotta have the money to pull all this off. Do you think that most Kubernetes clusters will never see a GPU
Viktor/Whitney
00:37:47.392
No, I, I, I think the opposite. I think that most, most clusters will see GPU, whether they will be able to afford full blown, kind of like. Frontier models or something smaller and stuff, but everybody's experi experimenting in some form or another. Right? I doubt that there are many companies that don't have a node with GPUs today. Today. Well, whether that's self-hosted, kind of like in my data center or GP on a w. There is some GU there in the short term, they all are gonna see a GPU, whether they're gonna keep using A GPU and see value in the cost. Yeah, you need to let them fail first. They're gonna be like, you know what? We had chat bots before that used fuzzy search and that worked out okay. Maybe we don't need AI behind this. It's gonna take 'em a while. Little while to get there.
Viktor/Whitney
00:38:39.238
Freaking amazing. Yeah. We switched it this time. I was, I did the demo. I monkey was the demo monkey, code monkey and he was the storyteller. Yeah, old dogs do learn new tricks sometimes. I would probably be not allowed to attend any, any more dogs he may have. This is my last talking chance here. No, I, I shut her down for being a woman. He did say Shut up woman. At the very beginning, my mic wasn't working, so I couldn't even talk back. Um, that's the best way to, to shut down women. That's the only time you feel bold enough enough to prevent from talking back. Exactly. Kind of like you think I'm, I'm just bursting without playing. Come on. You know.
Darin
00:39:23.241
Alright, so I'm trying to correct myself if I can, and I'm having a hard time finding the information. Uh, we're, I'm going back to home's, GPT to figure out, 'cause I initially said Microsoft, uh, I can't find any specific things. If so, rib Robusta may be still around right? I, I don't know. So if I'm wrong, I, I will go ahead and retract what I said before. Um, uh, well, I could blame Claude, but I should be reviewing what Claude gave me better. I did review things, but then I looked at other things and I ran out of time, so I didn't, I figured all those look pretty good 'cause I said, Hey, make sure you include the references where you saw it, but this section of where they gave me the sandbox project didn't include the references.
Viktor/Whitney
00:40:09.246
I think we can all relate to what you're telling us right now, what you're going through.
Darin
00:40:14.001
Okay. Yeah. Well, so how many people were there this year? Was it up or down from last year? We
Viktor/Whitney
00:40:24.261
I heard 12,000. I heard three 13,500. Oh, nice. I think that's 13. Was something around that number was announced in the keynote. It was huge. It's is the, uh, I mean, I don't know the numbers. Uh, they're always faked, but it felt like the biggest cube con. To walk from one end to the other. It took like 20 minutes. I, I was almost late to my own session 'cause I didn't leave enough time to, I didn't understand the scale of the building itself
Darin
00:40:55.941
Was it at the same venue as the Amsterdam, uh, conference before or was it a different one?
Viktor/Whitney
00:41:00.369
at the same venue. But before they were using different floors or something. It wasn't a sprawling, yeah. Yeah. Or maybe 'cause
Darin
00:41:06.759
the same, it wasn't the exact same layout as before, so it's hard to tell Yeah, the, the initial. pre-show number was like around 12,000 is what they were expecting. So, but again, I, Claude gave me 12,000. It's like, Hey look, did they actually report that? And I was like, no, I just saw that was a gas. I went, okay, you gotta take that out. 'cause No, that's, I don't know, but it sounds like it's bigger than what we've seen in the us.
Darin
00:41:38.109
Biggest ever. So why is that? Is it just because of the whole a Ai, ai chaos right now or is it something else?
Viktor/Whitney
00:41:46.447
I'm not sure to be honest. I'm not sure how much people are into AI to be honest, at, uh, attendee. I, I have a anti AI vibes, but I don't speak with fellow vendors in a way. I, I actually like had a conversation with some. Um, a random DevOps engineer who called himself a DevOps engineer. I know you love that title. I love it. Um, uh, who was saying like, for, they've been talking, he, it's his third coon. They've been talking about this, AI every coup con and this is the first time that his company actually has a use case for it and is excited and hungry to learn about how to actually implement. Very cool. Some of these things. Yeah. Very cool.
Darin
00:42:27.902
but I'm also saying, I don't know that I would've done it much before now, like thinking back to
Viktor/Whitney
00:42:42.977
Yeah. And they were talking about training back then, and that's not very relatable. Yeah. No, we just spoke about how you likely cannot afford inference a little training, training.
Viktor/Whitney
00:42:59.072
You cannot even train people who join your company. Who are you going? What are you going to train?
Darin
00:43:05.297
So what would you say overall that you thought of this cube con would do you, if people had thought about going to it not to go? Did they miss anything?
Viktor/Whitney
00:43:15.836
is my, one of my favorite CU coupons. Really? Yeah. Same. Uh, the only thing that I'm personally missing is that I don't feel so much Kubernetes related excitement as, as we as kind of like five years ago. Right. Uh, you know when, when you go to a keynote and then, then you learn five new things you never heard before, but you know that you need them, right? a mature platform, so, uh, that's the only thing I'm missing. But other than that, uh, this was an awesome, one of the best and the platform's mature, but all this AI stuff is still brand new and changing every second. Oh yeah. So you do get some of that exciting, like, oh, wow, I didn't think that was possible, or I didn't consider it that way. Yeah, yeah. But it's still kind of. You know, the, that's, that's still, it's not such a broad scope of users. Mm-hmm. Like if you talk about inference, that's exciting, but really kind of like, I don't have 500 k kind of Yeah. Just to run it. a situation. I, I think that, uh, there will be more focus in the future. I, I feel, uh, probably starting next CubeCon on, on agent. Mm-hmm. And that's more relatable, kind of everybody can create an agent. I saw quite a few sessions or a couple where we're talking about building agents or building or using skills, cloud code skills or I guess, uh, agentic skills as part of the solution to problems. And that was, that was all really fun. The new type of tooling that's available.
Darin
00:44:40.261
Yeah, a markdown file seems fun. Yeah, that, that seems just completely odd. Uh, by the way, Viktor, since I don't know if you saw it or not, uh, you were talking about Claude figuring out how to, you know, click around on my desktop. You didn't say it exactly like that, but trying to figure, they actually launched that this
Darin
00:44:57.816
I didn't know if you'd missed it. I didn't know if you'd seen it with you traveling or not,
Viktor/Whitney
00:45:17.431
It. I, I Automo. Okay. I, I read that it improves memories. Yeah. But I don't understand what that actually means. I mean, uh, cloth md Oh, okay.
Darin
00:45:28.681
So think of it. This, this is the way I explain it, even though it's not right. So throw it out. Um, is you, you did all your work for the day. You've had this auto memory going all day long, but. It's all over the place. Right? It's just, it's, it's the memory of what happened versus dream. Now consolidates all that into something more useful, concise. It's, it's compacting without losing everything, I guess is the way to think about it. Like what, what is the true thing of knowing and the auto mode, I don't know why we're talking about this, but the auto mode was somewhere in between the accept edits and run dangerously. It's, it's the right thing, I think, what a lot of people are looking for, but I still
Viktor/Whitney
00:46:15.216
Actually, I'm excited for that actually. Probably by accident. You know, I was, uh, involuntary, better tester like. A month ago or something like that. I noticed that, uh, I had one, one of them in auto mode, you know, write files automatically and all of a sudden started writing to, to CLO MD that is not in that project, but, uh, somewhere else kind of like, what the heck are you doing? What's, what's, what's being done to me? I check release notes. There is nothing. And then I started freaking out, uh, for a second involuntary beta tester. There we go. Mm-hmm.
Darin
00:46:49.636
So let me ask this question 'cause we, we talked all around ai. Were any of the big AI vendors there, philanthropic, open ai? Did you see any of them on the floor?
Viktor/Whitney
00:46:58.411
I, I think that the Tropic had at least one session, but not as exhibitor on the expo floor. I, at least I haven't seen them, I haven't seen open ai. Google is there, that's a vendor. I'd spent a good amount of time on the expo floor and I didn't notice philanthropic group in ai. I feel like I would have.
Darin
00:47:16.440
So that would be interesting to as a, as a way point for next year of, okay. Are they going to start showing up now or not? My gut tells me no, but.
Darin
00:47:27.285
I can't think of one at all, because their, their, customers, their true customers are the hyperscalers, in my opinion. That's,
Darin
00:47:38.175
and without the, the customers for the, the big AI companies, the foundation companies are the hyperscalers, I think. Right. And it's not so much that they're the
Viktor/Whitney
00:47:51.360
think about it, kind of like there, there are different areas, training and inference. Two of them. Why wouldn't tropic want you to rank me?
Viktor/Whitney
00:48:01.200
Uh, model yourself, right? Uh, they definitely. And the, the only area where they might show some interest is, is actually using their APIs and then through agents and not, but I feel that actually this crowd is too small for them. Yeah, right. It's a big business on inference models. Right. Uh, our inference, but using agents to say, okay, so in every company for every thousand employees there is, there are two ops people. Let's target them. let that be our business model, every developer is almost every developer, and I mean application developer, not software engineer is using them. And that's the bread and butter, right? That's 90% of technical part of the company. We are such a small minority in.
Viktor/Whitney
00:48:56.178
The only thing that I would add is that this, the, the whole AI thing reminds me so much of first, let's say five years of Kubernetes, where everybody's rightfully focused on very low level components, and then everybody else. Is somehow trying to figure out, okay, so I'm going to have to combine those 57 different things. This hurts my head. And the next wave will be probably higher level of abstractions Kind of like, okay, AI platform model, I dunno how to run inference. That includes those 57 different tools. now if you, if you go through the landscape and you say, okay, so none of them gives me inference. Mm-hmm. Maybe in a basic form, yes, but in general, I kind of fit everything I need and probably we will see, I could see that projects and companies saying, okay, this is the project that you use to run inference as example. Right? And that will, behind the scenes use bald and elegance that we see in CNCF or, or our projects built. An agent that can interact, we'll start seeing, instead of us trying to make our own agents, seeing agents being big unit of. F that we're all working with? Yeah.
Darin
00:50:12.289
I guess it'll get interesting, predictions for the US one this year. Is it just gonna be scaled down version of this? And is the EU really becoming the primary conference Cube Con?
Viktor/Whitney
00:50:25.539
Uh, because of the political climate in America right now, ICE agents are at airports. Um. a lot of people I've talked to are saying they're not gonna go to the America Conference this year. Yeah. But I feel that if no, if you exclude vendors mm-hmm. that probably does not apply to attendees because they, they go to one or the other. Yeah. A normal person doesn't go. Okay, that's fair. So I'm guessing that North America is pro, probably mostly people from USA Canada and everything south of USA Uhhuh. I will say several Canadians told me they're not gonna say put an American soil. Yeah. But I think that since CO US was always like, like 10% smaller. I dunno how significant that is, but and now it might be more than that. We'll see.
Darin
00:51:16.663
my real point on that was do we think we'll see larger announcements there, or is now is, has EU become the place where all the big announcements happen?
Viktor/Whitney
00:51:26.923
I don't think that companies, especially with today's space and battle ground in AI, that they can wait for, oh, I have something new. I'm not going to announce it until be you. Uh, so announcements are gonna come when they come, when they come, when there's something to announce. Uh, yeah. I, I don't even think that whomever has something to release two months from now will not wait for Salt Lake City either. they will keep announcing and then maybe, you know, usually, oh, if we have something that is announceable a month before, then we will wait. I wonder, you know, how like philanthropic has something, some new big release of some feature almost every day, several times a week it feels like, right? Like I wonder. Because they're using AI to build ai, it start, it's like an exponential speed of of Oh yeah, innovation. I wonder if that's gonna happen in our space too. It seems likely that it would. It'll definitely happen. The speed will increase for some, not all, but you know, some are still struggling with mainframe, so it's a culture
Viktor/Whitney
00:52:39.381
Okay, so here's my bet, and I'm willing to take it against anybody. It'll outlive ai, the mainframe. Oh, yeah, yeah. Eventually AI will be replaced with something else, so LL, let's say LMS will be replaced with something else, but mainframe is here to stay.