Chris
00:00:00.000
It's like if you gave a caveman a stone, flint stones that they could start a fire, or you gave them, a flamethrower. I think if they know how to use it, they'll say, "Oh, this is a lot easier than using the flint." Most of them, in my experience, overestimate their prowess, and they confuse frequency of use with proficiency. They might use it all day long writing emails or summarizing documents. That doesn't necessarily mean that they're getting the most out of the model. This is DevOps Paradox, episode number 372, Using AI Every Day Doesn't Make You Good at It
Darin
00:01:35.684
Viktor, I have a question for you. What do you think when any kind of C-level says, "Why can't we just have AI do that?" Do you actually think they believe what they're saying?
Viktor
00:01:50.690
Some do, most don't. Actually, no, and I've changed my answer. Most of them do. Your question was wrong. Do you think that they're right?
Viktor
00:02:02.604
Yeah, they believe that they're right. Yes, Yes. Look at it this way. CEO is really primary preoccupied about the finances of the company. They make changes because they think it's, is going to be more profitable. Whether they do the right changes or not, separate question, right? But if they do fire half of the people because they think that AI is going to replace them do think that
Darin
00:02:28.245
On today's show, we have Chris Diegel on from DeepBreath, chiefaiofficer.com. Gee, I wonder what he talks about at chiefaiofficer.com. Chris, thanks for joining us today.
Darin
00:02:42.013
What do you think of m- the question when a C-level says, "Why can't we just have AI do it?" What's really going through their head?
Chris
00:02:49.043
I think that they have talked to some peers who are hyperbolic about the impact in their own business, and they're believing what other CEOs are saying, and they think that it can actually… That it's magic
Viktor
00:03:07.814
excluding the scope of that AI is touching, exclude… So ignore the scope. Is that any different from previous cases where CEOs are convinced that switching to cloud will change everything, Or adopting Kubernetes or any of the other technological changes, right?
Chris
00:03:26.215
I do think it is different because their kid is using it, their wife is using it, their husband is using it, the frontline staff is using it, and it's making it accessible to anybody who can type in a chat box
Darin
00:03:40.360
What do you do with that though? Because you've got the wife, the kids using it, but that's not using it for a business case
Chris
00:03:49.234
They're having an oh my gosh reaction to the output or the speed of the output or whatever. So I think that they're understanding that, oh this translates into anything that I do. I could use it for business, I could use it for a hobby, I could use it for planning the diet. So I think that they do understand that it does translate.
Viktor
00:04:08.934
Are they wrong or what do you think? And I, And I'm not asking about their capability to implement that change, but will AI actually have such an impact?
Chris
00:04:21.409
It's like if you gave a caveman a stone, flint stones that they could start a fire, or you gave them, a flamethrower. I think if they know how to use it, they'll say, "Oh, this is a lot easier than using the flint." Most of them, in my experience, overestimate their prowess, and they, confuse frequency of use with proficiency. They might use it all day long writing emails or summarizing documents. That doesn't necessarily mean that they're getting the most out of the model. So I guess that would be my answer there.
Viktor
00:04:57.089
let's imagine for a second that they are right independently of whether they're capable or no. How much does it matter what your company is doing and how it is working to the whole story, my previous example maybe was not on par with AI, but I feel that most of the time the decisions are not wrong, just the capability to do what's needed for that decision to be successful was not there
Chris
00:05:25.587
I, I think that if they stumble through it the executive, the decision-maker kinda stumbles through it, they assume that everybody else can do the same. And if they were simply to just get a bunch of licenses and hand them out and say, "Hey, everybody, start using AI," there would be people that didn't touch it, there'd be people that used it once or twice a week, and there'd be, 2 to 5% that were like, "Oh my gosh, this is transformational." But as long as somebody touched it, they'd say, "Oh," they might say, "This is better than Google," or, "This is better than…" If they even know what Perplexity is. But yeah. I would say that would probably be a, an accurate assessment. The missing piece is, I tell this kind of story. I've got twin daughters, they're 17, they went to go get their driver's license. They had to take a written test first before they were able to conduct the vehicle, And I think that what a lot of businesses are doing… Now, we focus on lower middle market, is kind of our range, so we don't have a lot of experience in enterprise, and we don't really mess around with the solopreneurs or the small teams. It's just kind of where we've s- we've settled in the, food chain. I would say that most of them are going straight to trying to drive the car without understanding the mechanics of driving the car or the rules of the road. And to translate that, most of them are giving out licenses but not training the individuals on I call it thinking in AI, where they have made this, transformation from, "Oh, it's kinda like Google" to, "Oh my gosh. If it can do that, it can do this, it can do this." And the matrix unfolds and somebody starts dynamically thinking about, "How can I use this throughout the day?" As compared to, we kinda look at it like most people use it like a drive-through window. They go, they type in their answer, and they leave. Where the reality is it's, as Ethan Mollick from Wharton calls it, it's being a cyborg. It's a seamless use of the human and the tool. Doesn't mean they're always using the tool, but they're thinking in AI, they have the understanding of, what the tool's capable of, and they're making that decision a million times a day. "Oh, I can use AI. Oh, I could, I should," or, "I couldn't. Oh, no, this is better if I do it." So that's kinda my position on it.
Viktor
00:07:32.984
If I understand correctly what you're saying, basically y- you're pointing to lack of experience, and if that's the case, then should not matter whether it will be successful or not because given enough time, it will be, the way You're describing it
Chris
00:07:48.638
I would say that at some point with that frequency versus proficiency argument, Using it a lot means you're really good at the small scope of things that you're using it for. But you haven't necessarily asked yourself… A, a buddy of mine wrote a book called AI-Driven Leader, Jeff Woods. Jeff calls it like a thinking partner, If you're not using it in that capacity, you're only gonna be as good as what you apply it to. yes, If I got behind the wheel of a car and had never driven before, there'd be some stops and starts, but eventually I could get it on the road and keep it between the lanes. Versus if I went and spent a weekend in a driving class the minute that I went to go, do it on my own, I would have fundamentals and a foundation that would mean that I could go anywhere, as long as the roads were there. And I equate that to the training or the upskilling efforts. If you don't do that, your people will… Some will be better than others, some will figure some things out, but it will be accidental and organic as compared to intentional and impactful
Viktor
00:08:50.338
now my experience is more in larger enterprises, so I'm not sure how much that relate to yours. Is that actually enterprises, maybe not even consciously, do their best to remove the thinking from the equation among their workforce. I have a story that AI jo- jobs where we adopt fully AI agents and so on are actually almost exclusively thinking. So it almost goes in contrast with what people were taught to do, Typically in a big enterprise, "Oh, no you, you do only this. Don't worry, there are 57 other teams that do something else," You know only this tiny part of that." And that to me feels so fundamentally different than how I feel we should be working with AI.
Chris
00:09:43.534
Yeah. the timing's uncanny. This morning I interviewed a gentleman named Rana Gujral, who's got a book coming out called "The AI Instinct," and this is the entire general thesis of his book, and it's human plus tools plus rules. a big part of what he's suggesting is that if we only rely on the tools, we will have this fatigue or this atrophy of our decision-making and our, our judgment and our instinct. So a further addition to that was he said that the new role for individuals that are involved in agentic environments and things like that isn't the doing of the work, but it's the planning of the work. So that planning, if they don't have a lot of experience with delegation of those activities to another resource, whether that's artificial or, authentic that's a whole new skill set. And the thinking that I'm referencing is the individual doing the thinking, not necessarily the tool doing the thinking, but the individual being that, conductor of the vehicle.
Viktor
00:10:41.736
Yeah, I w- I was referring now to human thinking, which I feel somehow to be very absent in lar- large companies, right? Mo- mostly because of the division of work that I mentioned before, right? somehow leads me to think that ignoring now whether that's your role or not, that we are now more talking about types of roles like architects, managers, leads.
Viktor
00:11:09.563
Yeah. people who actually big picture, I'm capable of delegating rather than I'm capable of doing something.
Chris
00:11:18.481
I think the risk is for the people who are exclusively the doers as far as like the introduction of generative AI into the operations of a business because this is a whole new skill set that they would have to learn. if they pick up on it quickly, great. But if they are not capable of architecting multiple mechanisms that can act on their behalf to produce the deliverable their role will-- You mentioned in the enterprise they have a very discrete environment in which they work. That discrete environment will shrink even further to the point to where they may say, "Oh, it-- tokens are cheaper than Janet." So that's kinda how I'm seeing it.
Darin
00:12:03.037
I'm gonna go back a few minutes in what we were talking about. You said the word licenses, when it came to Sotnio getting some licenses for AI seats. What flashed through my mind, and if you were to go back and listen to the past five, six minutes and apply what I'm getting ready to say, it's going to feel very the same, sort of like what Viktor said about cloud Kubernetes. I heard Microsoft Word or Microsoft Excel in this last five to 10 minutes of conversation of the same thing applied there. We could figure out Word by playing with it. We would never be great at it. Same with, thing with Excel. But if we actually learned, took the test, and instead of it being organic, we were actually structural in how we went about using the tool. One thing I'm thinking is y- you talking a lot with executives and again, in a small and mid-market what are you telling those executives? Are they just trying to… Because th- they're sort of like the lesser cousins, I'm sorry, it's true, of the enterprises for most places. Some mid-market is… How do you classify mid-market? Let's start there real quick. What, What is the top end of mid-market to you?
Chris
00:13:07.445
I think like the official definition of even a small business by the, SBA is a billion dollars in revenue. Most of the companies that we're working with are doing between 20 and 500 million. They might have anywhere from, 50 employees to less than 1,000. So that's the sweet spot where 90% of my experience is gonna be
Darin
00:13:25.890
so unlike enterprise, in those cases, you've got people having to do multiple roles, right? It's, It's not sliced down as far. So they're already having to split their time just getting the day job done, and now we're saying, "Okay, hey, AI is going to help you and do it for you. Uh, It's only gonna take you two months to teach it what you're doing." I'm not gonna do that. Come on, I don't have time to do that. Do you, Do you still wanna make the revenue numbers for this month, even as a private company? Because it's even better when you're private because you don't have to… You're not beholden to the shareholders. what is their thinking now? And y- and let's step down beyond that again. Most of the time you speak with the executives or leadership, but I'm assuming you probably have some IT peopl- dropping in in those meetings too, just to try to keep the executives honest
Chris
00:14:14.707
you're an executive listening to this and you're thinking about doing something like what we do, I would suggest have your IT people there for a couple of reasons. One, so that you understand they're probably not the people that will introduce applied or operational AI into the business. It's not their role. But a lot of times it's not as much, but now w- we're seeing executives are recognizing that they're-- I don't even wanna say parallel role, at least what we do. Yes, it's a technology. Yes, it needs to be governed and those sorts of things. Yes, there needs to be security introduced into the conversation. But our approach to introducing AI is, to operationalize it. What do you do f- on a day-to-day basis? Let's identify pilots from that. So our I guess approach is much more from the process environment as compared to the technology. So and with the executives, what do we tell them? First give me about seven or eight hours. We do something called a kickstart, but it's an immersive day that we go on site. By the end of that day, they don't need the two-month ramp up. They are using AI at a quasi-expert level and doing things that truly 99% of other users, regardless of how long they've been using it, are doing. One of the biggest issues is that, okay, great, we gave everybody licenses to ChatGPT or Claude or whatever, or, those people are using an off-the-shelf tool that has to be seasoned, it has to learn about you and your preferences and those sorts of things. So one of the first things that, we do with an executive, and I would encourage anybody listening, if you haven't done this talk to the IT team, of course. But with the introduction of connectors which allow for that large language model to now have access to my inbox and my calendar at least, or, and maybe my Teams chat or Slack, one of the first things that I would do is I would ask the models to go through maybe the last 30 days of emails in the next two couple weeks, in the past couple weeks of my calendar events, and start to create a pattern of life, a work life for me. Who do I talk to? What kind of meetings do I do? What is my language style? What are the unit economics of the, industry that we're in? And give me output that I can put into certain areas of the settings of my LLM. So moving forward, there's no more guessing or me having to iterate nearly as much because the model has a deep understanding of me. It becomes from a plain vanilla to a much more bespoke application. Once that happens it's a completely different tool. And then throughout the rest of the day, we'll introduce them to some of the functionality that might seem intimidating but is extremely powerful, such as skills, such as workspace agents in ChatGPT, such as CoWork and Claude and things like that. So by the end of a day, that executive and even frontline staff who sat in that room are using the tool at a level that organically or stumbling through it, they might never get to. So it doesn't take nearly as long if you have the right framework to start with, I guess.
Darin
00:17:12.961
what's one of the more common pilots that you're seeing today? And we've heard about pilots not working out great in general, but that's true for any pilot over time. what pilots are you seeing that people are doing and that are actually reasonably successful?
Chris
00:17:29.889
To keep us to where I don't necessarily have to have a career in a certain industry, we focus on the sales, general, and administrative activities initially. Stuff that every business does. The operations, the HR, the finance, sales, marketing, those sorts of things. And where I normally recommend that the company start is in finance. It's not to replace the human, but to support the human a- and to chase up the receipts. And we, we never start with anything that's gonna be like, there was that MIT uh, study that came out in, in December of 2025, and indicating that 95% of pilots at the enterprise level were failing to impact the P&L. A month before that, Wharton had come out with a study of primarily lower middle market and SMB, and showing a 73% success rate or 75% success rate on pilots. So I think the difference is the environment. in enterprise, both you guys are much more familiar with enterprise. I was an Accenture consultant earlier in my career. I swam through the dysfunction that exists in all of those enterprises pretty regularly, so I wouldn't necessarily blame the technology. In the SMB environment, the people making the decisions about what to do or the mid-market environment, they're probably much closer to the process, and it's not as strategic, where strategic would be forcing top-down initiatives, and I guess in the SMB, it's much more like bottom up. "Hey, we know how this process works. I'm actually doing the process. Oh, this would help a lot." long answer, but I would say finance for sure. We do a lot of work with commercial construction light manufacturing, things like that. Why? I don't know. Because I think that one of the, guesses that I have is they're both not necessarily high margin environments, so everything they can do that focuses on SG&A, that's gonna hit EBITDA, it's gonna hit bottom line, so that's a very exciting prospect. But in those cases, a lot of it has to do with estimation compiling all the information, all the requirements from the prospect or the client, and being able to calculate, like, how do we make our profit margin here and still be competitive price-wise? generative AI, when we introduce it, is able to do some amazing things as far as speeding that process up removing, the majority of the human error, provided that I've got that, expert that's evaluating output before we ship it in a proposal. But those would be some areas. then generally, a framework would be, you want to pilot to win, identify something somebody doesn't like doing and they're not good at in their role, and start there That tends to be a very effective initial pilot approach. Small scope the morale is higher because, "Ooh, I hated doing that anyway," and, "Oh, this AI thing, it's pretty great. I don't have to copy and paste from this spreadsheet into this spreadsheet 40 times a day every time I do this." And there's a ton of that in every business. At enterprise, at SMB, there's a lot of that kind of like wasteful human being the Mechanical Turk as compared to AI being able to step in and do it.
Viktor
00:20:22.609
What's the end result of all this? Let's say that AI makes you twice as productive or twice better. So we now have half the people? Do we do twice as much as companies? Do we do twice as better? How does this look like, let's say, next year, two years from now, whenever, right? What's the outcome?
Chris
00:20:47.524
the paradox exists that people that are actively using AI are actually busier than they were. So this idea about, "I'm gonna have AI do my thing. I'll go take a long lunch," a two martini situation, that o- on paper would make a lot of sense But that's not what we're seeing, and that's not-- Generally analysis is indicating that people are actually doing more work. Not because the AI is doing stuff, but they're actually spending more time in front of the computer or spending more time at stuff. Where does this end up in a couple of years? Big question for sure. Fable 5.1 came out today that like I haven't even used it yet. Is it gonna be that much better? I don't know. But regularly the models are improving, not just incrementally but wow, this is a big difference. And I think that whatever we predict today, we're gonna undersell, unless you're Peter Diamandis or something, you're probably gonna undersell what the environment actually looks like. I think that and I'm not that, Pollyanna. I'm not-- But I'm also not a, a doom and gloomer or a EAC as they call them, acceleration at all costs kind of thing. I think It just doesn't make sense for me to continue to bring on more people on my team if the existing team is able to ingest more expectation from their role because AI is if planned correctly and architected or orchestrated correctly, AI is doing a lot more of it. So I don't know that the job… But then there's the was it Jevons paradox? That there will be more usage will mean more opportunities. I'm not an eco-economist or anything like that, especially a macroeconomist. I'm focusing on businesses where that decision maker, if AI is gonna replace a job, they know that person. At Enterprise, Dorsey, Jack Dorsey in, what was it, February, fired 40% of the staff in his company Block, jack didn't know those people, so it was a, a numbers situation. The math worked. so in Enterprise, I would expect to see a lot more of that. At lower middle market, which there's so many more businesses at lower middle market and small businesses, that's, you know, uh, numbers-wise at least volume-wise. I don't know. That's not something that I hear in the conversations. They're not like, "Hey, listen, psst, can you tell me how to get rid of people?" I think what they're looking for today and as far as they can see on the horizon, is more output doing less of the crap work faster turnaround, and just generally more enjoyment from the role for their employees. but also, "Oh, this is the new thing. We need to keep up." So there's a lit- a little bit of both.
Viktor
00:23:15.139
The reason why I'm asking that, this is for my personal curiosity, right? Is I'm wondering what is happening depending on the size of the company, right? I can easily see the future in which a s- relatively small company, 50 people, 100 people, maybe a few hundred, right? Can actually take into the ring giants. 50 of us and an army of agents type of situation, right? What I don't fully comprehend is what happens with massive companies. In which world a company needs 100,000 employees? And I'm not talking about manual m- type of work, right? Which would be a whole different conversation, but let's say, Knowledge work type of company. Yeah So w- what do we do with 100,000 people? to be honest I never understood what we do with 100,000 people before AI. To me, that was always a mystery. But i- it feels like a pl- a level playing situation
Chris
00:24:16.362
Yeah. So, you know, When I first started paying attention to this when ChatGPT 3.5 came out, there was this prediction that AI would create the first one-man billion-dollar company, right? And I think that that was achieved last year. Now, they were doing some shady stuff and blah, blah, blah, but that was done. So I think what's gonna happen with the smaller folks taking on the incumbents like I wouldn't wanna be necessarily like Salesforce five years from now. What we're seeing is that the smaller companies are like, "Ooh, I wish I had a dashboard that did this," and somebody says, "Hold on, hold my beer." Built, right? Oh my gosh, this is taking an infor- like that sort of, I'm not gonna say on-demand software, but v- extremely personal software. Micro SaaS is gonna be something that… 'Cause mentioned earlier, Darin, about I'm using Microsoft Excel, I'm using Microsoft Word. Yeah, but unless you've done the class, you're using about 5% of its capabilities. And I would say the same thing at least in the industries that we're working th- where they've got these, like commercial construction, there's some big players in that space, and the licensing is expensive and the training takes a long time, but they only use 20% to 30% of it, but they have to pay for the full software suite. So I'd say those companies that have-- And, would you wanna be… I know that y- both you guys have, are technically inclined. Would you want to be one of, 50,000 developers or 20,000 developers in a company? That's gonna be tough because you better be in the top 1% to 5% 'cause otherwise, like AI or, or an agentic environment is going to be able to "What are you doing?" "Oh I'm, I'm sleeping. I'm spending time with my family." The robot ain't, right? Like it's still coding. Yeah, and then the top 1% to 5% are just reviewing the output, very similar to what they do with the humans now. So those 100,000 employees, unless it is installing some sort of a, a line or a device or servicing some equipment natural selection to some degree is gonna occur. If you really wanna maintain economic viability as an individual and you're not using these tools or you're, thinking, "Oh, well I write, I use Copilot and I'm good to go," I don't know, man I don't see it. It's gonna be interesting. But then Elon and others have their positions on taxing the AI companies so that it can create, as Elon calls it, a, a universal high income. Boy, that sounds interesting, right? Instead of universal basic income. So don't know. And again I guess the, thing I've got going for me is that I'm paying enough attention to it real time to where if there's some massive shift, I'm not gonna be caught off guard. But my mom might, my uncle might, because they don't have the luxury or the privilege to be able to spend as much of their day thinking about the paradigm of generative AI and its application in knowledge work
Darin
00:27:05.488
You were talking about that top 1 to 5% that are checking the work. I've been thinking about this flow, and tell me if this sounds reasonable. It's like, and, 'cause this is my workflow when I'm working with AI, is I'll work, come up with good clarity around, okay, what are the, what's the plan? Let the AI do the thing, the thinking, the work, and then I come back at the end to validate that everything's okay. That seems to be the, like a, a 108010 type deal to me. Does that seem about right?
Chris
00:27:37.464
Yep. We actually use that same phrase, 108010. Yeah, But here's the difference. You know Enough about it to be able to plan what needs to happen, On the macro and the micro. the junior dev doesn't. So if they can't think about a higher strata of application like I just… Maybe they need to get coffee and, and bring it to me
Viktor
00:27:57.879
I agree with that, junior-senior division. But I also feel that there was never a time when a person could bridge that gap faster
Viktor
00:28:09.969
Right? Because the ability that I have, no matter whether I'm junior or senior, to figure out things that I don't know is infinitely greater today than it ever was, I don't need to read 57 books and spend 17 years with networking to reach the level where I am as productive as you now
Viktor
00:28:34.708
so maybe me being a junior is a deficiency, and I agree with that, but I also question how long do I stay a junior in this era?
Chris
00:28:44.258
That's great. And I would say that the deficiency would only be if that junior wasn't ambitious or wanted to, maintain economic viability. If they're showing up, they've got the CS degree or whatever, they're getting the job, but they're still interested in, "Oh, I'm gonna go have a weekend," and, "I'm gonna go take a vacation," right? In success in any field, I would certainly say the grind contributes to your success as long as it's directed. So it would be the behavior of the individual. However, that new person is probably… I'm not gonna say probably, but my assumption would be that at some of the higher tiers of the business on the technology side, they're aware of AI conceptually, they're aware of it theoretically, but they're not actually coding with it because they're running a department and they're making sure that the numbers are, hitting the targets and those sorts of things. So that junior dev who said, "You know what? Let me figure out Cursor over the weekend," or, "Let me get Claude Code plugged in and hit it from the terminal." And those individuals are now like, they have five or six agents running. I would imagine that they could probably run circles around at least in production, not quality of production we'd have to still compare on that, but they could run circles around the more senior who had all the tenure but didn't necessarily have the bandwidth or the ambition to use the tools. But then you see things like with SpaceX AI, I guess is the new company name. But their engineers are talking about how they're using GrokBot, which is a function of SpaceX AI's acquisition of Cursor and that sort of thing. And they're saying, "Hey, like most of our engineers are running 100 agents." Not, maybe not simultaneously, but they've got all of these specialist agents, and they've got… They show up to the table with, "Oh yeah, my team." What team? "Oh, it's me and my, my bots." And they're able to produce as if they were running that company you were talking about earlier of a few hundred employees as an individual.
Darin
00:30:33.991
That's great because they're not having to pay for it. How, How do you do that as an SMB? Forget mid-m- mid-market might be able to pull it off, but an SMB wouldn't be able to pull that off
Chris
00:30:44.424
But I don't know that they would need to. there's a few things that would be in their favor. If, If I showed up and I had this capability but I didn't have the token budget, I'm gonna go raise money, There's a ton of money looking for these early opportunities to leverage that. Secondly, I think that the you know, it's like "Willy Wonka & the Chocolate Factory" where the kids go into that room with the chocolate fountain and they can eat everything, right? If they're not paying for it, sure, it's exciting, but the minute that they have to settle the tab on those tokens or get shut down because their account level doesn't allow them that much usage I think that there will be lessons learned introducing I don't know more thriftiness perhaps in how they're, 'cause I, I had an old boss once, he told me, "There's nothing that brute force and sheer ignorance can't solve." And he's joking, of course. But if that's how you're-- Yeah.
Chris
00:31:33.858
With enterprise budgets at least, yeah. But if you're that SMB, I think that you will by constraint, you will be forced to be economical with the planning, what you do that does not require tokens and token allocation and priority of projects based on token spend
Viktor
00:31:53.968
I feel that the whole discussion about tokens being expensive is so exaggerated. I have a completely opposite story that they're extremely cheap. If you get 20% of the value of a person, that's already probably more tokens than that person costs.
Viktor
00:32:19.978
That's a lot of tokens. I, I don't understand why people complain that it is expensive. I mean, I understand, yeah, you can maybe as a person generate $100K in tokens a month, but if you fall down to reality for a second, it's cheap
Chris
00:32:38.038
If you're getting that type of output from it, yes. And I think right now there's a lot of experimentation, and with experimentation, to me at least, that would indicate that there is going to be some wasteful spend while I'm figuring things out. There's this whole idea of, time is money. Money is time. If I can throw more money at it, I can compress the cycle, of figuring things out or how much compute I can throw behind something. And I'm not a… i'm not what I would call an analytical AI expert that's, focused on data science and machine learning. I'm much more on the applied side, which is how do humans use it? But I think there's probably a lot of that, and that experimentation is expensive. But once somebody nails it, that experimental budget now gets, "Oh, we don't have to experiment anymore. This application, this activity, we know what it costs, and it costs $1,000, and John costs us five grand a month. Oh, let's use the agent."
Darin
00:33:27.568
You said one word a little bit earlier, constraint. How many people do you think at SMB level or even mid-market have even heard of theory of constraints? Unless they've come do- unless they've come through the enterprise path
Chris
00:33:39.321
Yeah. Either that or an MBA or they're just like a growth-minded individual. N- none of them, but I think that they might not be able to recite the theory, but I think that they would have their own homegrown version of explaining it because they've had to experience it
Darin
00:33:54.440
The other thing too is the Dora AI report talked about AI as an amplifier for both good and bad. Is that what you're seeing as well? You've seen people it's, it's worked out really good and you've seen it to where it's gone up in flames faster than the flames could fly?
Chris
00:34:08.984
We don't see a lot of going up in flames. We see a lot of results that are kind of anemic. So I wouldn't necessarily say that like it, it just screwed things up terribly or it, put the company into financial straits, but They were underwhelmed by what they-- the results they got based on the narrative that they're hearing in the marketplace. Now, on the upside, for sure, no question, anybody that knows how to do this stuff, even if it's not necessarily, you only produced 40 widgets and, or you're producing 40 and you used to produce 20. Sure, that's easily measure- measurable. But on the, the human activity side, somebody that knows how to use this tool, I mean, like after a day with us, they know how to use the tool the bulk of the knowledge work kind of activities that, that you could use it for, and they're doing things that aren't being calculated, The cycle time for anything that they do where they're able to leverage generative AI effectively, it drops significantly. How you measure that, but it's not just measuring, oh, he's writing the email faster, but it's also what's the time that they're getting back worth? What's that opportunity value? And I, I don't know that there's been a, an accurate rubric for measuring that right now that I've seen or algo for measuring that that I've seen. It's easy to say it used to take X resources, now it's X minus, but I, I don't see a lot of people talking about the opportunity value of that increased, that available bandwidth worth?
Darin
00:35:27.756
Because they're getting more coffee. That's all it is, is it's extra coffee time. Let me set up a scenario, see if-- I want to see how you answer it. Let's say that I'm the software developer at a place, and my boss's boss sat down with his eight-year-old kid and came up with a, a software plan for Monday to hand me. And I take a look at that plan, and it's technically unsound, It came back as like, "Oh, you're gonna be using COBOL, and we're gonna be running it on Kubernetes." It can be done, but, you know, unsound. I only have one meeting to deal with this. What do I say to that boss's boss so they actually hear me as competent and not doing the typical IT, "No, we're not gonna do that"?
Chris
00:36:11.912
I would blame the models. I would say, "Hey boss, this was a great idea. I put it into AI and said, you know, 'How do we build this?' And it came back with some observations that I think we should consider." So I'd just blame the models
Chris
00:36:28.782
Yep. Yep. But somebody that they trust because, oh, it's AI, it knows everything. It must be smarter than the, the professional
Darin
00:36:43.638
It is a format that works. I was hoping we weren't, wouldn't use that one again. Typically in past, we would just blame s- the consultants or we would blame the whatever. Now we're blaming the machine. Can we truly get away with blaming the machine?
Chris
00:36:56.276
If you've never used it or you don't use it to the degree that, say, a professional or a power user might use it, you don't know, right? You gotta take that, "Oh, boss, I use AI all the time." Or the boss already knows that. If you come in and say "Well, you know, I was able to coax this out of the large language model," that person who doesn't know thinks like I would assume that individual would go, "I'm gonna defer to the greater intelligence, which is not me, it's AI." So if AI says it's wrong, despite hallucinations and stuff like that, I don't think that would apply in this conversation. But the architecture or the stack that was used, And AI said it would be cheaper, boss, or it would be less likely to break down, or it would have, higher viability as a an option for our clients or whatever. Blame the model. Good and bad
Darin
00:37:49.478
of shadow AI in your engagements with SMBs and mid-market? So I, I, I'm used to seeing it in enterprise 'cause that's just what we do. I would expect it from mid-market. I've done enough mid-market too. But I wouldn't expect it a whole lot from SMB because there's sh- typically just not enough bodies around to do a lot of shadow
Chris
00:38:03.226
a definition that most people are, assuming with shadow AI is that somebody's in the stall, on their break using AI on their phone 'cause the company said you can't do it. Now that's different. the shadow AI that I'm seeing, and I don't even know if it'd be considered shadow, but it might fall under that umbrella, would be I got a free account, I'm using AI. Oh, he's got a free account. Oh, he's paying 20 bucks a month. But they're individual accounts, and here's the risks with that at the SMB level that they don't quite understand. Hey Chris, you know how to use AI, can you do this and this and this and this?" Sure, I'm using AI for that. And I go to, I don't know, a networking event or my uncle says, "Hey, you know AI? I'll pay you more. Come work for me." When I leave, all of the context that is in my account about your company leaves with me. So I think that's the bigger risk And most SMBs, and lower middle market, they're not scared of using it a- as a company. Individuals may be skeptical or may be fearful or whatever, but I don't see a lot of hesitancy at that level because they're talking to peers, and we d- we speak to YPO and Vistage and EO, so we're, talking to a lot of peer groups, so that may taint it. But one person's coming in there and everybody wants to brag about how smart they are with AI. They're hearing that and somebody else is going, "Oh, shoot, I know that guy. He's not that smart. We can do the same thing." So there's a lot of eagerness to use it as compared to fear, which at the enterprise level that might be existing.
Viktor
00:39:23.095
what you just mentioned is curious. You said, and I'm paraphrasing "If I use my own account, I leave the company, then knowledge goes with me," right? But there is a risk for me if it's the other way around as individual, right? Oh, if I, I cannot leave this company now because do I start from scratch?
Chris
00:39:44.785
Interesting. when we sit down with companies, we tell them, "Look, have a personal account and have a business account, and whatever you do in the personal account should not be business because you're gonna taint the context or vice versa." If I'm going to do the report, but I've been planning a Disney cruise in my company account, it might introduce Mickey into that output,
Viktor
00:40:04.441
But, you know, I was more referring more to, you know, before AI when I leave the company yeah, the, code stays with the company, but my value doesn't, Everything I learned stays here in the brain, right? But now it's not here anymore. I mean, Or only a, a big part of it is not here. So I might feel that I'm now locked into that company, 'Cause what's my value there if, infinite amount of knowledge that I provided and I cannot keep in my head stays out?
Chris
00:40:39.361
I would say that the skill that you developed by using AI that much is much more valuable than the actual synthetic output that you were able to produce. just like you said, if I leave, the code stays with the company, but I still know how to… Like, everything I learned in the process of development of that code goes with me. I would say it was very similar. Anything that I produce when I'm working for the company belongs to the company. But they can't take away the fact that I now know how to spin up a org chart of agents, and I know exactly which ones need to do what, all I have to do is ask the models to write the system prompt for those particular agents. If you know those things, you can say, "Look I'm starting from scratch, but give me a week I can replicate the majority of the architecture that I had at the pla- last place."
Viktor
00:41:20.925
I'm not actually convinced, to be honest, on that. Maybe today, but I can see how Claude, and I'm using it as example, how more and more it is aligned with me, How less and less I need to instruct it on the way I like to do things, Because there is always a combination of my company does things like this, but I do things like this. I'm developing that semi fear in a way that uh, I don't know, kind of… And it's not only about the company, if I leave the company I work in, it's even, okay, so what happens if I have to switch to GPT, right? It's kind of that memory, it's not necessarily that important today, but I can see how it is going from nothing to something extremely powerful
Chris
00:42:13.709
So that goes back to that, episode that we usually start our kickstarts with, which is that, hey, take a look at the context that I can give you, which is my email, my inbox and whatever, and learn about me. Because the alternative is I'm gonna spend six months starting from scratch with an off-the-shelf version, and I'm gonna season it over time. It's gonna, it's gonna learn about me over time from repetitive use. Or I can go into the new company, "Hey, ChatGPT, here's the connectors," build the information that, I need to put into the custom instructions and the more about you fields and all that sort of thing so that you don't necessarily use-- lose it. Now, if it's a company account and I don't have access to that, Outlook anymore, my, inbox yeah, different scenario
Viktor
00:42:57.897
Where I'm going with this is I think that you're completely right for somebody who is starting. But fast-forward 10 years, I'm not going to tell Claude, "Look at my emails and my chats and stuff like that," simply because actually 90% of the information is not there anymore. It's simply not there. My life in my inbox is going to be worth nothing. I mean, I will not be even writing those emails. I don't know whether I will be even sending emails. Most likely, my agent will talk to your agent directly
Chris
00:43:30.593
Yeah. Yeah. What I would do is I would be looking for some sort of almost we'll call it like a, quote-unquote, "database backup," where I would be able to extract some sort of context document from the model on a monthly basis, and I would build my own library that would have the history of the last 10 years of the projects that I worked on or whatever the case might be. Something like that would be a, a rudimentary and still manual, but I would be able to-- If I left, I would have that, let's call it a work diary almost, that went with me. Or maybe you don't do it. Maybe you've got a, a scheduled task in Claude or ChatGPT that, "Hey, once a month, go and add to this document and update it with all of the most recent, actions or developments or things that I've learned or whatever." I think that'd probably be a pretty easy workaround, and I would imagine that at some point, I wouldn't be surprised if they had some sort of a concept of a digital ID that was fed or trained on the behaviors that you showed within the models, as long as you had that, the subscription to that tool and you turned it on
Darin
00:44:37.713
I've got one more question. Your company name is ChiefAOfficer.com. I'm assuming there's a real company name somewhere else too, but that's, that's your marketing name. the title Chief AI Officer smells like what we had back in early 2010s of chief digital officer, It's a rapid adoption of this thing. It's a fuzzy mandate at best. We're bringing in consultants that really don't know anything, not only about what AI is, but also don't know the business that they're tr- trying to help out. And then those people that were the chiefs are, how can we say wished well in their future endeavors in about 18 to 24 months later. What's different this time?
Chris
00:45:17.899
A couple things. And how we came up with this name is, I referenced him earlier, but Peter Diamandis was on a podcast with Tom Bilyeu in March of 2023, Tom Bilyeu has a podcast called "Impact Theory," and he asked Peter, like in the first 60 seconds, he's like: "Look, as somebody who wants to go all in on AI, what should I do?" And Peter said, "I'm advising every company that I talk to, my own companies, that they need to hire what I call a chief AI officer. And it's not a data scientist. It's not somebody who's gonna fine-tune a large language model. In my definition, it's an individual who understands business and they're paying attention to the generative AI landscape, and they're able to translate the two." So for us, the chief AI officer, and I know that the term existed before ChatGPT 3.5 came out, but for us, the definition is that it's a non-technical business professional. to me, the best chief AI officer would be your ops person, your COO, or somebody that understood the operations of the business and could look around and go, "Oh, wait a minute. This tool does that? Ooh, we could plug it in here. That tool does this? Ooh, we could do this." So to me, it's that. Now, the role of COO hasn't gone away or the ops director or whatever, because operations is something that every business needs, whether that business is operated by agents or humans. So if you look at it in that term, and we can continue to spread that definition to where people assume that's the role, that it's a somebody that understands business operations and the capabilities of AI, I don't know that that role would go away. And if it did, they tell chief AI officer "Well, you're not called that anymore." That's okay, but I have a high degree of expertise in using the tool that every business wants to use. I'll keep the lights on regardless of what the title is
Darin
00:47:01.763
If only people actually believed that and didn't actually think the title meant I get to play golf more every day
Chris
00:47:09.823
You know, We don't see that. We see most of the people that wanna be a chief AI officer, they're very like impassioned by using the tools. it's not somebody who went the traditional route in their career. It's somebody who invested early in their own upskilling. we see this all the time. If you're that person, if you're listening to this and you're mid-tier or whatever, if you start to advertise… Listen, nothing bad will happen to your career if you let people know that like you know how to use AI really well. It's gonna be this thing hey Chris, you got a minute? We've got a meeting, they're gonna be bringing up AI. Can you jump in there?" Sure. you're gonna get exposure that you wouldn't get otherwise. And I can tell you that if you're expecting maybe in five years that chief AI officer is to the point to where they're doing a lot of strategy work. But right now that doesn't exist in most companies. the strategy is still being developed. So I would say that the chief AI officer needs to expect to be very busy and in high demand until the chief AI officer agent gets built,
Darin
00:48:10.099
We can find out more about Chris's company at chiefaiofficer.com, and all of Chris's information will be down in the episode description. Chris, thanks for being with us today
Chris
00:48:20.977
great conversation, guys. Thank you so much for the opportunity We hope this episode was helpful to you. If you want to discuss it or ask a question, please reach out to us. Our contact information and a link to the Slack workspace are at devopsparadox.com/contact. If you subscribe through Apple Podcast, be sure to leave us a review there. That helps other people discover this podcast. Go sign up right now at devopsparadox.com to receive an email whenever we drop the latest episode. Thank you for listening to DevOps Paradox