DOP 365: What the DORA AI ROI Report Really Says

Episode 365

Show Notes

#365: A bad engineer with a laptop is much worse than a bad engineer with a calculator. Now hand that engineer AI. DORA’s ROI report puts it politely - AI without engineering excellence just scales your problems.

Except it’s not more of everything. It’s more of whatever you were already doing, and there’s a statistical chance that what you’re doing is bad. Your agent already read your flaky tests and decided that’s the house style.

So what do you measure now? Viktor’s answer: every technical metric you’ve got is gamed.

Frequently Asked Questions

What does the DORA report say about AI's return on investment?

Darin Pope summarises the May 2026 DORA report on DevOps Paradox episode 365 in a single line: AI without engineering excellence just scales your problems. Productivity is not the issue. Organizations lacking mature platforms, clear workflows, and strong reviews amplify chaos when they switch AI on. The report also describes a J curve, with a dip in productivity across the first one to three months before output climbs.

Why does AI make a bad codebase worse?

Viktor Farcic explains on DevOps Paradox episode 365 that an agent asked to add a feature does not invent an approach. It reads how you already do things and produces more of the same. Flaky tests beget more flaky tests. He adds a second problem on top: once the agent works out what you actually want, it does that job well, and what you wanted may have been wrong in the first place.

What should you measure before adopting AI?

Darin Pope lists the DORA baselines on DevOps Paradox episode 365: platform quality, CI/CD speed, code review SLAs, incident mean time to resolution, and deployment frequency. Viktor Farcic dismisses most technical metrics as gameable, lines of code above all, and argues the only measurement that survives contact with reality is money. His preferred version is the ratio of prospects converting to customers, tracked over a long period.

Should you send your team on AI training courses?

Darin Pope’s advice on DevOps Paradox episode 365 is no. Give people the tools and access to your existing source code instead, and he reckons they will learn more in three days of that than in a three-day course. Viktor Farcic goes further: if AI is producing no benefit at your company, something in your system is broken, and the job is to find it rather than abandon the attempt.

What is agentic engineering?

Darin Pope explains on DevOps Paradox episode 365 that Andrej Karpathy, who coined the term vibe coding, later reframed the serious version as agentic engineering: design the system, specify the constraints, then use AI to accelerate something you have already reasoned through. Darin’s observation is that the first two steps describe every project in history. Viktor Farcic adds that he now uses agents during the reasoning phase as well.

How do you review more code than you can read?

Viktor Farcic admits on DevOps Paradox episode 365 that he no longer understands his own hobby projects at the code level, only at the level of architecture. His response is to improve validation rather than slow production down. On a CLI project he has agents record video clips of each test case, so his morning review starts by watching those recordings and then choosing which parts to run by hand.

What is the DevOps Paradox podcast?

DevOps Paradox is a weekly podcast co-hosted by Darin Pope and Viktor Farcic, covering DevOps, platform engineering, and modern software delivery. Episode 365 works through the DORA report on the return on investment of AI-assisted development, why AI amplifies whatever practices you already have, and what to measure before switching it on. Every episode page carries audio, video, and a full transcript.

Episode Transcript

Read the Full Transcript

Links from the Episode

Share and Download

Hosts

Darin Pope

Darin Pope

Viktor Farcic

Viktor Farcic

Viktor Farcic is a member of the Google Developer Experts and Docker Captains groups, and published author.

His big passions are DevOps, Containers, Kubernetes, Microservices, Continuous Integration, Delivery and Deployment (CI/CD) and Test-Driven Development (TDD).

He often speaks at community gatherings and conferences.

He has published DevOps Paradox and Test-Driven Java Development.

His random thoughts and tutorials can be found in his blog The DevOps Toolkit.