DOP 363: Is Your Website Agent-Ready?

#363: Three waves of the web, and you are late for the third one. The 90s were about getting a browser to render your page at all. The early 2000s were about SEO, or as Darin puts it, sell me all the ads ready. Now it is agent ready, and Cloudflare built a scoreboard for it at isitagentready.com. The devopsparadox.com site scored about 70 out of 100 and then went down when Cloudflare added new checks. Run yours. You will be sad.

Viktor thinks the framing is slightly off, though, and the correction is the good part. Optimizing for agents that browse your site is aiming at the wrong thing, because most requests never touch your server. Agent asks the model, model answers, agent shows you. So the target is not the crawler, it is the training data - and if the model does go looking, the question becomes whether you are the first answer or one of the five sites it was told to go analyze. Same game as Google. Different index. It is not Google index anymore, it is model training now.

Then the practical part. Five things Cloudflare scores you on: discoverability, content, bot access control, API, Auth, MCP & Skill Discovery, and Commerce. Content accessibility is where most of you are losing, because agents want Markdown and you are serving them a pile of HTML tags to strip. Both DOP and Viktor’s site are Hugo, so the Markdown is already sitting on disk next to the HTML - serve one or the other based on what the request asks for. Almost no effort. If you are still shipping a JavaScript-rendered site, Darin says it is game over, and humans do not like those either. On the blocking side, both of them are baffled by the same thing: if you do not want agents reading it, do not publish it. robots.txt is a suggestion at best. If you really want to block, actually block.

The API argument is the one that will annoy people. Viktor says CLIs and MCP servers are both auto-generated from a schema, so the real work is having a good API, and most companies do not. But who your audience is decides the wrapper - developers already have Bash, so give them a CLI and get out of the way. Everyone else needs MCP, because Viktor’s mom is not installing your binary. And somewhere in the middle of all this Darin asks whether documentation should live in the code now more than ever, and Viktor says no, less than ever - he wants it separate so he can review it, because agents made everything cheap to produce and review is now the only thing standing between him and 5,000 features a day. Also: WordPress should be the last thing you consider, not the first.