On August 18, 2026, Nathan Ingram, the agency trainer here at hosting.com, explored Cloudflare’s new agent readiness scanner “Is Your Site Agent-Ready?” in our “Is it agent-ready?" livestream. In it he scanned a website and reviewed the results live.
In the Office Hours session that followed two days later people naturally had a lot of questions about the scanner and the confusing results some of them were getting. So, for this blog, instead of picking one question and expanding on it, we pulled together every question from the Office Hours livestream about the scanner.
We’ll discuss what the scanner checks, the issues our viewers encountered, and the fixes that resolve them.
Why Cloudflare’s scanner is worth caring about
In the Weekly Brief section of the June 18, 2026, Office Hours livestream, Nathan pointed to Cloudflare’s own data. It shows that more than half of all requests that hit web pages are now made by bots and AI agents, around 57.5%.
The reality is, though, that most WordPress sites, and most sites beyond WordPress, were built with that remaining 42.5% of traffic in mind. This shift isn’t just a curiosity either. Nathan himself mentioned that his own agency was cited by name in a ChatGPT search for local web development agencies, without them doing anything to make that happen.
On the commerce side, Adobe found that AI-referred shoppers now convert 42% better than other traffic. That’s a full reversal from converting 38% worse just a year earlier. Whatever is happening with AI agents and search results, it’s already showing up in real leads and real sales.
Cloudflare’s Is Your Site Agent-Ready? tool is trying to help close that gap.
What does the “Is Your Site Agent-Ready?” scanner check?
When you submit your website for scanning, the tool evaluates your site in four major categories: Discoverability, Content, Bot Access Control, and API/Authentication/MCP.
The scan gives your whole site an overall score from 0 to 100 and a level from 1 to 5. Each category shows how many of its own checks you've passed. However, the level label is worth a quick caveat. It’s easy to read it as a maturity ladder for sites to climb, when, in reality, it isn’t. Nathan explained it like this during the “Is it agent-ready?” livestream:
We're not trying to level up a basic content site all the way up to a site that's Agent Native, with MCP servers and APIs and all of that. It's overkill, we don't want that.
And a bit later he added:
That level makes it seem like one level is better than the other. Really, what you should think about this is: these are types of sites, right? So, a basic web presence, it's fine that it doesn't have APIs, it's fine that it doesn't have integrated agents, because it's just a brochure site.
The terminology can be a little misleading because the levels aren’t a measurement of website quality. Instead, each level describes a different type of website based on its purpose and complexity.
A simple brochure site can be every bit effective as a complex e-commerce store. What matters is whether the site is well built and does what it needs to do for the business and it’s visitors. As for the categories themselves, here’s what each of them checks for:
Discoverability asks whether an agent can find the right information about your site in the first place. It checks for:
A robots.txt file that doesn’t block AI crawlers.
An XML sitemap.
Response headers pointing to things like your REST API.
A DNS record called DNS-AID (DNS for AI Discovery). It tells agents where to find AI-related resources for your domain.
Content primarily revolves around whether an agent can read your copy efficiently. More specifically, it checks if your site can hand over a clean markdown version of a page instead of forcing an agent to parse through a wall of HTML divs.
Bot access control covers whether you’ve explicitly disallowed any AI bots in robots.txt. It also covers a newer concept called content signals, which let you separately tell agents whether they can train on your content, search it, or use it to generate an answer.
API, authentication, and MCP is the advanced bucket. It checks whether your site publishes an MCP server card, an API catalog, agent skills, or supports OAuth or the emerging WebMCP browser standard. By default, no WordPress site checks any of these boxes, and for most smaller sites that’s fine.
Of course, always keep in mind that as you are interpreting the score, it’s Cloudflare’s opinion of what makes a site agent-friendly. It’s the same principle with a Google Lighthouse score: it’s Google’s opinion of what makes a page fast and well-built.
A high score is a useful signal and can show you real pain points on your website. However, chasing the perfect score doesn’t guarantee your site will be objectively better for it.
Troubleshooting a failed scan
This week’s Office Hours questions mostly involved the Cloudflare scanner and the same root issue, just with different symptoms.
One viewer had a manual robots.txt file sitting on the server, originally added months prior to allow the SEMrush bot through. The scan flagged it, and for a good reason. As Nathan explained it, a static robots.txt file in the root directory of your site overrides the ones your WordPress and SEO plugins generate dynamically.
A dynamic robots.txt file reflects whatever your current site’s settings are. Since those may change over time (such as marking a page as noindex or adjusting your sitemap through an SEO plugin), a dynamically generated robots.txt file updates automatically.
Because it silently overrides the dynamic version with no warning, it's often better not to have a static file at all.
Additionally, several other viewers had a trickier version of the same problem. The scan still failed, even though the robots.txt file was confirmed correct, dynamically generated and properly linked. The pattern showed across different SEO plugins and hosts, including sites where the sitemap loaded perfectly fine when tested manually.
So, when everything is fine on the WordPress side, it means something sitting between it and the scanner is the issue: a firewall rule, a security setting, or a WAF policy that blocks the scan itself.
One viewer confirmed that for their site, the host’s own firewall was quietly blocking the scan. It also explains why a couple of viewers saw their scans stall, even after applying the fix we’ll describe shortly. Nathan had a similar hunch, that something before the site was interfering:
It's got to be a security block, either at the network DNS level or at your host, preventing those files from being scanned, since they do exist.
So, here’s a quick checklist you can go through if your site is set up correctly, but the scan still fails:
Talk to your host about firewall and WAF rules that might be blocking automated scans or bot traffic. Ask specifically about allowing the scanner.
Check your DNS and CDN security settings if you are running anything in front of your host, since the same kind of blocking can happen there, too.
Compare a passing site against a failing one on the same server or account to narrow down whether the setting is server-wide or specific to that site.
Note that Cloudflare itself isn’t required for the scan to work. Sites with or without Cloudflare could still exhibit the same blocking pattern.
Chasing this issue down is worth it for more than just the score. If the scanner can’t read the robots.txt file or your sitemap, chances are legitimate AI agents are running into a similar wall. Both AI agents and a scanning tool make functionally the same kind of request.
The fix for most content sites: Mescio for Agents
The majority of the Discoverability and Content gaps we described above can be resolved very easily.
During the "Is it Agent-ready?" livestream, Nathan mentioned a plugin called Mescio for Agents. It's a small plugin that does most of the heavy lifting when it comes to fixing those issues. Once it's active, it automatically:
Generates an agents.txt file with sensible defaults. For example, agents can go anywhere, except wp-admin and the login page.
Sets the content signals in robots.txt that allow your pages to be cited in AI search results and included in AI-generated answers, without necessarily handing your content over for model training:
No to AI training.
Yes to AI search.
Yes to AI input.
Builds an llms.txt file (essentially a markdown sitemap for agents). Also creates a more detailed llms-full.txt file from your posts and pages.
Handles markdown content negotiation, so agents that request the markdown version of a page get it without any extra setup.
One viewer also asked about the extra fields on the post and page editor since installing the plugin (short description and full markdown content override). Nathan explained that users needn't fill those in. They are for extra control if you ever want to feed a different, markdown-formatted version of your content to agents than what appears on the front end.
If you are running an older plugin that only exposed markdown for browser viewing, it's worth switching over to Mescio for Agents rather than running both at the same time.
In case you are running a WooCommerce store
One odd thing about the Cloudflare agent readiness scanner is that it often doesn't recognize WooCommerce at all. It will report "No e-commerce signals detected," even on the shop page.
Nathan ran directly into this while testing, but the good news is that it's not something that will drag your score down. The entire e-commerce category is considered "optional."
Nonetheless, there is a fix for that, plus a couple of extra things you can do to ensure the scanner fully understands your site's intent.
The WooCommerce fix: WebMCP Bridge
The WebMCP Bridge plugin exposes store functionality to agents through the WordPress REST API. It allows AI agents to roughly operate the way a human visitor would through the browser. In other words, when it's active, agents can:
Search products and check stock and pricing.
Add items to a cart and apply coupons.
Discover the site's menu structure and search posts and pages.
Another upside is that it ships with a global rate limit: 120 interactions per 60 seconds by default. This means it doesn't open your site to being hammered by bot traffic. Combining this plugin with Mescio for Agents is usually enough to achieve more realistic results in the scan. As Nathan put it:
You have made your site incredibly agent ready, literally by activating two plugins.
Optional: the DNS-AID record
If you'd like to go further and optimize a bit more, the DNS-AID record is worth the extra setup. It's an SVCB record that requires DNSSEC to be enabled on your domain first.
If you are familiar with an MX record, the idea is the same. The same way the MX record tells a mail server where to find email services for your domain, the DNS-AID record tells AI agents where to find AI-related resources for your domain.
To get a bit more technical, it points to a directory on your website called "well-known" (usually hidden), which is where all your agent-facing instructions and resources live. It's also where things like SSL certificate verification are.
When an agent visits a site with this record in place, it knows to check that directory first, which unlocks everything else it needs to know about interacting with your website.
For a basic content site it won't add much, but stores or any sites with any real agent-facing functionality can benefit greatly from what is, ultimately, a relatively quick addition, assuming your DNS provider supports the prerequisites.
One last thing: the Cloudflare September 15th change
If you manage sites on Cloudflare, you probably already know about this, but it's still worth flagging.
Starting September 15, 2026, any new domain added to Cloudflare will, by default, block AI training bots on any page that serves ads. The catch is that some crawlers, including Googlebot, do double duty for both training and search. Consequently, an aggressive default block like that could end up hurting visibility in AI search results.
If you manage sites that might be affected by this change, it's worth reviewing the setting. Our blog post about these new Cloudflare AI crawler rules goes into detail on everything related to them.
The quick and short version
The Cloudflare agent readiness scanner is genuinely a good tool to give you an idea of how AI agent-friendly your site is.
As long as the required files that Nathan mentioned during the "Is it agent-ready?" livestream exist, are dynamically generated (instead of being static), and correctly linked, a failed scan isn't a disaster. It simply points to something other than WordPress blocking the scan, or a plugin problem.
For most content sites, the fastest way to fix this is to install Mescio for Agents, and for WooCommerce sites to also add WebMCP Bridge on top of that.
There's one thing that we must all remember, though: nobody yet has solid data on exactly how much of a difference all of this makes for AI search visibility or AI-driven sales. As Nathan cautioned during the livestream, don't trust anyone claiming to have concrete data. Watching your own traffic and order sources over the next few months is a better guide than any promised statistic.
And if you have any more questions for Nathan about agent readiness, the Cloudflare scanner, WordPress, or AI and hosting in general, join us for Office Hours every Thursday at 2 p.m. ET. Viewers often have questions on many different topics, so you are always bound to learn something new.
FAQ
Will making my site agent-ready hurt my regular SEO or Google rankings?
No, not your classic organic rankings. Human visitors still get the normal HTML version of your site, and the markdown negotiation only kicks in when an agent specifically requests it via a header, so it has no bearing on how Google indexes or ranks your pages for standard search results.
Content signals are a different story, and they're worth understanding separately rather than lumping in with "SEO." The "train" signal specifically exists to tell crawlers whether they can use your content to train AI models. That's a deliberate lever, not a side effect. Where it gets less certain is AI-driven features like Google's AI Overviews, since those may draw on the same crawled content that training signals are meant to govern.
So, the honest framing is: content signals shouldn't move your classic rankings, but they're explicitly designed to affect your visibility in AI-generated answers and training, that's a separate and real effect.
Is it safe to let AI agents access things like my cart or REST API data?
It's less risky than it sounds, mainly because agents can only do what a normal visitor could already do through the browser. Exposing product search or cart functionality through the REST API isn't handing over anything private; it's just giving agents a more efficient way to do what a human would do manually anyway. The bigger practical safeguard is rate limiting (WebMCP Bridge defaults to 120 interactions per 60 seconds site-wide) so you're protected from being overwhelmed by bot traffic rather than from any actual data exposure.
How would I know if any of this is making a difference?
Rather than chasing the scan score itself, the more useful signal is your own traffic and sales data. WooCommerce's own reporting already breaks down where orders come from (organic search, social, email, and so on), and that same breakdown now surfaces ChatGPT and Claude as referral sources when a sale originates from one. Watching that list over the next few months is a far more concrete way to gauge impact than relying on any scan score or third-party statistic.