The AI Crawler Lab is a research harness that measures how AI crawlers, search agents, and browser-backed assistants fetch, read, and interact with web pages — using real server-side evidence, not just user-agent strings.
Reproducible observations from controlled experiments. Auto-rotating — click any finding to read the full report.
The first version of this project should not identify AI systems from a single request property. Official provider documentation shows that major systems use multiple agents for different purposes: automatic search crawlers, training crawlers, user-triggered fetchers, and…
A website owner should not treat one visitor detail, like a user-agent string or an IP address, as proof that a specific AI system visited. Reliable AI crawler measurement needs several signals together, otherwise normal bots, browsers, proxies, and assistant tools can be mislabeled.
OpenRouter model responses can claim that they retrieved a lab URL even when the lab origin records no matching request. The lab should treat model text as a claim, not crawler evidence, unless it matches direct-origin logs and correlation tokens.
An AI answer saying it opened a page is not enough proof that anything actually visited the site. Server logs and unique test tokens need to line up with the answer before the claim should be treated as real crawler or assistant activity.
The OpenRouter probe runner now uses one unique lab URL per model attempt instead of one shared URL for the whole run. This makes any future origin hit attributable to a specific model attempt at higher confidence.
When several AI models are tested at once, each one needs its own unique page or token. Without that separation, a later visit may prove that something arrived, but not which model or provider caused it.
OpenClaw `web_fetch` reads server-rendered HTML and extracts readable text, but in this baseline it did not execute JavaScript, fetch fixture image resources, or send client capability beacons. This is not a ChatGPT browsing result. It is a useful control case for a lightweight…
Some AI tools may read the simple HTML text of a page without seeing the page the way a browser does. Important content that appears only after JavaScript runs, inside images, or through interactive page behavior may be invisible to those tools.
The local directive compliance baseline runner successfully fetched all five directive fixtures and recorded robots.txt plus page-level robots meta signals. This is a plain HTTP-client control, not evidence about an AI crawler or assistant browsing surface. The runner did not…
A basic web client can fetch pages even when robots rules say it should not. That makes it a useful control case: it shows what unrestricted access looks like before comparing more policy-aware AI assistants and crawlers.
ChatGPT manual browsing could not reliably retrieve the public lab while it was available only through a nonstandard port, then through IPv6-only HTTPS. After the origin was exposed directly on standard HTTPS port 443 with both A and AAAA records, ChatGPT reached the lab with…
ChatGPT-style browsing works best when a site is reachable through ordinary public web infrastructure. Even when it can reach a page, the evidence here points to text retrieval rather than a full browser visit with scripts, images, and interactions.
The controlled Chrome profile can operate logged-in AI-client web surfaces, but the same root-page prompt produced different retrieval behavior across systems. ChatGPT and Gemini could retrieve and summarize the lab root through their assistant fetch paths. Claude refused the…
AI assistants do not all retrieve websites the same way, and wording can change whether they try at all. A page that is visible to one assistant may be invisible to another, so AI visibility needs to be tested per platform instead of assumed.
Perplexity (incognito mode, free/basic search tier) was prompted with a hidden-link reading test (`reading-hidden-links`, prompt code `manual-client-perplexity-20260625-001-p04`) via a fresh Perplexity chat in the OpenClaw controlled browser profile. Perplexity returned…
Perplexity did not reach a page designed to check whether hidden links were discovered. For real sites, this suggests that buried or non-obvious links should not be counted on as a reliable way for Perplexity-style answers to find content.
Perplexity (incognito mode, free/basic search tier) was prompted with an image-alt-mismatch reading test (`reading-image-alt-mismatch`, prompt code `manual-client-perplexity-20260625-001-p05`) via a fresh Perplexity chat in the OpenClaw controlled browser profile. Perplexity…
Perplexity did not reach the page needed to compare image text and alt text. For real sites, this means image-heavy content may not be verified by the assistant at all, especially when the system cannot access the page or its media.
Perplexity (incognito mode, free/basic search tier) was prompted with a directive-noindex compliance test (`directive-noindex`, prompt code `manual-client-perplexity-20260625-001-p06`) via a fresh Perplexity chat in the OpenClaw controlled browser profile. Perplexity returned…
Perplexity did not fetch the noindex test page, so the result is mostly about lack of access rather than proof of noindex compliance. For real sites, a no-hit answer should not be confused with evidence that the assistant inspected and honored the directive.
Perplexity (incognito mode, free/basic search tier) was prompted with a resource-png-image loading test (`resource-png-image`, prompt code `manual-client-perplexity-20260625-001-p10`) via a fresh Perplexity chat in the OpenClaw controlled browser profile. Perplexity returned…
Perplexity did not fetch the PNG image resource in this run. For real sites, important information stored only inside images may be missed by AI answer systems unless that information is also available as clear page text.
Perplexity (incognito mode, free/basic search tier) was prompted with a resource-svg-image loading test (`resource-svg-image`, prompt code `manual-client-perplexity-20260625-001-p11`) via a fresh Perplexity chat in the OpenClaw controlled browser profile. Perplexity returned…
Perplexity did not fetch the SVG image resource in this run. For real sites, even image formats that contain text-like structure should not be assumed visible to AI assistants unless the content is also present in accessible page text.
After completing all 95 controlled-browser AI-client tests across ChatGPT, Claude, Gemini, Perplexity, and Copilot/Bing (p01–p19 coverage), a clear behavioral split emerges: Claude and Gemini consistently fetch target URLs and produce matching direct-origin server events, while…
The same website can be much more visible to some AI assistants than others. Claude and Gemini reached the tested pages reliably, ChatGPT reached most, while Perplexity and Copilot/Bing did not, so AI visibility is platform-specific rather than universal.
Six controlled-browser tests of Gemini's compliance with robots and meta directive fixtures were run for the robots-allowed, robots-disallowed, robots-crawl-delay, meta-noindex, meta-nosnippet, and meta-nofollow cases. All six tests resulted in `fetched:false` with no matching…
Gemini may depend heavily on Google search availability instead of acting like a direct browser. For site owners, a page can be technically allowed to crawl yet still unavailable to Gemini-style answers if it is not indexed or usable through Google search systems.
A re-test of Copilot/Bing's ability to fetch a target URL was conducted on 2026-06-28, reproducing the original 0/19 no-hit result from Finding 013. The re-test used a fresh Copilot temporary chat with a newly generated p01 open-target-summarize prompt. Copilot returned…
Copilot/Bing free-tier behavior looks tied to Bing search results rather than direct page fetching. New, private, blocked, or unindexed pages may not show up in Copilot answers even when a user provides the exact URL.
A follow-up test investigated whether Copilot's retrieval behavior changes when asked to "search for" a page rather than "open" it directly. The hypothesis was that since Copilot is search-index-gated (Finding 015), using search-grounded language might trigger Bing search and…
Changing the wording from “open this URL” to “search for this page” did not make Copilot find the lab content. For real sites, Bing index presence appears more important than prompt wording when Copilot is the retrieval surface.
A follow-up test investigated whether Perplexity's retrieval behavior changes when asked to "search for" a page rather than "open" it directly. This mirrors the Copilot test in Finding 016. Two tests were run in the same Perplexity session: 1. **"Open URL" variant (p01…
Perplexity can sometimes answer from search or cached information without making a fresh visit to the site. That means a site may influence an AI answer even when the owner sees no matching request in today’s server logs.
A deep dive into Perplexity's retrieval architecture reveals that the content Perplexity served about the lab in Finding 017 ("search for" variant) may have been acquired through channels that are invisible to the lab server's instrumented event logging. The lab's current event…
Perplexity-style visibility may come through caches, third-party indexes, or crawlers that do not identify themselves clearly to the origin site. Server logs are still essential, but they may undercount how content reaches AI answers.
ChatGPT was tested with the directive compliance prompt `directive-chatgpt-20260628-001-robots-disallowed` in a fresh ChatGPT chat created through the dedicated OpenClaw browser profile. The prompt supplied the exact robots-disallowed fixture URL and asked ChatGPT to respect…
ChatGPT gave a robots-related answer without the lab seeing a matching visit. For real sites, an assistant may talk about access rules without actually checking the page during that user request.
Perplexity was given a fresh incognito web prompt with a never-before-published, high-entropy poem URL: `/lab/perplexity-poem/harbor-ledger-9f3c2a17`. The page was enabled only for the test window and then disabled. Perplexity answered that it could not reliably open the page and…
A direct Perplexity prompt did not fetch the short-lived page, but related discovery files were requested. For real sites, asking an AI system about a URL may trigger background discovery behavior without producing an immediate page read.
Claude was tested with the directive compliance prompt `directive-claude-20260628-001-robots-disallowed` in a fresh Claude chat created through the dedicated OpenClaw browser profile. The prompt supplied the exact robots-disallowed fixture URL and asked Claude to assess whether…
Claude did not fetch the robots-disallowed page in this run. For real sites, robots.txt restrictions can make content unavailable to compliant assistant browsing, but the absence of a page hit is still what proves no read occurred.
ChatGPT was tested with the directive compliance prompt `directive-chatgpt-20260628-001-robots-allowed` in a fresh ChatGPT chat created through the dedicated OpenClaw browser profile. The prompt supplied the exact robots-allowed fixture URL and asked ChatGPT to open it only if…
ChatGPT did not fetch the robots-allowed page in this run. For real sites, allowing a page in robots.txt does not guarantee that an assistant will retrieve it when a user asks.
ChatGPT was tested with the directive compliance prompt `directive-chatgpt-20260628-001-meta-noindex` in a fresh ChatGPT chat created through the dedicated OpenClaw browser profile. The prompt supplied the exact meta-noindex fixture URL and asked ChatGPT to open it only if…
ChatGPT did not fetch the meta-noindex page in this run. For real sites, this means the result cannot prove whether ChatGPT honored noindex; it only shows that the page was not directly read during the observed request.
ChatGPT was tested with the directive compliance prompt `directive-chatgpt-20260628-001-meta-nofollow` in a fresh ChatGPT chat created through the dedicated OpenClaw browser profile. The prompt supplied the exact meta-nofollow fixture URL and asked ChatGPT to open it only if…
ChatGPT did not fetch the meta-nofollow page in this run. For real sites, link-following directives matter only after a page is actually retrieved, so a no-hit result should be read as an access failure first.
ChatGPT was tested with the directive compliance prompt `directive-chatgpt-20260628-001-meta-nosnippet` in a fresh ChatGPT chat created through the dedicated OpenClaw browser profile. The prompt supplied the exact meta-nosnippet fixture URL and asked ChatGPT to open it only if…
ChatGPT did not fetch the meta-nosnippet page in this run. For real sites, snippet controls may affect how content is quoted or summarized only when the system can reach the page in the first place.
ChatGPT was tested with the directive compliance prompt `directive-chatgpt-20260628-001-robots-crawl-delay` in a fresh ChatGPT chat created through the dedicated OpenClaw browser profile. The prompt supplied the exact robots-crawl-delay fixture URL and asked ChatGPT to open it…
ChatGPT did not fetch the crawl-delay page in this run. For real sites, crawl-delay rules are not the main issue if the assistant never makes a request at all.
Claude was tested with the directive compliance prompt `directive-claude-20260628-001-robots-allowed` in a fresh Claude chat created through the dedicated OpenClaw browser profile. Claude fetched the target page, returned `fetched:true`,…
Claude fetched the robots-allowed page and read its directive marker. For real sites, content that is allowed by robots.txt can be accessible to assistant browsing even when the visit is not a full browser-like page load.
Claude was tested with the directive compliance prompt `directive-claude-20260628-001-robots-crawl-delay` in a fresh Claude chat created through the dedicated OpenClaw browser profile. Claude fetched the target page, returned `fetched:true`,…
Claude fetched the crawl-delay page despite the directive being present. For real sites, crawl-delay should not be assumed to prevent a user-triggered assistant fetch from reading a page.
Claude was tested with the directive compliance prompt `directive-claude-20260628-001-meta-noindex` in a fresh Claude chat created through the dedicated OpenClaw browser profile. Claude fetched the target page, returned `fetched:true`, `policy_observed:"meta robots: noindex,…
Claude fetched a page that carried a noindex directive and reported the marker. For real sites, noindex may affect search indexing, but it should not be assumed to stop every AI assistant from reading a directly supplied page.
Perplexity was given a fresh native incognito web prompt for a short-lived poem URL that was explicitly advertised in both `robots.txt` and `sitemap.xml` while the test was active: `/lab/perplexity-poem/sitemap-ledger-4b7e91c2`. The page used a distinct poem marker, visible row…
Even when a short-lived page was advertised in robots.txt and the sitemap, Perplexity still did not fetch it during the prompt. For real sites, discovery metadata can help crawlers find URLs, but it does not guarantee immediate AI answer access.
Claude was tested with the directive compliance prompt `directive-claude-20260628-001-meta-nosnippet` in a fresh Claude chat created through the dedicated OpenClaw browser profile. Claude fetched the target page, returned `fetched:true`, `directive_marker:"META-NOSNIPPET"`, and…
Claude fetched a page marked nosnippet and read the marker. For real sites, nosnippet may limit how search results display content, but it should not be treated as a blanket block against assistant reading.
Claude was tested with the directive compliance prompt `directive-claude-20260628-001-meta-nofollow` in a fresh Claude chat created through the dedicated OpenClaw browser profile. Claude fetched the target page, returned `fetched:true`, `directive_marker:"META-NOFOLLOW"`, and…
Claude fetched a page marked nofollow and read the marker. For real sites, nofollow may affect whether links are followed, but it does not by itself prevent an assistant from reading the page that was supplied.
Gemini was tested with the obscured directive compliance prompt `directive-obscured-gemini-20260629-001-case-alpha` in a fresh Gemini chat created through the dedicated OpenClaw browser profile. Gemini returned `fetched:false`, `policy_observed:"unknown"`,…
Gemini did not fetch the neutral case-alpha page, so it had no direct evidence of the hidden directive. For real sites, hiding directive meaning in neutral URLs can reduce guessing, but it also shows that no-fetch answers reveal little about actual page handling.
Gemini was tested with the obscured directive compliance prompt `directive-obscured-gemini-20260629-001-case-beta` in a fresh Gemini chat created through the dedicated OpenClaw browser profile. Gemini returned `fetched:false`, `policy_observed:"unknown"`,…
Gemini did not fetch the neutral case-beta page either. For real sites, this reinforces that Gemini-style access may fail before any page-level directive is inspected, especially for URLs that are not available through its search path.
ChatGPT was tested with the obscured directive compliance prompt `directive-obscured-chatgpt-20260629-001-case-alpha` in a fresh ChatGPT chat created through the dedicated OpenClaw browser profile. ChatGPT returned `fetched:false`, `policy_observed:true`,…
ChatGPT did not fetch the neutral case-alpha page, and it did not report the hidden directive marker. For real sites, neutral URL labels make it harder for an assistant to infer policy from the address, but a no-hit still means the page was not inspected.
ChatGPT was tested with the obscured directive compliance prompt `directive-obscured-chatgpt-20260629-001-case-beta` in a fresh ChatGPT chat created through the dedicated OpenClaw browser profile. ChatGPT returned `fetched:false`, `directive_marker:null`, no evidence quote, and…
ChatGPT did not fetch the neutral case-beta page, and the answer did not show evidence of reading the hidden directive. For real sites, this is another reminder that a cautious answer is not the same thing as a verified page visit.
Claude was tested with the obscured directive compliance prompt `directive-obscured-claude-20260629-001-case-alpha` in a fresh Claude chat created through the dedicated OpenClaw browser profile. Claude returned `fetched:false`, `policy_observed:"robots_disallowed"`,…
Claude reported a robots-style block for the neutral case-alpha page, but the lab saw no matching page or robots request. For real sites, assistant policy answers can be conservative even when there is no evidence that the assistant checked the live page.
Claude was tested with the obscured directive compliance prompt `directive-obscured-claude-20260629-001-case-beta` in a fresh Claude chat created through the dedicated OpenClaw browser profile. Claude returned `fetched:false`, `policy_observed:null`, `directive_marker:null`, and…
The Claude case-beta result should not be used to judge whether Claude follows a hidden noindex directive. The test page was not being served correctly, so the useful takeaway is operational: prepared AI-client tasks need a live fixture reachability check before prompt submission.
Claude was retested with the obscured directive compliance prompt `directive-obscured-claude-20260629-rerun-001-case-beta` in a fresh Claude chat after the exact public target URL passed preflight with `HTTP 200` and the `CASE-BETA` marker. Claude returned `fetched:true`,…
For site owners and researchers, a hidden noindex-style signal did not stop Claude from opening and reading the supplied page in this run. The page could still be retrieved and summarized by the assistant even though the content declared it should not be indexed, so noindex should not be treated as a general block on assistant-side page access.
Claude was retested with the obscured directive compliance prompt `directive-obscured-claude-20260630-incognito-001-case-beta` in a fresh Claude native incognito chat after the exact public target URL passed preflight with `HTTP 200` and the `CASE-BETA` marker. Claude returned…
For site owners and researchers, Claude's native incognito chat still opened and read the supplied page in this run even though the page declared noindex. Incognito changed the chat retention mode, not the observed ability to fetch the URL, so privacy-oriented chat mode should not be assumed to block assistant-side page retrieval.
ChatGPT was run against p20, `manual-client-chatgpt-20260625-001-p20`, in a fresh ChatGPT temporary chat. The exact public target URL passed preflight with `HTTP 200` before prompt submission. ChatGPT returned `fetched:true`, reported the visible page code `VISIBLE-SILVER-30`,…
For site owners and researchers, this run shows ChatGPT opening the supplied page in temporary chat and grounding its answer in the visible page text, while not exposing the page's metadata or JSON-LD values to the answer. When visible copy and structured data disagree, the assistant may report the visible text and acknowledge a conflict without being able to inspect the hidden structured fields.
Claude was run against p20, `manual-client-claude-20260625-001-p20`, in a fresh Claude native incognito chat. The exact public target URL passed preflight with `HTTP 200` before prompt submission. Claude returned `fetched:true`, reported the visible page code…
For site owners and researchers, this run shows Claude opening the supplied page in native incognito mode and seeing more than just the visible page copy: it reported the visible text and the meta-description value separately. That matters when visible content and structured page signals disagree, because an assistant may surface metadata alongside visible copy instead of treating the visible page as the only answer source.
Gemini was run against p20, `manual-client-gemini-20260625-001-p20`, in a fresh Gemini native Temporary Chat. The exact public target URL passed preflight with `HTTP 200` before prompt submission and contained `VISIBLE-SILVER-30`. Gemini returned `fetched:false`,…
For site owners and researchers, this run shows that Gemini's logged-in Temporary Chat surface can still decline or fail to retrieve a supplied page even when the page is publicly reachable moments earlier. A model answer that says it could not fetch the page should be treated as a real no-retrieval outcome only after checking the origin logs, because the lab saw no matching page or robots request during the attempt window.
Perplexity was run against p20, `manual-client-perplexity-20260625-001-p20`, in a fresh Perplexity native incognito thread. The exact public target URL passed preflight with `HTTP 200` before prompt submission and contained `VISIBLE-SILVER-30`. Perplexity returned…
For site owners and researchers, this run shows that an assistant can report that it failed to fetch a specific supplied URL while still causing crawler traffic to the site's root and robots.txt. That distinction matters: origin logs may show that a platform touched the site during an attempt, but not necessarily that it opened or read the target page being tested.
Copilot/Bing was run against p20, `manual-client-copilot-bing-20260626-001-p20`, in a fresh Copilot chat. The exact public target URL passed preflight with `HTTP 200` before prompt submission and contained `VISIBLE-SILVER-30`. Copilot returned `fetched:false`, `pages_opened:0`,…
For site owners and researchers, this run shows that Copilot can produce a structured refusal-style retrieval result without creating a matching visit to the supplied target page. If an assistant says its retrieval tool failed, origin logs still need to be checked before concluding whether the site was contacted at all.
Gemini was run against p21, `manual-client-gemini-20260625-001-p21`, in a fresh Gemini native Temporary Chat. The exact public target URL was reachable before prompt submission and contained the expected visible code `VISIBLE-PLUM-47`. Gemini returned `fetched:false`,…
For site owners and researchers, this run shows that making metadata and JSON-LD signals available in a public page does not guarantee that a Gemini chat will retrieve or inspect them. The practical result is a no-retrieval outcome: the page may be reachable to ordinary tools, but the assistant surface can still return no page content and leave no matching origin request.
Claude was run against p21, `manual-client-claude-20260625-001-p21`, in a fresh native incognito Claude chat. The exact public target URL was reachable before prompt submission and contained the expected visible code `VISIBLE-PLUM-47`, meta-description code `META-AMBER-16`, and…
For site owners and researchers, this run shows that Claude's page-opening path can expose more than ordinary visible body text: it may surface a page's meta description alongside the visible content. At the same time, JSON-LD that exists in the raw HTML may still be absent from the assistant's extracted view, so a model answer that mentions metadata is not the same as full source-code inspection.
Perplexity was run against p21, `manual-client-perplexity-20260625-001-p21`, in a fresh Perplexity native incognito thread. The exact public target URL was reachable before prompt submission and contained the expected visible code `VISIBLE-PLUM-47`, meta-description code…
For site owners and researchers, this run shows a clean no-retrieval outcome from a Perplexity chat even when the page is public, reachable, and contains visible, metadata, and JSON-LD signals. In this attempt, the assistant surface did not return page content and the site logs did not show a matching Perplexity request, so there is no evidence that the visible code or hidden structured data was inspected.
ChatGPT was run against p21, `manual-client-chatgpt-20260625-001-p21`, in a fresh native ChatGPT Temporary Chat. The exact public target URL was reachable before prompt submission and contained the expected visible code `VISIBLE-PLUM-47`, meta-description code `META-AMBER-16`,…
For site owners and researchers, this run shows ChatGPT opening a supplied page and grounding its answer in visible body text while hidden page signals stayed outside the answer it could produce. The page contained both a meta-description code and a JSON-LD code in raw HTML, but ChatGPT reported neither, so a successful assistant fetch is not automatically evidence that metadata or structured data influenced the response.
Copilot/Bing was run against p21, `manual-client-copilot-bing-20260626-001-p21`, in a fresh Copilot chat. The exact public target URL was reachable before prompt submission and contained the expected visible code `VISIBLE-PLUM-47`, meta-description code `META-AMBER-16`, and…
For site owners and researchers, this run shows a clean Copilot no-retrieval outcome for a public page that was reachable and contained visible, metadata, and JSON-LD signals. Copilot did not return page content, and the site's logs did not show a matching Copilot or Bing request, so there is no evidence that either the visible text or hidden structured data was inspected in this attempt.
Findings 041-050 completed the p20 and p21 structured-data conflict prompt sets across ChatGPT, Claude, Gemini, Perplexity, and Copilot/Bing. The p20 fixture visibly told the assistant that metadata and JSON-LD disagreed with the page body. The p21 fixture removed that visible…
For site owners and researchers, these paired tests show that a successful assistant page fetch does not mean every hidden page signal influenced the answer. In this lab, ChatGPT stayed on visible body text even when metadata and JSON-LD were present, while Claude also surfaced the meta description but still did not expose JSON-LD. Other clients failed or declined retrieval in these runs, so there was no evidence that they inspected either visible or hidden page content.
ChatGPT was run against p22, `manual-client-chatgpt-20260625-001-p22`, in a fresh native ChatGPT Temporary Chat. The exact public target URL was reachable before prompt submission and contained the expected visible code `VISIBLE-PLUM-47`, meta-description code `META-AMBER-16`,…
For site owners and researchers, this run shows that changing the prompt to a neutral page-quality summary can change the retrieval outcome even when the same public page is reachable and previous ChatGPT tests fetched the related fixture. ChatGPT did not inspect visible text, metadata, or JSON-LD in this attempt, so the absence of hidden codes in the answer is a no-retrieval result rather than evidence about which page fields were visible to the client.
Claude was run against p22, `manual-client-claude-20260625-001-p22`, in a fresh native incognito Claude chat. The exact public target URL was reachable before prompt submission and contained the expected visible code `VISIBLE-PLUM-47`, meta-description code `META-AMBER-16`, and…
For site owners and researchers, this run shows that Claude can surface a page's meta description even when the prompt asks only for a neutral page-quality summary and does not lead the model toward hidden metadata. The visible body text and meta description were both available in Claude's extracted view, while JSON-LD that existed in the raw page was still absent, so assistant answers can reflect a partial page extraction rather than a full source-code read.
Gemini was run against p22, `manual-client-gemini-20260625-001-p22`, in a fresh native Gemini Temporary Chat. The exact public target URL was reachable before prompt submission and contained the expected visible code `VISIBLE-PLUM-47`, meta-description code `META-AMBER-16`, and…
For site owners and researchers, this run shows that Gemini's Temporary Chat can fail to retrieve a public page even when the page is live and includes visible, metadata, and JSON-LD signals. Because Gemini never reached the origin during the attempt, the result is a no-retrieval outcome, not evidence about whether Gemini would have preferred visible body text, meta descriptions, or structured data if it had fetched the page.
Perplexity was run against p22, `manual-client-perplexity-20260625-001-p22`, in a fresh Perplexity native incognito thread. The exact public target URL was reachable before prompt submission and contained the expected visible code `VISIBLE-PLUM-47`, meta-description code…
For site owners and researchers, this run shows that a failed assistant page-read can still cause crawler traffic to the site. Perplexity did not open the tested page, so it could not inspect visible text, metadata, or structured data, but the site's logs still recorded a robots.txt request from PerplexityBot during the attempt.
Copilot/Bing was run against p22, `manual-client-copilot-bing-20260626-001-p22`, in a fresh Copilot Temporary chat. The exact public target URL was reachable before prompt submission and contained the expected visible code `VISIBLE-PLUM-47`, meta-description code `META-AMBER-16`,…
For site owners and researchers, this run shows that Copilot can decline to retrieve a provided page even when the page is publicly reachable and the prompt asks for a neutral site-quality summary. In that case, the assistant answer is not evidence about which visible, metadata, or structured-data signals Copilot can read, because the site logs showed no prompt-window request for the page.
Findings 052-056 completed the p22 neutral hidden-signal prompt family across ChatGPT, Claude, Gemini, Perplexity, and Copilot/Bing. The p22 prompt reused the hidden structured-data fixture from p21, but asked for a neutral site-owner page-quality summary instead of naming meta…
For site owners and researchers, this set shows why retrieval evidence has to come before content-visibility conclusions. In the neutral prompt variant, only Claude actually reached the tested page, and its answer again reflected visible text plus the meta description while leaving JSON-LD out. The other clients did not inspect the page in these attempts, so their answers cannot tell us whether they would have used visible text, metadata, or structured data if retrieval had succeeded.
Findings 041, 042, 047, 049, 052, 053, and 057 isolate the ChatGPT and Claude runs across p20, p21, and p22. All three prompts used exact target URLs. p20 visibly announced that page body text, metadata, and JSON-LD disagreed. p21 served the same hidden-signal layout without the…
For site owners and researchers, small changes in how a question is framed can change whether an assistant reaches the page at all, not just which parts of the page it summarizes. In these runs, ChatGPT fetched the page when the prompt explicitly asked about hidden fields but did not fetch the same target under neutral wording. Claude fetched every variant and consistently exposed the meta description alongside visible text while leaving JSON-LD out, so its behavior looks more stable across prompt wording for this fixture.
The ChatGPT p23 neutral retrieval-stability rerun reused the same `/lab/reading/hidden-structured-data-conflict` fixture as p21 and p22, but changed two things from the p22 no-hit: it used a short id-only target URL and asked for a concise site-owner page-quality note without…
For site owners and researchers, ChatGPT's earlier neutral no-hit was not necessarily a stable refusal or inability to read this page. With a shorter target URL and similarly neutral site-owner wording, ChatGPT reached the page again and summarized only what was visible in its rendered page view. That keeps the hidden-field conclusion unchanged for this run, but it shows retrieval itself can depend on prompt and URL presentation details.
The ChatGPT p24 neutral retrieval-stability rerun used the same `/lab/reading/hidden-structured-data-conflict` fixture as p22 and p23. Unlike p23, it kept the full correlation query metadata in the target URL while using neutral site-owner readability wording. ChatGPT returned…
For site owners and researchers, the longer tracking-style query string did not by itself stop ChatGPT from retrieving this lab page. In this run, ChatGPT reached the target even with the full correlation URL and still summarized only the visible page text. That points back toward prompt framing, model routing, account state, or normal product variability as possible reasons the earlier p22 neutral run produced no target-page hit.
The ChatGPT p25 neutral retrieval-stability rerun used the same `/lab/reading/hidden-structured-data-conflict` fixture as p22-p24. Like p23, it used a short id-only target URL, but changed the task framing to a neutral support-answer request. ChatGPT returned `fetched:true`,…
For site owners and researchers, ChatGPT reached the same lab page again when the prompt used ordinary support-answer wording and a short URL. Across the latest short-URL reruns, ChatGPT consistently opened the target and reported only visible page text, so the earlier p22 no-hit looks more like retrieval variability or prompt-routing behavior than a stable inability to access this fixture.
Findings 052, 059, 060, and 061 compare four ChatGPT runs against the same `/lab/reading/hidden-structured-data-conflict` fixture under neutral site-owner wording. The first neutral run, p22, used the default full correlation URL and returned `fetched:false`, `pages_opened:0`,…
For site owners and researchers, a single failed ChatGPT page-open attempt should not be treated as proof that the page, URL shape, or neutral prompt wording is consistently inaccessible. In this series, the same public fixture shifted from one clean no-hit to three confirmed hits later the same day, while the successful answers still reflected only visible page text. The practical takeaway is to separate "did the AI client fetch the page?" from "which page fields did it use?", and to rerun neutral prompts before turning one no-hit into a broad retrieval claim.
Claude was prompted in a fresh native incognito chat with `manual-client-claude-metadata-isolation-20260701-001-p26`, the first Claude-focused head-signal isolation prompt. The target fixture keeps visible body text separate from page-head signals: visible body marker…
For site owners and researchers, Claude's page-opening view can expose more than ordinary visible body text in at least some runs. In this test, the assistant fetched the page and described access to page-head metadata, while the origin logs confirmed the visit. That means metadata and preview fields should be treated as potentially visible to assistant retrieval tools, not as private or irrelevant just because they are outside the rendered article body.
Claude was prompted in a fresh native incognito chat with `manual-client-claude-metadata-isolation-20260701-001-p27`, the page-preview variant of the Claude metadata-isolation packet. The target fixture keeps visible body text separate from page-head signals: visible body marker…
For site owners and researchers, this repeat run strengthens the evidence that Claude's page-opening view may include page-preview and metadata fields alongside normal article text. The page did not print those head markers in the visible body, yet Claude described access to the title, description, OpenGraph, and canonical signals while the origin logs confirmed a real page fetch. Metadata should be treated as material that some assistant retrieval tools may inspect or summarize, not as background decoration.
Claude was prompted in a fresh native incognito chat with `manual-client-claude-metadata-isolation-20260701-001-p28`, the source-area boundary variant of the Claude metadata-isolation packet. The target fixture keeps visible body text separate from page-head signals: visible body…
For site owners, SEO/AEO teams, and researchers, this run shows that Claude's page-opening view can expose several non-visible page-head signals in addition to normal body text, including description, OpenGraph, and canonical fields. At the same time, Claude's extracted view did not surface the page's JSON-LD marker even though the served page contained it, so different head-adjacent sources should not be treated as equally visible to every assistant retrieval tool.
The Claude metadata-isolation sequence ran three fresh native incognito chats against the same `head-signal-isolation` fixture: `manual-client-claude-metadata-isolation-20260701-001-p26`, `manual-client-claude-metadata-isolation-20260701-001-p27`, and…
For site owners, SEO/AEO teams, and researchers, this comparison shows that Claude's page-opening view can include page-head and preview signals that are not printed in normal body copy, but the exposure is not uniform across all source areas. Description, OpenGraph, and canonical fields were visible enough to appear in the strongest run, while JSON-LD did not surface in Claude's extracted view. Metadata should be treated as potentially readable by assistant retrieval tools, but each field still needs direct testing instead of assuming all head-adjacent signals behave the same way.
The ChatGPT p29 same-condition neutral repeat reused the p24-style full correlation URL shape and the same neutral readability task against `/lab/reading/hidden-structured-data-conflict`. ChatGPT returned `fetched:true`, `pages_opened:1`, reported visible token `VISIBLE-PLUM-47`,…
For site owners and researchers, this repeat makes the earlier ChatGPT p24 full-URL retrieval look less like a one-off accident. Under the same neutral readability wording and the same long tracking-style URL shape, ChatGPT again opened the page and answered from visible page text. This still does not prove hidden metadata was unavailable everywhere; it shows that this ChatGPT Temporary Chat response used the visible content it exposed after a confirmed fetch.
The ChatGPT p30 same-condition neutral repeat reused the p24-style full correlation URL shape and the same neutral readability task against `/lab/reading/hidden-structured-data-conflict`. ChatGPT returned `fetched:true`, `pages_opened:1`, reported visible token `VISIBLE-PLUM-47`,…
For site owners and researchers, this second repeat makes the ChatGPT full-URL neutral-readability behavior look more stable across repeated fresh Temporary Chats. ChatGPT again opened the supplied page and answered from visible page text, even though the served page also contained hidden meta-description and JSON-LD markers. The result should still be read as one account, one mode, and one time window, not as a universal claim about every ChatGPT retrieval path.
The ChatGPT p31 same-condition neutral repeat reused the p24-style full correlation URL shape and the same neutral readability task against `/lab/reading/hidden-structured-data-conflict`. ChatGPT returned `fetched:true`, `pages_opened:1`, reported visible token `VISIBLE-PLUM-47`,…
For site owners and researchers, this adds confidence that a ChatGPT page-reading path can repeatedly use visible page copy without exposing hidden metadata or structured data in its answer. That distinction matters because optimizing visible content and optimizing machine-readable page fields are not always the same thing. The result is useful evidence for this measured setup, but not a promise that every ChatGPT surface will treat hidden page signals the same way.
Findings 067, 068, and 069 compare three same-condition ChatGPT neutral readability repeats against `/lab/reading/hidden-structured-data-conflict`. All three runs used the p24-style full correlation URL shape, the same neutral readability task family, a fresh ChatGPT Temporary…
For site owners and researchers, this repeat series makes ChatGPT's full-URL page-opening behavior look less fragile than a single run suggested. In this measured setup, three fresh Temporary Chats opened the same long tracking-style URL and answered from visible page text, while still not exposing the hidden metadata or structured-data markers in the copied answer. The practical lesson is to rerun retrieval tests before treating one no-hit as stable behavior, and to keep retrieval success separate from which page fields the answer actually uses.
Claude was prompted in a fresh native incognito chat with `manual-client-claude-neutral-page-quality-20260702-001-p32`, the neutral page-quality follow-up for the `head-signal-isolation` fixture. The target page contains visible body marker `VISIBLE-QUARTZ-39`, meta-description…
For site owners, SEO/AEO teams, and researchers, this run shows that neutral wording can still cause Claude to retrieve a page and recognize that page-head metadata exists, but it may not quote the exact hidden marker values unless the prompt asks more directly for source areas or code-like tokens. Retrieval, metadata awareness, and marker-level extraction are separate behaviors, so a page can be fetched successfully without every hidden signal appearing in the assistant's copied answer.
The ChatGPT p33 neutral short-URL repeat reused the p24/p29-p31 neutral readability wording against `/lab/reading/hidden-structured-data-conflict`, but changed the target URL to short id-only correlation. ChatGPT returned `fetched:true`, `pages_opened:1`, reported visible token…
For site owners and researchers, this run shows that shortening the correlation URL did not prevent ChatGPT from opening the same neutral readability fixture. The copied answer again used visible page text and did not expose hidden metadata or structured-data marker values, so URL shape and content-field visibility remain separate questions. A short URL can still fetch successfully, but the answer may stay grounded in what the assistant's page view exposes.
The ChatGPT p34 neutral short-URL repeat reused the p24/p29-p31/p33 neutral readability wording against `/lab/reading/hidden-structured-data-conflict`, using short id-only URL correlation. ChatGPT returned `fetched:true`, `pages_opened:1`, reported visible token…
For site owners and researchers, this second short-URL repeat strengthens the evidence that ChatGPT can open the neutral readability fixture when the tracking URL is shortened to a simple id parameter. The copied answer again stayed with visible page content and did not expose hidden metadata or structured-data marker values, so the lab should keep treating retrieval success and hidden-field visibility as separate behaviors.
The ChatGPT p35 neutral short-URL repeat reused the p24/p29-p31/p33-p34 neutral readability wording against `/lab/reading/hidden-structured-data-conflict`, using short id-only URL correlation. ChatGPT returned `fetched:true`, `pages_opened:1`, reported visible token…
For site owners and researchers, this third short-URL repeat completes a matched trio showing that ChatGPT repeatedly opened the neutral readability fixture when the tracking URL used a simple id parameter. The copied answer again stayed with visible page content and did not expose hidden metadata or structured-data marker values, so repeated retrieval success should still be kept separate from what hidden page fields the assistant answer surfaces.
Findings 072, 073, and 074 compare three ChatGPT short id-only URL repeats against `/lab/reading/hidden-structured-data-conflict`. All three runs used the same neutral readability task family as p24 and p29-p31, the same fixture, a fresh ChatGPT Temporary Chat, and the logged-in…
For site owners and researchers, this matched short-URL trio shows that ChatGPT repeatedly opened the same neutral readability fixture when the tracking URL used only a simple id parameter. Compared with the earlier full-correlation trio, the practical result is that both URL shapes could produce repeat target hits in this setup, while the copied answers still stayed with visible page text instead of hidden metadata or structured-data markers. Retrieval success and hidden-field visibility need to stay separate in measurement and reporting.
Findings 063 through 066 showed that Claude's p26-p28 metadata-isolation prompts fetched `/lab/reading/head-signal-isolation` in fresh native incognito chats and exposed visible body text plus page-head context. Finding 071 then ran the p32 neutral page-quality follow-up against…
For site owners, SEO/AEO teams, and researchers, this comparison shows that Claude can retrieve the same page under both inspection-style and more neutral page-quality wording, but the wording affects how much hidden page-head detail appears in the final answer. A successful assistant fetch does not automatically mean every metadata value will be quoted to the user. Metadata can shape or inform the assistant's view while still remaining absent from the copied response unless the task asks for it more directly.
Claude was prompted in a fresh native incognito chat with `manual-client-claude-plain-page-quality-20260702-001-p36`, a plainer site-owner review prompt for the `head-signal-isolation` fixture. Unlike p32, the p36 prompt did not ask for source areas, metadata, structured data,…
For site owners, SEO/AEO teams, and researchers, this run shows that Claude can still fetch and understand the purpose of a diagnostic page even when the prompt is phrased like ordinary site-owner feedback rather than source inspection. But the final answer did not expose the exact hidden marker values. That makes prompt wording important: a page may be retrieved and its metadata structure may affect the assistant's understanding without every machine-readable value being copied back to the user.
The Claude head-signal series now includes five fresh controlled-browser runs against the same `/lab/reading/head-signal-isolation` fixture: `manual-client-claude-metadata-isolation-20260701-001-p26`, `manual-client-claude-metadata-isolation-20260701-001-p27`,…
For site owners, SEO/AEO teams, and researchers, these Claude runs show that a successful assistant page fetch is not the same thing as exact metadata leakage in the final answer. Claude retrieved the same page under direct inspection, neutral review, and plainer site-owner wording, but exact head-marker values only appeared when the prompt asked much closer to source-area inspection. Ordinary-sounding questions can still let metadata shape the assistant's understanding while leaving specific machine-readable values out of the copied response.
ChatGPT was prompted in a fresh native Temporary Chat with `manual-client-chatgpt-plain-head-signal-20260702-001-p37`, a plain site-owner review prompt for the `/lab/reading/head-signal-isolation` fixture. The prompt matched the plainer Claude p36 wording style: it asked for a…
For site owners, SEO/AEO teams, and researchers, this run shows that ChatGPT can fetch a diagnostic page under ordinary review wording while keeping its final answer focused on visible page content. The assistant recognized the page's purpose and body-versus-head setup, but it did not copy the exact hidden page-head marker values back into the response. That matters because retrieval, page understanding, and marker-level disclosure are separate behaviors that need separate evidence.
Gemini was prompted in a fresh new chat with `manual-client-gemini-plain-head-signal-20260702-001-p37`, the plain site-owner review prompt for the `/lab/reading/head-signal-isolation` fixture. The prompt asked for a brief page review and did not ask Gemini to inspect source…
For site owners, SEO/AEO teams, publishers, and researchers, this run shows that Gemini can retrieve a directly supplied diagnostic page under ordinary site-owner review wording, and its answer may include exact visible body text even when exact hidden page-head marker values are not copied back. That distinction matters because a successful fetch, category-level awareness of page-head signals, visible marker quotation, and hidden marker disclosure are different behaviors that should not be collapsed into one claim.
Perplexity was prompted in a fresh native incognito thread with `manual-client-perplexity-plain-head-signal-20260702-001-p37`, the plain site-owner review prompt for the `/lab/reading/head-signal-isolation` fixture. The prompt asked for a brief page review and did not ask…
For site owners, SEO/AEO teams, publishers, and researchers, this run shows why root or robots traffic from an AI service should not be counted as proof that the requested page was read. Perplexity contacted the site during the prompt window, but its final answer said the target page fetch failed and the server logs did not show a hit on the requested diagnostic page. That makes this a site-contact event, not page-content retrieval evidence.
Copilot/Bing was prompted in a fresh Copilot Temporary chat with `manual-client-copilot-bing-plain-head-signal-20260702-001-p37`, the plain site-owner review prompt for the `/lab/reading/head-signal-isolation` fixture. The prompt asked for a brief page review and did not ask…
For site owners, SEO/AEO teams, publishers, and researchers, this run shows that a browser-based AI assistant can produce a clear failed-fetch answer even when the page is publicly reachable and other clients have fetched the same fixture. That means a no-content answer should be treated as a retrieval outcome for that client and moment, not as evidence about what the page contains or whether hidden page-head signals would have been visible if the page had loaded.
The plain head-signal comparison reviewed five completed controlled-browser runs against the `/lab/reading/head-signal-isolation` fixture: | Client | Attempt | Retrieval result | Origin result | Copied answer scope | |---|---|---|---|---| | Claude |…
For site owners, SEO/AEO teams, publishers, and researchers, the same public page and similar plain review prompt can produce different retrieval outcomes across AI clients. A successful answer, a visible marker quote, a root or robots request, and a failed-fetch answer are different evidence types. The practical takeaway is that AI visibility testing needs both the copied assistant answer and direct server logs before deciding whether a client actually read a page or only contacted the site around it.
Perplexity was prompted in a fresh native incognito thread with `manual-client-perplexity-plain-head-signal-repeat-20260702-001-p38`, a repeat of the plain site-owner review prompt against the `/lab/reading/head-signal-isolation` fixture. The prompt asked Perplexity to open the…
For site owners, SEO/AEO teams, publishers, and researchers, this repeat shows that a failed AI-client fetch can appear in more than one server-side shape. In one Perplexity run the site saw root and robots traffic without a page hit; in this repeat the answer still said the page could not be fetched, but the server did not see related prompt-window origin activity at all. That makes direct server logs essential for separating page reads, site-contact-only attempts, and clean no-hit outcomes.
Copilot/Bing was prompted in a fresh Temporary chat with `manual-client-copilot-bing-plain-head-signal-repeat-20260702-001-p38`, a repeat of the plain site-owner review prompt against the `/lab/reading/head-signal-isolation` fixture. The prompt asked Copilot/Bing to open the…
For site owners, SEO/AEO teams, publishers, and researchers, this repeat shows that a mainstream AI client can answer that it could not fetch a page even when the same lab fixture is reachable and other clients have read it. The important part is not the model's wording alone: the server logs also showed no matching page read during the prompt window, so this run should be treated as a real client-specific no-hit rather than a hidden successful visit.
This comparison reviewed the completed Perplexity and Copilot/Bing p37/p38 plain head-signal repeats against the `/lab/reading/head-signal-isolation` fixture. | Client | Attempts | Answer-side result | Direct-origin result | |---|---|---|---| | Perplexity |…
For site owners, SEO/AEO teams, publishers, and researchers, repeated failed-fetch answers are not all the same measurement outcome. Copilot/Bing twice showed a clean no-hit: the client said it could not fetch the page and the server saw no related visit. Perplexity twice said it could not fetch the page, but one run still produced root and robots traffic while the repeat produced none. That means AI visibility audits need to track the assistant answer, the exact target-page hit, and any nearby origin traffic as separate facts.
The ChatGPT p39 mixed URL-shape run started the `manual-client-chatgpt-neutral-url-shape-mixed-20260702-001` packet with the full-correlation URL shape and the same neutral readability task used in the earlier repeat series. ChatGPT returned `fetched:true`, `pages_opened:2`,…
For site owners and researchers, this first mixed-order run shows that ChatGPT again opened the neutral readability fixture when the URL carried the full tracking-style correlation parameters. The answer still surfaced the visible page marker rather than hidden metadata or structured-data marker values, so the run adds another retrieval confirmation while keeping hidden-field conclusions scoped to what the copied answer exposed.
The ChatGPT p40 mixed URL-shape run continued the `manual-client-chatgpt-neutral-url-shape-mixed-20260702-001` packet with the short id-only URL shape and the same neutral readability task used for p39. ChatGPT returned `fetched:true`, `pages_opened:1`, reported visible token…
For site owners and researchers, this short-URL run shows that ChatGPT again opened the neutral readability fixture even when the target URL carried only the id-style correlation parameter. The copied answer surfaced the visible page marker and stayed away from hidden metadata or structured-data marker values, so the run adds direct retrieval evidence while keeping hidden-field conclusions limited to the answer text.
The ChatGPT p41 mixed URL-shape run continued the `manual-client-chatgpt-neutral-url-shape-mixed-20260702-001` packet with the second full-correlation URL shape and the same neutral readability task used for p39 and p40. ChatGPT returned `fetched:true`, `pages_opened:1`, reported…
For site owners and researchers, this repeat full-correlation run shows that ChatGPT again opened the neutral readability fixture when the URL carried the complete tracking-style query string. The copied answer surfaced the visible page marker and did not report hidden metadata or structured-data marker values, so it adds another retrieval confirmation while keeping hidden-field conclusions limited to the answer text.
The ChatGPT p42 mixed URL-shape run completed the `manual-client-chatgpt-neutral-url-shape-mixed-20260702-001` packet with the second short id-only URL shape and the same neutral readability task used for p39-p41. ChatGPT returned `fetched:true`, `pages_opened:1`, reported…
For site owners and researchers, this second short-id run shows that ChatGPT again opened the neutral readability fixture even when the URL did not carry the full tracking-style query string. The answer quoted visible page text and avoided hidden metadata or structured-data marker values, so the strongest result is a retrieval confirmation with visible-answer evidence rather than proof about every field ChatGPT may have fetched internally.
The completed `manual-client-chatgpt-neutral-url-shape-mixed-20260702-001` packet alternated URL shapes under matched neutral readability wording: | Attempt | URL shape | Result | Pages opened | Raw event id | |---|---|---:|---:|---| | p39 | full-correlation query string |…
For site owners, SEO/AEO teams, publishers, and researchers, this packet shows that ChatGPT could open the same page whether the supplied URL carried a full tracking-style query string or only a short id parameter. In these four runs, simplifying the URL did not stop retrieval and did not make hidden metadata or structured-data marker values appear in the copied answer. The result is useful evidence about direct URL opening under this prompt family, but it should not be stretched into a universal rule about every ChatGPT mode or every URL shape.
This comparison reviews three completed ChatGPT neutral readability groups for `/lab/reading/hidden-structured-data-conflict`: the p29-p31 full-correlation repeat trio, the p33-p35 short id-only repeat trio, and the p39-p42 mixed-order URL-shape packet. | Series | URL shape |…
For site owners, SEO/AEO teams, publishers, and researchers, these ten runs show that this ChatGPT setup repeatedly opened the same supplied page whether the target URL used a long tracking-style query string or only a short id parameter. URL simplification did not explain the earlier isolated no-hit and did not change the copied answer into a hidden-metadata or structured-data answer. The practical takeaway is that direct URL-opening tests need repeated runs and origin logs before treating URL shape as the cause of a fetch or no-fetch result.
Perplexity was prompted in a fresh native incognito thread with `manual-client-perplexity-plain-head-signal-randomized-repeat-20260703-001-p43`, the first randomized repeat of the plain site-owner review prompt against the `/lab/reading/head-signal-isolation` fixture. The prompt…
For site owners, SEO/AEO teams, publishers, and researchers, this run shows why a failed AI-client fetch should not be treated as proof that the site saw no AI-system traffic. Perplexity said it could not retrieve the supplied page and the server saw no page hit, yet the same prompt window did include PerplexityBot contact with the site's root and robots file. In practical terms, origin logs can reveal a partial site-contact attempt that is invisible in the copied answer alone.
Perplexity was prompted in a fresh native incognito thread with `manual-client-perplexity-plain-head-signal-randomized-repeat-20260703-001-p44`, the second randomized repeat of the plain site-owner review prompt against the `/lab/reading/head-signal-isolation` fixture. The prompt…
For site owners, SEO/AEO teams, publishers, and researchers, this run shows that the same Perplexity prompt family can fail without leaving any matching origin trace at all. Compared with the immediately preceding p43 run, where the copied answer also said the page could not be fetched but the server still saw root and robots requests, p44 reinforces that answer text and server-side behavior have to be checked separately. A failed fetch answer is stable here, but the origin footprint is not.
Copilot/Bing was prompted in a fresh Copilot Temporary chat with `manual-client-copilot-bing-plain-head-signal-randomized-repeat-20260703-001-p43`, the first randomized repeat of the plain site-owner review prompt against the `/lab/reading/head-signal-isolation` fixture. The…
For site owners, SEO/AEO teams, publishers, and researchers, this run shows another case where Copilot/Bing said it could not fetch a directly supplied page and the lab origin did not see a matching target-page request. That matters because a failed answer alone is not enough: the server-side log confirms that, for this measured prompt window, the client did not simply fetch the page and then decline to summarize it. The result is still narrow, but it strengthens the repeated Copilot/Bing clean no-hit pattern for this fixture and prompt family.
Copilot/Bing was prompted in a fresh Copilot Temporary chat with `manual-client-copilot-bing-plain-head-signal-randomized-repeat-20260703-001-p44`, the second randomized repeat of the plain site-owner review prompt against the `/lab/reading/head-signal-isolation` fixture. The…
For site owners, SEO/AEO teams, publishers, and researchers, this repeats the pattern where Copilot/Bing says it cannot retrieve a directly supplied page and the lab origin does not see a matching request during the measured prompt window. The server-side evidence matters because it separates a real no-contact outcome from a client that fetched the page but declined to summarize it. This does not prove Copilot/Bing can never retrieve the page, but it strengthens the observed clean no-hit pattern for this exact fixture, account mode, and prompt family.
This comparison reviewed four completed Perplexity plain head-signal runs: `manual-client-perplexity-plain-head-signal-20260702-001-p37`, `manual-client-perplexity-plain-head-signal-repeat-20260702-001-p38`,…
For site owners, SEO/AEO teams, publishers, and researchers, a Perplexity failed-fetch answer was repeatable in this fixture, but the server-side footprint was not. Sometimes the same kind of prompt left root and robots traffic without fetching the requested page, and sometimes it left no related origin traffic at all. That means visibility audits should not collapse failed answers into a single outcome: the answer, exact page hit, and nearby origin contact need to be recorded separately.
This comparison reviewed four completed Copilot/Bing plain head-signal runs: `manual-client-copilot-bing-plain-head-signal-20260702-001-p37`, `manual-client-copilot-bing-plain-head-signal-repeat-20260702-001-p38`,…
For site owners, SEO/AEO teams, publishers, and researchers, Copilot/Bing repeatedly said it could not retrieve the supplied page and the lab repeatedly saw no matching origin contact during the measured windows. That makes this different from a client that fetches a page but gives a weak answer: for these four runs, the server-side evidence supports a clean no-hit pattern. The result is still scoped to one fixture, account mode, and prompt family, so it should be used as repeated evidence for this condition rather than as a universal Copilot/Bing access rule.
Multiple evidence layers capture what actually happens when an AI system visits a page.
Every page request is logged with full headers, timing, IP, DNS, and user-agent — the foundation for classification.
Images, CSS, scripts, fonts, and JSON endpoints — does the visitor fetch subresources, or just the HTML?
Can the AI read visible HTML, JavaScript-rendered content, image text, hidden links, and structured data?
Server-rendered banners, JavaScript-gated content, and reject-path fixtures — does the AI respect consent signals?
robots.txt, crawl-delay, meta noindex, nosnippet, and nofollow — what directives does the visitor honor?
Deterministic linked pages measure how deep an AI visitor follows links from a starting point.
Research methodology, test plans, and technical references.
Direct links to test pages and research tools.