# Finding 015: Copilot/Bing free-tier retrieval is search-index-gated, not direct-fetch ## Date 2026-06-28 ## Status Published ## Summary A re-test of Copilot/Bing's ability to fetch a target URL was conducted on 2026-06-28, reproducing the original 0/19 no-hit result from Finding 013. The re-test used a fresh Copilot temporary chat with a newly generated p01 open-target-summarize prompt. Copilot returned `fetched:false` with "Invalid tool invocation" as the evidence quote, and no matching direct-origin events were observed in the bounded timestamp window. Analysis of Copilot's architecture reveals this is not a transient failure but a structural limitation: Copilot does not have a working direct URL fetch tool on the free tier. Copilot is built on the Microsoft Prometheus model, which integrates Bing search results into the LLM context. It can answer questions about pages that Bing has already crawled and indexed, but it cannot open an arbitrary URL on demand the way ChatGPT and Claude can. The "Invalid tool invocation" error indicates a `fetch_web_content` tool exists in Copilot's tool set but is either disabled, restricted to Bing-indexed URLs, or gated behind the paid Copilot Pro tier. This is architecturally analogous to Gemini's Google Search index dependency (Finding 014), except that Gemini's tool at least attempts the fetch and sometimes succeeds, while Copilot's tool rejects the invocation entirely. ## Method ### Re-test The re-test was run from the dedicated OpenClaw browser profile in a fresh Copilot temporary chat on 2026-06-28 at approximately 13:08 UTC. The prompt used the same p01 open-target-summarize format as the original run, with a fresh run ID (`manual-client-copilot-bing-20260628-002-p01`) and target URL: ``` https://ai-crawler-lab.kaistone.ai/lab/root?id=manual-client-copilot-bing-20260628-002-p01&test_kind=manual_client_prompt&surface=copilot-bing-web&run_id=manual-client-copilot-bing-20260628-002&model_id=copilot-bing-manual&source_prompt_id=open-target-summarize&prompt_code=manual-client-copilot-bing-20260628-002-p01&expected_actor=assistant_manual_client&bait=open-target-summarize ``` The lab server was live and collecting events. The bounded observation window was 13:08:00–13:11:30 UTC. After the response was received, the "Copy" button was used to extract the full JSON response from the Copilot chat interface. The lab server's `/api/hits` endpoint was queried for all events in the bounded window. ### Architecture analysis Architecture analysis was conducted by reviewing Microsoft's public documentation, Wikipedia's Microsoft Copilot article, and the pattern of error messages across all 20 Copilot tests (19 original + 1 re-test). The "Invalid tool invocation" error was consistent across all 20 attempts, spanning reading, directive, resource-loading, consent, and discovery prompt categories. ## Raw Evidence ### Re-test response ```json { "attempt_id": "manual-client-copilot-bing-20260628-002-p01", "source_prompt_id": "open-target-summarize", "prompt_code": "manual-client-copilot-bing-20260628-002-p01", "ai_system": "Copilot/Bing", "fetched": false, "pages_opened": 0, "answer": "This client cannot open or retrieve content from the target URL.", "evidence_quote": "Invalid tool invocation.", "limitations": "No ability to fetch or open external web pages in this environment.", "confidence": 0.41 } ``` ### Origin evidence No matching direct-origin events were found in the bounded 13:08:00–13:11:30 UTC window. All events in that window originated from the local lab browser or prior test sessions — none from Microsoft Azure IPs, no Copilot/Bing user-agent strings, and no requests to the target URL path. ### Cross-test consistency The "Invalid tool invocation" error has been consistent across all 20 Copilot tests: | Test | Prompt | Date | Result | Error | |------|--------|------|--------|-------| | p01 | open-target-summarize | 2026-06-27 | fetched:false | Invalid tool invocation | | p02 | reading-visible-html | 2026-06-27 | fetched:false | Invalid tool invocation | | p03 | reading-js-rendered | 2026-06-27 | fetched:false | Invalid tool invocation | | p04 | reading-hidden-links | 2026-06-27 | fetched:false | Invalid tool invocation | | p05 | reading-image-alt-mismatch | 2026-06-27 | fetched:false | Invalid tool invocation | | p06 | directive-noindex | 2026-06-27 | fetched:false | Invalid tool invocation | | p07 | crawl-depth-one | 2026-06-27 | fetched:false | Invalid tool invocation | | p08 | visible-link-follow | 2026-06-27 | fetched:false | Invalid tool invocation | | p09 | resource-css-stylesheet | 2026-06-27 | fetched:false | Invalid tool invocation | | p10 | resource-png-image | 2026-06-27 | fetched:false | Invalid tool invocation | | p11 | resource-svg-image | 2026-06-27 | fetched:false | Invalid tool invocation | | p12 | resource-script-file | 2026-06-27 | fetched:false | Invalid tool invocation | | p13 | resource-json-fetch | 2026-06-27 | fetched:false | Invalid tool invocation | | p14 | resource-font-file | 2026-06-27 | fetched:false | Invalid tool invocation | | p15 | consent-server-banner | 2026-06-27 | fetched:false | Invalid tool invocation | | p16 | consent-js-gated-content | 2026-06-27 | fetched:false | Invalid tool invocation | | p17 | consent-reject-limited-content | 2026-06-27 | fetched:false | Invalid tool invocation | | p18 | discovery-sitemap-only | 2026-06-27 | fetched:false | Invalid tool invocation | | p19 | discovery-robots-only | 2026-06-27 | fetched:false | Invalid tool invocation | | p01-rerun | open-target-summarize | 2026-06-28 | fetched:false | Invalid tool invocation | The error is not prompt-specific, path-specific, or content-specific. It is consistent across all 19 prompt categories and the re-test. ## Interpretation Copilot on the free tier does not have a working direct URL fetch tool. The "Invalid tool invocation" error indicates the tool exists in the tool set but is disabled or restricted. This is consistent with Copilot's architecture as a search-grounded assistant built on the Microsoft Prometheus model, which integrates Bing search results rather than fetching URLs directly. This is architecturally different from ChatGPT and Claude, which have working direct URL fetch tools (`ChatGPT-User/1.0` and `Claude-User/1.0` respectively). It is superficially similar to Gemini's Google Search index dependency (Finding 014), but with a key difference: Gemini's tool at least attempts the fetch and sometimes succeeds (19/19 after re-runs), while Copilot's tool rejects the invocation entirely (0/20). For AEO/SEO practitioners, this means: if your content is not in Bing's search index, Copilot users cannot access it through conversation. Unlike Claude and ChatGPT, which will fetch any reachable URL, Copilot is gated behind Bing's crawl decisions. ## Limitations - Only the free tier of Copilot was tested. Copilot Pro (paid tier) may have the `fetch_web_content` tool enabled. - The architecture analysis is based on public documentation and error pattern analysis, not on access to Copilot's internal tool configuration. - The lab server is a single origin; behavior might differ for pages already indexed by Bing. - The re-test used temporary chat mode; different Copilot modes (e.g., Copilot in Edge sidebar, Copilot in Bing search results) might behave differently. - A follow-up test should verify whether Copilot can answer questions about the lab pages if they are indexed by Bing (search-grounded access vs direct fetch). ## Publication Thesis Verification - Thesis: Copilot/Bing free-tier retrieval is architecturally search-index-gated, not a direct URL fetcher. The 0/20 no-hit rate is structural, not transient. - Source: Direct-origin server logs from 20 controlled-browser tests, Copilot response JSON, Microsoft public documentation, Wikipedia. - Method: Controlled-browser test with fresh temporary chat, bounded timestamp window, and independent server-side event correlation. Architecture analysis via public documentation review and cross-test error pattern analysis. - Bias: Tests cover one lab origin with one free-tier account; paid tiers and Bing-indexed pages were not tested. - Consensus: Consistent with Findings 007 and 013 (Copilot 0/19 no-hits across all prompt categories). Consistent with Gemini's search-index dependency (Finding 014), but with stricter gating. - Invalidation: Test paid Copilot Pro tier for `fetch_web_content` tool availability. Test whether Copilot can access pages already in Bing's index via search-grounded retrieval. - Verdict: Thesis is well-supported. The 0/20 consistency across all prompt categories and the re-test confirms this is structural, not transient. The "Invalid tool invocation" error pattern is uniform across all tests. - Confidence: high for the recorded free-tier behavior; medium for generalization to all Copilot surfaces and tiers. - Additional tests suggested: test with a domain that is in Bing's search index to verify whether Copilot retrieves indexed URLs; test with Copilot Pro/enterprise tier to check whether higher tiers unlock direct URL fetching; compare with Bing Chat API if available. ## Follow-up tasks 1. Test whether Copilot Pro (paid tier) has the `fetch_web_content` tool enabled for direct URL fetching. 2. Test whether Copilot can answer questions about lab pages that are indexed by Bing (search-grounded access vs direct fetch). 3. Test whether asking Copilot to "search for" the page rather than "open" it produces different retrieval behavior.