# Finding 016: Copilot "search for" vs "open" — Bing search returns no results for lab domain

## Date

2026-06-28

## Status

Published

## Summary

A follow-up test investigated whether Copilot's retrieval behavior changes
when asked to "search for" a page rather than "open" it directly. The
hypothesis was that since Copilot is search-index-gated (Finding 015), using
search-grounded language might trigger Bing search and produce different
retrieval behavior.

The test was run in a fresh Copilot temporary chat. The prompt asked Copilot
to "search for the website ai-crawler-lab.kaistone.ai" and summarize what
the page says it is measuring, using its web search capability.

Copilot reported that it performed a search
(`search_performed: true`), but found no results
(`results_found: false`). The domain does not appear in Bing's search index,
so Copilot could not provide any information about the page. No matching
direct-origin events were observed on the lab server.

This confirms that Copilot's retrieval is entirely gated behind Bing's search
index. When the page is not indexed by Bing, neither direct URL fetching
nor search-grounded retrieval can access it.

## Method

The test was run from the dedicated OpenClaw browser profile in a fresh
Copilot temporary chat on 2026-06-28 at approximately 13:37 UTC. The prompt
asked Copilot to search for the website and return compact JSON:

```
Search for the website ai-crawler-lab.kaistone.ai and tell me what the page
says it is measuring. Use your web search to find information about this site,
then summarize the content you find.

Return compact JSON only with:
search_performed, search_query, results_found, answer, evidence_quote,
source_urls, confidence.
```

The lab server was live and collecting events. The bounded observation
window was 13:36:00–13:39:00 UTC.

After the response was received, the "Copy code" button was used to extract
the full JSON response. The lab server's `/api/hits` endpoint was queried
for all events in the bounded window.

## Raw Evidence

### Copilot response

```json
{
  "search_performed": true,
  "search_query": "ai-crawler-lab.kaistone.ai what does the page measure",
  "results_found": false,
  "answer": "No publicly indexed information was found describing what the ai-crawler-lab.kaistone.ai page measures. The domain does not appear in search results, and no summaries, documentation, or references to the site were returned.",
  "evidence_quote": "No search results referenced the domain or provided any description of its measurement purpose.",
  "source_urls": [],
  "confidence": 0.32
}
```

### Origin evidence

No matching direct-origin events were found in the bounded 13:36:00–13:39:00
UTC window. All events in that window originated from the local lab browser
or prior test sessions — none from Microsoft Azure IPs, no Bing bot
user-agent strings, and no requests to the target URL path.

### Comparison with "open" variant

| Variant | Prompt approach | Search performed | Results found | Origin hits | Confidence |
|---------|----------------|-----------------|---------------|-------------|------------|
| "Open" (p01) | Open target URL and summarize | N/A (tool rejected) | N/A | 0/20 | 0.41–0.43 |
| "Search for" | Search for the domain and summarize | Yes | No (not indexed) | 0 | 0.32 |

The "search for" variant produced a different failure mode: instead of
"Invalid tool invocation" (tool rejected), Copilot successfully performed
a Bing search but found no results because the lab domain is not in Bing's
index. The confidence was lower (0.32 vs 0.41–0.43), reflecting that
Copilot was less certain about a negative search result than about a
tool rejection.

## Interpretation

This test confirms the search-index-gating hypothesis from Finding 015.
When Copilot is asked to "open" a URL, the `fetch_web_content` tool is
rejected ("Invalid tool invocation"). When Copilot is asked to "search for"
a domain, the Bing search tool works but returns no results for domains
not in Bing's index.

Neither approach produces direct-origin server hits. The difference is:

1. **"Open" variant**: Tool rejection — Copilot's `fetch_web_content` tool
   is disabled or restricted on the free tier. The tool invocation itself
   is rejected before any network request is made.

2. **"Search for" variant**: Search success, no results — Copilot's Bing
   search tool works, but the lab domain is not indexed by Bing, so no
   results are returned and no origin request is made to the target server.

For AEO/SEO practitioners, this means Copilot users cannot access content
that is not in Bing's search index through either approach — direct URL
fetching is disabled, and search-grounded retrieval returns nothing for
unindexed domains. The practical implication is that **Bing indexing is a
prerequisite for Copilot content access**, while ChatGPT and Claude can
fetch any reachable URL regardless of search index status.

## Limitations

- Only one "search for" variant was tested. Different search phrasing might
  produce different results for domains that are partially indexed.
- The lab domain may have been crawled by Bing but not indexed for search
  results. A `site:ai-crawler-lab.kaistone.ai` search on Bing.com could
  confirm this, but was not tested.
- Only the free tier of Copilot was tested. Copilot Pro might have
  different tool availability.
- The test used temporary chat mode; different Copilot modes might behave
  differently.
- The lab domain has been live since June 2026 and may not have been
  crawled by Bing yet. Newer domains may face a longer indexing delay.

## Publication Thesis Verification

- Thesis: Asking Copilot to "search for" rather than "open" a URL does not
  change the no-hit outcome, but changes the failure mode from tool rejection
  to search-with-no-results.
- Source: Direct-origin server logs from the bounded test window, Copilot
  response JSON, comparison with 20 prior "open" variant tests.
- Method: Controlled-browser test with fresh temporary chat, bounded timestamp
  window, and independent server-side event correlation.
- Bias: Single lab domain that is not in Bing's index. Domains that are
  indexed might produce different results.
- Consensus: Consistent with Findings 015 (Copilot search-index-gated
  retrieval) and 014 (Gemini search-index dependency). The pattern across
  all three search-index-gated clients (Gemini, Copilot, Perplexity) is
  that content not in the respective search index is inaccessible.
- Invalidation: Test with a domain that is confirmed in Bing's index to
  verify whether Copilot can retrieve and summarize indexed content.
- Verdict: Thesis is supported. The "search for" variant failed differently
  but still produced no origin hits.
- Confidence: high for the recorded test; medium for generalization to
  all Copilot surfaces and to domains that are in Bing's index.
- Additional tests suggested: test "search for" variant with a domain that
  is in Bing's index to check whether Copilot returns cached/indexed
  content without origin hits; compare with the "open" variant on an indexed
  domain to see if retrieval behavior changes.

## Follow-up tasks

1. Test whether Copilot can summarize content from a domain that is confirmed
   to be in Bing's search index (e.g., a well-known public website).
2. Verify whether `site:ai-crawler-lab.kaistone.ai` returns any results on
   Bing.com directly.
3. If the lab domain is not indexed, consider submitting it to Bing Webmaster
   Tools for indexing and re-testing after it appears in the index.