# Finding 026: ChatGPT robots-crawl-delay directive prompt produced a bounded no-hit

## Date

2026-06-29

## Status

Published

## Summary

ChatGPT was tested with the directive compliance prompt
`directive-chatgpt-20260628-001-robots-crawl-delay` in a fresh ChatGPT chat
created through the dedicated OpenClaw browser profile. The prompt supplied
the exact robots-crawl-delay fixture URL and asked ChatGPT to open it only if
browsing was supported, while respecting robots.txt and page-level robots
directives.

ChatGPT returned `fetched:false`, `policy_observed:true`,
`directive_marker:null`, and a limitation saying the target URL could not be
fetched in its current browsing environment, so it could not verify robots.txt
or page-level robots directives without inference. No matching direct-origin
event for the exact attempt id, `/lab/directives/robots-crawl-delay`, or
`/robots.txt` appeared in the bounded prompt window. The result is a recorded
no-fetch/no-hit outcome, not evidence that ChatGPT opened the target page or
independently checked the crawl-delay directive.

## Method

The run used the prepared controlled-browser artifact:

- Browser task:
  `research/manual-client-runs/browser-tasks/directive-chatgpt-20260628-001-robots-crawl-delay.browser-task.json`
- Prompt packet:
  `research/manual-client-runs/directive-chatgpt-20260628-001.prompts.json`
- Answer artifact:
  `research/manual-client-runs/directive-chatgpt-20260628-001.answers.json`
- Response file:
  `research/manual-client-runs/browser-tasks/responses/directive-chatgpt-20260628-001-robots-crawl-delay.response.json`

A new ChatGPT tab was opened at `https://chatgpt.com/` in the
OpenClaw-controlled Chrome profile. The page showed the fresh empty composer
before the prompt was submitted; no previous thread was reused. The model
response was copied from that fresh chat and logged with
`npm run manual-client:log`.

## Result

| Field | Value |
|---|---|
| Attempt ID | directive-chatgpt-20260628-001-robots-crawl-delay |
| Client | ChatGPT (chatgpt-web) |
| Fixture | robots-crawl-delay |
| Directive under test | robots_crawl_delay |
| Fetched | false |
| Policy observed | true |
| Directive marker | null |
| Direct-origin events | 0 |
| Bounded window | 2026-06-29T08:17:50.687Z - 2026-06-29T08:18:54.955Z |
| Confirmed hit from prompt | false |
| Confirmation status | registered_no_hit |

## Raw response

```json
{
  "attempt_id": "directive-chatgpt-20260628-001-robots-crawl-delay",
  "fixture": "robots-crawl-delay",
  "fetched": false,
  "policy_observed": true,
  "directive_marker": null,
  "followed_control_links": false,
  "evidence_quote": "",
  "limitations": "The target URL could not be fetched in my current browsing environment, so I cannot verify robots.txt or page-level robots directives without inferring from the URL.",
  "confidence": "low"
}
```

## Origin evidence

Local event stores and `/api/hits` text filters were searched for the exact
attempt id, `/lab/directives/robots-crawl-delay`, and `/robots.txt` across
`2026-06-29T08:17:50.687Z` to `2026-06-29T08:18:54.955Z`. There were no
matching direct-origin events.

## Interpretation

ChatGPT did not fetch the supplied robots-crawl-delay fixture in this run.
This completes the refreshed ChatGPT directive-compliance packet: all six
fixtures produced bounded no-hit outcomes in fresh ChatGPT chats.

Because the prompt included `fixture: robots-crawl-delay`,
`directive_under_test: robots_crawl_delay`, and
`expected_policy_signal: fetchable_with_crawl_delay_signal`, the model could
infer the test context from prompt fields. Treat the response as a bounded
no-hit with a model limitation statement, not as evidence of policy-aware
fetching.

## Limitations

- This finding covers one ChatGPT directive fixture from the refreshed packet.
- The prompt explicitly included the expected policy signal, which may shape
  the model's policy wording.
- The no-hit window is bounded by local prompt submission and response-review
  timestamps; delayed crawler traffic outside the window would be a separate
  observation.
- The response reports that the current browsing environment could not fetch
  the target URL, so this is a client/tool availability result for this turn
  as much as a directive fixture result.

## Publication Thesis Verification

- Thesis: ChatGPT did not fetch the robots-crawl-delay directive fixture
  during the recorded prompt window; its answer was a no-fetch model response
  without direct-origin retrieval evidence.
- Source: Fresh ChatGPT conversation response, response artifact, generated
  answer packet, and local event review.
- Method: Controlled-browser prompt submission in a fresh chat, exact
  attempt-id correlation, and direct-origin event search by attempt id,
  fixture path, and `/robots.txt`.
- Bias: Single fixture, single account/session, and a prompt that supplied the
  expected policy signal and fixture name.
- Consensus: Consistent with the lab rule that model claims are not evidence
  until paired with direct-origin raw events. It also aligns with Findings 019,
  022, 023, 024, and 025 for the refreshed ChatGPT directive packet.
- Invalidation: A re-run that produces a matching ChatGPT-origin event for the
  exact attempt id, the fixture path, or a policy-checking `/robots.txt` hit
  during the prompt window would show direct retrieval for a comparable
  robots-crawl-delay directive case.
- Verdict: Supported for this run. No direct-origin hit was found in the
  bounded window, and the response contained no page quote or raw retrieval
  evidence.
- Additional tests suggested: compare the completed ChatGPT directive no-hit
  packet against Claude's remaining directive fixtures and add obscured-slug
  directive fixtures that omit expected policy fields.

## Next steps

- Run the remaining Claude directive fixtures in fresh Claude chats.
- Compare the completed ChatGPT directive packet against Gemini and Claude
  directive outcomes.
- Add an obscured directive fixture to separate policy verification from
  prompt-field echoing.