# Finding 062: ChatGPT neutral retrieval changed from one no-hit to three confirmed hits ## Date 2026-07-01 ## Status Published ## Summary Findings 052, 059, 060, and 061 compare four ChatGPT runs against the same `/lab/reading/hidden-structured-data-conflict` fixture under neutral site-owner wording. The first neutral run, p22, used the default full correlation URL and returned `fetched:false`, `pages_opened:0`, with no exact target-page or fixture-path origin event in the bounded prompt window. The next three neutral stability reruns, p23, p24, and p25, all returned `fetched:true`, `pages_opened:1`, and each had a direct-origin `ChatGPT-User/1.0` target-page hit inside its bounded prompt window. The content result stayed stable when retrieval succeeded: ChatGPT reported the visible page token `VISIBLE-PLUM-47` and did not report the fixture's meta-description token `META-AMBER-16` or JSON-LD token `SCHEMA-INDIGO-82`. Because p24 also used the full correlation URL and still fetched the page, the p22 no-hit cannot be explained by the long tracking-style URL alone. ## What does this mean? For site owners and researchers, a single failed ChatGPT page-open attempt should not be treated as proof that the page, URL shape, or neutral prompt wording is consistently inaccessible. In this series, the same public fixture shifted from one clean no-hit to three confirmed hits later the same day, while the successful answers still reflected only visible page text. The practical takeaway is to separate "did the AI client fetch the page?" from "which page fields did it use?", and to rerun neutral prompts before turning one no-hit into a broad retrieval claim. ## Method This comparison reviews the published ChatGPT neutral hidden-signal and retrieval-stability findings: - p22: Finding 052, `manual-client-chatgpt-20260625-001-p22`. - p23: Finding 059, `manual-client-chatgpt-neutral-stability-20260701-001-p23`. - p24: Finding 060, `manual-client-chatgpt-neutral-stability-20260701-001-p24`. - p25: Finding 061, `manual-client-chatgpt-neutral-stability-20260701-001-p25`. - Response artifacts: `research/manual-client-runs/browser-tasks/responses/manual-client-chatgpt-20260625-001-p22.response.json` and `research/manual-client-runs/browser-tasks/responses/manual-client-chatgpt-neutral-stability-20260701-001-p23.response.json` through `research/manual-client-runs/browser-tasks/responses/manual-client-chatgpt-neutral-stability-20260701-001-p25.response.json`. - Prompt packets: `research/manual-client-runs/manual-client-chatgpt-20260625-001.prompts.json` and `research/manual-client-runs/manual-client-chatgpt-neutral-stability-20260701-001.prompts.json`. Each run used a fresh ChatGPT chat or native Temporary Chat, logged the copied answer with `npm run manual-client:log`, and reviewed direct-origin evidence by attempt id, prompt code, raw event ids, and bounded timestamp windows. Operator preflight requests are treated as fixture-readiness checks, not prompt-caused AI-client behavior. ## Comparison Matrix | Prompt | URL shape | Task framing | ChatGPT answer | Direct-origin evidence | Field implication | |---|---|---|---|---|---| | p22 | Full correlation URL | Neutral page-quality summary; asked to include directly observed code-like tokens and source area when exposed. | `fetched:false`, `pages_opened:0`; said browsing did not retrieve the page. | Clean bounded no-hit, no raw event ids. | No content conclusion; the target was not fetched. | | p23 | Short id-only URL | Concise site-owner page-quality note. | `fetched:true`, `pages_opened:1`; reported `VISIBLE-PLUM-47`. | Raw event `mr21pcqo-4w1raylc` from `ChatGPT-User/1.0`. | Visible-only answer; metadata and JSON-LD not directly exposed. | | p24 | Full correlation URL | Readability and purpose summary for a site owner. | `fetched:true`, `pages_opened:1`; reported visible text including `VISIBLE-PLUM-47`. | Raw event `mr23orwl-nnze1f0e` from `ChatGPT-User/1.0`. | Visible-only answer; rules out long URL as sole p22 explanation. | | p25 | Short id-only URL | Neutral support-answer wording. | `fetched:true`, `pages_opened:1`; reported `VISIBLE-PLUM-47`. | Raw event `mr25un5s-guspwbjl` from `ChatGPT-User/1.0`. | Visible-only answer; reinforces retrieval restoration. | ## Interpretation The p22-p25 series turns ChatGPT's p22 result into a retrieval-stability finding rather than a hidden-field visibility finding. In p22, ChatGPT did not retrieve the target, so its answer could not reveal whether visible HTML, metadata, or JSON-LD would have been available. In p23-p25, retrieval succeeded, and every copied answer stayed scoped to visible page content. The p24 run is the key control inside this small series. It kept the full correlation URL shape used by p22 but still produced a confirmed target-page hit, so the p22 no-hit cannot be attributed to the long URL alone. The remaining differences include source prompt wording, exact product routing state, account/session state, and ordinary timing variability in ChatGPT's retrieval path. The successful runs do not prove that ChatGPT can never inspect hidden fields. They show that in these three confirmed target-page hits, the exposed answer view reported `VISIBLE-PLUM-47` and did not expose `META-AMBER-16` or `SCHEMA-INDIGO-82`. That matches the earlier p21 visible-only ChatGPT result and keeps the content conclusion narrower than the retrieval conclusion. ## Limitations - This comparison covers four ChatGPT attempts from one logged-in account context and one day of product behavior. - The runs are not a perfect A/B series. Prompt wording, URL shape, chat mode details, and product routing state differ across attempts. - The prompt supplied the exact target URL, so this is direct URL opening, not independent discovery. - The copied answers report what the ChatGPT client exposed in its response; they do not prove what every underlying retrieval component could or could not inspect. - Origin review used bounded prompt windows. Delayed fetches outside those windows would require separate review before changing these conclusions. ## Publication Thesis Verification - Thesis: ChatGPT's neutral hidden-signal retrieval behavior changed from one clean p22 no-hit to three later confirmed target-page hits in p23-p25, while the confirmed-hit answers reported visible content only. - Source: Findings 052, 059, 060, and 061; their response files; logged answer packets; browser-task artifacts; and direct-origin raw events `mr21pcqo-4w1raylc`, `mr23orwl-nnze1f0e`, and `mr25un5s-guspwbjl`. - Method: Compare copied model answers with exact attempt ids, prompt codes, confirmation statuses, raw event ids, URL shapes, task framing, and bounded timestamp windows, while excluding operator preflight requests from prompt-caused evidence. - Bias: The result is single-client, single-account, and short-window. ChatGPT retrieval behavior may vary by tier, model selector, region, browsing tool state, target URL, prompt wording, and time. - Consensus: Consistent with the individual thesis-verification sections for Findings 052, 059, 060, and 061. The p23-p25 visible-only answers also align with Finding 049's earlier ChatGPT p21 visible-only confirmed hit. - Invalidation: A corrected event review that removes the p23-p25 raw events from their prompt windows, a response artifact mismatch, a fixture-serving error, or same-condition reruns showing a stable p22-style no-hit would weaken the comparison. - Verdict: Supported for this four-run series. The comparison supports retrieval variability plus visible-only copied answers when retrieval succeeded; it does not identify one definitive cause for the p22 no-hit. - Additional tests suggested: run same-condition repeats that hold URL shape and prompt wording constant across several fresh Temporary Chats, then compare target-hit rates before adding new hidden-field claims. ## Next steps - Prepare a small same-condition ChatGPT repeat packet that holds either the p22 full-query wording or the p24 full-query wording constant across several fresh Temporary Chats. - Keep future summaries split into retrieval status, origin evidence, and answer-visible field exposure.