Why the engine keeps using the old URL
AI engines do not cite from a live crawl of the web on every question. They cite from an index they already hold, plus a small number of pages fetched at answer time. A URL that was retrievable when it was indexed stays a candidate until the index is refreshed for that topic. If the page is now a 404, the engine may still cite it, because the index entry, not the page, is what was retrieved.
Search Engine Land's August 2026 description of ChatGPT's retrieval stack makes the mechanism concrete: there is an index, a cache of previously opened pages, and live fetches, and a page that was opened and cached is cited far more often than one that was only retrieved. That is why an old page with a cache entry can outlive a new page nobody has opened yet.
The two cases, and they need different fixes
| Case | What the reader sees | What actually fixes it |
|---|---|---|
| Dead URL (404, removed, moved without redirect) | A source link that goes nowhere | A live page at the same topic the engine can re-retrieve, ideally at the same URL or a 301 from it |
| Outdated URL (live, but the content is stale) | A source link to a page with old numbers or an old product name | Update the page in place, keeping the URL, so the trusted index entry inherits the new content |
The second case is common and easier. The engine already trusts the URL. Do not publish a new page beside it and hope the engine switches; edit the page it already cites. Keep the URL, keep the H1's job, and change the body.
What to do, in order
- Confirm the citation. Run the prompt three times on the engine in question and record the cited URL each time. If the old URL appears on all three, it is the index, not a one-off.
- Check what the old URL returns now. A 404, a soft redirect to the homepage, or a live but stale page each map to a row in the table above.
- If it is dead and the topic still matters, put a live page back at that URL, or 301 the old URL to the live equivalent. A redirect to the homepage does not count; the engine wants the same job answered.
- If it is live and stale, edit in place. Fix the numbers, the product names and the first paragraph. Leave the URL and the H1's job alone.
- If a third-party page is the one linking to your dead URL, ask that site to update the link. Engines often reach your page through theirs.
- Re-run the same prompt weekly. The swap shows up as a change in the cited URL column, usually within one to three crawl cycles. Record the date it changed.
What does not force the swap
- Putting the year in the title. "Guide 2026" tells a human it is new. It does not change whether the engine's index has the page. David Konitzny's September 2026 analysis of ChatGPT's search behaviour found that a freshness preference influences retrieval without strictly constraining it; a stale page can still be retrieved, and a date in the H1 does not make a page fresh.
- An llms.txt file. It is a courtesy index for crawlers, not an instruction to drop a cached URL. Keep it accurate; do not expect it to fix this.
- Publishing a near-duplicate at a new URL. Now the engine has two candidates, one of which it already trusts. You have made the old one harder to displace.
- Blocking the old URL in robots.txt. The engine cannot re-fetch it, so it cannot learn it is gone. Serve a 404 or 410 and let it be re-crawled.
A first-party observation of how long a citation persists
In a registered trial on this site, a Perplexity citation for one fixed prompt was absent on the first six observations, then present on nine of the next ten after it first appeared, with one later valid miss. Two lessons for stale URLs follow. First, once an engine has a page in its index it tends to keep citing it, which is why the old page is so sticky. Second, the swap to a new page is visible only if you run the same prompt repeatedly and log the URL. The result is engine-specific and observational; it says nothing about ChatGPT or Gemini, where the same prompt was never cited in the window.