Bingbot, IndexNow and Copilot: The Technical Connection
Understand how crawl discovery, IndexNow notifications and Bing's index support source freshness for Copilot without guaranteeing selection.
Bingbot, IndexNow and Copilot sit at different stages of the discovery chain. Bingbot crawls pages, IndexNow notifies participating engines that URLs changed, and Copilot may retrieve indexed web content when generating answers. A successful IndexNow submission means the notification was received—not that the page was indexed, selected or cited.
The discovery chain
Think in four stages:
- notification: sitemap, internal link or IndexNow signal;
- crawl: Bingbot or another agent requests the URL;
- index and retrieval: the engine processes and may retrieve the page;
- AI selection: a supported experience may cite it in an answer.
Each stage can succeed while the next fails.
Configure IndexNow
Generate a key between 8 and 128 allowed characters, host the key file as documented, and submit changed URLs through an IndexNow endpoint.
For one URL, use the GET format. For batches, POST JSON containing the host, key and URL list. Submit only URLs belonging to the verified host.
Interpret responses correctly:
- 200: URL submitted successfully;
- 202: received, key validation pending;
- 400: invalid request;
- 403: key problem;
- 422: host or schema mismatch;
- 429: too many requests.
Submit real changes
Trigger IndexNow when a canonical URL is added, materially updated or removed. Do not submit every URL on every deploy. Maintain a queue, deduplicate and retry transient failures with backoff.
IndexNow says participating search engines share submitted URLs with other participants. It does not promise indexing.
Keep sitemaps and links
IndexNow complements rather than replaces:
- XML sitemaps with canonical URLs and meaningful last-modified values;
- crawlable internal links;
- correct status codes;
- redirects for moved content;
- stable canonical tags.
These mechanisms provide structure and recovery when a notification is missed.
Verify Bingbot access
Check robots.txt, CDN rules, origin logs and response content. Ensure Bingbot receives the main page text, not an authentication wall or challenge. Avoid blocking required CSS or JavaScript where rendering matters.
Inspect important URLs in Bing Webmaster Tools and compare crawl or index status with submission logs.
Connect to AI Performance
Bing’s AI Performance report can show citations, cited pages, sampled grounding queries and trends across supported Microsoft AI experiences. Use it to observe the later selection stage.
A sequence such as “IndexNow submitted → crawled → indexed → cited” is useful for diagnosis, but it does not prove IndexNow caused the citation. Many other retrieval and selection signals intervene.
A technical monitoring table
Store per URL:
- canonical URL and template;
- last meaningful change;
- IndexNow submission and response;
- latest Bingbot request and status;
- Bing index state;
- AI citation activity;
- incident and owner.
This turns a vague Copilot complaint into a stage-specific investigation.
Common failure modes
- key file unavailable or on the wrong host;
- batch includes URLs from another domain;
- parameter URLs submitted instead of canonicals;
- successful notification followed by robots block;
- origin returns an empty client-side shell;
- updated page retains a stale canonical;
- team assumes no citation means no indexing.
Frequently asked questions
Does IndexNow replace Bingbot?
No. It sends change notifications; search engines still decide whether and how to crawl and process URLs.
Should deleted URLs be submitted?
Yes, the protocol supports notifications for added, updated and deleted URLs. The URL should also return the appropriate status or redirect.
Does IndexNow guarantee faster Copilot visibility?
No. It can improve change discovery, but selection in AI answers is a separate stage.
