How to Use IndexNow to Notify Search Engines About New URLs

Contents

Your sitemap is now submitted to Google Search Console and Bing Webmaster Tools. Both engines know your sitemap exists, and both will check it on their own schedule to find new and updated URLs.

That schedule is the problem.

Sitemaps work on a pull model. The search engines come to you when they decide to come. For a page you have just published and want indexed today, waiting for Google or Bing’s next crawl cycle can mean hours or days of delay. For a page you have just corrected, where the wrong version is already indexed, the delay is worse.

IndexNow is the protocol that solves this. Instead of waiting for the engines to come to you, you tell them the moment a URL changes. They visit within minutes rather than days. The protocol was pioneered by Bing, adopted by Yandex, Naver, Seznam, and several other engines, and is now honoured by some of the AI Search era crawlers as well.

This lesson covers what IndexNow is, which engines support it, how the protocol works, the three main ways to set it up (WordPress plugin, Cloudflare, manual), the key file you need to host, when pinging actually helps and when it can backfire, the AI Search era angle, and the common mistakes that catch people setting it up for the first time.

What IndexNow is and why it exists

IndexNow is a simple HTTP-based protocol that lets you notify participating search engines about URL changes on your site. When you publish a new page, update an existing one, or delete one, you ping the IndexNow endpoint with the URL, and the engines visit it shortly afterwards.

The protocol was created by Microsoft and Yandex in October 2021. Their reasoning was straightforward. The traditional crawl model wastes resources. Crawlers fetch pages they have already seen to check whether anything has changed, and they often miss new pages until their next routine crawl. A push-based system, where sites notify engines about changes directly, is more efficient for everyone. Engines crawl less wastefully, sites get indexed faster, and the web uses less bandwidth overall.

Microsoft and Yandex made the protocol open from day one. They published the specification at indexnow.org and invited other search engines to participate. Several did. The protocol is multi-engine by design now, which is part of what makes it more useful than a Bing-specific feature would have been.

Which search engines support IndexNow today

The list has grown since launch. As of 2026, the major engines that participate directly include:

Microsoft Bing, the original implementer.
Yandex, the second original implementer.
Naver, South Korea’s largest search engine.
Seznam, the largest search engine in the Czech Republic.

Several other engines and services either participate or honour the protocol indirectly through Bing’s index. DuckDuckGo, for example, draws its results from Bing and therefore benefits from IndexNow pings without being a listed partner itself. The current participant list is published at indexnow.org and is worth checking when you set up, because new partners get added periodically.

Google does not participate in IndexNow at the time of writing. Google has its own mechanisms (Search Console URL Inspection, sitemap fetches) and has not adopted the protocol. This is worth knowing because some people assume IndexNow is universal. It is not. For Google notifications, you still rely on the sitemap and URL Inspection.

But because Bing’s index feeds Copilot, ChatGPT search, DuckDuckGo, and several AI Search era assistants (covered in Lesson 8: How to Submit Your Website Sitemap to Bing Webmaster Tools), an IndexNow ping reaches a significant share of the wider search ecosystem in a single call, even without Google in the participant list.

How IndexNow works under the hood

The mechanic is simpler than it sounds. Three things have to be true.

You host a small text file on your domain that contains a unique key.
You send a ping to an IndexNow endpoint with the URL that changed, plus the same key.
The receiving engine fetches the key file from your domain to verify the ping came from someone with control over the site.

That is the entire protocol. Once an engine has verified the key file once, it caches that verification and trusts subsequent pings from the same site. You can ping individual URLs or batches at a time. The engines visit the URLs within minutes if their crawler is healthy.

A typical single-URL ping looks like an HTTP GET request to:

https://api.indexnow.org/indexnow?url=https://example.com/new-page&key=YOUR-KEY

For multiple URLs, you send a POST request with a JSON body containing the URL list. The api.indexnow.org endpoint forwards your ping to all participating engines, so you do not have to ping each one separately. You can also ping individual engines directly (Bing at bing.com/indexnow, Yandex at yandex.com/indexnow), but the central endpoint is the simpler default.

How to set up IndexNow on your website

You have three reasonable paths. The right one depends on what your site runs on and how much control you want over the implementation.

1. Using a WordPress plugin

If your site runs on WordPress, your SEO plugin probably already includes IndexNow support.

Yoast Premium added IndexNow in 2022 and pings automatically whenever you publish, update, or delete a post or page. Rank Math has a free IndexNow module that does the same. All in One SEO supports IndexNow in its plugin. There is also a standalone IndexNow plugin maintained by Microsoft that works alongside other SEO plugins if you do not want to handle IndexNow through your SEO suite.

The plugin path is the easiest. You install the plugin, enable IndexNow in its settings, paste in the key (or let the plugin generate one for you), and the plugin handles the key file, the pings, and the timing. No code, no manual work.

This is what I recommend for most WordPress sites unless you have a specific reason to manage IndexNow at the host level instead.

2. Using Cloudflare’s automatic integration

If your site sits behind Cloudflare, the platform offers a Crawler Hints feature that includes IndexNow. You enable it once in the Cloudflare dashboard under the Caching section, and Cloudflare handles pings automatically whenever your origin serves signals that something has changed.

The Cloudflare path is genuinely zero-configuration once enabled. You toggle it on and the rest happens at the edge. The trade-off is that Cloudflare decides when to ping, which is usually fine but means you have less visibility into what is being pinged and when.

For static sites built with Next.js, Astro, or similar, and hosted behind Cloudflare, this is the simplest path.

3. Setting it up manually

If you are running a custom stack (a static site without Cloudflare, a headless CMS with a custom backend, or something else), you can implement IndexNow manually.

The implementation has two parts. First, generate a key (any random string between 8 and 128 characters) and host it as a plain text file at the root of your domain. The file’s URL should be https://example.com/{key}.txt and its content should be the same key string. Second, send an HTTP request to the IndexNow endpoint whenever a URL changes on your site.

A simple example in PHP:

$url = 'https://example.com/new-page';
$key = 'abc123def456';
$endpoint = 'https://api.indexnow.org/indexnow?url=' . urlencode($url) . '&key=' . $key;
file_get_contents($endpoint);

Or in JavaScript with fetch:

const url = 'https://example.com/new-page';
const key = 'abc123def456';
await fetch(`https://api.indexnow.org/indexnow?url=${encodeURIComponent(url)}&key=${key}`);

For static sites with infrequent updates, you can run this as part of your build script. For dynamic sites, hook it into your publish-and-update workflow so the ping fires automatically when content changes.

The manual path gives you full control. It also gives you full responsibility for getting it right.

Understanding the IndexNow key file

The key file is the security mechanism that prevents anyone from pinging URLs on your behalf.

Here is what happens. When you send a ping with your key, the receiving engine fetches the key file from your domain at https://yourdomain.com/{key}.txt. The file’s content should be the same key string. If the engine successfully fetches the file and the content matches the key in the ping, the ping is accepted. If the file does not exist or the content does not match, the ping is rejected.

This works because only someone with control over your domain can host a file at your domain root. The key file is your proof of ownership for each ping.

Three things to keep in mind:

The key file must be served as plain text (Content-Type: text/plain).
The key in the file must match the key sent in the ping exactly.
The file must be accessible to crawlers, so it should not be blocked in robots.txt or sit behind authentication.

If a plugin handles your IndexNow setup, it usually places the key file in your site root automatically. If you are doing manual setup, you place the file yourself.

You can rotate keys later if you need to. The protocol supports multiple valid keys per domain, and rotating is as simple as generating a new key, hosting a new key file alongside the old one, and updating your ping code.

When to ping and when to hold back

IndexNow is useful, but it can backfire if used carelessly. The engines that receive pings track patterns over time. Sending too many pings, or pinging URLs that do not actually need re-crawling, signals low-quality usage. The engines can downgrade your pings’ priority in response.

Ping when:

You publish a new page that should be discovered quickly.
You update an existing page with substantive changes (new content, corrected facts, structural changes).
You delete a page that is currently indexed, so the engine knows to remove it.

Do not ping when:

You make trivial edits (typo fixes, minor wording changes, alt text tweaks).
You change something that does not affect what the engine should re-crawl (backend config that does not change the rendered page).
The same URL has been pinged recently and the change is minor.

The rule is simple. Ping when there is a real reason for the engine to revisit the URL. The protocol’s value lives in its signal-to-noise ratio. Pings that carry genuine new information get crawled fast. Pings that carry noise eventually get treated as noise.

IndexNow in the AI Search era

This is where IndexNow becomes more interesting than it was at launch.

Several AI Search era crawlers now honour IndexNow, either directly through their own infrastructure or indirectly through their dependence on Bing’s index. When you ping IndexNow about a new page, the page can show up in Copilot or ChatGPT search results faster than it would through traditional crawl alone. The same applies to DuckDuckGo and the other participating engines.

The pattern matters because AI assistants pull live results when their training data is insufficient. A page published five minutes ago can be cited in an AI answer if the underlying search index has picked it up. Sitemap discovery alone is too slow for that. IndexNow closes the gap.

This does not mean IndexNow guarantees AI citation. The same content quality, structured data, entity clarity, and topical authority signals still decide whether a page actually gets cited. But IndexNow ensures the page is at least available to be cited at the speed the AI Search era operates at.

For sites positioning around AI Search visibility specifically, IndexNow is no longer optional.

Common mistakes to avoid

Four issues come up regularly when people set up IndexNow for the first time.

1. Hosting the key file in the wrong location

The key file must sit at the root of your domain, accessible at the exact URL https://yourdomain.com/{key}.txt. Placing it in a subfolder (/wp-content/{key}.txt, for example) means the engines cannot find it and your pings get rejected. If a plugin manages this, it should handle the location automatically. If you are setting up manually, double-check the file is accessible at the root URL by visiting it in a browser.

2. Blocking the key file in robots.txt

A Disallow: / rule in robots.txt, or any rule that blocks the key file’s path, prevents the engines from verifying your pings. This is rare but happens when sites have aggressive robots.txt rules that lock down everything by default. If your robots.txt is restrictive, allow the specific key file URL explicitly.

3. Pinging URLs that do not exist

Pings for URLs that return 404 are treated as low-quality signals. If you delete a page, ping it so the engine knows to remove it from the index. But do not ping URLs that have never existed or that have been broken for a long time. The protocol expects URLs to actually exist, or to be returning the right status code for the engine to act on.

4. Spamming pings on minor changes

Pinging on every save, including minor edits and backend changes, dilutes the value of your pings over time. Configure your plugin (or your manual ping logic) to fire only on meaningful changes. Most SEO plugins handle this sensibly by default. Custom implementations sometimes ping too often and need their logic tightened.

Where this leaves us

Your sitemap is submitted to Google and Bing, and now backed up by IndexNow for fast notifications across several engines. Between the sitemap and IndexNow, the search engines that matter have the most efficient possible discovery of your URLs.

But none of that helps if the sitemap itself has errors. A submitted sitemap with broken XML, missing URLs, or invalid lastmod values is worse than no sitemap at all, because it sends the engines false signals about your site. The next lesson covers how to validate your sitemap properly, which tools to use, and what to check for before you rely on it.

Up next: How to Validate Your Website Sitemap →

This is Module 2: Lesson 9 of The Sitemap Series, a Technical SEO series on sitemaps from first principles, built for the AI Search era.