A sitemap lists the URLs of a site in a structured file intended for search engines. Knowing that it exists is not enough: the real question is to measure what changes concretely between a site that declares a clean sitemap and a site that does without it, particularly regarding indexing speed and page coverage in Google results.
XML Sitemap, HTML Sitemap, and Crawl Budget: What Each Format Modifies
| Criterion | XML Sitemap | HTML Sitemap | No Sitemap |
|---|---|---|---|
| Main Recipient | Crawlers (Googlebot, Bingbot) | Human visitors | – |
| Format | .xml file at the root of the site | Standard web page with clickable links | – |
| Impact on Indexing | Signals all URLs to explore, with last modified date | No direct effect on crawlers | Crawlers discover pages only through internal linking and external links |
| Impact on User Navigation | None (file unreadable for a human) | Consultable sitemap, reduces the number of clicks | Navigation limited to the menu and internal linking |
| Effect on Crawl Budget | Guides crawlers towards priority pages | Negligible | Crawlers may waste crawl budget on secondary pages |
The XML sitemap affects indexing, while the HTML sitemap impacts the navigation experience. Both formats address distinct needs, and combining them covers both aspects: technical SEO and usability.
To observe how these two formats coexist on a real site, The Web Brains’ sitemap page illustrates a navigation-oriented layout that complements the XML file declared to the engines.
Declaring the Sitemap After the End of Google Ping: robots.txt and Search Console
Since the end of 2023, Google no longer supports sitemap ping endpoints. Sending an HTTP request to Google to signal an update of a sitemap no longer produces any effect. This removal has rendered several plugins and automated scripts that relied on this mechanism obsolete.
The only two methods recognized by Google to declare a sitemap are now:
- Submission via the Sitemaps report in Google Search Console, which also allows checking for reading errors and the number of indexed URLs compared to the number of submitted URLs.
- The Sitemap: directive in the robots.txt file, placed at the root of the site. This line indicates to crawlers the exact location of the XML file without going through an interface.
A site that continues to ping Google without using one of these two methods risks having its new pages remain invisible for longer.
Large-Scale Recrawl: The Sitemap as a Lever for Large Sites
The Google Search Central documentation updated at the end of 2025 specifies a point that most guides overlook. For sites with a large volume of URLs, submitting a clean sitemap is the preferred method for requesting a large-scale recrawl. The URL inspection tool in Search Console works on a case-by-case basis, making it impractical when several hundred pages have been modified simultaneously.
A “clean” sitemap here means a file that contains only canonical URLs, without 404 error pages, without redirects, and without URLs blocked by robots.txt. Each unnecessary URL in the sitemap dilutes the signal sent to crawlers and consumes crawl budget without return.
Lastmod Tags and Actual URL Priority
The lastmod tag in the XML sitemap indicates the last modified date of a page. Google’s crawlers use this information to decide which pages to recrawl as a priority. However, if the date does not reflect a real content change, Googlebot eventually ignores the signal.
The priority tag, often set by default in SEO plugins, has no effect on Google’s behavior. It has not been taken into account for several years. Manually prioritizing it in the XML file does not change anything in the crawl.
Sitemap Errors That Hinder SEO Instead of Improving It
A poorly configured sitemap can produce the opposite effect of what is sought. Three technical errors frequently appear in SEO audits.
- Including URLs that return a 404 or 410 HTTP code. Crawlers crawl them, find the error, and the coverage report in Search Console accumulates warnings that obscure real issues.
- Listing non-canonical URLs (pages with sorting parameters, versions with and without final slash, HTTP and HTTPS versions). The sitemap must accurately reflect the canonical version of each page.
- Never updating the file after a redesign or content removal. An outdated sitemap sends crawlers to pages that no longer exist, wasting crawl budget and delaying the indexing of new pages.
Regularly checking the Sitemaps report in Google Search Console allows for detecting these inconsistencies. The report displays the number of submitted URLs, the number of indexed URLs, and reports reading errors in the file.
HTML Sitemap and User Navigation: An Underestimated Complement
The HTML sitemap does not change the behavior of indexing crawlers. Its role is different: it provides visitors with a tree view of the entire site, accessible in one click from the footer.
On a site with several dozen pages, this overview reduces the number of clicks needed to reach a deep page. For e-commerce sites or editorial portals, the HTML sitemap becomes a shortcut to categories or articles that the main menu does not highlight.
The indirect benefit for SEO exists: each link in the HTML sitemap strengthens internal linking and distributes PageRank to deep pages. It is not a substitute for the XML sitemap, but a complement that serves both the user and the site’s link structure.
The XML file declares the URLs to the crawlers, the HTML sitemap makes them accessible to visitors, and the robots.txt points to the XML file. These three elements form a coherent technical system. Removing one of them leaves a blind spot, either for search engines or for users looking for a page without going through the menu.
