Robots meta tag
A robots meta tag is a line of code in a web page's head section that tells search engine crawlers how to handle that page, such as whether to list it or show a preview. Google's spec says the same rules can be sent as an X-Robots-Tag header, which is how you set them on PDFs, images and other files.
Source: Google Search Central, Robots meta tag, data-nosnippet, and X-Robots-Tag specifications, checked October 29, 2026
What Robots meta tag means for your business
Noindex and nosnippet each have their own entry in this glossary. A few other rules are worth knowing. Nofollow asks Google not to follow the links on the page. Noimageindex asks Google not to index the page's images. Unavailable_after takes a page out of results after a date you pick, which suits a sale or an event page.
The tag can speak to every crawler or just one. Google reads the names googlebot and googlebot-news in that spot and ignores other names. When two rules clash, Google's spec says the stricter one wins, so a nosnippet beats a max-snippet on the same page. Google also lists rules it no longer reads, like noarchive, since its cached links are gone.
The X-Robots-Tag header does the same job from your server, as a line like X-Robots-Tag: noindex, so it works on files that have no head section to hold a tag. To see what Google actually got, the URL Inspection tool in Search Console shows the HTML Googlebot received. If a page or a file of yours isn't showing up in search the way you expect, a look at the rules your pages send to search engines is a quick check we can run.
Last reviewed October 29, 2026. All terms
How AI answers it today
On October 29, 2026, Google's AI Overview cited SEO blogs, Rank Math and Reddit, and Google's robots.txt guide instead of its meta tag spec. It and ChatGPT, which cited nothing, both said noarchive hides a cached copy, a rule Google says it no longer uses.
Our run through ChatGPT and Google's AI Overview, October 29, 2026. A sample, not what every buyer sees.
Related terms in crawlers and technical access
Find out where your brand stands.
Tell us what you're trying to figure out. We'll get back to you within one business day with a time to talk.