Meta Robots Tag

What Is A Meta Robots Tag?

A meta robots tag is an HTML element added to a web page’s <head> section that tells search engines and bots how to crawl that page.

Think of it like a piece of code that provides a set of rules and instructions for search engine bots to follow.

The meta robots tag is very similar to the Robots.txt file.

Its primary job is to tell crawlers like:

…and others whether they can crawl your website.

If the bot is programmed to follow the meta robots tag (which most major bots are), it should follow the rules you list.

This is what the Robots meta tag looks like:

<meta name=”robots” content=”index, follow”>

The tag tells search engines two things:

  1. Index this page and include it in the SERPs
  2. Follow the links on this page

This is a simple example of how the meta robots tag works.

It’s worth mentioning that you can put a number of different rules in your tag to instruct bots that visit your site.

More on that later.

learn how to increase your search traffic in 28 days

Why Are Meta Robots Tags Important?

Meta robots tags are important because they give you more control over how search engines and other crawlers interact with your website.

Let me explain:

You probably want most pages on your website to be crawled and indexed by Google. But there might be some sensitive pages that you don’t want Google to display in the organic search results.

This is where the meta robots tag comes in!

By adding a nonidex tag to the pages you don’t want to be indexed, you can keep specific pages out of the SERPs.

This gives you more control over what pages are displayed in the search results.

But that’s just one application…

Let’s say you run a larger site with tons of pages. Using a meta robots tag is great for managing crawl budget.

You can ensure search engines only crawl the important pages on your website. This way, they won’t spend time on pages you care less about.

Think of meta robots tags as your way of talking directly to search engines and other web crawlers. It’s one of the best ways to manage how (and where) your website appears online.

Want us to rank your site?

What Are The Robots Meta Tag Rules?

You can add a number of different rules to your meta robots tag to instruct search engines and crawlers in different ways.

Here is a complete list of all the meta robots tag rules and attributes:

  • All: This is a default value that allows crawlers to index and show pages without any restrictions.
  • Noindex: Prevents the page from being indexed and shown in search results.
  • Nofollow: Instructs search engines not to follow any of the links on the page.
  • None: This is essentially using both a noindex and nofollow tag.
  • Noarchive: Stops search engines from showing a cached link in SERPs.
  • Nositelinkssearchbox: Stops Google from displaying a sitelinks search box for the page in the SERPs.
  • Nosnippet: Prevents text snippets or video previews from being displayed in search results.
  • Indexifembedded: Allows indexing of some content embedded in other pages, even with a noindex rule.
  • Max-snippet: [number]: Limits the length of the text snippet in search results.
  • Max-image-preview: [setting]: Controls the maximum size of image previews in search results.
  • Max-video-preview: [number]: Limits the length of video snippets in search results.
  • Notranslate: Prevents Google from offering a translation of the page in search results.
  • Noimageindex: Stops images on the page from being indexed and shown in search results.
  • Unavailable_after: [date/time]: Sets an expiration date for the page in search results.

That’s a lot of rules, right?

That’s why robot meta tags are great for your site! You can instruct crawlers and search engines on what they can and can’t do with your content.

We have publish tons of SEO case studies

How To Add Robots Meta Tag In WordPress

The easiest way to add a robots meta tag in WordPress is to use a free SEO plugin like RankMath.

Here’s how to do it:

Install RankMath, navigate to the post/page you want to manage and click on the “RankMath” widget.

rank math seo plugin

Click on the “Advanced” tab and check the “No Index” option.

no index check box in advanced tab

RankMath will automatically add the Noindex rule to your meta robots tag for that page.

Keep in mind that this would STOP search engines like Google from crawling and indexing your page in the SERPs.

increase your organic search traffic in 28 days
Link Building

Link building you will be proud of.

Learn more
SEO Agency

We take full control of your traffic.

Learn more
Learn Portal

Free SEO tutorials to increase your traffic.

Learn more

Increase Your Search Traffic
In Just 28 Days…

CLICK HERE TO GET STARTED I’ll show you how step by step

Featured In: