We have been learning about meta tags the past few days. Some of these tags are important to optimize your SEO wordpress blog (meta description tags) and some are not as important (meta keyword tags).
So today we will be learning about meta robots tags. Meta robots tags tell the search-engine spider whether or not to index a Web page.
We will use steps from Kristopher B. Jones’s book Search Engine Optimization: Your Visual Blueprint for Effective Internet Marketing 2nd ed.
Meta Robots Tag:
Search-engine robots (like Googlebot above) need to be controlled. The meta robots tag allows you to tell the search engines what parts of your SEO blog to visit or skip.
Do NOT use this tag if you want your Web page to be indexed. Yes, you can use meta robots tag to tell the search engines to index your Web page, but they do it by default.
The meta robots tag is located in the header of an HTML document and it looks like this:
- <meta name=”robots” content=”index,follow”> or
- <meta name=”robots” content=”noindex,follow”> or
- <meta name=”robots” content=index,nofollow”> or
- <meta name=”robots” content=noindex,nofollow”>
Index – tells the spiders to index the page.
Noindex – tells the spiders to not index the page.
Follow – tells the spiders to follow the links on the page.
Nofollow – tells the spiders to not follow the links on the page.
You may also use “all” as a replacement for “index,follow” and you use “none” as a replacement for “noindex,nofollow“.
There are special meta robots tags for both Google and Bing search engine spiders. You can control each spider by attributing their name to the robots tag. So instead of using the name “robots,” you can use “Googlebot” or “MSNBot” to control each search engine spider.
So when should you use meta robots tags?
Use these tags when you want to block search engines from indexing content such as products or perhaps information that a user has to pay before accessing (like ESPN’s Insider content).