Which is better Robots.txt or Meta Robots Tag?

Started by Ruhikhan862, 08-29-2016, 00:54:32

Previous topic - Next topic

Ruhikhan862Topic starter

Hello friends,

I Want TO know that Which is better Robots.txt or Meta Robots Tag?


AmarInfotech

Robots meta tag gives option that crawl or not crawl specific page.It also informs about do-follow and no-follow links on the web page.whereas robots.txt is a list of a directories which mainly used for restriction about crawling of a specific web pages.so both used for different purposes and not better than each other.
Amar Infotech is a Laravel development company provides efficient Laravel development services at affordable rates.
  •  


RH-Calvin

Robots.txt is a text file that lists webpages which contain instructions for search engines robots. The file lists webpages that are allowed and disallowed from search engine crawling.

TomClarke

The meta tags option tells bots not to index individual files, whereas robots.txt can be used to restrict access to entire directories.
Sure, use A meta tag if you have the odd page in indexed folders that you want skipping, but generally, I would recommend you most of your non-indexed content in one or more folders and use robots.txt to skip the lot.
No, there isn't a problem in using both - if there is a  clash, in general terms, a deny will overrule an allow.

wellliving

i think robots.txt is better than meta robot. in robots.txt search engines get all information in a file. so we should go with robots.txt.