Seo Forum

Search Engine Optimization => SEO Basics => Topic started by: bababangali on 12-26-2016, 05:32:56

Title: Which is better Robots.txt or Meta Robots Tag?
Post by: bababangali on 12-26-2016, 05:32:56
Hello friends,

Which is better Robots.txt or Meta Robots Tag?
Title: Re: Which is better Robots.txt or Meta Robots Tag?
Post by: SamJ on 12-26-2016, 07:14:48
Hello,

The Meta tag option tells bots not to index individual files, whereas Robots.txt can be used to restrict access to entire directories and also show them the sitemap location.

I suggest you to write a robots.txt file.

You can also combine tags & robots.txt for your needs.

More informations here: https://www.weareinfront.com/learn-article/robots_txt/
Title: Re: Which is better Robots.txt or Meta Robots Tag?
Post by: luffy268 on 12-27-2016, 01:57:13
If you are trying to improve SEO it's ok to use both. You may want to and noindex with meta for the RSS feed, all back-end pages, comments and other pages without important content.
Title: Re: Which is better Robots.txt or Meta Robots Tag?
Post by: EnKonversations on 12-27-2016, 02:59:29
Robots.txt works fine, but keep in mind settings within robots.txt are just guidelines for search engines. They can choose to ignore them, so don't use this as a way to secure/hide web pages.

robots.txt file is a publicly available file. Anyone can see what sections of your server you don't want robots to use.
whereas meta tag ,you have to provide allow!/disallow to read the content of your web page !!bu default allow exists
Title: Re: Which is better Robots.txt or Meta Robots Tag?
Post by: damponting44 on 12-27-2016, 03:47:18
The purpose of a robots.txt file, also known as the robots exclusion protocol, is to give webmasters control over what pages robots (commonly called spiders) can crawl and index on their site. A typical robots.txt file, placed on your site’s server, should include your sitemap’s URL and any other parameters you wish to put in place.