Which is better Robots.txt or Meta Robots Tag?

[Novel Web Solution]

Hello friends,

Which is better Robots.txt or Meta Robots Tag?

[SamJ]

Hello,

The Meta tag option tells bots not to index individual files, whereas Robots.txt can be used to restrict access to entire directories and also show them the sitemap location.

I suggest you to write a robots.txt file.

You can also combine tags & robots.txt for your needs.

More informations here: https://www.weareinfront.com/learn-article/robots_txt/

[luffy268]

If you are trying to improve SEO it's ok to use both. You may want to and noindex with meta for the RSS feed, all back-end pages, comments and other pages without important content.

[EnKonversations]

Robots.txt works fine, but keep in mind settings within robots.txt are just guidelines for search engines. They can choose to ignore them, so don't use this as a way to secure/hide web pages.

robots.txt file is a publicly available file. Anyone can see what sections of your server you don't want robots to use.
whereas meta tag ,you have to provide allow!/disallow to read the content of your web page !!bu default allow exists

[damponting44]

The purpose of a robots.txt file, also known as the robots exclusion protocol, is to give webmasters control over what pages robots (commonly called spiders) can crawl and index on their site. A typical robots.txt file, placed on your site's server, should include your sitemap's URL and any other parameters you wish to put in place.