Use of Robots.txt

[rajanippt]

Robots.txt is file is very important to every website. you can allow or disallow the page by the help of robots.txt.
http://www.arounddelhi.net/?page_id=203

[Derickalex]

Robot.txt is a text file which tells which search engines to crawl the website & which links search engine  should not crawl and index

[webseomall]

It is used by webmasters to inform Google that which page need to not index and crawl. by default, all pages are indexed and crawled by search engines. While using robots.txt fine, use it wisely otherwise search engine can harm other important pages by non indexing and non crawling.

use it as -

User-agent: *
Disallow: /
This will instruct all the robots (*) not to index any file from your site (/).

[Richard Touchet]

Robot.txt files are the one to by which you can control the contents of your website from getting indexed by the search engine crawlers. You can hide the information from the search engine bot which you don't want that search engine should crawl with the help of Robot.txt.

[jesusbond]

Robots.txt is a text file which we can use in ON page optimization to tell the search engines to don't crawl some particular part of site like some folder or url. Generally robots.txt is used for ecommerce site. We can check the robots.txt list from webmaster tool.

[seowebagency]

Think Robots.txt as a traffic signal,which instructs the traffic(robots here)the way to go.It is usually used by webmasters to tell the different search engine crawlers which pages not to crawl.
Thanks 

[anilkumartgsb]

Robots.txt is a very important document file for any website. This document contains the information of pages which are allowed to follow by the search engine bots and pages which are disallowed.

For using it you need to keep it in the root directory of your website folder.

[ComoliWesta]

Robote.txt is useful for hiding your personal content from google crawler. For example, if you want any webpage or content to not show to the world, then mention it to your robot.txt file. Then it will not be crawled by search engine.