what robot txt?

Author Topic: what robot txt?  (Read 3503 times)

Offline europeanTopic starter

  • Trade Count: (0)
  • Semi-Newbie
  • *
  • Thank You 0
  • Posts: 13
  • Karma: 0
  • Gender: Male
  • Best immigration consultant in Ludhiana
    • Best immigration consultant in Ludhiana
what robot txt?
« on: 10-05-2015, 23:33:51 »
what robot txt? how to improve you seo with robot txt? why its is important for SEO?


Offline roger710

  • Trade Count: (0)
  • Semi-Newbie
  • *
  • Thank You 0
  • Posts: 37
  • Karma: 0
  • Gender: Male
    • Eminnet Infoweb
Re: what robot txt?
« Reply #1 on: 10-06-2015, 00:25:15 »
Robots.txt is common name of a text file that is uploaded to a Web site's root directory and linked in the html code of the Web site. The robots.txt file is used to provide instructions about the Web site to Web robots and spiders.

Offline josephinek

  • Trade Count: (0)
  • Semi-Newbie
  • *
  • Thank You 0
  • Posts: 18
  • Karma: 0
  • Gender: Female
    • Web development company
Re: what robot txt?
« Reply #2 on: 10-06-2015, 03:01:52 »
Robot.txt:
It is an HTML tag placed on the source of a web page which redirects search engine spiders which files to crawl on or not.

Offline jane

  • Trade Count: (0)
  • Jr. Member
  • **
  • Thank You 1
  • Posts: 52
  • Karma: 0
  • Gender: Female
    • Hair Transplant cost in India
Re: what robot txt?
« Reply #3 on: 10-06-2015, 04:33:14 »
Robot.txt file is directive file to search engine that directs it not to crawl information of a specific webpage. The file is used when you do not want your webpage indexed for some information. Webmaster use this file for keeping their some data secret. The file  is hence useful for prohibiting search engines to make a webpage's information public.

Offline RH-Calvin

  • Trade Count: (0)
  • Hero Member
  • *****
  • Thank You 52
  • Posts: 1018
  • Karma: -1
  • Gender: Male
    • Cheap VPS Hosting
Re: what robot txt?
« Reply #4 on: 10-06-2015, 05:25:42 »
Robots.txt is a text file defined on your website that contains instructions to crawlers. It lists webpages that needs to be crawled and disallowed by search engine crawlers.


Offline danielnash

  • Trade Count: (0)
  • Jr. Member
  • **
  • Thank You 6
  • Posts: 84
  • Karma: 0
  • Gender: Male
    • WordPress Development
Re: what robot txt?
« Reply #5 on: 10-07-2015, 05:11:35 »
Robots.txt is a text file that has some code about allow and disallow. It tells to search engine which page or directory should be crawl and or not. It saves time of search engine.

Offline jaysh4922

  • Trade Count: (0)
  • Sr. Member
  • ****
  • Thank You 10
  • Posts: 341
  • Karma: 0
Re: what robot txt?
« Reply #6 on: 10-09-2015, 00:09:58 »
Robot.txt is an on-page SEO technique and it is basically used to allow for the web robots also known as the web wanderers, crawlers or spiders. It is a program that traverses the website automatically and this helps the popular search engine like Google to index the website and its content. !-!

Offline samarkajinkya

  • Trade Count: (0)
  • Full Member
  • ***
  • Thank You 1
  • Posts: 133
  • Karma: 0
  • Gender: Male
    • Creative Digital Marketing Agency Mumbai
Re: what robot txt?
« Reply #7 on: 10-09-2015, 01:27:51 »
Hi,
A robots.txt file is a file you put on your site to tell search robots which pages you would like them not to visit. It indicates those parts of your site, you don’t want accessed by search engine crawlers.

Offline petersonangela

  • Trade Count: (0)
  • Jr. Member
  • **
  • Thank You 3
  • Posts: 83
  • Karma: 0
    • PVC & Commercial Vinyl Flooring; Artificial Leather Manufacturer & Producer
Re: what robot txt?
« Reply #8 on: 10-09-2015, 03:59:22 »
The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a standard used by websites to communicate with web crawlers and other web robots. reference website:https://en.wikipedia.org/wiki/Robots_exclusion_standard


Offline Akash Varma

  • Trade Count: (0)
  • Jr. Member
  • **
  • Thank You 0
  • Posts: 81
  • Karma: 0
  • Gender: Male
    • Packers and Movers in Pune
Re: what robot txt?
« Reply #9 on: 10-27-2015, 23:51:11 »
robots.txt file is used to regulate search engine crawlers. Its basic purpose is to give information to search engines about the certain path on the website which is blocked. If this file is not used then everything is allowed by default.
marketing manager at - orange-Packers-Movers-Pune

 

Related Topics

  Subject / Started by Replies Last post
1 Replies
982 Views
Last post 12-09-2013, 23:57:39
by emedianetwork
22 Replies
8809 Views
Last post 03-31-2016, 03:39:43
by ShreyaKoushik
0 Replies
553 Views
Last post 02-25-2016, 00:04:45
by qx_1789