What is robots.txt? Does it have any value for SEO?

Author Topic: What is robots.txt? Does it have any value for SEO?  (Read 10378 times)

Offline moritzbokk

  • Trade Count: (0)
  • Semi-Newbie
  • *
  • Thank You 2
  • Posts: 25
  • Karma: 0
Re: What is robots.txt? Does it have any value for SEO?
« Reply #10 on: 08-07-2012, 07:45:41 »
It's especially good, if you have a print preview for each of your sites. So to avoid dublicate contant you can determine with the robots.txt, that all the print previews shouldn't be crawled.


Offline ljennifer012

  • Trade Count: (0)
  • Semi-Newbie
  • *
  • Thank You 2
  • Posts: 24
  • Karma: 0
    • Home Solar Panels | Solar Installation - Solar Masta
Re: What is robots.txt? Does it have any value for SEO?
« Reply #11 on: 08-10-2012, 19:33:37 »
Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit.
Want to save up electricity up to 70%? Let Our  Solar Specialist  install Your Home Solar Panels

Offline milestoneuk

  • Trade Count: (0)
  • Newbie
  • *
  • Thank You 0
  • Posts: 6
  • Karma: 0
    • Web design Newcastle Under Lyme
Re: What is robots.txt? Does it have any value for SEO?
« Reply #12 on: 08-10-2012, 23:12:13 »
Robot.txt is general text pc computer file it is use to inform seo spider about which web page is index and neglect. It is helpful for SEO because it is help to listing website. This pc computer file is located in root directory of website.
newbielink:http://www.milestoneuk.co.uk [nonactive]

Offline kradha707

  • Trade Count: (0)
  • Jr. Member
  • **
  • Thank You 5
  • Posts: 57
  • Karma: 0
  • Gender: Female
    • Chennai to Madurai Bus Tickets
Re: What is robots.txt? Does it have any value for SEO?
« Reply #13 on: 08-13-2012, 01:11:53 »
In order not to crawl our website pages we are giving the information to spiders using robots.txt. If you website is having some sensitive information then you can restrict to crawl those pages with robots.txt.

I do have one more doubt here.

How can i know whether the pages on my site is indexed by google or not, please help me.
Book Bus Tickets to any destination in India @ TicketGoose.com

Offline arenakkg

  • Trade Count: (0)
  • Semi-Newbie
  • *
  • Thank You 1
  • Posts: 26
  • Karma: 0
Re: What is robots.txt? Does it have any value for SEO?
« Reply #14 on: 08-14-2012, 07:11:52 »
A robots.txt file on a website will function as a request that specified robots ignore specified files or directories when crawling a site. This might be, for example, out of a preference for privacy from search engine results, or the belief that the content of the selected directories might be misleading or irrelevant to the categorization of the site as a whole, or out of a desire that an application only operate on certain data.



Thanks...
web design courses


Offline sureshgupta1

  • Trade Count: (0)
  • Jr. Member
  • **
  • Thank You 12
  • Posts: 97
  • Karma: 0
  • Gender: Male
  • Because It's penguinlism
    • My Website
Re: What is robots.txt? Does it have any value for SEO?
« Reply #15 on: 08-14-2012, 21:37:20 »
Robots.txt is a notepad file that use for giving instruction to search engines for which page to crawl or not, Now days it becomes great tool to avoiding duplicates and other things

Have banned by Panda & Penguin, try our guest posting service and real poems

Offline 12dian04

  • Trade Count: (0)
  • Semi-Newbie
  • *
  • Thank You 1
  • Posts: 28
  • Karma: 1
  • Gender: Male
    • led street light
Re: What is robots.txt? Does it have any value for SEO?
« Reply #16 on: 08-15-2012, 00:38:10 »
robots.txt is a protocol, rather than a command. robots.txt is a search engine visit the Web site you want to view a file. The robots.txt file to tell the spider what files on the server can be viewed.
Eneletec LED High Bay Lights is IP65 and could be used in different applications.
led lights

Offline salty143

  • Trade Count: (0)
  • Jr. Member
  • **
  • Thank You 12
  • Posts: 95
  • Karma: 0
    • The Mattress Protector Company
Re: What is robots.txt? Does it have any value for SEO?
« Reply #17 on: 08-28-2012, 00:00:42 »
According to me Robots.txt file is a set of instructions that tell search engine robots which pages of your site to be crawled and indexed.

Offline moritzbokk

  • Trade Count: (0)
  • Semi-Newbie
  • *
  • Thank You 2
  • Posts: 25
  • Karma: 0
Re: What is robots.txt? Does it have any value for SEO?
« Reply #18 on: 08-29-2012, 04:58:07 »
with the robots.txt you can handle your linkjuice
for example the contact page is not so important so you tell the crawler with the robots.txt "hey man this page isn't important don't give it some of my linkjuice" ^^


Offline aadi14

  • Trade Count: (0)
  • Semi-Newbie
  • *
  • Thank You 5
  • Posts: 44
  • Karma: 0
  • Gender: Male
    • Rancor Infotech Pvt Ltd
Re: What is robots.txt? Does it have any value for SEO?
« Reply #19 on: 09-16-2012, 23:46:38 »
Robots.txt Is a file in which we specify/tell Google that which pages he don't want to get indexed.It is used to disallow the pages and by doing it the specified pages are left un-indexed.

 

Related Topics

  Subject / Started by Replies Last post
4 Replies
2720 Views
Last post 09-20-2018, 22:24:07
by MVMInfotech18
11 Replies
4268 Views
Last post 10-05-2016, 06:39:48
by mukeshkr
5 Replies
2228 Views
Last post 04-12-2016, 23:39:58
by jaysh4922
4 Replies
1904 Views
Last post 12-28-2016, 03:47:42
by wellliving
4 Replies
2857 Views
Last post 12-27-2016, 03:47:18
by damponting44