What is the difference between crawlers, spiders and robots in SEO

Author Topic: What is the difference between crawlers, spiders and robots in SEO  (Read 3508 times)

Offline EthanHawkeTopic starter

  • Trade Count: (0)
  • Newbie
  • *
  • Thank You 0
  • Posts: 1
  • Karma: 0

It has been asked as an interview question to me the difference between crawlers, spiders and robots in SEO. But i am not able to answer it as there is no such difference between the three all of them are search engine crawlers that searches the keywords and links in website for indexing. So if any one knows little bit information about the difference please reply me.

Offline RH-Calvin

  • Trade Count: (0)
  • Hero Member
  • *****
  • Thank You 92
  • Posts: 1243
  • Karma: 0
  • Gender: Male
    • Cheap VPS Hosting
They are all the same. They are automated search engine programs that are responsible to read through webpage source and provide the webpage with a cache certificate.

Offline fix.97

  • Trade Count: (0)
  • Jr. Member
  • **
  • Thank You 4
  • Posts: 56
  • Karma: 0
    • Fikreview
Spider - a browser like program that downloads web pages.

Crawler – a program that automatically follows all of the links on each web page.

Robots - An automated computer program that visit websites & perform predefined task. They are guided by search engine algorithms & are able to perform different tasks instead of just one crawling task. They can combine the tasks of crawler & spider together and help in indexing and ranking of websites on any particular search engine.

Offline TomClarke

  • Trade Count: (0)
  • Hero Member
  • *****
  • Thank You 55
  • Posts: 837
  • Karma: 0
  • Gender: Male
    • Custom Writing Service & Content Marketplace
Spider - scan and judge your website content
Crawler - crawl your content and index it in there searches according to Inbound links.
Both techniques helps Google to judge importance of website in terms of ranking.
Robots.txt  - if the file thta is included into web coding  so that Google not consider it in crawling ot to hide page from Google crawling.
Your ideas, our writings - Writology.com

Offline MVMInfotech18

  • Trade Count: (0)
  • Full Member
  • ***
  • Thank You 4
  • Posts: 110
  • Karma: 0
    • best website company in laos
Robots are programs that follow links on Web sites, gathering data for search engine indexes. They are considered robots rather than programs because they are somewhat independent: no programmer has told them what pages to find. Other names for these programs are spider or crawler (because they follow links on the Web), worm, wanderer , gatherer


Related Topics

  Subject / Started by Replies Last post
4 Replies
Last post 05-31-2010, 03:26:45
by coolman
8 Replies
Last post 12-07-2018, 21:59:25
by DeveloperOnRent
6 Replies
Last post 10-18-2019, 04:47:50
by lishmaliny
24 Replies
Last post 05-10-2012, 01:38:00
by ring2012
7 Replies
Last post 04-06-2017, 02:59:51
by sarahmaxwell789