What are Spiders, Robots and Crawlers ?

Author Topic: What are Spiders, Robots and Crawlers ?  (Read 2215 times)

Offline JoyadelfinTopic starter

  • Trade Count: (0)
  • Jr. Member
  • **
  • Thank You 5
  • Posts: 67
  • Karma: 0
    • Muslim Mantra For Love
What are Spiders, Robots and Crawlers ?
« on: 08-19-2015, 06:12:24 »
hello friends,


I want to know that What are Spiders, Robots and Crawlers and what are their functions? explain friends..


Offline Dewlance

  • Trade Count: (0)
  • Semi-Newbie
  • *
  • Thank You 2
  • Posts: 48
  • Karma: 0
    • Dewlance
Re: What are Spiders, Robots and Crawlers ?
« Reply #1 on: 08-22-2015, 07:36:25 »
I don't know to much but crawlers are programs which can copy our website text or full website and index site to their database.

Spiders are also a computer program but If you use common sense then you can easily understand what is meaning of Robots.
Cheap Windows VPS Hosting from Dewlance (Multiple Location)

Offline seoroy

  • Trade Count: (0)
  • Jr. Member
  • **
  • Thank You 7
  • Posts: 87
  • Karma: 0
  • Gender: Female
    • Wedding Planners in Delhi NCR
Re: What are Spiders, Robots and Crawlers ?
« Reply #2 on: 11-15-2015, 23:43:35 »
Spider - The browsers are like a program and to download the web page.


Robots - It had automated computer program can visit websites. It will guided by search engine algorithms It can combine the tasks of crawler & spider helpful of the indexing the web pages and through the search engines.

Crawler – The program is automatically to follow the links are web page.

Offline RH-Calvin

  • Trade Count: (0)
  • Hero Member
  • *****
  • Thank You 88
  • Posts: 1290
  • Karma: 0
  • Gender: Male
    • Cheap VPS Hosting
Re: What are Spiders, Robots and Crawlers ?
« Reply #3 on: 11-16-2015, 23:06:27 »
They are all the same and they are search engine programs that contain instructions to perform certain tasks online. They are mainly responsible to read through webpage sources and provide cache certificate to various webpages after successful crawling.

Offline seoroy

  • Trade Count: (0)
  • Jr. Member
  • **
  • Thank You 7
  • Posts: 87
  • Karma: 0
  • Gender: Female
    • Wedding Planners in Delhi NCR
Re: What are Spiders, Robots and Crawlers ?
« Reply #4 on: 11-23-2015, 03:04:50 »
Spider & crawler A bug is a program that visits Web destinations and peruses their pages and other data with a specific end goal to make passages for a web crawler record. The real web search tools on the Web all have such a project, which is otherwise called a "crawler" or a "bot." 

robots.txt It is extraordinary when web indexes as often as possible visit your website and record your substance however regularly there are situations when indexing parts of your online substance is not what you need. Case in point, on the off chance that you have two adaptations of a page , you'd rather have the printing adaptation prohibited from slithering, else you hazard being forced a copy content punishment.


Offline richardmsmith

  • Trade Count: (0)
  • Full Member
  • ***
  • Thank You 2
  • Posts: 229
  • Karma: 0
  • Gender: Male
    • Trucking App Solution
Re: What are Spiders, Robots and Crawlers ?
« Reply #5 on: 09-19-2018, 23:00:05 »
Robots are programs that follow links on Web sites, gathering data for search engine indexes. They are considered robots rather than programs because they are somewhat independent  no programmer has told them what pages to find.

Offline VPSnet

  • Trade Count: (0)
  • Semi-Newbie
  • *
  • Thank You 0
  • Posts: 13
  • Karma: 0
    • VPSnet
Re: What are Spiders, Robots and Crawlers ?
« Reply #6 on: 09-26-2018, 07:03:08 »
All the three programmes Spiders, Robots and crawlers are same and do similar work we call them with different names these are only service tools for search engine which helps to find and index new web links in search engine correctly

 

Related Topics

  Subject / Started by Replies Last post
4 Replies
1975 Views
Last post 05-31-2010, 03:26:45
by coolman
6 Replies
2166 Views
Last post 09-04-2018, 05:11:02
by richardmsmith
4 Replies
2286 Views
Last post 01-19-2012, 22:54:39
by cpaoutsourcing
4 Replies
2462 Views
Last post 09-20-2018, 22:24:07
by MVMInfotech18
7 Replies
2634 Views
Last post 04-06-2017, 02:59:51
by sarahmaxwell789