What is web Crawling?

Author Topic: What is web Crawling?  (Read 3630 times)

Offline kalmakuranTopic starter

  • Trade Count: (0)
  • Jr. Member
  • **
  • Thank You 3
  • Posts: 79
  • Karma: 0
    • Islamic Wazifa for Love
What is web Crawling?
« on: 01-23-2017, 05:21:33 »
Hello friends,

I want to know that What is web Crawling?


Offline hynds

  • Trade Count: (0)
  • Full Member
  • ***
  • Thank You 21
  • Posts: 168
  • Karma: 0
  • Gender: Male
  • Free Domain Coupon Codes at http://eCoupon.io/
    • Cheap Domain Promo Codes
Re: What is web Crawling?
« Reply #1 on: 01-23-2017, 22:28:40 »
Web Crawling basically means following a path. In the SEO world, crawling means following your links and “crawling” around your website. When bots come to your website (any page), they follow other linked pages also on your website.

This is one reason why we create site maps, as they contain all of the links in our blog and Google’s bots can use them to look deeply into a website.

The way we stop crawling certain parts of our site is by using the Robots.txt file.
Godaddy coupon just for $1/mo Hosting, 99 cents .COM Domain

Offline RoyJones

  • Trade Count: (0)
  • Jr. Member
  • **
  • Thank You 5
  • Posts: 59
  • Karma: 0
  • Gender: Male
    • esourceparts
Re: What is web Crawling?
« Reply #2 on: 01-23-2017, 23:34:15 »
A Web crawler is an Internet bot which Web crawlers help in collecting information about a website. They crawl one page at a time through a website until all pages have been indexed.

Offline gurujeet

  • Trade Count: (0)
  • Jr. Member
  • **
  • Thank You 2
  • Posts: 63
  • Karma: 0
Re: What is web Crawling?
« Reply #3 on: 01-24-2017, 02:27:29 »
A search engine crawler is a program or automated script that browses the World Wide Web in a methodical manner in order to provide up to date data to the particular search engine. While search engine crawlers go by many different names, such as web spiders and automatic indexers, the job of the search engine crawler is still the same.

 

Related Topics

  Subject / Started by Replies Last post
1 Replies
1704 Views
Last post 12-18-2011, 15:50:12
by rouled
0 Replies
1105 Views
Last post 10-28-2011, 01:20:40
by Walalayo
8 Replies
5836 Views
Last post 11-05-2016, 02:18:12
by Rammadhur
9 Replies
4436 Views
Last post 01-20-2017, 10:15:15
by claysmithpc
2 Replies
857 Views
Last post 02-24-2016, 03:04:37
by harsinha0018