Crawlers

Author Topic: Crawlers  (Read 1105 times)

Offline WalalayoTopic starter

  • Trade Count: (0)
  • Full Member
  • ***
  • Thank You 3
  • Posts: 180
  • Karma: 2
    • JavaScript Editor
Crawlers
« on: 11-01-2011, 21:13:17 »
how many levels/layers deep does a search engine crawler go, i have a site map with multiple pages on my site and i think the multiple pages is creating a problem for the crawlers. can anyone offer some insight to this?? thanks in advance


Offline C.Rebecca

  • Trade Count: (0)
  • Sr. Member
  • ****
  • Thank You 16
  • Posts: 360
  • Karma: 25
Re: Crawlers
« Reply #1 on: 11-02-2011, 02:41:31 »
A crawler can crawl all the links / pages on your sitemap...

For websites with large number of websites you can create a good internal linking strucutre. A good linking structure means your every webpage is linked to atleast one other web page of your website.
230% more traffic with 12+ Keyword Research Tools

Offline Hogward

  • Trade Count: (0)
  • Sr. Member
  • ****
  • Thank You 17
  • Posts: 420
  • Karma: 5
Re: Crawlers
« Reply #2 on: 11-02-2011, 03:04:30 »
Most commonly important product/service page of the site should be within 1 or 2 clicks from the home page. Further deep creates difficulty for both user and crawlers to look for the information.

Offline RaviFitness

  • Trade Count: (0)
  • Semi-Newbie
  • *
  • Thank You 0
  • Posts: 18
  • Karma: 1
  • Gender: Male
    • outdoor fitness
Re: Crawlers
« Reply #3 on: 11-03-2011, 23:44:33 »
Crawlers in the context of SEO, they are referring to the programs that search engines use to scan and analyze websites in order to determine their importance. Crawlers are very active, and often account for a great deal of the visitors to websites all over the internet.

Offline TM-Ali

  • Trade Count: (0)
  • Full Member
  • ***
  • Thank You 3
  • Posts: 107
  • Karma: 2
    • SEO Company India
Re: Crawlers
« Reply #4 on: 11-04-2011, 06:16:24 »
web crawler also perform deep link indexing.


Offline Vinil

  • Trade Count: (0)
  • Sr. Member
  • ****
  • Thank You 130
  • Posts: 301
  • Karma: 13
  • Gender: Male
  • I can and I Will
    • Boaters Directory
Re: Crawlers
« Reply #5 on: 11-05-2011, 00:01:18 »
Tell me how many pages max.  crawlers can crawl/Also If I have large pages ,then do i need to prepare multiple sitemaps ?

 

Related Topics

  Subject / Started by Replies Last post
4 Replies
1598 Views
Last post 01-19-2012, 22:54:39
by cpaoutsourcing
4 Replies
1378 Views
Last post 11-23-2015, 03:04:50
by seoroy
3 Replies
1163 Views
Last post 08-23-2016, 06:48:00
by TomClarke