Seo Forum

Search Engines => Google Search Optimization => Topic started by: manvbf on 11-01-2011, 03:59:46

Title: Google spider not good enough
Post by: manvbf on 11-01-2011, 03:59:46
Well, I feel like I've been forced to take matters into my own hands, and build my own site search. I've tried quite a few different things, from site maps to mod rewrite. It seems that google decides how many pages it wants to index on a site based on its own algorithm that determines the importance of a site. That's fine for Google, but the users of my site suffer. There are too many "supplemental" results and unindexed pages.

Take phpbb (just an example). Why should I need to use mod rewrite (which only partially improves the situation)? And then why would I want to put a Google site search box on my site, when the site is not properly indexed?

I've found a number of spiders that are available that will crawl my site better than google. With some fine tuning, I have a much more complete index than google. And with some login, I hope to have a better weighting algorithm as well.

Anyone else having the same problems?
Title: Re: Google spider not good enough
Post by: takeshiro on 11-02-2011, 09:48:53
yup, I agree with your stuff.