Seo Forum

Search Engine Optimization => On Site Optimization => Topic started by: Sap Solution on 09-29-2017, 12:16:04

Title: Stopping Google bot
Post by: Sap Solution on 09-29-2017, 12:16:04
How can we stop Google bot to visit on our website and in what cases this is requirement ?
Title: Re: Stopping Google bot
Post by: Jacob.k on 09-29-2017, 18:45:34
You can stop it in .htaccess... Not really sure how, but you'll find that easily on google.I think it's just one or two line code... just remember to backup your htaccess file, because if you do something wrong you're screwed  :-X
Title: Re: Stopping Google bot
Post by: SEO Travis on 10-01-2017, 01:12:01
Well, you could try the following:

Insert into HTML <head> of website: (Could be done through WP theme)
<meta name="robots" content="noindex, nofollow">
<meta name="googlebot" content="noindex, nofollow">

or you can block with robots.txt on your website.

User-agent:GoogleBot
Disallow: /

I hope this answers your question.
Title: Re: Stopping Google bot
Post by: guitarprince on 10-21-2017, 10:06:26
If you want to prevent Googlebot from crawling content on your site, you have a number of options, including using robots.txt to block access to files and directories on your server.

Once you've created your robots.txt file, there may be a small delay before Googlebot discovers your changes. If Googlebot is still crawling content you've blocked in robots.txt, check that the robots.txt is in the correct location. It must be in the top directory of the server (for example, www.example.com/robots.txt); placing the file in a subdirectory won't have any effect.

If you just want to prevent the "file not found" error messages in your web server log, you can create an empty file named robots.txt. If you want to prevent Googlebot from following any links on a page of your site, you can use the nofollow meta tag. To prevent Googlebot from following an individual link, add the rel="nofollow" attribute to the link itself.
Title: Re: Stopping Google bot
Post by: RH-Calvin on 10-24-2017, 22:39:02
You can stop Googlebot by editing the robots.txt file in your website. You can disallow the webpages that you don't want to get crawled by Google.