Can robots.txt block SEs from specific countries?

Author Topic: Can robots.txt block SEs from specific countries?  (Read 2073 times)

Offline oleshkaTopic starter

  • Trade Count: (0)
  • Semi-Newbie
  • *
  • Thank You 1
  • Posts: 48
  • Karma: 1
    • Фильмы по профессиональному Боксу
Is it possible to create a robots.txt file that will exclude all Australian based search engines (ie, google.com.au, au.bing.com, au.yahoo.com, etc) from indexing my website? If so, how would I do that?

Conversely, can you create a robots.txt file that ONLY allows Australian based search engines (such as the ones listed above) from indexing a site? And even more specifically, is it possible to create a robots.txt file that allows only search engines from Australia to index just one subdirectory of a site, while the rest of the site is open to search engines from any country?


Offline Charter

  • Trade Count: (0)
  • Semi-Newbie
  • *
  • Thank You 1
  • Posts: 36
  • Karma: 1
Re: Can robots.txt block SEs from specific countries?
« Reply #1 on: 05-29-2010, 01:37:43 »
User-agent: Australian based search engines bot
Disallow: /

Offline a3web

  • Trade Count: (0)
  • Semi-Newbie
  • *
  • Thank You 6
  • Posts: 49
  • Karma: 3
  • Gender: Male
    • Boaters Directory
Re: Can robots.txt block SEs from specific countries?
« Reply #2 on: 05-29-2010, 04:34:49 »
Go through Google webmaster tool and read about "robots.txt".

Offline coolman

  • Trade Count: (0)
  • Semi-Newbie
  • *
  • Thank You 2
  • Posts: 39
  • Karma: 2
Re: Can robots.txt block SEs from specific countries?
« Reply #3 on: 05-31-2010, 03:25:28 »
The robots.txt checker can check you site's robots.txt file and meta tags.

Offline coolrays

  • Trade Count: (0)
  • Semi-Newbie
  • *
  • Thank You 0
  • Posts: 34
  • Karma: 1
Re: Can robots.txt block SEs from specific countries?
« Reply #4 on: 06-01-2010, 02:58:58 »
You may go to about/robots.txt to know about it.  It explains clearly the phenomenon used.


 

Related Topics

  Subject / Started by Replies Last post
2 Replies
1478 Views
Last post 10-17-2011, 03:41:36
by Hogward
1 Replies
1929 Views
Last post 06-04-2015, 11:32:55
by MarcoLeitner
8 Replies
5124 Views
Last post 07-04-2016, 07:12:36
by seooffice
2 Replies
3088 Views
Last post 11-25-2016, 04:39:23
by CreativeDreamrz
5 Replies
442 Views
Last post 10-31-2017, 00:57:17
by swikriti.sharma09