Search Engine Optimization > Advanced Search Engine Optimization

Can robots.txt block SEs from specific countries?

(1/1)

oleshka:
Is it possible to create a robots.txt file that will exclude all Australian based search engines (ie, google.com.au, au.bing.com, au.yahoo.com, etc) from indexing my website? If so, how would I do that?

Conversely, can you create a robots.txt file that ONLY allows Australian based search engines (such as the ones listed above) from indexing a site? And even more specifically, is it possible to create a robots.txt file that allows only search engines from Australia to index just one subdirectory of a site, while the rest of the site is open to search engines from any country?

Charter:
User-agent: Australian based search engines bot
Disallow: /

a3web:
Go through Google webmaster tool and read about "robots.txt".

coolman:
The robots.txt checker can check you site's robots.txt file and meta tags.

coolrays:
You may go to about/robots.txt to know about it.  It explains clearly the phenomenon used.

Navigation

[0] Message Index