Search Engine Optimization > Advanced Search Engine Optimization

Can robots.txt block SEs from specific countries?


Is it possible to create a robots.txt file that will exclude all Australian based search engines (ie,,,, etc) from indexing my website? If so, how would I do that?

Conversely, can you create a robots.txt file that ONLY allows Australian based search engines (such as the ones listed above) from indexing a site? And even more specifically, is it possible to create a robots.txt file that allows only search engines from Australia to index just one subdirectory of a site, while the rest of the site is open to search engines from any country?

User-agent: Australian based search engines bot
Disallow: /

Go through Google webmaster tool and read about "robots.txt".

The robots.txt checker can check you site's robots.txt file and meta tags.

You may go to about/robots.txt to know about it.  It explains clearly the phenomenon used.


[0] Message Index

Go to full version