From Seo Wiki - Search Engine Optimization and Programming Languages
Mediabot visits those pages running AdSense ads that have not blocked its access via a robots.txt file and it's a Google recommendation that webmasters specifically add a command to their robots.txt file granting Mediabot access to the entire site. Here is how to do it:
User-agent: Mediapartners-Google* Disallow:
The Mediabot identifies itself with the user agent string "Mediapartners-Google/2.1".
The Mediabot revisits pages on a regular, but unpredictable basis. Changes made to a page therefore do not immediately cause changes to the ads displayed on the page. Note that ads can still be shown on a page even if the Mediabot has not yet visited it, in which case the ads chosen will be based on the overall theme of the other pages on the site. If no ads can be chosen, public service announcements are displayed instead.
|File:Crystal Clear app browser.png||This World Wide Web-related article is a stub. You can help Wikipedia by expanding it.|
|File:System-installer.svg||This computer software article is a stub. You can help Wikipedia by expanding it.|