# 5 April 2010 3.37pm # Exclude Files From All Robots: # Disallow all files in these directories for all spiders User-agent: msnbot Disallow: /scripts08/ Disallow: /services08/ User-agent: dotbot Disallow: / User-agent: Twiceler Disallow: / User-agent: Gigabot Disallow: / User-agent: Teoma Disallow: / User-agent: Robi Disallow: / User-agent: Webster Pro Disallow: / User-agent: Webster Disallow: / User-agent: Zeus Disallow: / User-agent: Slurp Disallow: / User-agent: MJ12bot Disallow: / User-agent: Yandex Disallow: / User-agent:* Disallow: /php/redirect.php User-agent: Googlebot-Image Disallow: / User-agent: baiduspider Disallow: / # End robots.txt file