# 20 July 2009 5.50pm # Exclude Files From All Robots: User-agent: dotbot Disallow: / User-agent: Twiceler Disallow: / User-agent: Gigabot Disallow: / User-agent: Teoma Disallow: / User-agent: Robi Disallow: / User-agent: Webster Pro Disallow: / User-agent: Webster Disallow: / User-agent: Zeus Disallow: / User-agent: Slurp Disallow: / User-agent: MJ12bot Disallow: / User-agent: Yandex Disallow: / User-agent:* Disallow: /php/redirect.php User-agent: Googlebot-Image Disallow: / User-agent: baiduspider Disallow: / # End robots.txt file