# robots.txt # Evil non-search engine robot that requests bad URLs and pummels server User-agent: http://www.almaden.ibm.com/cs/crawler Disallow: / User-agent: Slurp Crawl-delay: 60 User-agent: * Disallow: /cfg Disallow: /de Disallow: /en Disallow: /es Disallow: /images Disallow: /includes Disallow: /img Disallow: /media Disallow: /rtsp