# Robots.txt # Be nice. # User-agent: MSIECrawler Disallow: /*ARCHIVE* Disallow: / # User-agent: * Disallow: /*ARCHIVE* Disallow: /test Disallow: /testing # User-agent: WebVac Disallow: /*ARCHIVE*