# Specs at http://info.webcrawler.com/mak/projects/robots/robots.html # Also see http://info.webcrawler.com/mak/projects/robots/exclusion.html # The items below disallows all robots User-agent: * Disallow: /