# All robots will spider the domain User-agent: * Disallow: # Disallow directory /_edit/ User-agent: * Disallow: /_edit/ # Disallow directory /_admin/ User-agent: * Disallow: /_admin/ # Disallow directory /_special/ User-agent: * Disallow: /_special/ # Disallow directory /_history/ User-agent: * Disallow: /_history/