User-agent: Slurp Disallow: / User-agent: * Disallow: /cgi-bin Disallow: /datenbank Disallow: /hpsitter Disallow: /suche Disallow: /wordpress Disallow: /bot-trap # spammer User-agent: LinkWalker Disallow: / # archiver - no thanks User-agent: ia_archiver Disallow: / # company letting crawl the community: nono User-agent: grub-client Disallow: / # Inktomisearch go away User-agent: Slurp/2.0 Disallow: / # Inktomisearch - f**k up User-agent: Slurp.so/1.0 Disallow: / # Inktomisearch - no thanks User-agent: Slurp/si Disallow: / # Inktomisearch - cat? User-agent: Slurp/cat Disallow: / # Inktomisearch go away User-agent: Slurp Disallow: /