# robots.txt User-agent: ICCrawler - iCjobs Disallow: / User-agent: dotbot Disallow: / User-agent: baiduspider Disallow: / User-agent: baidu Disallow: / User-agent: Baidu Disallow: / User-agent: Baiduspider Disallow: /