#
User-agent: webreaper
User-agent: wget
User-agent: sitesnag
User-agent: HTTrack
User-agent: HTTrack 3.0x
Disallow: /
User-agent: *
Disallow: /rd/
User-Agent: *
Crawl-delay: 20