# /robots.txt file for http://webcrawler.com/ # mail support@essencha.com for constructive criticism #User-agent: webcrawler #Disallow: User-agent: * Disallow: /administer/ Disallow: /account/ Disallow: /shipping_service/ Disallow: /WebCharts/ Disallow: /includes/ Disallow: /Essencha.Business Disallow: /PDF/ Disallow: /OldPDFs/ Disallow: /UserControls/ Disallow: /bin/ Disallow: /survey/ Disallow: /kred-glass/ Disallow: /images/fvg318/ Disallow: /Test/