# Robots.txt file#User-agent: *Disallow: /wp-admin/
Sitemap: http://www.buzinga.com.au/sitemap_index.xml#common for all sites..## robots.txt for http://www.buzinga.com.au/## Please note: There are a lot of pages on this site, and there are# some misbehaved spiders out there. If you're# irresponsible, your access to the site may be blocked.## Trap Bots that do not respect robots.txtDisallow: /bot-trap/# don't want the following to crawl..User-agent: UbiCrawlerDisallow: /User-agent: DOCDisallow: /User-agent: ZaoDisallow: /# Some of these designed to copy# entire sites. Please obey robots.txt.User-agent: sitecheck.internetseer.comDisallow: /User-agent: ZealbotDisallow: /User-agent: MSIECrawlerDisallow: /User-agent: SiteSnaggerDisallow: /User-agent: WebStripperDisallow: /User-agent: WebCopierDisallow: /User-agent: FetchDisallow: /User-agent: TeleportDisallow: /User-agent: TeleportProDisallow: /User-agent: WebZIPDisallow: /User-agent: linkoDisallow: /User-agent: HTTrackDisallow: /User-agent: XenuDisallow: /User-agent: larbinDisallow: /User-agent: libwwwDisallow: /User-agent: ZyBORGDisallow: /User-agent: Download NinjaDisallow: /User-agent: wgetDisallow: /User-agent: grub-clientDisallow: /User-agent: k2spiderDisallow: /User-agent: NPBotDisallow: /User-agent: WebReaperDisallow: /User-Agent: rogerBotCrawl-Delay: 10User-Agent: msnbotCrawl-Delay: 1User-Agent: YandexBotCrawl-Delay: 10User-Agent: TweetedTimesCrawl-Delay: 30User-Agent: TweetedTimes Bot/1.0Crawl-Delay: 30User-Agent: MotorolaCrawl-Delay: 30User-Agent: Jakarta Commons-HttpClient/3.1Crawl-Delay: 30# END OF FILE