# since a 'nice' site sucked up 45meg off my server in one night - root folder only User-agent: * Disallow: /images/ Disallow: /downloads/ Disallow: /uploads/ # google # only the image robot User-agent: Googlebot-Image* Disallow: / # only allow rrroar*.htm and index.html, no pics & no other pages User-agent: Googlebot* Allow: /rrroar*.htm Allow: /index.html Disallow: /*.jpg Disallow: /*.htm # only allow rrroar*.htm and index.html, no pics & no other pages User-agent: Mozilla/5.0 (compatible; Googlebot* Allow: /rrroar*.htm Allow: /index.html Disallow: /*.jpg Disallow: /*.htm # denials - everyone else User-agent: ActiveBookmark* Disallow: / User-agent: Alexibot* Disallow: / User-agent: Aqua_Products* Disallow: / User-agent: Ask Jeeves/Teoma* Disallow: / User-agent: AskJeeves* Disallow: / User-agent: asterias* Disallow: / User-agent: Baiduspider* Disallow: / User-agent: BackDoorBot* Disallow: / User-agent: BecomeBot* Disallow: / User-agent: BlowFish* Disallow: / User-agent: Bookmark search tool* Disallow: / User-agent: BotALot* Disallow: / User-agent: BruinBot* Disallow: / User-agent: BuiltBotTough* Disallow: / User-agent: Bullseye* Disallow: / User-agent: BunnySlippers* Disallow: / User-agent: b2w* Disallow: / User-agent: CheeseBot* Disallow: / User-agent: CherryPicker* Disallow: / User-agent: Convera* Disallow: / User-agent: Copernic* Disallow: / User-agent: CopyRightCheck* Disallow: / User-agent: cosmos* Disallow: / User-agent: Crescent* Disallow: / User-agent: DittoSpyder* Disallow: / User-agent: dumbot* Disallow: / User-agent: eBingBong* Disallow: / User-agent: ejupiter* Disallow: / User-agent: EmailCollector* Disallow: / User-agent: EmailSiphon* Disallow: / User-agent: envolk* Disallow: / User-agent: EmailWolf* Disallow: / User-agent: Enterprise_Search* Disallow: / User-agent: EroCrawler* Disallow: / User-agent: es* Disallow: / User-agent: ExtractorPro* Disallow: / User-agent: FairAd Client* Disallow: / User-agent: findlinks* Disallow: / User-agent: Flaming AttackBot* Disallow: / User-agent: Foobot* Disallow: / User-agent: Gaisbot* Disallow: / User-agent: GetRight* Disallow: / User-agent: Gigabot* Disallow: / User-agent: GOFORITBOT* Disallow: / User-agent: grub* Disallow: / User-agent: Harvest* Disallow: / User-agent: Hatena Antenna* Disallow: / User-agent: HenryTheMiragoRobot* Disallow: / User-agent: hloader* Disallow: / User-agent: httplib* Disallow: / User-agent: http://www.SearchEngineWorld.com bot* Disallow: / User-agent: http://www.WebmasterWorld.com bot* Disallow: / User-agent: humanlinks* Disallow: / User-agent: ia_archiver* Disallow: / User-agent: ichiro* Disallow: / User-agent: IconSurf* Disallow: / User-agent: IlTrovatore-Setaccio* Disallow: / User-agent: InfoNaviRobot* Disallow: / User-agent: InternetArchive* Disallow: / User-agent: IRLbot* Disallow: / User-agent: Iron33* Disallow: / User-agent: Jeeves* Disallow: / User-agent: JennyBot* Disallow: / User-agent: Jetbot* Disallow: / User-agent: Kenjin Spider* Disallow: / User-agent: Keyword Density* Disallow: / User-agent: Konqueror* Disallow: / User-agent: larbin* Disallow: / User-agent: LexiBot* Disallow: / User-agent: libWeb/clsHTTP* Disallow: / User-agent: LinkextractorPro* Disallow: / User-agent: LinkScan* Disallow: / User-agent: LinkWalker* Disallow: / User-agent: LNSpiderguy* Disallow: / User-agent: looksmart* Disallow: / User-agent: lwp-trivial* Disallow: / User-agent: Lycos* Disallow: / User-agent: Mata Hari* Disallow: / User-agent: Metaspinner* Disallow: / User-agent: Microsoft URL Control* Disallow: / User-agent: MIIxpc* Disallow: / User-agent: Minuteman* Disallow: / User-agent: Mister PiX* Disallow: / User-agent: MJ12bot* Disallow: / User-agent: moget* Disallow: / User-agent: Mozdex* Disallow: / User-agent: mozilla* Disallow: / User-agent: Mozilla* Disallow: / User-agent: MSIECrawler* Disallow: / User-agent: msnbot* Disallow: / User-agent: MSProxy* Disallow: / User-agent: MSRBOT* Disallow: / User-agent: MVAClient* Disallow: / User-agent: naver* Disallow: / User-agent: NetAnts* Disallow: / User-agent: NetMechanic* Disallow: / User-agent: NetResearch* Disallow: / User-agent: NG* Disallow: / User-agent: nicebot* Disallow: / User-agent: NICErsPRO* Disallow: / User-agent: NimbleCrawler* Disallow: / User-agent: Nutch* Disallow: / User-agent: oBot* Disallow: / User-agent: Offline Explorer* Disallow: / User-Agent: OmniExplorer_Bot* Disallow: / User-Agent: onCHECK* Disallow: / User-Agent: OnetSzukaj* Disallow: / User-agent: Openfind* Disallow: / User-agent: Openbot* Disallow: / User-agent: Oracle Ultra Search* Disallow: / User-agent: PerMan* Disallow: / User-agent: PHP* Disallow: / User-agent: Pompos* Disallow: / User-agent: ProPowerBot* Disallow: / User-agent: ProWebWalker* Disallow: / User-agent: psbot* Disallow: / User-agent: Python-urllib* Disallow: / User-agent: QueryN Metasearch* Disallow: / User-agent: Radiation Retriever* Disallow: / User-agent: RepoMonkey* Disallow: / User-agent: RMA* Disallow: / User-agent: RufusBot* Disallow: / User-agent: SBIder* Disallow: / User-agent: scooter* Disallow: / User-agent: searchpreview* Disallow: / User-agent: Sensis* Disallow: / User-agent: Silk* Disallow: / User-agent: SiteSnagger* Disallow: / # yahoo User-agent: Slurp* Disallow: / User-agent: snap.com* Disallow: / User-agent: sohu* Disallow: / User-agent: sootle* Disallow: / User-agent: SpankBot* Disallow: / User-agent: spanner* Disallow: / User-agent: sproose* Disallow: / User-agent: Squigglebot* Disallow: / User-agent: StackRambler* Disallow: / User-agent: Stanford* Disallow: / User-agent: SurveyBot* Disallow: / User-agent: suzuran* Disallow: / User-agent: Szukacz* Disallow: / User-agent: Teleport* Disallow: / User-agent: Telesoft* Disallow: / User-agent: Teoma* Disallow: / User-agent: The Intraformant* Disallow: / User-agent: TheNomad* Disallow: / User-agent: toCrawl/UrlDispatcher* Disallow: / User-agent: True_Robot* Disallow: / User-agent: turingos* Disallow: / User-agent: TurnitinBot* Disallow: / User-agent: Twiceler* Disallow: / User-agent: updated* Disallow: / User-agent: URL Control* Disallow: / User-agent: URLy Warning* Disallow: / User-agent: URL_Spider_Pro* Disallow: / User-agent: VCI* Disallow: / User-agent: Vortex* Disallow: / User-agent: voyager* Disallow: / User-agent: Web Image Collector* Disallow: / User-agent: WebAuto* Disallow: / User-agent: WebBandit* Disallow: / User-agent: webcollage* Disallow: / User-agent: WebCopier* Disallow: / User-agent: WebEnhancer* Disallow: / User-agent: WebIndexer* Disallow: / User-agent: WebmasterWorld* Disallow: / User-agent: WebPic* Disallow: / User-agent: WebSauger* Disallow: / User-agent: Website Quester* Disallow: / User-agent: Webster Pro* Disallow: / User-agent: WebStripper* Disallow: / User-agent: WebVac* Disallow: / User-agent: WebZip* Disallow: / User-agent: Wget* Disallow: / User-agent: WWW-Collector-E* Disallow: / User-agent: Xenu's* Disallow: / User-agent: xirq* Disallow: / User-agent: YahooSeeker-Testing* Disallow: / User-agent: Zeus* Disallow: / User-agent: ZyBorg* Disallow: /