User-agent: * Disallow: /qwertyuiop-addresses/ Disallow: /cgi-bin/ Disallow: /tmp/ Disallow: /manual/ Disallow: /private/ Disallow: /usage/ # # Just by reading this file you've comitted yourself to obeying the rules # here. Do not visit any of the directories below or your IP address will # be blackholed and unpleasant things will happen to you. # # # This directory is here to bait robots which read this file just to locate # valuable protected directories to surf. Since we know they read this file, # and we know from parsing this file that that directory is off limits and # they should know better, they get banned. # #Disallow: /robot-trap # # This directory is referenced in hidden links around the site. Robots which # don't read these rules will surf into that directory and get caught. By # marking it disallowed here we make sure that good spiders know not to fall # for the trap. # #Disallow: /honeypot-trap # # Since so many hacked NT/IIS boxes are connecting to my site, I might as # well make them usefull as test subjects. These directories all lead to # tarpits. #Disallow: /scripts/ #Disallow: /MSADC/ #Disallow: /msadc/ #Disallow: /_vti_bin/ #Disallow: /_mem_bin/ #Disallow: /c/ #Disallow: /d/