18. June 2010

  1. Added support for url's that have been entered without http://
  2. Made it so that when you select a bot/spider/crawler the different sections (disallow, allow, sitemap, etc) are collapsed and expandable. Hopefully this will give a better overview about the desired behaviour of a specific bot.

17. June 2010

  1. Tool officially launched