Dit programma opent deuren die anders gesloten blijven omdat het zich voordoet als de Google- of de Yahoo bot.
[...] When Google visits the site, it gives something called a "Header". This header tells the site who the visitor is. Google's header is "Googlebot". The programmers of the site check to see if the header says "Googlebot", and if it does, it opens up all of its content for only googles eyes. [...]
BeTheBot
Technorati Tags:
