After I came up with the idea to log web application hits using jQuery, to my great surprise I found that Googlebot actually performs POSTs implemented as jQuery $.ajax() calls:
2014-01-15 09:46:04 POST /Log - - 18.104.22.168 Mozilla/5.0+(compatible;+Googlebot/2.1;++http://www.google.com/bot.html) - 200 0 0 255
Searching the Interwebs I found other people who observed this behavior, too:
- Does googlebot crawl urls in jQuery $.get() calls and can it be prevented?
- Prevent Googlebot from running a function
- Googlebot doesn’t see jquery generated content
Most importantly, the links in Wikipedia’s Googlebot article analyze the bot’s behavior in more detail:
- Googlebot makes POST requests via AJAX
- Googlebot Is Chrome
- Google… Stop Playing; The Jig Is Still Up!
The articles are about 2 years old, so the bot may now be even more capable than then.
Of course, the simplest solution to prevent bots from POSTing is to add the logger’s URL to robots.txt:
User-agent: * Disallow: /Log