Google AdSense Servers Tripling My Traffic?
Posted by Brad Wood
Nov 09, 2009 07:15:00 UTC
I threw some AdSense ads on my site the other day. I don't think I'll be getting rich any time soon though. I'm up to 58 cents so I figure I'll be seeing a check from Google in about 8 years. :) Anyway, while troubleshooting an error in my blogCFC I noticed I kept getting each error E-mail three times. Looking at the User Agent showed that "Mediapartners-Google" was hitting every URL I hit twice. Incidentally, I have two Google ads on every page of my site.I fired up MS Fiddler to confirm the extra requests weren't coming from my browser but they weren't. The IP address confirmed this. So apparently, for every ad on your site, Google hits your page real quick from their servers-- presumably to look for keywords in order to display the ads. I'm not sure if the extra hits to my site are really happening all day long for everyone. (A quick look at my Apache access log was inconclusive) One would think the great Google Mind would be smart enough to not hit the exact same URL twice in a row. And why do they even need to hit it at all when their search engine already has cached versions of all my pages? Hmm, maybe they were only hitting that particular page since I was making up a long random URL to cause the error. So, I guess THAT page wouldn't have been cached, but Google still hit that URL about 10 times in a row even though its contents wasn't changing. Overall I'm not too concerned about a little more traffic on a mostly-sleepy VPS, but what if the pages in question are very intense or incur server-side changes? Hmm, note to self-- Try this with a POST as well as a GET. Has anyone else noticed this. (And has anyone else made more than 58 cents so far? :)
The AdSense crawler is separate from the GoogleBot crawler, so I'd say that if you've just put ads on the site and the AdSense crawler hadn't had a chance to look at your site it would have needed to hit the page immediately on a page view (hitting it twice seems a little strange) so that it could put something in its index. After the initial hit it should just crawl and index your site every so often rather than as pages are being viewed.
@Justin: That may well be. it needs to index a little better though. Right now I'm looking at ads for workout equipment, the revolutionary war, Yahoo.com (in a Google ad??), wine, and recycling. I can't say any of those have much to do with a tech blog. Oh well. :)