Google Gods are Killing Me - PageRank 6 drops to PageRank 3

I decided perhaps it was just a fluke or bug in Google PageRank so I let some time go by. My PR6 still didn't return so then I decided to try the Google Webmaster Tools and a reinclusion request and see if Google reported any problems. Once again, nothing reported by Google. So once again I make some futile changes to my blog templates, which I know in my heart won't solve the problem, but it makes me feel better to try something... anything to get my glorious PR6 back. I wait a few months more and still PR3. Finally, I submit a reinclusion request again and this time Google tells me there is indeed a "problem": Here's what Google said:

Googlebot found an extremely high number of URLs on your site: http://blog.tmcnet.com/ July 20, 2008

Googlebot encountered problems while crawling your site http://blog.tmcnet.com/.

Googlebot encountered extremely large numbers of links on your site. This may indicate a problem with your site's URL structure. Googlebot may unnecessarily be crawling a large number of distinct URLs that point to identical or similar content, or crawling parts of your site that are not intended to be crawled by Googlebot. As a result Googlebot may consume much more bandwidth than necessary, or may be unable to completely index all of the content on your site.

More information about this issue

Here's a list of sample URLs with potential problems. However, this list may not include all problematic URLs on your site.

• http://blog.tmcnet.com/blog/tom-keating/images/trixbox-20-web-interface.jpg
• http://blog.tmcnet.com/green-blog/tag/apple+sdf+video+isdn+itexpo+soa+tdm
• http://blog.tmcnet.com/green-blog/tag/intel+environment+google+nec
• http://blog.tmcnet.com/blog/rich-tehrani/tag/sonus+3g
• http://blog.tmcnet.com/blog/rich-tehrani/tag/hostedcommunications+ipcommunications+att+ims
• More...

Google lists some old "tag" pages that used to work with a prior Movable Type plugin TMC used. Google points not just to my TMC blog, but several other TMC blogs which all still have higher PageRank than my blog. So if not returning 404s in the header is the reason for my low PR, why aren't all TMC blogs affected?

Until recently, I couldn't fix this issue, since we use Fastsearch, a 3rd party Movable Type plugin. I have no control over the search results it returns and it automatically handles missing/404 pages. Unfortunately, this plugin doesn't return a 404 error code for an invalid search - it always returns a 200 OK in the header. Nevertheless, many websites also don't return 404s for search queries. I knew that one popular blog ReadWriteWeb also uses Fastsearch so I decided to see if they have the same 404 issue for invalid queries. Well, they too don't report 404 errors. So in theory, I can create a massive HTML file with invalid outbound links to ReadWriteWeb, cause Google to index these "200 OK" pages which are all virtually identical and then eventually cause ReadWriteWeb a "Google penalty" for too many indexed pages.

Examples of ReadWriteWeb invalid pages that return 200 OK:
http://www.readwriteweb.com/tag/googlez
http://www.readwriteweb.com/tag/google+microsoft+voip+iphone
http://www.readwriteweb.com/tag/google+talk

Not sure if these pages are 200 or 404 in the header? Well, copy/paste any of these links to this online header checker tool.

The fake ReadWriteWeb pages I typed above all return 200 OK and they all use whitehat SEO'ed templates. So if you look at my screenshot, you'll see that ReadWriteWeb puts the tag search into the <title> tag (browser title bar), and in the body of the page. In theory, ReadWriteWeb could rank very well for these bogus pages I just made up once Google caches them:


I was skeptical that not returning 404 in the header was the cause for my PR drop, but decided to try and solve it anyway. So I spent about 15 hours reverse-engineering the PHP-based Fastsearch plugin to try and get it to report 404 errors for non-existent search results. Unfortunately, due to the templating mechanism built into Movable Type, it's not as simple as changing Fastsearch's PHP code to report 404 in the header for non-existent search results. I had to use Movable Type's SetVar command to set some variables and then "test" if no search results are returned and then use this PHP command in my dynamic Fastsearch template:

header("HTTP/1.1 200 OK");

This outputs a 404 in the header. It wasn't easy and required some serious Movable Type template tricks and conditional checks to get this to work.

So now these old (invalid) blog.tmcnet.com tag pages cached by Google now accurately report 404.  Since I'm reporting 404 errors, that should be good enough for the search engines to figure out not to index these pages. Alas, some time has past and still my PR hasn't returned.

On top of everything, I still can't access my damn Feedburner RSS feed account since the Google acquisition. I've tried everything, including this posting and this posting to the Google Groups, which is the "official" Feedburner support location.

I also emailed them the following:

To: 'aff-blog-feedback@google.com' Ever since Google acquired Feedburner, I haven't been able to access my account. The logon method uses Google accounts now, which is fine, but my Google account isn't "tied" to my existing feed.

When I logon using my Google account (ERASED@gmail.com) I don't have any feeds listed since it's a new account. I tried adding a new feed pointing to my blog's XML file (http://blog.tmcnet.com/blog/tom-keating/index.xml) and using the same Feedburner name (voipgadgets) but Feedburner tells me the feed already exists. I know it exists, but this feed is in the digital ether and I don't know how I can "claim" this feed as mine.

My feed is fortunately still updating (http://feeds.feedburner.com/voipgadgets/) but I have no way of accessing it to look at stats, add Adsense for Feeds, etc.

I don't want to change my feed name, since obviously many people are pointing to the current location.

I couldn't find any contact info on the Feedburner website to resolve. I tried the online help to no avail.

Please help.

I never heard back from my email nor my Google Groups posting. All this trouble with Google PageRank and Google Feedburner is enough to make me want to switch to Bing.com as my preferred search engine. I also recently considered a Google Android device, but their customer service really leaves something to be desired. Apparently, I am fated to live at PageRank 3 until the Google Gods decide I have had my just punishment. What the punishment is for is beyond me. I'd be happy to do a confessional ala Jimmy Swaggart style and confess my breaking of Google's 10 SEO Commandments if I knew which SEO Commandment I broke. Regardless of what sin I commited, let me pray for forgiveness... "Oh Great Google Gods, hear me now. Have mercy on this poor soul. Give us our daily PageRank... you can keep the daily bread since you've got enough bread to buy some countries, but I digress. Oh Great and Glorious Google, have pity on thee. For thou art a merciful search engine giant, are thee not? I will uninstall Yahoo! Toolbar, I will erase Bing from my Favorites, and I will use Google Talk instead of MSN Messenger. Deal?"