Is ‘Big Daddy’ Choking Google?
May 9, 2006
By Ben Charny
Web site operators are clamoring to understand what can best be described as an ongoing disturbance in the Google Force.
Google’s search engine, once a clean, lean indexing machine, from a Webmaster’s perspective has been slipping badly lately.
Starting about two months ago, site operators have complained that their Web sites have suddenly disappeared from Google’s index for no reason, tantamount to disappearing from the Internet.
Another common gripe is how a Web link mysteriously drops from a prominent place in search results to Page 43 or so. The list of gripes goes on.
Something is happening, although Google hasn’t said what it is yet and likely won’t on its annual Press Day, which is scheduled for May 10. Google usually makes a number of impressive announcements at this event. This year, there’s some “product demos” in the offing, it promises.
But in the backdrop of all the shiny new Google stuff is the 800-pound gorilla of a question: What’s going on? Many people suggest that Big Daddy’s to blame.
Big Daddy is, in effect, a brand-new data center that Google uses to perform core search engine tasks like cataloging Web sites or serving up localized features. Google operates thousands of data centers across the globe.
The Big Daddy project stands out because search engines infrequently upgrade the computing and networking hardware. Instead, the focus is on new features to lure more site visitors, which translates to more search engine revenues.
By upgrading, Google can better compete against rivals Yahoo, the world’s leading Internet destination but second to Google in search, and Microsoft, whose online MSN unit manages the world’s No. 3 search engine.
The ultimate goal is a bigger slice of the $13.8 billion expected to be spent on Internet advertising in 2006, and even more in years going forward.
Every search engine is constantly tinkering, or buying new companies and absorbing their intellectual property. It’s all to find new ways of attracting an ever bigger audience, which translates into higher advertising revenues.
Read more here about other Big Daddy sightings.
But as all the Big Daddy disruptions seem to indicate, there’s always growing pains as companies sharpen a competitive edge.
As screen-name JohnW writes in a SearchEngineWatch forum: Google “rolled out some new technology to fix some things, and in the process they seem to have broken some other things. There isn’t much to say about it other than it’s broken.it is what it is and will be over when it’s over.”
Because of Big Daddy, the theory goes, Google must completely erase the databases where it keeps its 8 billion-plus Web pages. Then it will add Big Daddy and kick-start the indexing.
Next Page: Big Daddy comes knocking.
That explains why sites disappear, for instance, a few days or weeks at a time. It’s Big Daddy knocking, say several posters at this search industry forum (password and username are required).
Another theory about the disruptions is based on comments about a Google “server crisis” from Google CEO Eric Schmidt to the New York Times. From those words comes the idea Google’s plumb out of data storage. That too explains the indexing issues.
There’s lots of speculation, and some quasi-confirmation from a prominent Google engineer, but Google didn’t immediately respond to an e-mail seeking comment on the topic.
Much of what’s known about Big Daddy comes from Google engineer Matt Cutts, who occasionally writes about the topic on his blog. A Google spokeswoman referred a reporter to Cutts’ blog in response to an inquiry about the recent problems.
At the Cutts blog, there’s an oft-cited March 27 Cutts-agram sent at the height of the initial wave of complaints. In it, Cutts claims Big Daddy’s been fully deployed and he mentions how it will “stabilize.”
He also warns that sites finding themselves off Google’s reservation can expect it to take just a few days to show up in the index again.
Meanwhile, at Press Day, the worst-kept secret has been about the debut of Google Health, one of dozens of Google-operated search engines focused on narrower topics.
Read more here about other likely new Google search silos.
A Google spokesperson confirmed that the company is testing the feature.
“We have been doing a variety of research in the health area, including how to improve the quality of health-related search results,” the spokesperson wrote in an e-mail. “We have nothing new to announce at this time.”
There’s also suspicion Google will unveil some more items from its partnership with computer manufacturer Dell Computers, based in Round Rock, Texas.
One dark horse rumor candidate is that Google will unveil a more sophisticated Internet phone service for its Google Talk instant message system. VOIP (voice over IP) is de rigueur for IM systems, yet Google Talk lags far behind competitors.
Microsoft’s MSN recently unveiled a new version with deeper phone capabilities, and Yahoo, of Sunnyvale, Calif., is also more and more phone-focused with each upgrade, so there’s certainly competitive pressures to do so.
Check out eWEEK.com’s Search Center for the latest news, views and analysis on enterprise search technology.
