Twitter is planning to improve its search engine in a radical way. Pandia argues that Google and Twitter need each other.
Twitter has many suitors, among them Apple and Microsoft. Still, regardless of how much we admire Apple, the match that really makes sense to us is the one between Twitter and Google.
The new Twitter search engine
This week Twitter’s Santosh Jayaram, previously manager of search quality operations for Google (GOOG), announced that the Twitter search engine will start indexing the content of pages included in tweets.
In practice this means that you can use the Twitter search engine not only to search the content of the microblog tweets themselves, but also the content of the articles people link to and recommend.
Twitter as a real-time intelligence source
This makes perfect sense, as the latest development has shown that people are using Twitter more and more as an intelligence tool, for following the latest trends and the hottest news, rather than for mindless “I am going to the store now” kind of chatter.
As Rafe Needleman of cnet observes:
“This will make Twitter Search a much more complete index of what’s happening in real time on the Web and make it an even more credible competitor to Google Search for people looking for very timely content.”
Twitter may strengthen Google search
Indeed! But this also demonstrates why a Google/Twitter marriage will make so much sense:
Google’s challenge (well, any search engine’s challenge) is to identify the most relevant and best quality pages for any query. In news search it is also important to find the latest happenings a.s.a.p.
At the moment Google uses three ways of doing this:
- For regular web search it is revisiting “important” and frequently updated sites more often than ever. Some are revisited on a daily basis, and popular sites that are know to present important news even more often.
- This also applies to the Google News search service. Google News is spidering a selected set of news sites on a daily basis. This set includes news sites like The New York Times and the Observer, but also smaller sites like Pandia.
- Google Blog search used to index the RSS web feeds of blogs on a regular basis (several times a day for productive blogs). Now it also indexes the content of the pages the RSS feed is pointing to.
Given that the regular Google search engine combines results from all of the above, Google does manage to capture much of what is happening right now.
The problem of identifying the best current news
But in some cases this is not good enough.
For regular, “old”, web search results, Google may rely on a network of links to determine the quality of a news article or blog post. The more inbound links from high quality, authoritative, sites, the more likely it is that an article is relevant and of a sufficient quality.
For news that happens right now, however, a reliable link structure has not been established, or it is fairly unreliable. Moreover, Google needs time to analyse this network of links.
This means that Google needs other kinds of input to tell what article or blog post is the best one right now for a particular query.
Twitter may provide that input, as the tweets give real time information about what the twitterati think is the most interesting stuff on the web this very minute.
Twitter doing it on its own
Now it may seem that Twitter is thinking of doing this on its own, turning Twitter search into a competitor to Google News and Google Blog Search.
If that works out and Twitter manages to develop a good enough search algorithm, Twitter Search will become a social search engine for current news.
Why Twitter needs Google
This is a big if, however. Twitter does not, for instance, have the same experience in combating spam as Google does, and spam will become a huge problem when black hat search engine marketers realize that people are using Twitter Search as a regular search engine.
They may set up a large number of interconnected “fake” Twitter accounts, sending out tweets linking to sites selling particular products or services.
Someone searching for Obama’s trip to Pakistan may soon find themselves on sites selling illustrated literature of the more unsavory kind.
Popular twitterers will also be encouraged to include endorsements and links to particular pages for a certain fee, and Twitter need to be able to distinguish between paid and regular links.
Combating spam
Twitter is already thinking of ways to ensure the quality of search results.
Jayaram says that when you do a search on a “trending” topic (a topic that is so big it gets its own link in the Twitter.com sidebar), Twitter will take into account the reputation of the person who wrote each tweet and rank search results in part based on that.
In other words: Twitter will boost pages that are linked to from persons who have a lot of followers and retweets.
Twitter knows that it is easy to generate a large number of followers on Twitter, and already has mechanisms in place that stop people from getting too many just by befriending others.
At the moment there is a limit of 2000 friends. If you have more than that, you will not be allowed to follow more people. This means that any twitterer that has more than 2000 followers have gained them because of his or her reputation, not because he or she has followed people found on the friend’s lists of other related twitteres.
Will Google acquire Twitter?
At the moment nobody seems to know. Twitter founder Biz Stone said Wednesday that Twitter is not for sale.
Hitwise reports that U.S. visits to Twitter Search have increased 514% from January to April, and given the present PR blizzard surrounding the brand, that increase will continue for quite a while.
Maybe Twitter reckons it has a product it can develop on its own, without Google.
Still, even such a strategy may leave room for Google. One important reason for focusing on search, is that Twitter need a revenue stream. The best way of doing that is to add text ads to search results, and Google remains the champion of pay per click search.
What about Apple and Microsoft?
There have also been a lot of rumors about Apple and Microsoft buying Twitter.
Microsoft desperately needs something that sets it apart from Google in the search engine field, and buying Twitter may give the company the advantage it needs.
However, Google has more to offer the Twitter engineers than Microsoft, which means that Microsoft will have to give them a lot of money, indeed, to compensate for that.
We find it hard to see why Apple should be interested in Twitter. Apple has no search engine of its own (unless you count iTunes as a search engine), so unless they want to become a competitor to Google a Twitter acquisition makes little sense.
And why should Apple want to compete with Google in the search field? Apple do well with their combination of beautiful hardware and functional software, but their attempts at developing services in the cloud (Mobile Me) have not been very successful. Nor do they have any social web sites to speak of. This isn’t Apple’s turf and they know it.
See also: Microsoft Must Buy Twitter (Silicon Alley Insider)
Follow Pandia at Twitter: @pandianews

Outsource SEO/PPC
$11 - $15 per hour

Original post by Per and Susanne Koch