New Search Engine Scoopler Searches in Real Time

As instantaneous as the Web can be, most search engines--Google included--suffer from a bit of a time lag. The search engines can only link to pages and sites that the search engines have scraped with their Web crawlers or spiders and other indexing tools. This means that the latest entries on the quick-changing-content sites like Twitter or Digg won't be searchable via Google or other similar search engines until well after the content has already been replaced with even newer entries.

Where Google leaves off, the new search engine, Scoopler looks to pick up. Scoopler is a search engine that indexes the content of a number of popular social-networking sites, and delivers real time search results:

"Scoopler is a real-time search engine. We aggregate and organize content being shared on the internet as it happens, like eye-witness reports of breaking news, photos and videos from big events, and links to the hottest memes of the day. We do this by constantly indexing live updates from services including Twitter, Flickr, Digg, Delicious and more. When you search for a topic on Scoopler, we give you the most relevant results, updated in real-time."

We put this to the test and searched for one of the most popular search topics of the day: "Star Trek." Most of results were from entries that were less than 1 minute old--in fact, one of the entries that turned up in our search was only 24 seconds old (as reported by Scoopler). Scoopler also continues to update the results in real time, so as new results become available they automatically appear on the top of the page and older results scroll down.

The overwhelming majority of search results for our "Star Trek" search were from Twitter, with about one new entry appearing about every five seconds. Considering the recent meteoric growth spurt and popularity of Twitter combined with yesterday's release of the geek-friendly Star Trek film, it is not surprising that Twitter is generating so much of the real-time traffic on this topic. The occasional entry from other sites, such as Digg, sprang up periodically. We tried a different search for an equally time-sensitive event, but with perhaps less "geekness" attached to it: "Mother's Day." The "Mother's Day" search results were not coming in quite as quickly as the "Star Trek" search results, but they still managed to average being about only 1 minute old (as reported by Scoopler) and surprisingly still seemed to all come from Twitter. Scoopler was starting to look to us like a real-time Twitter search engine. Finally, when we searched for "Knitting," the results were far fewer and far between that the results from our other searches, and these results came from sites such as Delicious and Flickr (but there still were some Twitter results as well).

The Scoopler search results page is broken up into three columns. The left most-column displays your three-most-recent searches as well as the current "Hot Topics." The middle column displays the scrolling, real-time search results. If a particular search result includes an entry that links out to additional content, such as a site or video, a "Peek" button appears, which allows you to preview the content in a "preview window," without needing to open a new browser tab or window. The right column shows "Popular" results relevant to your search, based on "All Content," "Videos," "Links," and "Images."

Scoopler was founded by software engineer, AJ Asver, and programmer, Dilan Jayawardane. The site has been in development for almost a year, and it launched as a private beta at the end of March 2009. Scoopler is now available for all to use and try out as a public beta: www.scoopler.com. Give it a try and tell us what you think in the comments section below.