Algorithmic Search and Discovery


When Rich Skrenta who has been a pioneer in Search with Topix (and just launched Blekko) writes the following, I take notice:

“Algorithmic search is sinking.

The only way to combat this and return trust and quality to search is by taking an editorial stand and having humans identify the best sites for every category. The algorithm can't find its way through the web's growing hall of mirrors anymore. And it's only going to get worse.”

However, Rich is only referring to one set of algos that are sinking. On the paid side algos are certainly rising, not sinking. A HUGE number of transactional queries end up clicking the ads – that are often more relevant (due to a/b testing of headlines and the like) and more useful than the underlying organic results.

Paid search aside, I take issue with Rich's solution of human editors/curation. I'd like to put the context of the problem in a different light. That of the multitude of wonderful mid/small retailers out there that people should not be afraid to buy from.

This has been the case since day 1 of the web. In fact, a few pretty nice sized web businesses have been built on the backs of these types of retailers. If we look at the current/latest major Google algo changes (May Day et al) the SERPs (once again) have skewed towards the Amazon's and Wal-marts of the world. This is the gentrification of the SERPs and I think it's a bad thing for Search.

At its core Search is about discovery. We should not be solving Search's problems by placing limits on our ability to discover especially if the solution – human curation – can be just as harmful and limiting in this regard as any algorithm. In fact, I'd argue human curation will be more limiting to discovery than Algorithmic Search.

Don’t think this type of human curation and editorial controls does not extend to content discovery the same way that it does commerce discovery. We’ve seen a great rise in curated content streams and it has an important place. However, any of us that have used Search to find out about information have surely stumbled onto helpful, useful and important blogs or other articles, essays, papers that we NEVER would have been exposed to without the algorithm.

The answer is not either or. The answer is both. The more tools we have on this web for information discovery the more we can discover and the better that experience can be. Yes, we’re navigating rough waters but the algorithm should not and will not end up as a wreck on the ocean floor, but rather a vessel that can and will help us discover new lands.





, , ,



4 responses to “Algorithmic Search and Discovery”

  1. wholesale designer clothing Avatar

    The more tools we have on this web for information discovery the more we can discover and the better that experience can be. Yes, we’re navigating rough waters. I think it’s a good idea.


  2. Ashish Avatar

    If we cannot write unbiased code, we sure cannot rate or review without making personally skewed decisions.
    The loopholes in those algorithms will always remain, it is just that an issue or two make them look bigger and uglier.


  3. Application Development Avatar

    Are you talking about google’s algorithm..;-) Joking!! It’s same searching something in Google and Sea. Both like same that we don’t know what is to be next!!


  4. AJ Kohn Avatar

    I couldn’t agree more. I’m catching up on my RSS reading before writing some blog posts and you’re on the same path as I.
    ‘The gentrification of SERPs’ is a great turn of phrase and is true from where I stand. A mid-level eCommerce client was shuffled back in the recent (October) algorithm changes. Brand names now dominate and I find it difficult to believe the user was better served.
    Regarding curation, I hadn’t thought about the limiting nature of curation – that the human mind is likely to be more biased and unable to know about all of the relevant content.
    I was thinking more about the entropy of that curated content. How long does someone keep that curated set of information current? Are they reviewing it periodically to see if some has gotten better and some worse? Has some of it been removed and has new content been introduced?
    Collective curation may help the latter, but the former (human limitations) could actually be magnified as only the popular become visible.
    I’m more disturbed than ever by the current algorithm, but like you, I think human feedback (call it curation if you want) simply needs a greater voice in that algorithm.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Create a website or blog at

%d bloggers like this: