October 2, 2005

What is search?

Posted in Personalization, Search at 10:36 pm by mj

Maybe this is a silly question, but I want to follow this line of thought.

Let’s start with this answer: search is a textbox, into which users enter text queries in order to retrieve relevant results. Sounds kind of boring, doesn’t it? Not that there aren’t a lot of interesting challenges here: content relevancy factors, social network analysis, targeted relevancy based on submitter behavior, and so on. In fact, a lot of resources have been poured into improving upon the “search is a textbox” mode of operation, and a lot of really cool innovations have come from it. The result is a firm mental model among users that search == textbox.

What about tagging? Tagging, like search, is an explicit activity: both in tag creation, and tag browsing. It is a bit more compelling than plain “search is a textbox,” because it offers additional navigation elements that are closely linked with the content. Some problems that are intermediate or difficult in the “search is a textbox” model of full-text indexing–such as clustering, related content, and so on–become technically simpler with tagging, and integrate well with the existing tag browsing UI. Users also gain more benefit from tag systems if they have a relationship with the organization. So if you’re a tag system developer, are you developing a search system? Does it matter whether the back-end is primarily MySQL, or Lucene, or Berkely DB?

The next level “above” tagging is faceted navigation, especially where tags (or metatags) become facets. Faceted navigation merges tagging and textbox searching into one UI. One way of thinking about faceted navigation is as an interactive boolean query builder. Faceted systems make a ton of queries into the back-end to fulfill their promise of providing context-sensitive navigation, and benefit from efficient search algorithms. A faceted navigation developer is obviously a search developer, even if all the context is pre-generated (good luck with that strategy, though).

The context-senstive links in faceted navigation are, of course, primitive (or not-so-primitive, depending on POV) recommendations, which brings us to recommender systems. Recommender systems usually rely heavily on statistical analysis, but then so do more traditional search features such as targeted/personalized relevancy and clustering. Recommender systems can be entirely driven by profiles consisting of a small set of keywords based on past (or recent) behavior. Recommender systems can be used to recommend just about anything: news stories, user-generated content, fellow members in a community, editorially “programmed” content, advertisers.

But now we’re talking about more than a textbox with a submit button. We’re talking dynamically generated content, which might even look like ordinary content, with ordinary navigation choices. That’s the heart of personalization, isn’t it? Isn’t that the dream we’re all (users, producers, pundits) sharing? That can’t all be driven by something as mundane as “search,” can it?



  1. […] I love tagging, not least of which because it simplifies some problems in information retrieval and provides a conveniently compelling user interface (see my previous entry What is search? for a bit more on my thoughts there). I also think it’s clear that Flickr is further along the “second generation” social networking journey than Webshots (see this previous entry on communities and social networks). These aspects account for Flickr’s amazing growth as much as the exploding market demand. (I do share Narendra’s skepticism that their APIs have a lot to do with it so far, though.) […]

  2. […] When the story originally broke, my response was to shrug it off. Yahoo’s acquisitions and newer experiments demonstrate that it’s headed in a different direction than Google, a direction in which Search-As-A-Textbook is less important than search as a platform, or, to put it another way, search as an anchor for community. […]

  3. […] If you know me, it’s this 4th (and possibly 5th, see my notes below) generation that I’m always harping about. I wrote a bit about the future of search last year. I make a bit of an ass about it sometimes, but it turns me on. […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: