October 2, 2005
What is search?
Maybe this is a silly question, but I want to follow this line of thought.
Let’s start with this answer: search is a textbox, into which users enter text queries in order to retrieve relevant results. Sounds kind of boring, doesn’t it? Not that there aren’t a lot of interesting challenges here: content relevancy factors, social network analysis, targeted relevancy based on submitter behavior, and so on. In fact, a lot of resources have been poured into improving upon the “search is a textbox” mode of operation, and a lot of really cool innovations have come from it. The result is a firm mental model among users that search == textbox.
What about tagging? Tagging, like search, is an explicit activity: both in tag creation, and tag browsing. It is a bit more compelling than plain “search is a textbox,” because it offers additional navigation elements that are closely linked with the content. Some problems that are intermediate or difficult in the “search is a textbox” model of full-text indexing–such as clustering, related content, and so on–become technically simpler with tagging, and integrate well with the existing tag browsing UI. Users also gain more benefit from tag systems if they have a relationship with the organization. So if you’re a tag system developer, are you developing a search system? Does it matter whether the back-end is primarily MySQL, or Lucene, or Berkely DB?
The next level “above” tagging is faceted navigation, especially where tags (or metatags) become facets. Faceted navigation merges tagging and textbox searching into one UI. One way of thinking about faceted navigation is as an interactive boolean query builder. Faceted systems make a ton of queries into the back-end to fulfill their promise of providing context-sensitive navigation, and benefit from efficient search algorithms. A faceted navigation developer is obviously a search developer, even if all the context is pre-generated (good luck with that strategy, though).
The context-senstive links in faceted navigation are, of course, primitive (or not-so-primitive, depending on POV) recommendations, which brings us to recommender systems. Recommender systems usually rely heavily on statistical analysis, but then so do more traditional search features such as targeted/personalized relevancy and clustering. Recommender systems can be entirely driven by profiles consisting of a small set of keywords based on past (or recent) behavior. Recommender systems can be used to recommend just about anything: news stories, user-generated content, fellow members in a community, editorially “programmed” content, advertisers.
But now we’re talking about more than a textbox with a submit button. We’re talking dynamically generated content, which might even look like ordinary content, with ordinary navigation choices. That’s the heart of personalization, isn’t it? Isn’t that the dream we’re all (users, producers, pundits) sharing? That can’t all be driven by something as mundane as “search,” can it?