One of the criticisms of modern life that I find fascinating is information overload. We all tend to be searching for something, and yet despite the overabundance of options set forth on the heaping table of life, many cannot find what truly sates their appetites. Some of this difficulty derives from not knowing what we really need, and some from just an inability to process so many options. Cable television is the classic cliche for this information explosion.

So is seems only natural that the solution to this dilemma is a facilitation of the “search” process. We see this in self-help books, t.v. guides, and various other traditional analog search tools. Of course, another criticism of modern life is its unquenchable thirst for speedy results. Long gone are the days when 6 to 8 weeks, or even just one week, was an acceptable period for delivery of a package, and now instant messaging is even replacing e-mail because of our need for instant gratification. This rapid turnaround has always been appreciated in the business world, as time is money, but just how compatible with the mundane realities of life can this process become? FedEx is nice for getting birthday gifts to loved ones within a day’s time, but can there ever be the equivalent of FedEx for delivering customized, detailed answers to business and personal inquiries? “Get your answer in under a minute or your next one is free!” seems to pop into my mind. The concept of the search engine tried to fill this niche, but as anyone who has used Google before knows, what you want only occasionally pops up on the first sweep. Language is just too ambiguous most of the time.

The initial promise of the digital revolution was fast and efficient access to all sorts of information. Indeed, information itself became capital, it became the building block of industries. However, to the disappointment and stark reality of many, our world is not always suited for easy and rapid digitization. Some fields, like radio, fit snugly into a new digital format, but the real nuances of life, slang, connotation, inferred references, and “reading between the lines” just don’t readily conform to digital processing. Put some XML tags on these things, and yes, perhaps the data can be manipulated, but what is really needed is a tool that “tags” real life, a tool that captures connotations without the user having to mark up his or her e-mail, or work report, or movie review. This in simplified terms is the goal of IBM’s new WebFountain tool aimed at breaching the barrier of machine understanding.

Whoever successfully bridges that divide will indeed be counted among the next victors of the digital age. The OS battles, even the battles for control over access and navigation of the Internet itself, are over. The real fight remains over the “digital high ground”, that ability to look out over the entire Internet, a digitized landscape of ideas and rumors and wonderful data hidden in the weeds, and see where there are breaks in the lines, and where opportunities exist to be acted upon. So far, few have been able to commit the needed resources to take the digital high ground. IBM seems to be near the verge of success, with beta testing of its WebFountain tool in progress, but even Big Blue admits that it faced huge monetary and server demands to get to its current level.

This is an intriguing aspect of the whole battle for domination of the “search” field in the next decade. Is it really just a huge math problem, the sorting and manipulating of unfathomable amounts of data gleaned daily from the ever expanding Internet and stored on servers, or is there really some algorithm that can accurately mimic human intuition and thought processes? In either case, majority control of such a tool in the hands of IBM smacks of the potential for sci-fi levels of societal control. People are worried that Google and its methods for ranking websites have shackled the true value of the Internet. Grokker2 and Nutch are examples of those opposed to search engine monopolies, but this process of structuring previously unordered heaps of information available on the Internet poses serious challenges not just to the very infrastructure of the net itself, but of how we begin to blend the digital and the analog.

When the laws of the digital realm start influencing the methods with which humans operate on a regular basis, then we know that a new revolution is on. Just as the very concept of “windows” and “folders” and “mousing” forever transformed the collective concept of what it means to use a computer, so too will the process of enabling computers to search for information in a manner consistent with human thought patterns forever change our concept of “searching”. WebFountain and its promise of facile collection and analysis of unstructured data is one step closer to that goal of being able to ask in plain English a question to our computers and receive an answer that we would expect from another human.

I can’t help but remember a prescient scene from Snow White starting, “Mirror, mirror, on the wall…”.