The Oxford Knowledge Company

If you thought you were just getting a grip on the whole web searching/ search engine registration issue, think again. Technology is being developed which insiders believe may have the ability to turn the whole searching model on its head.

 

For most people in the pharmaceutical industry, the ins and outs of web searching will have two main applications; to help you find information (which if you enjoy the benefits of a large organisation, you may still find is best and most reliably done by phoning the medical information department), and as one of the means by which you promote your Web-based marketing campaigns.

What if search engines became irrelevant?

If search engines didn't have such a phenomenally hard task we'd all hate them. As it is, to be able to make any sense out of the 1 billion randomly-filed, uncensored collection of pages that comprise the World-Wide-Web is something that for which the majority of Web users are very grateful. That doesn't mean we wouldn't like a better way to find information. And if you have anything to do with promoting a Web site to search engines you may have shared the frustration of many of your peers, that most search engines won't index your new Web pages for days, maybe weeks, and even then the content to which searchers are referred will be a couple of weeks out of date.

You may have heard of Napster, the MP3 file searching application that has turned the music industry on its head and is the subject of a copyright infringement suit. What Napster and its ilk herald is a whole new way to search for information, one that could theoretically do away with the need for search engines like Google and Altavista - provided that all Web publishers cooperate and actively share their hard drives with the rest of the world. And there's the rub.

Napster, and the more recent Gnutella are file-searching programs which work in a fundamentally different way to 'traditional' search engines. Google et al. use programs ('spiders'), which crawl all over the Web, deconstructing Web pages and recording their findings in their search index. When you search for a page in Google, what you locate is Google's reference to that page, and whatever Google found last time its crawler visited that page. With Gnutella, your search query is sent out to a network of computers that have been volunteered to participate, works its way around the network until it finds some information to satisfy your query upon which the results returned to you.

Gene Kan, one of the original developers of Gnutella and a team at a new start-up Gonesilent.com have gone on to develop new Internet searching technology known as Infrasearch. Instead of relying on a central server-based index of the Web as its first port of call, sites that agree to be part of the network simply run a piece of software inside their databases, and share the very latest search results, with people searching for it via the Infrasearch site.

For example, you might search for the latest news on "health economics" using InfraSearch about and if InPharm.com were running the Infrasearch search agent in its Web content database, you could immediately identify all of the stories about "health economics" that had been written on InPharm.com up to that minute. Real-time searching!

It is an old-fashioned idea in Internet terms - the idea that you share out the workload of processing across a network of computers. Known collectively as Peer-to-Peer software, companies developing P2P software such as Gonesilent and Applesoup are being talked about as the Next Big Thing. The exact business model is something no-one seems worried about at the moment - but when Marc Andreesen, one of the founders of Netscape invests his own cash and describes this as something that will do "for search what the Internet did for communications," you can understand why venture capitalists have become interested.

What might it mean to us, the regular users of the Internet, searchers of information and publishers of content?

This raises quite a few questions. In a world where centralised search engines aren't so important, it is likely that we'd have to learn new tricks to promote Web sites. Why invest in expensive search technology for your own site when the Infrasearch engine inside your database takes care of everything? On the other hand, how fast will such an engine really be when it has to go at the pace of some of the slower computers hooked into the network?. More importantly, how crucial a resource would something like Infrasearch ever become if enough people don't play along?

Unlike 'traditional' search engines that will list even pages whose authors don't announce the pages' presence to the search engines, Infrasearch only works when enough people 'buy in' to the concept. And here, "buy-in" means the whole bit; share and be truthful about your information - all the time. Did you ever hear about the practice of "Napster bombing"? This is where people mislabel their MP3 files to mislead people downloading files, so that you might receive porn when you were searching for something by the Spice Girls.

It is the same principle as search engine spamming. You can't completely stop this, but 'traditional' search engines, because they have ultimate control over their precious search index, can do a lot to mitigate the problem. It is hard to see how a decentralised search like Infrasearch could do the same. Imagine the opportunity to misrepresent something to increase sales or drive advertising. People lie; it's in our nature.

Larry Page, CEO of the search world's latest star, Google, is apparently not worried about the threat posed by this type of technology. I have to say that I'm with him here. I love what the Net can do and what it already has done for information sharing and communication. Nothing in business works properly if it completely requires everyone to play nicely with the other kids and eventually things only work well when the people running the system make money. Who, apart from the taxman, ever made a lot of money from getting people to behave?