I was drawn to this session because we're big advocates of the semantic web, particularly focused on making sure that we implement client sites in semantically-correct HTML with embedded data like microformats. I've also had a long-standing interest in Natural Language Processing, both out of general curiousity and as a user interface mechanism. It's one of those technologies, like virtual reality and voice recognition, which has been just around the corner for the last twenty bends in the road, so I was really curious to hear what the panelists had to say and about how close they are to releasing products based on it.

It was a great session with an excellent conversation between the panelists, who were all very knowledgeable about the topic. Panels often degenerate into promos for the represented companies but the hard (and soft) sells were kept to a miminum. A quick summary of the most salient points (raw notes follow below):

    
• The much-maligned labels 'Web1.0', 'Web2.0', and 'Web3.0' could really just refer to decades (1990-1999, 2000-2009, 2010-2019 respectively) in which the priorities of the web community shifted from inventing the backend of the web to improving the user eperience and frontend, to returning to the backend and building in intelligence in the form of semantic meaning (Nova). Think of it as the shift from plumbing to meaning, over a thirty year period (Barney).
    
• The web to date has been focused on producing content for human consumption rather than on content for machine consumption and processing (Barney). The web needs to move from being a file system to a knowledge system in order for humans to be able to more efficiently extract meaning from it (Nova).
    
• The semantic web is going to require humans to add all of the semantic information, which is where search engines like Mahalo are going (search results augmented with human-created information). Radar Networks is taking this approach by building community networks which help people add and augment the semantic understanding of the web (Nova). Sometimes people aren't very good at adding that information because of the complexity of the context required (e.g.: the difficulty is in things like email messages which just say "Yeah! Let's do it!", which could mean "Let's go for lunch!" or "Let's commit securities fraud!" but which can't be understood without the context of the related emails in the thread). Humans are only right about categorization about 60% of the time but Cataphora's software has been independently evaluated as being 99.6% accurate (Elizabeth).
    
• There's an element to the semantic web which is heavily based on the concept that the work one human does can have benefit for another human, and that giving your time in one place will save you time in another (e.g.: tagging, Wikipedia, etc.). This is the begining of the semantic web and future search software (like Powerset) will need to understand the meaning that we are now creating (Barney).
    
• One of the really interesting things in this space is finding the impossible problems and figuring out how to solve them. The question of "who said a specific thing" is really hard but could be a killer app for people like journalists (Barney).
    
• The basic technology at play here is nothing new and has been around for decades.     
• Trust is a big factor affecting the success of all of these systems. It's really hard to assess how much you trust a certain source and to have that trust affect the weighting of the graph for specific knowledge, especially when people will publish information which intentionally distorts the truth (Nova). It becomes even more difficult when you consider that some people will be highly trustful about certain topics and wrong (sometimes intentionally) about others, so there's no such thing as a single trust metric per source (Elizabeth).
    
• There's a difference between trust and authoritativeness, around which there is a bunch of research into hubs and authorities. This is the area that Google's PageRank is based on and is known as Bibliometrics. A measure of authoritativeness is not always the same as a measure of trust - you might trust your friend's opinion more than an authority because of your proximity to him or her (Nova). Sometimes authoritativeness is the opposite of what you might expect in that the document seen by more people might be less authoritative than the one seen only by a few (Elizabeth).
    
• General Magic had a vision for a semantic web before the web even existed, using a collection of autonomous agents to go out into the ether and do work for you, reporting back when they were done. Tim Berners-Lee's vision of the semantic web (Semantic Web Roadmap) was basically a restatement of the same vision. The more intelligence you add to the web (in the form of metadata), the less intelligent the agents need to be. Right now, we're more at a level where the agents aren't autonomous and are more like processes which run outside of your server, like the GoogleBot. We're probably 10 - 15 years away from truly autonomous agents (Nova). That said, we don't need them to be autonomous to be useful. We should expect the tools we use on a daily basis to get smarter about who we are and what relationships we have, as well as to understand richer inputs like natural language, in the next few years (Barney).
    
• AT&T deployed a natural language voice recognition IVR some years ago which understood only five words but which saved them hundreds of millions of dollars by directing service calls automatically instead of relying on human operators (who all lost their jobs). There was a belief in the 60s that the economy always creates more jobs than it destroys, so where will the new jobs be created (John)? Some of the call centers that moved to India are moving back because it turned out to be too difficult to train Indians to look, sound, and feel like Americans (Elizabeth). Humans do a much better job of adding semantic meaning to data, so the more data we acquire, the more knowledge coders we need (Nova).  Also, keep in mind that machines don't always replace humans but rather change the interaction. ATMs replaced some tellers but also allowed us to take out money at 3am, which didn't take away any jobs (Barney).
    
• The semantic web opens up whole cans of privacy issues that haven't been dealt yet with, not least of which is determining which conclusions you can share when some of them have been derived by traversing data sets which are private (Nova).
    
• There are a bunch of efforts underway to make it easier to share the underlying data behind the semantic web, both in the form of things like RDF which make it easier to combine disparate data sources by mashing text instead of re-calculating relationships (Nova) and in developing UDBRIs in addition to our existing URIs to identifiy and open up datasources (Barney).
    
• Sometimes the real meaning of things is found in what's not there. Cataphora bases part of their analysis on looking at things like people switching languages outside of a pattern in an attempt to hide from traditional search engines or English speaking searchers (e.g.: if you always write to your grandmother in German, it will ignore emails to her in that language but it will flag an email to a colleague in which you switch languages against the trend).

Full session notes, in a strange plain text/HTML format, are available at: Supernova2007-MakingComputersSmart.