Killer AP? Killer app?

Rumors of Google’s invincibility may have been greatly exaggerated. That’s the takeaway from recent developments in the world of online news and search — one a consequence of clashing business interests, the other a result of the never-ending quest to build the better information mousetrap.

Google News, Google’s algorithmically-generated news component, has for years feasted on news content from The Associated Press, the 163-year-old news cooperative. AP stories, along with those of hundreds of other news providers, form the spine of much of what Google News republishes.

But in an April 29 interview with Forbes, Associated Press Chief Executive Tom Curley threw down the legal gauntlet, threatening Google with a prohibition of use of its news content without Google taking action against those who misappropriate AP content with the aid of Google’s search rankings.

The threat follows News Corporation’s charge last month that Google commits copyright infringement when it borrows material from news stories to build its search rankings.

Robert Thomson, managing editor of the Wall Street Journal owned by News Corp., lashed out at Google recently. "There is a collective consciousness among content creators that they are bearing the costs and that others are reaping some of the revenues. There is no doubt that certain websites are best described as parasites or tech tapeworms in the intestines of the Internet.”

The AP signed a licensing deal with Google in 2006. Since then, Forbes reports,

Google has paid the AP undisclosed fees to carry AP content on the Google News section of the site. Search rankings on Google News give priority to recognizable news brands like the AP. But Google applies no such algorithmic discretion to general searches. The broader search rankings spread AP content out across the Web, says Curley, encouraging misappropriation by other sites. Curley wants Google to “protect content from unauthorized use and pay us for the longtail.” By “longtail,” Curley refers to the thousands of small sites that collectively drive vast herds of traffic using AP content.

Curley said that the 2006 agreement with Google was only reached only after the AP threatened to sue Google for copyright infringement. "They threw us a bone," he told Forbes, warning that if Google doesn't strike a better compensation deal with the AP soon, “They will not get our copy going forward.”


Industry analyst Ken Doctor of Outsell spoke to the point of fair compensation with the BBC on April 7, at the Newspaper Association of America's (NAA) annual conference in San Diego. “The real question is, 'Is it fair for news companies to produce all this content for Google and for Google to keep the lion's share of revenue? What we should be focusing on is 'fair share'.”

That issue will probably be resolved sooner rather than later. The no-doubt handsome fee Google pays the AP has to be welcome in this current economy — an economy that’s already seen the newspaper industry AP relies on having experienced serious contractions around the country.

As newspapers scale back operations, or close altogether, the AP loses some of its traditional base of content clients. With online as the evolving arena for the news and Google News already reaching more than 6 million users a month, one suspects that these two will reach an accommodation soon.

◊ ◊ ◊


Another challenge to Google’s dominance is still on the horizon, and will be a hell of a lot harder to dismiss.

The Wolfram|Alpha computational search engine has been the talk of geeks, media pundits, analysts and computer scientists for months now; that critical mass of critical thinkers has spilled over into the online media; some are calling Wolfram Alpha the first, real and serious challenge to Google, and, at the same time, hailing W|A as maybe the next big advance in computational technology. It's scheduled for public release sometime this month.

The W|A search engine was created and developed by physicist Stephen Wolfram, the mind behind Mathematica, the celebrated software program that enables users to visualize and manipulate applications of mathematical principles in theoretical and everyday uses, something with application in everything from flight simulators to statistics, music recording to disease diagnosis.

Wolfram, who introduced the "knowledge engine" on April 29 at a lecture at the Berkman Center for Internet and Society at Harvard University, used a classic show-and-tell to do it, breaking down the virtues of W|A in a classroom context — complete with fuzzy overhead projection. Consequently, you can’t glean that much from the YouTube video of Wolfram’s address; since it’s a video of a projection of screenshots, a lot is lost in translation.

But much of it is clear enough to showcase W|A’s ability to offer a range of computed answers to actual questions written in the most general terms. That means a fuller range of probable responses within a single page than that provided by Google, which responds to a query with a list of sources of possible answers arrayed across several web pages.



To go from Wolfram’s demo, W|A apparently also lets you enter search terms other than plain English and still get a workable search result based on the context of the query; for scientists used to speaking and thinking in the shorthand nomenclature of their chosen professions, this would no doubt be a timesaver.

For example, to go from Wolfram’s demonstration, typing (as he did) “6000C,” the W|A engine makes implicit assumptions from the start. In Wolfram’s demo, the search engine assumes that “C” represents a unit of something, then assumes that “C” represented a measurement of temperature (Centigrade or Celsius), then builds out a number of linear and graphic responses extrapolating from the initial assumptions.

If the real-world W|A engine works like the demo, then, typing in “C atomic number” would presumably bring up entries on carbon — the chemical element identified as C, and bearing the atomic number 6 — as well as a graphic of the periodic table carbon is part of, its byproducts (graphite and diamonds), a photograph of carbon, and other information displayed in linear and graphic form — all from dropdowns on the landing page. No drill-down to other pages required.

◊ ◊ ◊

There are some drawbacks: W|A is said to be weak on information for searches related to popular culture and entertainment. It’s been acknowledged that W|A’s release is meant to concentrate on the needs of professionals and academics. But that won’t last long — not if the damn thing is going to have any real traction in the mainstream of computing.

There’s a lot more everyday people than there are academics and professional number-crunchers; if W|A isn’t ultimately intended to be a "knowledge engine" for everyone … what’s the point?

Some aren’t that impressed yet.

Loftwork, writes in response to an article in The Independent (UK): “This appears to continue Harvard's Natural Language AI research, which has been ongoing for decades. It sounds interesting but I suspect its inherent difficulties with the apparent illogic of culturally based questions … where significance is largely determined by context and meaning is overloaded, may leave it as an alternative search methodology rather than a dominant system.”

Some are ready not to be impressed at all. “Hyped up pile of shite this!!” writes ffredson at YouTube.

◊ ◊ ◊

Others would beg to differ.

“For those of us tired of hundreds of pages of results that do not really have a lot to do with what we are trying to find out, Wolfram Alpha may be what we have been waiting for,” blogged Michael W. Jones at Tech.blorge.com.



“I would invest in this, literally and figuratively. If it is not gobbled up by one of the industry superpowers, his company may well grow to become one of them in a small number of years, with most of us setting our default browser to be Wolfram Alpha,” said Doug Lenat, on Semanticuniverse.com, back in March.

The truth will probably be somewhere between these extremes of reaction. But it’s been 13 years since Sergey Brin and Larry Page started Google, and about 11 years since it became a company. In Internet time, 11 years might as well be a millennium. Back then, a dozen years ago, most of us did pretty well (or thought we did) with the information we could get. Then, after Google was invented, we didn’t know how we got along without it.

Innovation, like history, has a way of repeating.
-----
Image credits: Google News logo, Google landing page: © 2009 Google. Associated Press logo: © 2009 The Associated Press. Wolfram Alpha landing page: © 2009 Wolfram Alpha LLC. Stephen Wolfram: Still from video of April 29 address at Berkman Center, Harvard University. Fair use rationale is implicit in the statute.