(Updated x2) Top Yellow Pages Searches

Ya gotta love the Yellow Pages - all printy fresh. In a press release issued today (the link is an instant download and I imagine you don't need the pdf clutter), they offered up their top searches for the year - the year 2002. In any case, it's interesting to see what the top "references" are - their terminology, best as I can tell, for the equivalent of a person taking action based on a Yellow Pages listing. This data comes from the Yellow Pages Integrated Media Association, which exists mainly to promulgate the idea that the Yellow Pages are a vibrant and long-lived source of leads for local business. Caveats herewith in place, the top ten are:

1. Restaurants
2. Physicians & Surgeons
3. Automobile Parts-New & Used
4. Automobile Repairing & Service
5. Pizza
6. Automobile Dealers-New & Used
7. Beauty Salons
8. Attorneys/lawyers
9. Dentists
10. Hospitals.

The data also includes the actual number of references for each term. The top term (Restaurants) had more than 1.3 billion references. In other words, folks used that category in the Yellow Pages 1.3 billion times in 2002. How on earth did they came up with this number?The study of course has no methodology attached to it. I mean, how *do* you track this? Compared to paid search, which is entirely trackable? Anyone know?! Guess I'll have to call the YPIMA and ask. Also, I've emailed folks at Google and Overture to ask what the equivalents are in paid search, which would be a fun comparison. ...

UPDATE: Click on the (more) link below to see the full response from the Yellow Pages Integrated Marketing Assocaition on how they got this data...Thanks to the IMA for this response!

John:

This survey was conducted by New Jersey-based Knowledge Networks/SRI. The surveys are based on 17,443 telephone interviews evenly spread throughout each of the previous two years and have been conducted every year for almost a decade. They also include 2,485 "yesterday" users. The research methods include the use of a random digit dial sample with one adult selected at random in each household. We have data on our Web site for the Top 300 Headings. You may find several free reports at:

http://www.yellowpagesima.org/research/index.cfm

The estimates of usage by heading represent the number of times individuals (for consumer or business purposes) look up a particular heading annually. It is important to note that these references do not include the particular product or service the individual is seeking. For example, many references to the "Automobile Dealers - New Cars" heading are for other dealer products, such as recent-model used cars, auto parts or repair services. While there may be individual headings for these products or services, a segment of consumers refers to the "New Cars" heading for all dealer services.

This usage data is taken from the Yellow Pages Industry Study (1994 - 2002), conducted for the Yellow Pages Integrated Media AssociationSM by Knowledge Networks/Statistical Research, Inc. Cranford, N.J. For more information on the study, contact Burt Michaels at Knowledge Networks/SRI 908-497-8060.

For questions about any of our Yellow Pages research, please contact our research director, Larry Small, at 303-690-0557.

Christopher G. Bacey
Director of Communications
Yellow Pages Integrated Media AssociationSM
Two Connell Drive, First Floor
Berkeley Heights, N.J. 07922
908-286-2394
www.yellowpagesima.org

UPDATE #2: I have heard from both Yahoo and Google, and they decline the opportunity to publish the top ten keywords in paid search. That's too bad, I think they'd be fun to know. ....

• Posted by John Battelle at 5:00 PM
• Permalink
• Comments (3)

Moving Beyond the Browser...

A Nielsen/NetRatings report shows how widespread the use of web-enabled applications (ie Chat, iTunes, etc) has become. MediaPost reports.

• Posted by John Battelle at 3:12 PM
• Permalink
• Comments (1)

Wired on How to Save the Internet

Back in 1997, Wired ran a cover story called "101 Ways to Save Apple." The cover remains my favorite for a number of reasons, the brilliance of the image, the genius of the singular imperative "Pray." I'm not sure the story, in which we polled a bunch of folks and created a list - Editors LOVE lists - was that great (actually, point #101 was pretty good: "Don't worry. You'll survive . It's Netscape we should really worry about.")

This month Wired is revising the 101 Ways meme with 101 Ways to Save the Internet. The story was written by Paul Boutin with input from a few key folks, including several readers of this blog. The voice is almost right, the politics line up, the issues are well chosen, but something about the list feels a bit off. I can't put my finger on it, but overall, it's a good rundown of all the issues the Net faces as we head into 2004.

• Posted by John Battelle at 1:37 PM
• Permalink
• Comments (2)

Rename it "Cardster" and Watch the VCs Come Running....

OK, here's a new idea: Search for people based on their business cards. I kid you not. CardBrowser is a web-based, paid registration database of...business cards gathered at various high tech conferences (more than 100 a year, they claim).

Now, nowhere on the site can I find exactly *how* they gather those cards, or if the folks represented on those cards are aware they are in a database, but...I've called to find out and will report back when I do.

The company behind CardBrowser is marketing the database as a way for companies to find "passive" job seekers - folks who already have good jobs in high tech who might not be actively raising their hands for new jobs. Recruiters can buy a subscription to the site and then contact potential recruits - and, the site boasts,have a pretty good chance of getting a response, as the information on a business card tends to be accurate.

This brings up a rather odd catch 22. Now, if the folks who are in this card database - and the company claims to have more than 17000 names, with some 2000-4000 added each month - *do* know they are being added to this database, then well, they ain't exactly passive anymore, are they? As an employer, I'd be less than happy to discover some of my key people in this database, and were I the distrustful type, I'd probably get a subscription just to check. If, on the other hand, the folks do *not* know they are in the database, seems to me we've got something of a privacy problem on our hands. The company has no privacy policy I could find, and does not address this issue anywhere. Could this be a simple oversight?

• Posted by John Battelle at 3:36 PM
• Permalink
• Comments (2)
• TrackBacks (1)

Another Oddly Named New One...

Tara over at ResearchBuzz has found this new entry: Ay-Up. Yup, Ay-Up. As she points out in her post summarizing its features, Ay-Up is unique in that it offers free site search. Worth a looksee. (And yet another new engine that needs the help of a logo specialist...)

• Posted by John Battelle at 10:22 AM
• Permalink
• Comments (3)

At Least It Doesn't Claim To Be The Next Google....

The latest entry in Odd Little Search Engines That Might...Sootle. Please, let me know if you want me to stop pointing you to this stuff. This engine is in deep Alpha, which might explain its name, logo, and terrible results (30 results for George Bush...) but not the lack of grammatical coherence in its "about" section...Given that the name of the Financial Director is "Peter Fiasco", I'm beginning to wonder if these new sites aren't elaborate jokes tossed up late at night by overworked engineers at Yahoo or Google....I mean....Sootle?

UPDATE: Within 12 hours of my posting this, both Peter Fiasco (my apologies, he's apparently a real guy) and the founder of Sootle, Sid Yadav, emailed me. They were quite kind, pointing out that my criticism of the site would inspire them to greater things with their new creation. Sid points out that my Bush search in fact found 30 *clusters*, and a total of 313 results. His index is only 11 days old, and is only starting to crawl ...literally. He calls Sootle "a hobby sort of project" and is working on a new logo and interface. Stay tuned....

• Posted by John Battelle at 10:12 PM
• Permalink
• Comments (5)
• TrackBacks (1)

Vertical Search: Sidestep The Obvious

Folks from time to time ask me if the game is up, if Google, Yahoo and a few others have locked up search forever. Bah!, I say. Search lives in every corner of the internet, insinuating itself into every fold of discoverable information. There are simply too many folds - large search companies can't profitably exploit every one of them. Hence the continued rise, in 2004, of the vertical search category. Examples? Sidestep, a travel search site that seems to be gaining traction of late. Why? The company's core promise to consumers can be found on its site: "SideStep is a search engine - not an online travel agency. " In other words, you can trust it, as it's not trying to sell you anything. It's focused entirely on its mission of finding the best travel deal, as opposed to selling you whatever inventory its partners might want to clear that day.

So what's the point? Have I fallen in love with Sidestep? No - it's still fish with feet - requiring you to download a software application that "watches" you do travel searches, then makes better suggestions. But I just love the idea it represents: search is a real time publishing opportunity. You can make a business of solving a person's ephemeral but specific information problem, addressing a person's simple but non-trivial query - "What's the cheapest hotel room in New York right now?" - and make a decent living at it to boot. Obritz, Expedia, Hotels.com - they all claim to do that - but they're not publishers, they're agents. Same with so many other first-generation vertical sites - Autobytel comes to mind. My experience is that they are all in the thrall of their partners and their inventory - they are in no way independent. (Just try asking Autobytel this question: "What's the cheapest Volvo c70 on the Web right now?" They send you to a dealer. Not exactly what you had in mind, eh?) I just love the idea that finding an honest answer to a reasonable question works as a business on the internet. Somehow, it feels like the essence of what publishing on the web can be - impartial (and complete) answers to honest questions. So I root for the Sidesteps of the world. The idea it represents scales to all sorts of opportunities, yet to be discovered.

• Posted by John Battelle at 9:52 PM
• Permalink
• Comments (0)
• TrackBacks (1)

And From The I Couldn't Resist Department...

• Posted by John Battelle at 6:37 PM
• Permalink
• Comments (0)

Australia's Answer to Google? Nah.

Yet another pretender to the throne, Mooter.com is a new search engine out of Australia. It uses clustering technology - not a new idea - but claims to have made it better. I tried it (quite cursorily) and it was, well, not awful. Scattered reports say the site sometimes fails to return results, but that hasn't stopped the Mooter CEO from saying they plan to go public on Google's coattails. At least she's being honest. According to a local news story on the IPO, Mooter has no profits. And if you hit their site and poke around, they sound darn flaky. From their Mission Statement:

As we move, as we track through the information now presented, as our brains cavort along their apparently random paths, increasingly powerful technologies will anticipate our needs. ....We must keep thinking. About who we really are. About what we really want. We must have a powerful tool for finding our way around the information world: a tool that does not impose value on us, but helps us find our own meaning.
If we do not, the mutated survivors will be the corporations who have managed the most manipulation, not the beauty of the human spirit in all its fierce joy of living and intensity of love for self and other sentients.
We must be mindful of what we plant, our children will bear the fruits.

Dude, pass the bong. It's sophomore year again.

And if they do go big time, they gotta do something about the logo. It looks like the mascot for a cold medication.

• Posted by John Battelle at 11:58 AM
• Permalink
• Comments (2)

Watch This Space: Comcast

There's been a lot of noise over Comcast lately, most of it about the company's rather restrictive terms of service for their broadband product. (It boils down to this: They make it hard to do anything but take a high bandwidth feed from them. Thus they are approaching the internet, predictably, the way they approach cable - a dumb system with intelligence, such that it is, embedded in the servers, rather than at the nodes). So watch this space: Comcast is continuing to flex its programming muscle. This means that Comcast the ISP will act more and more in the interests of Comcast the owner of entertainment programming. Which net net, isn't going to be good for the net, at least in the short term.

• Posted by John Battelle at 10:57 AM
• Permalink
• Comments (0)

Parts of Patriot II Slipped Into Law While No One Was Looking...

This is a very big deal. Not just because of the law itself - it's heinous - but because of the way it was passed by the Bush administration - on a Saturday, during Saddam's capture celebration, after an unaccountable voice vote on Thanksgiving with no debate. And of course the media did not pay attention, and of course, I hope, we will. I'll summarize the effect of this later - it gives the government extraordinary new powers of search (which is one reason it relates to this site) - but for now, please give your lawmakers hell.

• Posted by John Battelle at 4:27 PM
• Permalink
• Comments (6)
• TrackBacks (5)

Find O' The Day: WordNet

If you're a linguistic geek, or just like stoning out on how words work, check out WordNet. I was told of this site in an email discussion with a reader, it's an ongoing academic research project based out of Princeton.

This site (it's also available as downloadable software) is basically a database of interconnected word meanings. The site says: "WordNet® is an online lexical reference system whose design is inspired by current psycholinguistic theories of human lexical memory ." Er, in other words, it's a neat way to see the various "senses" a particular word might have. The online site has a Word Search function. Type in any word, say...."search"...and you'll see it has five senses as a noun, and four as a verb. You can then explore various aspects of the word's senses, including synonyms, derivations, and - really cool - hypernyms: "Search is a kind of..." and hyponyms "..is a kind of search."

I'll admit, this was my first time stumbling across the terms hypernym and hyponym and actually understanding why they matter. So why does this matter to Search writ large? Because one way to think about improving Search is for an engine to drill down on a particular query based on what sense of the word the searcher intended. In other words, when you type "jaguar" into the query box, which sense did you mean - the cat, the car, the team, the software? If a search engine can create "senses" of words on the fly, it might be able to create smart responses to difficult and high-results queries (AltaVista and others do something similar with clustering, but this technology has not been blessed by everyone as relevant enough..including Google). Think of Google's spell checker, but with "senses" of words, instead of spellings of words. "Did you mean the cat?...." etc. Now, I have no idea if this particular implementation would be useful to a search engine, it probably has all sorts of problems. But it's interesting to think about nevertheless.

Thanks to Steve K for this pointer and the conversation that provoked it.

(An aside - my email is jbat@battellemedia.com. I welcome email if a particular thought of yours is more comfortable in that medium as opposed to the site's comments area. I've learned a lot from such exchanges).

• Posted by John Battelle at 9:45 PM
• Permalink
• Comments (2)

China's Answer to Google

In the English language version of People' Daily (take it for what it's worth...) is a rather exuberant announcement for the launch of the "world's largest Chinese search engine", known officially in English as "China Search Online" (www.zhongsou.com). The page is reasonably clutter-free, as compared to most Chinese portals I've seen (I co-taught a course on weblogs and China last semester, the product of that course is a cool weblog called China Digital News.)
In any case, the folks behind the engine, HII (who went public earlier this month, see here) are compared to Google, they even have a no-human-editors-have-touched-this news product to boot.

• Posted by John Battelle at 11:32 AM
• Permalink
• Comments (0)

On Invisible Tabs (and Hands)

In an email conversation, Danny Sullivan (he of Search Engine Watch fame) and I recently were discussing last week's post on Froogle. Danny disagreed with my premise that Google's actions were inconsistent, in fact, he believes they may well be consistent with a new and evolving interface approach that he calls "invisible tabs." He explains the idea here. The gist: search engines will intuit what you are looking for behind the scenes, and deliver to you the results most consistent with that intuition, making the tab format redundant in the first place.

As Danny put it in an email to me:

The real departure is going to be if Google finally makes the jump and gives you back 10 product/Froogle results at some point, and suggest that you might also search the web, for some queries, rather than the web dominance we get now. That will be them fully putting into play this whole invisible tabs concept that I've been talking about recently.

Danny points out that Google already does this with News. Try searching for "George Bush," for example. You'll see News results at the top. Google is intuiting that you wanted news on George Bush, or at the very least, that news about George Bush is relevant to your search.

Same thing for Froogle results, Danny explains: "They're hitting the Froogle database in automated fashion, and if the automated system feels confident enough, you get Froogle results displayed. No different really in look, feel and operation than searching for "iraq" and getting news results."

Well, yes...and no. What I find interesting is this part of the idea: " If the automated system feels confident enough, you get Froogle results displayed." No matter what, code = architecture, and architecture = politics. Somebody had to code that Froogle algorithm to determine its confidence/intuition with regard to your search. Google, and any other search engine worth its shareholder's money, will never tell you how it makes those decisions. They are the Invisible Hands of the automated search process. The men behind the curtains.

And therein lies the interesting bits.

Regardless, we should all give Google a lot of credit for having neither paid inclusion nor referral fees in their shopping engine. That is leaving a lot of money on the table toward a greater end, and an indication of the philosophy which guides them.

• Posted by John Battelle at 10:36 AM
• Permalink
• Comments (1)

The Health of Magazines: Blame Cable As Much As Internet

More and more I'm noticing my cable lineup looks like a magazine rack. Used to be, television was a scarce resource. As late as five years ago, it was still being programmed for large audiences - at least a million, if not more. If folks wanted well-produced niche content, they had to go to magazines. Now they can go to the internet as well, but until recently, I thought magazines could still compete for a smaller audience's attention if they stood out as a voice for a particular community.

But I now believe magazines as we understand them are eroding, succumbing to the twin tides of niche cable and what might be called the second wave of Internet publishing.

First, TV. Cable seems to have finally realized that in a 500-channel universe, not every channel can garner a 20 rating. Hence a willingness to do focused, niche content that aspires to just several hundred thousand viewers at a time. This strategy can produce breakout mini-hits like Trading Spaces and Queer Eye, but in general, it seems cable has figured out how to make money selling audience sizes based on metrics quite similar to those of magazines. Thumbing up and down my cable menu, I feel like I'm at the magazine rack at Barnes & Noble - there's 25 different sports titles, scores of shelter books (that's the home/hearth category for you non-magazine folk out there), plenty of music/pop culture plays, even programmatic equivalents of "Guns&Ammo." None of these shows, save perhaps the pop culture stuff, do more than 500K in audience on any given day. In other words, TV has managed to segment audiences into the same demographic/psychographic buckets that once were the sole purchase of magazine land. PVRs only accelerate this trend, adding the convenience of search and storage to the magazine rack concept. Add in the fact that the average cable bill in the US is more than $40, and you have a subscription+ad model, just like magazines. I should also note that the advertising business has shifted in kind: production costs have been driven down by technology, and buyers now understand how to buy spot and niche cable. End game: TV wins head to head against print. Just ask the publishers of Life.

Now, the Internet. I've always thought you could create great magazines if you stayed away from competing solely on audience demographics/psychographics, and focused on the ineffable quality of publishing that might be called community. Because they serve deep and subtle content, magazines can create and/or declare community, a badge that folks wear proudly, a club in which they claim membership. Well, while it's quite difficult for a cable show to hold this rather ephemeral quality, the Internet has it in spades. Strike two.

If my beloved magazine readers are getting their high-bandwidth niche experience from cable, and their community succor from the web, what is left for magazines to do? Is Battelle saying magazines are dead?

Well, yes and no. We have to rethink what a magazine is. Again. After all, I teach magazine development, for goodness sake, so I can't very well believe magazines are a dying breed. There are exciting things to be done with the idea of magazines, if we can reinterpret them. For one, make them for smaller audiences, and compete on that point, rather than be ashamed of it. Two, figure out how to make magazines sing in the online world (nearly all attempts to date are awful). And three, figure out a way to get around traditional approaches to the twin evils of circulation marketing and distribution. I'll post more on my thoughts as to how at a later date, but I wanted to get that cable TV-as-magazine-rack meme out there, and see what you all thought.

• Posted by John Battelle at 9:50 AM
• Permalink
• Comments (3)
• TrackBacks (2)

A Software Wish

As long as I'm skipping down memory lane and all, I wanted to leave one tech/site related request - late though it is - for Santa this Christmas Eve. Or maybe I should say, for Ben and Mena over at Six Apart (the folks who brought us the software that makes this site, and many others, possible). I wish for Quark Xpress for the Web. I know this has been something of a pipe dream for many (and I'm not really tight into the discussion on this topic, so perhaps it's on its way), but now that I've mastered a few of the basics of Moveable Type, I really wish I could play with the software the way I did when desktop publishing was young, back in the late 1980s, when Quark and others made history. One of my first gigs was as a paginator for MacWeek, which claimed to be the first desktop-published four-color national weekly magazine in history. It was just really really cool to be a beta tester of software (Quark version .9) that you knew was going to change publishing forever. As great as MT is, it's still too hard to tweak the sites, to make them look better and perform better from a reader's point of view. I know making this stuff is extra hard. But I'd sure love to beta test the GUI version of MT, were it ever to come down the chimney....

And Happy Holidays to you all. Thanks for reading these past couple of months, and giving me so much to chew on. I look forward to 2004.

• Posted by John Battelle at 7:20 AM
• Permalink
• Comments (5)
• TrackBacks (3)

Why Yahoo, Interactive, and Google Love Local Search ...

Because it's poised to grow to nearly $3 billion in revenues by 2008, up from about $1 billion now. And because the mass of small business owners who currently don't use search would certainly switch if presented a compelling solution that actually brings in customers. Can you imagine your corner grocery store or dry cleaner buying keyword search? Me too. Move over, Yellow Pages....

• Posted by John Battelle at 1:22 PM
• Permalink
• Comments (1)

Corporate Search Is Sexy

Before there was the web, there was the corporate database. Remember those days? Back in the mid to late 80s, when the Local Area Network was the Next Big Thing, when everyone was madly installing client-server databases, when applications like dBase III and NetWare ruled the roost? You don't? Sigh. I must be showing my age. I was a cub reporter back then, covering the relatively new beat of "networking" as well as the corporate database market. Yup. Somehow I found that stuff fascinating. I thought this whole idea of connecting disparate networks of information was a hoot.

Anyway, to the point. About 1987 or so a new class of applications developed. Called Executive Information Systems (EIS), these were essentially interfaces to data, designed to live on top of corporate databases and cull the stuff Really Important Executives needed to know so as to make Really Important Decisions. The coolest part of the spec was the fact that the data was queried from the desktop - EIS promised easy and intuitive access those unintelligible databases the geeks kept buying. The idea was sexy, but the category never really took off. The design was too rules based, too top down. For them to work, you had to literally redesign your entire infrastructure. Oh, and the Executives in question had to give a shit.

Fast forward to now. As most of the world remains fascinated with search's more public face, a significant shift seems to be occurring in the corporate data world. I'm not saying EIS is back, exactly, but the overwhelming presumption of webwide search on your desktop is certainly rewiring how corporations think about their more private databanks. A robust market has grown up around "enterprise search," (some companies, such as FAST, were spun off from consumer search companies, and Google maintains a unit focused on the market). There's a crop of interesting startups to boot, including Tim Bray's company, Antarctica. It's entirely possible some of the next big ideas in search may well be developed in this more focused, less public field. Any readers out there have suggestions of cool companies in this space I may be overlooking?

• Posted by John Battelle at 8:10 AM
• Permalink
• Comments (1)
• TrackBacks (2)

The Mayo Database

Wired News reports on a massive database project from Mayo which has interesting, scary, and rather exciting implications for diagnosis and treatment. Genetic information will eventually be included. Excerpts:

During an office visit, a medic will be able to do enough quick data mining to ensure the most accurate diagnoses and most effective treatments while the patient waits, de Groen said. "Ideally the computer would query both our own database of patients (and) the complete medical literature." ....

Health-care professionals look forward to the eventual addition of patients' genetic information to databases like the MCLSS -- a field known as clinical genomics -- as a major advance in medicine. Among other things, such access would allow doctors to divine with great speed and accuracy what drugs have worked best on a certain type of person with a certain illness. ...

"It's really about applying the knowledge from many to the benefit of one," said Dr. Anne-Marie Derouault, director of alliances and distribution channel management for IBM Life Sciences, who heads the IBM teams working on the project. "Without genomics, it would be very hard to do that. Putting that kind of information with traditional information is potentially going to bring medicine to another level."

• Posted by John Battelle at 9:10 AM
• Permalink
• Comments (5)
• TrackBacks (1)

Thoughts on 2004

I am not sure why all of a sudden I am struck with the urge to prognosticate, but all weekend long I've been thinking about what might happen next year in the search/tech/media nexus. I think it has something to do with the book - my plan is to finish it by about mid year, then pray that nothing major changes for another six months while the manuscript wends its way through the vagaries of the publishing process. It's either that, or Jeremy envy.

So I've been thinking about a number of things, some small, some not so small, which might happen in the next twelve months. Given that I'm writing this on the eve of Winter's Solstice, I give you Battelle's First Annual Solstice Hopes and Predictions for 2004. I refuse to say which are hopes, and which predictions. This way, I can claim to be right next year one way or another. Take it for what it cost you on the way in.... (see list via link below)

1. The Web becomes a platform (again). Thanks to commerce and service APIs, RSS, and the ubiquitous interface of search, geeks around the world are again leveraging the web as a platform for cool new tools. 2004 will be the year these tools break out in something of a pre-cambrian explosion, reminiscent of the Mac in late 1980s, or CD-ROM in the early 90s. Only cooler. Examples: Grokker, Bloglines, Amazon API.

2. Along those lines (and no surprise to this readership, but still and all...), blog ecologies of like-minded folks will garner increasing cultural and social power. We've seen this happen first in the technology and media space, and recently politics has figured it out too. 2004 will see the rest of the world join in, especially in natural communities where power is projected: think professional verticals of finance, law, medicine, marketing. Folks who you never thought would ever blog will be coming online and claiming power. As a result, more blog ecologies will impose registration and/or subscription (the money kind, not the RSS kind...).

3. The Dutch auction/OpenIPO model will be validated. Not that it isn't already alive and working - WR Hambrecht is proving that - but 2004 is when a major player (and it need not be Google) will take the lead and fly the bird at traditional Wall Street approaches to going public.

4. Speaking of IPOs, we'll see a major IPO ($100 million+ sold to public) in search that isn't Google.

5. There will be a "Tylenol Scare" in search. One of the majors - AOL, Yahoo, MSN, Google - or possibly more than one will be caught up in a major privacy and/or corporate responsibility crisis. The press and consumers will freak as they realize how important - and imperfect - this thing called search is. There will be much harrumphing, then everyone will calm down, learn from the incident, and move on.

6. Once a month, a new search player will be crowned in the press as "the next Google." One of them, in fact, could be the next Google.

7. Second generation blog/RSS aggregation sites will come close to combining directory functions with LinkedIn- and recommendation-engine-like features - think Amazon+Yahoo for the blogosphere....

8. ...at about the same time Yahoo, AOL, MSN, and Google will build or buy second-generation blog/RSS aggregation sites.

9. The world will realize the importance of our digital artifacts, and takes further steps to to preserve them.

10. Cable companies will control more than 75% of the PVR market, but a backlash/new TiVo-like device (possibly from Apple) will develop by the end of the year.

11. Microsoft will have a surprise hit product that has nothing to do with Office or Longhorn, causing a minor fire drill in Redmond.

12. I'll finish my book, try to stop writing this blog, but find it impossible to do so. Meanwhile, a deeply cool, once-in-a-decade-magazine-I wish-I-had-thought-of will launch.

So what are your predictions?

• Posted by John Battelle at 9:00 PM
• Permalink
• Comments (9)
• TrackBacks (8)

Gurley on Cable Regulation: Counterintuitive Thinking

I subscribe to Bill Gurley's Above the Crowd newsletter, and always find his insights worth the read. (Bill is a VC at Benchmark, and a former partner of mine on the Internet Summit conferences.) His latest missive, "Cleaning Up After the Ninth Circuit in an Attempt to Save the Internet" is an exercise in counter-intuitive thinking. When I first read about this decision, I thought it was a victory against the evil cable companies, who I am always willing to believe have nothing but their own monopolies at heart. The ruling said, in short, that cable lines had to be viewed as telecommunication services rather than information services. The distinction was important, as it meant the cable lines were subject to the same sharing rules as phone lines. In other words, it could open cable up to competition, similar to what the RBOCs already face.

Given that for three of the past four years SBC felt my neighborhood was not profitable enough to offer DSL, and I had to get it from Speakeasy, which offers service on top of SBC lines, I thought this kind of a ruling was a good idea - maybe there would finally be someone like Speakeasy who could offer me cable modem speeds and who had a clue about the internet (Comcast clearly does not - don't get me started on that one).

Bill makes the case that while the intent of the court may have been good, the result could be crippling. I am not sure I agree with everything he writes - much of it is pretty rigid anti-regulation sentiment - but he makes some good points. Among them:

...Often in complex political systems, the objective of an action can be honorable, yet the impact of said action can be completely at odds with the objective. This is largely because the tools we use to encourage behavior in such systems are often crude and imprecise.

Attempting to increase competition by mandating that a company invest in infrastructure and then share that infrastructure with competitors is simply not a market-based solution. ...

We should all know by now that rather than increasing competition, regulation typically reinforces monopolies and oligopolies. Startups will not and cannot prevail in heavily regulated industries. ...

Bill concludes that either the Congress of the Supreme Court will have to clean up the Ninth's mess.

• Posted by John Battelle at 10:05 AM
• Permalink
• Comments (0)

LinkedIn+Vertical Blogs = Interesting Microcommunities

(Updated)
I think it'd be cool if you could join a network of folks who read the same blog(s). I've always maintained that any good publishing effort understands and reflects its community - that it is both a mirror to the community members, and a window into that community for folks who are interested in joining or understanding that community. Conferences have always been a neat way for readers of a publication to meet each other, for example. Foo Camp was one of the first I've been to where "blog ecologies" ended up meeting FTF, and it was quite something to see how folks who'd been connected mainly by blogs ended up working together in real space.

So think if you could "see" all the other people who read this site each day (and who opt-in to be seen, of course) - and invite them into a LinkedIn like network if you wished to. I wonder if that's in the cards for LinkedIn - to do vertical OEM stuff like that? Are there others working on stuff like this?
(Thanks for the meme, Matt...)

• Posted by John Battelle at 9:40 PM
• Permalink
• Comments (6)
• TrackBacks (3)

RSS Marches On...Net-based Aggregators

I keep meaning to do this...I've noticed a bunch of net-based aggregators out there, I'd be curious if anyone uses them. One of the earliest is Bloglines, founded by the same fellow behind eGroups. There's also a new one over at Feedster call myFeedster. And there's FastBuzz. And of course the one Meg's working on, Kinja (still in stealth).

• Posted by John Battelle at 8:08 AM
• Permalink
• Comments (13)
• TrackBacks (3)

The Salesforce IPO

So Marc's company will go first - the first of the companies on everyone's list of '04 IPOs to drop an S-1 at the SEC's doorstep. Here's the Merc's story on it...

• Posted by John Battelle at 7:12 AM
• Permalink
• Comments (0)

In a Name...

(caveat: site related)

A few readers of this site may recall the naming discussion we had back in 1997-8 when The Standard was born. Back then I was trying to come up with a name that properly captured the intent of the magazine, and I had several dogs in contention before Denise Caruso, Steven Johnson and I came up with The Industry Standard over (more than one round of) drinks in NYC. During that exercise, and during many subsequent and prior naming go-rounds, I always started email lists with trusted advisors, pinging them with naming ideas and getting their feedback.
This is a long way of saying that one of the reasons you see the "beta" tag on the banner of this site is that I've always been dissatisfied with its name. Somehow attaching the word "blog" felt dated, but I did it anyway, if only to signal the form the site would take. So with that in mind, I seek your counsel. You've been reading for a while now. Which of these names capture best the feel of this place? (I'm partial to the idea of keeping Search in the name, as you can tell).

-SearchThink
-SearchThought
-Searchlog
-SearchQuest

Have any others? If I rename the site to your suggestion, you get bragging rights forever, just like Steven and Denise have. Think of the whuffie....

And as long as I'm doing site news, I've now updated my templates to support RSS 2.0, so you can see links and pics in your readers, if they support 2.0.

• Posted by John Battelle at 6:55 AM
• Permalink
• Comments (10)

How the Information Age = The Dark Ages

This meme has popped up a lot over the past decade: We're increasingly putting everything on digital media, which is great, but we're failing to archive it properly, and even when we do, we're not archiving the machines we need to read the data. A librarian at the British Library makes this observation again in an article entitled The Great Digital Information Disappearing Act. (Thanks Gary for the link).

But we're missing a larger point. It's not just the data - the emails, MP3s, the websites. It's also the data about the data - the traces of our digital culture that are kept in the database of intentions. That data - what people wanted, when they wanted it, how they asked for it, what they got, where they went - represents no less than the cultural artifacts of our day, equivalent to the pottery shards and stone tools left by our forbearers. Imagine yourself an archaeologist 2,000 years from now. Wouldn't you like access to MSN or Google's database of intentions, so as to plumb the traces of an ancient culture emerging into the digital era? But to date, most search companies are either not archiving that data, or if they are, they are keeping it to themselves for competitive or legal reasons. There are huge privacy issues, of course, in insuring this data becomes part of our digital history, but they are solvable. What we don't have is the cultural will or foresight to realize what we're creating.

In any case, this is one of the larger informing concepts I am working on for the book. Your comments and input greatly appreciated.

• Posted by John Battelle at 9:30 AM
• Permalink
• Comments (2)

A New Class of Google Results, With Interesting Implications

I just read a piece by Bambi Francisco (it's here, scroll down in the story) in which she claimed in passing that typing "ugg boots" into the main page at Google returned a subcategory of results - above the regular "pure" search results but distinct from the advertising - called "Product Search." I tried it in quotes - "ugg boots" - and it didn't work. Without quotes - ugg boots - it did. I then tried "digital camera" and there it was again.

What do you know - a new class of results (from Froogle, see my earlier post which missed this) nested rather uncomfortably between the sponsored results at the top and on the right side, and the pure results below. Also interesting was the fact that the results were in the same white background as the regular "pure" search results, though they are distinct in look and feel.

This is a very interesting development with significant implications. Many would claim this is a major step toward Yahoo or MSN-like approaches on the part of Google. Others would argue that Google in fact is simply trying to do its users a favor, inferring that most searchers who type such queries are in fact in shopping mode. But it is a clear departure from the conceit - and I use that term nuetrally - that Google has always maintained, which is that the results offered by their engine are free of human intervention - that they in fact reflect the results of a carefully tended algorithmic secret sauce applied to every site without bias. Clearly, humans have decided to put that category of Product Search on top of the main results. And certainly those results are not subject to the same secret sauce which sifts the rest of the unwashed web. (One can imagine merchants racing to game Froogle, now that Froogle results are showing up - in first position no less - on the firehose of traffic that is the main results page of Google). No matter how you slice it, this marks a departure.

Interesting indeed. I am attempting to find out if this is just a test, a trial balloon of sorts, which Google does do quite often, or if this is a permanent feature. Stay tuned.

Update: Google's David Krane, who runs corp. communications, responded to my query early this morning. He said that the incorporation of Froogle results into the main search page is "a test of sorts, yes...just an opportunity to further integrate Froogle with what folks are looking for on Google. It's been up for a couple of weeks. We'll have more data on the response after the new year."

He offered to have me speak with the product manager in charge of this, and I will be talking to her at some point in the near future, give or take a Christmas holiday. Thanks for the quick reply, David!

• Posted by John Battelle at 10:50 PM
• Permalink
• Comments (2)
• TrackBacks (2)

Search IPO in Hong Kong

• Posted by John Battelle at 10:15 PM
• Permalink
• Comments (0)

Second Tier Players Strengthen

MediaPost reports that top members of the team behind Sprinks, which Google recently purchased and essentially folded into AdWords (why?see here), have landed at Kanoodle. Kanoodle also got a round of funding, and opened an office in NYC. They have a keyword search product, and will probably add contextual and local advertising products next. It's a sign of the robust market around paid search, and a good thing that companies besides Overture and Google can continue to thrive.

• Posted by John Battelle at 1:18 PM
• Permalink
• Comments (0)
• TrackBacks (1)

Search Print

Amazon is not the only one experimenting with making print available online. This is a Big Deal....

• Posted by John Battelle at 9:41 AM
• Permalink
• Comments (0)

Jakob Nielsen, Holiday Mixer

If you've ever driven down I-880 coming from points North toward San Jose, noticed the line of homes etched into the Eastern hills above Fremont, and wondered - "Who lives up there?" - the answer is Jakob Nielsen. Nielsen is a reknown user interface expert who's written numerous books and articles on web design, and who was an early Google advisory board member. I drove up to see him yesterday on my way down to the Google holiday party (well, the Google holiday party for members of the press, anyway). Jakob's house affords a sweeping view of the bay area from a more southern perspective, which is a bit disconcerting for someone who is used to seeing the bay from Mt. Tam or UC Berkeley.
Jakob has plenty to say about the state of search and design on the web. When I asked what was next in search, he said "Solving your problem, as opposed to finding you the best site." An interesting insight. He believes search is a critical thinking skill that should be taught in the elementary school system, and I agree. It's amazing how much smarter you can be online if you know a few basic search skills - use of quotes, and/or operators, a few basic syntax elements.
After Jakob I drove across the Valley to Google's new building, this time for a holiday media mixer. Just about everyone who's ever covered Google was there, from Dan Gillmor (SJMerc) to Fred Vogelstein (Fortune) to Kara Swisher (WSJ) to Stephanie Olsen (Cnet). And senior Google folks turned out in force, though some were obviously uncomfortable with being in a room teeming with journalists. I saw about ten folks I have interviewed for the book, it was good to reconnect. All in all a nice affair, and I had some interesting conversations with Craig Silverstein (employee #1), Krishna Bharat (created Google News and is now going to India to help start that project), Shona Brown (new at Google, running biz operations, wrote a good book back in the early bubble that may as well be a blueprint for Google right now), and many others. Larry, Sergey and Eric were there as well, they were predictably mobbed. I said hello and moved on. I hear Cory was in the house, though I didn't see him, but Joi Ito was there, looking younger than when I first met him at Wired in 1993. How the hell did he do that?

• Posted by John Battelle at 9:04 AM
• Permalink
• Comments (2)

It's Official: Time Weighs In

While at Wired, we used to joke that we'd know our time was up with Time Magazine did a major feature on us (they did, and it was, sort of - it marked a passing of our hype-driven era). Louis used to say he knew the hippy movement was through when it made the cover of Time.

Well Saddam made sure that Google wouldn't make the cover this week, but the magazine nonetheless managed a pretty comprehensive piece of its own. While much of it is retread, I have to give the author credit - it's one of the first, besides a nice piece in Wired last Feb, which gets to some of the JAM (joints after midnight) issues surrounding search. To wit:

"But for a minute forget about the big numbers, the millions of customers and the billions of dollars. Think about what's at stake culturally and socially in the search wars, and all those zeros start looking pretty paltry by comparison. The Internet is swiftly becoming the primary repository of the bulk of human information. Search is the way we get at that information, and companies like Google wield enormous power. They reflect our common interests and shape how we learn about the world with their rapid-fire search results. This isn't just about dotcom juggernauts duking it out for stock options and bragging rights. Whoever wins the search wars owns the keys to the kingdom of knowledge. That's a big responsibility. Are search engines up to it?

This is one of the bigger questions of my book (not only specific to Google, but to the database of intentions...).

Nonetheless, the media virus that has consumed nearly every outlet w/r/t Google may be played out, now that Time has had its say. Until the IPO, of course....

• Posted by John Battelle at 12:33 PM
• Permalink
• Comments (0)

The "Creeping Googlization" Meme

Alex Salkever of Businessweek gives Google an interesting business model once over, leading with a scenario in 2006 where Google runs just about everything web-related. He introduces the term "Googlization" - which I take means the creeping (his word) dominance of Google over nearly all forms of informational commerce on the web.

Salkever points out that all the new features Google is adding end up stealing traffic and revenue from other companies, starting with the example of Google's new ability to track FedEx packages:

"Perhaps more important, Google is providing this new shipment tracking service even though it doesn't have a partnership with FedEx. Rather, Google engineers have reprogrammed it to query FedEx directly with the information a user enters and provide the hyperlink direct to the customer's information.
No doubt, this is an ingenious way to keep people at Google longer. By extension, the search giant can create more online real estate to sell ads on. But with every new service, Google takes a slice of someone else's pie. Its ability to find pizza places within any given Zip code ultimately eliminates the use of YellowPages. Using it to find word definitions diminishes the business proposition of online dictionaries."

What's interesting here are the assumptions regarding Google's motivations - that the company is entirely motivated by the desire to garner more "real estate" against which it can sell ads. A reasonable assumption for a business magazine to make - that a company is being driven by the motivation to make more money. But that's quite distinct from what folks at Google state they are doing - leveraging searchable data to make a consumer's life easier. "Can't the two co-exist?" I can almost hear them asking. "Doesn't one (profits) follow from the other (helping consumers with a great service)?"

Well, yes, and with AdWords, certainly that was the case. But Salkever further paints an interesting scenario in which Google decides to add a feature where users type in "movies" and their zip code, then get Moviefone.com's results. He concludes:

"Google wouldn't really need to ask MovieFone's permission because the service is publicly available and easy to program into Google in the same manner as the FedEx service.
Yes, such a Google service would probably mean extra traffic for MovieFone.com. And that would be good for MovieFone, right? Perhaps not. The interaction takes from MovieFone at least one prime advertising asset, the usual initial visit to MovieFone's home page, and instead awards that impression to Google. That's because the movie-listing seeker will skip right to the MovieFone hyperlink with their Zip-code-specific information. And as more traffic arrives from Google, that would give it more leverage to ask MovieFone for a referral fee. That's not hard to envision if Google become a public company with a responsibility to, above all, boost profits."

But as seductive as the reasoning is, I don't think Google will sell referrals. Google will think twice, and maybe more than twice, about implementing such a business practice. That would make them a player in the buying and selling of pure intent - as opposed to the paid intent they so assiduously separate from their natural search results. Therein lies a significant conflict. Imagine if they did try to sell those referrals to Moviefone, and Moviefone told them to pound sand. Now what would they do - cancel the service? Keep the service but route the traffic to a different movie service, one that was willing to pay them (or pay them more)? Remember, this is within their pure, untainted-by-commerce natural search results. Maybe MSN would cut a deal like that, but such a scenario feels distinctly "unGoogly" to me. Google's entire reputation is staked on the purity of their search results.

I also don't buy into the idea that Google, or any portal, will become a consumer's sole point of entry and/or query to the web. First of all, there's too much context loss. Consumers like to have context surrounding their interactions with specific services. Second, there's syntax overload. It's amazing how powerful Google and other engines are, but more than 95% of all searches never take advantage of advanced search features. I don't want to have to remember the right syntax to get movies for my zip code, or to type in a particular syntax to get my Fed Ex package. Dedicated sites do this for me in an intuitive way with immediate options in case I get confused. Third, as a competitive differentiator, it's relatively easy to copy - Yahoo, MSN, and anyone else could add these features quickly if they started to gain traction. And lastly, if Google does become a standard referrer, Moviefone and others would simply have to adapt - it's not the first time content sites have depended on portals for referrals, and fought with them about who is delivering the true value to the consumer. It's AOL in 1994, Netscape in 1996, and Yahoo in 1998 all over again. The web will endure.

• Posted by John Battelle at 11:53 AM
• Permalink
• Comments (5)

Don Logan Speaks

An interesting interview in Ad Age with Don Logan, Chairman of Time Warner's Media and Communications Group (he runs the Time Inc. and AOL businesses, among others). He's an old Time Inc. guy, and that company has been running out of steam of late, reporting its first ever quarter without profit growth. So he's got two "troubled" units. He speaks about a number of issues, including putting Time Inc. content behind the wall at AOL, launching new magazines, and whether or not AOL should be sold (not if it remains a "sustainable business," hmmm...). (link via Rafat Ali, thanks).

• Posted by John Battelle at 9:48 AM
• Permalink
• Comments (0)

Go Toast Bought

One of the most talked about companies in the paid search analytics business is Go Toast, which makes a business of helping companies optimize and analyze their paid search campaigns. I always expected they would be in line for acquisition, perhaps by a large agency like IPG or Omnicom. Instead, Go Toast has been purchased by Atlas DMT, a unit of aQuantive (AQNT), which is a public roll up of several online agencies and services, including longtime survivor Avenue A.

• Posted by John Battelle at 9:32 AM
• Permalink
• Comments (0)

#9: Springtime in Geekland

Foo was fun, and it also showed that optimism was back in the geek inner circle.

THE MESSAGE
The Geeks Are All Right
What happens when 200 hackers and visionaries camp out in the hills of Northern California? If you have a stake in the future of business, you'll want to find out.

By John Battelle, December 2003 Issue

Stashed away in the rolling hills north of San Francisco, the town of Sebastopol, Calif., used to be remarkable for two things: Gravenstein apples (it was once the world's largest producer) and the Russian River appellation (excellent zinfandels). You can now add a third important growth industry whose roots are there: Foo Camp, a new breed of geek gathering organized (somewhat) by O'Reilly & Associates, a thriving technology publishing business.

This year's Foo Camp, held in early October, was extraordinary for many reasons, but perhaps mostly for its structure -- or lack thereof. Tim O'Reilly, Foo's founder, made sure that basics like food, showers, and meeting space were available, but then quickly turned over the weekend's agenda to the geeks (literally -- there was no agenda until Friday night, when the attendees made one up on the fly). The idea: Get 200 or so smart folks with a lot in common together in one place at one time, let them pitch tents, toss in a Wi-Fi network, and see what happens. Turns out, quite a lot. (more via link below)

At Foo, I was about as likely to bump into a founder of Google (both were there) as I was a vice chairman of Warburg Pincus. Tim Bray, a co-inventor of XML, huddled with Yahoo  (YHOO) engineer Jeremy Zawodny and Nutch founder Doug Cutting, and in no time they had posited a new approach to stopping spam (for details, head to www.tbray.org/ongoing/). Later, Zawodny teamed up with David Sifry, the founder and CEO of Technorati, a popular search engine for blogs, and others to propose a new way to organize the thousands of newsfeeds available from media outlets around the world. The new standard they hacked up, FDML, may well be adopted by major corporations and news outlets soon after this column hits newsstands.

Simon Cozens, an author and programmer from England, presented Twingle, a program that helps you find things in your e-mail archives (who doesn't need that?). Also receiving good geek buzz was an application called Dashboard, which automatically scans and indexes your hard drive, then displays documents related to whatever you're working on. And there were two sessions on how peer-to-peer networks over broadband connections are about to dramatically change the way entertainment is distributed to the home (for more on that, search Google for "BitTorrent"). For relaxation, campers drank microbrews, tossed Frisbees, and disassembled a Toyota Prius, then put it back together again (it was a rental). Clearly, this was not your average technology conference.

Talking with attendees, I couldn't help thinking that Foo was more than fun -- it was important, and not just to the characteristically self-involved lot who proudly wear the geek label. After nearly three years of nuclear winter in the technology industry, the folks who gathered in that Sebastopol orchard were ... well, they were happy again. Optimistic even. "It felt like we were poking our heads up out of the ice and seeing spring," noted Ross Stapleton-Gray, a security researcher.

If the Foo folk are any indication, our precious resource of geekdom is once again charged up to tackle big problems and create world-changing applications based on trends like social software, ubiquitous broadband and wireless connections, and a newly rational environment for investment and finance. These people believe they can fix the spam problem and make sense of the overwhelming flood of information now available to corporations and individuals. They sense lasting and real economic models and see potential in nearly every problem they encounter.

In other words, they are hard and happily at work moving the entire economy of the Web forward, to good end. Perhaps, I thought to myself as I drove home at the end of Foo Camp, it just takes a couple of days in the woods to realize that we are well on our way out of them.

John Battelle directs the business reporting program at UC Berkeley's Graduate School of Journalism. He founded the Industry Standard and was a co-founding editor of Wired.

Find this article at http://www.business2.com/b2/subscribers/articles/0,17863,548902,00.html

©2003 Business 2.0 Media Inc. All rights reserved.
Reproduction in whole or in part without permission is prohibited.

• Posted by John Battelle at 1:26 PM
• Permalink
• Comments (0)
• TrackBacks (1)

#8: Intent over Content, Fish with Feet

I do believe there is a huge market to be made in adapting video to weblike channels...

THE MESSAGE
Gone in 30 Seconds
The classic TV spot can't dominate advertising much longer. Microsoft's new MSN Video is bringing Web-based commercials to a computer monitor near you.

By John Battelle, November 2003 Issue

NOTE: This has been updated from the version published in the November 2003 issue of Business 2.0

Let's consider a few fun stats. First, broadband has reached nearly 39 percent of all Internet-connected households and is expected to be in 79 percent in five years. Next, research firm In-Stat/MDR predicts that corporate spending on broadband content-delivery services, essential to high-quality video streaming on the Web, will almost triple by 2007. Third, the vast majority (72 percent) of work connections are high-speed. Throw in the forecast that TiVo-like devices will be in 20 percent of American homes (and certainly a higher percentage of wealthy ones) by 2007. And finally, 99 percent of homes with incomes over $100,000 have Net access, and it's a safe bet that most of those have broadband connections.

Add it all together and here's what it means: A massive broadband marketing opportunity is upon us -- and, as a recent Yankee Group report concludes, the 30-second TV ad will soon lose top billing as our most valuable marketing vehicle. What will replace it? Web-based video advertising. (more via link below)

Leading the revolution is Microsoft's (MSFT) MSN, which has just introduced MSN Video, the Web's first high-quality broadband video service to be entirely free to consumers. Making its debut this winter in the U.S., it will feature on-demand, interactive news, sports and entertainment at no cost to viewers -- no subscription fees, no annual charges, nothing. Instead MSN Video will carry traditional 15-second video ads.

MSN Video is pioneering a form of advertising that will be vastly different from its TV ancestors: It will be measurable, trackable, and targetable to the nth degree of demographics. And to the extent that a given audience member cares to invite an advertiser into his or her private database of likes, dislikes, and desires (a profile on Amazon (AMZN) or Yahoo, for instance), marketers will be able to target specific intent to buy -- and psychographics to boot.

In essence, Web-based television advertising will be grafted onto the already booming market for streaming video on the Web, and instead of buying mass-market television shows (content, for lack of a better word), marketers will soon learn to buy pure demographics and consumer intent, courtesy of Web-based networks -- starting with MSN Video but eventually including AOL (AOL), and Yahoo (YHOO); preference engines like those in place at retailers like Amazon; and new distribution channels like TiVo (TIVO).

Yes, I'm on my hobbyhorse of content over intent again, but this is worth repeating: Marketers are in a slow, denial-laden shift from buying content-attached audiences, like those of TV shows, to buying intent-attached audiences, like those of search engines and personal video recorders. But imagine what happens when the two merge: Advertisers can buy the audience they want, just when the advertisers are most wanted by the audience.

I admit that the idea of television advertising migrating to the Web has always been a bit suspect. After all, the Web was supposed to provide a targeted, one-to-one medium that would far surpass the relatively blunt instrument of the 30-second TV spot. What's interesting is that it need not be an either-or proposition. It's still flawed, but video on the Net is wildly popular -- MSNBC alone streams between 14 million and 25 million clips a month on average (and 87 million in the first four weeks of the Iraq war).

Microsoft is now gambling that there's a business in selling that audience to advertisers. "MSN will be a cable-size property within a year," says Todd Herman, "streaming media evangelist" for MSNBC. "And MSN will be successful insofar as we make it easy for advertisers."

Microsoft already has early evidence that it's on to something big. Its MSNBC.com has sold out its inventory of 15-second advertising spots, which run adjacent to many of the news clips it streams each month. "We know there is pent-up demand," say Scott Moore, general manager of MSN News and Information. These 15-second clips are precursors of the multiple forms of marketing-driven video we will one day see on the Web -- the first fish with feet, so to speak. And because it can tie in to Microsoft's Passport registration technology, MSNBC.com will someday link that ad inventory to specific demographics or audience preferences. Advertisers that buy these spots are not buying shows; they're buying pure audience, as well as specific knowledge of that audience's intents. And that's something revolutionary.

There are any number of problems with selling television advertising on the Web, of course. Will Web surfers accept it? Will they hit the fast-forward button (or will they be able to)? Will advertisers see the Web as a valuable medium? Can they even buy it in a way that makes sense to them -- using the same reach and frequency metrics they apply to buying television and print? Can they let go of their decades-long dependence on the up-front and learn to love the Web?

I'd bet that the answer to all these questions is yes. But don't take my word for it. Thanks to Microsoft, we're going to find out sooner than most people think.

John Battelle directs the business reporting program at UC Berkeley's Graduate School of Journalism. He founded the Industry Standard and was a co-founding editor of Wired.

Find this article at http://www.business2.com/b2/subscribers/articles/0,17863,534114,00.html

©2003 Business 2.0 Media Inc. All rights reserved.
Reproduction in whole or in part without permission is prohibited.

• Posted by John Battelle at 1:23 PM
• Permalink
• Comments (0)
• TrackBacks (1)

#7: MSFT and Search

A longer piece on MSFT which came out of the news that Yahoo bought Overture.

THE MESSAGE
Mr. Gates and the Hunt for Search
Yahoo's grab for Overture seemed to outflank Microsoft. But wait: Bill's search-engine strategy is not what everyone thinks.

By John Battelle, September 2003 Issue

Like TV broadcasting and the automobile business before it, the Internet media industry has now resolved to the Rule of Three. Of the scores of companies that battled for dominance in the 1990s, only Yahoo (YHOO), Microsoft (MSFT), and Google remain serious contenders. They, along with AOL (AOL), own the lion's share of Internet advertising and worldwide English-language traffic. Yahoo's recent acquisition of Overture (OVER) reaffirms, if the idea needed reaffirming, that Internet media obeys the same urge to consolidate as every other industry.

It proves something else, too, about this phase of the Internet's evolution. The key driver is no longer content, but intent. The business is no longer about selling advertisers the eyeballs you've caught with news, images, games, and the like. It's about selling users at the moment they make their online desires known through their search queries. In plain terms, the engine of Internet media is once again search. (That, by the way, is why I'm leaving AOL out of this discussion. Preoccupied with many other problems, the dial-up giant -- a corporate sibling of this magazine -- is leaving its search functions to Google.)

Search will account for more than $2 billion in advertising sales this year. It's predicted to grow at 35 percent annually, to nearly $7 billion by 2007, according to U.S. Bancorp Piper Jaffray. Beyond the numbers, search has become the most important commercial application on the Web. Not only is it the defining task of any portal, but it's also the preferred doorway into e-tailers like Amazon (AMZN), as well as auto, home, and dating sites. An online consumer business can no longer afford to have poor search capabilities.

By the middle of last year, both MSN and Yahoo had realized that they needed to rethink their search strategies. To profit from search, a company needs three elements, all of which Google already had. First, you must have high-quality "algorithmic" search, which attempts to match users perfectly with what they're seeking. For years MSN and Yahoo have outsourced algorithmic search to companies such as Inktomi and Google. Second, you need a paid search network, which allows you to display links to paying advertisers alongside your editorial results. Both MSN and Yahoo had outsourced this to Overture. And third, you need your own distribution. In other words, you must own the site where the consumer makes his or her query and the results are displayed. Until recently this was the only element that either Yahoo or MSN truly owned.

By buying Inktomi last December and then Overture in July, Yahoo has taken control of the two missing elements, which arguably leaves MSN in the worst position of the Big Three. Besides having the only site of the three that is consistently unprofitable, it is outsourcing both its algorithmic and its paid search technology to a major competitor. Not exactly an ideal situation.

But aside from Netscape's early investors, not many people have ever gotten rich by underestimating Microsoft. (more via link below)

Earlier this summer, just before the Yahoo-Overture deal was announced, big news began rippling through the Silicon Valley grapevine: Bill Gates had seen the next great opportunity, and its name was search. Microsoft had recently posted scores of new job openings in the field, and a new "MSNbot" had been spotted crawling the Web. Within a week Microsoft officials had confirmed the rumor, and Valley dwellers once again invoked Netscape, this time as a verb -- as in "Microsoft is about to Netscape Google."

But Microsoft isn't out to crush Google, at least not directly. The best way to think about anything Microsoft does is to ignore the market leaders it might squash along the way and focus instead on the market lead it already commands. That has a one-word name: Windows.

Put another way, Microsoft's interest in search, or Internet media in general, has a lot less to do with beating Google or Yahoo and a lot more to do with ensuring that the next version of Windows has enough features to persuade customers to pony up. For now, potential search profits are a rounding error compared with the billions the company could earn from a new version of Windows.

What makes search an attractive battleground for Gates, apart from its importance to Internet media, is that it's still in its infancy as a computer science problem. That means there are extraordinary opportunities to create great new applications in the field. "Fifty to 60 percent of consumer search queries go unanswered by any search engine," points out Lisa Gurry, group product manager at MSN. "No one is successfully doing [search] today," she adds, echoing a view expressed by many engineers who specialize in search.

If Microsoft acts true to form, it will define "successfully doing search" as "making search a feature of Windows." Whether this is a wise approach remains to be seen, but the next Windows upgrade -- code-named Longhorn -- is planned for 2005 (it was originally due next year). And while Microsoft won't say it quite this way, it's clear that one of Longhorn's most marketable new features will be its integrated search functions. Sources both inside and outside the company say that Longhorn will blend local (your own hard drive or server) and Internet search, and will even go so far as to crawl your favorite websites overnight and cache them on your hard drive, creating an extraordinarily fast Internet experience.

If all Microsoft wanted was a great search engine over at MSN.com, it might as well keep its current arrangement with Yahoo/Inktomi/Overture -- or fire a shot across the bow of Yahoo CEO Terry Semel and cut a new deal with Google. MSN's contracts with Inktomi and Overture are staggered across multiple markets and product offerings, and are set to expire by 2005 at the latest. Microsoft could probably wriggle out of them all should it wish to -- its execs have already taken to referring to their Overture partnership in the past tense. But in the short term, the company will likely continue outsourcing its search functions while armies of Microserfs labor away on integrating search into Longhorn.

Given the critical nature of search, it makes sense for Microsoft to develop its own code. CEO Steve Ballmer may accelerate the pace through acquisition of a second-tier search engine like FindWhat, LookSmart, or Ask Jeeves -- all three companies saw their stock prices jump in response to the Overture deal. But the most far-fetched scenario has Microsoft acquiring Google. While Google founding lights Larry Page and Sergey Brin privately say such a deal will never happen, Ballmer & Co. have $49 billion in the bank, and upon closer inspection, the two companies are not that incompatible. Both have engineering-driven cultures that prize intelligence over all else. To further cloud the horizon, the Yahoo-Overture deal won't be final until late this year. There's always the chance that Microsoft -- or someone else, for that matter -- could make Overture a better offer.

So where does that leave Google? It never pays to underestimate the leverage that Redmond wields with its Windows monopoly, but once search is integrated into Longhorn, the folks at Google should be smart enough to learn from Netscape's mistakes and step quickly out of Microsoft's ponderous path. They can simply play to their strength, offering all those frustrated Windows users a better way to find things on the Web. Chances are good there'll always be a market for that.

John Battelle directs the business reporting program at UC Berkeley's Graduate School of Journalism. He founded the Industry Standard and was a co-founding editor of Wired.

Find this article at http://www.business2.com/b2/subscribers/articles/0,17863,515970,00.html

©2003 Business 2.0 Media Inc. All rights reserved.
Reproduction in whole or in part without permission is prohibited.

• Posted by John Battelle at 1:20 PM
• Permalink
• Comments (0)

#6: A Growing Early Lockdown Threat: Super DMCAs

Think you control your own entertainment system? Think again.

THE MESSAGE
Don't Stifle Progress
A number of states are trying to do what the federal government couldn't: Kill innovation on the broadband Internet.

By John Battelle, August 2003 Issue

What if it were illegal to connect your computer to the Internet without asking permission? It seems unthinkable, but an ongoing legislative imbroglio has brought that very question to the fore.

The fight pits the cable industry and (surprise!) the Motion Picture Association of America against a battery of consumer and civil liberties groups. It revolves around a set of bills that opponents have labeled "Super DMCAs" -- state-level legislation modeled on the much-reviled federal Digital Millennium Copyright Act of 1998.

During the past 36 months, these bills have been passed in the legislatures of seven states, including Michigan and Pennsylvania, with nine more waiting in the wings. In each case, the legislation is cloaked in the seemingly benign language of preventing cable signal theft and has been quietly fed to a somnambulating legislative body. But all of these bills also contain extraordinary language that gives the access industry (cable, satellite, and phone) complete control over what devices you can connect to their broadband networks. They also make many things we take for granted -- such as anonymity, firewalls, even instant messaging -- potentially illegal.

In other words, these bills could create "early lockdown" of the broadband Internet, meaning that every aspect of how the network is used could be dictated by the access provider. And when the rules governing a robust system prevent innovation and favor one set of players above all others, the system withers. Even the Internet.
(more via link below)

With apologies to the MPAA, we've seen this movie before. Way back in the middle of the last century, the U.S. phone system (also known as AT&T (T)) was a government-sanctioned monopoly. In a classic example of early lockdown, no devices could be connected to AT&T's network without the company's approval. The bulk of what we think of as today's Internet -- personal computers, modems, and all -- was rendered illegal before it was even conceived.

Luckily, an obscurely named American startup called Hush-A-Phone took on Ma Bell. Back in the 1920s, folks were just getting accustomed to the idea of using a telephone in a public place. Privacy was a serious concern, so people would cup their hands over the phone receiver to mute their speech. Hush-A-Phone came up with a rectangular plastic attachment that covered one end of the phone. The user spoke into a hole in the middle, ensuring a private conversation. Hush-A-Phone was a hit, selling more than 125,000 units.

Then AT&T got wind of it and sued. It had not approved this innovation. The case went as far as the U.S. Court of Appeals in D.C., and in 1956 sanity won out -- Hush-A-Phone prevailed. The case laid the groundwork for the eventual breakup of AT&T and the "attachment" of all kinds of devices to the phone network, including answering machines, faxes, personal computers, and even MCI.

With Super DMCAs, it's Hush-A-Phone all over again. While the bills vary from state to state, they're all similar enough to rally a powerful Net-based counterstrike, led by some obvious names (the Electronic Frontier Foundation) and some not-so-obvious ones (the Consumer Electronics Association). Under Super DMCA legislation, it's illegal to connect an unapproved personal video recorder (a TiVo-type device) to your television; only those approved by your cable company are allowed. Might the cable company prefer a box that doesn't allow you to skip commercials or that lets you record only certain programs at certain times? Hmmm.

But there's more at stake than our right to bear PVRs. Innovation, like capital, flows to wherever it's freest. Places like South Korea and Japan are encouraging open, unfettered broadband networks. There's no rule stating that the next great leap in technology and media will happen in the United States. We learned this lesson once. Let's not have to learn it again. There are rays of hope, including Colorado governor Bill Owens, who recently vetoed his state's Super DMCA bill. If other elected officials follow his lead and take a stand against the cable and movie lobbyists, the Internet will remain a safe place to innovate.

John Battelle directs the business reporting program at UC Berkeley's Graduate School of Journalism. He founded the Industry Standard and was a co-founding editor of Wired.

Find this article at http://www.business2.com/b2/subscribers/articles/0,17863,515853,00.html

©2003 Business 2.0 Media Inc. All rights reserved.
Reproduction in whole or in part without permission is prohibited.

• Posted by John Battelle at 1:16 PM
• Permalink
• Comments (0)

#5: Nutch Presages a New Kind of Search Engine

Open source search - in an age where innovation is increasingly silo'd in large companies, this is a good idea whose time has come. I didn't like the "watch out" angle, but...it gets a reader's interest.

THE MESSAGE
Watch Out, Google
Nutch could rewrite the rules of search development -- especially with an impressive roster of Internet luminaries now lining up behind it.

By John Battelle, August 08, 2003 (Web Special)

Ask anyone in Silicon Valley what the hottest application on the Internet is today and you can bet their answer will be search. The dealmaking has been nothing short of torrid. Only a year ago there were at least half a dozen major players. Now there are just three: Yahoo (YHOO), which last month bought search giant Overture (OVER) in a $1.6 billion deal; Google, the undisputed king of search; and Microsoft (MSFT), which is busy building a search platform of its own. They're all fighting to dominate the huge and ballooning market, already worth $2 billion and expected to generate between $6 billion and $8 billion in revenues by 2007.

Search is a game of intellectual property, innovation, and market position. The three combatants all keep jealous watch over their patents (Yahoo, for one, has more than 60), engineering talent (hundreds of Ph.D. holders work at Google), and market advantages (Microsoft -- need we say more?). Indeed, search is such a complicated and expensive undertaking that analysts have pegged the cost of market entry at well over $100 million.

All that could change this fall, when a new player strides onto the field.
(more via link below)

Meet Nutch, the open-source search engine. Open-source applications are unusual in that the code upon which the software runs is not owned by a private, commercial company but rather bound by a simple license that allows anyone to use, modify, and even profit from it free of charge, as long as they pledge to contribute their own innovations back into the code base. Because of this, anyone will be able to access Nutch's code and use it to their own ends, without paying licensing fees or hewing to a particular company's set of rules. Perhaps more important, Google takes a "trust us" approach to search; they say they don't skew their PageRank formula to favor certain sites, but we have no way of knowing for sure. With Nutch, the indexing and page-ranking technologies are all open and visible; you can check them yourself if you have a problem with your page's ranking. Just as Linux has taken on Windows, revolutionizing the rules of search-engine development and distribution, Nutch could pose an enormous threat to Google and other search giants.

"Search is interesting again," says Doug Cutting, a founder and core project manager at Nutch. Cutting, whose development chops were honed at Xerox (XRX) PARC, Excite and Apple (AAPL), is building Nutch (that's his toddler's all-purpose word for "meal") with a small team of engineers based around the country. But Cutting says they hope that once Nutch is loosed on the world, tinkerers from Romania to China to Palo Alto will help build it into a robust platform, in the spirit of Linux or Apache (which has garnered more than 60 percent of the Web-server software market in just the last couple of years). "Search is the first thing people use on the Web now, and there are fewer and fewer alternatives," Cutting says. With Nutch, "researchers, university folks, and anyone else can have a test bed to make search better. There are a lot of smart people out there that Google can't hire."

Mitch Kapor, who helped found Lotus Development and the Electronic Frontier Foundation and is founder and president of the Open Source Applications Foundation, certainly agrees. He's thrown his weight behind the project by joining Nutch's nonprofit board, as has Tim O'Reilly, the CEO of O'Reilly & Associates. Brewster Kahle, the visionary behind the Internet Archive, has also lended his support. Nutch is moving its servers to Kahle's high-bandwidth location this weekend, a crucial step toward readying the engine for its public debut.

"I love Google," Kapor says, "but this will push search to places that are not immediately obvious. In terms of research and innovation, there is a clear need for an open platform for search." Kapor and others imagine new kinds of applications springing from Nutch, ideas that commercially driven companies like Yahoo or Microsoft would never fund. "Search is close to a duopoly," Kapor points out. "Historically we know there are risks when that happens. It's too important an application to not be transparent."

Cutting won't commit to a specific launch date for the engine, but he said he expects it to go live at Nutch.org sometime early this fall. Due to the move to Kahle's facility and insufficient hardware (Cutting is looking for additional sponsors), Nutch's demo -- based on an initial crawl of more than 100 million webpages -- is not yet open to the public. But Cutting, who together with his development partners has built an impressive resume in the search field, is confident his latest creation will be a contender once it launches. "It's fun to go toe-to-toe with market leaders," he says. "It's always a challenge to build a better mousetrap."

John Battelle (john@battellemedia.com) is a visiting professor at the UC Berkeley Graduate School of Journalism, where he directs the business reporting program. He was the founder of the Industry Standard and a co-founding editor of Wired.

Find this article at http://www.business2.com/b2/subscribers/articles/0,17863,515961,00.html

©2003 Business 2.0 Media Inc. All rights reserved.
Reproduction in whole or in part without permission is prohibited.

• Posted by John Battelle at 1:13 PM
• Permalink
• Comments (0)

#4: That Wonderful Honda Ad

When I saw this ad, I knew I was watching the future of online video advertising unfold.

THE MESSAGE
Downloading the Future of TV Advertising
With a plink and a plunk and 86 moving parts, Honda reminds the ad world of the value of great content -- and teaches it something about the power of interactivity.

By John Battelle, July 2003 Issue

In April 2003, Honda U.K. debuted an extraordinary two-minute television advertisement called "Cog."Through the simple act of releasing a remarkable television commercial onto the Web, the U.K. wing of automobile giant Honda (HMC) has unleashed something of a typhoon in the advertising business. Though it has yet to fully play out, Honda's ad proves the value of content and could stand as a turning point in the history of the television spot -- proof that interactivity won't kill television advertising, as many are now predicting, but may instead be instrumental in saving it.

Back in April, Honda U.K. debuted an extraordinary two-minute television advertisement called "Cog." Aired only in the United Kingdom, the film -- and that really is the best term for it -- is a Rube Goldbergian ballet, a synchronized dance of 86 distinct parts from a Honda Accord that roll, pirouette, and fly along the floor in a mesmerizing production of meticulously intended consequence. The spot begins with a sequence of three cogs rolling along a plank; one falls to the floor, and a cam shaft rolls, setting an exhaust tube slowly spinning, which knocks three precisely placed grommets down the slope of a hood, and so on. (Download the ad for yourself at Honda UK.)
(more via link below)

To watch this film is to want to watch it again, which is what I did, repeatedly, after a friend e-mailed me the link. Not only was the work beautiful, but it was advertising -- it functioned in ways that television ads simply weren't intended to function.

"Cog" made me think well of Honda, so the branding was effective. But more interesting was the way I came across it -- through word of mouth -- and the expectations I brought as I downloaded it: I was taking the action to view the message; it was my intent that drove the transaction. This ad was content I wanted, not a sales message I wished to ignore. The experience was peculiar -- this isn't how advertising models for television are supposed to work.

But work it has. Since the film made its debut on British television in April, Honda U.K. has been besieged by repeat viewers. As was its custom, Honda had already put the ad on its site; it was instantly blogged and Slashdotted. Every major paper in the United Kingdom wrote it up, noting its painstakingly analog production (filming required more than 600 takes) and unusual length. Traffic to the Honda site quadrupled; in the first few weeks, nearly 1 million people downloaded the film. By mid-May, the number was twice that -- and millions more, no doubt, have seen the film as an e-mail attachment. The Honda marketing folks are clearly tickled by the response. "We think this campaign has managed to reposition Honda more toward the quality and sophistication of the European makers," says Nigel Bobs, a marketing executive at Honda U.K. "We certainly had no idea it would take off like this."

"Cog" reminds us of the power of great content, and it may well shift the tired debate regarding what many marketers deride as "vanity ads," which capture awards rather than results. It proves that great content can be combined with intent-based marketing like direct mail or paid search. Imagine a film like "Cog" as the payoff to clicking a paid link for Honda or "buy car" on Google. Or as a downloadable promotion on a service like TiVo. In fact, Honda U.K. tested "Cog" on BSkyB's fledgling interactive television system in England. Although a relatively small number of people saw the film, more than 10,000 viewers requested additional information on the Honda Accord. To get that kind of response through regular direct mail, Honda would have had to spend close to a million bucks. On BSkyB it spent about $32,000.

Could it be that "Cog" presages a time when the television spot evolves to an Internet-based format, courtesy of the fertile mix of broadband, search engines, and PVRs? Memo to advertising agencies: Pay attention to "Cog." You can have your vanity ad and ROI it too.

John Battelle directs the business reporting program at UC Berkeley's Graduate School of Journalism. He founded the Industry Standard and was a co-founding editor of Wired.

Find this article at http://www.business2.com/b2/subscribers/articles/0,17863,515786,00.html

©2003 Business 2.0 Media Inc. All rights reserved.
Reproduction in whole or in part without permission is prohibited.

• Posted by John Battelle at 1:10 PM
• Permalink
• Comments (1)

#3: Why Contextual Paid Ads Matter

Oh, if only we had this three years ago, at thestandard.com...

THE MESSAGE
Putting Online Ads in Context
Overture and Google have figured out how to sell the Web. Paid search has already saved Yahoo -- and your business might be next.

By John Battelle, June 2003 Issue

The long-awaited "30-second spot for the Web" -- a way for ads to finally work online – may well be at hand. Overture (OVER), the company some say saved Yahoo's (YHOO) bacon, will shortly roll out a service that opens up the entire Web to a new form of advertising. "It's potentially revolutionary," says Scott Moore, who oversees Slate and MSNBC.com for Microsoft (MSFT). How revolutionary? Moore says using Overture's new service, or one like it, could well push his sites to sustained profitability.

The breakthrough, which I'll call "contextual advertising," involves commercial links that appear adjacent to relevant content on websites. Say you're at Caranddriver.com, reading a review of the Acura MDX. In place of banners for everything from cell phones to cars you don't care about, you would see paid text links advertising the Acura website, the Edmunds auto comparison site, and leasing companies vying for your business. These are the same links you'd see if you typed "Acura MDX" into Overture's client portals, like MSN or Yahoo.

Google also plays in this new market with an offering called "content-targeted advertising." The beauty of both is their ease of use for publishers: Overture and Google automatically analyze the publishers' pages and insert relevant links on the fly. All the publishers have to do is collect a check. It's close to manna from heaven.
(more by clicking on link below)

This natural evolution of the search engine business closes the loop linking search, content, and ad dollars. In the past few years, marketers of all stripes (about 175,000, at last count) have learned to buy paid listings, or sponsored links placed next to "pure" search results. This phenomenon has created billions in annual revenues and a growth rate approaching triple digits. The reason is simple: Paid search is an incredibly efficient way to bring in sales leads -- it's the Yellow Pages, classifieds, and direct mail rolled into a single just-in-time pitch.

Overture resells its paid link services to portals like Yahoo and MSN. The resulting revenues have accounted for the lion's share of Yahoo's recent earnings success: a net income of $46.7 million in the first quarter of 2003. Yahoo's gain may be Overture's loss; though Overture's revenues shot up 57 percent in the first quarter, its shares fell 30 percent in April when the company pared back its earnings forecast, largely because it plans to spend more to develop contextual ads, and partners like Yahoo and MSN are now getting a larger share of the revenues.

Google -- a private but by all accounts profitable company -- makes a mint from selling paid links on its own site, where it keeps all the revenue. But it also sells its paid search results to AOL (AOL), Amazon.com (AMZN), Disney (DIS), and other Web publishers.

The torrid growth of Google and Overture -- in the face of the Internet "collapse," no less -- presents an extraordinary problem: "The No. 1 question we get from our advertisers is 'Can we get more listings?'" notes Susan Wojcicki, director of product management at Google. "That's one of the main reasons we developed content-targeted ads." The logic is unmistakable: Search is the best-monetized portion of the Web, but it represents a fraction of all usage. Billions of webpages are waiting for a revenue model, and popups for the X10 camera aren't going to do it. Major publishers, like Moore at MSN, have already figured this out.

But it's not with the big guys that I see the real breakthrough. If you're a publisher with a high-quality site about, say, jazz, you can't succeed with banner ads. But you just might be able to make it on paid links for albums and trumpet lessons. Google's offering is limited to sites with large amounts of traffic, like Google Groups, Knight Ridder's newspaper sites, and Blogger.com. But soon contextual advertising will open up Overture's and Google's vast databases of advertisers to the entire universe of Web publishing. Let that soak in: This is a new revenue source for the entire Web, one that not only is unobtrusive but, because it's based on relevance, might even be useful to readers. Contextual advertising "could be much larger than the paid search market," claims Bill Demas, senior vice president at Overture. Google's Wojcicki seconds his assertion. For the sake of independent, high-quality content on the Web, we can only hope they are right.

John Battelle directs the business reporting program at UC Berkeley's Graduate School of Journalism. He founded the Industry Standard and was a co-founding editor of Wired.

Find this article at http://www.business2.com/b2/subscribers/articles/0,17863,515706,00.html

©2003 Business 2.0 Media Inc. All rights reserved.
Reproduction in whole or in part without permission is prohibited.

• Posted by John Battelle at 1:06 PM
• Permalink
• Comments (0)
• TrackBacks (1)

Piece #2: Please, Steve, Buy TiVo

For this piece, I daydreamed about how TiVo might be saved by Steve Jobs. I wish this were the case. With cable coming on strong, I am not feeling so cheery about TiVo's chances.

THE MESSAGE
Is TiVo NeXT?
The beleaguered personal video recorder company is ripe for an Apple takeover.

By John Battelle, May 2003 Issue

Everyone who has TiVo (TIVO) loves TiVo; it is to television what Macintosh was to computing -- a revelation. Which is exactly why Apple (AAPL) should buy TiVo and once again redefine the intersection of culture and technology.

Folks love TiVo for the same reason they loved the Mac in 1984 and the iPod in 2001: It gives control back to the end user. TiVo viewers call the shots regarding when, how, and -- soon -- even where they watch. Once content or access is purchased, the end user is in charge, just like with the iPod.

But unlike the iPod, TiVo and systems like it are in serious trouble. The culprit is the entertainment industry. TiVo has an abeyant Napster-like quality -- and the content business is scared silly that it will not only destroy advertising revenues but become the platform for video swapping on the Internet. Case in point: A coalition of entertainment companies recently sued TiVo competitor Sonicblue into bankruptcy.
(more by clicking on link below)

TiVo's own undoing may come in the form of a recently reported cable service. AOL Time Warner (AOL) (the parent company of this magazine) has realized that its control-based business model will be in jeopardy should TiVo succeed. So last May it neutered its relationship with TiVo (in 2000 it announced plans to invest as much as $200 million with the intent of distributing the services through the now-defunct AOLTV), and it's now focusing instead on a service currently called Mystro TV. Unlike TiVo, which is an intelligent "node" on the edge of the television network, Mystro would be embedded in AOLTW's servers, much like pay-per-view. While it would give viewers some TiVo-like features (pause and rewind, for example), Mystro TV decides which programs can be recorded and whether ads can be skipped. And it requires no storage device, thereby exorcising the ghost of Napster. Comcast (CMCSK), the largest cable provider (Time Warner Cable is the second-largest), has rolled out a similar service.

Mystro is a crippled model, one that in the long run could fail for any number of reasons. (Here are two: Since TiVo knows so much about you, it's a marketer's paradise. And the technology Mystro employs may not scale.) But in the near term, Mystro could strangle TiVo before it can hit critical mass. If customers know they can buy, with a click of the remote, what seems like a similar service for $5 a month, it's unlikely that they'll shell out $299 for a box that's difficult to set up and may be moribund within the year.

Wall Street has caught on to this fact, and despite healthy subscriber gains, TiVo's stock, once at $70, is stuck in the single digits. Some pundits are predicting that the company will soon run out of cash.

So it's time for Apple to step in. Steve Jobs is the only man in techland who can stand up to the content companies on his own terms. Not only does he understand the entertainment industry -- his other company, Pixar, is a Hollywood hit machine -- but he also deeply understands the consumer. Apple's "Rip. Mix. Burn." approach has captured the essence of how consumers feel about music: It's theirs.

Beyond that, Jobs used the iPod to help curb music piracy: The device is wedded to one computer at a time, making tune theft more trouble than it's worth.

TiVo should find a soft spot in Jobs's heart for other reasons. In January, TiVo announced that upcoming devices would use Apple's Rendezvous networking technology to allow TiVo-equipped TVs to play music and display photos stored on a Mac. Also, TiVo is similar to Jobs's erstwhile NeXT Software -- an expensive and risky endeavor, but eerily prescient. When Jobs returned to Apple, he brought NeXT with him, and its core technologies are burrowed deep into OS X, the elegant operating system at the center of Apple's new "iLife" media strategy.

Jobs could do the same with TiVo. With a depressed market cap and nearly 625,000 customers, TiVo is a steal. Jobs would have to unwind some messy licensing agreements, but he's done that before. His next step would be to apply Apple's design elegance and create an "iTV" device that integrates with Macintosh OS X, the Internet, and your cable or satellite box. Talk about a revolution. Once Apple turned on the marketing and PR offensive, we'd have one hell of a Hollywood drama unfolding. And with Jobs in the lead role, it'd be awfully fun to watch.

John Battelle directs the business reporting program at UC Berkeley's Graduate School of Journalism. He founded the Industry Standard and was a co-founding editor of Wired.

Find this article at http://www.business2.com/b2/subscribers/articles/0,17863,515587,00.html

©2003 Business 2.0 Media Inc. All rights reserved.
Reproduction in whole or in part without permission is prohibited.

• Posted by John Battelle at 1:02 PM
• Permalink
• Comments (0)

My First Piece in 2.0

I wrote this a year ago (there's a three-month lag from writing to pub date with monthly magazines). I was sick of the press hammering the internet, with the presumption that everything we did over the past four years was a waste. Turns out, it wasn't.

THE MESSAGE
Alive and Well
From content providers to dog-food retailers, Internet business has moved to a new stage of stable growth. That's the real story -- so why aren't we hearing it?

By John Battelle, March 2003 Issue

Any avid reader of the business press has seen endless variations on this tired theme: Internet business is dead. It was all a dot-con, and it dot-bombed.
Eager for an easy target and brimming with schadenfreude, many business reporters (and a few opportunistic book authors) continue to tear down the Net with nearly the same enthusiasm they displayed while building it up. But the facts tell a different story.
Let's start with the flashing VCR clock of all dot-bomb maxims, the Internet pet-food industry. There's no better proof of dotcom stupidity than the fact that venture capitalists funded not one but at least four pet-food websites at the same time. Thank God for us all, they are dead.
Except ... they're not. In fact, type "buy pet food" into Google and you'll get scores of active merchants selling pet food online. I put in a call to one of them, Geoffrey Walker, CEO of PetFoodDirect.com. Surprise: His business grew 22 percent last year, and he expects similar growth this year. In fact, he and his three or four biggest competitors -- yup, there are still that many players in this category -- are all doing well. As for Pets.com, the now-defunct Sock Puppet company, Walker is thrilled about all that exposure, which let consumers know that they no longer had to lug around 40-pound bags of kibble. Pets.com now redirects to Petsmart (PETM), whose stock price has rung up a 44 percent increase since a year ago. Woof! (for more click link below)

OK, so maybe that dog does hunt, but surely the Internet content business is dead. Nope, publishing on the Web is starting to work. Businesses from Yahoo (YHOO) to the New York Times (NYT) are making money online. Internet advertising -- which had a "terrible" year in 2002, with some $6 billion in billings -- is estimated to grow by 5 to 10 percent or more this year, on track for faster growth than any other medium. And consumers seem to be willing to pay for content: A recent survey reports that they spent $975 million in just the first three quarters of 2002, up from $670 million for all of 2001.

Well, at least the online travel business is a mess. Wrong. Expedia reported more than $30 million in profit last quarter -- not bad in a post-9/11 world.

OK, how about the biggest joke of them all: broadband interactive TV? Fact is, broadband is taking off. Recent figures in the Wall Street Journal show a net addition of 100,000 DSL and cable modem subscribers a week, with total U.S. penetration closing in on 19 percent of all households toward the end of this year. Couple that with the fact that one of the hottest segments in entertainment right now is interactive games, with PlayStation 2 alone selling more than 50 million units so far. Personal video recorders like TiVo (TIVO) and ReplayTV have forced both Hollywood and New York to rethink their business strategies. And lest we forget, MSNBC.com, CNN.com, and others are streaming millions of video feeds a week, most to broadband connections at the workplace.

Certainly the dotcom bubble hurt many individual investors and erased a huge amount of market wealth. There were too many hucksters and too many suckers. Anyone who seriously believed that Priceline at $104 would pay their kids' college tuition has learned a painful lesson. The rest were speculators who knew what they were getting into.

Now-defunct dotcoms tried a lot of dumb things. But that doesn't mean the fundamental ideas they believed in are dumb. Thanks to the more than 100 million Americans who now have access to the Net, and the hundreds of millions more Web surfers worldwide, Internet business has moved on to a new stage of stable growth. That's a real story, and it's only going to get more interesting.

John Battelle (john@battellemedia.com) is a visiting professor at the UC Berkeley Graduate School of Journalism, where he directs the business reporting program. He was the founder of the Industry Standard and a co-founding editor of Wired.

Find this article at http://www.business2.com/b2/subscribers/articles/0,17863,515401,00.html.
  
©2003 Business 2.0 Media Inc. All rights reserved.
Reproduction in whole or in part without permission is prohibited.

• Posted by John Battelle at 12:58 PM
• Permalink
• Comments (0)
• TrackBacks (1)

Note: Column Upload to Commence

Yeah, I've been threatening to post the past year's Business 2.0 columns, and I'm about to start. I've decided to post then in sequence as full text posts to the home page, then classify them as "Columns" using Moveable Type's "Categories" tools. That way, once they've moved off the home page, they will live forever under a "Columns" link on the left. This way, future columns can be posted and added to the list automatically.

(As long as I'm posting site notes, if you can't see a picture in the upper left hand corner, make your browser wider by pulling the lower right hand corner to the right till the picture shows up....ahh, that's better).

So take note, a blizzard of posts (well, about a dozen or so) will follow, and you may want to simply ignore them for the next little while.

• Posted by John Battelle at 12:46 PM
• Permalink
• Comments (0)

Holiday Shopping: Froogle Update

SearchDay summarizes Google's recent update to Froogle. The update is not a total redesign, nor a clear statement by Google that it wants to jump in full throttle, like Yahoo has (Google has yet to take off the "beta" moniker). Shopping engines in the main present something of a challenge for Google - by their nature they invite commercially-driven presentation of results, and Google is best known for its "pure" or "natural" results. As Ninj pointed out to me last week, at Yahoo Shopping they believe that shoppers *expect* to be marketed to while in shopping mode - the metaphor she used was Yellow Pages. Those who paid more, have more credibility and larger ads, and that means something to shoppers. Such an approach is not exactly dead center to Google's overall brand values, but neither does it directly contradict them. It is worth noting that the company updated Froogle in time for the holiday season and has been promoting it for several days in a row. To wit: the phrase "Shopping? Try Froogle - Google's product search service" appears on the main page. Google product managers tell me it's extremely difficult to get a promo for anything onto that home page, as they count every bit on that page, each additional bit slows down performance. Clearly, a decision has been made that this is important to the company, and by extension, to the company's users.

• Posted by John Battelle at 11:53 AM
• Permalink
• Comments (7)

Remember the Internet Generation?

• Posted by John Battelle at 9:03 AM
• Permalink
• Comments (5)

Google, India, Yahoo, Paul

I spent most of yesterday working on the book, talking to folks at Yahoo, Google, and with Paul Saffo. I met with Terry Winnograd, who was Larry Page's professor at Stanford, worked with Larry and Sergey early in the project, and is still on Google's technical advisory board (scroll down). While I was there I ran into Yossi Vardi, who is always a joy to see. The man is always beaming, and this was no exception, he was touring Google's new building with Sergey and telling enthusiastic stories about Sergey's recent trip to Israel (Yossi of course played host). In any case, internationalization is clearly a theme at Google. The company recently announced a new R&D center in India. I asked David Krane (Director of CC) why, and he said he company can't expect everyone to come here, and there's a lot of talent in Bangalore. Makes sense.

Terry is an energetic man, he was a founding member of CPSR and has done a lot of work on natural language and HCI (human computer interface). He recently took a sebbatical from Stanford to work at Google full time, and is now moving back into the academic life. I won't go into all we discussed (gotta save something for the book!) but it was a good meeting, and I heard some funny tales of the early days, among other things.

At Yahoo I met with Srinija Srinivasan, Yahoo's Editor in Chief. She's been there almost since the beginning, and we had a robust and lengthy conversation about the role of editorial in search, the role of the directory at Yahoo (very interesting) and challenges ahead for the industry and the company. I really enjoyed Ninj, as she is known. She's only 32, but has spent nearly 9 years thinking about this stuff as EIC at Yahoo. Prior to that, she worked on Doug Lenat's CYC project, which for those of you unfamiliar with it was a very brave, arguably foolish, and ultimately unsuccessful attempt to solve AI's "brittleness" problem. CYCorp still exists, applying some of its early work to corporate data issues.

Lastly, I spent the early evening with Paul Saffo, who is a walking idea machine. Paul is something of a metaphor graffiti artist - he could earn a living tagging metaphors across anything that catches his fancy - wait a minute, that *is* how he makes a living. He came up with a few really good ones as it relates to the Database of Intentions and various other Big Ideas in my book. Hint: Think Middle Ages.

• Posted by John Battelle at 10:08 AM
• Permalink
• Comments (0)

Marchex

Marchex, a Seattle-based SEO/Pay per click firm headed by folks from Go2Net, has filed to go public. Here's a story on it. This is interesting in many ways - the company is not profitable (though it is cash flow positive), and it's not well known, but it's in the search/advertising space, which is feeling overheated. Worth watching.

• Posted by John Battelle at 9:31 AM
• Permalink
• Comments (4)
• TrackBacks (1)

Glimmers of Hope In Video Search

• Posted by John Battelle at 9:58 AM
• Permalink
• Comments (4)

It's Not The Name, It's The Verb

• Posted by John Battelle at 9:26 AM
• Permalink
• Comments (2)

Yet more funky little search companies

* Brainboost, an Ask Jeeves like Q&A engine (employs "cutting edge AI!")
* Think Tank 23 (finds "ideas")
* Nervana. " You define the context of what you need, and Nervana delivers semantically relevant results from a multitude of sources (such as documents, emails, knowledge communities, intranets, and the Internet). " (via Gary Price's Resourceshelf)

There must be hundreds of these small search-related companies already, with scores more on the brink of launch.

• Posted by John Battelle at 1:08 PM
• Permalink
• Comments (0)

Lycos Top Search Terms for 2003...

And the Top Ten are (with last year's ranking in parens):

1. KaZaA (2)
2. Britney Spears (4)
3. Dragonball (1)
4. Paris Hilton (-)
5. IRS (7)
6. Kobe Bryant (-)
7. Christmas (9)
8. NFL (6)
9. Pamela Anderson (10)
10. Brooke Burke (34)

Check out the various slices and dices in Lycos' version of the Database of Intentions...

• Posted by John Battelle at 12:12 PM
• Permalink
• Comments (0)

Google+Friendster+Plaxo = Eurekster

I've not fully grokked it yet, but here's a new concept in search: Eurekster. From the site: "Eurekster uses the six-degrees of separation concept to learn from your extended network of contacts and deliver you prioritised results based on the success and proximity of the searches they have done." The site is in beta and you need a password to get in, a friend who will remain anonymous gave me his. I signed up to be official, we'll see if I get the nod.

Meanwhile, I see the folks behind this are RealContacts and SLI Systems. SLI is the tech that was behind Snap!, and survived it (and now powers some portions of the NBC webspace).

I can't tell is this is simply silly - take the hottest stuff on the web right now and combine it! - or for real. More when I get smarter.

• Posted by John Battelle at 11:47 AM
• Permalink
• Comments (2)
• TrackBacks (1)

Grokking Groxis' Grokker

So last night was the launch soiree for version 2.0 of Grokker, a search interface tool built right in my own backyard of Marin. It's backed by Paul Hawken, and sports some impressive advisors, investors, and board members, including Paul Saffo and John Seely Brown. Grokker "gives the big picture" on large collections of data - it's a visualization tool, and from what I saw last night, the new version is quite elegant. I saw a demo of Version 1.0 about a year ago and they've really made a lot of progress. I am a huge fan of the word "grok" - from its original Heinlein origins through Wolfe and Wired (where I helped the author of Wired Style write the definition) and the Standard (where we launched a series of newsletters and a magazine called "Grok"). It's neat to see it making another round through the vernacular.

In any case the party was quite subdued by usual product hype standards, with an odd assortment of well-to-do investor types wandering shoeless through an extraordinary Mill Valley home, and Bonnie Raitt also in attendance (Bonnie Raitt?!), sporting a very down-to-earth and pleased-to-be-here manner. Groxis CEO RJ Pittman gave me a demo. Grokker takes datasets created by a keyword or phrase queries on sites like Amazon or "the Web" (it hits six search engines) and runs them through a second filter which displays results as clusters of nested orbs, each with tags derived on the fly from the data. It's quite a seductive interface, and I can see it working for any number of search needs. And as a trend, I applaud this kind of development - building new applications based on search as a platform, rather like an OS. (Tim Bray among others have noted in the past how search today seems stuck at the level of the DOS C: prompt. Groxis might be seen as an attempt to go GUI.)

But the most interesting thing about Groxis is how it is approaching versioning - this release comes after more than a year out in the open, soliciting feedback (some of it quite negative) from the search community. With Version 2.0, Groxis plans to again listen and learn from how the product is used, rather than try to force it into a particular bucket of revenue (though they do have enterprise and licensing deals). This is due to the angel investors behind the company, Pittman said, who are not demanding a rigid, pre-determined approach to how the product will make money. That was exactly how Google did it in the beginning (and, it seems, how Friendster is doing it now). The Mac version will be ready to beta in a few weeks. When I get it, I'll post more.

• Posted by John Battelle at 9:48 AM
• Permalink
• Comments (3)
• TrackBacks (1)

Three Years Ago, Where Were You?

Last night at the Webby Business Awards dinner (caveat, I was a judge), while having a lively conversation about the future of technology and business online, it struck me that it might be interesting to ask the folks at the table not what they thought was coming, but rather where they were three years ago. Because, as it happens, three years ago this month was pretty much when the worm turned in the internet industry. Late 2000, the money ran out, the businesses started to fall apart, the media coverage got bitter and angry, and a lot of people started losing their jobs. Not surprisingly, most of the folks at my table had entirely different jobs three years ago, as did I (there were folks from Google and Salesforce present, and as you might expect, they were in fine fettle).
Three years ago I was running the Standard, facing the first layoffs in the company's history, and dealing with a dramatic reversal in our advertising forecasts: a drop of about 75%. So where were you, and what were you doing? And...more importantly, how are you doing now? Are you happier? Interestingly, every single person I asked that question last night said they were. Worms keep turning, and eventually the soil again bears fruit.

• Posted by John Battelle at 5:41 PM
• Permalink
• Comments (1)
• TrackBacks (1)

It's "Like," The Pioneers Get the Arrows

I'm scanning my media news this morning and I see an item titled "Cablevision to Offer Tivo-Like Service." Sigh. It's so sad to watch a great idea, and a pretty good execution, fade right in front of your eyes, as pale imitators create "Tivo-like" services.

The problem with this, as I've ranted before (in a previous 2.0 column that I swear I am about two days from posting here), is that while TiVo puts its intelligence at the edge of the cable network, ostensibly under your control, the cable company boxes have opaque, client-server architectures, owned and controlled by the cable system. In other words, they decide what you can fast forward, record, cut and paste (well, you can't do that anyway), etc. It's a terrible, closed system, and because cable companies are monopolies, for the most part, there's not much to be done about it. It's Larry Lessig's worst nightmare (and mine too).

And how might the cable companies handle their newly found control over your PVR? Well, here's a hint from Tom Rutledge, president of cable operations for Cablevision: "We didn't think it was a good idea (to add PVRs to the system) and didn't want to do it," Rutledge said. In other words, we are with the MPAA and the content industry on this one, and are only doing this because the market is asking for it. Endgame: If you get a cable PVR, expect the keys to be in someone else's hands. My suggestion: go satellite and buy a TiVo. This year might be your last chance.

• Posted by John Battelle at 11:15 AM
• Permalink
• Comments (3)
• TrackBacks (1)

The RSS Naming Party: Chunky or Smooth?

Clearly Searchblog's most popular post of all time (OK, so we're only a month old) is this one, in which we attempted to take Scott Rosenberg up on his challenge for a better name for RSS. There is even a contest happening over at another site, and I am sure this meme will evolve for some time to come. So far the conversation has included names as far fetched as "chunkyweb" (the web was plain, RSS makes it chunky), Feedcast (which I rather like), Flow (which prompted some interesting comments about gushing loins), Readster (more of a product name), and a thought that perhaps what we are really looking for is a better name for the Semantic Web. I say, keep on talking. It's fun to whiteboard ideas this way.

• Posted by John Battelle at 1:10 PM
• Permalink
• Comments (0)

Cnet Gets Some Good News

An interesting coup for Cnet: MediaPost reports it topped all other search engines, by a rather wide margin, in a recent WebSideStory study which compared vertical shopping engines on a key metric: whether visitors from the engine who clicked through to a commerce site actually convert into buyers. Cnet's conversion rate of 1.64% was better than AOL, the #2 engine, by 28%.
I'd like to know why this is. I've sent a note to folks at Cnet, and I congratulate them - it's been a rough few years for them as they weathered the tech nuclear winter, but I've always had a soft spot for the company - it launched roughly at the same time as Wired, and has really seen it all.

Order Conversion Rates: Searches To Gift/Electronics Sites

Conversion Ratio
CNET 1.64%
AOL 1.28%
Overture 1.25%
iWon 1.15%
LookSmart 1.14%
MSN 1.04%
Yahoo 1.01%
Netscape 0.93%
Lycos 0.92%
Google 0.85%

Update: I must've been rushed and didn't think this through, as I was driving to a Webby business awards tonight I realized of course Cnet has better conversion because it's a vertical portal that aggregates folks with intent to buy particular technology items. Matt points this out in the comments section as well.

• Posted by John Battelle at 12:54 PM
• Permalink
• Comments (1)

Spookle

Google has landed a deal with the CIA for its enterprise search solution, Government Computer News reports via IESDB. I hope this doesn't meant the CIA will next set its sights on Google's version of the Database of Intentions....

• Posted by John Battelle at 12:25 PM
• Permalink
• Comments (0)

Blocking Paid Search Ads

.
EWeek reports that a new software application is now available that blocks paid search ads on most popular search engines. I wonder if this will take off, or is ephemeral? It depends on how consumers react to the next wave of search engines - how Yahoo and MSN do their next implementations, for example. I don't think anyone really objects to how Google does ads right now, in fact, I think most feel the ads are even useful. I'd be curious to know if any readers are using this product and if so, if it in fact is useful. For more on Intermute, the company making this software, here's their site.

• Posted by John Battelle at 12:19 PM
• Permalink
• Comments (1)
• TrackBacks (3)

The Search Papers: Bray on Search

Tim Bray has a series called On Search over at his Ongoing blog, and I find it worthy of a read'n'muse. He starts with this backgrounder on himself and search issues as he sees them, and has a ton of entries on any number of subjects, too numerous to go into here. Highlights: he writes on interface issues (warning, not for the faint of geek), how best to search XML (answer: we don't know yet, recall he was a co-author of same), and on result rankings, with a quick refresher on why PageRank works, and good advice on paying attention to your own logs. Also worthy: his primer on how search works, and his discussion of the technical search terms precision and recall (with an interesting note on the absence of top companies in the research community - see my post on this here), and lastly (whew), his mini-rant on intelligent search, and why it's a long way off. An excerpt:
"If we want better search (and we do), we’d better not count on AI voodoo or linguistic juju or semantic mojo. We need to work with good sound statistical techniques, and be clever about generating and using metadata, and we need to get our APIs right. All of these things are hard, and there is good work being done in all of them."

• Posted by John Battelle at 12:59 PM
• Permalink
• Comments (1)
• TrackBacks (1)

Jeremy Z. on 2004...

...and with his conclusions, I must say I agree....

• Posted by John Battelle at 12:07 PM
• Permalink
• Comments (0)

The Search Papers: Challenges in Web Search Engines (A Google Paper, 2002)

This paper "presents a high-level discussion of some problems in information retrieval that are unique to web search engines," according to its abstract in the ACM library. (A reminder as to what this whole "Search Papers" thing is about: read this.) "The goal is to raise awareness and stimulate research in these areas," it continues. How might such a lofty incitement be backed up? Well, it's written by two senior employees of Google, Monika R. Henzinger and Craig Silverstein (I've met with Craig, he was employee #1 after Larry and Sergey, and a nice guy to boot), as well as Rajeev Motwani, a professor at Stanford (Craig was his graduate student).

The paper is dated September, 2002, so it does not rank as a missive from the early, more geeky phase of Google's life, but rather a more corporate product - the two Google authors knew they bore the weight of "being Google" when they wrote this paper, and it's worth keeping that in mind when reading through it.

This is particularly clear in the paper's scope and focus. It lays out six challenges for search engines - and they read like a laundry list of Google's headaches. The paper then goes on to offer suggested paths for more research on the topics, which I could imagine might read either as genuine or a tiny bit patronizing, depending on who you are. (The paper does not tackle a range of other issues it says are already the subject of abundant research - natural language queries, image/audio search, improving text-based retrieval, language issues, or interface/clustering, for example.)
(more in the extended entry, click link below)

First among the stated problems is spam - folks who try to game search engine listings for their own commercial gain (this is clearly Google’s biggest problem, dominating a lot of their time). Second and third are content quality and quality evaluation - how to determine the relative value of content on a web page, and how to determine if your algorithms w/r/t same are working. Fourth is something they call "web conventions" - how to create useful search engines given the fact that the web follows loose conventions rather than strict rules. Fifth is the problem of duplicate hosts – two hosts that serve the same content (eliminating these would unclog search results, Google has sometimes been criticized by its competitors for having too many duplicate pages). And sixth is the wonderfully termed “vaguely structured data” – XML is mentioned, but dismissed – the authors instead suggested there is value in understanding conventions of HTML presentation (the way a page looks) and somehow using that to make searches better.

So as not to bore the lot of you, I won’t go into the detail on each. Suffice to say, this paper was interesting and a worthy read if you are a student of the company and/or the field. I have only now begun to read the more recent public papers from Google scientists, so I can’t compare them as a corpus. A few notes: It’s not clear who this paper was really written for, as there are notes that seem for less technical readers (ie one note explains what a crawler is – are there really folks in the research community who are not web savvy?). The paper toots Google’s horn a few times (it says PageRank is not vulnerable to some types of spam), admits where Google has weaknesses, gives props to Jon Kleinberg’s HITS algorithm (upon which some say PageRank is based) and even seems to float some trial balloons to the research community (on how to detect spamming tactics, for example, in section 2.4). I did take issue with some of the editorial assumptions in the “Content Quality” section, but I won’t go into all that here. Drop me a line if you want to discuss. And…if you are a researcher in this field, or know one, I’d be interested in what the academic community thinks of this paper, and any others I post on as well. Thanks!

• Posted by John Battelle at 11:53 PM
• Permalink
• Comments (0)

Blogging For Dollars

• Posted by John Battelle at 9:19 AM
• Permalink
• Comments (7)

A Morning with Brewster

• Posted by John Battelle at 9:35 AM
• Permalink
• Comments (0)

RSS Pushed One Step Closer to the Limelight

Funny how am idea gathers momentum. As I was penning my Implications of RSS For Business column for 2.0 (awaiting publication in dead tree form in three weeks), Scott Rosenberg was writing a pean as well, published in Salon this morning. He suggests we need a name for what RSS represents, just as the Web became the mainstream's understanding of HTML, we need a name for RSS. He reminds us we've been here before (remember Push? I was a reluctant contributor to this 1996 article, which began as an email thread in the Wired offices...)

In any case, I agree with Scott, we need a name. All the businesses in this space are still in the pre-market phase. RSS allows us to connect more efficiently, to grok information as we like it, when we like it - but what do we call it? I like to say my reader and blogs/news sources is my personal ecology - is there an idea in there somewhere? In any case, it's exciting to see the idea start to take popular flight. Watch for the NYT treatment soon.

• Posted by John Battelle at 8:44 AM
• Permalink
• Comments (37)
• TrackBacks (2)

Zeitgiest in Action: "Miserable Failure"

Type "miserable failure" into Google. Out pops Google's top pick: George W. Bush. This comes to me via Farber's IP list, but many others have crossed my desk over the past year, including one on "weapons of mass destruction" which - if you hit "I'm Feeling Lucky" - still gaves a 404: Not Found-like error thanks to the vagaries of web humor, linking, and PageRank.

UPDATE: This was a case of Googlebombing, see here....

• Posted by John Battelle at 7:38 AM
• Permalink
• Comments (0)
• TrackBacks (2)

Monier, Yahoo, Gadgets

Today I bounced around a bit, from a very stimulating two hours with Louis Monier, founder of Alta Vista and current head of R&D/Search at eBay, to attending a CMO roundtable discussion at Yahoo where co-founder David Filo spoke, to a Churchill Club dinner in SF where Walt, Kara, Greg Harper and Larry Page talked about their favorite gadgets.
I asked Louis to react to the ideas in my monoculture post - that the best minds are now silo'd in private corporations. He had a great response: "Airlines are not built by the academy." In other words, very complex and expensive stuff by definition is done by the private sector. Monier was, in fact, a veritable quote machine, and were it not so late, and were I not so tired, I'd create a whole post on the subject. Perhaps tomorrow. One more: "Google is, I think, the intellectual descendant of Alta Vista....I always said, with search you need a sharp pencil, that's all." In other words, don't give me more than I need, just focus just on solving the search problem. It is true, Alta Vista was briefly THE search engine that mattered - about 1996-97, before it was ruined by Compaq and later CMGI.
The CMO discussion focused on Yahoo's brand. It featured a panel of senior Yahoo execs talking about what makes the brand special, with an emphasis on how they live the brand from the inside out - how the company and its employees act in a way consistent with what the brand means to the outside world. A bit flat, but there was some neat stuff - I always like to hear Libby Sartain speak (she's the head of HR there) - she's infectious and quite inspiring. David told the early founding story, which was good to hear from the founder's perspective.
The Churchill Club was livelier. There's nothing like geeks with toys to get a crowd going, and Larry for one outdid himself. He really does love this toys, and seemed quite in his element - relieved, I imagine, to for once be in a public speaking role where he does not have to directly bear the weight of being a founder of Google. Cool stuff included a blue tooth ear plug for cell phones, LED flashlights (no bulbs, quite bright, long battery life), a 4 million pixel webcam that is its own POP server and looks rather like HAL from 2001, and touchless, digital toilets from Japan that require no toilet paper (they wash and dry you at the touch of a button. Really.)

• Posted by John Battelle at 11:00 PM
• Permalink
• Comments (0)

Welcome FeedsterFolk

So I am offline today (down in the valley meeting Louis Monier and various folks at Yahoo) and I checked into a local Starbucks, and hey, I'm the feed of the day on Feedster! Welcome all of you who might stop by thanks to the pointer. For a tour of what the site's all about, check out this post on the book and for more on me, see this as well.

Thanks Scott!

• Posted by John Battelle at 2:57 PM
• Permalink
• Comments (0)

On RSS, Blogs, and Search

I've been thinking lately about the role of blogs and RSS in search, and that, of course, has led me to both the Semantic Web and to Technorati, Feedster, and many others. Along those lines, I recently finished a column for 2.0 on blogs and business information. I can't reveal my conclusions yet (my Editor'd kill me) but suffice to say, I find the intersection of blogging, search, and the business information market to be pretty darn interesting.
I'm certainly not alone. Moreover has created "Enterprise-Grade Weblog Search" - essentially, a zietgiest mining tool for corporations. One can imagine similar products from any of the RSS search engines, or even from the major marketing agencies of the world. On the other end of the spectrum (making blogs easier to read for consumers, as opposed to easier to mine for product marketers), Meg Hourihan is fast at work on Kinja, which is going to be a blog of blogs that will make finding and following blogs easier. Bloglines is doing similar work.
What makes this interesting from the perspective of search is the structured nature of what is being searched - blog postings and news articles, for the most part (or maybe I should say the data is "vaguely structured" - a term in a paper I just read - more on that later). Anyway, I sense a pretty potent market shaping up. Anyone know of other folks, companies, or news I should know about?

• Posted by John Battelle at 6:03 PM
• Permalink
• Comments (4)

A Dance, Or A Hurricane?

For those of you who don't follow the vagaries of search engine index updates, the past few weeks may have been pretty uneventful. But for the businesses and marketers who make their living by ranking well in Google's listings, it's been a pretty tumultuous month. That's because once again the Google Dance has swept through the search engine markets, and this last one was unique.

"Google Dance" refers to the process by which Google updates its index - the master code that determines which listings you see as a result of your queries. Because it is a massive index, and because Google often wants or needs to incorporate various tweaks and refinements to its site-ranking secret sauce (in particular to fight spam), it can take days or even weeks for a new index to settle across the web. The folks over at WebmasterWorld track this stuff quite closely, and have taken to naming each update alphabetically, following the nomenclature usually reserved for hurricanes. This past one happens to be F, and has been dubbed (dub-ya'd?) Florida.

When the Google index dances, many sites which once enjoyed top listings can fall off the first page of results, for seemingly arbitrary reasons. Danny Sullivan has written extensively about this, (including today's newsletter) as have many others. What makes this most recent update fascinating is that for the first time, Google watchers have figured out a somewhat consistent way to track what's changed since the last update, and many of them are crying foul. They're claiming that Google has tweaked its algorithms to favor terms which have commercial value, and to prove it, they've hacked up "Scroogle," a tool that shows the before and after results for any given query. Danny's given an interesting example in his article of what happens to the query "laptop rentals" in a before and after test.

While a case might be made, I can't imagine Google is favoring its own commercial interests over its stated philosophy of focus on search, the profits will follow. It'd be suicide for a company that's made its name on "not being evil." The uproar over the Florida update, however, points to the increasing responsibility the company bears as a holder of public trust. Updates like Florida have major implications for an entire economy of Google-dependent merchants, and the two parties (Google and the merchants who depend on its listings) are only growing more interdependent. A dance, indeed.

All the braying about Florida aside, it's interesting to note what you get when you search for "laptop rentals" on MSN. At least MSN is consistent - whoever pays MSN the most, you see first!

• Posted by John Battelle at 5:03 PM
• Permalink
• Comments (1)
• TrackBacks (1)