Viewing the news as data

adrian holovaty at ONA 2007Live from the ONA conference in Toronto…

Adrian Holovaty looks at a photo of the world’s ugliest dog and sees 1s and 0s.

Displaying a photo of the hideous animal, Holovaty –the big brain behind and the Python framework Django— says there are loads of data in the ugly dog image. Who/what is the subject? Who took the photo? Where was it taken? When was it taken? What kind of camera was used? What colors are in the photos?

So how can that image, if tagged with metadata, give your site a big advantage? A good example is Flickr, which allows users to search photos by all sorts of non-traditional criteria. The result is a site that is stickier and allows a high degree of browsability, a trait that users nowadays are beginning to view as essential, Holovaty said.

News organizations have reporters attending city council meetings, high school sports games and covering local crimes — something Google doesn’t or can’t do. But rather than just having reporters gather facts and fuse them into a “blob” that is unreadable by machines (aka, a news story), Holovaty wants to also see news organizations compiling that information into a database format that can be easily browsed by users.

“We have all those killer advantages, but the tragedy is that we haven’t actually leveraged the information we collect,” Holovaty said.

How to get started

Being that crime databases are all the rage these days –no thanks to Holovaty– here is a set of processes and tips to get one rolling:

-Analyze the raw data you get from the police department.

-List the data’s fields (Date, time, crime type, address, etc.)

-Identify the key concepts. If a user clicks on a field, is it useful to that user to see the data sorted by that criteria? For instance, browsing by date ishelpful, but case numbers are unique and not really browseable.

-Make list pages with multiple records that are browseable by a certain criteria.

-Then, make detail pages for individual crimes.

-Every piece of information needs to have a permalink. Linkability/bookmarkability is critical, not just for users but also for search engines. “Your Google juice will go up,” Holovaty said.

online news association ona logoLive from the ONA conference in Toronto…

Lots of cool folks are now blogging the Online News Association conference:

-Amy Webb is gay meeting areas near me on the blogging ethics working group going on right now (I know this because I’m peeking over her shoulder… muahaha).

-Bryan Murley dating sites south africa and even took a picture of couple seeking male.

-BlogTalkRadio has been broadcasting live from the conference.

-Steve Safran over at Lost Remote has touched on some of the community journalism issues.

-Matthew Ingram blogged the future of news panel (and entirely on a Blackberry no less).

-Alfred Hermida blogged the future of publishing panel, focusing on mobile devices.

-JD Lasica –who showed off his mad moderator skills during the community evangelist panel– blogs about the future of news discussion.

-The student newsroom is chugging away with stories, photos and video.

-And last but not least, we’ve got Tweets galore here.

Getting started with online data

david milliron at ONA 2007, TorontoLive from the ONA conference in Toronto…

This shall be the year of the “data center.”

Gannett’s online data initiative has hit it big with online news industry types who are now itching to put up databased treats such as school report cards, crimes, property sales, public employee salaries and restaurant inspections.

Easier said than done.

David Milliron, now at Caspio but formerly a data guru at the Atlanta Journal-Constitution, provided some insights on what’s needed to get a news organization started putting up great searchable databases.

Of the many things about which Milliron spoke (getting advertising on board, worrying about server load), the thing that struck me the most was the need for getting educated the right way and having a solid network of other pros to lean on. When the crap hits the fan and your site goes down because of a bad query, it’s essential to have the “under the hood” knowledge of how the database and application work. That means understanding the programming language behind the framework, Milliron said.

Furthermore, having a network of pros –whether its at your shop or otherwise–means you tap those other folks’ wisdom, have them give you great ideas on how to write good, efficient code and have them spot-check your work. For some without a mentor in-house, that may mean finding a local users group or joining a listserv (such as the ones offered by NICAR and other organizations).

Some things Milliron says you need to consider before starting database projects:

-Are there other people in your shop or at other shops in your industry developing with the programming language and framework you’re picking?

-Can you get someone knowledgable to review your code?

-If the person developing your applications gets hit by a train or hired away, will you be able to bring in someone else?

-What are the costs and time associated with maintaining the database, not just what it takes to slap it up.

As Milliron said, it’s a rare thing to find one person who knows how to gather data, analyze it, clean it, draw meaningful conclusions from it, put it up on a server, build a rocking database application and then fix it when the thing blows up. And if you do run across such an individual, they probably already have a startup or work at Google…

Now if you want to get your hands dirty with code, Milliron suggests checking out your local community college or professional development programs for database classes. That comment struck me because yours truly has been going to a local community college since August to learn some honest-to-goodness programming (in C++ of all things). I can tell you that after banging my head against a PHP/MySQL book for weeks and making little progress, having a pro hold my hand through those concepts helped me build a useful PHP app for our web producers in about four hours.

Creating interesting, functional, user-friendly databases is a big undertaking. Just don’t be afraid to get your hands dirty, and don’t fear failure. You will probably write a bad query, crash a server and have some whiz kid rewrite your 70 lines of code into seven.

But if it means gaining the knowledge to build great apps, it’ll all be well worth it in terms of page views for your site, value for your readers and –most poignantly– your job security.

Journalism is becoming a high-tech industry

Live from the ONA conference in Toronto…

Lisa Williams, founder of H20Town and Placeblogger, says journalism is becoming a high-tech industry that is moving toward the way technology firms operate.

“You guys have mooched much closer to me,” said Williams, a veteran of the technology industry.

When Williams began H20Town, she took delight at working faster than the local newspaper and gaining an audience “because I could hit ‘Enter’ first.” But now, she says, the local newspaper is publishing breaking news and implementing more Web 2.0 tools.

Williams offered several strategies for media organizations trying to navigate the online world. Among my favorites:

-Take something that used to cost money and make it free. Why let the next Craig Newmark steal your readers?

-The Web rewards “narrow comprehensiveness.” Create a product that is “everything about something” very specific.

-Limits are good. Because newspapers try hard to be all things to all readers, media organizations subsequently port that same mentality to their online products. Instead, focus on creating great, key features that will gain many fans, such as Twitter has done with their service. Get more info at carrefour-maires .

I think most mainstream media organizations have slurped the Kool-Aid and realized that we need to embrace technology. But how?

Sitting in this room full of journalists from some of the biggest news organizations in the world, I can’t help but realize that we just might be somewhat like the blind leading the blind. Nearly every group of journalists I come across asks the same questions. We desperately need to seek out the wisdom of other industries to help us navigate through the dark.

Let’s be totally shameless and emulate the ways technology companies operate. Let’s start swiping great talent from Apple, Microsoft and Google. Let’s gain some street cred among technology enthusiasts so they can help us evangelize our journalism. Let’s become educated about the techniques used to develop software. And, as many others have said before, let’s nurture real entrepreneurship in our news organizations backed with actual rewards.

Twin Cities Daily Planet rounds up niche papers, takes on the Star-Tribune

Live from the ONA conference in Toronto…

twin cities daily planet A coalition of small, niche community publications can become a premier source of news in Minnesota, says Jeremy Iggers, director of of the Twin Cities Media Alliance at

The Twin Cities Daily Planet –inspired by OhMyNews — “is conceived as an experiment in participatory journalism, built on a partnership between professional journalists and individual citizens. One goal of the Daily Planet is to harness that community intelligence and enable individuals to share information and work together for the common good. [More here.]”

The Planet partners with other small, niche media outlets to cover Minnesota — specifically areas they feel are underserved by the Minneapolis Star-Tribune. Iggers labeled his former employer, the Star-Tribune, as “the newspaper of the most desirable zip codes” for its giving coverage priority to affluent neighborhoods.

Currently, TCDailyPlanet is receiving an assortment of grants, including a Knight-Batten award and a grant from the McCormick-Tribune foundation. They also plan to hand out small amounts of funds –in $50 to $100 amounts– to assist some of its partner publications in publishing stories that have a broad appeal.

But the question, as always, is whether TCDailyPlanet –and publications of its ilk– can sustain financially in the long run. A call for financial help from the public resulted in only about a dozen donations, Iggers said. He hopes that can adopt the NPR style of public fundraising within the next two to three years.

Will online niche community publication grow to the point where such ventures can be solidly profitable? Or will TCDailyPlanet and the many microlocal blogs remain passion projects dedicated to being an alternative to the big newspapers and TV stations in town? Passion project or otherwise, it’s the readers who likely stand to win.

ONA conference begins tomorrow

online news association ona logoAfter a day of carousing about Toronto, being nearly attacked by really fat, hungry seagulls at Nathan Phillips Square and experiencing the tender mercies of “We Will Rock You” (featuring a performer flashing her maple leaf-clad underwear at the crowd), I’ll be heading tomorrow morning to the 2007 Online News Association conference.

I can feel in my bones that the CitMedia workshop I’m attending tomorrow is going to be awesome. Gannett’s Jennifer Carroll,’s Rob Curley, NYU’s Jay Rosen, Morris Communications’ Steve Yelvington and many more are going to be speaking. That’s more community journalism knowledge in one room than you can shake a stick at.

Web connection and time permitting, I’ll be blogging from the conference. John Havens at BlogTalk Radio will be recording and streaming live interviews from the conference.

Also, if you’ve got any suggestions for good eats or not-so-touristy things to do here in downtown Toronto, drop me a line in the comments, ay!