8 Ways to add Data Features to your Site

Computer-assisted reporting whiz Matt Wynn from the Arizona Republic has published an awesome list of eight ways to get interactive data on your site.

Matt has essentially reviewed various vendors that offer embeddable databases (such as Caspio and Zoho), as well as gone over some of the more hands-on options, such as directly using programming languages and their frameworks.

I’ll add my two cents to Matt’s post: One of the best way to truly grow your site and do innovative (and revenue-gaining) work is to hire some smart, dedicated developers for your site. Vendors, though useful for certain projects, are only going to get your site so far. Give those developers the tools they need: time, training, exposure to different departments, cover fire and a reliable server with which to work that isn’t under choking restrictions. Then, you’ll be cooking with gas.

[More: Matt Wynn: Eight ways to get interactive data on your site]

Catnip for Online Designers with SND’s Best of Multimedia Entries

SND VegasWant to see the best online information design the news industry is producing? Then you might want to tune into the SND Update Blog for the next few days as SND highlights entries from their worldwide Best of Multimedia Design competition.

There’s more awesome design here than you can shake a t-square at, so make sure to take a look!

Check out the entries from:

Entertainment/Lifestyle Off Deadline (The category with the most entries)
Breaking News (Most of the entries here seem to allow advance time to produce, but really nice work nevertheless.)
Lifestyle On Deadline
(Only one survivor here)
Non-Breaking News
Sports Off Deadline

[More at the SND Update Blog]

Google’s ‘Knol’ is Direct Challenge to Media Companies

Google today announced the launch of Knol, a Wikipedia-esque site that hosts articles from contributors. Google hopes the articles become among the most authoritative on the Web — which represents a direct challenge to media sites.

In a June 2006 interview with the LA Times, Google CEO Eric Schmidt was asked whether Google is a media company or a technology company. His response:

“It’s better to think of Google as a technology company. Google is run by three computer scientists, and Google is an innovator in technology in our space. We’re in the advertising business – 99% of our revenue is advertising-related. But that doesn’t make us a media company. We don’t do our own content. We get you to someone else’s content faster.

If today’s launch of Google’s Knol is any indication, this line of thinking has fundamentally changed. Google, in short, is becoming a full-fledged media company in direct competition with established news and knowledge sites.

Knol –short for “knowledge– is Google’s new Wikipedia-esque site that hosts authoritative articles on a wide variety of subjects. Knol recruits contributors to write articles on subjects such as medical conditions, sports and more. Article authors earn money by running AdSense campaigns on their content. Knol also offers a suite of collaboration tools that allows other users to suggest changes to the original article.

This move is a fundamental shift from Google’s traditional directive of helping users find content, as opposed to creating and hosting the content. It is a shift that has continued as Google acquired Blogger, launched Google Page Creator, allowed users to publish documents with Google Docs and began hosting Associated Press articles and user commentary on its Google News service (as opposed to linking to AP affiliates’ stories and leaving comments to the news sites).

However, Google also brings an enormous amount of traffic to news sites — traffic that means big advertising dollars. Most media companies worth their salt have significant search engine optimization efforts in place to make sure those who seek information are likely to find it on a news site. It’s for that reason that news organizations’ view of Google approaches the realm of bipolar disorder. News sites beg for the Google traffic but are also being encroached upon by Google features, such as Knol and new search boxes that let users bypass news sites’ own search features (which does help people actually find stuff for a change).

The Guardian’s Jack Schofield summarizes it well when he writes that “Knol represents an attack on the media industry in general.” TechCrunch’s Michael Arrington believes that Knol may be “a step too far.” Search engine expert and author Danny Sullivan also agrees, saying that:

“[Google’s] Search, Ads, & Apps mantra that CEO Eric Schmidt has repeated on several occasions underscores that offering content tools is fair game within its mission. But does hosting content turn it into a competitor with other content providers and set up an unfair advantage in gaining traffic that might otherwise flow to them?”

Knol also represents a potential conflict of interest in Google’s own search results. If Knol articles are meant to be “authoritative articles about specific topics,” those familiar with search engine optimization will see the red flag. Google’s incentive to make Knol articles the most authoritative on the Internet puts it in direct competition with topic-specific columnists, news stories on a plethora of subjects and web sites such as About.com and Wikipedia. Because Google itself creates the algorithms that define what is “authoritative,” Google would have an unfair advantage over other sites — even if it is simply in terms of using in-house knowledge as opposed to somehow altering the actual ranking algorithms.

Nevertheless, BuzzMachine’s Jeff Jarvis has made the point that it is unproductive to argue whether Google is a friend or foe. He is correct in that we should try to emulate Google, rather than become disgruntled and blame the news industry’s woes on the search giant. Google’s endeavors at hosting and creating content doesn’t mean news sites can’t continue the symbiotic relationship with search engines.

But make no mistake; Google isn’t becoming a direct competitor to traditional news and media sites. It already is.

Audio from Future of Web Apps Miami available

fowa-thumb.gifJust got an e-mail letting me know that audio from the Future of Web Apps Miami conference is available. Woot!

For us online news types, these talks are a great chance to get exposed to what’s happening in web technology and to think about how we can apply it to our situation.

These were the talks I thought were the best IMHO:

Cal Henderson from Flickr talks about the software development process and gives boatloads of excellent advice you need to hear if you’re developing applications.

Gary Vaynerchuk from Wine Library TV talks about the importance of passion in growing a community (this one really applies to news sites).

Kevin Marks from Google talks about Google’s OpenSocial and the future of social networking.

Blaine Cook from Twitter talks about the future of Twitter and its architecture.

Leah Culver from Pownce talks about opening your web app to the masses (something news sites need to get better at, without jeopardizing their revenue that is..)

Microsoft to give away development software for students

visualstudio.pngFor all you aspiring journo-programmers, Microsoft has a treat for you: free software! Woot!

Microsoft is launching a new initiative, DreamSpark, to offer up development and design software free to students, probably in hopes of weaning them from open-source solutions. The following expensive software will be offered free to students as part of the program:

It doesn’t get any better than pricey software for free, my friends, especially if ASP.NET (Microsoft’s flavor of Web development) is your bag. Kinda makes you want to go take a community college course for that student ID just to take advantage of this free stuff (plus 2 bucks off movies, of course).

Journalistopia crime map directory updated

crime-maps.jpg

Hey Journalistopia denizens, just a quick heads up that I’ve been updating the Journalistopia Crime Maps Directory with all sorts of great, new stuff ever since it first appeared back in September. News organizations have definitely been embracing crime data in a big way.

So check out the new additions, and keep the submissions rolling! Crime map directory is here.

Adrian Holovaty’s EveryBlock goes live

everyblock.gifAfter being awarded a two-year Knight Foundation grant, journo-programmer maximus Adrian Holovaty has launched the highly anticipated EveryBlock.com, a site dedicated to gathering as much data about communities as possible. Holovaty has often preached about viewing the news as data. Well folks, you’re lookin at it.

(NOTE: Poynter’s Al Tompkins has posted an interview with Adrian about EveryBlock here.)

EveryBlock features news stories, crime reports, user photos, business inspections and a whole lot more, all geocoded and highly organized. The navigation style is reminiscent of ChicagoCrime.org, which means it’s highly browseable and not dependent on search fields and dropdown menus.

What’s awesome about EveryBlock is the sheer amount of data that’s being collected and aggregated. You can spend quite a while jumping fluidly from one kind of data to another. And, the effort it must’ve taken to acquire all of the site’s data in an automated fashion is a big achievement in itself. UrbanBurger can guide in a better way.

However, I notice something that I’ve also encountered in my work on Orlando-area neighborhood pages and data features: It’s tough to put all of that data into context and provide more historical information such as a community’s history, landmarks and evolving story. For instance, having a highly detailed view of crimes in a neighborhood is really cool, but how does my neighborhood compare to another? How is crime in the neighborhood trending? That’s going to be the next big challenge for news organizations who want to do features such as this.

Nevertheless, EveryBlock is an awesome effort that bears close study. It’s a tremendous exercise in how to aggregate huge amounts of news data and organize it in a digestible fashion. Sigh, if only Clothes sites were as well organized as the swiss avenue dot com.

Go check out EveryBlock, and drop feedback to Holovaty and crew here. Big kudos to the EveryBlock team for a successful launch.

Viewing the news as data

adrian holovaty at ONA 2007Live from the ONA conference in Toronto…

Adrian Holovaty looks at a photo of the world’s ugliest dog and sees 1s and 0s.

Displaying a photo of the hideous animal, Holovaty –the big brain behind ChicagoCrime.org and the Python framework Django— says there are loads of data in the ugly dog image. Who/what is the subject? Who took the photo? Where was it taken? When was it taken? What kind of camera was used? What colors are in the photos?

So how can that image, if tagged with metadata, give your site a big advantage? A good example is Flickr, which allows users to search photos by all sorts of non-traditional criteria. The result is a site that is stickier and allows a high degree of browsability, a trait that users nowadays are beginning to view as essential, Holovaty said.

News organizations have reporters attending city council meetings, high school sports games and covering local crimes — something Google doesn’t or can’t do. But rather than just having reporters gather facts and fuse them into a “blob” that is unreadable by machines (aka, a news story), Holovaty wants to also see news organizations compiling that information into a database format that can be easily browsed by users.

“We have all those killer advantages, but the tragedy is that we haven’t actually leveraged the information we collect,” Holovaty said.

How to get started

Being that crime databases are all the rage these days –no thanks to Holovaty– here is a set of processes and tips to get one rolling:

-Analyze the raw data you get from the police department.

-List the data’s fields (Date, time, crime type, address, etc.)

-Identify the key concepts. If a user clicks on a field, is it useful to that user to see the data sorted by that criteria? For instance, browsing by date ishelpful, but case numbers are unique and not really browseable.

-Make list pages with multiple records that are browseable by a certain criteria.

-Then, make detail pages for individual crimes.

-Every piece of information needs to have a permalink. Linkability/bookmarkability is critical, not just for users but also for search engines. “Your Google juice will go up,” Holovaty said.

Getting started with online data

david milliron at ONA 2007, TorontoLive from the ONA conference in Toronto…

This shall be the year of the “data center.”

Gannett’s online data initiative has hit it big with online news industry types who are now itching to put up databased treats such as school report cards, crimes, property sales, public employee salaries and restaurant inspections.

Easier said than done.

David Milliron, now at Caspio but formerly a data guru at the Atlanta Journal-Constitution, provided some insights on what’s needed to get a news organization started putting up great searchable databases.

Of the many things about which Milliron spoke (getting advertising on board, worrying about server load), the thing that struck me the most was the need for getting educated the right way and having a solid network of other pros to lean on. When the crap hits the fan and your site goes down because of a bad query, it’s essential to have the “under the hood” knowledge of how the database and application work. That means understanding the programming language behind the framework, Milliron said.

Furthermore, having a network of pros –whether its at your shop or otherwise–means you tap those other folks’ wisdom, have them give you great ideas on how to write good, efficient code and have them spot-check your work. For some without a mentor in-house, that may mean finding a local users group or joining a listserv (such as the ones offered by NICAR and other organizations).

Some things Milliron says you need to consider before starting database projects:

-Are there other people in your shop or at other shops in your industry developing with the programming language and framework you’re picking?

-Can you get someone knowledgable to review your code?

-If the person developing your applications gets hit by a train or hired away, will you be able to bring in someone else?

-What are the costs and time associated with maintaining the database, not just what it takes to slap it up.

As Milliron said, it’s a rare thing to find one person who knows how to gather data, analyze it, clean it, draw meaningful conclusions from it, put it up on a server, build a rocking database application and then fix it when the thing blows up. And if you do run across such an individual, they probably already have a startup or work at Google…

Now if you want to get your hands dirty with code, Milliron suggests checking out your local community college or professional development programs for database classes. That comment struck me because yours truly has been going to a local community college since August to learn some honest-to-goodness programming (in C++ of all things). I can tell you that after banging my head against a PHP/MySQL book for weeks and making little progress, having a pro hold my hand through those concepts helped me build a useful PHP app for our web producers in about four hours.

Creating interesting, functional, user-friendly databases is a big undertaking. Just don’t be afraid to get your hands dirty, and don’t fear failure. You will probably write a bad query, crash a server and have some whiz kid rewrite your 70 lines of code into seven.

But if it means gaining the knowledge to build great apps, it’ll all be well worth it in terms of page views for your site, value for your readers and –most poignantly– your job security.