Investigative journalism is underprovided in the market, but new combinations of data and algorithms may make it easier for journalists to discover and tell the stories that hold institutions accountable.


DEMOCRACY'S DETECTIVES: The Economics of Investigative Journalism
By James T. Hamilton

Investigative journalism involves original work, about substantive issues, that someone wants to keep secret. This means it is costly, underprovided in the marketplace, and often opposed. It gets done when a media outlet has the resources to cover the costs, has an incentive to tell a new story, cares about impacts, and overcomes obstacles. Changes in media markets have put local investigative reporting particularly at risk. But new combinations of data and algorithms may make it easier for journalists to discover and tell the stories that hold institutions accountable.

CJ Lab co-founder Jay Hamilton writes all about this in his new book, published Fall 2016.

Available from Harvard University Press

         

Winner of 2017 Goldsmith Book Prize for the best academic book on media, politics and public policy.

     

"The Goldsmith Book Prize is awarded to the academic and trade books that best fulfill the objective of improving democratic governance through an examination of the intersection between the media, politics and public policy. The Goldsmith Book Prize for best academic book will be awarded to James T. Hamilton for Democracy’s Detectives: The Economics of Investigative Journalism (Harvard University Press)."


 

Winner of Frank Luther Mott - Kappa Tau Alpha Journalism & Mass Communication Research Award for best research-based book about journalism or mass communication published in 2016.

     

"Hamilton provides a fresh and compelling look at the value of investigative journalism in our democracy," said Jeff Fruit, president of Kappa Tau Alpha and a contest judge. "He not only develops and analyzes a unique data set of investigations, but also clearly demonstrates the impact that investigative journalism can have through a case study of one Pulitzer Prize-winning reporter."


 

Winner of the 2017 Tankard Book Award.

     


"The Tankard Book Award was established to honor James W. Tankard, Jr. of Texas at Austin. A former editor of Journalism Monographs, the award recognizes his many contributions to the field of journalism and mass communication education."

 
       

What is computational journalism?


Computational journalism is an evolving field.

Definitions include:

“What is computational journalism? Ultimately, interactions among journalists, software developers, computer scientists and other scholars over the next few years will have to answer that question. For now though, we define computational journalism as the combination of algorithms, data, and knowledge from the social sciences to supplement the accountability function of journalism.” James T. Hamilton and Fred Turner, “Accountability through Algorithm: Developing the Field of Computational Journalism” (a report from Developing the Field of Computational Journalism, a Center for Advanced Study in the Behavioral Sciences Summer Workshop, July 27–31, 2009), 2. 514-

Computational journalism, “Broadly defined ... can involve changing how stories are discovered, presented, aggregated, monetized, and archived.” Sarah Cohen, James T. Hamilton, and Fred Turner, “Computational Journalism,” Communications of the ACM 54 (2011): 66

In analyzing the dimensions of computation that could advance sensemaking in journalism, Terry Flew and coauthors note:

“Automation alleviates activities such as data gathering and interpretation, number crunching, network analysis, sorting, and processing that would other wise need to be done manually; algorithms allow operators to follow predefined steps needed to accomplish certain goals, identify problems, find suitable solutions in a large set of alternatives, and verify information in a reliable, consistent and efficient manner; and abstraction enables the qualification of different levels or perspectives from which an idea may be presented or new directions that may be explored.” Terry Flew, Christina Spurgeon, Anna Daniel, and Adam Swift , “The Promise of Computational Journalism,” Journalism Practice 6 (2012): 159

Though computational journalism builds on and incorporates elements of computer-assisted reporting and data journalism, this new approach often involves larger data sets and more sophisticated algorithms. Recent advances in computational journalism center on reporting by algorithms, about algorithms, and through algorithms. Stories generated by computer algorithms include those by Narrative Science and Automated Insights, companies seen as leaders in “Automation in the Newsroom.” The Wall Street Journal and ProPublica have each done reporting that looks at the disparate impacts of private and government algorithms on different groups in society. Reporters are also using algorithms to mine for stories, as the Atlanta Journal Constitution did in using web scraping and machine learning to identify potential cases across the country of doctors involved in sexual misconduct.

The papers and video (to come) from the 2016 Computation + Journalism conference at Stanford show recent computational advances in story discovery, telling, and distribution.


The Stanford Computational Journalism lab is currently focused on two research questions:

How do you lower the costs of discovering stories through better use of data and algorithms?

How do you tell stories in more personalized and engaging ways?


                                   

Projects involving story discovery include:

Projects involving telling stories in more personalized and engaging ways include:

For more on computational journalism’s history and potential, see Chapter 8, “Accountability and Algorithms,” in "Democracy’s Detectives."

         

By the numbers:


 
   
   
   
   
   
 

What can be done?


Three changes in government policy would readily reduce the hassle costs involved with investigative work:

  • (1) real reform of the implementation of FOIA laws;
  • (2) adding journalism as a field to support in federal R & D competitions focused on algorithms, data, and tech and in IRS rules that would make it easier for online public affairs sites to gain nonprofit status; and
  • (3) truer implementation of open government and transparency policies.

Advances in computational journalism can improve the economics of accountability journalism in two ways:

  • On the supply side, new ways of combining data and algorithms could lower the costs of discovering stories.
  • On the demand side, telling stories in more personalized and engaging ways for readers and viewers can increase the likelihood of revenues through subscription or advertising.

The set of people whose decisions could significantly expand the field of computational journalism include computer scientists, philanthropists, entrepreneurs, journalism educators, and reporters.

         

In the press:


         

Book Reviews:


“In riveting detail, Hamilton meticulously examines the storied history of investigative journalism in America, chronicles its current malaise, and makes a convincing case that pouring resources into gumshoe reporting makes economic sense for sclerotic news organizations. Why? Because readers hunger for more of it and are willing to pay to read it.” —Walter V. Robinson, Pulitzer Prize–winning investigative journalist and Editor-at-Large at the Boston Globe

“This is an outstanding book, the product of careful thinking, of remarkable and painstaking gathering of data on investigative reporting—past and present—that no one in academia or in journalism has ever undertaken before. It is a moving, evidence-based affirmation of the value of journalism to democracy.” —Michael Schudson, Columbia University

“Democracy’s Detectives is essential reading for anyone interested in the economics of news, and it is a master class in methodological creativity and ingenuity in conducting social science research.” —Philip M. Napoli, Duke University

“Relatively little serious academic scholarship has focused on investigative journalism ... . Fortunately, Stanford University professor James T. Hamilton helps rectify this imbalance with an important, deeply researched, and original analysis of this vital subject.” —Mark Feldstein, University of Maryland