AP investigative reporter offers tips for seeking public records

The Associated Press is committed to fighting for access to information the public has a right to know. AP journalists across the country routinely file Freedom of Information Act requests to uncover critical information that the government would have preferred to keep secret. Washington investigative reporter Jack Gillum recently broke the story that Hillary Clinton used a private email server at her home, and he mined information on Instagram to track Illinois Rep. Aaron Schock’s spending. Gillum frequently draws from records requests to report exclusives. Here, he explains why they should be part of every journalist’s toolkit:

Investigative reporter Jack Gillum (AP photo).

Investigative reporter Jack Gillum (AP photo).

How important to your work are the Freedom of Information Act and open records statutes in the states?
Public records requests have been invaluable in my reporting. FOIA requests to U.S. agencies such as the Federal Aviation Administration recently uncovered that the government knew local police in Ferguson, Missouri, put in place flight restrictions to keep the news media away during demonstrations following the shooting death of 18-year-old Michael Brown.

Other requests can illuminate how government officials conduct their affairs, such as when we found senior U.S. officials using alternative email accounts – raising questions about their obligations to turn over documents to lawmakers and the public.

My request for 911 tapes made during the Newtown, Connecticut, shootings — the subject of a lengthy legal fight — revealed how public safety officials responded to one of the worst school shootings in U.S. history. The records didn’t come easy, with one state prosecutor opposing their release and telling a judge that neither I nor the AP represented the public. The judge ultimately sided with the AP.

How often do you file FOIA requests?
I usually file at least one request a week. That doesn’t include the countless records requests other AP journalists file with governments in the United States and around the world.

What challenges do you encounter in the process?
The federal FOIA is chock full of delays, leaving journalists to wait for information long after the news value of those documents has passed. The U.S. State Department, the defendant in a new FOIA lawsuit by the AP seeking documents about former Secretary of State Hillary Clinton, has an average wait time of nearly a year and a half for certain requests.

Public records laws can vary from state to state. Some laws are antiquated and don’t properly address electronic records, leading to excessive charges (25 cents per page) just to view a public official’s emails or her schedules. Other agencies — state, federal or municipal — simply can’t or won’t perform adequate searches, especially with regard to databases and other forms of digital communications.

What advice do you have for other journalists who are learning to navigate the system?
Even if you’re not a lawyer, become an expert on your state’s freedom-of-information laws. Be prepared to fight any denial; don’t “file and forget” the request. And file requests often — not just when you need information on a big, breaking story. After all, governments produce a lot of material that could be newsworthy and important for the public to see.

Become familiar with electronic records and how they’re stored, especially since documents in manila folders are becoming less commonplace. Ask for database “record layouts” – a virtual map of a database that can reveal what information is being kept – and request other forms of electronic communication besides email (like text messages, chat transcripts or Twitter direct messages).

Where to find AP at SXSW Interactive

The Associated Press is joining thousands of digital and creative professionals from around the world converging at the 2015 SXSW Interactive festival in Austin, Texas, which runs March 13 through 17.  Here’s where you’ll find AP:

Saturday, March 14

Social Media: Breaking News or Fixing News?

AP Social Media Editor Eric Carvin (AP Photo).

AP Social Media Editor Eric Carvin (AP Photo).

  • AP Social Media Editor and Online News Association board member Eric Carvin (@EricCarvin) joins a panel on the impact of social media on the news ecosystem. The hour-long session presented by The Knight Foundation also features Kai Ryssdal, host of “Marketplace”; Michael Roston, social media editor at The New York Times; and Alison Lichter, the Wall Street Journal’s social media editor. The session will take place from 9:30 a.m. – 10:30 a.m. at the Hyatt Regency Austin, Zilker Ballroom. Follow the conversation on Twitter with hashtags: #sxsw #breaking.
  • AP is sponsoring a photo booth at the popular SXSW event, “The Awesomest Journalism Party Ever. V.” The event is from 5 p.m. – 8 p.m. and RSVP is required.

Sunday, March 15

When robots write the news, what will humans do?

Lou Ferrara, vice president and managing editor

Lou Ferrara, vice president and managing editor (AP Photo).

  • When AP announced it would use technology from Automated Insights to automate corporate earnings stories the reaction was dramatic, and many wondered: what will happen to human journalists? AP Vice President and Managing Editor Lou Ferrara (@LouFerrara) will discuss why AP decided to automate some business and sports news content, the impact of automation on the news business and how journalists have adapted to the change. He’ll be joined by Robbie Allen, CEO of Automated Insights. The session will take place from 3:30 p.m. – 4:30 p.m. at the Hyatt Regency Austin, Texas Ballroom 4-7. Follow the conversation on Twitter using hashtags: #sxsw #newsrobots.

AP journalists will also be providing news coverage of SXSW interactive, film and music. Follow business writer Mae Anderson (@maetron) and Music Editor Mesfin Fekadu (@MusicMesfin) for updates.

Dig into data journalism with AP

The Associated Press will be sharing expertise and learning from other data journalists from around the country at a computer-assisted reporting conference put on by NICAR and Investigative Reporters & Editors (IRE). The annual event runs today through March 8 in Atlanta.

AP Editor for Interactive Technology Troy Thibodeaux (AP Photo)

AP Editor for Interactive Technology Troy Thibodeaux (AP Photo)

“Every year we pick up tools and techniques that become essential to our data journalism toolkit. And every year, I personally steal a few teaching ideas that help me bring the material back to the broader AP,” said AP Editor for Interactive Technology Troy Thibodeaux. “All this, and we get to compare notes and ask questions of the people in other news organizations whose work most inspires and challenges us.”

Here’s some of the hands-on training AP is providing:

THURSDAY, MARCH 5

FRIDAY, MARCH 6

SATURDAY, MARCH 7

  • Getting started with SQLite
    This workshop led by Thibodeaux will provide an introduction to the world of SQL (Structured Query Language), “the lingua franca of relational databases.”

Automated earnings stories multiply

The Associated Press, working with Automated Insights and Zacks Investment Research, is now automatically generating more than 3,000 stories about U.S. corporate earnings each quarter, a tenfold increase over what AP reporters and editors created previously. Here, Assistant Business Editor Philana Patterson, who has been overseeing the rollout of this process in the newsroom, gives an update on AP’s automation efforts that began last summer.

Assistant Business Editor Philana Patterson (AP Photo).

Assistant Business Editor Philana Patterson (AP Photo).

What changes has AP made to the automation process?
Since automation began in July, AP has added a number of enhancements to the stories. Descriptions of businesses have been added and the stories now include forward-looking guidance provided by the companies. We are running smoothly, and always looking for opportunities, along with Zacks and AI, to improve what we are producing with automation.

What has the reaction been?
There has been a great deal of interest about how automation works from both members and readers, and overall the reaction has been incredibly positive. AP members are getting more stories about companies in their markets than ever before. We want this process to be as transparent as possible so we have added an explanation of how earnings automation works. It can be found on Automated Insights’ landing page: http://www.automatedinsights.com/ap/.

That link, and one from Zacks, is provided in the tagline of each story. We’ve also encouraged our members and subscribers to make these links available to readers when using the stories, especially online.

Internally, the reaction has been positive from staff, largely because automation has freed up valuable reporting time and reduced the amount of data-processing type work they had been doing.

How does AP ensure quality control?
Quality control was critical from the outset. We worked with Zacks and AI to make sure that every step of the process would produce stories without errors. When we launched last summer, a fair number of errors were discovered in the testing process. We then worked with Zacks and AI on solutions to ensure they wouldn’t happen again. Today, mistakes are rare. Pretty much the only time we will now have an error is if a number is entered incorrectly into the system at the beginning. Once you set up automation, and go through a rigorous testing process, you reduce the prospect of errors. In fact, we have far fewer errors than we did when we were writing earnings reports manually.

Has automation allowed staff to focus more on reporting?
Absolutely. Like all media, we are working with limited resources and it’s critical that we maximize the time reporters have to do journalism and break news. We estimate the automation of earnings reports has freed up about 20 percent of the time that we had spread throughout the staff in producing earnings reports each quarter. It is enabling us to reconfigure our business breaking news operations to be more in sync with social media and user-generated content, and focus more reporters on higher-end enterprise stories that break news that no one else has. Our goals are to break more business news than our competitors, aim higher on investigative and explanatory journalism and focus more of our work on the general consumer. We’ve got some big projects in the works. Automation is helping us free up resources to do all of these things.

What’s next?
This quarter, we are testing the automation of earnings from Canadian and European companies. We expect to add further enhancements and more companies in future quarters. My colleagues in the sports department are also exploring small-audience sports for automation in order to free staff to report news that fans and consumers do not get on the field or a broadcast. We expect to be talking about automation through the year, including at this year’s SXSW Interactive in Austin, Texas.

Doubling down on state government coverage

Building on The Associated Press’ unmatched presence in all 50 U.S. statehouses, we are adding to our competitive advantage by creating a team of state government specialists.

As announced today to the AP staff, the specialists will collaborate with statehouse reporters, as well as on their own projects and stories focused on government accountability and strong explanatory reporting. Their over-arching goal will be “to show how state government is impacting the lives of people across the country,” said Brian Carovillano, managing editor for U.S. news.

The California State Assembly met for an organizational session where lawmakers took the oath of office at the Capitol in Sacramento, Calif., Monday, Dec.  1, 2014.  Both houses of the Legislature will reconvene after the new year. (AP Photo/Rich Pedroncelli)

The California State Assembly met for an organizational session where lawmakers took the oath of office at the Capitol in Sacramento, Calif., Monday, Dec. 1, 2014. Both houses of the Legislature will reconvene after the new year. (AP Photo/Rich Pedroncelli)

Tom Verdin, AP’s administrative correspondent in Sacramento, will assume a new position leading the team of specialists full time. He’s supervised a number of high-impact projects, including AP’s coverage of the rollout of the Affordable Care Act.

Joining Verdin on the team will be National Writer David Crary, reporters David LiebRyan Foley and Christina Almeida Cassidy, as well as Central Enterprise Editor Tom McCarthy.

The New York-based Crary is an expert on many of the social issues state governments are tackling, from gay rights to abortion and adoption, and he’ll continue to focus on many of those issues. Lieb has owned the state government beat in Missouri. From Chicago, McCarthy has been Lieb’s editor and partner on some of his best recent work, and he will serve as editor for many of the stories the State Government Team produces.

Cassidy has been AP’s state government reporter in Georgia. And Foley, based in Iowa, is among AP’s strongest watchdog reporters.

Here, Carovillano answers a few key questions about today’s announcement:

Managing Editor for U.S. News Brian Carovillano (AP Photo).

Managing Editor for U.S. News Brian Carovillano (AP Photo).

How will the state government specialists differ from the AP reporters already assigned to all 50 statehouses and state bureaus?
The team will complement what our excellent state government correspondents do every day across the country and allow us to bring extra reporting firepower in on the most important stories. Let’s say there’s a trend emerging from several statehouses that our folks on the ground identify. The state government team will work with reporters in those states — and with the data team, if necessary — to bring depth and a national perspective to that issue and show how it’s playing out across the country.

They’ll be a resource to our statehouse reporters looking for help broadening the scope of their reporting, and a projects team that will partner with folks in the states to pursue bigger and more ambitious enterprise on the business of state government. And the focus really needs to be on how that impacts peoples’ lives. We don’t cover state government for the state government; we cover it for all the people of the state. The message here is that state government coverage is essential to AP and its members, and we are doubling down on that commitment, which should benefit the entire cooperative.

How else has AP expanded and strengthened state news coverage across the country?
We’ve hired 13 statehouse reporters over the past year. A few of those are new positions; a few filled positions that had been vacant. We are and will remain committed to staffing every statehouse. And we’ll add about 40 additional contract reporters to cover legislative sessions next year, in addition to the permanent staff.

AP President and CEO Gary Pruitt has identified state news coverage as a companywide priority. What other steps are being taken to bolster AP’s state news franchise?
Well, we have made some hires in key locations. I mentioned the 13 statehouse reporters we’ve hired this year. We’ve also made hires on some essential beats, such as politics, immigration, courts/crime and education. Beyond that, we are really pushing our state bureaus to focus their time and effort on content that is exclusive to AP and that our members and subscribers can’t get anywhere else. That needs to be our guiding principle. We do that exactly as we always have: by developing sources and breaking stories, being fastest on big breaking news, and by providing explanation, analysis and depth on the stories that have the biggest impact on peoples’ lives.

To help the bureaus recommit to this kind of high-value content, we’re setting up centralized operations in each region to handle “shared” news from the cooperative. These are the stories and images we pick up from one member and redistribute to the other members in that state. We’re also going to be putting more resources into social media newsgathering, and especially user-generated content, in each of the four U.S. regions. This lets us be in a lot more places than ever before, but it’s critical that we do it without compromising at all on the AP’s reputation for accuracy and fairness.

New AP tool helps journalists manage data

troy_thibodeaux_headshot

Troy Thibodeaux (AP Photo)

The Associated Press’ data team and developers from civic technology company DataMade have created a new tool to make it easier for journalists to add context to a data set. Here, AP Editor for Interactive Technology Troy Thibodeaux, who conceived and launched the tool called Geomancer, explains its potential:

Why did AP create Geomancer?
AP has been a pioneer in data journalism and is committed to helping journalists use data more efficiently to find and tell important stories. We won a grant last year from the Knight Prototype Fund to build an open-source tool to help journalists make sense of data by mashing it up with other data sets about the same geographic location.

For reporters who work with data, it’s a common and laborious task to look up population or demographic data about the counties or ZIP codes represented in a given data set. Geomancer puts this data just a few clicks away. Our goal is to remove the drudgery from data so reporters can focus on finding the story.

How does it work?
Currently, the Geomancer prototype includes two data sources: the U.S. Census Bureau’s 2013 American Community Survey (via CensusReporter.org) and federal contracts from the U.S. Office of Management and Budget’s USASpending.org. It allows users to find data based on 10 geographic levels, including state, county and congressional district. There has already been some community interest in adding new data sets and geographical types.

The Geomancer team has created a working demo (geomancer.io) to show the tool’s potential and simple instructions that any newsroom can follow to install its own Geomancer and build its own warehouse of geography-based data sources. The Geomancer blog at geomancer.ap.org includes links to example data sets for getting started.

What’s next?
We’ll be working to help journalists inside and outside of the AP use Geomancer, and we are excited to see what stories it will help them produce. We’re looking forward to getting feedback from users and will be exploring ways to refine the tool, which is in a beta version.

Uninspected wells: Finding local dangers in a sea of federal data

A team of Associated Press journalists across the states worked together to break an exclusive national story and help member news organizations leverage data to produce unique, local reports tied to AP’s findings. In this memo to staff, AP Vice President and Managing Editor for U.S. News Brian Carovillano explains:

In this June 9, 2014 photo, a petroleum industry worker stands on an oil and gas rig on a well pad, in New Castle, a small farming and ranching settlement on the Western Slope of the Rockies, in Colo. (AP Photo/Brennan Linsley)

In this June 9, 2014 photo, a petroleum industry worker stands on an oil and gas rig on a well pad, in New Castle, a small farming and ranching settlement on the Western Slope of the Rockies, in Colo. (AP Photo/Brennan Linsley)

The report from the Government Accountability Office was intriguing: The government had failed to inspect thousands of oil and gas wells on federal and Indian lands classified as potentially high risk for water contamination and other environmental damage.

But the details were missing. Where were these wells? And did the lack of inspections contribute to any environmental damage?

The Bureau of Land Management was reluctant to provide details, but Washington-based reporter Hope Yen, who broke the story on the GAO report, pressed the agency over the course of several weeks, citing the public’s right to know.

The GAO’s findings came as the use of hydraulic fracturing, or fracking, has been increasing around the country. While fracking has produced major economic benefits, it also has raised fears among environmentalists that chemicals used in the process could spread to water supplies.

When BLM finally released the data to AP, it was in the form of nearly a dozen spreadsheets. Phoenix-based Interactive Editor Dan Kempton, a member of the data journalism team, consolidated them into two master files, allowing calculations to determine which wells on federal and tribal lands were considered higher risk for water contamination and other environmental problems, and whether or not they were inspected by BLM within the given time period, 2009-2012.

Kempton identified, and BLM later confirmed, that its data had duplicate entries and other inconsistencies. Kempton consolidated the duplicates and merged the missing entries to create the most complete and accurate list available of well inspection data. The consolidated spreadsheets were then distributed in advance to AP bureaus and members in states with drilling operations on public and Indian lands, so they could start working on localized stories to accompany Yen’s national overview.

But the data alone was dry. Absent was the human impact. What was the reaction of people living near these uninspected wells?  With Colorado among the top states with uninspected wells, Denver reporter Thomas Peipert and photographer Brennan Linsely literally knocked on door after door to gather reaction and get photos to illustrate the story.

The story was used on the front pages of more than a dozen newspapers from Denver to Akron, Ohio, to Williamsport, Pa., and Tuscaloosa, Ala. It was featured as a Yahoo showcase, and in the 24-hour period following its release, it was tweeted out nearly 600 times. It was also one on the most widely viewed stories on AP Mobile. About a dozen AP bureaus produced state separates, and many members did their own stories using data provided by AP (The Salt Lake Tribune, Times Leader).

It was yet another example of how data journalism offers AP an opportunity to work with its members to provide the tools for local, granular coverage of national issues.

For their enterprising and exclusive journalism, and for furthering AP’s efforts to help members localize our coverage, Yen, Kempton, Peipert and Linsely win this week’s $300 Best of the States prize.

How reporter produced revealing closeup of Gov. Brown’s prison plan

In a memo to Associated Press staffers, Managing Editor for U.S. News Brian Carovillano describes how a story spotted on a locally focused website prompted a high-impact investigation by AP of whether California Gov. Jerry Brown’s prison realignment plan is working as advertised. The story in the Turlock City News reported that Brown had visited officials in rural Stanislaus County. It caught the attention of AP Sacramento Correspondent Tom Verdin. Carovillano continues:

It seems Brown had been quietly dropping in on sheriffs and county officials around the state to gauge the effectiveness of one of the signature achievements of his latest tenure as governor: a law that reduces California’s prison population by sentencing lower-level offenders to county jails.

The governor’s office almost never announced the visits ahead of time, and he rarely spoke publicly afterward so reporters could assess how the visits went. Verdin contacted the governor’s office to find out why they hadn’t been listed on Brown’s official schedule. Brown’s spokesman told him it did not need to be because such visits with public officials were “private,” and that the official photographs distributed by the governor’s staff via Twitter would suffice for public disclosure.

Reporter Don Thompson has been aggressively covering prison realignment as part of his statehouse beat, resulting in a number of other newsbreaks and AP exclusives. In the seemingly innocuous local news item, Verdin and Thompson saw an opportunity for more accountability journalism.

Thompson began by requesting the list of counties Brown had visited. Not wanting the publicity of a formal public records request, the governor’s office complied, and Thompson began making calls to sheriff’s departments and county supervisors on the list. Over several weeks, he contacted half the counties Brown had visited, a representative sample that included urban and rural, coastal and inland.

Across the spectrum, the message was consistent: Local officials said they needed more money and that the governor had not yet followed through on his statements and promises. Two of them said Brown’s office had not gotten back to them on concerns they had raised: “I haven’t heard a thing,” said one local official.

Thompson’s reporting showed that the statements Brown was making in public  _ that “realignment is working”? _ contradicted what he was hearing from county officials.

Additionally, Thompson got an advance look at data showing the jail population for all of California’s 58 counties, before and after realignment, before they were released publicly. That chart moved in advance so members could localize the story if they wished.

Several major California dailies put Thompson’s story on their front pages, including The Fresno Bee and Santa Barbara News-Press. “The advance notice on that story was great,” said Santa Maria Times Editor Marga Cooley, whose newspaper ran it across the top of Sunday’s A1 with a localized sidebar. “The story was timely and of significant interest in our area.”

Photographer Rich Pedroncelli also was able to gain access to a jail in one of the counties Brown had visited, and the package ran with a chart showing the inmate population before and after the realignment law in all 58 counties.

Two days after the story ran, the moderator cited Thompson’s reporting during a public policy forum on prisons. One of the panelists, state Assemblywoman Melissa Melendez, evoked Brown’s “it’s working” quote by saying “It depends on your definition of working, I guess.”

For striking a blow for transparency, holding the governor to account for his misleading statements on an an important accomplishment of his administration, and advancing AP’s efforts to share data with members so they can localize our state- and nation-level reporting, Thompson wins this week’s $300 Best of the States award.

Q&A: How we localized flood insurance investigation for states and small towns

For many years, the federal government offered subsidized flood insurance on homes and businesses constructed before there were many rules about building close to the water. But premiums have been insufficient to cover the payouts, leaving the National Flood Insurance Program billions of dollars in debt. There has been public outcry over some actions taken in Congress to support the program.

The west branch of the Susquehanna River flows past Jersey Shore, Pa. on Sunday March 23, 2014. About a third of the borough (population 4,300) is in a flood hazard zone and nearly 470 homes in town are expected to see flood insurance premium hikes because of changes to the National Flood Insurance Program. (AP Photo/Ralph Wilson)

The west branch of the Susquehanna River flows past Jersey Shore, Pa. on Sunday March 23, 2014. About a third of the borough (population 4,300) is in a flood hazard zone and nearly 470 homes in town are expected to see flood insurance premium hikes because of changes to the National Flood Insurance Program. (AP Photo/Ralph Wilson)

The Associated Press today published an important investigative project by reporter David B. Caruso that details how rising flood insurance premiums across the country will have devastating, long-term impact on many homeowners and communities. Caruso’s report was distributed along with an interactive and AP sidebars from each of the 50 states.

In addition, data shared by AP with member news organizations – such as the New Haven Register and The News Journal (Delaware) – helped them to further localize their coverage.

“This report is a great example of how AP can work with its members and clients to help them produce exclusive, highly local stories that can’t be found elsewhere,” said Brian Carovillano, managing editor for U.S. news. “Going forward, many of our data-driven investigations will include national and state reporting from AP journalists, and our content partners around the country can provide local perspective using data gathered, formatted and distributed by AP.”

Here, National Investigative Editor Rick Pienciak describes how AP tackled the ambitious reporting, which includes data on more than 18,000 communities across the country.

National Investigative Editor Rick Pienciak

National Investigative Editor Rick Pienciak (AP Photo)

What prompted AP to explore this issue?
David Caruso, who has written a great deal about post-Superstorm Sandy insurance issues, proposed looking at the big rate hikes in store for 1.1 million participants in the National Flood Insurance Program, 20 percent of all participants. We figured we could perform a nice public service for a large number of people – in all 50 states.

What were the most striking findings and concerns?
For one, we learned that the public outcry was so strong against the 2012 law to make everyone start paying true-risk premiums (increases as much as 15-fold), that Congress might push back some of those increases, many of which had started to take effect in October. In fact, compared to the usual speed of Congress, a bill was passed by the House and the Senate earlier this month, spreading out the increases. And President Barack Obama signed the bill on Friday. (It is worth noting there are congressional elections in November.)

Instead of paying the full rate immediately, depending on the type of property, those impacted will face annual increases of up to 18 percent or a mandatory 25 percent. In analyzing the data for more than 18,000 communities, we were taken aback by the impact these premium increases will have – even spread out over years – on small, old river towns. The numbers help tell the story that some of these places might very well turn into ghost towns.

What challenges did you face?
One of the biggest challenges was distributing a large amount of data for so many communities. First, we had to decide what to use and what to skip from the Federal Emergency Management Agency data because it wasn’t relevant. And, as anyone who has ever worked with a large, complicated data set might say, “messy” is an understatement. So David, AP’s top data guru Troy Thibodeaux and I spent a great deal of time talking through what we would use, how we’d use it and then those fellows spent a lot more time “cleaning” the data and getting it ready for distribution.

One other challenge was the need to distribute the data ahead of time, via a password-protected FTP site, and then prepare our national piece and our state sidebars – one for every state and the District of Columbia – all without knowing exactly when the president would sign the bill to ease the rates of increase. He did so Friday, so we were able to update each story just before they actually hit the wire. We also fielded countless emails and phone calls from member editors and offered them assistance on using the data. Being able to present this package just a couple of days after the president signed the bill is a big accomplishment.

What did AP offer member newspapers to localize this story?
We offered 28 columns of key data for 18,423 separate towns, cities or unincorporated sections of counties. That is a lot of information.

We provided a unique identifying number for each entity, its name, location by county and state, population, number of policies receiving rate discounts, the number of policyholders facing annual increases of up to 18 percent, the number of policies facing annual increases of 25 percent (that category is generally for vacation homes and businesses).

We also provided the percent of all flood insurance policies in each community facing premium increases. Just from that information, a local reporter can get a really good idea of little towns where large chunks of the citizens are going to face hard times until Congress comes up with a long-term fix.

We provided data state by state, and by type of structure (a business, residential home or 2-to-4-family building). And then there were columns for claim history, numbers of active policies in each community, the total annual premium in a community, total payments since the community joined the federal insurance program and total number and dollar amounts of claims paid out to each respective community. Local reporters could really dig in deep and write their own stories about their town or city, regardless of size.

Q&A: Uncovering the dirty cost of green energy

The Associated Press today published a major investigative report by Washington bureau journalists Dina Cappiello and Matt Apuzzo showing that the ethanol era has proved far more damaging to the environment than the government has acknowledged.

As farmers rushed to find new places to plant corn, they touched off a cascade of unintended consequences, including wiping out millions of acres of conservation land, polluting water and destroying habitat.

Ted Bridis

Ted Bridis

Here, Ted Bridis, the lead editor on the months-long investigation, describes what went into reporting the hidden, dirty cost of this green energy source:

What prompted AP to explore the topic of ethanol?
This year, among other stories, the Washington investigative team has been exploring some of the little-known costs and consequences of green energy. This project developed out of reporting by and conversations with Dina Cappiello, our excellent environmental reporter, who was “loaned” to our investigations team in the newsroom away from her everyday specialty beat responsibilities to work on some longer-term reporting efforts.

What were the most striking findings?
We were able to conservatively quantify how many acres farmers had set aside for conservation purposes but, driven in part by Washington’s biofuels mandate, had been converted to crops. The figure was at least five million acres _ more than Yellowstone, Everglades and Yosemite National Parks combined _ that have vanished on President Barack Obama’s watch.

We also conducted by computer a satellite-imagery analysis to show how many more acres of pristine grassland in the Corn Belt had been plowed into cornfields or soybeans since 2006, the year before the ethanol mandate passed. Insiders revealed to our reporters how the government’s analysis of ethanol’s carbon dioxide footprint was flawed and influenced by input from the industry. But in many ways no one in government is keeping track of its environmental toll.  

I also think one of our findings was how friendly and accommodating we found farmers and others in the Midwest when our journalists visited and explained the story we were investigating; many farmers, especially, explained they were genuinely conflicted by economic forces, chasing corn profits at the expense of continued conservation.

What were the biggest challenges you faced in reporting this project?
The tale of U.S. ethanol policy is complicated for many different audiences: farmers, motorists, policy makers, politicians, environmentalists, farmers, scientists and others. Turning months of reporting about agricultural and energy policies, scores of interviews and hefty data analyses into a narrative that readers will find compelling and informative is always a challenge.

How did you draw on AP’s resources — around the world and in the 50 states — to do it?
This project represents the efforts of dozens of AP journalists, photographers, video producers, data experts, editors and others who helped in its production

See the AP reporting here, which includes companion photos, video and a detailed interactive. Join a conversation about the AP’s reporting on ethanol today at 3 p.m. ET on Reddit.