KEY FINDINGS

• Overall, 32 percent of the questions asked in debates were non-substantive, i.e. they concerned campaign gaffes, political tactics, or issues of personality.
• Only 9 percent of the questions concerned the economy. There were only six questions about the mortgage meltdown that touched off the current economic crisis.
• As the primary campaign progressed, the debates became less and less substantive. By the primary campaign's final period, non-substantive questions outnumbered substantive questions.
• The debates hosted by PBS and Univision were the most substantive, with 100 percent and 82 percent substantive questions respectively. Fox and ABC hosted the least substantive debates; fewer than half the questions in each of those networks' debates were substantive.
• The front-running candidates in both parties were more likely to be asked questions focused on trivia, while the candidates trailing badly in the polls were more likely to be asked substantive questions. For example, 73 percent of the questions posed to Dennis Kucinich were substantive, while only 51 percent of the questions given to Barack Obama were substantive, and only 59 percent of the questions for John McCain were substantive.
• Of those who moderated more than one debate, Maria Elena Salinas and Jorge Ramos of Univision asked the most substantive questions (88 percent and 73 percent, respectively). George Stephanopoulos of ABC (39 percent) and Chris Wallace of Fox (33 percent) asked the fewest substantive questions.
• Twenty-one percent of the questions did not even touch on a policy issue.

INTRODUCTION

After Sens. Hillary Rodham Clinton (D-NY) and Barack Obama (D-IL) participated in their final primary debate on April 16, the reviews were scathing. But it was not the candidates who were the target of these critiques. Instead, it was the moderators, Charles Gibson and George Stephanopoulos of ABC News, who were criticized. Tom Shales of the Washington Post wrote that the two “turned in shoddy, despicable performances.” Greg Mitchell of Editor & Publisher magazine called it “perhaps the most embarrassing performance by the media in a major presidential debate in years.” Many critics focused on the fact that the debate was nearly half over before a single question on a policy issue was asked; instead, the moderators focused on rehashing old gaffes and absurd questions about patriotism.

Though the ABC News debate was in many ways the worst in memory, it was actually the last step in a progression throughout the primary season. While the initial round of debates was reasonably substantive, as time went on and more debates took place, the moderators became more and more focused on political maneuvering and outright trivia, and less concerned with actual policy issues.

Media Matters Action Network analyzed all the debates broadcast on national television to chart this progression. We categorized the questions as either substantive or non-substantive, with further subcategories for the non-substantive questions. A third category -- substantive but flawed -- was used for questions that had a substantive basis but that forced candidates to respond to inflammatory hypothetical situations or to attack each other.

A variety of types of questions fell into the non-substantive category. These included “gotcha” questions focusing on prior controversial statements; questions in which a candidate was forced to defend someone else, such as a supporter; questions about a candidate's political motives for some statement or position; questions revolving around poll results, electability, and so on; and questions about personal matters not directly related to policy or the duties of the president.

At one time or another, candidates on both the Republican and Democratic sides took issue with the questions that were being asked. Overall, there was not much difference in the way Republicans and Democrats were treated: 57 percent of the questions asked of Democrats were substantive, while 60 percent of the questions asked of Republicans were substantive.

The overall picture masks the fact that the questions asked became less substantive over time. We divided the debate season into four periods: the first half of 2007, which featured six debates; the second half of 2007, which featured 13 debates; the month of January 2008, which included the beginning of actual voting and featured nine debates; and finally, the period from February 2008 on, which featured three debates between Obama and Clinton (Sen. John McCain (AZ) had secured the Republican nomination by that point). During the first period, 66 percent of the questions were substantive, while only 23 percent were non-substantive, meaning that there were three substantive questions for every non-substantive question. But as time went on, the debates moved steadily away from policy and toward discussions of campaign maneuvering, remarks made by the candidates' supporters, personality issues, and other types of trivia. By the last debate period, the proportion of non-substantive questions (52 percent) had exceeded the proportion of substantive questions (39 percent).

ISSUES

Given the large proportion of the questions that were not substantive, it should be little surprise that 21 percent of the questions touched on no policy issue at all (Under our coding system, a question could be non-substantive but still relate to some policy issue. For instance, Republicans were asked about the political consequences of having a pro-choice nominee (Rudy Giuliani), a non-substantive question (as it focused only on the horse race) that nonetheless touched on a policy issue (abortion)). The most common topic of discussion by far was foreign policy and national security, which included the Iraq war. No other issue area was the subject of more than 10 percent of the questions.

This is particularly notable with regard to the economy, which has become the most important issue in the fall campaign. In the primary debates, it was almost an afterthought, accounting for a mere 9 percent of questions. Moderators spent much of the time they did devote to the economy attempting to pin down candidates on whether they would use the word “recession” to describe the country's situation, or asking about the large incomes of hedge fund managers. Brit Hume of Fox asked how candidates would keep the economy going if terrorists attacked our shopping malls. But only six questions out of the total of 2,304 (less than three-tenths of one percent) touched on the growing crisis in the mortgage industry, which was already making headlines in 2007. Only three questions mentioned the minimum wage Ð one of which was when Chris Dodd was asked if he would work for minimum wage as president. Only two questions touched on the issue of declining wages. In other words, when they weren't ignoring the economy completely, the moderators missed the aspects of the issue that are of greatest concern to many Americans.

Other critical issues were likewise given short shrift. There were 61 questions about abortion - half as many as all other health care questions combined. There were dozens of questions about oil prices, but only three questions about conservation and renewable energy (one of which was when the candidates were asked if they themselves were using compact fluorescent bulbs). Although candidates were asked whether Dick Cheney had too much authority in the Bush administration, there was not a single question about the administration's unprecedented use of signing statements, its dramatic claims of executive privilege, or its extraordinary secrecy. These 31 debates featured only one question about wiretapping and only two questions about the prison at Guantánamo.

How did the priorities of the news organizations conducting the debates compare with the priorities of the American public? To find out, we can compare the attention given various issues in the debates to the priorities expressed by the public over the course of this period. The debates took place between April 2007 and April 2008. In its polls, Gallup asks Americans what they consider “the most important problem facing this country today.” We took the average of three of these Gallup polls, from April 2007, October 2007, and April 2008 as a point of comparison.

While one could reasonably argue that debates shouldn't necessarily be a perfect reflection of the public agenda, we can say nonetheless that the public had somewhat different priorities than the debate moderators (Because the Gallup questions were open-ended, they did not correspond precisely to our categories. For means of comparison, we combined the Gallup results on Iraq, terrorism, foreign aid, international issues, and conflict in the Middle East to compare to our Foreign Policy & National Security category, which includes questions about Iraq. For our Economy category, we combined responses under economy in general, high cost of living, unemployment, lack of money, and wage issues. The one change across the three polls was that the economy rose in importance between the second and third polls, while foreign policy declined between the second and third polls). This was particularly true of the economy, which was the basis for only 9 percent of debate questions, but was cited as the most important issue by substantial proportions of the public, particularly in 2008. In Gallup's April 2008 poll, 56 percent of respondents cited an economic issue as the country's most important problem.

In short, when the debate moderators weren't focusing on personal and trivial questions, their priorities were far out of line with what the public found important, and those issues that have come to dominate the general election.

UNEQUAL TREATMENT

Though there was not much difference between the treatment of Democrats and Republicans overall, all candidates were not treated equally. To a great degree, it appears that the better a candidate was faring in the polls and the greater the chance that that candidate might actually win the nomination, the less likely he or she was to be asked substantive questions. It is particularly troubling that the candidates with less of a perceived chance to win could expect to be asked substantive policy questions, while the main contenders were more likely to be asked to respond to trivia.

The candidates who were asked the most substantive questions -- Chris Dodd, Fred Thompson, Dennis Kucinich, and others -- were not front-runners. If we look only at each party's strongest candidates (McCain, Mitt Romney, Mike Huckabee and Rudy Giuliani for the Republicans; Obama, Clinton, and John Edwards for the Democrats), we see that as a group, they ranged from a low of Romney's 51 percent and Clinton's 50 percent substantive to a high of Giuliani's 60 percent substantive, with an average of 54 percent. Among the weaker candidates, almost all did better than the leading candidates, and as a group, they averaged 65 percent substantive questions.

Moreover, while McCain was asked a higher proportion of substantive questions overall than Obama or Clinton, the questions the two Democrats were asked became steadily less substantive over time, while McCain was actually asked more substantive questions during the second half of 2007 than he had been during the first half of that year. This may or may not have something to do with the fact that during the second half of 2007, he was essentially written off as having little chance to win; once he climbed in the polls, the questions he was asked became less substantive.

THE NETWORKS: THE GOOD, THE BAD, AND THE UGLY

Though there is plenty of legitimate criticism to be made of this election's primary debates, it would be a mistake to assume that all the networks were equally culpable. In fact, there were dramatic differences between the networks that hosted these debates. PBS hosted one nationally televised debate that we analyzed (PBS also hosted a Republican debate on September 27, 2007. Because John McCain, Rudy Giuliani, Mitt Romney, and Fred Thompson did not attend, we chose not to include this debate in our data. But with the exception of an introductory question asking the attending candidates why you chose to be here tonight and what you say to those who chose not to be here to-night, all the questions in that debate were also policy-oriented), and every single question asked of the Democratic candidates in that debate was substantive.

The second-most substantive network was Univision, which was the sole host of two debates and the co-host, with CNN, of one debate (For that debate, we assigned the questions asked by Jorge Ramos of Univision to the Univision figures, and the questions asked by Campbell Brown and John King of CNN to the CNN figures). At 82 percent, Univision's proportion of substantive questions was far above that of the English-language commercial networks.

If PBS and Univision receive good grades for their debate performances, then poor grades go to CNN and MSNBC/NBC, each of which hosted nine debates. Fewer than two-thirds of the questions on each of those networks' debates were substantive. Finally, Fox and ABC receive failing grades; fewer than half of the questions on each of those networks' debates were substantive.

Even within networks, some moderators' questions were more substantive than others'. Though MSNBC's numbers overall were poor, Chris Matthews scored better than all but two moderators; almost three of every four questions he asked were substantive. At the other end of the scale were Chris Wallace of Fox and George Stephanopoulos of ABC, the hosts of those two networks' prestigious Sunday-morning talk shows. Thirty-nine percent of the questions Stephanopoulos asked were substantive, while only 33 percent of Wallace's questions were substantive (Since a number of journalists moderated one debate or asked a few questions in more than one debate, this chart includes only those who asked a total of 50 questions or more. All the journalists included in this chart moderated at least two debates).

The two moderators who asked the most substantive questions were both from Univision, the Spanish-language network. Univision sponsored one Republican and one Democratic debate, both of which were moderated by Maria Elena Salinas and Jorge Ramos; Ramos also moderated a joint Univision/CNN debate. Eighty-eight percent of the questions Salinas asked were substantive, as were 73 percent of Ramos' questions.

CONCLUSION

Debates can be extraordinarily valuable for citizens looking to assess the men and women who would be president. Unlike reports about the campaign on the evening news, where everything a candidate says in a day is reduced to a six-second sound bite, in a debate, the public has the opportunity to hear candidates speak at length. We can learn about their ideas, their proposals, and their interpretation of current events. That is, we can do so if those responsible for bringing the debates to us -- the television networks -- moderate the debates in such a manner as to make substantive discussion possible.

But when they focus on trivia, attempt to get candidates to attack one another, and find ways to avoid discussion of the policy issues the next president will face, the networks do the public a profound disservice. There is little doubt that while some networks and moderators did better than others in the 2008 primary season, on the whole, they failed.

METHODOLOGY

We coded all nationally broadcast primary debates in which a majority of the active candidates in a party participated, a total of 31 debates:

We divided the questions into the following categories:

Substantive questions: These were legitimate inquiries about the candidates' records, opinions, and positions on policy issues and the next presidency.

Non-substantive questions: These fell into a number of categories, as follows:

Candidate “gotchas”

Rehashing gaffes -- questions that forced the candidate to explain a previous controversial statement

Defend another -- questions that forced the candidate to defend the actions or words of someone else

Playing politics -- questions that accused the candidate of engaging in evasiveness, flip-flopping, harsh campaign tactics, etc.

Frivolous/irrelevant questions

Campaign politics -- questions about polling, voter appeal, electability, etc.

Personal/character -- questions involving character traits, lifestyle choices, etc.

Triviality -- questions utterly unrelated to any policy or personal issue

Other kinds of problematic questions:

Hypothetical questions -- questions that offered extreme or absurd hypothetical situations and asked the candidate to respond

Quarrel -- questions that sought to instigate conflict and argument between the candidates

A substantive question could also be coded for Hypothetical or Quarrel, in which case it was categorized as “substantive but flawed.”