Can we trust any of these claims about piracy?
The US doesn't think so. In a new report out yesterday, the government's own internal watchdog took a close look at "efforts to quantify the economic effects of counterfeit and pirated goods." After examining all the data and consulting with numerous experts inside and outside of government, the Government Accountability Office
concluded (PDF) that it is "difficult, if not impossible, to quantify the economy-wide impacts."
More specific studies that focus only on single industries don't fare much better because "the illicit nature of counterfeiting and piracy makes estimating the economic impact of IP infringements extremely difficult." And when it comes time to choose a substitution rate (how much of the infringing activity should be counted as a lost sale), we're left only with "assumptions... which can have enormous impacts on the resulting estimates."
The GAO then went on to slam three particular reports often linked to the government. They're all commonly cited, they're all bogus, and at least one is still being used officially.
Three commonly cited estimates of U.S. industry losses due to counterfeiting have been sourced to U.S. agencies, but cannot be substantiated or traced back to an underlying data source or methodology.
First, a number of industry, media, and government publications have cited an FBI estimate that U.S. businesses lose $200-$250 billion to counterfeiting on an annual basis. This estimate was contained in a 2002 FBI press release, but FBI officials told us that it has no record of source data or methodology for generating the estimate and that it cannot be corroborated.
Second, a 2002 CBP press release contained an estimate that U.S. businesses and industries lose $200 billion a year in revenue and 750,000 jobs due to counterfeits of merchandise. However, a CBP official stated that these figures are of uncertain origin, have been discredited, and are no longer used by CBP. A March 2009 CBP internal memo was circulated to inform staff not to use the figures. However, another entity within DHS continues to use them.
Third, the Motor and Equipment Manufacturers Association reported an estimate that the U.S. automotive parts industry has lost $3 billion in sales due to counterfeit goods and attributed the figure to the Federal Trade Commission (FTC). The OECD has also referenced this estimate in its report on counterfeiting and piracy, citing the association report that is sourced to the FTC. However, when we contacted FTC officials to substantiate the estimate, they were unable to locate any record or source of this estimate within its reports or archives, and officials could not recall the agency ever developing or using this estimate. These estimates attributed to FBI, CBP, and FTC continue to be referenced by various industry and government sources as evidence of the significance of the counterfeiting and piracy problem to the U.S. economy.
The GAO then sets its sights on several private industry reports. The Business Software Alliance claimed a loss of $9 billion to piracy in 2008, but its study "uses assumptions that have raised concerns among experts we interviewed, including the assumption of a one-to-one rate of substitution and questions on how the results from the surveyed countries are extrapolated to non-surveyed countries."
Next up was the MPAA, which has already publicly taken its lumps for that flawed 2005 survey we mentioned above. But even when you set aside the mistaken initial conclusion about collegiate downloading, the study still shouldn't be used by lawmakers; it's a black box.
"It is difficult, based on the information provided in the study, to determine how the authors handled key assumptions such as substitution rates and extrapolation from the survey sample to the broader population," says the GAO.
More than they bargained for
Why is the government even looking into this issue? It's all due to the
PRO-IP Act, which passed under President Bush and has led President Obama to appoint an Intellectual Property Enforcement Coordinator within the White House. Part of the IPEC's duties include gathering data on piracy and counterfeiting, and current IPEC Victoria Espinel is
now rounding up that data. The GAO report is part of this process, and it certainly doesn't make industry estimates look compelling.
This is ironic for a bill that was backed by the big rightsholders; even its acronym, the PRO-IP Act, shows what it was supposed to do. But, by hauling the black art of "piracy surveys" into the light, the PRO-IP Act is forcing rightsholders to tone down some of their more specific and alarmist rhetoric.
What about all that data Espinel asked for, including detailed methodologies? The content industries basically punted, pointing to three surveys done by a single guy, Stephen Siwek of the Institute for Policy Innovation. GAO looked specifically at
Siwek's work, all of which seeks to model effects of piracy on the entire US economy.
The government concluded that "most of the experts we interviewed" were reluctant to embrace Siwek's methodology; his approach comes from the Commerce Department, but it simply wasn't designed to measure what's being measured here. For instance, these studies ignore the obvious points that pirating goods leaves consumers with more disposable income, which is likely spent elsewhere in the economy. Effects on the economy as a whole, then, are terribly speculative and seem more likely to be simply redistributive.
None of this is to say that piracy and counterfeiting aren't real problems. The GAO accepts that the problem is "sizeable," but it also points out just how much bad data is used to produce these studies. Actual dollar figures and job loss numbers should be handled with extreme care and a good bit of skepticism; the GAO also noted that numerous experts told it that "there were positive effects [from piracy on the economy] and they should be assessed as well."
This is a helpful, level-headed review from the GAO, one that (hopefully) brings some of the debates over digital infringement into saner territory.
