The AP and Ars Technica and about a hundred other news sites are covering the MPAA’s admission this week that its commissioned study on P2P movie* piracy contained inaccurate data — almost a 30 point error in the calculations. One of many, many things that’s wrong with the entire MPAA study saga — from 2005 through to this minute — is that there has been no shoulder-to-shoulder community response from the leaders of the high tech industry analyst firms.
Background: MPAA had the study conducted in 2005 by LEK Consulting. The MPAA has been using findings from the study for political lobbying, public positioning, industry positioning — basically, every possible use for a set of “proof points”. They never released the methodology or raw data, and so no credible source could refute the findings in a point-counterpoint manner. Many credible sources did express concern, individually, about the accuracy of the data points that were released.
Granted, you can pass this off as another example of what a well financed organization can get away with when they hand off “facts” from a confidential research study to a bevvy of lawyers, lobbyists, and PR specialists. Plus, LEK Consulting is a custom research company, not an industry analyst firm.
But look again. What’s at issue here is a research study into consumer tech behaviors, devices, services, and content. Points of debate include questionable data gathering methods, questionable data analysis and conclusions, and questionable uses of data. This is the industry analysts’ backyard.
Consider the impact a joint statement would have carried, if signed by the companies recognized for their expertise in technology market research: Gartner, Forrester, IDC, Burton Group, Info-Tech Research Group, Informa TM, JupiterResearch, M:Metrics, Parks Associates, RedMonk, Juniper Research, NPD Group, Yankee Group, and tens to hundreds of reputable high tech research firms and indies.
Cooperating on a common position statement would have made a big impact. It could be a single statement presenting divergent viewpoints. However, speaking one at a time, as though no other analyst exists on the planet, had no visible impact whatsoever.
I’m not calling for pandemic love among analyst firms worldwide. I am saying that cooperating for the good of the public is good for business.
* Original post corrected from “music” to “movie” - BF
Tags:
January 23rd, 2008 at 3:16 pm
Were the analyst firms to come together as suggested, such a step could set a precedent for future gang-ups. Who can predict if one of these signatories might be on the receiving end of such treatment in the future, should it become fashionable? It doesn’t have to stop with hinky sponsored studies.
Commissioned research already carries a high degree of controversy in the media, even where the methodologies and the raw data are also made public. The MPAA’s troubles will only increase the reflexive skepticism among journalists.
This problem will self-correct.
January 24th, 2008 at 11:40 am
Rob, You’re right that fear of analyst gang-ups is real. It does exist. But, two questions back to you — and everybody:
> Does fear of analyst gang-ups exist outside the tech companies?
> Should the possibility of competitive censure stop analysts from cooperating on issues of public importance?
My answers to these questions are No, and No. I have never seen anything but normal competitive behavior among analyst firms. They seem well equipped to devour their own, on their own. No assistance needed there.
We know that the combined influence of a group of analysts is greater than the influence of an individual analyst or firm. We see this at work every day. I’m suggesting that the analysts channel this secondary effect for the public good.
What do you think?
January 28th, 2008 at 3:13 pm
I’m paraphrasing this, but there’s an old saying that when someone invokes the public good as a justification for a course of action, we should double-check our wallets to make sure they are still where they should be.
It’s a fair question about whether the IT analyst industry should be more inclined to self-regulation. That said, self-regulation is usually an industry’s reaction to the threat of legislative regulation that’s come about from prior instances of severe market distortion, scandal or fraud. The CFA Institute is an example.
I don’t think that addressing controversial practices in the IT analyst industry will require anything more formal than the old-fashioned market force of reputation. If a client comes to believe that the integrity of an IT analyst’s data or recommendations can’t be trusted, that client will cease doing business with them. And word invariably gets around, accelerating the downward spiral.
Bloggers and peer-to-peer networking venues will only make analyst firms more keen to proactively scrutinize their own business practices to lower the risk of reputation degradation.