One of the questions that lands in my inbox every month is, “Who is on your list of ‘Top 5′ industry analyst firms?” I used to treat this question without hesitation, and rattle off the five firms with the highest revenues. Like most people, my logic was straightforward: either you slice and dice by a general coverage/geographical area, or you name the global top 5 based on who’s making the most money.

Money’s not enough of a basis for the “global top 5″ anymore. As an industry, we live in more interesting times and this is changing the ICT research and advisory playing field. And, let’s face it: years of analyst watching has changed this watcher. Today will mark my 10,000th post on Industry Analyst Reporter.

Analyst company revenues and state-of-the-business continue to be important metrics in my eyes. As always, I say the best source for this kind of info is Louise Garnett at Outsell. She obtains information from privately held companies as well as the handful of public ones, and she correlates it against research into Fortune 500 corporate purchasers.

What other criteria do I use for naming a “global top 5″? I like these attributes:

Baseline coverage: does the company have the scope and depth of topical expertise and primary research for addressing all the basics of enterprise IT, networking and telecom?

Advisory/consulting availability: does the client/expert ratio facilitate meaningful discussions on applying research to decisions and practices?

Track record: what indicates proven success in research quality, accuracy, consistency, customer loyalty, and customer satisfaction?

Research expertise: does the company recruit professional market research skills and topical expertise, and invest in ongoing professional development in both research skills and topical expertise?

Research sources: what is the caliber and extent of company’s primary and secondary research sources?

Ethics: has the company made a public commitment to uphold market research industry standards and ethics, as defined by appropriate regional market research or competitive intelligence associations?

How does that line up with your thinking?

This is a great idea, James. Shall I set up a wiki? Seems a good way to go.

Besides enterprise technologists and their providers, this will be useful for analysts too.

I agree with points Alan made in his comment. This business model seems difficult for some people to grasp. Plus, there is some natural uncertainty as to the client experience — whether all research is free, whether there is an expectation for contributing back to the research community as in open source code (as if…).

A guide to the companies comparing their implementations would be a good supplement to the individual explanations in the wild. I still refer people to RedMonk posts like this one for an intro, and lately also to Brenda’s Elemental Links launch, as a good case in point.

Some other people I’d like to tag to contribute to the project would be Dana Gardner, the mwd Neils, and the analyst who sizes the largest analysts and trends buffeting them, Louise Garnett.

There are many other analysts engaged in open source analysis of various sorts in various disciplines — networking, security, etc. If we mandate use of a creative commons license or similar mechanism, then exclusion is simple. Good points for discussion, exploration.