What would you do, if you were an enterprise IT leader faced with perfectly opposing analyst views on the question of SOA and ROI? As of this morning, this is the dilemma facing IT leaders evaluating SOA, courtesy of Macehiter Ward-Dutton and Nucleus Research.

One the one hand, a Nucleus Research/KnowledgeStorm benchmark survey on SOA deployments and ROI finds low overall adoption of SOA and not much evidence of ROI where it has been deployed.

On the other hand, Macehiter Ward-Dutton takes a dim view of ROI-of-SOA exercises to begin with, in their blog post, The pointless search for SOA ROI.

These analysts are not simply disagreeing on shades of gray. Their arguments nullify each other.

This situation is not all that common. When it does happen, the resulting press coverage tends to focus on point-counterpoint between the analysts. That makes for nice vanity press, perhaps — if you like a good cat fight. However, I’m curious about how this type of situation affects thinking among IT management and IT management consultants.

Maybe enterprise thought leaders like James McGovern and James Taylor (when he is back from holiday) will weigh in on this — not only with their views on the particular question of whether SOA ROI is or is not a meaningful pursuit, but also on the impact of being faced with diametrically opposed analyst opinions on a topic as complex as SOA ROI.

The communal project, “analyzing the analysts guide - open source analysis edition”, starts today!

This is an informal, community effort to develop a guide to working with industry analyst firms using open source licensing / analysis as part of their business model. The project was proposed last month by James McGovern at Enterprise Architecture: Thought Leadership.

This is a project of peers; no one is in charge. I simply volunteered to set up a wiki and tap some analysts on the shoulder. Subject matter experts are welcome to participate. This includes industry analysts, professionals who are experienced clients of industry analysts, and related subject matter experts.

I don’t have email addresses for everyone James mentioned in his original post. So if you did not get a late-night email inviting you to join, please contact anyone already involved to join. (You can put a fork in my head next time you see me.) Check your spam filter, too — I think the email was sent from the wiki, not from me.

It’s at wikispaces and it’s called the “osaa” wikispace.

This is a great idea, James. Shall I set up a wiki? Seems a good way to go.

Besides enterprise technologists and their providers, this will be useful for analysts too.

I agree with points Alan made in his comment. This business model seems difficult for some people to grasp. Plus, there is some natural uncertainty as to the client experience — whether all research is free, whether there is an expectation for contributing back to the research community as in open source code (as if…).

A guide to the companies comparing their implementations would be a good supplement to the individual explanations in the wild. I still refer people to RedMonk posts like this one for an intro, and lately also to Brenda’s Elemental Links launch, as a good case in point.

Some other people I’d like to tag to contribute to the project would be Dana Gardner, the mwd Neils, and the analyst who sizes the largest analysts and trends buffeting them, Louise Garnett.

There are many other analysts engaged in open source analysis of various sorts in various disciplines — networking, security, etc. If we mandate use of a creative commons license or similar mechanism, then exclusion is simple. Good points for discussion, exploration.

James McGovern, an outspoken enterprise IT blogger, recently outlined his latest frustration with software vendor sales tactics. His candid post offers an example of how good industry research can go bad in the hands of some vendors. Let’s take his beef with security software vendors.

Mr. McGovern is frustrated with security software vendors pitching SOX compliance as a reason to buy. To him, it’s cliche. Fair enough. Yet, I’ve seen several research studies linking enterprise security with compliance, and IM security with compliance, and so on…

Read the rest of this entry »