In this blog I look beyond the obvious and try to find out why people and companies do what they do - and what it means for the rest of us. 
An e-mail archiving company, ZL Technologies, Inc., has sued, been dismissed, and re-sued Gartner – basically claiming that ZL’s placement in Gartner’s “Magic Quadrant” has caused the company damage – namely, that since Gartner places ZL in the “niche” spot, large customers don’t consider them, although the company contends their offerings are superior to those listed in a more prominent spot. Gartner counter claims that the suit is without merit because the MQ represents opinion, and therefore there is no legal leg for ZL to stand on.
There are interesting things at play here.
First, the disclosures:
- ESG, the company I founded, is a Research, Analysis, and Strategy firm. As such, in some ways we theoretically compete with Gartner. In practicality, we don’t.
- A more honest disclosure of the above is that ESG doesn’t compete with Gartner, as we serve very different purposes. A truly honest disclosure is that I am wildly impressed and completely jealous of Gartner’s ability to drive revenue the same way the mafia does – by veiled threat. The difference is that the threat of the mafia (in a protection scheme) is physical while the threat of Gartner is market muscle/economic. I would love to be that powerful – although if I were I’d like to believe I would not abuse the power – which in my opinion is the true crux of this issue.
- I have never met, nor heard of ZL Technologies until this lawsuit – although our analysts have. I am told, that the company has some cool stuff, but is run by a lunatic. I do not know if CEO Kon Leong is actually insane, or brilliant, or acting, or something else. But I love the fact that he’s spending a lot of money on this issue keeping me entertained.
Here are the realities of the situation – IN MY OPINION (for those of you in the legal profession):
The laws of the land protect the opinion of people. You can’t legislate what people think. This, in essence, is Gartner’s defense, and ultimately why it will most likely prevail in this matter.
What is more interesting is that in its defense, Gartner exposes what everyone in the industry already knows, but are incapable of doing anything about – the fact that Gartner’s opinions are largely unqualified. They market expertise and data – i.e., fact, but defend with opinion.
I can only speak about the Gartner I know – the IT Gartner that plays in the industry where I play. For all I know the Gartner analysts, research, etc. in their other markets might be absolutely brilliant, and the value they bring to their customers and society in general unparalleled. Judging by what I see in their IT practice, I find it doubtful, but one never knows.
What Gartner is, but can never say, is a market “influencer,” to an envious degree. What the Magic Quadrant is is an absolutely unscientific – but awesomely powerful – “tool” with no rhyme or reason, no scientific merit, and no disclosure. It’s a self-interpretive slide that leaves the reader to judge the meaning of seemingly random dot placement. Gartner does not defend (or contends it need not defend) the placement of those dots, for that placement is the opinion of the placee – namely the analyst responsible for said market analysis.
The problem, as I see it, is one for a much different court than a civil court. It’s really about the FTC – because opinion isn’t the issue here. I’m all for being able to toss your opinion around as one wishes. The problem is one of deceptive business practices.
If you net out the whole issue it comes down to this: Gartner has created a massively powerful perception of what I’ll term “qualified expertise” in the eyes of the mass market IT buyer (the people who write checks to vendors for billions of dollars). Note: They are absolutely brilliant at being able to accomplish this – and my guess is that at one time, that perception was merited. As such, they wield incredible influence on companies that do business (or attempt to do so) within those market segments. They have the power of business life and death sometimes. They are the emperor – they can actually affect who wins, who loses, who lives, and who dies.
No rational person would look at the effective equivalent of a rorshach test (the MQ) and make any substantive decision based upon it, because there is no valid meaning in it. It is subjective. It, for all intents and purposes, may be thought provoking but it is by itself meaningless.
Further, if you were to look at an industry segmented Magic Quadrant that showed 10 companies or technologies with some clear winners and clear losers in an area that you were responsible for, you would put SOME level of merit into what the results/placement of the dots held – ranging from none to total belief. The level of merit you place on that would depend on your TRUST of the expertise of those who present it. If you have a trust relationship with a brand or a person then by default, the (perceived or real) credibility of that “data” or “input” is higher than those who have no such relationship.
For example, if you were investigating a major server purchase, you might very well look to see what Gartner says about the players in that market. You would probably take a look at their MQ. If that MQ showed vendor A up and to the right and vendor B down and to the left, you might reasonably assume that the brilliant experts at Gartner feel vendor A is superior to vendor B. You might place a significant amount of credence on that assumption and it might truly influence your decision making. If, however, you were to KNOW for a fact that the dots were placed by chimpanzees as part of a scientific experiment, you might not hold that “data” in such high esteem. Since you don’t know that, you might be making a decision based solely on the perceptive expertise of the provider, with no actual insight into whether that perception is merited. You simply ASSUME that the expertise behind the decision is valid and merits the result.
This, in my opinion (saying that a lot, aren’t I?) is the whole enchilada. Is perpetuating and even actively attempting to foster the perception of expertise where little or none actually exists tantamount to fraud – or at least deceptive business practices? I don’t know the law, but I know what a skunk smells like.
Allow me to say what most cannot or won’t – there is virtually no one within the IT industry itself that believes Gartner has any value outside of its market influence. No one that I know pays Gartner for their expertise in any aspect of business. There are no CEOs who call on Gartner for advice as to how to run their business. There are no people in R&D calling on Gartner to determine what they should build into their next set of offerings. There are no marketing big wigs calling Gartner to get help with messaging and positioning. There are no market researchers who call on Gartner to sift through data for analysis as to why something will or will not happen, only that it did. They use Gartner data if it makes them look good, and don’t if it doesn’t. They use a quote if it makes them look good, and don’t if it doesn’t. End of story.
Further, people in industry generally do not like dealing with Gartner – they don’t pay them for help, they pay them because they feel they have to. They get no value generally, but the potential of garnering negative value is too great a risk for them to ignore. In short, you pay Gartner to hope to end up in the top right of the quadrant.
I know this first hand – I was a customer of Gartner. Virtually every ESG practitioner was also a customer of Gartner. The stories are fairly universal – and they are the same stories told privately by most every executive of every vendor in the IT space.
This came from one of our analysts – and I think it’s a fair representation of many in our industry:
“In a prior vendor-side role, I was in ZL’s position – my company/product was consistently placed in the niche quadrant for a few years running without an interview or review by Gartner (no facts to base their opinion on). I was able to prove (with a Gartner ombudsman) Gartner’s complete incompetence in constructing his (the analyst’s) MQ year-over-year as I had red-lined documents and an e-mail trail pointing out incorrect information in the previous years’ reports that he regurgitated year after year verbatim. His response was that we should be happy we made the MQ at all (Gartner was giving us ink). I lost all respect for Gartner after that. The analyst (a very long time, very entrenched “name”) was “let go” soon after … no idea if it was related. Still, the company I worked for had little recourse. The worst that we could do was to drop our Gartner subscription.”
No one wants to pay protection money, yet they do. Given a choice, they wouldn’t. No one stands up to the mob, out of fear.
Allow me, at this point, to reaffirm my absolute unabashed jealousy of Gartner. Who wouldn’t want to wield such power? There are not many companies I can think of where the customer hates them, gets no value, and yet still gives them piles of money. The only ones I can think of are monopolies – or governments. Hmmm. That is power. Of course not all power is used for good – much is used for evil.
My problem with Gartner is simple – they simply do not live up to their perceptive level of expertise – at least in the markets where I live. Their people are not practitioners of the art normally, which is why they can’t really bring any legitimate value to industry. Their people simply have not stood in the shoes of the people who give them money. That is not to say they don’t have very smart people – they do. They also have a lot of complete and utter idiots. The problem is that it’s very hard to tell where the “intelligence” comes from – unqualified idiot or brilliantly qualified person.
So the real question is, if Gartner’s influence is based on an invalid perception of expertise, and Gartner knows it, does that equate to a deceptive business practice? Since the result of this ends up in the case ZL is bringing – that Gartner is using an unqualified invalid perception of expertise to negatively influence the ability of ZL (and everyone else in that position) to conduct commerce.
What is the recourse for those affected by this practice? Nothing really. You could sue the mafia for shaking you down for protection money, but you would lose. It’s your choice to give them your money. You decide the value. ZL is suing the mafia. The judge will most likely say, sorry – you don’t have to pay them if you don’t want. That’s the only recourse industry has – they can stop doing business with Gartner. The reason they won’t is obvious – fear. It’s the same reason they do business with them to begin with.
The really bad part of all of this is that Gartner doesn’t have to be doing things like this. They could refill their ranks with qualified experts. They could add logic, reason, and disclosures to their MQ choices. They could legitimize themselves – and then they would deserve the influence they command.
The reality is that they probably won’t – and the reason why is simple: money. They are public. Idiots are cheaper than experts. Plus, they don’t have to. They are protected as long as their influence on the buyers exists. There is no incentive for them to do anything except propagate the myth of expertise. People are going to pay them because they don’t want a “labor dispute.” It’s a cost of doing business. Why add value if it doesn’t matter? The only way this “family” is broken up is either through a massive industrial backlash (i.e., all the big guys band together and publicly call Gartner out) or a government intervention. Otherwise, I’m afraid ZL and all the rest are hosed.
The only way ZL wins is if someone with enormous pockets, who cares less about Gartner, and who themselves wield huge power and influence in the market decide to join the party. I’m thinking Larry Ellison would be perfect for this role. Oracle and ZL have a relationship currently. That would make things really interesting.
Related posts:
blogs
Steve – this is a thought provoking post.
Let’s take your mafia comparison – there has to be a market. The end users are complicit in handing the keys of the kingdom to Gartner. And so are the vendors.
Someone once said that special knowledge and information – something that “I” know that you don’t – even the implication of that insight – is very powerful.
Where can IT professionals go to get relevant information to make buying decisions? There is no gold standard for this – there is no ultimate source. So they go to Gartner, the trade press, other analysts, bloggers and their channel partners. They don’t go just Gartner but they are the biggest voice in the IT industry. And they don’t just go to these sources to make their decisions – they often use this content to justify decisions they already made. One IT professional said to me that sometimes when his boss asks why he bought a particular solution, he shows them various analyst reports and articles. He called it his CYA reports.
Vendors are complicit as well. Sure they complain when they aren’t in the Magic Quadrant but you know all of them use Gartner and IDC research to show investors and potential customers how big and important their markets are.
On another note – regardless of whether ZL wins or not – someone is challenging the game and I admire that.
Our IT world is shifting and I don’t think the trade press or the analyst community is prepared for these changes And yes, it will take some time for this to occur but we are clearly in the midst of this transition. It isn’t just Gartner that should be concerned about this but all press and analysts.
A fair point(s). I don’t mind the research aspect however, that is legitimate value. A vendor pays to use those research finding to their benefit, so I don’t put that in the same category. It’s why I didn’t take on other big guys like IDC or Forrester, for example. Vendors pay tons to IDC for backward looking market share numbers – but they don’t “have” too. They want too, and while I question the value of those numbers, they clearly don’t. If, on the other hand the only way that IDC would put you into a market share study was if you paid them too, it would be totally different. Forrester, on the other hand, has a large end-user business, but does not seem to have the same stigma that Gartner does (positively or negatively). I can only attribute that to what I hear – that they do a very good job and have very credible people.
I completely appreciate the CYA point. That could very well be the value to the end-user. However, it doesn’t mitigate the issue if covering your butt (CYB) means you rely on complete horseshit from a 3rd party.
Finally, you bring up a really great point in “where else can end-users go?” to get legitimate information. I contend that there is so much crap out there, of unknown or cloudy origin and expertise, that sooner or later people who need real input are going to pay for it. I hope I’m around when that happens!!! Cheers.
Steve one of your point rings out louder than the others, and that is Gartner once had a top notch bench of people from the trenches.
People that long ago when we were on the customer or vendor side of the table as Gartner customers saw as being in tune.
However lately, Gartner appears to be trading more on reputation, hiring more sales people to beef up their sales per analyst ratios at a time when others are beefing up their knowledge workers/teams capabilities.
I too am envious of the Gartner model from a business perspective.
After all, if charging the amount they do per customer or vendor access to an analyst for a briefing or call or interview, I would have a fleet of high-end sports cars including one for each my dogs.
Maybe that sums up the whole story.
That is, being a public company, there is more emphasis on sales revenue per head, thus the need for story tellers to propel the myth of the value, drive margins.
However what’s next then, outsource the G2 gathering and analysis to other third parties?
Hmm, maybe that fleet of sports cars is not that far out of reach!
Cheers gs
This blog post sums up my feelings on Gartner as well, although I have met way too many end users who put far too much credence in the MQ. Just last month I was talking to a potential client who told me that the short list for their backup and archive project was already set and was done so using the MQs. If you use the MQs for backup and archive (two different MQs), I believe you end up at only one vendor. Nice. As far as this customer’s management is concerned, they only need to make sure that this company’s products work. If they do, why look anywhere else? Madness, I tell, madness.
So what you are saying is that I really should create the ESG Circle of Irrelevance, place all vendors in a segment randomly in the circle, and allow users to interpret it as they see fit! Then they wouldn’t need the MQ to justify their decision, they could justify any action whatsoever. All I’d need to do is charge each vendor a fortune for being included randomly in the CoI and voila! They can claim anything they want too.
Seems like everyone would be happy then. Hmmm.
@Tony
The company I’m starting, http://www.truthinit.com is looking to fill the void you’re talking about. Our purpose in life is to help companies know what products actually work and which ones don’t — which ones are good for certain purposes and not for others. We’re just getting started and are currently in beta, but that is my goal. If you want a MQ based on goodness-knows-what, go to Gartner. If you want to know what actually works, come to Truth in IT, Inc.
Interesting name by the way……
Do tell – how does Truth in IT make money by testing things properly and truthfully?
As a current ESG and former Forrester analyst, I share your admiration and mistrust of the Gartner MQ. Let me provide a recent observation that calls the Gartner MQ into question.
In May of this year, Gartner published its MQ on Security Information and Event Management (SIEM). One of the vendors in the desired upper right hand quadrant was Cisco with its Monitoring, Analysis, and Response System (MARS) product. This struck me as odd since I’d be hearing for a while that Cisco was not keeping up with the competition on product updates, angering customers and alienating the salesforce.
If I knew that MARS was falling way behind, my feeling was that Gartner must also have known this.
In October of this year, roughly 5 months after the SIEM Magic Quadrant, Gartner issues a research note alerting customers that Cisco was putting MARS on maintenance and would no longer support 3rd party devices. As such, Gartner told its customers to plan on migrating to another enterprise SIEM platform.
To me, this is an example of the questionable nature of the Gartner MQ. How could Cisco get a top position if everyone knew that MARS had become a second-tier product at best? How could Gartner come back 5 months later and reverse its position (note that it never mentioned its SIEM MQ in its October research note)?
If I purchased MARS based upon the Gartner SIEM MQ, I would be extremely angry and would at least explore legal recourse. Gartner is in an enviable position where it makes a lot of money on its opinion (i.e. MQ), keeps its methodology secret, and somehow avoids any responsibility for bad calls. Even organized crime figures go to jail or get “whacked” now and then.
Perfect example. How can the opinion be based on expertise, if anyone really following the industry knew Cisco was behind? Why would they continue to show up as leaders, when not only were they not – but it was even worse??? Customers found out shortly thereafter that those who purchased recently, were screwed.
On one hand I feel bad for the IT guys who used the MQ to justify their decision and who subsequently got fired, but on the other hand I don’t. If you use Gartner (or me) as an excuse to be lazy, you get what you deserve. If you count on any “expert” for actual guidance, then you deserve better – certainly better than this kind of b.s.
I can say with strong conviction after personally doing a demo of many products multiple times on ‘magic quadrant’ performers that, usually it’s smoke and mirrors on the reviews. We demoed bluecoat, riverbed, citrix, cisco, packeteer (before acquisition by bluecoat) on a wanop project. Bluecoat was hands down the best for our company but cisco cut costs to make the deal happen for our company. Riverbed (the MQ leader) had a horrible demo experience. When we called them with a legitimate issue the arrogance (presumably stemming from pride in their product) on the other end of the phone was such a fast turnoff we honestly didn’t give them a fair shake based on the support for the device.
The value Gartner DOES add is their communication to keep executives in the loop on warnings and awareness for certain technologies. Anyone who has attended an exec level tech expo/conference has had buzz word poisoning. Every year there’s a new list of 10 or so hot new technologies (this year seems to be all about the cloud). Gartner does a good job quelling the reckless drive for technologies like this in places it’s simply unjustifiable.
Another solid point they do is price checking. They’re a trusted source of purchasers willing to go out and grind the sales reps down for you. Most of the time they’re pretty close to what you can find on your own if you have a decent relationship with your vendor.
Good points. They do have executive reach so they have the ability to cull fact from crap. In terms of price checking, once a completely valid service, the age of social media has erased that value. If buyers don’t know if they are paying too much, they should connect to the internet. It’s not hard any longer.
Much of the comments can be summed up as Gartner doesn’t have the influence that ZL says they do” and “no responsible IT manager would rely on Gartner to the extent ZL says they do.”
I’ve been on both sides of aisle and cannot imagine that IT professionals are that influenced by Gartner. No offense to ESG et al but no analyst does. It is helpful information, especially if you have a “been there done that” analyst, but nothing more. So, if Gartner is but a shadow of former self then ZL is especially off base.
Here’s a challenge: all analysts go to your clients and ask to see their purchase decision matrices (they have them) and see how prominent a position Gartner or any firm has.
One of the interesting things to note about responses to the article is that nearly all comments are from industry pundits, and not end-users or those identified with IT vendors. Even if there is a silent high % of people who feels strongly on this topic, one way or another, the fear that their opinion may be seen by a colleague or by Gartner is likely what is holding back their public comments.
At its core, it cannot be debated that in some circles, Gartner holds a significant amount of influence. I would argue that the exclusivity of this influence is waning, thanks to the growth of competitive analyst firms (IDC, ESG, etc.) and the growth of peer to peer media, but there will always be shops that give the magic quadrants and other Gartner material significant credence. Vendors, typically those who provide Gartner the most income, and coincidentally perform the best in the reports, are happy with their position and will argue its accuracy. Those who pay less, have smaller share and lower marks, may be happy to be included, but know they can never crack “the upper right”, so they utilize other data to present worth and value.
Putting aside the “pay for positioning” argument, there is also the “pay for citation” and “review all mentions” policy that is especially frustrating. Assuming that ZL were to achieve elite “upper right” status with Gartner, they would need to first, become a client to gain access to the report, swear on the dotted line that the report would not be distributed, and then, if they wanted to mention said report, they would need to run all potential public mentions by Gartner, who is notorious for editing every word in releases and collateral.
That’s why most people refer to Gartner’s copyright squad as the “Gartner Police”. It’s not the mafia, but it’s something, indeed.
I personally believe, not speaking for any relationships I have with vendors, past or present, that the value of these studies and charts is decreasing. I believe that the best IT shops will discover potential vendors based on their own research and word of mouth, and do the necessary testing and compares prior to making purchasing decisions. But aggregate data from IDC, Gartner, ESG, or any other firm is something that IT vendors cannot do on their own – and everybody loves PowerPoint slides that show growth in their market (as validated by Analyst XX from Firm YY). That sheer data crunching and year over year number charting is hard to replace.
Well said. On a side note it is funny, I’ve probably had no less than 20 emails/tweets on the subject from industry folk agreeing but afraid to mention their names/companies. I understand – if anything it furthers the point. Further, not many users will stand up and be counted less they be found out by their boss!!! Oh, the power of power!
I’d be interested to see this same post on something like slashdot where more “hands on” people live. I am a hands on guy and I don’t see myself as the typical person to follow a blog of this type.
I’d also like to point out that an analyst is a “made or fade” kind of career based on diplomacy and tact. The “hands on guy” not so much as a matter of fact I think the standard blunt and harsh on most trench dweller’s resume.
@Steve
Think combining Consumer Reports w/traditional consulting. A low annual fee from those interested in knowing works, combined with experts in the technology on our end. Instead of the traditional consulting model (which of course, I know very well) where you charge one company $50K just to give them that answer (which I did a lot at my former employer), we charge 50 companies $1K and give them all the same answer. One of the secrets of consulting is that everybody asks the same questions.
Customers will have access to experts in their subscribed fields, and they’ll be able to ask what I would call “billable questions” of them if they agree to have their questions anonymized and published back out to the populous. (We’ll, of course, have private, custom services available for discounted rates as well.)
One key, though, is NO vendor money, which is what we get from Consumer Reports. They accept no advertising and neither will we. I know that there are people out there that will pay for real, unbiased information. We just need to find them.
Bless you if you can pull it off. Hard to get a little money out of a lot of people, but i admire the plan.
[...] My friend and colleague at ESG just blogged about an interesting court case happening in the USA on The Bigger Truth. “An e-mail archiving company, ZL Technologies, Inc., has sued, been dismissed, and re-sued [...]
Mr. O’Donnell probably has it right – except when he’s wrong. He didn’t look at the MQ, but surely others do. Or is this all a wasted exercise? Maybe no one cares and I’m wasting everyone’s time!
Steve
Here’s my take, nobody who knows anything about IT would pay any attention to Gartner anyway, the Magic Quadrant is uni-dimentional focussed on solving a particular silo of technology problem. Enterprise IT guys have a completely different focus – on Business issues and business problems. CIOs look for three key measures when considering technology: – Business Risk, Business Agility (Cycle Time) and Business Cost.
In my career I probably spent a few more billions of dollars on IT than the average and I can honestly say that I never paid any attention at all. Magic Quadrants just don’t help with any of that so are more important to vendors as bragging rights than useful to buyers.
Steve
I hope for an open source approach to tackle their monopoly, much along the Redmonk style. With the proliferation of blogs and other social tools, this is not just a fantasy and it can (will) happen eventually. It is tough but doable.
OK, so I’m posting as a semi-anonymous poster. Why? Because I’m a vendor caught between knowing that GartnerIT is a ship of fools as well as knowing that they still have enough influence that we have to play nice-nice.
I’ve briefed analysts at Gartner focusing on 2 areas of the storage industry. After 7+ years, one is still a clueless idiot in her field of ‘expertise’. She truly doesn’t get it and everyone in the industry knows it. The other is so arrogant and rude that they make my blood boil, even telling me that my terms for my technology were WRONG, and here are their terms that MUST be used. Didn’t matter that the industry wasn’t using those terms, and never has.
If I need #s, I go to IDC. I believe their research methodology and presentation of data is far superior.
If I need advice, feedback or reviews, I’ll go to ESG and your peers that employ knowledgeable, experienced people who can give me the real feedback I need to set my strategy, pricing, positioning, etc.
ZL won’t win, for all the points you and other mentioned. What’s awesome about it is that someone is finally opening up the flood gates for the long overdue reality check of what Gartner has really become.
You will all be sorry come March…
Steve,
Thank you for having the courage to openly discuss one of the IT industry’s biggest pain points.
I agree with your concerns. In my opinion and experience with Gartner vendors that do not pay them, ask for their opinion and “court” their analysts are forever doomed in their respective MQ’s.
Technology blogs and social tools combined with industry experts who can add relevant opinions to the conversation are absolutely where the market is headed. Vendor-neutral firms that can provide data and informed opinions will emerge as the market leaders.
I have interviewed higher education leaders for many years; and more and more often I am hearing that they are dropping their subscriptions. Vendors I work with are also telling me that they are reducing their spend in 2010. A “consumer reports” model is emerging and technology and business innovators are at work to support the reality.
Vicki Tambellini
The Tambellini Group
edu1world.org
[...] insisted I look into an article written by Steve Duplessie of Enterprise Strategy Group titled: ZL vs. Gartner – Interesting at many levels. It all started when archiving vendor ZL Technologies sued Gartner Group for what is calls lost [...]
As a former Gartner analyst, VP and Research Director, I would like to set a few things correct in the above article. This is in no way a defense or a promotion of Gartner, just simple fact corrections.
1. Most Gartner analysts are required to do phone calls with clients. They are typically 30 minutes a time and most of them are with C-level’s (CEO, CIO, CTO). So the statement that “There are no CEOs who call on Gartner for advice as to how to run their business.” is completely wrong. I previously did between 5-8 of these calls a day, nearly every day for 5+ years. Some were for R&D purposes also, understand what different markets needed in particular and developing a product adjustment or change to fit the market. The company size ranged from small to very large. Many product managers are also making these calls to understand how better to address markets and how better to compete against their competition in different markets. As a former analyst, this was one of my greatest tools for gaining knowledge also. What better way to understand market trends than talking to the leading people directly responsible for different strategic parts of key IT businesses. What is sad about this service is that many subscribe to it, but fail to use it. Those that did use it, used it over and over. And those that used it were easier for the sales people to get renewals from because they got better value out of it than just simply reading a research report. Each year, Gartner has over 10,000 client interactions with organizations world-wide, 200,000 one-on-one client discussions and 12,000 vendor briefings (source: http://www.gartner.com/technology/why_gartner.jsp) So either they are lying (which I know for a fact they are not) or someone is getting value.
2. You may argue that the advice is not of value, and that fees are paid only for “mafia” type reasons. But if this was the case, you would not get companies and individuals lining up for Gartner conferences which are outside of this. The above mentioned 200,000 one-on-one client discussions happen at these events. People talking to people.
3. Gartner does not charge for vendor briefings. You simply have to go through the right process, fill out the forms and which analysts (if you know the appropriate ones) you wish to talk to and a time will be allocated. You get about 30 minutes, sometimes an hour if you have your act together and your story is compelling. Not only that, when a research report is written that includes a vendor, each vendor is sent a copy of the report ahead of time to ensure factual correctness. If they fail to respond, or respond with non-factual information (i.e. “I disagree because we are better”) then that is their own fault. Every research report that I ever wrote that included a vendor went through this process and a report could not be published without the vendors response or a period of absence of to a response.
4. With respect to the MQ, yes, it’s a great marketing tool. But at the same time, its not just throwing a dart at the grid. A considerable matrix is built out of different criteria, which is then used to place the dot on a X-Y basis. Each point is debated fiercely between many analysts. At least that was the case whenever I was involved in a MQ. Yes, this is still subjective, but it’s the collective opinion on numerous analysts who determine each point, which when added up determines where the dot is placed. Its not a subjective single point of “I think I will put the dot here”.
5. With respect to expertise, at least during my time there, I was surrounded by some of the most talented industry experts I have ever met. People that went through the ranks of corporate and did hard time, then were recruited into the role at Gartner. The interview process alone is very grueling. With panels of experts ensuring you know your stuff, can talk about it properly and with authority, not shooting from the hip or getting help from others. Even a research paper has to be written as part of the process and then you must stand up and defend it against a panel. This is tough and many don’t make it through the interview stages.
6. With respect to distrust, look at the history between Oracle, Microsoft and Gartner. These are some big spending clients and yet still much negative information is published. As former lead analyst for Microsoft is Asia, I wrote many negative articles and was backed fully by management. I also wrote positive articles when they deserved it. It did not make me popular in Microsoft, but it got me their respect and the right people to talk to. The one thing that I loved when I worked there was being able to write my thoughts. Gartner does not do take money for research to be written. Those companies who buy larger subscriptions get more analyst time, but that does not mean a research report. That means more time to discuss product and strategy. If the analyst finds it relevant to Gartner clients (let us not forget that’s who pays a lot, not the vendors only, end users of IT), then a report may be written.
Now to “my opinion”.
At the end of the day, market trends cannot be based on fact. They can only be based on an interpretation of the facts as they have yet to happen. Even what happened in previous years is often modeled for key parts of the market as the data may or may not be available. But this is done with a great degree of thought and process. But Gartner is not about fact, it is about advice, whether written or verbal. From my perspective, it’s a well educated opinion offered in the form of advice based on industry experience, the research of many and discussions with industry peers. An individual is free to take or ignore the advice, or for that matter even challenge it.
As a former Gartner analyst, I made many predictions on what I felt different markets would do – these were based on my opinion and derived from experience and the facts at hand. Many of these predictions came true, many did not. But they were predictions that were scrutinized by many other analysts before they were ever published. These analysts are in a position of trust because of the Gartner brand. As such, they do not want to lose that trust. Analysts that are bluffing it do not last long. Both Gartner and the industry are not that foolish. No substantive decision is made on a MQ alone, but it certainly helps save a lot of time for many. If the companies don’t promote their products, their messages and their capability well, there is probably a reason they are not on the MQ. Each time I was involved in the MQ, I talked directly to the vendors on the issues and ranking points.
If Gartner is as bad as you are portraying them, then that would mean that the industry is fools for allowing them to have this position and then continuing to buy. Gartner is not the one making a choice. The customer is making a choice to pay for Gartner. Sure the big guys like MS spend a lot of money and of course they get analyst time because that gives them the appropriate level of access via their subscription to talk to and provide information. But those same opportunities are available to smaller companies, whether they are customers of Gartner or not. In fact of Gartner’s customers are pure consumers of the research, not vendors at all. I live in Thailand today and there are a lot of companies here that are using Gartner via a paid subscription. This includes banks, airlines and others. They have no reason to buy a Gartner subscription if they did not get value. The same goes for companies in many other markets.
Companies that want to influence Gartner analysts don’t need to buy the research, they need to get organized. Have an analyst relations strategy, make regular contact, not just via email, but schedule briefings. Build a relationship with the analysts. Understand what they are looking for. Don’t hype your product, get to the meat. The number of hype sessions that did not have meat behind them when I was doing vendor briefings was incredible. The down to earth guys that had solid product, solid strategy, spent the time and energy to understand what the analysts were seeking, not ramming marketing propaganda down their throats got the respect of the analysts and got the messages across to them that needed to be. Saying “my product is great” is not helpful. Let the analyst decide if it is great or not. Show your product, not your ego and you will go a long way further. But if you can’t convince the analyst that it’s great, don’t blame them, take a look closer to home. Maybe it is great, but your messaging did not convey it enough or maybe your opinion of your own product is inflated. Steve, if you were able to take something up to the Gartner ombudsman and it was factual not just hype, then I believe it would have been acted on. I had several vendors challenge statements I made, including some of the biggest, the ombudsman took if very seriously, but each time while the vendor thought “they were right” failed to offer anything other than opinion or disagreement with the analyst and other analysts opinions. Each research report goes though many stages of approval and review. It’s not the better product, it is the entire offering, company, strategy and more. You had a chance at vendor review to correct those issues. That is the reality. And while I don’t agree with the response you say you were given, I personally would have never said that, many don’t make it on the MQ at all and any position on the MQ is still better than not being on. It shows that you have some influence in the market, but if the bottom left, shows that you need to get more serious about certain aspects of the business. In many cases a niche player was a great fit for many businesses. But even with great product if a company is not competitive on support, able to articulate a clear strategy, show the right case studies, unable to get beyond the hype then they are ranked lower. Some of the MQ that I partook in the analysis on had 150+ fields of criteria. Just because you don’t know them does not mean they don’t exist.
At the end of the day, it is expert advice that people subscribe to Gartner for most. That advice comes in the form of analyst opinions. The bio of each analyst is on the Gartner site. If you don’t think they are qualified, don’t take their advice.
I will probably get flamed for this comment. But it seems that there are a lot of “research firms” that are willing slap Gartner for their size while at the same time saying they respect the company. The author of this article is a good example. Perhaps as a researcher you should check facts a little more before publishing. This seems a little too heavy on the envy site to be credible. I am not saying Gartner is perfect mind you. Every company has its flaws. But many described here are simply factually incorrect. My opinion is that this entire “anti-Gartner rant” is little more than a marketing ploy from a “theoretically” competitor.
Yes, Gartner does hold a role of dominance. Maybe they deserve it, maybe they used to deserve it, maybe they don’t today. That is a point for the end user IT companies to decide and for the moment they are still buying. Vendors pay money, but a huge number of customers get nothing from Gartner but opinionated advice and are not vendors.
I am several years out from Gartner and still have a lot of respect for their internal processes and thoroughness. I am not a client, shareholder and have no relationship to Gartner at all other than remaining in occasional touch with a bunch of great people that I still consider friends, many who have also gone out into the world after doing their time at Gartner. I left after I decided I wanted to grow beyond the Gartner shores and get beyond advice and go back to being in the field. I started my own company like you, not my first. This is not uncommon with many of these great people that you have decided to stereotype as 650 “non-expert” analysts. And thus my reason for replying to your blog. I too was one of these people that you bundled in this group not so long ago. So yes, this is personally insulting and insulting to all current and former Gartner analysts. If I and all others you bundle together are not qualified just because we worked at Gartner, then neither are you my friend. As you said “I know what a skunk smells like”. Again to quote you “Is perpetuating and even actively attempting to foster the perception of expertise where little or none actually exists” Take a look in the mirror and read your own bio. (http://www.enterprisestrategygroup.com/steve-duplessie/)
Marketing speak is easy. Steve, your own bio has plenty. Saying you are respectful and then going out on a full out attack that is based in many parts on non-factual information shows just a little bias don’t you think? If you are the professional that you claim to be, you would not go on the attack like this, you would show what you have, not what they have. The reality is that this is ESG is a competitor to Gartner, albeit a smaller one, and this is a nice bit of marketing showing your own opinion on Gartner, more than on the case at hand, in order to get some attention to the ESG website and your blog. Meanwhile a captive audience of your own customers for the most part who read your blog respond in the comments. The irony is that there are so many factual errors in this blog article, yet it is entitled “the bigger truth”
While this was a long, mostly well thought out response, I think it makes my point more than counters it. I’ll get specific on it later, but I was very clear in stating that “IN MY AREA” my comments were accurate – as is clearly supported by the other 25 or so comments. I made no claim other than to doubt much potential difference among the other thousand areas Gartner covers. Having said that, I will tell you that I received dozens of direct emails and a few calls on the subject from VERY large companies where Gartner does millions and millions in business annually – all affirming this position. They don’t want to go on record, for the very reason I stated, but they sure do seem to support the primary argument I was making.
Conversely, outside of yourself, why didn’t anyone who was receiving legitimate value from Gartner come forward? Why didn’t any user or vendor anonymously come forward to say “hey, that might be true elsewhere, but I love my Gartner analyst and trust them with guidance for critical decisions I need to make.”? No one. I haven’t had anyone (until now) defend the value received from Gartner. Not one. The closest thing to a supporting argument came from one end-user who basically said that all analysts suck and no real user would base any decision on anything anyone of us published or says.
Why has no one jumped on Gartner’s side of this? Do you think they are afraid of me? Are they concerned that I might use my absurdly limited power to somehow subvert them if they stood up for their relationship with Gartner? First, like your comment, I will post it. Second, I am GLAD when people get legitimate value, not mad. Gartner’s ability to drive value to our common customer has no bearing on me – other than anything that helps an ESG client out is ok in my book. We don’t compete, regardless of your thoughts. I challenge you to find any common customer who really thinks we do. Gartner and ESG are not used in the same way, by the same people, even within the same customer. For what it’s worth, I’m also jealous of IDC’s power – and guess what, while we provide research, we don’t compete with them either. As with Gartner, it would be a fools game. We either do something different, or dilute ourselves. I might be a jerk, but I’m not that dumb.
If you think for a minute that there are real CEO’s who are on the phone with a Gartner analyst for 30 minutes for any reason other than “face time” or “penance” you are nuts. If you believe that those same CEO’s count on that precious time for actual insight that leads them to make actual business decisions based on those conversations, you are delusional. I challenge you to name one. Big or small – name just one. If you can tell me how Joe Tucci or Dan Warmenhoven or Larry Ellison ever took ONE single action directly based on intelligence and guidance received on one of these calls, I’ll shout it from the top of this blog! Pick a small CEO (but an actual legitimate business person), same deal. In ten years and a thousand companies, we just haven’t seen it. Could it happen outside my purview? Absolutely. Probably not much, but it absolutely could.
Further, IN MY SPACE, I challenge you to find any individual or team of Gartner analysts that are more knowledgeable or bring more “business” value to the table than an ESG analyst (value determined by the customer for making BUSINESS decisions). If they existed, I’d hire them. If I could I’d show you the pile of resumes we’ve collected from those experts over the years, and I assure you that you would find the cream of the Gartner crop among them. It isn’t because we can’t afford them that they don’t work here. It’s because what they might be great at is not what we do.
If you take the time to re-read my post, I think you’ll find that many of the things you accuse me of, or contend I am wrong about, are well covered – including my envy. I’m enormously envious of Gartner’s success and power.
If you have any specific areas where you feel I’m just plain wrong, I welcome you bringing them up. Thanks for taking the time. Steve
I rest my case on the marketing front – well done. With respect to the CEOs, I have a Rolodex of several hundred, that I have personally advised, including some of the biggest and smaller – even including government ministers and prime ministers. Have a search on products like Windows XP Starter Edition – look at their origins. Yes, you are wrong. Much more face time was given than 30 minutes. But lets not make this a pissing match, it does not matter who I advised. I don’t need to drop names. If you want to know, a simple Google search will solve that issue. So by your account, I guess I am disillusion.
I have already listed 5 areas where I think you are just plain wrong. But one more is to bundle all analysts that have or still work for Gartner as “non-experts” in their fields. All 600+ of them apparently.
I expect if you are right then those 10,000 client interactions and 200,000 one-on-one client discussions are a thing of the past and they will all just go away. If the advice was not of any value they would not be taking the time. I can assure you there are thousands of C-levels, including CEOs that interact with Gartner regularly.
And if you are trying to be different, then your website surely does not show it:
From the front page of ESG:
–IT Professionals gain clarity and understanding
–Count on ESG for straight talk to The Media
–Technology Companies rely on our guidance
–Our 360° view is sought by Investment Pros
From the front page of Gartner:
–We provide IT Professionals with the insight they need to succeed.
–You can access Gartner’s expertise in a variety of ways to help you make the right decisions about technology solutions, align your IT initiatives with business objectives, and connect with industry peers and best practices.
–Gartner is the IT Professional’s best first source for addressing virtually any IT issue
If you cant differentiate on first impressions, then your messages are wrong. Perhaps this is the same reason you had issues when talking to a Gartner analyst.
Anyway, thanks for posting my comment and thanks for the reply. Happy marketing.
“Anyway, thanks for posting my comment and thanks for the reply. Happy marketing.”
I’d guess the Gartner analyst is appreciative of his post being shown because where he’s from they’d moderate/censor that type of thing.
You sort of shoot yourself in the foot here Steve. Yes there are a handfull of people complaining about Gartner here, but the remainder of the people making those 210k interactions with Gartner analysts must be too busy interacting and getting the advice they keep asking for to read your blog and responding.
I have sold software for many firm. Some small and some large… Firms often use Gartner to pick what software they should POC and if you are not in the MQ you have your work cut out to get people/firms to look at your software.
I have sold software for many firms. Some small and some large… Firms often use Gartner to pick what software they should POC and if you are not in the MQ you have your work cut out to get people/firms to look at your software. I think it is great ZL is going after them… People need to know the truth about what Gatner does.
I am managing director of a legal technology analysis firm. While we do provide relative market positions of vendors, we are conscious of the possible visual prejudice that may be caused by a single graph with all vendors overlayed. More important than perhaps disadvantaging vendors, our clients might overlook a great solution for them, that perhaps just does not yet have all the features or maturity that we need to consider in the broader market assessment. But that’s just us…