September 12, 2008 at 17:58About analytics. How IT companies felt honesty
It is funny to consider some index of development of a specific enterprise for internal use, publishing only industry average values, etc. And a lot of positive emotions, and quarreling with respondents is not necessary. It is more difficult to consider indicators publicly and impartially. Especially if the level of trust is evaluated. Recently I found out about a minor scandal due to such personal indexes. It was a little over a year ago.
The analytical company IDC publishes Russia IT Services Forecast and Vendor Shares , where it draws conclusions on the status of the provision of IT services in Russia, on the shares of different providers of these services and forecasts how the market will change in the coming years. In 2007, IDC involved LINEX in this study .(League of Independent IT Experts). This company has introduced a “confidence index”, which shows how real the data on the revenue of the IT company in the IDC questionnaire is. Those. at LINEX they tried to measure whether the respondent lied strongly. Actual index. And not only because it catches liars with a word, but because the respondent could not understand the question in a trite way, make a mistake when filling out. The index is over the top - it means you should call and clarify.
This measure of honesty ended in collective writingfrom not at all the latest IT companies Compulink, CROC, Ai-Teco, LANIT and R-Style to the president of IDC. In it, signatory executives were outraged by the index and insisted that the IDC impede its publication. As I understand it, there was no publication. There was a mention of the existence of the index at a press conference. In addition, the angry group demanded from now on to coordinate with the respondents the transfer of data from the questionnaires to a third party. If estimates of the accuracy of the data provided fall into the media, then all of the above will publicly break with the IDC.
How did it all end? IDC gently sent our IT companies, saying that it was happy with the subcontract, and that the rights to the results of the study were not transferred to anyone. LINEX continues to consider the discrepancy of the data provided and updated. They are already converging more precisely than before, twice as accurate. Those who do not want to participate in the study do not participate. Who is better? Rating - no, its representativeness has suffered. Although my, for example, respect for IDC has grown slightly, because at first glance, they included analysts unrelated to their company in the calculations. The accuracy is likely to increase. For Gartner and IDC, I sometimes find, to put it mildly, a difficult attitude among managers. Forrester is better. Managers of outraged companies also did not feel better, I think if they really get bonuses for getting into the IDC rating closer to its beginning.
It’s impossible to figure out this story in a couple of days, I’ll try to keep it in sight. I’m not sure of the usefulness of the ratings in general, but I’m sure of what exactly - to fight for the adequacy of their place on Olympus, we need to constructively criticize the researchers and point out no errors in their calculations. And do not demand the removal of specific analysts, leaving only "those that were considered earlier and the way we liked." The lists of the best, in my opinion, should be a way of self-assessment and control for companies, and not a sales tool.
The analytical company IDC publishes Russia IT Services Forecast and Vendor Shares , where it draws conclusions on the status of the provision of IT services in Russia, on the shares of different providers of these services and forecasts how the market will change in the coming years. In 2007, IDC involved LINEX in this study .(League of Independent IT Experts). This company has introduced a “confidence index”, which shows how real the data on the revenue of the IT company in the IDC questionnaire is. Those. at LINEX they tried to measure whether the respondent lied strongly. Actual index. And not only because it catches liars with a word, but because the respondent could not understand the question in a trite way, make a mistake when filling out. The index is over the top - it means you should call and clarify.
This measure of honesty ended in collective writingfrom not at all the latest IT companies Compulink, CROC, Ai-Teco, LANIT and R-Style to the president of IDC. In it, signatory executives were outraged by the index and insisted that the IDC impede its publication. As I understand it, there was no publication. There was a mention of the existence of the index at a press conference. In addition, the angry group demanded from now on to coordinate with the respondents the transfer of data from the questionnaires to a third party. If estimates of the accuracy of the data provided fall into the media, then all of the above will publicly break with the IDC.
How did it all end? IDC gently sent our IT companies, saying that it was happy with the subcontract, and that the rights to the results of the study were not transferred to anyone. LINEX continues to consider the discrepancy of the data provided and updated. They are already converging more precisely than before, twice as accurate. Those who do not want to participate in the study do not participate. Who is better? Rating - no, its representativeness has suffered. Although my, for example, respect for IDC has grown slightly, because at first glance, they included analysts unrelated to their company in the calculations. The accuracy is likely to increase. For Gartner and IDC, I sometimes find, to put it mildly, a difficult attitude among managers. Forrester is better. Managers of outraged companies also did not feel better, I think if they really get bonuses for getting into the IDC rating closer to its beginning.
It’s impossible to figure out this story in a couple of days, I’ll try to keep it in sight. I’m not sure of the usefulness of the ratings in general, but I’m sure of what exactly - to fight for the adequacy of their place on Olympus, we need to constructively criticize the researchers and point out no errors in their calculations. And do not demand the removal of specific analysts, leaving only "those that were considered earlier and the way we liked." The lists of the best, in my opinion, should be a way of self-assessment and control for companies, and not a sales tool.