The world's first case of cheating in AI research



    Imagenet Challenge is an international competition for pattern recognition systems. According to the conditions, a computer program should recognize and classify a number of objects in digital images, identifying one of 1000 thematic categories.

    Many large companies participate in the competition, which allocate large budgets for such developments. Therefore, the news about the cheating of the Chinese company Baidu caused a real scandal: they say this is the first case of fraud in the development of artificial intelligence systems. As a result, Baidu is excluded from the Imagenet competition for next year.

    The Imagenet Challenge is compared with the Turing test for AI systems, although the task here is not in a meaningful conversation, but in pattern recognition. But this is also a very important task for AI.

    In a scientific article , Baidu engineers describe a supercomputer that supposedly shows a better result in object recognition than previous designs, including those from Google and Microsoft. The Chinese said their program showed a record low error rate of 4.58%, while Google’s figure was 4.82% and Microsoft’s 4.9%.

    Baidu posted the publication on Arxiv.org electronically. Articles are published on this site without prior verification. The situation was aggravated by the fact that a couple of days later the news about the "achievement" of Chinese engineers was reprinted by the MIT Technology Review and other scientific publications. Only three weeks later, the fact of cheating was revealed, and the article was updated.

    The investigation showed that Baidu has registered at least 30 accounts to participate in the Imagenet Challenge and over the past six months has submitted more than 200 applications with results, including more than 40 applications over a five-day period in March 2015.

    Baidu later apologized publicly for its action.

    This story once again shows that research in the field of pattern recognition is now one of the main directions in IT, even if large companies are not embarrassed to use such tricks to put themselves ahead of the competition.

    Also popular now: