Wikipedia learned to automatically detect incorrect article edits using AI

Source: Wikimedia The

online encyclopedia Wikipedia has received a new tool - a service with AI elements that will help to automatically detect incorrect edits of resource materials. The ORES service (Objective Revision Evaluation Service) will check all edits for spam or trolling. The creator of ORES is the Wikimedia Foundation. The developers say that the new service works like X-ray glasses (hence the announcement picture. The system will now highlight everything that looks suspicious and then send it to the human editor for verification. If the administrator decides not to use the edit, the user who proposed it will receive a notification This system is more user friendly since no notifications are being sent to users right now.

The encyclopedia team taught the system to distinguish between unintentional edits and what is called “damaging edits”. The training was conducted on examples of real materials. Now the new service can already be used.

An example of the service is shown below. It demonstrates how editors see materials (left) and what ORES sees (right). The probability that the text is normal is 0.0837. The probability of intentional damage to the text is 0.9163. As a result, the human editor understands that they really tried to ruin the text. In fact, the sentence “Llamas grow on trees” cannot be called correct editing.

ores.wmflabs.org/scores/enwiki/damaging/642215410



And here is an example of the assessment of the "human factor"

ores.wmflabs.org/scores/enwiki/damaging/638307884



This is not the first such tool that works for the benefit of the online encyclopedia. Previously, such systems also tried to teach how to work, but old services did not see the difference between “harmful editing” and a common human error.

The average text analysis time is about 100 milliseconds. Now supported by 14 languages. Already held 45 million evaluations. This is not so much, since the number of edits of Wiki per day reaches half a million. The new service will help editors quickly and efficiently evaluate all the edits offered by users.