Transformation of threats in the information space: from technological to social. Part III
Is it permissible that a biased interpretation of the opinions of 1,448 Americans, with active information pressure from the US state media, would mark the beginning of World War III? And does humanity have protection against the “interested” actions of a small group of politicians? Modern Online Big Data & Analytics technologies have violated the omniscience of the “hawks” about the opinion of the entire population, as they allow you to get a real assessment of the statements and moods of residents of countries and around the world.
Yes, of course, the creation of a new “Shield for Peace” leads to the next development of the “Sword of War” (the use of bots, individual targeted impact on LOMs (leaders of public opinion), situational information impact, etc.) - the confrontation between the Shield and the Sword never stops. But let's talk about this another time. Or we won’t talk - the topic of National Information Security in 2013 became a priority for all developed countries of the world and many previously public developments receive the signature “national security”.
It’s still “possible”, we will reveal some technological features and social aspects of the Shield.
In July 2013, according to a Reuters / Ipsos poll, over 70% of the US population opposed an armed attack on Syria. Active information processing "Assad used chemical weapons against the civilian population" and the subsequent (August 19-23) poll of the "population" (as many as 1,448 Americans) brought the situation to the brink of war: "Already only half (56%) of Americans oppose the attack on Syria" .
The result of informational artillery preparation - on August 31, the world stood still waiting for Obama to speak: will the “Nobel Peace Laureate” declare war against Syria?
In order to find out how much the picture in the media corresponds to the real situation, we conducted an express study, after 2 days collecting and analyzing 1.7 million messages, we do not publish the full version so as not to clog the Habr with politics, you can find brief results at the end of the article.
And now we will open the veil of secrecy of the SHIELD, which allows us to receive, in real time, a reliable analysis of situations and events.
The peak thematic information flow determines the real-time capabilities of OBD & A systems for:
The case with Syria allowed “in combat mode” to test the capabilities of the iLook Platform and OBD & A BrandAnalytics class systems , and determine the technological limitations of the system (MongoDB, Elasticsearch, Gearman, Memcached, MySQL, PHP, C #):
1. The topics are collected by keywords in various languages - in total, the system recognizes 17 languages, the use of distance search and other operators of the language of search queries allows you to fine-tune the search. For example, we can set the search in this way “Obama Syria” ~ 7 - this will allow you to find messages that mention Obama and Syria within 7 words between them. Thus, the system will add the message “Obama’s decision to launch a direct air strike on Syria” into the topic of analysis, and news digests like “1. Obama walks around Paris: photo! 2. ... 3. ... 10. George Clooney starred in a film about Syria "- no.
2. Messages are analyzed by the emotional coloring (tonality) of messages - a linguistic object tonality allows you to automatically determine the emotion of messages in relation to a given object. For example, the message “Ivan is a wonderful person, he told Peter that Fedor was a scoundrel”: in relation to the object “Ivan” - positively, to the object “Peter” - neutral, and to “Fedor” - negatively.
3. Messages are geolocated - a multi-parameter analysis of the author’s public profile data, texts of his public messages and the environment is used, the information is updated with each new message by the author. The received data is processed through its own geo-dictionaries. Thus, if the author’s profile indicates Miami, and he is checked in at the coffee house in Butovo every day, we will geolocate him in Moscow.
4. Collected messages are automatically analyzed for reposts / retweets / fuzzy takes - we identify popular points of view, get rid of spammers and throws.
5. Analyzed information about the audience - the number of subscribers of each author at the time of publication of the message. The calculation algorithm is individual for each social network and takes into account its features.
6. Automatically generated message ratings - by audience, takes, comments, etc.
Analysts have a minimum of effort - to evaluate these systems, identify existing trends and predict the development of the situation.
What do we get at the output?
The graph shows the number of messages for August 31: the peak is Obama's expectation and speech on appeal to Congress to resolve the attack on Syria:
The purpose of the study is to answer questions:
1. How actively residents of different countries discuss the situation in Syria.
2. What is the attitude of the inhabitants of the largest countries of NATO and Turkey to a possible attack on Syria.
3. Who is considered guilty in this situation: Assad or the opposition.
Characteristics of the study:
1. Duration: 48 hours (2 days)
2. Period: from 18:00 on August 30 - 18:00 on September 1
3. Number of messages: 1,745,549
4. Number of unique authors: 605,484
5. Number of countries: 241 countries (11,397 settlements)
The research was carried out in the main social networks: Facebook, Twitter, Vkontakte, Livejournal, Youtube, etc. The references to the conflict in Syria and the preparation of the military operation of the Western countries against this country were studied.
The most active discussion on the situation in Syria has unfolded in the US Internet space (46% of reports). In second place is the United Kingdom (14%). In Russia (Russian-language traffic makes up about 1% of the world), Syria’s theme is 4% of the global one.
The vast majority of users of social networks in all countries spoke out against the invasion of Western countries in Syria. The most pronounced rejection of intervention in Russia (in 94% of communications), France (90%), Germany (88%), slightly lower - in the UK (84%) and the United States (83%). In Turkey, criticism of the upcoming Western operation against Syria is contained in 64% of reports.
In the US social media segment, users tend to blame the Assad government (64% of posts) rather than rebels (36%) in the current situation in Syria, as do Germany (58% and 42% respectively) and Turkey (70% versus 30%) respectively). In Britain, by contrast, most perceive the guilt of the rebels (57%) rather than the regime (43%). Opinions were divided almost equally in France (51% versus 49%, respectively). In Russia, rebels are unambiguously considered guilty (96% of reports).
1. The most actively discussing the situation with Syria in the US Internet space
2. The vast majority (over 85%) speak out against the attack
3. In the USA and Germany, the population is inclined to blame the Assad government in this situation, in Russia and the UK - the majority consider the rebels guilty.
In October, opinion polls in various NATO countries showed 70-82% of opponents of the invasion of Syria. You can draw your own conclusions.
In 2014, we plan new “world” studies for which it is necessary to raise the bar of technological thresholds several times, and also invite new partners with experience in visualizing (Online) Big Data, building social graphs and coloring connections, applied artificial intelligence, automatic construction of templates for isolating meaningful content, semantic analysis of the text, application of ontological models, identifying new trends, adaptive emotional coloring, sociologists, “high-loaders” - everyone who works in related fields.
The nearest “big game” for OBD & A and operational sociology will undoubtedly be the 2014 Olympics, we plan to tell about the results here at Habré, as well as at the annual Grushinsky Readings (not to be confused with the Grushinsky music festivals :)) - the largest annual Russian sociological conference , where for the first time there will be a special section devoted to the use of modern technologies for online monitoring and sociological research in social media.
It’s great that Russian technological developments in the OBD & A segment are at the world level, and “butt” will be interesting - with world leaders (a month ago, Apple bought TopSy, a company that specializes in collecting and analyzing Twitter messages, widely known for monitoring the US presidential election, for $ 200 million) ) - so, friends, welcome! - we are waiting for interesting projects and new technological solutions :)
Yes, of course, the creation of a new “Shield for Peace” leads to the next development of the “Sword of War” (the use of bots, individual targeted impact on LOMs (leaders of public opinion), situational information impact, etc.) - the confrontation between the Shield and the Sword never stops. But let's talk about this another time. Or we won’t talk - the topic of National Information Security in 2013 became a priority for all developed countries of the world and many previously public developments receive the signature “national security”.
It’s still “possible”, we will reveal some technological features and social aspects of the Shield.
Combat start
In July 2013, according to a Reuters / Ipsos poll, over 70% of the US population opposed an armed attack on Syria. Active information processing "Assad used chemical weapons against the civilian population" and the subsequent (August 19-23) poll of the "population" (as many as 1,448 Americans) brought the situation to the brink of war: "Already only half (56%) of Americans oppose the attack on Syria" .
The result of informational artillery preparation - on August 31, the world stood still waiting for Obama to speak: will the “Nobel Peace Laureate” declare war against Syria?
In order to find out how much the picture in the media corresponds to the real situation, we conducted an express study, after 2 days collecting and analyzing 1.7 million messages, we do not publish the full version so as not to clog the Habr with politics, you can find brief results at the end of the article.
And now we will open the veil of secrecy of the SHIELD, which allows us to receive, in real time, a reliable analysis of situations and events.
Technological characteristics of OBD & A
- Technological data collection: 5.000 messages per second
- Information flow (without duplicates): 17-18 million messages per day (200 / sec)
- Peak thematic information flow: 60 messages / sec
The peak thematic information flow determines the real-time capabilities of OBD & A systems for:
- Automatic tone detection;
- Location geolocation;
- Calculation of (un) clear duplicates (retweets, reposts);
- Calculation of the audience of authors;
- A variety of top ratings (frequency of words, authors, countries, settlements, etc.).
The case with Syria allowed “in combat mode” to test the capabilities of the iLook Platform and OBD & A BrandAnalytics class systems , and determine the technological limitations of the system (MongoDB, Elasticsearch, Gearman, Memcached, MySQL, PHP, C #):
- Thematic information flow - 10 million messages per day;
- Automatic tone detection (slowest module) - 100 kb / s in one stream.
How it works from a “human” point of view
1. The topics are collected by keywords in various languages - in total, the system recognizes 17 languages, the use of distance search and other operators of the language of search queries allows you to fine-tune the search. For example, we can set the search in this way “Obama Syria” ~ 7 - this will allow you to find messages that mention Obama and Syria within 7 words between them. Thus, the system will add the message “Obama’s decision to launch a direct air strike on Syria” into the topic of analysis, and news digests like “1. Obama walks around Paris: photo! 2. ... 3. ... 10. George Clooney starred in a film about Syria "- no.
2. Messages are analyzed by the emotional coloring (tonality) of messages - a linguistic object tonality allows you to automatically determine the emotion of messages in relation to a given object. For example, the message “Ivan is a wonderful person, he told Peter that Fedor was a scoundrel”: in relation to the object “Ivan” - positively, to the object “Peter” - neutral, and to “Fedor” - negatively.
3. Messages are geolocated - a multi-parameter analysis of the author’s public profile data, texts of his public messages and the environment is used, the information is updated with each new message by the author. The received data is processed through its own geo-dictionaries. Thus, if the author’s profile indicates Miami, and he is checked in at the coffee house in Butovo every day, we will geolocate him in Moscow.
4. Collected messages are automatically analyzed for reposts / retweets / fuzzy takes - we identify popular points of view, get rid of spammers and throws.
5. Analyzed information about the audience - the number of subscribers of each author at the time of publication of the message. The calculation algorithm is individual for each social network and takes into account its features.
6. Automatically generated message ratings - by audience, takes, comments, etc.
Analysts have a minimum of effort - to evaluate these systems, identify existing trends and predict the development of the situation.
What do we get at the output?
Case 4. The military operation of the West against Syria: global monitoring of public opinion in social media
The graph shows the number of messages for August 31: the peak is Obama's expectation and speech on appeal to Congress to resolve the attack on Syria:
The purpose of the study is to answer questions:
1. How actively residents of different countries discuss the situation in Syria.
2. What is the attitude of the inhabitants of the largest countries of NATO and Turkey to a possible attack on Syria.
3. Who is considered guilty in this situation: Assad or the opposition.
Characteristics of the study:
1. Duration: 48 hours (2 days)
2. Period: from 18:00 on August 30 - 18:00 on September 1
3. Number of messages: 1,745,549
4. Number of unique authors: 605,484
5. Number of countries: 241 countries (11,397 settlements)
The research was carried out in the main social networks: Facebook, Twitter, Vkontakte, Livejournal, Youtube, etc. The references to the conflict in Syria and the preparation of the military operation of the Western countries against this country were studied.
The most active discussion on the situation in Syria has unfolded in the US Internet space (46% of reports). In second place is the United Kingdom (14%). In Russia (Russian-language traffic makes up about 1% of the world), Syria’s theme is 4% of the global one.
Western Invasion of Syria: Attitude of Social Network Users
The vast majority of users of social networks in all countries spoke out against the invasion of Western countries in Syria. The most pronounced rejection of intervention in Russia (in 94% of communications), France (90%), Germany (88%), slightly lower - in the UK (84%) and the United States (83%). In Turkey, criticism of the upcoming Western operation against Syria is contained in 64% of reports.
Who is to blame for this situation: Assad or opposition
In the US social media segment, users tend to blame the Assad government (64% of posts) rather than rebels (36%) in the current situation in Syria, as do Germany (58% and 42% respectively) and Turkey (70% versus 30%) respectively). In Britain, by contrast, most perceive the guilt of the rebels (57%) rather than the regime (43%). Opinions were divided almost equally in France (51% versus 49%, respectively). In Russia, rebels are unambiguously considered guilty (96% of reports).
General brief analytics:
1. The most actively discussing the situation with Syria in the US Internet space
2. The vast majority (over 85%) speak out against the attack
3. In the USA and Germany, the population is inclined to blame the Assad government in this situation, in Russia and the UK - the majority consider the rebels guilty.
Sanity results
In October, opinion polls in various NATO countries showed 70-82% of opponents of the invasion of Syria. You can draw your own conclusions.
Plans and collaboration
In 2014, we plan new “world” studies for which it is necessary to raise the bar of technological thresholds several times, and also invite new partners with experience in visualizing (Online) Big Data, building social graphs and coloring connections, applied artificial intelligence, automatic construction of templates for isolating meaningful content, semantic analysis of the text, application of ontological models, identifying new trends, adaptive emotional coloring, sociologists, “high-loaders” - everyone who works in related fields.
The nearest “big game” for OBD & A and operational sociology will undoubtedly be the 2014 Olympics, we plan to tell about the results here at Habré, as well as at the annual Grushinsky Readings (not to be confused with the Grushinsky music festivals :)) - the largest annual Russian sociological conference , where for the first time there will be a special section devoted to the use of modern technologies for online monitoring and sociological research in social media.
It’s great that Russian technological developments in the OBD & A segment are at the world level, and “butt” will be interesting - with world leaders (a month ago, Apple bought TopSy, a company that specializes in collecting and analyzing Twitter messages, widely known for monitoring the US presidential election, for $ 200 million) ) - so, friends, welcome! - we are waiting for interesting projects and new technological solutions :)