April 15, 2018 at 12:44Gartner Data & Analytics Summit 2018
On March 19-21, the Gartner Data & Analytics Summit conference was held in London . I have been a visitor to this event and I want to share my thoughts and observations with you.
I was familiar with the activities of Gartner long before the conference thanks to the famous “magic quadrants” and “hype cycles”. Going to the conference, I formulated the following personal goals:
The expectation that the organizers will be able to surprise me with something new has formed purely speculatively. A company that is studying a high-tech and dynamic digital market must have had to demonstrate the best practices of this market during the conference itself.
The first thing I want to note is the conference’s mobile application - Gartner Events Navigator . At first, looking at the required privileges, I did not want to install it, because I was afraid that I would not get anything valuable for myself (besides what is already on the site), but I would provide a lot of data about myself to a third-party company. But, having read all the features of the application, I installed it and, as a result, I was very pleasantly surprised at how popular and thoughtful the authors made it. Judge for yourself, in the application you get:
a) A personal planner with the ability to watch the schedule, schedule a visit to your favorite presentations and download materials in PDF to your phone
b) Social network and chat of participants
c) Location of the halls
d) Feedback tools
e) A recommendation service that will select the best presentations to attend depending on from your interests.
I actively used the application for all three days and I think something happened to me recently that experts from the field of ethics of big data spoke about - users will be willing to share their personal data with the application themselves if they understand what value they receive in return.
The second thing I would like to note is the digitization of human flows at the conference.
Each conference participant wore a badge with a bar code, and at the entrance to each room and at each demo booth, the organizers using this device scanned all visitors who showed interest in this topic:
Considering that each participant indicated enough professional information about themselves during registration, there were about 1,500 participants in total, and the event lasted 3 days, this whole Internet ofthings of people generated a very interesting data array, which, in addition to direct study, can certainly be monetized.
Plus, I liked it more - each performance began and ended strictly on time, minute per minute.
Turning to the conference itself, it is impossible not to note the super-saturated (pdf)subpoena. There were several types of events: presentations, round tables and master classes and an exhibition with more than 50 vendors. It was necessary to accept in advance that it would not work to visit all 100% of the events, from the very beginning it was necessary to tune into a compromise and make a painful choice.
It’s interesting for me to post-factum analyze the statistics of topics covered at the conference (presentations and round tables):
In my opinion, such a disproportion in favor of Data Governance is somewhat artificial. It seemed to me that the true leader in terms of user interest was the topic “AI & ML”, in any case, I could not get to any round table or master class on ML due to the fact that registration for them was already at the beginning of the event were closed while there were free slots at Data Governance events.
Below are a few abstracts / terms / thoughts that I remembered after the last three days of presentations and reports:
Organizational structure- The most common hybrid model. There is a Chief Data Officer who reports to the CEO or CIO. As a rule, there are two types of Data Scientists in an organization: with a specific business function or in a separate Center of Excellence. The purpose of the Center of Excellence is RnD and innovative tasks that cannot be attributed to any of the existing businesses. CDO is responsible for coordinating Data Scientists. He defines the goals and development strategy.
Only 20% of visitors surveyed have the role of CDO in their regular structure. Gartner forecast - by 2020, this figure should grow to 40-50%.
Data Lake - the following implementations are most common:
1. Hadoop
2. Object Store (for example, Amazon S3)
3. RDBMS
Data Lake is not the “killer" of Enterprise DWH, but complements it on the corporate IT landscape.
Security in Data Lake / Hadoop is a real risk, both because there are no standard tools unlike RDBMS, and because almost no one invests in implementing security policies in Hadoop: authentication, authorization, audit.
Data Literacy (literacy with t.z. work with data) - an asset in which you need to invest. Only about 5% of organizations can boast that they implement end-to-end programs to increase the level of Data Literacy at all levels - from Executive Management to Junior specialists.
Logical DWH- a method of organizing access to data in which there is no centralized data storage, as, for example, in DWH / Data Lake. Nevertheless, such an organization is completely transparent for users, and they can use the data quite calmly without thinking about how access is organized to them. (It turns out that there are already a lot of solutions on this topic, see below.)
Open Source- many vendors begin to make Community versions of their commercial products and distribute them for free, thus recognizing the viability of such a model, and at the same time earning a loyal fan base (However, not all. I had a dialogue with one vendor on how I can try Their answer was such that in order for them to send me activation keys, I must write them a letter, sign an NDA, draw up a PoC agreement, in which to write down the criteria for success and the obligation to purchase their product, if all the criteria are satisfactory shnosti are met. ¯ \ _ (ツ) _ / ¯)
In addition to presentations and reports that came in a few streams, in the conference center of the large hall is constantly acting exhibition area in which it was possible to get acquainted with the company and see a demo of their products .
In a structured way, these are the companies that managed to see there:
Major well-known vendors:
- Oracle - presented their cloud platform
- Qlik - Self-service BI
solutions - Tableau - Self-service BI
solutions - Ataccama - Data Quality, Master Data Management, Data Govenance solutions
- Google Cloud - cloud platform
- IBM - presented solutions for Data Science: DSX & SPSS
- Attunity - data replication / CDC solution
- Teradata
- SAS
- SAP
- Microstrategy
- Informatica
- Information Builders
Hadoop commercial assemblies:
- Cloudera
- MapR
- Hortonworks
Solutions for Data Science:
- Dataiku
- Rapid Miner
- R Studio
- Angoss
Solutions for Data Governance:
- Alation
- Backoffice Associates
- Collibra
- Semarchy
- Stibo Systems
Solutions for Self Service BI:
- Arcadia Data
- Looker
- Sinequa
- ThoughtSpot Building
software logical DWH:
- Actian
- BI Builders
- Denodo
- Domo
- Dremio
-Iguazio
- Sisense
- Snowflake
- Trifacta
- Wherescape Of
course, these are not all the solutions that were presented, but looking at them all in such a short time in between presentations is almost an impossible task, especially considering that after the third or fourth company everything is in my head begins to mix.
Summary of Conclusions
The set goals are generally achieved. They got an idea of the direction the market is developing, what players are on it, and where we are now in terms of technologies (Hadoop, Data Lake, Streaming) and processes (Governance, Security, staff training, etc.) regarding what they’ve heard on the summit. These findings will form the basis of immediate development plans.
What surprised me most was the large number of relatively young vendors who aim at the Logical DWH segment, and which, in general, are not kept up by the giants. For me, “Logical DWH” is definitely a topic that needs to be dealt with in more detail and in depth.
Well, in general, I confirmed for myself the conclusion that such events are useful in terms of broadening one's horizons and understanding in which direction a progressive community is moving, so that it is easier to understand where to develop further.
I was familiar with the activities of Gartner long before the conference thanks to the famous “magic quadrants” and “hype cycles”. Going to the conference, I formulated the following personal goals:
- discover something new at Gartner;
- Search for new ideas in the field of working with data;
- Well, it’s a bonus to learn the art of public speaking from world-class specialists (but this topic is beyond the scope of this article).
The expectation that the organizers will be able to surprise me with something new has formed purely speculatively. A company that is studying a high-tech and dynamic digital market must have had to demonstrate the best practices of this market during the conference itself.
The first thing I want to note is the conference’s mobile application - Gartner Events Navigator . At first, looking at the required privileges, I did not want to install it, because I was afraid that I would not get anything valuable for myself (besides what is already on the site), but I would provide a lot of data about myself to a third-party company. But, having read all the features of the application, I installed it and, as a result, I was very pleasantly surprised at how popular and thoughtful the authors made it. Judge for yourself, in the application you get:
a) A personal planner with the ability to watch the schedule, schedule a visit to your favorite presentations and download materials in PDF to your phone
b) Social network and chat of participants
c) Location of the halls
d) Feedback tools
e) A recommendation service that will select the best presentations to attend depending on from your interests.
I actively used the application for all three days and I think something happened to me recently that experts from the field of ethics of big data spoke about - users will be willing to share their personal data with the application themselves if they understand what value they receive in return.
The second thing I would like to note is the digitization of human flows at the conference.
Each conference participant wore a badge with a bar code, and at the entrance to each room and at each demo booth, the organizers using this device scanned all visitors who showed interest in this topic:
Considering that each participant indicated enough professional information about themselves during registration, there were about 1,500 participants in total, and the event lasted 3 days, this whole Internet of
Plus, I liked it more - each performance began and ended strictly on time, minute per minute.
Turning to the conference itself, it is impossible not to note the super-saturated (pdf)subpoena. There were several types of events: presentations, round tables and master classes and an exhibition with more than 50 vendors. It was necessary to accept in advance that it would not work to visit all 100% of the events, from the very beginning it was necessary to tune into a compromise and make a painful choice.
It’s interesting for me to post-factum analyze the statistics of topics covered at the conference (presentations and round tables):
In my opinion, such a disproportion in favor of Data Governance is somewhat artificial. It seemed to me that the true leader in terms of user interest was the topic “AI & ML”, in any case, I could not get to any round table or master class on ML due to the fact that registration for them was already at the beginning of the event were closed while there were free slots at Data Governance events.
Below are a few abstracts / terms / thoughts that I remembered after the last three days of presentations and reports:
Organizational structure- The most common hybrid model. There is a Chief Data Officer who reports to the CEO or CIO. As a rule, there are two types of Data Scientists in an organization: with a specific business function or in a separate Center of Excellence. The purpose of the Center of Excellence is RnD and innovative tasks that cannot be attributed to any of the existing businesses. CDO is responsible for coordinating Data Scientists. He defines the goals and development strategy.
Only 20% of visitors surveyed have the role of CDO in their regular structure. Gartner forecast - by 2020, this figure should grow to 40-50%.
Data Lake - the following implementations are most common:
1. Hadoop
2. Object Store (for example, Amazon S3)
3. RDBMS
Data Lake is not the “killer" of Enterprise DWH, but complements it on the corporate IT landscape.
Security in Data Lake / Hadoop is a real risk, both because there are no standard tools unlike RDBMS, and because almost no one invests in implementing security policies in Hadoop: authentication, authorization, audit.
Data Literacy (literacy with t.z. work with data) - an asset in which you need to invest. Only about 5% of organizations can boast that they implement end-to-end programs to increase the level of Data Literacy at all levels - from Executive Management to Junior specialists.
Logical DWH- a method of organizing access to data in which there is no centralized data storage, as, for example, in DWH / Data Lake. Nevertheless, such an organization is completely transparent for users, and they can use the data quite calmly without thinking about how access is organized to them. (It turns out that there are already a lot of solutions on this topic, see below.)
Open Source- many vendors begin to make Community versions of their commercial products and distribute them for free, thus recognizing the viability of such a model, and at the same time earning a loyal fan base (However, not all. I had a dialogue with one vendor on how I can try Their answer was such that in order for them to send me activation keys, I must write them a letter, sign an NDA, draw up a PoC agreement, in which to write down the criteria for success and the obligation to purchase their product, if all the criteria are satisfactory shnosti are met. ¯ \ _ (ツ) _ / ¯)
In addition to presentations and reports that came in a few streams, in the conference center of the large hall is constantly acting exhibition area in which it was possible to get acquainted with the company and see a demo of their products .
In a structured way, these are the companies that managed to see there:
Major well-known vendors:
- Oracle - presented their cloud platform
- Qlik - Self-service BI
solutions - Tableau - Self-service BI
solutions - Ataccama - Data Quality, Master Data Management, Data Govenance solutions
- Google Cloud - cloud platform
- IBM - presented solutions for Data Science: DSX & SPSS
- Attunity - data replication / CDC solution
- Teradata
- SAS
- SAP
- Microstrategy
- Informatica
- Information Builders
Hadoop commercial assemblies:
- Cloudera
- MapR
- Hortonworks
Solutions for Data Science:
- Dataiku
- Rapid Miner
- R Studio
- Angoss
Solutions for Data Governance:
- Alation
- Backoffice Associates
- Collibra
- Semarchy
- Stibo Systems
Solutions for Self Service BI:
- Arcadia Data
- Looker
- Sinequa
- ThoughtSpot Building
software logical DWH:
- Actian
- BI Builders
- Denodo
- Domo
- Dremio
-Iguazio
- Sisense
- Snowflake
- Trifacta
- Wherescape Of
course, these are not all the solutions that were presented, but looking at them all in such a short time in between presentations is almost an impossible task, especially considering that after the third or fourth company everything is in my head begins to mix.
Summary of Conclusions
The set goals are generally achieved. They got an idea of the direction the market is developing, what players are on it, and where we are now in terms of technologies (Hadoop, Data Lake, Streaming) and processes (Governance, Security, staff training, etc.) regarding what they’ve heard on the summit. These findings will form the basis of immediate development plans.
What surprised me most was the large number of relatively young vendors who aim at the Logical DWH segment, and which, in general, are not kept up by the giants. For me, “Logical DWH” is definitely a topic that needs to be dealt with in more detail and in depth.
Well, in general, I confirmed for myself the conclusion that such events are useful in terms of broadening one's horizons and understanding in which direction a progressive community is moving, so that it is easier to understand where to develop further.