About how open data is arranged in Russia

    It is no secret that in recent years in Russia, on the one hand, there has been more open data, on the other, a lot of criticism has accumulated.

    It is no coincidence that the topic of open data is what I have been doing for a long time and I know about most of the initiatives in this area not by hearsay, but from my own experience of participation. Here I will try to do without links to my projects, so as not to violate the rules of the site, they will be easy to find by their names in any case. But I will try to give all the links to official resources.

    Therefore, instead of the preface, I will introduce myself.

    My name is Ivan Begtin, for several years now I have been promoting the idea of ​​open data in Russia, I am leading a small non-profit organization (Infoculture) on behalf of which and together with my comrades we will organize competitions and promote openness of data among government agencies.

    Among other things, I enter the Open Data Council under the Government Commission for the Coordination of the Open Government and a number of public and expert councils under the authorities and are aware of many successful (and failed) government initiatives in this area.

    Well, since I, including in the comments on Habré, read a lot of questions posed explicitly or implicitly, I will answer the most frequent ones in this article, as well as those that will be in the comments.

    What is happening in Russia with open data?

    To answer this question, two concepts must be separated from the very beginning. Open data - as an open form of presentation of any databases. Whether collected by activists, prepared by commercial companies or published by government agencies. Open data is a comprehensive term that encompasses all possible sources of data emergence, the main thing is that the data is accessible for free reuse and is machine-readable (plus compliance with 8 principles of open data that has already been written about here.

    But open government data- This is a subspecies of open data that is produced from within the state machine. This can be information about both the state itself and what state agencies collect in the performance of their functions.

    Based on these definitions, it can be said that what happens in Russia with open data as a whole is 100% dependent on ourselves, how we collect, publish, persuade, and so forth to publish data, this is what happens. There are many projects that exist independently from the state - this is the GIsLab community, these are our OpenGovData and Hubofdata projects, and almost all crowdfunding projects on MediaWiki can be attributed to open data initiatives because they provide an API for receiving data for everyone.

    But with regard to open government data , there is no way to do without the state itself. In 2013 was adoptedFederal Law of the Russian Federation of June 7, 2013 N 112-ФЗ “On Amending the Federal Law“ On Information, Information Technologies and the Protection of Information ”and the Federal Law“ On Providing Access to Information on the Activities of State and Local Government ” .

    This law contains a set of amendments to the federal law of February 9, 2009 N 8-ФЗ “On ensuring access to information on the activities of state bodies and local governments” and the federal law of July 27, 2006 N 149-ФЗ “On information, information technologies and information protection “that data should be published in open data formats.

    Further at the disposal of the Government of the Russian Federation of July 10, 2013 N 1187-rA list of the data that should be published first was presented.

    A complete list of laws, orders and decrees is compiled, for example, on the Moscow open data portal in the Documents section . There are many documents, both federal and those adopted by the Moscow City Hall.

    2. Who in Russia is responsible for open government data?

    One could say that the Government as a whole, but this is not entirely true. The approval of openness plans of departments, which include work on open data, is carried out by the Open Data Council. The Open Data Council is the name of the working group under the Government Commission for the Coordination of Open Government Activities.

    I am also a member of this council and I can say that despite the big name, the Council has practically no authority. And it cannot be in mind the Russian government structure in which all the councils are exclusively in a consultative form. In fact, the council’s only resource is the opportunity to convey one or another position to its chairman, Minister Abyzov.

    On the other hand, since the law on open data was amendments to 8-FZ, the methodological support of data disclosure and the creation of a federal open data portal is carried out by the Ministry of Economic Development of Russia .

    Here we must make a small digression. In 2012-2013, when the topic of open data in Russia was just taking the form of a state initiative, there was a departmental dispute between the Ministry of Communications and the Ministry of Economic Development about which department will oversee this topic. As a result, the Ministry of Economic Development won then, and in the future it was it that was the main executor of efforts to open data.

    The result of this was a lot of consequences, the main of which was that the emphasis on data disclosure shifted from state information systems as the Ministry of Communications wanted, and the official websites of government agencies were always asked by the Ministry of Economic Development.

    3. How much money do all of these initiatives have?

    The topic of open data is almost completely absent in all government programs, including the Information Society program. Over the past few years, only the Moscow Government has made significant investments in making open data accessible, which created the Moscow open data portal and even advertised it on the streets of the city.

    At the federal level, there was nothing like this. All members of the Open Data Board work without any compensation for their efforts.
    And the costs of creating a federal open data portal are low.

    In fact, all the council meetings that were held and attended by representatives of government agencies during this time came down to convincing government agencies to spend efforts on opening the data. In some cases, it brought results, and in some, all the conversations were divided into counter-requests "... and you help us to allocate a budget of XXX million rubles and we will open it all ...". The most striking example of such behavior was Rosstat, a similar situation was with the CEC of Russia and many other departments.

    Similarly with all subjects of the federation and municipalities. None of them received money to implement the practice of publishing open data; all efforts were limited to persuading them to spend their own money on these initiatives. Some subjects advanced in this, while some openly scored and simply sabotaged, revealing not their data, but their simulation.

    4. Does anyone in Russia use open data?

    In short, yes.
    Yes, the data is used more than actively. Activists and businesses use it, use it to commercial projects, public and personal, use it to improve existing products, and to create new ones.

    There are several areas and data sets that are more than in demand. First of all, these are such data as:
    - The base of state and municipal procurements;
    - Operational data of the Central Bank of the Russian Federation on currencies and open data on banks;
    - FIAS database (federal address information system);
    - Data on the GLONASS satellite constellation
    and much more.

    There are several dozens of projects for tracking new state orders (Purchases360, Bikotender), for analyzing contractors (Spark.Interfax, Kontur.Fokus, Kommersant.Kartoteka) that use the data directly. Aggregating data on companies, contracts, purchases, licenses and more. By mixing open data and those that they receive from commercial sources, they create commercial products. As a result, established commercial products exist and develop on the basis of open data.

    On the basis of the FIAS base and before it, on the basis of the KLADR base, a significant number of services are operating throughout the country. This database is usually used not to create new products, but to improve the convenience of address verification in numerous commercial information systems.

    Central Bank data is used by almost all sites showing official exchange rates and all sites for the analysis of banks and the banking system as a whole.

    In addition to these data, there are many others - much more highly specialized and which are also actively used, but most often their users do not advertise themselves and are little known.

    5. And yet, why are there so few examples of those who use data?

    The main reason is that the main data users are commercial companies that are completely not interested in either disclosing their income or telling about their business model. And indeed, most of them do not want to spend time on this. All of this is mixed with their fear that when they talk about how and how much they earn on certain data, they will immediately begin to have problems and they will have to pay for the data that they receive for free. Fear is not only unjustified, but also not unjustified.

    For this reason, all those examples that I call are examples of which I know and those examples that are in sight.

    6. And still there is very little useful data. Everything else is either meaningless data sets, or with a nasty description

    There is really little useful data. And I will say more, the most useful and sought-after data was published even before the term “open data” itself appeared as a state initiative.

    Central Bank of the Russian Federation - has been publishing data for over ten years as an API through SOAP services. The FIAS database appeared in the form of open data much earlier. And data on government contracts have been published in machine-readable form since 2008.

    Everything is very simple. There is the data that government agencies publish in response to a public or corporate request and the data that they publish on a loan in different order .

    The requirements that were formulated in the law on open data and government orders for most government agencies - this is another annoying fly. They don’t understand why to do this, they don’t understand who needs it, they don’t understand what is the use of opening data and they don’t understand what is the benefit of it. As a result, we get a lot of examples of “bad data” when data are published in sections on government websites without a description, without diagrams, with empty data files or other features. Most often, such data is published by representatives of the press service of the authorities and they do it all on the principle of "formal closure" , meaningless publication that allows you to check the results.

    7. Why is everything bad?

    This is not to say that everything is very bad. Russia does not have the last places in data openness ratings like Open Data Index and Open Data Barometer. But there are really few reasons to rejoice, and there are many reasons for everything going so hard.

    One of the reasons is that open data in Russia has very few clearly declaring their interest to consumers.
    The fact is that in the world the main consumers of open data are:
    - non-profit organizations and activists (civil society);
    - journalists;
    - commercial companies;
    - universities.

    We have our own problems with each of the consumers in Russia.
    Independent Nonprofit Organizations теперь почти нет. Все что жили на зарубежные гранты теперь, либо закрываются, либо перестраиваются под госгранты, либо резко сворачивают свою активность. Наша некоммерческая организация существует исключительно по то причине что все учредители это люди занимающиеся бизнесом и у нас была и есть возможность тратить немножко средств на её поддержание и в том что самые большие проекты мы делали при поддержке Фонда Кудрина. Единственного, пожалуй, который из фондов в России поддерживает общественные проекты на открытых данных. Других возможностей фактически нет. Государственные грантооператоры не выделяют средств на подобные проекты, коммерческих спонсоров для проектов на открытых данных найти очень сложно ну а брать иностранные гранты теперь табу.

    Журналистика почти вся в России весьма далека от нейтральной подачи информации. Но даже провластные издания не взаимодействуют с госорганами на предмет открытости их данных для того чтобы их использовать в работе. Увы, за все эти годы направление журналистики данных так и не обрело заметных очертаний в нашей стране и до сих пор журналисты оказывают очень малое влияние на открытости государства в тех или иных вопросах. Пронзительные экономические аналитики, исследователи демографии, словоохотливые социологи — все они на страницах СМИ не формируют запроса к органам власти на открытость.

    Коммерческие компанииeverything is simple here - there are very few of them. Commercial companies using open data can be divided into two categories. The first group is those who live off state orders and they need the data to show their competencies in order to receive more orders. And the second group living in the information market is extremely small and does not want to interact with government agencies without an urgent need. Mainly for reasons of deep distrust of the state machine.

    And finally, higher education institutions. Alas, there is simply no request from universities for the data to be available for research, for students to be able to use relevant data in their work. If in the world universities form their own databases on the areas of research and the work of their faculties, then in Russia this is, if any, in very limited forms.

    8. Will it be better?

    Before the imposition of sanctions, the law on foreign agents and other deterioration of foreign and domestic policy, I would say that definitely yes. When it all started Russia was in the G8, signed the G8 Open Data Chapter, claimed to participate in the Open Government Partnership. Now the topic of open data is developing with high resistance from the state apparatus. The most requested data are disclosed with great difficulty. Detailed educational, criminal and other municipal statistics are extremely difficult to access. State geodata is still hard to open. But much more is possible. These are climatic, transport and other data demanded in practice.

    9. Why am I personally involved in all this?

    There are many reasons. The main thing is that I got involved in open data back in 2009 with anger that this topic is developing all over the world, but in Russia there is nothing. The other is that the device of the “state”, not just the Russian one, is my old hobby. Despite the fact that the greatest efforts were made and it was necessary not only to convince officials of the openness of data, but to independently collect data from various sources and turn it into open data.
    - I’m attaching a couple of polls to the note and, as promised, I’m ready to answer all the substantive questions in the comments.

    Only registered users can participate in the survey. Please come in.

    Is it necessary to lobby for the openness of state data?

    • 84.3% Yes, you need 291
    • 7.5% Only if it does not take too much time and resources 26
    • 7.5% Better to spend time on public data projects 26
    • 0.5% None of this is necessary 2

    How do you rate the work of the Open Data Council?

    • 10.3% Positive 33
    • 2.5% negative 8
    • 31.5% I can’t understand what he is doing 101
    • 55.6% I have not heard or read anything about his work 178

    How do you assess the work of the Ministry of Economic Development of Russia according to open data?

    • 4.3% Positive 14
    • 11.2% Negative 36
    • 32.8% I can’t understand what he is doing 105
    • 51.5% I have not heard or read anything about his work. 165

    What is most important in the future of open data?

    • 53.4% More data 185
    • 89.5% Data quality and qualitative description of their structure 310
    • 28.3% Ability to communicate with responsible government bodies 98
    • 47.1% Possibility to request missing data 163
    • 62.1% API instead of 215 files

    Also popular now: