Artificial intelligence in the real world

Original author: Microsoft blog editor
  • Transfer
The development of complex artificial intelligence systems in the laboratory is a difficult task. But what happens when such systems get into the real world and start interacting with real people? This is what researchers are trying to figure out, including Dr. Fernando Diaz, senior research manager at Microsoft Research Montreal. Today, Dr. Diaz will share his thoughts and answer our questions about artificial intelligence and its impact on society.

Together with colleagues, Fernando is trying to understand how AI systems affect society, getting into the real world, and how to cope with the consequences.

He will also talk about how technology can shape musical tastes, and explain why it is today so important to teach students who are educated in computer science, not only algorithms, but also ethical principles.


When I run an experiment, I ask myself: which experiments on users are considered ethical, and which ones are not? How widely do you need to inform people that they have become participants in the experiment? How to recognize and eliminate the bias of data used by machine learning technologies? This comes to mind first. In the following years, many other questions will arise, such as how to develop AI systems in such a way as to show respect for users.

You are listening to the Microsoft Research podcast. Here we introduce you to cutting-edge technology research and the scientists behind it. With you lead Gretchen Huizinga (Gretchen Huizinga).

The development of complex artificial intelligence systems in the laboratory is a difficult task. But what happens when such systems get into the real world and start interacting with real people? This is exactly what researchers like Dr. Fernando Diaz, senior research manager at Microsoft Research Montreal, are trying to figure out. Together with colleagues, Fernando is trying to understand how AI systems affect society, getting into the real world, and how to cope with the consequences.

Today, Dr. Diaz will share his thoughts and answer our questions about artificial intelligence and its impact on society. He will also talk about how technology can shape musical tastes, and explain why it is today so important to teach students who are educated in computer science, not only algorithms, but also ethical principles. This and much more is in the new release of the Microsoft Research podcast.

Fernando Diaz, welcome to the podcast.


You are a senior research manager at Microsoft Research Montreal and are working in the field of artificial intelligence, search, and information. But you also learn the principles of honesty, accountability, transparency, and ethics (Fairness, Accountability, Transparency and Ethics, FATE). That is, if we talk in general (we will move on to the specifics later): what makes you wake up in the morning? What serious questions are you looking for answers to, what important problems do you want to solve?

Many of the systems we build are highly successful. Information retrieval, web search, computer vision - all these technologies have been developed over the years. Today they are actively filling the mass market, and people are starting to use them every day. However, some IT professionals have not thought through how to design these technologies: in what social context they will be used.

And in this case, I'm just trying to understand what social prerequisites existed for the creation of these systems, how the social context in which they operate affects not only our indicators, for example, the accuracy and completeness of the returned data, but also the society as a whole. . It seems to me that this issue comes to the fore for IT professionals, since many of these technologies, developed in isolation, are only now beginning to enter the market.

So, you are an IT specialist, you have investigated the algorithms for obtaining information, machine learning and statistical methods. Recently, however, you are interested in the interaction of artificial intelligence technologies with society, in particular the consequences of their wide distribution or, as some say, their release. Why are you interested right now? What worries you? What spurred your interest in this area?

Great question. At first I, of course, enrolled in the magistracy, received a degree. I studied all these systems, so to speak, on an abstract level, experimenting with static data obtained offline. Shortly after graduation, I went to the industrial research laboratory. Here I worked together with production workers, we were engaged in the practical implementation of technologies that I studied at the university.

And then I began to understand: when we scale these algorithms and provide them to real users, most of the basic assumptions put forward in the laboratory are completely inapplicable in reality. For me, it was a kind of final test of all my research, a return to the basic principles and an attempt to understand what the problem is, how I can accurately assess the results and achieve specific indicators.

That is, you have already worked in Microsoft Research, then you left, and then returned again. You started in New York, and now you have moved to Montreal. Why are you back?

After university, I began research work in Montreal and for a number of reasons I was forced to leave there. But living there, I realized that in this city - as in Canada as a whole - the tradition of research in the field of IT and machine learning is quite strong. And in my heart I really always wanted to come back here to participate in this work. And when I had the opportunity to return to the Microsoft Research lab in Montreal, I gladly took it. Especially when you consider that the laboratory is fully engaged in developments in the field of artificial intelligence. Montreal has been very active in this area, and I wanted to be a part of it all, to make my own contribution.

Let's say a few words about Montreal. This city has become a real Mecca in all that relates to artificial intelligence, and the laboratory of SMR Montreal has a very specific task - to teach machines to read, think and communicate in the same way as people do. Tell us how far you have come along and how your own research correlates with the work of the Montreal laboratory.

I think a special AI research lab was created because a lot of questions arose regarding the development of such systems, and they haven’t yet been answered. And I think that this requires not only specialists in the processing of natural languages, not only experts in dialogic training or in stimulating training. In fact, they should all work closely together. And it seems to me that this is what makes our laboratory truly unique.

My task today is to come to the laboratory, talk to specialists as much as possible and tell them how these systems can behave when real people start interacting with them. As I said earlier, such systems are quite easy to develop in isolation from reality, in isolation. But when their practical implementation begins, it turns out that too many assumptions were made during the experiments. Now I am forming a team whose task is to anticipate the emergence of such questions, optimize the development of the system, increase its stability in conditions, say, differences between the population groups with whom we interact, or variations within the knowledge base from which I draw information.

What team do you want to form?

I'm trying to create a kind of “sister” of the FATE group, which we organized in New York a few years ago. We will focus on the social consequences of integrating artificial intelligence into society. Our team will include specialists not only in IT, but also in related disciplines, such as sociology. For IT professionals to better understand the implications for society, we need sociological experts, anthropologists, and so on. They will be able to tell a lot of useful things about such things that we have not yet evaluated or taken into account.

Yes, let's talk about this in more detail. The application of FATE principles in conducting various studies in the field of artificial intelligence and machine learning today is of paramount importance. As you have said, the reason is that not all controversial issues can be properly studied in the laboratory. Together with the planned result, completely unexpected, shocking consequences may arise. So, the researchers of this community have different specialization and education. What is the contribution of each specialist when it comes to the principles of honesty, accountability, transparency and ethics?

Yes, sociologists have a much better understanding of various aspects of the application of technology in general, they know about the possible positive and negative consequences, about how people react to one or another of the tools we offer. Specialists with a law degree will be able to comment on the political background of individual technologies being developed and will help us to better understand the concept of “honesty,” for example.

IT professionals, in turn, better understand the essence of the systems being developed, are able to translate concepts such as “honesty” into a viable concept and incorporate it into the system. However, the presence of very different points of view on the same problem is simply necessary in order to design systems more efficiently.

Let's go back to what you did in the past and what you continue to work on now: information access systems, search engines, and information retrieval. In the document you have compiled, you speak about the existence of a certain gap between the study of such systems and their practical implementation, but at the same time you make a provocative statement that employees of educational institutions will cope better with some problems than industry technical specialists. What are these problems and why do you think so?

Let's look at the situation in the field of access to information research. There are scientific employees of educational institutions who have done a lot for the public good, but today many studies, say in the field of web search, are carried out by large search giants who have data, information about users, and so on. And in most cases, researchers do not have access to such data, a platform for conducting experiments. Therefore, the research possibilities are clearly unequal.

And in my article I wrote that research officers of educational institutions do not have a large amount of data, but they have the opportunity to attract diverse specialists, which the search giants cannot do. The university has sociologists and experts in other sciences. There are teachers of economic disciplines. All these potential “accomplices” of research will help to look at the problem more broadly, to study it from different points of view, instead of resting on one single one, which some search giant sticks to.

I think that generating data sets is just one of the strategies. Another approach, or type of scientific platform, not available for educational institutions, is experiments. I can conduct A / B tests. You can set up controlled experiments involving a large sample of users that is not available in the data set.

Yes it's true.

And yet, it seems to me, it is worth exploring how, in fact, we provide educational institutions with access to their resources for conducting such controlled experiments.


All this was happening chaotically, haphazardly, and it seems to me that this is what we, industry researchers, need to think about: how to make access to such features more simple and convenient.

So, let's go back to the data. Let's say a few words about them. Specialists who were engaged in machine learning, agreed that it was not enough to have just “big data” - I specifically say “big data” in quotes. Among other things, high-quality and objective data are required. We know that all these big data sets lack objectivity to one degree or another.

And we need to fix it. Today, there is a lot of talk about how to increase the objectivity of data through, for example, search engine audits, equality algorithms and the like. How to do it?

One of the reasons for our concern about data bias: a model trained on the basis of this data will be biased when deployed. That is the most important thing - to be able to determine how objectively artificial intelligence acts. And if he acts biased, you need to go back and retrain the algorithm or add restrictions to it that will not allow you to adopt bias from the data. My work today is mainly focused on evaluation and measurement.

We want to understand the users accessing the system, understand what they need, and evaluate whether the system is objectively or non-objectively taking into account who these users are and what population group they belong to. This requires a wealth of experience, accumulated by specialists in obtaining information, who, since the beginning of such research in the 50s of the 20th century, had time to think through all the estimation and measurement algorithms. This is what makes it possible to find a natural balance between auditing, evaluating and obtaining information.

As we have said, non-objectivity is something like a buzzword among researchers in the field of data processing and analysis and artificial intelligence. However, you say that in addition to bias there are other social problems that need to be addressed. What are these problems, and how can research help solve them?

Yes, I really think non-objectivity is a very important issue, but why did I even start talking about the social context of using artificial intelligence? Because I believe that bias is just one of the social problems that we can identify. Of course there are others. Obviously, one of them is related to transparency. How can I make the decisions made by the algorithm transparent for users, make them feel that they can control the situation, participate in the work of the algorithm?

The second problem is the cultural background of the algorithms. This all happens in the context of, say, the selection of recommendations of films or music. For example, I create a great system for selecting music recommendations for users. What will be the consequences of the deployment of this algorithm for culture, if I know, for example, that by adding individual performers to a recommendation, you can form someone’s musical tastes in a certain way? What will it mean to create or preserve culture in the long run?

There is another aspect of this problem: the algorithms for selecting music recommendations can have a significant impact on the authors or performers themselves. As an IT specialist, I can say that this is the best algorithm for selecting music recommendations. And I'm going to bring it to the market. But we, IT professionals, did not even think about how this algorithm will affect the authors. For me personally, this is especially important.

How, then, are you going to do research that will take all this into account?

Let's go back to the example with a selection of music recommendations. Imagine that you communicate closely with musicians and understand how important this is for them. How will they know that they are excluded from the system? How does it feel that their life is governed by a system of selection of recommendations, and they absolutely can not affect it? As an IT specialist, I just need to sit at the table with sociologists and anthropologists, experts in mass media, in order to better understand what such a significant group of the population is like musicians.

And then I, an IT specialist, will be able to sit down and think about how to create an algorithm that will satisfy the needs of both listeners and musicians. Now it seems to me that such a formulation sounds too simplistic. That is why I want a specialist in other disciplines to tell me: "Fernando, you know, you haven't thought about this, that, and more about that."

Given the nature of your research and the results obtained, can you say that we have something to worry about? Are there things that keep you awake?

Personally, I am concerned about the fact that many of the technologies developed in our research community go out into the world, go into mass circulation in a matter of days or weeks. And this is done by people who themselves have not conducted any experiments. There is nothing terrible in “open science”, but I think that we should learn more and understand better the consequences of the use of algorithms before introducing them somewhere. And it also worries me that we are releasing more and more new algorithms very quickly, not fully appreciating and understanding their consequences.

Laboratory Microsoft Research is famous for its close cooperation with educational institutions. And I know that you pay great attention to education. Let's talk about what happens with education in terms of the principles of FATE - honesty, accountability, transparency and ethics. Tell us how you see the future of educational programs in this area.

You know, when I entered the magistracy ... or even earlier, while studying at the IT department, we were practically not taught the principles of ethics and did not talk about the implications for society of the technologies that we are developing. In part, this is probably due to the fact that students study operating systems or ways of obtaining information at a certain abstract level. They do not think about the real context in which these technologies will be deployed. Let's be honest, students have a hassle with basic algorithms.

In my opinion, when I was studying, and even today, the question of the consequences for society and social responsibility, which engineers and computer technology specialists should bear, did not even arise. I think now people are coming to understand that this is important. It seems to me that today IT departments are starting to organize their work taking into account ethics, and students are also beginning to learn this. However, I am still concerned about the fact that a huge number of people who have never studied this are still developing systems. In addition, to establish a company, you know, you do not need a doctoral degree in IT.

And this side of the question - the existence of a proper education - really bothers me. However, in terms of education, I went through a good school in the industry research laboratory and can share my experience with students in the classroom so that they better understand not only the practical side of large-scale deployment of machine learning systems, but also social and ethical implications.

I have already raised these questions before: “When I launch an experiment, I ask myself: which experiments on users are considered ethical, and which ones are not? How widely do you need to inform people that they have become participants in the experiment? How to recognize and eliminate the bias of data used by machine learning technologies? ”. This comes to mind first. In the next few years, many more questions will arise, such as how to develop systems in such a way as to demonstrate due respect for users.

You know, you can probably compare it with the famous Hippocratic Oath "do no harm." Systems that are created to help people must ensure that people are not harmed. In the first place, of course, we relate this principle to physical harm. But it is all the more gratifying to understand that today we are starting to think about this when it comes to large-scale deployment of artificial intelligence systems. Although there is no analogue to the management of drug and food control for artificial intelligence, we are still moving in the right direction.

Yes. And I think it is important to understand that this has already happened in other areas, as you have already said, and we are not the first to go through this. And we can benefit from the support of medical ethics specialists or sociologists, they help us realize what it means to be a responsible IT specialist. That is, we do not need to start from scratch. We have someone to ask for assistance from, specialists from other disciplines will help us solve this problem.

So, at the end of our conversation, I will ask: what should the next generation of researchers in your field of knowledge expect? I know that it is impossible to give a short simple answer to this question, but at least you can advise our listeners, many of whom are also engaged in research. Perhaps they are also interested in the consequences of using AI technologies for society?

I think for those who are interested in this area ... many of the studies conducted today are just the first step on a very long way. This is incredibly interesting, because we are starting to ask serious, very important questions. And it seems to me that in the future we will have to look for answers to other, equally important questions, and today's newcomers, young and energetic researchers will just be engaged in this.

It seems to me that this study will acquire enormous significance for society, since today many such systems are being actively used. And the opportunity to participate, for example, in creating a search engine, to correct all the shortcomings that we talked about today, and to correct the consequences for society is simply an incredibly powerful tool of influence. Great time for young researchers.

For more on Dr. Fernando Diaz’s biography and his research on honesty, accountability, transparency and ethics (FATE) in IT, see the website .

A minute of advertising for colleagues from Voronezh

October 12 in Voronezh, the guys from the AI-Community will organize a workshop called " Digital Plant ". We are speaking on it as a methodological partner.

At the workshop, teams will be formed to generate modern solutions to real business cases that will remotely and effectively manage production processes and entire enterprises, monitor the physical condition of employees at the facilities, train and coordinate their actions “in the field” using virtual and augmented reality systems .

Come or tell friends from Voronezh, participants will receive a certificate!

check in

Also popular now: