Unmanned cars. Intel expert answers



    In this post, we publish the answers of Intel expert Ivan Kuzmin to questions from the Readers on the topic “Artificial Intelligence and Autonomous Vehicles”. There were a lot of questions, as usual, and we left Ivan to choose those that he can and wants to answer. For various clarifications, we invite you to comment, but we do not guarantee quick answers.
    At the very end, according to tradition, we announce the name of the Habrauser, receiving a prize for the best question.

    We thank Ivan Kuzmin for the selfless work and time spent! Lingvo

    questions
    Is AI used in reality in unmanned vehicles that are actually being tested today, or is it all marketing tales and is the car autopilot really based on clear mathematical algorithms and logic?

    Answer: in the control of an unmanned vehicle, various mathematical algorithms are used. All algorithms will be very difficult to count, because the control task itself is complex and must both take into account various parameters and calculate data coming from different sources. The algorithms of artificial intelligence or machine learning can include both classical algorithms (trees, logistic or linear regression, the support vector method), and deep learning algorithms (convolutional and recurrent neural networks). The combination of various algorithms allows you to effectively reliably solve the control problem.

    How is the certification of this software for compliance with safety standards adopted for automotive software and is it done at all? Is the autopilot considered the same critical control unit of the car as ESP or airbags, where there is also hardware and software?

    Answer: Of course, the autopilot is as critical a unit as airbags or other devices. There is a standard ISO 26262 or its equivalent GOST R ISO 26262 in Russia, which imposes a requirement on the entire process of developing security-related systems. The developer of such systems at the end of development should show that he followed this standard.

    Does autopilot work in hard real time or is everything on Linux / Windows for now?

    Answer: In general, we have to meet less than 140 milliseconds from receiving signals or images to sensors or cameras to transmitting a signal to the machine’s control system. This problem can be solved in hard real-time mode, but also a number of car companies and partners are considering the option of developing the direction of the Linux version, which could meet all the requirements of the industry.

    Does Intel plan to bring its achievements to OpenSource and attract more developers, or will it keep everything closed and sell it to interested customers?

    Answer: the company has always supported and supported the ideas of open source and is actively involved in the development of a large number of OpenSource projects. At the same time, in the development of unmanned vehicles, an important issue is compliance with the standard, which may limit the participation of a large number of developers. In any case, we will weigh all the pros and cons for each individual product and make decisions individually. There is no general strategy in this regard yet.

    Are development tools used for modeling and automatic code generation (for example Matlab / Simulink), or is everything written with pens in C / VHDL? If so, what percentage of total software coverage?

    Answer: the automotive standard requires that the developer have a complete understanding of the code that is written. This limits the use of code generators. At the same time, models built in tools such as Matlab are quite used to verify the written algorithm.

    What kind of hardware is used to “quickly prototype” the developed algorithms in a real car — processors, FPGAs, ready-made boards, sensors / lidars / radars / parking sensors, actuators?

    Answer: if we are talking about rapid prototyping, then still it happens more likely not in the car, but in the data center. Here, Intel has many solutions both in hardware and in software, including our Intel Distribution for Python package, which significantly accelerates Python, as the most convenient language for rapid prototyping. When we talk about the car, here the language is C and C ++, and the platform itself, as a rule, is set in advance, taking into account the cost, power consumption and many other factors.

    Are there already industry standards for connecting the specified peripherals to a computing platform?

    Answer: yes, such standards already exist, and they are actively used in existing ADAS machines and solutions.

    questions kababok
    Does Intel refer to Tier-1-Supplier or Tier-2-Supplier?

    Answer: Intel and Mobileye work together with more than 50 different partners both directly and through suppliers. This list is constantly updated and the latest information about partners can always be found on our website .

    Are these Intel solutions compatible with AUTOSAR architecture (s)?

    Answer : Intel is one of the AUTOSAR partners and actively cooperates with this organization.
    Official releases reasonably indicated that Intel's autonomous driving solutions are optimized for Intel processors. Is it possible to use / port solutions from Intel to processors from other manufacturers: for example, Texas Instrument, Infineon, nVidia, Altera, etc.?

    Answer: in optimizing mathematical libraries, we not only rely on the compiler, but also actively use the instructions of the processor on which the code is executed. Theoretically, you can try to map instructions from IA to other architectures, but here it is important to remember that the standard imposes a requirement on the entire development process and it will be necessary to transfer not only the code, but also the tests, and it is also possible to revise the guarantees that are given. So, I think, porting the code will be extremely difficult.

    Have Intel solutions mentioned received Automotive certification? If so, what certificates?

    Answer: development is carried out in accordance with ISO 26262 and fully complies with it.

    Does it mean using Intel solutions based on only a separate ECU in the architecture of electronic components of a car or is it possible to deliver solutions in the form of separate libraries, which are then connected via dedicated interfaces to the software functions of other ECUs of other auto suppliers (Suppliers / Zulieferer).

    Answer: Intel has a large number of components (FPGA, CPU, 5G, Mobileye and so on) that can be used both separately and as a full-fledged platform that the company provides.

    Does Intel interact directly with OEM with its mentioned solutions?

    Answer: of course, we work with a large number of partners. All information about those with whom we work can be found on our website .

    Does Intel use any of the SiL / HiL / XiL stands in its own development processes?

    Answer: yes, in some cases

    In the automotive industry, at the level of software / hardware solutions for the development / commissioning / testing / acceptance and other parts of R&D processes, Vector Informatik and ETAS have long been de-facto leaders. Are Intel's negotiated solutions optimized for working with products from these companies?

    Answer: we work and interact with many leaders in this industry.

    Does Intel provide ready-made simulation blocks in Matlab / Simulink?

    Answer: at the moment, such blocks are not provided. Information on what is included in the Intel GO SDK can be found here .

    What processors, communication interfaces, operating system are used on the control unit in Audi A8?

    Answer: this car uses FPGA solutions and the VxWorks operating system.

    Is it possible to get a private solution from Intel in the form of an evaluation kit or a complete test kit from Intel GO Automotive SDK?

    Answer: You can request access via this link .

    Is Intel working on using autonomous / robotic control solutions in the aviation industry?

    Answer: Intel WindRiver is developing an operating system that is already in use on various aircraft, including the Boeing 787 Dreamliner or Airbus A400M. Some details can be found here .

    What are the specific features of the Python language used in Intel solutions? And for what - C ++ tools? What are the differences in the use of different languages ​​within the same product?

    Answer: Python is used at the stage of rapid prototyping of solutions in a data center. Directly in the car, it is proposed to use the C or C ++ languages

    Is ASIL D certification sufficient for the AURIX processor only — if other components fail to achieve this level?

    Answer: according to the requirement of the standard, we must reach the ASIL D level for the entire system as a whole; however, you can apply the decomposition of the general system into submodules, some of which may be lower than ASIL D.

    Questions Nick_mentat
    Are the possibilities of developing cars without machine vision systems on non-networks considered? If not, what kind of hardware will be used, and will the necessary computing power not overcharge the cars too much?

    Answer:There are various approaches to the creation of unmanned vehicles and 2 opposite camps can be distinguished. Some argue that you need to build an autopilot based solely on cameras and vision algorithms. Others believe that you can use radars and lidars, and get information about signs from maps or from low-resolution cameras. There are also many who are in the middle and actively use various means for orientation on the roads. We suggest connecting a processor to each camera, which will calculate the image. Its power can be limited, since in the process of calculation we do not need to recognize all the objects, but it is quite enough to classify them and assign the road, obstacle, sign, pedestrian and so on to some class. We also care about the final cost, together with car manufacturers,

    Is it possible to organize a sufficiently reliable communication system so that miner clusters of some cryptocurrency that does not yet exist, are engaged in settlements for cars currently traveling in real time around the world? If so, what kind of communication channel will it be, and what equipment will be needed to support it?

    Answer: we plan to use a 5G modem in the platform, which will allow us to download high-resolution cards in real time, interact with other machines, and also allow passengers to connect to high-speed Internet. The planned channel will be wide enough, but I’m not sure that the machine’s powers that could be used for some third-party calculations will remain unused.

    questions noonv
    Please tell me which architecture for building an unmanned vehicle control system do you consider to be the most promising? Fully based neural networks or modular - where do AI-based solutions work with classic algorithms?

    Answer: I would say that it is modular. The car uses many algorithms that are used collectively to achieve the final result. Of course, for pattern recognition, neural networks today have the best accuracy, but at the same time they require a lot of data for training, are more computationally expensive and not always stable. On the other hand, for the task of calculating the trajectory and its smoothing, classical algorithms can be completely dispensed with.

    What was the most unexpected problem you encountered when working on unmanned systems?

    Answer: probably, as such, there is no one problem, there is the total volume and complexity of the task, as well as the responsibility that we assume. We are talking about a fairly complex platform in which there are many different components that interact with each other. Understand and analyze these dependencies, make sure that they work as they should, and at some point say, yes it is ready, you can release it. For me, when I say that it is ready, I think not only about how many times and what tests we ran and how formally we met all the requirements, but also whether I am ready to put my family and children in an unmanned vehicle and remove my hands from steering, entrusting the management of what we did with the team.

    Question Rost99
    She is interested in how it is planned to collect / have collected data for training, and how much data for this can be considered minimally sufficient.

    Answer: each car that travels on the road during the day collects about 4 TB of data. We plan to release on the road several dozen cars that will collect this data during the year, which will later be used for training. So we are talking about the fact that there will be really a lot of data and probably it will be that sufficient level to confidently let the car onto the road. Is this enough? I think not, and additional training will continue to happen as machines become larger, algorithms will improve, rules will change, and so on. Asympro

    question
    How do you test your products, sensors and decision making?

    Answer: there are various testing methods. Some of them are imposed by the requirement directly by the standard: tests built on the basis of the requirement of functionality, checking the operability of interfaces, checking for resource consumption, checking for working with faulty data, and so on. In addition to all this, we accumulate a large amount of data taken from sensors and cameras during the movement of the car with the driver, the data are digitized, and the algorithm is checked in conditions similar to real ones. In addition, work is also underway on the use of virtual reality.

    questions sith
    What prevents (and what does not prevent) the most from releasing an affordable unmanned family car on public roads today?
    As options:
    • Lack of inexpensive / essential radars in the market.
    • Lack of inexpensive / necessary calculators.
    • Imperfections / lack of reliable algorithms for the computer for all situations.
    • Not good road conditions (poor marking, sudden obstacles on the road, etc.).
    • Not always clear road conditions sometimes even for a live driver.
    • Unpredictable actions of other road users.
    • Large energy losses on the computer (especially relevant for EV).
    • Lack of accuracy and accessibility (tunnels) of data from navigation systems (GPS)
    • Strange appearance (protruding radars).
    • Other inconvenience (trunk partially occupied by the computer).
    • The laws.

    Answer:there are many factors and virtually all of them have been listed. First of all, this, of course, is the ability to efficiently and accurately process data and calculate the trajectory of movement, while remaining within the framework of electricity consumption, which is acceptable for a car (in the long term electric). There are many different partial solutions that are actively used (row adjustment, adaptive cruise control, and so on). The creation of a full-fledged unmanned car requires a qualitative transition in terms of the types and number of algorithms used in the final solution, multiple duplication of their work with other algorithms, and so on. For example, we say that neural networks have an accuracy of 90+ percent for pattern recognition. This is a very good result, but to entrust management,

    Are radars really needed if a person can quite successfully control a car in general in any situation, even closing one eye and directing it with four buttons on the keyboard, watching the process on a flat screen with a very low frame rate and a black and white 640x480 picture?

    Answer: indeed, the question is very interesting. As I said, in answering another question, there are various trends. Some argue that one camera of high accuracy is enough and conduct research on algorithms that can restore image depth. Others, on the contrary, argue that radars and lidars can easily cope on their own and that cameras are not needed at all. I am inclined to think that at the very beginning of the path of unmanned vehicles that will drive in the next few years, we will see solutions that combine both of these components, yet the risk of error is too great and excessive duplication will not interfere. At the same time, I am sure that work on optimizing the solution and reducing the total number of elements will continue actively. Where science will take us, let's see in 30 years.

    QuestionsAreso
    Is it possible to produce such cars on roads with unpredictable traffic? Say, some parts of Russian roads, or a significant part of Southeast Asia?

    Answer: movement in conditions of heavy traffic will need to be tested separately. Work in this area, including already underway.

    How will such a car react to a rider moving in an aisle?

    Answer: like a motorcyclist riding aisle :) the car is not programmed for the specific behavior of other participants in the movement. She has her own strategy and adapts it depending on the behavior of other participants in the movement.

    Will the car violate the rules (traffic, parking) if the owner asks for it?

    Answer: it is not yet assumed that the driver can tell how to go by car. Choose one of the proposed route options yes, but do not say how fast to go or when you need to turn around. This is the meaning of unmanned vehicles, that such situations become unlikely.

    questions vassabi
    Can such a car move off-road (along the beach, in the field)?

    Answer: he probably could, but I think a little different algorithms and strategy are needed here. Now, first of all, they solve the control problem on public roads.

    How does a car determine its position on the road if there is no road marking? or if there is a conflicting marking (the old road was 3 lane, the new was two lanes, and sometimes the old marking is visible)?

    Answer: there are different approaches. The simplest: we can know how many lanes there should be based on signs or based on a map and build virtual markings, given the total width of the road. A person does something similar when there is no markup. By the way, the situation of the lack of markup is more typical than it seems to us. Many countries, and not only Russia, are located in latitude, where snow falls periodically and it is simply necessary to navigate in such a situation.

    Does such a car see hatches (sewer, stormwater)? can he distinguish an open hatch from a closed one?

    Answer: yes, he must see all this in order to plan the trajectory of movement correctly. Moreover, it is planned that the car will communicate with other cars through 5G and can, for example, find out what is happening behind the hill - that a person cannot see and know at all now. It is expected that cars will exchange information and also take it into account when moving.

    How does a car behave if there is a large puddle ahead? (for example, the right lane in heavy rain)

    Answer: just like a person, it slows down to and passes quietly for such a time so as not to lose control. In some cases, it can go around the adjacent lane, if it is free. It all depends on the situation.

    Does the atopilot see signs: on poles, to the left \ to the right of the road (detour direction), on the lane (accident warning, road works), barriers or tapes stretched across the road?

    Answer: the car sees all the signs and objects on which it has been programmed. In addition, he uses the data about the road and signs that come to him from the maps.

    Can an autopilot retrain a particular road by itself? (if traveling, driving along a small number of routes)

    Answer: so far the issue of retraining driving directly is not planned to be carried out directly in the car, but it is expected that the algorithms and solutions will be updated and they will periodically come into the car.

    Question Ar4er7
    Will the car respond to the atypical behavior of other autopilot cars? For example, to calculate the trajectory of avoiding a head-on collision?

    Answer: the strategy for avoiding a car from a collision is constantly being calculated. Goodzonchik

    Question
    A very interesting situation is when a car gets from one country to another, where there are multidirectional movements. From France (right-hand), the car passes through the ferry to England (left-hand). And if an autopilot learns, what will happen to him during such a transition, if for a person such a transition can be difficult and lead to very good consequences?

    Answer: the situation is really very interesting, maybe the car will have to download software for driving in another country, because in the end, each country is involved in the process of allowing the car on its roads. How exactly this will happen is difficult to say now. There is something to think about and fantasize

    Question akovalenko2368
    In principle, any autopilot object should have a navigation system and a complex of technical vision on board. If we hold the realities of our roads and roads in European countries, it will be difficult for me to imagine: how will autopilot manage outside large megacities?
    1. What navigation system is currently used by Intel, given that it is mandatory to use the GLONASS system in the Russian Federation?
    2. «Зрение» автопилота для «глубинок» предусматривает обучение в части объездов препятствий?
    3. С какой максимальной скоростью в настоящее время Вы смогли обеспечить пилотирование авто и предполагаемая (чисто гипотетически) стоимость «АвтоПилот» для установки вне завода)?

    Answer: The highway management task is actually simpler than the management task in a metropolis. When moving out of town, we usually make far fewer maneuvers.
    Any solution that will be created will be certified directly in the market in which it will be used.
    Yes, it provides for detouring both static obstacles and a reaction to unexpectedly appearing obstacles.
    The cost of a solution always depends on the volume, but a number of parts manufacturers plan to sell individual components. I can not say about the cost.

    Question Turisas
    How can an autopilot solve problems on a narrow road where 2 cars cannot leave? If people are behind the wheel, then they’ll figure someone closer to the “pocket” and he will hand over back. Can the autopilot solve such problems?

    Answer: two autopilots can inform each other in advance that they are approaching such a site and “agree” on how to leave.

    Question quverty
    After the recent news, I would like to know the prospects for using such an advanced hardware. Or are these too expensive “toys”? With quantum computers, the prospects are generally unclear. On the other hand, the same Volkswagen concludes some agreements with Google, and Honda with IBM.

    Answer: now this direction is at the very beginning of its development, but gradually it will become more accessible and may well begin to be used in a variety of devices.

    Question Pilat
    Who will be responsible for the accident if two AIs are involved, in the case of AI and a live driver? How will such an incident be dealt with?

    Answer: This question is open and resolved in each country separately. For example, in a number of Scandinavian countries, the question is being considered that the state will directly take responsibility, since it issues a permit to allow cars on the roads.

    Question Drakosha
    Will such an autopilot car drive independently on service matters (refueling, repair) without people / driver in the cabin?

    Answer: yes, I think it can, but if we are talking about electric cars, now work is underway to charge the batteries in a non-contact way, which can greatly simplify the whole process. SizovAlex

    Questions
    At what stage is the transition of the unmanned ground transport industry from laboratory "greenhouse" experiments to real practical applications?

    Answer: I think we will see solutions directly on the roads in the next 5 years

    Is there room for small groups of developers in this industry, or is advanced research now only within the reach of large players in this market?

    Answer: Recently, we have seen a large number of small companies that enter the market and offer some of their solutions. Many of these solutions are at the prototype stage, but they themselves are quite innovative. As for the final solution, which will work directly in the car, there is still room for only major players, primarily because of the cost of the final solution, in compliance with all requirements of the automobile standard.

    Does INTEL have its own prototypes of unmanned cars and will they be presented in the upcoming technology competition “Winter City”?

    Answer: samples are available, but now they are presented abroad. One of the latest developments could be seen at CES in Las Vegas, earlier this year. I myself did not go in it, but according to the stories of those who tried to ride, the sensations are amazing.

    The prize from an Intel expert goes to kababok , who asks 27 questions! It is immediately clear that the person is in the subject and digs deeply. Accept our small but pleasant present.

    Also popular now: