Why is CarPrice controlled by artificial intelligence?
A lot has been written and said about neural networks for the last three years. Thinking, we also decided to tell you how we use “artificial intelligence” in our daily work. Moreover, with many routine operations, he copes much better than people.
In car sales, all major operations are traditionally tied to people - emotional and to varying degrees reliable. Every year, CarPrice holds up to 150,000 auctions, which means that terabytes of statistics for each model of car, from its real state and up to price dynamics depending on the place of sale and the time of day, accumulate in the depths of the company. Is it possible, by analyzing arrays of information, to increase the conversion to sale? Can and should be!
At first we wanted to create a tool that will help the manager with the work. But in the process of testing, we were convinced that the neural network is completely tolerable without a person. But first things first.
So, below we will talk about several tools created on the basis of neural networks, which allow us to increase the efficiency of work. All of them work constantly, online.
Smart margin is one of the key tools to increase profitability. The system knows for how much we can sell each car, taking into account its age, mileage, equipment, damage, time of day, color, day of the week, and even the floor of the seller. There are a lot of such parameters, about 600.
Understanding how much dealers will give for a car and what amount is most likely to suit the seller, the neural network independently calculates the optimal size of the auction margin. Smart margin is set up to create conditions under which the probability of selling a car would be maximum. Sometimes, for a guaranteed sale, the neural network assigns the lowest possible margin, because the machine is highly liquid, in good condition and the seller will quickly sell it elsewhere. On another car, the margin will be higher, because it is unreliable and expensive to repair, which means there are more risks for CarPrice.
You can say something in the spirit of "just make the margin minimal, then sales will grow" and ... make a mistake. There are cars whose owners will not sell their car, even if we pay extra. There are cars whose owners are not at all sensitive to price - service and security of a deal are more important to them. Therefore, simply reducing margins in most cases means that we will receive less revenue. Again, the main task of this tool is to create conditions for the car to be sold. If, for example, if the margin is reduced by a certain percentage, the probability of selling a car increases by a factor of 2-3, then we will do it. As a result, due to a sharp increase in sales conversion, the company's revenues increase.
Here are some statistics. Before implementation, we performed A / B testing. Below is a rough margin chart. The black line is a test group with a smart margin. Green - control group, without smart margin. It can be seen that according to the recommendations of the neural network, the marginality is lower.
And this is a graph of the state of the purchased cars, which is reflected in our “stars”. It turns out that if all the factors are properly taken into account by the neural network, we buy back more good machines than without a neural network. Better car - less complaints.
Conversion chart. The test group with a smart margin is higher:
Higher and the average price of the purchased machine. That is higher and the volume of auction proceeds:
Finally, we compare the average returns for groups as a whole. With the application of smart margin, it is several tens of percent higher simply because the conversion is increasing. Due to the “smart” reduction of the margin on certain cars, we get a higher conversion to the sale, which, of course, dramatically increases the company's revenues.
Is the neural network wrong in determining the optimal margin? Today there is almost no, but at the testing stage errors got out constantly.
Suppose we hired an inspector to inspect cars. He worked for several months and spent several hundred auctions. The neural network analyzes the results of its work and finds out which type of car or customer it works best with. For example, one beautifully buys cars from girls with iPhones. And the other copes with the model range of Volkswagen. Someone special in "Japanese", and someone perfectly buys everything, but only on Monday or Friday.
Such patterns are monitored by the neural network. Budget cars or expensive, “Germans” or “Koreans” - whoever comes to us, the system knows which employee will provide the best conversion. Having registered on the website and having left data on the car, the neural network appoints the employee who will cope better than any other. As in the first case, a lot of parameters are taken into account, including the client’s phone model (if the entry was via the mobile version of the site).
After the introduction of smart compatibility, conversion at auctions, where the inspector was recommended, was 2-5 percentage points higher than at auctions without a recommendation. And the average margin of the auction is higher by 10-15%. This is a lot, especially considering that such an increase in efficiency does not incur any costs.
This is a smarter neural network. By signing up for a car sale, the client determines the address and time. As I said, we understand in advance what the probability is that the owner will sell the car through us. At the slot allocation stage, we give a higher priority time to such a customer / car pair, whose potential marginality or conversion will be higher.
How does it look in practice? If, according to the analysis, the probability of a client conversion is very high, then all slots are free for him when recording - I don’t want to choose. And if the owner of a car arrives with a set of characteristics that historically is poorly converted in our country, then only unclaimed slots will be available for selection. For example, late evening. Because if you give time to a customer with a low probability of conversion, the customer with a higher probability of conversion will not be able to sign up and sell the car. If, on a slot occupied by a not too liquid car, a liquid competitor appears, then we transfer the first car to a less popular watch with the resources of the call-center.
Here it is also important to take into account that not every client will eventually come to our office. For example, we are surprised that women come twice as obligatory as men. And people with iPhones reach CarPrice 30% better than people with phones on Android. We take this and more into account when giving a customer the opportunity to choose the best time.
Below is traditional statistics. We divided the cars into three groups according to the probability of their arrival, estimated by the neural network - green, yellow and red. As soon as this tool earned, the number of visits of "green" cars began to grow. As you can see, the system was not mistaken.
And this is the conversion of the arrival in redemption. It can be seen that the volume of "green" cars is also growing.
Our earnings in points with smart slots are now 27% higher than in points without them. And again, at no cost. Except for the costs of algorithms and programming, of course.
Every dealer who buys cars from us has certain preferences. Someone loves expensive models, someone buys only "Logans" and "Solaris" ... Dealers are browsing a lot of cars, and if you take into account their consumer preferences when forming the auction feed, you can dramatically increase the conversion. It seems to be obvious? However, everything is a little more complicated.
Dealer preferences are variable. Business and customer preferences are changing, so they can move from one segment to another. The neural network on clicks, transactions and transactions determines this and reconfigures the car ribbon. For example, the entire December, a dealer Ivanov from Vologda bought “tricks” for 300-500 thousand rubles. But suddenly, in January, he began to buy expensive SUVs with prices ranging from one and a half to two million. The tape immediately rebuilt, offering him the most relevant cars. In addition, the system itself sends notifications to it, sensitively reacting to the reaction.
Below are a few typical dealer profiles. Those who buy cheap cars, as a rule, never buy expensive cars. Why then do they show them?
This is the easiest filter. A neural network, while forming a personalized auction feed, simultaneously analyzes hundreds of similar attributes.
By forming the auction tape individually, we get higher auction rates. A dealer who, say, needs a three-year-old "logan" is more likely to fight for him and will most likely bet higher than others. Just by showing the buyers the cars that they are most interested in, we get an increase in conversion to buy-out and an increase in the average margin for the auction.
Of course, we are developing other neurotools, some of which today are in a state close to implementation. Why is this so important? First, the neural network allows us to earn more from the existing stream of customers. That is, in order to increase revenue, you do not need to increase marketing costs. Secondly, the neural network provides more satisfied customers - the more people sold cars through CarPrice, the higher the NPS. And in the long run, this is probably much more important than revenue.
For those who prefer the video format, we offer a presentation by Denis Dolmatov, the general director of CarPrice, dedicated to our neural networks.
And finally, about vacancies. Now we are looking for DevOps / Linux administrator in Moscowto the car auction team, as well as senior PHP developers to the internal services team. We welcome your resume.
In car sales, all major operations are traditionally tied to people - emotional and to varying degrees reliable. Every year, CarPrice holds up to 150,000 auctions, which means that terabytes of statistics for each model of car, from its real state and up to price dynamics depending on the place of sale and the time of day, accumulate in the depths of the company. Is it possible, by analyzing arrays of information, to increase the conversion to sale? Can and should be!
At first we wanted to create a tool that will help the manager with the work. But in the process of testing, we were convinced that the neural network is completely tolerable without a person. But first things first.
So, below we will talk about several tools created on the basis of neural networks, which allow us to increase the efficiency of work. All of them work constantly, online.
Smart Margin
Smart margin is one of the key tools to increase profitability. The system knows for how much we can sell each car, taking into account its age, mileage, equipment, damage, time of day, color, day of the week, and even the floor of the seller. There are a lot of such parameters, about 600.
Understanding how much dealers will give for a car and what amount is most likely to suit the seller, the neural network independently calculates the optimal size of the auction margin. Smart margin is set up to create conditions under which the probability of selling a car would be maximum. Sometimes, for a guaranteed sale, the neural network assigns the lowest possible margin, because the machine is highly liquid, in good condition and the seller will quickly sell it elsewhere. On another car, the margin will be higher, because it is unreliable and expensive to repair, which means there are more risks for CarPrice.
You can say something in the spirit of "just make the margin minimal, then sales will grow" and ... make a mistake. There are cars whose owners will not sell their car, even if we pay extra. There are cars whose owners are not at all sensitive to price - service and security of a deal are more important to them. Therefore, simply reducing margins in most cases means that we will receive less revenue. Again, the main task of this tool is to create conditions for the car to be sold. If, for example, if the margin is reduced by a certain percentage, the probability of selling a car increases by a factor of 2-3, then we will do it. As a result, due to a sharp increase in sales conversion, the company's revenues increase.
Here are some statistics. Before implementation, we performed A / B testing. Below is a rough margin chart. The black line is a test group with a smart margin. Green - control group, without smart margin. It can be seen that according to the recommendations of the neural network, the marginality is lower.
And this is a graph of the state of the purchased cars, which is reflected in our “stars”. It turns out that if all the factors are properly taken into account by the neural network, we buy back more good machines than without a neural network. Better car - less complaints.
Conversion chart. The test group with a smart margin is higher:
Higher and the average price of the purchased machine. That is higher and the volume of auction proceeds:
Finally, we compare the average returns for groups as a whole. With the application of smart margin, it is several tens of percent higher simply because the conversion is increasing. Due to the “smart” reduction of the margin on certain cars, we get a higher conversion to the sale, which, of course, dramatically increases the company's revenues.
Is the neural network wrong in determining the optimal margin? Today there is almost no, but at the testing stage errors got out constantly.
What is “under the hood” of a smart margin
При разработке модели смарт-маржи используется алгоритм машинного обучения MultiLayer Feedforward Perceptron. Нейросеть, полученная в результате применения этого алгоритма, в нашем случае выглядит следующим образом:
X1, X2,…,Xn - это набор входных данных, который нам известен:
1) о клиенте:
2) его машине:
3) о точке продаж CarPrice, куда приехал клиент:
4) о цене, которую на аукционе дают дилеры за данную машину.
В набор входных данных нейросети входит день недели и время запуска аукциона, а также процент маржи, зарабатываемый CarPrice.
На выходе (outputs) нейросеть выдает вероятность согласия клиента продать нам свое авто. В итоге задача сводится к максимизации критерия ожидаемой абсолютной маржи:
Смарт-маржа работает как отдельный WebAPI сервис, в который поступает набор входных данных, перечисленных выше. В качестве результата возвращается процент маржи, при котором ожидаемая абсолютная маржа достигает максимума.
При разработке модели смарт-маржи используется алгоритм машинного обучения MultiLayer Feedforward Perceptron. Нейросеть, полученная в результате применения этого алгоритма, в нашем случае выглядит следующим образом:
X1, X2,…,Xn - это набор входных данных, который нам известен:
1) о клиенте:
- пол;
- возраст;
- маркетинговый канал, откуда пришел клиент на сайт CarPrice (Offline, Calls, CPA, Context и пр.);
- из какого района города приехал клиент.
2) его машине:
- марка;
- модель;
- год выпуска;
- модификация;
- пробег;
- состояние авто (кузов, салон, техника).
3) о точке продаж CarPrice, куда приехал клиент:
- профессиональный опыт сотрудника CarPrice, который работает с клиентом;
- общие показатели точки продаж CarPrice, куда приехал клиент.
4) о цене, которую на аукционе дают дилеры за данную машину.
В набор входных данных нейросети входит день недели и время запуска аукциона, а также процент маржи, зарабатываемый CarPrice.
На выходе (outputs) нейросеть выдает вероятность согласия клиента продать нам свое авто. В итоге задача сводится к максимизации критерия ожидаемой абсолютной маржи:
<dealer price>*<margin>*<purchase probability> - dealer price — максимальная цена, которую дают за авто на аукционе дилеры
- margin — процент маржи, зарабатываемый CarPrice
- purchase probability — вероятность согласия клиента продать своё авто
Смарт-маржа работает как отдельный WebAPI сервис, в который поступает набор входных данных, перечисленных выше. В качестве результата возвращается процент маржи, при котором ожидаемая абсолютная маржа достигает максимума.
Smart compatibility
Suppose we hired an inspector to inspect cars. He worked for several months and spent several hundred auctions. The neural network analyzes the results of its work and finds out which type of car or customer it works best with. For example, one beautifully buys cars from girls with iPhones. And the other copes with the model range of Volkswagen. Someone special in "Japanese", and someone perfectly buys everything, but only on Monday or Friday.
Such patterns are monitored by the neural network. Budget cars or expensive, “Germans” or “Koreans” - whoever comes to us, the system knows which employee will provide the best conversion. Having registered on the website and having left data on the car, the neural network appoints the employee who will cope better than any other. As in the first case, a lot of parameters are taken into account, including the client’s phone model (if the entry was via the mobile version of the site).
After the introduction of smart compatibility, conversion at auctions, where the inspector was recommended, was 2-5 percentage points higher than at auctions without a recommendation. And the average margin of the auction is higher by 10-15%. This is a lot, especially considering that such an increase in efficiency does not incur any costs.
What is "under the hood" in smart compatibility
В процессе анализа данных нам удалось выявить различия в навыках менеджеров при выкупе машин. Этот инсайт лёг в основу нейросети, которая использует следующий набор входных параметров:
На выходе нейросети считается вероятность выкупа машины. Оптимизируемым критерием здесь является:
Для каждого клиента, приехавшего на точку продаж, нейросеть выбирает менеджера, который выкупит авто с наибольшей вероятностью.
- конверсия менеджера в разрезе ценовых диапазонов авто
- конверсия менеджера в разрезе цены – года выпуска авто
- конверсия менеджера в разрезе марок – моделей авто
- конверсия менеджера в разрезе пола/возраста клиента
- конверсия менеджера за последние 7 дней
- конверсия менеджера в разрезе маркетинговых каналов, откуда пришел клиент
На выходе нейросети считается вероятность выкупа машины. Оптимизируемым критерием здесь является:
<Probability to purchase> Для каждого клиента, приехавшего на точку продаж, нейросеть выбирает менеджера, который выкупит авто с наибольшей вероятностью.
Smart slotting
This is a smarter neural network. By signing up for a car sale, the client determines the address and time. As I said, we understand in advance what the probability is that the owner will sell the car through us. At the slot allocation stage, we give a higher priority time to such a customer / car pair, whose potential marginality or conversion will be higher.
How does it look in practice? If, according to the analysis, the probability of a client conversion is very high, then all slots are free for him when recording - I don’t want to choose. And if the owner of a car arrives with a set of characteristics that historically is poorly converted in our country, then only unclaimed slots will be available for selection. For example, late evening. Because if you give time to a customer with a low probability of conversion, the customer with a higher probability of conversion will not be able to sign up and sell the car. If, on a slot occupied by a not too liquid car, a liquid competitor appears, then we transfer the first car to a less popular watch with the resources of the call-center.
Here it is also important to take into account that not every client will eventually come to our office. For example, we are surprised that women come twice as obligatory as men. And people with iPhones reach CarPrice 30% better than people with phones on Android. We take this and more into account when giving a customer the opportunity to choose the best time.
Below is traditional statistics. We divided the cars into three groups according to the probability of their arrival, estimated by the neural network - green, yellow and red. As soon as this tool earned, the number of visits of "green" cars began to grow. As you can see, the system was not mistaken.
And this is the conversion of the arrival in redemption. It can be seen that the volume of "green" cars is also growing.
Our earnings in points with smart slots are now 27% higher than in points without them. And again, at no cost. Except for the costs of algorithms and programming, of course.
What is "under the hood" in smart slotirovaniya
Базовым алгоритмом нейросети здесь является тот же MLP, входными параметрами для которого являются:
По набору этих параметров нейросеть считает вероятность события выкупа авто у клиента или, другими словами, прогнозируемую сквозную конверсию из заявки в выкуп.
В зависимости от рассчитанной величины вероятности выкупа и ожидаемой маржи, которую заработает компания, клиенты разделяются на 3 группы по ценности. Критерий разделения на группы выглядит следующим образом:
Клиенты с самой высокой величиной этого критерия относятся к первой группе, с самой низкой – к третьей. Нам важно, чтобы было больше записей клиентов первой группы ценности, так как зарабатываем мы на них гораздо больше. Поэтому по мере формирования слотов мы даем больше опций по выбору удобного слота для первой группы, чуть меньше для второй и значительно меньше для третьей группы.
Для планирования заполненности слотов и во избежание очередей на точках продаж разработана предсказательная модель на основе дерева решений, которая вычисляет вероятность приезда клиента на точку. Вот как выглядит одно из правил расчета вероятности приезда клиента:
Здесь переменные cr_ — конверсии по параметрам клиента. Например cr_apcon2m_source_chan — это средняя конверсия клиентов, пришедших с того же маркетингового канала. При выполнении условий выше расчётная вероятность приезда клиента равна 0.14.
- марка/модель/год выпуска авто
- маркетинговый канал, с которого зашёл клиент на сайт CarPrice
- модель устройства, используемое клиентом для оценки авто на сайте
- день недели/часы суток когда клиент зашел на сайт
По набору этих параметров нейросеть считает вероятность события выкупа авто у клиента или, другими словами, прогнозируемую сквозную конверсию из заявки в выкуп.
В зависимости от рассчитанной величины вероятности выкупа и ожидаемой маржи, которую заработает компания, клиенты разделяются на 3 группы по ценности. Критерий разделения на группы выглядит следующим образом:
<ProbabilityAppointment To Purchase>*<Expected Margin>Клиенты с самой высокой величиной этого критерия относятся к первой группе, с самой низкой – к третьей. Нам важно, чтобы было больше записей клиентов первой группы ценности, так как зарабатываем мы на них гораздо больше. Поэтому по мере формирования слотов мы даем больше опций по выбору удобного слота для первой группы, чуть меньше для второй и значительно меньше для третьей группы.
Для планирования заполненности слотов и во избежание очередей на точках продаж разработана предсказательная модель на основе дерева решений, которая вычисляет вероятность приезда клиента на точку. Вот как выглядит одно из правил расчета вероятности приезда клиента:
cr_apcon2m_source_chan <= 0.5672744316784764 AND cr_apcon2m_weekday_conf > 0.5210736783538652 AND cr_apcon2m_hour_conf > 0.5068323664539807 AND cr_apcon2m_source_chan > 0.4755808440018966 AND cr_apcon2m_brand_model > 0.037602487984167376 AND cr_apcon2m_brand_model <= 0.1464285714285714 AND cr_apcon2m_hour_conf > 0.14705882352941177Здесь переменные cr_ — конверсии по параметрам клиента. Например cr_apcon2m_source_chan — это средняя конверсия клиентов, пришедших с того же маркетингового канала. При выполнении условий выше расчётная вероятность приезда клиента равна 0.14.
Smart tape
Every dealer who buys cars from us has certain preferences. Someone loves expensive models, someone buys only "Logans" and "Solaris" ... Dealers are browsing a lot of cars, and if you take into account their consumer preferences when forming the auction feed, you can dramatically increase the conversion. It seems to be obvious? However, everything is a little more complicated.
Dealer preferences are variable. Business and customer preferences are changing, so they can move from one segment to another. The neural network on clicks, transactions and transactions determines this and reconfigures the car ribbon. For example, the entire December, a dealer Ivanov from Vologda bought “tricks” for 300-500 thousand rubles. But suddenly, in January, he began to buy expensive SUVs with prices ranging from one and a half to two million. The tape immediately rebuilt, offering him the most relevant cars. In addition, the system itself sends notifications to it, sensitively reacting to the reaction.
Below are a few typical dealer profiles. Those who buy cheap cars, as a rule, never buy expensive cars. Why then do they show them?
This is the easiest filter. A neural network, while forming a personalized auction feed, simultaneously analyzes hundreds of similar attributes.
By forming the auction tape individually, we get higher auction rates. A dealer who, say, needs a three-year-old "logan" is more likely to fight for him and will most likely bet higher than others. Just by showing the buyers the cars that they are most interested in, we get an increase in conversion to buy-out and an increase in the average margin for the auction.
What is the result?
Of course, we are developing other neurotools, some of which today are in a state close to implementation. Why is this so important? First, the neural network allows us to earn more from the existing stream of customers. That is, in order to increase revenue, you do not need to increase marketing costs. Secondly, the neural network provides more satisfied customers - the more people sold cars through CarPrice, the higher the NPS. And in the long run, this is probably much more important than revenue.
For those who prefer the video format, we offer a presentation by Denis Dolmatov, the general director of CarPrice, dedicated to our neural networks.
And finally, about vacancies. Now we are looking for DevOps / Linux administrator in Moscowto the car auction team, as well as senior PHP developers to the internal services team. We welcome your resume.