May 25, 2017 at 07:07What is hidden inside the car auction platform
Services for the sale of used cars are usually built on the same principle: "posted an ad, called, showed a car, sold it." In such projects, high technology is concentrated mainly around the site itself. But with us everything is different. We have created a platform on which anyone can sell their used car (even an emergency one) to car dealers through the auction system, with a check of the technical condition of our specialists. Behind the apparent simplicity of the description is a complex ecosystem of self-written and third-party software products that we want to talk about.
Hello! My name is Ilya Pyatin. I am responsible for IT at CarPrice and today I want to give you a general outline of the insides of our platform and what we make of it.
Our story will follow this pattern, so from time to time we will repeat it so that you do not have to often return to the beginning:
So, a person wants to sell his car. He visits our public site (there are closed parts of the system, more about them later), fills out a short form and signs up for a meeting at one of our branches (more than 45 of them are in 21 cities of Russia).
A calculator is also available on the public website , with the help of which you can roughly estimate how much you can earn per car at CarPrice auction. Prices are calculated using an analytical algorithm based on our existing sales statistics for specific brands, models, technical condition, demand in various regions of the country and so on.
After entering, the data goes to our call center. They call up the owner of the car, clarify additional information and warn what documents you need to bring with you.
When a person arrives at the selected branch, he is met there by our employee - a specialist in vehicle inspection. It checks documents using our internal system called AutoCheck. Information from the TCP, STS and passports is entered there, and AutoCheck checks information on all available public service bases and collects data on various prohibitions, restrictions, fines and other legal problems that may arise when selling this car.
If problems are found, then we report them to the owner of the car and explain how they can be solved. If we are only talking about unpaid fines, then we can pay them for it. If everything is in order, then our colleague - the inspector - with the help of our developed mobile application, which we call the “Inspector application”, enters all the information about the car into the database. This is done according to a certain algorithm: he selects the make, model, year of manufacture, takes photos of the body, interior, indicates damage, equipment and so on. The inspection algorithm is the same for all cars.
When the inspector finishes the inspection, he clicks the “OK” button in the application, and all the information goes over the network to our online platform.
The main features of the application:
When the inspector has finished work, the data collected by him turns into a lot at the auction: they are aggregated and displayed in a readable form. And dealers who may be interested in this machine receive a notification. By the way, a neural network is partially responsible for notifying dealers, analyzing their behavior and past purchases.
In fact, dealers do not see the car itself. They only have access to the card in the auction system, with the result of the inspection and photographs taken by our inspector. At this time, the car is in one of our offices.
One more explanation: when the inspection results get into our system, in order not to dump hundreds of points for dealers, the data are aggregated and reduced to a point system - the machine receives “stars”. Only four criteria, for each of which there can be up to 5 stars:
Why did we decide to introduce "stars"? The fact is that the checklist of the inspection application consists of more than 1000 items. Each item has its own weight. In addition, the list is regularly reviewed: something is added, something is removed. Therefore, the "stars" are vital for dealers to quickly get an idea about the condition of the car, as well as to formulate a guarantee policy for this car.
After the card enters the auction system, the owner of the car waits 30 minutes. He can watch all the current auctions in real time on a separate TV - dashboard, he sees how dealers are betting on his car, what cities they are from, how much time is left until the end of the auction. Most watch the auction with great excitement. If the time is almost over and a new bid has arrived, the platform extends the auction for a while. And such extensions can occur several times, so that the owner can sell his car more profitably, if there is a demand for it.
The main part of the platform is written in PHP, but we did a number of the most critical components for speed and load on Golang: accepting bets, robots trading for dealers, and so on. The notification system, one of the components that are very important to us, is implemented on Slanger . This is an open source protocol-based notification system from Pusher with intermediate storage in Redis.
From the auction platform, data is sent to the logistic, statistical and marketing systems (used, inter alia, for A / B testing). A separate push interface was created for the sales department to monitor the progress of auctions.
Dealers can enter the auction either through a browser or through a mobile application. Many of them do not sit in the office, but constantly go to meetings, watch cars, so the application is more relevant for them. If the company is large, then, on the contrary, it often sets aside an individual employee who sits and purposefully buys cars at CarPrice that meet certain criteria he has voiced.
By choosing a card of a car of interest, the dealer can look through the photos, see the package, assess damage:
In the card made on the React + Redux bundle, a timer is displayed showing how much is left until the end of trading on this lot, and you can bet. If you participated in online auctions, for example, on eBay, then you can well imagine how in the last seconds the fight between dealers for tidbits flared up.
By the way, we implemented on Go a service of “auto deliveries” for dealers. In fact, these are robots to which you can say: "Haggle for this car up to such an amount," and go see other lots. As soon as your bet is outbid, the bot will automatically place a bet higher.
The dealer does not just need to win the auction, he needs to offer the price at which the owner agrees to sell his car. After all, if one of the dealers wins the auction with the highest offer, but it does not suit the owner, he can simply refuse and leave. By the way, for the car owner, the entire procedure for evaluating a car and putting it up for auction is completely free, regardless of its final decision.
But the dealer is in a favorable position: he has not paid anything yet, but at the same time receives the goods with our technical expertise, with high-quality photographs, no need to go somewhere and communicate with someone. By the way, now we are introducing an algorithm that automatically determines the quality level of the photograph that the inspector took when inspecting the vehicles taken to inform him of the need to retake it if the picture turned out to be "not very".
Key features of the dealer application:
Suppose that one of the dealers won in a hot fight, won the auction, and the proposed amount suited the owner of the car. In this case, we immediately give him the full amount, draw up documents, and happy, now the former owner is leaving to celebrate the deal.
The car is temporarily with us - we have already paid for it and now we want to give it to the winner. For example, at the moment we own more than 1000 cars. Naturally, such a geographically distributed fleet with constant "fluidity" requires strict accounting. We did not find a suitable third-party solution and wrote our own warehouse system, with which all branches work. It also allows you to track all cars moved between branches, plan freight for car transporters, and so on.
So, the dealer won the auction on some lot, and the car arrived at the warehouse of the corresponding city. Here he goes through the inspection procedure - QA, you all know the quality assurance. Only in this case we are not talking about checking a new feature in the product, but about re-evaluating the condition of the car. Was the acceptance assessment accurate enough? Are there any missing defects?
QA employees also have their own mobile application with a checklist, in which the results of the first acceptance inspection are uploaded. Since the cars are inspected not in greenhouse conditions with Wi-Fi connection, but in the warehouse-parking lots, and often "far from civilization", we adapted the application to work in low-level communications. This is a good LTE or 4G coverage in cities with over one million people, and even then not all. And in smaller cities, and even on the outskirts, and Edge may be rare. Before the inspection, the employee uploads data on the machines in the office, which he will inspect - and this is up to 500 MB of data and photos for each car, and goes to the fields. There he makes corrections to the information, and when a connection appears, automatic synchronization with the platform is performed.
Key features of QA applications:
Each dealer has a personal account with an online reservation system. It can be booked separately for inspection and for the issuance of cars.
Why separately? Dealers are businessmen. Before paying, they want to first look at the product, make sure that it matches the description on the lot card. After all, cars are not cheap, the risks are quite high. The dealer inspects the car with our employee, and the results of the inspection are recorded in the QA application. If the dealer has no complaints, the document management system invoices him for payment. After that, he pays for it and can make an appointment to collect his purchase.
This concludes our core business chain.
It happens that a car arrived at the auction, which for some reason didn’t attract the attention of dealers within half an hour or the amount offered was too low, and we understand that the owner will not agree to it. But at the same time, we know that in our client base there are those whom the car may interest, or who are willing to offer more for it, just now they could not participate. In this case, we call our “domestic dealer”, ask him to take risks and redeem the car, offering the owner a larger amount than dealers. After completing the documents, we put the car at the so-called long auction, which does not have a limit of 30 minutes. By the way, many dealers prefer just such lots, because there is no need to rush, you can safely evaluate the car, come to the parking lot, see, think.
How do we know how much to offer the owner? Why do we think that a car will be bought from us later? Our specialists are responsible for this, focusing on an analytical system that looks at the history of bidding for similar cars, analyzes demand and the customer base, and, based on many criteria, selects dealers who have not yet seen this lot, but who may be interested in it. Today, we have about 30,000 dealers in our client base, and not all of them use smartphones and computers: there are many who still go with push-button phones. Call them regularly, talking about interesting lots - no call centers will be enough. Nevertheless, these are successful businessmen who can regularly send digests with potentially lots for them from a long auction.
We also introduced an automatic recommendation system. The neural network analyzes all lots, and not just those for which there are no bids or too few are offered. This allows you to specifically call for auction those dealers who are most interested in a particular machine.
Based on many hundreds of criteria describing a car, a neural network identifies those cars that may be of interest to a particular dealer. Moreover, based on the history of dealers' behavior, the neural network can highlight preferences that the dealer himself has not formulated for himself. For example, she might guess that the dealer is not interested in cars that have more than two owners of a TCP or a non-leather interior. And another dealer often looks right-hand drive "Japanese", they need to show him in the first place.
To summarize the above: lengthy auctions help offer the car owner a fair price, and then find a dealer who is willing to pay a decent amount for it.
Perhaps you have a long question, but why dealers buy used cars, competing with each other at auction and raising the price? Everything is very simple: we only carry out a technical assessment of cars, but do not carry out any technical work. Dealers then do all this: repair, replace, clean, paint, and straighten. In general, they increase the value of a car and prepare it for it to find a new owner, and then sell it at a premium. Thus, with our help, they save at the stage of selection and purchase.
Already released a personal online account for car owners on React / Redux. When a person decides to sell a car and registers with CarPrice, then his personal account displays useful information about what documents need to be prepared, how to check for unpaid fines, how best to prepare a car for sale, how to get to the nearest CarPrice office and much more.
The results of all systems are aggregated into the BI system for further in-depth analytics. As the company grows, analytics begins to play an increasingly important role in a wide variety of business processes and decision making, so now we have a whole department dedicated to the analysis of our own data.
The stack of the main technologies used in CarPrice is as follows:
In addition to the above applications and systems, we have created many other IT products, as well as integrated third-party solutions that help us to develop faster: Huginn , Zapier , Exponea .
All data exchange between services and systems is carried out directly through internal APIs protected by jwt tokens. All key services run on RabbitMQ queues.
The load on services is regulated by nginx-balancers, and, of course, all key systems are protected by an anti-DDoS umbrella.
We have a lot of interesting things, and we plan to slowly talk about the solutions used. It will be great if in the comments you mention what you should talk about first.
Thanks!
Hello! My name is Ilya Pyatin. I am responsible for IT at CarPrice and today I want to give you a general outline of the insides of our platform and what we make of it.
Our story will follow this pattern, so from time to time we will repeat it so that you do not have to often return to the beginning:
Car reception
So, a person wants to sell his car. He visits our public site (there are closed parts of the system, more about them later), fills out a short form and signs up for a meeting at one of our branches (more than 45 of them are in 21 cities of Russia).
Technical Reference
Visible to everyone, the Bitrix client site in the current microservice architecture is, in fact, one of the “fronts”. Information entered by car owners is transferred to internal storage, analytics system, call center, marketing systems and so on.
A calculator is also available on the public website , with the help of which you can roughly estimate how much you can earn per car at CarPrice auction. Prices are calculated using an analytical algorithm based on our existing sales statistics for specific brands, models, technical condition, demand in various regions of the country and so on.
After entering, the data goes to our call center. They call up the owner of the car, clarify additional information and warn what documents you need to bring with you.
Technical Reference
The call center is built on the Oktell / MSSQL platform and works in our private cloud. It is integrated with CSI, an advanced user polling system, client CRM, and various mailing and notification services. Primary information about the car owner, the legal status of the car, and so on, is also recorded there.
When a person arrives at the selected branch, he is met there by our employee - a specialist in vehicle inspection. It checks documents using our internal system called AutoCheck. Information from the TCP, STS and passports is entered there, and AutoCheck checks information on all available public service bases and collects data on various prohibitions, restrictions, fines and other legal problems that may arise when selling this car.
If problems are found, then we report them to the owner of the car and explain how they can be solved. If we are only talking about unpaid fines, then we can pay them for it. If everything is in order, then our colleague - the inspector - with the help of our developed mobile application, which we call the “Inspector application”, enters all the information about the car into the database. This is done according to a certain algorithm: he selects the make, model, year of manufacture, takes photos of the body, interior, indicates damage, equipment and so on. The inspection algorithm is the same for all cars.
When the inspector finishes the inspection, he clicks the “OK” button in the application, and all the information goes over the network to our online platform.
The main features of the application:
- A universal script is downloaded from the server, which affects the sequence of actions of the inspector inspecting the car.
- When viewed, the application can work completely offline, as our employees work in very different conditions, sometimes on the client’s territory.
- Checking restrictions on registration and other actions through the traffic police, FSSP, Autocode, etc.
- Built-in system for smart compression and multithreaded transfer of photo and video to the cloud storage.
- Own computer diagnostics through the OBD-2 scanner.
- Work with mobile printers.
- Electronic index of car damage.
Technical Reference
The application architecture is built on the MVVM pattern using RxJava, Dagger , Retrofit 2 , Realm , Amazon SDK and so on.
Put up for sale
When the inspector has finished work, the data collected by him turns into a lot at the auction: they are aggregated and displayed in a readable form. And dealers who may be interested in this machine receive a notification. By the way, a neural network is partially responsible for notifying dealers, analyzing their behavior and past purchases.
In fact, dealers do not see the car itself. They only have access to the card in the auction system, with the result of the inspection and photographs taken by our inspector. At this time, the car is in one of our offices.
One more explanation: when the inspection results get into our system, in order not to dump hundreds of points for dealers, the data are aggregated and reduced to a point system - the machine receives “stars”. Only four criteria, for each of which there can be up to 5 stars:
- body
- technical condition
- salon
- related factors
Why did we decide to introduce "stars"? The fact is that the checklist of the inspection application consists of more than 1000 items. Each item has its own weight. In addition, the list is regularly reviewed: something is added, something is removed. Therefore, the "stars" are vital for dealers to quickly get an idea about the condition of the car, as well as to formulate a guarantee policy for this car.
After the card enters the auction system, the owner of the car waits 30 minutes. He can watch all the current auctions in real time on a separate TV - dashboard, he sees how dealers are betting on his car, what cities they are from, how much time is left until the end of the auction. Most watch the auction with great excitement. If the time is almost over and a new bid has arrived, the platform extends the auction for a while. And such extensions can occur several times, so that the owner can sell his car more profitably, if there is a demand for it.
Technical Reference
Dashboards are SPAs on Vue.js and push notifications. In fact, this is a page that receives partial information about all bids and in a compact form displays data from current auctions for the branch in which the client is located.
The main part of the platform is written in PHP, but we did a number of the most critical components for speed and load on Golang: accepting bets, robots trading for dealers, and so on. The notification system, one of the components that are very important to us, is implemented on Slanger . This is an open source protocol-based notification system from Pusher with intermediate storage in Redis.
From the auction platform, data is sent to the logistic, statistical and marketing systems (used, inter alia, for A / B testing). A separate push interface was created for the sales department to monitor the progress of auctions.
Dealers and Auction
Dealers can enter the auction either through a browser or through a mobile application. Many of them do not sit in the office, but constantly go to meetings, watch cars, so the application is more relevant for them. If the company is large, then, on the contrary, it often sets aside an individual employee who sits and purposefully buys cars at CarPrice that meet certain criteria he has voiced.
By choosing a card of a car of interest, the dealer can look through the photos, see the package, assess damage:
In the card made on the React + Redux bundle, a timer is displayed showing how much is left until the end of trading on this lot, and you can bet. If you participated in online auctions, for example, on eBay, then you can well imagine how in the last seconds the fight between dealers for tidbits flared up.
By the way, we implemented on Go a service of “auto deliveries” for dealers. In fact, these are robots to which you can say: "Haggle for this car up to such an amount," and go see other lots. As soon as your bet is outbid, the bot will automatically place a bet higher.
The dealer does not just need to win the auction, he needs to offer the price at which the owner agrees to sell his car. After all, if one of the dealers wins the auction with the highest offer, but it does not suit the owner, he can simply refuse and leave. By the way, for the car owner, the entire procedure for evaluating a car and putting it up for auction is completely free, regardless of its final decision.
But the dealer is in a favorable position: he has not paid anything yet, but at the same time receives the goods with our technical expertise, with high-quality photographs, no need to go somewhere and communicate with someone. By the way, now we are introducing an algorithm that automatically determines the quality level of the photograph that the inspector took when inspecting the vehicles taken to inform him of the need to retake it if the picture turned out to be "not very".
Key features of the dealer application:
- The ability to bid anywhere.
- A list of expected auctions allows you to plan the time of dealers.
- Internal and push notifications of new auctions.
- Built-in participation fee.
- Displays all the damage that was indicated when inspecting the car on an interactive map.
Warehouses and logistics
Suppose that one of the dealers won in a hot fight, won the auction, and the proposed amount suited the owner of the car. In this case, we immediately give him the full amount, draw up documents, and happy, now the former owner is leaving to celebrate the deal.
The car is temporarily with us - we have already paid for it and now we want to give it to the winner. For example, at the moment we own more than 1000 cars. Naturally, such a geographically distributed fleet with constant "fluidity" requires strict accounting. We did not find a suitable third-party solution and wrote our own warehouse system, with which all branches work. It also allows you to track all cars moved between branches, plan freight for car transporters, and so on.
Technical Reference
This is Laravel / MySQL / Vue's own service with tons of external integrations.
QA
So, the dealer won the auction on some lot, and the car arrived at the warehouse of the corresponding city. Here he goes through the inspection procedure - QA, you all know the quality assurance. Only in this case we are not talking about checking a new feature in the product, but about re-evaluating the condition of the car. Was the acceptance assessment accurate enough? Are there any missing defects?
QA employees also have their own mobile application with a checklist, in which the results of the first acceptance inspection are uploaded. Since the cars are inspected not in greenhouse conditions with Wi-Fi connection, but in the warehouse-parking lots, and often "far from civilization", we adapted the application to work in low-level communications. This is a good LTE or 4G coverage in cities with over one million people, and even then not all. And in smaller cities, and even on the outskirts, and Edge may be rare. Before the inspection, the employee uploads data on the machines in the office, which he will inspect - and this is up to 500 MB of data and photos for each car, and goes to the fields. There he makes corrections to the information, and when a connection appears, automatic synchronization with the platform is performed.
Key features of QA applications:
- Workday planning for warehouse staff.
- The current list of dealers who want to pick up their cars.
- Current list of cars in stock.
- Inspection of the vehicle according to the inspection certificate uploaded with the dealer.
- Fixing unaccounted damage.
- Launching internal business processes.
Technical Reference
The application is made on MVP with Retrofit 2, EventBus, Realm, Amazon SDK, JobQuery and so on.
Issuing cars to dealers
Each dealer has a personal account with an online reservation system. It can be booked separately for inspection and for the issuance of cars.
Why separately? Dealers are businessmen. Before paying, they want to first look at the product, make sure that it matches the description on the lot card. After all, cars are not cheap, the risks are quite high. The dealer inspects the car with our employee, and the results of the inspection are recorded in the QA application. If the dealer has no complaints, the document management system invoices him for payment. After that, he pays for it and can make an appointment to collect his purchase.
This concludes our core business chain.
Technical Reference
The workflow system is a separate module for 1C: Bitrix. It generates packages of documents for each transaction, taking into account all kinds of conditions.
Long auction
It happens that a car arrived at the auction, which for some reason didn’t attract the attention of dealers within half an hour or the amount offered was too low, and we understand that the owner will not agree to it. But at the same time, we know that in our client base there are those whom the car may interest, or who are willing to offer more for it, just now they could not participate. In this case, we call our “domestic dealer”, ask him to take risks and redeem the car, offering the owner a larger amount than dealers. After completing the documents, we put the car at the so-called long auction, which does not have a limit of 30 minutes. By the way, many dealers prefer just such lots, because there is no need to rush, you can safely evaluate the car, come to the parking lot, see, think.
How do we know how much to offer the owner? Why do we think that a car will be bought from us later? Our specialists are responsible for this, focusing on an analytical system that looks at the history of bidding for similar cars, analyzes demand and the customer base, and, based on many criteria, selects dealers who have not yet seen this lot, but who may be interested in it. Today, we have about 30,000 dealers in our client base, and not all of them use smartphones and computers: there are many who still go with push-button phones. Call them regularly, talking about interesting lots - no call centers will be enough. Nevertheless, these are successful businessmen who can regularly send digests with potentially lots for them from a long auction.
We also introduced an automatic recommendation system. The neural network analyzes all lots, and not just those for which there are no bids or too few are offered. This allows you to specifically call for auction those dealers who are most interested in a particular machine.
Technical Reference
The neural network was first modeled in MATLAB, and then ported to Tensorflow, there are places on NumPy . All this works in the form of a separate microservice, which collects information about specific lots and selects from the database of dealers those who may be interested in these lots. She analyzes the past behavior of dealers: who, when and what lots looked, for which cars he traded, won or did not win, how many cars are bought now, are they waiting for any payment bills.
Based on many hundreds of criteria describing a car, a neural network identifies those cars that may be of interest to a particular dealer. Moreover, based on the history of dealers' behavior, the neural network can highlight preferences that the dealer himself has not formulated for himself. For example, she might guess that the dealer is not interested in cars that have more than two owners of a TCP or a non-leather interior. And another dealer often looks right-hand drive "Japanese", they need to show him in the first place.
To summarize the above: lengthy auctions help offer the car owner a fair price, and then find a dealer who is willing to pay a decent amount for it.
Why all this to dealers?
Perhaps you have a long question, but why dealers buy used cars, competing with each other at auction and raising the price? Everything is very simple: we only carry out a technical assessment of cars, but do not carry out any technical work. Dealers then do all this: repair, replace, clean, paint, and straighten. In general, they increase the value of a car and prepare it for it to find a new owner, and then sell it at a premium. Thus, with our help, they save at the stage of selection and purchase.
Other developments
Already released a personal online account for car owners on React / Redux. When a person decides to sell a car and registers with CarPrice, then his personal account displays useful information about what documents need to be prepared, how to check for unpaid fines, how best to prepare a car for sale, how to get to the nearest CarPrice office and much more.
The results of all systems are aggregated into the BI system for further in-depth analytics. As the company grows, analytics begins to play an increasingly important role in a wide variety of business processes and decision making, so now we have a whole department dedicated to the analysis of our own data.
Technology
The stack of the main technologies used in CarPrice is as follows:
In addition to the above applications and systems, we have created many other IT products, as well as integrated third-party solutions that help us to develop faster: Huginn , Zapier , Exponea .
All data exchange between services and systems is carried out directly through internal APIs protected by jwt tokens. All key services run on RabbitMQ queues.
The load on services is regulated by nginx-balancers, and, of course, all key systems are protected by an anti-DDoS umbrella.
We have a lot of interesting things, and we plan to slowly talk about the solutions used. It will be great if in the comments you mention what you should talk about first.
Thanks!
