May 15, 2018 at 11:00How we defeated chaos in the central warehouse
Hello, Habr! My name is Alexei Shikhov, I lead the CarPrice development team in Kirov. Now Carprice takes the second place in Russia in car sales on the secondary market, and almost all cars bought in Moscow go through one huge central hub, from where buyers (dealers) pick them up or send them to the regions by car carriers. There are always at least 700 cars in the warehouse that do not linger there for more than five days. In this article I will tell you how our team of friendly developers and caring testers fought the chaos that was inevitable for such an anthill and defeated it.
The central CarPrice hub is not just a guarded territory where we dump cars in anticipation of someone arriving at them. All cars that arrived at the warehouse undergo a technical inspection, they prepare documents and put them in order - they charge the batteries, add fuel, washing liquid, etc. And before the sale they organize additional examinations for dealers.
Dealers arrive unpredictably - the flow was very unstable. Warehouse employees either ran out sticking their tongues from car to car, or “smoked bamboo”, not knowing what to do with themselves. It happened that one dealer immediately took a dozen cars - it took a whole day, from early morning to late at night.
To streamline the work, we created a simple universal service for recording services (aka booking, aka electronic queue). In it, the receipt of the car is divided into two steps - inspection and issuance. The first includes preparation for the inspection and the inspection of the car by the dealer in the presence of the warehouse manager. Next, the dealer pays for the car and proceeds to the issuing step, at which documents are signed and keys are transferred. Inspection and issuance can take place on different days, some dealers are recorded immediately for issuance without inspection, and some inspect one car several times.
The recording service at first was not a complex system, it solved simple and understandable tasks - it calculated free slots depending on the type and duration of the service (stage), taking into account the work schedule of warehouse employees.
The dealer logs into your personal account, selects one of the cars won at the auction from the final warehouse, the type of service (inspection or delivery) and is recorded on a free slot.
This can be done both from the desktop and through the mobile application. If for some reason the dealer cannot register on his own, he can be recorded by a personal manager or warehouse employee.
Such schemes successfully work in many companies, and we hoped that as soon as we integrate the service, all problems would disappear.
After some time, we understand that the system does not work in practice. The reason is that most dealers are not very punctual: they arrive without an appointment, much later or earlier than the appointed time, and sometimes even on another day. At the same time, they are recorded for extradition, and come for inspection. They indicated the inspection of one car, and they demand to show five more others. Some dealers only want to work with a specific warehouse manager. However, many dealers are very emotional and are not able to put up with inconsistencies. With such unpredictability, the warehouse staff do not understand which of the crowd of furious dealers needs to be hired, which cars to prepare for inspection. We thought, thought and decided to introduce a new solution.
The idea was promising. Managers do not need to think about whom to hire, - clicked the “next” button, and go. Dealers understand how busy the warehouse is, see their position in the queue and know how much to wait. Based on the results of monitoring the service time, you can dynamically adjust the slots, and in case of SLA violation, notify management. And for inspection / issuance, you can register directly from the terminal in the warehouse.
The backend should implement the entire logic of the application, including the distribution of tasks between operators. Front components:
From a technical point of view, the dealer, making an appointment to inspect a car in his personal account, makes a request for a booking service on the backend, in response to receiving a list of possible cars for recording, free days and time. In addition to the main one, the system had to process other, "vital" cases:
There were several additional requirements:
The requirements did not seem transcendental, so I did not want to immerse myself in it and fill up the bumps. We thought that the electronic line-up is already a well-developed topic, and decided to find a contractor with a ready-made boxed solution who knows how to approach the given business cases.
The search for a contractor yielded nothing. Most turnkey solutions have a very primitive algorithm. Coupons are added to one general queue or categorized, and then operators alternately pull coupons out of the queue. All. There is no flexible configuration, there is no logic for distributing coupons between operators, and the system usually works only in Windows. I won’t even talk about easy integration with internal services. At the same time, the price tags for everyone are cosmic. Nevertheless, we started working with one contractor, but he had ten percent of the necessary functions, and everything else would have to be cut for a long time. So we broke up and decided to write an electronic queue on our own.
At this point, we already clearly understood our needs with all the nuances and immediately started implementation. The backend was made on the basis of the previously created booking service - using Laravel, PHP 7, RabbitMQ, Percona, Websocket, and everything in Docker. The entire front was implemented on web sockets, which made it possible to make realtime interfaces.
The terminal was assembled from a metal case in which the iPad is hiding. The case protects the tablet from theft and fixes it in the right position.
Guide access is activated on the tablet and in a full-screen browser without controls, a third-party application on Vue.js (SPA) is launched. First, the dealer is authorized in the terminal via SMS (JWT). After authorization, he can look at his records for inspection / issuance and receive coupons for them. Dealers have access to VIP services with a lock in their personal wallet.
Each record for inspection and delivery is pre-linked to a specific warehouse manager and, if possible, is automatically grouped with other records of the same dealer. When binding a manager, the schedule and workload of employees, the availability of other planned records at the dealer and other factors are taken into account.
When you make an appointment on the dealer’s account, money is blocked, and when you receive a coupon, they are returned. Entries for which a coupon was not received on time are automatically canceled, and money for a false entry is debited from the dealer’s account. From the received coupons, a queue is formed, which is distributed between the working managers, taking into account the recording time and dealer expectations, type of recording, etc.
There is a separate queue for VIP - the dealer can take the coupon for "right now" and immediately go to a meeting with the manager if there is no queue from the same VIP. The service, although paid, is in great demand among dealers who arrived without an appointment or were late and do not want to wait.
Another component of the system - a dashboard - shows which ticket is served by which operator. In addition, he displays the current queue and invites the dealer to proceed to the operator in a voice. In fact, this is a regular TV with a nettop connected, on which a full-screen browser is launched and the Vue.js (SPA) application is opened with the parameters of a specific warehouse.
The operator has a remote control for work - a laptop in the workplace.
The operator’s interface is open on the laptop, in which, after authorization, he can create, modify and cancel records, record the result of service, transfer customers and postpone work with them. Using this interface, if you have certain rights, you can configure the work schedule of managers and their possible services.
Through the same interface in each warehouse individually, you can configure SLA in detail. This makes it easy to scale our electronic queue to any number of new warehouses.
There are many different settings for SLA, we monitor the operation of warehouses according to many indicators.
The new electronic queue system helps to resolve various difficult cases. Here are a few for example:
If you carefully looked at the screenshots of the SLA settings above, you saw lines related to the messenger there. It used to be that warehouse employees instead of work left to smoke for an hour, and their colleagues had to switch to emergency mode. Now, if an employee goes to work and does not take dealers for a long time, the management receives a notification of the following type:
Or such warnings about dealers:
In addition to managers working with dealers, there are technicians in the warehouse who prepare the car for the scheduled inspection time: they dig it out of the snowdrift , fill in fuel, wash, charge the battery, warm up the engine and drive the car from the closed parking lot. A separate channel helps them learn about the features of starting the engine and complete training faster. Here are some interesting posts:
But this option is perhaps the most original:
With the help of the new electronic line, we defeated the chaos in the warehouse, simplified the life of dealers and gave them a number of additional services. If we list the business indicators, then the dealers' waiting time has decreased threefold, the number of negative feedback is quadrupled, and the warehouse with the same number of employees can now let through not 3000, but 3000 cars per day. While we are observing the system and in plans, except perhaps to put a printer that will print coupons on paper.
Now we are looking at our Moscow office for a QA specialist and backend developer . We welcome your feedback!
The central CarPrice hub is not just a guarded territory where we dump cars in anticipation of someone arriving at them. All cars that arrived at the warehouse undergo a technical inspection, they prepare documents and put them in order - they charge the batteries, add fuel, washing liquid, etc. And before the sale they organize additional examinations for dealers.
Dealers arrive unpredictably - the flow was very unstable. Warehouse employees either ran out sticking their tongues from car to car, or “smoked bamboo”, not knowing what to do with themselves. It happened that one dealer immediately took a dozen cars - it took a whole day, from early morning to late at night.
Booking system
To streamline the work, we created a simple universal service for recording services (aka booking, aka electronic queue). In it, the receipt of the car is divided into two steps - inspection and issuance. The first includes preparation for the inspection and the inspection of the car by the dealer in the presence of the warehouse manager. Next, the dealer pays for the car and proceeds to the issuing step, at which documents are signed and keys are transferred. Inspection and issuance can take place on different days, some dealers are recorded immediately for issuance without inspection, and some inspect one car several times.
The recording service at first was not a complex system, it solved simple and understandable tasks - it calculated free slots depending on the type and duration of the service (stage), taking into account the work schedule of warehouse employees.
The dealer logs into your personal account, selects one of the cars won at the auction from the final warehouse, the type of service (inspection or delivery) and is recorded on a free slot.
This can be done both from the desktop and through the mobile application. If for some reason the dealer cannot register on his own, he can be recorded by a personal manager or warehouse employee.
Such schemes successfully work in many companies, and we hoped that as soon as we integrate the service, all problems would disappear.
Problems do not disappear
After some time, we understand that the system does not work in practice. The reason is that most dealers are not very punctual: they arrive without an appointment, much later or earlier than the appointed time, and sometimes even on another day. At the same time, they are recorded for extradition, and come for inspection. They indicated the inspection of one car, and they demand to show five more others. Some dealers only want to work with a specific warehouse manager. However, many dealers are very emotional and are not able to put up with inconsistencies. With such unpredictability, the warehouse staff do not understand which of the crowd of furious dealers needs to be hired, which cars to prepare for inspection. We thought, thought and decided to introduce a new solution.
Electronic queue
The idea was promising. Managers do not need to think about whom to hire, - clicked the “next” button, and go. Dealers understand how busy the warehouse is, see their position in the queue and know how much to wait. Based on the results of monitoring the service time, you can dynamically adjust the slots, and in case of SLA violation, notify management. And for inspection / issuance, you can register directly from the terminal in the warehouse.
The backend should implement the entire logic of the application, including the distribution of tasks between operators. Front components:
- A terminal with authorization by phone number, recording services, viewing available cars, connecting paid services and issuing recording coupons
- Dashboard with visualization of the queue and current coupons at work
- Operator console (warehouse manager)
From a technical point of view, the dealer, making an appointment to inspect a car in his personal account, makes a request for a booking service on the backend, in response to receiving a list of possible cars for recording, free days and time. In addition to the main one, the system had to process other, "vital" cases:
- Dealer signed up, arrived / not arrived
- The dealer signed up, arrived on time, put into the slot 30 minutes
- The dealer signed up, arrived later / earlier at 15, 40, 100 min
- Dealer is registered / arrived for VIP-issue
- The dealer signed up for the slot in 30 minutes - in fact it turned out 10 minutes / 2 hours
- The dealer signed up for the inspection, arrived, examined, went to pay, after 40 minutes he returned to pick up the car
There were several additional requirements:
- Flexible adjustment of the warehouse staff schedule indicating the services that anyone can provide. And all this in the conditions of overlapping work schedules of employees - we have two shifts, from 9 to 18 and from 13 to 22
- Quick connection to the new warehouse system
- Integration with other services - current booking, dealer authorization, etc.
- Easy vertical and horizontal scaling
- Realtime interfaces
The requirements did not seem transcendental, so I did not want to immerse myself in it and fill up the bumps. We thought that the electronic line-up is already a well-developed topic, and decided to find a contractor with a ready-made boxed solution who knows how to approach the given business cases.
We develop ourselves
The search for a contractor yielded nothing. Most turnkey solutions have a very primitive algorithm. Coupons are added to one general queue or categorized, and then operators alternately pull coupons out of the queue. All. There is no flexible configuration, there is no logic for distributing coupons between operators, and the system usually works only in Windows. I won’t even talk about easy integration with internal services. At the same time, the price tags for everyone are cosmic. Nevertheless, we started working with one contractor, but he had ten percent of the necessary functions, and everything else would have to be cut for a long time. So we broke up and decided to write an electronic queue on our own.
At this point, we already clearly understood our needs with all the nuances and immediately started implementation. The backend was made on the basis of the previously created booking service - using Laravel, PHP 7, RabbitMQ, Percona, Websocket, and everything in Docker. The entire front was implemented on web sockets, which made it possible to make realtime interfaces.
The terminal was assembled from a metal case in which the iPad is hiding. The case protects the tablet from theft and fixes it in the right position.
Guide access is activated on the tablet and in a full-screen browser without controls, a third-party application on Vue.js (SPA) is launched. First, the dealer is authorized in the terminal via SMS (JWT). After authorization, he can look at his records for inspection / issuance and receive coupons for them. Dealers have access to VIP services with a lock in their personal wallet.
Each record for inspection and delivery is pre-linked to a specific warehouse manager and, if possible, is automatically grouped with other records of the same dealer. When binding a manager, the schedule and workload of employees, the availability of other planned records at the dealer and other factors are taken into account.
When you make an appointment on the dealer’s account, money is blocked, and when you receive a coupon, they are returned. Entries for which a coupon was not received on time are automatically canceled, and money for a false entry is debited from the dealer’s account. From the received coupons, a queue is formed, which is distributed between the working managers, taking into account the recording time and dealer expectations, type of recording, etc.
There is a separate queue for VIP - the dealer can take the coupon for "right now" and immediately go to a meeting with the manager if there is no queue from the same VIP. The service, although paid, is in great demand among dealers who arrived without an appointment or were late and do not want to wait.
Another component of the system - a dashboard - shows which ticket is served by which operator. In addition, he displays the current queue and invites the dealer to proceed to the operator in a voice. In fact, this is a regular TV with a nettop connected, on which a full-screen browser is launched and the Vue.js (SPA) application is opened with the parameters of a specific warehouse.
The operator has a remote control for work - a laptop in the workplace.
The operator’s interface is open on the laptop, in which, after authorization, he can create, modify and cancel records, record the result of service, transfer customers and postpone work with them. Using this interface, if you have certain rights, you can configure the work schedule of managers and their possible services.
Through the same interface in each warehouse individually, you can configure SLA in detail. This makes it easy to scale our electronic queue to any number of new warehouses.
There are many different settings for SLA, we monitor the operation of warehouses according to many indicators.
The new electronic queue system helps to resolve various difficult cases. Here are a few for example:
- The dealer signed up for several cars, was very late and half of them became overdue. It turns out that the record is partially expired, so the dealer still sees it in the terminal and can receive a ticket on it. The manager in the operator’s console sees that some of the services are expired, can end work with the coupon without them, or delete some of the active ones and add expired services instead.
- The dealer came to inspect and issue the car, but the car showed defects. Now the dealer does not need extradition, but the formation of a claim for a refund. In the operator’s console, the manager can do this with one button - change the type of service from issuing to claim.
- The dealer came to inspect one car, but wants to inspect another. The manager in the operator’s console can replace the machine for inspection in the ticket.
Monitoring SLA via Telegram
If you carefully looked at the screenshots of the SLA settings above, you saw lines related to the messenger there. It used to be that warehouse employees instead of work left to smoke for an hour, and their colleagues had to switch to emergency mode. Now, if an employee goes to work and does not take dealers for a long time, the management receives a notification of the following type:
Or such warnings about dealers:
In addition to managers working with dealers, there are technicians in the warehouse who prepare the car for the scheduled inspection time: they dig it out of the snowdrift , fill in fuel, wash, charge the battery, warm up the engine and drive the car from the closed parking lot. A separate channel helps them learn about the features of starting the engine and complete training faster. Here are some interesting posts:
But this option is perhaps the most original:
Summary
With the help of the new electronic line, we defeated the chaos in the warehouse, simplified the life of dealers and gave them a number of additional services. If we list the business indicators, then the dealers' waiting time has decreased threefold, the number of negative feedback is quadrupled, and the warehouse with the same number of employees can now let through not 3000, but 3000 cars per day. While we are observing the system and in plans, except perhaps to put a printer that will print coupons on paper.
Now we are looking at our Moscow office for a QA specialist and backend developer . We welcome your feedback!