How I tried to fix the search on the cards for drivers
This is a story about how I tried to solve a strange problem that prevented me from doing it myself. Looking ahead, I will say - I am satisfied with the resulting decision and brought the application to its logical end. However, in order to run it fully, you need more resources, so I decided to take a pause and ask people if it was needed by someone else. To this end (and also just to speak out) and write here.
Two words about myself: I live in Dublin, Ireland, I work as a programmer. It’s not exactly sitting in place, so in my free time I saw different projects at home, mostly on the table. I write on Habré for the first time, although I have been reading for many years.
Quite a long time ago, when at work I had to travel a lot to unfamiliar places, I began to notice that the standard search on any maps today is absolutely inapplicable to drivers. Look: you are driving behind the wheel in an unfamiliar area, and you have a gasoline arrow at zero. Your actions? If at this moment I am not alone in the car, then I tell the passenger: “Well, look near the gas station while I am on my way”. Because if you do it yourself, then you need to perform the following actions:
In my opinion, a nightmare. First, you have to stop, or at least wait for the traffic light. Because the maps are complex, and the icons are small. Secondly, the map does not tell you what the next gas station is. In this regard, Google is the worst: even in the results in the form of a list, he stubbornly pushes up not the nearest place, but I have no idea what is the most rated / with photos / paid /.
Well, the third factor: we are moving. The result, which the cards were issued, was relevant for a certain point, but we are already far from it. Have you noticed that even the route laid out by Google is not automatically updated if we have shifted? Only if the navigation has already been launched - then it will rebuild.
In general, the problem is clear. Logic, which works for pedestrians and their needs, for drivers about nothing. I do not care about the rating of the place and what kind of kitchen there is - I need, without being distracted from the road, in real time to get a route to the nearest gas station, charging, parking, ATM and so on.
Let us now try to determine the ideal search script. The following criteria:
The first thing that suggests itself is to replace the standard one-time search with a scan. That is, a chain of actions is approximately as follows:
You define the search criteria in advance - in fact, this is the type of location and the scan radius. Further, while the car moves, the application works as a radar. It quickly became clear that, firstly, the radius was not round, but in the form of an isoline. Secondly, it should be built not by distance, but by time, because minutes are much easier for perception than kilometers.
Then I thought about the method of issuing results. More precisely, do I need a map at all? The driver looks at the application with one eye and interacts with one finger - he doesn’t need a map, but large text and large buttons. Therefore, I immediately decided that the main screen would be a list, and I might add a map at first and see if I left it.
Knowing my peculiarity to stretch projects, I decided to take 2 months for myself - as a result, I got into 3. In principle, the application is quite simple:
Sounds like a lung easier. I already had some previous experience in cartography (a couple of years ago I made a real estate portal where the search was on the map), so the stack on the backend was immediately clear:
On the client, on reflection, I decided to use the new Google framework - Flutter. It is cross-platform, quite flexible, and allows you to make full-fledged applications with a minimum of code. Of course, it is raw and incomprehensible in production, but it looks perfect for prototyping. It is necessary to clarify that by this time I had the experience of native development for android (was the team leader) and decided, so to speak, to look the enemy in the face. The enemy was not so terrible.
The first prototype of the application was ready very quickly - Flutter has a low threshold of entry and an understandable redux-like philosophy. The declarative description of the interfaces, oddly enough, also liked, as well as the hot reboot (React Native, your card is a bit). In general, it seemed that the Google users took into account most of the congenital diseases of previous attempts. However, I understand people who may not have it all - someone doesn’t like darts, a limited number of widgets, and the “visual debug” that is being offered here is something very raw.
On the backend, I did the following:
I did the update of the results by shifting the device coordinates by 5 meters. The map is static - I just used Google’s google maps (as you can see, this is the only place where the corporation crawled into our cozy world of sorts). The first implementation looked like this:
 Having
played with the application, I decided to hide the map. She did a good job of understanding which search site was being made, and she looked entertaining - it was interesting to see how this octopus changes shape in real time. However, this entertainment did not help the application to perform its functions and occupied a third of the screen.
It also occurred to me to add an arrow, which indicated the direction to each result. It worked like this:
This very simple trick makes it much easier to understand whether we are already heading towards a place or need to turn around.

By this time I was quite pleased with the resulting application and decided to try to deploy it to several countries. Still, Ireland is a very small state, respectively, and the elastic index and the graph of roads were small. For the test, I decided to connect the neighboring UK. It is about 4 times longer and has a much denser road grid (especially the capital and major cities). And there was a problem.
Elasticsearch expectedly digested the increased index well, but with OpenTripPlanner there was a complete failure. It is written in Java and, as I said above, generates a graph of roads, so that after loading it into the RAM. The count for Ireland was 1 gigabyte, for the UK it was already 5. You could, of course, break into countries, regions and even areas, and then redirect to the desired graph, depending on the coordinates of the user. However, this made it impossible to create routes between areas, and most importantly - did not solve the need to keep all these graphs in memory. Finally, simply compiling each such object took up VERY many resources and lasted forever. For interest, I started the assembly of the count of France in my car (16 GB frames), waited 24 hours and canceled.
It is obvious that the technology, which performed well in small tasks, is absolutely not designed for scaling (at least not with my resources). So one should either admit defeat or crawl to another technology. I took a pause for a couple of days and began to study what other open source solutions exist in the world. It turned out that there are basically two of them:
If the first is written in Java and loads the graph of roads into RAM, then OSRM - the Open Source Routing Machine - is already written on the pros and keeps its (not less monstrous) intermediate files on the disk. Thus, the need to have a huge amount of RAM has changed to the requirement of a large and fast disk. This is already more real.
After a couple of nights of tinkering with the documentation, the entire server code was transferred to a new solution. It really worked, and worked well. It was possible to connect several countries, and even the search speed increased. The general principles were the same: from the extract of OpenStreetMaps, intermediate files were compiled for the “machine” profile (the profile is a set of weights and instructions for the edges of the graph - there are “foot”, “bicycle”, etc.). Then these files were added to the directory, and OSRM api already read them from the disk. By the way, Api turned out to be rather big - both the contour lines and the routing with various nuances were supported, there was even a generation of tiles for the map. At the last I decided to stop in more detail.
Returning to the application and continuing to test it, I realized a couple more things:
I gladly threw out a Google map (hooray, now 100% open source and my data), simplified the menu, moved it down. I started thinking what to do with the cards. And here, by the way, turned up the api tiles, which I mentioned above. It allows you to generate a vector tile for given coordinates and zoom level. The result is given in the form of a binary blob of the type application / x-protobuf — a rather inconvenient data type for manipulations. I will not go into details (I had to sweat a little), but in short my actions looked like this:
In the course of the action, there were different nuances, for example, indenting a bounding box or marking points with colored rings (and making their radius constant for all zoom levels). The resulting picture looked like this:

When I attached a visual route to each result, the list began to play with new colors. In addition, realizing that each picture defaults to the north, I made them spinning in relation to the compass. Thus, in addition to the visual effect, this chip also became functional - replacing the direction arrow. Now, when you move behind the wheel, you can see exactly which side of you or other result is located.
The third month of development was expiring, and it was already necessary to round out. The more you add, the more you want, so at some point you just need to pull yourself together and release the project. I have a little more corrected and painted the interface, and for the feeling of completion I sketched the application logo:
and the introductory page:
And finally, the final version of the application:
Well, thank you for your attention. I hope this stream of consciousness will be interesting to someone, and maybe even useful. At this stage, I consider the application ready: it is fast, without any special bugs, and will be able to work in any country in the world. By the way, you may have noticed that the screenshots were from both the iPhone and the android, because thanks to Flutter, the application works exactly the same on both platforms.
Nevertheless, so far I decided to freeze everything - I changed jobs, new concerns appeared. After a couple of months I shook off the dust and decided to write a retrospective. Your feedback is interesting: whether you would like it, use it, what can be changed.
PSOf course, about the readiness of the application is nonsense. It is ready as a prototype - if you approach a serious production, then you need to do data synchronization scripts with OpenStreetMaps, check the work at the zoo devices, localize interfaces and so on. The same backend on the node and python will fall under any serious load.
Two words about myself: I live in Dublin, Ireland, I work as a programmer. It’s not exactly sitting in place, so in my free time I saw different projects at home, mostly on the table. I write on Habré for the first time, although I have been reading for many years.
Problem
Quite a long time ago, when at work I had to travel a lot to unfamiliar places, I began to notice that the standard search on any maps today is absolutely inapplicable to drivers. Look: you are driving behind the wheel in an unfamiliar area, and you have a gasoline arrow at zero. Your actions? If at this moment I am not alone in the car, then I tell the passenger: “Well, look near the gas station while I am on my way”. Because if you do it yourself, then you need to perform the following actions:
- Stay
- In the maps application, enter “gasoline” in the search (or click on one of the quick buttons that some cards now offer)
- The application does a search and shows you a huge map with a dozen refills
- You try to figure out which one is closest to you, and click on it to build a route.
In my opinion, a nightmare. First, you have to stop, or at least wait for the traffic light. Because the maps are complex, and the icons are small. Secondly, the map does not tell you what the next gas station is. In this regard, Google is the worst: even in the results in the form of a list, he stubbornly pushes up not the nearest place, but I have no idea what is the most rated / with photos / paid /.
Well, the third factor: we are moving. The result, which the cards were issued, was relevant for a certain point, but we are already far from it. Have you noticed that even the route laid out by Google is not automatically updated if we have shifted? Only if the navigation has already been launched - then it will rebuild.
In general, the problem is clear. Logic, which works for pedestrians and their needs, for drivers about nothing. I do not care about the rating of the place and what kind of kitchen there is - I need, without being distracted from the road, in real time to get a route to the nearest gas station, charging, parking, ATM and so on.
Idea
Let us now try to determine the ideal search script. The following criteria:
- the interaction is short and understandable so as not to distract the driver
- transparent distance-based delivery
- real time update
The first thing that suggests itself is to replace the standard one-time search with a scan. That is, a chain of actions is approximately as follows:
- Run a scan
- You go, you look at the updated results in real time
- When I liked something, I clicked and paved the route.
You define the search criteria in advance - in fact, this is the type of location and the scan radius. Further, while the car moves, the application works as a radar. It quickly became clear that, firstly, the radius was not round, but in the form of an isoline. Secondly, it should be built not by distance, but by time, because minutes are much easier for perception than kilometers.
Then I thought about the method of issuing results. More precisely, do I need a map at all? The driver looks at the application with one eye and interacts with one finger - he doesn’t need a map, but large text and large buttons. Therefore, I immediately decided that the main screen would be a list, and I might add a map at first and see if I left it.
Plan
Knowing my peculiarity to stretch projects, I decided to take 2 months for myself - as a result, I got into 3. In principle, the application is quite simple:
- A client of a pair of screens (search, list and map)
- Periodically sends to the server its coordinates, search radius and type of places
- The server builds the isoline in time (by the way, it has its name in English - isochrone), does a search in places and returns a list
Sounds like a lung easier. I already had some previous experience in cartography (a couple of years ago I made a real estate portal where the search was on the map), so the stack on the backend was immediately clear:
- import data from OpenStreetMaps to Elasticsearch
- OpenTripPlanner for contouring
On the client, on reflection, I decided to use the new Google framework - Flutter. It is cross-platform, quite flexible, and allows you to make full-fledged applications with a minimum of code. Of course, it is raw and incomprehensible in production, but it looks perfect for prototyping. It is necessary to clarify that by this time I had the experience of native development for android (was the team leader) and decided, so to speak, to look the enemy in the face. The enemy was not so terrible.
Implementation
The first prototype of the application was ready very quickly - Flutter has a low threshold of entry and an understandable redux-like philosophy. The declarative description of the interfaces, oddly enough, also liked, as well as the hot reboot (React Native, your card is a bit). In general, it seemed that the Google users took into account most of the congenital diseases of previous attempts. However, I understand people who may not have it all - someone doesn’t like darts, a limited number of widgets, and the “visual debug” that is being offered here is something very raw.
On the backend, I did the following:
- I put Nominatim, filled OpenStreetMaps data extract into its database (I took it here ), using its native utility osm2pgsql. Why did you turn to Photon , a small but very pleasant open source geocoder . Earlier, I used it in a couple of projects - it generates the Elasticsearch index, imports data from the Nominatim database and searches by this index. I like it with speed and clean mapping (for example, I tried Pelias and I liked it less). His main problem is the old version of elastic, but in my case I didn’t need the functionality of the geocoder itself, only the data, so after importing, I transferred the index to the latest version of the elastic installation with a pure soul. By the way, why did I choose Elasticsearch? It is very fast, and it has a function to find coordinates by polygon.
- Polygon - it is isochrone - for me initially generated OpenTripPlanner . This is quite a good open source route planner. It works as follows: it takes the same extract of OpenStreetMaps and compiles it into a large graph of roads, which, as a separate object, saves to disk. When the server starts, this graph is loaded into RAM and all routes are searched through it. Pros: quickly raise, rich functionality (for example, generates isolines out of the box) and a good speed of work. Minuses: this speed of work directly depends on the amount of RAM, and the documentation is extremely disgusting. Just monstrous documentation. Vietnamese flashbacks.
- On the python, I sketched a small api, which takes the type of place and the search radius in seconds, requests the polygon from OpenTripPlanner, then searches for it in Elasticsearch. For each place found, it requests a route (again from OpenTripPlanner), takes its length and time. After that, all the collected data beautifully packs and returns.
I did the update of the results by shifting the device coordinates by 5 meters. The map is static - I just used Google’s google maps (as you can see, this is the only place where the corporation crawled into our cozy world of sorts). The first implementation looked like this:
 Having played with the application, I decided to hide the map. She did a good job of understanding which search site was being made, and she looked entertaining - it was interesting to see how this octopus changes shape in real time. However, this entertainment did not help the application to perform its functions and occupied a third of the screen.
It also occurred to me to add an arrow, which indicated the direction to each result. It worked like this:
- We remember our previous coordinates
- At offset we plot the route from the previous position to the current one.
- We take the last segment of our route and compare it with the first segment of the route of each result. Since they are laid on the same road grid, with a 99% probability the angle between them is close to either 0 or 180.
This very simple trick makes it much easier to understand whether we are already heading towards a place or need to turn around.

By this time I was quite pleased with the resulting application and decided to try to deploy it to several countries. Still, Ireland is a very small state, respectively, and the elastic index and the graph of roads were small. For the test, I decided to connect the neighboring UK. It is about 4 times longer and has a much denser road grid (especially the capital and major cities). And there was a problem.
Elasticsearch expectedly digested the increased index well, but with OpenTripPlanner there was a complete failure. It is written in Java and, as I said above, generates a graph of roads, so that after loading it into the RAM. The count for Ireland was 1 gigabyte, for the UK it was already 5. You could, of course, break into countries, regions and even areas, and then redirect to the desired graph, depending on the coordinates of the user. However, this made it impossible to create routes between areas, and most importantly - did not solve the need to keep all these graphs in memory. Finally, simply compiling each such object took up VERY many resources and lasted forever. For interest, I started the assembly of the count of France in my car (16 GB frames), waited 24 hours and canceled.
It is obvious that the technology, which performed well in small tasks, is absolutely not designed for scaling (at least not with my resources). So one should either admit defeat or crawl to another technology. I took a pause for a couple of days and began to study what other open source solutions exist in the world. It turned out that there are basically two of them:
- Opentripplanner
- Project OSRM
If the first is written in Java and loads the graph of roads into RAM, then OSRM - the Open Source Routing Machine - is already written on the pros and keeps its (not less monstrous) intermediate files on the disk. Thus, the need to have a huge amount of RAM has changed to the requirement of a large and fast disk. This is already more real.
Finish line
After a couple of nights of tinkering with the documentation, the entire server code was transferred to a new solution. It really worked, and worked well. It was possible to connect several countries, and even the search speed increased. The general principles were the same: from the extract of OpenStreetMaps, intermediate files were compiled for the “machine” profile (the profile is a set of weights and instructions for the edges of the graph - there are “foot”, “bicycle”, etc.). Then these files were added to the directory, and OSRM api already read them from the disk. By the way, Api turned out to be rather big - both the contour lines and the routing with various nuances were supported, there was even a generation of tiles for the map. At the last I decided to stop in more detail.
Returning to the application and continuing to test it, I realized a couple more things:
- the menu at the top is no good
- I don’t need a shared map, it just ties me to Google
- results cards boring and monotonous
I gladly threw out a Google map (hooray, now 100% open source and my data), simplified the menu, moved it down. I started thinking what to do with the cards. And here, by the way, turned up the api tiles, which I mentioned above. It allows you to generate a vector tile for given coordinates and zoom level. The result is given in the form of a binary blob of the type application / x-protobuf — a rather inconvenient data type for manipulations. I will not go into details (I had to sweat a little), but in short my actions looked like this:
- Pick up the line of the constructed route to the point in the form of polyline
- Polyline -> GeoJSON
- Get the bounding box of this shape.
- Request all tiles that this bounding box captures
- Convert tile data from binary format to GeoJSON
- Glue the tiles, trim along the bounding box, combine with the route line, paint
- GeoJSON turned into a raster image
In the course of the action, there were different nuances, for example, indenting a bounding box or marking points with colored rings (and making their radius constant for all zoom levels). The resulting picture looked like this:

Finishing touches
When I attached a visual route to each result, the list began to play with new colors. In addition, realizing that each picture defaults to the north, I made them spinning in relation to the compass. Thus, in addition to the visual effect, this chip also became functional - replacing the direction arrow. Now, when you move behind the wheel, you can see exactly which side of you or other result is located.
The third month of development was expiring, and it was already necessary to round out. The more you add, the more you want, so at some point you just need to pull yourself together and release the project. I have a little more corrected and painted the interface, and for the feeling of completion I sketched the application logo:

and the introductory page:

And finally, the final version of the application:

Total
Well, thank you for your attention. I hope this stream of consciousness will be interesting to someone, and maybe even useful. At this stage, I consider the application ready: it is fast, without any special bugs, and will be able to work in any country in the world. By the way, you may have noticed that the screenshots were from both the iPhone and the android, because thanks to Flutter, the application works exactly the same on both platforms.
Nevertheless, so far I decided to freeze everything - I changed jobs, new concerns appeared. After a couple of months I shook off the dust and decided to write a retrospective. Your feedback is interesting: whether you would like it, use it, what can be changed.
PSOf course, about the readiness of the application is nonsense. It is ready as a prototype - if you approach a serious production, then you need to do data synchronization scripts with OpenStreetMaps, check the work at the zoo devices, localize interfaces and so on. The same backend on the node and python will fall under any serious load.