Tell the mayor where to go (using Open Street Map)
Do not you think that our officials are somewhat divorced from reality? What is not enough to look at the controlled territory solely from the office window in the city center? What would it be nice for the city head to walk around the city personally from time to time, to communicate with people and answer questions in the mode of live communication?
So it seemed to the well-known blogger Ilya Varlamov, too, so he launched the coolest project with the defiant name BDSM, and had the Big Road With the Mayor ( link to the announcement):
Stop walking around Russian cities alone - it's time to invite mayors, governors, city managers.
The rules are simple.
The computer randomly selects two addresses in your city. After that we go for a walk from one address to another. Condition one: do not use the car. Only legs, bike, public transport, scooter, dog sled. No preparation, no agreement on the route. Sometimes the walks will be long, through the whole city, sometimes you have to walk just a few blocks in the center. This is the main condition: without showing off, without preparation.
In the course of the walk, we, together with the head of the city, study the infrastructure and communicate with the residents. The official will have a unique opportunity to tell the whole of Russia about his city, to explain the subtleties of city management, to get closer to the people!
A frame from the first video of the project, with the participation of my service for selecting random points
What does it have to do with Habr, you ask? And the whole thing in the phrase about a random choice of the address of the beginning and end of the route. This is not an easy task if you don’t want to end up in the wilds of any industrial zone or get lost in the endless fences of the private sector. Simple poking at the card with a blindfold will not work here!
About how I solved this problem and what discoveries gave me such cities as Ivanovo or Kirov - under the cut.
It just so happened that I already had the necessary experience with maps and navigation thanks to my service for building tourist routes. It was through an article about him that in Habré I was found and offered to develop a system that selects the starting and ending points of the route on the map of the city.
Idea
As conceived by Varlamov’s team, it should have looked like this: on the map of the city, markers of the start and end points jump back and forth, clearly demonstrating the randomness of the process. The mayor himself at any time pushes the button and stops the search. After that, they choose the route (it is a pity that manually, then my walking route service could offer them something interesting) and go for a walk around the city.
Initially, the requirements for points were as follows:
- The distance between the points is not less than 3 kilometers
- The points are not in the industrial zone and not in the private sector.
- The points are necessarily located in different parts of the city, so that the route between them leads through the center.
Then, during the development, another requirement was added: the service should be able to interact with the hardware Big Red Button, like this:
The button simply sends a click signal to the connected computer.
The whole difficulty was how to make this route in such a way that, on the one hand, to avoid the wilds and, on the other, to cover all points of the city with possible points, and not just a few "decent" neighborhoods in the center. At the same time, I initially had a certain set of stereotypes in my head, which are a consequence of the fact that I myself live in St. Petersburg, besides from more or less large Russian cities there were only a few million of people like Moscow and Kazan. Here are just a project mainly concentrated on cities with a population of 200-500k people (apparently the mayors of larger cities are too busy to walk the streets). In general, I once again received confirmation of the thesis that "Moscow is not Russia." All the considerations below are incorrect:
1) Multi-storey houses in the center, and industrial zones and private houses on the outskirts.
2) At least there is a division by quarters - in one block only high-rise buildings, in the other - only private houses. Well, who will build Khrushchev among wooden huts?
3) The territory of the city is one simple polygon on the map, inside of which there are all buildings
4) Ok, there may be holes in the polygon, but at least one
5) Well, not one but at least two unrelated pieces
6) Cities in OSM the data is tagged with the appropriate landuse tags and choosing residential areas will just be
In general, I had to walk around the rake.
Point selection algorithm
To begin with, I had to decide exactly how the starting and ending points would be chosen. The idea of throwing a point randomly at arbitrary coordinates within the city’s borders I dropped almost immediately. Most of the city on the map is a different kind of impassable territory: polygons of houses, rivers, simply unmarked territories on which it is not clear what is located. Honestly, a random point will very often fall into them, you will either have to shift it (which will affect the randomness quite a lot), or transfer it until you get to a suitable place (and this will affect the speed of work).
Since the walk does not need an arbitrary point, but one that can be reached on foot, I decided to take points only along the roads. As a result, my algorithm takes all OSM roads (Way with a highway tag and values corresponding to pedestrian roads and sidewalks), breaks up too long edges adding intermediate points (so that adjacent points on one edge are not more than one hundred meters apart ), then removes points that are too close to each other (closer than 50 meters), for example, on several parallel roads.
Marked points for Yaroslavl, carved quarters of private houses and industrial zone along the railway
The result is a more or less uniform grid covering the entire city. Further, when requesting the next point, it is no longer necessary to throw and do heavy geometric calculations many times; it is enough to choose a suitable random pair from a previously calculated set of points.
How to choose them? We have two requirements for this: no closer than three kilometers from each other and in different parts of the city, so that the route must pass through the center. If the first condition is simple, then for the second I needed to determine the center itself. From the maps such information is not particularly vyzhmesh (geometric center of the city is not at all a fact that it is its historical center).
For this, I decided to go the easy way, and just mark the center of the city manually. Added loading of polygons in GeoJSON, which I edited using the very convenient site geojson.io (seriously, if you need to work on your project with a simple 2D geometry, then geojson.io allows you to do everything very quickly and not to steam with the creation of your editors or using any heavyweight GIS applications).
Marked Kirov, with marked center (yellow), manually removed areas (red) and added piece (green)
Thus, the first version of the algorithm was ready in two days and the testing time began. And here reefs have got.
Private sector
Under the terms of the problem, the route should not begin and end in the private sector. By my naivety, a resident of a large city could not have imagined that private wooden houses could be on a nearby street from the central square of the city. And not a couple of monuments of antiquity, like ours, but naturally whole blocks.
Fighting these quarters took most of the time. At first I tried to do with pure OSM data. There is a special meaning of the residental = rural tag to denote the countryside and landuse = allotments for all gardening.
Yes, even in St. Petersburg there are embedded gardening within the city limits
However, it quickly became clear that these tags are set quite rarely, and are usually satisfied only with the most common landuse = residental (residential area).
The next step was an attempt to weed out such quarters manually. I added support for manual removal of areas to the geojson file with the city center. Unfortunately, it quickly became clear that a bunch of iterations of the cycle "loaded the map - looked at the eyes - found the private sector - cut it out at geojson.io - loaded again" takes a lot of time, especially in cities where private houses go along with high-rise buildings and are scattered in small groups and here. For example in Ivanovo
Private and apartment buildings are densely mixed with each other just a couple of blocks from the center.
I had to invent an empirical algorithm. I took a quarter (landuse = residental area, although there is no direct requirement for its use, usually it is not marked by the whole city or district, but by separate quarters), I took all the buildings within its boundaries (polygons with the building tag) and considered the average area of such a polygon. If it was less than 400m2 (the figure was taken from the ceiling, with the condition of coarse marking and counting and individual long houses found in such quarters), the quarter was considered the private sector and was thrown out of the calculations.
As a result, the combination of these three methods made it possible to more or less effectively filter out unwanted areas, although not without the involvement of manual labor.
But of course then there were exceptions. For example, Evpatoria, where in general the entire historical center is in essence the narrow streets of the private sector. If you throw it all away, there will be nothing left of the city.
The typical landscape of Evpatoria in a couple of hundred meters from the sea
Especially for such cases, we had to add to GeoJSON not only manually deleted sectors of the city, but also manually added sectors. Those. even if the quarter did not fall under other conditions, it was still included in the list of searchable points.
Industrial zones and commercial sites
With industrial zones, everything is much easier than with the private sector. As a rule, they are already marked up with the landuse = industrial tag and there are no problems with them. Unless there are some exceptions, when not all blocks are laid out, but separate buildings. But since I used the ready-made code for working with maps from Sight Safari in the project, it was already taken into account there.
With commercial sites landuse = retail had to tinker manually. Since in half of the cases, large markets or shopping centers are laid out in this way, and in the second half, wholesale warehouses are somewhere on the outskirts, or in general cargo terminals where no entry is allowed. Standardization, as usual, is lame. Fortunately, there are quite a few such zones, this tag is not very popular, so it was not difficult to deal with them manually.
At the same time I added my list of funny place names, which are often found in industrial zones. For example in Yaroslavl there is a brake street. If I lived there, I would definitely start an office there for my IT office.
Cartographic Surprise
Even though I already had some experience with OSM, it was still constantly emerging that something new.
For example, Yaroslavl had an open external contour. Visually, it looks closed, but refers to the missing nodes and lines. Possible export problem.
A lot of drug addiction was found within the boundaries of cities. In my naivety, I thought that the border of a city is always a simple closed broken line. But no.
There are cities with holes. For example Obninsk. That is, the real piece inside the city is not administratively owned by him.
Obninsk with a hole
There are cities consisting of several unrelated areas, such as Kirov. And if on average there are several blocks of Kirov, then the left, the smallest one, is just a piece of the road leading to the airport and four houses. What was the salt cut such a piece and attach it to the city?
Kirov consists of three unrelated areas
For sure if the BDSM program will continue and I will continue to participate in it - I will get acquainted with some non-trivial cartographic situations that I cannot imagine right now.
Problems with the layout of maps
A couple of words about OSM markup.
In the last cities that I processed, I was faced with the fact that people do not put landuse tags, or put them somehow. In fact, a person who is far from IT may not be very clear about the deep meaning of these tags. Well, if you think about it, they change the color of the card (gray residential areas, pink industrial zones, etc.), it would seem to be unprincipled. And therefore many treat them rather casually.
However, in my opinion, the power of OSM lies precisely in the ability to machine processing large amounts of data, such as what I did here or in Sight Safari. There are a lot of maps on the Internet, but I don’t know any more maps with the ability to extort all the geometry and tags and use them freely in my algorithms. And just for such purposes, these tags begin to play an important role. Since almost always tasks look like “find something there in areas of a certain type,” and the landuse tag is the first and easiest way to identify and find the necessary areas.
The same, but to a lesser extent, applies to a bunch of other tags, for example specifying the type of use (well, it's not just a residential area, but a low-rise building). They are often not bothered to bet, probably because they do not see in them a great benefit. And in fact, if you use only navigators and online maps, they are not visible there. But it is for machine processing of map data that they can provide valuable hints and simplify algorithms.
In general, put the tags, gentlemen! Oh please!
And yes, on the basis of the problems found, I myself went over the maps and added some tags. So the community benefits from this project too.
So where did I send the mayor?
Since I did not take part in the shooting itself, and most of my work was completed before the shooting of the first episode, I had to be content with the same fragmentary information from public sources as everyone else.
In general, it was a bit dumb, if my algorithm would lead them completely into some wilderness. It became especially dumb when I read something like this in a post about Ivanovo:
Vladimir Sharypov, the mayor of the city, left a pleasant impression. An open and frank person. I would say too frank for the official. The conversation turned out to be interesting. Went to Sort, where a meeting with some voters took place spontaneously.
All anything, but at 14 o'clock on the Sorting there are no sober. Under the terms of the show, the mayor must walk unguarded. I even thought at some point that there would be a fight, as some citizens were obviously in inadequate condition. But the cost.
Well, finally came the first series on the channel, you can watch.
When the algorithm chose Tovarny Lane on the Evpatoria map, it seems to Illya that doubts crept in, in any case he asked if the mayor was afraid. The name is something that immediately evokes thoughts about some gloomy abandoned factories, dirty roads and dark personalities with a heavy look (forgive me, residents of Evpatoria, if this is not so).
Conclusion
Work on this project was interesting. Without leaving home, I went on a small trip through the cities of my country, izlazil them with a magnifying glass in all possible maps and became acquainted with the local cartographic coloring. Along the way, I also finalized the Sight Safari code.
As you can see, this is not such a trivial task - to dig into maps and select random points. And for every obvious algorithm, life is ready to give you a dozen cities where it does not work.
As advertising: I am doing all this on the basis of the Institute of Design and Urbanism at ITMO. We are doing there a lot of interesting pieces related to IT and urban data, if someone is interested in this topic - come to us to study for a magistracy or work.
During the preparation of the project, the code and experience of the Sight Safari project - a navigator for searching tourist routes was actively used .
And finally: what do you think, is this a useful initiative - such walks with mayors? Will they change anything? What questions would you ask the mayor of your city?