How I made my “Yandex.Transport” with timetables and buses

    Author: Mikhail Aksenov, .NET Developer, DataArt

    How it all began

    In the fall of 2014, there was joyful news that Yandex.Transport was launched in Voronezh, which shows the movement of buses and trolleybuses. It was very cool right at the bus stop to open the schedule and find out what your bus will do in 5 minutes.

    But after a couple of months everything went according to a quite predictable scenario. Since all transport in Voronezh from the trolley bus to the bus works “in cash”, each driver is interested in carrying as many passengers as possible. As a result, they arrange a race among themselves.

    New service drivers used to follow each other on the map, overtaking and cutting competitors on the road. Some began to fall revenues, they began to complain to the owners of the routes, and they went to the people selling equipment with GLONASS and GPS and sending data about the movements of cars in Yandex. At the request of the fleet owners, they stopped sending information, and the buses immediately began to disappear from the map. Now they have become a little bigger, but it is still much less than it actually is.

    For those who are too lazy to read further, I offer a video report on this topic:

    For the rest - I will describe the project in the form of text.

    Where to get the data?

    At the same time in Voronezh there is a place where all the data is collected - the Center for the Organization of Road Traffic (CTO). They have a big and heavy web application on the site with non-debugged JavaScript and other artifacts, while everything in it comes with POST requests and nothing is cached. Using the site itself, you can easily put a service that collects data on the movement of buses.

    There was an option that allows you to poke on the map and find out when the bus will come and what it will be like. Provided that you approximately get to a stop (± 50 meters). In the menu, you could open a list of routes and see exactly where the buses go, but there was a problem - they drove strangely. For example, in Koltsovskaya, a street in the center, where you can’t especially accelerate, the bus moved forward and backward at a speed of about 200 km / h.

    Why python?

    I chose Python because it is cool, the batteries are bundled, and so on. And 3.6 because there is formatting of string literals, typing, that's all. Let's look at what I basically used:

    • Many complained that it was very difficult to drive the exact name of the stop with all punctuation marks. Then I realized the possibility of a fuzzy search and created the first unit test for it.
    • The whole system is hosted on Heroku. Free, because I do not have a database - I hope to get by with the TSODD database. I agreed on its use with the Center when I realized that the data that I collect through the web interface is not enough.
    • Actually, the hosting goes through the Tornado web server. I think it is familiar to anyone who has encountered web queries in Python.
    • I needed the Pytz package, because the Heroku server is in a different time zone, and the data comes to me without specifying a time zone. Therefore, I myself took care of localization.
    • To create a bot, I used the recommended library Python-telegram-bot . The documentation here is quite adequate, and in general the library meets the requirements of the minimum viable product. Initially, the entire project was a Telegram bot, which in response to sending your location sent a bus schedule.
    • Firebird, the former Interbase - Open source database, which, I believe, many of you have worked with. Of course, it is not as cool as the same PostgreSQL, but for a very large number of queries it is enough. In our case, no more is needed.
    • Caсhetools is a very simple module that allows you to cache calculations. This is, of course, not about memecached or caching web pages, but about cases when you need memoization for long queries. Just take, add the appropriate decorator - and it works. The cache has different options, I use the TTL version, which saves data for a specified time, because I know that the data will not be updated more often than at certain intervals (in my case - 30 seconds).

    Bot functionality

    • / nextbus stop name — expected arrival time;
    • / last route numbers separated by a space - the last stops;
    • Sending location — estimated time of arrival for the next three stops;
    • Free entry - route numbers and distance to buses (when sending location).

    I tried to describe the teams in sufficient detail so that people would not ask how to use them. There are basically two basic commands: nextbus, when you write the name of the stop, and sending the location.

    Site functionality

    I made the website as minimal as possible. Of the third-party libraries I used only two, to support fetch and promise, because the old versions of the pre-installed browsers in the phones cannot do without them. If you do not support these methods, you get quite heavy noodles from callbacks for the requests themselves. With fetch, everything looks sleeker.

    Actually, the functionality here consists of 4 items:

    1. Arrival. You can just see the stop. By clicking on the button (by the way, it may be worthwhile to do more - I will think about it!), You send your location, the system searches for the three stops nearest to you and shows information on them. You can enter the name of the stop and get information on it. Since you hardly need all the 20–30 buses that can come to a bus stop, you can filter them by routes.
    2. Buses. The second page contains information on buses. This is connected with another story that pushed me towards this project. One day I forgot my hat on the bus, called the control room, explained where it happened. The dispatcher offered me to catch my bus in the center, where he would return, turning on the final one. I remembered that I have a login and password from the Traffic Management Center system, but it turned out that I still can’t track the location of the car I need via the web interface. I returned the cap, but with great difficulty.
      I thought that the search for the bus could be much easier. I wrote the route number, and the system shows you the cars on the line, their current location and the time when they passed the stops along the way. And even if you know the route number, but you know the bus number (this also happens), you can still find it.
    3. Map. Map on the site without animation. Here you can choose buses from the list or enter numbers manually, you can filter them. Up to 600 buses can be shown on the map, although after 20 it is difficult to find something in the center, simply because the routes are all concentrated here. Therefore, it is better to limit the search parameters more rigidly.
    4. About the project. Now the project has a website, there is a Telegram-bot, groups in social networks.

    Fortunately, Python is a language with batteries out of the box. For simple things, you don't even need to download anything, just read the documentation carefully.

    Of course, such services can be done for other cities, moreover, in some they already work. Much depends on the local data processing center, in some places, for example, in St. Petersburg there is even an API for developers. But the most important thing is the citizens who want to change their city for the better.

    Also popular now: