March 29, 2018 at 13:25

An effective way to load a large number of points (geometries) onto a map

    Hi, Habr.


    Today I will tell you how to efficiently download a large number of points on a map (web or in mobile applications).


    Quick dive


    From the point of view of most modern cartographic solutions, several basic approaches to map rendering can be distinguished:


    • raster (static files - pictures with tiles stored on CDN and having their own TTL)
    • vector

    Here you need to make a remark, a tile, for those who are not comfortable, this is a special block coordinate grid (parallel to the tiles are also called static images), which is characterized by three coordinates (z - zoom, x, y), at each level the number of tiles $ 2 ^ {(2 \ times zoom)} $


    The most interesting today is the vector format.


    Why? Actually, you need to start with the problems associated with the raster format:


    • Discrete zoom levels (0 to 20 in most cases)
    • Rigid style and data binding on tile layers
    • Classic raster flaws

    What prospects does the vector format offer? Actually, there are several of them:


    • Zoom is not clearly associated with tiles on a particular layer, but can vary with any accuracy and smoothness.
    • Since often it is the client (without considering the intermediate rendering option) that renders the map, it is he who determines what data and in what style he wants to draw
    • Excellent quality regardless of resolution, etc., which is, in general, the classic advantage of a vector over a raster

    Actually, everything with the vector format is good, but there are also drawbacks. At the very least, the vector format is harder to cache, it is not as obvious as in most cases with the raster format, where enough pre-generated tiles are folded onto a CDN, set up TTL and regeneration and no longer think about it.


    In general, you can talk about this for a very long time, my task was to give a brief description of the actual mechanisms of work, now we can move on to the topic of the post.


    Decision


    Actually, from the point of view of possible solutions, there are several approaches that have different support among ready-made libraries (services):


    1. Download the whole GeoJSON and render it (most fall off on this option :)). Actually, the advantage is that almost all the popular libraries Google Maps, Yandex Maps, Mapbox, Leaflet, etc., support it one way or another. However, not everyone does it efficiently, for example, on Google’s request for an effective solution for loading a large number of points in Mapbox, the first link will be their solution - supercluster, which in fact is a server utility that downloads, prepares GeoJSON data and returns it to the customer.


    2. Download only in viewport. The idea is quite simple, we only load what is in the viewport (* in some cases, the nearby tiles are lazily loaded). The implementation of this is in the box for example in Yandex Maps - LoadingObjectManager , including which it can do clustering on the client. They also have RemoteObjectManager, which already expects clustered data from the server.


    3. Pregeneration of static tiles and loading them. It also has a place to be, especially in the case of highly loaded services, but not particularly interesting. Most often, tiles are generated using a tool such as Mapnik and poured onto a CDN, or a third-party utility / library is used that does all this work on its own.


    4. Mapbox Vector Tiles, actually one of the vector formats for maps, which describes various geometries, their style, etc. ... The idea when using this method is to create your own MVT endpoint, which actually can:
      • Return geometry
      • Cluster points
      • Simplify geometry at low zoom levels
      • Other

    This format is perfectly supported by solutions from Mapbox, there are also libraries for Leaflet.


    From the point of view of storage for GIS data, which would also make it possible to efficiently perform operations on them, you can choose PostGIS, which by the way gives all the necessary operations for the above, including the generation of MVT (Mapbox Vector Tiles):


    • Including geometry in a tile (rectangle) using the operator @
    • MVT Generation - ST_AsMVT, ST_AsMVTGeom

    Thank you for your attention, I hope for a constructive feedback, if it is interesting I can tell you more about some aspects of working with cards!


    Examples of the above are available on my github

    Tags:

    Also popular now: