March 3, 2015 at 12:35Visualize it
- Tutorial
A few years ago I made a fishing site and published a map of the area around the city, marking interesting places. I was so interested in maps that today I am engaged in the design of geoservices, and at my leisure I am experimenting with cartographic visualizations. Recently, I visualized the statistics of the Moscow bicycle rental , and earlier I published a high-rise model of the city in the form of a multi-colored grid of streets.
It's great when large amounts of data about the world around us become visual, tangible. What attracts me to this is not the multitude of colorful lines or dots against a dark background, but the opportunity to visually show and tell others an interesting story, to make this data useful to people. I want more people who are keen on this topic - to share knowledge with each other, discuss ideas.
Houses in the center of Moscow are of different colors depending on the area of the building.
Data: © Participants of OpenStreetMap
I have already talked about the design in cartography and will continue to tell, because you can tell a lot of interesting things about this. In this story, it’s not about traditional cards.
I decided to collect what I know about cartographic visualizations and talk about it. As a result, I got an online course "Visualization of geodata" - a series of mini-lectures on general principles and tools for working with geodata. For each part, I collected links to additional materials and examples of working files so that you can dive into the details and try to do something yourself. This post is compiled from course materials.
In fact, there is no fundamental difference between ordinary paper maps and newfangled cartographic visualizations - the principle of any map: a visual representation of our knowledge of the real world.
I would single out several characteristic components of good cartographic visualization:
By combining well-chosen data, technology and design, many interesting visualizations can be made. Let me show you a few for example:
One of the brightest examples of a good history with geodata is the visualization of all Dutch buildings by year of construction . In addition to entertainment, the story tells very clearly on this map how cities were built up.
All buildings in the Netherlands are stylized by years of construction.
In addition to visualizing the Dutch buildings, there are similar projects: Brooklyn , New York , Moscow .
MapBox guys visualized 1,500,000 RunKeeper tracks - jogging, walking, riding bicycles. Popular places for walking are clearly visible. I even found cycling routes from home to work in the summer.
1,500,000 RunKeeper tracks Map data: © OpenStreetMap contributors
A good example where data, technology and design are connected is Watercolor Maps from Stamen Design , a California-based design studio that has been successfully engaged in cartographic visualizations for a long time. In the case of this card, we can confidently say that it turned out to be a magnificent art object.
© Watercolor Maps by Stamen . Map Data: © OpenStreetMap Members
No card can be a card if it does not contain data. Data is received by the prefix "geo" at the moment when the information appears georeferenced and can be displayed on the map. Usually, an object is linked to the terrain using geographic coordinates - longitude and latitude, and where a three-dimensional representation is required, the height is also indicated.
Geodata is divided into two main types: raster and vector.
Raster geodata, as you might guess, are ordinary geo-referenced raster images. The most familiar example of raster geodata is satellite imagery.
Cloud Streets on the Black Sea, image January 8, 2015 © NASA Earth Observatory
In addition to satellite imagery, the raster is used for digital terrain models, where each pixel in the image contains information about the altitude at this point in the terrain. For better recognition of parts of the city, I supplemented the picture with a grid of main streets.
High-altitude model of Moscow © US Geological Survey , SRTM30, map data © OpenStreetMap participants
Raster data may not be static images, technologies allow you to embed video in an interactive map.
Vector geodata is described by a set or sequence of coordinates, geometry, and attribute values. There are three main types of vector data:
In addition to the type of geometry and location, attributive information is equally important. In vector data, each object may contain additional attribute information. Using these attributes, you can make selections of objects and apply various style rules to them. Using the example of a road layer from OpenStreetMap: by the “type” attribute, you can select roads with a value corresponding to “primary” and select them on the map with a special style.
An example of a layer of roads, the main streets are highlighted by the value of the “type” attribute. Map Data © OpenStreetMap Members
For the convenience of working with attributes, most tools have the ability to view data in a table. By the way, attributes can be filled in advance, or can be calculated depending on any parameters of the object. The cover image for this post is the buildings from OpenStreetMap in the center of Moscow, the color of which depends on the area of the house.
A large number of various formats have been developed for vector geodata, I will talk about the most popular:
The topic of cartography only seems specific, complex and confusing, now more and more various technologies and tools for “household cartography” are becoming available to ordinary users. I’ll try to give a brief overview of what I often use myself.
Quantum GIS (abbreviated as QGIS) is a true GIS in the classical sense. It is a cross-platform product with open source code and is an excellent alternative to expensive GIS packages.
Working with cartography, QGIS has become an indispensable tool for working with geodata for me. First of all, it is the ability to view, edit, import and export various formats, as well as the ability to analyze and work with samples of objects. For example, I often need to filter objects by some feature or select them in a specific area.
Quantum GIS 2.6.1: Building Layer © OpenStreetMap with categories by area, viewing object attributes.
In addition, QGIS has the ability to stylize a map, export map images for printing and publishing on the Internet. Quantum has a large community of developers interested in developing the project, and to date, a large number of plugins have been developed that significantly expand the functionality of the program.
CartoDB is an online service for working with your geodata, in fact a geodata hosting service with the ability to visualize and publish your own data. The basic scheme of working with the service is this: you need to download data, configure its display, and you can already use the ready-made visualization - publish a link to the project or place a map on the site.
Setting up earthquake visualization in CartoDB
In the screencast about CartoDB, I showed how you can create an animated visualization of earthquakes in a few minutes (happened in 30 days, according to USGS). At CartoDB, everything is very conveniently done so that a user with any level of training can figure out the service, and for those who need help, the guys launched a special site, The Map Academywhich publishes examples and screencasts that teach you how to use the service, and the blog publishes the best map of the week created using this service. For more advanced features, CartoDB has an API.
In fact, the company MapBox , which developed TileMill * over the past couple of years has already run far ahead: a more functional Design Studio product has been released to customize maps , the online service also has rich geodata capabilities, and a wide range of APIs are offered for developers . The team regularly publishes impressive visualization examples on its blog .
MapBox, like CartoDB, allows you to store your geodata in the cloud and publish it on various platforms. The main difference: CartoDB allows you to style and display them on top of any ready-made map, and in MapBox services you can customize both the map and the objects that you would like to display on it. At the same time, in CartoDB you can connect the tiles of the map prepared in MapBox.
The main idea of styling maps in TileMill is as follows: you add layer-by-layer data to the map, and you can stylize to customize the display style of the layer using CartoCSS, a look - and- feel language very similar to CSS. Pictures in a post about data types, or a cover picture were just prepared in TileMill, and once you figure it out and understand the principle, it is done conveniently, quickly and easily.
MapBox TileMill
A prepared map can be exported either as a static image, or published to the cloud for future use of the map on the Internet or on mobile devices. Data storage in MapBox is a paid subscription service, and there is a free tariff plan that is enough to familiarize yourself with the basic features of the service.
* - MacOS 10.10 Yosemite users need to download the dev version
Yandex Maps API - incredibly rich in various map features. For the first acquaintance with the JavaScript API, I recommend looking at examples from the Sandbox - here you can quickly figure out how to add an interactive map to a page with various parameters, configure the behavior of the map, or add various objects.
In the screencast about the API, I downloaded the information about the bike rental stations in CSV format, converted using QGIS these data to GeoJSON and then visualized them using the Yandex.Map API in several ways:
By the way, you can work with GeoJSON files in a slightly more elegant way, using ymaps.GeoQuery . For example, all loaded objects can be immediately added to the collection of geo objects and control display parameters for the entire data set.
The code for these examples can be found on GitHub in the screencast materials .
Clustering of bike rental points
I used clustering and drawing circles in the experiment with visualizing public toilets - I was very interested in showing 5-minute radii, and I used the heat map module in the draft statistics of the Moscow bike rental .
Heat map in the projectstatistics of Moscow Velobikes
This, of course, is far from all the API features. If you wish, you can compile a fairly interesting interactive project based on the Maps API. You can find interesting examples of visualizations in a blog , a development club , a sandbox , or in the documentation .
The topic of cartography is endless, and the more I immerse myself in it, the more limitless this cartographic universe seems to me. Particularly attentive readers probably noticed that the story turned out about data and about technology, and little is said about design in this story. We will go on this adventure later - for now, I am collecting materials on the topic, structuring and comprehending what has been collected.
So, the most important links for further immersion in the topic of geodata visualization.
Record screencasts:
Links and working files for examples: github.com/minikarma/geotalk
To discuss the topic, I started a Facebook group - “ Household Mapping ”. It will be great to find like-minded people and discuss similar topics there, to share experiments and experiences.
I plan to continue to replenish the materials as far as possible and I will be glad to additions and wishes.
Preparing information for the course, I collected a certain set of links for independent study. I share with you. It will be great if you share something interesting with me in the comments.
Interesting examples:
It's great when large amounts of data about the world around us become visual, tangible. What attracts me to this is not the multitude of colorful lines or dots against a dark background, but the opportunity to visually show and tell others an interesting story, to make this data useful to people. I want more people who are keen on this topic - to share knowledge with each other, discuss ideas.
Houses in the center of Moscow are of different colors depending on the area of the building.
Data: © Participants of OpenStreetMap
I have already talked about the design in cartography and will continue to tell, because you can tell a lot of interesting things about this. In this story, it’s not about traditional cards.
I decided to collect what I know about cartographic visualizations and talk about it. As a result, I got an online course "Visualization of geodata" - a series of mini-lectures on general principles and tools for working with geodata. For each part, I collected links to additional materials and examples of working files so that you can dive into the details and try to do something yourself. This post is compiled from course materials.
In fact, there is no fundamental difference between ordinary paper maps and newfangled cartographic visualizations - the principle of any map: a visual representation of our knowledge of the real world.
I would single out several characteristic components of good cartographic visualization:
- data - a good visual history is based only on quality data;
- technology - it so happened that technology simplifies the processing of large amounts of data and makes possible something that could not be done manually;
- design - the process of creating, constructing a map and the conscious desire to make the map convenient and understandable for users.
By combining well-chosen data, technology and design, many interesting visualizations can be made. Let me show you a few for example:
One of the brightest examples of a good history with geodata is the visualization of all Dutch buildings by year of construction . In addition to entertainment, the story tells very clearly on this map how cities were built up.
All buildings in the Netherlands are stylized by years of construction.
In addition to visualizing the Dutch buildings, there are similar projects: Brooklyn , New York , Moscow .
MapBox guys visualized 1,500,000 RunKeeper tracks - jogging, walking, riding bicycles. Popular places for walking are clearly visible. I even found cycling routes from home to work in the summer.
1,500,000 RunKeeper tracks Map data: © OpenStreetMap contributors
A good example where data, technology and design are connected is Watercolor Maps from Stamen Design , a California-based design studio that has been successfully engaged in cartographic visualizations for a long time. In the case of this card, we can confidently say that it turned out to be a magnificent art object.
© Watercolor Maps by Stamen . Map Data: © OpenStreetMap Members
Geodata
No card can be a card if it does not contain data. Data is received by the prefix "geo" at the moment when the information appears georeferenced and can be displayed on the map. Usually, an object is linked to the terrain using geographic coordinates - longitude and latitude, and where a three-dimensional representation is required, the height is also indicated.
Geodata is divided into two main types: raster and vector.
Raster geodata, as you might guess, are ordinary geo-referenced raster images. The most familiar example of raster geodata is satellite imagery.
Cloud Streets on the Black Sea, image January 8, 2015 © NASA Earth Observatory
In addition to satellite imagery, the raster is used for digital terrain models, where each pixel in the image contains information about the altitude at this point in the terrain. For better recognition of parts of the city, I supplemented the picture with a grid of main streets.
High-altitude model of Moscow © US Geological Survey , SRTM30, map data © OpenStreetMap participants
Raster data may not be static images, technologies allow you to embed video in an interactive map.
Vector geodata is described by a set or sequence of coordinates, geometry, and attribute values. There are three main types of vector data:
- points - a pair of coordinates is enough to set a point object, an example of point objects on a map is POI (points of interest)
- lines - the geometry of the line is defined by a sequence of coordinate pairs, the most familiar example of linear objects - roads on the map
- polygons - coordinates of vertices are set, buildings - an example of polygonal geodata
In addition to the type of geometry and location, attributive information is equally important. In vector data, each object may contain additional attribute information. Using these attributes, you can make selections of objects and apply various style rules to them. Using the example of a road layer from OpenStreetMap: by the “type” attribute, you can select roads with a value corresponding to “primary” and select them on the map with a special style.
An example of a layer of roads, the main streets are highlighted by the value of the “type” attribute. Map Data © OpenStreetMap Members
For the convenience of working with attributes, most tools have the ability to view data in a table. By the way, attributes can be filled in advance, or can be calculated depending on any parameters of the object. The cover image for this post is the buildings from OpenStreetMap in the center of Moscow, the color of which depends on the area of the house.
A large number of various formats have been developed for vector geodata, I will talk about the most popular:
- Shapefiles - initially this format was used only for Esri GIS packages, but it turned out to be convenient in use and became the standard for other geographic information applications;
- KML (Keyhole Markup Language) is an XML-based geodata markup language. For a very long time, Google Earth was the most affordable tool for working with geodata on the Internet, so KML files are widely used on the Internet and map services;
- GPX is a text format, again based on XML, which is mainly used for recording GPS tracks. An example GPX file can be exported from RunKeeper;
- GeoJSON - a text format, due to the convenience of using this format in JavaScript, has recently become actively used for interactive mapping;
- CSV - due to its simplicity, the text format is a common format for storing geodata, the coordinates are indicated in the columns of the table and, as a rule, CSV is used for point features.
Instruments
The topic of cartography only seems specific, complex and confusing, now more and more various technologies and tools for “household cartography” are becoming available to ordinary users. I’ll try to give a brief overview of what I often use myself.
QuantumGIS
Quantum GIS (abbreviated as QGIS) is a true GIS in the classical sense. It is a cross-platform product with open source code and is an excellent alternative to expensive GIS packages.
Working with cartography, QGIS has become an indispensable tool for working with geodata for me. First of all, it is the ability to view, edit, import and export various formats, as well as the ability to analyze and work with samples of objects. For example, I often need to filter objects by some feature or select them in a specific area.
Quantum GIS 2.6.1: Building Layer © OpenStreetMap with categories by area, viewing object attributes.
In addition, QGIS has the ability to stylize a map, export map images for printing and publishing on the Internet. Quantum has a large community of developers interested in developing the project, and to date, a large number of plugins have been developed that significantly expand the functionality of the program.
Cartob
CartoDB is an online service for working with your geodata, in fact a geodata hosting service with the ability to visualize and publish your own data. The basic scheme of working with the service is this: you need to download data, configure its display, and you can already use the ready-made visualization - publish a link to the project or place a map on the site.
Setting up earthquake visualization in CartoDB
In the screencast about CartoDB, I showed how you can create an animated visualization of earthquakes in a few minutes (happened in 30 days, according to USGS). At CartoDB, everything is very conveniently done so that a user with any level of training can figure out the service, and for those who need help, the guys launched a special site, The Map Academywhich publishes examples and screencasts that teach you how to use the service, and the blog publishes the best map of the week created using this service. For more advanced features, CartoDB has an API.
Tilemill
In fact, the company MapBox , which developed TileMill * over the past couple of years has already run far ahead: a more functional Design Studio product has been released to customize maps , the online service also has rich geodata capabilities, and a wide range of APIs are offered for developers . The team regularly publishes impressive visualization examples on its blog .
MapBox, like CartoDB, allows you to store your geodata in the cloud and publish it on various platforms. The main difference: CartoDB allows you to style and display them on top of any ready-made map, and in MapBox services you can customize both the map and the objects that you would like to display on it. At the same time, in CartoDB you can connect the tiles of the map prepared in MapBox.
The main idea of styling maps in TileMill is as follows: you add layer-by-layer data to the map, and you can stylize to customize the display style of the layer using CartoCSS, a look - and- feel language very similar to CSS. Pictures in a post about data types, or a cover picture were just prepared in TileMill, and once you figure it out and understand the principle, it is done conveniently, quickly and easily.
MapBox TileMill
A prepared map can be exported either as a static image, or published to the cloud for future use of the map on the Internet or on mobile devices. Data storage in MapBox is a paid subscription service, and there is a free tariff plan that is enough to familiarize yourself with the basic features of the service.
* - MacOS 10.10 Yosemite users need to download the dev version
Yandex.Maps API
Yandex Maps API - incredibly rich in various map features. For the first acquaintance with the JavaScript API, I recommend looking at examples from the Sandbox - here you can quickly figure out how to add an interactive map to a page with various parameters, configure the behavior of the map, or add various objects.
In the screencast about the API, I downloaded the information about the bike rental stations in CSV format, converted using QGIS these data to GeoJSON and then visualized them using the Yandex.Map API in several ways:
- simple labels with the default style; in standard styles there are still several presets ;
- tags with your badge , and since we have data on the capacity of bike rentals, you can, for example, vary the size of the badge depending on the capacity of the bike station;
- tag clustering , clustering is a very convenient thing that groups closely spaced tags into special group tags;
- rental points can be displayed as circles with a radius specified in meters;
- with the help of a special module, bike rental displays as heat maps ;
By the way, you can work with GeoJSON files in a slightly more elegant way, using ymaps.GeoQuery . For example, all loaded objects can be immediately added to the collection of geo objects and control display parameters for the entire data set.
The code for these examples can be found on GitHub in the screencast materials .
Clustering of bike rental points
I used clustering and drawing circles in the experiment with visualizing public toilets - I was very interested in showing 5-minute radii, and I used the heat map module in the draft statistics of the Moscow bike rental .
Heat map in the projectstatistics of Moscow Velobikes
This, of course, is far from all the API features. If you wish, you can compile a fairly interesting interactive project based on the Maps API. You can find interesting examples of visualizations in a blog , a development club , a sandbox , or in the documentation .
The topic of cartography is endless, and the more I immerse myself in it, the more limitless this cartographic universe seems to me. Particularly attentive readers probably noticed that the story turned out about data and about technology, and little is said about design in this story. We will go on this adventure later - for now, I am collecting materials on the topic, structuring and comprehending what has been collected.
So, the most important links for further immersion in the topic of geodata visualization.
Record screencasts:
- “ Introduction to the Course and Examples ” (2:25)
- “ Types and formats of geodata ” (3:50)
- « The Quantum GIS » - an overview of the basic features (9:51)
- " CartoDB " - an online data visualization service (11:38)
- “ MapBox TileMill ” - map styling with CartoCSS (19:27)
- “ Yandex.Map API ” - showing points on a map in JavaScript (10:25)
Links and working files for examples: github.com/minikarma/geotalk
To discuss the topic, I started a Facebook group - “ Household Mapping ”. It will be great to find like-minded people and discuss similar topics there, to share experiments and experiences.
I plan to continue to replenish the materials as far as possible and I will be glad to additions and wishes.
References
Preparing information for the course, I collected a certain set of links for independent study. I share with you. It will be great if you share something interesting with me in the comments.
Interesting examples:
- Watercolor maps
- Space station maps
- Burningmap stamen
- Pinterest maps
- Foursquare Pulse - visualization of user checks
- Map of 6 billion tweets
- Animated Weather Visualization - Earth
- 1.5 million tracks from RunKeeper
- Amsterdam buildings by years
- Taxi in New York and another visualization
- Maps Mania (formerly Google Maps Mania)
- MapBox Blog
- CartoDB Blog
- Esri Blog
- Axismaps
- Maptime.io
- Stamen design
- “Brief Introduction to GIS” translation of the book “A Gentle Introduction to GIS” into Russian from GIS-lab
- Anatomy of the Web Map - A Good Presentation on the Basics of Web Mapping
- QGIS documentation in Russian.
- A very short introduction to QGIS by Alan McConchie - presentation
- QGIS tutorials - a large selection of QGIS examples
- Maps and the Geospatial Revolution - Coursera course course on cartography
- Geodesign: Change Your World - a course on map design on Coursera, starting July-2015
- From GPS and Google Maps to Spatial Computing (Coursera)
- Ordnance Survey Location Toolkit Links
- Presentation by Andy Kirk - about data visualization in general
- Links to open geodata and OSM data extracts on the GIS-Lab website
- Metro Extracts - OpenStreetMap data for major cities from Mapzen
- Links to other projects where you can download OpenStreetMap data
- Moscow Government Open Data Portal
- data.gov.uk - United Kingdom
- data.gov - US Open Data
- NYC OpenData - New York City Municipal Open Data Site
- Useful resources on open data - a large selection of links to open data in Russia
- List of Russian government open data portal
- Geonames is an open base of geographical names (about 8 million objects).
- A good presentation about the basic data types from the maptime.io community
- Geographic coordinates - Wikipedia article
- GIS Formats - Wikipedia article in English
- JSON.org and GeoJSON.org and - more about JSON and GeoJSON formats
- GeoJSON specification - translation of the specification of the GeoJSON format on the site