How to use GoPro frame-by-frame photography for the Mapillary service

Original author: Amanzi
  • Transfer
  • Tutorial
Over the past six months, the popularity of the Mapillary web service has grown: over half a million photos have been uploaded, more than 100 thousand pieces - only in the last 10 days!

At first glance it seems that only a regular smartphone is suitable for this case. But what if you could use an action camera, for example, GoPro, to collect photographs for Mapillary? Mounted on a bicycle, helmet or even a car? As it turned out, this is possible.

And so, last week I took a short trip and collected about 1,700 photos in half an hour using the TimeLapse function. On my 32 gigabyte SD card, it only ate 3.4 gigabytes. I could still ride for more than 4 hours and take about 150 thousand photos before my memory card was full!

However, this is not a completely trivial matter, therefore, I believe, it is necessary to explain in more detail how all this is done. The main problem is that GoPro does not contain a GPS receiver. How does a map service know where to display photos on a map? We ourselves must pass this information to him, so we will have to use a separate GPS-receiver. Personally, I used my HTC One smartphone - the same one that I used to take regular photos for Mapillary.

In short, the procedure is as follows:
  • Mount the camera on a bicycle or car using a variety of mounting devices.
  • Run a program to record a GPS track. I used the geopaparazzi program, but there are many others that work just as well.
  • Set GoPro to frame-by-frame shooting. I chose the settings: 5MP, medium, narrow, 1s.
  • Launch GoPro and ride.
  • Upload photos and GPX track to computer.
  • Using gpx2exif, attach photos to geographic coordinates, synchronizing the shooting time with a GPX track.
  • Upload photos to Mapillary.

It doesn't seem to be too difficult. We will describe these actions in more detail using my trip last week as an example.

image

I mounted the GoPro on the front left fender of my Mazda RX8 to get a good view of the middle of the road.

image

Then I fixed my HTC One smartphone in the car interior to a special mount. Using the GoPro App", I joined the camera and displayed a preview of the area as GoPro sees it. This is convenient, but, in fact, it’s not really necessary, because with the start of recording, the preview will disappear. It’s only necessary that the smartphone was in the car in the place where the GPS satellite signal is well caught. If you use the GoPro App, take a little time to set the time on the camera: the difference in current time on devices should be as small as possible. As you will see later , you have to take into account the time difference in order to synchronize the track and the photo, and the smaller the correction you have to make, the easier it will be to complete the work.

image

Let's check the camera settings again: the time-lapse photos mode of single-frame shooting should be turned on. This is easy to do with the GoPro App, but you can do it on GoPro itself. I chose 5MB, Medium, Narrow, to get a view similar to the one I’m shooting with my phone. The equivalent focal length (in terms of a 35 mm camera) is approximately 20 mm. GoPro default settings provide a wide field of view, with large optical distortion, so on this trip I refused this mode. I plan to describe another scenario in another article when I took pictures for Mapillary from a wide-angle 4K video stream. It was harder, so for now skip the story.

With the selected setting 1 frame per second, the distance between the photographs will be 10-20 meters at a speed of 40-80 km / h. The Mapillary recommendation about one frame every two seconds is more suitable for a bicycle, but I plan, of course, to move faster than on a bicycle!

Launch the application for recording a GPS track. In this case, I used the geopaparazzi program . This application has a button to start recording. I pressed this button and confirmed the suggested file name to save the track. OK. Now everything is ready. It remains only to start shooting, and go!

image

Having rolled, we will stop recording on the camera and on the GPS receiver. Now the real work begins! We need to perform georeferencing before we can upload our material to Mapillary. In Geopaparazzi, I exported the track to a GPX file and sent it to myself by mail. From GoPro, I just pulled out a card and copied photos from it onto my laptop.

The first thing I wanted to do was to see what my trip looked like. I executed the command:

geotag -g väla_to_billesholm.gpx \
       -o väla_to_billesholm.png \
       -D 1 -s 2048x2048


The result is a good high-resolution image of the entire path map. note the use of the -D option to specify a larger spacing between the markers on the map. This is necessary, because geotag is configured by default for short tracks, like, for example, a fast bike ride. The resulting image shows the time and position of the key points of the trip. It is necessary to enlarge the scale in several characteristic places where we can manually check the coincidence of time on the camera and the GPS receiver and more accurately determine the clock error between them. At the synchronization stage, we will be able to correct this difference, and it is important to determine it correctly.

I executed the command:

geotag -R 20140505T12:39:00+02-20140505T12:41:00+02 \
       -g väla_to_billesholm.gpx \
       -o krop.png -D 0.1
, which generated this picture:

image

I can find a photo where I enter under the bridge and check the clock.

image

The EXIF ​​data of this photo shows that it was taken at 12:39:11. Looking at the map, we see that we passed under the tunnel at 12:39:14. Thus, the error is 3 seconds. You can use this value in the georeferencing process, but it’s best to first look at another photo.

image

I generated a map of the path through Mörarp, because I can identify objects such as buildings and intersections. You should not use the intersections where you stopped (such as my turn to the right). Look for landmarks near which you were on the move. I was looking for the first passage on the right, at the top of the map, and found a photograph taken at 12:46:10.

image

I took note of several landmarks about which I know exactly where they are: a warning sign in the middle, white markings at the intersection, a street lighting pole.

Another interesting way to check, which can also be used - if you were in a place where there are Google street view pictures, you can compare two images.

image

The Google street view shots show a road sign, markings at an intersection, a tree and a lighting pole. But note that there is no fence, Christmas trees grow in its place. It is clear that the construction began after the Google car drove in this place. Google says the picture was taken in September 2011 - about 2.5 years ago - of course, something has changed.

From the data of the GPX track through Merarp it can be seen that the intersection passed at 12:46:13, which is 3 seconds later than recorded by the camera in the photo. And again, we have an error of 3 seconds. It pleases: it turns out that the error is the same throughout the track. You can continue and synchronize all 1,500 photos using the command:

geotag -g väla_to_billesholm.gpx \
       20140505_TimeLapse/*JPG -t 3 -v


I set the time offset using the "-t -3" option and used the "-v" option to monitor the process. Since the script is a wrapper over the exif_file command-line program, several processes start processing each file; this takes some time, but at the end all your photos will contain geo-referencing information taken from the GPX.

image

When the geo-information linking to the photos is completed, you can upload the photos to mapillary.com. Enter the site, click on your name, select “Upload images”, click the “Choose files” button. After selecting all the files, scroll down the page and click on the “Start Uploading” button. The color scheme on the site is such that it is not always clear whether the download has started. Just scroll up

image

Finally, when the download is completed, click on your name, select “my uploads” 'and you will see new images for your track.

image

Click on your latest upload to view photos on Mapillary!

image

Full processing of photos may take some time, so do not be alarmed if they are not immediately available. Return to them a little later.

And now - slow motion video

Settings in GoPro for the described method of photographing are not in vain called “Time Lapse” - “slow motion”. From photos you can make a video. Since we shot one frame per second - if we make a video at a frequency of 25 frames per second, we will get 25x acceleration. This is great! See what I got:



This video was made like this ::
  • Rename all the photos so that the file names have numbers starting with 0000. I gave names like foo-0000.jpeg. To make my life easier, I wrote a ruby ​​script that creates hard links with the necessary names. You can then use the ffmpeg command to compile the video:

    ffmpeg -f image2 -i foo-%04d.jpeg \
           -r 25 -s 1280x960 ../myvid.mp4
    

  • This command compresses 7-megapixel 4: 3-format photos into a 960p HD movie.
  • Then, using the OpenShot video editor, I cropped to 16: 9, added sound, added a card and squeezed it in 720p resolution with medium quality for faster downloading to the web.


Gpx2exif installation

The geotag team has been widely used in this article. This command is included in the ruby ​​gem "gpx2exif". The article used the capabilities available in version 0.3.6. However, at the time of publication of the article, version 0.3.1 was available. Therefore, I will explain how to install the latest version.

Install Ruby on Ubuntu

The first thing you need is Ruby. Installation depends on the operating system. I use Ubuntu 14.04 and RVM, here I will give instructions suitable for my OS. And I recommend that you go to the sites ruby-lang.org and rvm.io for advice that is suitable for your platform.

sudo apt-get install curl
curl -sSL https://get.rvm.io | sudo bash -s stable --ruby
sudo usermod -g rvm craig
# logout and login to get RVM groupsource /etc/profile.d/rvm.sh

The part of the program that creates PNG images uses ImageMagick for its work. For Ubuntu, this means you need to install several dependencies first:

sudo apt-get install imagemagick imagemagick-doc libmagickwand-dev
gem install rmagick # Requires libmagickwand-dev to compile


Installation from rubygems.org

When ruby ​​is installed, just install gem:
gem install gpx2exif

Then output the list with the gem list command to find out which version was installed as a result. If it is older than 0.3.5, then follow the instructions below.

Installation with github

Install git and then execute the commands:

git clone https://github.com/craigtaverner/gpx2exif
cd gpx2exif
bundle install
rake build
gem install pkg/gpx2exif-0.3.6.gem

If everything went well, then you have built and installed the latest version of Ruby Gem.

Remarks from the translator :
1) What is Mapillary? As Zverik put it : "Mapillary for snapshots is like OpenStreetMap for maps." In short, this is an analogue of Google street view and Yandex panoramas, working on the principle of crowdsourcing. Users upload their geo-referenced photos, which are then displayed on the map along with the line traveled. The content license allows the use of information obtained from photographs in OpenStreetMap.
2) For geo-referencing photos, there is also such a Python script: https://github.com/mapillary/mapillary_tools/blob/master/python/geotag_from_gpx.py
3) I learned about everything mentioned in points 1 and 2 from article " The People's pan 'in user blog Zverik.
4) The most convenient way to record the discrepancy in the clock is to photograph the clock on the GPS device.

Also popular now: