Skip to content

Latest commit

 

History

History
62 lines (42 loc) · 3.24 KB

atlas_map_pipeline.md

File metadata and controls

62 lines (42 loc) · 3.24 KB

Atlas OSM Map Pipeline

  • Matt Young.
  • Henry Batt.

Introduction

This document contains the full pipeline for extracting map data from OpenStreetMap and importing it into Atlas.

Downloading OSM Brisbane extract

Download and save the extract of Brisbane: click https://github.com/DECO3801-Segfault-Coredump/atlas_data_raw/raw/master/map/brisbane.osm.pbf

If teh above link does not work, try this one instead: https://www.dropbox.com/scl/fi/7bo3ktiiwzl11ez35mhtb/brisbane.osm.pbf?rlkey=ggwu5f7tdsm3m8myy9s05m6zv&dl=0

You should save this somewhere easy to access with its original name (brisbane.osm.pbf). For example, /home/matt/Downloads/brisbane.osm.pbf

This was generated by me using the method below, but you can just download and import the above version to save you some time :)

Matt's notes

The extract I used for Brisbane is: https://app.protomaps.com/downloads/osm/ed673440-97cd-4a6e-a7ec-082c12616543

  1. Go to https://app.protomaps.com/downloads/osm
  2. Draw bounding polygon for Brisbane
  3. Click download to get a .osm.pbf (OpenStreetMap binary Protobuf file)

Serving tiles

  1. Create data volume: docker volume create osm-data
  2. Create tile cache volume: docker volume create osm-tiles
  3. Import data: docker run -v <PATH_TO_DOWNLOADED_EXTRACT>/brisbane.osm.pbf:/data/region.osm.pbf -v osm-data:/data/database/ overv/openstreetmap-tile-server import (you'll need to change paths for wherever you saved "brisbane.osm.pbf")
  4. Serve tiles: docker run --name atlas-tileserver -p 8080:80 -p 5432:5432 -e THREADS=$(nproc) -v osm-data:/data/database -v osm-tiles:/data/tiles -d overv/openstreetmap-tile-server run (just run this once)
  5. Access the tile server locally at http://localhost:8080/ and zoom in on Brisbane

Based on instructions here: https://switch2osm.org/serving-tiles/using-a-docker-container/

Thanks to Henry for fixing the old Docker command infinitely allocating new containers 💀

(Optional, but recommended) Pre-rendering tiles

The map can be pretty slow on the first run. We can pre-render the tiles to save time. Note, this is necessary for good UX

  1. Login to the container: docker exec -it atlas-tileserver /bin/bash
  2. Download the script: cd /tmp && wget https://raw.githubusercontent.com/alx77/render_list_geo.pl/master/render_list_geo.pl && chmod +x render_list_geo.pl
  3. Generate tiles: time ./render_list_geo.pl -n 32 -z 11 -Z 20 -y "-27.780999227328973" -Y "-26.899691691210972" -x "152.741860574895" -X "153.37445809991218"

Warning: This will take a significant amount of time (1h on a very powerful desktop), consume all your CPU and also use about 10 GB of disk space. I suggest running it overnight. It's worth it though, as it makes the map much more responsive and reduces OSM CPU usage when running Atlas.

You should also change -n 32 to however many CPU cores you have, I have 32 so I used that.

Matt's notes

min coord: -27.780999227328973, 152.741860574895 max coord: -26.899691691210972, 153.37445809991218\

note for some reason lat/long has to be swapped for this stupid ass script

./render_list_geo.pl -n 32 -z 11 -Z 20 -y "-27.780999227328973" -Y "-26.899691691210972" -x "152.741860574895" -X "153.37445809991218"

Source: Overv/openstreetmap-tile-server#15