Geographical data is amazing, if you have a decent dataset you can find out a lot of things by running a diverse range of analysis. Most of the times you will be using shapefiles in order to analyze geographical data but the standard filetype to display geographical data in web browsers is GeoJSON. We will create a simple layer using a complex GeoJSON file.

In this tutorial we are going to be using GeoJSON which is a format for encoding a variety of geographic data structures, as you may already be guessing GeoJSON is based on JavaScript Object Notation. If you want a better understanding of how GeoJSON works you should check the GeoJSON Format Specification. You can find the code at github but you will have to create your own GeoJSON (shown in this tutorial).

The dataset

First of all we need to find a nice dataset, we will be using geojson-vt a highly efficient JavaScript library for slicing GeoJSON data into vector tiles on the fly, primarily designed to enable rendering and interacting with large geospatial datasets on the browser side. I live in Mexico and one nice dataset is the land use for our country, you can find the file of Mexico’s land use at the INEGI’s website.

Lets download the file and unzip it:

1
2
3
4
5
6
7
8
unzip usoSueloVegetacion.zip -d uso
# Archive: usoSueloVegetacion.zip
# inflating: uso/Uso del suelo y vegetacion.shp
# inflating: uso/Uso del suelo y vegetacion.shx
# inflating: uso/Uso del suelo y vegetacion.dbf
# inflating: uso/Leeme.txt
# inflating: uso/retrieve.htm
# inflating: uso/Uso del suelo y vegetacion.prj

One issue with this dataset is that the current projection is not the one we need to use in order to render in leaflet. In order to change the projection we may use ArcGIS or a similar geospatial tool, in this case we want to avoid as much as possible using tools that are an overkill for the job we want to get done. A great tool that helps with the transformation of data is mapshaper.

We can install mapshaper by running npm install -g mapshaper. This will install the command line tool and the mapshaper gui, lets use the mapshaper-gui so we can explore the data we just downloaded. Run mapshaper-gui and load the shapefile. We should see something like this:

Thats Mexico and the land use layer which INEGI created, thats perfect but we still need to know if the projection used in this shapefile is compatible with the approach we are going to use to render the data in the browser. If we use mapshaper’s command line tool we can get a little bit of information of the projection and a sneek peak of the data we have:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
mapshaper 'Uso del suelo y vegetacion.shp' -info
# Layer name: Uso del suelo y vegetacion
# Records: 26,866
# Geometry
# Type: polygon
# Bounds: 572600.2255049599 124.6882307715714 4427354.750182984 2767389.2458159914
# Proj.4: +proj=lcc +x_0=2500000 +lon_0=-102 +lat_1=17.5 +lat_2=29.5 +lat_0=12 +ellps=GRS80
# Attribute data
# Field First value
# AREA 17478.5
# CLAVEFOT 'H2Om'
# ENTIDAD 'CUERPO DE AGUA'
# EROSION 'No aplicable'
# FC 6212
# OBJECTID 1
# PERIMETER 648.272
# REVISION_ 2
# REVISION_I 13479
# SHAPE_area 17484.9065181
# SHAPE_len 647.982509239
# TIPO 'Cuerpo de Agua Perenne maritimo'
# VEGSEC 'No aplicable'

We find that the projection is not the one we’ll be needing, we need our data to be projected using WGS84. We can export the file to a GeoJSON format and simpify while we are changing the projection. We should also just store the attributes we care for. The attribute TIPO is the one that we are interested in so lets keep that in mind. If you are interested in getting to know the options mapshaper offers you should go check the documentation.

1
2
3
4
mkdir wgs84
mapshaper 'Uso del suelo y vegetacion.shp' -proj wgs84 -simplify 30% \
-filter-fields TIPO -o format=geojson ./wgs84
# Wrote wgs84/Uso del suelo y vegetacion.json

alt text

We are changing the projection to WGS84, simplyfying to a 30% (Percentage of removable points to retain. Accepts values in the range 0%-100% or 0-1), selecting the attribute data TIPO so it can be stored in the properties of our features and finally exporting to the GeoJSON format. If we load the newly projected data in the mapshaper-gui we should see how the projection changed:

alt text

Our projection is now correct and we have our GeoJSON ready, we still need to find all the values of the attribute data TIPO and create a set with those. Lets use the each flag provided by mapshaper and some shell utilities to find the unique values present in this attribute:

1
2
3
4
5
6
7
8
9
10
11
mapshaper 'Uso del suelo y vegetacion.shp' -each 'console.log(TIPO)' | sort | uniq
# Agricultura de Humedad
# Agricultura de Riego
# Agricultura de Riego Eventual
# Agricultura de Temporal
# Agricultura de Temporal, Pastizal cultivado
# Agricultura de Temporal, Pastizal inducido
# Agricultura de Temporal, Selva Baja Caducifolia
# Agricultura de Temporal, Selva Mediana Subcaducifolia
mapshaper 'Uso del suelo y vegetacion.shp' -each 'console.log(TIPO)' | sort | uniq | \
sed 's/\(.*\)/"\1",/g' > types.txt

Keep the files we generated (GeoJSON and types.txt), we can now go and start building our GeoJSON explorer with geojson-vt.

You can export and simplify using either the mapshaper-gui or the command line interface. I prefer using the gui since you can have a visual feedback of how is the data being simplified and you can control how much you are willing to give in order to reduce the size of the file.
geojson-vt

If we want to use geojson-vt we need to create a javascript application. Use this package.json to start our project. The important parts here are the scripts and dependencies, the start script will run the webpack-dev-server without webpack-dev-server or webpack being installed globally. Don’t forget to run npm install to install all the dependencies in our package.json

package.json
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
{
"name": "dc-003-geojson-vt",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"start": "webpack-dev-server",
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "Omar Eduardo Torres Chavez <omar@disciplinecode.com> (http://disciplinecode.com)",
"license": "MIT",
"dependencies": {
"axios": "^0.15.3",
"babel-core": "^6.21.0",
"babel-loader": "^6.2.10",
"babel-preset-es2015": "^6.18.0",
"d3-scale": "^1.0.4",
"geojson-vt": "^2.4.0",
"leaflet": "^1.0.2",
"webpack": "^1.14.0",
"webpack-dev-server": "^1.16.2"
}
}

Lets create a folder structure like the following:

1
2
3
4
5
6
7
8
9
10
.
+-- dc-003-geojson-vt/
| +-- dist/
| +-- node_modules/
| +-- shapefiles/
| +-- src/
| | +-- static/
| | +-- index.js
| +-- package.json
| +-- webpack.config.js

  • dist: Ths folder will contain our bundle and the files needed to run the application.
  • node_modules: The folder in which all our dependencies will be installed
  • shapefiles: The folder containing the shapefiles (This is optional)
  • src: The folder contains all the files for the development of our application.
    • static: The content of this folder will be copied automatically to our dist folder on npm start or npm run build.
    • index.js: The entrypoint of the application.
  • package.json: The package.json we previously created.
  • webpack.config.js: webpack configuration for development.

Copy the GeoJSON previously created into the static folder.
Create an index.html file in the static folder.

Use the following template to populate our index.html:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="x-ua-compatible" content="ie=edge">
<title>geojson-vt</title>
<meta name="description" content="">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- <link rel="apple-touch-icon" href="apple-touch-icon.png"> -->
<!-- Place favicon.ico in the root directory -->
<link rel="stylesheet" href="https://unpkg.com/leaflet@1.0.2/dist/leaflet.css" />
<style>
html, body {
margin: 0;
padding: 0;
height: 100%;
}
#geojson__container {
height: 100%;
width: 100%;
}
</style>
</head>
<body>
<!--[if lte IE 9]>
<p class="browserupgrade">You are using an <strong>outdated</strong> browser. Please <a href="http://browsehappy.com/">upgrade your browser</a> to improve your experience and security.</p>
<![endif]-->
<div id="geojson__container"></div>
<script src="bundle.js"></script>
</body>
</html>

Our html has some important parts:

  • Requiring leaflet.css: We are calling the leaflet CSS so our maps can be nicely rendered.
  • Adding simple CSS: We will include a little bit of CSS in our header in order to avoid the creation of another file just to make simple style modifications.
  • Create container div: We create a div with id ‘geojson__container’ which will be the container of our map.
  • Requiring bundle.js: We are calling the bundle we will create in a bit.

Lets create a webpack.config.js file so we can create our bundle (I assume you have a little webpack knowledge):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
const path = require('path');
const CopyWebpackPlugin = require('copy-webpack-plugin');
module.exports = {
entry: [
'webpack-dev-server/client?http://localhost:9000/',
path.resolve(__dirname, 'src/index.js')
],
output: {
path: path.join(__dirname, 'dist'), // Output path
filename: 'bundle.js', // Name of the bundle file generated
},
devServer: {
port: 9000, // Development server port
host: '0.0.0.0', // Host
outputPath: path.join(__dirname, 'dist')
},
plugins: [
new CopyWebpackPlugin([
{
from: path.join(__dirname, 'src/static'),
to: path.join(__dirname, 'dist')
}
])
],
target: 'web',
module: {
loaders: [
{
test: /\.js$/,
exclude: /node_modules/,
include: [ // Include just the necessary files
path.resolve(__dirname, 'src/index.js')
],
loader: 'babel',
query: { presets: ['es2015'] }
}
]
}
};

Lets add the following into our index.js file:

src/index.js
1
2
3
4
5
6
7
8
9
10
11
12
13
// imports
import { map, latLng, TileLayer, GridLayer, DomUtil } from 'leaflet';
// create map instance
const mymap = map('geojson__container', {
center: L.latLng(19.432904,-99.1568927),
zoom: 12
});
// setup layer
const osmUrl='http://{s}.basemaps.cartocdn.com/light_all/{z}/{x}/{y}.png';
const osmAttrib = '&copy; <a href="http://www.openstreetmap.org/copyright">OpenStreetMap</a> contributors, &copy; <a href="https://carto.com/attributions">CARTO</a>'
const osm = new TileLayer(osmUrl, {minZoom: 2, maxZoom: 15, attribution: osmAttrib});
// add layer to map
mymap.addLayer(osm);

The previous snippet should render a map with tiles from Carto.

Now lets call our GeoJSON file with axios, in our webpack configuration we stated that our static files will be at the folder dist. So our bundle.js will be able to find our GeoJSON at ‘./uso.json’

src/index.js
1
2
3
4
5
6
7
8
// imports
import { map, latLng, TileLayer, GridLayer, DomUtil } from 'leaflet';
import axios from 'axios';
// ...
axios.get('./uso.json') // Call our GeoJSON
// Object {type: "FeatureCollection", features: Array[26866]}
.then(response => console.log(response.data))
.catch(err => console.log(err));

We have our GeoJSON loaded, we can now use geojson-vt! Lets declare a function expression which will handle the creation of our index of tiles, lets import the geojsonvt module at the top of our file and create our tile index in the function expression body. In order to create the tile index you will only need to give as firt argument the GeoJSON and as second argument a options object. You can tweak the options object, but the defaults should work with the dataset we are using.

src/index.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
// imports
import { map, latLng, TileLayer, GridLayer, DomUtil } from 'leaflet';
import axios from 'axios';
import geojsonvt from 'geojson-vt';
// ...
// declare a function expression
const createLayer = response => {
const data = response.data;
const tileIndex = geojsonvt(data, {
maxZoom: 12, // max zoom to preserve detail on
tolerance: 3, // simplification tolerance (higher means simpler)
extent: 4096, // tile extent (both width and height)
buffer: 64, // tile buffer on each side
debug: 0, // logging level (0 to disable, 1 or 2)
indexMaxZoom: 4, // max zoom in the initial tile index
indexMaxPoints: 1000000, // max number of points per tile in the index
solidChildren: false // whether to include solid tile children in the index
});
};
//
axios.get('./uso.json')
.then(createLayer)
.catch(err => console.log(err));

alt text
How To: Bing Maps Custom Tile Overlay – Google Maps, viewed 27 December 2016, http://blogs.microsoft.co.il/shair/2012/06/24/how-to-bing-maps-custom-tile-overlay-google-maps.

A layer is composed of a group of tiles, each tile will be a square of 256 pixels x 256 pixels. When we zoom a map each succeeding zoom level will divide the map into 4 N tiles, where N refers to the zoom level. The leaflet docs explain that in order to create a custom layer, we must extend GridLayer and implement the createTile() method, which will be passed a Point object with the x, y, and z (zoom level) coordinates to draw our tile.

src/index.js
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
const createLayer = response => {
// ...
const CanvasLayer = GridLayer.extend({
createTile: function(coords) { //
// console.log(coords); { x: Number, y: Number, z: Number }
// leaflet will run this method for each tile needed in the viewport
// create a canvas (we will use this canvas to draw our features)
let tile = DomUtil.create('canvas', 'leaflet-tile leaflet-sedesol');
// Set the tile size
tile.width = 256;
tile.height = 256;
// at the moment we will just return an empty canvas
return tile;
}
});
// Add layer to the canvas
mymap.addLayer(new CanvasLayer());
};

If you inspect the html you can find how our tiles are being rendered, navigate to the div with class leaflet-tile-pane. You will see that this node has two child nodes with class leaflet-layer, the first child will be the container of the tiles from Carto. The second child will be our tiles, navigate the child nodes and try to understand whats happening, just remember that at the moment all the canvas we created are empty.

1
2
3
4
5
6
7
8
9
<div class="leaflet-tile-container leaflet-zoom-animated" style="transform: translate3d(0px, 0px, 0px) scale(1);">
<canvas class="leaflet-tile leaflet-geojson-vt leaflet-tile-loaded" width="256" height="256" style="width: 256px; height: 256px; transform: translate3d(47px, -136px, 0px); opacity: 1;"></canvas>
<canvas class="leaflet-tile leaflet-geojson-vt leaflet-tile-loaded" width="256" height="256" style="width: 256px; height: 256px; transform: translate3d(-209px, -136px, 0px); opacity: 1;"></canvas>
<canvas class="leaflet-tile leaflet-geojson-vt leaflet-tile-loaded" width="256" height="256" style="width: 256px; height: 256px; transform: translate3d(47px, 120px, 0px); opacity: 1;"></canvas>
<canvas class="leaflet-tile leaflet-geojson-vt leaflet-tile-loaded" width="256" height="256" style="width: 256px; height: 256px; transform: translate3d(-209px, 120px, 0px); opacity: 1;"></canvas>
<canvas class="leaflet-tile leaflet-geojson-vt leaflet-tile-loaded" width="256" height="256" style="width: 256px; height: 256px; transform: translate3d(-209px, -392px, 0px); opacity: 1;"></canvas>
<canvas class="leaflet-tile leaflet-geojson-vt leaflet-tile-loaded" width="256" height="256" style="width: 256px; height: 256px; transform: translate3d(47px, -392px, 0px); opacity: 1;"></canvas>
<!-- more canvas elements -->
</div>

We will use the getTile method of the geojson-vt tileIndex object so we can get the features that are within the tile we are fetching. Lets understand this a little bit better:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
// The coords passed into our createTile method will contain x, y and z.
// The z will be the zoom
// The x will be the
const tile = tileIndex.getTile(10, 228, 455);
// this will get the tile at zoom 10, x 228 and y 455
// we want to render our features, but how are we supposed to get them?
// if we use our tile variable we will be able to see if our tile has any data
// available, if there is no data available the getTile method will return
// null
// in case there is data available we should get an object like the following
{ features: Array[1], numPoints: 5, numSimplified: 5, numFeatures: 13, source: null… }
// in the object returned by our tileIndex we have an array of features
// if we inspect this array we will see that each object has the following values
{ geometry: Array[1], tags: Object, type: 3 }
// so now we know where our features and geometries are located
// lets find a way in which we can render the features in our canvas

Remember that every tile we are creating is a canvas, which will allow us to draw on them with the Canvas API. Explaining how the canvas API works is not covered in this tutorial, you can check this tutorial if you want to deepen your knowledge in this subject. Lets create a simple example on how to draw a feature in our canvas:

Lets explain a bit of the code found in the jsfiddle. Each feature has a geometry, which is an array of arrays. Each element of the array has an array with two values a x and a y. We are using a sample geometry to illustrate how the canvas rendering will work. We then need to get the context of the canvas.

The canvas is initially blank. To display something, a script first needs to access the rendering context and draw on it. The <canvas> element has a method called getContext(), used to obtain the rendering context and its drawing functions. getContext() takes one parameter, the type of context. For 2D graphics, such as those covered by this tutorial, you specify “2d” to get a CanvasRenderingContext2D.

When we have the context of the canvas we can start modifying it:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
const geometry = [ // The geometry of one of our features
[-64, 4160],
[-64, 3735],
[105, 3783],
// ...
];
const ctx = document.getElementById('test').getContext('2d'); // get the context
ctx.strokeStyle = '#c1c1c1'; // set the stroke color
ctx.fillStyle = '#e65e5e'; // set fill style
ctx.beginPath(); // start path
// Add instructions
geometry.forEach((point, index) => {
const pad = 0;
const extent = 4096; // Extent declared in our options object for geojson-vt
const x = point[0] / extent * 256; // Calculate relative x
const y = point[1] / extent * 256; // Calculate relative y
if (index) ctx.lineTo(x + pad, y + pad); // If index is truthy render line
else ctx.moveTo(x + pad, y + pad); // If index is falsy just move
});
// fills the current or given path with the current
// fill style using the non-zero or even-odd winding rule.
// https://en.wikipedia.org/wiki/Even%E2%80%93odd_rule
ctx.fill('evenodd');
// strokes the current or given path with the current stroke
// style using the non-zero winding rule.
ctx.stroke();

alt text
Dobot: Robotic Arm for Everyone! Arduino & Open Source, viewed 27 December 2016, https://www.kickstarter.com/projects/dobot/dobot-robotic-arm-for-everyone-arduino-and-open-so.

In order to keep it simple we will create a mapping for our variables and colors, the logic behind the color will be your choice.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
// Select the types of land use you are interested from our types.txt and assign a color
const colorMappings = {
"Agricultura de Humedad": '#454259',
"Agricultura de Riego": '#45516A',
"Agricultura de Riego Eventual": '#426279',
"Agricultura de Temporal": '#3C7387',
"Agricultura de Temporal, Pastizal cultivado": '#348590',
"Agricultura de Temporal, Pastizal inducido": '#2F9697',
"Agricultura de Temporal, Selva Baja Caducifolia": '#34A799',
"Agricultura de Temporal, Selva Mediana Subcaducifolia": '#44B897',
"Agricultura de Temporal, Vegetacion secundaria de Selva Alta Perennifolia": '#5CC893',
"Agricultura de Temporal, Vegetacion secundaria de Selva Baja Caducifolia": '#7AD88C',
"Agricultura de Temporal, Vegetacion secundaria de Selva Mediana Subcaducifola": '#9CE683',
"Agricultura de Temporal, Vegetacion secundaria de Selva Mediana Subperennifolia": '#C1F37B',
};

Now lets really dig into the tile creation!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
// ...
const CanvasLayer = GridLayer.extend({
createTile: function(coords) {
let tile = DomUtil.create('canvas', 'leaflet-tile leaflet-geojson-vt');
tile.width = 256;
tile.height = 256;
// Get context of our current canvas
const ctx = tile.getContext('2d');
// Based in the coords get the tile we want to render
const tileToRender = tileIndex.getTile(coords.z, coords.x, coords.y);
// Method of the Canvas 2D API sets all pixels in the rectangle defined
// by starting point (x, y) and size (width, height) to transparent
// black, erasing any previously drawn content.
ctx.clearRect(0, 0, tile.width, tile.height);
// If tileToRender is null return just a clear canvas
if (!tileToRender) {
return tile;
}
// If tileToRender is not null find all the features
const features = tileToRender.features;
// Set the stroke for the polygons
ctx.strokeStyle = 'grey';
// Iterate features
features.forEach(feature => {
// Find all geometries for the feature
const geometries = feature.geometry;
// Set a color based on the TIPO attribute
let featureColor;
if(feature.tags.TIPO) {
// Use our mapping
featureColor = colorMappings[feature.tags.TIPO];
}
// If TIPO couldn't be mapped return alpha 0 rgba
ctx.fillStyle = featureColor || 'rgba(0,0,0,0)';
// ctx.globalAlpha = 0.5; (optional)
// Start path
ctx.beginPath();
// Iterate geometries
geometries.forEach(geometry => {
const type = geometry.type;
// Iterate points
geometry.forEach(ctxDrawPolygon.bind(null, ctx));
});
// Fill
ctx.fill('evenodd');
// Strokes the current or given path
ctx.stroke();
});
// Return the canvas
return tile;
}
});
// Use the same principle as the jsfiddle
const ctxDrawPolygon = (ctx, point, index) => {
const pad = 0;
const extent = 4096;
const x = point[0] / extent * 256;
const y = point[1] / extent * 256;
if (index) ctx.lineTo(x + pad, y + pad)
else ctx.moveTo(x + pad, y + pad)
};

You should get something like this:
alt text

#Bundling

Ok, so we have everything we need, we just need to bundle our project. Run npm run build

1
2
3
4
5
6
7
8
9
10
11
12
> webpack
Hash: 2065a4d1f0fb6c6d2758
Version: webpack 1.14.0
Time: 3943ms
Asset Size Chunks Chunk Names
bundle.js 698 kB 0 [emitted] main
index.html 1.02 kB [emitted]
uso.json 57 MB [emitted]
[0] multi main 40 bytes {0} [built]
+ 110 hidden modules

That’s it, you now can watch your dataset in your browser. I hope you find this useful, if you have any doubts you can ask anything at my twitter account @torresomar