Adding more layers for overture derived tiles

[michaelkirk]

We currently use planetiler to generate tiles for headway (a self hosted map stack) using osm.pbf as input.

For https://maps.earth (our headway deployment) that input data comes from https://daylightmap.org 's planet.pbf.

With the pending shutdown of https://daylightmap.org (https://daylightmap.org/2024/05/03/sunsetting-daylight.html), I've been looking at overture as a potential successor for QA'd source data.

I've successfully used https://github.com/onthegomap/planetiler-examples/blob/main/Overture.java as a starting point, but that example profile includes only a few layers - e.g. it doesn't have a POI layer or land use layer. Similar data exists in overture in some form or another (though I think there may be some significant differences in the data).

Is there an existing effort to implement more of the openmaptiles schema from overture? Or interest in incorporating such a thing?

[acalcutt]

Just an FYI, there is also https://github.com/onthegomap/planetiler-examples/blob/main/OvertureLandcover.java that has landcover layers from overture. Based on https://gist.github.com/dschep/9a4c875715e62c6b8e7d5697e33780d4 I used this code in my style.

    {
      "id": "land_cover",
      "type": "fill",
      "source": "overture_land_cover",
      "source-layer": "land_cover",
      "layout": {
        "visibility": "visible"
      },
      "paint": {
        "fill-outline-color": ["rgba", 0,0,0,0],
        "fill-color": [
          "case",
          [ "==", [ "get", "subtype" ], "urban" ],
          ["rgb",167, 162, 186],
          [ "==", [ "get", "subtype" ], "forest" ],
          ["rgb",134, 178, 137],
          [ "==", [ "get", "subtype" ], "barren" ],
          ["rgb",245, 237, 213],
          [ "==", [ "get", "subtype" ], "shrub" ],
          ["rgb",239, 218, 182],
          [ "==", [ "get", "subtype" ], "grass" ],
          ["rgb",254, 239, 173],
          [ "==", [ "get", "subtype" ], "crop" ],
          ["rgb",222, 223, 154],
          [ "==", [ "get", "subtype" ], "wetland" ],
          ["rgb",158, 207, 195],
          [ "==", [ "get", "subtype" ], "mangrove" ],
          ["rgb",83, 171, 128],
          [ "==", [ "get", "subtype" ], "moss" ],
          ["rgb",250, 230, 160],
          [ "==", [ "get", "subtype" ], "snow" ],
          ["rgb",255, 255, 255],
          ["rgba",255, 255, 255, 0]
        ]
      }
    },

You can see this landcover mixed with OSM data at
https://tiles.wifidb.net/styles/WDB_OSM_LAND/#6/42.378/-71.927

In the past alpha version of overture I had started to make a rough style that was a mix between osm basic and protomaps
https://github.com/acalcutt/wifidb-tileserver-gl/tree/master/tileserver-gl/styles/WDB_BASIC_OVERTURE
but you can see at https://tiles.wifidb.net/styles/WDB_BASIC_OVERTURE/#6/42.378/-71.927 it needs a bit of work with label clipping, though I am not sure if the newer overture profile improved things since that was made on the old alpha data and overture profile before it was merged.

[acalcutt]

It looks like my alpha version was based on https://github.com/onthegomap/planetiler/blob/overture-generic/planetiler-core/src/main/java/com/onthegomap/planetiler/overture/Overture.java which looks quite different from the newer example. It did have landuse, which I used in my style

It was from https://osmus.slack.com/archives/C031V9E9RMG/p1703037524824199

[msbarry]

Hello! There are a couple of different options here that I can think of:

1. Create a brand new overture profile from scratch: This would be continuing with what I started with Overture.java and might not be that bad since Overture already does a lot of the work that openmaptiles has to do to sort and filter features into thematic layers so it's mostly deciding on appropriate min zooms for features in each layer. This wouldn't be compatible with existing openmaptiles styles, but it would also let you ditch the extra copyright notice and cruft that's accrued in the schema over time.
2. Modify openmaptiles profile to add a flag that pulls data from overture: This would maintain compatibility with existing styles, and could be done one layer at a time, for example for onthegomap.com I made my openmaptiles fork use OSM for all layers, except pull buildings from Overture.
3. Add an Overture-to-OSM-compatibility layer inside planetiler: This would map Overture elements and attributes to what the canonical tags would look like if they came from OSM and pass that through so any OSM-based profile could switch over to using Overture. This would be quite a bit of work, but benefit all planetiler users on whatever schema they are using
4. Create an Overture-to-OSM.pbf translation tool: Similar to (3) except this would write out an osm.pbf from overture data using the same rules to generate canonical osm tags that (3) would have used on an in-memory transformation. (3) could easily be turned into (4) by just writing elements to disk instead of just passing them through a profile. This would be the most work (initial build-out and ongoing maintenance as Overture evolves), but have maximum benefit for all tools that currently process OSM data.

Thinking long-term I think (1) would be ideal to start to give people more options than just openmaptiles. You could also imagine an OSM.pbf-to-Overture translation tool that would let you plug live data into a pipeline instead of Overture snapshots, which would also be more straighforward to build as Overture publishes their translation rules in a machine-readable format.

For my own purposes, I'd probably continue on the path of (2) since it's the fastest way to start getting incremental benefit from incorporating bits of Overture data. I could contribute changes from my fork upstream.

But if someone else were willing to help then any of these options would be possible. What do you think?

[michaelkirk]

Thank you for the thorough response!

(1) ... would also be more straighforward to build as Overture publishes their translation rules in a machine-readable format.

I didn't know that. Can you link me to anything saying more about this?

To be honest, my initial instinct was (4.) — it'd be great to be able to build an .osm.pbf from overture data, and then continue to use the myriad of existing tools that accept osm.pbf.

But I have some hangups that might be more personal than technical here. For example, I'm still invested in supporting OSM as the ultimate source of truth. For Buildings, OSM entries are preferred, and ML buildings are only used if one doesn't exist in OSM, which works for me.

But for places:

The [Places] theme is derived from a conflation of Meta and Microsoft data

So I don't have a clear cut way to direct my users to "fix this data".

It still might be worthwhile to pursue, I'm just kind of thinking out loud.

As my first baby steps into planetiler, I think I'll start by trying to add some layers to https://github.com/onthegomap/planetiler-examples/blob/main/Overture.java (approach 1.)

And then if that goes well, maybe I'll try to see about extracting that into something that could be reusable for (approach 2.).

At the end of the day, I want to be able to use overture data for the tiles on https://maps.earth

[msbarry]

I didn't know that. Can you link me to anything saying more about this?

They publish these sql queries that map OSM data to overture: https://github.com/OvertureMaps/schema/blob/main/task-force-docs/base/land_use.sql - worst case you would have to port that, but ideally that would be a more language-independent format that other tools could read.

I'm also not sure how practical (4) is until more kinds of data from OSM are reflected in overture, or at least until OSM POIs are conflated with meta/microsoft like buildings. That's why I'm thinking a hybrid approach that replaces selective features from OSM with overture (ie. water, buildings) might be a practical compromise. I suppose you could also have a tool that takes OSM.pbf + overture and returns a new OSM.pbf with selective features added/replaced?

As my first baby steps into planetiler, I think I'll start by trying to add some layers to https://github.com/onthegomap/planetiler-examples/blob/main/Overture.java (approach 1.)

Sounds good, there are 2 versions of that one. The one you linked requires building with maven, whereas this one lets you modify/run the java profile without any build tools: https://github.com/onthegomap/planetiler-examples/blob/main/Overture.java. I'm thinking we might want to standardize on that one?