Metal Renderer: iOS & macOS Early Access

[louwers]

Warning
Metal on iOS is not production-ready yet. But we are interested to hear your feedback!

Long-awaited Metal support MapLibre Native has landed! You can download a pre-release (in the form of an XCFramework) and use it in your project. More details below. We are working on an automated release to the Swift Package Index and to Cocoapods.

This thread is used to coordinate testing.

Changes

• The internal rendering architecture of MapLibre Native has been overhauled. Please refer to the following documents for more details:
  • Rendering Modularization Design Proposal
  • Metal Port Design Proposal
  • Rendering Modularization Design Results (recent)
• The prefix MGL has been changed to MLN. See the example below.
• The iOS SDK has received very little changes otherwise, with the exception that (custom) OpenGL layers are (obviously) no longer supported. A new way to write custom layers is in the works (CustomGL Layer Path #1272).

Usage Instructions

See this comment.

Known Issues

• 4 render tests are failing. Re-enable ignored metal render tests #1899
• Snapshotter functionality still needs to be implemented. Metal Snapshotter #1862

All issues with metal + bug labels

Reporting Issues

When report an issue, the following are required:

• Style: If your style is public or can be made available, that will help a lot.
• Lat / Long / Zoom: Where exactly does the issue occur?
• Screenshot
• iOS version
• MapLibre Native version
• Device

When available:

• Stack Trace / Metal Error / Core Dump
• Minimal style: Can you provide a minimal style that reproduces the issue?

[sjg-wdw]

I still think "early access" sounds cooler.

[louwers]

@sjg-wdw Fixed. 😄

[ianthetechie]

✌️

[georgbachmann]

Sweet

[nnhubbard]

Sounds good!

[louwers]

Instructions for trying out MapLibre Native with Metal support below.

1. Download the latest pre-release. We push out new releases on every merge to main. Be sure to include the version, which includes a git SHA, when reporting issues.

2. Create a new iOS project with Xcode.
3. Unzip the XCFramework.
4. Follow this guide to add the XCFramework to the project (make sure to select "Embed & Sign").
5. Create a new SwiftUI view called MapView. Paste the following code:

import SwiftUI
import MapLibre

struct MapView: UIViewRepresentable {
    func makeUIView(context: Context) -> MLNMapView {
        // Build the style URL
        let styleURL = URL(string: "https://demotiles.maplibre.org/style.json")

        // Create the map view
        let mapView = MLNMapView(frame: .zero, styleURL: styleURL)

        mapView.logoView.isHidden = false

        mapView.setCenter(
            CLLocationCoordinate2D(
                latitude: 23.16, longitude: -109.50), animated: false)

        mapView.setZoomLevel(1, animated: false)
        return mapView

    }

    func updateUIView(_ mapView: MLNMapView, context: Context) {
        // Update the view if needed
    }
}

Note that the prefix was changed from MGL to MLN. You also need to import the library with import MapLibre.

6. Add the MapView to the root scene of the app.

@main
struct MapLibreTestAppApp: App {
    var body: some Scene {
        WindowGroup {
          MapView()
        }
    }
}

}

7. Build & Run!

Advanced: Building MapLibre Native Yourself

We have come a long way making it easier to set up a development environment for MapLibre Native iOS development. If you want to build the library yourself, make sure bazelisk is installed and use:

git clone --recurse-submodules https://github.com/maplibre/maplibre-native.git
cd maplibre-native
npm install --ignore-scripts
cp platform/ios/bazel/example_config.bzl platform/ios/bazel/config.bzl
bazel build //platform/ios:MapLibre.dynamic --spawn_strategy=local --//:renderer=metal

Now there will be a XCFramework in bazel-bin/platform/ios/MapLibre.dynamic.xcframework.zip.

To open the project in Xcode:

bazel run //platform/ios:xcodeproj --@rules_xcodeproj//xcodeproj:extra_common_flags="--//:renderer=metal"
xed platform/ios/MapLibre.xcodeproj

[birkskyum]

Using the pre-compiled version worked beautifully 👍

When trying to build myself, in the bazel build step I get the below errors, but changing unary_function to __unary_function as they suggest allow the compile to complete:

ERROR: /Users/admin/repos/maplibre-native/BUILD.bazel:89:11: Compiling src/mbgl/util/http_header.cpp failed: (Exit 1): wrapped_clang_pp failed: error executing command (from target //:mbgl-core) external/local_config_cc/wrapped_clang_pp '-D_FORTIFY_SOURCE=1' -fstack-protector -fcolor-diagnostics -Wall -Wthread-safety -Wself-assign -fno-omit-frame-pointer -O0 -DDEBUG '-std=c++11' ... 

vendor/boost/include/boost/container_hash/hash.hpp:130:33: error: no template named 'unary_function' in namespace 'std'; did you mean '__unary_function'?
        struct hash_base : std::unary_function<T, std::size_t> {};
                           ~~~~~^
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS17.0.sdk/usr/include/c++/v1/__functional/unary_function.h:46:1: note: '__unary_function' declared here
using __unary_function = __unary_function_keep_layout_base<_Arg, _Result>;
^

When running bazel run, I get this erros:

ERROR: Unrecognized pre-command argument: '--@rules_xcodeproj//xcodeproj:extra_common_flags=--//:renderer=metal'

[louwers]

@birkskyum Ah yes. Known issue with Xcode 15 right now.

We are in the process of updating Boost, which will resolve this issue. In the meantime, you can use an older Xcode version or go to the file where it errors and change unary_function to __unary_function as a quickfix.

[birkskyum]

@louwers , sound good. Do you have a tip for the bazel run issue as well?

[louwers]

@birkskyum Command had a typo. Try again.

[ianthetechie]

Hey, hey! Big thanks to everyone who made this possible! 🚀

A few notes from me to any other devs wanting to hack on this. The instructions from @louwers are absolutely applicable to most cases, but if you're weird like me and happen to be developing an Swift package (not an app target but a library for redistribution) that itself depends on MapLibre, here's how to get it into your Swift package:

1. Download a pre-release as noted.
2. Compute its checksum: swift package compute-checksum Downloads/MapLibre.dynamic.xcframework.zip. Suggestion: could we auto-compute this checksum in release pipelines? The only reason a Swift packager needs to do steps 1-2 is to get this.
3. In your Package.swift file, add a binary target to your targets list like so, using the URL of the XCFramework from the GitHub release and the checksum you computed in step 2:

.binaryTarget(name: "MapLibre",
                      url: "https://github.com/maplibre/maplibre-native/releases/download/ios-v6.0.0-predd74d1e84a781a41691cfd0de592d153c8795b65/MapLibre.dynamic.xcframework.zip",
                      checksum: "0a9c5a898f699e4acaa1650761f8908213fb5d638c389ed714a2f784349dd3b8")

4. Make sure to add .target(name: "MapLibre"), to the dependencies of any targets in your package that need MapLibre.

For a complete working example, I've updated the MapLibre SwiftUI DSL. The process took less than 10 minutes ;) Big thanks again for all the work that went into the release process!

[username0x0a]

Tried in our full-fledged app 👀 the performance's not great (at least in the Simulator) when compared to our MetalANGLE-wrapped package (which is speedy as hell, so expectations are high! 😎), but that's understandable as it's WIP, yet the integration was flawless (except for some calls to downstream changes we do in the library), looking for further progress! 🙌 🎉

[halset]

Thank you for the good work on Metal Renderer. It looks correct here in the Simulator on a pretty complex vector tile map. The hill shade in this map is a raster layer with PNGs with pre-rendered hill shade. The hill shade laggs a bit when panning the map with the Metal Renderer, but the vector map is not laggy. This looks like a very good start from my perspective at least. Promising!

[georgbachmann]

It looks very promising! After a certain zoom level it seems not render everything any more, but with my map (also a raster hillshade-layer and a LOT of layers) renders ok. Performance is not as it used to be, but I'd say good start! Awesome work!!!

[birkskyum]

Does the Bazel scripts need a refactor before this can be used with macOS? Naively opening macos.xcodeproj gives:

/maplibre-native/platform/darwin/src/NSValue+MLNAdditions.h:4:9 'MLNLight.h' file not found

[JesseCrocker]

There seems to be some significant issues with annotations.

After adding MLNPolyline Annotations it takes about 2 seconds to show up.

Point annotations show up faster, but when panning the map it seems to take a second before the map starts to move, then moves smoothly, but the annotation seems to wander a little bit.

I dont have a test project for this, but can come up with one if needed.

[sjg-wdw]

There seems to be some significant issues with annotations.

After adding MLNPolyline Annotations it takes about 2 seconds to show up.

Point annotations show up faster, but when panning the map it seems to take a second before the map starts to move, then moves smoothly, but the annotation seems to wander a little bit.

I dont have a test project for this, but can come up with one if needed.

I'm not sure how much testing we do with annotations. @stefankarschti does one of our test cases cover that?

[stefankarschti]

I don't think any of the cases covered annotation performance specifically, but I'm looking into it.

[stefankarschti]

There seems to be some significant issues with annotations.

After adding MLNPolyline Annotations it takes about 2 seconds to show up.

Point annotations show up faster, but when panning the map it seems to take a second before the map starts to move, then moves smoothly, but the annotation seems to wander a little bit.

I dont have a test project for this, but can come up with one if needed.

@JesseCrocker : how many polylines / points are you adding?

[tomasharkema]

Awesome! In conjunction with the porting of MapLibre Navigation iOS library to SPM for Flitsmeister, I took the liberty to open a branch using this metal implementation.

https://github.com/maplibre/maplibre-navigation-ios/tree/feature/spm-metal

Depend on this version of the navigation lib like so:

.package(url: "https://github.com/maplibre/maplibre-navigation-ios", branch: "feature/spm-metal")

[ianthetechie]

EDIT: The issue is fixed in the latest release and the paths now show up.

Line symbols are not implemented

Just to make sure I'm not reporting a duplicate issue, I'm seeing something funny with lines that I don't think is related to symbols. If this is a known issue, lemme know. Otherwise I can do a full writeup.

What I see:

What I expect to see:

Style: Stadia Outdoors. Here's a URL which should magic link to the map I screenshot above and has URL and zoom in the params: https://stadiamaps.com/explore-the-map/#map=17.31/37.509221/127.100405&style=outdoors

MLN version: ios-v6.0.0-predd74d1e84a781a41691cfd0de592d153c8795b65 (old I know; i can try out a newer version later).

[sjg-wdw]

Quick update on the Metal support.

We've got a version that's pretty close to OpenGL performance.
If the last version you grabbed was 3x or 4x, this one is closer to 1x, so give it a look and let us know what you find.

We're continuing to pursue performance improvements, but we'd love to find the corner cases if you have one.

[username0x0a]

@sjg-wdw I'll try it on an app that relies on & utilises custom annotations quite a lot 👍 I think the stress options in the example app are quite useful for this specifically, when hundreds+ are added 😁 and I'll cross-test it against a MetalANGLE-routed fork we use right now. 👀 Just waiting for a freshly baked build! 🙏

[louwers]

@username0x0a The latest build is 3 days old and contains the most recent changes for iOS. 🍞

https://github.com/maplibre/maplibre-native/releases?q=ios&expanded=true

[username0x0a]

Okay, I did a quick test in the Simulator ('cause that's the most usable for actual performance comparison) and I made some recordings of both the Metal Native & MetalANGLE binaries. Better download it, the YouTube-like previews in the browser are gross in quality.

With the Metal Native one, I'm experiencing tearing, when navigating around it occasionally freezes for up to a second (during swipe-moving around), also there's some visual glitching/tearing what seems to be around the tile edges. 🤔 FPS definitely still lower as well.

The MetalANGLE is really swifty, that's why we've settled with it for quite some time even in production.

The annotations alone are not part of the tiles data & are loaded asynchronously (from an API serving sorta-tiled Places infos – that is, ID, name + coords), usually 4–16 tiles (up to 64 annotations each, not all added to the map) are fetched (if not cached already), there's also an animation of the annotation as well as fade-in of a thumbnail image over the icon-font symbol layer once it's loaded. 😄 So quite a good case for a production app benchmark.

Hope you find this quick info useful. 👍

You can also try our MetalANGLE XCFramework binary just to have it for a side-by-side comparison, it's basically what used to be here in the obsoleted Metal support PR – should be replaceable in a testing project linking the xcframework with no changes needed.

[ianthetechie]

Good work @sjg-wdw! I'll do some more extensive testing over the coming days, but so far it looks like the rendering accuracy problems I had with the initial releases are solved!

[acalcutt]

I would love any suggestions on getting to node macos build working. I was seeing if I could get this to work in the cmake based process in #1793 by setting the options for metal rendering and restoring the macos cmake file that builds the node libs, but it doesn't quite build.

Other than restoring the existing cmake process, could this bazel build generate the node lib files, do what https://github.com/maplibre/maplibre-native/blob/main/platform/node/CMakeLists.txt does after building the macos components?

[sjg-wdw]

We're just starting to look at reviving the MacOS version. Give us a week or two and see where it's at.

[sjg-wdw]

Okay, I did a quick test in the Simulator ('cause that's the most usable for actual performance comparison) and I made some recordings of both the Metal Native & MetalANGLE binaries. Better download it, the YouTube-like previews in the browser are gross in quality.

With the Metal Native one, I'm experiencing tearing, when navigating around it occasionally freezes for up to a second (during swipe-moving around), also there's some visual glitching/tearing what seems to be around the tile edges. 🤔 FPS definitely still lower as well.

The MetalANGLE is really swifty, that's why we've settled with it for quite some time even in production.

The annotations alone are not part of the tiles data & are loaded asynchronously (from an API serving sorta-tiled Places infos – that is, ID, name + coords), usually 4–16 tiles (up to 64 annotations each, not all added to the map) are fetched (if not cached already), there's also an animation of the annotation as well as fade-in of a thumbnail image over the icon-font symbol layer once it's loaded. 😄 So quite a good case for a production app benchmark.

Hope you find this quick info useful. 👍

You can also try our MetalANGLE XCFramework binary just to have it for a side-by-side comparison, it's basically what used to be here in the obsoleted Metal support PR – should be replaceable in a testing project linking the xcframework with no changes needed.

That's a pretty interesting corner case. The simulator can be a bit weird so I'm curious what that looks like on hardware.
I'd also be curious how it performs without the annotations.

What could be happening there is a syncing problem we're not necessarily seeing in the tests.

[username0x0a]

@sjg-wdw

I've uploaded a couple of vids from the device (prefixed device-*). 👍

The annotations syncing issue is probably an important one here, indeed. Tearing & frame drops can be seen in zooming (quite a lot) as well as swiping around, often the map (I guess?) finishes the transition where annotations are a bit slower, sometimes resulting in tearing again.

The tile glyphs aren't present on the device (maybe some occasional flashing of the whole map layer), curiously.

Clear map performance is better on sight, but the tearing issues are still there, most notably when zooming in & out extensively.

MetalANGLE stats: FPS almost constant 60, CPU topping around 50%
Metal Native stats: FPS between 35–55-60, CPU topping around 75% with higher mean average

I'd definitely concentrate on the Simulator, too. For once, it's visibly more prone to performance issues (= easier to spot, profile, etc.), plus imagine that devs usually better spend their time during development day in the Simulator. 😅 Which is luckily way speedier than before when using Metal top-to-bottom. Using OpenGL Mapbox on M1 was a hell, so kicking in MetalANGLE for us was like a shift to the heaven. 🚀

Also I believe there are many projects utilising annotations, annotation clusters, custom shapes, etc. as by far not all data (if any even, if one uses external tiles provider) can be served via tiles, might not be even usable if the presented data are very dynamic/needs complex local filters and more.

[sjg-wdw]

Thanks! Those are excellent examples.

[georgbachmann]

May I ask about the state of macOS? I am building a little internal analysis tool and would love to try it out. The pre-release versions seem not to contain macOS slices yet?

Or can anybody tell me how to build it myself? Would be really great to try out...

[sjg-wdw]

We haven't put MacOS support back yet, but soon.

[halset]

Will it be possible to hook in to the Metal Renderer to draw custom dynamic non-tiled data by code?

[sjg-wdw]

That's the intent of the Custom Layer support.

[ianthetechie]

@sjg-wdw just curious if there's any movement on macOS support yet ;)

[sjg-wdw]

January now it looks like.

[louwers]

I set up automated releases to Swift Package Index and CocoaPods.

If you use the Swift Package Manager, maybe someone can test the pre-release that was made. I don't think it is possible to depend on a pre-release with Xcode, but if you use a Package.swift file you can add:

    dependencies: [
        .package(url: "https://github.com/maplibre/maplibre-gl-native-distribution.git", from: "6.0.0-pre9599200f2529de44ba62d4662cddb445dc19397d")
    ],

For CocoaPods the pre-release version available is 6.0.0-pre1811e91e3d40d14c860e9ff4e53d95fae68945c7. If any CocoaPods user could try it out and report back that would be great.

[ianthetechie]

Thanks @louwers! I just tested it out with our Swift package and it appears to be working.

[r3econ]

I followed the steps to include the framework in my iOS app. I used this prerelease.

I'm getting an error:

There is no Info.plist found at '..../MapLibre.dynamic.xcframework/Info.plist'.

Is there something I'm missing to make it work?

[louwers]

@r3econ The .zip is called MapLibre.dynamic.xcframework.zip. This contains a directory called MapLibre.dynamic.xcframework, the actual .xcframework is indide this directory. This is a bit confusing... Please try to use the .xcframework inside the directory, and it should work. Let us know.

[louwers]

Thanks for trying out the pre-releases everyone!

We are planning on making the official release of MapLibre Native for iOS with Metal support tomorrow.

[georgbachmann]

Is macOS still planned to have available via SPM? :-)

[louwers]

@georgbachmann If you want to work on it, pull request welcome. But it's not something I plan to work on.