Rebased iOS Metal support branch

[username0x0a]

TL;DR:

• reconstructed metal-support branch
• rebased on top of up-to-date main
• reverted some rendering flow changes
  • so that it runs on the ± same code like the current metal-support branch
  • without crashes experienced in the current main which might be caused by the same set of changes
• built around a bundled MetalANGLE.xcframework
  • binary with no need for building
  • stripped non-64-bit architectures
• a couple of WIP commits with changes required to run the framework with basic features

In this state it's definitely not meant to be merged, more like added as an official branch after some polishing. Feel free to post suggestions & push improvements so that the Metal support could be pushed forward for once. 👍

From my experience, the Metal-backed runtime is quite smooth, yet I've experienced some FPS drops to ~40fps, maybe caused by the reverted rendering flow code & synchronisation of UIKit/CoreAnimation layers. Hopefully somebody way more experienced in OpenGL/Metal field will be able to help optimising it.

A couple of link references so that this PR gets some attention:

• iOS Crash: Simulator Crash (glDrawElements) on custom style when camera is moved #268 – primary Metal issues tracking issue
• XCode 13.3, iOS 15.4 - MGLMapView renderSync mapbox/mapbox-gl-native-ios#650 – similar issue in Mapbox repository
• Example app crashes on iOS 15.4, emulator only rnmapbox/maps#1792 – yet another in the React Native fork
• Launch Issue with macOS Ventura + Xcode 14 + iOS 16 Simulator #337 – Ventura/Xcode 14/iOS 16 crash ticket

P.S.: When checking the code, skip the vendor folder as that currently contains the bundled MetalANGLE framework with quite heavy header files. 😅 That's why the abysmal +/- stats of this PR.

[roblabs]

I've kicked off the GitHub Action check(s). It looks like we will have a new branch called metal-support-new, which we can tag like we did with the Metal Support branch (example, ios-v5.13.0-pre.1).

I recommend that we lock the metal-support branch and let this new branch be the latest effort for Metal Support.

[ntadej]

I vote that we merge this into main, but this means we should not attach full MetalANGLE library. This can be an external dependency.

[username0x0a]

All [wip] comments are pretty much hackins which need to be addressed somehow, either in main as it's a general change (like non-64-bit dropping or invalid references) or by updating this PR.

MetalANGLE could be integrated either as a source (Git submodule) or as a precompiled binary (xcframework hosted somewhere), that's to be decided. I switched to the bundled binary so that I can provide it really optimised (and don't have to compile it all the time 💥). The size increase caused by it isn't really dramatical, in my MinSizeRel builds I even got a smaller footprint than with previously built Mapbox binaries, but somewhere I also seen an idea to only use what's needed from the MetalANGLE source to keep it as thin as possible – but that's definitely not necessary from the beginning & I'm also a bit skeptical that a significant part of MetalANGLE would be thrown away.

[nvanfleet]

It seems like this file should be renamed MGLMapView+OpenGL.mm

[username0x0a]

The code is still pretty much OpenGL 🤔 it's really just the overriding header files and bundled MetalANGLE doing the transition magic. I think there's a plan to rewrite the SDK in native Metal, that would be a good opportunity for changes – till that time, the SDK is formally still OpenGL.

[nvanfleet]

I seems like platform/darwin/src/gl_functions.cpp and platform/darwin/src/headlessbackend_eagle.mm still have a dependency on OpenGLES. Is this something that needs to also be migrated to use Metal?

https://github.com/maplibre/maplibre-gl-native/blob/main/platform/darwin/src/gl_functions.cpp
https://github.com/maplibre/maplibre-gl-native/blob/main/platform/darwin/src/headless_backend_eagl.mm

[nvanfleet]

I tried out a build of this and I see the following things in the terminal. It seem to all come out of the SDK when the map is initially initialized. The SDK warnings require you to hook into collecting logs from the SDK itself but the other things appear to just appear in Xcode without that.

• SDK Log Warning 1

filepath = "-[MGLStyle initWithRawStyle:stylable:]"
line = 121
filepath = "virtual bool mbgl::MGLCoreLoggingObserver::onRecord(mbgl::EventSeverity, mbgl::Event, int64_t, const std::string &)"
message = "[event]:OpenGL [code]:-1 [message]:Not using Vertex Array Objects"

• SDK Log Warning 2 (see the line it refers to is different)

filepath = "-[MGLStyle initWithRawStyle:stylable:]"
line = 27
filepath = "virtual bool mbgl::MGLCoreLoggingObserver::onRecord(mbgl::EventSeverity, mbgl::Event, int64_t, const std::string &)"
message = "[event]:OpenGL [code]:-1 [message]:Not using Vertex Array Objects"

• Terminal error

[Unknown process name] CGBitmapContextGetColorSpace: invalid context 0x2814edf80. If you want to see the backtrace, please set CG_CONTEXT_SHOW_BACKTRACE environmental variable.

• Metal compiler warning 1

2022-09-14 15:32:16.621421-0700 Lyft[6245:469664] [Metal Compiler Warning] Warning: Compilation succeeded with: 
program_source:79:11: warning: unused variable '_uv_linesofar'
    float _uv_linesofar = (floor(in._ua_data.z / 4.0) + (in._ua_data.w * 64.0)) * 2.0;
          ^

• Metal compiler warning 2 (see the line it refers to is different)

2022-09-14 15:32:16.897243-0700 Lyft[6245:469664] [Metal Compiler Warning] Warning: Compilation succeeded with: 

program_source:104:11: warning: unused variable '_uv_linesofar'
    float _uv_linesofar = (floor(in._ua_data.z / 4.0) + (in._ua_data.w * 64.0)) * 2.0;
          ^

[nvanfleet]

I also get a blue or black map on simulator. I'm not sure if you get that as well. On device it appears to work fine though.

[username0x0a]

Thanks for your feedback, @nvanfleet!

Ad the vertex array warnings:

bool Context::supportsVertexArrays() const {
    // TODO: Metal: Metal does not work with vertex arrays.
    return false;
}
...
        if (!supportsVertexArrays()) {
            Log::Warning(Event::OpenGL, "Not using Vertex Array Objects");
        }

Citation from the WIP Metal code here, I'd say this warning could be just dropped at this point if linked against Metal? The guys will figure out.

---

Ad Metal/shader code warnings: I know about these, but I haven't yet found where these are stored, I'm suspecting these to be coded somewhere (possibly in some shared context) so I'm not sure how to handle that against all platforms – again, somebody smarter needs to do it right. 😄

---

Ad blue or black map: do you refer to the iosapp in the MapLibre project? 🤔 The provided default styles can't resolve the domain at this point so you'd probably need to provide some working style instead. But last time I tested it, those styles worked (before domain was unreachable), the same goes to our internal (heavily customised) style. 👌

Update: the up-to-date MapLibre Basic style works, in terms of showing something relevant when running. 😄 The branch is now rebased with it.

---

My job here (for our project's purpose) is just to make it buildable & usable while considering its state at this point, but I'm sure MapLibre guys will polish it as needed. 👌

[username0x0a]

Rebased once again, replacing the bundled MetalANGLE.xcframework with a wrapping Swift package referencing it as a binary target so that the Pull request content is reduced to a necessary minimum for further polishing & optimisation. 👌

Later on, it would probably be best to include MetalANGLE directly as a submodule & bake it right into the library (it builds pretty fast, at least on my M2 Air 😅) so that there's minimal overhead & both static and dynamic libraries are full-fledged in all cases with no another framework to carry around/embed.

[louwers]

Please see the proposals below for the design of the upcoming Metal support:

• https://github.com/maplibre/maplibre-native/blob/main/design-proposals/2022-10-27-rendering-modularization.md
• https://github.com/maplibre/maplibre-native/blob/main/design-proposals/2022-11-29-metal-port.md

A team of graphics engineers is implementing this approach right now in favour of MetalANGLE, so I don't think it makes sense to merge this at this point in time.