Ortho4XP bundled application

[shred86]

Thanks to @ColinM9991's idea, I made some changes to the latest dev branch that enable the use of PyInstaller so we can bundle all the dependencies into one package making it easier for users. You can clone the dev branch and build it using:

pyinstaller --noconfirm \
    --paths=src \
    --add-data="./Utils:./Utils" \
    --add-data="./Extents:./Extents" \
    --add-data="./Filters:./Filters" \
    --add-data="./Licence:./Licence" \
    --add-data="./Patches:./Patches" \
    --add-data="./Previews:./Previews" \
    --add-data="./Providers:./Providers" \
    --hidden-import=PIL \
    Ortho4XP.py

Or if you are a Mac ARM user, I also uploaded a packaged version which can be downloaded using this link.

It's still a work in progress so definitely welcome any help or suggestions.

Some things I know that still need to be done:

• Figure out if there's a way to have the files/folders generated by Ortho4XP separate from _internal, which is where everything goes for a pyinstaller onedir build. I tried a few things but haven't had any luck with this. In the end, it's not a huge deal because if you can set a custom base directory in Ortho4XP for your tiles. Everything else is just cache data and there's a way to delete it from the GUI.
• Reduce the file size. Since these are OS/architecture specific, we can obviously remove stuff that isn't necessary for that particular build but I did that on the macOS build I posted and it's still in the 300MB range.

[ColinM9991]

Depending on whether you're up for the challenge, you could create a GitHub action that automatically builds and packages the output for each OS type (Mac, Linux and Windows)

Happy to help with this. Should have plenty of time at the weekend to dig in

[shred86]

Yeah that's a good point, although I'm not sure how often we'll be pushing new changes so not sure how much effort it's worth (might be pretty easy though). One issue right now I'm seeing is the file size. The macOS version I built is in the 300MB range and Github has a 100MB file size limit. Looking through the files, there's nothing that really stands out in terms of being able to really reduce the file size. The map images for the tile management window is about 22MB and I tried compressing the images further but they're already fairly optimized. There's just a lot of dependencies that add up.

Distribution wise, I was thinking maybe just posting torrent files to X-Plane.org to let people help distribute. Google Drive might also work since there probably won't be very many downloads per day.

[shred86]

The other thing I'm debating is how much, if any, changes to the baseline functionality of Ortho4XP do we pull in. For example, I made a few small changes/fixes that I've incorporated into the dev branch because they either fix a problem or bug. My plan is to push all of these changes to the master branch soon and just continue to updating this forked version but not diverge too far from the original version (i.e. no UI redesign, although I'm tempted because I am not a huge fan of how it's laid out right now 😄).

[w8sl]

Surprisingly well for a 1.0! Congratulations!
GDAL 3.9 is not working currently on macOS but GDAL 3.8.5 and below works perfectly on Linux. I guess it works on Windows also.
Compiled O4XP is cross compatible between Linux distributions, with few issues (Linux only): Earth2_ZL6_6_....jpg files cannot be found, in spite of correct PATH including upper/lower case.
PIL._tkinter_finder module is missing on Linux.

I would suggest:
OSM_dir = resource_path("MyOrtho/OSM_data")
Mask_dir = resource_path("MyOrtho/Masks")
Imagery_dir = resource_path("MyOrtho/Orthophotos")
Elevation_dir = resource_path("MyOrtho/Elevation_data")
Geotiff_dir = resource_path("MyOrtho/Geotiffs")
Tile_dir = resource_path("MyOrtho/Tiles")
Overlay_dir = resource_path("MyOrtho/yOrtho4XP_Overlays")

As it makes easier to link from _internal just one directory

[shred86]

I had issues with GDAL on macOS but I seemed to get it working. I documented it in an issue here.

I think a separate directory for the Ortho4XP data makes sense but I was really trying to avoid any changes that would affect the organization for users who just download the repo and run it directly. However, it might be worth it because it is kind of a pain having to sort through all the dependency files, especially when you're updating Ortho4XP or trying to back it up.

[w8sl]

Perfect - thanks!

[shred86]

I think this might be an ideal solution:

def resource_path(relative_path):
    """Get absolute path to resource."""
    # Required for using pyinstaller
    if getattr(sys, 'frozen', False) and hasattr(sys, '_MEIPASS'):
        base_path = sys._MEIPASS + "/Ortho4XP_Data/"
    else:
        base_path = os.path.abspath(".")
    return os.path.join(base_path, relative_path)

Basically, if we're running from a standalone, we add everything that Ortho4XP generates into the Ortho4XP_Data folder. If it's a "manual" setup, then everything remains as is.

Seems to be working okay but I want to make sure I'm not missing anything before I recompile and push this into master.

[shred86]

After testing this a bit, it seems to be a good solution that doesn't break anything for users who want to do a manual setup. I went ahead and pushed into dev and merged over to master. I also rebuilt the packages for Windows, Mac and Linux. The Linux package was built on Debian (Trixie) but I'm assuming it will work on any Linux distro since its a standalone package.

[w8sl]

This is working better on Linux:

pyinstaller --noconfirm
--paths=src
--add-data="./Utils:./Utils"
--add-data="./Extents:./Extents"
--add-data="./Filters:./Filters"
--add-data="./Licence:./Licence"
--add-data="./Patches:./Patches"
--add-data="./Previews:./Previews"
--add-data="./Providers:./Providers"
--collect-all=PIL
Ortho4XP.py

[w8sl]

I have compiled your new build on ubuntu 24 with Python 3.12 and GDAL 3.8.4 Then moved compiled O4XP to Arch Linux. Everything incl. GDAL works out of the box on Ubuntu and Arch with no errors/warnings. O4XP_Data is a perfect solution!

BTW: Pyinstaller commmand for Arch linux: python -m PyInstaller --noconfirm \

Regarding PIL - this is happening with import all option:

8755 INFO: Analyzing hidden import 'PIL.SunImagePlugin'
8757 INFO: Analyzing hidden import 'PIL.TarIO'
8758 INFO: Analyzing hidden import 'PIL.TgaImagePlugin'
8762 INFO: Analyzing hidden import 'PIL.WalImageFile'
8763 INFO: Analyzing hidden import 'PIL.WebPImagePlugin'
8769 INFO: Analyzing hidden import 'PIL.WmfImagePlugin'
8772 INFO: Analyzing hidden import 'PIL.XVThumbImagePlugin'
8773 INFO: Analyzing hidden import 'PIL.XbmImagePlugin'
8774 INFO: Analyzing hidden import 'PIL.XpmImagePlugin'
8777 INFO: Analyzing hidden import 'PIL.main'
8777 INFO: Analyzing hidden import 'PIL._tkinter_finder'
8777 INFO: Analyzing hidden import 'PIL.report'

I have tried to add just: hidden import 'PIL._tkinter_finder' with no luck

Once again thanks for your hard work!

[shred86]

Nice, thanks for testing that out! I just also tested it on a clean Debian (trixie) install and it ran fine. Also setup the latest version of Ubuntu on my Windows machine using WSL and it worked there as well.

I'm using it on my main computer (macOS) and have built a few new and rebuilt a few old times. So far, everything seems to work as expected which is good.

As for PIL, I'm still not quite sure what's going on. I compared the PyInstaller build logs between macOS and Linux, and they're identical (pulling in all the same PIL hooks). In the meantime, not too worried about it since we have a solution that works (--collect-all=PIL) and doesn't take up much more space. Wiki should be up-to-date for installation and developer notes.

[w8sl]

It seems like this is also working on Linux:
--hidden-import=PIL
--collect-submodules=PIL
Not sure what is the problem with PIL. Collect all is indeed a bit strange.

[shred86]

Interesting. I just compared the PIL and pillow.libs folders after using just --hidden-import=PIL and both --hidden-import=PIL and --collect-submodules=PIL. It appears using the latter pulls in an additional file in the PIL folder: _imagingmorph.cpython-312-x86_64-linux-gnu.so

On Windows and macOS, that file is not present but the map loads file... 🤷 I haven't noticed any issues using Ortho4XP on macOS so far.

I think it's "cleaner" to use --collect-submodules=PIL so I'll update the documentation. Thanks for the find. Still no idea why Linux is behaving differently though... must be something with how Linux renders graphics that requires it to call an extra function.

[w8sl]

I am still figuring this out. Even --collect-submodules=PIL._tkinter_finder works, but it is only 0,1 mb less than simpler command: --collect-submodules=PIL, which should work on all systems