State of project on Linux

[tamastimar]

It's not entirely clean for me how it can be used on Linux.

Should I copy the necessary Apple SDKs to somewhere on the Linux machine? If not, where Cocoa symbols come from?

Thanks!

[grp]

Right now, xcbuild runs and passes its tests on Linux, but it's only a small piece of what you'd actually need to build. A more complete list is below, but even that hasn't been tested in full. Linux support is definitely still a work-in-progress, but is a part of xcbuild we'll continue to develop on.

1. You'll need to build xcbuild itself on Linux. This seems to require GCC 5 right now although it would be great to lower that requirement.
2. You'll need a .xctoolchain built for Linux. I don't know of any public examples here but you might have luck with a public iOS toolchain for Linux that you arrange to look like one.
3. You'll need specifications (.xcspec files) for how to run the tools. Right now, these are found in Xcode, but we'd like to make a custom set to support xcbuild standalone.
4. You'll need the .platform and .sdk you want to build for. These would come from Xcode.
5. Some tools are also not usually included in these since they don't have open source implementations (momc, mapc, actool, and ibtool are common ones) which you'd need to avoid for any Linux-based builds.

You'd need to arrange all of those into a Developer directory, roughly structured to match what xcbuild expects. Then, set the DEVELOPER_DIR environment variable to point to that directory and run xcbuild. With all the pieces, in theory it should do the right thing.

[tamastimar]

Thanks for your detailed answer.

As far as I see, the weakest point is the .xctoolchain (2). By a quick googling I've found this: https://github.com/tpoechtrager/osxcross

Maybe it'd be helpful for others to indicate the state of project on Linux in README (e.g. a link pointing your comment). :)

[mstorsjo]

FWIW, I've managed to set up a partially functional xcodebuild environment on linux.

What I do is basically this:

• Copy Xcode.app from an OS X machine
• Build xcbuild
• Get a vanilla clang version
• Build other apple tools from https://github.com/tpoechtrager/cctools-port
• Replace apple binaries in Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin with (symlinks to, or scripts invoking) native linux ones

Instead of vanilla clang, one could also try building e.g. https://opensource.apple.com/tarballs/clang/clang-703.0.31.tar.gz, but this fails building for me.

Large sections of the Xcode app bundle can be stripped out to save disk space. You basically need Contents/PlugIns, Contents/Toolchains and Contents/Platforms.

As for the toolchain binaries, I've made lipo, libtool and dsymutil symlinks to the ones from cctools-port and clang/llvm (dsymutilneeds to be linked to llvm-dsymutil).

When using vanilla clang instead of the apple patched one, I've used a wrapper script like this:

#!/bin/bash
ARGS=()
TARGET_SET=""
SYSROOT_SET=""
while [ $# -gt 0 ]; do
        a=$1
        if [ "$a" = "-arch" ]; then
                shift
                ARGS+=(-target $1-apple-darwin16)
                TARGET_SET=1
                shift
        else
                if [ "$a" = "-isysroot" ]; then
                        SYSROOT_SET=1
                fi
                ARGS+=("$a")
                shift
        fi
done
if [ -z "$TARGET_SET" ]; then
        ARGS+=(-target x86_64-apple-darwin16)
fi
if [ -z "$SYSROOT_SET" ]; then
        # Is there a better way to find the default sdk?
        SDKS=$DEVELOPER_DIR/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs
        SDK=$SDKS/$(ls $SDKS | head -1)
        ARGS+=(-isysroot $SDK)
fi
export PATH=/path/to/cctools/bin:$PATH
if [ "$(basename $0)" = "clang++" ] || [ "$(basename $0)" = "c++" ] || [ "$(basename $0)" = "g++" ]; then
        EXE=/path/to/clang/bin/clang++
else
        EXE=/path/to/clang/bin/clang
fi
$EXE "${ARGS[@]}"

The copypng tool in in Contents/Developer/usr/bin tries to call xcrun which isn't available in my path (it's part of xcbuild, which I don't have in the main system paths), so I've just sidestepped this by making copypng a symlink to /bin/true, effectively skipping this step. (I mainly use this for build testing so far, not for actually producing release binaries.)

Apple haven't opensourced the version of ld used in Xcode 8, and Xcode 8 uses a new version of the file format for tbd library stubs which isn't supported by the previous versions, so if you need to do linking (i.e. not only building static libraries), you probably want to go with an Xcode 7 app bundle. (EDIT: Nowadays this is handled.)

I've got all of this wrapped up in a reusable dockerfile at https://github.com/mstorsjo/xcode-cross. It requires you to pack up a real Xcode installation and it only works with Xcode 9.2 or older though.

Edit: Updated the clang wrapper script a little.

Edit2: Mentioned that the tbd library format is possible to use these days, and pointed to a prepackaged version of all of this.

[acecilia]

Checking again, now that we are in 2019.

The readme says that xcbuild runs on linux (and it does) but Xcode projects are not being build because of the following reasons:

• There is no support for Xcode 10 (and what about the new build system?) | Make failed！ #293.
• ibtool (and other tools like momc, mapc or actool) is not open source and can not be compiled and run on linux (as far as I have seen), so even if you make xcbuild work for your project, it will not build if you are using storyboards | xcbuild: add linux SDK Definition #67, State of project on Linux #37.
• There is no clear explanation about how to do the linux setup, and the only information here is old (from 2016) and pretty hacky.

The first and second points (the first one can be fixed, but the second I do not see how) make impossible to build a Xcode project on linux.

@grp What do you think? Do you know anybody that managed to build a Xcode project on linux?
Thanks!

[mstorsjo]

Do you know anybody that managed to build a Xcode project on linux?

Yes, I have. However it only works for certain projects that fit within the limitations (no interface builder/storyboards or assets) you mentioned. Works fine for certain CI tasks, but probably not very useful for a full modern app. An app that e.g. does everything the "manual" way of creating GUI via code might work. My setup just symlinks ibtool to /bin/true which is ok for CI build testing but won't build a real working app.

I updated my post about a linux setup though, with a reference to https://github.com/mstorsjo/xcode-cross, which has got an as-full-as-possible full version (without Xcode itself) of what I'm using, which should be possible to reproduce pretty easily.

[acecilia]

@mstorsjo awesome! Thanks for the fast response and the super useful insight

[acecilia]

@mstorsjo do your approach work with Xcode 10 and the new build system?

I could not build and run xcbuild successfully in Mac Mohave, getting the issue asdescribed in #293 when building.

1. I tried the suggested solution in Make failed！ #293 (comment) (adding add_compile_options(-Wno-delete-non-virtual-dtor) to line 67 in CMakeLists.txt) and I managed to compile xcbuild
2. When running it on my project (Xcode 10) I get this other issue.

[mstorsjo]

There's actually three orthogonal issues involved here:

1. The xcbuild source code seems to have issues when built with newer C++ tools (Make failed！ #293). That should probably be pretty straightforward to fix. This isn't related strictly to Xcode 10 itself, but you should face the same if you try to build xcbuild on linux with an equally new clang and potentially GCC as well.

2. The issue with graph cycles isn't related to Xcode 10 itself but to what the build project looks like. In my case I was able to restructure the project slightly to avoid the issue.

3. The Xcode build tools use a new file format TBDv2 for text library stubs since Xcode 9.3, and as far as I know this isn't yet opensourced from apple. This means that you can't link against the libraries bundled as part of Xcode. See how https://opensource.apple.com/release/developer-tools-93.html says "tapi-902.0.9 (coming soon!)". This is only an issue with building from linux where all build tools have to be rebuilt from scratch, while if you just would run plain xcbuild on macos, it shouldn't be an issue.

[aaronmagi]

@mstorsjo I wanted to know with your container version can you use xcbuild to do a "test". IE build a particular xcode project and then preform the build then test which installs and runs the app on the real iOS device?

Thank you I thought I would ask prior to going through the setup.

[mstorsjo]

I haven't tried that, and I doubt that it works. Even outside of xcbuild, I haven't dug much into how well it is possible to install code on iOS devices from linux, and even if there is some solution for that, I don't think it's hooked up into xcbuild at the moment.

[aaronmagi]

@mstorsjo Thank you for your update and quick response.

[huang1988519]

https://heroims.github.io/2017/09/10/Linux%20%E6%9E%84%E5%BB%BA:%E7%BC%96%E8%AF%91IOS:Mac%E7%A8%8B%E5%BA%8F/

[lechium]

3. The Xcode build tools use a new file format TBDv2 for text library stubs since Xcode 9.3, and as far as I know this isn't yet opensourced from apple. This means that you can't link against the libraries bundled as part of Xcode. See how https://opensource.apple.com/release/developer-tools-93.html says "tapi-902.0.9 (coming soon!)". This is only an issue with building from linux where all build tools have to be rebuilt from scratch, while if you just would run plain xcbuild on macos, it shouldn't be an issue.

Looks like newer tapi versions have been posted to opensource.apple.com hopefully that can help alleviate that particular problem: https://opensource.apple.com/tarballs/tapi/ someone made a github mirror as well https://github.com/tpoechtrager/apple-libtapi

[mstorsjo]

1. The Xcode build tools use a new file format TBDv2 for text library stubs since Xcode 9.3, and as far as I know this isn't yet opensourced from apple. This means that you can't link against the libraries bundled as part of Xcode. See how https://opensource.apple.com/release/developer-tools-93.html says "tapi-902.0.9 (coming soon!)". This is only an issue with building from linux where all build tools have to be rebuilt from scratch, while if you just would run plain xcbuild on macos, it shouldn't be an issue.

Looks like newer tapi versions have been posted to opensource.apple.com hopefully that can help alleviate that particular problem: https://opensource.apple.com/tarballs/tapi/ someone made a github mirror as well https://github.com/tpoechtrager/apple-libtapi

Yes - I've taken that into use in my packaged linux setup of it all at https://github.com/mstorsjo/xcode-cross. It currently works well for both Xcode 9.3 and Xcode 10. Xcode 11 however seems to require quite a bit of fixes to xcbuild (which I unfortunately haven't gotten round to posting issues for).

[lechium]

awesome 👍 glad to hear it.

EDIT: i updated my setup locally with the new tapi libraries and everything works properly noe as well with newer tbd files. still need to get xcbuild paths setup next though.