Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consolidate LocateNativeCompiler target #103375

Merged
merged 4 commits into from
Oct 16, 2024
Merged

Consolidate LocateNativeCompiler target #103375

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
04f086a
Consolidate LocateNativeCompiler target
am11 Oct 12, 2024
9a97e5e
Set install_name on all Apple platforms
am11 Oct 13, 2024
6297fdf
Update src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Nativ…
am11 Oct 16, 2024
97a667f
Merge branch 'main' into feature/eng/internal-nativeaot-build.targets
am11 Oct 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 5 additions & 27 deletions eng/testing/linker/project.csproj.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,14 @@
<NETCoreAppMaximumVersion>{NetCoreAppMaximumVersion}</NETCoreAppMaximumVersion>
<UseMonoRuntime>{UseMonoRuntime}</UseMonoRuntime>
<RuntimeIdentifier>{RuntimeIdentifier}</RuntimeIdentifier>
<PublishAot>{PublishAot}</PublishAot>
<AppHostSourcePath>{AppHostSourcePath}</AppHostSourcePath>
<SingleFileHostSourcePath>{SingleFileHostSourcePath}</SingleFileHostSourcePath>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>

<!-- Implicitly force 'UseNativeAotForComponents' for AOT tests -->
<PublishAot>{PublishAot}</PublishAot>
<UseNativeAotForComponents Condition="'$(PublishAot)' == 'true'">true</UseNativeAotForComponents>

<!-- wasm specific -->
<MonoAOTCompilerDir>{MonoAOTCompilerDir}</MonoAOTCompilerDir>
<MonoProjectRoot>{MonoProjectRoot}</MonoProjectRoot>
Expand Down Expand Up @@ -59,14 +62,9 @@
<IlcSdkPath>{IlcSdkPath}</IlcSdkPath>
<IlcFrameworkPath>{IlcFrameworkPath}</IlcFrameworkPath>
<IlcFrameworkNativePath>{IlcFrameworkNativePath}</IlcFrameworkNativePath>
<SysRoot Condition="('$(CrossBuild)' == 'true' or '$(BuildArchitecture)' != '$(TargetArchitecture)') and '$(ROOTFS_DIR)' != ''">$(ROOTFS_DIR)</SysRoot>
<CoreCLRBuildIntegrationDir>{CoreCLRBuildIntegrationDir}</CoreCLRBuildIntegrationDir>
</PropertyGroup>

<ItemGroup>
<CustomLinkerArg Condition="'$(CrossBuild)' == 'true' and '$(_hostArchitecture)' == '$(_targetArchitecture)' and '$(ROOTFS_DIR)' != ''" Include="--gcc-toolchain=$(ROOTFS_DIR)/usr" />
</ItemGroup>

<ItemGroup>
{RuntimeHostConfigurationOptions}
</ItemGroup>
Expand All @@ -81,28 +79,8 @@
</ItemGroup>
</Target>

<Target Name="LocateNativeCompiler"
Condition="'$(PublishAot)' == 'true' and '$(_hostOS)' != 'win'"
BeforeTargets="SetupOSSpecificProps">
<PropertyGroup>
<CppCompilerAndLinker Condition="'$(CppCompilerAndLinker)' == ''">clang</CppCompilerAndLinker>
</PropertyGroup>

<Exec Command="sh -c 'build_arch=&quot;$(TargetArchitecture)&quot; compiler=&quot;$(CppCompilerAndLinker)&quot; . &quot;$(RepositoryEngineeringDir)/common/native/init-compiler.sh&quot; &amp;&amp; echo &quot;$CC;$LDFLAGS&quot;' 2>/dev/null"
EchoOff="true"
ConsoleToMsBuild="true"
StandardOutputImportance="Low">
<Output TaskParameter="ConsoleOutput" PropertyName="_CC_LDFLAGS" />
</Exec>

<PropertyGroup>
<CppLinker>$(_CC_LDFLAGS.SubString(0, $(_CC_LDFLAGS.IndexOf(';'))))</CppLinker>
<_LDFLAGS>$(_CC_LDFLAGS.SubString($([MSBuild]::Add($(_CC_LDFLAGS.IndexOf(';')), 1))))</_LDFLAGS>
<LinkerFlavor Condition="$(_LDFLAGS.Contains('lld'))">lld</LinkerFlavor>
</PropertyGroup>
</Target>

<Import Project="{NativeSanitizersTargets}" />
<Import Project="$(RepositoryEngineeringDir)toolAot.targets" />

<ItemGroup>
<Content Include="@(SanitizerRuntimeToCopy->'{SanitizerRuntimeFolder}/%(Identity)')" CopyToOutputDirectory="PreserveNewest" />
Expand Down
2 changes: 1 addition & 1 deletion eng/testing/linker/trimmingTests.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@

<MSBuild Projects="@(TestConsoleApps)"
Targets="Publish"
Properties="Configuration=$(Configuration);BuildProjectReferences=false;TargetOS=$(TargetOS);TargetArchitecture=$(TargetArchitecture);_IsPublishing=true" />
Properties="Configuration=$(Configuration);BuildProjectReferences=false;HostOS=$(HostOS);TargetOS=$(TargetOS);TargetArchitecture=$(TargetArchitecture);_IsPublishing=true" />
</Target>

<Target Name="ExecuteApplications"
Expand Down
33 changes: 2 additions & 31 deletions eng/toolAot.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,30 +28,6 @@
<PackageReference Include="runtime.$(ToolsRID).Microsoft.DotNet.ILCompiler" Version="$(MicrosoftDotNetILCompilerVersion)" />
</ItemGroup>

<!-- Needed for the amd64 -> amd64 musl cross-build to pass the target flag. -->
<Target Name="_FixIlcTargetTriple"
AfterTargets="SetupOSSpecificProps"
Condition="'$(CrossBuild)' == 'true' and '$(HostOS)' != 'windows'">
<!-- Compute CrossCompileRid, and copy the downstream logic as-is. -->
<PropertyGroup>
<CrossCompileRid>$(RuntimeIdentifier)</CrossCompileRid>

<CrossCompileArch />
<CrossCompileArch Condition="$(CrossCompileRid.EndsWith('-x64'))">x86_64</CrossCompileArch>
<CrossCompileArch Condition="$(CrossCompileRid.EndsWith('-arm64')) and '$(_IsApplePlatform)' != 'true'">aarch64</CrossCompileArch>
<CrossCompileArch Condition="$(CrossCompileRid.EndsWith('-arm64')) and '$(_IsApplePlatform)' == 'true'">arm64</CrossCompileArch>

<TargetTriple />
<TargetTriple Condition="'$(CrossCompileArch)' != ''">$(CrossCompileArch)-linux-gnu</TargetTriple>
<TargetTriple Condition="'$(CrossCompileArch)' != '' and ($(CrossCompileRid.StartsWith('linux-musl')) or $(CrossCompileRid.StartsWith('alpine')))">$(CrossCompileArch)-alpine-linux-musl</TargetTriple>
<TargetTriple Condition="'$(CrossCompileArch)' != '' and ($(CrossCompileRid.StartsWith('freebsd')))">$(CrossCompileArch)-unknown-freebsd12</TargetTriple>
</PropertyGroup>

<ItemGroup>
<LinkerArg Include="--target=$(TargetTriple)" Condition="'$(TargetOS)' != 'osx' and '$(TargetTriple)' != ''" />
</ItemGroup>
</Target>

<ItemGroup Condition="'$(NativeAotSupported)' == 'true'">
<CustomLinkerArg Condition="'$(CrossBuild)' == 'true' and '$(_hostArchitecture)' == '$(_targetArchitecture)' and '$(_hostOS)' != 'windows'" Include="--gcc-toolchain=$(ROOTFS_DIR)/usr" />
</ItemGroup>
Expand All @@ -61,6 +37,7 @@
BeforeTargets="SetupOSSpecificProps">
<PropertyGroup>
<CppCompilerAndLinker Condition="'$(CppCompilerAndLinker)' == ''">clang</CppCompilerAndLinker>
<SysRoot Condition="'$(CrossBuild)' == 'true'">$(ROOTFS_DIR)</SysRoot>
</PropertyGroup>

<Exec Command="sh -c 'build_arch=&quot;$(TargetArchitecture)&quot; compiler=&quot;$(CppCompilerAndLinker)&quot; . &quot;$(RepositoryEngineeringDir)/common/native/init-compiler.sh&quot; &amp;&amp; echo &quot;$CC;$LDFLAGS&quot;' 2>/dev/null"
Expand All @@ -77,10 +54,4 @@
</PropertyGroup>
</Target>

<!-- Cross-build settings -->
<PropertyGroup>
<LinkerFlavor Condition="'$(CrossBuild)' == 'true' and '$(TargetsLinux)' == 'true'">lld</LinkerFlavor>
<SysRoot Condition="'$(CrossBuild)' == 'true' and '$(HostOS)' != 'windows'">$(ROOTFS_DIR)</SysRoot>
</PropertyGroup>

</Project>
</Project>
5 changes: 3 additions & 2 deletions src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,8 @@ The .NET Foundation licenses this file to you under the MIT license.
<IlcRPath Condition="'$(IlcRPath)' == '' and '$(_IsApplePlatform)' != 'true'">$ORIGIN</IlcRPath>
<IlcRPath Condition="'$(IlcRPath)' == '' and '$(_IsApplePlatform)' == 'true'">@executable_path</IlcRPath>

<SharedLibraryInstallName Condition="'$(SharedLibraryInstallName)' == '' and '$(_IsApplePlatform)' == 'true' and '$(NativeLib)' == 'Shared'">@rpath/$(TargetName)$(NativeBinaryExt)</SharedLibraryInstallName>

<EventPipeName>libeventpipe-disabled</EventPipeName>
<EventPipeName Condition="'$(EventSourceSupport)' == 'true'">libeventpipe-enabled</EventPipeName>

Expand Down Expand Up @@ -92,7 +94,6 @@ The .NET Foundation licenses this file to you under the MIT license.
<_AppleTripleAbi Condition="$(_targetOS.EndsWith('simulator'))">simulator</_AppleTripleAbi>

<TargetTriple>$(CrossCompileArch)-apple-$(_AppleTripleOS)$(AppleMinOSVersion)-$(_AppleTripleAbi)</TargetTriple>
<SharedLibraryInstallName Condition="'$(SharedLibraryInstallName)' == '' and '$(NativeLib)' == 'Shared'">@rpath/$(TargetName)$(NativeBinaryExt)</SharedLibraryInstallName>
</PropertyGroup>

<PropertyGroup Condition="'$(_targetOS)' == 'osx'">
Expand Down Expand Up @@ -214,7 +215,7 @@ The .NET Foundation licenses this file to you under the MIT license.
<LinkerArg Include="-Wl,--strip-debug" Condition="$(NativeDebugSymbols) != 'true' and '$(_IsApplePlatform)' != 'true'" />
<LinkerArg Include="-Wl,-rpath,'$(IlcRPath)'" Condition="'$(StaticExecutable)' != 'true' and !$([MSBuild]::IsOSPlatform('Windows'))" />
<LinkerArg Include="-Wl,-rpath,&quot;$(IlcRPath)&quot;" Condition="'$(StaticExecutable)' != 'true' and $([MSBuild]::IsOSPlatform('Windows'))" />
<LinkerArg Include="-Wl,-install_name,&quot;$(SharedLibraryInstallName)&quot;" Condition="'$(_IsiOSLikePlatform)' == 'true' and '$(NativeLib)' == 'Shared'" />
<LinkerArg Include="-Wl,-install_name,&quot;$(SharedLibraryInstallName)&quot;" Condition="'$(SkipInstallName)' != 'true' and '$(_IsApplePlatform)' == 'true' and '$(NativeLib)' == 'Shared'" />
am11 marked this conversation as resolved.
Show resolved Hide resolved
<LinkerArg Include="-Wl,--build-id=sha1" Condition="'$(_IsApplePlatform)' != 'true'" />
<LinkerArg Include="-Wl,--as-needed" Condition="'$(_IsApplePlatform)' != 'true'" />
<LinkerArg Include="-Wl,-e,0x0" Condition="'$(NativeLib)' == 'Shared' and '$(_IsApplePlatform)' != 'true'" />
Expand Down
4 changes: 0 additions & 4 deletions src/native/managed/compile-native.proj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,6 @@
<ItemGroup>
<!-- additional native compilation-specific properties to pass down to the ProjectReference -->
<SubprojectProps Include="NativeLib" Value="$(NativeLibKind)"/>

<SubprojectProps Condition="'$(SysRoot)' != ''" Include="SysRoot" Value="$(SysRoot)" />
<SubprojectProps Condition="'$(LinkerFlavor)' != ''" Include="LinkerFlavor" Value="$(LinkerFlavor)" />
<SubprojectProps Condition="'$(CustomLinkerArgToolchainArg)' != ''" Include="CustomLinkerArgToolchainArg" Value="$(CustomLinkerArgToolchainArg)" />
</ItemGroup>

<Import Project=".\subproject.props" />
Expand Down
19 changes: 0 additions & 19 deletions src/native/managed/native-library.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,28 +9,9 @@
<ControlFlowGuard>Guard</ControlFlowGuard>
</PropertyGroup>

<!-- set the shared library name. this helps the native linker correctly reference this shared
library in dependents -->
<PropertyGroup>
<!-- in net9.0 we can do this, but only on mobile apple platforms, not OSX -->
<SharedLibraryInstallName>@rpath/$(MSBuildProjectName).dylib</SharedLibraryInstallName>
</PropertyGroup>
<ItemGroup Condition="'$(TargetsOSX)' == 'true'">
<LinkerArg Include="-Wl,-install_name,@rpath/$(MSBuildProjectName).dylib" />
</ItemGroup>
Comment on lines -14 to -20
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We generally need to specify the @rpath/-prefixed name on macOS for libraries. Is that done somewhere else now? What is the output for otool -l before and after?

Copy link
Member Author

@am11 am11 Oct 13, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that done somewhere else now?

Seems to me like an avoidable duplicate:

runtime/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets

Line 217 in 7b69459

<LinkerArg Include="-Wl,-install_name,&quot;$(SharedLibraryInstallName)&quot;" Condition="'$(_IsiOSLikePlatform)' == 'true' and '$(NativeLib)' == 'Shared'" />

Can we use it for all apple platforms? Or are there use-cases when we explicitly don't need it?

Copy link
Member

@filipnavara filipnavara Oct 13, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use it for all apple platforms? Or are there use-cases when we explicitly don't need it?

I'd say we can. We would need to check with Xamarin workloads to ensure that it doesn't break anything.

Note that generally Xamarin workloads have an SDK code on the consumption side where it fixes up the paths with install_name_tool if they are wrong, so I don't expect it to break anything there. I am more concerned that the workload may also specify -install_name on the library compilation side and it would collide if we add it in the NativeAOT linker args.

Copy link
Member Author

@am11 am11 Oct 13, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With 9a97e5e the behavior is:

-      cmdsize 72
-         name bin/Release/net9.0/osx-arm64/native/helloworld.dylib (offset 24)
+      cmdsize 48
+         name @rpath/helloworld.dylib (offset 24)
  • User can skip setting install_name with SkipInstallName
  • If user has <LinkerArg Include="-Wl,-install_name,@rpath/TheirName" /> type of thing specified in their project, TheirName takes precedence with SkipInstallName=true

What do you think?

cDAC (the project being modified) is currently only used for testing in the repo, which is working, so I think we can live with it without waiting for next SDK update which won't be long (10.0 alpha1).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @lambdageek

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

<ItemGroup Condition="'$(TargetsUnix)' == 'true' and ! ('$(TargetsAppleMobile)' == 'true' or '$(TargetsOSX)' == 'true')">
<!-- If there is no soname, ld on Linux and some other Unixes will embed the full (build-time!)
path to this shared library into any binary that links against it. We don't want that. -->
<LinkerArg Include="-Wl,-soname=$(MSBuildProjectName).so" />
</ItemGroup>

<PropertyGroup>
<!-- if IsRuntimeComponent is true, we will put the native library into the specified locations under `artifacts/bin/$(RuntimeFlavor)/os.arch.config/` -->
<IsRuntimeComponent Condition="'$(IsRuntimeComponent)' == ''">true</IsRuntimeComponent>
</PropertyGroup>

<ItemGroup>
<!-- passed by compile-native.proj to set - -gcc-toolchain=$(ROOTFS_DIR)/usr -->
am11 marked this conversation as resolved.
Show resolved Hide resolved
<CustomLinkerArg Condition="'$(CustomLinkerArgToolchainArg)' != ''" Include="$(CustomLinkerArgToolchainArg)" />
</ItemGroup>
</Project>
2 changes: 1 addition & 1 deletion src/tests/Common/XUnitLogChecker/XUnitLogChecker.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<TargetFramework>$(NetCoreAppToolCurrent)</TargetFramework>
<GenerateRuntimeConfigurationFiles>true</GenerateRuntimeConfigurationFiles>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
<PublishAot Condition="'$(NativeAotSupported)' == 'true'">true</PublishAot>
<PublishAot Condition="'$(UseNativeAotForComponents)' == 'true'">true</PublishAot>
<SuppressTrimAnalysisWarnings>false</SuppressTrimAnalysisWarnings>
</PropertyGroup>

Expand Down
Loading