(String) Unicode Data Table Support via PDCPlugin?

[tyetrask]

With String landing in Embedded Swift, it's been really nice working in PlaydateKit but I still find myself wanting more, if not the full, String API 😅 (comparison, iteration, string keys in dictionaries, etc.).

I'm curious if you're interested and/or were planning to add this as an option to the PDC plugin so that users could choose whether to opt into linking the data tables?

I've been experimenting with how to link the unicode data tables to get this but I'm not having much luck so far (I have very little experience with compilation/linking so I apologize in advance if what I've tried isn't making much sense).

(I've posted about it in the Playdate Squad discord here in case any onlookers might be able to assist as well.)

After looking at the Swift Embedded user manual, I understood the gist and set out to modify PDCPlugin's clang and cc command flags.

let triple = "armv7em-none-none-eabi"
let toolchainPath = "\(home)Library/Developer/Toolchains/swift-latest.xctoolchain"
let unicodeDataTablesPathDevice = "\(toolchainPath)/usr/lib/swift/embedded/\(triple)"
let unicodeDataTablesPathSimulator = "\(toolchainPath)/usr/lib/swift/embedded/arm64-apple-none-macho"
let unicodeDataTablesLibName = "swiftUnicodeDataTables" // libswiftUnicodeDataTables.a

For the simulator build, this seems to work:

try clang([
    "-nostdlib", "-dead_strip",
    "-Wl,-exported_symbol,_eventHandlerShim", "-Wl,-exported_symbol,_eventHandler",
    "-Wl,-L\(unicodeDataTablesPathSimulator),-l\(unicodeDataTablesLibName)",
    productSimulatorPath.string, "-dynamiclib", "-rdynamic", "-lm",
    "-DTARGET_SIMULATOR=1", "-DTARGET_EXTENSION=1",
    "-I", ".",
    "-I", "\(playdateSDK)/C_API",
    "-o", sourcePath.appending(["pdex.dylib"]).string,
    "\(playdateSDK)/C_API/buildsupport/setup.c"
])

However, the gcc build for device, does not:

try cc([setup, productDevicePath.string] + mcFlags + [
    "-Wl,-L\(unicodeDataTablesPathDevice),-Bstatic,-l\(unicodeDataTablesLibName)",
    "-T\(playdateSDK)/C_API/buildsupport/link_map.ld",
    "-Wl,-Map=\(context.pluginWorkDirectory.appending(["pdex.map"]).string),--cref,--gc-sections,--no-warn-mismatch,--emit-relocs",
    "-o", sourcePath.appending(["pdex.elf"]).string
])

(I've tried without -Bstatic as well as moving around the order of the arguments in case that would seem to matter 🙈.)

With the latter, I still see the compiler errors about failed linking:

(...)
.$ss7UnicodeO14_NFCNormalizerV7_resume12consumingNFDAB6ScalarVSgAH6scalar_AB9_NormDataV04normI0tSgADzXE_tF04$ss7a4O14_b21V6resume9consumingAB6f25VSgAIyXE_tFAH6scalar_AB9_hI20V04normH0tSgADzXEfU_AIIgd_Tf1cn_nTf4ng_n+0x112): undefined reference to `_swift_stdlib_getComposition'
/usr/local/playdate/gcc-arm-none-eabi-9-2019-q4-major/bin/../lib/gcc/arm-none-eabi/9.2.1/../../../../arm-none-eabi/bin/ld: //<compiler-generated>:(.text.$ss7UnicodeO14_NFCNormalizerV7_resume12consumingNFDAB6ScalarVSgAH6scalar_AB9_NormDataV04normI0tSgADzXE_tF04$ss7a4O14_b21V6resume9consumingAB6f25VSgAIyXE_tFAH6scalar_AB9_hI20V04normH0tSgADzXEfU_AIIgd_Tf1cn_nTf4ng_n+0x132): undefined reference to `_swift_stdlib_getComposition'
/usr/local/playdate/gcc-arm-none-eabi-9-2019-q4-major/bin/../lib/gcc/arm-none-eabi/9.2.1/../../../../arm-none-eabi/bin/ld: /Users/<home>/Projects/PDXSemiGameSwift/.build/plugins/PDCPlugin/outputs/Source/pdex.elf: hidden symbol `_swift_stdlib_getDecompositionEntry' isn't defined
/usr/local/playdate/gcc-arm-none-eabi-9-2019-q4-major/bin/../lib/gcc/arm-none-eabi/9.2.1/../../../../arm-none-eabi/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status
[2K
[31m[1merror: [0mccFailed(exitCode: 1)

I've tried passing -Wl,--verbose to see if I can suss out what's happening but I'm just not familiar enough. I did manage to verify that ld seems to at least be finding the library correctly but I don't know enough to know why the linking still fails.

[paulstraw]

@finnvoor do you have any thoughts on this issue/Tye's general approach? I've been playing with this without luck so far, and wanted to check in before spending time on a deeper dive.