libpkg: track lib32 and Linuxulator shlibs

[ifreund]

When scanning ELF files to generate the shlibs_provided/required lists, pkg now generates entries for lib32 and Linux compat shlibs rather than ignoring all non-native shlibs.

The following format is used for shlibs_provided/required entries:

libfoo.so.1.0.0          - native (no change to status quo)
libfoo.so.1.0.0:32       - compat 32
libfoo.so.1.0.0:Linux    - compat Linux
libfoo.so.1.0.0:Linux:32 - compat Linux 32

This is only done if targeting FreeBSD for now.

I've marked this as a draft because it depends on the unmerged #2386 and because I would like to add more tests before it is merged.

References: #2331

[ifreund]

Note: this is not yet sufficient to fully close #2331 as lib32 and Linuxulator shlibs are not yet backed up if BACKUP_LIBRARIES is set.

[ifreund]

Hmm, this PR causes the backup_lib test to fail on Debian for some reason. I will investigate tomorrow.

[ifreund]

I've realized another blocker to resolve before merging this PR: We need to scan /usr/lib32 and build a list of base system-provided 32bit shlibs if pkgbase is not in use. (Just like we scan /lib and /usr/lib for native shlibs since #2386).

[ifreund]

The backup_lib test is failing on Debian because pkg fails to determine that libempty.so is a linux shlib despite it being compiled for the Debian host system using the Debian toolchain. Manually examining a libempty.so produced the same way as in the test on Debian make it clear why pkg's OS detection fails, the NT_GNU_ABI_TAG style note in the ELF notes section is not present:

build@build:/tmp$ readelf -n libempty.so.1 

Displaying notes found in: .note.gnu.build-id
  Owner                Data size 	Description
  GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)
    Build ID: d4398ec29c5fe1c5204bd739c77505f78c15c98e
build@build:/tmp$ 

Furthermore, the OS set in the elf header is very generic, so it wouldn't make sense for pkg to fall back on checking that:

build@build:/tmp$ readelf -h libempty.so.1
ELF Header:
  Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 
  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              DYN (Shared object file)
  Machine:                           Advanced Micro Devices X86-64
  Version:                           0x1
  Entry point address:               0x0
  Start of program headers:          64 (bytes into file)
  Start of section headers:          13384 (bytes into file)
  Flags:                             0x0
  Size of this header:               64 (bytes)
  Size of program headers:           56 (bytes)
  Number of program headers:         8
  Size of section headers:           64 (bytes)
  Number of section headers:         23
  Section header string table index: 22

This will also be an issue for detecting Linuxulator shlibs on FreeBSD though we don't have tests for that yet. I will investigate if there is a reliable way to detect ELF files compiled for Linux though I'm a bit worried that there is no universal solution here due to the fragmentation of the Linux ecosystem.

[ifreund]

From my research, it seems to be impossible to reliably determine if an ELF shared library is targeting Linux.

It is possible to reliably determine that an executable targets Linux by checking DT_INTERP or NT_GNU_ABI_TAG but DT_INTERP is specifc to executables and NT_GNU_ABI_TAG is only required for executables.

The EI_OSABI value in the ELF header is not consistently set to ELFOSABI_LINUX, it usually seems to be left set to ELFOSABI_NONE aka "UNIX System V ABI."

I see 2 obvious paths we could take:

1. Assume shared libraries for which we cannot determine an OS are Linux shlibs.
2. Require that the user of pkg (i.e. the ports system) use brandelf(1) or similar to ensure all Linux ELF files have the necessary metadata.

I implemented option 1. in my latest commit to this branch but would be happy to take a different approach if someone has a better idea.

[bapt]

I don't think we can do better that implementing 1 for now.
but what I think we should do it add an option to enable/disable it for now, so we issue a release with out tracking the linux libs first and then we can make some experimental repos with the tracking activated.

If we are happy with the result, then we can enable it and remove the option to disable it.

TBH I don't know a case where 1 would be wrong.

[ifreund]

@bapt that seems reasonable to me, I added a TRACK_LINUX_COMPAT_SHLIBS option that defaults to FALSE.