Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stage2: improved glibc stubs #10330

Merged
merged 1 commit into from
Dec 13, 2021
Merged

stage2: improved glibc stubs #10330

merged 1 commit into from
Dec 13, 2021

Conversation

andrewrk
Copy link
Member

@andrewrk andrewrk commented Dec 13, 2021

This commit upgrades glibc shared library stub-creating code to use the
new abilists file which is generated by the new glibc-abi-tool project:
https://github.com/ziglang/glibc-abi-tool/

The abilists file is different in these ways:

  • It additionally encodes whether a symbol is a function or an object,
    and if it is an object, it additionally encodes the size in bytes.
  • It additionally encodes migrations of symbols from one library to
    another between glibc versions.
  • It is binary data instead of ascii.
  • It is one file instead of three.
  • It is 165 KB instead of 200 KB.

This solves a handful of bugs.

Fixes #7667
Fixes #8714
Fixes #8896

cc @LemonBoy

I have not yet verified that all these issues are fixed; I will do that before merging the PR.

@motiejus
Copy link
Contributor

motiejus commented Dec 13, 2021

I don't think it will fix #5882, as it's a header problem, not a glibc-stub problem. Testing an example from #9485 with https://github.com/ziglang/zig/tree/glibc-stubs:

$ ~/code/zig/build/zig version
0.9.0-dev.1951+0d7331b6f
$ ~/code/zig/build/zig cc --target=x86_64-linux-gnu.2.27  main.c
ld.lld: error: undefined symbol: fcntl64
>>> referenced by main.c
>>>               /home/motiejus/.cache/zig/o/201c47b92a2605ce99b0da8a4c160de3/main.o:(main)

@marler8997
Copy link
Contributor

marler8997 commented Dec 13, 2021

I confirmed what @motiejus said by verifying that issue #9485 still reproduces with this branch. It appears that in order to really fix this issue, Zig would need a way to version the glibc headers as well (not just the libraries we link to). Maybe a new project is in order to solve this? I think it may be fairly ambitious to do this in an efficient way (i.e. not including an instance of the glibc headers for every version) but maybe we could start there and make incremental improvements? Maybe compression would be a good start to see how feasible it would be?

P.S. Actually "git" already has a good mechanism for this. If we thought of the glibc headers as a repo, and we only stored the "diff" between each version that might be a good solution. The zig compiler could use the initial version and the "diff database" to instantiate any version of the glibc headers as needed.

@andrewrk
Copy link
Member Author

I verified #7667 is fixed, thanks to this repro: #7667 (comment)

Not sure how to repro #8714 but it does look to be fixed. Zig now emits accurate glibc .so files that contain lseek64 in multiple libraries if that's what happened in that version.

#8896 is definitely fixed; I can see in the output of readelf -s that the objects are objects and they have sizes.

Thanks for letting me know about #5882 and #9485. Those will have to be addressed with yet another creative piece of engineering, separate from this PR.

This commit upgrades glibc shared library stub-creating code to use the
new abilists file which is generated by the new glibc-abi-tool project:
https://github.com/ziglang/glibc-abi-tool/

The abilists file is different in these ways:
 * It additionally encodes whether a symbol is a function or an object,
   and if it is an object, it additionally encodes the size in bytes.
 * It additionally encodes migrations of symbols from one library to
   another between glibc versions.
 * It is binary data instead of ascii.
 * It is one file instead of three.
 * It is 165 KB instead of 200 KB.

This solves three bugs:

Fixes #7667
Fixes #8714
Fixes #8896
@andrewrk andrewrk merged commit 1442aa7 into master Dec 13, 2021
@andrewrk andrewrk deleted the glibc-stubs branch December 13, 2021 23:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants