Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test #1

Draft
wants to merge 25 commits into
base: master
Choose a base branch
from
Draft

test #1

wants to merge 25 commits into from

Conversation

thesamesam
Copy link
Owner

No description provided.

@thesamesam thesamesam force-pushed the ci-rework branch 2 times, most recently from 005e455 to 06bdcb2 Compare April 16, 2024 04:06
thesamesam added a commit that referenced this pull request Apr 16, 2024
TODO: fix the actual change

Arithmetic on a null-pointer is UB in C, even if it is okay on most
platforms.

With Clang 17 and UBSAN, we get:
```
$ src/xz/xz -T1 --block-list=1MB
args.c:173:11: runtime error: applying non-zero offset 1 to null pointer
    #0 0x556f6bc848c1 in parse_block_list xz/src/xz/args.c:173:11
    #1 0x556f6bc848c1 in parse_real xz/src/xz/args.c:602:4
    #2 0x556f6bc830cb in args_parse xz/src/xz/args.c:787:2
    tukaani-project#3 0x556f6bc8ceef in main xz/src/xz/main.c:188:2
    tukaani-project#4 0x7f69f160de69  (/usr/lib64/libc.so.6+0x25e69)
    tukaani-project#5 0x7f69f160df24 in __libc_start_main (/usr/lib64/libc.so.6+0x25f24)
    tukaani-project#6 0x556f6bc59a00 in _start (xz/src/xz/.libs/xz+0x12a00)
```

First noticed and prompted investigation via Clang's scan-build which pointed
out by scan-build which reported that a possibly-NULL pointer could be passed
to `strchr()` - but it was kind of missing the point.

See also the discussion on bug-gnulib from December 2023 [0].

[0] https://lists.gnu.org/archive/html/bug-gnulib/2023-12/msg00002.html

See also 30e95bb.

Fixes: 88ccf47
thesamesam added a commit that referenced this pull request Apr 19, 2024
TODO: fix the actual change

Arithmetic on a null-pointer is UB in C, even if it is okay on most
platforms.

With Clang 17 and UBSAN, we get:
```
$ src/xz/xz -T1 --block-list=1MB
args.c:173:11: runtime error: applying non-zero offset 1 to null pointer
    #0 0x556f6bc848c1 in parse_block_list xz/src/xz/args.c:173:11
    #1 0x556f6bc848c1 in parse_real xz/src/xz/args.c:602:4
    #2 0x556f6bc830cb in args_parse xz/src/xz/args.c:787:2
    tukaani-project#3 0x556f6bc8ceef in main xz/src/xz/main.c:188:2
    tukaani-project#4 0x7f69f160de69  (/usr/lib64/libc.so.6+0x25e69)
    tukaani-project#5 0x7f69f160df24 in __libc_start_main (/usr/lib64/libc.so.6+0x25f24)
    tukaani-project#6 0x556f6bc59a00 in _start (xz/src/xz/.libs/xz+0x12a00)
```

First noticed and prompted investigation via Clang's scan-build which pointed
out by scan-build which reported that a possibly-NULL pointer could be passed
to `strchr()` - but it was kind of missing the point.

See also the discussion on bug-gnulib from December 2023 [0].

[0] https://lists.gnu.org/archive/html/bug-gnulib/2023-12/msg00002.html

See also 30e95bb.

Fixes: 88ccf47
thesamesam added a commit that referenced this pull request Apr 20, 2024
TODO: fix the actual change

Arithmetic on a null-pointer is UB in C, even if it is okay on most
platforms.

With Clang 17 and UBSAN, we get:
```
$ src/xz/xz -T1 --block-list=1MB
args.c:173:11: runtime error: applying non-zero offset 1 to null pointer
    #0 0x556f6bc848c1 in parse_block_list xz/src/xz/args.c:173:11
    #1 0x556f6bc848c1 in parse_real xz/src/xz/args.c:602:4
    #2 0x556f6bc830cb in args_parse xz/src/xz/args.c:787:2
    tukaani-project#3 0x556f6bc8ceef in main xz/src/xz/main.c:188:2
    tukaani-project#4 0x7f69f160de69  (/usr/lib64/libc.so.6+0x25e69)
    tukaani-project#5 0x7f69f160df24 in __libc_start_main (/usr/lib64/libc.so.6+0x25f24)
    tukaani-project#6 0x556f6bc59a00 in _start (xz/src/xz/.libs/xz+0x12a00)
```

First noticed and prompted investigation via Clang's scan-build which pointed
out by scan-build which reported that a possibly-NULL pointer could be passed
to `strchr()` - but it was kind of missing the point.

See also the discussion on bug-gnulib from December 2023 [0].

[0] https://lists.gnu.org/archive/html/bug-gnulib/2023-12/msg00002.html

See also 30e95bb.

Fixes: 88ccf47
thesamesam added a commit that referenced this pull request Apr 23, 2024
TODO: fix the actual change

Arithmetic on a null-pointer is UB in C, even if it is okay on most
platforms.

With Clang 17 and UBSAN, we get:
```
$ src/xz/xz -T1 --block-list=1MB
args.c:173:11: runtime error: applying non-zero offset 1 to null pointer
    #0 0x556f6bc848c1 in parse_block_list xz/src/xz/args.c:173:11
    #1 0x556f6bc848c1 in parse_real xz/src/xz/args.c:602:4
    #2 0x556f6bc830cb in args_parse xz/src/xz/args.c:787:2
    tukaani-project#3 0x556f6bc8ceef in main xz/src/xz/main.c:188:2
    tukaani-project#4 0x7f69f160de69  (/usr/lib64/libc.so.6+0x25e69)
    tukaani-project#5 0x7f69f160df24 in __libc_start_main (/usr/lib64/libc.so.6+0x25f24)
    tukaani-project#6 0x556f6bc59a00 in _start (xz/src/xz/.libs/xz+0x12a00)
```

First noticed and prompted investigation via Clang's scan-build which pointed
out by scan-build which reported that a possibly-NULL pointer could be passed
to `strchr()` - but it was kind of missing the point.

See also the discussion on bug-gnulib from December 2023 [0].

[0] https://lists.gnu.org/archive/html/bug-gnulib/2023-12/msg00002.html

See also 30e95bb.

Fixes: 88ccf47
thesamesam added a commit that referenced this pull request Apr 23, 2024
TODO: fix the actual change

Arithmetic on a null-pointer is UB in C, even if it is okay on most
platforms.

With Clang 17 and UBSAN, we get:
```
$ src/xz/xz -T1 --block-list=1MB
args.c:173:11: runtime error: applying non-zero offset 1 to null pointer
    #0 0x556f6bc848c1 in parse_block_list xz/src/xz/args.c:173:11
    #1 0x556f6bc848c1 in parse_real xz/src/xz/args.c:602:4
    #2 0x556f6bc830cb in args_parse xz/src/xz/args.c:787:2
    tukaani-project#3 0x556f6bc8ceef in main xz/src/xz/main.c:188:2
    tukaani-project#4 0x7f69f160de69  (/usr/lib64/libc.so.6+0x25e69)
    tukaani-project#5 0x7f69f160df24 in __libc_start_main (/usr/lib64/libc.so.6+0x25f24)
    tukaani-project#6 0x556f6bc59a00 in _start (xz/src/xz/.libs/xz+0x12a00)
```

First noticed and prompted investigation via Clang's scan-build which pointed
out by scan-build which reported that a possibly-NULL pointer could be passed
to `strchr()` - but it was kind of missing the point.

See also the discussion on bug-gnulib from December 2023 [0].

[0] https://lists.gnu.org/archive/html/bug-gnulib/2023-12/msg00002.html

See also 30e95bb.

Fixes: 88ccf47
thesamesam added a commit that referenced this pull request Apr 25, 2024
TODO: fix the actual change

Arithmetic on a null-pointer is UB in C, even if it is okay on most
platforms.

With Clang 17 and UBSAN, we get:
```
$ src/xz/xz -T1 --block-list=1MB
args.c:173:11: runtime error: applying non-zero offset 1 to null pointer
    #0 0x556f6bc848c1 in parse_block_list xz/src/xz/args.c:173:11
    #1 0x556f6bc848c1 in parse_real xz/src/xz/args.c:602:4
    #2 0x556f6bc830cb in args_parse xz/src/xz/args.c:787:2
    tukaani-project#3 0x556f6bc8ceef in main xz/src/xz/main.c:188:2
    tukaani-project#4 0x7f69f160de69  (/usr/lib64/libc.so.6+0x25e69)
    tukaani-project#5 0x7f69f160df24 in __libc_start_main (/usr/lib64/libc.so.6+0x25f24)
    tukaani-project#6 0x556f6bc59a00 in _start (xz/src/xz/.libs/xz+0x12a00)
```

First noticed and prompted investigation via Clang's scan-build which pointed
out by scan-build which reported that a possibly-NULL pointer could be passed
to `strchr()` - but it was kind of missing the point.

See also the discussion on bug-gnulib from December 2023 [0].

[0] https://lists.gnu.org/archive/html/bug-gnulib/2023-12/msg00002.html

See also 30e95bb.

Fixes: 88ccf47
@thesamesam thesamesam force-pushed the ci-rework branch 4 times, most recently from 5223a2a to 0f136e1 Compare May 2, 2024 06:29
@thesamesam thesamesam force-pushed the ci-rework branch 2 times, most recently from a6619ed to ce9fbef Compare June 3, 2024 02:55
Larhzu and others added 11 commits June 3, 2024 11:44
The main feature was broken because an old variable name hadn't
been updated to match the rest of the script.
Solaris' GCC can't understand that our use is fine, unlike modern compilers:
```
list.c: In function 'print_totals_basic':
list.c:1191:4: error: format not a string literal, argument types not checked [-Werror=format-nonliteral]
  uint64_to_str(totals.files, 0));
  ^~~~~~~~~~~~~
cc1: all warnings being treated as errors
```

It's presumably because of older gettext missing format attributes.

This is with `gcc (GCC) 7.3.0`.
Inspired by RsyncProject/rsync@3f2a38b.

It runs on Solaris 5.11 via a VirtualBox VM.
Without "set -e" the job will always be successful.

See vmactions/freebsd-vm#72
There's no real value in doing it via commit for official GH actions. We
can keep using pinned commits for unofficial actions. It's hassle for no
gain.

Maybe going forward we can limit this further by only being paranoid
for the jobs with any access to tokens.
The only non-trivial part here was macOS where 'gcc' is actually Apple Clang
by default, so we have to go out of our way to get GCC from Homebrew, and do
the same for Clang.
thesamesam and others added 14 commits June 3, 2024 12:29
scan-build reports a possible dangling reference in `uncompress()`, where
`strm->avail_in` points to no-longer-valid (stack) `out_buf` on return.

This shouldn't be a problem in reality because we should be in LZMA_FINISH
at this point and therefore callers shouldn't do anything with `strm->avail_in`,
but it's simple enough for us to NULL-out `out_buf`, so let's do it.

Ditto for the case with LZMA_STREAM_END.

TODO: check this reasoning is right!
Solaris' sed appears broken and chokes on the old version:
```
running build-aux/version.sh
chunk 1 (sed)
5
7
0
alpha
chunk 2 (sed + tr)
5|7|0|alpha|
chunk 3 (sed + tr + sed)
chunk 4 (sed + tr + sed + tr)
```

vs (good):
```
running build-aux/version-gsed.sh
chunk 1 (gsed)
5
7
0
alpha
chunk 2 (gsed + tr)
5|7|0|alpha|
chunk 3 (gsed + tr + gsed)
5.7.0alpha
chunk 4 (gsed + tr + gsed + tr)
5.7.0alpha
```

Rather than requiring GNU sed, rewrite in POSIX awk. Thank you to Kerin
Millar for the rewrite.

Unfortunately, we cannot rely on the xpg4 POSIX versions, because of:
"""
Only the core-os packages are guaranteed to be present on a Solaris 11
regular installation, thus only oawk/awk and nawk are there. In particular,
when you create a new non global zone, it contains by default the
solaris-small-server group package so neither the xpg4 nor the gnu awk
binaries are available.
"""

As a bonus, it's IMO far more legible now. Given Recent Events, the fewer
immediately-obvious sed expressions in the world, the better.

Thanks-to: Kerin Millar <[email protected]>
```
lzmainfo.c: In function ‘my_log2’:
lzmainfo.c:92:1: warning: function might be candidate for attribute ‘const’ if it is known to return normally [-Wsuggest-attribute=const]
   92 | my_log2(uint32_t n)
      | ^~~~~~~
```

GCC is right.
xz is already part of oss-fuzz, but oss-fuzz provides & encourages [0]
its 'CIFuzz' service to test individual PRs and commits.

I think it can also run on a schedule but I haven't added that in (yet?)

[0] https://google.github.io/oss-fuzz/getting-started/continuous-integration/
SUSv2 doesn't seem to allow semicolons to separate commands;
only newlines work. The old version didn't work with /bin/sed
on Solaris although it did with /usr/xpg4/bin/sed.
This addresses the issue I mentioned in 6c095a9
and speeds up the Valgrind job a bit, because non-xz tools aren't run
unnecessarily with Valgrind by the script tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants