Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update kernel digests for DSM 7.2 #6317

Merged
merged 7 commits into from
Nov 10, 2024

Conversation

emanspeaks
Copy link
Contributor

@emanspeaks emanspeaks commented Nov 10, 2024

Description

Synology updated the kernel sources from DSM 7.2 on 08.11.2024.
All kernel 7.2 sources are updated, except for kvmx64 that is still from 05.10.2023.

Something seems to have changed the checksums for the Denverton 7.2 kernel sources on the Synology site (https://global.download.synology.com/download/ToolChain/Synology%20NAS%20GPL%20Source/7.2-64570/denverton/linux-4.4.x.txz), which was causing 7.2 builds to fail. This may be a wider issue with 7.2 builds, but can only attest for Denverton.

Fixes #6314
Resubmission of #6316

Additional updates

  • remove all generic kernel files
  • add armv7-7.2 to build matrix for DSM 7.2
  • declare rtd1619b as unsupported in synokernel-* packages

Checklist

  • Build rule all-supported completed successfully
  • New installation of package completed successfully
  • Package upgrade completed successfully (Manually install the package again)
  • Package functionality was tested
  • Any needed documentation is updated/created

Type of change

  • Bug fix
  • New Package
  • Package update
  • Includes small framework changes
  • This change requires a documentation update (e.g. Wiki)

@emanspeaks emanspeaks changed the title update kernel digests for denverton 7.2 Update kernel digests for denverton 7.2 Nov 10, 2024
@emanspeaks emanspeaks closed this Nov 10, 2024
@emanspeaks emanspeaks deleted the update-denverton-7.2-kernel branch November 10, 2024 17:39
@emanspeaks emanspeaks restored the update-denverton-7.2-kernel branch November 10, 2024 18:17
@emanspeaks emanspeaks reopened this Nov 10, 2024
@hgy59
Copy link
Contributor

hgy59 commented Nov 10, 2024

@emanspeaks all kernel source packages for DSM 7.2 were updated on 8.11.2024, not only denverton

@hgy59
Copy link
Contributor

hgy59 commented Nov 10, 2024

@th0ma7 while analyzing the kernel Makefiles, I found that we have those for the generic archs aarch64 and armv7 too (but not for x64).
IMHO we must not build generic packages when kernel sources are required.
When I am wrong we sould add x64 configs for kernel too,
but if I am right, we sould remove the armv7 and aarch64 kernels (for all DSM versions).

@emanspeaks
Copy link
Contributor Author

@hgy59 is there a simple way to update all the digests? I generated these manually for Denverton but don't want to sign up for doing that for all archs unless there's a script or something I can run to automate it

@hgy59
Copy link
Contributor

hgy59 commented Nov 10, 2024

Here a differency report of the denverton 7.2 kernel sources

Binary Comparison of 2024_denverton-linux-4.4.x to 2023_denverton-linux-4.4.x
57142 files in 3953 folders

57087 files match exactly
-------------------------

38 files don't match
--------------------
	arch\x86\boot\compressed\head_64.S
	drivers\block\virtio_blk.c
	drivers\scsi\sd.c
	drivers\scsi\sd.h
	drivers\scsi\virtio_scsi.c
	fs\namespace.c
	include\linux\syno.h
	include\linux\synobios.h
	include\linux\synolib.h
	init\initramfs.c
	kernel\sysctl.c
	lib\synolib\syno_kexec_test.c
	scripts\kallsyms.c
	SynoBuildConf\build
	SynoBuildConf\depends
	SynoBuildConf\install-dev-virtual-headers
	synoconfigs\apollolake
	synoconfigs\armada37xx
	synoconfigs\broadwell
	synoconfigs\broadwellnk
	synoconfigs\broadwellnkv2
	synoconfigs\broadwellntb
	synoconfigs\broadwellntbap
	synoconfigs\coffeelake
	synoconfigs\denverton
	synoconfigs\epyc7002
	synoconfigs\geminilake
	synoconfigs\geminilakevs
	synoconfigs\icelaked
	synoconfigs\kvmcloud
	synoconfigs\kvmx64
	synoconfigs\nextkvmx64
	synoconfigs\purley
	synoconfigs\r1000
	synoconfigs\rtd1296
	synoconfigs\rtd1619
	synoconfigs\skylaked
	synoconfigs\v1000

19 folders and files only on left
---------------------------------
	crypto\hydrogen\impl\
	crypto\hydrogen\impl\gimli-core\
	crypto\hydrogen\impl\gimli-core\portable.h
	crypto\hydrogen\impl\gimli-core\ssse3.h
	crypto\hydrogen\impl\common.h
	crypto\hydrogen\impl\core.h
	crypto\hydrogen\impl\gimli-core.h
	crypto\hydrogen\impl\hash.h
	crypto\hydrogen\impl\hydrogen_p.h
	crypto\hydrogen\impl\kdf.h
	crypto\hydrogen\impl\kx.h
	crypto\hydrogen\impl\random.h
	crypto\hydrogen\impl\secretbox.h
	crypto\hydrogen\impl\sign.h
	crypto\hydrogen\impl\x25519.h
	crypto\hydrogen\hydrogen.c
	include\crypto\hydrogen.h
	include\linux\syno_model_info.h
	SynoBuildConf\depends-virtual-headers

The mandatory changes are the additional crypto\hydrogen\
and the kernel configurations have some differences:
Some configs are removed with the update

CONFIG_SYSTEM_DATA_VERIFICATION=y 
CONFIG_SYNO_KEXEC_TEST=y
CONFIG_MODULE_SIG=y
CONFIG_SYSTEM_TRUSTED_KEYRING=y

@hgy59
Copy link
Contributor

hgy59 commented Nov 10, 2024

@hgy59 is there a simple way to update all the digests? I generated these manually for Denverton but don't want to sign up for doing that for all archs unless there's a script or something I can run to automate it

Yes (I am already working on this).
you can call make digests in the kernel folders, but unfortunately when calling make digests or even make clean the toolchain is triggered to download (and rust toolchain is installed too).

So it takes some time.

I am already working on this, and want to cleanup the generic archs too (see my question above).

@hgy59
Copy link
Contributor

hgy59 commented Nov 10, 2024

@emanspeaks If you don't mind, I can update the digests for the other kernels in your working branch.

@emanspeaks
Copy link
Contributor Author

@hgy59 yes that is ok with me! Let me know if I need to do something to allow you.

@th0ma7
Copy link
Contributor

th0ma7 commented Nov 10, 2024

while analyzing the kernel Makefiles, I found that we have those for the generic archs aarch64 and armv7 too (but not for x64).
IMHO we must not build generic packages when kernel sources are required.
When I am wrong we sould add x64 configs for kernel too,
but if I am right, we sould remove the armv7 and aarch64 kernels (for all DSM versions).

From a resulting kernel modules, no, generic kernel folders has no reason to exists. Kernel modules must exactly match the target kernel and as such generic are useless.

I have a feeling these are no longer needed unless i had used it for looping over sub-arches, but even that i should have used toolchain definitions (I don't have access to my build system so cannot confirm).

@th0ma7
Copy link
Contributor

th0ma7 commented Nov 10, 2024

@hgy59 is there a simple way to update all the digests? I generated these manually for Denverton but don't want to sign up for doing that for all archs unless there's a script or something I can run to automate it

Yes (I am already working on this).
you can call make digests in the kernel folders, but unfortunately when calling make digests or even make clean the toolchain is triggered to download (and rust toolchain is installed too).

So it takes some time.

I am already working on this, and want to cleanup the generic archs too (see my question above).

That may be tricky as by default it does prepare the kernel so it is in a usable state when performing a basic install as its not only a matter of extracting but rather setting up the config and a make mrproper or similar being called.

Although upon download indeed it would be practical to skip the toolchain part.

@hgy59
Copy link
Contributor

hgy59 commented Nov 10, 2024

I have a feeling these are no longer needed unless i had used it for looping over sub-arches, but even that i should have used toolchain definitions (I don't have access to my build system so cannot confirm).

Yes, the loop is over generic toolchain definition, kernel Makefiles have only one arch defined

@hgy59
Copy link
Contributor

hgy59 commented Nov 10, 2024

The build of synokernel-* packages for rtd1619b does not work (for DSM 7.1 and DSM 7.2)

error: Cannot resolve BTF IDs for CONFIG_DEBUG_INFO_BTF, please install libelf-dev, libelf-devel or elfutils-libelf-devel
make[4]: *** [Makefile:1247: prepare-resolve_btfids] Error 1
make[4]: Leaving directory '/spksrc/spk/synokernel-cdrom/work-rtd1619b-7.1/linux-rtd1619b-7.1'
make[3]: *** [../../mk/spksrc.cross-kernel-module.mk:57: kernel_module_prepare] Error 2
make[3]: Leaving directory '/spksrc/kernel/syno-rtd1619b-7.1'

It might be the same reason as epyc7002 is known to be unsupported.

NB the models with rtd1619b and epy7002 have linux kernel 5.x
all other models have kernel 4.x, 3.x or 2.x

Found issue:
When building synokernel-* for aarch64, the build for rtd1619b fails, but the build does not break, the package is successfully created and contains modules for rtd1296 and armada37xx only.

@hgy59
Copy link
Contributor

hgy59 commented Nov 10, 2024

@mreid-tt just found that with REQUIRED_MIN_DSM = 7.2 only x64 and aarch64 packages are built.
It sould also build for armv7.

@th0ma7
Copy link
Contributor

th0ma7 commented Nov 10, 2024

The build of synokernel-* packages for rtd1619b does not work (for DSM 7.1 and DSM 7.2)

error: Cannot resolve BTF IDs for CONFIG_DEBUG_INFO_BTF, please install libelf-dev, libelf-devel or elfutils-libelf-devel
make[4]: *** [Makefile:1247: prepare-resolve_btfids] Error 1
make[4]: Leaving directory '/spksrc/spk/synokernel-cdrom/work-rtd1619b-7.1/linux-rtd1619b-7.1'
make[3]: *** [../../mk/spksrc.cross-kernel-module.mk:57: kernel_module_prepare] Error 2
make[3]: Leaving directory '/spksrc/kernel/syno-rtd1619b-7.1'

It might be the same reason as epyc7002 is known to be unsupported.

NB the models with rtd1619b and epy7002 have linux kernel 5.x
all other models have kernel 4.x, 3.x or 2.x

Found issue:
When building synokernel-* for aarch64, the build for rtd1619b fails, but the build does not break, the package is successfully created and contains modules for rtd1296 and armada37xx only.

Epyc7002 was known, i had issues with cross-compiling the libelf mandatory dependency (actually wonder if that wasn't native). Although i thought rtd1619b was working out, but may be now facing the same issue with new updates to 7.2 kernel sources.

@hgy59
Copy link
Contributor

hgy59 commented Nov 10, 2024

Epyc7002 was known, i had issues with cross-compiling the libelf mandatory dependency (actually wonder if that wasn't native). Although i thought rtd1619b was working out, but may be now facing the same issue with new updates to 7.2 kernel sources.

the error message for epyc7002:

error: Cannot generate ORC metadata for CONFIG_UNWINDER_ORC=y, please install libelf-dev, libelf-devel or elfutils-libelf-devel
make[4]: *** [Makefile:1238: prepare-objtool] Error 1
make[4]: Leaving directory '/spksrc/spk/synokernel-cdrom/work-epyc7002-7.2/linux-epyc7002-7.2'
make[3]: *** [../../mk/spksrc.cross-kernel-module.mk:57: kernel_module_prepare] Error 2
make[3]: Leaving directory '/spksrc/kernel/syno-epyc7002-7.2'

both have the libelf dependency error.

@hgy59
Copy link
Contributor

hgy59 commented Nov 10, 2024

both have the libelf dependency error.

even with installed libelf-dev, the synokernel packages fail to build for epyc7002 and rtd1619b for DSM 7.1 and 7.2
with the same error:

  CC       /spksrc/spk/synokernel-usbserial/work-rtd1619b-7.1/linux-rtd1619b-7.1/tools/bpf/resolve_btfids/libsubcmd/help.o
In file included from help.c:12:
In function ‘xrealloc’,
    inlined from ‘add_cmdname’ at help.c:24:2:
subcmd-util.h:56:23: error: pointer may be used after ‘realloc’ [-Werror=use-after-free]
   56 |                 ret = realloc(ptr, size);
      |                       ^~~~~~~~~~~~~~~~~~
subcmd-util.h:52:21: note: call to ‘realloc’ here
   52 |         void *ret = realloc(ptr, size);
      |                     ^~~~~~~~~~~~~~~~~~
subcmd-util.h:58:31: error: pointer may be used after ‘realloc’ [-Werror=use-after-free]
   58 |                         ret = realloc(ptr, 1);
      |                               ^~~~~~~~~~~~~~~
subcmd-util.h:52:21: note: call to ‘realloc’ here
   52 |         void *ret = realloc(ptr, size);
      |                     ^~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors

@hgy59 hgy59 changed the title Update kernel digests for denverton 7.2 Update kernel digests for DSM 7.2 Nov 10, 2024
@mreid-tt
Copy link
Contributor

mreid-tt commented Nov 10, 2024

@mreid-tt just found that with REQUIRED_MIN_DSM = 7.2 only x64 and aarch64 packages are built. It sould also build for armv7.

Really good catch @hgy59, the users of Jellyfin on the following models could theoretically upgrade beyond v10.8 now:

DS419slim, DS218j, RS217, DS116, DS216j, DS216, DS416slim, RS816, DS416j, DS216play

Given the finding in #6309, this should not affect these models as you mentioned that DSM 7.2 has GLIBC 2.36.

EDIT: Actually, no these are still 32-bit archs which were previously excluded for performance issues as well as no support for software or hardware transcoding.

@th0ma7
Copy link
Contributor

th0ma7 commented Nov 10, 2024

both have the libelf dependency error.

even with installed libelf-dev, the synokernel packages fail to build for epyc7002 and rtd1619b for DSM 7.1 and 7.2 with the same error:

  CC       /spksrc/spk/synokernel-usbserial/work-rtd1619b-7.1/linux-rtd1619b-7.1/tools/bpf/resolve_btfids/libsubcmd/help.o
In file included from help.c:12:
In function ‘xrealloc’,
    inlined from ‘add_cmdname’ at help.c:24:2:
subcmd-util.h:56:23: error: pointer may be used after ‘realloc’ [-Werror=use-after-free]
   56 |                 ret = realloc(ptr, size);
      |                       ^~~~~~~~~~~~~~~~~~
subcmd-util.h:52:21: note: call to ‘realloc’ here
   52 |         void *ret = realloc(ptr, size);
      |                     ^~~~~~~~~~~~~~~~~~
subcmd-util.h:58:31: error: pointer may be used after ‘realloc’ [-Werror=use-after-free]
   58 |                         ret = realloc(ptr, 1);
      |                               ^~~~~~~~~~~~~~~
subcmd-util.h:52:21: note: call to ‘realloc’ here
   52 |         void *ret = realloc(ptr, size);
      |                     ^~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors

Yup, I was facing this exact same error... I tried playing with the config to disable ORC* and other items but didn't manage to find a proper workaround.

@hgy59
Copy link
Contributor

hgy59 commented Nov 10, 2024

@mreid-tt just found that with REQUIRED_MIN_DSM = 7.2 only x64 and aarch64 packages are built. It sould also build for armv7.

Really good catch @hgy59, the users of Jellyfin on the following models could theoretically upgrade beyond v10.8 now:

DS419slim, DS218j, RS217, DS116, DS216j, DS216, DS416slim, RS816, DS416j, DS216play

Given the finding in #6309, this should not affect these models as you mentioned that DSM 7.2 has GLIBC 2.36.

EDIT: Actually, no these are still 32-bit archs which were previously excluded for performance issues as well as no support for software or hardware transcoding.

but dotnet on 32-bit archs has other issues...

@hgy59 hgy59 merged commit 94e0fd0 into SynoCommunity:master Nov 10, 2024
15 checks passed
Copy link
Contributor

@th0ma7 th0ma7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - any intent at releasing updated synokernel packages?

urosch pushed a commit to urosch/spksrc that referenced this pull request Dec 9, 2024
* update kernel digests for denverton 7.2

* update further kernel 7.2 digests

* remove kernel sources for generich archs (armv7, aarch64, x64)

* add armv7-7.2 to build matrix for dsm72

* declare rtd1619b as unsupported for synokernel-*

---------

Co-authored-by: hgy59 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Kernel checksums have changed for Denverton 7.2
4 participants