Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added new spk for activebackup-prometheus-exporter #4292

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

fleaz
Copy link

@fleaz fleaz commented Dec 8, 2020

Motivation:
I work at a company where we are responsible for the infrastructure in some office buildings and we use Disk- and Rackstations together with the backup tools from the ActiveBackup suite to backup virtual machines, physical hosts and Google Drives. Because this was a blank spot in our monitoring I started to build activebackup-prometheus-exporter which is an Prometheus exporter which presents the data scraped from the internal sqlite databases generated by the ActiveBackup tools to get informations about the backup jobs. So if you have a running Prometheus server, you can just add your Synology NAS as a target and scrape the data about the backup jobs after installing this tool and create alerts when e.g. the last backup is older than 24 hours or something like that.

I'm new to this whole Synocommunity thing and this is my first spk package I build. So if I did something wrong, please let my know and I will fix it :)

Due to the similarities I used the spk and cross paccakge from the normal Prometheus node-exporter as a starting point and changed it to my needs. The packages was tested on my DS218+ (apollolake) and on a RS1619xs+ (broadwellnk) both running DSM 6.2.

The only thing that is currently not working is that it's not recognized as "running" after an installation, even though it's running (checked with ps and I can curl it). Maybe someone more experienced can give me a hint on how to fix this?

Best regards,
Felix

Checklist

  • Build rule all-supported completed successfully
  • Package upgrade completed successfully
  • New installation of package completed successfully

@fleaz
Copy link
Author

fleaz commented Dec 15, 2020

Not sure what the problem is, but all of the failed Actions run sucessfully when I build the spk in the official Docker container... 😕

@hgy59
Copy link
Contributor

hgy59 commented Dec 16, 2020

@fleaz only x64 build succeeded, so it seems to be a cross compilation issue. May be @th0ma7 may help with this, as he created mk/spksrc.cross-go.mk.

@th0ma7
Copy link
Contributor

th0ma7 commented Dec 17, 2020

@fleaz only x64 build succeeded, so it seems to be a cross compilation issue. May be @th0ma7 may help with this, as he created mk/spksrc.cross-go.mk.

Sadly not, I did some work on the spksrc.cross-cmake.mk and now working on the meson side of things but nothing on the go side of things.

For the status output, if your code uses the regular src/service-setup.sh then you should end-up with a package.pid file under /var/package/<packagename>/target/var/<packagename>.pid. Automatically generated scritpts that are looking at the status (along with start|stop) are located under /var/package/<packagename>/scripts. You'll find out how it is proceeding to fetch the application status and may be able to figure why something's wrong (one possibility being pid not matching the real app running pid for instance).

@fleaz
Copy link
Author

fleaz commented Dec 17, 2020

@fleaz only x64 build succeeded, so it seems to be a cross compilation issue. May be @th0ma7 may help with this, as he created mk/spksrc.cross-go.mk.

As statet in my first comment, all archs build sucessfully when I build them on my machine in the spksrc Docker container (Just copied the armv7 binary to one of my RaspberryPis to confirm that the binary is intact).

I just found out how to see the full logs of the GH Actions and found this line to be the error:

arm-unknown-linux-gnueabi-gcc: error: unrecognized command line option '-m64'

I have sadly no idea what the GH actions do different than my local container when I run make aarch-armv7 shrug

@th0ma7
Copy link
Contributor

th0ma7 commented Jan 12, 2021

I did found where the build error is:

===>  Building Go for host system
# GO111MODULE disable go.mod per https://github.com/golang/go/issues/26996
cd /home/spksrc/spksrc-fleaz/spksrc/native/go/work-native/go/src && GO111MODULE=off GOARCH="" GOROOT_BOOTSTRAP=/home/spksrc/spksrc-fleaz/spksrc/native/go/work-native/../../go-1.4/work-native/go ./make.bash
Building Go cmd/dist using /home/spksrc/spksrc-fleaz/spksrc/native/go/work-native/../../go-1.4/work-native/go. (go1.4-bootstrap-20170531 linux/amd64)
Building Go toolchain1 using /home/spksrc/spksrc-fleaz/spksrc/native/go/work-native/../../go-1.4/work-native/go.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
go tool: no such tool "asm"
go tool dist: FAILED: /home/spksrc/spksrc-fleaz/spksrc/native/go/work-native/go/pkg/tool/linux_amd64/go_bootstrap install -gcflags=all= -ldflags=all= -i cmd/asm cmd/cgo cmd/compile cmd/link: exit status 2
make[3]: *** [Makefile:26: go_native_compile] Error 2
make[3]: Leaving directory '/home/spksrc/spksrc-fleaz/spksrc/native/go'
make[2]: *** [../../mk/spksrc.depend.mk:51: depend_target] Error 2
make[2]: Leaving directory '/home/spksrc/spksrc-fleaz/spksrc/cross/activebackup-prometheus-exporter'
make[1]: *** [../../mk/spksrc.depend.mk:51: depend_target] Error 2
make[1]: Leaving directory '/home/spksrc/spksrc-fleaz/spksrc/spk/activebackup-prometheus-exporter'
make: [../../mk/spksrc.spk.mk:417: arch-aarch64-6.1] Error 2 (ignored)

Now its a matter of defining the why....

Here is how to reproduce:

  1. clean-up (e.g. make clean) your spksrc/native/go directory
  2. clean-up your spksrc/spk/activebackup-prometheus-exporter
  3. build a non x86 arch at first build: will fail

If you start with the x86_64 build, the native/go part will run successfully and be available for all other cross-compiled builds afterwards. Although if you start with any other cross-compile build first, without having pre-compiled it on the native platform, it will fail miserably.

Something seems to be wrong in the environment when called from a cross-compiled build. To investigate.

@th0ma7
Copy link
Contributor

th0ma7 commented Jan 12, 2021

Ok, got a tentative fix at #4383 which forces an amd64 host setting along with enforcing docker image gcc and g++.
Can you give it a shot and tell if cross-compiled images on non-x86_64 platform do work OK?

This requires you to clean-up first your native/go (e.g. cd spksrc/native/go && make clean) prior to invoking any make arch-* to confirm native depend is always built ok and end-resulting binaries do run on the expected target platform. If you can test building all of them but at least run one in particular would be beneficial as myself I run a x86_64 so I can't test it all through myself.

@fleaz
Copy link
Author

fleaz commented Jan 12, 2021

Thanks @th0ma7 for taking a look. I will try your fix and report if it worked :)

@fleaz
Copy link
Author

fleaz commented Jan 13, 2021

I just cherry-picked your commit into my branch and can confirm that it's working as intended. I cleaned my own package and the native/go package, and directly build e.g. armv7 without any problems 👍

@th0ma7
Copy link
Contributor

th0ma7 commented Jan 17, 2021

@fleaz I just merged my fix. Can you rebase + force push to confirm it now builds all-right on all platforms?

@fleaz fleaz force-pushed the add_activebackup-exporter branch from 802df89 to 3c3211f Compare January 18, 2021 10:38
@fleaz
Copy link
Author

fleaz commented Jan 18, 2021

Done :)

Copy link
Contributor

@ymartin59 ymartin59 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for contribution. I have a minor improvement request

spk/activebackup-prometheus-exporter/src/service-setup.sh Outdated Show resolved Hide resolved
@fleaz
Copy link
Author

fleaz commented Feb 3, 2021

I still have the problem with the status of the Service. It's running fine in the background but the webinterface tells me "Status: Stopped".
I checked the generated PID file and it contains the correct pid of the running process. And when I click Action>Run in the webinterface I can see that the old process gets killed, a new one gets started, and the id of the process in the PID file gets updated but the status still is "Stopped". Sadly i'm not that fluent in bash to fully understand what the generated start-stop-status script is doing (e.g. where does $PID_FILE come from?). Some help would be really appreciated because I think this should be fixed beforehand.
Also I'm still waiting for our company designer to create a logo :D

@ymartin59
Copy link
Contributor

@fleaz About the logo, we care about credits to author (in SVG metadata we store in Git in addition to png). Please also care about license for usage and distribution, we recommend to select one of creative commons.

@ymartin59 ymartin59 self-assigned this Feb 9, 2021
@ymartin59
Copy link
Contributor

@ymartin59 I will test and check your process status issue with PID file... framework should do the job seamlessly... except if exporter has a specific behavior like internal forking...

@fleaz fleaz force-pushed the add_activebackup-exporter branch from c58c284 to f7c2999 Compare February 20, 2023 13:19
@fleaz
Copy link
Author

fleaz commented Feb 20, 2023

Hey,
long time no see :D

I recently had to update this package for our own NAS because I made an update to the packages software itself, and meanwhile I also rebased this PR and I can confirm that the problem described further up with the wrong detection of the running state is gone. One of the nearly 700 commits on master in between have probably fixed it :D

I just build it for our NAS and the state is working fine and I can also start/stop via the GUI without any problems.

Last thing is the missing icon. But if this is not a requirement for you, this PR is finished from my side :)

@fleaz fleaz changed the title [WIP] Added new spk for activebackup-prometheus-exporter Added new spk for activebackup-prometheus-exporter Feb 20, 2023
@hgy59 hgy59 added the new-package PR/WIP for a new package label Apr 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
new-package PR/WIP for a new package
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants