[docu] Migrate doxygen Makefile to CMake

[ferdymercury]

This Pull request:

Changes or fixes:

This is in a draft status, work in progress...

The goals are:

• Migrate old Makefile to modern CMake. See legacy Makefiles and Configures #9090
• Allow building the documentation from a git source repository with read-only permissions. I.e. try to not pollute the sources within the process. This follows the Make to CMake migration philosophy. See [doc] Building Doxygen documentation pollutes source directory #8947
• As done by ALICE, use a 'dynamic' Doxyfile declaration, that only specifies what needs to be changed with respect to the default one. This makes easier the maintenance, as you do not need to constantly update the Doxyfile when a new doxygen version arises, and prevents warnings when running in older doxygen versions. This will hopefully contribute to next point:
• Automated documentation test for new PR #9953
• Allow in the future for automatic meta-documentation of the CMake flag system. See Doxygen documentation of CMake flags #8999
• Potentially add a flag in the main ROOT CMakeLists.txt, to enable the building of the documentation via a normal "add_subdirectory()", so that one does not need to build as a separate process.
• Make the doxygen documentation thread-safe
• Don't let doxygen search for "input" files in the output directory (e.g. it looks now for Mathjax output files that one just "generated")

Checklist:

• tested changes locally
• We would need to update https://github.com/root-project/web/blob/main/for_developers/doxygen/index.md
• Update rootspi scripts to build docu in CI accordingly
• tutorial generation does not work yet as "out of source". To-do...
• require min doxygen version 1.9.4
• pre-check that output directory is empty ?
• if GVIZ not found, disable dot
• Stamp file generation to only recompile when changes
• Remove unneeded unnecessary files Images.dat etc ? (or prevent that they appear in root source)
• Make filter.cxx threadsafe but giving unique names to ImageSizes.dat .root etc
• set NUM_PROC_THREADS
• listLibs pass ROOT_CMD as variable
• check why there are stdout.dat and NumberOfImages not being removed
• Consider enabling https://www.doxygen.nl/manual/config.html#cfg_clang_assisted_parsing
• Fix strip_from_path in root-tag ROOT tag file contains hard coded paths #10059
• Check compress [doxy] qch and tag as compressed files #10066
• Upgrade mathjax [doxy] upgrade to Mathjax3 #10067

[phsft-bot]

Can one of the admins verify this patch?

[couet]

Wow ! that's a big change. You removed Doxyfile ? You marked it as "Draft" so I guess it is not aimed to be merged.

[ferdymercury]

It's not ready yet because the tutorials part still generates stuff inside the source tree.

Yes, the Doxyfile is removed. Instead, Cmake creates it for you on the fly. The Doxyfile options are thus directly set in the Cmakelist. This will allow later to overwrite some of them via command line by passing cmake -DDOXYGEN_INPUT=touchedFile.h which would be useful for the CI script.

[couet]

I do not understand how you generate the INPUT list ? Note that we have a special mechanism with makeinput.sh absolutely needed to make sure the Python frame of some docs appears at the end of the doc. Note also that Makefile runs some extra scripts like listlibs.sh which is a trick to generate the graphs of libraries needed by a class. We absolutely needs these two scripts. How will you run those without a Makefile ?

[ferdymercury]

The CMakeLists is doing all these steps now. See:

[couet]

Ok I see. That's good. :-)

[ferdymercury]

Status update:
With the latest modifications for multithreading (latest doxygen master build from today is needed, where some bugs were fixed), and EXCLUDING the tutorials in the make-input.sh, I managed to reduce the build time down to 42 minutes with 8 threads.

[phsft-bot]

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, ROOT-ubuntu2004/soversion, mac1015/python3, mac11/cxx17, windows10/cxx14
How to customize builds

[phsft-bot]

Build failed on ROOT-ubuntu16/nortcxxmod.
Running on sft-ubuntu-1604-3.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Errors:

• [2022-03-22T13:11:46.706Z] stderr: error: Failed to merge in the changes.
• [2022-03-22T13:11:53.203Z] CMake Error at /mnt/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1082 (message):

[phsft-bot]

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, ROOT-ubuntu2004/soversion, mac1015/python3, mac11/cxx17, windows10/cxx14
How to customize builds

[phsft-bot]

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, ROOT-ubuntu2004/soversion, mac1015/python3, mac11/cxx17, windows10/cxx14
How to customize builds

[phsft-bot]

Build failed on ROOT-performance-centos8-multicore/default.
Running on olbdw-01.cern.ch:/data/sftnight/workspace/root-pullrequests-build
See console output.

Failing tests:

• projectroot.runtutorials.tutorial_roostats_rs101_limitexample

[phsft-bot]

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-ubuntu16/nortcxxmod, ROOT-ubuntu2004/soversion, mac1015/python3, mac11/cxx17, windows10/cxx14
How to customize builds

[ferdymercury]

@couet I think it's ready for first tests on your end (though not ready to be merged yet).

[phsft-bot]

Starting build on ROOT-debian10-i386/soversion, ROOT-performance-centos8-multicore/cxx17, ROOT-ubuntu18.04/nortcxxmod, ROOT-ubuntu2004/python3, mac1015/cxx17, mac11/cxx14, windows10/cxx14
How to customize builds

[ferdymercury]

@jalopezg-r00t I have rebased on top of your changes. (Thanks for the improvements!).

Would you mind taking a look? I am not sure on the new logic of replaceCollaborationDiagrams, I have added it as a final-step, but it's not finding it.

Also, in my previous version, I had parallelized makeCollaborationDiagrams.sh, see https://github.com/root-project/root/blob/9e1f918b5c0b6eee4ff22155e1f2bdaaf3172bb6/documentation/doxygen/makelibs.sh

I do not see something similar when you call bash makeCollaborationDiagrams without -j. Is this because that step is taken care later by replace? Or could my parallelization be backported into your new script?

Cheers!

[phsft-bot]

Starting build on ROOT-debian10-i386/soversion, ROOT-performance-centos8-multicore/cxx17, ROOT-ubuntu18.04/nortcxxmod, ROOT-ubuntu2004/python3, mac1015/cxx17, mac11/cxx14, windows10/cxx14
How to customize builds

[phsft-bot]

Build failed on ROOT-performance-centos8-multicore/cxx17.
Running on olbdw-01.cern.ch:/data/sftnight/workspace/root-pullrequests-build
See console output.

Failing tests:

• projectroot.runtutorials.tutorial_spectrum_Background_incr

[phsft-bot]

Starting build on ROOT-debian10-i386/soversion, ROOT-performance-centos8-multicore/cxx17, ROOT-ubuntu18.04/nortcxxmod, ROOT-ubuntu2004/python3, mac1015/cxx17, mac11/cxx14, windows10/cxx14
How to customize builds

[phsft-bot]

Build failed on windows10/cxx14.
Running on null:C:\build\workspace\root-pullrequests-build
See console output.

Errors:

• [2022-08-22T08:27:26.806Z] LINK : fatal error LNK1104: cannot open file 'C:\build\workspace\root-pullrequests-build\build\bin\libCore.dll' [C:\build\workspace\root-pullrequests-build\build\core\Core.vcxproj]

[phsft-bot]

Build failed on mac1015/cxx17.
Running on macitois21.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build
See console output.

Failing tests:

• projectroot.roottest.python.distrdf.dask.roottest_python_distrdf_dask_test_all

[ferdymercury]

Closing. Rebased / Moved to #15160