Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

APRIL and SDHCALContent integration #29

Open
wants to merge 24 commits into
base: master
Choose a base branch
from

Conversation

tanguypasquier
Copy link

BEGINRELEASENOTES

  • Add ability to link with new PFA contents : SDHCALContent for SDHCAL specific corrections and APRILContent for APRIL PFA algorithms.
  • Allow to compile with or without linking to new PFA Contents, default is to build without new contents.
  • Add processor option to switch between Pandora and APRIL PFAs.
    ENDRELEASENOTES

Copy link
Contributor

@tmadlener tmadlener left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the long delay, this slipped through during summer / vacation time. Overall this looks pretty good to me. I think we should also try and get APRILContent and SDHCalContent into the Key4hep nightly builds to iron out all potential dependency issues and to also enable at least building these options in CI.

I have effectively mainly a few minor comments for the cmake configuration and some code formatting. The biggest thing I would change is the implementation of the usage of the hit factory that currently initializes creates two factories, when only one would be necessary.

CMakeLists.txt Outdated
Comment on lines 69 to 79
IF( APRILContent_FOUND )
INCLUDE_DIRECTORIES( SYSTEM ${APRILContent_INCLUDE_DIRS} )
LINK_LIBRARIES( ${APRILContent_LIBRARIES} )
ADD_DEFINITIONS ( ${APRILContent_DEFINITIONS} )
ADD_DEFINITIONS( "-DAPRILCONTENT" )
ENDIF()
IF( mlpack_FOUND )
INCLUDE_DIRECTORIES( SYSTEM ${mlpack_INCLUDE_DIRS} )
LINK_LIBRARIES( ${mlpack_LIBRARIES} )
ADD_DEFINITIONS ( ${mlpack_DEFINITIONS} )
ENDIF()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
IF( APRILContent_FOUND )
INCLUDE_DIRECTORIES( SYSTEM ${APRILContent_INCLUDE_DIRS} )
LINK_LIBRARIES( ${APRILContent_LIBRARIES} )
ADD_DEFINITIONS ( ${APRILContent_DEFINITIONS} )
ADD_DEFINITIONS( "-DAPRILCONTENT" )
ENDIF()
IF( mlpack_FOUND )
INCLUDE_DIRECTORIES( SYSTEM ${mlpack_INCLUDE_DIRS} )
LINK_LIBRARIES( ${mlpack_LIBRARIES} )
ADD_DEFINITIONS ( ${mlpack_DEFINITIONS} )
ENDIF()
INCLUDE_DIRECTORIES( SYSTEM ${APRILContent_INCLUDE_DIRS} )
LINK_LIBRARIES( ${APRILContent_LIBRARIES} )
ADD_DEFINITIONS ( ${APRILContent_DEFINITIONS} )
ADD_DEFINITIONS( "-DAPRILCONTENT" )
INCLUDE_DIRECTORIES( SYSTEM ${mlpack_INCLUDE_DIRS} )
LINK_LIBRARIES( ${mlpack_LIBRARIES} )
ADD_DEFINITIONS ( ${mlpack_DEFINITIONS} )

If you have REQUIRED in your find_package call there is no need to check again if it is actually found, because CMake will actually stop if it is not found.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
IF( APRILContent_FOUND )
INCLUDE_DIRECTORIES( SYSTEM ${APRILContent_INCLUDE_DIRS} )
LINK_LIBRARIES( ${APRILContent_LIBRARIES} )
ADD_DEFINITIONS ( ${APRILContent_DEFINITIONS} )
ADD_DEFINITIONS( "-DAPRILCONTENT" )
ENDIF()
IF( mlpack_FOUND )
INCLUDE_DIRECTORIES( SYSTEM ${mlpack_INCLUDE_DIRS} )
LINK_LIBRARIES( ${mlpack_LIBRARIES} )
ADD_DEFINITIONS ( ${mlpack_DEFINITIONS} )
ENDIF()
IF( APRILContent_FOUND )
INCLUDE_DIRECTORIES( SYSTEM ${APRILContent_INCLUDE_DIRS} )
LINK_LIBRARIES( ${APRILContent_LIBRARIES} )
ADD_DEFINITIONS ( ${APRILContent_DEFINITIONS} )
ADD_DEFINITIONS( "APRILCONTENT" )
ENDIF()
IF( mlpack_FOUND )
INCLUDE_DIRECTORIES( SYSTEM ${mlpack_INCLUDE_DIRS} )
LINK_LIBRARIES( ${mlpack_LIBRARIES} )
ADD_DEFINITIONS ( ${mlpack_DEFINITIONS} )
ENDIF()

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deleted the IF(... FOUND) in the commit "CMakeLists updates" (d49c7a0)

I tried to delete the -D in the add definitions but it seems that the compiler wasn't understanding the definition as a macro anymore and was trying to find it as file or directory, thus creating errors during compilation

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The special handling of adding -D might only be available for target_compile_definitions(?) @andresailer

CMakeLists.txt Outdated
Comment on lines 57 to 62
IF( SDHCALContent_FOUND )
INCLUDE_DIRECTORIES( SYSTEM ${SDHCALContent_INCLUDE_DIRS} )
LINK_LIBRARIES( ${SDHCALContent_LIBRARIES} )
ADD_DEFINITIONS ( ${SDHCALContent_DEFINITIONS} )
ADD_DEFINITIONS( "-DSDHCALCONTENT" )
ENDIF()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
IF( SDHCALContent_FOUND )
INCLUDE_DIRECTORIES( SYSTEM ${SDHCALContent_INCLUDE_DIRS} )
LINK_LIBRARIES( ${SDHCALContent_LIBRARIES} )
ADD_DEFINITIONS ( ${SDHCALContent_DEFINITIONS} )
ADD_DEFINITIONS( "-DSDHCALCONTENT" )
ENDIF()
INCLUDE_DIRECTORIES( SYSTEM ${SDHCALContent_INCLUDE_DIRS} )
LINK_LIBRARIES( ${SDHCALContent_LIBRARIES} )
ADD_DEFINITIONS ( ${SDHCALContent_DEFINITIONS} )
ADD_DEFINITIONS( "-DSDHCALCONTENT" )

If you have REQUIRED in your find_package call there is no need to check again if it is actually found, because CMake will actually stop if it is not found.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
IF( SDHCALContent_FOUND )
INCLUDE_DIRECTORIES( SYSTEM ${SDHCALContent_INCLUDE_DIRS} )
LINK_LIBRARIES( ${SDHCALContent_LIBRARIES} )
ADD_DEFINITIONS ( ${SDHCALContent_DEFINITIONS} )
ADD_DEFINITIONS( "-DSDHCALCONTENT" )
ENDIF()
IF( SDHCALContent_FOUND )
INCLUDE_DIRECTORIES( SYSTEM ${SDHCALContent_INCLUDE_DIRS} )
LINK_LIBRARIES( ${SDHCALContent_LIBRARIES} )
ADD_DEFINITIONS ( ${SDHCALContent_DEFINITIONS} )
ADD_DEFINITIONS( "SDHCALCONTENT" )
ENDIF()

cmake will add -D when needed, and in places where we have to treat definitions programatically it is easier if we don't have to deal with some with -D and some without.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above

Comment on lines 14 to 18
FIND_PACKAGE_HANDLE_STANDARD_ARGS(mlpack DEFAULT_MSG mlpack_CORE_HPP_DIR)

IF(NOT mlpack_FOUND)
MESSAGE(FATAL_ERROR "The mlpack package not found.")
ENDIF()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
FIND_PACKAGE_HANDLE_STANDARD_ARGS(mlpack DEFAULT_MSG mlpack_CORE_HPP_DIR)
IF(NOT mlpack_FOUND)
MESSAGE(FATAL_ERROR "The mlpack package not found.")
ENDIF()
include(FindPackageHandleStandardArgs)
FIND_PACKAGE_HANDLE_STANDARD_ARGS(mlpack DEFAULT_MSG mlpack_CORE_HPP_DIR)
  • Make sure that FIND_PACKAGE_HANDLE_STANDARD_ARGS is actually always available
  • FIND_PACKAGE_HANDLE_STANDARD_ARGS already does the correct handling of REQUIRED, QUIET, ... (the arguments that are passed to find_package).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in the "Fixed Findmlpack.cmake" (d8f1d2c)

cmake/Findmlpack.cmake Outdated Show resolved Hide resolved
/**
* @brief Register the LCContent plugins outside their original function. Temporary fix. Would be better to change the plugins registration directly in LCContent
*/
pandora::StatusCode PandoraHack(const pandora::Pandora &pandora);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
pandora::StatusCode PandoraHack(const pandora::Pandora &pandora);
pandora::StatusCode AprilLCContentPluginRegistration(const pandora::Pandora &pandora);

(or a more fitting name)

Would at least hide a bit that this is a "hack", while also making it rather clear from the function name what is happening (on a high level).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated the name in the commit "Fixed the factory issue" (d793e70)

@@ -70,6 +80,7 @@ DDCaloHitCreator::~DDCaloHitCreator()

pandora::StatusCode DDCaloHitCreator::CreateCaloHits(const EVENT::LCEvent *const pLCEvent)
{
ChooseFactory(); //New method to choose the factory to use
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this have to happen every event? This could be done only once in init where all the necessary information should also be available, right?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, I moved it to the DDCaloHitCreator constructor so it's now called only once in the init

#endif

//Added by T.Pasquier
pandora::ObjectFactory<object_creation::CaloHit::Parameters, object_creation::CaloHit::Object>* caloHitFactory; //General factory to initialize
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
pandora::ObjectFactory<object_creation::CaloHit::Parameters, object_creation::CaloHit::Object>* caloHitFactory; //General factory to initialize
pandora::ObjectFactory<object_creation::CaloHit::Parameters, object_creation::CaloHit::Object>* m_caloHitFactory; //General factory to initialize

To make it easier to spot that this is a member variable. (Requires some changes in the .cc file as well)

Given the overall usage pattern in the code, I would actually make this

std::unique_ptr<pandora::ObjectFactory<object_creation::CaloHit::Parameters, object_creation::CaloHit::Object>> m_caloHitFactory{nullptr};

(#include <memory> for unique_ptr)

Then ChooseFactory would become something like

  #ifdef APRILCONTENT
    if(m_settings.m_useAPRIL)
    {   
        caloHitFactory = std::make_unique<april_content::CaloHitFactory>();
    }
    else
    #endif
    {
        caloHitFactory = std::make_unique<pandora::PandoraObjectFactory<object_creation::CaloHit::Parameters, object_creation::CaloHit::Object>>();
    }

And the DefaultCaloFactory as well as the m_pAPRILCaloHitFactory member variables could go.

Usage in the implementation would then be

PANDORA_THROW_RESULT_IF(pandora::STATUS_CODE_SUCCESS, !=, PandoraApi::CaloHit::Create(m_pandora, caloHitParameters, *m_caloHitFactory));

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much! I fixed it and it works perfectly

src/DDPandoraPFANewProcessor.cc Outdated Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mlpack should come with a cmake config file, so this findmlpack should not be needed
https://github.com/mlpack/mlpack/blob/master/CMake/mlpack-config.cmake.in

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tanguypasquier did you check, whether you can simply remove this file and things keep working?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I indeed tried to simply remove the file but it generates some errors during the compilation. I get :

`By not providing "Findmlpack.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "mlpack", but
CMake did not find one.

Could not find a package configuration file provided by "mlpack" with any
of the following names:

mlpackConfig.cmake
mlpack-config.cmake

Add the installation prefix of "mlpack" to CMAKE_PREFIX_PATH or set
"mlpack_DIR" to a directory containing one of the above files. If "mlpack"
provides a separate development package or SDK, be sure it has been installed.`

I checked in the mlpack files for the ones mentioned in the CMake error but couldn't find any of them.

I had already encountered this error during the APRIL installation and I then came across this issue (mlpack/mlpack#444) which seemed to indicate that the best option was to have the Findmlpack.cmake file.
As it solved the problem, I didn't really tried other ways of doing it since then

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where are you getting your mlpack from? It looks like it should be shipping a cmake configuration since v3.2.2 in principle. It might just be a question of pointing cmake into the right direction.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the long delay, I had some other projects going on

I'm getting mlpack from their github repo (https://github.com/mlpack/mlpack) and I'm currently working with the v.4.0.0.
I tried a lot of different things in the last few weeks to make it work without the Findmlpack but didn't succeed..

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems that the creating of the mlpack-config file got lost during their refactoring :/
They do write a pkgconfig file that one can also use via cmake, if PkgConfig is available.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the quick reply!

I succeeded in using PkgConfig via cmake to find mlpack without needing the Findmlpack file.
I will update the README with the new instructions and push this new version right away.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent!

CMakeLists.txt Outdated
Comment on lines 57 to 62
IF( SDHCALContent_FOUND )
INCLUDE_DIRECTORIES( SYSTEM ${SDHCALContent_INCLUDE_DIRS} )
LINK_LIBRARIES( ${SDHCALContent_LIBRARIES} )
ADD_DEFINITIONS ( ${SDHCALContent_DEFINITIONS} )
ADD_DEFINITIONS( "-DSDHCALCONTENT" )
ENDIF()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
IF( SDHCALContent_FOUND )
INCLUDE_DIRECTORIES( SYSTEM ${SDHCALContent_INCLUDE_DIRS} )
LINK_LIBRARIES( ${SDHCALContent_LIBRARIES} )
ADD_DEFINITIONS ( ${SDHCALContent_DEFINITIONS} )
ADD_DEFINITIONS( "-DSDHCALCONTENT" )
ENDIF()
IF( SDHCALContent_FOUND )
INCLUDE_DIRECTORIES( SYSTEM ${SDHCALContent_INCLUDE_DIRS} )
LINK_LIBRARIES( ${SDHCALContent_LIBRARIES} )
ADD_DEFINITIONS ( ${SDHCALContent_DEFINITIONS} )
ADD_DEFINITIONS( "SDHCALCONTENT" )
ENDIF()

cmake will add -D when needed, and in places where we have to treat definitions programatically it is easier if we don't have to deal with some with -D and some without.

CMakeLists.txt Outdated
Comment on lines 69 to 79
IF( APRILContent_FOUND )
INCLUDE_DIRECTORIES( SYSTEM ${APRILContent_INCLUDE_DIRS} )
LINK_LIBRARIES( ${APRILContent_LIBRARIES} )
ADD_DEFINITIONS ( ${APRILContent_DEFINITIONS} )
ADD_DEFINITIONS( "-DAPRILCONTENT" )
ENDIF()
IF( mlpack_FOUND )
INCLUDE_DIRECTORIES( SYSTEM ${mlpack_INCLUDE_DIRS} )
LINK_LIBRARIES( ${mlpack_LIBRARIES} )
ADD_DEFINITIONS ( ${mlpack_DEFINITIONS} )
ENDIF()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
IF( APRILContent_FOUND )
INCLUDE_DIRECTORIES( SYSTEM ${APRILContent_INCLUDE_DIRS} )
LINK_LIBRARIES( ${APRILContent_LIBRARIES} )
ADD_DEFINITIONS ( ${APRILContent_DEFINITIONS} )
ADD_DEFINITIONS( "-DAPRILCONTENT" )
ENDIF()
IF( mlpack_FOUND )
INCLUDE_DIRECTORIES( SYSTEM ${mlpack_INCLUDE_DIRS} )
LINK_LIBRARIES( ${mlpack_LIBRARIES} )
ADD_DEFINITIONS ( ${mlpack_DEFINITIONS} )
ENDIF()
IF( APRILContent_FOUND )
INCLUDE_DIRECTORIES( SYSTEM ${APRILContent_INCLUDE_DIRS} )
LINK_LIBRARIES( ${APRILContent_LIBRARIES} )
ADD_DEFINITIONS ( ${APRILContent_DEFINITIONS} )
ADD_DEFINITIONS( "APRILCONTENT" )
ENDIF()
IF( mlpack_FOUND )
INCLUDE_DIRECTORIES( SYSTEM ${mlpack_INCLUDE_DIRS} )
LINK_LIBRARIES( ${mlpack_LIBRARIES} )
ADD_DEFINITIONS ( ${mlpack_DEFINITIONS} )
ENDIF()

@tmadlener
Copy link
Contributor

I think you will also have to rebase this onto the latest version of iLCSoft/DDMarlinPandora, as it looks like there is a merge conflict (which may resolve itself automatically during rebasing). Additionally, that will pick up the latest configuration for the CI.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tanguypasquier did you check, whether you can simply remove this file and things keep working?

CMakeLists.txt Outdated
Comment on lines 69 to 79
IF( APRILContent_FOUND )
INCLUDE_DIRECTORIES( SYSTEM ${APRILContent_INCLUDE_DIRS} )
LINK_LIBRARIES( ${APRILContent_LIBRARIES} )
ADD_DEFINITIONS ( ${APRILContent_DEFINITIONS} )
ADD_DEFINITIONS( "-DAPRILCONTENT" )
ENDIF()
IF( mlpack_FOUND )
INCLUDE_DIRECTORIES( SYSTEM ${mlpack_INCLUDE_DIRS} )
LINK_LIBRARIES( ${mlpack_LIBRARIES} )
ADD_DEFINITIONS ( ${mlpack_DEFINITIONS} )
ENDIF()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The special handling of adding -D might only be available for target_compile_definitions(?) @andresailer

CMakeLists.txt Outdated
FIND_PACKAGE( SDHCALContent REQUIRED )
INCLUDE_DIRECTORIES( SYSTEM ${SDHCALContent_INCLUDE_DIRS} )
LINK_LIBRARIES( ${SDHCALContent_LIBRARIES} )
ADD_DEFINITIONS ( ${SDHCALContent_DEFINITIONS} )
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SDHCALContent doesn't seem to export any definitions(?): https://github.com/tanguypasquier/SDHCALContent/blob/main/cmake/SDHCALContentConfig.cmake.in

Suggested change
ADD_DEFINITIONS ( ${SDHCALContent_DEFINITIONS} )

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, I deleted it in one of the last commits.

CMakeLists.txt Outdated
FIND_PACKAGE( mlpack REQUIRED )
INCLUDE_DIRECTORIES( SYSTEM ${APRILContent_INCLUDE_DIRS} )
LINK_LIBRARIES( ${APRILContent_LIBRARIES} )
ADD_DEFINITIONS ( ${APRILContent_DEFINITIONS} )
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ADD_DEFINITIONS ( ${APRILContent_DEFINITIONS} )

APRILContent doesn't seem to export any definitions(?): https://github.com/SDHCAL/APRILContent/blob/current/cmake/APRILContentConfig.cmake.in

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as for SDHCALContent, I also deleted it

Comment on lines 274 to 275
/* DDCaloHitCreator& operator=(const DDCaloHitCreator&) = delete; // Disallow copying
DDCaloHitCreator(const DDCaloHitCreator&) = delete; */
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/* DDCaloHitCreator& operator=(const DDCaloHitCreator&) = delete; // Disallow copying
DDCaloHitCreator(const DDCaloHitCreator&) = delete; */
public:
DDCaloHitCreator& operator=(const DDCaloHitCreator&) = delete;
DDCaloHitCreator(const DDCaloHitCreator&) = delete;

Since you know have a unique_ptr member copying won't work in any case. Marking these as = delete will improve the compiler error message. (I would move them to the rest of the constructors a bit further above).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I uncommented it and moved it as suggested!

tanguypasquier and others added 23 commits November 18, 2024 13:35
…ation OK. 3 over 4 crash at runtime : commented out until cleaning in APRIL is done
…ectly in the Marlin call without having to compile. Modified the README
…d particleId plugins from LCContent when using APRIL
…ndmlpack file. Modified the README according to the new CMake command
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants