ReFrame 2.7

ReFrame 2.7 Release Notes

API Changes

• New composable and flexible syntax for sanity and performance checking
• Old syntax of RegressionTest.sanity_patterns and RegressionTest.perf_patterns is deprecated
• RegressionTest.strict_check attribute behavior changed

Framework

• New complete documentation with tutorial and reference guides
• New command-line option --strict
• New syntax in the configuration file
• New resourcesdir entry in the systems section of the configuration file
• Removed command-line option --relax-performance-check
• More robust unit tests of the asynchronous execution policy
• prefix_apps was removed from the settings.
• Standardised the behaviour of scoped dictionaries
• Code quality improvements
• Fixed misleading failure reports in asynchronous execution policy
• Fixed behaviour of the --nocolor option
• Fixed frontend crash during failure report when a test fails early in its setup phase
• Fixed crash when using execution modes

ReFrame 2.6.1

ReFrame 2.6.1 release notes

API changes

None.

Framework

• Automatic cancellation of jobs blocked in a pending state due to reasons that require intervention of system administrators
• Support for canceling jobs
• Print detailed failure info for the sanity and performance checking algorithm
• Bugfix: Module load instructions are now emitted correctly
• Refactoring and coding style adjustments
• CI improvements

Reframe 2.6

Reframe 2.6 release notes

API changes

• New field exclusive_access is added that lets a check require exclusive access to compute nodes by the underlying scheduler

Framework

None

Reframe 2.5

Reframe 2.5 release notes

API changes

• sourcesdir may now be None in which case the regression test is resourceless

Framework

• Treat properly stage and output directories if they alias each other
• Improvements of the internal launcher API to allow users to create custom launchers
• Addition of a VisitLauncher for the VisIt visualisation software
• Support for custom execution modes
• --notimestamp and --timefmt options replaced by a single --timestamp TIMEFMT option; no timestamp directory components are appended anymore
• Bug fix: crash when printing system listing and the current system was not auto-detected
• Bug fix: Environment names with slashes are now treated correctly
• Bug fix: asynchronous execution policy unit tests are now more robust
• Bug fix: job launcher options were not emitted in the job script file

Reframe 2.4

Reframe 2.4 release notes

API changes

• Support for OS variable expansion in the variables field
• Fixed environment loading: modules are loaded first and variables are set
  afterwards
• Default value of the use_multithreading field is now None, in which case
  no hint is emitted

Framework

• Brand new frontend
  • Google Test look-and-feel
  • Less verbose output
  • Better error handling and management
  • Better test statistics management
  • Support for pluggable execution policies of regression tests
  • New command line option --exec-policy for setting the execution policy of
    the regression tests
  • Regression checks are not reused anymore across partitions and programming environments. Instead, they are cloned for every case.
• Support for asynchronous execution of regression tests
  • The running phase of regression tests is executed in parallel
  • Support for different maximum pending jobs per partition
• Full logging support
  • Multiple logging levels
  • Multiple log record handlers
  • Easily configured in reframe/settings.py
  • New command line option --save-log-files for saving the produced log files
    in the output directory of the run
• More robust user check validation
  • The regression test file is first parsed for the correct definition of the
    _get_checks() method and is then loaded and further validated

Reframe 2.3

Reframe 2.3 release notes

API changes

• Sanity checking returns True if sanity_patterns are empty
• Introduction of readonly_files variable in RegressionTest API to allow symlinking of target files
• New propagate member variable of ProgEnvironment that enables/disables the propagation of compilation flags to the underlying make invocation
• Default value for programming environment flags is now None. See Reference Guide for more information on the actual behaviour.

Framework

• More robust behaviour when no modules are loaded
• Support for virtual copying of directories
• Ability to invoke make without passing it the environment's flags
• Fixed sandboxing of system partitions
• Unit test and CI improvements
• Other bug fixes

Reframe 2.2

First public release.