ReFrame 3.6.0

ReFrame v3.6.0 Release Notes

New features and enhancements

• Add command-line option to generate a JUnit XML report (#1925)
• Add a --list-tags command-line action to list all the available test tags (#1945)
• Add a configuration option to disable automatic module conflict resolution (#1941)
• Deprecate the use of the @parameterized_test decorator (#1934)
• Issue a warning if a module operation is attempted on the nomod modules system (#1933)
• Abbreviate node lists in FAILURE INFO reports (#1912)
• Re-implement pipeline hook mechanism and add a post-init hook (#1865)
• Improve error message when parameters are undefined (#1888)
• Add test API methods for skipping tests (#1875)
• Show a preview of the standard error in cases of build failures (#1873)
• Improve access to the namespaces of both parameter and variable built-ins (#1819)
• Add a time_limit partition configuration parameter and a build_time_limit variable for regression tests (#1860)
• Allow setting the test description and executable name directly in the class body (#1862)
• Minimize the list of doc requirements (#1847)

Bug fixes

• Fix regression in setting the graylog server address (#1956)
• Warn when a check search path does not exist (#1949)
• Properly check exit status of ensurepip in bootstrap.sh (#1953)
• Fix crash when OSError occurs during dumping of test cases (#1926)
• Fix crash when sourcesdir is set to None in class body (#1910)
• Fix missing path to build stdout when collecting EasyBuild installed modules (#1891)
• Fix squeue backend scheduler crash (#1907)
• Fix rescheduling condition for asynchronous execution policy (#1904)
• Color bootstrap.sh only if stdout is a terminal (#1898)
• Fix bootstrap script for pip-less Python installations (#1900)
• Set stdin to null when launching processes from ReFrame, so that processes do no think they are on a tty (#1879)
• Fix the call order on the __set_name__ hook for built-in variables (#1881)
• Do not reset sourcesdir to None if it is set to a URL (#1874)
• Fix typo in variable name (#1866)

Other

• Pull base images for CI instead of building them from scratch (#1909)
• Update documentation of built-ins and update tutorial examples removing the use of __init__ (#1940)
• Make basic configuration example more intuitive w.r.t. to modules systems (#1932)
• Change favicon to the ReFrame logo (#1924)
• Add URL to Twitter badge (#1922)
• Add -pthread to the multithreaded tutorial test (#1870)
• Add tutorial about reusable tests (#1895)
• Update tutorial examples to use the parameter builtin (#1857)
• Various documentation fixes (#1859)

ReFrame 3.5.3

ReFrame v3.5.3 Release Notes

New features and enhancements

• Deprecate the use of the @parameterized_test decorator (#1934)
  • The decorator is set to deprecate with ReFrame 3.6.0
• Issue a warning if a module operation is attempted on the nomod modules system (#1933)
• Abbreviate node lists in FAILURE INFO reports (#1912)

Bug fixes

• Fix crash when OSError occurs during dumping of test cases (#1926)

Other

• Pull base images for CI instead of building them from scratch (#1909)
• Make basic configuration example more intuitive w.r.t. to modules systems (#1932)
• Change favicon to the ReFrame logo (#1924)
• Add URL to Twitter badge (#1922)

ReFrame 3.5.2

ReFrame v3.5.2 Release Notes

New features and enhancements

• Re-implement pipeline hook mechanism and add a post-init hook (#1865)
• Improve error message when parameters are undefined (#1888)

Bug fixes

• Fix crash when sourcesdir is set to None in class body (#1910)
• Fix missing path to build stdout when collecting EasyBuild installed modules (#1891)
• Fix squeue backend scheduler crash (#1907)
• Fix rescheduling condition for asynchronous execution policy (#1904)
• Color bootstrap.sh only if stdout is a terminal (#1898)
• Fix bootstrap script for pip-less Python installations (#1900)
• Do not skip test if sanity_patterns is not set (#1897)
• Set stdin to null when launching processes from ReFrame, so that processes do no think they are on a tty (#1879)
• Fix the call order on the __set_name__ hook for built-in variables (#1881)

Other

• Add -pthread to the multithreaded tutorial test (#1870)
• Add tutorial about reusable tests (#1895)

ReFrame 3.5.1

ReFrame v3.5.1 Release Notes

New features and enhancements

• Add test API methods for skipping tests (#1875)
• Skip test if sanity_patterns is undefined (#1876)
• Show a preview of the standard error in cases of build failures (#1873)
• Improve access to the namespaces of both parameter and variable built-ins (#1819)
• Add a time_limit partition configuration parameter and a build_time_limit variable for regression tests (#1860)
• Allow setting the test description and executable name directly in the class body (#1862)
• Minimize the list of doc requirements (#1847)

Bug fixes

• Do not reset sourcesdir to None if it is set to a URL (#1874)
• Fix typo in variable name (#1866)

Other

• Update tutorial examples to use the parameter builtin (#1857)
• Various documentation fixes (#1859)

ReFrame 3.5.0

ReFrame v3.5.0 Release Notes

New features and enhancements

• Add processor and device information to the system partitions and make them accessible from the tests (#1773)
• Add support for building tests using EasyBuild (#1719)
• Add a new prepare_cmds partition configuration parameter which allows to emit commands before loading any environment (#1803)
• Improve the container platform API exposed to tests (#1775)
  • Respect the entrypoint and command defined inside the container image.
  • Deprecate workdir and commands fields.
  • Introduce the command field.
  • Allow to skip pulling of the container image from a container registry.
  • Expand container platform tutorial example.
• Log reference values of all performance variables in a single line (#1816)
• Allow deprecation of fields in a future version and issue deprecation warnings also in prereleases (#1804)
• Support parameter and variable access directly from the class body (#1806)
• Add built-ins to manage test variables and parameters (#1699)
• Allow logging arbitrary job attributes (#1788)
• Comply with the semantic versioning specification (#1761)
• Add new action for generating dynamic Gitlab pipelines (#1641)
• Add auto completion support to ReFrame (#1685)
• Add a new family of sanity functions that looks for patterns inside text instead of files (#1705)
• Support multiple job submissions when slurm's job submit limit is reached (#1704)
• Add --maxfail option to stop execution after a certain number of failures (#1676)
• Set executable bit in generated job script files (#1716)

Bug fixes

• Fix passing of stagedir to container platforms (#1842)
• Properly pass num_tasks_per_socket to the scheduler backends (#1837)
• Remove VERSION variable from docs Makefile (#1831)
• Fix status printout in case tests are retried (#1814)
• Deepcopy inherited variables from the parent classes (#1810)
• Make attrs() utility function pass when an AttributeError is raised (#1802)
• Print a FAILURE result if not all test cases have completed (#1801)
• More documentation fixes (#1798)
• Document osext.cray_cdt_version() more precisely (#1796)
• Fix AttributeError crash when handling OS errors while writing a test case file (#1785)
• Move shell completion scripts under share/completions/ (#1787)
• Add Python version check to bootstrap script (#1762)
• Various documentation fixes (#1771)
• Fix bug and parameterize cuda samples tests (#1758)
• Add missing required packages in 'setup.py' (#1767)
• Fix validation of the generated Gitlab pipeline files (#1764)
• Fix skipping of test cases in case of unresolved dependencies (#1751)
• Fix "New in version" annotation in documentation for certain new features (#1754)
• Fix parameter value specification in the parameter() directive (#1722)
• Fix handling of single character scheduler backend options passed through -J (#1718)

ReFrame 3.4.2

ReFrame v3.4.2 Release Notes

New features and enhancements

• Add built-ins to manage test variables and parameters (#1699)
• Allow logging arbitrary job attributes (#1788)
• Comply with the semantic versioning specification (#1761)

Bug fixes

• Print a FAILURE result if not all test cases have completed (#1801)
• Document osext.cray_cdt_version() more precisely (#1796)
• Fix AttributeError crash when handling OS errors while writing a test case file (#1785)
• Move shell completion scripts under share/completions/ (#1787)
• Add Python version check to bootstrap script (#1762)
• Various documentation fixes (#1771, #1798)
• Add missing required packages in 'setup.py' (#1767)
• Fix validation of the generated Gitlab pipeline files (#1764)

ReFrame 3.4.1

ReFrame v3.4.1 Release Notes

New features and enhancements

• Add new action for generating dynamic Gitlab pipelines (#1641)
• Add auto completion support to ReFrame (#1685)
• Add a new family of sanity functions that looks for patterns inside text instead of files (#1705)
• Support multiple job submissions when slurm's job submit limit is reached (#1704)
• Add --maxfail option to stop execution after a certain number of failures (#1676)
• Set executable bit in generated job script files (#1716)

Bug fixes

• Fix skipping of test cases in case of unresolved dependencies (#1751)
• Fix "New in version" annotation in documentation for certain new features (#1754)
• Fix parameter value specification in the parameter() directive (#1722)
• Fix handling of single character scheduler backend options passed through -J (#1718)

ReFrame 3.4

ReFrame v3.4 Release Notes

New features and enhancements

• Add sanity function equivalents of some of os.path functions (#1695)
• Introduce new syntax for allowing the creation of extensible parametrised tests (#1614)
• Support CUDA Fortran file extensions in SingleSource (#1677)
• Support spack load for loading software packages (#1624)
• Remove the --use-feature=2020-resolver from bootstrap (#1652)
• Add ability to restore a test session and rerun selected tests (#1538)
• Drop support of pipeline stage method override (#1636)
• Drop support for old configuration syntax (#1630)
• Allow using colon separated paths with the --module-path option (#1623)
• Remove deprecated pre_run, post_run, prebuild_cmd and postbuild_cmd attributes (#1625)

Bug fixes

• Skip test cases recursively in case of dangling dependencies (#1714)
• Correct the advisory rerun message on test failures (#1706)
• Ensure that job options passed from the command-line are always the last in the generated job script (#1701)
• Fix sanity function print to accept a single object only (#1698)
• Fix numbering of failed tasks when reframe execution is aborted (#1673)
• Revert feature to preserve symbolic links in sources directory since it can lead to dangling links in the stage directory; users should use readonly_files instead (#1670)
• Fix documentation of sanity functions accepting a filename (#1666)
• Do not evaluate deferred expressions when dumping in JSON (#1665)
• Pass the job id when raising job-related errors (#1662)
• Restore module paths specified in command line when restoring module collections (#1656)
• Properly expand environment variables for the filelog log handler (#1659)
• Fix module backend implementation for Lmod versions < 8.2 (#1654)
• Fix formatting of lists of non-string objects when logging (#1649)
• Send test attributes correctly with the Graylog handler (#1644)
• Fix detailed listing message for node allocation scheme (#1637)
• Fail properly when the source code language cannot be guessed (#1619)

ReFrame 3.4-dev1

v3.4-dev1

ReFrame 3.4-dev1

ReFrame 3.4-dev0

v3.4-dev0

ReFrame 3.4-dev0