Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

False positive test pass #303

Open
z-sztrom opened this issue Jan 7, 2025 · 0 comments
Open

False positive test pass #303

z-sztrom opened this issue Jan 7, 2025 · 0 comments
Labels
bug Something is wrong and needs fixing.

Comments

@z-sztrom
Copy link
Contributor

z-sztrom commented Jan 7, 2025

Description

Body of method testParser is wropped in try/catch block and the catch block only logs an exception. As a consequence, if an exception is thrown during the test execution, it is not propagated to a caller method, and a test is treated as a SUCCESS.

As an example, test testEventParserWithEventEiffelIssueDefinedEvent succeeds although it cannot open file expected_parsed_EiffelIssueDefinedEvent.json, because it's incorrectly named expected_parsed_EiffelIssueDefined.json.

There's a stack trace displayed by the test method in a log, but it's very easy to overlook it in a flood of other messages. However, the test should be treated as FAILed and listed in final overview of test results.

This particular test testEventParserWithEventEiffelIssueDefinedEvent has been fixed as part of #302, however it would be good to remove the catch section from method testParser and check that other test results are reported properly.

Additional Context

No response

Logs

No response

Expected Behavior

If any failure ocures during a test, the test should be marked as FAILed, unless it is an expected failure.

Steps To Reproduce

Command mvn clean package reports

11:24:50.582 [main] ERROR com.ericsson.eiffel.remrem.publish.service.EventTemplateHandlerTest - File 'src/test/resources/expectedParsedEvents/expected_parsed_EiffelIssueDefinedEvent.json' does not exist
java.io.FileNotFoundException: File 'src/test/resources/expectedParsedEvents/expected_parsed_EiffelIssueDefinedEvent.json' does not exist
	at org.apache.commons.io.FileUtils.openInputStream(FileUtils.java:2115)
	at org.apache.commons.io.FileUtils.readFileToString(FileUtils.java:2234)
	at org.apache.commons.io.FileUtils.readFileToString(FileUtils.java:2251)
	at com.ericsson.eiffel.remrem.publish.service.EventTemplateHandlerTest.testParser(EventTemplateHandlerTest.java:360)
	at com.ericsson.eiffel.remrem.publish.service.EventTemplateHandlerTest.testEventParserWithEventEiffelIssueDefinedEvent(EventTemplateHandlerTest.java:350)

Nevertheless overall conclusion of the build is

[INFO] Reactor Summary for eiffel-remrem-publish 2.1.5:
[INFO] 
[INFO] eiffel-remrem-publish .............................. SUCCESS [  0.530 s]
[INFO] publish-common ..................................... SUCCESS [  7.473 s]
[INFO] publish-service .................................... SUCCESS [ 14.239 s]
[INFO] publish-cli ........................................ SUCCESS [  3.736 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS

and exit status of the process is 0.

The version of this project/repo, if applicable

2.1.5

The version/edition of the Eiffel Protocol used, if applicable

All, not related to particular protocol

@z-sztrom z-sztrom added the bug Something is wrong and needs fixing. label Jan 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something is wrong and needs fixing.
Projects
None yet
Development

No branches or pull requests

1 participant