Throw AssertionError instead of Error in case of approval failure

[nicerloop]

Approval failures currently throw Error, but should throw AssertionError to actually distinguish functional failure (AssertionError) from technical failure (everything else).

See PR #595

[LarsEckart]

hm, that AssertionError comes from java.lang though. There is also https://github.com/ota4j-team/opentest4j which was my first thought when I saw the subject of this Issue. Are you aware of this, wouldnt this not the preferred Exception to throw?

I've seen your PR, we (or other maintainers) will have a look the next time we get together. (In my case that's 9.12.)

[nicerloop]

As explained in https://github.com/ota4j-team/opentest4j#status-quo the base common ground is AssertionError which should be used in Approval instead of Error in case of failure.

Using opentest4j is cleaner but requires an external dependency, which would be a larger change than just using AssertionError from the base Java runtime.

[LarsEckart]

hello again, we've been thinking about this and haven't quite understood what problem does it solve when we switch from throwing Error to AssertionError ?

[LarsEckart]

We couldn't let it go, and we rather not change it as we don't see what it solves. Regardless, with release 24.11.0 you can now customize it on your side if this is necessary for you.

[nicerloop]

hello again, we've been thinking about this and haven't quite understood what problem does it solve when we switch from throwing Error to AssertionError ?

The problem is that a test can fail due either to a technical problem or a functional problem. As explained in https://github.com/ota4j-team/opentest4j#status-quo AssertionError is the simplest Java provided throwable describing a failed assertion, i.e. a functional test failure. Error on the other hand signals a generic technical failure.

As an example, using Eclipse test runner, or Serenity BDD, the test is reported differently is Error or AssertionError is thrown: Error marks the test as (technically) broken, AssertionError marks the test as (functionally) failed.

I am currently integrating ApprovalTests.Java with Cucumber and Serenity, and I don't want my (functional) failures to be reported as (technical) broken tests.