Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use custom functions to evaluate equality #39

Closed
daphee opened this issue Oct 17, 2017 · 5 comments
Closed

Use custom functions to evaluate equality #39

daphee opened this issue Oct 17, 2017 · 5 comments

Comments

@daphee
Copy link
Contributor

daphee commented Oct 17, 2017

I'd like to propose to add a syntax to allow Expectation creating functions other than Expect.equal. The case I'd need such a syntax for is to test examples containing floating point numbers, e.g.

cos (2*pi * 90/360)
--> 0  

This would fail because the result is actually something really small but different from 0.
I need to check for equality with a certain tolerance. The test should look like this:

cos (2*pi * 90/360) |> Expect.within (Absolute 0.000000001) 0

instead of:

cos (2*pi * 90/360) |> Expect.equal 0

I think this could be achieved by modifying the syntax to include another special character which marks a line that contains the function to be used, e.g. like this:

cos (2*pi * 90/360)
| Expect.equalWithingThreshold
--> 0  

What do you think? Would this obfuscate the examples too much? Do you maybe have a better idea on how to test examples like that? Is this doable?

@ianmackenzie
Copy link
Contributor

I'd add that floating-point equality is an important case but it would also be great to support fully custom expectations on custom types. For example, in opensolid/geometry the documentation examples often look like

point =
    Point2d.fromCoordinates ( 1, 0 )

Point2d.rotateAround Point2d.origin (degrees 45) point
--> Point2d.fromCoordinates ( 0.707, 0.707 )

and a corresponding test might use

import OpenSolid.Geometry.Expect as Expect

Point2d.fromCoordinates ( 1, 0 )
    |> Point2d.rotateAround Point2d.origin (degrees 45)
    |> Expect.point2dWithin 0.001
        (Point2d.fromCoordinates ( 0.707, 0.707 )

Could this potentially be implemented by adding some more options to elm-verify-examples.json to specify what expectation functions to use by default in a particular file, or perhaps based on a regex match with the result expression or something? (For example if the result expression starts with Point2d.fromCoordinates then I know it's a Point2d and should use Expect.point2dWithin 0.001...)

@stoeffel
Copy link
Owner

⚠️ This is not a replacement for tests, this tool should be used for improving your documentation. (from the readme)

This is the reason, why I'm against adding special syntax for more complex expectations. Adding special syntax (--> is just a comment) and customised expectations make it hard to understand what's actually going on in the example. It feels like the examples you're describing are actually tests.

What do you think? Would this obfuscate the examples too much? Do you maybe have a better idea on how to test examples like that? Is this doable?

I think you could just do:

cos (2*pi * 90/360)
    |> truncate
--> 0  

@daphee
Copy link
Contributor Author

daphee commented Oct 24, 2017

I think e.g. this example is perfectly within the scope of the library. Still it isn't possible to verify it without introducing some additional function to truncate the numbers, like in your example.

Doing so would introduce the same amount of clutter than the syntax I proposed does.

Another possibilty would be to define which comparison function should be used in a seperate non-documentation comment above the example, like that:

{- @verify Expect.equalWithingThreshold -}
{-| 
     cos (2*pi * 90/360)
     --> 0  
-}

What do you think about that?

@stoeffel
Copy link
Owner

I think e.g. this example is perfectly within the scope of the library.

Agreed, but I think it's easier for people new to a library to grok an example if it doesn't hide the code for verifying a result. You can use intermediate definitions to simplify an example and still have a function to verify it. This way it's all explicit and you can separate the function from the example using a line-comment. Would that work for your use-case?
https://github.com/stoeffel/elm-verify-examples/#intermediate-definitions

Separately, I think it's better to keep the verified example as close to actual elm code as possible. The current syntax is totally valid elm and doesn't introduce any new syntax feature.

@gampleman
Copy link
Collaborator

I think there are 2 separate issues going on here:

  1. Floating point can't be tested in elm-test using Expect.equal. This (as a side-effect of that decision) makes elm-verify-examples unusable for verifying examples for numerical code. Let's keep discussion of that issue in Impossible to test equality of floating point numbers #83.

  2. Customising assertions for other stuff. I think @stoeffel has argued quite eloquently that there are good reasons for why that's out of scope. You can always do:

example input >= 3 --> True

or some other way to make a custom predicate, and it will be easier for users to understand what you are saying.

As such, I think we can close this issue. If there is some other angle we feel hasn't been covered, we can of course re-open this discussion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants