Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve error messages in the unit tests #135

Open
asishallab opened this issue Jul 8, 2020 · 2 comments
Open

Improve error messages in the unit tests #135

asishallab opened this issue Jul 8, 2020 · 2 comments
Assignees

Comments

@asishallab
Copy link
Member

Currently the unit tests when failing print a non readable error message that is too long and without white-spaces and thus is completely useless. Improve this

Use linux shell's diff and grep

Assume you have the full generated code (generated.txt) and the expected code (expected.txt) and want to see common lines and where the expected code deviates from what has been generated. Using *nix command line tools we can easily highlight which expected lines are missing from the generated code. Consider the following example:

diff -u expected.txt generated.txt | grep -P "^[\s-]" | sed -e '1d'

which generates the following example output

   input searchTranscript_countInput {
     field: transcript_countField
     value: typeValue
     operator: Operator
-    searchChanged: [searchTranscript_countInput]
   }

The - sign indicates the expected line of code missing from the generated code.

Helper function to solve this

Implement a simple test helper function that receives two arguments, the expected and the generated code including white-spaces. The helper then invokes the system and uses the above combination of commands to generate a comprehensive error message. This can then be passed to the mocha unit tests as custom error messages.

Security check

Investigate whether node is aware of the underlying operating system. If so, the above helper function should fall back to the default mocha error messages on Windows systems. We don't really support Win OS, do we?

Adapt existing test cases

All tests that actually compare expected with generated code should be updated to use this new comprehensive custom error messages.

@TimRehberg
Copy link

TimRehberg commented Jul 8, 2020

It took a bit of fiddling to get exactly the output we want, but I've achieved the desired results with jsdiff in a standalone proof-of-concept (~20 lines of code), so we're platform independent.

We can also colour code the changes (makes it easier to read in my opinion) and/or hilight individual words/indentifiers (might be useful on long lines? Will test.)

@TimRehberg
Copy link

I've had to spend a little more time on the helper because a not-syntax-aware diff is of limited use if you combine a lot of duplicated code with modified function signatures. I'll merge the improved version into the issue135 branch when I'm done fixing the Cassandra unit tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants