Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cost of ownership of PiTest needed #35

Open
paul-hammant opened this issue Jul 13, 2017 · 4 comments · May be fixed by #115
Open

Cost of ownership of PiTest needed #35

paul-hammant opened this issue Jul 13, 2017 · 4 comments · May be fixed by #115

Comments

@paul-hammant
Copy link

Some indication is needed of how long mutation testing could take for a particular testbase. I'd suggest something like:

"PiTest will take about 4x longer than your regular unit test suite duration."

A suggestion too as to how to operationalize it in CI would be cool. Like so:

"Don't put PiTest in your primary CI build. Instead make another job that runs at some frequency (say hourly) that tests everything at HEAD revision then. Take the rollup number from the log, and track that one on a timeline. Perhaps using the Plot plugin https://wiki.jenkins.io/display/JENKINS/Plot+Plugin"
@StefanPenndorf
Copy link
Contributor

"PiTest will take about 4x longer than your regular unit test suite duration."

I don't think giving numbers here is reliable. The time for mutation testing depends on your hardware and whether you use parallel unit tests already.
Maybe one could add such a warning but couldn't find a good place so the warning is neither too demotivating nor overlooked.

@paul-hammant
Copy link
Author

How about ...

"PiTest will take about 1.5x to 20x longer than your regular unit test suite duration."

I agree that it could be discouraging. Maybe we should just close this issue - people can run PiTest where they like.

2 similar comments
@paul-hammant
Copy link
Author

How about ...

"PiTest will take about 1.5x to 20x longer than your regular unit test suite duration."

I agree that it could be discouraging. Maybe we should just close this issue - people can run PiTest where they like.

@paul-hammant
Copy link
Author

How about ...

"PiTest will take about 1.5x to 20x longer than your regular unit test suite duration."

I agree that it could be discouraging. Maybe we should just close this issue - people can run PiTest where they like.

StefanPenndorf added a commit to StefanPenndorf/pitest-site that referenced this issue Apr 26, 2020
@StefanPenndorf
Adding some hint or warning about cost of ownership
0e3f39b 
Running pitest comes with some cost: cost of CPU we could make this a little bit more transparent and guide users to use mutation testing as part of their CI process.

fixes hcoles#35
StefanPenndorf added a commit to StefanPenndorf/pitest-site that referenced this issue Dec 7, 2020
@StefanPenndorf
Adding some hint or warning about cost of ownership
620fa90 
Running pitest comes with some cost: cost of CPU we could make this a little bit more transparent and guide users to use mutation testing as part of their CI process.

fixes hcoles#35
@StefanPenndorf StefanPenndorf linked a pull request Dec 7, 2020 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Adding some hint or warning about cost of ownership StefanPenndorf/pitest-site
2 participants
@paul-hammant @StefanPenndorf