Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pitest/descartes fail to run test on xwiki-commons-extension-repository-maven #117

Open
vmassol opened this issue Jan 31, 2020 · 8 comments
Open

pitest/descartes fail to run test on xwiki-commons-extension-repository-maven #117

vmassol opened this issue Jan 31, 2020 · 8 comments
Labels
test-execution

Comments

@vmassol
Copy link
Contributor

vmassol commented Jan 31, 2020

We have a weird problem in xwiki-commons/xwiki-commons-core/xwiki-commons-extension/xwiki-commons-extension-repositories/xwiki-commons-extension-repository-maven when executing pitest/descartes:

...
[INFO] Defaulting target classes to match packages in build directory
2:38:25 PM PIT >> INFO : Verbose logging is disabled. If you encounter a problem, please enable it before reporting an issue.
2:38:25 PM PIT >> INFO : Sending 16 test classes to minion
2:38:25 PM PIT >> INFO : Sent tests to minion
2:38:25 PM PIT >> INFO : MINION : 2:38:25 PM PIT >> INFO : Checking environment

2:38:34 PM PIT >> INFO : MINION : 2:38:34 PM PIT >> INFO : Found  24 tests

2:38:34 PM PIT >> INFO : MINION : 2:38:34 PM PIT >> INFO : Dependency analysis reduced number of potential tests by 0

2:38:34 PM PIT >> INFO : MINION : 2:38:34 PM PIT >> INFO : 24 tests received
                                                                                                                                                                                                         -2:38:38 PM PIT >> INFO : MINION : 2:38:38 PM PIT >> SEVERE : Description [testClass=org.xwiki.extension.repository.aether.internal.SystemHTTPProxyTest, name=[engine:junit-vintage]/[runner:org.xwiki.extension.repository.aether.internal.SystemHTTPProxyTest]/[test:testProxy(org.xwiki.extensio
2:38:38 PM PIT >> INFO : MINION : n.repository.aether.internal.SystemHTTPProxyTest)]]
java.lang.AssertionError: The repository did not requested the proxy server
        at org.junit.Assert.fail(Assert.java:89)
        at org.junit.Assert.assertTrue(Assert.java:42)
        at org.xwiki.extension.repository.aet
2:38:38 PM PIT >> INFO : MINION : her.internal.SystemHTTPProxyTest.testProxy(SystemHTTPProxyTest.java:69)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccess
2:38:38 PM PIT >> INFO : MINION : orImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
        at org.junit.internal.runners.model.ReflectiveCallable.run(Re
2:38:38 PM PIT >> INFO : MINION : flectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at org.xwiki.test.mockito.MockitoComponentMockingRule$1.
2:38:38 PM PIT >> INFO : MINION : evaluate(MockitoComponentMockingRule.java:188)
        at com.github.tomakehurst.wiremock.junit.WireMockRule$1.evaluate(WireMockRule.java:62)
        at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
        at org.junit.runners.BlockJUnit4ClassRunner$1.evalu
2:38:38 PM PIT >> INFO : MINION : ate(BlockJUnit4ClassRunner.java:100)
        at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4C
2:38:38 PM PIT >> INFO : MINION : lassRunner.java:63)
        at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
        at org.junit.runners.ParentRunner                                                                                                                                                                \2:38:38 PM PIT >> INFO : MINION : .access$100(ParentRunner.java:66)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
        at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
        at org.junit.runner.J
2:38:38 PM PIT >> INFO : MINION : UnitCore.run(JUnitCore.java:137)
        at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
        at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:40)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
        at ja
2:38:38 PM PIT >> INFO : MINION : va.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
        at java.util.Iterator.forEachRemaining(Iterator.java:116)
        at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
        at java.util.stream.AbstractPipeline
2:38:38 PM PIT >> INFO : MINION : .copyInto(AbstractPipeline.java:481)
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
        at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluate
2:38:38 PM PIT >> SEVERE : Description [testClass=org.xwiki.extension.repository.aether.internal.SystemHTTPProxyTest, name=[engine:junit-vintage]/[runner:org.xwiki.extension.repository.aether.internal.SystemHTTPProxyTest]/[test:testProxy(org.xwiki.extension.repository.aether.internal.SystemHTTPProxyTest)]] did not pass without mutation.
2:38:38 PM PIT >> INFO : MINION : Sequential(ForEachOps.java:174)
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
        at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTe
2:38:38 PM PIT >> INFO : MINION : stEngine.java:80)
        at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:71)
        at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:229)                                                                                                            |at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$6
2:38:38 PM PIT >> INFO : MINION : (DefaultLauncher.java:197)
        at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:211)
        at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:191)
        at org.junit.platform.launcher.core.Def
2:38:38 PM PIT >> INFO : MINION : aultLauncher.execute(DefaultLauncher.java:128)
        at org.pitest.junit5.JUnit5TestUnit.execute(JUnit5TestUnit.java:88)
        at org.pitest.coverage.execute.CoverageDecorator.execute(CoverageDecorator.java:50)
        at org.pitest.testapi.execute.containers.UnContainer.e
2:38:38 PM PIT >> INFO : MINION : xecute(UnContainer.java:31)
        at org.pitest.testapi.execute.Pitest.executeTests(Pitest.java:57)
        at org.pitest.testapi.execute.Pitest.run(Pitest.java:48)
        at org.pitest.coverage.execute.CoverageWorker.run(CoverageWorker.java:51)
        at org.pitest.coverage.exec
2:38:38 PM PIT >> INFO : MINION : ute.CoverageMinion.main(CoverageMinion.java:90)

2:38:38 PM PIT >> INFO : Calculated coverage in 12 seconds.
2:38:38 PM PIT >> SEVERE : Tests failing without mutation: 
Description [testClass=org.xwiki.extension.repository.aether.internal.SystemHTTPProxyTest, name=[engine:junit-vintage]/[runner:org.xwiki.extension.repository.aether.internal.SystemHTTPProxyTest]/[test:testProxy(org.xwiki.extension.repository.aether.internal.SystemHTTPProxyTest)]]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  40.802 s
[INFO] Finished at: 2020-01-31T14:38:38+01:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.pitest:pitest-maven:1.4.11:mutationCoverage (pitest-check) on project xwiki-commons-extension-repository-maven: Execution pitest-check of goal org.pitest:pitest-maven:1.4.11:mutationCoverage failed: 1 tests did not pass without mutation when calculating line coverage. Mutation testing requires a green suite.
[ERROR] See http://pitest.org for more details.

It seems that somehow pitest/descartes fails when executing the xwiki tests without any mutation which is strange since the tests work fine when executing without pitest.

@oscarlvp any idea? Thx!
vmassol added a commit to xwiki/xwiki-commons that referenced this issue Jan 31, 2020
@vmassol
[Misc] Exclude a test from pitest since it's a false positive. See ST…
d02e9f9 
…AMP-project/pitest-descartes#117
@vmassol
Copy link
Contributor Author

vmassol commented Jan 31, 2020

Same problem for xwiki-platform/xwiki-platform-core/xwiki-platform-rendering/xwiki-platform-rendering-parser:

...

3:08:09 PM PIT >> INFO : MINION : 3:08:09 PM PIT >> SEVERE : Description [testClass=org.xwiki.rendering.internal.parser.DefaultContentParserTest, name=parseHasNoMetadataSource]
java.lang.AssertionError: 
Expected: null
     but: was "wiki:space.page"
	at org.hamcrest.MatcherAssert.assertTh
3:08:09 PM PIT >> INFO : MINION : at(MatcherAssert.java:20)
	at org.junit.Assert.assertThat(Assert.java:964)
	at org.junit.Assert.assertThat(Assert.java:930)
	at org.xwiki.rendering.internal.parser.DefaultContentParserTest.parseHasNoMetadataSource(DefaultContentParserTest.java:85)
	at sun.
3:08:09 PM PIT >> INFO : MINION : reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method
3:08:09 PM PIT >> INFO : MINION : .invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExpl
3:08:09 PM PIT >> INFO : MINION : osively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.xwiki.test.mockito.MockitoComponentMockingRule$
3:08:09 PM PIT >> INFO : MINION : 1.evaluate(MockitoComponentMockingRule.java:188)
	at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:258)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4Cla
3:08:09 PM PIT >> INFO : MINION : ssRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runCh
3:08:09 PM PIT >> INFO : MINION : ild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runn
3:08:09 PM PIT >> INFO : MINION : ers.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at or
3:08:09 PM PIT >> INFO : MINION : g.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
	at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:40)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.
3:08:09 PM PIT >> INFO : MINION : java:184)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at java.util.Iterator.forEachRemaining(Iterator.java:116)
	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
	at java.util.stream.
3:08:09 PM PIT >> INFO : MINION : AbstractPipeline.copyInto(AbstractPipeline.java:481)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
	at java.util.stream.ForEachOps$ForEachO
3:08:09 PM PIT >> INFO : MINION : p$OfRef.evaluateSequential(ForEachOps.java:174)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
	at org.junit.vintage.engine.VintageTestEngine.executeAllCh
3:08:09 PM PIT >> INFO : MINION : ildren(VintageTestEngine.java:80)
	at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:71)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:229)
	at org.junit.platform.launcher.core.DefaultLauncher.
3:08:09 PM PIT >> INFO : MINION : lambda$execute$6(DefaultLauncher.java:197)
	at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:211)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:191)
	at org.junit.platform.l
3:08:09 PM PIT >> INFO : MINION : auncher.core.DefaultLauncher.execute(DefaultLauncher.java:128)
	at org.pitest.junit5.JUnit5TestUnit.execute(JUnit5TestUnit.java:88)
	at org.pitest.coverage.execute.CoverageDecorator.execute(CoverageDecorator.java:50)
	at org.pitest.testapi.execute.containe
3:08:09 PM PIT >> INFO : MINION : rs.UnContainer.execute(UnContainer.java:31)
	at org.pitest.testapi.execute.Pitest.executeTests(Pitest.java:57)
	at org.pitest.testapi.execute.Pitest.run(Pitest.java:48)
	at org.pitest.coverage.execute.CoverageWorker.run(CoverageWorker.java:51)
	at org.pite
3:08:09 PM PIT >> INFO : MINION : st.coverage.execute.CoverageMinion.main(CoverageMinion.java:90)

\3:08:09 PM PIT >> SEVERE : Description [testClass=org.xwiki.rendering.internal.parser.DefaultContentParserTest, name=parseHasNoMetadataSource] did not pass without mutation.
|3:08:09 PM PIT >> INFO : Calculated coverage in 2 seconds.
3:08:09 PM PIT >> SEVERE : Tests failing without mutation: 
Description [testClass=org.xwiki.rendering.internal.parser.DefaultContentParserTest, name=parseHasNoMetadataSource]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  18.985 s
[INFO] Finished at: 2020-01-31T15:08:09+01:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.pitest:pitest-maven:1.4.10:mutationCoverage (pitest-check) on project xwiki-platform-rendering-parser: Execution pitest-check of goal org.pitest:pitest-maven:1.4.10:mutationCoverage failed: 1 tests did not pass without mutation when calculating line coverage. Mutation testing requires a green suite.
[ERROR] See http://pitest.org for more details.

@oscarlvp to reproduce I execute: mvn clean install -Pquality -Dxwiki.pitest.skip=false

@vmassol
Copy link
Contributor Author

vmassol commented Jan 31, 2020

And we also have it on xwiki-rendering:

[INFO] --- pitest-maven:1.4.10:mutationCoverage (pitest-check) @ xwiki-rendering-test ---
[INFO] Found plugin : Default csv report plugin
[INFO] Found plugin : Default xml report plugin
[INFO] Found plugin : JSON report plugin
[INFO] Found plugin : Pseudo and partially-tested methods JSON report
[INFO] Found plugin : Pseudo and partially-tested methods HTML report
[INFO] Found plugin : Default html report plugin
[INFO] Found plugin : Static initializer code detector plugin
[INFO] Found plugin : Static initializer filter plugin
[INFO] Found plugin : Excluded annotations plugin
[INFO] Found plugin : Inlined finally block filter plugin
[INFO] Found plugin : Try with resources filter
[INFO] Found plugin : Implicit null check filter
[INFO] Found plugin : Method reference null check filter
[INFO] Found plugin : For each loop filter
[INFO] Found plugin : Logging calls filter
[INFO] Found plugin : Infinite for loop filter
[INFO] Found plugin : Long running iterator loop filter
[INFO] Found plugin : For loop counter filter
[INFO] Found plugin : Kotlin junk mutations filter
[INFO] Found plugin : Max mutations per class limit
[INFO] Found plugin : Equals shortcut equivalent mutant filter
[INFO] Found plugin : Trivial return vals equivalence filter
[INFO] Found plugin : Mutant export plugin
[INFO] Found plugin : Stop methods filter
[INFO] Found plugin : Avoids null mutation in non-nullable methods, as marked by the Kortlin compiler
[INFO] Found shared classpath plugin : Default mutation engine
[INFO] Found shared classpath plugin : Extreme mutation engine
[INFO] Found shared classpath plugin : JUnit plugin
[INFO] Found shared classpath plugin : TestNG plugin
[INFO] Found shared classpath plugin : JUnit 5 test framework support
[INFO] Adding org.pitest:pitest to SUT classpath
[INFO] Adding org.pitest:pitest-junit5-plugin to SUT classpath
[INFO] Adding eu.stamp-project:descartes to SUT classpath
[INFO] Mutating from /root/workspace/xwiki-rendering_pitest/xwiki-rendering-test/target/classes
[INFO] Defaulting target classes to match packages in build directory
9:48:29 PM PIT >> INFO : Verbose logging is disabled. If you encounter a problem, please enable it before reporting an issue.
9:48:30 PM PIT >> INFO : Sending 18 test classes to minion
9:48:30 PM PIT >> INFO : Sent tests to minion
9:48:30 PM PIT >> INFO : MINION : 9:48:30 PM PIT >> INFO : Checking environment

9:48:30 PM PIT >> INFO : MINION : 9:48:30 PM PIT >> INFO : Found  4 tests

9:48:30 PM PIT >> INFO : MINION : 9:48:30 PM PIT >> INFO : Dependency analysis reduced number of potential tests by 0

9:48:30 PM PIT >> INFO : MINION : 9:48:30 PM PIT >> INFO : 4 tests received

9:48:30 PM PIT >> INFO : MINION : Jan 30, 2020 9:48:30 PM org.junit.vintage.engine.descriptor.RunnerTestDescriptor warnAboutUnfilterableRunner
WARNING: Runner org.junit.internal.runners.ErrorReportingRunner (used on class org.xwiki.rendering.test.cts.RenderingTest) does not support filteri
9:48:30 PM PIT >> INFO : MINION : ng and will therefore be run completely.

9:48:30 PM PIT >> INFO : MINION : 9:48:30 PM PIT >> SEVERE : Description [testClass=org.xwiki.rendering.test.cts.RenderingTest, name=initializationError]
org.junit.runners.model.InvalidTestClassError: Invalid test class 'org.xwiki.rendering.test.cts.RenderingTest':
  1. Test class should h
9:48:30 PM PIT >> INFO : MINION : ave exactly one public zero-argument constructor
	at org.junit.runners.ParentRunner.validate(ParentRunner.java:525)
	at org.junit.runners.ParentRunner.<init>(ParentRunner.java:102)
	at org.junit.runners.BlockJUnit4ClassRunner.<init>(BlockJUnit4ClassRunner.
9:48:30 PM PIT >> INFO : MINION : java:84)
	at org.junit.runners.JUnit4.<init>(JUnit4.java:23)
	at org.junit.internal.builders.JUnit4Builder.runnerForClass(JUnit4Builder.java:10)
	at org.junit.vintage.engine.discovery.DefensiveAllDefaultPossibilitiesBuilder$DefensiveJUnit4Builder.runnerFor
9:48:30 PM PIT >> INFO : MINION : Class(DefensiveAllDefaultPossibilitiesBuilder.java:128)
	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:70)
	at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:37
9:48:30 PM PIT >> INFO : MINION : )
	at org.junit.vintage.engine.discovery.DefensiveAllDefaultPossibilitiesBuilder.runnerForClass(DefensiveAllDefaultPossibilitiesBuilder.java:56)
	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:70)
	at org.junit.vintage.engin
9:48:30 PM PIT >> INFO : MINION : e.discovery.ClassSelectorResolver.resolveTestClass(ClassSelectorResolver.java:66)
	at org.junit.vintage.engine.discovery.ClassSelectorResolver.resolve(ClassSelectorResolver.java:57)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestReso
9:48:30 PM PIT >> INFO : MINION : lution.lambda$resolveUniqueId$3(EngineDiscoveryRequestResolution.java:161)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at java.util.ArrayList$ArrayListSpliterator.tryAdvance(ArrayList.java:1359)
	at java.util.stream.Refer
9:48:30 PM PIT >> INFO : MINION : encePipeline.forEachWithCancel(ReferencePipeline.java:126)
	at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:499)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:486)
	at java.util.stream.AbstractPipeline.w
9:48:30 PM PIT >> INFO : MINION : rapAndCopyInto(AbstractPipeline.java:472)
	at java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:152)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.
9:48:30 PM PIT >> INFO : MINION : java:531)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.resolve(EngineDiscoveryRequestResolution.java:174)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.resolveUniqueId(EngineDiscoveryRe
9:48:30 PM PIT >> INFO : MINION : questResolution.java:161)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.resolve(EngineDiscoveryRequestResolution.java:118)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.access$100(Engine
9:48:30 PM PIT >> INFO : MINION : DiscoveryRequestResolution.java:53)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution$DefaultContext.resolve(EngineDiscoveryRequestResolution.java:227)
	at org.junit.vintage.engine.discovery.MethodSelectorResolver.resolvePare
9:48:30 PM PIT >> INFO : MINION : ntAndAddFilter(MethodSelectorResolver.java:56)
	at org.junit.vintage.engine.discovery.MethodSelectorResolver.resolve(MethodSelectorResolver.java:47)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.lambda$resolveUniqueId$3(E
9:48:30 PM PIT >> INFO : MINION : ngineDiscoveryRequestResolution.java:161)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at java.util.ArrayList$ArrayListSpliterator.tryAdvance(ArrayList.java:1359)
	at java.util.stream.ReferencePipeline.forEachWithCancel(Re
9:48:30 PM PIT >> INFO : MINION : ferencePipeline.java:126)
	at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:499)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:486)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.j
9:48:30 PM PIT >> INFO : MINION : ava:472)
	at java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:152)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:531)
	at org.junit.platform.
9:48:30 PM PIT >> INFO : MINION : engine.support.discovery.EngineDiscoveryRequestResolution.resolve(EngineDiscoveryRequestResolution.java:174)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.resolveUniqueId(EngineDiscoveryRequestResolution.java:161)
	at org
9:48:30 PM PIT >> INFO : MINION : .junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.resolve(EngineDiscoveryRequestResolution.java:118)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.resolveCompletely(EngineDiscoveryRequestResolution
9:48:30 PM PIT >> INFO : MINION : .java:87)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.run(EngineDiscoveryRequestResolution.java:80)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolver.resolve(EngineDiscoveryRequestResolver.j
9:48:30 PM PIT >> INFO : MINION : ava:112)
	at org.junit.vintage.engine.discovery.VintageDiscoverer.discover(VintageDiscoverer.java:42)
	at org.junit.vintage.engine.VintageTestEngine.discover(VintageTestEngine.java:62)
	at org.junit.platform.launcher.core.DefaultLauncher.discoverEngineRoot
9:48:30 PM PIT >> INFO : MINION : (DefaultLauncher.java:177)
	at org.junit.platform.launcher.core.DefaultLauncher.discoverRoot(DefaultLauncher.java:164)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:128)
	at org.pitest.junit5.JUnit5TestUnit.execute(JUnit
9:48:30 PM PIT >> INFO : MINION : 5TestUnit.java:88)
	at org.pitest.coverage.execute.CoverageDecorator.execute(CoverageDecorator.java:50)
	at org.pitest.testapi.execute.containers.UnContainer.execute(UnContainer.java:31)
	at org.pitest.testapi.execute.Pitest.executeTests(Pitest.java:57)
	a
9:48:30 PM PIT >> INFO : MINION : t org.pitest.testapi.execute.Pitest.run(Pitest.java:48)
	at org.pitest.coverage.execute.CoverageWorker.run(CoverageWorker.java:51)
	at org.pitest.coverage.execute.CoverageMinion.main(CoverageMinion.java:90)

9:48:30 PM PIT >> SEVERE : Description [testClass=org.xwiki.rendering.test.cts.RenderingTest, name=initializationError] did not pass without mutation.
9:48:30 PM PIT >> INFO : MINION : Jan 30, 2020 9:48:30 PM org.junit.vintage.engine.descriptor.RunnerTestDescriptor warnAboutUnfilterableRunner
WARNING: Runner org.junit.internal.runners.ErrorReportingRunner (used on class org.xwiki.rendering.test.integration.RenderingTest) does not support
9:48:30 PM PIT >> INFO : MINION :  filtering and will therefore be run completely.

9:48:30 PM PIT >> INFO : MINION : 9:48:30 PM PIT >> SEVERE : Description [testClass=org.xwiki.rendering.test.integration.RenderingTest, name=initializationError]
org.junit.runners.model.InvalidTestClassError: Invalid test class 'org.xwiki.rendering.test.integration.RenderingTest':
  1. Tes
9:48:30 PM PIT >> INFO : MINION : t class should have exactly one public zero-argument constructor
	at org.junit.runners.ParentRunner.validate(ParentRunner.java:525)
	at org.junit.runners.ParentRunner.<init>(ParentRunner.java:102)
	at org.junit.runners.BlockJUnit4ClassRunner.<init>(BlockJU
9:48:30 PM PIT >> INFO : MINION : nit4ClassRunner.java:84)
	at org.junit.runners.JUnit4.<init>(JUnit4.java:23)
	at org.junit.internal.builders.JUnit4Builder.runnerForClass(JUnit4Builder.java:10)
	at org.junit.vintage.engine.discovery.DefensiveAllDefaultPossibilitiesBuilder$DefensiveJUnit4B
9:48:30 PM PIT >> INFO : MINION : uilder.runnerForClass(DefensiveAllDefaultPossibilitiesBuilder.java:128)
	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:70)
	at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitie
9:48:30 PM PIT >> INFO : MINION : sBuilder.java:37)
	at org.junit.vintage.engine.discovery.DefensiveAllDefaultPossibilitiesBuilder.runnerForClass(DefensiveAllDefaultPossibilitiesBuilder.java:56)
	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:70)
	at org.jun
9:48:30 PM PIT >> INFO : MINION : it.vintage.engine.discovery.ClassSelectorResolver.resolveTestClass(ClassSelectorResolver.java:66)
	at org.junit.vintage.engine.discovery.ClassSelectorResolver.resolve(ClassSelectorResolver.java:57)
	at org.junit.platform.engine.support.discovery.EngineDisc
9:48:30 PM PIT >> INFO : MINION : overyRequestResolution.lambda$resolveUniqueId$3(EngineDiscoveryRequestResolution.java:161)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at java.util.ArrayList$ArrayListSpliterator.tryAdvance(ArrayList.java:1359)
	at java.u
9:48:30 PM PIT >> INFO : MINION : til.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126)
	at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:499)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:486)
	at java.util.stream.Ab
9:48:30 PM PIT >> INFO : MINION : stractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
	at java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:152)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.findFirst(Re
9:48:30 PM PIT >> INFO : MINION : ferencePipeline.java:531)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.resolve(EngineDiscoveryRequestResolution.java:174)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.resolveUniqueId(E
9:48:30 PM PIT >> INFO : MINION : ngineDiscoveryRequestResolution.java:161)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.resolve(EngineDiscoveryRequestResolution.java:118)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.a
9:48:30 PM PIT >> INFO : MINION : ccess$100(EngineDiscoveryRequestResolution.java:53)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution$DefaultContext.resolve(EngineDiscoveryRequestResolution.java:227)
	at org.junit.vintage.engine.discovery.MethodSelectorReso
9:48:30 PM PIT >> INFO : MINION : lver.resolveParentAndAddFilter(MethodSelectorResolver.java:56)
	at org.junit.vintage.engine.discovery.MethodSelectorResolver.resolve(MethodSelectorResolver.java:47)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.lambda$res
9:48:30 PM PIT >> INFO : MINION : olveUniqueId$3(EngineDiscoveryRequestResolution.java:161)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at java.util.ArrayList$ArrayListSpliterator.tryAdvance(ArrayList.java:1359)
	at java.util.stream.ReferencePipeline.forE
9:48:30 PM PIT >> INFO : MINION : achWithCancel(ReferencePipeline.java:126)
	at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:499)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:486)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(Ab
9:48:30 PM PIT >> INFO : MINION : stractPipeline.java:472)
	at java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:152)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:531)
	at org
9:48:30 PM PIT >> INFO : MINION : .junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.resolve(EngineDiscoveryRequestResolution.java:174)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.resolveUniqueId(EngineDiscoveryRequestResolution.j
9:48:30 PM PIT >> INFO : MINION : ava:161)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.resolve(EngineDiscoveryRequestResolution.java:118)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.resolveCompletely(EngineDiscoveryR
9:48:30 PM PIT >> INFO : MINION : equestResolution.java:87)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.run(EngineDiscoveryRequestResolution.java:80)
	at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolver.resolve(EngineDiscoveryR
9:48:30 PM PIT >> INFO : MINION : equestResolver.java:112)
	at org.junit.vintage.engine.discovery.VintageDiscoverer.discover(VintageDiscoverer.java:42)
	at org.junit.vintage.engine.VintageTestEngine.discover(VintageTestEngine.java:62)
	at org.junit.platform.launcher.core.DefaultLauncher.di
9:48:30 PM PIT >> INFO : MINION : scoverEngineRoot(DefaultLauncher.java:177)
	at org.junit.platform.launcher.core.DefaultLauncher.discoverRoot(DefaultLauncher.java:164)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:128)
	at org.pitest.junit5.JUnit5TestUn
9:48:30 PM PIT >> INFO : MINION : it.execute(JUnit5TestUnit.java:88)
	at org.pitest.coverage.execute.CoverageDecorator.execute(CoverageDecorator.java:50)
	at org.pitest.testapi.execute.containers.UnContainer.execute(UnContainer.java:31)
	at org.pitest.testapi.execute.Pitest.executeTests(Pi
9:48:30 PM PIT >> INFO : MINION : test.java:57)
	at org.pitest.testapi.execute.Pitest.run(Pitest.java:48)
	at org.pitest.coverage.execute.CoverageWorker.run(CoverageWorker.java:51)
	at org.pitest.coverage.execute.CoverageMinion.main(CoverageMinion.java:90)

�/�-�\9:48:30 PM PIT >> SEVERE : Description [testClass=org.xwiki.rendering.test.integration.RenderingTest, name=initializationError] did not pass without mutation.
�|9:48:30 PM PIT >> INFO : Calculated coverage in 0 seconds.
9:48:30 PM PIT >> SEVERE : Tests failing without mutation: 
Description [testClass=org.xwiki.rendering.test.cts.RenderingTest, name=initializationError]
Description [testClass=org.xwiki.rendering.test.integration.RenderingTest, name=initializationError]
[JENKINS] Archiving disabled
...

vmassol added a commit to xwiki/xwiki-commons that referenced this issue Jan 31, 2020
@vmassol
[Misc] Exclude a test from pitest since it's a false positive. See ST…
8a2e046 
…AMP-project/pitest-descartes#117

* Removed leftover comment
vmassol added a commit to xwiki/xwiki-platform that referenced this issue Jan 31, 2020
@vmassol
[Misc] Exclude a test from pitest since it's a false positive. See ST…
8c490b4 
…AMP-project/pitest-descartes#117
@vmassol
Copy link
Contributor Author

vmassol commented Feb 1, 2020

Yet another problem:

[ERROR] Failed to execute goal org.pitest:pitest-maven:1.4.11:mutationCoverage (pitest-check) on project xwiki-commons-extension-repository-xwiki: Execution pitest-check of goal org.pitest:pitest-maven:1.4.11:mutationCoverage failed: 1 tests did not pass without mutation when calculating line coverage. Mutation testing requires a green suite.

@vmassol vmassol mentioned this issue Feb 2, 2020
Closed
@oscarlvp
Copy link
Member

oscarlvp commented Feb 3, 2020

@vmassol I'm unable to build this module in my computer. Some enforcer rules are failing. Is there an easy way to skip this verification?

UPDATE:

found the xwiki.enforcer.skip option, but now I get test failures like:

Caused by: java.lang.NoSuchMethodError: 'org.reflections.util.ConfigurationBuilder org.reflections.util.ConfigurationBuilder.filterInputsBy(java.util.function.Predicate)'

Also, I saw that you updated to PIT 1.4.11. Check issue hcoles/pitest#720 it seems that there is a regression in that version. I would suggest you to go back to 1.4.10.

vmassol added a commit to xwiki/xwiki-commons that referenced this issue Feb 3, 2020
@vmassol
[Misc] Exclude a test from pitest since it's a false positive. See ST…
aa7f3bd 
…AMP-project/pitest-descartes#117
@vmassol
Copy link
Contributor Author

vmassol commented Feb 3, 2020 •
edited
Loading

Also, I saw that you updated to PIT 1.4.11. Check issue hcoles/pitest#720 it seems that there is a regression in that version. I would suggest you to go back to 1.4.10.

I tried that but I still get errors. For example:

[INFO] --- pitest-maven:1.4.10:mutationCoverage (pitest-check) @ xwiki-commons-extension-repository-xwiki ---
...
3:35:10 PM PIT >> INFO : Calculated coverage in 2 seconds.
3:35:10 PM PIT >> SEVERE : Tests failing without mutation: 
Description [testClass=org.xwiki.extension.repository.xwiki.internal.SystemHTTPProxyTest, name=[engine:junit-vintage]/[runner:org.xwiki.extension.repository.xwiki.internal.SystemHTTPProxyTest]/[test:testProxy(org.xwiki.extension.repository.xwiki.internal.SystemHTTPProxyTest)]]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  17.552 s
[INFO] Finished at: 2020-02-03T15:35:10+01:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.pitest:pitest-maven:1.4.10:mutationCoverage (pitest-check) on project xwiki-commons-extension-repository-xwiki: Execution pitest-check of goal org.pitest:pitest-maven:1.4.10:mutationCoverage failed: 1 tests did not pass without mutation when calculating line coverage. Mutation testing requires a green suite.
[ERROR] See http://pitest.org for more details.

@vmassol
Copy link
Contributor Author

vmassol commented Feb 3, 2020

@oscarlvp You can run it with mvn clean install -Pquality -Dxwiki.pitest.skip=false -Djacoco.skip=true -U (the -U is to force getting the latest snapshot versions since we rolledback some dependencies that was causing problems). Thx!

@vmassol
Copy link
Contributor Author

vmassol commented Feb 3, 2020

Note that for xwiki-commons-extension-repository-xwiki, you need to remove the following from the POM (that's my workaround FTM):

    <pluginManagement>
      <plugins>
        <!-- Exclude tests breaking pitest, see https://github.com/STAMP-project/pitest-descartes/issues/117
             TODO: remove when fixed -->
        <plugin>
          <groupId>org.pitest</groupId>
          <artifactId>pitest-maven</artifactId>
          <executions>
            <execution>
              <id>pitest-check</id>
              <goals>
                <goal>mutationCoverage</goal>
              </goals>
              <configuration>
                <excludedTestClasses>
                  <param>org.xwiki.**.SystemHTTPProxyTest</param>
                </excludedTestClasses>
              </configuration>
            </execution>
          </executions>
        </plugin>
      </plugins>
    </pluginManagement>

@vmassol
Copy link
Contributor Author

vmassol commented Feb 7, 2020

Ok I've found the issue. It's a regression in pitest-junit5-plugin in v0.11. It works fine with v0.10. And still fails in 0.12.

@vmassol vmassol mentioned this issue Feb 7, 2020
Open
@monperrus monperrus mentioned this issue Apr 9, 2021
Closed
@Rohitesh-Kumar-Jain Rohitesh-Kumar-Jain mentioned this issue Apr 24, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
test-execution
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@vmassol @oscarlvp