Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support NUnit XML format #38

Merged

Conversation

bryanbcook
Copy link
Contributor

@bryanbcook bryanbcook commented Nov 11, 2023

Addresses #4

  • NUnit V2 Support
  • NUnit V3 Support

The formats for test-suites and test-case between V2 and V3 are similar, but have some structural and summary differences.

V2:

  • top-level node is test-results. The attribute total represent the number of executed, where as not-run were not executed. There is no total for passed, it must be inferred from the total by subtracting errors+failures+inconclusives.
  • test-suites are nested within results nodes, with each part of the namespace represented as a different node. As a result, the depth of the structure is not fixed.
  • Names for test-suites are partial, whereas the name of the test-case is the fully-qualified namespace + class name + method.
  • The test-case nodes are held primarily in test-suites that have a TestFixture type, with the exception of ParameterizedTest.
  • Properties/Categories can be set at the fixture of test-case level.
  • No id attributes on nodes.
<test-results total="n" errors="n" failures="n" not-run="n" inconclusive="n" ignored="n" skipped="n" invalid="n">
  <test-suite type="Assembly" name="/full/path/to/assembly.dll">
    <results>
      <test-suite type="Namespace" name"="NamespaceName" >
        <results>
          <test-suite type="TextFixture" name="ClassName">
             <categories>
              <category name="..." value="..." />
             </categories>
             <properties>
               <property name="..." value=".." />
             </properties>        
             <results>
               <test-case name="NamespaceName.ClassName.MethodName"
                          executed="{true|false}" 
                          result="{Success|Failure|Ignored|Error|Inconclusive|NotRunnable}" 
                          success="{true|false}">
                  <categories>
                    <category name="..." value="..." />
                  </categories>
                  <properties>
                    <property name="..." value=".." />
                  </properties>
                  <!-- either failure or reason (NotRunnable) -->
                  <failure>
                    <message><![CDATA[ ... ]]></message>
                    <stacktrace><![CDATA[ ... ]]></message>
                  </failure>
                  <reason>
                    <message><![CDATA[ ... ]></message>
                  </reason>
               </test-case>
             </results>
          </test-suite>
        </results>
      </test-suite>
    </results>
  </test-suite>
</test-results>

V3:

  • top level node is test-run. Additional attributes for testcasecount (previously total + not-run), passed (previously total - errors+failures+inconclusive).
  • test-suite nodes are not deeply nested, though
  • test-suite now has fullname
  • test-case no longer has executed or success attributes. The Label attribute contains Error, Cancelled or Invalid if there's a Failed state.
  • test-case result can be Passed (instead of Success), Failed (instead of Failure), Inconclusive or Skipped.
  • V2 previously included test-cases for invalid test-fixtures, which were marked as executed="False". V3 does not include these test-case nodes, but the test-suite counts them in the testcasecount attribute, so it's possible to have an empty test-suite that indicates it has test-cases but it does not.
  • Support for <properties> but not <categories>. In addition to fixture and test-case level, properties are also supported at the test-suite Type="Assembly" level.
  • id attributes are present.
<test-run id="n" result="{Passed|Failed}" testcasecount="n" total="n" passed="n" failed="n" inconclusive="n" skipped="n" >
  <test-suite id="n" type="Assembly" fullname="/path/to/assembly.dll">
    <properties>
      <property name="..." value="..." />
    <properties>
    <test-suite id="n" type="TestFixture" fullname="NamespaceName.ClassName"
                result="{Passed|Failed|Inconclusive|Skipped}">
      <properties>
        <property name="..." value="..." />
      <properties>
      <test-case id="n" name="MethodName" fullname="NamespaceName.ClassName.MethodName"
                 result="{Passed|Failed|Skipped}"
                 label="{Error|Cancelled|Invalid}">
        <!-- either failure or reason (Invalid) -->
        <failure>
          <message><![CDATA[ ... ]]></message>
           <stacktrace><![CDATA[ ... ]]></message>
        </failure>
        <reason>
          <message><![CDATA[ ... ]></message>
        </reason>
      </test-case>
    </test-suite>
  <test-suite>
</test-run>

This implementation automatically detects whether you are using nunit v2 or nunit v3 format.

@ASaiAnudeep ASaiAnudeep merged commit 67f113f into test-results-reporter:main Nov 12, 2023
6 checks passed
@bryanbcook bryanbcook deleted the feature/4-nunit-support branch November 13, 2023 13:05
This was referenced Feb 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants