Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce MSTest Support #44

Merged

Conversation

bryanbcook
Copy link
Contributor

@bryanbcook bryanbcook commented Nov 13, 2023

This PR introduces basic support for MS Test TRX format:

<TestRun xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
  <Results>
    <UnitTestResult testId="..." duration="..." outcome="...">
      <Output>
        <ErrorInfo>
          <Message>...</Message>
          <StackTrace>...</StackTrace>
        </ErrorInfo>
       </Output>
    </UnitTestResult>
    <UnitTestResult .../>
  </Results>
  <TestDefinitions>
    <UnitTest name="..." id="...">
      <TestCategory>
        <TestCategoryItem TestCategory="..." />
      </TestCategory>
      <!-- Note Properties are supported by the Schema but are not emitted on all platforms -->
      <Properties>
        <Property>
          <Key>...</Key>
          <Value>...</Value>
        </Property>
      </Properties>
      <TestMethod className="..." name="..." />
    </UnitTest>
    <UnitTest .../>
  </TestDefinitions>
</TestRun>

Known issues:

  • %VSInstallDir%/Xml/Schemas/vstst.xsd defines multiple data types under /TestRun/Results.

    Element Type Supported
    UnitTestResult Yes
    GenericTestResult No
    ManualTestResult No
    WebTestResult No
    LoadTestResult No
  • %VSInstallDir%/Xml/Schemas/vstst.xsd defines multiple data types under /TestRun/TestDefinitions:

    Element Type Supported
    UnitTest Yes
    UnitTestElement Deprecated
    ManualTest No
    WebTest No
    CodedWebTest No
    OrderedTest No
    LoadTest No
    GenericTest No
  • Although Properties are defined in the schema, they are not emitted by Visual Studio. see Add Attributes to the TestDefinition in the trx file. microsoft/vstest#2480

  • Could not replicate a test that had an "Error", so errors count on testresult and testsuite will always be empty. Invalid tests that cannot be run are not included in the test output, and details regarding their load failure are listed as plain-text in the ResultSummary element.

  • Test durations are currently not calculated. MSTest results do not include durations #43

Support for the elements listed above could be introduced if we had sample xml (trx) outputs.

@ASaiAnudeep ASaiAnudeep merged commit 4255d9e into test-results-reporter:main Nov 15, 2023
6 checks passed
@bryanbcook bryanbcook deleted the feature/5-mstest-support branch November 15, 2023 12:53
@czimbortibor
Copy link

Any chance this gets published in a new release?

@bryanbcook
Copy link
Contributor Author

Waiting on @ASaiAnudeep re #48

@bryanbcook
Copy link
Contributor Author

@czimbortibor a new release was pushed earlier today.

@czimbortibor
Copy link

Awesome! Thank you very much for this great tool. Appreciated

@bryanbcook bryanbcook mentioned this pull request Feb 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants