Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ignorePending not excluding pending test cases from testrail test run results #62

Open
chrisgray37 opened this issue May 24, 2024 · 2 comments

Comments

@chrisgray37
Copy link

chrisgray37 commented May 24, 2024

I am using this in conjunction with cypress/grep. All the pending test cases are because they were skipped due to tagging. However, they are still showing up as untested in testrail run.

I am on this version of cypress-testrail:
"cypress-testrail": "^2.10.0",

Here is my cypress.config.js:

const {defineConfig} = require('cypress');
const TestRailReporter = require('cypress-testrail');

module.exports = defineConfig({
    video: true,
    reporter: 'cypress-multi-reporters',
    reporterOptions: {
        configFile: 'reporter-config.json',
    },
    scrollIntoView: {
        offset: {
            top: -150,
        },
    },
    viewportWidth: 1800,
    viewportHeight: 1000,
    chromeWebSecurity: false,
    e2e: {
        setupNodeEvents(on, config) {
            new TestRailReporter(on, config).register();
            require('@cypress/grep/src/plugin')(config);
            return config
        },
    },
    env: {
        grepFilterSpecs: true,
        grep: {
            showPending: true,
            showSkipped: true,
        }
    },
});

require('@applitools/eyes-cypress')(module);

And here is my testrail section of the cypress.env.json:

  "testrail": {
      "domain": "example.testrail.io",
      "username": "[email protected]",
      "password": "examplePW",
      "projectId": "42",
      "milestoneId": "",
      "suiteId": "",
      "runName": "Cypress Automation Pipeline Run: (__datetime__)",
      "ignorePending": true,
      "closeRun": true,
      "screenshots": true
  },

And here is my console when running the tests:

 1 passing (3s)
  16 pending

 Updating TestRail run R7088. Adding test cases: 9425, 9426, 9427, 9428, 9438, 9429, 9430, 9431, 9432, 9433, 9434, 9435, 9437, 9551, 9552, 9553, 9436 
 TestRun updated in TestRail: 7088 
 Ignoring pending test: C9425: Test Actuator Info 
 Ignoring pending test: C9426: Test Actuator Health 
 Ignoring pending test: C9427: Test Copy Test Data 
 Ignoring pending test: C9428: Test Generate, Reset, Then Delete Test Data 
 Ignoring pending test: C9438: Test Extract 
 Ignoring pending test: C9429: Test Details 
 Ignoring pending test: C9430: Test Resume 
 Ignoring pending test: C9432: Test Restore Collection 
 Ignoring pending test: C9433: Test Move Assets 
 Ignoring pending test: C9434: Test Delete Job 
 Ignoring pending test: C9435: Test List Jobs 
 Ignoring pending test: C9437: Test Find Job 
 Ignoring pending test: C9551: Test Delete Asset - Bad collectionName 
 Ignoring pending test: C9552: Test Delete Asset - Bad profileName 
 Ignoring pending test: C9553: Test Delete DFC 
 Ignoring pending test: C9436: FFV E2E 
 TestRail >> Sending case results to run R7088: C9431 
 Results sent to TestRail R7088 for: C9431 

  (Results)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Tests:        17                                                                               │
  │ Passing:      1                                                                                │
  │ Failing:      0                                                                                │
  │ Pending:      16                                                                               │
  │ Skipped:      0                                                                                │
  │ Screenshots:  0                                                                                │
  │ Video:        true                                                                             │
  │ Duration:     2 seconds                                                                        │
  │ Spec Ran:     api-ffv.cy.js                                                                    │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
@chrisgray37
Copy link
Author

@boxblinkracer

@chrisgray37
Copy link
Author

chrisgray37 commented May 24, 2024

Also, if I change ignorePending to false, all the pending test results are marked as blocked in testrail; and if I just delete the line altogether, I get the same result as if i did "ignorePending": true,, where it shows all pending tests as untested

@chrisgray37 chrisgray37 changed the title ignorePending not excluding pending test cases from testrail suite ignorePending not excluding pending test cases from testrail test run results May 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant