Does anyone use user defined tests in their ingest packages?

[znatty22]

If you recall, you can define tests inside your ingest package that will get executed via pytest as part of ingest validation. The goal was to give an ingest package developer a way to further test their extract and transform stage output.

Does anyone use this feature? If not, I'd like to get rid of that code and its depedencies (pytest). The version of pytest that it uses often conflicts with other code bases that use the ingest library and also use pytest for unit testing.

[youngnm]

No, I am not using it!

[fiendish]

even if they did, integrating it isn't necessary. the old data structures are gone now. someone could script running the first two stages and looking at the output before running the last stage.

[chris-s-friedman]

I never have used it - but actually wanted to do it for the first time the other day

[znatty22]

@fiendish @liberaliscomputing @chris-s-friedman @youngnm @nicholasvk Ok so then should we get rid of it? React, 👍 for yes or 👎 for no/we should keep it.

[chris-s-friedman]

👎 if i can get a tutorial on how to write user defined tests 😜

[fiendish]

people who have never used it want to keep it, lol. no, come on, get rid of it plz. 😆

[fiendish]

Like if you want to do ET and only L on some condition, the library and CLI already support that behavior without wedging in a dependency on pytest by letting you run just ET. So you can run just ET, look at the output however you want, and then decide to run L after.

And if someone wants to do that check with pytest, that's fine. And if someone wants to script that process so they don't have to do it manually, great too. And if they want to add it to the library more generically instead of this, that's also ok. But we don't have to make the library itself depend on pytest for it.

But if the problem is version conflicts, we could also just unpin the version.