Support modeling activities that finish without doing anything

[magnusbaeck]

Summary

As someone who attempts to model a CI system
I want a way to model an activity that finished early because it realized that there's nothing to do
So that I can compute correct metrics or visualize the behavior

Context

There are cases where activities need to start executing and won't complete in the normal fashion, but rather short-circuit themselves and not do anything. This would be an additional possible outcome of an activity, expressing "yes, technically I started but it turned out that there was nothing for me to do", which isn't the same thing as "successful", "failure", "abort", or any of the other possible current outcomes.

In CDEvents, the testSuiteRun finished event has an outcome field whose value space currently is "pass", "fail", "cancel", and "error", where "cancel" is what's currently missing in Eiffel. Since "cancel" already means something in the context of Eiffel activities, we should choose another term. Perhaps "skipped"?

Exemplification

An activity tasked with completing an artifact build could have a caching mechanism that, after an analysis, concludes that nothing needs to be done except send an EiffelArtifactReusedEvent. It would be useful to outside observers to be able to distinguish such activities from the usual ones that take much longer and eventually result in an EiffelArtifactCreatedEvent.

Drawbacks

No response

Out of Scope

No response

Further links

No response

Acceptance Criteria

No response

Implementation Ideas

The data.outcome.conclusion member of EiffelActivityFinishedEvent should be given an additional enum value, e.g. "SKIPPED".

The same thing should be considered for EiffelTestCaseFinishedEvent, but a concrete use case is less clear to me. It's my impression that test cases execution is usually predetermined, but I'm sure there are cases where they need to skip their execution when they've started.

[e-backmark-ericsson]

See also this commit in a CDEvents PR related to noop activities: https://github.com/cdevents/spec/pull/238/files/da08d70d23d08bc799232a587051026e43eaf4ca..53e8fe1de6a62ccf0eb61d803bb5eb71bb45c5e6

[magnusbaeck]

This was discussed at the 2024-12-05 community meeting:

• There are legitimate real-world cases where activities need to start but where they reach the conclusion that nothing actually needs to be done.
• We could call this an abortion, but then we won't be able to distinguish them from manual activity abortions.
• Maybe cancellation is a better term, since it aligns well with the ActC event. The difference between those two cases is basically the timing of the activity's demise. In the ActC it happens before the activity starts, and here we're talking about what happens if it happens after the start.
• For test cases, we have both the conclusion of the test (SUCCESSFUL, FAILED, ABORTED, TIMED_OUT, INCONCLUSIVE) and its verdict (PASSED, FAILED, INCONCLUSIVE). Is there a value in having a similar model for activities? They don't have a pass/fail status, but they could be successful yet skipped/cancelled.
• Separate weirdness that perhaps should be corrected: For TSF, neither data.outcome.conclusion or data.outcome.verdict are required, meaning that it's legal to set data.outcome to an empty object.
• Comparing to the test events in CDEvents, a queued event must be followed by either started or finished, where both can have an "cancelled" outcome.
• If we were to split the outcome for ActF (mirroring TCF), "verdict" doesn't sound right.
• We currently can't describe activities that complete but with some errors ignored, i.e. they're not "perfect" but they're different from failed activities.
• Maybe the split isn't necessary? It would affect many producers and consumers, and the marginal benefit is limited. Perhaps we should look into concrete examples of the value spaces of a conclusion and a verdict (or whatever it should be called) and see if we need the expressiveness.
• No major objections against a new CANCELLED enum value for data.outcome.conclusion.
• TCF also lacks a CANCELLED conclusion. For tests, SKIPPED would be a better name to align with standard testing terminology.
• Leaning towards skipping a split of outcome.