Architectural redesign and concurrent execution

[tyranron]

Pre-history and motivation

We hardly use cucumber_rust in our production projects having ~800 scenarios and recently have started to suffer from its limitations: no concurrent runner, inability to customize its parts without changing upstream code.

This PR is our long-going effort to make the cucumber_rust extensible, customizable, while ergonomic and rational out-of-the-box for most cases. It represents an almost complete redesign and rewrite of the library with introducing some new features along and fixing many existing issues. It was battle-tested in our code base for quite a while, and showed significant improvements of test execution times. All the documentation, images and the Book are made up-to-date with the changes.

New architecture

The new design of the library splits it to the 3 main components in the following pipeline:

Parser -> Runner -> Writter

where:

• Parser is responsible for sourcing Scenarios for execution.
• Runner is responsible for executing the stream of Scenarios.
• Writer is responsible for outputting execution results.

All the 3 components may be customized or changed to a custom implementation, which solves any questions about implementing custom output, or custom execution order, or anything else.

Still the library provides reasonable defaults:

• parser::Basic - reads from .feature files and supports # language: comment.
• runner::Basic - executes all the scenarios concurrently by default (max 64 at the same time), and customizable to run fully (max 1 concurrent) or partially sequential (via @serial tag).
• writer::Basic - simply outputs execution results to STDOUT, while writer::Normalized and writer::Summarized are out-of-the-box wrappers introducing output order normalization and writing summary of execution (for anyone implementing custom Writter to no bother about these problems).

Output

We've also stripped from the default output the filenames and line numbers for successful steps, as from our experience they make the output unreasonable bloated and hard-to-read. We still keep them for failing steps, though.

Codegen

macros feature is now enabled by default, as provides more ergonomic and understandable way to write step matchers.

Code style

rustc lints of the project are somewhat hardened and clippy::pedantic lints are checked on CI now. We believe that tightening code style is beneficial for open source libraries and future contributions, despite it may mandate some strange or verbose situations in code.

Releasing and CI

Release process now is fully automated via GitHub actions. Just prepare the repository for a release, commit and push the appropriate version tag.

⚠️ ACTION REQUIRED: You should set the CRATESIO_TOKEN to the project's GitHub secrets to release via CI.

CI now fully checks and tests the project, including the Book.

Resolved issues

• Allow cropping or suppressing printed docstrings on steps #56: writer::Basic has really minimal output for success steps now.

• Return a demonstration gif to the readme #62: All the GIFs are ap-to-date now/

• Add proper documentation in the crate #63: We've tried our best to document everything pretty well, and enabled Clippy and rustc lints to keep an eye on that.

• Support for example values - or better examples #64: Possible by using regex now.

• Emoji in default output causes width calculation to be incorrect #71: Fixed and not applicable anymore.

• Test fails without meaningfull error #80: Parsing errors are now propagated to the Writer.

• Scenario outline doesn't work inside a Rule scope #86: Just fixed.

• Deadlock when captured stdout is too large #88: Fixed and not applicable anymore.

• Add default language to Cucumber runner builder #102: Now possible with Cucumber::language().

• Add default matching tags to Cucumber runner builder #103: Can be done by Cucumber::filter_run_and_exit().

• Colors when output is not a TTY #119: writer::Basic pretty-prints only if terminal was detected.

• Add a configuration option for making not-implemented tests failures #123: Now possible with writer::Summarised::fail_on_skipped().

• v. 0.9.0: Setting the default language in the Cucumber builder is not applied during test execution #126: Now uses # language: comment to deduce language dynamically.

• (partially) Feature: xunit/junit output #50 and Produce output JSON for integration with Cucumber for Jira #127: While not provided out-of-the-box, may be implemented via custom Writer.

Contribution and maintaining.

We understand that this PR brings a huge impact to the library and may be undesirable by its author. If the later is true, we're OK to go on with our own fork and maintain it. However, from the Rust ecosystem perspective, it would be better to not introduce any ecosystem splits and disperse contributors efforts. That's why we also ask @bbqsrc to add me (@tyranron) and @ilslv to project maintainers, so we can continue to maintain the project and contribute to it more easily and actively. Ideally, we see the project to migrate to some GitHub org (like cucumber-rs or smthng), where new maintainers may step-in to the project more easily in the future.

[bbqsrc]

I will review this somewhat over the weekend, provide some feedback, and definitely add you as co-maintainer in either case. 😄

[ms-ati]

@bbqsrc Hi Brendan, I just want to high-five you on all your work on this project so far, and also the open and welcoming way in which you chose to engage with this massive PR.

Such a positive example of open source community interaction to see in my email this morning!

[rob-mur]

Looks like a lot of changes and sounds very exciting! I had my eye on working further on the json output for jira, and implementing the configuration option for skips=fail. In principle this redesign sounds like that work would be easier with these changes so I'll await the review for now. Thanks for your hard work!

[bbqsrc]

The gherkin parser already handles setting the language if a language comment is found, so I'm wondering what this code is for?

[ilslv]

@bbqsrc oh, I got confused by #126, fixed this

[tyranron]

@rob-mur after merging this PR we have plans for some subsequent works including JSON+Jira and JUnit output. Considering skip=fail behavior, it's already done in this PR. May it will require some additional documentation in Book, though.

[rob-mur]

@tyranron ok cool! I'm happy to test/write documentation if you're handling the implementation, just let me know.

[bbqsrc]

hihi, I haven't forgotten, just an extremely busy two weeks for me, I will get to this again soon 😄

[bbqsrc]

You have both been added as contributors. I will deal with the rest of the bureaucracy (crates.io team, changing org on GitHub, etc) some time this week.

[bbqsrc]

There's likely some more issues that I've missed, but I won't have time to do a proper review for at least a few more days, so merging now so at least movement can be made on the other issues people care about before doing a release next week.

Sound good?

[tyranron]

@bbqsrc we would like to polish it a little bit more for a week or two before releasing.

[svenfoo]

Is there some sort of a Porting from cucumber-rust 0.9 to cucumber 0.10 document available? It doesn't seem straight-forward to do the update.

[ilslv]

@svenfoo It heavily depends on what library features you were using before the update.

In case you were already using attribute macros then it's just a matter of changing your main function to MyWorld::run("path").await

Otherwise, if you were using approach from the example repository, then you should change couple of things:

1. Use WorldInit derive macro on your World deriver.

#[derive(WorldInit)]
enum MyWorld {
    // ...
}

2. Change the way you register your steps

From

pub fn steps() -> Steps<crate::MyWorld> {
    let mut steps: Steps<crate::MyWorld> = Steps::new();

    steps.given("a string with some particular value", |_world, _ctx| {
        MyWorld::SomeString(default_string())
    });

    steps.when(
        "I append a known suffix to the value",
        |world, _ctx| match world {
            MyWorld::SomeString(x) => MyWorld::SuffixedString(interesting_appendage(&x)),
            _ => panic!("Invalid world state"),
        },
    );
}

To

#[given("a string with some particular value")]
fn value(w: &mut MyWorld) {
//          ^^^^ World is now passed by mutable reference instead of value. 
    *w = MyWorld::SomeString(default_string());
}

#[when("I append a known suffix to the value")]
fn suffix(w: &mut World) {
    let with_suffix = if let MyWorld::SomeString(x) = &w {
        interesting_appendage(x)
    } else {
         panic!("Invalid world state");
    }
    *w = MyWorld::SuffixedString(with_suffix),
},

All addition features of attribute macros (like regex support and more) are described in the docs and The Book

3. Change your main function to

#[tokio::main] // or some other way to run async main, we are runtime-agnostic
async fn main() {
    MyWorld::run("path").await;
}

---

There is no separate book chapter on porting to 0.10 because I think that all those steps are understandable from README or Getting Started. If you are still struggling to understand something please give some feedback, so I can improve docs/write additional chapter on porting.