Tests fail when requiring a non-stable Matomo version in plugin.json

[tsteur]

I've tried to require below Matomo version in plugin.json

    "require": {
        "matomo": ">=4.12.0-rc2,<5.0.0-b1"
    },

All automated tests then fail because it ends up using the master branch.

Looking into the issue I believe this is due to this line https://github.com/matomo-org/travis-scripts/blob/master/matomo_version_parser.php#L31

|| version_compare($version, \Piwik\Version::VERSION) > 0).

By default, travis will check out the 4.x-dev branch. But because in https://github.com/matomo-org/matomo/blob/4.x-dev/core/Version.php#L24 the latest version is 4.12.0-b it won't let you use this Matomo version.

This must have been a regression from using the next_release branch where the version number is 4.12.0-rc2: https://github.com/matomo-org/matomo/blob/next_release/core/Version.php#L24

Maybe this check is not actually needed or could be simplified?

As a result, it's currently using master branch and then the tests don't run at all as it's a very old version. It should instead check out the correct version.

[sgiehl]

Actually I think, we don't need to adjust anything if we directly merge all changes done to next_release back to 4.x-dev.

[tsteur]

@sgiehl the issue will eventually appear though again once next_release and 4.x-dev aren't up to date? Even if they defer only a few minutes and a build runs at the same time it can cause issues?

I'm also not sure why this check is there in the first place. It doesn't seem helpful. Like once 4.x-dev changes to 5.0 version then you could also technically require any invalid version like 4.29.59

[sgiehl]

I'm also not sure why this check is there in the first place. It doesn't seem helpful. Like once 4.x-dev changes to 5.0 version then you could also technically require any invalid version like 4.29.59

Yes, don't know what that check is good for either. Guess we can simply remove it.

Also we could maybe improve the default checkout. As we don't use the master branch anymore, we could try to find the branch name using Version::MAJOR_VERSION and adding x-dev to it maybe...

[tsteur]

Also we could maybe improve the default checkout. As we don't use the master branch anymore, we could try to find the branch name using Version::MAJOR_VERSION and adding x-dev to it maybe...

Actually, it was kind of good the tests failed that way as we otherwise wouldn't have noticed the tests aren't running against the right branch. It was supposed to run against min required Matomo meaning it would have executed the same tests as the ones for latest matomo. Maybe could even throw an error or just print a proper error message like "wrong branch configured" or something? We can also use the x-dev though

[sgiehl]

Closing this one as we meanwhile migrated to GitHub actions. We've applied some improvements around that topic. It also won't fall back to run tests against master there anymore.