Feature/restructure tests

[ladsmund]

Refactored and Organized Test Modules

• Moved test modules and data from the package directory to the root-level tests directory.
• Updated directory structure to ensure clear separation of source code and tests.
• Updated import statements in test modules to reflect new paths.
• Restructured the tests module:
    - Renamed original automatic tests to e2e as they primarily test the main CLI scripts.
    - Added unit directory for unit tests.
    - Created data directory for shared test data files.

This comprehensive refactoring improves project organization by clearly separating test code from application code. It facilitates easier test discovery and enhances maintainability by following common best practices.

[PennyHow]

Why is the test module outside of src/pypromice? I would have thought it would be better to be inside pypromice and distributed with the package so that users can also test their installations (and we can test installations on new instances).

[PennyHow]

I've just opened a PR to this branch #'270, just to update the function naming conventions in e2e.

[ladsmund]

Why is the test module outside of src/pypromice? I would have thought it would be better to be inside pypromice and distributed with the package so that users can also test their installations (and we can test installations on new instances).

The tests are located outside the package to maintain a clear separation of concerns. This way, the pypromice package is solely focused on functionalities intended for production and end-users. The tests, run during continuous integration (CI), ensure that the distributed package works as intended. This approach also reduces the package size by excluding test code and data. I understand your point about using tests to verify an installation. The CI script is designed to test scenarios where pypromice is installed in a new environment, addressing this concern.

I am open for discussion, but these are my current thoughts.

[PennyHow]

This approach also reduces the package size by excluding test code and data.

I thought the test datasets were pretty light-weight, so do not make the package particularly bulky.

I understand your point about using tests to verify an installation.

I think this will be useful (to us, and general users) especially now we have the bufr processing tests in here, as the eccodes bindings can sometimes not install correctly. Running the unit tests within the pypromice installation in say, a docker image, would be really useful.

Let's discuss more next time we talk

[PennyHow]

Looks good @ladsmund!