Skip to content

Commit

Permalink
fix(repo): Remove non-refactor modules
Browse files Browse the repository at this point in the history
  • Loading branch information
devsjc committed Nov 22, 2024
1 parent cf42210 commit ccbb822
Show file tree
Hide file tree
Showing 85 changed files with 51 additions and 7,914 deletions.
75 changes: 0 additions & 75 deletions ARCHITECTURE.md

This file was deleted.

53 changes: 46 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,26 +42,64 @@ $ docker pull ghcr.io/openclimatefix/nwp-consumer

## Example usage

**To create an archive of GFS data:**
**To download the latest available day of GFS data:***

TODO
```bash
$ nwp-consumer consume
```

## Documentation
**To create an archive of a month of GFS data:**

> [!Note]
> This will download several gigabytes of data to your home partition.
> Make sure you have plenty of free space (and time!)
TODO: link to built documentation
```bash
$ nwp-consumer archive --year 2024 --month 1
```

Documentation is generated via [pydoctor](https://pydoctor.readthedocs.io/en/latest/).
## Documentation

Documentation is generated via [pdoc](https://pdoc.dev/docs/pdoc.html).
To build the documentation, run the following command in the repository root:

```bash
$ python -m pydoctor
$ PDOC_ALLOW_EXEC=1 python -m pdoc -o docs --docformat=google src/nwp_consumer
```

> [!Note]
> The `PDOC_ALLOW_EXEC=1` environment variable is required due to a facet
> of the `ocf_blosc2` library, which imports itself automatically and hence
> necessitates execution to be enabled.
## FAQ

### How do I authenticate with model repositories that require accounts?

Authentication, and model repository selection, is handled via environment variables.
Choose a repository via the `MODEL_REPOSITORY` environment variable. Required environment
variables can be found in the repository's metadata function. Missing variables will be
warned about at runtime.

### How do I use an S3 bucket for created stores?

The `ZARRDIR` environment variable can be set to an S3 url
(ex: `s3://some-bucket-name/some-prefix`). Valid credentials for accessing the bucket
must be discoverable in the environment as per
[Botocore's documentation](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html)

### How do I change what variables are pulled?

With difficulty! This package pulls data specifically tailored to Open Climate Fix's needs,
and as such, the data it pulls (and the schema that data is surfaced with)
is a fixed part of the package. A large part of the value proposition of this consumer is
that the data it produces is consistent and comparable between different sources, so pull
requests to the effect of adding or changing this for a specific model are unlikely to be
approved.

However, desired changes can be made via cloning the repo and making the relevant
parameter modifications to the model's expected coordinates in it's metadata for the desired model
repository.

## Development

Expand All @@ -77,7 +115,8 @@ $ python -m ruff check .
```

Be sure to do this periodically while developing to catch any errors early
and prevent headaches with the CI pipeline.
and prevent headaches with the CI pipeline. It may seem like a hassle at first,
but it prevents accidental creation of a whole suite of bugs.

### Running the test suite

Expand Down
7 changes: 5 additions & 2 deletions src/nwp_consumer/cmd/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,11 @@ def parse_env() -> Adaptors:
case "metoffice-datahub":
model_repository_adaptor = \
repositories.model_repositories.MetOfficeDatahubModelRepository
case _ as model:
log.error(f"Unknown model: {model}")
case _ as mr:
log.error(
f"Unknown model repository '{mr}'. Expected one of "
f"['gfs', 'ceda', 'ecmwf-realtime', 'metoffice-datahub']"

Check failure on line 37 in src/nwp_consumer/cmd/main.py

View workflow job for this annotation

GitHub Actions / lint-typecheck

Ruff (COM812)

src/nwp_consumer/cmd/main.py:37:74: COM812 Trailing comma missing
)
sys.exit(1)

notification_repository_adaptor: type[ports.NotificationRepository]
Expand Down
56 changes: 0 additions & 56 deletions src/nwp_consumer/cmd/test_main.py

This file was deleted.

91 changes: 0 additions & 91 deletions src/nwp_consumer/internal/cache.py

This file was deleted.

31 changes: 0 additions & 31 deletions src/nwp_consumer/internal/config/__init__.py

This file was deleted.

Loading

0 comments on commit ccbb822

Please sign in to comment.