redo logging #415
redo logging #415
Conversation
|
|
|
hm ok for me but i'd like to have kinda "where we're at" info - maybe a new "[STEP]" tag?
|
|
but no strong opinion |
Added 
|
| @@ -575,7 +584,7 @@ def compute(self, golds: list[str], predictions: list[str], **kwargs) -> dict: | |||
| dict: Scores over the current sample's items. | |||
| """ | |||
| if self.bert_scorer is None: | |||
| hlog_warn("The first metric computation step might be a bit longer as we need to download the model.") | |||
| logger.info("The first metric computation step might be a bit longer as we need to download the model.") | |||
|
Wil you rebase it on main? |
|
No I will merge it into the CLI refacto (this will create a big PR But easy to manage as the are separated I think). Doing this because I wanted a clean demo to show |
|
Ok, then:
|
|
At the moment, this one is based on your closed source PR, can you rebase it to main? |
|
mhh this PR is based on the refacto CLI, you mean rebase the cli refacto to main ? |
|
Yep! |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
| @@ -25,7 +25,6 @@ appropriate extras group. | |||
|
|
|||
| | extra name | description | | |||
| |--------------|---------------------------------------------------------------------------| | |||
| | accelerate | To use accelerate for model and data parallelism with transformers models | | |||
There was a problem hiding this comment.
We'll want to offer a no parallelism option I think
There was a problem hiding this comment.
not sure what you mean, parallelism is not enabled by default, we have to set model_parallel=True
There was a problem hiding this comment.
Using accelerate is mandatory to launch lighteval, but we might want a setup with no DP/MP/PP of any kind, without gpu use
decent logging