Add swiss legal evals as new community tasks

[JoelNiklaus]

Adds new community tasks with swiss legal evaluations. Currently translation tasks are supported but others may follow in the future.

[clefourrier]

@hynky1999 tagging you if you've got a couple minutes to check the templating when back from the offsite

[hynky1999]

Re templates:
We don't have any template for translation tasks atm.
There are many variants to go with (see the image below), but I would prefer going with the [src]: [input] [tgt]: (A variant). Since translation is inherently cross-lingual tasks and it's not clear which language we should use (target or source?), such template allows us to be independant on language (the language labels are kinda standardized, but yeah they will be in latin).

@JoelNiklaus
Have you experimented different prompt formats?

Source: https://arxiv.org/pdf/2301.07069

I can quickly make a PR for the translation template and we can convert it to that.

[JoelNiklaus]

I haven't experimented with prompts yet. Yes, going with variant A sounds good.

Thanks so much!

[JoelNiklaus]

Btw. what is the reason you are not using the metrics from evaluate?

[clefourrier]

Evaluate is no longer actively maintained (it's indicated in the Github readme). We also wanted lighteval to be light, and not rely on a heap of dependencies.

[JoelNiklaus]

I see. I used the direct implementation for COMET and METEOR, rather than evaluate.

[NathanHB]

PR looks great ! Do the results on your evals look sound ?
Also, you can use the pre-commit hooks to format the files and fix the CI :)

pip install pre-commit
pre-commit install
pre-commit run --all-files

[JoelNiklaus]

Great, thanks!
Just ran the pre-commit hooks.

Couldn't run the evals yet because of the judge prompt. Hope to do that soon.