Modify generation temperature for code generation keyword args to 0.2

[mansheej]

What does this PR do?

This PR changes the generation temperature from the default 1.0 to 0.2 for code evals. For generation of code, lower temperature leads to more deterministic and better outputs. Also, this is the standard in other frameworks, see: https://huggingface.co/spaces/bigcode/bigcode-models-leaderboard.

What issue(s) does this change relate to?

Before submitting

• Have you read the contributor guidelines?
• Is this change a documentation change or typo fix? If so, skip the rest of this checklist.
• Was this change discussed/approved in a GitHub issue first? It is much more likely to be merged if so.
• Did you update any related docs and document your change?
• Did you update any related tests and add any new tests related to your change? (see testing)
• Did you run the tests locally to make sure they pass?
• Did you run pre-commit on your change? (see the pre-commit section of prerequisites)

[hanlint]

Wouldn't this break backwards compatibility for users for ICL code evals, and such that users cannot recover the temperature=0.1 behavior without editing the code? Instead, we should add a temperature argument to the class that gets passed to the generation kwargs.

[mansheej]

Wouldn't this break backwards compatibility for users for ICL code evals, and such that users cannot recover the temperature=0.1 behavior without editing the code? Instead, we should add a temperature argument to the class that gets passed to the generation kwargs.

Yes, that is true, it would break backward compatibility. However, the default temperature right now is 1.0 not 0.1 which is something I would consider more of a "bug" that should be fixed/a warning should be raised saying that the temperature is not standard and the results will be suboptimal.
However, I do agree that we should add generation kwargs and I think there is another PR for that.
I will leave this open so that @abhi-mosaic can use this branch as a temporary workaround and will close it once the other PR gets merged in.

[hanlint]

If it's a bug, and the other PR makes it in before a next release, that's fine to merge.

[dakinggg]

@mansheej im guessing this should be closed now?