Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add patches to reduce variance for coherence scoring #64

Closed
donaldknoller opened this issue Oct 2, 2024 · 1 comment
Closed

Add patches to reduce variance for coherence scoring #64

donaldknoller opened this issue Oct 2, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@donaldknoller
Copy link
Contributor

donaldknoller commented Oct 2, 2024 •
edited
Loading

Suggested implementations for reducing variance of the coherence score:

  • When calling openAI API for coherence_score, there will be less variance if the temperature is set to 0 over the default value.
  • Add a more explicit handler for openAI API errors
@donaldknoller donaldknoller added the enhancement New feature or request label Oct 2, 2024
@donaldknoller donaldknoller mentioned this issue Oct 8, 2024
Closed
@donaldknoller donaldknoller changed the title Set temperature to 0 when calling openAI for coherence_score Add patches to reduce variance for coherence scoring Oct 9, 2024
This was referenced Oct 9, 2024
Closed
Merged
@donaldknoller
Copy link
Contributor Author

This has been patched via #68

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@donaldknoller