Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add node to use OpenAI's GPT-3 for QA #2605

Merged
merged 17 commits into from
Jul 8, 2022
Merged

Add node to use OpenAI's GPT-3 for QA #2605

merged 17 commits into from
Jul 8, 2022

Conversation

tholor
Copy link
Member

@tholor tholor commented May 28, 2022 •
edited by bogdankostic
Loading

Proposed changes:

Integrating OpenAI's GPT-3 model via their API.
The node can take documents as input and generates top_k answers by calling the remote API from OpenAI.
Users will need to signup at OpenAI and supply their api_key to the node.

Example:

node = OpenAIAnswerGenerator(api_key="<your-key>")
docs = [Document(content="Puppy A is happy"), Document(content="Puppy B is sad")]
res = node.predict(query="Which puppy is happy?", documents=docs, top_k=2)
print(res)

Limitations:
OpenAI's API doesn't return scores for the answers

Open tasks:

  • implement predict_batch() (implemented in BaseGenerator)
  • add a test similar AzureConverter that only runs when an api_key is set in the CI
  • basic error handling

Status (please check what you already did):

closes: #2344

@tholor tholor changed the title first draft of openai node for QA Add node to use OpenAI's GPT-3 for QA May 28, 2022
@ZanSara ZanSara added type:feature New feature or request topic:modeling labels Jun 8, 2022
@ZanSara
Copy link
Contributor

ZanSara commented Jun 15, 2022

Hey @tholor this would be super-cool to have in master! Shall we take over or you think you'll be able to keep working in this?

I'll merge master in here in the meantime to get the new CI working here too.

@tholor
Copy link
Member Author

tholor commented Jun 15, 2022

@ZanSara Would be amazing if you could take over!

@julian-risch julian-risch mentioned this pull request Jun 22, 2022
Closed
@bogdankostic bogdankostic marked this pull request as ready for review July 7, 2022 07:19
@bogdankostic bogdankostic requested a review from julian-risch July 7, 2022 07:19
@bogdankostic
Copy link
Contributor

As the answers endpoint is deprecated, I adapted this node to make use of the completions endpoint instead.

julian-risch
julian-risch requested changes Jul 8, 2022
View reviewed changes
Copy link
Member

@julian-risch julian-risch left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks very good to me already. 👍 I would recommend two small changes in docstrings: 1) explicitly mention that an API key is needed and explain how to get one and 2) small typo (see my comment below).

haystack/nodes/answer_generator/openai.py Outdated

def _build_prompt(self, query: str, documents: List[Document]) -> Tuple[str, List[Document]]:
"""
Builds the prompt for the GPT-3 model in order for ir to generate an answer.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo "ir" -> "it"

haystack/nodes/answer_generator/openai.py Outdated

class OpenAIAnswerGenerator(BaseGenerator):
"""
Uses the GPT-3 models from the OpenAI API to generate answers based on supplied documents (e.g. from any retriever in Haystack).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docstring should mention that there is an API key needed and should also link to a web page with a description how to get an API key.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added this to the doc strings.

@bogdankostic bogdankostic requested a review from julian-risch July 8, 2022 09:24
julian-risch
julian-risch approved these changes Jul 8, 2022
View reviewed changes
Copy link
Member

@julian-risch julian-risch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 👍

@bogdankostic bogdankostic merged commit ba08fc8 into master Jul 8, 2022
@bogdankostic bogdankostic deleted the openai branch July 8, 2022 11:59
@ZanSara ZanSara mentioned this pull request Jul 11, 2022
Closed
Krak91 pushed a commit to Krak91/haystack that referenced this pull request Jul 26, 2022
@tholor @github-actions
@ZanSara @bogdankostic
Add node to use OpenAI's GPT-3 for QA (deepset-ai#2605)
3d593a5 
* first draft of openai node for QA

* Update Documentation & Code Style

* fix mypy. add node to inits

* Update Documentation & Code Style

* fix linter

* Adapt OpenAIGenerator to completions endpoint

* Update Documentation & Code Style

* Fix pylint

* Fix doc strings

* Make use of temperature

* Make use of api key in tests

* Adapt doc strings

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: ZanSara <[email protected]>
Co-authored-by: bogdankostic <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bogdankostic bogdankostic bogdankostic left review comments

@julian-risch julian-risch julian-risch approved these changes

Assignees
No one assigned
Labels
topic:modeling type:feature New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Feature request: Use gpt-3 (or other services) as Reader
4 participants
@tholor @ZanSara @bogdankostic @julian-risch