Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introducing RAG module into package #64

Merged
merged 27 commits into from
Mar 29, 2024
Merged

Conversation

ZiTao-Li
Copy link
Collaborator

@ZiTao-Li ZiTao-Li commented Mar 13, 2024


Introducing RAG module into AgentScope package

Description

Introducing retrieval augmented generation techniques into AgentScope package to enrich the ability of agents.

To accommodate the RAG process of different packages, we abstract the RAG process into four stages:

  • data loading: loading data into memory for following processing;
  • data indexing and storage: document chunking, embedding generation, and off-load the data into VDB;
  • data retrieval: taking a query and return a batch of documents or document chunks;
  • post-processing of the retrieved data: use the retrieved data to generate an answer.

Checklist

Please check the following items before code is ready to be reviewed.

  • Code has passed all tests
  • Docstrings have been added/updated in Google Style
  • Documentation has been updated
  • Code is ready for review

@ZiTao-Li ZiTao-Li requested a review from DavdGao March 13, 2024 06:41
Copy link
Collaborator

@xieyxclack xieyxclack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see the inline comments, thx

src/agentscope/rag/langchain_rag.py Outdated Show resolved Hide resolved
src/agentscope/rag/llama_index_rag.py Outdated Show resolved Hide resolved
examples/rag/rag_agents.py Outdated Show resolved Hide resolved
src/agentscope/rag/rag.py Outdated Show resolved Hide resolved
Copy link
Collaborator

@DavdGao DavdGao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see inline comments

examples/rag/rag_agents.py Outdated Show resolved Hide resolved
examples/rag/rag_agents.py Outdated Show resolved Hide resolved
examples/rag/rag_agents.py Outdated Show resolved Hide resolved
examples/rag/rag_agents.py Outdated Show resolved Hide resolved
examples/rag/rag_agents.py Outdated Show resolved Hide resolved
examples/rag/rag_agents.py Outdated Show resolved Hide resolved
examples/rag/rag_agents.py Outdated Show resolved Hide resolved
examples/rag/rag_agents.py Outdated Show resolved Hide resolved
examples/rag/rag_example.py Outdated Show resolved Hide resolved
src/agentscope/rag/langchain_rag.py Outdated Show resolved Hide resolved
Copy link
Collaborator

@DavdGao DavdGao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Plz see inline comments, and I think the RAG module is still not ready for using.

  1. For me, I cannot find how to set rag_config in docstring and example: what parameters should be included and their constraints, and what format should the parameters follow.
  2. RAG-related documents are suggested, maybe add a RAG section in tutorial.

src/agentscope/agents/__init__.py Outdated Show resolved Hide resolved
src/agentscope/agents/rag_agents.py Outdated Show resolved Hide resolved
src/agentscope/rag/rag.py Outdated Show resolved Hide resolved
src/agentscope/agents/rag_agents.py Outdated Show resolved Hide resolved
Copy link
Collaborator

@DavdGao DavdGao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@DavdGao DavdGao merged commit f762dd6 into modelscope:main Mar 29, 2024
4 checks passed
myh-0521 pushed a commit to myh-0521/agentscope that referenced this pull request Nov 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants