feat(settings): remove Config Manager and optimize initialization (#83)

- Implemented lazy loading.
- Introduce lazy loading to optimize performance by delaying resource initialization until needed.
- Remove `config_manager` and config file saving functionality.
- Update default model names and remove context window handling.
- Introduce `llama_index.llms.openai` for LLM communication to enhance integration and compatibility with updated LLM protocols.
- Add `ChatPromptTemplate` to manage prompts more efficiently and improve prompt consistency.
- Streamline implementation within `ChatEngine` for improved readability and maintainability.
- Fix issue where logger failed to capture full error stack trace in certain cases.
- Remove `tiktoken` dependency, directly retrieve token usage from API response.
- Add import sorting for improved readability and consistency.
- Remove deprecated test cases.
- Removed deprecated `config.toml.template` file
- Removed `sync_func` to prevent TypeError and ensure file saving.
- Removed model name restriction.
- Updated README to mention GitHub Actions support.
- Migrated to llamaindex for enhanced functionality.
- Integrated `llama_index.vector_stores.chroma` and `llama_index.embeddings.openai` for improved vector storage and embeddings.
- Added semantic chunking for better document segmentation.
- Introduced `ChatPromptTemplate` and `PromptTemplate` for customizable chat prompts.
- Added `initialize_with_params` method for dynamic initialization
- Changed `print_hierarchy` from a subcommand to an option in the `run` command.
- Defined `chat-with-repo` as an optional dependency in `pyproject.toml`.

.gitignore

@@ -10,7 +10,6 @@ __pycache__/
 *$py.class
 chroma_db/
 .vscode
-.project_doc_record
 # C extensions
 *.so
 
@@ -177,9 +176,11 @@ prompt_output/
 .vscode/
 
 # RepoAgent
-.project_doc_record
 log.txt
 .chroma_db
 config.yml
 config.toml
-.pre-commit-config.yaml
+.project_doc_record
+
+# Gradio
+.gradio

README.md

@@ -55,6 +55,10 @@ Traditionally, creating and maintaining software documentation demanded signific
 
 ### Installation Method
 
+#### Using GitHub Actions
+
+This repository supports GitHub Actions for automating workflows such as building, testing, and deploying. For detailed instructions on setting up and using GitHub Actions with this repository, please refer to the [actions/run-repoagent](https://github.com/Umpire2018/setup-repoagent/actions).
+
 #### Using pip (Recommended for Users)
 
 Install the `repoagent` package directly using pip:
@@ -108,28 +112,12 @@ set OPENAI_API_KEY=YOUR_API_KEY # on Windows
 $Env:OPENAI_API_KEY = "YOUR_API_KEY" # on Windows (PowerShell)
 ```
 
-Use `repoagent configure` if you need to modify the running parameters.
-
-```sh
-Enter the path to target repository: 
-Enter the project hierarchy file name [.project_doc_record]: 
-Enter the Markdown documents folder name [markdown_docs]: 
-Enter files or directories to ignore, separated by commas []: 
-Enter the language (ISO 639 code or language name, e.g., 'en', 'eng', 'English') [Chinese]: 
-Enter the maximum number of threads [4]: 
-Enter the maximum number of document tokens [1024]: 
-Enter the log level (DEBUG, INFO, WARNING, ERROR, CRITICAL) [INFO]: 
-Enter the model [gpt-3.5-turbo]: 
-Enter the temperature [0.2]: 
-Enter the request timeout (seconds) [60.0]: 
-Enter the base URL [https://api.openai.com/v1]: 
-```
-
 ## Run RepoAgent
 
 Enter the root directory of RepoAgent and try the following command in the terminal:
 ```sh
 repoagent run #this command will generate doc, or update docs(pre-commit-hook will automatically call this)
+repoagent run --print-hierarchy # Print how repo-agent parse the target repo
 ```
 
 The run command supports the following optional flags (if set, will override config defaults):
@@ -145,12 +133,10 @@ The run command supports the following optional flags (if set, will override con
 - `-l`, `--language` TEXT: The ISO 639 code or language name for the documentation. Default: `Chinese`
 - `-ll`, `--log-level` [DEBUG|INFO|WARNING|ERROR|CRITICAL]: Sets the logging level for the application. Default: `INFO`
 
-
 You can also try the following feature
 
 ```sh
 repoagent clean # Remove repoagent-related cache
-repoagent print-hierarchy # Print how repo-agent parse the target repo
 repoagent diff # Check what docs will be updated/generated based on current code change
 ```
 
@@ -217,14 +203,15 @@ We conceptualize **Chat With Repo** as a unified gateway for these downstream ap
 Here we demonstrate a preliminary prototype of one of our downstream tasks: Automatic Q&A for Issues and Code Explanation. You can start the server by running the following code.
 
 ```sh
+pip install repoagent[chat-with-repo]
 repoagent chat-with-repo
 ```
 
 ## ✅ Future Work
 
 - [ ] Generate README.md automatically combining with the global documentation
 - [ ] **Multi-programming-language support** Support more programming languages like Java, C or C++, etc.
-- [ ] Local model support like Llama, chatGLM, Qwen, GLM4, etc.
+- [x] Local model support like Llama, chatGLM, Qwen, GLM4, etc.
 
 ## 🥰 Featured Cases
 

README_CN.md

@@ -102,28 +102,12 @@ set OPENAI_API_KEY=YOUR_API_KEY # on Windows
 $Env:OPENAI_API_KEY = "YOUR_API_KEY" # on Windows (PowerShell)
 ```
 
-如果需要修改运行参数，使用 `repoagent configure` 
-
-```sh
-Enter the path to target repository: 
-Enter the project hierarchy file name [.project_doc_record]: 
-Enter the Markdown documents folder name [markdown_docs]: 
-Enter files or directories to ignore, separated by commas []: 
-Enter the language (ISO 639 code or language name, e.g., 'en', 'eng', 'English') [Chinese]: 
-Enter the maximum number of threads [4]: 
-Enter the maximum number of document tokens [1024]: 
-Enter the log level (DEBUG, INFO, WARNING, ERROR, CRITICAL) [INFO]: 
-Enter the model [gpt-3.5-turbo]: 
-Enter the temperature [0.2]: 
-Enter the request timeout (seconds) [60.0]: 
-Enter the base URL [https://api.openai.com/v1]: 
-```
-
 ## 运行RepoAgent
 
 进入RepoAgent根目录并在终端尝试以下命令：
 ```sh
 repoagent run # 这条命令会生成文档或自动更新文档 (pre-commit-hook 会自动调用它)
+repoagent --print-hierarchy # 此命令将打印repoagent解析出的目标仓库
 ```
 
 run 命令支持以下可选标志（如果设置，将覆盖配置默认值）：
@@ -143,7 +127,6 @@ run 命令支持以下可选标志（如果设置，将覆盖配置默认值）
 
 ```sh
 repoagent clean # 此命令将删除与repoagent相关的缓存
-repoagent print-hierarchy # 此命令将打印repoagent解析出的目标仓库
 repoagent diff # 此命令将检查基于当前代码更改将更新/生成哪些文档
 ```
 
@@ -206,7 +189,8 @@ RepoAgent hook会在git commit时自动触发，检测前一步您git add的文
 在这里，我们展示了我们的下游任务之一的初步原型：自动issue问题解答和代码解释。您可以通过在终端运行以下代码启动服务。
 
 ```sh
-repoagent chat_with_repo
+pip install repoagent[chat-with-repo]
+repoagent chat-with-repo
 ```
 
 # ✅ 未来工作
@@ -237,4 +221,3 @@ repoagent chat_with_repo
       primaryClass={cs.CL}
 }
 ```
-

display/book_tools/generate_repoagent_books.py

@@ -1,6 +1,5 @@
 import os
 import shutil
-
 import sys
 
 

markdown_docs/display/book_tools/generate_repoagent_books.md

@@ -1,39 +1,33 @@
 ## FunctionDef main
-**main**: The function of main is to copy a folder of markdown documents into a specific book structure, creating the necessary directories and a README.md file if it does not exist.
+**main**: The function of main is to create a directory structure for a book and copy Markdown documentation files into it.
 
-**Parameters**: This function does not accept parameters directly through its definition. Instead, it retrieves parameters from the command line arguments:
-- The first argument (`sys.argv[1]`) specifies the folder containing markdown documents.
-- The second argument (`sys.argv[2]`) specifies the name of the book.
-- The third argument (`sys.argv[3]`) specifies the repository path where the markdown documents are located.
+**parameters**: The parameters of this Function.
+· parameter1: markdown_docs_folder - The name of the folder containing Markdown documentation files to be copied.
+· parameter2: book_name - The name of the book for which the directory structure is created.
+· parameter3: repo_path - The path to the repository where the Markdown documentation folder is located.
 
-**Code Description**: The `main` function begins by retrieving the necessary paths from the command line arguments. It constructs the destination directory path by combining a base path (`./books`), the book name, and a subdirectory (`src`). It also constructs the path to the source directory containing the markdown documents.
+**Code Description**: The main function begins by retrieving command-line arguments that specify the folder containing Markdown documentation, the desired book name, and the repository path. It constructs the destination directory path where the book's source files will be stored, specifically under './books/{book_name}/src'. It also constructs the source directory path for the Markdown documentation files based on the provided repository path and the specified folder name.
 
-The function checks if the destination directory exists. If it does not, the directory is created, and a message indicating the creation of the directory is printed to the console.
+The function then checks if the destination directory exists. If it does not exist, it creates the directory and prints a confirmation message indicating that the directory has been created. 
 
-Next, the function iterates over all items in the source directory. For each item, it checks if it is a directory or a file. If the item is a directory, it uses `shutil.copytree` to recursively copy the entire directory structure and its contents to the destination. If the item is a file, it uses `shutil.copy2` to copy the file to the destination, preserving metadata. After each copy operation, a message is printed to the console indicating the action taken.
+Next, the function iterates over each item in the source directory. For each item, it constructs the full source and destination paths. If the item is a directory, it uses `shutil.copytree` to recursively copy the entire directory to the destination. If the item is a file, it uses `shutil.copy2` to copy the file to the destination. For each copy operation, a message is printed to confirm the action taken.
 
-Additionally, the function includes a nested function named `create_book_readme_if_not_exist`, which is responsible for checking if a README.md file exists in the destination directory. If the file does not exist, the function creates it and writes a header with the book name.
+Additionally, the function defines a nested function called `create_book_readme_if_not_exist`, which checks for the existence of a README.md file in the destination directory. If the README.md file does not exist, it creates one and writes the book name as the title in Markdown format.
 
-Finally, the `create_book_readme_if_not_exist` function is called with the destination directory as its argument to ensure that a README.md file is present in the book structure.
+Finally, the main function calls `create_book_readme_if_not_exist` to ensure that a README.md file is created for the book if it is not already present.
 
-**Note**: This function relies on external libraries (`os`, `sys`, `shutil`) for file system operations and command line argument processing. It is designed to be executed from the command line, and the user must provide the correct paths and book name as arguments. The function does not perform error handling for incorrect paths or permissions issues, which should be considered when integrating into larger applications or scripts.
+**Note**: It is important to ensure that the specified paths and folder names are valid and accessible. The function relies on the presence of the `shutil` and `os` modules, which must be imported for the code to execute successfully. Additionally, the function assumes that the command-line arguments are provided in the correct order and format.
 ### FunctionDef create_book_readme_if_not_exist(dire)
 **create_book_readme_if_not_exist**: The function of create_book_readme_if_not_exist is to create a README.md file in a specified directory if it does not already exist.
 
-**Parameters**:
-- **dire**: This parameter specifies the directory where the README.md file should be created.
+**parameters**: The parameters of this Function.
+· dire: A string representing the directory path where the README.md file should be created.
 
-**Code Description**:
-The `create_book_readme_if_not_exist` function is designed to ensure that a README.md file exists within a specified directory. It takes a single parameter, `dire`, which represents the directory path where the README.md file is to be checked for existence and potentially created.
+**Code Description**: The create_book_readme_if_not_exist function is designed to check for the existence of a README.md file in a specified directory. It takes one parameter, 'dire', which is the path to the directory where the README.md file is intended to be created. 
 
-The function begins by constructing the full path to the README.md file within the specified directory using the `os.path.join` method. This method combines the directory path provided in the `dire` parameter with the filename 'README.md' to create a complete path to where the README file should exist.
+The function first constructs the full path to the README.md file by joining the provided directory path with the filename 'README.md' using the os.path.join method. It then checks if the file already exists at that path using os.path.exists. If the file does not exist, the function proceeds to create it. 
 
-Next, the function checks if a file at the constructed path already exists using the `os.path.exists` method. If the file does not exist (`os.path.exists` returns False), the function proceeds to create the README.md file at the specified path.
+Within a context manager (using the 'with' statement), the function opens the README.md file in write mode ('w'). This ensures that if the file is created, it will be properly closed after writing. The function writes a header line to the file, formatted as '# {book_name}', where 'book_name' is expected to be a variable that holds the name of the book. However, it is important to note that 'book_name' must be defined in the scope where this function is called, as it is not passed as a parameter to the function itself.
 
-To create the README.md file, the function opens a new file at the constructed path in write mode (`'w'`). It then writes a markdown header to the file, which includes a placeholder for a book name. The placeholder is formatted as `# {}`, where `{}` is intended to be replaced with the actual name of the book. However, it's important to note that in the provided code snippet, the variable `book_name` is referenced but not defined within the function or passed as a parameter, which suggests that the code snippet may be incomplete or relies on external context for the `book_name` variable.
-
-**Note**:
-- The function assumes that the `dire` parameter provided is a valid directory path and does not perform any checks to verify the existence of the directory itself.
-- The function relies on an external variable `book_name` for the content of the README.md file, which is not defined within the function or passed as a parameter. Users of this function will need to ensure that `book_name` is defined in the appropriate context for the function to execute successfully.
-- This function only creates a README.md file if it does not already exist. If a README.md file is already present in the specified directory, the function will not modify or overwrite the existing file.
+**Note**: It is essential to ensure that the variable 'book_name' is defined before calling this function, as it is used in the content written to the README.md file. Additionally, the function does not handle exceptions that may arise from file operations, so it is advisable to implement error handling if necessary.
 ***