-
Notifications
You must be signed in to change notification settings - Fork 82
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Moon bit test agent #1053
Open
KirbytroNic0528
wants to merge
101
commits into
moonbitlang:main
Choose a base branch
from
KirbytroNic0528:MoonBit-Test-Agent
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+1,320
−0
Open
Moon bit test agent #1053
Changes from 3 commits
Commits
Show all changes
101 commits
Select commit
Hold shift + click to select a range
7396fb3
workflow for MoonBit Test Agent
KirbytroNic0528 557f851
MoonBit Test Agent Code
KirbytroNic0528 7518a5c
Merge branch 'main' into MoonBit-Test-Agent
tonyfettes 7ddda72
Merge branch 'main' into MoonBit-Test-Agent
tonyfettes ce0bacc
Update gettest.py
KirbytroNic0528 25c1f82
Update readcoverage.py
KirbytroNic0528 8ac8123
Update requirement.txt
KirbytroNic0528 7ce6b71
Update testagent.py
KirbytroNic0528 3d4b2af
Update writedown.py
KirbytroNic0528 fb1510c
rename requirement.txt to requirements.txt
KirbytroNic0528 8fd3751
Update requirements.txt
KirbytroNic0528 2022af8
Update coverage_and_test.yml
KirbytroNic0528 2d4f6ef
Update requirements.txt
KirbytroNic0528 7f340c8
Update coverage_and_test.yml
KirbytroNic0528 d01f5d1
Update coverage_and_test.yml
KirbytroNic0528 6a92863
Update testagent.py
KirbytroNic0528 f768ad9
Update coverage_and_test.yml
KirbytroNic0528 88853da
Update coverage_and_test.yml
KirbytroNic0528 12505b9
Update coverage_and_test.yml
KirbytroNic0528 afdc122
Update coverage_and_test.yml
KirbytroNic0528 ab62eee
Merge branch 'main' into MoonBit-Test-Agent
tonyfettes 9bd8ccb
Merge branch 'main' into MoonBit-Test-Agent
tonyfettes 1393173
Add files via upload
KirbytroNic0528 274c190
Update coverage_and_test.yml
KirbytroNic0528 790d0fd
Update coverage_and_test.yml
KirbytroNic0528 3004f3e
Update coverage_and_test.yml
KirbytroNic0528 3bc771a
Update testagent.py
KirbytroNic0528 1d6060e
Update coverage_and_test.yml
KirbytroNic0528 739d460
Update coverage_and_test.yml
KirbytroNic0528 258c99a
Update coverage_and_test.yml
KirbytroNic0528 5005c2a
Update coverage_and_test.yml
KirbytroNic0528 59462c0
Update coverage_and_test.yml
KirbytroNic0528 63d8f00
Update coverage_and_test.yml
KirbytroNic0528 ed18621
Update coverage_and_test.yml
KirbytroNic0528 fd7d1f2
Update coverage_and_test.yml
KirbytroNic0528 7b9eed6
Update coverage_and_test.yml
KirbytroNic0528 fbcc04c
Update coverage_and_test.yml
KirbytroNic0528 259bc54
Update coverage_and_test.yml
KirbytroNic0528 36276b7
Update coverage_and_test.yml
KirbytroNic0528 7b1a02e
Update coverage_and_test.yml
KirbytroNic0528 68e3e81
Update coverage_and_test.yml
KirbytroNic0528 9f15f9c
Update coverage_and_test.yml
KirbytroNic0528 219e1c2
Update coverage_and_test.yml
KirbytroNic0528 a340c03
Update coverage_and_test.yml
KirbytroNic0528 8a3293e
Update readcoverage.py
KirbytroNic0528 a658717
Update coverage_and_test.yml
KirbytroNic0528 cf5e5b5
Update coverage_and_test.yml
KirbytroNic0528 bb12c48
Update testagent.py
KirbytroNic0528 4184704
Update coverage_and_test.yml
KirbytroNic0528 2bac8fd
Update coverage_and_test.yml
KirbytroNic0528 3a3be85
Update coverage_and_test.yml
KirbytroNic0528 8590315
Update coverage_and_test.yml
KirbytroNic0528 d31dd70
Update coverage_and_test.yml
KirbytroNic0528 b1777a1
Add files via upload
KirbytroNic0528 d1d1ca7
Update gettest.py
KirbytroNic0528 3ccd47c
Update readcoverage.py
KirbytroNic0528 921d1df
Update testagent.py
KirbytroNic0528 16221df
Update writedown.py
KirbytroNic0528 a806d21
Update coverage_and_test.yml
KirbytroNic0528 81e5026
Merge branch 'main' into MoonBit-Test-Agent
KirbytroNic0528 33bbce7
Update testagent.py
KirbytroNic0528 e132cbd
Update coverage_and_test.yml
KirbytroNic0528 a05e348
Update coverage_and_test.yml
KirbytroNic0528 68f2fde
Update testagent.py
KirbytroNic0528 fc4bfd7
Update requirements.txt
KirbytroNic0528 8ba79bb
Update coverage_and_test.yml
KirbytroNic0528 766f02b
Update coverage_and_test.yml
KirbytroNic0528 40cfc6e
Update requirements.txt
KirbytroNic0528 9b466c8
Update coverage_and_test.yml
KirbytroNic0528 d2aef31
Update zhipuai.py
KirbytroNic0528 e8e5da2
Update zhipuai.py
KirbytroNic0528 fc1a2e0
Update writedown.py
KirbytroNic0528 ff999d1
Update gettest.py
KirbytroNic0528 d1ae519
Update readcoverage.py
KirbytroNic0528 4d33546
Update gettest.py
KirbytroNic0528 695c11b
Update readcoverage.py
KirbytroNic0528 edd5bd6
Update readcoverage.py
KirbytroNic0528 bab0ab5
Update readcoverage.py
KirbytroNic0528 e0453d0
Update readcoverage.py
KirbytroNic0528 ac4c362
Update readcoverage.py
KirbytroNic0528 5a095ee
Update gettest.py
KirbytroNic0528 e51fd0c
Update testagent.py
KirbytroNic0528 fe1c886
Update coverage_and_test.yml
KirbytroNic0528 fbb94b0
Update testagent.py
KirbytroNic0528 112339e
Update testagent.py
KirbytroNic0528 0761a65
Add files via upload
KirbytroNic0528 808ef45
Update testagent.py
KirbytroNic0528 4346c59
Update writedown.py
KirbytroNic0528 ad5de51
Update gettest.py
KirbytroNic0528 c991054
Update and rename zhipuai.py to zhipuai_model.py
KirbytroNic0528 2febd8d
Update readcoverage.py
KirbytroNic0528 6ac2c08
Update coverage_and_test.yml
KirbytroNic0528 5267b75
Update main.py
KirbytroNic0528 1fd3b07
Update readme.md
KirbytroNic0528 44bb4ac
Update coverage_and_test.yml
KirbytroNic0528 ff0d308
Merge branch 'main' into MoonBit-Test-Agent
KirbytroNic0528 50bb8eb
Update coverage_and_test.yml
KirbytroNic0528 cd10baf
Merge branch 'main' into MoonBit-Test-Agent
tonyfettes f1d10b3
Update coverage_and_test.yml
KirbytroNic0528 d359c97
Update gettest.py
KirbytroNic0528 8de233b
Update readcoverage.py
KirbytroNic0528 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,142 @@ | ||
name: coverage_and_test | ||
|
||
on: | ||
push: | ||
branches: | ||
- main | ||
pull_request: | ||
|
||
jobs: | ||
build: | ||
strategy: | ||
matrix: | ||
os: [ ubuntu-latest ] | ||
runs-on: ubuntu-latest | ||
continue-on-error: true | ||
steps: | ||
- uses: actions/checkout@v4 | ||
|
||
- name: install | ||
run: | | ||
curl -fsSL https://cli.moonbitlang.com/install/unix.sh | bash | ||
echo "$HOME/.moon/bin" >> $GITHUB_PATH | ||
- name: moon version | ||
run: | | ||
moon version --all | ||
moonrun --version | ||
|
||
- name: moon check | ||
run: moon check --deny-warn | ||
|
||
- name: moon info | ||
run: | | ||
moon info | ||
git diff --exit-code | ||
|
||
- name: Set ulimit and run moon test | ||
run: | | ||
ulimit -s 8176 | ||
moon test --target all | ||
moon test --release --target all | ||
|
||
- name: moon bundle | ||
run: moon bundle --all | ||
|
||
- name: check core size | ||
run: find ./target -name '*.core' | xargs ls -lh | ||
|
||
- name: format diff | ||
run: | | ||
moon fmt | ||
git diff --exit-code | ||
|
||
- name: Set up Python | ||
uses: actions/checkout@v4 | ||
with: | ||
python-version: '3.9' | ||
|
||
- name: Install dependencies | ||
run: | | ||
sudo apt install python3-pip | ||
pip install -r scripts/requirements.txt | ||
|
||
test: | ||
runs-on: ubuntu-latest | ||
continue-on-error: true | ||
steps: | ||
- uses: actions/checkout@v4 | ||
|
||
- name: initial moon test | ||
run: moon test --enable-coverage | ||
|
||
- name: initial coverage report | ||
run: | | ||
moon coverage report -f summary summary > coverage_summary.txt | ||
cat coverage_summary.txt >> "$GITHUB_STEP_SUMMARY" | ||
|
||
|
||
- name: loop coverage improvement | ||
id: loop-coverage | ||
run: | | ||
prev_coverage=$(cat coverage_summary.txt | grep 'Total Coverage' | awk '{print $NF}' | tr -d '%') | ||
max_iterations=5 | ||
iteration=0 | ||
coverage_improved=true | ||
|
||
while [ "$coverage_improved" = true ] && [ $iteration -lt $max_iterations ]; do | ||
python Agent.py --api_key ${{ secrets.API_KEY }} | ||
|
||
moon test --enable-coverage | ||
|
||
moon coverage report -f summary summary > coverage_summary.txt | ||
|
||
new_coverage=$(cat coverage_summary.txt | grep 'Total Coverage' | awk '{print $NF}' | tr -d '%') | ||
|
||
if [ $(echo "$new_coverage > $prev_coverage" | bc) -eq 1 ]; then | ||
prev_coverage=$new_coverage | ||
echo "Coverage improved to $new_coverage%" | ||
else | ||
coverage_improved=false | ||
echo "Coverage did not improve. Stopping loop." | ||
fi | ||
iteration=$((iteration + 1)) | ||
done | ||
|
||
echo "Final coverage: $new_coverage%" | ||
echo "::set-output name=final_coverage::$new_coverage" | ||
|
||
- name: coverage report | ||
run: | | ||
cat coverage_summary.txt >> "$GITHUB_STEP_SUMMARY" | ||
moon coverage report -f coveralls -o codecov_report.json --service-name github --service-job-id "$GITHUB_RUN_NUMBER" --service-pull-request "${{ github.event.number }}" --send-to coveralls | ||
env: | ||
COVERALLS_REPO_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
|
||
typo-check: | ||
runs-on: ubuntu-latest | ||
timeout-minutes: 10 | ||
env: | ||
FORCE_COLOR: 1 | ||
TYPOS_VERSION: v1.19.0 | ||
steps: | ||
- name: download typos | ||
run: curl -LsSf https://github.com/crate-ci/typos/releases/download/$TYPOS_VERSION/typos-$TYPOS_VERSION-x86_64-unknown-linux-musl.tar.gz | tar zxf - -C ${CARGO_HOME:-~/.cargo}/bin | ||
|
||
- name: Checkout repository | ||
uses: actions/checkout@v4 | ||
with: | ||
ref: ${{ github.event.pull_request.head.sha }} | ||
|
||
- name: check typos | ||
run: typos | ||
|
||
license-header-check: | ||
runs-on: ubuntu-latest | ||
env: | ||
HAWKEYE_VERSION: v5.5.1 | ||
steps: | ||
- uses: actions/checkout@v4 | ||
- name: Download HawkEye | ||
run: curl --proto '=https' --tlsv1.2 -LsSf https://github.com/korandoru/hawkeye/releases/download/$HAWKEYE_VERSION/hawkeye-installer.sh | sh | ||
- name: Check License Header | ||
run: hawkeye check |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
from langchain_community.chat_models import ChatZhipuAI | ||
from langchain_core.output_parsers import StrOutputParser | ||
from langchain_core.prompts import ChatPromptTemplate | ||
|
||
def generate_test_code(moonbit_code,path,api_key): | ||
test_prompt = ChatPromptTemplate.from_template( | ||
"""作为一名MoonBit语言工程师,你的任务是编写一系列测试用例来验证项目的正确性。 | ||
请根据以下提供的格式,结合文件名{path}理解函数的用途,对给出的MoonBit函数编写对应的测试用例, | ||
以下是你需要提供的测试用例格式参考: | ||
test {{ | ||
assert_eq!(f(x)) | ||
assert_eq!(f(x)) | ||
}} | ||
需要提供测试用例的MoonBit函数为{moonbit_code}。 | ||
注意,你的输出中只需要包含测试用例的代码,不需要包括分析过程以及任何其他语句。 | ||
同时,你生成的是moonbit语言的测试用例,请不要将moonbit语言与其他语言混淆 | ||
""" | ||
) | ||
|
||
test_llm = ChatZhipuAI( | ||
api_key=api_key, | ||
model="glm-4-plus", | ||
temperature=0.5, | ||
max_tokens=2048 | ||
) | ||
|
||
test_retriever_chain = ( | ||
test_prompt | ||
| test_llm | ||
| StrOutputParser() | ||
) | ||
test_code = test_retriever_chain.invoke({ | ||
"moonbit_code": moonbit_code, | ||
"path":path | ||
}) | ||
return test_code |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
from langchain_community.chat_models import ChatZhipuAI | ||
from langchain_core.output_parsers import StrOutputParser | ||
from langchain_core.prompts import ChatPromptTemplate | ||
|
||
def read_coverage(moonbit_code, index, api_key): | ||
read_prompt = ChatPromptTemplate.from_template( | ||
"""你是一位技术高超的moonbit工程师,现在你需要读入moonbit语言的代码以及测试未覆盖代码行数的索引 | ||
根据索引找出测试未覆盖的代码并返回该行代码所在函数的函数 | ||
需要读入的MoonBit代码:{moonbit_code},未覆盖代码行数的索引:{index} | ||
注意,你的输出中只需要包含未覆盖的函数,不需要包含分析过程和任何其他语句。 | ||
""" | ||
) | ||
|
||
read_llm = ChatZhipuAI( | ||
api_key=api_key, | ||
model="glm-4-plus", | ||
temperature=0.5, | ||
max_tokens=2048 | ||
) | ||
|
||
read_retriever_chain = ( | ||
read_prompt | ||
| read_llm | ||
| StrOutputParser() | ||
) | ||
response = read_retriever_chain.invoke({ | ||
"moonbit_code": moonbit_code, | ||
"index": index | ||
}) | ||
return response | ||
|
||
|
||
|
||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
langchain | ||
argparse | ||
langchain_community | ||
glob | ||
pyjwt |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
import argparse | ||
import json | ||
from Agent.readcoverage import read_coverage | ||
from Agent.gettest import generate_test_code | ||
from Agent.writedown import writedown_test_files | ||
def main(): | ||
with open('coveralls.json', 'r') as file: | ||
data = json.load(file) | ||
|
||
parser = argparse.ArgumentParser(description="用于加载API密钥。") | ||
parser.add_argument('--api_key', default="4a478b99108ee30c1ae4aaa0aefe6632.X8sj7A6gaBgWh9AE", type=str, help='API密钥') | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is the default key a valid key? Do I need to add variable |
||
args = parser.parse_args() | ||
zhipuai_api_key = args.api_key | ||
|
||
for source_file in data['source_files']: | ||
with open(source_file['name'], "r") as codefile: | ||
index = [index for index, value in enumerate(source_file['coverage']) if value == 0] | ||
if index!=[]: | ||
print(index) | ||
moonbit_code = codefile.read() | ||
response = read_coverage(moonbit_code,index, zhipuai_api_key) | ||
print("未覆盖的函数声明为"+response) | ||
response = generate_test_code(response,source_file['name'],zhipuai_api_key) | ||
test_code = response.replace("```moonbit\n", "").rstrip("```") | ||
writedown_test_files(source_file['name'], test_code) | ||
print("测试用例为"+test_code) | ||
|
||
|
||
if __name__ == "__main__": | ||
main() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
import os | ||
|
||
def writedown_test_files(path, test_code): | ||
with open(path, 'a') as file: | ||
file.write(test_code + '\n') | ||
|
||
print(f"test_code has been written to {path}") |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
typo? There is a
script/requirement.txt
in your PR.