Skip to content

Commit

Permalink
Update index.md
Browse files Browse the repository at this point in the history
  • Loading branch information
mtian8 authored Jul 15, 2024
1 parent 868b441 commit 69dac59
Showing 1 changed file with 5 additions and 1 deletion.
6 changes: 5 additions & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,11 @@ In designing test cases for evaluation, we incorporate domain-specific test case
![Image Title](figures/SciCode_chart.png)
**Left:** Distribution of Main Problems **Right:** Distribution of Subproblems


## Experiment Results
We evaluate our model using zero-shot prompts. We keep the prompts general and design different ones for different evaluation setups only to inform the model about the tasks. We keep prompts the same across models and fields, and they contain the model’s main and sub-problem instructions and code for previous subproblems. The standard setup means the model is tested without background knowledge and carrying over generated solutions to previous subproblems. The scientists' annotated background provides the necessary knowledge and reasoning steps to solve the problems, shifting the evaluation’s focus more towards the models’ coding and instruction-following capabilities.
![Image Title](figures/Standard_Setup.png)
![Image Title](figures/Standard_Background.png)
![Image Title](figures/Performance_Gain.png)

### Numerical Linear Algebra
1_Conjugate_Gradient
Expand Down

0 comments on commit 69dac59

Please sign in to comment.