From 9264a4b36e5cc1aa3e7dcc8fddbc7ee3f6e82a5d Mon Sep 17 00:00:00 2001 From: Minyang Tian <69544994+mtian8@users.noreply.github.com> Date: Mon, 4 Nov 2024 00:59:43 -0600 Subject: [PATCH] Update leaderboard.md --- docs/leaderboard.md | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git a/docs/leaderboard.md b/docs/leaderboard.md index ea06bf1..342ed7f 100644 --- a/docs/leaderboard.md +++ b/docs/leaderboard.md @@ -2,7 +2,24 @@
-Under construction +# SciCode Leaderboard + +| Model | Main Problem Resolve Rate | +|------------------------|---------------------------| +| OpenAI o1-preview | 7.7% | +| Claude3.5-Sonnet | 4.6% | +| Deepseek-Coder-v2 | 3.1% | +| GPT-4o | 1.5% | +| GPT-4-Turbo | 1.5% | +| OpenAI o1-mini | 1.5% | +| Gemini 1.5 Pro | 1.5% | +| Claude3-Opus | 1.5% | +| Claude3-Sonnet | 1.5% | +| Qwen2-72B-Instruct | 1.5% | +| Llama-3.1-405B-Instruct| 0% | +| Llama-3.1-70B-Instruct | 0% | +| Mixtral-8x22B-Instruct | 0% | +| Llama-3-70B-Chat | 0% | @@ -11,4 +28,4 @@ Under construction
!!! tip "How to submit" - Want to submit your own model? Head over to the [documentation](docs/index.md). \ No newline at end of file + Want to submit your own model? Head over to the [documentation](docs/index.md).