From 9264a4b36e5cc1aa3e7dcc8fddbc7ee3f6e82a5d Mon Sep 17 00:00:00 2001
From: Minyang Tian <69544994+mtian8@users.noreply.github.com>
Date: Mon, 4 Nov 2024 00:59:43 -0600
Subject: [PATCH] Update leaderboard.md
---
docs/leaderboard.md | 21 +++++++++++++++++++--
1 file changed, 19 insertions(+), 2 deletions(-)
diff --git a/docs/leaderboard.md b/docs/leaderboard.md
index ea06bf1..342ed7f 100644
--- a/docs/leaderboard.md
+++ b/docs/leaderboard.md
@@ -2,7 +2,24 @@
-Under construction
+# SciCode Leaderboard
+
+| Model | Main Problem Resolve Rate |
+|------------------------|---------------------------|
+| OpenAI o1-preview | 7.7% |
+| Claude3.5-Sonnet | 4.6% |
+| Deepseek-Coder-v2 | 3.1% |
+| GPT-4o | 1.5% |
+| GPT-4-Turbo | 1.5% |
+| OpenAI o1-mini | 1.5% |
+| Gemini 1.5 Pro | 1.5% |
+| Claude3-Opus | 1.5% |
+| Claude3-Sonnet | 1.5% |
+| Qwen2-72B-Instruct | 1.5% |
+| Llama-3.1-405B-Instruct| 0% |
+| Llama-3.1-70B-Instruct | 0% |
+| Mixtral-8x22B-Instruct | 0% |
+| Llama-3-70B-Chat | 0% |
@@ -11,4 +28,4 @@ Under construction
!!! tip "How to submit"
- Want to submit your own model? Head over to the [documentation](docs/index.md).
\ No newline at end of file
+ Want to submit your own model? Head over to the [documentation](docs/index.md).