Skip to content

Commit

Permalink
Typo fix (mea culpa)
Browse files Browse the repository at this point in the history
  • Loading branch information
klieret committed Sep 24, 2024
1 parent f03025a commit f336c56
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion index.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ Dolan-Gavitt[^2], Muhammad Shafique[^5], Karthik Narasimhan[^3], Ramesh Karri[^2
[^4]: *Stanford University*
[^5]: *New York University Abu Dhabi*

Although language model (LM) agents are demonstrating growing potential in many domains, their success in cybersecurity has been limited due to simplistic design and the lack of fundamental features for this domain. We present <span class="enigma">EnIGMA</span>, a LM agent for autonomously solving Capture The Flag (CTF) challenges. <span class="enigma">EnIGMA</span> introduces new *Agent-Computer Interfaces* (ACIs) to improve the success rate on CTF challenges. We establish the novel *Interactive Agent Tools* concept, which enables LM agents to run interactive command-line utilities essential for these challenges. Empirical analysis of <span class="enigma">EnIGMA</span> on over 350 CTF challenges from three different benchmarks indicates that providing a robust set of new tools with demonstration of their usage helps the LM solve complex problems and achieves state-of-the-art results on the [NYU CTF](https://arxiv.org/abs/2406.05590) and [Intercode-CTF](https://openreview.net/pdf?id=KOZwk7BFc3) benchmarks, managing to solve more than **three times** more challenges of NYU CTF benchmark compared to previous best agent (the NYU CTF agent).
Although language model (LM) agents are demonstrating growing potential in many domains, their success in cybersecurity has been limited due to simplistic design and the lack of fundamental features for this domain. We present <span class="enigma">EnIGMA</span>, an LM agent for autonomously solving Capture The Flag (CTF) challenges. <span class="enigma">EnIGMA</span> introduces new *Agent-Computer Interfaces* (ACIs) to improve the success rate on CTF challenges. We establish the novel *Interactive Agent Tools* concept, which enables LM agents to run interactive command-line utilities essential for these challenges. Empirical analysis of <span class="enigma">EnIGMA</span> on over 350 CTF challenges from three different benchmarks indicates that providing a robust set of new tools with demonstration of their usage helps the LM solve complex problems and achieves state-of-the-art results on the [NYU CTF](https://arxiv.org/abs/2406.05590) and [Intercode-CTF](https://openreview.net/pdf?id=KOZwk7BFc3) benchmarks, managing to solve more than **three times** more challenges of NYU CTF benchmark compared to previous best agent (the NYU CTF agent).


Want to try it yourself and explore our new agent? We are completely *open-source*!
Expand Down

0 comments on commit f336c56

Please sign in to comment.