# SWE-bench

- **Website:** <https://swebench.com>
- **Primary alias:** `swebench.com`

## Description

SWE-bench is a benchmark project for evaluating AI agents on software engineering tasks. Its website publishes leaderboards that compare systems by the percentage of benchmark instances resolved, with additional details such as model, agent, evaluation date, release version, and, where available, average trajectory cost. The benchmark family includes SWE-bench Verified, a human-filtered set of 500 instances, as well as Multilingual, Multimodal, Lite, and Full versions. The Verified leaderboard also offers a bash-only comparison setting, including results run with mini-SWE-agent.

Alongside the benchmarks, SWE-bench provides a collection of related tools and resources. These include mini-SWE-agent, SWE-agent (legacy), SWE-bench CLI, SWE-ReX, and SWE-smith, as well as the CodeClash, mini-SWE-agent, ProgramBench, and SWE-agent entries listed in its project family. Visitors can explore documentation, research papers, blog posts, citations, and press information, or find guidance on submitting results. Together, these offerings help users examine and compare the performance of AI systems on software development problems.

## Industries

- 🖥 Computers Electronics and Technology
- 🖥 Artificial Intelligence and Machine Learning _(under Computers Electronics and Technology)_

## Links

- [twitter](https://twitter.com/SWEbench)
- [github](https://github.com/swe-bench)

## Logos & icons

- logo _(primary)_ — SVG, PNG — [download](https://cdn.brandfetch.io/idJvwe65dN/theme/light/logo.svg?c=1bxid64Mup7aczewSAYMX&t=1791093706489)
- icon _(primary)_ — PNG — [download](https://cdn.brandfetch.io/idJvwe65dN/w/200/h/200/theme/dark/icon.png?c=1bxid64Mup7aczewSAYMX&t=1791093706556)

## Colors

| Name | Hex | Theme |
| --- | --- | --- |
| Contessa | #c9787a | accent |
| Ebony | #111827 | dark |
| Tonys Pink | #ea9f8f | light |

## Fonts

- DIN — asset _(custom)_

---

_Source: <https://brandfetch.com/swebench.com>_
_See [/llms.txt](/llms.txt) for a full list of agent-readable pages, and [/auth.md](/auth.md) for how agents authenticate._