March 17th, 2024: US-based startup Cognition introduced Devin, an AI-powered tool the company claims is the “world’s first fully autonomous AI software engineer.”
Devin is designed to solve engineering tasks independently using its own shell, code editor, and web browser.
According to demonstrations provided by Cognition, Devin can utilize its web browser to access and learn from API documentation, enabling it to plug into various APIs.
When the AI agent encounters an error, it automatically adds a debugging print statement to the main code within its code editor interface and reruns the code.
Cognition has showcased Devin’s capabilities in building and deploying apps, identifying and fixing bugs in codebases, and even fine-tuning AI models.
To assess Devin’s accuracy, Cognition tested the AI agent on SWE-bench, a benchmarking platform that challenges agents to resolve real-world issues found in open-source projects on GitHub.
Devin successfully resolved 13.86% of the issues end-to-end, surpassing the performance of GPT4 (1.74%) and the previous best score held by Anthropic’s Claude 2 (4.80%).
Notably, Devin achieved this without assistance in locating the relevant files within the repository.
While Microsoft offers AI-powered developer tools like GitHub Copilot, which provides code completion and assistive features for programmers, it cannot complete codes end-to-end without human interference or assistance.
In contrast, Devin is capable of autonomously completing coding tasks.
Today we’re excited to introduce Devin, the first AI software engineer.
— Cognition (@cognition_labs) March 12, 2024
Devin is the new state-of-the-art on the SWE-Bench coding benchmark, has successfully passed practical engineering interviews from leading AI companies, and has even completed real jobs on Upwork.
Devin is… pic.twitter.com/ladBicxEat
Cognition is currently offering early access to Devin for businesses who wish to utilize the AI agent for engineering work. Interested customers can request early access through the company’s website.
With its impressive performance on the SWE-bench platform and its ability to operate independently, Devin represents a significant step forward in the development of AI-powered software engineering solutions.
