Based between San Francisco and London, Cosine, a human reasoning lab building artificial general developers, raised a €2.2 million round led by US venture firms Uphonest and SOMA Capital, with participation from Lakestar, Focal, amongst others.
The company also announced it has achieved a 30% score on SWE-Bench, the industry standard for evaluating software engineering skills in AI models. This represents a 56% improvement over the previous best score, held by Factory at 19% and 2196% improvement over OpenAI’s GPT4 score of 1.31%. The benchmark, which includes real-world human tasks in software architecture, debugging and the implementation of new features in existing codebases, assesses an AI model’s ability to understand, modify, and generate complex code. According to Cosine, this marks the highest score achieved by any company to date.
“Our breakthrough in codifying human reasoning is allowing us to train AI models to operate far beyond the narrow range of tasks and tightly restricted prompts currently available to teams developing software,” said Cosine CEO, Alistair Pullen, who published and monetised his first software application aged 9.
According to the company, Cosine’s Artificial Developer, Genie, works like a very good human developer. For instance, it is able to solve bugs, build features, refactor code, and everything in between either fully autonomously or collaboratively with other developers. By fine-tuning models to emulate human reasoning, Cosine’s approach has beaten out rivals like AWS’s Amazon Q Developer and Cognition’s Devin, both of which scored under 20% on the same benchmark, with Cognition recently valued at $2bn after raising from Peter Thiel’s Founders Fund.
Founded in 2022, Cosine’s software was created out of the founder’s realization of the potential in using LLMs to perform complex tasks in the coding space by imitating human software developers’ behaviors. It is uncannily ‘human’ in its approach to reasoning as a result, with the founders’ primary goal to create truly resilient AI capable of tackling open-ended problems across various domains.
“We are focused on creating a colleague, not a co-pilot,” commented Sam Stenner, CIO at Cosine. “After we figured out how to generate data sets that codify human reasoning which can then be used to train LLMs, we knew the potential for what we had built and worked with OpenAI to fine tune their largest context window LLMs. We’re confident we now have the capabilities to consistently beat our own top score”.
“Cosine is not just improving AI; they’re fundamentally teaching AI to reason, providing companies with a true AI colleague,” added Ellen Ma, Partner at Uphonest Capital.
