👑New AI Software Engineer King

Cosine’s AI software engineer Genie bests Cognition’s Devin

Key Points 

• Genie is trained on data that mirrors human engineers' thought processes and workflows, enabling it to solve complex problems with high precision and efficiency.

• It set a new benchmark on SWE-Bench with a score of 30.08%, which is 57% better than the previous top scores from Amazon’s Q and Code Factory.

👨🏻‍💻News - Just a few months after Cognition launched Devin, Cosine, the human reasoning lab, has unveiled Genie, which they claim is the most advanced AI software engineering model out there.

Genie is designed to mimic how human engineers think and work, so it can tackle complex problems with great precision and efficiency. Alistair Pullen, the founder of Cosine, explained that to get a model to act like a software engineer, it has to be trained on how real engineers operate.

In line with this, Genie’s training included data that reflects the thought processes and workflows of human engineers. This sets it apart from other AI tools that just add features to existing models. 

🤖Performance evaluation - During its development, Genie was tested with SWE-Bench and HumanEval to see how well it could solve software engineering problems and retrieve the right code. It set a new benchmark on SWE-Bench with a score of 30.08%, which is 57% better than the previous top scores from Amazon’s Q and Code Factory. It also achieved a 64.27% success rate in retrieving necessary code lines, identifying 91,475 out of 142,338 required lines. While this is a big leap forward, Cosine admits there's still room for improvement.

🤓Here's what you can use it for - You can use Genie for a variety of coding tasks, such as fixing bugs, building features, refactoring code, and running thorough tests. It can work either fully autonomously or in collaboration with users.

It’s also worth noting that Genie supports a range of programming languages, including JavaScript, Python, TypeScript, Java, C#, C++, Rust, Scala, Kotlin, Swift, Golang, PHP, and Ruby.

Report: What 1,000 US consumers think about AI in customer service

Key Points 

• Only 32% of customers have successfully resolved service issues with AI, and 63% are frustrated with AI self-service, preferring human interaction.

• Despite 62% of consumers expecting AI to dominate customer service in the future, only 27% currently believe AI can match the quality of live agents, highlighting a significant confidence gap.

☕News - In Forbes' latest research on customer service and CX, over 1,000 US consumers were surveyed about their experiences with AI and digital support. Here are the main findings for 2024:'

• 62% of consumers expect AI and related technologies to become the primary mode of customer service in the future. However, opinions on AI’s current readiness for customer service vary.

• 38% believe AI will lead to more personalized customer experiences, leveraging customer data to enhance interactions.

• 49% think AI technologies could improve the overall customer experience, though challenges remain in achieving full customer adoption.

🙃On the flip side

• Only 32% of customers have successfully resolved a service issue using AI or ChatGPT technologies, indicating limited effectiveness or recognition of AI’s role.

• 56% of customers are apprehensive about AI technologies, influenced by negative media portrayals or past bad experiences.

• 63% of customers are frustrated with self-service options using AI and similar technologies, preferring human interaction for support and service.

📒In conclusion - When considering these findings as a whole, it’s clear that inconsistency is the main reason AI has not yet become a reliable customer support option. Customers’ varied experiences with AI across different companies have contributed to their fear and frustration. As of early this year, only 27% of customers believe AI-powered self-service can deliver the same quality of experience as a live agent. This number may improve over time, but the progress will be gradual.