Anthropic's upgraded Claude 3.5 Sonnet can now control desktop apps

Key Points 

• Claude 3.5 Sonnet can now control desktop applications using the new “Computer Use” API, performing human-like actions like keystrokes, clicks, and mouse gestures.

• Claude 3.5 Sonnet surpasses OpenAI’s o1 in coding tasks, and has the ability to self-correct and handle complex, multi-step objectives more effectively.

👨🏻‍💻News - Anthropic has launched an upgraded version of its Claude 3.5 Sonnet model that can interact with desktop applications. This new version, released on Tuesday, is powered by the “Computer Use” API, now in open beta. It can mimic human-like actions on a PC, including keystrokes, clicks, and mouse gestures, making it feel as if a person is physically using the computer. In a blog post, Anthropic explained that Claude can "see" the screen, analyze it, and perform actions based on screenshots of the user interface.

🤨Is it safe? While browsing the web and using apps isn’t new for AI models, it’s a first for Anthropic’s Claude. That being said, the model remains under human control at all times. Users provide specific prompts to direct Claude’s actions, such as “use data from my computer to fill out this form.” Humans enable and limit access as needed, ensuring the AI can only carry out tasks as instructed. 

🧐What makes it different? While desktop task automation isn't new, Claude 3.5 Sonnet introduces an “action-execution layer” that allows it to perform more complex commands. Anthropic claims this model outperforms OpenAI’s flagship model, o1, in coding tasks. It can also self-correct and work through long, multi-step objectives, even when it encounters obstacles.

🤔Are there any limitations? Despite its advancements, Claude 3.5 Sonnet still struggles with basic tasks like scrolling and zooming. It can also miss short-lived actions or notifications due to the way it processes and stitches together screenshots, which may cause it to overlook brief changes on the screen.

Canva launches new text-to-image generator “Dream Lab” and more AI features

Key Points 

• Canva’s new “Dream Lab” lets users generate images from descriptions in various styles, offering better photographic quality and multi-subject images compared to the platform’s Stable Diffusion-based generator.

• The platform’s updated AI features include improved accuracy and one-click autocomplete for the Magic Write text generation tool.

👾News - Canva has announced the addition of several new AI features to its design platform. Among these updates is “Dream Lab,” a new image generator that emerged from Canva’s acquisition of the generative AI startup Leonardo.ai.

Powered by Leonardo's Phoenix model, Dream Lab allows users to create images based on descriptive prompts in various styles, including “3D render” and “Illustration.” 

While Canva already has a Stable Diffusion-based image generator, Dream Lab offers notable improvements in producing photographic styles and multi-subject images. It also features a tool that references existing images, providing users with greater control over the output.

✨What’s more? In addition to Dream Lab, Canva has updated its broader “Magic” AI tool suite. These updates include enhanced accuracy and one-click autocomplete for the Magic Write text generation feature. 

Whiteboards now support AI-powered summarization and real-time reaction stickers to facilitate collaboration. Additionally, new video editing tools automatically generate captions and introduce innovative animation effects, further expanding the platform's versatility for creative projects.