In the latest Claude 3.5 sonnet AI model update, Anthropic infused a new feature that can control a computer by looking at a screen, moving the cursor, clicking buttons, and typing text.
This is a new feature called “Computer Use” and is available today on the API, allowing developers to direct Claude to work on a computer like we or humans do.
You can check out the embedded video from the company below.
The Copilot Vision of Microsoft and OpenAI’s desktop app for ChatGPT have shown what their AI tools can do based on what they see on your computer screen. Google also has the same capabilities in its Gemini app on Android devices. However, these companies have not released these tools widely that allow them to click around and perform tasks for users. Even Rabbit R1 promised such capabilities.
They are saying that this feature is still “cumbersome and error-prone.” “We’re releasing computer use early for feedback from developers, and expect the capability to improve rapidly over time.”
Here is what developers think about this.
“There are many actions that people routinely do with computers (dragging, zooming, and so on) that Claude can’t yet attempt. The “flipbook” nature of Claude’s view of the screen—taking screenshots and piecing them together, rather than observing a more granular video stream—means that it can miss short-lived actions or notifications.”
Also, this version of Claude has been ordered to keep away from social media. There are “measures to monitor when Claude is asked to engage in election-related activity, as well as systems for nudging Claude away from activities like generating and posting content on social media, registering web domains, or interacting with government websites.”
Anthropic also says that the new Claude 3.5 sonnet model has improved in many benchmarks and is offered to customers at the same price.
Thanks for choosing to leave a comment. Please keep in mind that all comments are moderated according to our comment Policy.