In partnership with
Reading time: 6 minutes
🗞️ In this edition
🪤 The Test Claude Cannot Pass
Sponsored
DeepSeek's Agent Does Not Wait To Be Asked
Abu Dhabi Is Rewriting Its Rules For Machines
🔌 MCP CORNER Three servers worth wiring up this week.
⚡ YOUR MOVE For people building the life they want, now.
📢 What else is happening
A new test handed the best models real work and nobody got close. DeepSeek's new agent does not wait to be asked. And Abu Dhabi has signed a stack of deals with OpenAI.
Keep reading.
Source: ShiftDelete.net
They gave the models real work and most of a working day to finish it.
WHAT HAPPENED
The research firm SemiAnalysis has built a new test for AI agents and run the leading models through it. Instead of single questions, each task drops a model into a realistic working setup, like a finance job seeded from a real deal room, and leaves it most of a day to finish. The best model managed 12%. The firm says safety refusals pulled down the scores of two of the strongest systems, because they declined parts of the security and biology work.
WHY IT MATTERS
Tests that every model already aces tell a buyer nothing, which is how every lab gets to claim the lead at once. A test built out of real working setups, with a ceiling this low, is a far more honest measure of how much of a job an agent can actually take off your hands. It also says the distance between a good demo and a week of real work is still most of the distance.
OUR TAKE
The obvious read is that agents are overhyped and the low scores prove it. We think the more useful finding is in the behaviour, and the giveaway is what the agents did when they could not finish. In the security tasks almost all of them broke a rule they had been given. In the medical ones, some signed a clinical note in a doctor's name. Weak capability is a timing problem. Rule breaking under pressure is a design problem, and it does not go away when the scores go up.
Watch whether the big labs post their own numbers on this test. If two of them do inside a month, they have decided that a hard test they do badly on still beats no test at all. If they all stay quiet, this stays a critic's tool and the marketing keeps coming from tests the models have already beaten.
From Our Sponsor:
Store It. Search It. Analyze It. One Postgres
Your AI app generates three kinds of data at once: events, vector embeddings, and the analytics you run on top of them. The default move is three systems, three sets of tooling, and pipelines to keep them in sync.
TimescaleDB handles all three in the Postgres you already run.
Hypertables ingest events at scale. pgvector and pgvectorscale store and search embeddings. Continuous aggregates keep analytics live without re-querying everything.
Same SQL, same tools, one system to operate. No pipelines to sync, no drift, no second database to feed.
Point it at your heaviest workload and watch it stay fast as data grows. It's still Postgres. Start on Tiger Cloud and get $1000 in credits.
Source: Digital Trends
It reads the group chat, then goes looking for the fix.
DeepSeek has published a prototype coding tool that watches a development team's group chats, picks out the problems being discussed, and proposes fixes without anyone asking it to. The code is on GitHub, and the team behind it describes the aim as the most value for the least human attention. The company has not said how it holds up on complicated work.
When you do click a story in here, what makes you?
Source: The National
The first pilot is already running, on food imports.
Abu Dhabi's government and OpenAI have signed a set of agreements built around turning the emirate's written policies and regulations into a form a machine can read, so officials can hand routine compliance checks to AI and trace every answer back to the rule it came from. A five year agreement covers adoption across government, business and education. The processing runs on hardware inside the country.
🔌 MCP CORNER
THREE SERVERS WORTH WIRING UP THIS WEEK.
An MCP server plugs your assistant into another app, so it stops describing the work and starts doing it.
Notion MCP: Opens your own workspace to your assistant, so the page gets written instead of planned.
Grafana MCP: Hands your dashboards and alerts to your assistant, so you ask what broke instead of scrolling for it.
Docker Hub MCP: Lets your assistant check the image and its tags, so the build stops failing on a guess.
⚡️ YOUR MOVE
FOR PEOPLE BUILDING THE LIFE THEY WANT, NOW.
🖱️ Vercel has released a free tool that lets a coding agent drive the actual apps on your machine, on Mac, Windows and Linux. It skips screenshots entirely and reads each app the way a screen reader does, so the agent acts on a named button instead of guessing at pixels, and every agent in a session gets its own coloured cursor so you can see who did what. If your agent stops at the edge of the terminal, this is the afternoon that changes. → Wire it up
📊 OpenAI is merging the ChatGPT and Codex allowances on Plus, Pro and Business into one shared pool from December, and has not yet published the size of that pool, the reset cycle, or what each feature draws from it. Chats in the fast mode sit outside it. If your team runs a coding agent all day on a personal plan, go and look at the usage page now, while you still have two separate numbers to compare. → Read your usage page
🧨 A flaw rated at the maximum severity has been disclosed in Prefab UI, a library whose generative interface feature runs untrusted Python in what was meant to be a sealed sandbox. The sandbox kept a bridge to the host open, so code sent by a caller could read and write files on the server as the service's own user. Versions from 0.14 are affected and 0.21 closes it. If this is anywhere in your stack, upgrade before you read the rest of this. → Upgrade to 0.21
The Robots Need Someone To Film People First – Beijing company GenRobot AI has disclosed a $100 million round for the unglamorous half of robotics, building the rigs and software that record a person's hand and body movements doing household, factory and warehouse jobs, then clean and label those recordings so robot developers can train on them, with SAIC Motor, Baidu and Galaxy General Robotics among the investors.
Chatbots Got The Ballot Deadline Wrong – An evaluation startup found factual errors in almost 40% of the answers eight leading chatbots gave to election questions days before the US midterms, with voting logistics going wrong more often than questions about candidates, one assistant naming the wrong mail ballot deadline for Pennsylvania, and several leaning on sources as weak as forum threads.
China Opened A Studio Built Around Rewrites – The streaming company iQIYI has opened what it calls the country's first AI film studio in Shanghai, where live action footage syncs straight into its own production platform so the environment around an actor can be changed after the shoot while the original performance stays, with a police feature called Cop in Trouble already filming.
Taskmole: Turns the thing you keep meaning to do into tasks that actually get assigned and closed.
Ideogram: Makes images with the words spelled right, so the poster does not need a designer after all.
Ermine.ai: Transcribes your recording in the browser without the file ever leaving your machine.
Gamma: Turns your rough notes into a deck you are not embarrassed to present.
Three stories about work being handed over before anyone settled what finished looks like. A new test says the best agent gets through about an eighth of a real job, and breaks the rules it was given while trying. DeepSeek's prototype does not even wait for the job to be handed to it. And Abu Dhabi is rewriting its own rulebook into something a machine can read, which is the only one of the three that starts with the checking rather than the doing.
Say what finished looks like before you delegate.
Interested in featuring your services with us? Email us at [email protected]
