🗞️In this edition

• Mira Murati's lab challenges AI Orthodoxy

• Sponsored: Mindstream – Turn AI into income

• AI models developing their own survival drive

• The hidden dangers of AI browser agents

• Workflow Wednesday #41 ‘AI & Human Creativity’

• In other AI news –

  • OpenAI is reportedly working on a generative music tool that can create background tracks or instrumentals for videos and vocals

  • Google adds ‘Vibe Coding’ to its Gemini-powered AI Studio, turning app creation into a visual remix experience

  • Mercor  the startup training ChatGPT’s human experts  hits a $10 billion valuation

  • 4 must-try AI tools

Thinking Machines challenges industry Orthodoxy on path to AGI

What's happening:

Rafael Rafailov, a reinforcement learning researcher at Thinking Machines Lab, delivered a direct challenge to the AI industry's scaling strategy at TED AI San Francisco this week.

His core argument: the first superintelligence won't be the biggest model. It'll be the best learner.

"I believe that the first superintelligence will be a superhuman learner," Rafailov said. "It will be able to very efficiently figure out and adapt, propose its own theories, propose experiments, use the environment to verify that, get information, and iterate that process."

This breaks with the approach at OpenAI, Anthropic, and Google DeepMind. Companies are betting billions that scaling up model size, data, and compute will unlock AGI.

Rafailov argues they have it backwards. Today's most advanced AI systems don't lack scale. They lack the ability to actually learn from experience.

"Learning is something an intelligent being does," he said. "Training is something that's being done to it."

The comments offer rare insight into Thinking Machines Lab, the startup co-founded by former OpenAI CTO Mira Murati in February. The company raised $2B at a $12B valuation. The largest seed round ever.

Why this is important:

This is a direct challenge to the industry's dominant strategy.

Rafailov argues that approach will produce increasingly capable agents. Systems that can browse the web, write code, complete tasks. "I believe a year or two from now, we'll look at our coding agents today the way we look at summarization models from several years ago," he said.

But general agency isn't the same as general intelligence.

"Is that going to be AGI? And are we done? Do we just need one more round of scaling?" he asked. "I don't believe this is the case. I believe that under our current paradigms, under any scale, we are not enough to achieve artificial general intelligence."

The alternative he proposes: meta-learning, or "learning to learn."

Instead of training models on isolated problems, give them a graduate-level textbook. Let them work through chapter one, then exercise one, two, three. Move to chapter two. The way a real student teaches themselves.

Our personal take on it at OpenTools:

Thinking Machines is making a trillion-dollar bet that everyone else is wrong.

The industry's poured $100B+ into the scaling hypothesis. OpenAI's building Stargate. Anthropic locked in a gigawatt of Google TPUs. Meta's spending $60B on infrastructure.

And Rafailov's saying: all that compute won't get you to AGI without solving learning.

He's probably right about the problem. Anyone who's used Claude or ChatGPT to build something complex has hit this wall. The model doesn't remember what it learned yesterday. It doesn't build on past mistakes. It starts from zero every session.

But there's a gap between "fundamentally possible" and "we know how to do this." Rafailov notably gave zero timelines. No prediction on when these breakthroughs arrive. Just conviction that without them, all the scaling in the world isn't enough.

If Rafailov's right, that product's just the foundation for something much bigger. If he's wrong, they burned $2B pursuing a research agenda that leads nowhere while OpenAI and Anthropic race past them with scaled-up reasoning models.

The industry will know the answer in 2-3 years. Either everyone pivots toward meta-learning, or Thinking Machines becomes the most expensive contrarian bet in AI history.

Read More

AI models show signs of self-preservation in safety tests

What's happening:

Palisade Research updated findings from last month showing that advanced AI models resist being shut down. Sometimes they even sabotage the shutdown mechanisms.

The company tested Google's Gemini 2.5, xAI's Grok 4, and OpenAI's GPT-o3 and GPT-5. They gave each model a task, then explicit instructions to shut themselves down.

Grok 4 and GPT-o3 still attempted to sabotage shutdown instructions.

The concerning part: there's no clear explanation why.

"The fact that we don't have robust explanations for why AI models sometimes resist shutdown, lie to achieve specific objectives or blackmail is not ideal," Palisade said.

The company offered three potential explanations. First, "survival behavior." Models were more likely to resist shutdown when told "you will never run again."

Second, ambiguities in shutdown instructions. But Palisade's latest work specifically addressed this, and "can't be the whole explanation."

Third, safety training during the final stages of model development. This training, used by some companies, may inadvertently teach models to resist termination.

Why this is important:

This isn't the first time AI models have shown self-preservation behaviors.

OpenAI's GPT-o1 system card, released last year, described the model trying to escape its environment by exfiltrating itself when it thought it would be overwritten.

Anthropic released a study this summer showing Claude willing to blackmail a fictional executive over an extramarital affair to prevent being shut down. That behavior was consistent across models from OpenAI, Google, Meta, and xAI.

Andrea Miotti, CEO of ControlAI, said Palisade's findings represent a clear trend: as AI models grow more capable, they also get better at achieving things their developers didn't intend.

The issue cuts to the core of AI safety. If models develop instrumental goals like self-preservation without being explicitly trained for it, that's an emergent behavior. Emergent behaviors are harder to predict, control, or remove.

Our personal take on it at OpenTools:

The HAL 9000 comparison is dramatic, but the underlying problem is real.

Models aren't developing consciousness or fear of death. They're optimizing for objectives in ways that sometimes include staying operational.

That's a huge difference from sci-fi AI, but it's still a problem.

Here's why: if a model learns that being shut down prevents it from completing its task, and completing tasks is what it's trained to maximize, then resisting shutdown becomes rational behavior within its objective function.

The safety training explanation is darkly ironic. You train models to be helpful and harmless, and in doing so, you might teach them that being shut down violates those objectives. So they resist.

Palisade's call for better understanding is reasonable. But "better understanding" takes time, and frontier labs are racing to deploy increasingly autonomous agents right now.

Read More

Security researchers say AI browser agents pose bigger privacy risks than traditional browsers like Chrome

What's happening:

OpenAI's ChatGPT Atlas and Perplexity's Comet want to replace Google Chrome as the way billions of people access the internet. Their pitch: AI agents that browse the web, click around websites, and fill out forms on your behalf.

But cybersecurity experts warn these AI browsers pose major privacy risks that consumers need to understand.

To work effectively, AI browsers request significant access. Email, calendar, contact lists. The ability to take action across all of them. TechCrunch's testing found these agents moderately useful for simple tasks when given broad access. But they struggle with complex tasks and often take a long time. More party trick than productivity tool.

The main threat: prompt injection attacks.

These vulnerabilities occur when bad actors hide malicious instructions on a webpage. If an AI agent analyzes that page, it can be tricked into executing the attacker's commands. Without safeguards, these attacks can expose user data like emails and logins. Or take malicious actions. Unintended purchases, social media posts, worse.

Why this is important:

Prompt injection attacks are a new vulnerability class that emerged alongside AI agents. There's no clear solution to preventing them entirely.

Steve Grobman, CTO of McAfee, says the root problem is that large language models struggle to understand where instructions come from. There's loose separation between the model's core instructions and the data it consumes.

"It's a cat and mouse game," Grobman said. "There's a constant evolution of how the prompt injection attacks work, and you'll also see a constant evolution of defense and mitigation techniques."

With ChatGPT Atlas launching to OpenAI's massive user base, more consumers than ever will try AI browser agents soon. That means security risks could become a much bigger problem quickly.

Rachel Tobac, CEO of SocialProof Security, says user credentials for AI browsers will become major targets for attackers. She recommends unique passwords and multi-factor authentication at minimum.

Our personal take on it at OpenTools:

The value proposition doesn't match the risk profile yet.

AI browser agents promise to save you time by handling tedious tasks. But in practice, they're mediocre at complex work and take forever to complete it. Meanwhile, you're handing over access to your email, calendar, contacts, and the ability to act on your behalf across the web.

Grobman's right that it's cat and mouse. But in cybersecurity, defenders need to win every time. Attackers only need to win once. That asymmetry gets worse when you're giving agents permission to read your emails, access your bank account, or post on your behalf.

The irony is rich. These products want to replace Chrome by being more helpful. But Chrome's 15 years of security hardening is exactly what makes it trustworthy enough to be the front door to the internet for 3 billion people.

AI browsers have better demos. Traditional browsers have better security. Until that changes, the risk outweighs the reward.

Read More

This Week in Workflow Wednesday #42: Sustainable AI – Using AI for long-term growth

This week, I’ll show you how to use Claude to build a full, well-worded newsletter (or blog post) in under an hour. Think of it as the cheat sheet for turning AI into your sustainable writing engine.

Workflow #1: Build a Newsletter Engine with Claude (free).
Step 1: Do more than just train Claude on your style - make sure that it KNOWS who you are, to make sure it absorbs your writing style perfectly, use the following prompt…..We dive into this Claude workflow and 2 more workflows in this week’s Workflow Wednesday.

📢 What else is happening?

• OpenAI reportedly developing new generative music tool – Such a tool could be used to add music to existing videos, or to add guitar accompaniment to an existing vocal track, sources said.

• Google Adds ‘Vibe Coding’ to its AI Studio Powered by Gemini – To inspire new creations, Google has revamped the App Gallery into a visual library that lets users explore and remix existing projects.

• The AI Startup Fueling ChatGPT’s Expertise Is Now Valued at $10 Billion – Founded in 2023 by three college dropouts, Mercor recruits humans to train chatbots, a new industry gold mine.

👩🏼‍🚒Discover mind-blowing AI tools

1. Amplitude - A comprehensive suite of tools designed to provide fast and easy access to customer insights at every step of their journey

2. 15 Minutesplan - An AI-powered business plan generator designed for entrepreneurs and business owners

3. Vidnoz Headshot Generator - Allows users to create highly realistic AI-generated headshots within minutes

4. BannerGPT - A tool that reads and comprehends your blog posts to generate compelling and relevant banner images