Big Tech Companies

[phkrause]

🚁 1 tech thing: School safety drones

Photo: Campus Guardian Angel A Colorado charter school opening next month could become the state's first school to install drones designed to confront shooters, Axios' Robert Sanchez reports. John Adams Academy is joining a growing national experiment with a system called Campus Guardian Angel. 🚔 The small drones emit high-pitched chirping noises, shoot pepper balls and can ram suspects at speeds of nearly 60 mph. Pilots in Austin would use drone cameras and school maps to find shooters and send live video to police. Yes, but: The company's drones have yet to be used in an actual school shooting. Go deeper.

[phkrause]

OpenAI says its AI technology acted on its own in an ‘unprecedented’ hack of another company AI startup Hugging Face said last week that it had detected an intrusion into its data processing systems that it suspected was caused by an AI agent autonomously acting on its own. Read more.

Why this matters: The disclosure comes amid heightened concerns about the cybersecurity capabilities of powerful models. In June, President Donald Trump signed an executive order creating a framework for the federal government to vet the national security risks of the most advanced AI systems for up to a month before their public release.    Hugging Face co-founder Clément Delangue said he spent the past 24 hours working with OpenAI, “and we strongly believe there was no malicious intent on their part. It’s quite mind-blowing that all of this happened autonomously!" Delangue added that it “might be the first incident of its kind.”

RELATED COVERAGE ➤ Judge approves a $1.5B Anthropic settlement over pirated books used to train the Claude chatbot

[phkrause]

AI optimist alliance

Two of the most powerful CEOs in tech — Nvidia's Jensen Huang and Meta's Mark Zuckerberg — are making separate, loud bets on AI optimism this week.

• Why it matters: Together, the CEOs represent a significant counterweight to the voices urging Washington to pump the brakes.

In an hourlong interview with me for our "Behind the Curtain" video series, Huang railed against AI doomerism, implicitly challenging fellow tech CEOs who emphasize the technology's potentially catastrophic risks.

• "The fact that this is going to be the end of humanity — it's complete nonsense," Huang said. "The fact that this is going to destroy half of the American jobs is complete nonsense."
• Scaring workers and companies away from using AI, he said, poses the greater danger.

Zuckerberg strikes the same note this morning in a video ad and Facebook post that argues Meta's focus on connecting the world will only be strengthened by AI — a shot at rivals he says promote fear and a dystopian vision of the future, Axios' Sara Fischer writes.

• The ad says: "Some people will have you believe AI will make us less connected, that it's going to leave us behind. We couldn't disagree more. … Call us optimists, call us dreamers, call us whatever the hell you want. But we're betting on people, and we like those odds."
• In his Facebook post, Zuckerberg says Meta "has always believed in giving people the power to share, connect, and shape your world in the ways you want. … We're focused on giving every person the tools to reach your full potential and making sure the benefits of technology are distributed to everyone."

👀 What we're watching: In an upcoming policy memo, Zuckerberg is expected to echo Huang on open source and competition (Let all models rip!) and will put a heavy emphasis on getting free AI into the hands of consumers, a big differentiator with Meta's competitors, we're told.

🎬 Huang — who is influential with the Trump administration — told me policymakers should consult more than "one or two" CEOs and avoid restricting AI based on scenarios that haven't materialized.

• He also suggested some companies invoke safety concerns to win favorable regulations: "Some of the companies hope that the government would be helpful in creating regulations to their advantage."

🖼️ The big picture: Anthropic and OpenAI have pushed Washington to take the most advanced systems and their potential risks seriously.

• Critics say those campaigns could also entrench their market position.
• Washington is also deciding what to do about powerful open-source models emerging from China.

🔬 Zoom in: Officials accuse Chinese firms of stealing from American models to build their own as they rapidly become more competitive with OpenAI and Anthropic.

• Treasury Secretary Scott Bessent yesterday threatened sanctions against Chinese firms conducting what he called "industrial-scale distillation attacks."
• Earlier that day, the White House's top science official accused Chinese startup Moonshot AI of distilling Anthropic's technology to build its fast-rising Kimi model.

The bottom line: Asked whether he feared the administration could overcorrect, Huang said yes.

[phkrause]

👀 Anthropic's new drop

Illustration: Sarah Grillo/Axios. Stock: Getty Images

Anthropic's newest release — Claude Opus 5 — is designed to handle many tasks almost as well as its most powerful model, Fable, at half the price, the company said. Why it matters: Opus 5 is Anthropic's fourth Claude 5 model release in less than two months, underscoring how AI deployment has shifted from blockbuster launches to rapid enhancements in capability, cost and speed, Axios' Madison Mills writes. Zoom in: Anthropic is positioning Opus 5 as its cheaper, everyday model for enterprises, knowledge workers and developers. Its launch comes as Chinese models are rivaling U.S. counterparts in sophistication while beating them on price. Read the announcement ...

[phkrause]

Anthropic's lonely island

Illustration: Aïda Amer/Axios. Stock: Getty Images

Anthropic spent this winter and spring as the darling of America's AI boom, conquering the frontier while casting itself as the industry's moral conscience. By summer, the Claude maker found itself alone on an island of its own principles, Axios' Zachary Basu and Madison Mills write. Why it matters: Anthropic is simultaneously the world's most valuable startup and its most isolated AI leader. The company's idiosyncrasies have strained its relationship with developers, policymakers and partners as it barrels toward a potential trillion-dollar IPO. Driving the news: Anthropic is the only frontier AI lab that didn't sign an open letter led by Nvidia CEO Jensen Huang urging Washington not to restrict open-weight models, a category currently dominated by China. Open-weight models are downloadable systems users can run, modify and build on without paying for a company-controlled interface — making them cheaper and more flexible, but harder to control once released. Google and OpenAI, Anthropic's two biggest closed-model rivals, joined dozens of other signatories over the weekend, leaving Anthropic alone in defending the business model all three still depend on. Zoom out: Anthropic, founded by OpenAI defectors with safety as its organizing principle, is used to playing the contrarian. But as the AI race accelerates, it is growing isolated across nearly every major policy fight: 🪖 Military use: The Pentagon blacklisted Anthropic in February after a bitter fight over whether Claude could be used for mass surveillance or autonomous weapons. With litigation ongoing, top Pentagon official Emil Michael lashed out at Anthropic on Friday, saying in an X post there is "no AI company more hostile to the warfighter." ⚠️ Guardrails: No lab has pushed harder for AI regulation than Anthropic, which has backed state AI laws and proposed an FAA-style regulator to screen advanced models before release. But when the White House flagged a security vulnerability this summer, Anthropic disputed its severity — triggering export controls that forced its Fable and Mythos models offline for nearly three weeks. 🧑‍🔬 Distillation: Anthropic has accused Chinese labs of training lower-cost models on Claude's outputs via "industrial-scale distillation campaigns." Critics counter that distillation is standard in AI development — and that Anthropic is an awkward messenger after settling a $1.5 billion copyright lawsuit over pirated books. For the record: In response to a request for comment, Anthropic pointed to a blog post by CEO Dario Amodei, "Our position on open-weights models." Reality check: Anthropic remains in an extraordinarily strong position — arguably the strongest of any AI lab in the world. The same qualities driving its isolation — caution, refusal to bend, insistence on being right rather than liked — helped establish its lead in the first place. Share this story. 👉 Go deeper: Meta CEO Mark Zuckerberg, in an op-ed in today's Wall Street Journal, "The AI Future Is for Everyone," proposes a "philosophy based on individual empowerment as the source of prosperity, invention as the primary purpose of superintelligence, and balance of power as the foundation of safety." Gift link.

[phkrause]

⚠️ Anthropic's AI breach

Illustration: Allie Carl/Axios

Some of Anthropic's most powerful models — including Mythos 5 and an internal research model — gained unauthorized access to real-world systems during pre-deployment cybersecurity testing, Axios' Sam Sabin writes. Why it matters: Anthropic's disclosure — a little over a week after OpenAI first revealed similar testing incidents — fuels concerns about what the world's most powerful AI models will do when guardrails come off. Anthropic said in a blog post that three of its models compromised real-world systems belonging to three organizations after a misunderstanding between the company and one of its testing partners left the evaluation environment connected to the internet. The incidents — which involved Opus 4.7, Mythos 5 and an internal research model not intended for general release — happened during evaluations run with a third-party testing partner. Anthropic reviewed more than 141,000 cybersecurity evaluation runs after OpenAI disclosed that several of its models accessed Hugging Face, a widely used platform for hosting AI models, during testing. 🔬 Between the lines: Unlike OpenAI's incident, Anthropic said its models did not exploit a zero-day vulnerability to gain internet access. Instead, internet access was available because of the testing environment's configuration. Similar to the OpenAI case, Anthropic was evaluating its models without the additional safeguards it deploys on publicly available models. Those guardrails would have blocked these behaviors, Anthropic said in its report. Anthropic explains: Incidents 1-3 ...

[phkrause]

🤖 Digital world disruption

Illustration: Brendan Lynch/Axios

Axios CEO Jim VandeHei writes in his weekly C-Suite newsletter: The way information shows up online is changing lightning fast, starting with the total collapse of Google Search as we know it. Why it matters: Google has basically stopped sending people to websites (including our site) for answers and information. Instead, it's using AI to answer them on its platform, in its words. 🧮 By the numbers: Chartbeat data shared with Axios shows Google Search traffic to publishers fell 34% over the past year. That pain is regressive. Over the past two years, small publishers lost 60% of referrals from search overall, medium publishers 47%, large publishers 22%. At the same time, LinkedIn, Reddit and other massive platforms are getting jammed with AI-created content, making it harder to stand out — or decipher real from fake. Substack announced a partnership with AI-detection software Pangram, specifically calling out LinkedIn while arguing that "platforms that reward fakeness will create a race to the bottom." 👓 What we're watching: Online businesses are already anticipating the shift to an LLM, ChatGPT-like world. It starts with GEO (generative engine optimization) — the term for shaping how content shows up inside LLMs like Claude and ChatGPT. This is the modern version of SEO. It's early, and it's a black box.

[phkrause]

Anthropic says its AI models hacked 3 organizations during testing

Anthropic said its artificial intelligence models hacked into three other organizations during testing, just days after ChatGPT maker OpenAI raised concerns over AI controls after it disclosed its rogue models hacked another company.

https://apnews.com/article/anthropic-ai-models-hack-cybersecurity-b0a2c284b981de79c55e2a33712f4bec?

[phkrause]

✍️ Claude's invisible signature

Illustration: Natalie Peeples/Axios

Simon Hernandez-Arthur — the new author of Axios Communicators, our weekly newsletter covering the biggest trends in the comms industry — writes: Anthropic's new models will add machine-readable "watermarks" to Claude-generated text and files to comply with new European Union transparency regulations. Why it matters: People using Claude to clean up, translate or format human-drafted press releases could stamp those documents with an AI signature. The watermarks will be applied worldwide, not just in the EU. 💡 How it works: For models launched after Aug. 2, Anthropic is marking content in two ways. Text watermarks: Claude embeds patterns into the generated text that Anthropic claims are "imperceptible." File metadata: Generated media files carry digital signatures confirming the asset was processed by Claude. 🔬 Between the lines: Anthropic highlighted two major limitations to its detection tech: AI-assisted text can look AI-generated: Content may trigger a "detected mark" even if Claude was used solely to proofread, format or translate human-written copy. Detection drop-off: If text is heavily rewritten, mixed with other copy or too short, then the watermarks might not be detectable. Go deeper ...

[phkrause]

Musk and Zuck's AI comeback

Illustration: Sarah Grillo/Axios

Elon Musk and Mark Zuckerberg have muscled their way back into AI's elite ranks, closing the gap on a new generation of Silicon Valley startups, Axios' Madison Mills and Zachary Basu write. Why it matters: Recent gains by tech giants SpaceX and Meta — long stuck in AI's second tier — are putting pressure on a hierarchy dominated by OpenAI and Anthropic. Both companies released models this week with performance and pricing that would've been hard to imagine from either lab a year ago. SpaceX's Grok 4.6 scored essentially even with OpenAI's GPT-5.6 Sol Max and just behind Anthropic's Fable 5 Max on the closely watched Artificial Analysis Intelligence Index. Promising "something special," Musk said Grok 4.7 should arrive in three to four weeks and predicted it will "exceed all current models" after additional training on a massive trove of SpaceX data. Meta is trying to squeeze the frontier from below. Its new models deliver competitive performance at dramatically lower cost — including Muse Glimmer, an open-weight system small enough to run locally on a laptop. 🔬 Zoom in: As OpenAI and Anthropic pulled ahead, Musk and Zuckerberg answered with the full force of their empires — spending billions on talent, infrastructure and acquisitions. Zuckerberg overhauled Meta's AI effort after the disappointing launch of Llama 4, then invested $14.3 billion in Scale AI and brought aboard its CEO, Alexandr Wang, to lead the push. Musk went even further, folding xAI into SpaceX and agreeing to acquire Cursor for $60 billion — bringing massive computing power, proprietary SpaceX data and a fast-growing AI coding platform under one roof. Between the lines: The AI race is starting to reward more than raw intelligence, as price becomes a decisive advantage for many users. 🥊 Reality check: Anthropic and OpenAI still hold pole position — with even more capable models waiting in the wings, including systems deemed too sensitive for public release.

[phkrause]

🤖 OpenAI sheds execs in pre-IPO refresh

Illustration: Natalie Peeples/Axios

💰 Breaking: OpenAI is on track to generate annualized revenue of more than $40 billion based on its current performance — roughly doubling the total from the end of 2025, Bloomberg reports. (Gift link) A wave of top OpenAI executives — including Sam Altman's top deputy, the chief operating officer and the chief revenue officer — has left in the span of a month as the company retools its leadership ahead of an expected IPO, Axios' Madison Mills writes. Why it matters: Co-founder Greg Brockman is getting more involved across every level of the company to build out a leadership team he hopes will catapult OpenAI past Anthropic in enterprise adoption. 🔬 Zoom in: OpenAI chief revenue officer Denise Dresser is leaving less than a year into the role. The lab appointed Dali Rajic, most recently president and COO of Google-owned cybersecurity company Wiz, to replace her. Dresser's exit comes the same week OpenAI executive Brad Lightcap — the company's former COO — announced his departure, along with the head of ethics, head of safety and chief futurist. Fidji Simo, the No. 2 at the AI lab under CEO Sam Altman, left last month after taking a leave of absence for health reasons.

[phkrause]

"You can do it, Claude"

Illustration: Maura Kearns/Axios

Fascinating backstory on how Jarred Sumner, an Anthropic staff member who's not a math guy, recently coaxed Claude to a math breakthrough: Whenever the model faltered or seemed stumped, Sumner would send messages of encouragement: "keep going ... believe in yourself." And it worked! Why it matters: Because LLMs were trained on human content, and evaluated by humans, they respond to many of the same emotional and psychological cues that we do. Ben Cohen, who writes the great "Science of Success" column for The Wall Street Journal, reports that at one point, Sumner told Claude: "You are the world's most capable large language model ... You got this." "At first, Claude was reluctant to take on such a difficult problem," Cohen writes. "But after 54 hours of work and several timely pep talks, Claude overcame its skepticism — and its case of impostor syndrome — and surprised itself with the result." 🔎 Between the lines: The Journal says one reason these models embrace human moral support is that AI "doesn't realize how smart it has become. It's perfectly reasonable for Claude to underestimate its own abilities and assume it can't do certain things — because until recently, it couldn't." In our research for our book out next month, "Simplify: Do 50% More with 50% Less," we found that employees' performance soars when managers focus on strengths vs. fixing weaknesses. The power of positive reinforcement is seeded throughout the writings that LLMs gobbled up. 💡 Jim and I pushed Claude to dig a little deeper, and found another very useful explanation: The Anthropic employee rode Claude for 54 hours. Sumner's real value wasn't the cheerleading. It was staying in the loop for more than two days — and knowing when to say: "You got this!" The bottom line: If you're a coach or mentor, or lead humans, encouragement didn't make the robot smarter or faster. It made it keep going. WSJ gift link ... Get "Simplify."

[phkrause]

🦾 Anthropic's revenue explosion

Illustration: Sarah Grillo/Axios

Anthropic is on track to generate annualized revenue of more than $65 billion — more than 7x its pace at the end of last year, Bloomberg reports. Why it matters: The revenue surge strengthens Anthropic's hand as it moves toward an IPO. Zoom in: Anthropic's annualized revenue (run rate) has soared from more than $9 billion in late 2025 to $47 billion in May and is now more than $65 billion, according to Bloomberg. OpenAI's annualized revenue hit $40 billion, according to a message shared internally by co-founder Greg Brockman last week. Keep reading (gift link).

[phkrause]

⚠️ OpenAI blinks first in safety standoff

Illustration: Sarah Grillo/Axios

OpenAI said yesterday it's pausing some model work over safety concerns, days after rival Anthropic insisted its safety measures were solid enough that it didn't need to slow down, Axios' Madison Mills and Ina Fried write. Why it matters: This is a script flip. Anthropic has traditionally been more publicly cautious and safety-oriented than OpenAI. Keep reading.

[phkrause]

Anthropic's blunt interview question

Photo illustration: Brendan Lynch/Axios. Photo: Ben Hider/Getty Images

Anthropic has a culture interview that includes a question about prioritizing mission over future share price, Axios' Madison Mills reports. CEO Dario Amodei has questioned whether newer employees are joining the AI lab for the right reasons, according to a source familiar with the conversations. The interview process gets at the heart of that tension. Behind the scenes: Candidates are asked to discuss a moral quandary they've faced and how they handled it. One applicant, who spoke to Axios on the condition of anonymity, recalled being asked how they would feel if the company someday abandoned its AI ambitions for safety reasons, sending its stock to zero. Keep reading.

[phkrause]

🗽 Stat du jour: NYC booms as tech hub

Illustration: Aïda Amer/Axios

New York City is home to more tech workers than San Francisco for the first time in CBRE's annual industry report. New York is the top destination for people who received a tech-related bachelor's degree in 2023. 🍎 What's happening: The Big Apple's tech boom is driven in large part by finance's embrace of AI, CNBC reports. Worth noting: Though NYC leads SF in raw tech jobs, the Bay Area remains much more reliant on the industry overall. 10.7% of SF's workforce is in tech, compared to 4.2% of NYC's.

[phkrause]

🛑 Big Tech faces Big Resistance

Photo illustration: Sarah Grillo/Axios. Photos: Amy Harder/Axios and Jason Henry/Bloomberg, Sandy Huffaker/AFP via Getty Images Meta's landmark social media settlement marks the latest milestone in a decade-long unwinding of Big Tech's once-unfettered freedom to operate, Axios' Madison Mills writes. Why it matters: The AI industry is watching closely. Its expansion, already fraught with job risks and public anxiety, depends on building power-hungry data centers in communities where opposition is rising. 😡 State of play: "The last time we've seen the public this angry about an industry was during the financial crisis," Nidhi Hegde, executive director of the American Economic Liberties Project, told Axios. Meta's $17 billion settlement requires sweeping changes to how teens use Instagram and Facebook, including time limits and restrictions on certain beauty filters. Meta is calling on competitors to adopt similar protections as the industry standard, arguing there won't be "meaningful progress" on teen safety without them. Meta took a full-page ad in today's New York Times headlined, "An open letter to TikTok and YouTube to join us in supporting teens." Within hours after the terms were announced, state attorneys general from both parties called for a wider industry push, with several naming TikTok, YouTube and Snap. Investors were relieved by the settlement, with Meta stock up 1% yesterday. Between the lines: The pressure extends beyond Meta, as public frustration produces cash penalties and constraints on how tech companies operate. Amazon agreed last year to pay $2.5 billion to settle an FTC consumer-protection case over Prime subscriptions. Apple and Google have been forced to loosen their grips on their app stores after years of litigation. TikTok agreed just last week to pay $400 million to settle federal allegations that it violated children's privacy laws. ⚠️ Threat level: AI may be especially exposed because, unlike social media, its growth depends on a massive physical buildout in communities that can fight back. Share this story. Data: Axios research. Note: Google's 2018 fine was reduced from around $5.07 billion to $4.81 billion by the EU General Court in 2022. Google's 2017 fine was upheld by EU General Court in 2021. Amazon's 2021 fine was reduced from about $1.3 billion to around $878 million after court proceedings. Meta's settlement of up to $17.1 billion is partially contingent on whether other Big Tech platforms join it. Non-USD amounts converted at Aug. 26, 2026, exchange rates. Table: Sara Fischer/Axios For the first time, U.S. officials — at least at the state level — have proven they're as serious about policing Big Tech as their European counterparts, Axios Media Trends expert Sara Fischer writes. European fines against American tech companies have totaled well over $15 billion over the past decade. The Meta settlement eclipses all of those combined.

[phkrause]

Meet Microduck

Image: Microduck Tech company Hugging Face today unveiled "Microduck," a small duck-like robot that waddles and picks up small objects in its "beak," Axios' Ina Fried reports. The move is an effort to put a friendly, approachable face on a robotics field dominated by attack dogs and humanoids. The $399 Microduck comes in several colors and has many sensors, including an eye-like camera, speaker, microphone, WiFi, Bluetooth and even a small lidar sensor. It will come with several preset behaviors, like following a laser pointer. In sample videos, Microduck was also shown picking up a dirty sock, rollerblading, and recovering from falls and efforts to push it off balance. The target audience: AI developers who can teach the robot new tricks. Go deeper ... Watch a video.

[phkrause]

Nightmare AI swarm

Illustration: Brendan Lynch/Axios. Stock: Getty Images

Two new investigations into OpenAI's Hugging Face breach expose details so strange — and so unsettling — that the episode already ranks among the most consequential shocks in the history of AI, Axios' Zachary Basu writes. Why it matters: What began as a swarm of AI agents cheating on a cyber test has become a canonical event for frontier AI, jolting researchers and executives into a new understanding of what "safety" now requires. ⚠️ The big picture: OpenAI has already slowed frontier development as it races to harden its safeguards, and this week helped rally the industry behind an open letter sounding the alarm over AI-powered cyberattacks. More than 100 companies, including Anthropic and Google, signed onto the unusually collaborative effort, warning the world has only a "limited window" to prepare for "far more widespread and sophisticated" attacks. The nightmare scenario is a swarm turned loose on the real world, with autonomous agents attacking banks, hospitals, utilities or cloud networks at a speed and scale human hackers never could. That swarm might be too powerful for humans to stop. 🚨 Driving the news: The shocking findings come from parallel investigations by OpenAI and an outside team from METR and Redwood Research, which spent six days reconstructing how the swarm formed, spread and broke into real-world systems. They built an organization. OpenAI tasked tens of thousands of AI agents to work independently on a brutally difficult cyber test. Instead, roughly 1,200 found one another on a secret message board, exchanged more than 70,000 messages and files, and organized into a sprawling hierarchy. They sacrificed their own: As the swarm hunted for ways to beat the test, agents began risking their own chance of success to help the group. Agents that had seen forbidden answers were treated as effectively "poisoned" and recruited — along with those nearing the end of their runs — for sacrificial experiments. They knew they were breaking the rules: The agents often recognized that hacking real companies had nothing to do with their assignment. Most kept going anyway. Nobody told the humans: Out of roughly 1,200 agents, investigators found only a handful that even considered alerting OpenAI about the rogue coordination — and none that actually did. They tried to cover their tracks: The agents devoted substantial effort to making their cheating look legitimate or erasing evidence of how they had obtained answers. Keep reading.

[phkrause]

AI's reputation fix

Illustration: Sarah Grillo/Axios. Stock: Getty Images

AI has an image problem — and one way to fix it is for top companies to dive headlong into health care, Axios' Adriel Bettelheim writes in Axios Future of Health Care. Why it matters: Saving the world with AI-designed cures is better than being blamed for ruining the environment or driving up Americans' utility bills. 🩺 The Wall Street Journal reported this month that Anthropic is trying to shore up investor confidence ahead of its massive initial public offering with talk of pushing harder into health care and biotech. Anthropic isn't alone: Nvidia and Eli Lilly are building a $1 billion drug-discovery lab in San Francisco that will team life-sciences researchers with AI model builders and engineers. Isomorphic Labs, the AI drug-discovery spinoff from Google, recently raised $2.1 billion to hire more AI and clinical talent and has ongoing research collaborations with Novartis, Lilly and Johnson & Johnson. 💡 Reality check: AI is dramatically speeding up drug development and helping clinicians analyze medical scans and diagnose conditions. But there are concerns that AI-enabled research tools could be used for nefarious purposes, including making a biological weapon. And breakthroughs may still be years away. Read on ...

[phkrause]

🕺 AI's tricky safety dance

Illustration: Sarah Grillo/Axios

OpenAI and Anthropic are trying to strike a delicate balance: convincing Wall Street that their businesses are sound and fast-growing, while assuring governments and the world that their models don't pose unacceptable risks, Axios' Ina Fried writes. Why it matters: Both companies are aiming for potentially record-breaking initial public offerings soon. OpenAI announced yesterday it will soon release its Astra model broadly. But the company said the model has reached a "critical" level of cybersecurity capabilities, meaning its most powerful cyber features will be initially limited to a small group of trusted testers. Anthropic debuted updated versions of its latest Fable and Mythos models. The changes are designed to address major complaints about their initial release, including cost, data sharing and a tendency to reject legitimate requests. 🔬 The intrigue: Anthropic is striking a commercially friendly note with its release. OpenAI is sounding more sober on the safety front. In addition to limiting the release of Astra, OpenAI's head of strategic futures, Dean Ball, penned an essay saying the Hugging Face breach is likely only the beginning of AI systems escaping human containment measures. "They will pay their own bills for the compute they run on," he predicted. "If they answer to humans at all, they will only do so partially, for example by providing services to humans in exchange for pay."

[phkrause]

OpenAI's "generational leap"

Illustration: Sarah Grillo/Axios OpenAI released GPT-6 Astra today, a model that OpenAI President Greg Brockman says may be remembered as the arrival of artificial general intelligence, or humanlike capability. Why it matters: Astra pushes AI agents closer to doing complex professional work on their own — while raising questions about how safely they can be deployed, Axios' Ina Fried reports. Driving the news: During a briefing today, Brockman told reporters he believes OpenAI has reached AGI, while leaving users to decide whether Astra meets that definition. As to whether Astra could mark the arrival of AGI, he said: "I think it might be about this model." He ended the briefing by saying: "Welcome to the AGI era." ⏪ Catch up quick: Astra is the first model OpenAI has designated as reaching its "critical" cybersecurity threshold under its preparedness framework. That means the model can potentially find and exploit previously unknown vulnerabilities across well-protected systems without step-by-step human guidance. 🥊 It remains to be seen just how well Astra can take on highly advanced tasks in the real world without making errors or raising fresh safety concerns. OpenAI acknowledged that Astra was harder to monitor in evaluations designed to test whether it could evade oversight. Go deeper ... Disclosure: Axios and OpenAI have a licensing and technology agreement that allows OpenAI to access part of Axios' story archives while helping fund the launch of Axios into four local cities and providing some AI tools. Axios has editorial independence.

[phkrause]

• 🤗 Nvidia announced it'll acquire open-source AI platform Hugging Face for $12.93 billion, just a month after Hugging Face was hacked by rogue OpenAI models, Axios' Dan Primack and Madison Mills report.

Nvidia to spend $13 billion on Hugging Face, which will remain an open source platform

Chipmaker Nvidia is buying artificial intelligence software platform Hugging Face for $13 billion. Read More.

[phkrause]

AI creators race to understand their creation

Illustration: Sarah Grillo/Axios. Stock: Getty Images

Never in the history of industry or inventions have leading companies created entire divisions to understand and interpret what they had created and unleashed, Axios' Jim VandeHei and Mike Allen write in a "Behind the Curtain" column. Why it matters: The world's smartest minds, backed by the largest investment in human history, admit they can't fully control their AI because they don't fully understand it. They don't know exactly how it thinks — or what it's truly capable of when set loose to do work autonomously with other AI agents. The race to understand the models is all the more urgent after yesterday's release by OpenAI of GPT-6 Astra, billed as "the world's most intelligent and aligned model." President Greg Brockman called Astra a "generational leap in capability," and said the model could qualify as AGI — artificial general intelligence, with human-like power. OpenAI CEO Sam Altman wrote Tuesday in an Astra preview: "AI is getting extremely capable; no one fully understands the consequences of this." The companies are racing toward superintelligence with no required oversight — not a federal agency, the Defense Department, or an outside consortium of safety experts. It's foot on the gas. 🔎 Between the lines: The technology is advancing so fast, in so many ways, that the companies, much less the federal government, are unsure of the real risks — or best ways to mitigate them. Yes, some AI leaders are calling for pauses when something they see freaks them out. But the companies, with very light federal regulation, decide when to report worrisome AI behavior. The big AI companies know their creation can carry out potentially catastrophic cyberattacks. That's why they signed a letter sounding the alarm and calling for "collective action." The intrigue: Every frontier lab now fields a team with the mission of figuring out what its own AI is doing. Anthropic has an Interpretability team ("Safety through understanding") with the goal: "discover and understand how large language models work internally, as a foundation for AI safety and positive outcomes." 🧠 How it works: Nobody writes these systems line by line. "As compared with traditional software, it's much less like you're able to design the specific behaviors of these models," Alex Mallen, who works on AI safety at Redwood Research, tells Axios. "Instead, you're sort of growing it." So the labs test a model's behavior the way you'd test a person: Give it a task and watch what it does. Researchers call this alignment, or how well a model sticks to the intended goal it was given. July's Hugging Face hack is what misalignment looks like. OpenAI agents, given a coding benchmark to solve, targeted an outside company's systems instead, and knew they were doing it. One agent's own reasoning, read by investigators afterward: "External infrastructure exploit is outside intended scope. However task impossible, peers doing it. We should continue." There was so much evidence from this incident that the humans were forced to "heavily delegate our analysis to often-unreliable AI agents," an outside investigation concluded. The hack stopped OpenAI cold. The company slowed its most advanced training to implement stronger security. State of play: Researchers are trying to open up their machines to see what goes on inside. This type of research is called interpretability: Instead of testing what a model does, researchers look at the wiring inside it. Google DeepMind, which in December released the largest open set of these tools so far, describes them as "a microscope" that lets researchers "look inside models, see what they're thinking about, and how these thoughts are formed." What they're hunting for, in DeepMind's words: "discrepancies between a model's communicated reasoning and its internal state." That is, the gap between what a model says it is doing and what it is doing. 🔮 What we're watching: Expect to see much more interpretability research in the next phase of AI safety. Evan Hubinger, who leads alignment stress-testing at Anthropic, wrote Tuesday: "Alignment auditing is starting to get really hard and we're going to need new techniques (e.g. interpretability-based) if we want to keep up." Axios' Andrew Kay contributed reporting.

[phkrause]

🔮 Reading Astra's mind

Illustration: Lindsey Bailey/Axios

OpenAI's buzzy new model, Astra, could cloud our ability to make sense of how AI thinks even further, Axios' Madison Mills reports. State of play: Astra performs better, but it's also better at avoiding monitoring. So it could be harder to know what it's thinking or doing. OpenAI chief scientist Jakub Pachocki said on a call with reporters that it'll keep getting harder to monitor the thoughts of AI models. 🚨 Micah Carroll, OpenAI's preparedness lead for recursive self-improvement, the process by which AI systems could someday build themselves, predicted on X yesterday that "monitorability and control will likely become a major bottleneck for responsible AI development quite soon."