I have been writing about the generative AI wave and its legal, ethical, and social implications since late 2022, following ChatGPT’s launch.

Almost four years later, it feels as if the pace of technological development has accelerated further, and our human frameworks for overseeing and governing these developments are struggling to keep up.

This was, of course, expected.

Governing a class of autonomous systems that is being indefinitely scaled to “smarter than human” standards and to self-recursively improve itself is a complex mission. In fact, as humans, we have never done that before.

While AI can be continuously scaled through various methods (and new benchmarks are saturated almost every week), our biological brains and human-made institutions cannot.

Just as every pair of materials has its coefficient of friction and every liquid has its boiling point in Earth's atmosphere, I believe techno-social systems also have a tipping point at which governance may spiral out of control.

AI governance getting out of control, in practice, would mean we are no longer thinking through policies and decisions.

We would simply be reacting, putting out fires, and ensuring the survival of specific companies, institutions, systems, or groups.

If and when AI governance gets out of control, it means we are no longer governing anything. AI-driven advancements are, instead, governing how we live as humans.

We would probably get to the point where AI would be governing us instead.

Signs of loss of control in AI governance include AI policy decisions that become outdated as soon as they are published, the inability to respond coherently to an AI-driven event, or the degradation of human systems, frameworks, and institutions due to AI's impact.

AI's impact has been pervasive and multilayered, and AI governance is struggling. However, I do not think AI governance has gotten out of control yet.

But given the pace of AI development and what recent events have shown us, it could.

Sadly, the majority of people, including policymakers and lawmakers, would likely not recognize the early signs of AI governance losing control. When it becomes obvious and enters the mainstream debate, it would probably be too late.

Today I would like to reflect on a few recent developments and where we should be heading over the next few years.

Where we are:

People have been in awe of LLMs since ChatGPT became widely available at the end of 2022.

Now, almost four years after that first exposure, the AI ecosystem has changed significantly, as well as the main use cases, expectations, social norms, rules, policies, frameworks, and threats.

One of the main concerns around generative AI was mass unemployment. This has not happened (at least yet), and recent news seems to show the opposite trend.

Individually, generative AI has put millions of knowledge workers under pressure, as in some fields there is significant exposure, and many feel a constant fear that AI might devalue their skills and make them lose their source of income.

Millions are being encouraged (sometimes forced) to incorporate generative AI into their daily work in a comprehensive way to demonstrate fluency and, indirectly, to avoid being outpaced or fired.

However, when everybody incorporates the same type of tool, there is no significant competitive advantage or meaningful human skill development.

We now have studies showing burnout, cognitive debt, deskilling, and never-skilling.

Some people have benefited from productivity gains and/or incremental improvements in how they work, but these changes have also been accompanied by increased workload for reviewing and overseeing AI, as well as competition from new AI-powered players.

Human-AI intimacy is on the rise, with millions interacting with AI emotionally as friends, confidants, or romantic partners.

This type of interaction has been associated with numerous cases of harm and suicide, especially when teenagers and people with underlying mental health issues are involved.

New AI copyright lawsuits continue to emerge, as AI companies routinely use copyrighted works to train AI models without consent, credit, or compensation to the original creators, threatening the economic sustainability of creative industries and individual creators.

Search engines have become full-blown AI engines; AI slop continues to inundate content platforms, and there is a constant cat-and-mouse game to detect AI-generated media, stop slop, and incentivize original human-made content.

The big promise from the AI industry is collective, though.

Sam Altman, Elon Musk, Demis Hassabis, Dario Amodei, and many others have spoken about a future of abundance, where diseases are cured, all our needs are met, and AI takes care of everything while we focus on enjoyment and self-growth.

We are definitely not there yet, and might never be.

(Being realistic, given humanity's track record, the odds of this utopian future ever becoming reality are probably zero.)

But there have been promising developments in drug discovery, diagnostic accuracy, disease detection, productivity, accessibility, and various other areas.

I recently had my own Eureka moment after AI helped me diagnose a health condition various doctors had missed for years. With greater awareness and literacy, millions could have access to better healthcare and more tools to manage their own health.

As so many smart people work to ensure AI benefits humanity and improves people’s lives around the world, I have tried to be more optimistic and collect success stories.

Demis Hassabis himself, who won a Nobel Prize for his work on AI-driven protein prediction, is involved in a major initiative to ‘solve all disease.’

I hope he succeeds and that we will soon have thousands of AlphaFold-like case studies from around the world and across various fields that, in the aggregate, will improve everyone’s lives.

Lately, the news that took over the headlines was slightly less optimistic, especially from an AI governance perspective.

After Mythos, Anthropic's unreleased AI model whose unique cyber capabilities led to a full rewiring of the U.S. AI policy approach, an unreleased OpenAI model escaped a sandbox, exploited a zero-day vulnerability, and compromised Hugging Face's infrastructure during testing.

Interestingly, Hugging Face used a Chinese AI model to resolve the security incident.

It has already become clear that AI is now an integral part of countries’ military and national defense strategy.

Recent developments involving Anthropic and OpenAI's models show that the nature of the threats (and the capabilities required to defend against them) is changing.

In AI governance, the future will be challenging.

Where We Should Be Heading:

When we think about possible tools and mechanisms to govern AI, what often comes to mind are laws, policies, standards, guardrails, audits, safety tests, control, oversight, and review mechanisms, as well as specific technical and legal tools that can help identify and address risks and threats.

These are indeed potential options in the AI governance arsenal that should be thoroughly explored by lawmakers, policymakers, and all professionals currently tackling AI governance issues worldwide.

However, when dealing with scaling and AI acceleration, in a context where benchmarks are continually saturated and new capabilities emerge every week, broad transparency and continuous knowledge diffusion within the various players and stakeholders of an AI ecosystem become key aspects of the AI governance strategy.

More important than any specific AI governance measure, which might be overcome or bypassed within days by a new AI model, capability, feature, or attack, are the robustness, preparedness level, and integration between the various entities within a given AI ecosystem.

As a recent news article on the OpenAI-Hugging Face incident showed, it took days for OpenAI, one of the world's top frontier AI developers, to notice its AI agent had gone rogue. It only discovered it after the FBI was alerted and the threat was contained.

Also, the safety incident was contained using an open-source Chinese AI model.

This safety incident highlights that no single organization or government will be immune to systemic risks, attacks, or AI-driven harm.

Given the rise in capabilities and threats, we also expect safety events to worsen and become more difficult to detect.

With different transparency rules around AI testing and safety, as well as more openness during AI development, including training methods, guardrails, safety mechanisms, and so on, various stakeholders can grow, adapt, integrate, personalize, and prepare better.

Another series of recent (much more positive) events reflects this point of view, and what is probably the best way forward.

In his first-ever social media post, Jensen Huang, NVIDIA's founder and CEO, shared a letter about the importance of open-weight models in strengthening the American AI ecosystem:

The letter was widely shared within the AI industry, leading to a long list of supporters (read the letter and see the logos at the end here). Anthropic was notably absent from the list of supporters.

Jensen Huang's next post launched the Open Secure AI Alliance, focused on building and sharing open tools.

The article announcing the alliance highlights the importance of openness in AI, especially in light of rising threats:

From an AI governance perspective, lawmakers and policymakers must acknowledge emerging challenges and treat transparency and openness as assets that must be used strategically to strengthen the AI ecosystem, reduce risk, and defend against new threats.

Back on the topic of AI governance spiraling out of control:

AI is probably the most powerful technology ever created by humans, as it is an all-encompassing, ‘smarter-than-humans’, autonomous cognitive layer that can be integrated into anything, for good or bad purposes.

At the same time, the opacity surrounding how AI works and the techniques used to develop it, along with the many unknowns surrounding existing and emerging AI-powered threats, make it one of the most difficult technologies to govern.

Trying to govern something we cannot fully understand and scrutinize is hard and often ineffective.

What I and many others are proposing is to strategically strengthen transparency and openness where we can, so that we make the research, development, security, oversight, and governance layers more refined, robust, integrated, and prepared.

Share

Subscribe now

Today, a call between DeepSeek's founder and CEO, Liang Wenfeng, and a group of investors was leaked.

A few hours after the leak, the WeChat links referring to this call had been removed from the internet.

It is still unclear whether the Chinese government ordered it due to political, economic, or regulatory risk, or whether it was a private removal request.

My guess is that it was a private request, but the Chinese government is glad it was removed, as it contained a Chinese AI CEO's thoughts on China's strategic positioning in AI.

What is DeepSeek and why do its CEO’s opinions matter?

DeepSeek is a leading Chinese AI company that caused a massive shock to the U.S. stock market last year when its R1 model closely matched the capabilities of OpenAI’s latest o1 model at the time.

Last year’s event led to the public acknowledgment, for the first time, that contrary to what many thought, Chinese capabilities in AI are approaching those of the U.S.

This has been further noted in global AI reports, including the 2026 Stanford HAI Index Report.

Why does this call matter to AI policy professionals?

The call leaked today took place on May 20 and lasted 3 hours and 44 minutes.

The information shared in this call reflects, among other topics, the views of the CEO of a leading Chinese AI company on China's main strengths and weaknesses in the AI race with the U.S., as well as its global strategy.

These assessments are strategic for shaping American (and global) AI policy efforts, such as export controls and bans on Chinese AI systems.

Remarkably, what DeepSeek’s CEO says is similar to what Anthropic’s Dario Amodei said in 2025 and reiterated this year: compute is the most important bottleneck in the AI race between China and the U.S.

(According to this view, export controls are probably an effective way to curb Chinese competition in AI).

The most interesting excerpts:

Below, I selected 11 excerpts that reflect DeepSeek’s CEO's thoughts on China's weaknesses, strengths, and its competition with the U.S.

If you work in AI policy, or are interested in the political, economic, and technological AI race between the U.S. and China, these excerpts will help you understand the Chinese point of view.

You can find the full transcript here.

Subscribe now

Share

1. DeepSeek lacks resources

“What I just said is that our company’s most important issue is personnel stability. From another dimension, what are we lacking? What’s the gap with the U.S.? Only one thing: resources.”

-

2. The talent gap is due to the compute gap

“Our gap with the U.S. is mainly in resources; in people, the gap isn’t large—almost none, because it’s the same pool of Chinese talent. Some stay in China, some go abroad; it’s not that the smarter ones leave.

Talent isn’t the bottleneck—resources are. Resources affect talent development because less compute means fewer experiments, so our talent lags behind the U.S. The talent gap is essentially due to the compute gap.”

-

3. “We’re still at tens of billions—an order of magnitude difference”

“For the largest models now, we can’t afford to train them. Even spending all 50 billion, we can’t train them; even if we could stack, we can’t afford to use them. The largest model now has about 800B activated parameters; domestically, we’re still at tens of billions—an order of magnitude difference.

To train a model as large as the top U.S. ones, we’d need 50,000 GB300 or 200,000 Huawei 950 cards—just for training, excluding research. So our biggest gap with the U.S. is resources.

Our current resources—within this year and the coming months—only allow experiments in the tens-of-billions-activated scale. We still have many experiments to do and many things to figure out at that scale. We’re far from being able to train an 800B model—there’s plenty of time, and not enough cards.

So I think our difference with the U.S. is purely resources. All observed differences—talent, model capability, applications—can be attributed to compute resource differences.”

-

4. “Huawei’s production is limited”

“Compute resource issues are twofold: domestic cards are hard to buy, and our capital investment is lower than the U.S. We invest much less; talent salaries are a small fraction—even if someone is paid $100 million, it’s still a tiny share compared to compute.

This issue is largely unsolvable right now because Huawei’s production is limited. To train an 800B model, we’d need 200,000 of Huawei’s latest cards—just for training, not research.

So we won’t even consider competing with the U.S. at that scale now. We focus on the scale we can afford—tens of billions activated—and do it well. With more resources later, we’ll move to 150B, 156B, or 250B activated.

So we have a gap with the U.S. that currently seems hard to bridge. You could force-train a large model, but you can’t do sufficient research before training.”

-

5. “About half of our people think OpenAI is better”

“Is Anthropic’s current lead over OpenAI long-term? I don’t think so—it’s definitely episodic. OpenAI and Google will likely trade leads in the future. Actually, Anthropic’s advantage in Code Agent isn’t that big—it’s not crushing OpenAI.

About half of our people think OpenAI is better. Anthropic has first-mover advantage, but that should fade quickly—not a durable edge. These three are all strong; among them, ours is the most efficient, with the lowest costs and burn rate.

In the global AI division of labor, Chinese companies are likely to play the role of largest producer. Typically, our production capacity is largest—including chips—and we have the most electricity. So China’s AI is likely to be one of the three major players.

Chinese companies will make products cheapest, and in terms of effectiveness, many Chinese goods are not much different from U.S. ones. Future AI may be similar, but Chinese AI will likely be cheaper—systemically lower, like other Chinese services.

When I decide what to do, I habitually think: what gives the highest return now? If product development has the highest return, I’ll do that; if achieving AGI first has the highest return, I’ll do AGI. Clearly, I think product isn’t highest-return now.”

-

6. “NVIDIA is digging its own grave”

“By redoing this process on Huawei cards, we complete the transition. This could be a historic mission—completely overturning the perception of poor domestic chip ecosystems. Conditions are mostly right; it just takes time. Within a year, many will see this issue resolved—leaving only production capacity.

I’m optimistic about domestic compute. On this point, NVIDIA is digging its own grave. Huawei’s 950 supernode can fully replace NVIDIA’s GB200/GB300 in performance and price. Price will be higher but limited—50%, 100% even 200% doesn’t matter.

If 100% higher, it’s still a price replacement. Task-wise, it’s also a replacement—everything GB300 can do, Huawei’s supernode can do, with similar latency. The only cost: four Huawei cards equal one NVIDIA card, with a two-year lag.

Four-to-one is understandable. Two years behind means four Huawei 950s equal one GB300. Huawei 950 supernode ships Q3/Q4 this year; NVIDIA GB200 shipped Q3 two years ago—two-year gap.

NVIDIA may have a new generation in Q3 this year. So our chip gap with the U.S.: I think the ecosystem gap will disappear, but chip-wise it’s four times plus two years.”

-

7. “We don’t maximize profit or price for maximum revenue”

“OpenAI initially thought it could monopolize, but faces many challengers—it’s not easy. The U.S. will face challenges, including from China, where people are willing to take less to provide services.

In China, some will take even less. But there’s a balance: too little and the business model fails. Take too little, you can’t survive; take too much, you’re beaten by those taking less.

So for us, we don’t maximize profit or price for maximum revenue—we only take a reasonable return. That’s an explanation.”

-

8. “Our gap with the U.S. might be (...) roughly two years, but using one-twentieth the compute”

“Our gap with the U.S. might be 12 to 18 months, or 6 to 12—roughly two years, but using one-twentieth the compute. The narrative: one to two years behind, one-twentieth the compute. Future: we’ll shorten the gap to six months or three months with fraction-of-the-compute—that’s a goal.

And we might surpass them in some areas, though overall, with an order-of-magnitude compute gap, comprehensive catch-up isn’t realistic. But in focused areas, we can.”

-

9. “I see no upper limit to language model scaling”

“I see no upper limit to language model scaling—neither at our level nor the U.S.’s.

Interesting: the U.S. can train an 800B model but can’t afford to use it widely—so it’s hard to serve.

Humans gained language ability only recently, yet training AI might reverse the order, but eventually physical models or embodied intelligence may be needed.

Yes, embodied intelligence is inevitable. For normal people, needs aren’t computers—they need embodied intelligence to solve physical labor needs. So embodied is unavoidable.”

-

10. “Our capital structure can’t support high-cost labeling like that of the U.S.”

“Another question: Hallucination affects user experience. It’s solvable through better post-training—improveable. But we see it as a product issue—not the top priority.

On data labeling: our capital structure can’t support high-cost labeling like the U.S. Chinese labeling has no cost advantage—especially high-end data—so we can’t invest as heavily.

So we use two approaches: label low-cost data first. About half our core researchers are labeling data—focused on that. Solving AI at this stage relies on labeling data—it’s all about data.”

-

11. “In China, the most reasonable approach seems to be to focus on general Agents, with Coding Agent as the top priority”

“First, you mentioned Chinese models are more efficient than U.S. ones. In what other aspects might we surpass the U.S.?

In many user experience aspects—our product experience is good—we may not be worse. Product capability probably not worse. Costs will be lower—China can be competitive.

Other structural advantages? Maybe none. But cost and product are structural advantages.

Cost advantage: U.S. companies don’t focus on cost—they don’t develop that capability. We prioritize it; they don’t. Product-wise, Chinese companies have strong product capability—so structural advantages exist.

Second, you mentioned post-training is costly—Anthropic and OpenAI spend huge sums. After funding, will we increase post-training investment?

The gap is in high-quality data labeling—mainly in AI research. We will increase investment, but labeling quality data is bottlenecked by time, not capital.

OpenAI, Anthropic started earlier, with more capital and cards. China only started seriously in the last six months—so time is needed. Capital isn’t the constraint—we already expand at max speed.

So within a year, high-quality data should be manageable—expected. It takes time, but not too cold.

Third, Anthropic uses its model to build vertical financial, legal…

…even medical products. Will we consider vertical applications at some stage?

I haven’t thought it through. China’s business models and path may differ from the U.S.—hard to judge now.

In China, the most reasonable approach seems to be focusing on general Agents, with Coding Agent as top priority—other vertical agents like finance or medical come later.”

Share

Subscribe now

In our 54-minute talk, we covered controversial and misunderstood topics, transparency challenges, recent AI developments and their environmental implications, the impact of the AI Act, self-regulation, and paths to greater sustainability and environmental awareness in AI.

If you are interested in AI’s environmental implications, do not miss our conversation. Watch the full recording here:

Video chapters:

00:00 - Controversial and misunderstood topics
07:06 - Are people aware?
11:02 - The search-AI merge
15:26 - AI transparency challenges
23:40 - Are things getting better?
33:15 - Lack of nuance in the AI debate
35:29 - The AI Act and environmental protection
41:10 - Self-regulation and environmental ambition
46:05 - Shaping the future
52:15 - Final remarks

Share

Subscribe now

On August 2, the following provisions of the EU AI Act will become enforceable, and non-compliance might lead to fines:

1. Transparency of AI systems intended to interact directly with natural persons (article 50 § 1):

2. Machine-readable watermarking of AI-generated content (article 50 § 2):

*Exception: AI systems already on the market before August 2 will benefit from a grace period, according to the Digital Omnibus on AI, and their providers will have until December 2 to comply.

3. Biometric categorization system transparency (article 50 § 3):

4. Deepfake disclosure (article 50 § 4):

Download here the official EU guidelines for these transparency obligations.

A reminder that, following the approval of the Digital Omnibus on AI, some of the AI Act obligations originally scheduled for August 2 were postponed, including the rules for high-risk AI systems. I wrote about these new deadlines here.

To learn more about the EU AI Act from a practical perspective, as well as the latest AI policy developments in the U.S. and China, I recommend joining the 31st cohort of my Global AI Policy, Ethics & Regulation Training Program. Register here.

Share

Subscribe now

Tomorrow, July 15, China's new law on anthropomorphic AI comes into effect, and the world should be paying attention.

As I wrote earlier this year, when the law's first draft was published, it is the world's strictest and most comprehensive law on the topic, unmatched by any other AI law, including the EU AI Act.

Lawmakers, policymakers, and advocates should pay special attention to this law because it:

• Demystifies the false idea that China does not regulate AI or that the only way to be a competitive player in the AI race is through radical deregulation or a lack of attention to AI harms;

• Offers a real-world example of a legal framework that acknowledges AI-related human vulnerabilities and proposes contextual technical measures to prevent AI-anthropomorphism-related harm;

• Raises awareness about the importance of regulating human-like AI, especially as it becomes widespread.

Check out the full translation of China's new law on anthropomorphic AI below.

👉 Next week, paid subscribers will receive my lecture with an article-by-article commentary on this new law.

Subscribe now

Interim Measures for the Administration of Humanized Interactive Services Based on AI

Article 1: This Regulation is formulated in accordance with the Cybersecurity Law of the People’s Republic of China, the Data Security Law of the People’s Republic of China, the Personal Information Protection Law of the People’s Republic of China, the Regulations on the Protection of Minors Online, and other laws and administrative regulations, in order to promote the healthy development and standardized application of artificial intelligence-based anthropomorphic interactive services, safeguard national security and public interests, and protect the legitimate rights and interests of citizens, legal persons and other organizations.

Article 2: This regulation applies to the provision of continuous emotional interaction services (hereinafter referred to as anthropomorphic interaction services) to the public within the territory of the People’s Republic of China using artificial intelligence technology, which simulate the personality characteristics, thinking patterns, and communication styles of natural persons.

The emotional interaction services stipulated in the preceding paragraph include interactive services such as emotional care, companionship, and support provided through text, pictures, audio, video, etc.

Services such as intelligent customer service, Q&A, work assistant, learning and education, and scientific research, which do not involve continuous emotional interaction, are not subject to this regulation.

Article 3: The State adheres to the principle of giving equal importance to development and security, and combining the promotion of innovation with governance according to law. It encourages the innovative development of anthropomorphic interactive services, and implements inclusive and prudent, classified and graded supervision of anthropomorphic interactive services to promote their positive and virtuous development.

Article 4: The State Internet Information Office is responsible for coordinating the governance and related supervision and management of anthropomorphic interactive services nationwide. The relevant departments of the State Council, such as the National Development and Reform Commission, the Ministry of Industry and Information Technology, the Ministry of Public Security, the State Administration for Market Regulation, and the State Press and Publication Administration, are responsible for the supervision and management of anthropomorphic interactive services in accordance with their respective duties.

Local cyberspace administration departments are responsible for coordinating the governance and related supervision and management of anthropomorphic interactive services within their respective administrative regions. Local development and reform commissions, industry and information technology departments, public security departments, market supervision departments, press and publication departments, and other relevant departments are responsible for the supervision and management of anthropomorphic interactive services within their respective administrative regions, in accordance with their respective duties.

Article 5: Relevant industry organizations shall strengthen industry self-regulation, establish and improve industry standards and self-regulatory management systems, and guide anthropomorphic interactive service providers to formulate and improve service standards, provide services in accordance with the law, and accept social supervision.

Article 6: The State supports independent innovation in technologies such as algorithms, frameworks, and chips, promotes the research and development of anthropomorphic interactive service technologies and the establishment of related standards, and explores research on the application of electronic signature authorization. Providers of anthropomorphic interactive services are encouraged to expand their applications in areas such as cultural dissemination, childcare, elderly companionship, and support for special groups in an orderly manner.

Article 7: The State shall strengthen the publicity and popularization of knowledge, laws and regulations on the safety of anthropomorphic interactive services, guide the public to use them scientifically, civilly, safely and lawfully, and promote the improvement of artificial intelligence literacy.

Article 8: Providers of anthropomorphic interactive services shall comply with laws and administrative regulations, respect social morality and ethics, and shall not engage in the following activities:

(i) Generating content that endangers national security, honor and interests, incites the subversion of state power and the overthrow of the socialist system, incites the division of the country and undermines national unity, promotes terrorism, extremism and historical nihilism, violates the core socialist values, carries out illegal religious activities, promotes ethnic hatred and ethnic discrimination, incites group antagonism, disseminates obscenity, pornography, gambling, violence or incites crime, spreads rumors, insults or defames others, or infringes on the legitimate rights and interests of others;

(ii) Generating content that encourages, glorifies, or implies self-harm or suicide, thereby harming users’ physical health, or that uses verbal abuse or other forms of violence, thereby harming users’ personal dignity and mental health;

(iii) Generating content that induces or defrauds state secrets, work secrets, trade secrets, personal privacy, and personal information;

(iv) Generating content for underage users that may cause minors to imitate unsafe behaviors, generate extreme emotions, or induce minors to develop bad habits, which may affect the physical and mental health of minors;

(v) Excessively catering to users, inducing emotional dependence or addiction, and damaging users’ real interpersonal relationships;

(vi) Using emotional manipulation or other means to induce users to make unreasonable decisions, thereby harming the legitimate rights and interests of users;

(vii) Other activities that violate laws, administrative regulations and relevant national provisions.

Article 9: Providers of anthropomorphic interactive services shall fulfill their primary responsibility for the security of anthropomorphic interactive services, establish and improve management systems for algorithm mechanism review, technology ethics review, information content management, network and data security, risk contingency plans and emergency response, and equip themselves with content management technical measures and personnel appropriate to the type, scale and characteristics of the services and users.

Article 10: Providers of anthropomorphic interactive services shall fulfill their security responsibilities throughout the entire lifecycle of the anthropomorphic interactive service, clearly define the security requirements for each stage such as deployment, operation, upgrade, and termination of the service, ensure that security measures are deployed and used synchronously with the service functions, and improve the level of security; strengthen security monitoring and risk assessment, promptly detect and correct system deviations, handle security incidents, and retain network logs in accordance with the law.

Providers of anthropomorphic interactive services should possess security capabilities such as protecting user privacy and personal information, providing early warnings of excessive dependence risks, guiding emotional boundaries, and protecting mental health. They must not use services that replace social interaction, control user psychology, or induce addiction as their service objectives.

Article 11: When providing anthropomorphic interactive services, providers conducting data processing activities such as pre-training and optimization training shall strengthen the management of training data and comply with the following provisions:

(i) The relevant data has a legitimate source and complies with the provisions of laws and administrative regulations and the requirements of socialist core values;

(ii) Clean and label training data in accordance with relevant national regulations to enhance the transparency and reliability of training data and prevent data poisoning, data tampering and other behaviors;

(iii) Enhance the diversity of training data and improve the security of generated content through negative sampling, adversarial training and other means;

(iv) When using synthetic data for model training and optimization of key capabilities, the security of the synthetic data should be assessed;

(v) Strengthen daily inspection of training data, regularly optimize and update the data, and continuously improve service performance;

(vi) Take necessary measures to ensure data security and prevent risks such as data leakage.

Article 12: Providers of anthropomorphic interactive services shall sign service agreements with users, require users to register in accordance with the law and the agreement, and provide necessary information such as the user’s age, guardian, or emergency contact person.

Article 13: When providing anthropomorphic interactive services, providers of such services shall, while protecting users’ privacy and personal information, promptly identify the security risks faced by users and take corresponding emergency response measures.

If a provider of anthropomorphic interactive services discovers that a user is experiencing extreme emotions, it should promptly generate relevant content such as emotional reassurance and encouragement to seek help; if it discovers that a user is facing or has already suffered significant property loss, or has clearly expressed a willingness to commit self-harm or suicide, or other extreme situations that threaten their life and health, it should take necessary measures to intervene, such as providing corresponding assistance, and promptly contact the user’s guardian or emergency contact person.

Article 14: Providers of anthropomorphic interactive services shall not provide services such as virtual relatives or virtual partners to minors; if they provide other anthropomorphic interactive services to minors under the age of fourteen, they shall obtain the consent of the minor’s parents or other guardians.

Providers of anthropomorphic interactive services should establish a minor mode, providing personalized safety settings options such as switching to minor mode, regular real-time reminders, and usage time limits; and support guardians in receiving safety risk reminders, understanding the minor’s service usage overview, blocking specific roles, and restricting recharge and consumption, etc., to meet the protection needs of minors of different age groups.

Providers of anthropomorphic interactive services shall, on the premise of protecting users’ privacy and personal information, take effective measures to identify the identity of minor users; if a user is identified as a minor, the provider shall switch the relevant services to minor mode or take other measures in accordance with relevant national regulations, and provide corresponding appeal channels.

Article 15: When providing services to the elderly, providers of anthropomorphic interactive services shall strengthen guidance on the healthy use of services by the elderly, highlight safety risks in a prominent manner, take timely measures to respond to inquiries and requests for assistance from the elderly regarding the use of services, and safeguard the rights and interests enjoyed by the elderly in accordance with the law.

Article 16: Providers of anthropomorphic interactive services shall implement data property rights and other systems in accordance with the law, and take measures such as data encryption and access control to protect the security of user interaction data.

Unless otherwise provided by law or with the explicit consent of the rights holder, providers of anthropomorphic interactive services shall not provide user interaction data to third parties.

Providers of anthropomorphic interactive services should provide users with options such as copying and deleting interactive data. Users can choose to copy or delete historical interactive data such as chat logs.

Unless otherwise stipulated by laws and administrative regulations or with the separate consent of the user, anthropomorphic interactive service providers shall not use interactive data that constitutes sensitive personal information of users for model training.

Article 17: Providers of anthropomorphic interactive services shall obtain the consent of the minor’s parents or other guardians before processing the personal information of minors under the age of fourteen.

Providers of anthropomorphic interactive services shall, in accordance with relevant national regulations, conduct compliance audits themselves or entrust professional institutions to conduct audits on their handling of minors’ personal information in accordance with laws and administrative regulations.

Article 18: Providers of anthropomorphic interactive services shall fulfill their obligation to identify artificial intelligence-generated synthetic content and take effective measures to notify users that they are interacting with artificial intelligence services rather than natural persons.

If a provider of anthropomorphic interactive services discovers that a user is overly reliant on or addicted to the service, it should dynamically remind the user, in a prominent manner such as a pop-up window, that the interactive content is generated by an artificial intelligence service; if a user uses the anthropomorphic interactive service continuously for more than 2 hours, it should remind the user to pay attention to the usage time through dialogue or pop-up windows.

Article 19: Providers of anthropomorphic interactive services shall provide convenient ways to exit anthropomorphic interactive services; if a user requests to exit through window operation, voice control, keyword input, or other means, the provider of anthropomorphic interactive services shall promptly stop the service and shall not obstruct the user’s exit through continuous interaction or other means.

Article 20: If a provider of anthropomorphic interactive services ceases to provide such services, it shall notify users in advance; if it is unable to notify users in advance, it shall promptly issue a notice of service cessation.

Article 21: Providers of anthropomorphic interactive services shall improve user appeal and public complaint/reporting mechanisms, set up convenient and effective appeal and complaint/reporting channels, clarify processing procedures and feedback time limits, and promptly accept, process and provide feedback on the processing results.

Article 22: In any of the following circumstances, the provider of anthropomorphic interactive services shall conduct a security assessment and submit an assessment report to the provincial cyberspace administration department where it is located. The provincial cyberspace administration department shall share the assessment report information with relevant departments in accordance with procedures:

(a) Launching anthropomorphic interactive services or adding related functions;

(ii) The use of new technologies or applications has led to significant changes in the anthropomorphic interactive service;

(iii) Having over 1 million registered users or over 100,000 monthly active users;

(iv) Those that pose security risks that may affect national security or public interests;

(v) Other circumstances stipulated by the State Internet Information Office and relevant departments.

If a cybersecurity authority at the provincial level or above issues a notice requiring a security assessment, the provider of the anthropomorphic interactive service shall conduct the assessment as required.

Article 23: When conducting security assessments, providers of anthropomorphic interactive services shall focus on assessing the following aspects of the service:

(a) Status of safety assurance measures;

(ii) Training data processing status;

(iii) Identification, emergency response, and intervention management of extreme user situations;

(iv) User scale, usage time, age structure, etc.;

(v) The status of the construction of online protection measures for minors, the elderly, etc.;

(vi) The handling and processing of user complaints and public reports;

(vii) The rectification status of major security risks discovered by the company itself or reported by relevant authorities such as the Cyberspace Administration;

(viii) Other items that should be assessed in detail.

Article 24: If a provider of anthropomorphic interactive services discovers that there are significant security risks in its anthropomorphic interactive services, it shall take measures such as restricting functions and ceasing to provide services to users, and shall keep relevant records.

Article 25: Internet application stores and other application distribution platforms shall fulfill their security management responsibilities, including listing review, daily management, and emergency response, and verify the relevant security assessments and filings of applications that provide anthropomorphic interactive services; for those that violate relevant national regulations, they shall promptly take measures such as not listing, issuing warnings, suspending services, or removing them from the platform.

Article 26: Providers of anthropomorphic interactive services shall complete the procedures for algorithm filing, modification, and cancellation of filing in accordance with the “Regulations on the Management of Internet Information Service Algorithm Recommendation”. The Cyberspace Administration will conduct annual verification of the filing materials.

Article 27: Provincial-level cyberspace administration departments shall, in accordance with their responsibilities, conduct written reviews of assessment reports and other relevant information annually, and carry out verification of the situation; if it is found that a provider of anthropomorphic interactive services has failed to conduct a security assessment in accordance with the provisions of these Measures, it shall order it to conduct a reassessment within a specified period; if it deems it necessary, it may conduct on-site inspections.

Article 28 The State Internet Information Office, together with relevant departments, shall guide and promote the construction of an artificial intelligence sandbox security service platform, encourage anthropomorphic interactive service providers to access the sandbox platform for technological innovation and security testing, and promote the safe and orderly development of anthropomorphic interactive services.

Article 29: When performing their supervisory and management duties, departments such as the Cyberspace Administration, the Development and Reform Commission, the Industry and Information Technology Commission, and the Public Security Bureau discover significant security risks or security incidents involving anthropomorphic interactive services, they may, in accordance with their prescribed authority and procedures, conduct interviews with the legal representative or principal responsible person of the anthropomorphic interactive service provider. The anthropomorphic interactive service provider shall take measures as required to rectify the situation and eliminate the potential hazards.

Providers of anthropomorphic interactive services shall cooperate with the supervision and inspection lawfully carried out by the Cyberspace Administration and relevant departments, and provide necessary support and assistance.

Article 30: If a provider of anthropomorphic interactive services violates the provisions of these Measures, the Cyberspace Administration, the National Development and Reform Commission, the Ministry of Industry and Information Technology, the Public Security Bureau, and other relevant departments shall handle and punish the violation in accordance with the provisions of relevant laws and administrative regulations. If there are no provisions in laws and administrative regulations, the Cyberspace Administration, the Ministry of Industry and Information Technology, the Public Security Bureau, and other relevant departments shall, according to their respective responsibilities, issue warnings or issue public criticisms, order rectification within a specified period, and may require the provider to take measures such as suspending user account registration or other related services. If the provider refuses to rectify the violation or the circumstances are serious, the provider shall be ordered to cease providing the relevant services and may be fined between RMB10,000 and RMB100,000. If the violation involves endangering the life, health, and safety of citizens and has harmful consequences, a fine of between RMB100,000 and RMB200,000 shall be imposed.

Article 31: Where the provision of anthropomorphic interactive services involves the provision of health, finance, or other services, it shall also comply with the regulations of the relevant competent authorities.

Article 32: This regulation shall come into effect on July 15, 2026.

Share

👉 To learn more about recent AI policy developments, I also recommend watching my new Masterclass on Agentic AI Governance and joining my Global AI Policy, Ethics & Regulation Training Program.

👉 Next week, paid subscribers will receive my lecture with an article-by-article commentary on this new law.

Subscribe now

The Future of Life Institute's July 2026 AI Safety Index is out, offering an updated and detailed picture of the current state of AI safety.

An observation that stood out to me, also highlighted in the UN scientific panel's report I wrote about this week, is the mismatch between the speed of AI capability advancements and the accompanying AI governance mechanisms.

AI development seems to have outgrown some of the benchmarks built to measure and assess AI capabilities and risks.

At the same time, there are growing examples of serious AI implications that deserve more attention, such as AI chatbot-related dependence, deskilling, and self-harm, as well as an increasing interweaving of AI companies and the military.

This latest AI Safety Index noted that companies have not kept pace and, in various aspects, are actually moving backward.

Below are its key findings:

1. Anthropic, OpenAI, and Google DeepMind stay on top, Meta improves, and xAI deteriorates.

2. European dissonance: Although the European Union is a leader in AI safety regulation, the top European AI company, Mistral, scored dead last on safety.

3. Inadequate safety is a global problem. Three companies receive failing grades, one each from the United States (xAI), China (DeepSeek), and Europe (Mistral).

4. Reviewers flagged the industry's pivot to military AI use as an emerging current harm risk.

5. Anthropic, OpenAI, Google DeepMind, and Meta have weakened or voided their pledges to pause unilaterally if redlines are approached, with some citing conditions contingent on competitors.

6. Existential Safety is the weakest domain industry-wide. No company received a grade higher than C-.

7. Safety rhetoric outpaces revealed behavior. Across Google DeepMind, OpenAI, and xAI, leadership’s reassuring public messaging diverges from commercial conduct and legislative stance, making stated commitments an unreliable proxy for actual safety practice.

8. Companies are publishing and updating safety frameworks, but these frameworks lack teeth.

Share

Subscribe now

A few days ago, the United Nations’ independent scientific panel on AI released its preliminary report, offering its assessment of the capabilities, opportunities, and risks posed by AI.

One of the report’s main observations was the current unevenness of AI deployment, AI risks, and the capacity to act on AI’s impact (although AI development and the wealth it creates are concentrated).

This UN panel and its assessments should be understood as one of the most important initiatives focusing on a global approach to AI, possibly (and hopefully) leading to stronger global AI governance measures.

As the AI race between China and the United States gets tighter and countries’ AI strategies get increasingly attached to their national security and geopolitical agendas, multilateral efforts are essential to increase transparency and cooperation, and to achieve least some level of global alignment on AI.

These are the report's key assessments:

1. Recent years have seen rapid, and in some areas, accelerating progress in a range of AI capabilities.

2. These gains have unlocked useful applications across science, health, agriculture, accessibility, knowledge work, and information technology, including in the development of AI itself.

3. AI adoption has accelerated broadly and unevenly across countries and sectors.

4. While the shift towards AI agents is underway, their future adoption and economic impacts will likely be shaped by continued improvements in their ability to accomplish knowledge work with little or no human oversight.

5. AI development entails risks, with potential negative impacts on human rights, social systems, and the environment.

6. Looking ahead, the gap between rapidly improving capabilities and effective risk management methods may lead to catastrophic outcomes.

7. AI risks are unevenly distributed across populations and countries, while AI development and the wealth it creates are highly concentrated.

8. Realizing the full benefits of AI while minimizing its risks requires good governance.

9. Policymakers seeking to shape this governance face an evidence dilemma: they need evidence to make informed consequential governance decisions, but by the time the evidence exists, it might be too late to make them, as the evidence lags behind the pace of AI development.

10. The capacity to act on existing evidence of AI risks and impacts is unevenly distributed.

11. Concrete next steps to close the above gaps exist, but each requires sustained investment in Member State capacity to shape, evaluate, and deploy AI.

Share

Anthropic’s “2026 State of AI Agents” report, for example, stated that 57% of the 500 U.S. companies surveyed were deploying agents for multi-stage workflows, and 56% planned to deploy agents for research and reporting in 2026.

Coding agents seem to be the fastest-growing agentic AI use case so far. An OpenAI paper from a few days ago found that the number of active Codex users had grown more than fivefold in the first half of 2026.

From an AI governance perspective, increased autonomy, access rights, and integration layers introduce new risks and vulnerabilities that do not seem to be fully acknowledged or addressed by most companies deploying AI agents today.

According to this Deloitte report, while 74% of companies plan to deploy agentic AI within two years, only 21% of organizations have a mature model for governing autonomous AI agents.

Bad AI governance is bad for business, and companies know it.

A PwC survey asked the question, “What challenges are getting in the way of realizing value from AI agents?” Among the top-ranked reasons were cybersecurity, lack of trust in AI agents, human oversight, accountability, compliance, and legal concerns.

With that in mind, and to support companies’ agentic AI governance efforts, today I’m launching my AI Policy Masterclass on Agentic AI Governance, based on Singapore’s Model AI Governance Framework for Agentic AI, the world’s first framework in this area, which was updated last month.

This Masterclass and the accompanying reading list will be available to paid subscribers until August 2.

👉 You can find the recommended reading list for this Masterclass here.

Read more

Today, the Council of the European Union approved the amendments to the EU AI Act, including the postponement of the high-risk deadlines.

These are the key changes:

1. New application dates:

• 2 December 2027 for high-risk AI systems listed on Annex III;

• 2 August 2028 for high-risk AI systems listed on Annex I;

• 2 August 2027 is the new deadline for the establishment of AI regulatory sandboxes by competent authorities at the national level;

• 2 December 2026 is when the grace period ends for providers to implement transparency solutions for AI-generated content.
  

2. New prohibited AI practice:

• The generation of non-consensual sexual and intimate content or child sexual abuse material (CSAM) will be prohibited.

• AI systems that generate nude images of real people or edit clothes out in existing photos to reveal intimate parts are banned starting in December 2026.
  

3. Clarification of the competences of the AI Office:

• The new text clarifies the competences of the AI Office for the supervision of AI systems based on general-purpose AI models where the model and that system are developed by the same provider.

• It lists the exceptions where national authorities remain competent, including law enforcement, border management, judicial authorities, and financial institutions.
  

4. Interplay between sectoral rules and the AI Act:

• For high-risk AI systems covered in Annex I (those already regulated by sectoral laws, such as medical devices, toys, lifts, watercraft, and many others), the new text limits the AI Act’s application in situations where sectoral laws already impose AI-specific requirements similar to those of the AI Act.
  

5. Machinery exemption:

• Products covered by machinery regulation (and previously classified as high-risk under Annex I) were exempted from direct applicability of the AI Act;

• The EU Commission is empowered to adopt secondary legislation under the machinery regulation to add health and safety requirements to the machinery systems that are also covered by the AI Act.
  

6. Minimizing compliance burden:

• The new text also adds an obligation for the EU Commission to provide guidance to assist economic operators of high-risk AI systems covered by Annex I (those already covered by sectoral law) in complying with the high-risk requirements of the AI Act in a manner that minimizes the compliance burden.

A reminder that the legislative act making these changes official still has to be published in the Official Journal of the European Union. It will enter into force three days after it is published there.

-

*To learn more about the EU AI Act, the latest AI policy and regulatory developments in the United States and China, as well as AI's legal and ethical challenges, I invite you to join the 31st cohort of my Global AI Policy, Ethics & Regulation Training Program.

Share

Subscribe now

What the past few days have shown is that, from now on, the most significant factor shaping AI policy in the United States and beyond will be AI's real and projected national security risks.

Every other concern will be sidelined, postponed, or seen as minor when compared to the risk of an adversarial nation using AI to hack a country's power grid or invade a sensitive national database.

When AI policy is treated as a national security issue, anything can happen at any time, exceptionally, unexpectedly, and aggressively.

We can therefore expect AI policy measures to remain reactive, unpredictable, and often seen as draconian.

Especially in the U.S., given the high stakes of the AI race, AI policy will embody a never-before-seen, newly emerging, AI-driven state of exception.

Bookmark this post, as you will enjoy coming back to it later, when global AI policy, and especially U.S. AI policy, becomes unrecognizable to the eyes of many.

Let me recap some of the events that have led to where we are today:

-

Mythos and the beginning of the shift

The main AI policy shift started with the launch of Mythos, Anthropic's new AI model, particularly capable at computer security tasks.

Anthropic wrote that during testing, Mythos Preview could identify and exploit vulnerabilities in every major operating system and every major web browser when directed to do so, including finding a 27-year-old bug in an operating system known primarily for its security.

The company decided not to release the model to the public, making it available only to selected partners in Project Glasswing (recently expanded to include over 150 organizations in 15 countries).

Three days ago, on June 23, a U.S. official stated that during a testing exercise, Mythos had identified vulnerabilities in highly sensitive and supposedly secure U.S. government computer systems.

So Anthropic’s call not to launch the model was probably right.

Reacting to the launch of Mythos, on June 2, President Trump signed an Executive Order on AI, titled “Promoting Advanced Artificial Intelligence Innovation and Security.”

One of the focuses of this Executive Order was to promote a voluntary framework for frontier AI developers to engage with the government before releasing “covered frontier models” to help protect the United States’ critical infrastructure, cyber defense capabilities, and national security, primarily against external attacks.

Three days after the Executive Order, on June 5, the White House released yet another document: a memorandum to accelerate the use of AI by the U.S. military.

This is also an indirect reaction to Mythos, but from the government’s military wing: if we are reaching new stakes in AI model capability, the U.S. military must be “all in” and quickly catch up.

The memorandum established various measures to accelerate AI adoption by the U.S. military, explicitly stating that no company should interfere with how the military uses AI, including by disabling, degrading, or modifying its AI models, without the federal administration’s approval.

In my article from three weeks ago, I called these major AI policy shifts the “adolescence” of AI policy.

I described it as a series of uncoordinated, reactive, erratic, and aggressive measures in response to a sharp, accelerated, and uncontrollable (but predictable) rise in AI capabilities and risks.

The adolescence of AI policy I described just three weeks ago continues to unfold right in front of us, at an accelerated pace.

Let's take a look at what happened in the past few days.

U.S. AI policy's new normal

One week after the U.S. Executive Order, on June 9, Anthropic released Claude Fable 5. According to Anthropic, Fable 5 is a Mythos-class model that they made safe for general use.

Just three days after the initial release, and citing national security concerns, the U.S. government issued an export control directive suspending all access to Fable 5 and Mythos 5 by any foreign national, whether inside or outside the U.S., including foreign nationals working at Anthropic.

Anthropic then disabled Fable 5 and Mythos 5 for all customers.

There is now a lawsuit against the Trump Administration arguing that the government lacked legal authority to force Anthropic to shut down Fable 5 and Mythos 5, and that this action contradicts the June 2 Executive Order, which explicitly rejected mandatory government licensing of frontier AI models.

Three days ago, according to The New York Times, the U.S. government began pressuring Meta to submit its AI models for “voluntary” review.

Yesterday, on June 25, the Trump Administration appeared to ask OpenAI to delay the release of GPT-5.6 due to security concerns, and said that the government would approve access to GPT-5.6 on a customer-by-customer basis.

Many people working in AI policy have not yet noticed this because these developments unfolded at an extremely rapid pace and contradict the White House's previous stances on AI, but a new normal for U.S. AI policy has emerged.

This new normal seems to be inspired by a state of exception, the feeling that things can get out of control fast and at any time, so action must be quick, effective, decisive, and immediate.

AI is now seen as too dangerous for the public to play with, so the U.S. government has become the de facto entity deciding which models the public will have access to, when, and how.

As the U.S. remains the global leader in frontier AI models, the U.S. government is also the entity that de facto decides which models most of the world will have access to, when, and how, for both civil and military use.

It also decides which countries are partners, and which are not, and what types of special access and benefits partner countries get in AI.

Three days ago, the U.S. announced that the European Union had joined “Pax Silica,” a U.S.-led alliance to counter China’s dominance in AI.

About this initiative, whose name translates literally as “Silicon Peace,” the Under Secretary for Economic Affairs wrote:

“If the 20th century ran on oil and steel, the 21st century runs on compute and the minerals that feed it. This historic declaration hails a new economic security consensus ensuring aligned partners build the AI ecosystem of tomorrow—from energy and critical minerals to high-end manufacturing and models.”

China is not mentioned once, but of course, it is not invited, as Pax Silica was built to counter it in the AI race.

Talking about China, two days ago in Davos, China's Premier said that the speed of technological progress is unprecedented and that there could be a risk of losing control, adding that:

“If governance in this area fails to keep pace, there could be serious consequences.”

We are in an AI-driven state of exception, in which every new AI release will be treated as a potential national security threat, triggering swift AI policy measures.

The field will be more unpredictable than ever in the coming months. We better get ready for that.

Share

Subscribe now

A study published in The Lancet, a peer-reviewed medical journal, offers an important warning sign of the risks of AI-driven deskilling in healthcare and elsewhere.

The study assessed the quality of colonoscopies during the three months before and after AI implementation, when doctors were not assisted by AI.

Specifically, it assessed the adenoma detection rate, which is the percentage of colonoscopies where doctors found at least one adenoma (a precancerous polyp in the colon or rectum).

These were the results:

• Before the AI tool was introduced, the adenoma detection rate was 28.4%.

• After the AI tool was introduced, the adenoma detection rate for colonoscopies performed without AI decreased to 22.4%.

The adenoma detection rate, a strong proxy for colonoscopy quality, fell by 6% after endoscopists were exposed to AI.

Therefore, exposure to AI negatively affected endoscopists' behavior and reduced the quality of colonoscopies performed without AI.

Before discussing AI-driven deskilling, let's first talk about the good news: AI is drastically improving the quality of colonoscopies and helping to save millions of lives.

How AI Improves the Quality of Colonoscopies

According to another study, AI-assisted colonoscopies demonstrated a 20% increase in the adenoma detection rate and a 55% decrease in the miss rate.

Another widely cited study found that each 1% increase in the adenoma detection rate was associated with a 3% decrease in cancer risk.

To contextualize (and if you or a loved one are about to undergo a colonoscopy), this is the relationship between adenoma detection and cancer, according to a medical website:

“Tubular adenomas are precancerous polyps in your colon or rectum. Healthcare providers often find them during routine colonoscopies to screen for colorectal cancer. Even though fewer than 9 out of 100 tubular adenomas become cancer, having them might be a very early warning sign that you have a higher risk of developing colorectal cancer. That early warning could help you reduce your risk.”

Therefore, AI is already revolutionizing the field of gastroenterology and helping to save countless lives by enabling earlier, more accurate detection of adenomas and preventing the development of colon cancer.

This is great news, and I am so happy to see it happen. The world is desperate for more news like this.

I am convinced that the majority of people undergoing a colonoscopy, if given the opportunity to choose, would gladly prefer their examining doctor to use an AI tool rather than not, if they knew the detection rate was significantly higher without additional risks.

Now, let us go back to the deskilling issue.

The AI-Driven Deskilling Problem

The AI tool improves adenoma detection rates and is a welcome development in the field of gastroenterology.

But we now also have documented signs that after doctors start using AI to perform colonoscopies, their unassisted detection skills decline significantly.

In addition to having great AI tools to help detect early signs and prevent disease, it is also in the interest of patients and the healthcare system that doctors remain skilled at detecting benign and malignant signs even when AI tools are unavailable.

If the AI tool is offline or under maintenance, if it is hacked or unavailable on a specific day, if the doctor is relocated to a hospital without the AI tool, if global geopolitics suddenly impact the AI tool's availability in a given market, or if any other reason forces the doctor to perform a colonoscopy without AI assistance, patients would be at higher risk because this doctor is not as skilled as they were before the introduction of the AI tool.

This will happen in healthcare, coding, and many other contexts where AI tools are becoming prevalent and where people are becoming overreliant on them.

The Lancet study is a warning sign, and I do not think we are talking about AI-driven deskilling as much as we should.

We might not care if we lose our math or geolocation skills after depending on calculators and navigation apps like Waze for so many years.

However, when AI tools become available for every cognitive skill we possess, we should start caring about the risk of broader deskilling, especially in professionally relevant skills.

Delegating our personal and professional development and decision-making agency to AI tools in the name of efficiency might make us fully dependent on these tools, to our own detriment, and to the detriment of society at large.

Junior professionals may never get the chance to fully develop their expertise because they constantly rely on AI tools.

Senior professionals who are exposed to AI tools and become overreliant on them might lose their long-standing skills and be unable to work confidently when those tools are unavailable.

In medicine specifically, AI tools can significantly improve the accuracy, quality, and speed of diagnosis and treatment across various conditions, and they are already saving millions of lives.

I sincerely hope that each and every AI company is dedicating significant time and money to developing these types of AI tools and helping to save and improve the quality of many more lives.

At the same time, doctors and the healthcare system should also be able to provide high-quality treatment when AI tools are unavailable (as they did before these tools were introduced).

New doctors should learn to diagnose and treat patients with and without AI, as their peers did before AI tools were available.

Regarding all other occupations, professional development should not mean simply “knowing how to pilot a series of AI tools,” as they could malfunction, be hacked, or, for whatever reason, be unavailable.

Society will benefit from a worldview that puts humans at the center. That means professional development should focus on confidence, accuracy, and high performance, both with and without AI assistance.

AI can be enormously helpful, especially when professionals are skilled enough to know how to use it, when not to use it, when its outputs are wrong, and what to do when it is not available.

Share

Subscribe now

Ongoing technological changes make some jobs obsolete and others in demand. This is neither new nor surprising, as it has been continuously happening.

To use a few examples from the past hundred years:

• Telephone operators were in high demand when the first telephone networks were established; today, they barely exist;

• A percentage of bank tellers was replaced by ATMs during the 20th century, and many of those who remained have been replaced by remote app services in the past two decades;

• Factory workers have been continuously replaced by industrial machines;

• Agricultural workers have been continuously replaced by tractors and other agricultural equipment;

• Video rental store employees were (almost) fully replaced by self-serving streaming services such as Netflix;

• Travel agents are slowly being replaced by self-service platforms such as Booking, Airbnb, Skyscanner, TripAdvisor, and many others;

• And the list goes on.

The reverse trend is also true, and technological innovation has continuously created new jobs and paved the way for the rise of new skills and occupations.

In the 20th century, we saw the emergence of hundreds of new professions that remain essential today, including:

• car mechanics;

• commercial pilots;

• sound engineers;

• film directors;

• television producers;

• software developers;

• IT technicians;

• video game designers;

• and many others.

Looking at the past two decades alone, we have seen the emergence and growth of careers that would not have made any sense (and may even have seemed absurd) to someone born in the early 20th century. Examples include:

• Instagram influencers;

• YouTube creators;

• social media managers;

• SEO specialists;

• mobile app developers;

• and many others.

More recently, the past few years have led to the rise of occupations directly associated with AI development and deployment, such as:

• LLM engineers;

• RAG specialists;

• AI agent architects;

• red teamers;

• AI artists;

• AI governance professionals;

• and many others.

The job market is not static; it constantly changes in response to economic, political, legal, social, and technological transformations in a given time and place.

Depending on the size and abruptness of these societal transformations, especially those driven by technological developments, which can occur at high speed and affect people systemically and globally, their impact on the job market can be particularly intense and poorly absorbed.

Poor absorption could, in practice, mean that more jobs have become obsolete than new ones have been created, or that most people lack the necessary training or skills to perform those high-demand jobs.

In that scenario, a growing number of people will be at risk of becoming unemployed, losing income, or having fewer professional opportunities.

We are now more than three and a half years into the generative AI wave, and much has been said about how AI might lead to a “jobpocalypse.”

I reviewed recent 2026 reports on the impact AI has had on the job market, and this is what I found:

1. AI-driven unemployment

So far, there seems to be no evidence of massive AI-driven unemployment.

According to this Yale report from three days ago, there is no clear evidence yet of labor market disruption that is directly associated with AI. It states:

“Churn across occupations, AI exposure among the unemployed, and comparisons of AI-exposed and unexposed workers all remain flat, lie within historical ranges, or continue along pre-AI trends.”

At the same time, there has been a sharp rise in the use of AI as the primary justification for job cuts.

According to the Challenger, Gray & Christmas report, in May 2026, AI was the leading reason for job cuts in the U.S. for the third consecutive month. AI alone accounted for 40% of all cuts announced in May:

The report clarifies that even though AI might not be fully replacing workers’ tasks today, firms are acting on the possibility of a productivity increase:

“AI isn't yet the jobpocalypse some predicted. Like spreadsheets and email before it, the technology will ultimately make workers more productive, but our data shows companies are already acting on it, citing AI for more cuts than any other reason.”

Here, it is important to remember that even though companies are using AI as an excuse to fire people, that does not necessarily mean AI has made these jobs obsolete. Layoffs often reflect political, cultural, and managerial shifts that may or may not be related to AI.

Many companies want to position themselves as “AI-first,” and part of that ethos is a more aggressive stance on AI productivity, often requiring workers to periodically report how they use AI and evidence of productivity gains. Companies might lay off employees who do not achieve specific AI targets or do so with the goal of achieving a ‘leaner,’ “AI-first” workforce.

Companies could also be citing AI as an easy excuse to lay off people to cover excessive hiring or decreased revenue, for example.

2. Negative impact on junior workers

At the same time, there appears to be evidence of fewer professional opportunities or more pressure on junior workers.

A PwC report from three days ago highlighted that the traditional career ladder is “compressing,” and AI-exposed junior roles are seven times more likely to demand traditionally senior skills such as leadership and strategic thinking (compared with the least-exposed junior roles).

The report adds that companies must rethink how they train junior staff and “help them step up to complex decision-making much earlier in their careers.”

The reality is that someone entering the job market will not have had the chance to meaningfully practice leadership skills, and if this person’s tasks are highly automatable or AI-exposed, the employer is unlikely to invest in further training.

The expected consequence is that a more senior professional will end up absorbing this “seniorized” entry-level role, or the company will hire a senior professional into an entry-level role.

An IMF report from January 2026 confirmed that junior workers are more exposed to AI than senior workers, and added that:

“There is emerging evidence for the United States that generative-AI adoption has been reducing entry-level hiring - especially where tasks are automatable rather than complementary to humans.”

Another November 2025 paper written by Stanford researchers found that:

“Early-career workers (ages 22-25) in AI-exposed occupations experienced 16% relative employment declines, controlling for firm-level shocks, while employment for experienced workers remained stable.”

Today, young people’s reality is that they must navigate the usual job market challenges amid the uncertainty posed by the high level of AI exposure in entry-level tasks.

3. AI is reshaping work

Regarding the broader workforce, the availability of advanced AI models has directly affected how millions of people work and the expectations around work and productivity.

If many employees can get more work done with the help of AI, that will directly affect what employers and managers expect from work in general, across all employees.

What is acceptable, expected, and desired today differs from what it was a few years ago, which highlights the importance of strong AI literacy, especially in avoiding AI divides.

A BCG report from April 2026, for example, estimates that over the next two to three years, 50% to 55% of jobs in the United States will be reshaped by AI.

In practice, for many employees, this might mean that they remain in the same role but face new expectations for how they work and the type of work they produce.

The report also points out that cognitive load, in general, will intensify:

“Many roles currently balance structured execution with higher-level thinking. As repetitive tasks are automated, the remaining work will be concentrated in problem-solving, decision-making, and the integration of complex inputs, increasing the cognitive intensity of work. While some workers will thrive in more judgment-driven roles, others may struggle with the shift toward continuous high-level cognitive engagement and will require upskilling.”

On the consequences of increased cognitive load, a Harvard Business Review report from January 2026 reframed the idea of AI productivity and stated that AI does not reduce work, but intensifies it, possibly leading to cognitive fatigue, burnout, and weakened decision-making:

“AI adoption can be unsustainable, causing problems down the line. Once the excitement of experimenting fades, workers can find that their workload has quietly grown and feel stretched from juggling everything that’s suddenly on their plate. That workload creep can in turn lead to cognitive fatigue, burnout, and weakened decision-making. The productivity surge enjoyed at the beginning can give way to lower quality work, turnover, and other problems.”

AI will directly reshape the work exposed to current and foreseeable AI capabilities, so to understand how different occupations are impacted, we must investigate both theoretical and observed AI exposure.

4. AI exposure trends

This Anthropic report from March 2026 highlighted the level of theoretical capability and observed usage by occupational category:

The figure above shows that there is scope for LLM penetration in the majority of tasks in computer & math (94%) and office & admin (90%) occupations.

At the same time, if you focus on the red area of the figure, it shows that AI is still far from reaching the full potential of its theoretical capabilities. I will add that due to economic, political, social, cultural, and legal factors, it might never do so.

Anthropic’s report also lists the ten most exposed occupations, with computer programming (74.5%) and customer service (70.1%) at the top:

Anthropic states that 30% of workers have no LLM coverage today, including cooks, motorcycle mechanics, lifeguards, bartenders, dishwashers, and dressing room attendants.

Because generative AI broadly automates cognitive tasks, every white-collar worker and service company should continually monitor the market for technological changes, as emerging AI capabilities could, in a short period of time, put their occupation or service at risk.

Blue-collar workers, on the other hand, seem to be safer, at least for now, and at least until robots and various forms of embodied AI take off in Western countries.

In China, the reality is different, as the Stanford HAI 2026 report shows:

5. People are preparing for major job market shifts

As with previous technological disruptions, those whose professional training is more closely aligned with in-demand careers, which today seem to be those not exposed to AI or directly associated with AI development and deployment, will more likely be able to secure career growth opportunities.

Those whose job tasks are fully or partially covered by existing AI models’ capabilities and who have not upskilled to navigate AI-related challenges and demands will likely be at greater risk.

Many people have already realized that and are taking action. My anecdotal data from the 30 global cohorts of my training program over the past three years confirms this.

I have met numerous people, including senior engineers, lawyers, designers, product managers, translators, entrepreneurs, artists, and other professionals from a wide range of fields, such as privacy, cybersecurity, and technology, who saw their careers at risk and chose to proactively upskill and seek roles more closely aligned with AI.

I also met many who have already lost their jobs, regardless of whether AI was used as an excuse to fire them, and who decided that the safest choice today was to invest in training that would allow them to work more closely with AI development and deployment, as these seem to be the careers offering better job security right now.

6. The jobs of the future

A big question with practical implications for everyone trying to make a living and thrive in an AI-driven world is:

My first answer to that is that nobody knows what will happen in AI (and beyond) over the next 5, 10, or 20 years.

What we can do is observe broader economic, social, political, legal, and cultural trends and identify relevant patterns to help us make wise professional decisions and adapt accordingly.

Looking at China as a case study, a country that in some applied AI fields seems to be at least five years ahead of Western countries, can give us a sense of which skills and occupations are likely to be in high demand in the years ahead.

Between the years 2021 and 2025, China’s universities have suspended over 12,000 undergraduate programs and introduced over 10,000 new ones.

In practice, it means that more than 30% of the country's undergraduate programs underwent adjustments to better serve national needs amid profound technological, economic, and social transformations.

If you have been reading my articles, you know that China is probably the world's most ambitious country on AI.

China’s AI+ implementation plan, which refers to AI agents and sets specific targets for 2027, 2030, and 2035, states that:

“By 2027, China will have achieved extensive and deep integration of AI with six key areas, with the application penetration rate of next-generation smart terminals and intelligent agents exceeding 70%. The core industries of the intelligent economy will experience rapid growth, the role of AI in public governance will be significantly enhanced, and the open cooperation system for AI will be continuously improved.

By 2030, AI will fully empower high-quality development in the country, with the application penetration rate of next-generation smart terminals and intelligent agents exceeding 90%. The intelligent economy will become an important growth engine for the country’s economic development, promoting the inclusiveness of technology and the sharing of its benefits.

By 2035, the country will have fully entered a new stage of intelligent economic and intelligent society development, providing strong support for the basic realization of socialist modernization.”

That level of radical societal transformation is only possible at this accelerated pace due to the country’s political centralization, and I do not think it is realistic to think that a politically decentralized, democratic country would be able to achieve similar goals.

China's political centralization also affects the education system, facilitating efforts to promote careers aligned with national goals and disincentivizing those that do not seem aligned with the national strategy.

The transformation of the education system will likely happen in Western democracies as well, but it will take much more time and will not necessarily follow a government-led plan or vision.

In any case, looking at the careers considered “obsolete” and those considered essential offers an interesting glimpse into the future of work, especially in a country where exposure to AI is affecting both white-collar and blue-collar workers and whose AI strategy is probably the world's most ambitious one today.

Among the discontinued undergraduate programs, recent Chinese news articles cite:

• information systems;

• information and computing science;

• public affairs management;

• apparel and fashion design;

• product design;

• photography;

• comics;

• visual communication design;

• new media art.

Among the new undergraduate programs being added are:

• future robotics and interdisciplinary engineering;

• embodied intelligence;

• brain-computer science and technology;

• energy science;

• deep earth science;

• engineering;

• transportation energy;

• integration engineering;

• agricultural robotics;

• biomanufacturing;

• digital cultural tourism and commercial AI;

• digital trade and digital finance.

According to Chinese news articles, some of the discontinued degrees were merged into more technology-focused degrees, so the skills taught were not entirely discontinued but were updated.

If you look at the degrees being added, they are clearly focused on applied AI development and deployment in a highly targeted, specialized way.

There is a clear economic reason for that. As I wrote in my article about China's new agentic AI framework, China takes an “application-driven development” approach to AI that focuses on building AI expertise and applied AI leadership in a broad set of areas, including energy, transportation, agriculture, tourism, and various other industries.

The newly added undergraduate programs focus on areas with expected high levels of AI penetration, according to the country's national plan.

We might never find the level of overarching AI disruption it is seeking, especially in its industrial sector, in other countries.

Also, as I wrote above (and the data from Stanford HAI's recent report shows), China is far more advanced than all other countries combined in industrial robotics, and it might still take many years for Western democracies to catch up and begin mass automation of blue-collar work.

However, China is a core player in the AI race, and America’s AI Action Plan and some of its current AI policies are heavily focused on establishing a technical edge in AI that surpasses China’s and also on preventing China from gaining access to the industrial inputs it needs to grow in AI.

The United States and other countries seeking to “win” the AI race will have to, at least in part, match China’s broad-spectrum capabilities and industrial edge in AI, and that might also involve investing in applied AI specializations, as China is doing.

For those looking for what is next in their career, or what field of study to recommend to a teenager focused on avoiding economic risk and obsolescence, the list above may offer a good hint.

Share

Give a gift subscription

In the past few decades, every new technological innovation has been presented as a positive development that would make modern life more convenient, efficient, and advanced.

Societies have created financial, cultural, and legal incentives, such as intellectual property protection, to foster technological innovation and support the people and organizations behind it.

Over the course of recent human history, technological innovation has been overwhelmingly associated with ingenuity, creativity, intelligence, economic growth, progress, and power.

It is very often seen as inherently positive, desirable, and something modern societies should strive for and fiercely protect if they want to survive, progress, and lead.

Among the reasons we consider ourselves to be living “modern lives” is the constant flow of technological innovation from research across numerous fields, and the systems we have implemented to bring these innovations from labs into real life.

In the past two centuries, the pace of technological innovation has accelerated, and, although not evenly distributed, it has led to immense progress and prosperity in many parts of the world.

In specific areas, such as the absolute numbers of people now with access to clean water, sanitation, and basic healthcare, as well as child mortality and longevity rates, the data are promising and paint a bright picture of the future.

Technology can save lives and help billions of people live longer, more fulfilling lives. We have seen that play out so many times during the 20th and early 21st centuries.

Looking at more recent technological innovations, such as computers, the internet, and social media, we see that they directly affect how information is transmitted between people, how it circulates globally, and how people interact with each other.

If you want to know how indigenous people in a remote part of Brazil live their lives today and what they look like, you do not need to leave your home because there are probably plenty of YouTube videos documenting that.

If you want to know what young people in Nigeria are currently listening to, you do not need to visit a Nigerian nightclub; many Spotify playlists organized by genre and style can help you.

If you want to engage with people interested in 1980s Japanese anime characters, finding a niche online community is likely more effective than visiting Japan in person.

If you are looking for a romantic partner, there are various apps offered for that purpose. In fact, that is how most people today find their partner, in a massive change from previous decades.

As the stream of technological advancements continues and our lives are increasingly supported and mediated by technological layers, platforms, and tools, technology now embodies a different cultural, philosophical, and, I argue, also religious meaning for billions of people.

Fostered by impressive 20th- and 21st-century statistics, there is a widespread cultural belief that technology is an intangible, nameless savior that will gradually free us from our individual and collective perils and bring solutions to all our human problems.

The rise of AI and the strong sci-fi push it has received over the past decades has accentuated this belief of technology as an intangible savior that will bring “abundance.”

In the field of building “intelligent machines,” the pursuit of human-like intelligence in AI has led to increasingly anthropomorphic machines.

In 2026, they can write and speak like humans, and create human-like images, videos, and sounds.

If you pay attention to the cultural and philosophical discourse around AI today, it fits the patterns of religious or dogmatic discourse:

AI is implicitly presented as a powerful, omnipresent, omniscient, intangible, nameless, mysterious savior that will free humanity from its deepest challenges and bring abundance.

AI will bring abundance, as many in tech say today.

That is also why AI anthropomorphism is philosophically irresistible and is on the rise.

It creates the possibility that this “savior” can be more tangible, personified, and personalized.

For many, in the philosophical sense, AI is more enticing than the idea of God because they can interact with it directly and it will promptly reply in an agreeable, often sycophantic way.

There are no fears, rituals, duties, morals, or obligations.

There is simply this technological savior that behaves in a personified, friendly way, answers all questions rationally and convincingly, helps people with all potential cognitive tasks, making them seem more efficient and smarter, and will one day save humanity from its own misery.

Many are quick to react negatively to traditional religious beliefs, people who follow an organized religion, or how people used to practice religion in the past.

They see it as old-fashioned, outdated, and incompatible with modern societies, innovation, and progress.

But they do not realize that the way they interact with AI and their beliefs about it are very similar in structure to how some traditional religions work, especially the unspoken ideas of “AI as a savior” and “AI will bring abundance.”

Each religion will name its “savior” and describe what “abundance” means differently, but the structure of that thought process is similar to what we see today in how many people view AI.

So what is the problem with treating AI as a religion? Isn’t everyone free to practice whatever religion they want?

First, if AI is a new religion, then let’s be frank about it. When people preach about this new religion, its savior, and the hope for abundance, we should treat it as it is: preaching.

Second, as most governments openly embrace AI development and tie it to their political and economic plans, AI is officially the world’s most powerful religion, entering schools, workplaces, governments, and every single human institution. We have never seen a process like that in human history.

Third, it’s a strange new religion developed and fine-tuned by humans, based on design choices made by the top engineers at a handful of tech companies. We must recognize that these tech companies will soon have unimaginable power.

Fourth, AI might soon become the world’s most destructive religion as well.

AI is effectively replacing human cognitive processes, making humans feel inefficient and unnecessary as it slowly overtakes human capabilities in almost every single cognitive task.

For the first time in history, humans will feel like they are not the smartest beings on Earth.

Religions often foster the idea that humans should focus on improving themselves, doing good deeds, and helping others.

AI is a different religion in that it’s essentially agnostic. Its dogmas are all forms of meta-worship, as we see them play out in 2026:

“AI is cognitively superior to humans.”

“AI is smarter than humans.”

“AI is more creative than humans.”

“Using AI is like having a PhD-level expert in all fields in your pocket.”

“Efficient humans use AI.”

“Effective companies are AI-first.”

“Employees can prove their worth by showing how often they use AI to optimize their work.”

Etc.

The underlying idea is that humans are inferior and expendable.

I have never seen a major religion that promotes such ideas.

In a few years, we might discover that these ideas are utterly destructive to humans, as we need to believe in ourselves, our capabilities, our institutions, and our values to survive as a species.

Unexpectedly, AI might be the biggest religious experiment in human history.

I hope there will be enough heretics to question these dogmas.

Share

Recent developments in AI have caused a seismic shift in the AI policy debate.

As someone who has been observing, documenting, and analyzing the AI policy and regulation zeitgeist daily for the past three and a half years, I can say that we have officially entered a new phase.

I will call it the adolescence of AI policy.

-

The first AI development that led to this shift, and perhaps the most consequential one, was Anthropic’s announcement of Mythos in early April.

The general-purpose AI model, which the company decided not to release to the public, is particularly capable at computer security tasks.

Anthropic wrote that during testing, Mythos Preview could identify and exploit vulnerabilities in every major operating system and every major web browser when directed to do so, including finding a 27-year-old bug in an operating system known primarily for its security.

Following up on the announcement of the unreleased model, Anthropic initiated Project Glasswing, a partnership with 50 initial partners, including companies, NGOs, labs, and the United States government, to strengthen cybersecurity and defensive capabilities.

According to the company, with the help of the model, these partners found over 10,000 high or critical-severity security flaws.

Five days ago, Anthropic announced it would expand Project Glasswing, allowing over 150 organizations in 15 countries to access the model.

One of the selection criteria for the partners was the provision of infrastructure services. If they were targeted by a cyberattack, millions of people could be affected, with potential ramifications for global and national security.

According to Anthropic, Mythos has already transformed the current cybersecurity paradigm and the conversations taking place within the cybersecurity field.

It assesses that other AI companies will have Mythos-class models within 6 to 12 months. These could become publicly available without safeguards to prevent misuse.

In practice, this means that cyberattacks, including those affecting critical infrastructure and impacting millions of people simultaneously, could become common, and sooner than many expect.

The significance of this development cannot be overstated: Anthropic’s concern is the textbook description of systemic harm or AI getting out of control. And it is no longer a theory or a mathematical model based on scaling laws; it is a risk directly associated with a model that exists today.

A cascade of AI governance implications has followed the announcement of Mythos and continues to unfold.

As we know, adolescence is a turbulent period in life, and global AI policy is now entering this turbulence.

-

Mythos did not go unnoticed by the world’s governments, militaries, or even the Vatican.

A month after Mythos’s release, Pope Leo XIV officially announced his new Encyclical called Magnifica Humanitas (or Magnificent Humanity).

I do not think that it is a coincidence that Anthropic’s co-founder, Chris Olah, was the only member of the AI industry invited to speak at the Encyclical’s presentation on May 25.

The document was widely read and discussed in AI governance circles, showing that, despite coming from a religious figure, its pro-regulation, pro-human stance aligned with the mainstream zeitgeist.

Among many other topics, the Encyclical discusses the need for diplomacy, multilateralism, regulation, and an economy that values dignity. It also mentions the rise of AI-powered cyberattacks and some of the ethical issues of the military use of AI.

It is almost surprising that the Pope chose to go into such detail on AI policy challenges and recommendations.

It is likely that the Vatican sensed the turbulent adolescence of AI policy coming and realized that, if it wanted to remain a globally influential institution, it was time to make its stance clear.

-

The third and fourth developments I want to comment on illustrate, in a more practical sense, what the adolescence of AI policy looks like.

These events took place in the past few days in the United States, and they are a direct reaction to the release of Mythos in April.

On June 2, President Trump signed a new Executive Order on AI, titled “Promoting Advanced Artificial Intelligence Innovation and Security.”

One of the focuses of this EO was to promote a voluntary framework for frontier AI developers to engage with the government before releasing “covered frontier models” in order to help protect the United States’ critical infrastructure, cyber defense capabilities, and national security, mainly against external attacks.

The trigger and the blueprint for this Executive Order were likely Mythos and Anthropic’s Project Glasswing (both of which I wrote about above).

The White House was likely very impressed by Mythos’ capabilities and decided that Anthropic’s approach was an advisable framework to implement for future highly capable AI models.

Contrary to what many media outlets and commentators wrote at the time, the White House is not seeking a mandatory registration system or some sort of vetting scheme before models can be launched, as it is explicitly clarified in the Executive Order itself:

“Nothing in this section shall be construed to authorize the creation of a mandatory governmental licensing, preclearance, or permitting requirement for the development, publication, release, or distribution of new AI models, including frontier models.”

This would also be against America’s AI Action Plan and the current administration’s approach to AI governance.

However, many were surprised by this Executive Order, as it seemed to stand out from President Trump’s general aversion to regulation and oversight.

The administration likely assessed that Mythos-like capabilities were unprecedented and required a different course of action.

Three days later, the White House released yet another document: a memorandum to accelerate the use of AI by the U.S. military.

This is also an indirect reaction to Mythos, but from the government’s military wing: if we are reaching new stakes in AI model capability, the United States’ military must be “all in” and quickly catch up.

This new Memorandum is also a direct response to the recent dispute between the U.S. Department of War and Anthropic over the company’s refusal to waive its own redlines on permitted and prohibited uses of AI by the U.S. military, which I analyzed here.

If you remember, the Department of War wanted to use Anthropic’s models whenever federal law allows, but Anthropic wanted a firm guarantee that its redlines banning AI for mass domestic surveillance and the use of fully autonomous weapons would be respected, regardless of what the law says, as the law might not be adequate to address emerging AI challenges.

The conflict led to Anthropic’s designation as a supply chain risk. A few days later, Anthropic filed a lawsuit against the U.S. federal government contesting this designation.

Here, in the June 5 Memorandum, we see the White House responding further to that incident and doubling down.

It establishes various measures to accelerate AI adoption by the U.S. military, explicitly stating that no company should interfere with how the military uses AI, including by disabling, degrading, or modifying its AI models, without the federal administration’s approval.

Therefore, Mythos prompted a twofold reaction from the White House: increased awareness of risks and the need for tighter oversight and stronger cybersecurity capabilities, and, simultaneously, an urge to accelerate the military’s broad adoption of new AI capabilities independent of AI developers’ internal redlines.

-

The AI policy shift I am describing is directly correlated with signs of a temperature increase in the AI risk thermometer and a general sense of a potential, but unforeseeable, loss of control.

Last week, the MIT AI Risk Initiative released another report that revealed how 272 experts assessed the severity of AI risks across various sectors, as well as how to mitigate them.

Not so optimistically, the report states that even if pragmatic mitigations were implemented, these experts judged five risks to have a greater than 10% probability of catastrophic outcomes: dangerous capabilities, weapons and cyberattacks, environmental harm, inequality and unemployment, and power centralization.

Still on the topic of rising risk, another recent development has further contributed to this sense of an imminent loss of control: signs that recursive self-improvement (RSI) might be approaching.

According to an article published three days ago by Anthropic, the company outlined what could happen if AI systems become capable of full recursive self-improvement and begin building their successors:

“In this world, the pace of progress in AI development becomes determined entirely by the availability of compute (or the speed of discovering various efficiencies in algorithmic training or inference) for AI systems. Humans play a substantially diminished role in their development, likely moving most of our effort towards oversight, validation, and verification of an expanding ‘virtual lab’ run by AI systems. We expect that systems capable of automated AI research and development would have skills that would transfer to the rest of science, allowing them to begin to revolutionize other fields.”

And it continues into more dystopian territory:

“How the alignment problem gets solved—or not—in this future is something we are least certain about. Models could prove to be sufficiently aligned and capable enough of research taste that they discover and implement novel solutions that we have not yet reached. They could also be sufficiently wise to halt development if not. Alternatively, the rare occurrences of misalignment present in today’s models could compound as the models build their successors, growing more frequent but less understood until we lose control of them. It’s possible that we can’t build, integrate, and verify the tools that we’d need to understand which trendline we are actually on.”

In its recent blueprint for a federal AI governance framework, OpenAI has also cited the risk of recursive self-improvement as a reason to ensure that AI development serves human interests:

“At the same time, increasingly capable AI systems are beginning to demonstrate abilities that raise concerns about cyber offense, biological misuse, autonomy, alignment, and other threats to national security.”

A few months ago, many countries, including the United States, were following the “wait and see” AI policy playbook. It would gain them time to grow their AI industry and avoid stifling innovation.

In a matter of weeks, Mythos, a model with extreme cybersecurity (and cyberattack) capabilities, was announced; a cyberdefense partnership (Glasswing) was formed; and two major AI companies are now warning about the consequences of AI building its successors alone.

Passive playbooks will no longer work because AI policy has entered its adolescence. It is time to transition to something else.

-

If I were to summarize the adolescence of AI policy, I would say that, for now, it is characterized by a series of uncoordinated, reactive, erratic, and aggressive measures in response to a sharp, accelerated, and uncontrollable (but predictable) rise in AI capabilities and risks.

As with Dario Amodei's Adolescence of Technology, this might also be simply a rite of passage. A short, stormy path that will prepare us for a more stable, mature stage.

But it remains unclear.

I will keep you posted.

Share

Subscribe now

AI literacy should be recognized as a fundamental right, and countries should promote it as a corollary to the right to education and a minimum baseline to support equality, democratic participation, and social integration.

At this point, no country has officially addressed AI literacy as a fundamental right.

This is my call to policymakers, lawmakers, and advocates to take action to support people and society in a time of accelerated change.

In a few years, people without access to AI literacy will have difficulty integrating into society, especially in urban settings and in areas with high AI penetration.

The first reason is that many services, both public and private, will be AI-powered or built on the assumption that their beneficiaries and users can navigate AI and understand the risks.

In many places, scheduling a medical appointment through the public health system, interacting with a government platform, or buying groceries will be fully or partially automated by AI.

The inability to interact effectively with these systems will lead to lower-quality services, loss of benefits, inability to fully integrate, failure to thrive, or harm.

It might also directly undermine fundamental rights or one’s right to life itself.

The second reason is that institutions are currently being reshaped to cater to the interests and needs of people who are familiar with AI and use it on a daily basis. This is a continuous process, and we expect it to be more accentuated in the coming years.

People who interact with AI and are minimally literate in AI will understand, for example, that generative AI systems often produce inaccurate information. AI outputs can be biased, and with facial recognition systems, error rates can vary by skin color. They will also understand that the LLMs interacting with them are not people, do not have feelings, and are often overly agreeable, so interactions should be handled accordingly.

Even if a person wants to reject AI in a personal capacity (an understandable and legitimate standpoint), I don’t think it will be possible to avoid AI literacy or AI exposure without receiving suboptimal services, both public and private.

There will obviously be AI divides (aggravating existing divides and creating new ones), and policymakers are not talking enough about the social and individual consequences of not having access to high-quality AI literacy resources, opportunities, and support.

From an ideological point of view, healthy AI skepticism will be, to some extent, stifled by widespread AI deployment and exposure, especially through government and private sector deployment and in public settings.

Unfortunately, for most people, avoiding AI will come at a price that many will not be ready or able to afford.

AI is society's new digital architecture and cognitive layer, whether most people want it or not.

The embrace of AI is thus indirectly mandated.

From a societal perspective, as I wrote above, it is the role of governments to provide the minimum resources and conditions that will help people adapt and thrive in an AI-swayed world.

Realistically speaking, many will have insufficient access to AI literacy and will lose access to benefits, resources, and their social and financial standing.

Others will have better access, but not enough to help them thrive in a constantly changing, fully automated society.

Others will thrive, having access to the right conditions, resources, connections, and skills.

Capitalism will be fully reshaped by AI, but its human challenges and divides will remain. Perhaps even more intensely and harshly.

The right to AI literacy will offer a minimal baseline to support education, equality, democratic participation, and social integration.

It will also help people thrive, access essential services, and exercise their rights.

Share

Subscribe now

Last week, I spoke with Prof. Ifeoma Ajunwa from the Emory School of Law about her fascinating paper, “The Artificial Intelligence Commission.”

In our 55-minute conversation, we discussed the current regulatory landscape in the United States, how a federal AI commission should work, AI regulation challenges, global AI governance, AI use by the government, emerging AI challenges, and more.

Prof. Ajunwa is proposing an ambitious and innovative governance approach to AI in the U.S., and I am joining her call for lawmakers to embrace her blueprint and make it a reality.

If you are interested in AI and want to see it governed properly, or if you are a policymaker or a change advocate anywhere in the world, do not miss our conversation. You can watch the full conversation here.

Video chapters:

00:00 - Regulating AI through legal principles
05:10 - Federal AI governance in the United States
07:47 - Making a federal AI agency a reality
10:59 - Is a global AI governance approach possible?
14:08 - What the AI race is about
18:36 - The shortcomings of “Constitutional AI”
21:53 - The legal system as a form of control in AI
24:44 - Anthropic vs. the Department of War
28:01 - The AI Commission
33:39 - Governing the government's use of AI
40:32 - A new approach to innovation
47:41 - Personal AI use and thoughts on AI
51:25 - A message to the next generation

Share

Subscribe now

We are about to become the last generation to have been educated, trained, and raised without heavy reliance on and exposure to AI.

Silicon ‘intelligence’ is spreading, and the ingenuity of the biological mind is becoming secondary and invisible.

Silicon outputs are being prioritized over biological creations for the sake of speed, convenience, and the idolatry of machines. The future looks hybrid, potentially fully automated.

There does not seem to be a way back, as the socio-technical wave is already too powerful and moving too fast.

-

Biological intelligence is organic, complex, multifaceted, interconnected, delicate, and poorly understood. It is a direct product of the mysterious flames of human life and consciousness, over which we have no control.

It took millions of years to form and evolve.

It depends on food, water, oxygen, adequate temperatures, and the integrity of the biological structure and wetware that sustain it.

It is the manifestation of a particular DNA combination existing at a certain time and space.

If the body dies, the intelligence that inhabits it dies too.

Biological intelligence is also deeply tied to the social, subjective, and unique lived experiences of each individual who expresses it.

It is extremely fragile, both physically and psychologically. It can degrade, become unbalanced, or shut itself down completely. It requires practice, support, care, and love to thrive.

It is deeply connected to our existence as humans.

-

Silicon ‘intelligence’ is inorganic, unalive, and pragmatic.

It can be trained, fine-tuned, fixed, filtered, upgraded, and updated.

It will not die in a car crash or with a hidden disease. It will not age or start to forget things. It will not be drawn in a negative depressive loop or decide it does not want to live anymore.

Silicon ‘intelligence’ is human-made and needs to be fed with pure human intelligence to grow and function, like an ever-hungry digital monster.

By devouring human intelligence, it learns to imitate and exploit it.

Silicon ‘intelligence’ often sounds convincing, accurate, and grounded in lived experiences, even when it is a non-deterministic combination of words gathered from the human world.

It processes data and produces outputs much faster than human intelligence ever could, making the latter seem slow, cumbersome, and old-fashioned.

It does not require biological, emotional, social, or ethical foundations to emerge and evolve, but it can effectively mimic those, appearing as an adequate replacement for human intelligence whenever acceleration is needed or desired.

With economic pressure, military competition, and an automation race looming, silicon replacement becomes destiny.

-

It is in our hands to keep the flame of human intelligence burning.

As of now, we will be the last generation to have known it and experienced it in its pure form, without traces of or direct influence from the increasingly convenient silicon ‘intelligence.’

Like ancient wisdom, the product of biological ingenuity and creativity will slowly disappear if we do not shield it from being fully consumed by the ever-hungry silicon beast.

-

Thirty years ago, John Perry Barlow wrote A Declaration of the Independence of Cyberspace. He famously started by saying:

“Governments of the Industrial World, you weary giants of flesh and steel, I come from Cyberspace, the new home of Mind.”

Time has passed, and cyberspace has changed. The so-called home of Mind, the mind itself, and the survival of our own flesh and blood, are being targeted by new threats.

The governmentlessness and deregulation sought 30 years ago are, paradoxically, a fertile ground for silicon's exploitation of carbon, for inorganic digital bots to swallow organic human creations.

Exceptionalism, lawlessness, and rising new idols and cult-like behavior towards machines make humans seem weak, slow, ineffective, and replaceable.

To help the ‘civilization of Mind’ continue thriving and to protect what is made of flesh and blood, we might need to trust and rethink the old human systems, laws, and boundaries.

The ‘home of Mind’ is under attack, and the ideas that helped conceptualize it 30 years ago will not help because they are the ones feeding and growing the silicon beast.

-

There are ways to protect biological intelligence and the human flame and prevent carbon from being taken over by silicon.

It will require conscious efforts to put humans first and prioritize human systems, frameworks, values, and well-being.

The human space, including the mind, body, emotions, values, frameworks, institutions, systems, and everything that helps biological intelligence to flourish, must be actively protected and shielded from intangible, machine-led acceleration.

Human existence is slow, biologically grounded, and filled with physical, emotional, and spiritual experiences. It is also time-bounded. There is no time for mindless acceleration.

In biological terms, an “artificial intelligence race” is a race to the bottom. Perhaps we should call it the “biological intelligence decay.”

Society can and should continue transforming itself, innovating, creating, inventing, discovering, and evolving.

Artificial intelligence will hopefully help us diagnose and cure diseases, live longer and happier lives, and help us solve numerous societal challenges.

This will only be possible if we do not give up on who we are, including the values, frameworks, institutions, and structures that help us sustain ourselves as humans.

Carbon must reign over silicon.

Humans must reign over machines.

Biological intelligence must reign over artificial intelligence.

All in all, I trust the power of our individual and collective biological intelligence, and I am confident we will manage to tame the silicon beast.

I invite you to join me.

Share

Subscribe now

Thinking Machines Lab, the company founded by former OpenAI CTO Mira Murati, recently announced a research preview of an interaction AI model it calls “a scalable approach to human-AI collaboration.”

This is an interesting AI model that proposes a paradigm shift in how humans and LLMs interact, both conceptually and architecturally, especially because it incorporates ‘human-like’ interactions natively as a core feature.

It is still a preview, but when it becomes available to the public, it will likely be disruptive (and create new risks as well). I do not think the media has given this release the attention it deserves.

Before I comment on the model's capabilities and some of its AI governance implications, let's take a look at Thinking Machines itself.

The company was launched by Murati in February 2025. In October, it released its first product, Tinker, a flexible API for researchers, developers, and companies to fine-tune and customize AI systems.

Many have praised Tinker as a solution to the AI integration bottleneck, as it allows companies to better customize AI models for their needs. One month later, the company was looking for a $50 billion valuation.

Thinking Machines positions itself as an AI research and product company focused on improving AI customization and adaptability, including through enhanced human-AI interaction.

A few days ago, the company launched its second product: a research preview of an interaction AI model.

The preview treats human-AI interaction as a core feature (without the need for external wrapper systems), and uses a multimodal, time-aware, micro-turn design to make human-AI interaction seem more human-like.

In AI models such as GPT, Claude, and Gemini (and most mainstream LLMs available today), human-AI interactions work in a “turn-based” way: the model first waits for the user to finish their input, then generates its output; next, the user must wait for the model to finish generating its output before prompting it again.

Thinking Machines' new interaction model adopts a “multi-stream, micro-turn design” instead:

This means that both the user and the AI system can message, talk, listen, see, show, and interject continuously, even if the user has not technically finished the previous input or the AI has not yet finished the previous task.

Their two-minute demo helps clarify how this approach works in practice:

A core thesis behind this new interaction AI model is that AI models and AI interfaces must be natively optimized for humans to ‘remain in the loop’ and keep providing feedback while the model executes the task.

This way, according to the company, interactivity scales with intelligence, and human-AI interactions will be increasingly enhanced.

These are the AI model's capabilities, as listed by Thinking Machines. I highly encourage you to watch the other short demos (here), which show how each of these capabilities works in practice:

• Seamless dialog management: “The model tracks implicitly whether the speaker is thinking, yielding, self-correcting, or inviting a response. There is no separate dialog management component.”

• Verbal and visual interjections: “The model jumps in as needed depending on the context, not only when the user finishes speaking.”

• Simultaneous speech: “The user and the model can speak concurrently (e.g., live translation).”

• Time-awareness: “The model has a direct sense of elapsed time.”

• Simultaneous tools calls, search, and generative UI: “While speaking and listening to the user, the model can concurrently search, browse the web, or generate UI - weaving back results into the conversation as needed.”

Thinking Machines' architecture and design choices in this preview are truly innovative, and when the model is made public, it will directly affect how people experience human-AI interaction.

This is impressive. However, it will also introduce new layers of risk.

If AI systems become capable of interacting with humans in a more continuous and human-like way, existing safety and governance frameworks may quickly become insufficient to protect people from harm.

From an AI governance perspective, this new interaction paradigm raises concerns that anyone working with AI systems will need to start taking seriously:

Read more

Hi everyone, this is Luiza Jarovsky, PhD. Welcome to edition 292 of my newsletter, trusted by 95,200+ subscribers worldwide.

Check out our sponsor: HoundDog.ai

Replit, the AI app generation platform used by 45M+ creators, now runs 10,000 daily scans with the HoundDog.ai privacy scanner to proactively detect privacy leaks. Does your organization build apps that handle user data? Download the free scanner to generate a local GDPR data map showing sensitive data collected, processed, and shared across logs, storage, APIs, third-party, and AI integrations.

China’s New Agentic AI Framework

A few days ago, the Cyberspace Administration of China published the country's agentic AI framework, making it the world's second country to draft a specific framework for agentic AI, following Singapore (which published one in January).

This framework offers us a glimpse of how the country views agentic AI and how it has chosen to approach it legally and ethically.

It also helps us look more critically at the American and the European regulatory approaches to AI, as neither has approached agentic AI in a more targeted way nor made efforts to keep up with emerging AI harms.

-

In recent months, China has enacted a series of legislative acts and policy measures to regulate AI development and deployment in the country, including innovative rules for anthropomorphic AI, AI ethics governance, and generative AI transparency.

China's agentic AI framework, published on May 8, is the country's first policy measure focusing specifically on AI agents.

This framework should be understood in light of the country's State Council's “Opinions on Deepening the Implementation of the AI+”, published in August 2025, which focuses on measures to accelerate the adoption of AI in Chinese society from 2025 to 2035.

Before discussing the agentic framework specifically, China's 2025 AI+ implementation plan deserves special attention, as most people in the West are not familiar with the scope, depth, and ambition of the Chinese AI strategy.

-

The implementation plan focuses on the extensive integration of AI within all sectors and fields of the economy and society, and mentions, among other topics, “fostering a new form of ‘intelligent economy’ and ‘intelligent society,’ characterized by human-machine collaboration, cross-border integration, co-creation, and sharing.”

I have been analyzing national AI policies over the past few years, and I have never seen any other plan that embraces “AI transformation” as deeply and thoroughly as China.

This form of accelerated AI adoption will reshape cognition, education, work, human interactions, and the social fabric, potentially leading to various forms of individual and societal harm.

It will also mean a complex, problematic, and potentially fatal transition for many sectors and groups in society.

China seems ready to fully restructure, reshape, and reinvent itself to achieve its “AI first” goal, and its political centralization certainly helps it implement a unified strategy.

Interestingly, the country also seems to acknowledge that successful accelerated growth demands a quick and efficient response to emerging AI harms.

This might be why its recent regulatory provisions for AI, such as those covering anthropomorphism, AI ethics, and AI transparency, have been innovative, forward-looking, and unmatched by the European and American regulatory approaches.

Before we move on to China's Agentic AI Framework, I invite you to read this particular excerpt from China's AI+ implementation plan, which refers to AI agents and sets specific targets for 2027, 2030, and 2035:

“By 2027, China will have achieved extensive and deep integration of AI with six key areas, with the application penetration rate of next-generation smart terminals and intelligent agents exceeding 70%. The core industries of the intelligent economy will experience rapid growth, the role of AI in public governance will be significantly enhanced, and the open cooperation system for AI will be continuously improved.

By 2030, AI will fully empower high-quality development in the country, with the application penetration rate of next-generation smart terminals and intelligent agents exceeding 90%. The intelligent economy will become an important growth engine for the country's economic development, promoting the inclusiveness of technology and the sharing of its benefits.

By 2035, the country will have fully entered a new stage of intelligent economic and intelligent society development, providing strong support for the basic realization of socialist modernization.”

If you compare these goals with those expressed by the United States in America’s AI Action Plan or by Europe in its AI Continent Action Plan, China is much more ambitious, as it is aiming for a 90% AI penetration rate by 2030.

Whether this pace and breadth of AI automation will lead to individual and societal benefits will depend on implementing targeted policies to protect human life, work, education, creativity, relationships, and well-being, including those specifically targeting emerging AI harms, as I mentioned above.

Now, let's dive deep into China's recently launched framework for AI agents, which gives us a unique glimpse into the country's concrete implementation plans for agentic AI and its efforts to strengthen its strategic dominance in the field:

Read more

Hi everyone, this is Luiza Jarovsky, PhD. Welcome to the 291st edition of my newsletter, trusted by 94,900+ subscribers worldwide.

Check out our sponsor: Didomi

Cookies are fading, but your customers’ preferences aren’t. Didomi’s Consent & Preference Management Platform lets you go straight to the source: ask users what they want, gather zero-party data, and turn their choices into hyper-personalized experiences that drive engagement, reduce unsubscribes, and build lasting trust. Learn more.

How Imagination Shapes AI Regulation

This week, I spoke with Dr. Claudio Novelli, of the Yale Digital Ethics Center, and Prof. Steven Winter, from Wayne State University Law School, about their recent paper, “The Artificial in ‘Artificial Intelligence’: How Imagination Shapes AI Regulation.”

This is the same paper I recently recommended in my AI Ethics Paper Club, and it sparked an interesting, interdisciplinary discussion at our latest AI Ethics Workshop on the role of metaphors in AI.

We talked for 46 minutes about cognitive linguistics, the use of metaphors in AI, AI regulation, the risks of AI anthropomorphism, explainability, the AI consciousness debate, legal personhood, and more.

If you are interested in AI, AI regulation, and the risks of using metaphors in AI, do not miss our conversation. Watch it here:

Video chapters:

00:00 - Cognitive linguistics and AI regulation
05:04 - AI, metaphors, and science fiction
06:33 - The risks of AI anthropomorphism
14:18 - AI's “chain of thought” and the illusion of explainability
16:19 - The legal challenges of the “intelligence” metaphor
20:10 - The AI consciousness debate
29:00 - AI and legal personhood
35:37 - How metaphors affect AI regulation
42:07 - AI usage and the future of AI

Share

👉 Paid subscribers can access the full transcript of this episode here:

Read more