🗝️ 5 Things AI Companies Don't Want You To Know

AI's Legal and Ethical Challenges | Edition #199

👋 Hi, Luiza Jarovsky here. Welcome to this newsletter's 199th edition!

[If you haven’t subscribed yet, we're a community of over 59,400 people working to ensure AI is properly governed. Join us, have your voice heard, and help us shape the future of AI.]

Don't miss: AI Governance Training | Learning Center | Chat | Job Board

---

🗝️ 5 Things AI Companies Don't Want You To Know

The field of AI has existed at least since the 1950s.

However, with the rise of the Generative AI wave in 2022, the term “AI” has dominated headlines and has been directly affecting the economy, politics, law, culture, and society to an extent we had never seen before.

As this newsletter celebrates 3 years this week, I reflect on how interesting it has been to watch (and dissect) the legal and ethical aspects of this techno-social swirl.

As usual, when there is so much money, media coverage, and political interests involved, competition becomes fierce, and companies will do whatever is within their reach to “win” the race.

Inspired by what I've observed in the last 3 years, below are five things AI companies don't want people to know (so make sure to share widely):

1. Personal Data Leakage

AI models, such as GPT or LLaMA, are trained using web scraping. Web scraping relies on bots to collect publicly accessible content from the internet without consent, including personal and potentially sensitive information.

This means that the pictures, videos, and messages you naively posted on Facebook back in 2007 (and had forgotten about them) might actually be baked into AI models. There is no 100% guarantee that they won't be leaked or regurgitated.

Additionally, when people interact with AI systems, the default option is usually to use the information being input by users to train the model. Many people are unaware of how AI works and input personal and sensitive information, increasing their risk.

2. Misinformation About People

Every existing Generative AI model has a hallucination rate, meaning that in a percentage of outputs, it will invent information.

GPT 4o's hallucination rate, for example, is 1.5%. If you consider the number of user prompts ChatGPT receives a day, the absolute number of false information being output is actually extremely high.

These fake outputs might involve individuals; ChatGPT, for example, has falsely accused a Norwegian user of being a murderer. There is no way to ensure that your name will never be falsely associated with harmful accusations.

3. AI Risk's Unpredictability

Different from other types of products, AI is adaptive and “learns” from the interaction with its environment. AI systems are trained to behave in a certain way, but even “normal” use might involve unexpected and unpredictable outcomes.

AI's unpredictability also includes negative and catastrophic events. Governments and companies have been investing in “AI safety” labs, but given AI's opacity and adaptability, it's unclear how effective these ex ante efforts will be.

Given the fierce AI race going on, companies’ priorities will likely be profits, not safety.

4. The Destruction of the Internet

Today, it's cheap and easy to create content using Generative AI: text, images, audio, videos, whole websites, podcasts, books, music, AI influencers, AI bots, viral deepfakes, and so on.

Additionally, AI capabilities are being integrated into existing systems, and companies are pushing people to rely on AI for every task (do you remember how many times you read “create it with AI” today?).

As a consequence, the internet is being inundated with AI-generated content, lowering the quality, accuracy, and authenticity of the information ecosystem. We only have one internet, and we might be destroying it forever. AI companies know that and don't seem to care.

5. The AI Literacy Divide

As I wrote earlier in this newsletter, a topic that concerns me deeply is AI literacy, and the AI literacy divide that is being formed, amplifying existing inequalities and creating new ones.

AI is evolving fast, transforming industries, and sometimes replacing people, especially knowledge workers. It has become a professional necessity, but many don't have the awareness, time, support, or resources to understand how AI impacts their work, upskill, and stay competitive.

AI companies, especially those that claim to support responsible AI, should help promote AI awareness, transparency, education, and literacy, and ensure that this transition happens smoothly. Instead, they completely ignore AI's social impact.