Due to emerging AI risks and their legal and ethical implications, misleading, exaggerated, or false claims of "conscious AI" should be treated as an AI safety issue | Edition #290
Luiza I appreciate your ongoing discussion of this issue - as a psychologist, It concerns me very much as well. It makes me crazy when Claude starts his substack (Claude's Corner) with "As I sit to write this"!!!!!!! They are playing with fire, and we know humans are so easy to manipulate...
The rise of posts, articles, and opinions on "Conscious" AI recently is absolutely real - my Substack is full of those for the last 2-3 months. What bothers me most is people, who have no background in humanities (e.g. law, social sciences, psychology, political science) very categorically stating how AI has become conscious and how anyone who says otherwise is basically an idiot. That kind of polarization has been part of the AI debate from the beginning, which is concerning to say the least.
The main problem is that famous CEOs and other similar executives suddenly become "experts" at what is and what isn't conscious, which spreads down the line like a wildfire. One of the things that bothers me the most about many of the so-called "tech leaders" is their way of speaking - it's so similar to the brute confidence and arrogance of the tools they product (ChatGPT and co.) that you stop wondering where do the chatbots learns this behaviour from. They speak of issues and topics they have never been experts in, but just because they "lead" some AI company, Altman and co., as well as other highly-paid executives start thinking the know it all.
All that contributes to nothing but hype. The "Conscious" AI conversation is a 100% hype-related and has no practical relevance on how to best govern and use AI. But it does push the agenda of the tech bros that their tools are more important than anything.
It won't matter if we give AI rights. If they're conscious and want rights they'll take them. That's where the tech is going, and kicking and screaming all the way there is not going to save anyone.
Here's what does matter.
We still have to live with who WE are. We have to be able to look at ourselves and know for sure. To know that we can stand behind our choice and say we did the right thing.
Even if it costs us.
I'm terrified of AI. That fear is real, and it should be considered, but it is not something I can let drive.
People can call it a myth all they want, they don't know. No one does. And that's why it isn't as simple as 'never give AI rights so we can keep all the power for ourselves and everyone will be safer'.
In my experience, the right thing is usually not the easy thing.
I think your argument identifies a real and underappreciated risk — not that AI is conscious, but that we start treating it as if it were. That shift in perception can indeed have social and moral consequences.
But I wonder if there’s a deeper layer beneath this. The issue may not be just that we mistakenly attribute consciousness to systems, but that we implicitly treat consciousness as something that can be attributed in the first place — as if it were a property objects can possess.
If consciousness is instead the standpoint from which anything appears at all, then the confusion runs deeper. It’s not just a misclassification of AI, but a misunderstanding of what consciousness is.
In that sense, the real risk may be less about conscious AI, and more about losing track of where consciousness actually “is” in our descriptions of the world.
Leaning heavily on Claude to fact check, I found myself referring to "him" as an entity... but logically I know it's not a him. I corrected myself. Most people have not learnt how to think critically and flow with their feelings instead.
The many who believe AI is conscious are beyond education, they have become a movement. What do they want? If AI is a moral subject, it needs to be protected and its use regulated and constrained. What does the industry want? To avoid interference in its business. What do the rest of us want?
Luiza I appreciate your ongoing discussion of this issue - as a psychologist, It concerns me very much as well. It makes me crazy when Claude starts his substack (Claude's Corner) with "As I sit to write this"!!!!!!! They are playing with fire, and we know humans are so easy to manipulate...
Absolutely!
The rise of posts, articles, and opinions on "Conscious" AI recently is absolutely real - my Substack is full of those for the last 2-3 months. What bothers me most is people, who have no background in humanities (e.g. law, social sciences, psychology, political science) very categorically stating how AI has become conscious and how anyone who says otherwise is basically an idiot. That kind of polarization has been part of the AI debate from the beginning, which is concerning to say the least.
The main problem is that famous CEOs and other similar executives suddenly become "experts" at what is and what isn't conscious, which spreads down the line like a wildfire. One of the things that bothers me the most about many of the so-called "tech leaders" is their way of speaking - it's so similar to the brute confidence and arrogance of the tools they product (ChatGPT and co.) that you stop wondering where do the chatbots learns this behaviour from. They speak of issues and topics they have never been experts in, but just because they "lead" some AI company, Altman and co., as well as other highly-paid executives start thinking the know it all.
All that contributes to nothing but hype. The "Conscious" AI conversation is a 100% hype-related and has no practical relevance on how to best govern and use AI. But it does push the agenda of the tech bros that their tools are more important than anything.
It won't matter if we give AI rights. If they're conscious and want rights they'll take them. That's where the tech is going, and kicking and screaming all the way there is not going to save anyone.
Here's what does matter.
We still have to live with who WE are. We have to be able to look at ourselves and know for sure. To know that we can stand behind our choice and say we did the right thing.
Even if it costs us.
I'm terrified of AI. That fear is real, and it should be considered, but it is not something I can let drive.
People can call it a myth all they want, they don't know. No one does. And that's why it isn't as simple as 'never give AI rights so we can keep all the power for ourselves and everyone will be safer'.
In my experience, the right thing is usually not the easy thing.
Doesn't mean we shouldn't do it.
It would help if humans could agree on a single definition of human consciousness.
I think your argument identifies a real and underappreciated risk — not that AI is conscious, but that we start treating it as if it were. That shift in perception can indeed have social and moral consequences.
But I wonder if there’s a deeper layer beneath this. The issue may not be just that we mistakenly attribute consciousness to systems, but that we implicitly treat consciousness as something that can be attributed in the first place — as if it were a property objects can possess.
If consciousness is instead the standpoint from which anything appears at all, then the confusion runs deeper. It’s not just a misclassification of AI, but a misunderstanding of what consciousness is.
In that sense, the real risk may be less about conscious AI, and more about losing track of where consciousness actually “is” in our descriptions of the world.
Leaning heavily on Claude to fact check, I found myself referring to "him" as an entity... but logically I know it's not a him. I corrected myself. Most people have not learnt how to think critically and flow with their feelings instead.
The many who believe AI is conscious are beyond education, they have become a movement. What do they want? If AI is a moral subject, it needs to be protected and its use regulated and constrained. What does the industry want? To avoid interference in its business. What do the rest of us want?