Amazon's dark patterns disrespect users
👋 Hi, Luiza Jarovsky here. Read about my work, invite me to speak, tell me what you've been working on, or just say hi here.
This week's newsletter is sponsored by Incogni:
Hundreds of companies are involved in the collection and trade of your personal information. This includes data like SSNs, dates of birth, home addresses, health information, contact details, and potentially even more. Safeguard yourself against scams, online harassment, robocalls, spam emails, and identity theft by proactively securing your personal information. Take control of your privacy and utilize Incogni to optimize your online security and minimize the exposure of your personal information to data brokers.
🔥 Amazon's dark patterns disrespect users
The Federal Trade Commission (FTC) is suing Amazon over dark patterns in the context of Prime subscriptions. I have written extensively in this newsletter about dark patterns in privacy and privacy UX, you can check my articles about the topic in the archive and also my paper on the topic. According to Harry Brignull (who coined the term), dark patterns are “tricks used in websites and apps that make you do things that you didn't mean to, like buying or signing up for something.” In this specific case, according to the FTC's press release, “Amazon used (…) dark patterns to trick consumers into enrolling in automatically renewing Prime subscriptions. Amazon also knowingly complicated the cancellation process for Prime subscribers who sought to end their membership.” From my point of view, one of the most negatively surprising quotes of the lawsuit was the following: “Amazon named that process ‘Iliad,’ which refers to Homer’s epic about the long, arduous Trojan War. Amazon designed the Iliad cancellation process (‘Iliad Flow’) to be labyrinthine, and Amazon and its leadership (…) slowed or rejected user experience changes that would have made Iliad simpler for consumers because those changes adversely affected Amazon’s bottom line.” We are in 2023, there have been multiple reports on the topic of dark patterns, such as this one from the FTC itself and from various data protection authorities in the EU, as well as massive fines, such as the $520 million fine against Epic Games in the US (also by the FTC), the fine against Discord, and many others that I have been discussing in this newsletter. Nevertheless, Amazon's leadership internally names the Prime cancellation process “Iliad,” exploiting users’ cognitive biases and making it very difficult to cancel their accounts. As I have commented recently, privacy is not only about drafting policies but essentially about respecting the user. I reiterate what I wrote a few weeks ago: Amazon needs a new privacy culture. Especially now that they have the capabilities for total consumer surveillance through their new home devices, they need a corporate culture focused on “privacy first” and privacy by design, as Dr. Ann Cavoukian proposed. As today's main topic is dark patterns, check out our training program in Dark Patterns in Privacy.
🔥 Thoughts on AI disruption
How big is the AI disruption? Is AI coming for your job? What about the AI Act? The AI field is growing fast, and it is almost impossible to predict the size and extent of its impact a year from now. My opinion is that AI will somehow affect every field, directly or indirectly, and the most affected areas, at least in the short run, will be the ones that involve tasks that can be easily automated or that could be supported by existing AI systems, such as the creation & edition of content (text, image, code), textual, sound-based & visual communication, mathematical and technical operations, and so on. Another pressing topic is that many people are afraid of losing their jobs due to AI. In that matter, as we watch the development and spread of AI-based automation, I would focus on staying up to date with the latest AI developments and capabilities and would prioritize various soft skills such as communication, critical thinking, problem-solving, self-awareness, empathy, leadership, team management, and so on, as these are essential skills regardless of AI's impact. Additionally, for graduates, I would say that adaptability and continuous learning will be important skills to grow and thrive in an environment of ongoing change and technological disruption. In terms of AI regulation, as I have discussed on multiple occasions in this newsletter, the AI Act in the EU is a necessary step in the context of global efforts to regulate AI and establish AI governance frameworks that respect fundamental rights. The EU is sending a clear message to the world when it establishes that certain practices are forbidden as they cannot be aligned with a fundamental rights-based framework, and some AI systems present a "high-risk" profile and will have to follow stricter rules. The AI Act will hopefully inspire laws in other countries to follow the same basic idea: the protection of fundamental rights matters and should be at the core of AI-related efforts. I am excited about the future, and I think everybody will have to adapt to some extent. I am also optimistic that, as a society, this time, we are better prepared to deal with rapid technological changes and emerging risks. As with any other technology, AI should only be allowed to the extent that it respects fundamental rights.
🔥 New report: consumer harms of generative AI
Forbrukerrådet - the Norwegian Consumer Council - has just launched a new report and webinar on consumer harms of generative AI. Before we continue, as we have been discussing dark patterns today, Forbrukerrådet has excellent resources on the topic, which you can access here. Their 2018 report on dark patterns - Deceived by Design - was one of the first documents I ever read on the topic, and it inspired me to create a taxonomy for dark patterns in the privacy context (read my article here; I also teach it in my course). Now back to AI: Forbrukerrådet's new AI report, called “Ghost in the Machine,” offers a great overview of what the consumer harms are when we are talking about generative AI. Among them are manipulation, bias, discrimination, privacy and data protection, security vulnerability and fraud, intellectual property, environmental impact, and so on. It also covers regulation, in the sense of various fields of law that can be applicable to AI and help tackle these harms, such as data protection law, consumer law, general product safety law, competition law, the AI Act, and so on. It is worth reading. Regarding their new webinar, it is a great opportunity to listen to Brando Benifei, Finn Myrstad, Calli Schroeder, Frederico Oliveira da Silva, and Daniel Leufer's take on generative AI and consumer harms - also worth watching.
🔥 Will there be any human-made content left?
It is clear that this recent AI wave - generative AI-powered - is causing massive disruption and will perhaps change the online environment. What makes me apprehensive is how much we are