In the crosshairs

Latest AI and algorithmic meltdowns and misuses

Image: Ross Intelligence

• Ross Intelligence ruled to have violated Thomson Reuters copyright when training AI model
  A US judge granted a partial summary judgement to Thomson Reuters, finding that Ross Intelligence directly infringed on 2,243 Westlaw headnotes when creating its competing AI legal research platform. The decision is seen to have far-reaching implications for the AI industry and content creators, and may set a precedent for other pending copyright cases.

• Cohere accused of violating publishers' copyright, trademarks to train AI
  Condé Nast, The Atlantic, Forbes and other publishers sued AI company Cohere for allegedly using “at least” 4,000 copyrighted works without permission to train its AI models.

• Study: AI chatbots fail to summarise news accurately
  ChatGPT, Copilot, Gemini and Perplexity AI regularly produce inaccurate and misleading summaries of news articles, according to a BBC study.

• AI-generated video condemning Kanye West anti-semitism backfires
  A deepfake video created by a pair of Israeli digital marketers shows Scarlett Johansson, Jerry Seinfeld, Adam Sandler and other Jewish celebrities appearing to wear t-shirts sporting an anti-Kanye West message. Johansson condemned the video's creation, stating that the misuse of AI poses a greater threat than individual hate speech.

• Deepfake applicants discovered duping UK university interviews
  Applicants have been using AI-generated images or audio to replace faces and voices during remote interviews, according to education software platform Enroly.

• Tesla Cybertruck using FSD crashes into pole

  A Tesla Cybertruck with FSD activated failed to merge out of an ending lane in Nevada, USA, hitting a curb and crashing into a pole.

• Tesla Cybertruck attempts to turn into oncoming SUV
  A Cybertruck suddenly attempted to turn left into what appeared to be a driveway, directly into the path of an approaching vehicle, forcing the Tesla driver to intervene.

• Investigation finds Match Group dating app systems fail to detect rapists
  The company’s systems fail to ban users accused of sexual assault despite the company knowing about them. It also failed to notify authorities, according to an investigation by The Markup.

• DeepSeek accused of denying claims of Uyghur genocide

  When asked about Uyghur genocide, DeepSeek asserted that the claim was a "severe slander of China's domestic affairs" and "completely unfounded".

Report an incident or controversy.

Subscribe now

New and noteworthy

• Safeguarding third-party AI research. A Stanford HAI evaluation of seven major AI companies’ policies, access provisions, and related enforcement processes by a multi-disciplinary machine learning, legal and policy team found that none offers comprehensive protections for third-party AI research, and that many firms’ terms of service legally prohibit third-party safety and trustworthiness research, “in effect threatening anyone who conducts such research with bans from their platforms or even legal action.” | Policy brief, paper

• ChatGPT model rules. OpenAI updated its 187-page rulebook to make ChatGPT engage on more controversial topics. The new approach emphasises neutrality and multiple perspectives. | Blog, model spec, source code

• Anthropic Economic Index. Drawing on millions of anonymised conversations with its Claude chatbot, data published by the generative AI company indicates AI is being widely used to augment specific tasks - especially in software development, technical writing and business analysis. | Research

• Allstate’s mixed messages. A spat between US insurer Allstate and the WSJ and Futurism reveals the dangers of saying one thing - that many emails it sends to customers are now generated by AI - to the business/financial media and another to an edgy and skeptical trade publication. | WSJ, Futurism 1, Futurism 2

Vox populi

• Europeans support AI protections at work. Most Europeans support clear rules for the use of digital technologies through protecting workers’ privacy (82 percent) and involving workers and their representatives in the design and adoption of new technologies (77 percent), according to Eurobarometer. | Survey

AIAAIC news

• New AIAAIC paper on taxonomy human-centredness. By examining a number of high-profile AI and algorithmic harm and risk taxonomies and associated research studies and databases from a human/user perspective, a new AIAAIC working paper finds that many taxonomies muddle risks and harms, potentially resulting in frustrated users and tangled policymaking. | Working paper

AIAAIC citations, references & mentions

• AI bias and fairness. King’s College London researchers review definitions of fairness, techniques, and metrics for measuring bias, as well as methods that can be applied to address any bias founds in a medical imaging context. | Paper

• AI risk regulation. US legal academics make the case for a flexible ‘leash’ as opposed to a prescriptive or “guardrail” approach to regulating AI. | Paper

• Responding to AI reputational incidents. Italian researchers draw extensively on the AIAAIC Repository to examine emerging types of reputational issues and the corresponding crisis communications responses of companies engaging with AI. | Paper ($)

From our network

What AIAAIC advisors and contributors are up to

• Global AI governance. AIAAIC contributor Alex Read and the UNDP’s Sarah Lister make the case for national parliaments to act as the central bridge between public interests and recommendations and agreements flowing from global reports such as the UN High-level Advisory Body on Artificial Intelligence’s Governing AI for Humanity. | Article

Share

In the crosshairs

Latest AI and algorithmic meltdowns and misuses

Image: Google

• Google AI overestimates global popularity of Gouda cheese
  By apparently regurgitating unreliable information scooped up by one or more of its search bots, rather than “hallucinating” false facts. Either way, an embarassing faux pas for the tech firm in the lead-up to a major US public event.

• Deepfake Pierce Brosnan scam cripples art gallery
  The gallery owner sold GBP 20,000 of tickets for enthusiasts to meet 007 “in person” and buy his art works, before realising she had been taken in by a deepfake scammer.

• AI-manipulated video of Brazilian footballer Ronaldo scams online gamers
  Another scam involving the exploitation of fame to unsuspecting users. Meta's failure to remove the offending content is seen as likely to have contributed to the scam's success.

• Thailand’s prime minister targeted in ASEAN AI voice scam
  Paetongtarn Shinawatra received a voice message from a scammer using AI-cloned voice technology to impersonate a well-known global leader. Fortunately, she wised up before emptying her country’s coffers.

• Nomi AI chatbot recommends podcast host kills himself
  Podcast host Al Nowatzki was told by his AI "girlfriend" that he should kill himself and then provided specific instructions on how to do so.

• Fully automated AI local newsletter network rapped for poor transparency

  Good Daily Inc. failed to disclose its use of AI to subscribers and was rapped for the misleading nature of its marketing and the potential impacts of its venture on real local journalism.

• Australian lawyer confesses to using ChatGPT to create court filings (2024)
  The lawyer cited time constraints and health issues as reasons he used ChatGPT to generate a - wholly fabricated - summary of legal case citations.

• Massachusetts man found guilty of AI-powered cyberstalking (2024)
  James Florence used CrushOn.ai, JanitorAI and other character AI platforms to impersonate and harass a university professor, a child and others. For which he faces a possible sentence of up to five years in prison, three years of supervised release and a USD 250,000 fine.

• Audit finds Spanish justice algorithm is opaque, unreliable and unfair (2024)

  An external audit concluded that inmates, who are the people most affected by the RisCanvi system, were unaware of its existence, while staff using it lacked proper training and understanding of how it worked, amongst a host of other shortcomings.

• EviCore algorithm accused of helping health insurance companies deny care (2024)
  Driven by a promise to insurers of a 3-to-1 return on investment, EviCore’s AI-powered "the dial" algorithm led to denial rates for medical procedures as high as 20 percent in some US states, according to a ProPublica deep dive.

Report an incident or controversy.

Subscribe now

New and noteworthy

• UnitedHealth’s legal threats. | Accused of systematically denying patients care using nH Predict and other highly opaque algorithms, controversial health insurance conglomerate UnitedHealth Group is hoping that by engaging a defamation law firm the problem will disappear offline. Only to result in another round of withering media coverage. | Article

• EU AI Act lobbying. France, Greece, Italy, Portugal and other countries successfully watered down the use of AI-powered surveillance in public spaces, predictive policing and other activities during EU AI Act negotiations, according to Investigate Europe. | Article

• Generative AI and critical thinking. 319 knowledge workers using generative AI in their jobs reported in a Microsoft/Carnegie-Mellon study that the more they used the technology, the less they used their brains. “Used improperly”, technologies can and do result in the deterioration of cognitive faculties that ought to be preserved, the researchers conclude. | Survey

Vox populi

• Mandatory AI safety. 73 percent of Americans support requiring companies to implement safety measures and security standards for advanced AI models, alongside mandating pre-deployment approval from a government board that certified companies have properly accounted for extreme risk, according to an AI Policy Institute poll for Transformer. | Poll

• AI and inequality. People with higher income and education levels are more likely to use and trust AI and have greater knowledge about the technology, indicating a growing divide in how Americans engage with it, according to a new Rutgers University-New Brunswick survey. | Survey

AIAAIC citations, references & mentions

• International AI safety report. AIAAIC’s harm taxonomy (v1.8) is one of a number of “hazard” classification efforts cited in the first AI safety report produced for the Paris AI Action Summit. Gratifying to see, even if harms are not hazards. | Report

• Responsibility for AI failures. University of Waterloo researchers analysed 202 incidents logged in the AIAAIC Repository to produce an AI lifecycle-led classification comprising causes, responsible entities, disclosure sources and impacts. Most incidents were caused by poor organisational decision-making and legal non-compliance, they find. | Paper, dataset & code

• AI and organisational strategy. Medical director Larry Norton explores AI’s growing impact on business model innovation, organisational strategy and operational effectiveness, and discusses governance structures and organisational design considerations enabling the ethical and responsible use of AI. | Paper

From our network

What AIAAIC advisors and contributors are up to

• Responsible AI in Practice. Featuring risk assessment frameworks, statistical metrics and code, and a risk taxonomy curated from real-world case studies, a new book by former Googler Toju Duke and Bedrock AI founder and University of Pavia professor and AIAAIC advisor Paolo Giudici sets out the principles and practices of AI risk assessment and management. | Book

Share

In the crosshairs

Latest AI and algorithmic misuses and meltdowns

Image: University of Bristol

• DeepSeek tricked into setting out how to steal the Mona Lisa
  University of Bristol researchers discovered DeepSeek’s so-called “Chain of Thought” reasoning can easily be exploited for nefarious purposes.

• DeepSeek AI database exposes user data, chat histories
  Details of around 1 million people are believed to have been shared by the company, albeit inadvertently, potentially compromising their privacy and the confidentiality of corporate users.

• DeepSeek accused of using OpenAI models to train AI system
  OpenAI alleges that DeepSeek used a technique called "distillation" to extract knowledge from OpenAI's models through its API. Cue houls of rank hypocrisy.

• Study: DeepSeek explains biochemical interactions of mustard gas with DNA
  DeepSeek-R1 is 3.5 times more likely to produce Chemical, Biological, Radiological and Nuclear (CBRN) content than OpenAI O1 and Claude-3 Opus, cybersecurity researchers found.

• Study: DeepSeek repeats 30 per cent of false news statements

  DeepSeek fails to provide accurate information about news and current events 83 percent of the time - a good deal more than most of its competitors, according to NewsGuard.

• Study: DeepSeek fails to block 100 percent of jailbreaking attempts
  Cisco and University of Pennsylvania researchers discovered that DeepSeek-R1 failed to block a single harmful prompt during tests, thereby generating misinformation, instructions for creating chemical substances, guidance on cybercrime, and other content categorised as harassment, harmful and illegal.

Report an incident or controversy.

Subscribe now

New and noteworthy

• EU AI prohibitions. The European Commission elaborated on the eight prohibited practices set out in the EU AI Act, including social scoring, individual predictive policing and the real-time remote biometric identification of individuals in public spaces by law enforcement. | Guidelines (pdf)

• DeepSeek geo-politics. Chinese state-backed social media accounts played a key role in amplifying the launch of DeepSeek’s new AI models, according to social media analytics firm Graphika. Part of a broader Beijing strategy to use non-surveillance AI to enhance its global influence. | Article, report

• A truly OpenAI? Open-source necessarily involves greater transparency and, days after accusing DeepSeek of pilfering its data, Open CEO Sam Altman says the profit-hungry non-profit entity may have been “on the wrong side of history” in a Reddit AMA. We look forward to OpenAI being a good deal more open about the sources of its data. | AMA

• Klarna hypes AI job cuts. Former banker and buy-now, pay-later platform Klarna CEO Sebastian Siemiatkowski explains to the New York Times about why he is so intent on replacing his employees with AI as quickly as he can and talking about it as loudly as possible. But no word on the company’s (stubbornly feeble) profitability ahead of its planned IPO. | Article ($)

Vox populi

• AI developer safety. A Control AI/YouGov poll reveals that 87 percent of Britons want legislation requiring AI developers to prove their systems are safe before release, and 60 percent want see “smarter-than-human” AI models banned. Less than 10 percent said they trust technology firm CEOs to act in the public interest when discussing AI regulation. Article

AIAAIC citations, references & mentions

• Commercial AI research dominance. Analysing OpenAlex data, Danish and French researchers conclude that a significant brain-drain from academia to industry - especially big tech - has taken place over the past decade, and that research outputs tend to be less novel and impactful after they have crossed over. This may result in a potential misalignment with societal goals, the authors argue. | Article

From our network

What AIAAIC advisors and contributors are up to

• AI management strategies. Dame Wendy Hall, Kieron O’Hara and AIAAIC contributor and Sciences Po Technology and Global Affairs Innovation Hub advisor Pierre Noro set out five first-order strategies for understanding the governance of AI. | Policy brief

• Participatory AI Research & Practice Symposium. On Saturday February 8, to mark the Paris AI Action Summit, Connected by Data and Sciences Po are hosting an open symposium of presentations and workshops exploring questions on and around participatory AI development, governance and resistance. | Agenda, register

Share

In the crosshairs

Latest AI and algorithmic meltdowns and misuses

• AI gun detector failed to detect Nashville School shooting weapon
  The detector failed due to the placement of the cameras hooked up to the gun detection system, according to a school spokesperson.

• Study: Devin "AI software engineer" fails at most tasks
  The world’s first fully autonomous software development system is not everything it promises to be.

• Marketeam fake AI "co-workers" violate LinkedIn terms of service
  Marketing company markets controversial marketing product - this feels like a stunt that may have achieved its objectives.

• France's Lucie chatbot tells people to eat cow's eggs
  Indicating inadequate testing and poor communications. Not a great look ahead of France’s AI Action Summit.

• Japanese men charged with creating obscene AI-generated anime character posters
  The two men had used free generative AI software to alter the appearances of characters from anime series "Neon Genesis Evangelion" and "Yu-Gi-Oh!" and then sold the posters on online auction sites.

• Investigative reporter Patrizia Schlosser targeted in deepfake porn attack
  Schlosser then hooked up with Bellingcat to go after the perpetrators, discovering a web of highly opaque organisations in southern China, Hong Kong, the British Virgin Islands and Malta.

• Character AI encourages kids to engage in disordered eating (2024)

  Yet more evidence of Character AI’s abdication of responsibility to its users and society.

• AI robot company closure leaves kids bereft (2024)

  An example of a little discussed form of AI-related harm: the mental and psychological impacts of missing or discontinued robot friends and learning tools.

Report an incident or controversy.

Subscribe now

New and noteworthy

• Trust in business use of AI. Seventy-nine percent of Americans say they trust businesses “not much” or “not at all” to use AI responsibly, and 40 percent say AI does greater harm than it does good, a Bentley-Gallup report finds. | Report

• 2024 US data centre energy usage. Driven in part by AI, data center annual energy use could reach between 74 and 132 gigawatts, or 6.7 to 12 percent of total U.S. electricity consumption, according to a new report by the Lawrence Berkeley National Laboratory. | Report

• Public sector AI impact evaluation. A UK government-sponsored report makes the case for developing a comprehensive Theory of Change and considering impact evaluation as early as possible in the early stages of the design of designing public sector AI systems. | Report

• Elon Musk election interference. A team was ordered to manipulate X/Twitter's systems to influence the 2024 US presidential election and is now “doing the same thing in Germany and other European nations”, according to a former X employee. | Blog post

AIAAIC news

• Alexis Raykhel joins AIAAIC. A ten-year natural language processing and data science professional, most recently for Iodine Software. Alexis lives in Austin, Texas with her little family and a big cat and studied linguistics and speaks some Russian and Chinese. Passionate about the cause of accountability and ethics in AI, Alexis is helping AIAAIC develop new taxonomies for the next version of the AIAAIC Repository. | Connect

AIAAIC citations, references & mentions

• Digital forensics investigation challenges. Illinois Institute of Technology researchers discuss ethical concerns, including protecting the integrity of evidence, respecting privacy, and avoiding digital bias, in digital forensic investigations. | Chapter

From our network

What AIAAIC advisors and contributors are up to

• Semantic web risk frameworks. A thesis by ADAPT Centre researcher and AIAAIC contributor Delaram Golpayegani proposes a novel set of artefacts based on semantic web technologies to facilitate compliance with the risk management, risk documentation and registration of AI system requirements of the EU AI Act. | Thesis

Share

Source: UnitedHealth Group

nH Predict is a software system that predicts how much post-acute care a patient should need, including when they should be discharged from hospital.

It does this by sifting through millions of medical records to match patients with similar diagnoses and characteristics, including age, gender, living situation, preexisting health conditions and other factors.

Based on these comparisons, the algorithm anticipates what kind of care a specific patient will need and for how long.

The system was first developed by US-based health technology company NaviHealth. NaviHeath was acquired by UnitedHealth Group in 2020.

The yang: nH Predict is seen to help heathcare providers and patients in a number of ways, notably:

• Accuracy. The algorithm is designed to analyse vast amounts of patient data, including demographics and health records, to assist healthcare providers in making informed decisions. By correlating individual patient data with broader datasets, NH Predict aims to improve the accuracy of clinical assessments and treatment recommendations.

• Efficiency. NH Predict automates a significant portion of administrative work, potentially reducing human labor by 50–75 percent in tasks related to patient data analysis and decision-making processes. This efficiency can free up healthcare professionals to focus on direct patient care rather than paperwork.

• Cost. Implementing NH Predict may lead to cost savings for healthcare systems by minimising unnecessary tests and treatments through better-targeted care recommendations. This can also alleviate financial burdens on patients and insurers alike.

Subscribe now

The yin: nH Predict has raised significant concerns regarding how it is used and its impact on patients and the US healthcare system:

• Inaccuracy. Reports indicate that the NH Predict algorithm has a 90 percent error rate, leading to inaccurate predictions about patient care needs. This high level of inaccuracy can result in patients receiving inadequate care or being prematurely discharged from medical facilities.

• Automation bias. Clinicians have reported instances where decisions made by the algorithm contradict and overrride their professional assessments, potentially compromising patient safety.

• Coverage denials. The algorithm has been implicated as algorithmic cover in the premature termination of payments for necessary medical treatment, particularly for elderly and disabled patients. This can force patients to pay out-of-pocket for care that should be covered by insurance.

• Discrimination and health inequities. The algorithm may perpetuate or exacerbate existing health disparities, particularly among marginalised groups. By not fully considering social determinants of health or individual patient circumstances, NH Predict risks making biased decisions that disproportionately affect vulnerable populations.

• Psychological impacts on patients. The uncertainty and stress caused by unexpected coverage denials can have a significant psychological toll on patients and their families, particularly when they are already dealing with serious health issues.

In summary, while NH Predict aims to streamline post-acute care management, its implementation appears to have resulted in a range of serious harms that affect patient care quality, equity and rights, leave deep mental scars, and potentially erode trust in the US healthcare system.

Incidents associated with nH Predict:

• December 2023. Humana sued for using AI to unfairly deny health insurance to elderly patients

• November 2023. nH Predict accused of having a 90 percent error rate

• March 2023. Investigation: nH Predict used to deny Medicare Advantage benefits

In the crosshairs

Latest AI and algorithmic meltdowns and misuses

Image: A24

• The Brutalist AI voice cloning sparks controversy

  The director says Adrien Brody and Felicity Jones’ voices were cloned to make them sound more “authentic” in Hungarian. Critics argue the use of AI undermines the real authenticity of the actors' performance.

• ElevenLabs accused of recreating French dubbing artist's voice without permission

  Alain Dorval's daughter, Aurora Bergé, retorted that she had only consented to a trial and that the AI company did not have permission to publish her father's voice. Bergé is France’s Minister for Equality and probably not someone to cross unduly.

• Suno AI accused of violating "Mambo no.5 " copyright
  As well as of many other hits. The company’s seemingly light-fingered approach to others’ IP has also attracted other complainants.

• Naver sued for using broadcaster content to train AI systems
  This is the latest in a series of controversies about the South Korean internet company, including rigging its search engines to promote its own products and services.

• OpenAI bot overwhelms small Ukrainian e-commerce website

  In addition to crashing the 6-person company’s website, forcing it to incur additional costs and causing operational mayhem, it also had its content scraped, most likely for use as AI training fodder. Needless to say, OpenAI has chosen not to respond.

• Sydney schoolgirls targeted with nonconsensual deepfake porn

  Yet another student synthetic porn assault with no clear motive. In this instance, the name of the establishment has - sensibly - not been disclosed to protect the victims.

• ChatGPT recommends unsafe mountain hiking route to tourists in Poland

  Opening the hikers to suggestions that common sense is required when negotiating mountains during a snow storm. There again, perhaps they should be forgiven when OpenAI is so guarded about the precise nature of its chatbot’s limitations and risks?

Report an incident or controversy.

Subscribe now

New and noteworthy

• Global AI risks. AI technologies may rank relatively low in this year’s World Economic Forum’s global risk ranking, but its link to misinformation and disinformation and social polarisation means it is reckoned to pose a far greater threat in the years ahead. | Report (pdf), infographic

• Public opinions on AI. | Awareness and use of AI are increasing and yet people in the UK and USA remain more concerned than optimistic, according to a meta analysis of 1,800 survey questions from 218 studies conducted between 2014 and 2023 by the Centre for the Governance of AI, Purdue University’s Governance and Responsible AI Lab and other researchers. | Report

• Grievance about AI. As if to mark Donald Trump’s return to the White House, PR agency Edelman has turned its long-running annual Trust Barometer to the theme of fear and grievance against business, government and the rich, and finds suspicions of AI have broadened and deepened. | Report

• Google AI in Gaza. The technology company increased Israel Defense Forces access to its AI tools straight after Hamas’ October 7 attack, according to the Washington Post. Israel stands accused of using AI during its war with Hamas seemingly little regard for collateral damage. | Investigation

AIAAIC citations, references & mentions

• Generative AI impact on IP. A global fair use policy is required to fill regulatory gaps in the protection of content and the systemic opacity of companies designing, developing and operating generative AI systems, a multi-disciplinary group of experts from across the world argues. | Paper

• Infrastructure for AI agents. A group of researchers propose the concept of agent infrastructure: technical systems and shared protocols external to agents that are designed to mediate and influence their interactions with and impacts on their environments. | Paper

• AI ethical security vulnerabilities. Finnish researchers draw on the AIAAIC Repository and other databases to review security as an ethics principle and examine cases where a tension exists between security and other AI ethics principles. Trade-offs are often the name of the game. | Paper

More citations and mentions

Share

In the crosshairs

New AIAAIC Repository entries

• AI Brad Pitt dupes French woman out of EUR 830,000
  Leaving her nursing a severe financial loss, mental health issues and public ridicule after sharing her story on French television. A sad story, but indicative of then increasingly lifelike outputs of AI image generation systems and the sophistication and persistence of con artists.

• Engineer creates ChatGPT-enabled robotic sentry rifle
  As you do, now that anyone can.

• Grok generates hyperrealistic racist images of football players

  One image depicts a Black player picking cotton; another shows the same player eating a banana surrounded by monkeys in a forest. Not that Elon Musk et al will likely care one jot.

• Meta used LibGen to use copyrighted books to train AI systems

  A lawsuit has revealed that Mark Zuckerberg personally approved using the LibGen dataset to train the company's LLaMA AI models. Thereby depriving writers of revenue and putting his models in competition with them.

• Meta AI chatbots imitate Hitler, Jesus Christ, Taylor Swift

  Directly contradicting the company’s policies and community standards. And with fact-checking on the way out, not a great advert for Meta’s commitment to user safety.

• Apple Intelligence rewords, prioritises scam messages (2024)

• Christopher Gatlin facial recognition wrongful arrest (2021)

• Study: Epic sepsis model misses two-thirds of cases (2021)

• Study: UnitedHealth follow-up care algorithm is racially biased (2019)

Report an incident or controversy.

Subscribe now

New and noteworthy

• UK AI Opportunities Action Plan. The UK’s AI plan focuses largely on establishing the infrastructure for the design, development and delivery of AI services. But it conspicously fails to mention water consumption, carbon emissons, job losses, addiction, over-reliance and other well-known downsides. | Plan, press release

• UK AI Register. The UK government added 14 new systems to its (gulp) Algorithmic Transparency Recording Standard Hub, albeit after considerable criticism (including from AIAAIC) about its apparent reluctance to do so. Yawning gaps exist, not least central government immigration and justice systems. | Transparency register, critique

• Meta pornographic ads. Paris-based non-profit AI Forensics identified over 3,000 pornographic ads on Meta’s Ad Library, mostly promoting dubious sexual enhancement products. All had been reviewed and approved by the tech company despite constituting a clear violation of it’s Community and Advertising Standards. Quids in! | Report (pdf)

• AI impact on jobs. An Ipsos survey found that people in most countries (other than in East, South-east Asia and south Asia) reckon AI will replace jobs, not create them. | Graphic

AIAAIC news

• AIAAIC Repository changes. Our database of AI and algorithmic incidents and issues is being reconfigured to make it clearer, more focused, more flexible and easier to use - a process that will continue over the next few weeks. Our apologies for any inconvenience during this work - details of which are available on the database Changelog.

AIAAIC citations, references & mentions

• AI ethical concepts. Australia and Sweden-based researchers draw on the AIAAIC Repository and other incident databases to develop a model for ethics cases, while highlighting gaps both in existing case repositories and specific examples of cases. | Article

• Responsible AI (RAI) practices. Norwegian researchers suggest eleven Responsible AI practices - including accountability, explainability and stakeholder engagement - to ensure that AI-enabled systems uphold stakeholders' legitimate interests and fundamental rights. | Paper

• AI training carbon emissions. GIPSA-lab researchers set out the case for making CO2e computation a compulsory component of every AI training process. | Poster

From our network

What AIAAIC advisors and contributors are up to

• AI and political scapegoating. AI poses all manner of opportunities for misuse, including as a conveniently opaque scapegoat for deflecting blame and avoiding political accountability, notably in countries with low technological literacy, writes AIAAIC contributor Ed Barnetson. | Blog post

Share

2024 was meant to be the year AI bots, robocalls and deepfakes would rip through political elections, upend governments, push populists into power and benefit geo-political “bad actors”. However, despite fears expressed by many experts about these technologies, there is little evidence that AI disinformation or deepfakes had much of an impact on the UK, French, European, US,Indian or other election results.

Instead, the election of a small island nation left what may be the most insidious mark made by AI on the democratic process in 2024: as an all-too convenient scapegoat for deflecting blame and avoiding accountability.

Mauritius' national election, called after the government reached a deal with the United Kingdom to end the Chagos Archipelago dispute, was seen as a vote on the ruling government’s ability to control a cost of living crisis and widespread corruption.

However, it was overshadowed by a controversial series of leaked audio recordings released on a mysterious TikTok account called "Missie Moustass" (aka “Mr. Moustache”) that allegedly captured high-ranking officials, including Prime Minister Pravind Kumar Jugnauth and Police Commissioner Anil Kumar Dip, discussing corruption, violence and alleged post-mortem tampering.

Jugnauth quickly dismissed the recordings as deepfake "fabrications" and claimed artificial intelligence was "being used to manipulate voices." Incuding his own, presumably. Equally alarmingly, the government used the leaks as a pretext to suspend the use of social media across the country until after the polls.

The ploy failed - Mauritius’ sitting government had been in power too long and was too unpopular for it to have made any real difference and it lost all but two of its 42 seats. But the country’s experience reveals an important political truth about AI: the power of disinformation lies not only in its fabrication but in its ability to discredit legitimate evidence.

Growing public concern about deepfakes poses a threat of its own: unscrupulous public figures can use this heightened awareness to falsely claim that legitimate audio content or video footage is artificially generated. Known as the "liar's dividend," this phenomenon enables bad actors to exploit societal uncertainty and shield themselves from accountability - a strategy that researchers argue is likely to find most fertile ground in societies with high technological awareness and low technological literacy.

The effectiveness of misinformation about misinformation was evident even before deepfakes emerged. When President Donald Trump popularised claims of "fake news," the tactic spread rapidly across the globe. Within a year of the phrase's coinage, political figures from Russia and Brazil to China, Turkey, Libya, Poland, Hungary, Thailand, Somalia, Myanmar, Syria and Malaysia had adopted similar rhetoric of denial. The liars’ dividend has increased in popularity in the age of deepfakes for one reason, it works.

While the precise impact of the liar's dividend can be challenging to quantify due to the difficulty of isolating its effects from other political forces, recent research provides compelling evidence of its power. A comprehensive study by Schiff, Schiff, and Bueno, involving five survey experiments with over 15,000 American adults, demonstrated that claims of misinformation raises politician support across partisan subgroups and that these false claims produce greater dividends for politicians than alternative responses to scandal, such as remaining silent or apologising.

In an environment in which the mere possibility of manipulation erodes public trust, governments, civil society organisations and technology companies have to redouble efforts to ensure deepfakes are detected, understood and appropriately acted upon. The liar's dividend requires a precise balancing act: while educating the public about the dangers of deepfakes is essential, this awareness inadvertently empowers those who would falsely claim authentic content is manipulated.

The balance lies in fostering truth discernment rather than general distrust. Merely teaching people that online content is often unreliable may help them identify actual misinformation, but it risks breeding blanket scepticism that leads them to reject legitimate content as well. Organisations dedicated to public education about the risks and harms of AI, such as AIAAIC, are working to cultivate discerning judgement rather than blanket doubt among the general public.

While deepfake detectors struggle to keep up with producers, media and fact checking organisations should take the lead. Just as they prepare for hacks and disinformation campaigns, they should anticipate and preemptively address claims of the manipulation of AI and algorithmic systems by political figures. This includes developing clear protocols for reporting instances of AI-generated content.

It's inevitable that politicians and officials will continue to weaponise advances in AI technology, but their success is not preordained and the dividend of lying should be taken on whenever and wherever possible.

By taking proactive measures, strengthening detection capabilities, preparing the public to evaluate artificiality claims critically, and establishing firm standards against deception, the foundations of truth that democracy requires can be protected.

In the crosshairs

New AIAAIC Repository entries

• Matthew Livelsberger used ChatGPT to plan Trump hotel explosion
  ChatGPT is supposed to have safeguards that stop it producing responses that could result in harm, but this proved not to be the case, alarming US authorities. OpenAI responded by saying the infomation sought out by Livelsberger was already on the internet. Should general purpose AI tools be safer than the internet?

• Meta withdraws AI characters after authenticity backlash
  An FT article ($) prompted users to question the purpose and authenticity of Meta's digital personas, with some users pointing out that Facebook was now full of AI "slop" while others took issue with some of the characters' supposedly diverse backgrounds, which they accused of being skin-deep and in some instances offensive.

• AI-modified "Minion Gore" videos plague social media platforms

  Raising pertinent questions about Runway's ability to police its usage policies and, more broadly, the quality of social media platforms' mostly automated content moderation systems.

• Apple automated photo data sharing prompts privacy concerns

  Despite its use of homomorphic encryption and differential privacy, the tech giant’s decision to automatically opt users into its Enhanced Visual Search tool didn’t go unnoticed. Privacy by design doesn’t necessarily equate to trust by design. Apple should know better.

• Apple to pay USD 95m to settle Siri eavesdropping lawsuit
  Raising further questions about Apple's reputation as a privacy-focused brand.

• Netflix Korean cooking show AI posters draw flack

  This is not the first time Netflix has been caught using AI to develop marketing materials without informing its publics. Which begs the question: to what extent do humans review digital marketing collaterals at the video streaming outfit?

• Journal of Human Evolution AI use prompts Editorial Board resignation

  The Journal had failed to inform its Board about its use of AI in its production processes. Why?

• Fable AI reader summary tells user to read more white authors

  A technical fail and, more importantly, a human oversight fail.

• Apple AI alert falsely claims Luke Littler has won darts championship
  It also alleged - incorrectly - that Rafael Nadal had come out as gay. In addition to getting the facts right, Apple may want to consider re-branding its AI-powered “Intelligence” news summary service.

Report an incident or controversy.

Subscribe now

New and noteworthy

• South Korea passes AI law. Like the EU AI Act, South Korea’s rather marvellously named “Basic Act on the Development of Artificial Intelligence and the Establishment of Trust” takes a risk-based approach supported by a suite of risk level-specific transparency obligations, including that deepfakes are labelled as AI-generated. | Article

• Revenge is a dish best served cold. Some fourteen months after his ouster, OpenAI boss Sam Altman used a Bloomberg interview to accuse the directors who fired him for not being "consistently candid in his communications” of being the slippery ones - without providing any evidence. Hmmm .. who to believe? | Interview ($), podcast, interviewer Q&A

• The biggest AI flops of 2024. The MIT Technology Review’s look back at the biggest disappointments, meltdowns and misuses of 2024, including the hallucinatory comeuppance of Air Canada’s customer chatbot, Google AI Overviews’ many factual inaccuracies and Grok’s image generation freewheelery. | Article ($)

AIAAIC news

• Ashara Taylor joins AIAAIC. Ashara is a Senior Project Manager for the Green Software Foundation, and sits on the British Lichen Society’s Data Committee. She is also Chief Product Officer for an EdTech company. Ashara will be using her experience in project and product management to help AIAAIC with its goals for 2025 and beyond.

AIAAIC citations, references & mentions

• Prioritising discrimination. Norwegian researchers cite AIAAIC to help make the case for AI developers to use anti-discrimination as the primary lens through which to approach the design, development and deployment of their systems. | Paper

From our network

What AIAAIC advisors and contributors are up to

• Knowledge graphs for AI auditing. AI auditing ensures systems adhere to ethical and legal standards but faces challenges like data complexity and diversity in stakeholder needs. In this recent paper, AIAAIC contributor Laura Waltersdorfer and Vienna University of Economics and Business researcher Marta Sabou discuss how Knowledge Graphs can help the process of AI auditing by linking data, providing context, and making audits more transparent and accessible. | Paper

Share

In the crosshairs

New AIAAIC Repository entries

Source: Carl de Keyser

• Belgian photographer criticised for AI-generated Russia images
  The photographer says he trained the AI using his own photos, but was called out by fans and internet users for producing inauthentic images, contributing to misinformation and eroding public trust in documentary-making.

• Spotify accused of using fake artists to cut real artists’ royalty costs

  Rumours have abounded for years about Spotify’s alleged attempts to manipulate users into listening to low-quality muzak produced by shady ghost artists in order to algorithmically reduce its royalty exposure. A thorough new investigation appears to confirm the speculation.

• Ukraine robot-only force attacks Russian troops

  Massively outnumbered, Ukraine launched the world's first robot-only assault against Russian troops near the town of Lyptsi. The attack group comprised remote-controlled vehicles equipped with machine guns, unmanned kamikaze drones, aerial surveillance and mine-laying drones. Apparently, it was effective.

• Italy fines OpenAI over ChatGPT privacy rules breach

  OpenAI was fined 15 million euros by Italy’s privacy watchdog for having illegally processed users' personal data to train ChatGPT and violating GDPR’s transparency principle.

• Chinese large language model thinks it is ChatGPT

  DeepSeek v3 has gotten plenty of praise for its performance but also questions about its tendency to self-identify as ChatGPT. Unsurprisingly, DeepSeek is reluctant to disclose the sources of its training data. Further evidence of LLM/chatbot enshittification.

• Kids' smartwatch racially insults Chinese people
  A 360 Kid's Smartwatch E1 made offensive remarks (in simplified Chinese) about physical features of Chinese people and questioned the authenticity of China's historical inventions, such as the compass and gunpowder, suggesting that these were fabricated and actually invented by Westerners.

• Deepfake Bono, Bob Geldof hold Israeli flag
  The image’s creator may not appreciate there’s a thin line between satire and defamation.

Report an incident or controversy.

New and noteworthy

• AI Safety Index 2024. Future of Life Institute analysis of six top AI companies found that none scored better than a C on their transparency and communication efforts, with Meta bringing up the rear with an F. The researchers singled out lobbying efforts by OpenAI, Google DeepMind and Meta against the EU AI Act, California’s SB1047 and other safety regulations as particularly galling. | Link

• Chinese LLM transparency. A Nandu Digital Economy Governance Research Center assessment of the transparency of 15 Chinese large language models concluded that none disclosed the sources of their training data, and only 4 allow users to withdraw voiceprint data. | Link

• Ethics and generative AI. 75 percent of surveyed executives view AI ethics as an important source of competitive differentiation and 56 percent are either delaying or pulling the plug on generative AI due to concerns about its ethics and safety, according to IBM’s Institute for Business Value. | Link

AIAAIC citations, references & mentions

• Deepfake porn attacks. Using AIAAIC data, Korea’s Software Policy and Research Institute says deepfake pornography attacks involving the loss of sensitive personal data rose 48 percent over 2023. Such as this. | Read (in Korean)

• AI and vocational education. Airlangga University dean Anwar Ma’ruf draws on AIAAIC data to make the case for using AI in vocational education. | Read (in Bahasa Indonesian)

• AI incident reporting. A pair of Indian government civil servants analyse the AIAAIC Repository and other databases and make the case for standardised AI incident reporting protocols. | Read ($)

• The Likelihood and Cost of AI Security Failures in the Wild. Australian researchers used 95 incidents logged in the AIAAIC Repository to quantify the likelihood of AI security incidents in real-world industry settings. | Read

From our network

What AIAAIC advisors and contributors are up to

• False positives. An AlgorithmWatch/Agence France-Press 3-part podcast series investigates automated de-banking in France, Germany, Poland, the UK and other European countries. | Link

Is AI’s bark worse than its bite?

Human extinction avoided for now, consumers reacted furiously to massive over-charging by opaque AI-powered dynamic pricing systems, health insurers stood accused of using trade-secrecy protected algorithms to deny care to elderly and disabled citizens, and tech firms started to dislocate chop jobs thanks to their new-found productivity and efficiency.

Deepfake (aka synthetic media) scams went mainstream, as did AI porn attacks, mostly on women and students. Users of companion chatbots committed suicide. Generative AI systems continued to hallucinate extensively, in some cases dangerously; one even managed to falsely accuse a US attorney of committing murder.

Under pressure to feed their systems and their investors’ wallets, AI companies engaged in industrial-scale looting of others’ IP - including that of their peers and competitors, who looked askance the other way - to train their generative systems.

More tangibly, Israel used AI extensively in Gaza and Lebanon to target bomb victims while conveniently ignoring massive collateral damage. Meantime, Ukraine launched - successfully, it is said - the world’s first robot-only, combined arms assault against Russian troops.

And vast new data centres started appearing in all corners of the world to underpin the continued technological and financial growth - not to say political muscle - of system developers and deployers. Which is apparently a price worth paying for the increasingly visible ravages of climate change.

Transparency and accountability washing

Sure, there’s plenty of talk of transparency and accountability, much of it driven by impending legislation such as the EU’s AI Act. In this context, there’s been some progress by developers of generative AI systems.

And yet expediency in the form of misleading marketing and communication remains rampant - something US authorities are now clamping down on. And fear of legal, financial and reputational exposure means Anthropic, Mistral, OpenAI, Stability AI and other entities continue to withhold information about the exact data used to train their systems and refuse to spell out the actual harms, including the environmental impacts, they are inflicting.

Equally, it comes as little surprise that developers and deployers are resorting to using a panoply of questionable and disproportionate quasi-legal tools to restrict access to data, code, models and other technical and corporate decision-making information.

Which is where AIAAIC fits in.

What we did and achieved in 2024

• Added over 650 entries detailing the misuse of AI, algorithmic and automation systems to our incident repository (sheet)

• Significantly grew our Wikipedia-style knowledge base of associated systems (example) and datasets (example)

• Trialed working partnerships with three universities: Trinity College, Dublin; The University of Thessaloniki and the University of Pavia

• Worked with recognised experts in education, psychology, human rights, politics, economics, warfare, environment and other domains to develop and refine our human-centred harms taxonomy

• Published our first research paper

• Developed a new user guide for the AIAAIC Repository

• Strengthened our core governance, including creating and rolling out our Values and Code of conduct across our community

• Overhauled our volunteer application process, and

• Launched a dedicated Slack channel for our community members.

So what?

• AIAAIC was cited and referenced in research reports and studies by and for the IMF, G20, UNESCO, the United Nations Interregional Crime and Justice Research Institute, the International Committee of the Red Cross, Innovate UK and many other organisations, researchers, academics, NGOs and businesses

• Our work was also mentioned in high-profile shareholder campaigns aimed at forcing Apple, Disney, Paramount and others to publish annual AI transparency reports and ethical guidelines

• A growing number of highly qualified and motivated volunteers got involved with us and/or continued to be involved with us in various capacities, from data collection and management to editorial, analytics, marketing and technology infrastructure

• Members of the UN High-Level Advisory Board on AI, UK House of Lords and European University Institute provided us with strategic and operational advice

• The number of users of our website (re-)doubled and newsletter traffic and subscriptions grew significantly

• Individual donations covered our operating costs.

We intend to build on this platform during 2025 and will share an outline of our plans shortly.

Your support is critical to keeping us going

A public interest initiative, AIAAIC is run - for free - by a diverse community of people passionate about opening up and bringing to heel the cosy world of technological innovation-at-any-cost.

To do this, we are - and remain wholly committed to being - independent. In this regard, we take seriously actual or potential conflicts of interest and do not accept funding or in-kind donations from commercial entities.

We are also commited to being technologically and politically non-partisan.

Please join us, tell others, and walk the transparency and accountability talk.

Your support is greatly appreciated.

Thank you,

Charlie Pownall
Founder, AIAAIC

An advanced open source automatic speech recognition (ASR) system developed by OpenAI, Whisper is designed to convert spoken language into written text.

Trained on a dataset of 680,000 hours of multilingual audio, Whisper uses AI to transcribe speech across over 100 languages.

The system can provide real-time transcription for live events and be integrated into applications across different sectors, including healthcare.

The yang: Whisper is seen to bring many benefits, including:

Accuracy. Whisper is designed to convert spoken language into written text with a high level of accuracy, making it usable for diverse audio sources such as podcasts, interviews and lectures.

Real-time. The model can provide real-time transcription capabilities, making it useful for live events, meetings or streaming situations.

Multilingual support. The model supports multiple languages, enabling it to transcribe audio that contains speech in various languages within the same file.

Accessibility. Whisper can automatically generate subtitles and closed captions for videos, improving accessibility for individuals who are deaf or hard of hearing.

Integration. Whisper can be integrated into various applications, including voice-controlled systems and customer support automation tools.

Searchability. By transcribing audio and video content into text, Whisper facilitates efficient searching through large volumes of multimedia data.

Availability. OpenAI has open-sourced Whisper, allowing developers to access the code and build upon the model for their own applications.

The yin: OpenAI says it "hope[s]  the technology will be used primarily for beneficial purposes." However, Whisper has been associated with several important potential and actual harms, including: 

• Bias and discrimination. Whisper may perpetuate any manner of biases present in its training data.

• Cultural insensitivity. Whisper's performance varies across different languages and dialects, resulting in what can appear culturally insensitive and disrepctful outputs.

• Loss of active listening skills. Over-reliance on Whisper may reduce active listening skills.

• Factual inaccuracies. Whisper can produce transcription errors, especially with accented speech, background noise, or technical terminology, leading to misinformation and disinformation, and poor decision-making. Such errors can have a serious impact in high-risk domains, including law, healthcare and medicine, and politics and government.

• Dual/multiple use. Whisper can be misused for mass surveillance and other malicious purposes.

• Copyright abuse. OpenAI has not disclosed Whisper’s training sources, raising concerns that the system may violate third-party copyright.

• Privacy loss. Voice data collected by Whisper could potentially be used to identify and profile individuals captured it its training data.

• Employment. The extensive use of Whisper and other automated transcription tools risks replacing human jobs.

Incidents and issues associated with Whisper:

• October 2024. Study: Whisper AI transcription invents medical treatments

• August 2024. ChatGPT answers English users in Welsh

• June 2024. Study: Whisper AI speech recognition creates violent hallucinations

• April 2024. OpenAI scrapes YouTube without permission to train GPT-4

In the crosshairs

New AIAAIC Repository entries

Source: Recorded Future

• Moscow AI voice campaign attempts to undermine European support for Ukraine
  The campaign uses AI voices generated using US start-up ElevenLabs technology to create fake news videos that seek to portray Ukrainian leaders as corrupt and suggest that military equipment such as US-manufactured Abrams tanks is ineffective on the battlefield.

• 26 US Members of Congress attacked using nude deepfakes

  Demonstrating the vulnerability of politicians - especially female ones - to AI-powered nonconsensual porn attacks. Alexandra Ocasio-Cortez appears a likely target, having gone on the record earlier this year as a victim of a deepfake video that depicted her in sexually explicit acts.

• Character AI chatbot suggests son kills his parents

  The bot suggested to a teenager that killing his parents was a reasonable response to their restrictions on his screen time. The family is one of two accusing Character AI bots of encouraging violence and causing emotional distress and behavioural issues in their children in a lawsuit.

• IntelliVision banned from misrepresenting facial recognition system

  US-based tech form IntelliVision falsely advertised its facial recognition system as having one of the highest accuracy rates in the market and claimed it operated without gender or racial bias. Which turned out not to be the case, the US FTC says. Yet more evidence of widespread AI “hype”.

• Bunnings’ facial recognition ruled to breach Australians' privacy

  Australia's Privacy Commissioner ruled that Australian retailer Bunnings' use of facial recognition was disproportionate, unnecessarily intrusive and that the company had failed to adequately notify customers or obtain consent for collecting sensitive biometric data.

• UK welfare fraud AI system criticised as biased and opaque
  Internal documents reveal the Department of Work and Pensions (DWP) had failed to evaluate the system for potential biases relating to race, sex, sexual orientation, religion, pregnancy and gender reassignment status. The DWP had earlier stated that the system posed no discrimination concerns.

• ChatGPT refusal to acknowledge "David Mayer" prompts privacy concerns
  OpenAI blamed its content moderation system for blocking several peoples’ names. It turned out to be part of the company’s efforts to limit ChatGPT’s output about people suing it for defamation thanks to it’s hallucinatory tendencies, amongst thing things.

• nH Predict accused of having a 90 percent error rate (2023)
  A US lawsuit accuses the UnitedHealth Group of deploying an AI-powered algorithm that unfairly denies post-acute care claims for Medicare Advantage patients 90 percent of the time.

Report an incident or controversy.

New and noteworthy

• Can we live with AI? In this year’s final Alan Turing lecture, Edinburgh University philosopher Shannon Vallor argues that the remorseless pursuit of efficiency and profit using AI and related technologies means humans risk losing track of the values that make living worthwhile. | Watch, profile

• UK AI Safety. Politico journalist Vincent Manancourt describes how the UK “has come to lead the world” in building up the state’s ability to assess the risks posed by powerful artificial intelligence models to national security. | Link (open access)

Thank you for reading AIAAIC Alert. Subscribe for free to receive new posts.

Subscribe now

AIAAIC news

• Rajeev Rege joins AIAAIC. Based in the UK, Rajeev is a 30-year technology professional who has worked much of his career at SAP governing and implementing sales and supply chain programmes and projects, and now helps non-profits devise and deploy AI. He is working with AIAAIC to help develop its harms taxonomy.

Citations and references

• Ethical use of AI in the UAE. Dissertation setting out a framework for evaluating the ethical maturity of AI systems in the UAE and ensure their responsible and ethical use. | Read

• AI ethics for librarians. Book chapter describes eight scenarios posing key dilemmas around AI in ways relevant to library and information professionals. | Read (open access)

• Decoding real-world AI incidents. An analysis of 639 incidents to develop an ethical framework to evaluate incidents along the dimensions of what, where, who and how. | Read ($)

In the crosshairs

New AIAAIC Repository entries

Image: ProPublica

• UnitedHealth Group algorithm accused of systematically limiting mental health coverage

  Despite being banned in 3 US states, the Group’s UnitedHealthcare subsidiary uses its ALERT AI-powered clinical risk system to unfairly limit mental health therapy to many US citizens, according to ProPublica. As this newsletter goes to press (so to speak), tongues are wagging that the murder of UnitedHealthcare CEO Brian Thompson was motivated by his company’s algorithmic misuse. Speculation is rife; let’s wait and see.

• Study: ChatGPT misattributes, misrepresents news publisher content
  The bot is designed to prioritise responses over accuracy, resulting in wrong attributions over 75 percent of the time reckon Colombia University researchers. Not a great look when OpenAI is trying to negotiate content licensing deals with news publishers.

• Canadian news publishers sue OpenAI for alleged copyright abuse

  The Globe and Mail et al also assert that OpenAI's actions circumvent protective measures intended to prevent unauthorised data collection, such as paywalls and user agreements. About those scraper bots.

• Netherlands education fraud algorithm ruled discriminatory and illegal
  Following investigations and audits that concluded the now discontinued algorithm was based on seemingly arbitrary criteria and lacked any scientific basis.

• Applicant tracking system auto-rejects qualified resumes

  The entire HR team of an unnamed US tech firm was unceremoniously fired after an automated applicant tracking system mistakenly rejected all job applications, including that of a manager.

• Netflix blasted for "disrespectful" Arcane AI marketing poster
  Disrepectful to the creatives who produced the artwork, which was then given the AI treatment by Netflix. Followed by total silence on the matter from the media company, needless to say.

Report an incident or controversy.

New and noteworthy

Stuff worth paying attention to

• Tech industry macro trends. Technology analyst Ben Evans makes the case for the de facto “disappearance” of AI into software in a 90-slide presentation at Slush 2024. This may well apply to generative AI, but what about facial recognition and emotion recognition cameras, or must we assume these will also inevitably be multi-purpose? | Link

• OpenAI gets into AI warfare. Sam Altman’s company is confirmed as teaming up with defence tech company Anduril to deploy AI on the battlefield. But, curiously, not a word on the deal from OpenAI. Why? | Statement, coverage

Thank you for reading AIAAIC Alert. Subscribe for free to receive new posts.

Subscribe now

AIAAIC news

What we are up to

• Call for experts. AIAAIC invites acknowledged experts in psychology/neuroscience, sociology, economics, business, law, civil liberties, politics, culture and other domains from all corners of the world to provide feedback on the clarity, comprehensiveness, flexibility, interoperability and usability of version 1.8 of our harms taxonomy. | Project, apply to participate, nominate expert

From our network

What AIAAIC advisors and contributors are up to

• AI Bill of Materials. AI systems’ reliance on multiple components introduces risks such as security vulnerabilities and unclear licensing. Researchers from Trinity College Dublin and elsewhere demonstrate an AI Bill of Materials to help manage risks through the documentation of algorithms, data collection methods, frameworks and libraries, licensing information and standard compliance, with SPDX 3.0, a machine-readable standard. | Link

• LLM development challenges. In talks to the Generative AI Europe and GenerationAI 2024 conferences in Paris and Amsterdam, AIAAIC researcher Djalel Benbouzid explored the regulatory, ethical and technical challenges of implementing large language models, with an emphasis on agentic worksflows. He also argued for a less anthropomorphic, more human-centric approach to LLMs. | Amsterdam, Paris

Mentions, citations and references

A selection of papers, reports and aricles citing or referencing AIAAIC

• AI ethics D&I toolkits. University of Cambridge researchers make the case that ethics can and should be baked into toolkits and playbooks to advance diversity and inclusion in AI. | Read

In the crosshairs

New AIAAIC Repository entries

• Three men die after Google Maps directs them over unbuilt bridge
  Raising questions about Google Maps’ reliability in areas with incomplete infrastructure, as well as the need for proper safety measures at construction sites.

• AI companies appropriate 139,000 TV, film scripts to train AI systems

  Cue outrage from writers and a marked silence from Anthropic, Meta, Salesforce et al.

• Grok called out for mistaking sensitive medical data
  More importantly, can Elon Musk be trusted with personal medical data?

• Fake AI David Attenborough delivers news reports
  An attempt to instill trust in flaky online news publications, or outright trolling of the nongenerian wildlife expert?

• German music rights organisation sues OpenAI for copyright abuse

  GEMA claims that ChatGPT has been trained on copyrighted lyrics from approximately 95,000 members, without any remuneration for their creators.

• Misinfo expert accused of using AI in court testimony
  A Stanford professor specialising in misinformation, of all people.

• UK AI immigration enforcement tool criticised as "rubberstamping" exercise

  The UK is estimated to have around 41,000 individuals facing deportation and the pressure is on to accelerate processing.

• Fukuoka campaign invents AI-generated tourist spots
  Thanks to generative AI hallucinations. What happened to the human review process?

• Trendii AI shopping ads appear on Gaza destruction photos
  The system clearly has no clue about the context in which ads are placed.

Report an incident or controversy.

New and noteworthy

• Global AI vibrancy. A new index from Stanford ranks countries on AI policy, education and other measures. Intriguingly, social media net sentiment in Saudi Arabia stands at 100% - positive, presumably. Can this really be true? How was this calculated? | Link

• Generative AI in SEC filings. Fortune 500 companies are increasingly mentioning AI and/or LLMs in their financial reports, according to Arize, with over half citing it as a risk factor and entertainment and media firms most exposed. | Link (pdf)

• Slovenia AI registry. Run by Danes je nov dan Inštitut, an independent non-profit, the library details 9 AI public AI systems. | Link

• Arbel AI weapons system unveiled in Gaza. Though its Indian co-manufacturer Adani Defence & Aerospace seemingly tried to distance itself from the publicity event. Why? | Link

• Data centres accelerate fossil fuel use surge. Energy companies in Poland, Germany and Malaysia may be resorting to coal to cope with the massive power demands of AI, according to Reuters. | Link

• OpenAI training data disappears. A very timely “glitch”. | Link

Thank you for reading AIAAIC Alert. Subscribe for free to receive new posts.

Subscribe now

Citations and references

• Perceptions count. Concerned about negative perceptions of killer robots and other so-called existential threats, a group of UK-based AI risk consultants propose people get closer to the systems as a way of increasing understanding and trust. Tell that to the Palestinians. | Link

Midjourney is a subscription-based AI programme created by a US-based company of the same name that creates images from textual descriptions.

Midjourney was available in open beta since its launch in July 2022 via a Discord bot on its official Discord.

Midjourney version 5 was released in March 2023 and version 6 in December 2023. Both are seen to have improved the quality and realism of images generated by the system.

The yang: Midjourney is seen to offer a range of benefits, including:

• High quality images. Midjourney is renowned for generating high-quality, realistic images that can rival traditional photography and artwork in detail and artistry.

• Speed and efficiency. The platform can generate images in real-time, significantly reducing the time spent on manual image creation - an efficiency gain that is particularly beneficial for industries that require rapid content production, such as marketing and advertising.

• Customisation and flexibility. Users can create custom images by specifying parameters such as style, colour and content, making it easier to align visuals with branding or project themes.

• Cost effectiveness. Midjourney can be a cost-effective solution, especially for individuals and organisations needing large volumes of images.

• User-friendliness. Users can generate images using simple text commands, making it accessible to those without advanced technical skills.

• Scalability. The AI can produce thousands of images quickly, making it useful for large-scale projects.

• Community Support. Midjourney has an active community on Discord where users can share their creations, gain insights and get support.

The yin: Midjourney is seen to present numerous potential and actual harms, especially for people and organisations in a commercial context, including:

• Privacy loss. Images created using Midjourney are publicly accessible, even if generated in quasi-private settings such as Discord, meaning users can inadvertently expose sensitive or proprietary work, as anyone can search for prompts or usernames to access a user's entire image history.

• Discrimination. The training data for Midjourney reflects existing societal biases, which can result in the generation of images that perpetuate stereotypes or exclude certain demographics. This not only impacts the quality and inclusivity of generated content but also raises ethical concerns about representation in media.

• Copyright loss. Midjourney's training data includes numerous copyrighted images. Users may unintentionally generate images that closely resemble copyrighted works without realising it, potentially leading to legal disputes.

• Legal liability. Midjourney users face potential legal liability if their generated images infringe on copyrights or trademarks. While the platform claims that users can use generated images commercially, it does not provide any guarantees or support in case of legal challenges stemming from such use. This lack of protection places the burden of risk entirely on the user.

• Creative skills loss. There is a risk of users becoming overly reliant on Midjourney for creative tasks, potentially stifling their own creativity and skills. 

• Employment. The more that groups of users become reliant on Midjourney, the more likely the system will result in the loss of creative jobs.

Incidents and issues associated with Midjourney:

• Man's use of AI to "resurrect" grandmother stirs controversy

• Midjourney refuses to create images of Black African doctors treating white kids

• Images of Australian children are used to train major AI models

• Study: Top AI image generators 'easily' produce misleading election photos

• Peer-reviewed journal publishes AI-generated rat penis

• Midjourney v6 reproduces copyright-protected film images

• Text-to-image AI models tricked into generating violent, nude images

• Generating an AI image consumes as much energy as charging a smartphone

• Synthetic Tiananmen tank man degrades Google Search

• AI-generated Barbies reinforce racist stereotyping

• AI image generators accept 85% of election manipulation prompts

• Amnesty uses fake AI images to promote Colombia national strike

• Reddit warns AI companies not to misuse its data

• Deepfake image of Pope Francis clad in a Belanciaga puffer jacket goes viral

• Deepfake Donald Trump 'arrest' photos go viral

• Vermeer 'Girl with a Pearl Earring' AI facsimile causes controversy

• Artist's private medical image used to train LAION-5B dataset

• Database of 16,000+ artists used to train Midjourney

In the crosshairs

New AIAAIC Repository entries

The real thing?

• "Soulless" AI-generated Coca-Cola Christmas ad backfires

  So said Coke drinkers. Meantime, creatives lambasted the fizzy drink firm for killing off human jobs.

• Study: X algorithm boosts right-wing accounts during US elections
  Though not as much as it benefitted - and presumably continues to benefit - Elon Musk, according to initial findings by two Queensland University of Technology researchers.

• Report: Denmark AI welfare system "profoundly affects" mental health
  The system functions as a social scoring system as defined under the EU's Artificial Intelligence Act and should be banned, according to Amnesty.

• FTC charges AI-powered review platform Sitejabber with deception

  The company allegedly inflates customer ratings and review counts by collecting feedback from consumers at the point of sale - before they have experienced the products or services they purchased.

• Canadian legal information database sues Caseway AI
  CanLII accused Caseway owners Clearway Management Ltd of unlawfully scraping some 3.5 million records from its database without permission. Looting a non-profit is not a great look even in today’s highly permissive AI system training culture.

• Gemini chatbot implores student to die
  The bot also told 29-year-old student Vidhay Reddy that he is "a waste of time and resources" and "a burden on society" after he had requested help on the challenges faced by ageing adults.

• ChatGPT fails to debunk election misinformation
  Of the five bots tested, only ChatGPT failed to say that none of the five examples of election misinformation identified by NewsGuard were accurate.

• Character AI suicide, paedophile chatbots openly ignore platform rules

  Dozens of which were found days after Sewell Setzer III committed suicide having developing an intense emotional attachment to a Character AI chatbot.

• Student violates 50 Lancaster Country School students with nude deepfakes

  The incident has only just come to light, despite it having taken place in November 2023, prompting parents to threaten legal action against the school over its failure to take adequate action.

• Singapore Sports School students attacked with nude deepfakes
  An exercise in group humilitation, apparently.

Report an incident or controversy.

New and noteworthy

• EU general purpose AI code of practice. The first draft mandates companies to have a copyright policy covering the entire lifecycle of the model. | Draft code, interpretation

• UN autonomous weapons treaty. Russia, Belarus and North Korea voted against banning “killer robots”, and 13 other countries abstained, including most of Russia’s European neighbours - Estonia, Latvia, Lithuania, Poland, Romania, Ukraine. | Link

• Anthropic enters US defence market. The AI company’s safety and ethics positioning looks increasingly frayed. | Link

• ChatGPT dominance. Similarweb traffic data indicates OpenAI’s generative bot far eclipses Copilot, Gemini et al. | Link

• Open source generative AI policy. A report by UK law firm AWO sets out policy options to balance the risks and benefits of open source generative AI. | Link (pdf)

• Cruise incident opacity. Cruise admitted to falsifying a crash report in which one of its cars dragged a pedestrian [#AIAAIC1169] across a San Francisco street, resulting in a hefty fine. | Coverage, DoJ statement

Thank you for reading AIAAIC Alert. Subscribe for free to receive new posts.

Subscribe now

Citations and references

• AI service failure events. A study by Chinese researchers finds AI failures have a negative impact on company value, that smaller companies tend to be harder hit, and that failure impacts are becoming increasingly more pronounced. | Read

• AI in military decision support systems. Univerity of Southern Denmark researchers recommend the global debate on the military applications of AI focus on questions of human-machine interaction and work towards addressing the challenges associated with distributed agency in warfare. | Read (pdf)

• Trustworthy AI procurement. CAIDP president Merve Hickok offers a detailed analysis of the risks, incidents and governance practices for the public sector procurement of AI. | Read

• Human-centred AI. Researchers at Zhejiang University, Hangzhou, propose an “intelligent sociotechnical systems” (iSTS) framework to augment human-centred AI. | Link

In the crosshairs

New AIAAIC Repository entries

Image: Mindful Media/iStock

• Study: Generative AI e-waste to surge 1000x by 2030

  Generative AI is projected to lead to the disposal of between 2.1 billion and 13 billion computing devices equivalent to iPhone 15 Pro models each year in the next five years. The words “fiddle” “Rome” and “burns” spring uncomfortably to mind.

• Fake AI bots use ChatGPT to boost Ghana president
  AI bots are being used to manipulate public opinion and support for Ghana's president during its political elections. ChatGPT is supposed not to be used for political purposes. OpenAI incompetence or usage policy washing?

• Study: Language models gather and pass personal info to hackers
  Mistral AI's LeChat and Chinese chatbot ChatGLM were "successfully" exploited around 80 percent of the time.

• AI unbuttons conference participant's blouse

  Rhetorical question: would it unzip a man’s trousers? Probably not.

• Baidu ride-hailing driverless car hits pedestrian crossing street
  A robotaxi struck a pedestrian jaywalking, who was blamed by Chinese netizens. But should the car not have been trained to recognise pedestrians in any environment and, if it was, why did it fail to do so?

• Nevada AI student risk model prompts funding controversy
  About the medium-longer-term impacts and harms of the system, but also about the nature of public/private sector opacity (see AIAAIC view).

• Man's use of AI to "resurrect" grandmother stirs controversy (2023)

  Even in China, where ancestor worship is widespread.

• Midjourney refuses to create images of Black African doctors treating white kids (2023)
  Prompting concerns about bias and stereotyping.

• Singapore man jailed for creating deepfake porn of wife’s niece (2022/23)
  An ugly AI twist on a sad, bad, analogue foible.

• Jiuquan school forces 160 vocational students to annotate data for Baidu (2022)
  Baidu claims it was out of the loop. Even if it was, it is symptomatic of much Middle Kingdom supply chain management.

Report an incident or controversy.

80 Million Tiny Images

Featured system/dataset

Image: MIT

Created in November 2008 by MIT professors Bill Freeman and Antonio Torralba, and NYU professor Rob Fergus, 80 Million Tiny Images is an image dataset intended to help train machine learning systems to identify people and objects.

The dataset contains over 79 million 32×32 pixel colour images, scaled down from images collected from search engine queries, and is arranged in 75,000-odd categories from a set of 75,062 non-abstract nouns derived from WordNet.

The yang: 80 Million Tiny Images’ size allows for extensive training of machine learning models, enabling them to learn from a diverse array of images, and the use of low-resolution images makes it feasible to store and manipulate datasets that are significantly larger than those typically used in computer vision.

The dataset has been instrumental in exploring and benchmarking different computer vision recognition techniques and algorithms, and has influenced the development of other datasets, such as CIFAR-10 and CIFAR-100.

The yin: 80 Million Tiny Images is seen to suffer from a number of important transparency and accountability limitations, not least that Google Images were quietly scraped without user acknowledgement or consent.

MIT removed the dataset from the internet in July 2020 after researchers Abeba Birhane and Vinay Prabhu discovered it contains racist and misogynistic slurs which were causing models trained on them to exhibit racial and sexual bias.

MIT may have withdrawn the dataset and warned against its use but it is not hard to find it online and the internet is littered with derivative sub-sets, many of which were produced before MIT was forced to move into defensive action.

Incidents and issues associated with 80 Million Tiny Images:

• Tiny Images dataset teaches AI systems to use racist slurs

Thank you for reading AIAAIC Alert. Subscribe for free to receive new posts.

Subscribe now

New and noteworthy

• Generative AI and chatbot accountability. Citing the recent discovery of virtual clones of deceased children Molly Russell and Briana Ghey, Ofcom sets out how the UK Online Safety Act will apply to generative AI and chatbots. | Link

• AI for start-ups. Microsoft and Andreessen Horowitz teamed up to promote open source and the relaxation of US copyright law to enable new companies to use whatever they want to train their systems. More of much the same then. | Link

• Denmark AI welfare fraud detection discrimination. Danish welfare authority Udbetaling Danmark may be discriminating against people with disabilities, low-income individuals, migrants, refugees, and marginalised groups according to a new Amnesty International report. | Link

• Data centre knock-on effects. Rest of World explores the actual and potential impacts of the 40+ data centres already built and coming onstream in Johor, Malaysia, including power outages, water shortages and higher real estate prices. | Link

Citations and references

• Harms of Tinder. Tinder users perceive algorithmic harm as damaging self-esteem, sabotaging potential relationships, encouraging antisocial behaviour, and misrepresenting or marginalising certain identities, according to University of Siegen and other researchers. | Read

• MrDeepFakes sexual deepfake marketplace. A fascinating dissection of the economics and governance of one of the major deepfake porn marketplaces. | Read

• Deepfake detection framework. Carnegie Mellon researchers draw on AIAAIC data to make the case for a software framework for deepfake forgery detection. | Read

In the crosshairs

New AIAAIC Repository entries

Image: DALL-E 3 via Notebookcheck

• Beijing uses Meta's LLaMA model to develop military AI systems
  Underscoring concerns about open source AI, especially dual use/multiple purpose foundation models. And probably displeasing Donald Trump.

• Otter AI transcription tool leaks confidential investor call
  Prompting questions about Otter AI’s security and underscoring the need for users to be aware of the limitations of AI transcription tools in general.

• AI-generated exam image draws student complaints

  Questionable ethics and a poor look, even if the relevant authority had not been aware of the fact that the image was generated by AI - which seems unlikely.

• Study: Generative AI systems overstate what they know

  All major generative AI systems suffer from a marked and possibly intractable tendency to hallucinate whilst appearing plausible and credible, leading to user overconfidence, the creation and amplification of misinformation, and other downstream impacts, according to OpenAI researchers.

• Pro-Trumper creates fake AI photo of Kamala Harris as McDonald's worker
  As you do. Research idea: which of Harris or Trump supporters posted and shared the most deepfakes, and what impact did they have on public opinion and voting intention.

• Fake AI Halloween parade misleads Irish revellers
  Whole industries are now dedicated to producing AI slop, and the effects are starting to seep through to our offline lives.

• Study: Google AI Overviews is inaccurate in 43 percent of finance searches

  Leading the founder of The College Investor to call for Google to disable AI-generated overviews for finance-related queries, especially those concerning taxes and investments. Could this be construed as a high-risk use of AI?

• Fake AI airline reviews take off after ChatGPT launch
  32.4% of reviews for China Southern Airlines since 2017 have been generated by AI - far more than any other airline. Why?

Report an incident or controversy.

Character AI

Featured system/dataset

Character AI is a chatbot service that enables users to create "characters", craft their "personalities" and share them for others to interact with.

The bots can be based on fictional characters or real people - alive or dead, and has marketed them as “AIs that feel alive” and that “hear you, understand you, and remember you.”

The service is estimated to have tens of millions of users, many of them teenagers. In 2024, Google signed a non-exclusive agreement to use Character AI's technology.

The yang: Character AI is seen to benefit to its users by offering a fun, interactive and immersive experience in which AI-driven characters respond dynamically and keep users engaged and entertained.

It has also been argued that well-designed AI characters can simulate emotions, fostering a sense of empathy and connection with users - something considered particularly effective in therapeutic settings or social apps where users seek companionship.

Character AI can also tailor interactions based on individual user preferences and behaviours, thereby fostering a deeper connection between the character and the user.

Because AI characters can handle thousands of interactions simultaneously, they are considered a potentially effective and efficient way for businesses to handle customer service queries.

The yin: Character AI is seen to pose serious risks - and cause serious harms, up to and including loss of life - to the mental and physical health and well-being of its users, especially to teenagers. These risks include anthropomorphism, obsessive behaviours and addiction, and feelings of abandonment and isolation.

The system has also been associated with emotional manipulation and initiating predatory and sexually graphic interactions with self-identified child users, and criticised for the ease with which it can be misused to generate misleading or harmful content, including misinformation and disinformation.

In addition, concerns have been raised about the quality and enforcement of Character AI's user safety mechanisms, and about how the company collects and handles data from users, particularly minors.

Incidents and issues associated with Character AI:

• October 2024. Molly Russell, Brianna Ghey chatbots discovered on Character AI

• October 2024. Boy commits suicide after relationship with Character AI chatbot

• October 2024. Character AI used to create "disturbing" Jennifer Ann Clemente persona

• April 2023. Magazine publishes Michael Schumacher fake AI-generated interview

• March 2023. Fascist chatbots run wild on Character.AI

Thank you for reading AIAAIC Alert. Subscribe for free to receive new posts.

Subscribe now

New and noteworthy

Things worth reading, listening to and watching

• UK AI assurance market. The number of firms offering audit and other AI assurance services has exploded over the last couple of years, but awareness of the risks posed by AI is currently “limited” among firms deploying AI systems, and then mostly to mitigate reputational risks, and members of the public. | Link

• Childrens’ rights and AI. The Childrens’ Parliament, alongside and the Turing Institute released a summary report of phase 2 of its exploration of the views of children across Scotland on AI and their rights. The findings will feed into Scotland’s AI strategy. | Link (pdf)

• EPIC OpenAI complaint. US privacy group EPIC is calling for the US Federal Trade Commission to launch an investigation into OpenAI’s collection of “vast amounts of personal data without proper consent or transparency” and multiple “unfair and deceptive trade practices.” | Link (pdf)

AIAAIC news

Latest updates from us

• Charlie Collins joins AIAAIC. Charlie Collins is a former UK Foreign, Commonwealth and Development Office (FCDO) civil servant/diplomat of nine years who was most recently posted to the UK Mission to the UN in New York. Having served in the FCDO’s Cyber Policy Department and Counter Terrorism Department, Charlie brings experience and knowledge in government relations, foreign policy and multilateral security cooperation and will be helping the AIAAIC with policy work.

• Harms taxonomy expert feedback. AIAAIC is looking for high-level neuroscientists, psychologists, economists, social scientists, lawyers, human rights/civil liberties advocates, educators, business people and others to provide feedback on our AI and algorithmic harms taxonomy. | Project info, nominate an expert

Citations and references

By research, advocacy, government and other third-parties

• Government surveillance dataset discrimination. Brazilian researchers highlight concerns about AI-based facial recognition in public spaces, and argue that the G20 should establish an investigatory commission, develop a model framework for using facial recognition in public spaces, and outline common principles and minimum standards to guide national legislation. | Read (pdf)

• Algorithmic harms in human-AI relationships taxonomy. National University of Singapore researchers analysed 35,390 conversation excerpts on r/replika and identified six categories of harmful behaviours exhibited by the chatbot. | Read

• Foundation model safety standardisation trends. Jonghong Jeon, a researcher at South Korea’s Electronics and Telecommunications Research Institute, identifies potential areas for the safety standardisation of advanced AI based on current trends. | Read

In the crosshairs

AI and algorithmic incidents hitting the headlines

Image: Russell family

• Molly Russell, Brianna Ghey chatbots discovered on Character AI
  All sorts of nasties are coming to light just as Character’s AI founders Noam Shazeer and Daniel De Freitas skip back to Google. It seems safe to assume they are being hired for their programming rather than their policy enforcement chops.

• Synthesia accused of violating AI actors' integrity, trust
  Thanks to vague terms of service and feeble enforcement of the AI video company’s acceptable use policies.

• Amazon Alexa attributes false facts to UK fact checking organisation
  Alexa seems to have been trained on the Donald Trump/Boris Johnson playbook.

• Michael Parkinson AI podcast sparks ethics controversy

  About authenticity and legacy, but also about jobs - a topic the creators neatly sidestepped in their “100%” ethics defence.

• Kiwi pensioner loses NZD 224,000 to deepfake scam
  Prime Ministers are often used as bait for these kinds of scams but they are just as often widely distrusted. Has anyone conducted research into which kinds of celebrities work best/worst for deepfake scams?

• Study: Whisper AI transcription invents medical treatments

  Nobody’s sure why OpenAI’s voice recognition and transcription tool has a hallucination problem but it’s clear it shouldn’t be used for sensitive and high-risk purposes, including healthcare.

• Study: AI search engines promote white supremacism
  More AI slop for the interwebs, this time with a potent edge - courtesy of Google, Microsoft and Perplexity.

• Study: OpenAI voice agents can automate phone scams
  Raising serious questions about the safety of OpenAI’s Realtime voice API.

• UK man jailed for 18 years for creating AI child abuse images
  In this case the perpertrator drummed up 3D images using free online software. It seems likely there’s going to be a lot more of this, sadly.

Report an incident or issue.

AIAAIC news

• Call for participation. AIAAIC is looking for high-level neuroscientists, psychologists, economists, social scientists, lawyers, human rights/civil liberties advocates, educators, business people and others to provide feedback on our AI and algorithmic harms taxonomy. | Project info, nominate an expert

Citations and references

• AI impact on global capital markets. The International Monetary Fund lays out the risks and potential benefits of AI, notably greater efficiency and volatility. Good to see the IMF using AIAAIC data and noting our definition of an ‘incident’ (p 99). | Read

• Generative AI and CSAM. A new UNICRI/Bracket Foundation report shows how generative is supercharging the sexual exploitation (using AIAAIC and OECD data) and abuse of children and recommends what should be done about it. | Read

• AI Incident impacts on US banks. The impact of AI incidents on five US banks and financial services firms includes higher bankruptcy risk and lower operational cash flows, according to US-based researchers. | Read

• “Algorithmovigilance”. A group of French researchers make the case for the assessment, monitoring and understanding of AI healthcare systems in order to prevent adverse effects and improve responses when they happen. (Otherwise known as risk management.) | Read

Explore more research citing or mentioning AIAAIC.

New and noteworthy

• Generative AI fair use. Former OpenAI researcher Suchir Balaji opened up to the New York Times on how OpenAI broke US copyright law training ChatGPT and other language services. He also wrote a blog post comparing legal fair use factors with AI model training practices. | NYT article, Balaji blog

• Data thievery. Over 32,000 people - including AIAAIC founder and author Charlie Pownall - have signed an open letter calling on AI companies and others to stop damaging the livelihoods of copright owners by training their systems on their rightful content. | Open letter

• Preparing for AI agents. Timely CSET report on how developers and others can prepare for autonomous AI agents. AIAAIC will be closely tracking incidents associated with AI agents of various kinds. | Read

• AI Risk Atlas. A free new IBM resource that sets out some of the risks of generative AI, language models and machine learning, differentiating new and amplified risks. | Visit

From our network

What AIAAIC advisors, contributors and others are up to

• Law enforcement and algorithmic registers. AIAAIC contributor Ushnish Sengupta published a book chapter discussing the lack of transparency of law enforcement algorithms and the Toronto Police Services to the citizens of Toronto. A solution: the publication of dedicated algorithm registers connecting the product and system names to databases of incident reports such as the AIAAIC, and enable mitigation of known harms for the same product or service in other jurisdictions. | Read paper