Can “AI” do your science for you? Should it be your co-author? Or, as one company asks, boldly and breathlessly, “Can we automate the entire process of research itself?”

Major scientific journals have banned the use of tools like ChatGPT in the writing of research papers. But people keep trying to make “AI Scientists” a thing. Just ask your chatbot for some research questions, or have it synthesize some human subjects to save you time on surveys.

Alex and Emily explain why so-called “fully automated, open-ended scientific discovery” can’t live up to the grandiose promises of tech companies. Plus, an update on their forthcoming book!

References:

Sakana.AI keeps trying to make 'AI Scientist' happen
- The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery

Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers

How should the advent of large language models affect the practice of science?

Relevant research ethics policies:

ACL Policy on Publication Ethics

Committee On Public Ethics (COPE)

The Vancouver Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work

Fresh AI Hell:

Should journals allow LLMs as co-authors?

Business Insider "asks ChatGPT"

Otter.ai sends transcript of private after-meeting discussion to everyone

"Could AI End Grief?"

AI generated crime scene footage

"The first college of nursing to offer an MSN in AI"

FTC cracks down on "AI" claims

You can check out future streams at http://twitch.tv/dair_institute.

Follow Emily at https://twitter.com/EmilyMBender and https://dair-community.social/@EmilyMBender
Follow Alex at https://twitter.com/alexhanna and https://dair-community.social/@alex

Music: Toby Menon.
Production: Christie Taylor.
Graphic Design: Naomi Pleasure-Park.

Do you feel like this summer has brought just unending terrible ideas for how to use “AI”? You’re not alone! The time has come, yet again, for Emily and Alex to clear out the poison, purge some backlog, and take a journey through AI hell.

From surveillance of emotions...to continued hype around using 'AI' to automate the scientific process. But also, more promising, signs that people can see that the AI emperor is indeed without clothes.

Fresh AI Hell:

Synthetic data for Hollywood's test screenings

NaNoWriMo's AI fail
- But AI is built on classist exploitation
- NaNoWriMo is sponsored by an AI writing company
- NaNoWriMo's AI writing sponsor still creates bad writing

Company's AI assistant starts rickrolling customers

Programming LLMs with "fiduciary duty"

Canva increasing prices thank to "AI" features

Ad spending by AI companies

Clearview AI hit with largest GDPR fine yet, Dutch regulator considers holding execs personally liable

'AI detection' in schools harming neurodivergent kids

CS prof proudly stating unethical use of ChatGPT in the Atlantic

College recruiter chatbot can't "discuss politics"

"The AI-Powered Nonprofits Reimagining Education"

Teaching AI at art schools

Professors' 'AI twin' chatbots as teaching assistants

A teacherless AI classroom

Another 'AI Scientist'

LLMs still biased against African American English

AI "enhances" photo of Black people into photo of white-appearing people

Eric Schmidt says go ahead and steal data, write apps with ChatGPT

The environmental cost of Google's "AI Overviews"

A "Grand Challenge" for AI in climate and environment -- brought to you by Jeff Bezos

What I found in an AI-company's e-waste

Musk's xAI accused of worsening Memphise smog with unauthorized gas turbines at data center

Smile surveillance of workers

Rail carrier used AI for "emotion recognition" of passengers

Chatbot-generated harassment scenario reveals real harassment victim's personal information

AI Has Increased Workloads And Hampered Productivity

Survey says: "AI" in a product description makes people want to purchase it less

Is tripe kosher? It depends on the religion of the cow.

You can check out future streams at http://twitch.tv/dair_institute.

Follow Emily at https://twitter.com/EmilyMBender and https://dair-community.social/@EmilyMBender
Follow Alex at https://twitter.com/alexhanna and https://dair-community.social/@alex

Music: Toby Menon.
Production: Christie Taylor.
Graphic Design: Naomi Pleasure-Park.

Foone🏳️‍⚧️

2024-08-19 19:27:14

ugh. I picked up a shitty NUC from ewaste and it had a label on it for an AI company.
ahh, another startup that burnt out trying to build some silly AI project on crap hardware. I wonder what they did? I check their URL:
ahh. healthcare. great, great.

The Washington Post is going all in on AI -- surely this won't be a repeat of any past, disastrous newsroom pivots! 404 Media journalist Samantha Cole joins to talk journalism, LLMs, and why synthetic text is the antithesis of good reporting.

References:

The Washington Post Tells Staff It’s Pivoting to AI: "AI everywhere in our newsroom."
Response: Defector Media Promotes Devin The Dugong To Chief AI Officer, Unveils First AI-Generated Blog

The Washington Post's First AI Strategy Editor Talks LLMs in the Newsroom

Also: New Washington Post CTO comes from Uber

The Washington Post debuts AI chatbot, will summarize climate articles.

Media companies are making a huge mistake with AI

When ChatGPT summarizes, it does nothing of the kind

Fresh AI Hell:

"AI" Alan Turning

Google advertises Gemini for writing synthetic fan letters

Dutch Judge uses ChatGPT's answers to factual questions in ruling

Is GenAI coming to your home appliances?

AcademicGPT (Galactica redux)

"AI" generated images in medical science, again (now retracted)

You can check out future streams at http://twitch.tv/dair_institute.

Follow Emily at https://twitter.com/EmilyMBender and https://dair-community.social/@EmilyMBender
Follow Alex at https://twitter.com/alexhanna and https://dair-community.social/@alex

Music: Toby Menon.
Production: Christie Taylor.
Graphic Design: Naomi Pleasure-Park.

When is a research paper not a research paper? When a big tech company uses a preprint server as a means to dodge peer review -- in this case, of their wild speculations on the 'dangerous capabilities' of large language models. Ali Alkhatib joins Emily to explain why a recent Google DeepMind document about the hunt for evidence that LLMs might intentionally deceive us was bad science, and yet is still influencing the public conversation about AI.

Ali Alkhatib is a computer scientist and former director of the University of San Francisco’s Center for Applied Data Ethics. His research focuses on human-computer interaction, and why our technological problems are really social – and why we should apply social science lenses to data work, algorithmic justice, and even the errors and reality distortions inherent in AI models.

References:

Google DeepMind paper-like object: Evaluating Frontier Models for Dangerous Capabilities

Fresh AI Hell:

Hacker tool extracts all the data collected by Windows' 'Recall' AI

In NYC, ShotSpotter calls are 87 percent false alarms

"AI" system to make callers sound less angry to call center workers

Anthropic's Claude Sonnet 3.5 evaluated for "graduate level reasoning"

OpenAI's Mira Murati says "AI" will have 'PhD-level' intelligence

OpenAI's Mira Murati also says AI will take some creative jobs, maybe they shouldn't have been there to start out with

*****

You can check out future streams at http://twitch.tv/dair_institute.

Follow Emily at https://twitter.com/EmilyMBender and https://dair-community.social/@EmilyMBender
Follow Alex at https://twitter.com/alexhanna and https://dair-community.social/@alex

Music: Toby Menon.
Production: Christie Taylor.
Graphic Design: Naomi Pleasure-Park.

Alex and Emily put on their social scientist hats and take on the churn of research papers suggesting that LLMs could be used to replace human labor in social science research -- or even human subjects. Why these writings are essentially calls to fabricate data.

References:

PNAS: ChatGPT outperforms crowd workers for text-annotation tasks
* Beware the Hype: ChatGPT Didn't Replace Human Data Annotators
* ChatGPT Can Replace the Underpaid Workers Who Train AI, Researchers Say

Political Analysis: Out of One, Many: Using Language Models to Simulate Human Samples

Behavioral Research Methods: Can large language models help augment English psycholinguistic datasets?

Information Systems Journal: Editorial: The ethics of using generative AI for qualitative data analysis

Fresh AI Hell:

Advertising vs. reality, synthetic Willy Wonka edition
* https://x.com/AlsikkanTV/status/1762235022851948668?s=20
* https://twitter.com/CultureCrave/status/1762739767471714379
* https://twitter.com/xriskology/status/1762891492476006491?t=bNQ1AQlju36tQYxnm8BPVQ&s=19

A news outlet used an LLM to generate a story...and it falsely quoted Emily
* AI Invents Quote From Real Person in Article by Bihar News Site: A Wake-Up Call?

Trump supporters target Black voters with faked AI images
* Seeking Reliable Election Information? Don’t Trust AI

You can check out future streams at http://twitch.tv/dair_institute.

Follow Emily at https://twitter.com/EmilyMBender and https://dair-community.social/@EmilyMBender
Follow Alex at https://twitter.com/alexhanna and https://dair-community.social/@alex

Music: Toby Menon.
Production: Christie Taylor.
Graphic Design: Naomi Pleasure-Park.

The United States Congress spent 2023 busy with hearings to investigate the capabilities, risks and potential uses of large language models and other 'artificial intelligence' systems -- and Senate Majority Leader Chuck Schumer led an entire series of such forums, inviting heads of companies, experts, and others to speak from their experience. Alex and Emily are joined by journalist Justin Hendrix to talk about the limitations of these hearings, from the alarmist fixation on so-called 'p(doom)' to the languishing legislation on data privacy that could make a meaningful difference in peoples' lives.

Justin Hendrix is the editor of Tech Policy Press, a nonprofit media outlet that focuses on how tech intersects with power, ethics, racism, and democracy.

References:

Tech Policy Press tracker for the US Senate 'AI Insight Forum' hearings

Joint Oversight & Investigations and Research & Technology Subcommittee Hearing - Balancing Knowledge and Governance: Foundations for Effective Risk Management of Artificial Intelligence (featuring Dr. Emily M. Bender)

Charter for the above hearing

Opening remarks of Dr. Emily M. Bender at Rep. Bobby Scott’s virtual roundtable on AI

Transcript: Senate hearing addressing the national security implications of AI

Video: Rep. Nancy Mace opens hearing with ChatGPT-generated statement. Advances in AI: Are we ready for a tech revolution? (House Committee on Cybersecurity, Information Technology, and Government Innovation)

Brennan Center report on Department of Homeland Security: Overdue Scrutiny for Watch Listing and Risk Prediction

Tech Policy Press: Senate Homeland Security Committee Considers Philosophy of Artificial Intelligence

Tech Policy Press: White House Executive Order on AI Gives Sweeping Mandate to DHS

Listen to Emily & Alex's appearance on the Tech Policy Press Podcast

Fresh AI Hell

US asylum seekers are being held up by AI-powered translation apps

UK officials are using AI to decide on issues from benefits to marriage licenses

Refreshing chaser: Prior MAIHT3K guest Dr. Sarah Myers West testifying in front of Senate Judiciary Committee on AI concentration

You can check out future streams at http://twitch.tv/dair_institute.

Follow Emily at https://twitter.com/EmilyMBender and https://dair-community.social/@EmilyMBender
Follow Alex at https://twitter.com/alexhanna and https://dair-community.social/@alex

Music: Toby Menon.
Production: Christie Taylor.
Graphic Design: Naomi Pleasure-Park.

Emily and Alex talk to UC Berkeley scholar Hannah Zeavin about the case of the National Eating Disorders Association helpline, which tried to replace human volunteers with a chatbot--and why the datafication and automation of mental health services are an injustice that will disproportionately affect the already vulnerable.

Content note: This is a conversation that touches on mental health, people in crisis, and exploitation.

Hannah Zeavin is a scholar, writer, and editor whose work centers on the history of human sciences (psychoanalysis, psychology, and psychiatry), the history of technology and media, feminist science and technology studies, and media theory. Zeavin is an Assistant Professor of the History of Science in the Department of History and The Berkeley Center for New Media at UC Berkeley. She is the author of, "The Distance Cure: A History of Teletherapy."

References:

VICE: Eating Disorder Helpline Fires Staff, Transitions to Chatbot After Unionization

… and then pulls the chatbot.

NPR: Can an AI chatbot help people with eating disorders as well as another human?

Psychiatrist.com: NEDA suspends AI chatbot for giving harmful eating disorder advice

Politico: Suicide hotline shares data with for-profit spinoff, raising ethical questions

Danah Boyd: Crisis Text Line from my perspective.

Tech Workers Coalition: Chatbots can't care like we do.

Slate: Who's listening when you call a crisis hotline? Helplines and the carceral system.

Hannah Zeavin: The Third Choice: Suicide Hotlines, Psychiatry, and the Police

New York Times hype: Using "AI" to diagnose mental disorders based on voice recordings, April 2022

Fresh AI Hell:

ChatGPT for city government work
https://www.japantimes.co.jp/news/2023/06/06/national/yokosuka-adopts-chatgpt/

ChatGPT as manuscript pre-screener for publishing companies:
https://www.publishersweekly.com/pw/by-topic/digital/content-and-e-books/article/92471-ai-is-about-to-turn-book-publishing-upside-down.html)
https://twitter.com/jgnoelle/status/1665729550922252288

A lawyer tried to use ChatGPT for “research”:
https://storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.nysd.575368.32.1.pdf
Legal scholar (and former MAIHT3K guest) Kendra Albert breaks down the story about a lawyer using ChatGPT for court proceedings:
https://dair-community.social/@kendraserra/110441210244994168

ChatGPT as investment advisor:
https://venturebeat.com/ai/jpmorgan-plans-for-a-chatgpt-like-investment-service-are-just-part-of-its-larger-ai-ambitions/

Professor asks “ChatGTP” if it wrote student’s assignments, and then gave them a 0 if it said yes.
https://twitter.com/Linkletter/status/1658591316991938560

You can check out future streams at http://twitch.tv/dair_institute.

Follow Emily at https://twitter.com/EmilyMBender and https://dair-community.social/@EmilyMBender
Follow Alex at https://twitter.com/alexhanna and https://dair-community.social/@alex

Music: Toby Menon.
Production: Christie Taylor.

Kendra Albert

2023-05-27 14:52:10

Are you just catching up on the bonkers story about the lawyer using ChatGPT for federal court filings? This is a thread for you.

Alex and Emily are joined by law and technology scholar Kendra Albert for a conversation about ChatGPT, legal expertise, and how we do and don't measure someone's ability to practice law.

References:
Social Science Research Network paper “written” by ChatGPT:
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4294197

Joe Wanzala, “ChatGPT is ideal for eDiscovery”:
https://nlsblog.org/2023/02/21/chatgpt-implications-for-criminal-defense-litigation-practice/

Legal applications for ChatGPT:
Shot: https://the-decoder.com/gpt-4-could-pass-bar-exam-ai-researchers-say/
Chaser: https://www.theonion.com/chatgpt-forced-to-take-bar-exam-even-though-dream-was-t-1850036337

AI for law:
https://www.dailyjournal.com/articles/370354-the-future-of-artificial-intelligence-in-law-firms

“AI can legally run a company”:
https://twitter.com/johnjnay/status/1622319172955901952

Generative AI Is Coming for the Lawyers:
https://www.wired.co.uk/article/generative-ai-is-coming-for-the-lawyers

“This is a decision by a Colombian court in Cartagena (dated January, 30, 2023).
As far as we know, it is the first time that a judicial decision has been taken by explicitly resorting to #ChatGPT @sama @OpenAI. The Court poses a series of specific questions to #ChatGPT":
https://twitter.com/ethics_law/status/1628793562698510344

"Don't Be A Lawyer" song:

Rep. Ted Lieu introduces legislation written by
https://lieu.house.gov/media-center/press-releases/rep-lieu-introduces-first-federal-legislation-ever-written-artificial

DoNotPay offers money to anyone willing to use their AI to argue in court: https://twitter.com/jbrowder1/status/1612312707398795264?s=20&t=g6_oG7cPzEVX5H36DoHtNQ

Vanderbilt University responds to MSU shooting with e-mail written using ChatGPT:
https://vanderbilthustler.com/2023/02/17/peabody-edi-office-responds-to-msu-shooting-with-email-written-using-chatgpt/

Science fiction magazine closes submissions due to LLM spam:
https://twitter.com/ShackletonCI/status/1627730312925966336

The 1st International Workshop on Implicit Author Characterization from Texts for Search and Retrieval (IACT’23):
https://en.sce.ac.il/news/iact23

Journalist Dave Lee's Signal number given out by ChatGPT:
https://mastodon.social/@daveleeFT/109876410735127795

You can check out future streams at http://twitch.tv/dair_institute.

Follow Emily at https://twitter.com/EmilyMBender and https://dair-community.social/@EmilyMBender
Follow Alex at https://twitter.com/alexhanna and https://dair-community.social/@alex

You can check out future streams at http://twitch.tv/dair_institute.

Follow Emily at https://twitter.com/EmilyMBender and https://dair-community.social/@EmilyMBender
Follow Alex at https://twitter.com/alexhanna and https://dair-community.social/@alex

Music: Toby Menon.
Production: Christie Taylor.

Dave Lee

2023-02-16 20:56:14

Having fun on Twitter unpicking a curious thing that happened to me this morning. Someone reached me on Signal thinking I was ChatGPT. Turned out it had given them my number.