March 31, 2023

I had heard a bit about this in discussions of inaccurate information from AI engines like ChatGPT. But hearing it directly from TPM Readers AT (a research librarian) and JC brought it into sharper relief for me. Inaccurate information can mean a lot of things. You can say Thomas Jefferson died in 1846 when in fact he died in 1826. Other inaccuracies are more complex and pernicious.

In my earlier post, I noted that when I asked ChatPGT for data on the trajectory of COVID case mortality rates, it came pretty close to my ideal result: a series of studies from reputable journals which gave me a start on pulling this information together. I noted that it was a bit annoying that it didn’t include the links. But I’m told that’s part of the guardrails since the whole application is still in a testing phase.

I assume that’s true. But the lack of links probably obscured something pretty important.

As AT and JC explained to me, many of these studies turn out not to exist. They’re simply made up, or rather they are the kinds of studies and articles that could exist but don’t. I might have published an article in the American Historical Review in 1998. It fits with various details of my academic background, the fact that I entered a PhD program in American history in 1992. But I didn’t. I wrote one in 1995 in a different academic journal. AT notes that she’s had a number of cases where academics come into the library looking for help locating one of the articles they’ve been referred to by ChatGPT. But the articles don’t exist.


If this were a person writing an article or a school paper this would be textbook evidence of willful deception. Anyone can get the dates of a president’s life wrong, either through error or sloppiness. But you can’t accidentally come up with the details — journal name, date, volume, page numbers, author — of an academic article. You can only make it up. The very facticity of the detail makes it fabrication rather than error.

So what’s going on here?

I understand very little about how AI works. So I’ll keep this all at a very high level of generality. But at a basic level, the technology isn’t great at distinguishing between things that are real and things that are not real. One of the things people do is ask ChatGPT to write a poem in the style of this or that poet. Or write a short essay on Thomas Jefferson — the kind a student might turn in as a homework assignment. In that case it’s literally making things up based on putting together facts and connections and statistical patterns it’s absorbed by consuming vast amounts of information on the web. The whole magic of it is that it’s not just finding a few paragraph essay somewhere on the web and showing it to you. It’s creating something new, something that didn’t exist. But clearly the technology isn’t always able to distinguish between creating something in a certain style or format as a person might write it (but didn’t) and producing claims that might have been true but are in fact not true.

There are whole fields of human intellection devoted to how these things differ and how we understand those differences — formal logic, cognitive and neuroscience, machine learning, linguistics, computer science and maybe 20 other fields. It’s complicated. So maybe we shouldn’t be so hard on ChatGPT. But they are foundational. So foundational that they’re hard to spot at first.

TPM Reader JL, who has some distant academic background in the field, told me that for all the amazing, stunning leaps forward in the technology, “to a large degree it’s really just a clever trick. ChatGPT is ultimately just stringing chunks of text together based on statistical observations without any understanding of what it’s talking about. It’s amazing how far that gets you but the idea that the errors it makes are just kinks to be worked out seems like wishful thinking.”

today s free spins coins daily coin master rewards 2023 3000 free tiktok coins generator hacks 2023 without downloading redeem code for dreame app surf pm de free tiktok coins generator 2023 24 view rent amenities features and contact 12353 mountain road unlimited free coins tiktok tiktok coins generator 8e0 taringa ai art generator tiktok free shirdi sai baba temple new tiktok coins hack tiktok coins generator get tiktok genuine free tiktok coins generator generator revealed free how to get free tiktok coins in 2023 updated v 294577 tiktok coins hack tiktok coins generator get instantly free tiktok coin generator no human unlimited free coins tiktok tiktok coins generator 8e0 taringa free tiktok coins how to earn coins on tiktok in 2023 mq 307564 tiktok views free coffee academy lazio how to get free tiktok coins itech hacks how to get free tiktok coins in 2023 tecnoguias free tiktok coin generator 2023 no human verification free tiktok coins generator 2023 no human verifition md how to get free tiktok coins 2023 product hunt coin birds crystal hack 99copytrading tiktok coins free 2023 2023 tiktok coins generator tiktok free tiktok coins generator 2023 photographie free tiktok coins generator 2020 free tiktok coins how to get free tik tok coins in tiktok coins hack tiktok coins converter to convert tiktok coins into cash using 3000 free tiktok coins generator hacks 2023 without downloading how to get free tik tok coins in tiktok coins hack 5000 free tiktok followers no verification free tiktok coins generator

Leave a Reply

Your email address will not be published. Required fields are marked *