ChatGPT discussion

Rand al'Thor

Jan 29, 2024 08:17

Having read Gareth's analysis of an AI-generated answer ... wow. It's kind of scary how a totally false but seemingly well-researched answer can be spat out in a fraction of a second by these bots.

verbose

m yeah

Rand al'Thor

Did you read Gareth's analysis of a ChatGPT-generated answer on this site? The only way a human answer could be that bad would be if someone was maliciously trolling, inventing spurious chapter/page numbers to back up false claims in a plausible-looking way. I'd consider that sort of behaviour potentially grounds for suspension even if no AI was involved. — Rand al'Thor ♦ 2 mins ago

@verbose Goodnight, monodont!

verbose

oh yeah that's a good point, I'dn't thought about how such an answer would cause a user to be suspended

Mithical

@verbose FWIW, that history isn't visible to anyone without the privilege of seeing deleted posts (4k rep).

Rand al'Thor

*2k

Mithical

Jan 29, 2024 08:24

right

Rand al'Thor

Jan 29, 2024 09:01

@Bookworm The HNQ's a pleasant place, / Its private life is a disgrace: / I really could not tell to you / The awful state of HNQ.

Bookworm

Jan 29, 2024 10:23

0

Q: Why must "she" perform impossible tasks in order to be "a true love of mine" in "Scarborough Fair"?

"Scarborough Fair" is, according to Wikipedia, a traditional English ballad, with many different versions. However, one thing that the versions have in common, is setting impossible tasks and saying "then she'll be a true love of mine". For instance, in the version sung by The Hound + The Fox, t...

song-lyrics

Tsundoku

Jan 29, 2024 10:39

What would you find in Charles Dickens’s pantry?
The best of thyme, the worst of thyme.

Why is John Milton terrible to invite to game nights?
Because whenever he's around, there's a pair of dice lost.

Alex

Jan 29, 2024 12:20

@Randal'Thor Humans apparently make things up too. Even respected literary critics.

15

Q: Why did literary critic Harold Bloom say something that didn't correspond to reality regarding Harry Potter and the Sorcerer's Stone?

In this opinion piece by renowned late literary critic Harold Bloom, we see him levy the following charge against Harry Potter and the Sorcerer's Stone: I went to the Yale bookstore and bought and read a copy of “Harry Potter and the Sorcerer’s Stone.” I suffered a great deal in the process. The...

harry-potter literary-criticism harold-bloom

Alex

Jan 29, 2024 12:39

Or this one, since you pointed it out:

14

A: Where in the books does it say the first victor of District 12 was female?

It doesn't, in the original trilogy. We don't really know anything about the other District 12 victor who wasn't Haymitch, except the facts that they existed and that they died prior to the start of the first book. It's important to be aware that the Hunger Games Wikia is extremely unreliable. ...

Alex

Jan 29, 2024 12:59

Regarding the point about page numbers:

1

A: Source for Frodo/Gandalf exchange about wounds in "The Return of the King"

Beginning of Chapter 7 (Homeward Bound)... This happens to be, in my edition, page 325, though is probably different in yours because there can be many different editions. I suggest reading the Table of Contents to determine this. Here's an excerpt: Chapter 7 Homeward Bound At last the hobbits h...

Gareth Rees

Jan 29, 2024 13:38

@Alex Do you really consider that a fair comparison with the ChatGPT answer?

DLosc

Jan 29, 2024 19:25

@verbose I like Christie, but I have to agree that Passenger to Frankfurt was really bad. I assumed that it was because she was trying a new genre and it didn't go well.

Gareth Rees

Jan 29, 2024 20:03

More thoughts on language model policy. Alex's argument is that language models are just tools, and any tool can be used in error. We don't ban people from using libraries even though their shelves contain myths, hoaxes, misattributions, etc.

This is an attractive argument but in my opinion it doesn't adequately engage with the nature of language models. Our main defence against myths, hoaxes, misattributions, etc. is diligent use and careful checking of citations. But language models can generate arbitrarily many plausible claims, replete with pseudo-citations, at the touch of a button, overwhelming anyone's ability to check up on them.

The way to appreciate this is to spend time carefully fact-checking and correcting a sample of language model output. This ought to bring home the terrible ratio between the ease of generating nonsense and the difficulty of checking and refuting it.

Tsundoku

Jan 29, 2024 21:59

In addition to confidently providing incorrect information, LLMs can also violate copyright. Generative AI has been caught plagiarising content: Generative AI Has a Visual Plagiarism Problem.

Tsundoku

Jan 29, 2024 22:11

I disagree with the statement that LLMs are "just tools" for another reason: they rely on the same type of cheap labour as other parts of our extractive economy: OpenAI Used Kenyan Workers on Less Than $2 Per Hour to Make ChatGPT Less Toxic.

For people tired of the AI hype, read for example Dan McQuillan's articles We come to bury ChatGPT, not to praise it. and ChatGPT: The world’s largest bullshit machine.

Alex

Jan 29, 2024 23:49

@GarethRees I agree that AI makes it easier to quickly generate lots of bad content, but I don’t think that fundamentally changes the way we should look at it. If this was a bigger site and we were swamped with such posts on a daily basis, my approach might not be as practical, but with our current volume it seems to be easily workable.

Bookworm

0

Q: A children's book with a rhyme naming cities in western England

I remember reading a few details about three children's books I read in elementary school, presumably borrowed from the school library instead of the local library, because I remember them in connection with a school I attended in November 1961 to June 1962. And it is possible that maybe those de...

identification-request

Alex

My Meta answer about this was already at -4 a few hours after I posted it, and that was with it not being on the main site and not being a new question, which would seem to indicate that the site has the manpower to sufficiently downvote posts as they come along.

So when someone posts something invented by ChatGpT we can just downvote it to indicate that it is not a good answer.

@Alex Though I suppose it could be the reverse, and on Meta it’s easier to accumulate the negative score, since the main site might have hordes of users who just blindly upvote things.

Bookworm

Jan 30, 2024 00:15

0

Q: Who is John H. Crabb"

I am trying to find out about John H. Crabb? Crabb wrote a historical novel entitled In the Crescent's Dark Shadow in 1952.

identification-request

Peter Shor

@verbose Refugees by Macneice

Gangways – the handclasp of the land. The resurrected,

The brisk or resigned Lazaruses, who want

Another chance, go trooping ashore. But chances

Are dubious. Fate is stingy, recalcitrant.

Refugee Blues, by Auden:

Saw a poodle in a jacket fastened with a pin,

Saw a door opened and a cat let in:

But they weren’t German Jews, my dear, but they weren’t German Jews.

(Compare and contrast.)

CDR

Jan 30, 2024 01:04

I'm afraid to make the tag, considering that this site has pretty strict guidelines about tag-format and stuff; would a [mysterious-author] tag be useful? It seems to me that nothing else could fit on this recent question, and the question about Djuna could also maybe use it. If [mysterious-authors] is a poor fit, what else could the first question I linked be tagged with?

Bookworm

Jan 30, 2024 01:47

0

Q: A childen's book with a rhyme about a badger's home

I remember a few details about three children's books I read in elementary school in the USA, presumably borrowed from the school library instead of the local library, because I remember them in connection with a school I attended in November 1961 to June 1962. And it is possible that maybe those...

identification-request

Bookworm

Jan 30, 2024 02:56

0

Q: A children's book with a Beast in a forest

I remember a few details about three children's books I read in elementary school in the USA, presumably borrowed from the school library instead of the local library, because I remember them in connection with a school I attended in November 1961 to June 1962. And it is possible that maybe those...

identification-request

verbose

Jan 30, 2024 05:50

@DLosc She had written at least two books in that genre before: They Came to Baghdad and Destination Unknown

@PeterShor Thank you. Both extraordinarily moving poems.

@Mithical On the one hand, fair point; on the other, quibble.

@Randal'Thor cute!

@Alex The point is that ChatGPT generated answers are hard to identify as bad because they are so plausible. Yes, ChatGPT is a tool, but it is not a valid, useful, or reliable tool for generating accurate facts or defensible analyses. It's a con artist, presenting complete drivel as impeccable argument.

2

Your defense of it basically boils down to "it's fine if someone doesn't want to do the work of coming up with an answer and just uses ChatGPT, because others (those who care about the site) should take on the responsibility of evaluating such answers."

Why should I waste my time doing the sort of investigative work Gareth did on the answer? Why should Gareth? You're asking us to take on asymmetric effort on the grounds that ChatGPT is "a tool." That's like saying "a hammer is a tool, and it's fine if someone takes a hammer to your house, because you're the one responsible for rebuilding your house with other tools."

Nah, I'd say, make it known that the use of the tool is destructive and we don't want people using it here.

I put in a lot of work into my answers for this site. It's demotivating for me to do all that work if my answers have to fight for space alongside machine generated bullshit.

5

Rand al'Thor

Jan 30, 2024 08:39

@Bookworm Impossible to be a true love of HNQ.

@Alex That's one problem, plus meta votes often indicate simple agreement/disagreement. Main-site posts will often be upvoted for being well-researched, even if the voter hasn't actually gone to verify all the information in them. ChatGPT posts can seem well-researched even if they're totally false.

@PeterShor Oh, I remember this one. It was one of the poems included in my English Lit GCSE (or English Lang maybe).

@CDR Hmm. The obvious thing to tag that question with is simply [john-h-crabb], but that's not helpful for a new user who doesn't have enough rep to create the tag. [author-identification] exists as a synonym of [identification-request], but I'm not sure if that really fits here. I feel like a tag name of [mysterious-author] might give a wrong impression (author of mystery stories? might be misused on any ID question?) but I can't think of anything better ...

verbose

@Bookworm Crabbwise into the HNQ

@Randal'Thor author-biography? Though I don't see an issue with just leaving it john-h-crabbe

Bookworm

Jan 30, 2024 10:18

0

Q: What makes the Sydney barber's remark rude?

In Banjo Paterson's poem "The Man from Ironbark", a city barber makes a supposedly rude remark to his country bumpkin customer: And as he soaped and rubbed it in he made a rude remark: "I s'pose the flats is pretty green up there in Ironbark." It's not clear to me what makes this rude. I can ima...

meaning poetry

Gareth Rees

Jan 30, 2024 10:35

@Alex The process that you seem to be imagining is: (i) language model output gets posted as an answer; (ii) diligent members of the site fact-check the language model output and point out mistakes in comments; (iii) the language model output gets down-voted.

I don't think this properly engages with the nature of language models. Posting language model output is easy but fact-checking it is difficult and time-consuming, so the fact-checkers will burn out. I strongly recommend having a go at it yourself, for example, try your skills on the monograph question.

CowperKettle

Jan 30, 2024 10:50

> Albeit nurtured in democracy,
And liking best that state republican
Where every man is Kinglike and no man
Is crowned above his fellows, yet I see,
Spite of this modern fret for Liberty,
Better the rule of One, whom all obey,
Than to let clamorous demagogues betray
Our freedom with the kiss of anarchy.
Wherefore I love them not whose hands profane
Plant the red flag upon the piled-up street
For no right cause, beneath whose ignorant reign
Arts, Culture, Reverence, Honour, all things fade,
Save Treason and the dagger of her trade,

Wilde of the day

Tsundoku

Jan 30, 2024 14:22

> For you yourself, sir, should be old as I am
if, like a crab, you could go backward.

CowperKettle

Jan 30, 2024 14:52

Контракт / Contract (1985) HD [EN subtitles]

►

Soviet cartoon based on a sci-fi story by Robert Silverberg

verbose

Jan 30, 2024 19:12

(Sigh) The Bodleian wants me to give them £10 for looking at that Larkin/Pym letter. I’m a weakling and can barely lift ten pounds. How am I supposed to carry it all the way to Oxford?

Rand al'Thor

Jan 30, 2024 19:46

@verbose Odsbodikins, what an odd Bodleian. Quomodocunquising is not meet for an institution of academe, methinks.

Alex

Jan 30, 2024 22:12

@GarethRees I don’t know enough about monographs to evaluate the accuracy of the claims in the answer, but I would approach it like any other answer — it is not very useful because it doesn’t provide any sources or evidence for any of the things it claims. So I wouldn’t upvote it and I would possibly downvote it. However, it is useful in the sense that it provides specific ideas that (if I was interested on the topic) I could research further, with more ease than if I was starting from scratch.

If the site was getting overrun with ChatGPT posts, it might be a different story. But as it stands now, it seems that this is pretty rare, so I don’t think it warrants drastic action.

verbose

@Randal'Thor Ikr, it's like they wanna extract their pound of flesh

@Alex It does claim to provide sources and evidence. That is the problem.

verbose

Jan 30, 2024 22:42

not the monograph q specifically, I mean; ChatGPT will happily make up references, provide page numbers, etc.

verbose

Jan 30, 2024 23:29

@CowperKettle Personally I'm not that fond of my state's Republicans (Kevin McCarthy, for example) 🙃

@Alex I don't think updating our off-topic guidance to include ChatGPT generated questions/answers constitutes "drastic action"

Alex

Jan 30, 2024 23:56

@verbose I was talking about that specific post. Since it has no citations/evidence it is not very useful, even had it been written by an actual person. But where it does provide references, they can be checked, which is what I would generally do if a human posted something as well.

verbose

That's what I mean when I say it shifts the burden of proof from the answerer to the asker/reader. If I ask a question here and I get a plausible-seeming answer with references to volumes that I have no access to, then I have to put in a lot of work to verify that answer ... to the point that it would have been easier to answer the question myself in the first place.

I think that is against the spirit of Stack Exchange, where we should be able to trust that those who answer know what they're talking about.

3

Alex

Jan 31, 2024 01:23

@verbose It's an anonymous internet site. You can't trust anything unless you either verify it, or it comes from someone who has already earned your trust (e.g. by posting lots of good answers already)

In that sense, a new user is no different from ChatGPT.

Also, a policy of banning ChatGPT answers presupposes that they can be identified easily enough. If that is the case then we can just ignore them if we don't want to have to do further research to know if the answer is accurate.

verbose

Jan 31, 2024 01:39

@Alex No, it doesn't? It tells the person posting an answer that "we know you think ChatGPT gives good answers to the questions people ask here. It doesn't. So we don't want people posting ChatGPT generated answers here."

I don't see how "a new user is no different from ChatGPT". You're assuming bad faith users. I'm assuming good faith ones. That assumption is basic to Stack Exchange. Nobody would post questions and answers on here unless we believed the answers were plausible.

If you want to say, "caveat lector," then fine; why bother with Stack Exchange at all? We advertise ourselves as being a forum for quality questions and answers. ChatGPT does not provide quality questions and answers.

To put it another way: we can assume a ChatGPT posted answer is (a) plausible (b) hard to verify (c) bad. If we made those assumptions for Stack Exchange answers generally, then the site loses its raison d'être

Alex

@verbose If the policy under discussion is simply an announcement that ChatGPT answers are not very beneficial and we don't like them very much, I'm all for it. My understanding, however, was that we are talking about a ban, which includes deleting such answers.

@verbose In that case, assuming good faith users assumes that they wouldn't post unverified (plausible-sounding) nonsense from ChatGPT either.

verbose

Jan 31, 2024 01:54

@Alex I think that's a hair-splitting distinction. Yes, I think we should delete chatGPT generated answers. I think that given the prevailing assumption that answers here are reliable, allowing answers from a known unreliable source is pointless.

Alex

But I don't think "assuming good faith" means to accept things without verifying them. It means that if something can be interpreted charitably or uncharitably we should generally opt for the charitable interpretation.

verbose

@Alex Not necessarily. I posted what I thought was a good answer to an identification-request question. I didn't just rely on ChatGPT. I verified that the book ChatGPT identified as the correct one did exist, but I accepted that the rough outlines of the story according to ChatGPT were correct. Luckily I had said in my answer that I used ChatGPT, so someone suggested I look deeper.

Turns out that ChatGPT just picked a book with some relevant matches seemingly at random, paraphrased the details in the question in its own words, and presented that as an answewr

@Alex There is no reason to extend charity to ChatGPT.

Alex

@verbose Having a couple of ChatGPT answers scattered across the site would not materially affect the overall quality of the site. Also, I don't think I agree that all ChatGPT answers are automatically bad quality. I have a fair amount of experience with ChatGPT, and while it does make various errors (both factual and logical), more often than not I have found it to be very helpful. That is why I suggest treating each post on its own merits.

verbose

I've found ChatGPT helpful too—but not for literary analysis, or even for facts about literary history. Such answers from ChatGPT are poor quality.

And those are the questions our site traffics in.

Yes, ChatGPT answers here can be assumed to be bad quality.

I've read stories from humanities professors saying "this was my prompt, this was what ChatGPT came up with! A solid B or better!" And my response has always been, if that's what you're handing out a B to, then speaking as a teacher of writing and literature, your standards are too low. And in fact, the prevailing sentiment among humanities faculty these days matches my sense that ChatGPT just presents word salad with no actual argument or even fidelity to facts.

Alex

@verbose I make the same assumptions for all answers of unknown pedigree. If an answer doesn't demonstrate its correctness, or provide citations/evidence for its claims then it's not worth much regardless of how the answer was generated. If it does demonstrate its correctness, or provide citations/evidence then it is useful regardless of how it was generated.

verbose

Jan 31, 2024 02:07

@Alex But the point is that ChatGPT can never write an answer that demonstrates its correctness or provide citations/evidence. I don't see why you keep saying it's a tool like any other. "Regardless of how it was generated" is reasonable only if you assume that ChatGPT is neutral. It is not the case that ChatGPT is neutral.

Alex

@verbose There too I would treat ChatGPT like anyone else. I would not accept that the rough outlines of the story were correct just because ChatGPT said so, or because a book review by a person said so. Where possible, I would verify the contents of the book myself, and if not possible, I would put a disclaimer on the answer saying that "according to So-And-So this book is about XYZ".

@verbose Not extending charity to ChatGPT; extending charity to the user posting it.

verbose

Gareth has painstakingly shown that the answers chatGPT generates are not to be trusted. Your stance, that we should trust no answers, seems to miss the whole point and purpose of ChatGPT

@Alex It is charitable to prevent someone from falling into avoidable error. How is it not charitable to say: Don't rely on ChatGPT for your questions/answers, because our experience has shown such questions/answers to be hot messes despite their plausibility?

Alex

@verbose Sure it can. Just because it sometimes (often?) makes up citations, doesn't mean that in never provides real citations.

@verbose If the policy under discussion is merely a recommendation not to rely on ChatGPT for your questions/answers, then I wholeheartedly support it.

verbose

@Alex Why should the burden of verifying those citations fall on the reader rather than the poster? It's against the spirit of Stack Exchange to say, "don't assume we know what we're talking about." And you're saying "it's fine for the poster to just use ChatGPT, and let the rest of us put in the hard work the poster is unable or unwilling or too lazy to do."

2

Yeah, it's clear you and I aren't going to see eye-to-eye on this.

Alex

@verbose I don't think anyone is forced to undertake a burden. It's perfectly fine for someone to read an answer and say "this looks like a ChatGPT answer, which are usually nonsense, so I will downvote and move on".

@verbose I enjoy hearing other people's perspectives, especially high ranking members of the site.

verbose

Jan 31, 2024 02:21

Figuring out whether something is real or ChatGPT is a task that's burdensome. If you don't care to put in that work, it's fine; but in my view, not doing the work would put the entire site at risk. The plausibility of ChatGPT answers make them all the more likely to be upvoted; the answer I posted got two or three upvotes in the hour or two it was up before I realized it was just wrong.

If I care about the quality of this site, then yes, weeding out bad answers is a responsibility I assume. And if ChatGPT posts answers that are poor quality but endowed with enough truthiness for upvotes, then yes, it's risky to allow those answers here.

And we have no enforcement mechanism for a "recommendation", do we? A ban is more clearcut. I don't think ChatGPT adds any value to the site, so why not ban it? Your stance that it's a neutral tool, and/or that answers from ChatGPT are no better or worse than other answers on here and should be treated the same way, is not one I can agree with. ... so here we are.

Alex

That's the nature of a free site where anyone can post what they want. There were plenty of upvoted wrong answers before ChatGPT came along. A few months ago I saw an answer (on another site on the network) which (I believe) is demonstrably wrong, and I posted an answer that (I believe) demonstrates that, yet the original answer is accepted with a score of +38. (And the user who posted that answer is the site's top user.)

If determining whether an answer is written by ChatGPT is a burden, it will be the same burden whether the consequence is deleting it or just downvoting it.

"weeding out bad answers" is done by downvoting, not by deleting.

verbose

These things do happen; I can think of three or four examples of my own answers here and on Stack Exchange which are, in my view, better than the upvoted/accepted answers. My point is that that is not a good thing overall, and allowing ChatGPT answers will make that problem worse, to the point that the site will become useless.

@Alex If we state clearly that using ChatGPT is not allowed, we can at least assume that answers don't use ChatGPT. That reduces the burden. Sure, someone can post a ChatGPT generated answer anyway, but that is a risk we can't avoid. And in that case, we can put in the work of determining it was ChatGPT generated, and delete the answer.

Alex

I mentioned in my Meta post that a key point is that we are not being overrun by such answers. I have yet to organically come across such an answer, and the Meta post arguing the other side identified just one such answer on the site. If it was getting to the point that the site was becoming useless, I think it would be a different discussion.

verbose

@Alex We've had more than just the one. We are getting to the point where we need to get ahead of the problem. I don't see any downside to being proactive. "I can see the storm clouds gathering, but I'll leave my umbrella at home. If it starts to rain while I'm out, I can buy another."

2

I think we're rehashing an argument mods had with the organization that led to the strike ...

Alex

Jan 31, 2024 02:46

I wouldn't want to be the cause of anyone striking....

The Reading Room

Participants