More thoughts on language model policy. Alex's argument is that language models are just tools, and any tool can be used in error. We don't ban people from using libraries even though their shelves contain myths, hoaxes, misattributions, etc.
This is an attractive argument but in my opinion it doesn't adequately engage with the nature of language models. Our main defence against myths, hoaxes, misattributions, etc. is diligent use and careful checking of citations. But language models can generate arbitrarily many plausible claims, replete with pseudo-citations, at the touch of a button, overwhelming anyone's ability to check up on them.
The way to appreciate this is to spend time carefully fact-checking and correcting a sample of language model output. This ought to bring home the terrible ratio between the ease of generating nonsense and the difficulty of checking and refuting it.
In addition to confidently providing incorrect information, LLMs can also violate copyright. Generative AI has been caught plagiarising content: Generative AI Has a Visual Plagiarism Problem.
@GarethRees I agree that AI makes it easier to quickly generate lots of bad content, but I don’t think that fundamentally changes the way we should look at it. If this was a bigger site and we were swamped with such posts on a daily basis, my approach might not be as practical, but with our current volume it seems to be easily workable.
I remember reading a few details about three children's books I read in elementary school, presumably borrowed from the school library instead of the local library, because I remember them in connection with a school I attended in November 1961 to June 1962.
And it is possible that maybe those de...
My Meta answer about this was already at -4 a few hours after I posted it, and that was with it not being on the main site and not being a new question, which would seem to indicate that the site has the manpower to sufficiently downvote posts as they come along.
So when someone posts something invented by ChatGpT we can just downvote it to indicate that it is not a good answer.
@Alex Though I suppose it could be the reverse, and on Meta it’s easier to accumulate the negative score, since the main site might have hordes of users who just blindly upvote things.
I'm afraid to make the tag, considering that this site has pretty strict guidelines about tag-format and stuff; would a [mysterious-author] tag be useful? It seems to me that nothing else could fit on this recent question, and the question about Djuna could also maybe use it. If [mysterious-authors] is a poor fit, what else could the first question I linked be tagged with?
I remember a few details about three children's books I read in elementary school in the USA, presumably borrowed from the school library instead of the local library, because I remember them in connection with a school I attended in November 1961 to June 1962.
And it is possible that maybe those...
I remember a few details about three children's books I read in elementary school in the USA, presumably borrowed from the school library instead of the local library, because I remember them in connection with a school I attended in November 1961 to June 1962.
And it is possible that maybe those...
@DLosc She had written at least two books in that genre before: They Came to Baghdad and Destination Unknown
@PeterShor Thank you. Both extraordinarily moving poems.
@Mithical On the one hand, fair point; on the other, quibble.
@Randal'Thor cute!
@Alex The point is that ChatGPT generated answers are hard to identify as bad because they are so plausible. Yes, ChatGPT is a tool, but it is not a valid, useful, or reliable tool for generating accurate facts or defensible analyses. It's a con artist, presenting complete drivel as impeccable argument.
2
Your defense of it basically boils down to "it's fine if someone doesn't want to do the work of coming up with an answer and just uses ChatGPT, because others (those who care about the site) should take on the responsibility of evaluating such answers."
Why should I waste my time doing the sort of investigative work Gareth did on the answer? Why should Gareth? You're asking us to take on asymmetric effort on the grounds that ChatGPT is "a tool." That's like saying "a hammer is a tool, and it's fine if someone takes a hammer to your house, because you're the one responsible for rebuilding your house with other tools."
Nah, I'd say, make it known that the use of the tool is destructive and we don't want people using it here.
I put in a lot of work into my answers for this site. It's demotivating for me to do all that work if my answers have to fight for space alongside machine generated bullshit.
@Alex That's one problem, plus meta votes often indicate simple agreement/disagreement. Main-site posts will often be upvoted for being well-researched, even if the voter hasn't actually gone to verify all the information in them. ChatGPT posts can seem well-researched even if they're totally false.
@PeterShor Oh, I remember this one. It was one of the poems included in my English Lit GCSE (or English Lang maybe).
@CDR Hmm. The obvious thing to tag that question with is simply [john-h-crabb], but that's not helpful for a new user who doesn't have enough rep to create the tag. [author-identification] exists as a synonym of [identification-request], but I'm not sure if that really fits here. I feel like a tag name of [mysterious-author] might give a wrong impression (author of mystery stories? might be misused on any ID question?) but I can't think of anything better ...
In Banjo Paterson's poem "The Man from Ironbark", a city barber makes a supposedly rude remark to his country bumpkin customer:
And as he soaped and rubbed it in he made a rude remark:
"I s'pose the flats is pretty green up there in Ironbark."
It's not clear to me what makes this rude.
I can ima...
@Alex The process that you seem to be imagining is: (i) language model output gets posted as an answer; (ii) diligent members of the site fact-check the language model output and point out mistakes in comments; (iii) the language model output gets down-voted.
I don't think this properly engages with the nature of language models. Posting language model output is easy but fact-checking it is difficult and time-consuming, so the fact-checkers will burn out. I strongly recommend having a go at it yourself, for example, try your skills on the monograph question.
> Albeit nurtured in democracy, And liking best that state republican Where every man is Kinglike and no man Is crowned above his fellows, yet I see, Spite of this modern fret for Liberty, Better the rule of One, whom all obey, Than to let clamorous demagogues betray Our freedom with the kiss of anarchy. Wherefore I love them not whose hands profane Plant the red flag upon the piled-up street For no right cause, beneath whose ignorant reign Arts, Culture, Reverence, Honour, all things fade, Save Treason and the dagger of her trade,
(Sigh) The Bodleian wants me to give them £10 for looking at that Larkin/Pym letter. I’m a weakling and can barely lift ten pounds. How am I supposed to carry it all the way to Oxford?
@GarethRees I don’t know enough about monographs to evaluate the accuracy of the claims in the answer, but I would approach it like any other answer — it is not very useful because it doesn’t provide any sources or evidence for any of the things it claims. So I wouldn’t upvote it and I would possibly downvote it. However, it is useful in the sense that it provides specific ideas that (if I was interested on the topic) I could research further, with more ease than if I was starting from scratch.
If the site was getting overrun with ChatGPT posts, it might be a different story. But as it stands now, it seems that this is pretty rare, so I don’t think it warrants drastic action.
@verbose I was talking about that specific post. Since it has no citations/evidence it is not very useful, even had it been written by an actual person. But where it does provide references, they can be checked, which is what I would generally do if a human posted something as well.
That's what I mean when I say it shifts the burden of proof from the answerer to the asker/reader. If I ask a question here and I get a plausible-seeming answer with references to volumes that I have no access to, then I have to put in a lot of work to verify that answer ... to the point that it would have been easier to answer the question myself in the first place.
I think that is against the spirit of Stack Exchange, where we should be able to trust that those who answer know what they're talking about.
@verbose It's an anonymous internet site. You can't trust anything unless you either verify it, or it comes from someone who has already earned your trust (e.g. by posting lots of good answers already)
In that sense, a new user is no different from ChatGPT.
Also, a policy of banning ChatGPT answers presupposes that they can be identified easily enough. If that is the case then we can just ignore them if we don't want to have to do further research to know if the answer is accurate.
@Alex No, it doesn't? It tells the person posting an answer that "we know you think ChatGPT gives good answers to the questions people ask here. It doesn't. So we don't want people posting ChatGPT generated answers here."
I don't see how "a new user is no different from ChatGPT". You're assuming bad faith users. I'm assuming good faith ones. That assumption is basic to Stack Exchange. Nobody would post questions and answers on here unless we believed the answers were plausible.
If you want to say, "caveat lector," then fine; why bother with Stack Exchange at all? We advertise ourselves as being a forum for quality questions and answers. ChatGPT does not provide quality questions and answers.
To put it another way: we can assume a ChatGPT posted answer is (a) plausible (b) hard to verify (c) bad. If we made those assumptions for Stack Exchange answers generally, then the site loses its raison d'être
@verbose If the policy under discussion is simply an announcement that ChatGPT answers are not very beneficial and we don't like them very much, I'm all for it. My understanding, however, was that we are talking about a ban, which includes deleting such answers.
@verbose In that case, assuming good faith users assumes that they wouldn't post unverified (plausible-sounding) nonsense from ChatGPT either.
@Alex I think that's a hair-splitting distinction. Yes, I think we should delete chatGPT generated answers. I think that given the prevailing assumption that answers here are reliable, allowing answers from a known unreliable source is pointless.
But I don't think "assuming good faith" means to accept things without verifying them. It means that if something can be interpreted charitably or uncharitably we should generally opt for the charitable interpretation.
@Alex Not necessarily. I posted what I thought was a good answer to an identification-request question. I didn't just rely on ChatGPT. I verified that the book ChatGPT identified as the correct one did exist, but I accepted that the rough outlines of the story according to ChatGPT were correct. Luckily I had said in my answer that I used ChatGPT, so someone suggested I look deeper.
Turns out that ChatGPT just picked a book with some relevant matches seemingly at random, paraphrased the details in the question in its own words, and presented that as an answewr
@Alex There is no reason to extend charity to ChatGPT.
@verbose Having a couple of ChatGPT answers scattered across the site would not materially affect the overall quality of the site. Also, I don't think I agree that all ChatGPT answers are automatically bad quality. I have a fair amount of experience with ChatGPT, and while it does make various errors (both factual and logical), more often than not I have found it to be very helpful. That is why I suggest treating each post on its own merits.
I've found ChatGPT helpful too—but not for literary analysis, or even for facts about literary history. Such answers from ChatGPT are poor quality.
And those are the questions our site traffics in.
Yes, ChatGPT answers here can be assumed to be bad quality.
I've read stories from humanities professors saying "this was my prompt, this was what ChatGPT came up with! A solid B or better!" And my response has always been, if that's what you're handing out a B to, then speaking as a teacher of writing and literature, your standards are too low. And in fact, the prevailing sentiment among humanities faculty these days matches my sense that ChatGPT just presents word salad with no actual argument or even fidelity to facts.
@verbose I make the same assumptions for all answers of unknown pedigree. If an answer doesn't demonstrate its correctness, or provide citations/evidence for its claims then it's not worth much regardless of how the answer was generated. If it does demonstrate its correctness, or provide citations/evidence then it is useful regardless of how it was generated.
@Alex But the point is that ChatGPT can never write an answer that demonstrates its correctness or provide citations/evidence. I don't see why you keep saying it's a tool like any other. "Regardless of how it was generated" is reasonable only if you assume that ChatGPT is neutral. It is not the case that ChatGPT is neutral.
@verbose There too I would treat ChatGPT like anyone else. I would not accept that the rough outlines of the story were correct just because ChatGPT said so, or because a book review by a person said so. Where possible, I would verify the contents of the book myself, and if not possible, I would put a disclaimer on the answer saying that "according to So-And-So this book is about XYZ".
@verbose Not extending charity to ChatGPT; extending charity to the user posting it.
Gareth has painstakingly shown that the answers chatGPT generates are not to be trusted. Your stance, that we should trust no answers, seems to miss the whole point and purpose of ChatGPT
@Alex It is charitable to prevent someone from falling into avoidable error. How is it not charitable to say: Don't rely on ChatGPT for your questions/answers, because our experience has shown such questions/answers to be hot messes despite their plausibility?
@verbose Sure it can. Just because it sometimes (often?) makes up citations, doesn't mean that in never provides real citations.
@verbose If the policy under discussion is merely a recommendation not to rely on ChatGPT for your questions/answers, then I wholeheartedly support it.
@Alex Why should the burden of verifying those citations fall on the reader rather than the poster? It's against the spirit of Stack Exchange to say, "don't assume we know what we're talking about." And you're saying "it's fine for the poster to just use ChatGPT, and let the rest of us put in the hard work the poster is unable or unwilling or too lazy to do."
2
Yeah, it's clear you and I aren't going to see eye-to-eye on this.
@verbose I don't think anyone is forced to undertake a burden. It's perfectly fine for someone to read an answer and say "this looks like a ChatGPT answer, which are usually nonsense, so I will downvote and move on".
@verbose I enjoy hearing other people's perspectives, especially high ranking members of the site.
Figuring out whether something is real or ChatGPT is a task that's burdensome. If you don't care to put in that work, it's fine; but in my view, not doing the work would put the entire site at risk. The plausibility of ChatGPT answers make them all the more likely to be upvoted; the answer I posted got two or three upvotes in the hour or two it was up before I realized it was just wrong.
If I care about the quality of this site, then yes, weeding out bad answers is a responsibility I assume. And if ChatGPT posts answers that are poor quality but endowed with enough truthiness for upvotes, then yes, it's risky to allow those answers here.
And we have no enforcement mechanism for a "recommendation", do we? A ban is more clearcut. I don't think ChatGPT adds any value to the site, so why not ban it? Your stance that it's a neutral tool, and/or that answers from ChatGPT are no better or worse than other answers on here and should be treated the same way, is not one I can agree with. ... so here we are.
That's the nature of a free site where anyone can post what they want. There were plenty of upvoted wrong answers before ChatGPT came along. A few months ago I saw an answer (on another site on the network) which (I believe) is demonstrably wrong, and I posted an answer that (I believe) demonstrates that, yet the original answer is accepted with a score of +38. (And the user who posted that answer is the site's top user.)
If determining whether an answer is written by ChatGPT is a burden, it will be the same burden whether the consequence is deleting it or just downvoting it.
"weeding out bad answers" is done by downvoting, not by deleting.
These things do happen; I can think of three or four examples of my own answers here and on Stack Exchange which are, in my view, better than the upvoted/accepted answers. My point is that that is not a good thing overall, and allowing ChatGPT answers will make that problem worse, to the point that the site will become useless.
@Alex If we state clearly that using ChatGPT is not allowed, we can at least assume that answers don't use ChatGPT. That reduces the burden. Sure, someone can post a ChatGPT generated answer anyway, but that is a risk we can't avoid. And in that case, we can put in the work of determining it was ChatGPT generated, and delete the answer.
I mentioned in my Meta post that a key point is that we are not being overrun by such answers. I have yet to organically come across such an answer, and the Meta post arguing the other side identified just one such answer on the site. If it was getting to the point that the site was becoming useless, I think it would be a different discussion.
@Alex We've had more than just the one. We are getting to the point where we need to get ahead of the problem. I don't see any downside to being proactive. "I can see the storm clouds gathering, but I'll leave my umbrella at home. If it starts to rain while I'm out, I can buy another."
2
I think we're rehashing an argument mods had with the organization that led to the strike ...
@PeterShor sadly, I'm old enough to remember phone books. And landlines. And rotary dials. And booking long distance calls ("trunk calls") via an operator. Sigh.
@Alex I never imagined that you were an expert on the history of doctoral monographs! But neither are any of us. So in order to fact-check this language model output, someone has to do the research in order to gain that expertise. That's why I've been emphasizing how difficult this is. You can't, in fairness, propose a policy under which other people need to do the work that you are not willing or interested in contributing to.
In the monograph case, one might start with the Doctor of Philosophy article on Wikipedia and follow the references to get some general background.
Ruano-Borbalan claims that the requirement to submit a written dissertation (the doctoral monograph) dates only from the "middle of the 19th century", which contradicts ChatGPT's vague and unsupported claim in paragraph 3, that the thesis "has been a central component of academic traditions for centuries".
Does that adequately dispose of paragraph 3? Well, maybe, but Ruano-Borbalan doesn't give a citation for this claim either. So more research is needed. And that's just one paragraph of eight.
@verbose and the table of domestic telephone call prices with four columns based on time of the day and day of week, and four rows based on how close the two landlines are.
@b_jonas I have to confess I don't remember those. All I remember is that we could only make calls within the city limits of Bombay. Anything beyond was a "trunk call." Then at some point, we didn't need to book a call on a trunk line via an operator, because the telephone service gave its entire customer base an STD: Subscriber Trunk Dialing.
D'y'all think @Tsundoku will respond to the "paging the 'doku! paging the 'doku!" shrieks in the last sentence of this answer?: "Hopefully someone who has access to Rolland's monograph and/or is an expert on La Fontaine and/or has a more competent grasp of French (and, for Corradi, Italian), will provide a better answer soon."
actually since Peter has access to academic libraries and speaks French, he'd be a good candidate for an answerer too.
@verbose By the time I was young, all the country was on automatic dialing, even long distance. The long distance calls were just more expensive. Calls via operator still existed, but you only rarely needed to use them. You could try as a curiosity to show your children via old phones that didn't have a rotary dial yet, or use it to initiate reverse fee calls, or perhaps to call into foreign countries that didn't have automated dialing yet.
Also even after that was gone, there was and perhaps is some services for international reverse fee dialing, which works by having a toll-free number in select countries that you call to make (necessarily operator-initiated) reverse fee calls into Hungary. The number has to be per initiating country because international toll-free numbers, while technically have a country code reserved, don't really exist as far as I can tell.
@GarethRees I am not proposing a policy that forces anyone to do research that I am not willing to do. What I am proposing is that in a case like the monograph answer you should downvote since a bunch of paragraphs with no citations or evidence is not a good answer.
If the topic interested you, you could then do your own research and either post your answer (which you would probably have done even if the other answer didn’t exist) or refute the existing answer.
If the topic does not interest you (or you don’t have the time) you can just leave the existing answer with a downvote (and hopefully a comment pointing out its unusefulness).
This should be the same approach if a human had written an answer like that; indeed as you point out here the human claim on this topic has no citations either, so it’s not much better (except to the extent that he may be a recognized expert on the topic and his name alone lends credence to the claim, thougheven experts should not be blindly trusted without evidence).
You generally shouldn’t downvote something without reading it, as you wouldn’t know if it is a good answer or not. If you did read an answer, and it makes a series of unsupported claims, and particularly if you have reason to believe that the claims may have just been invented out of thin air, then you should downvote, or at the very least not upvote.
But why advocate for the site to entertain such posts? We would prefer humans write proper answers. Sure, they don't always do, even if they put thought into it. But why make it worse by inviting robots to write answers we know make no sense because zero thought was put into them to begin with?
If an answer that doesn't support its claims might not be any good, then why encourage answers that can't support their claims at all because they don't even make reasonable claims?
If you want the humans on the site to post better answers, then encourage them to do so rather than inviting a ton more terrible answers saying "well, the answers are already terrible anyway". It really makes no sense.
I don't think we should encourage such answers. I agreed earlier that we should actually discourage them. I just don't think they should be banned and deleted.
I think that goes against the fundamental nature of the site. We don't delete answers for being bad, except to the extent that the badness is that it does not address the question. If an answer is bad we downvote it, comment on its flaws, and/or post a better answer.
We very much delete answers that we know don't adress the question and/or are simply spam.
If the asker wanted a robot to guess words, he'd asked one. He came to SE for having thought and intent put into his question. If that minimum of human decency can't be guaranteed, this site has lost its entire purpose.
And if you don't know anything about Henry Wadsworth Longfellow's poems and don't care enough to go learn about them, then you just have no business answering a question about these. Go look for another one.
I don’t think that is correct. I have asked many questions to ChatGPT, and its texts have meaning, and more often than not it provides useful or correct information. Of course, it does better with certain kinds of topics, but I don’t think one can say that it is inherently impossible for a chat GPT to provide a good answer.
@NapoleonWilson The questioner wants a human to write a good answer as well, but there is no site requirement for that. The very nature of a free and open site carries the inherent possibility that not all answers will be of stellar quality.
@Alex Of course it can guess correctly and it often does. It's not a terrible piece of software engineering. But meaning is only something you can extract out of its word jumble.
@Alex Indeed. But there is a guarantee that someone actually read and thought about your question. If they then go on being wrong about it, tough luck. But if they don't even engage the question sensibly, we delete their ramblings, be they as hiddenly magical revelations as they want.
We describe our site as follows: "Literature Stack Exchange is a question and answer site for scholars and enthusiasts of literature." LLMs are not in the category of intended users.
I am trying to find out about John H. Crabb? Crabb wrote a historical novel entitled In the Crescent's Dark Shadow in 1952.
