« first day (4558 days earlier)      last day (321 days later) » 

nobody
nobody
7:17 AM
39
Q: GPT on the platform: Data, actions, and outcomes

PhilippeIn a meeting with some moderators last week, I committed to releasing the data sets from our initial studies around the efficacy and false positive rates of ChatGPT detectors to them. Tuesday afternoon, we did so. This post contains as much information from that discussion as we are able to share...

discussion chatgpt community-data
How to not do data analysis
 
 
7 hours later…
Fire Quacker
Fire Quacker
2:07 PM
@nobody Woulda been nice if they'd posted that in the first place instead of making changes and explaining later
Now that people are pointing out mistakes in their reasoning, they have the unenviable task of either admitting they were wrong or trying to cover it all up/doubling down
Things would have gone so much better if they only started a discussion about their concerns and then asked the community for ideas
But I suspect the real problem is that they have declining traffic, leading to declining ad revenue, and they don't want to talk about that in public
 
CaffeineAddiction
CaffeineAddiction
2:48 PM
eh, I mean I kinda predicted some of those outcomes and the stats they are listing seem to back my theories.

IMO, there are 2 types of people who answer questions
1) People who know a lot and feel like sharing with the masses
2) People who asked a question, got it answered and feel like they should give a bit back to the community that helped them.

People in the first group are prob feeling threatened by GPT in the sense of ... if GPT can give an answer indistinguishable from theirs with out deep knowledge of the subject and investigation ... why should they bother.
IMO, long term se's days are numbered. I also thing that fighting against GPT is like fighting against people who use google as a tool to do research for their answer ...

I think ultimately a different tact should be taken. I think instead of punishing answers for giving good answers regardless of the source ... answers should be nudged in the direction of citing their sources ... even if the source is gpt.

I think the best outcome in change of rules reguarding GPT would be that if suspected a mod could ask a user to quote the section given by GPT so readers of the answer could have the…
@RoryAlsop @schroeder, as mods what is your feeling ... if I gave an answer on sec.se and quoted a section of my answer as clearly coming from GPT ... would I be facing a 3 week ban?
 
CaffeineAddiction
CaffeineAddiction
3:18 PM
Their is also another benefit of citing your source as being from a GPT in that this aids in preventing false trails of self referential "knowledge" (fake news) through time. Eg, if future GPTs are trained with knowledge for SE it would be good for them to be able to include or exclude data that was generated by another GPT based upon the communities reaction to it.

If an answer has a cited GPT answer and comments on the answer say that its wrong that is useful to the community even though the answer isnt.
 
 
3 hours later…
nobody
nobody
6:16 PM
@CaffeineAddiction I think you're missing the point. If GPT answers are allowed, lazy people will fill up Stack Overflow with completely unverified answers, and there won't be enough eyes available to review them properly.
And that defeats the entire reason SO was successful: A relatively high signal to noise ratio.
 
CaffeineAddiction
CaffeineAddiction
I disagree, same could be said of google ... ever heard of the term "Its on the internet thus it must be true"? I remember teachers I had in grade school that said any source that came from the internet was totally 100% unreliable ... can you imagine a week at work with out google today?
views change over time ... what im saying is if you cite your source ... it shouldnt matter where the answer came from ... if its garbage it will be voted as garbage ... if its useful it will be voted as such
 
nobody
nobody
@CaffeineAddiction No, but only because I use google to find SO answers :)
@CaffeineAddiction Right, but if the only thing on I get on SO is GPT generated answers, I might as well as just go to GPT
 
CaffeineAddiction
CaffeineAddiction
I mean, honestly ... prefixing my google questions w/ "reddit" usually gets me a better solution these days depending on what I am looking for.
@nobody A large portion of people are already there ... "If GPT gives me an equiv answer to SO ... why bother w/ the semi-toxic userbase?"
 
nobody
nobody
@CaffeineAddiction yeah, people will move because dealing with an llm is better than dealing with people
Stack overflow will only remain to answer questions gpt cannot
 
CaffeineAddiction
CaffeineAddiction
problem is ... llm are being trained off things like SO ... so if SO goes away you just killed the golden goose
OpenAI found that they could use LLM to generate training data for learning language ... but I doubt that applies to technical knowledge
Github doesnt have this problem, and LLMs will be able to continue to use that as a source ... but that is not a long term technical solution for non-code issues
 
nobody
nobody
6:30 PM
@CaffeineAddiction right, and putting gpt answers on SO will kill it completely
 
CaffeineAddiction
CaffeineAddiction
see thats where we have a difference of opinion
 
nobody
nobody
You get the worst of both worlds. Hallucinations and you cannot even ask it to fix them
Plus the regulars will likely leave
 
CaffeineAddiction
CaffeineAddiction
GPT answers are in most cases are the equiv of closing a question a dupe and pointing to another question
cause GPT was trained on the other question ... and thus is largly regurgitating the answer to that w/ maybe some added info from the rest of the internet at large (best case) or worst case (total giberish that it made up)
in the case of the best case ... imo, its a better user outcome than closing a question and pointing to someone elses question

in the worst case ... its no worse than a bad answer which will be voted down into oblivion
both case help the community and GPT
 
nobody
nobody
@CaffeineAddiction Yeah, except they ruin the search experience. Google something and the front page is filled with basic questions being answered dozens of times
@CaffeineAddiction You assume their will be people around to downvote
 
CaffeineAddiction
CaffeineAddiction
so then fix google SEO ... thats not a GPT issue
 
nobody
nobody
6:36 PM
And how do you do that?
 
CaffeineAddiction
CaffeineAddiction
you assume that this problem will magically go away

this is the equiv of my grandpa wishing computers would go away so life could be like the good old days.

Even if SO dies another question/answer site will rise from the ashes ... and it will face the same GPT issue
@nobody hell if I know ... post the question and I will give a blately wrong answer ... surly someone will jump at the chance to correct me ;)
lol, i forgot a type of user:
3) There is someone on the internet ... and they are WRONG!!!! I must correct this!
 
nobody
nobody
Lol
@CaffeineAddiction We're trying out best to keep it in check :)
Also, I doubt another QA site will rise
Probably just going to tirn into asking humans for help in smaller communities
Discord servers and the.like
Fuck, the mobile chat is horrible
 
CaffeineAddiction
CaffeineAddiction
@nobody so like BBC and IRC of the past? ;P
 
nobody
nobody
Yeah, but of the future :D
 
CaffeineAddiction
CaffeineAddiction
“The Wheel of Time turns, and Ages come and pass, leaving memories that become legend. Legend fades to myth, and even myth is long forgotten when the Age that gave it birth comes again.”
 
nobody
nobody
6:52 PM
@CaffeineAddiction didn't know there was a lord of the rings remake lol
 
CaffeineAddiction
CaffeineAddiction
I would hardly call "The Weel of Time" a lord of the rings re-make ... but meh
Lord of the Rings -> 576,459 words
The Wheel of Time -> 4,410,036 words
The Wandering Inn -> 40-60 thousand words per week
all totally different animals
 
nobody
nobody
Oh
It looked like one from Wikipedia
 
 
1 hour later…
Fire Quacker
Fire Quacker
8:14 PM
Obscure CAs do some of the weirdest things
 
nobody
nobody
Why do we even have obscure CAs???
Since the system is built on trust, shouldn't there be just a handful of well-known CAs that can be audited more frequently?
 
Fire Quacker
Fire Quacker
8:43 PM
Yep
-7
A: June 2023 Data Dump is missing

Jody BaileyStack Overflow senior leadership is working on a strategy to protect Stack Overflow data from being misused by companies building LLMs. While working on this strategy, we decided to stop the dump until we could put guardrails in place. We are working on setting up the infrastructure to do this co...

In case anybody missed it
> Hey, welcome to SO! You're new here, but someone should have told you: traditionally we used the Reddit dumpster fires as examples of what not to do. – Shog9 30 mins ago
 
CaffeineAddiction
CaffeineAddiction
LOL, this feels like a News Paper company trying to stay relivant
tbh, their best move atm ... is sell SO to one of the companies doing LLMs
because their revenue stream is about to vastly change from one of advertisement and being mad at people w/ adblockers to one of selling data to companies training LLMs
 
Fire Quacker
Fire Quacker
From the CEO's blog posts, it's possible that SO is trying to make their own LLM, but it's hard to tell from all the vague buzzword-speak
 
CaffeineAddiction
CaffeineAddiction
Reddit sees the writing on the wall already and thats why their api is so expensive ... its because SOMEONE is going to pay it
and its not going to be the users ... and its such a big payday they dont even care about screwing over the third party apps the users are using
which is very short-sided ... but it must be one hell of a payday
with all the people subbed to ChatGPT4 for 20$ a month ... im very surprised OpenAI doesnt just buy Reddit outright
prob dont want to manage the dumpsterfire and would rather pay for the privilege of not having too
 
Fire Quacker
Fire Quacker
Is $20/month enough to offset server costs? I'm sure they have a ton of GPUs running overtime to keep up with the demand
Plus all the time spent in development
 
CaffeineAddiction
CaffeineAddiction
im sure 20$ per month has PLENTY of padding in it for all types of things ...
 
Fire Quacker
Fire Quacker
8:55 PM
But I'm sure it's not hard to get investors, so maybe they could have cash on hand to buy something if they wanted to
 
CaffeineAddiction
CaffeineAddiction
again, the GPUs are used to train it ... once its trained it doesnt take that much to run it
same reason once you train a face recognition machine learning code you can put it on a arduino and it will work just fine
 
Fire Quacker
Fire Quacker
@CaffeineAddiction Moderation is... harder than people think. OpenAI has to deal with that enough with people using DAN to get it to say crazy stuff
 
CaffeineAddiction
CaffeineAddiction
learning is the expensive part, using it once the learning is done is trivial in comparison
@FireQuacker True, but they can use another LLM to filter out the majority of it ... and then only have to worry about the new stuff and then train the new stuff in.
 
Fire Quacker
Fire Quacker
Okay. I assumed that all AI stuff was GPU expensive, because it was rough on my computer when I ran StableDiffusion
 
CaffeineAddiction
CaffeineAddiction
thats a whole diff ballgame, but yes
image generation has historically always been computationally expensive
 
Fire Quacker
Fire Quacker
8:59 PM
Yeah, that makes sense
 
CaffeineAddiction
CaffeineAddiction
but that being said, check out midjourny ... they are cranking out TONS of user generated content ... and their sub is comparable to OpenAIs
and im sure both companies have TONS of padding in the sub
 

« first day (4558 days earlier)      last day (321 days later) »