Shog9 + Story ID

Shog9

Apr 7, 2017 18:32

Regarding ID questions (@rand @napo et al)... This came up just yesterday in a call, and I was reminded yet again how easily these discussions go south because folks have such a very different experience on SFF

I think the % of ID questions on SFF vs other sites is really striking, and almost certainly explains why SFF-regulars tend to consider them a fun little passtime and fail to understand why they're irritating just about everywhere else. So, I put together a query...

data.stackexchange.com/scifi/query/655052#graph

First line is the % of ID questions asked each month, second is just the ID questions that are not deleted (still as a percentage of all questions asked that month)

Not only have ID questions always been a minority... Nearly all of them are considered acceptable.

Rand al'Thor

@Shog9 SFF isn't the only example. RPG also allows ID questions and seems to have no serious problems with them (cc @BESW)

DForck42

@Shog9 it's returning all 0's for me

Rand al'Thor

@DForck42 Yep, me too.

Shog9

wrong link. Try: data.stackexchange.com/scifi/query/655052#graph

(forgot tags are stored differently in the public data)

@Randal'Thor RPG gets almost no ID questions: data.stackexchange.com/rpg/query/655052#graph

Rand al'Thor

@Shog9 Which proves the point that ID questions can be allowed without either overrunning the site or needing to be heavily policed.

Proves it better than SFF, in fact, where ID is by far the top tag.

Shog9

Apr 7, 2017 18:38

@Randal'Thor proves they can be allowed if they don't overrun the site

DForck42

@Shog9 modified for movies, our tags are slightly different: data.stackexchange.com/movies/query/655057#graph

Shog9

@DForck42 yup. Immediate jump to over 20% by the end of the first year, and just kept on climbing.

Gallifreyan

@DForck42 Oh sweet Morpheus, poor you :'(

Shog9

For a while, half of the questions on the site were ID questions.

Also note the vast difference between "questions asked" and "questions retained"

Rand al'Thor

Yep, that's pretty striking.

DForck42

Apr 7, 2017 18:41

@Randal'Thor it's part of why we've had to be stricter on them, to keep them from consuming the site

Rand al'Thor

@DForck42 Oh, I know. I've spent quite a while in the M&TV Close Votes queue.

But here on Lit, ID makes up 4% of questions asked so far.

Shog9

I mean, one could be forgiven, upon coming to MTV in '15, for thinking ID questions were the primary purpose of the site.

That's where a hell of a lot of the current frustration and bitterness arose

DForck42

@Shog9 yup

Rand al'Thor

And like I said to DForck earlier, Lit is much more comparable to SFF than to M&TV in terms of ID questions.

DForck42

honestly, movie's is STILL healing from that. I wasn't even active when it happened and I still see it

Rand al'Thor

Apr 7, 2017 18:44

Book ID questions tend to be waaaaay better than film ID questions, on average.

SFF's best ID questions are all about written stuff, and the worst tend to be about films or TV.

DForck42

@Randal'Thor maybe

Rand al'Thor

Which explains why M&TV is swamped with shite (if you'll pardon the word) when it comes to ID.

DForck42

@Shog9 is there any way for you to get this data set for the old lit site/ especially for both id and recommendation?

Gallifreyan

@Randal'Thor The worst I've seen so far were about games

Shog9

I suspect... (though of course I cannot prove...) that a lot more people passively watch and half-remember movies while drunk / stoned / kids than pick up books and read a few pages here and there then immediately forget most of what they read.

Rand al'Thor

Apr 7, 2017 18:46

M&TV gets the equivalent of SFF's bad ID questions, whereas Lit is more likely to get the equivalent of the good stuff.

Shog9

@DForck42 data dump should be available

Rand al'Thor

@Shog9 Agreed.

Shog9

(so, YOU can...)

DForck42

@Shog9 true

Rand al'Thor

I also mentioned some theories and anecdotes about why book ID tends to be better than film ID, in my Lit meta post on the issue.

But regardless of the why, it's a fact that there is a difference there.

Hence:

3 mins ago, by Rand al'Thor

And like I said to DForck earlier, Lit is much more comparable to SFF than to M&TV in terms of ID questions.

Shog9

Apr 7, 2017 18:48

For completeness, this is what the graph for Anime & Manga (which completely banned ID-requests last year) looks like:

Rand al'Thor

Sheesh.

DForck42

@Shog9 interesting

Shog9

And they really tried hard to make those work.

Gallifreyan

@Shog9 @DForck42 @NapoleonWilson and whomever it may concern - watch and learn :D

Shog9

They had custom tag pop-ups, meta posts with tools and tips and so on, notices...

DForck42

Apr 7, 2017 18:50

@Gallifreyan lol

JNat

For anyone interested in seeing the plan to get rid of 'em:

22

Q: What we'd like to do about those gosh darn "identification requests" questions

This discussion is now closed. Here's phase 1 of what we've decided to do about identification-requests (deprecating). Here's phase 2 (blacklisting). Here's phase 3 (delete and lock). The topic of id-request is a very subjective one. I believe that it's difficult to provide an object...

Rand al'Thor

@Gallifreyan You don't need to tell those guys about the idea of banning ID questions. They already know.

Gallifreyan

@JNat @DForck42 @NapoleonWilson - read and learn :D

JNat

@Gallifreyan They're aware ;)

Rand al'Thor

(And they might swear at you for mentioning it.)

Gallifreyan

Apr 7, 2017 18:55

@Randal'Thor Sure, but here's an example of how this could be done swiftly

Rand al'Thor

@Shog9 That is significantly worse than M&TV's stats, to be fair.

DForck42

@Randal'Thor so true, that's been a bubbling topic for a while

Shog9

@Randal'Thor worse?

they never actually cleared 50%...

Rand al'Thor

@Shog9 I.e. it looks like A&M's ID questions were even lower quality than M&TV's.

Shog9

@Randal'Thor well, they went and deleted nearly all of them during the cleanup

Rand al'Thor

Apr 7, 2017 19:07

@Shog9 Yeah, but their retention rate looks much lower.

Shog9

59

Q: 2016: a year in closing

A moderator asked me for data on the percentage of questions closed across all sites. This doesn't strike me as information that needs to be restricted to moderators, so for your idle enjoyment I present: Questions asked, closed and reopened for all sites in 2016 Site Name ...

discussion questions closed-questions statistics reopened-questions

remember that? 3K questions closed last year.

Rand al'Thor

Oh, that graph doesn't actually show how many were being closed/deleted at the time?

Shog9

@Randal'Thor yeah, when you delete nearly all questions in a tag it tends to look that way ;-P

@Randal'Thor right

@Randal'Thor here's what it looks like if we consider "not deleted in 30 days" to be retained: data.stackexchange.com/anime/query/655061#graph

Hamlet

@Shog9 my thing about identification questions is that I see this site's purpose as a place where people come to learn more about and discuss literature.

Rand al'Thor

@Shog9 Ah. Well, that's less shocking.

Hamlet

Apr 7, 2017 19:11

And I don't really see story-id questions as doing that: when you answer a story-id question, you identify the book, but you and the readers don't learn anything other than [story x with plot y] exists.

Which is why I think your food metaphor was good (although I enjoy eating radishes, so I might have used the word junk food instead).

Shog9

@Randal'Thor contrast with movies - note the real drop in retention came in as they crossed 30%

Rand al'Thor

@Hamlet And, in the process, learn something new about literature. (Not to mention helping people with an actual practical problem.)

Shog9

@Hamlet I've used a puzzle analogy in the past; some folks buy newspapers for the crossword, but most newspapers don't exist to host the crossword.

Rand al'Thor

Not every question here has to be some deep academic-level analysis question.

Hamlet

@Randal'Thor I agree with that

Rand al'Thor

Apr 7, 2017 19:14

Mar 29 at 22:48, by Rand al'Thor

@Hamlet More importantly, I wish you wouldn't be such a spoilsport. If you don't feel a question is interesting to you personally, downvote it; but leaving comments on such questions to imply that they're worthless because "what do they tell you about the book" willl discourage people from asking questions and ultimately could kill the site. It's a literature site, not a literary analysis site, and this is a question about literature which at least five people find interesting/useful. What's your problem? — Rand al'Thor 3 mins ago

Shog9

(but I kinda wonder if newspapers are too much of an anachronism these days)

Hamlet

@Randal'Thor not many of the questions on this site fall under that category anyway, I would be dumb if I came here expecting otherwise.

Shog9

Metaphors aside... I stand by what I wrote back when the site was still in private beta: this isn't something y'all should worry about unless they become exceedingly common - if that happens, you need to treat them as a serious problem.

2

Hamlet

But story-id questions fail the basic test of a good question: is it helpful to future readers? can you learn something from it?

Shog9

@Hamlet that's one of those tests that's a lot harder to apply than it seems like it should be

It's why we ended up dropping "Too Localized" as a close reason

Gallifreyan

Apr 7, 2017 19:18

@Hamlet I've personally seen some comments like "OMG I've been looking for this book for ages thankyou somuch cheers hugs"

Shog9

You'd get folks saying, "Drawing an ASCII christmas tree? That's not gonna help anyone, ever". Turns out, it's a common assignment and LOTS of folks have questions about it.

Gallifreyan

@Hamlet And there's probably more, but we don't see it since non-users and users with <50 rep can't post thank you comments

@Shog9 VTC, homework question :D

Shog9

@Gallifreyan :shrug: that's tomorrow's userbase you're throwing under the bus; if it's a reasonable question, let it be

...which hits on the real problem here: these still have to be good questions.

Something I've seen a LOT of on MTV are ID questions where there's no way to know if any answer is correct!

2

Gallifreyan

@Shog9 Oh come on, I had even put a smiley there :) We even allow homework questions here, though within reasonable standards

Shog9

@Gallifreyan I took it seriously 'cause I wanted to build on it... ;-)

Bad ID question: there aren't enough details to narrow it down. The asker doesn't remember enough to confirm or reject any answer. The description and title are broad enough that, yeah, someone is probably gonna find it on Google, but it's a crapshoot as to whether the answer will be what they're looking for.

That's not just a problem for the site - that's actively making the Internet worse, noisier, harder to search.

Rand al'Thor

Apr 7, 2017 19:22

@Hamlet As shown in my answer on meta and what I said earlier in chat, yes.

Gallifreyan

@Randal'Thor @Hamlet my first 3 [surviving] questions are story-ID, and I'm still here

I think it's about the asker, not the question. Reflects the level of self-discipline of an Internet user.

Hamlet

@Randal'Thor at the end of the day, this is just my opinion. I can't force through any policy.

Shog9

@Gallifreyan this is also something worth monitoring. To go back to Anime, my original guidance for them was to try & use ID questions as a hook to get folks interested in the site. Eventually, it became clear (JNat ran numbers on it) that the vast, vast majority of askers never returned or did anything else contructive on the site - so nearly half of the site's effort was going toward folks who gave nothing back.

That doesn't have to be how it plays out, and hopefully it won't here. But again... Keep an eye on it.

What's the point of having all these sites if you can't learn the pitfalls & watch for them on new ones?

Napoleon Wilson

@Gallifreyan Learn what? Please don't think we've not been over this a thousand times already.

@Gallifreyan Ditto.

34 mins ago, by JNat

@Gallifreyan They're aware ;)

Gallifreyan

@NapoleonWilson I know you're aware, I've glanced at your meta. I'm just saying it's entirely justified to deploy radical solutions in this case.

Napoleon Wilson

Apr 7, 2017 19:30

@Randal'Thor Not necessarily. The incident where I did swear when you mentioned it recently had quite a different reason, and you're pretty much aware of that.

Shog9

The critical (admirable!) factor in Anime's case is that they didn't take radical action until they'd methodically tried everything else. They gave 'em a fair shot, put a ton of work into moderation, and when it didn't help... Then they shut it down.

Rand al'Thor

@NapoleonWilson I don't recall the exact context. But as mentioned earlier, I'm not quite as strongly pro-ID as you seem to think :-)

Last time the question of outright banning ID came up on M&TV, I was against it. Now ... well, I'd probably still be against it, but I'd be more ambivalent at least.

Shog9

MTV has had two complementary problems:
1. They pushed for banning a little bit too quickly, and...
2. ...they had a bunch of folks from a site where ID questions *aren't a problem and have never been a problem* jumping in with "helpful" advice at the worst possible moment.

I am optimistic that they can mitigate both issues.

Napoleon Wilson

@Gallifreyan Oh, we also know that. We know that we approached this problem way less proactively than we should have and could have.

@Randal'Thor That might be true, it wasn't about that either, though. But assuming good intentions it seems you really didn't know what was so problematic about that earlier comment and I'm rather letting it rest now.

@Shog9 That's encouraging to hear. I try to be optimistic too.

Rand al'Thor

@Shog9 And over here we're having a slight case of the opposite: a bunch of folks from a site where ID questions are a major problem advising that they be banned here without giving full consideration to the differences between the sites.

Not that all the anti-ID people here are from M&TV, but a fair few of them are.

Catija

Apr 7, 2017 19:43

@Randal'Thor I haven't been around much recently to say anything... but I think my (our) measured approach on A&C was a good place to start... I'm pretty vocal about disliking ID on M&TV but I also know that, in some cases (on some sites), they're totally fine.

Rand al'Thor

@Catija What approach did you end up taking on A&C? I remember there was a meta post where you and I both posted answers, but I don't recall what the conclusion was.

Catija

@Randal'Thor People seemed to like my answer - granted, we haven't had it tested too much. We're not very high-traffic.

But we're talking about physical things... it's easy to require something like a photo.

7

Q: What guidelines should we set for "identification"-style questions?

I know that ID questions are very contentious on different sites - some sites love them and others hate them. I believe that they can be successful here if we set some strict ground rules to make the questions less of a "guessing game" and more of an actual, answerable question. So, to help the...

discussion on-topic

DForck42

@Randal'Thor :-D

b_jonas

Apr 7, 2017 22:10

@Randal'Thor Are the rest of them from Arqade or Anime?

@Shog9 And there are a lot of interesting ways to go about them, such as perlmonks.com/?node_id=734447

Rand al'Thor

@b_jonas No, Hamlet is one of them.

user15026

FWIW, I don't hate ID questions. They didn't work on Arqade for a variety of reasons, but I can see the value in trying them here.

b_jonas

@Randal'Thor This, yes. At least, the good questions will be in a higher ratio than on Sci Fi. Three quarters of the ID questions on Sci Fi are bad.

@Hamlet I've seen many comments on Sci Fi about what people benefited from ID questions. The last time was when I managed to recommend Hal Clement's book to someone and he read it and enjoyed it: scifi.stackexchange.com/questions/96700/…

@Shog9 By that measure, my post history.stackexchange.com/q/35918/24029 is a bad question too, because I don't remember enough to be able to confirm any answer. But I did learn from the answers anyway.

Shog9

@Randal'Thor this is why it's critical to remind folks to look into the why of their opinions. Opinions are useless; the "why" is useful.

2

@b_jonas I don't remember anything about WinINET, but I've still written useful answers about it. Memory is something to be used and discarded; it is enough that I once knew, when I needed to know.

Catija

@b_jonas That really depends on who you ask...

Shog9

Apr 7, 2017 22:24

You can't leave your memories at the door even if you wanted to, @Stijn. Instead, try to remember why we do things the way we do them elsewhere... And if you don't know, find out before you blindly repeat the rituals here. Tool recommendations are often useful questions, but they still need to focus on an actual problem and the answers need to solve that problem. The issue we were unable to overcome on SO was a long history of folks asking for the "best" library, with aging answers and circling wakes of spammers trying to feed off of the rotting corpses; that doesn't have to happen. — Shog9 ♦ Feb 28 at 23:27

different boogieman, still useful strategy

user15026

I mean yeah, if someone goes "I remember this book, it was blue", thats a terrible ID question.

b_jonas

I personally think we should allow ID questions on Lit and Sci Fi, but we have to continuously keep a lookout at their quality, and if we start to see that they're hurting the site, then we should consider what measures we can take, and eventually possibly even ban them if that's the best solution.

user15026

I figure Lit's a young site, no use throwing the baby out with the bathwater. We're trying a bunch of things here already like allowing song lyrics as literature, I figure ID questions fall under the same sort of "let's try it and see". For all we know, we're going to get awesome detailed stuff out of it.

Rand al'Thor

@Shog9 Absolutely. The advice about explaining why and how in the GSBS blog post and the associated help centre page can apply equally well to meta discussions as main-site posts.

b_jonas

@Shog9 Interesting. Hey, do you happen to know a library for multi-dimensional numeric root finding and optimization that's under a non-copyleft free software license?

user15026

Apr 7, 2017 22:27

We've not really even been around long enough for the site to find it's legs.

Shog9

user15026

(It helps that a lot of story-id here is really unlikely to be "what's the best book for this", and more "I remember a story with the following details")

Catija

By the way, @Shog9 I don't know that it's fixing everything but I think that nudge to the ID question box you made is helping... the 2 day close ratio is only 25% down from the 90 day percentage of 38%... it will be interesting to see when we get to a full week.

Rand al'Thor

@b_jonas A good balanced approach :-)

Shog9

@Catija yeah... Let's keep watching.

I was kinda shocked just how strong that heuristic was for past questions

b_jonas

Apr 7, 2017 22:29

@Shog9 I should look at whether scicomp SE takes them first

Shog9

probably, yeah

there's gotta be a Python library

Rand al'Thor

@Ash It's much harder to bite my tongue about grocers' apostrophes when we're on a literature site :-P

The Reading Room

Participants