The Assembly

user2334

1:22 AM

Skeptics election is now underway

awoodland

5:01 PM

Thanks @ whoever handled the related series of large scale spam flags I raised yesterday

user2334

8:32 PM

Holy carp

user2334

Astronomy, Economics, Literature, Firearms, Healthcare IT, and Theoretical Physics getting shut down

4

Gilles

8:44 PM

@MarkTrapp and without warning that I could see

user2334

Well, the writing was on the wall for a long time

user2334

I have been saying SE needs to be more forthright with this stuff though

2

Shog9

9:00 PM

@MarkTrapp Tough call... Posting, "Your site looks bad and we might have to shut it down" can be sorta... demotivating. Also covers like 20% of the sites, any one of which could turn it around. Mods have access to a ton of stats, and there's this Area51 wall where if there's a lot of "Needs work" you should at least... y'know... work on it.

Gilles

@Shog9 many sites have “needs work” in terms of traffic pretty much right up to the point they graduate

@Shog9 “your site looks bad and we're shutting it down now” is a lot more demotivating

A lot of good contributors won't be back for a second round

Shog9

@Gilles At that point, you kinda want folks to stop posting.

@Gilles Quite possibly. Which is why we're not shutting down 16 sites.

Gilles

“Your site looks bad and here's what you should do, if you don't we might have to shut it down” would be a lot better.

I would be angry if you shut down French Language and Usage without warning, after all the energy I've put into it. I understand that others would feel the same way about other sites.

Shog9

@Gilles Well, that's what I'm gonna be trying on a few of the other ones that were in the original list. Without the "or we'll shut you down" bit.

@Gilles Stats on FLU are decent at present. Biggest potential issue is the low number of questions per day, and even that isn't abysmal. Home page shows just over two weeks worth of activity. If we're into August and it still looks like that, we might be a bit concerned, but hopefully it will continue to grow.

user2334

10:20 PM

@Shog9 The problem I have with the whole process is people get linked to the same 2 blog posts from ages ago whenever they ask for feedback up until the point they graduate or get shut down.

user2334

If SE could check in once a month or even once every two months with a meta post saying "this is where your site is at, here's what we'd like you see work on", I dunno: I feel like it'd be a lot less jarring

user2334

You guys did that once for Healthcare IT when you couldn't find any moderators and it lit a fire under them for a bit

Shog9

@MarkTrapp Well, that's what we're working on doing with quality evals, which should start showing up with more regularity as time progresses.

@MarkTrapp Yeah... I wouldn't exactly classify that as a success story

user2334

@Shog9 I still don't understand the point of them. Some of them have SE's thoughts on them, others don't.

user2334

I'm suggesting SE needs to be direct about what it thinks of the sites

user2334

10:29 PM

You guys didn't wake up this morning and decide Literature needs to shut down, for example

Shog9

@MarkTrapp There's been a lot of experimentation. I'm a huge fan of including my own evaluations within them, certain other team members are... less enthused.

user2334

@Shog9 Fair enough. But even with the feedback, they're a check for quality: sites that have quality issues tend to be shut down in private beta. Public beta sites going back to Atheism and Gadgets get shut down for lack of content and traffic. But it's hard to tell what SE considers acceptable in those metrics

user2334

Like take BH: it's doing worse than half the sites that shut down today

user2334

Or Poker for that matter

Shog9

10:45 PM

Poker's too young yet, but it doesn't look good.

BH is an interesting case - I have a meta post planned there too, as they were on the list at one point.

There are a lot of sites that are not doing particularly well. Our goal here was to shut down the ones we saw as clearly having no chance. As @Gilles notes, there's a huge risk of just driving people away when doing this - waiting until they've burned out trying to make it work and left in disgust doesn't improve upon that. Not a huge lot of SE 1.0 site owners active in SE 2.0...

@MarkTrapp We're looking very hard at abandoning, or at least making some fairly substantial changes to the current lifecycle. Part of this will absolutely involve some better metrics for "progress" - the Area51 stats are the best we have right now, but there's no clear division between a site that's doing poorly and a site that's all but dead.

For one thing, there are a bunch of stats that aren't exposed, or are exposed in a really hard-to-read fashion - just try getting per-question participation - a hugely important aspect - out of these sites anywhere.

user2334

Indeed. One thing I see even on launched sites like Programmers and Gaming (but also on sites like BH, Christianity, and the Workplace) is that people focus on traffic and question closures and a lot of metrics that don't seem to be at all relevant to a site's success

Shog9

Quality evals are the other part of this. One thing you'll see (if you look very closely) on these sites is that down-voting is... rare. If you're only getting one answer per question, and no one ever points out wrong answers, you get a site that looks (on A51 at least) a lot better than it really is.

user2334

And I think a large part of that is due to people guessing at what counts rather than being told, "hey, this is what we're looking at. Focus on this instead of those red herrings". BH not closing is indicative or something other than traffic saving it (question quality perhaps?)

Shog9

@MarkTrapp Short answer for BH is that the single most troubling metric there was time-to-answer, which given the subject matter I don't actually consider a bad thing. FGITWing something like that (or Linguistics, which fell into the same bucket) is probably more harmful than good.

Gilles

@MarkTrapp where “guessing what counts” usually (I've seen it on a lot of beta) means looking at the Area 51 stats and concentrating on making everything green. @Shog9 The Area 51 stats may be doing more harm than good if they're making people focus on the wrong things

Shog9

10:57 PM

Which illustrates one of the problems with just making all of these stats public - it's tempting to optimize for the wrong thing. The A51 stat is "questions per day", but even that is troublesome, since what's really important there is that the folks who're showing up on the site to answer have something useful to do.

Gilles

@Shog9 interesting, this is the first time I see this metric mentioned

Shog9

@Gilles I'm pretty sure that's exposed somewhere, but I can't recall where

user2334

@Shog9 Right, and you see people suggesting the way you get more questions per day is to allow things like lists or fun questions or whatever

user2334

Or my favorite "don't close as much"

Gilles

@Shog9 yup, that's something that worries me on French Language and Usage sometimes. We have a bunch of highly motivated users who keep fixing typos, seemingly just to have something to do, but I'm afraid we lose the “middle people”

Are there publicly accessible or mod-accessible tools to evaluate user retention?

Shog9

10:59 PM

@MarkTrapp Exactly. The root problem with most of these is simply that we launched sites that don't have enough of an overlap with the existing community on SE, and haven't been able to attract many outsiders. Everything else is just symptoms of that.

Gilles

@Shog9 I don't claim that it's never been said, but it's certainly not a recurring theme, I expect it's never appeared on most metas

Shog9

@Gilles I don't believe so. You can sorta approximate it by looking at the rep leagues, but the numbers on MIA users are currently employee-only.

That might be a good one to expose further.

@Gilles It ties into FGITW, which is usually a good thing - but then you have this huge difference between the audience and subject matter on sites like SO or Gaming and CSTheory or BH

Gilles

@Shog9 well, I think FGITW is a bad thing on SO

Shog9

If gaming looked like either of the latter, it'd be dead. But I could easily see sites like CST, Ling and BH becoming something more akin to a public mailing list, where you might only get a question a day forever, and that's fine because that's still a significant part of the total discussion.

@Gilles Heh... Well, it's one of the things that makes SO appealing to a lot of users. But it's only valuable when you really do need an answer RIGHT NOW and someone is just sitting on it anyway.

Gilles

@Shog9 interesting, that's the model I tend to push on French. We do have a culture of rejecting dictionary questions and justifying answers with references, not systematic or by policy Skeptics-style, but enough to set us apart from most language forums out there

@Shog9 I guess that because my interests are not the high-rate tags, I get the downsides of both: FGITW pushes a culture of lowest-common-denominator answers and draws attention away, but when there are only a handful of people on SO who might answer a question, I'm not going to get an answer in 5 minutes anyway

Shog9

11:09 PM

@Gilles Yeah, I think I mentioned this somewhere on CS during the private beta, and the response was "no duh" from the CST expats - but that's a pretty huge departure from the SO model.

Gilles

@Shog9 “this” being? Our streams are crossing

Shog9

@Gilles meh. There'll always be this ocean of questions that really only need 5 minutes of work or less to answer. The bit so many people forget is that on a site the size of SO, you can provide a comprehensive answer to some obscure question and eventually it still ends up getting quite a bit of exposure.

Gilles

What's the status of tag wildcard search?

Shog9

?

Gilles

this is status-completed, but it's so old it has whiskers

Let's say I want to search for all the questions with a tag that begins with star-trek on Science Fiction and Fantasy

scifi.stackexchange.com/questions/tagged/star-trek%2a

How do I get to this URL from the UI?

Shog9

11:16 PM

@Gilles Hmm... I expected [star-trek*] to work, but that fails and produces [star-trek] only.

Passed this on to the devs. Not sure when that broke; I was just playing with that recently, but... Editing the URL directly

Rebecca Chernoff

scifi.stackexchange.com/questions/tagged/stargate-~

Gilles

@RebeccaChernoff oh, right, but the question was how to reach it from the UI

@Shog9 thanks!

Shog9

11:37 PM

FYI:

I agree - the current system for tracking progress and providing feedback is broken; the Area 51 page tries to give you an overview of where you stand, but completely fails at indicating progress (or a lack thereof). I don't want to get into the habit of nagging sites that are doing their best, but some sort of regular status update is something we owe you - this is as much of a wake-up call for us as anything. So lemme just say again, that I do - we all do - greatly appreciate the time and effort you've put in, and stay tuned... — Shog9 1 min ago

Transcript for