Stats based thoughts on hot questions

user41796

Feb 14, 2014 16:57

I have been reading over the stats and whatnot from Shog on the hotness questions. Mighty cool that he gathered all that info for you.

user55340

I've got an answer to that M.P.SE bit about why the question was deleted in the editor locally that I'll finish up at some point.

user55340

@GlenH7 Kind of wish that he also got the top answer too... though I didn't ask for that initially.

user41796

I'm trying to take a step back and think about the ramifications of that data

user41796

In most cases, there is a high correlation between people who up vote and people who have already answered at least one question on the site

user41796

To me, that implies folk who were previously active or are currently active on the site and got drawn in to that question for whatever reason

user41796

Feb 14, 2014 17:02

Using me as an example - I normally watch Progs, but I track Lego and a few others. So I would fit that pattern if I went over to a hot Q at Lego and up voted it. In other words, I'm a semi-casual user of the site in question. I have an interest, but it's not a regular active interest.

user41796

Does that conjecture seem to hold up?

user55340

Not sure... (and I really wish there was a graph of "days active out of last 100" rather than just a "previous"). I think days active is sufficiently anonymous.

user41796

@MichaelT yeah, that's a hole in my logic.

user55340

I'll also point out that this is a shotgun of trying to find which data might be useful and have further drill downs into it.

user41796

Engineering is the art of making decisions based upon the data you have available and accepting that you may either a) need to request more data when you discover the need or b) end up with incorrect conclusions from lack of data. Or c) both. :-D

user41796

Feb 14, 2014 17:06

But it's important to call out the assumptions

user55340

I don't know of any controversial questions that get absurd hot attention.

user41796

It would be interesting to know what the activity level of hot question visitors were after seeing the hot question.

user41796

Back to hypothetical me - I see a Lego hot question, go in and up vote or whatever. So what happens to my activity on Lego after that question?

user41796

If I never go back until the next hot question, there wasn't any stickiness

user55340

I'd use that psudo-random one and maybe another one as tests to get data on that maybe in a month or so.

user41796

Feb 14, 2014 17:09

But if I go back and answer or up vote other things, there's some stickiness

user41796

And I'll conject that voting is almost as valuable as providing answers.

user41796

A site dies if the silent readers aren't voting.
Likewise, a site will die if people don't answer.

user55340

"how many votes cast in the past month for users that had cast a vote on the question" and "how many days active in the past month for users that had cast a vote on the question"

user41796

If possible, I would broaden that to "how many returned and did something elsewhere on the site regardless of any activity on the hot question"

user55340

(the pseudo random one on SU is on the second page of all time views there)

user41796

Feb 14, 2014 17:11

in other words, I may see the question, deem it has enough votes and not do anything. But if I come back, there was stickiness.

user55340

activity a month prior and after the vote.

user41796

And I'm modifying my earlier conjecture - voting is important. Voting and answering is more important.

user55340

Or do 100 days... I like 100, it makes for nice numbers.

user41796

or even 1 month and then 3 months +/- of the hot question.

user41796

3 months == 100 days in my book. :-)

user55340

Feb 14, 2014 17:12

What is the delta of votes for N days prior and after.

user55340

If someone cast 5 votes in the 100 before and 50 votes after - you've got a +45.

user41796

Those sorts of questions would really determine the value of hot questions

user41796

@MichaelT and I'm willing to call that a good thing. It's more actively participating in the community.

user55340

A high number of high +s would show "people go there and participate" while low numbers would indicate that people look at it like buzzfeed.

user55340

Certainlly. I'm of the opinipn that the hot questions is more buzzfeed like and doens't actually promote the community.

user41796

Feb 14, 2014 17:14

Agreed - and we don't want / need buzzfeed.

user41796

@MichaelT That's the working hypothesis. But we don't have stats (yet) to prove that.

Jimmy Hoffa

@aliteralmind alright I can't keep my eyes straight to wander through that code - but I wouldn't approach it the way you appear to suggest with that fieldable interface and such. I know a very simple generalizable approach to extending classes that use a creational pattern I've typically relied on when needing to extend them. When you pop around (and I have time) I can tell you about it. Hint: FP. (to those here who are surprised - I KNOW RIGHT? SHOCKER!)

user55340

( @aliteralmind remember what I said about it being more active during the North American work day? )

user41796

@JimmyHoffa ugh. Warn us when you do so we can temporarily mute you. :-D

user55340

@GlenH7 monad, hypomrophic polyzogomorphism.

user41796

Feb 14, 2014 17:16

Ponderings #2 re hot questions....

user41796

I think we need stats to back up further action, but it seems increasingly unlikely that significant changes can or will be made to the existing formula.

user41796

so the question becomes, how do we more effectively deal with those as a community

user41796

One challenge is that they frequently happen late in the UTC day when most of us are out of close votes. So we can't shut things down in a timely fashion

user41796

Asking for more close votes for 10k users seems a reasonable request.

user41796

Another way we can affect things is with more 20k voters

user41796

Feb 14, 2014 17:19

Yes, we're kind of on our way to that one, but we need more.

Jimmy Hoffa

@MichaelT In Haskell builders are usually done using the applicative interface, works great and allows a lot nicer decorations and annotations on the creational approach. But that's not how I extend classes using creational patterns in C# - I've actually done this a hand full of times in C#. Couldn't use the applicative approach nicely in C# or Java because they lack the syntax for the necessary functional indirection to be done without looking like gobbledygook

user41796

it's easy enough to build up a site culture to push crap answers down to -3. 20k voters can then move to VTD and get rid of them.

user41796

removing answers from a hot question is the #1 most effective way to throttle the collider score it earns

user55340

They are also < 2 days old, so 15k+ can't protect.

user41796

@MichaelT That's another good one to lobby for

user55340

Feb 14, 2014 17:21

There's an auto-protect MSO question out there already.

user41796

once it hits the collider, a flag needs to be set so 15k+ can protect

Jimmy Hoffa

Haskell let's you rely on partial application/currying that your type constructors already have so builder is sort of built into the language.

user55340

If hotness > N, autoprotect.

user41796

politically, I think the auto-protect is a non-starter request. SE proper won't hear it.

user41796

but asking to give the community the tools to judge for themselves about protecting the question may fly

user55340

Feb 14, 2014 17:22

auto-protect helps save the people who do and get annoyed messagings and 'evil mods won't let me post crap answers' type thing.

user41796

in other words, just because I can protect a hot question doesn't mean I have to protect it. We understand SE proper's concerns, so we can let it ride a bit.

Jimmy Hoffa

@Ampt and yes you saw the fluent interface and thought "oo, Monad?" Good intuition. Not necessarily accurate but similar. You've learned something in here to have thought that - muaha I have infected your mind, you can't unlearn that and you will go take it with you into the industry! The industry will bow to Haskell if I keep this up!

user55340

2

A: Better criteria for the hot questions list

Executive Summary All SE sites are currently far smaller than Stack Overflow. The amount of eyes the Hot Questions can bring to a smaller or younger site can overwhelm budding community moderation. Hopefully any new system will be sure to minimize harm to the communities these questions were (I ...

Jimmy Hoffa

Or...the industry won't, but it's a nice thought.

user41796

@MichaelT my skin is fairly thick from that point of view. :-)

user41796

Feb 14, 2014 17:23

And my primary concern would be to prevent it from hitting CW status from lemming answers

user55340

24

A: The Anatomy of a Hot Question

As a mod on UX.stackexchange (a graduated site) we get the same issue. Something hits the HQ top 3 and we get loads of comments, 'answers' and all sorts and invariably the solution is to delete a bunch of posts and protect the question. So my suggestion is — when a question hits the top 3 of the...

user41796

Likewise, with more 20k voters, we can remove the -3 answers and also keep it out of CW land.

user55340

@JimmyHoffa It would be interesting.

user41796

@JimmyHoffa - conversations in that vein never hurt.

user41796

@gnat - thoughts on our conversation about hot questions?

user41796

Feb 14, 2014 17:25

As the obvious next request is to suggest you DV a little less and get yourself up over 20k

user41796

more close votes
more 20k users
ability for 15k+ users to protect hot questions

user41796

And we may have the tools we need to throttle hot questions here a little better

user41796

Smaller sites will still be hard up, but I can't save everyone. ;-)

Jimmy Hoffa

@MichaelT Very. The biggest piece would be not knowing the Java libraries at all.

user41796

And we need stats to guide our behavior. If the hot questions really do improve community involvement, then we need to work with a light hand. But if they are just buzzfeed (likely outcome) then we are safe to work more diligently in maintaining their quality (if any).

The Whiteboard

Participants