« first day (2495 days earlier)      last day (1328 days later) » 

[ SmokeDetector | MS ] Username similar to website in answer (63): Docker problem with Springboot Batch-REST-app based on two DBs by artsgard on stackoverflow.com
 
@Makyen The TPs on this one are...weird. I'm not actually convinced they're all TP, like I suspect this one might be FP, for instance, and there's mixed feedback on others.
The only two clear-ish TP that wouldn't have been caught by other rules are this one which was judged (reasonably, but borderline) to be TP based on the sarcastic/insulting tone, and this one which could be hypothetically caught by a watch on "beat your meat" (0 FP on MS/SE, only 1 TP though)
 
@RyanM beat[\W_]*+[\w']*+[\W_]*meat catches one more (not on Islam), probably worth a watch
 
8:05 PM
fp- feedback received
 
@NobodyNada oh nice, +1 for watching that
 
@ThomasWard looks good to me!
 
@RyanM in that first one, the word "masturbate" wasn't the offensive part -- it was really coincedental that we caught that post at all
 
@NobodyNada Yeah, agreed. Even if it weren't a coincidence, a missed tp or two in a sea of hundreds of fp just isn't worth it.
 
user435118
Ok. I'll stick to extremely accurate automated feedback until I think/get to developing a better system.
 
!!/watch beat(?:|s|ing)[\W_]*+[\w']*+[\W_]*meat
(more future-proofing)
 
@Daniil I think the consensus is that we don't want automated feedback at all
 
user435118
@NobodyNada You won't know...
 
@Daniil yes, which is why we must operate on trust
 
8:12 PM
@Daniil I mean there are lots of things we could do to undermine the system without detection or making it hard to reverse.
That's not a reason why we should do those things though
 
If we did have automated feedback, it would be absolutely essential that we be able to easily separate it from human feedback in order to audit it and evaluate its accuracy.
 
user435118
Fine. I'll keep logs locally if you want.
 
user435118
Which will be made available for a good reason.
 
8:14 PM
@Daniil You have been given reviewer, flagger, core, and blacklist manager privileges, which means we trust you to behave sensibly and follow the rules set by the project admins. To explicitly violate those rules would be a breach of that trust
 
@Daniil No. We're saying we don't want automated feedback.
 
user435118
@Makyen You're not saying why though. It's more accurate than a human.
 
Isn't the best way forward to collaborate on this and implement this improvement in the system? Why not pursue that option first at the very least being seemingly the most sensible as well as seemingly the most agreeable path... Why the focus only on autoreviewing instead of other options?
 
8:16 PM
@Daniil Accuracy isn't the issue, the issue is whether it's a human or not
 
@Daniil If I might suggest a compromise: if you were to use your automation to produce a list of reviews that it shows to you and that you look at and evaluate by hand to confirm they're all fp before giving that feedback, I expect that would be fine.
 
@Daniil It's also highly controversial
 
@Daniil If you want to experiment and run something that just generates logs, but does not actually give feedback to MS, and you don't just run through applying the feedback for the bot without actually reviewing the post, then just keeping the logs would be OK.
 
but also if it's that accurate then this seems like a really useful thing to prevent the posts from being reported in the first place, as Makyen noted above.
 
8:20 PM
Or if it's not accurate enough for that, marking posts (by means other than how humans give feedback) as "this was judged by the system to not require a second review" after one FP feedback would be similarly useful.
 
@Daniil We have given you reasons, opinions, and a decision, you just haven't liked them, or considered them relevant. We generally trust you not to do this. The way you're continuing with this, in particular that you're implying that you will actually move forward and do it, and we just won't know, makes us trust you less.
 
user435118
There's no point of me generating logs and then feedbacking myself when I have a bigger possibility of giving inaccurate feedback due to a misclick.
 
8:22 PM
@Daniil If that's your concern, you could automate the giving-feedback part, but the human review is still a must, as some of these are quite subtle.
 
@Daniil You appear to be assuming that you will just be applying the bot's feedback by hand, without reviewing them yourself.
 
(AFK)
 
@NobodyNada yep i thought as much. But what we really need is some retooling to make sure this actually behaves right, and passes in the escaped (raw from API?) body for testing. That I don't have the skills to add as a rule, but I can at least write the programming logic for it
 
@RyanM @Daniil Do not take this statement by @RyanM to indicate you should have the bot apply the feedback itself.
 
@NobodyNada (hence that code with a testsuite and multiple test examples - the 'nonprintable characters' part was an addition I thought of today, someone trying to use a tab or a linefeed or something to obfuscate)
 
user435118
@RyanM It's more accurate than a human.
 
@Makyen apologies if I wasn't completely clear with my suggestion: I was not suggesting automated feedback that is not directed by a human who has read the post and judged it to be fp before the feedback is given
 
user435118
@Makyen Will if there's 99.99+% accuracy I won't review by hand.
 
user435118
@RyanM The initial version only feedbacks if it has one feedback.
 
@Daniil is it 99.99% accurate without relying on having a piece of human feedback to copy?
 
@Daniil How are we judging accuracy then?
 
user435118
8:32 PM
3 mins ago, by Daniil
@RyanM The initial version only feedbacks if it has one feedback.
 
@Daniil I think the gist of the question was, does the one feedback feed into the calculation or is it entirely independent?
And have you tested it on 10,000 posts where it independently came to an fp conclusion validated by the one human review on the same post in all but at most 1 case?
 
user435118
Ok, how about this. I'll try this through some periods next week, if you can spot a few posts where it was automated and it's for a reason, not guessing and no cheating I'll stop.
 
user435118
@DanielWiddis It is pretty dependent on the feedback, not 100%
 
@Daniil I see. Then that really doesn't leave much choice on our part in how to respond.
@Daniil Let me make this crystal clear: I have disabled your access to metasmoke. You have explicitly stated you will do something we've said we don't want you to do. Your continued statements here make me not trust that you will not set up automated feedback.
 
8:37 PM
@Daniil That is simply not going to fly. Applying automation to areas of the system is not your choice alone to make. People have told you that they do not agree. Why are you continuing as if you will do it regardless?
And even if it were "can a human tell the difference" is not remotely a good way to measure the success of the bot.
 
user435118
@Makyen I didn't say anything.
 
You literally just said:
2 mins ago, by Daniil
Ok, how about this. I'll try this through some periods next week, if you can spot a few posts where it was automated and it's for a reason, not guessing and no cheating I'll stop.
 
We can kinda try this out on test.MS I guess
 
If the feedback accuracy is equal or better than human, why would we rather trust a less accurate entity than a more accurate entity?
 
8:39 PM
which is going to be recloned on Friday
(so anything yo udo now on test.MS gets blasted Friday evening EST)
 
If it is just about 'because we are human' then it does not really mathematically make sense
 
Metasmoke will be down for approximately one hour on September 18th around 22:00 UTC while we re-clone a copy of Metasmoke to the test MS server. Metasmoke will be unavailable during the cloning process, but will reactivate after the clone is complete.
11
 
@ThomasWard :-P
 
@user12986714 I'm unclear how "truth" is defined here in terms of accuracy, if not the human review. I could develop a 100% accurate automation to vote fp on every post that a human voted fp on. Would you believe it was 100% accurate?
 
8:41 PM
What about we have @Daniil create an account on test.MS after the reclone and try the thing out and get the actual stat?
 
@user12986714 how do you define the stat?
 
@DanielWiddis So it is like we wipe all human feedback on test.MS, and try out the bot, and compare the bot feedback with human feedback
 
So Metasmoke will be taking a smoke break?
 
@Machavity lol
 
@DanielWiddis Same. I'm also uncomfortable if feedback is being used to judge accuracy and it takes feedback into consideration this is circular logic.
 
8:42 PM
If it is really indistinguishable I think it kinda passed the turing test
 
@user12986714 The point of having human feedbacks is that we don't EVER want to rely solely on heuristics, because heuristics can be wrong/not context-sensetive/self-reinforcing
2
 
@user12986714 The turing test isn't even that great of a test for chat bots, I'm not sure why it would be useful here.
 
Human feedbacks aren't perfect -- which is why we have two of them -- but they're not just piles of regexes, which is the important part
 
@user12986714 I agree in principle. However, Daniil explicitly stated it used the human review as part of the judgment: "... pretty dependent on the feedback, not 100%"
 
user435118
8:44 PM
@ThomasWard Sounds like an good time to delete my MS sock and the new spam user.
 
user435118
@Makyen Can you please reactivate my MS key?
 
@NobodyNada Yeah, but we make mistakes. It is like we still use Fermat's prime test in SSL libraries even it is not 100% accurate. This is analogous to the chance Fermat's test producing incorrect result less than background radiations flipping a bit in memory causing errors.
 
@user12986714 The point is that "human feedback" vs "machine heuristics" are kept entirely seperate throughout our system
We use lots of both, and we don't treat either one as being infallible, but we always distinguish between them
 
8:46 PM
Yeah so that is why I emphasize on 'indistinguishable'
 
naa feedback received on [MS] Activex control for web print
 
I wonder if it would be useful to have an AI prioritize posts for which human review is most likely to improve the overall accuracy, in case there is a backlog?
 
:+1:
 
@JohnDvorak This whole conversation belongs on the post on teams.
 
8:49 PM
@Daniil Yes. Removing the Flagger role and deactivating your MS API key was not needed. The issue is writes. I've restored the Flagger role to your account and re-activated your userscript's MS API key.
 
@NobodyNada Well, I don't quite understand the reasoning here. If the machine passed the turing test, what is the difference between it and a human in the field of giving feedbacks?
 
@user12986714 The term for something that's "really indistinguishable" from a human is "human"
Unless Daniil's invented a true AGI, there always will be some difference
 
user435118
@Makyen What about Core?
 
8:52 PM
[ SmokeDetector | MS ] Potentially bad ns for domain in body (1): Assinar pdf com delphi by Marcos De Oliveira Dário on pt.stackoverflow.com
 
user435118
sdc amicodeprivileged
 
@Daniil ╳ No, you are not a blacklist manager privileged user.
 
user435118
...
 
user435118
And that @Makyen ^
 
@NobodyNada It is more in a probabilistic sense. Say if human feedback is 98% accurate and the machine is 98% accurate, giving that there is only one machine (so no double wrong feedback), it is just equivalent to human feedback I think.
The problem is about having more than 1 machine, because there are differences between people but not necessarily so between machines.
 
8:53 PM
fp feedback received on [MS] Mywifiext.net Setup Login Method
 
user435118
Review seems to have gone as well
 
@user12986714 But the 2% are generally going to be different
 
fp feedback received
 
@Daniil As I said, I restored the "Flagger" role to your account.
 
@NobodyNada Yeah, so if there is only 1 machine feedback, it does not really cause harm.
 
user435118
@Makyen But that's not the only role I had...
 
@user12986714 The problem isn't the machine feedback in and of itself, it's the machine feedback masquerading as a human feedback
We intentionally keep those seperate
 
@NobodyNada So what about we create a bot account and use it to do things like that?
 
8:56 PM
1 hour ago, by NobodyNada
Depending on the problem we're trying to solve, we can do it better by just applying the criteria elsewhere
 
[ SmokeDetector | MS ] Mostly punctuation marks in answer (36): Разряд прилагательных by ZaCk - Brawl stars on rus.SE
 
1 hour ago, by Makyen
Seriously, if we have code that's good enough to be giving automated FP feedback and be 100% accurate, then why not have that code in SD to prevent the report-which-will-be-FP in the first place?
 
Can I get consensus thoughts on this post? It's a (wrong) answer to the question and redirects to a user-uploaded file download site with lots of ads to install mal/bloat/something-ware.
 
1 hour ago, by ArtOfCode
@Daniil I would want more information on that before deciding. That said, what NobodyNada said makes a lot of sense - perhaps feedback isn't the correct point at which to apply this automation.
 
user435118
@SmokeDetector f
 
@NobodyNada Well, implementing it on SD side does not allow that manual feedback to direct machines.
I guess the machine is like if the post have a fp and passed <tests> flag it fp
 
@user12986714 It depends on the problem we're trying to solve. If the problem is too many old FPs in review, then we should modify the review queue to kick out old FPs
 
Not to say "two human reviewed this post", when in reality only one human did
 
fp feedback received on [MS] how use hindi font in pdf
 
@NobodyNada This is like a 'auto review job' v2
 
@user12986714 I'm not sure what you're referring to, was there a v1?
 
Of course, the v1 is not as accurate
 

« first day (2495 days earlier)      last day (1328 days later) »