« first day (3080 days earlier)      last day (1928 days later) » 

Rob
Rob
2:01 AM
Shog has posted some Stack Exchange stats in The Tavern.
 
 
7 hours later…
Martijn Weterings
Martijn Weterings
9:16 AM
Do we have a canonical question about the *correctness* of the question below and discussing significance conceptually?


How do I compute whether my linear regression has a statistically significant difference from a known theoretical line?
https://stats.stackexchange.com/questions/388448/how-do-i-compute-whether-my-linear-regression-has-a-statistically-significant-di

I have some doubts about the ways that "statistical significance" is being used.
 
 
2 hours later…
Scortchi
Scortchi
11:14 AM
@MartijnWeterings: Probably ...
What are your doubts?
 
Martijn Weterings
Martijn Weterings
12:01 PM
I have my doubts in the comments. In summary:
(1) viewing statistical significance as a binary variable (the regression has or has not statistical significance)
(2) using a hypothesis test to test a theoretical model (y=x), but ignoring the model actually test a statistical model (y=x+e)
(3) ignoring the power of the test (comparing y=x versus y=a+bx, but not other than 1st order polynomial models)
(4) misrepresenting a potential complex data set by a single relationship (e.g. some sort of Simpson's paradox may be luring around the corner)
(3b) testing the significance when the power is low due to small sample size, or not stating what sort of variation from theory is considered important.
(3c) wrong representation of significance due to non-normality of errors
I may need to reorder or place those doubts in a better scheme. But say, one can make type I and type II errors for other reasons than statistical significance, and that question breaths a simple and layed back interpretation of significance and hypothesis testing.
 
Scortchi
Scortchi
12:31 PM
@MartijnWeterings: I see: I'd wager that all of those matters have been dealt with on CV, but I don't think we've a general "Do I really want to perform a significance test?" thread. Float the idea on Meta.
2
In the question you reference, the axis labels on the graph are what suggest the model might be mis-specified & the wrong kind of alternatives considered: seems like a classic XY problem.
 

« first day (3080 days earlier)      last day (1928 days later) »