@CowperKettle we have a number of Russian-fluent statisticians on site; one might see the above conversation

@Glen_b Nice! I switched to another translation now, but I'll try to find some basic book on clinical statistics or soume Coursera course, and learn the basic ropes.

Can you tell why I cannot say that a parameter gets within n% confidence interval with probability n%? You say that interval either contains the interval (100%) or it does not (and corresponding confidence is 0%). This answer, IMO, defeats the very meaning of probability. Tossing a coin, I either get a heads or tails but not their probability.

Particularly, you seem not to apply the same logic to the credible interval.

@LittleAlien: See What's the difference between a confidence interval and a credible interval? & Why does a 95% Confidence Interval (CI) not imply a 95% chance of containing the mean?.

The mathematical apparatus of Probability Theory is applied quite differently in frequentist & in Bayesian approaches to inference: in particular epistemic uncertainty about the true value of a parameter is not represented by a probability distribution in the frequentist approach.

@LittleAlien, consider it from a different perspective. You toss a coin. It lays on the table in front of you. You can see that it is heads. Why do you want to say that there is a 95% chance that it's heads? That preserves "the very meaning of probability" for you, but I don't see any value in it.

@Scortchi Yet, you prefer to explain the flaw of "there is 90% prob that value of interest in that CI" by simply saying that either value in the CI or it is not. You say it here and in the reference PDF. I wonder why do you switch to epystimology to support this argument. You either have heads or tails. There is no probability. Why do you need the probability?

Epistimologically, Bayesian credibility may be much stronger grounded but shy should it not let me to apply the same argument, "your value of interest is either within the credible interval or outside" to say that you cannot speak of probability of your interval capturing the true parameter value.

I do not speak about all the epistemology, I just want to you explain why this particular argument can be applied to the confidence intevals, to demonstrate them nonsense, but cannot be applied to the credible interval. It is about double standards and justice, not about other enything else, not about other aspects of epystemology which can really perish the confidence procedures displaying the credible intervals the only meaningful ones.

@LittleAlien: Saying that either the true value of the parameter falls in a given interval or it doesn't isn't an argument - as you point out it applies regardless of whether that interval's labeled with "credible" or "confidence". Rather it's all you can say unless you define what you mean by a probability distribution over parameter values; until you've done that you can't make sense of statements like "there's a 90% probability that the true value of the parameter falls in this interval".

@Scortchi 1. All you can say that you will have either heads or tails. So, the notion of probability is meaningless. 2. I think that credible/confidential intervals are pretty specific what they mean but this does not stop you to say that you either have value in the CI or not and deny the meaning of CI on this ground, overlooking application of the same logic to the credible interval.

@gung Thanks for your replies about d-prime. I did not want to abuse the comments any longer, but here is what confuses me. When wikipedia says that "For normally distributed signal and noise with mean and standard deviations ..., d prime is defined as ..." - is it then simply wrong?

Because d prime is defined via hit and false alarm rates and hence even for normal signal and noise d prime can be very different for different detectors

However, I guess under some assumptions about the detector, this formula is correct, and this assumption I guess can be formulated as some optimality condition. That's the only way I see to rescue that wikipedia formula.

But you are right that there can be many different optimality conditions, and clearly not all of them will imply this formula.

Does it make sense?

@amoeba, I certainly wouldn't phrase it the way it is on Wikipedia. I notice that the "definition formula" references a scientific paper in the Journal of Neuroscience, rather than a methodological paper or a textbook. I'm not that Wikipedia savvy, I tried to scroll through previous iterations of the page to figure out how it came to its current form, but I don't see much.

@gung Funny thing is that I found that answer of yours because I am reading a neuroscience paper that used d prime without defining it or giving any references. From how they are presenting it, it seems to me that they are using this "wikipedia definition".

The paper I am reading is from 2016 and from one of the top journals in the field

"Wikipedia definition" is basically Cohen's d for equal sample sizes.

That isn't exactly what it originally meant. There is a methodological literature on this (although, I mostly haven't read it); there are canonical sources. It may be that people in neuroscience have come to use & think of it in a specific way, perhaps analogous to something like linguistic drift.

I am looking into that 2010 jneurosci paper right now...

and then they give some refs, let me see..

The most authoritative and the oldest source that they cite is Green DM, Swets JA (1966) Signal detection theory and psychophysics. New York: Wiley

I found the pdf of this book but I think I am not motivated enough to keep investigating it right now.

Yeah, Green & Swets is standard.

@LittleAlien: That either the true value of the parameter is in a given interval or it isn't may be an indupitable fact, but I've tried to explain why it's not the reason why you can't say a confidence intervals' having 90% coverage implies the true value of the parameter has a 90% probability of being in that interval.

@gung Checked one more citation from the 2010 paper - it's a 1999 jneurosci paper. They use the same definition of d prime (that is essentially cohen's d) and cite only green & swets.

So I wonder if this definition is already contained in there.

I'd have to find that stuff and look it up. Conceptually, we assume that, but we don't ever have it & it isn't measured / computed.

It seems that you were right about linguistic drift.

It's not really all that different. It's easy to see how it gets used that way. That was the point of my answer: they're ultimately the same, but used in different contexts.

I added a full wiki to the tag.

If you're wondering about this, &/or the issue of 'optimality' the best thing would be to ask a question--it would take a bit to explain. (Not sure how fast I can get to it, though.)

I gave an answer for a "best guess" estimate to the problem recently posed here ( stats.stackexchange.com/a/234854/127790 ). The question was ambiguous, but interesting I think. I would appreciate any ideas on error estimates for my answer. (Perhaps @whuber or @Glen_b have some pointers?)

In this answer, you say that you cannot say that CI contains a value because CI either contains it or not and there is not probability. It is an answer to a cited paper, which actually gives this reason against the CI in it beginning. In the end, it proposes the credible interval saying that credible intervals preserve the expected probabilistic interpretation. I wonder why?

@LittleAlien The short answer is that credible intervals are a Bayesian method that treats the population quantity being estimated (e.g., the population mean) as random, rather than fixed.

@GeoMatt22 You were right to change from probability DVs to binomial DVs, since the "probabilties" are only observed proportions. However, the assumption of independence between the events might be too unrealistic. OP was wrong to give the "generalized" version of their problem instead of the real one.

@Kodiologist Thank you for the answer. It starts to make some sense now. It is doubtful however since the question "what is the probability that parameter gets into interval?" is asked by frequntialists, which implies that there is a randomness in matching between estimated parameter and the confidence interval. It should be fixed in the Bayesian statistics, since bayesianists seem to be not interested in this probability.

@LittleAlien There is a randomness on the frequentist side; it's just that the interval is random, not the parameter. The Bayesian approach has it the other way round, by treating the data as fixed and the parameter as random.

@Kodiologist thanks. Actually in terms of the independence, I realized that if the elements of the collection were truly independent, then the prior on the collection's "Binomial-p" should probably be highly positively skewed, i.e. very non-uniform, as the size of the collection grows.

(e.g. if the prior is uniform on the element probabilities, the prior on the collection would be like math.stackexchange.com/questions/659254/… )

If I had a nickel for every time somebody replied to my criticism of a method with "This is standard in $FIELD", which seems to be academic-speak for "This is the sacred way of our people", I'd be a wealthy man.