Discussion on question by William: Is there any *mathematical* basis for the Bayesian vs frequentist debate?

Whether or not questions attendant on this debate are ill-posed, there are real issues involved. They come down to two things, neither of which is mathematical (although some Bayesians would like to pretend so by hiding the issues behind sets of axioms): (1) the interpretation of probability and (2) how one models phenomena. The former is insoluble because probability has many useful interpretations; no single one will ever rule. The latter is interesting insofar as "Frequentists" will apply Bayesian methods when they can, but are more reluctant to invent or assume priors than are Bayesians.

"Mathematically correct" is a useless criterion for selecting statistical procedures--as in any quantitative field, it's merely a precondition for any procedure. What matters is applicability. A classical statistician would be glad to help a client make a decision by means of a minimax procedure, for instance, while a Bayesian would eschew that as being inconsistent with their axioms and assumptions. Classical statisticians compute confidence intervals all the time; Bayesians have no use for them at all.