(click the image for a larger version)

The values in the far low tail of the Z are for samples most like ("most nearly equivalent" to) normal.

You don't want to call those "not equivalent".

Now we come to something I screwed up; I had almost all the basic notions right but forgot to account for the fact that the hypotheses flip over; you're essentially putting all the non-equivalence possibilities as the things to reject in order to show equivalence.

So yes, you're correct - with a one way ANOVA situation you end up setting a value by looking to the left side of a cutoff of the distribution to reject non-equivalence -- but it's not the left side of the distribution under the null. On the equivalence boundary you're dealing with points under the alternative for the NHST, and it's the distribution under those specific alternatives on the boundary you have to worry about.

It's important to be clear on this. You specify a collection of alternatives that make up non-equivalence. and if no non-equivalences hold in the population, you must have equivalence. If things are neat enough you can use a sample to reject non-equivalence by testing each possibility on the boundary between equivalence and non-equivalence (you reject non-equivalence if any of them don't fall in the non-equivalence region). To do that in one way anova you need the distribution ...

of your test statistic under each of the alternatives that make up that boundary. You can do that for one way anova: it's just a noncentral F, whose noncentrality parameter is obtainable from the population ψ² -- i.e. it's the same noncentral F distribution for every point on that boundary.

For the W that's an even bigger problem than the second thing I was worrying about (though the two do have a connection). Royston's transformation to approximate normality ln(1-W) applies under the null, but you instead need the distribution under all possible alternatives in a nonequivalence region (or if you're lucky, on an equivalence boundary, were it possible to specify one; there's that connection). This isn't doable - the distribution is different for every kind of non-normality.

A uniform and a particular t-distribution might both be equally as far from normal on some metric but the distribution of the statistic under those two alternatives will be quite different.

[It's complicated even further - omnibus goodness of fit tests are generally biased (there are alternatives with lower rejection rates than under the null). It's an annoying, unavoidable but (usually) minor nuisance in NHST, but can be a somewhat bigger issue for equivalence testing.] The basic issues are (a) actually specifying alternatives (the space of alternatives is not governed by the size of a finite number of

parameters), and then (b) computing the distribution under them

I don't see any likely way forward.

I want to see in detail what Welleck does for the Mann-Whitney but I still don't think there will be a solution for W. I'll have to look at it on the weekend if I can find an hour or so; I am too tired to read it now.

I'm curious because the distribution of U under the alternatives will not be distribution free (distribution free tests are only distribution free under the null, not under the alternative), so again there will be an issue of how to do this with U (but a considerably simpler issue, so there may indeed be a suitable way to deal with it for U that I haven't thought of).