I've got a question about lme4 over on Stack Overflow, if anybody wants to take a swing at it.

@Kodiologist I don't really understand what you are trying to do - I have the same problem as the first commenter there, and your reply does not make it clearer. Can you describe here what you want to do with more details?

You have 5 predictors and want to estimate five betas but such that they come from a common gaussian distribution? I.e. you want to estimate the mean and the variance of this distribution over five betas?

@amoeba Yes, although the mean and variance are just nuisance parameters. I only care about the coefficients themselves.

I'm not sure what details there are to add.

Hey @Kodiologist

@amoeba Hey.

I think I see what you mean, but I am not sure this can be framed as a mixed model at all. Perhaps it can - but it seems at least nontrivial

What's the issue?

Isn't it pretty much ridge regression?

well, when you say "random slopes", I think of a model y = b_i x + a_i where b_i can be different for each group i

That's a mixed model

But you don't have any groups. You have y = \sum b_i x_i, and you want b_i \sim N(0,s). Maybe I am being stupid, but I don't see how to write it as a "usual" mixed model

b_i \sim N(mu, s)

Right, the unusual thing about this situation being that each random slope is shared by all observations instead of just a subset of observations (a "batch").

@Kodiologist @Kodiologist How is that random?

@GavinSimpson In that they come from a common normal distribution.

In OLS the slope is a random variable. I'm not seeing any distinction here

Yes. In any case, this is definitely not a programming, but a statistics Q, so you should rather post it here - and I would frame it a bit differently as well (not "where is the mistake in my code", but rather "is it a mixed model at all and if yes how can I approach it with lme4")

In OLS, the slopes are not expected or constrained to be similar to each other in any way.

But there is one slope for all observations, right?

You mean you want slopes for the 5 covariates to be similar?

Yes to the second one.

@Gavin, here is the model: y = a + \sum b_i x_i, b_i \sim N(mu, s). If you write it like that, b_i are random variables

Ah, OK.

Is the problem that in a mixed model, the coefficients for distinct covariates can't come from a common batch?

I will admit that it looks like it should be a mixed model, but I have no idea how to write it as one.

Are the x_i related in some way? I mean why should the slopes be similar?

@GavinSimpson Depends on the problem, but even if you don't expect the population values to be similar somehow, you might want the shrinkage.

Now thinking about it, when we have usual random slopes term (b|subject) in a lmer model, it technically refers to several binary dummy predictors ("subject") and their betas ("b"), and your situation is identical with the only difference that instead of dummy predictors you have five real predictors.

But how to write it in a lmer syntax - no idea

Cool question.

Do consider bringing it to CV.

@amoeba Good point. The most substantial difference I can think of is that in the usual case, there is only at most one random slope from each batch in each observation.

Here, each observation has 5 slopes from the same batch.

@amoeba Wouldn't you need to put the data into long form so have a column x that is categorical. Then you can do it. Whether this makes any sense is another matter

Gavin, I don't see how it would work. Each y should be a linear combination of all five x's, and long format would destroy that, no

?

The problem currently is that you have separate x1, x2 etc. These need to be a single x categorical variables with value as numeric containing the stacked x1, x2, etc

@amoeba Right, but the only way to get lme4 to do (b | x) is to pull the data into that long form. if that doesn't make sense, then I don't think lme4 can do it.

But a model y ~ (b|x_number) is not a model Kodiologist wants to fit.

Maybe there is a way to supply "design matrix" for random effects manually? Instead of relying on the formula? This is definitely beyond my lme4 skills, btw

In particular, the model you're proposing, Gavin, wouldn't allow additive effects of the five predictors.

It's tricky not that that tricky

@amoeba I know; I'm just working through the implications of your earlier point that suggested similarity between (value | x) with value numeric and x categorical indexing the covariate. That isn't what is wanted but it is the only thing lme4 can do.

I'll caveat that by adding "without getting deep under the hood of lme4". The underlying machinery is quite flexible I am told if you understand what Doug Bates was doing with the code. A colleague has used the underlying machinery to fit a wider range of models than the lme4 interface allows via lmer(). But doing that requires almost intimate knowledge of the lme4 internals.

Interesting, Gavin! Do you know what kind of a "wider range" was needed?

@amoeba he was fitting multivariate models, latent variable models, and models with spatial and temporal correlations as specific examples IIRC. There even was/is an R package somewhere on github

This was what Steven has been working on:

github.com/stevencarlislewalker/lme4ord

Not the model being discussed here, but all things you can get lme4 to fit if you dig deep enough

If the b's are supposed to be equal, all you need to do @Kodiologist is to put the sum of the independent variables on the right hand side. This constrains the slopes (and the variance for mixed effects) to all be drawn from the same distribution.