Ten fold

12:06 AM

@NickStauner WLS isn't really 'based on 4th order moments'. WLS is (arguably) based on first and second order moments of some appropriately standardized quantity. It might potentially count as method of moments in some circumstances.

Nick Stauner

i see...seems like the article I read that in had several theoretical holes...

wouldn't the fourth moment be kurtosis?

Glen_b

It may be that the article was saying something correctly, but in an unclear way.

The fourth moment would be related to kurtosis, but isn't itself kurtosis.

Nick Stauner

maybe; "based on 4th-order moments" is a direct quote IIRC!

OK, good to know, thanks!

Glen_b

Is the paper on-line?

Nick Stauner

Hmm...maybe not...

Glen_b

12:09 AM

kurtosis is a standardized fourth moment (or equivalently, a fourth-moment of a standardized variable, as long as we're not talking about excess kurtosis, which adds a shift)

Just looking for the context in which the quote arises.

Nick Stauner

this is the article anyway:

ncbi.nlm.nih.gov/pubmed/16279297

"Neither weighted least squares nor diagonally weighted least squares estimation makes assumptions about the nature of the distribution of the variables and both
methods produce asymptotically valid results. Nevertheless, because weighted least squares estimation is based on fourth-order moments, this approach frequently leads to practical problems and is very computationally demanding. This means that weighted least squares estimation may lack robustness when used to evaluate models of medium, i.e., with 10 indicators, to large size and small to moderate sample sizes."

Glen_b

Thanks. I have the pdf now.

Normally, WLS implies "diagonally" (in that the weight-matrix is diagonal). If there are off-diagonal elements it's normally called GLS not WLS). I might need to take a look to make sure I understand the situation

Nick Stauner

oh, also good to know! I think the idea (for DWLS) is to use only the diagonal when inverting the covariance matrix...?

Glen_b

There may be something about this particular application which makes it related to moments of the fourth order.

... what?

Wow

Nick Stauner

No idea if that makes sense :D Way out of my league now because I don't remember much of anything from matrix algebra besides how to find a 2x2 matrix's determinant

the general application is just confirmatory factor analysis, or simple SEM in general. The main idea was to test if WLS is better than ML for estimating these models with ordinal (Likert) data.

Glen_b

12:18 AM

Okay, this is what I get for having insufficient context. They're using the terms in a somewhat unusual way. It's a common issue when different areas all use their own bits of methodology.

SEM and 'simple' don't go together in my mind.

Nick Stauner

@Glen_b bah! That figures...SEM people seem to be in their own little world sometimes...

Glen_b

Don't get me started on economists and 'fixed effects'...

Nick Stauner

hahaha...that sounds like a good diatribe!

@Glen_b I'm rather surprised at how simple it can be to perform SEM (without having a damn clue as to how the math works "under the hood")

Glen_b

I'm going to say "I don't know what's going on in that paper well enough to comment yet"

It's simple to do almost anything if someone will write code to do it.

Nick Stauner

fine by me :) that paper is part of the answer I had in mind anyway. they give some tentative evidence (that I've seen supported in one other oddball article) that DWLS estimation works better for ordinal SEM than ML even with moderate sample sizes.

Glen_b

12:22 AM

These covariance matrices are estimates (based on data), right?

I confess I'm likely out of my depth on this one.

Nick Stauner

yep. With ordinal data, I think a polychoric correlation matrix is substituted somehow though. Part of what they did was compare ML vs. WLS estimation of a polychoric matrix, and then vs. ML on an ordinary covariance matrix.

Glen_b

I have several nits with the first page of the article, but this is literature with which I have almost no familiarity. I've read Joreskog & Sobom, but that was decades ago. The rest of it I doubt I've seen at all.

Nick Stauner

Here's another article on roughly the same topic; slightly newer and better methodologically, but still not the most shining example of a methodological article...

infonomics-society.org/IJDS/…

Glen_b

I've done a little with partial least squares, but I'm pretty much an ignoramus on SEM applications all round.

I've read a few papers and helped a few people wrestle with software and interpret some results to the best I could manage.

Nick Stauner

@Glen_b I'm curious about those nits. I don't have a clue what might be wrong with the first page.

Glen_b

12:27 AM

Nearly all of it about 20 years ago (sometimes more).

In particular, in the discussion about Pearson correlation; again they might mean something reasonable but on the face of it their phrasing suggests something that's not quite right. But I'll hold back on that until I have a better picture of the thing. I'd be interested to read what you do on this.

Nick Stauner

I'd love to bounce it off you soon! I don't know if I'll get any deeper into it than these articles do though, so it might read like a bunch of SEM-speak by the time I'm ready to share it...

Glen_b

What is the correct "magnitude of the correlation between ordinal variables" and how do they know?

Nick Stauner

@Glen_b That one I think I can answer. See Bollen & Barb (1981) for a sim study of continuous vs. binned variables and Pearson's r. Basically, binning continuous data attenuates correlations, much like we know about dichotomization.

Glen_b

I imagine it will look like SEM-speak. I will likely learn something from it.

Especially if there's someone I can ask dumb questions of.

Nick Stauner

Ha! It would be my pleasure to answer dumb questions on SEM :) that's basically what brought me here in the first place!

Glen_b

12:33 AM

But in the case of binning there's an underlying continuous variable about which we can say "this is the 'correct' correlation". In the case of ordinal variables, that's not always the case.

I ask lots of dumb questions. It's basically how I learn.

Nick Stauner

true; the underlying continuum is an assumption of typical latent trait models in psychology. We generally assume that our Likert scale ratings are manifest indicators of (caused by, basically) these latent continuums. (continua?)

Glen_b

So the argument goes "in a study where there's an underlying continuous variable, the correlation between the continuous variables was underestimated by binning and then calculating a correlation on the binned variables. If we assume there's underlying continuous variables behind our ordinal ones, we'll underestimate that correlation" ... do I have that right?

Nick Stauner

yeah, sounds about right. basically, it's thought to be more efficient/reliable to ask people to rate themselves on ordinal scales rather than continuous scales, but these impose a sort of binning in the process of deciding, "Do I agree or strongly agree?"

Glen_b

Okay, makes sense, it's actually the case.

But there are other issues - even if the underlying variables were linearly related there's no reason to expect any binned version to be.

Nick Stauner

I may be giving traditional usage of Likert scales too much credit for being justified in some way...

@Glen_b no reason? or just not enough justification to assume without a doubt? there are certainly lots of inferential jumps in the process, but I think it's at least likely that an evenly binned version would preserve a linear relationship well enough...

Glen_b

12:39 AM

On the original issue, my guess is that in this application, the WLS they're talking about probably isn't method of moments.

Nick Stauner

Okay. So much for that answer then!

Glen_b

Sorry, I just had to go batten down all the hatches

Nick Stauner

no prob; I should probably go, come to think of it. Gotta prepare my stuff in storage for the moving truck! I'm moving halfway across the country in a week or so for that postdoc (at Case Western Reserve in Cleveland, Ohio).

Glen_b

I mean - in the absence of justification at least - 'no reason'. It would require assumptions I see no basis for believing.

Well, I'll let you go (though hatches are now battened and the storm is slowing back to ordinary rain by the sound of it).

I hope to chat with you some more at some point.

Nick Stauner

I'm really eager to finish this transition and get to work. I expect to generate a lot of questions in late September–October...so I'll be around :)

Glen_b

12:47 AM

Few of which I doubt I'll have much clue on, I'm guessing.

I know a bit about simulation though, so I may be able to put in the odd word here or there.

Weird, it's hardly raining at all now. It was near torrential ten minutes ago.

Nick Stauner

Yeah, that's going to be one big task: sampling data from simulated populations with known correlational structure (other than simply uncorrelated) is something I haven't figured out yet, and will need to!

Rain really comes and goes in the summer it seems; same thing happening here in California and in Minnesota in the past month. Almost had to cancel a canoe trip down the St. Croix river because of heavy rain that passed before we'd even left the area!

Glen_b

If Ohio is halfway across the country ... California ?

Oh, yes, I see, California.

Nick Stauner

presently, yes. "Inland Empire" = Riverside, San Bernardino, Redlands, Loma Linda, etc.

I'm in Beaumont though, which barely shows up on Google Maps :)

Glen_b

I can help with that part (simulated correlated data) at least.

I've spent a little time in Minnesota. Mostly Minneapolis and St.Paul though

Nick Stauner

I was born in Minneapolis! Spent 23 years in that metro area.

Glen_b

12:53 AM

I have a friend in St Paul. When I was in Minneapolis at a conference, he showed me around a little.

Nick Stauner

Feel free to ask questions in comments on any questions I come up with if you can't answer. Even if I can figure out the bit about simulating correlated data, I'll check for a good question here and ask one if I can't find one.

I just got to give a one-week tour of Minnesota when I was there a couple weeks ago (hence the canoe trip, among other things). I think I saw more of it in that week than I have in months of living there!

Glen_b

The usual approach is via Cholesky decomposition, though any matrix square root will work. I can teach you how to do it, but most packages will do generation of multivariate correlated data for you.

Nick Stauner

okay, that sounds pretty manageable. I figured it would be.

better go fight the storage wars...thanks again for your help!

Glen_b

Hmm, ... I must be careful; here by 'square root' I mean anything of the form M=AA', not literally A^2

OKay bye.

@quit

Nick Stauner

heh...note to self: be careful! :)

Glen_b

12:56 AM

Damn. I did that again

So used to a different forum

Bye again

Nick Stauner

2:15 AM

@Glen_b Which one, out of curiosity?

Glen_b

2:53 AM

A forum I doubt you'd have ever heard of, but the command itself is common enough; it's one used used in a variety of IRCs and MUDs and MOOs and other ancient relics of Internets gone by.

@NickStauner Sorry, forgot to nudge you there.

gung

3:30 AM

@NickStauner, for info on generating correlated random numbers you may want to check out this thread: stats.stackexchange.com/q/38856/7290

Nick Stauner

@Glen_b I'm sure I haven't heard of it! I have used IRC and played a few MUDs though. I know quit works in R too, though I tend to just Alt+F4 things...

@gung looks great! thanks!

Glen_b

4:46 AM

@Nick Yes, it's probably R that keeps it "in my mind"

Transcript for