All right! The fact is that EFA does not make any assumption regarding the distribution of the responses. So you can't use confidence interval, model fit indices and the like. But if the factior structure is close enough there's not much to worry about.
Sure, at least you know what to respond to a reviewer would be picky about that. IN this case, using EFA alone I would simply summarize the difference between each loadings. If they don't vary too much (say, 0.1 on average), that's probably ok. Add to this the correlation between individual scores on run 1 and 2, with a confidence interval.
The proper way to do that would be to use a confirmatory factor analysis with two groups (here, defined by the time of administration) and check whether the loadings are identical. That's why I said that EFA alone is not the best option.
By the way, I took the liberty to clean up the comment threads by putting questions as an edits to your original post, and my answers as an edit to my reply. If you're happy with that, I suggest you delete all but the first and eventually last comment of yours.
An alternative approach would be to carry out a principal component analysis, assuming individuals scores are stored as numerical values. Look at the scree plot (the distribution of eigenvalues) and the correlation circle (which summarizes the correlation between all variables/items). How do they compare?
First idea: compute the correlation matrix on all items for the first run, then for the second run. Do they look close one to each other? This will indicate whether individuals keep answering in a coherent way to all items.
As I understand it, you want to ensure that your scale adequatly reflects the construct it purports to measure, taking into account multiple administrations of that very same scale using stimuli of different polarity. Is that right?
I have very good memories of the start and development of this site, of all these first contributors, some of whom are still here, as well as of the moderators who succeeded one another. Thank you @whuber for the introduction and the compliment, I think it is you who has become the living memory of this site by now. I hope that the community will remain as enthusiastic as it was in the beginning and I wish all the best to the new moderators.
Minor correction: Hopefully, you're still there, and contributing. The last point is important since I for one stopped to answer question for real once I was elected.
@whuber I assume so. Mine is pretty calm (and sometimes boring, because mostly technical, but that's okay). Thanks again for your first welcome message, btw.
Oh, nice! I posted the same message twice! And I can delete either one. You're right @Scortchi-ReinstateMonica, that's what I probably miss the most: editing/deleting my/others' typos.
@gung-ReinstateMonica Thanks, @gung! Shared pleasure. It's really too bad that you resigned (even if I can understand why) but your voice probably still counts as much as it did in the past. Looking forward to learning from you again.
Happy to see that we're still in a ping-pong game from time to time in this room too. Anyway, I just wanted to say hi since I've been away for such a long time. I don't expect to be as active as I've used to be in the past, but I'm happy to see the site is going well. Best wishes
@amoeba As it stands, this does not really fit SE standards. Either we need to convert the latest thread as a comment, or you will need to use a dedicated chat room (or create a new session of our (dead) Journal Club :-).
@Glen_b You're welcome! I think your answer deserves more attention because you clearly described how you approached the problem (as @whuber did), which is something I really like.
Since we are just keeping this room open to unfroze it every two weeks, I was wondering if it would be worth posting a new message on our Meta or add a banner on the main site to call for contributions. /cc @whuber @Glen_b @mbq @cardinal @peterflom
Discussing about the site itself is a good idea, IMO. Book reviews, conference announcement/wrap-up, or case studies would also be welcome. Indeed, these topics would not overlap with the main site. So, I would be happy with the above proposal, which I only noticed today. My bad!
Ten fold
Ten fold