Required sample size and power for SEM
MacCallum et al. (1996, 1997, 2006) and Kim (2005) propose methods to calculate the required sample size (given a desired power) or the achieved power (given a sample size) to assess the fit of structural equation models based upon different fit indices (e.g. RMSEA or AGFI). The code on this page implements these routines in R and SPSS.
Instructions: Choose the calculation of interest below, fill in the necessary information and press the download button. To run the code, copy and paste it in your R or SPSS syntax window on your computer.
Calculate required sample size
-
For RMSEA (1)
-
For RMSEA (2)
-
For GFI
-
For AGFI
-
For Steigers γ
-
For McDonalds MC
-
For test of no difference between nested models (RMSEA)
-
For test of small difference between nested models (RMSEA)
Calculate power
-
For RMSEA
-
For GFI
-
For AGFI
-
For test of no difference between nested models (RMSEA)
-
For test of small difference between nested models (RMSEA)
Note:
Apparently Preacher and Coffman (2006) have already published similar R routines a while ago. You can run their code online, without having to install R on your computer. Check it out.
Additionally Friendly (2006) wrote a SAS Macro to conduct these power estimations.
Literature
- Friendly, M. (2006, January).The csmpower macro - Power estimation for Covariance Structure Models [Computer software]. Available from http://www.math.yorku.ca/SCS/sasmac/csmpower.html (2008-08-30).
- Kim, K. H. (2005). The Relation Among Fit Indexes, Power, and Sample Size in Structural Equation Modeling. Structural Equation Modeling, 12, 368-390. doi: 10.1207/s15328007sem1203_2.
- MacCallum, R. C., Browne, M. W., & Cai, L. (2006). Testing differences between nested covariance structure models: Power analysis and null hypotheses. Psychological Methods, 11, 19-35. doi: 10.1037/1082-989X.11.1.19.
- MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1, 130-149. doi: 10.1037/1082-989X.1.2.130.
- MacCallum, R. C. & Hong, S. (1997). Power Analysis in Covariance Structure Modeling Using GFI and AGFI. Multivariate Behavioral Research, 32, 193-210. doi: 10.1207/s15327906mbr3202_5 .
- Preacher, K. J. & Coffman, D. L. (2006, May). Computing power and minimum sample size for RMSEA [Computer software]. Available from http://www.quantpsy.org (2008-08-28).
English
Deutsch
Hi,
I have 42 items and SEM is the tool to be used.As the thumb rule says, 10 samples for each items, i need 420. Here I need who has proposed the thumb rule and kindly explain in detail what is the procedure, ie, how to work with the thumb rule. Alo state me the authors and articles where I can refer so as to defend my sample size determination for SEM.
Kindly explain the workouts in detail and cite me the author and article where the sample size for SEM is worked out.
Hi!
For starters, you might want to take a look at the Statnotes and any introductory textbook on SEM, e.g.: Byrne, B. (2008). Structural Equation Modeling with AMOS. New York: Psychology Press.
Timo
Does anyone know of any work to extend this to multilevel models?
Jeff
I'm sorry, I haven´t worked with multilevel models yet.
Timo
I received the following error message. Can you help me find what is wrong?
Error # 4095. Command name: exe
>The transformations program contains an unclosed LOOP, DO IF, or complex file
>structure. Use the level-of-control shown to the left of the SPSS commands to
>determine the range of LOOPs and DO IFs.
>This command not executed.
Katherine
Please send me your syntax file by email. There seems to be missing period. Which version of SPSS are you using?
Timo
Sorry for my delay in responding. I became ill.
I started again, taking special care not to exclude any periods. I got another message that I realized, from reading one of your other answers to another post, that I had the RMSEA values reversed. I made H0 smaller than H1 and it worked. However, I am concerned that I am misinterpreting something because it doesn't make sense to me that a larger RMSEA would be the H1. Further concerning me is the suspiciously small sample size I am getting for H1 RMSEA=.05, H0 RMSEA=15, which is 11.33. Changing them to .05 and .08 respectively, yields an estimated N of 66.
Am I thinking about this incorrectly? Thank you for your responsiveness.
Katherine
Hi Katherine,
I assume you are referring to the first script on the page to determine the required sample size based upon a RMSEA difference (MacCallum et al., 1996)?
The RMSEA(H0) indicates the RMSEA for models you would accept (e.g., with RMSEA(H0) set to .05, the following H0 is tested: RMSEA <= .05). The RMSEA(H1) is an arbitrary value higher than the RMSEA(H0) for bad fitting models (e.g., .08). The difference between the two RMSEA values can be interpreted in terms of an effect size as in common power analysis. For details see the source article (MacCallum et al., 1996).
A RMSEA(H1) of .15 does not make much sense in my opinion as this indicates a rather bad fitting model. I would say, RMSEA values of no more than .08 are indicative of acceptable model fits. If the calculations result in a small sample size this might be due to an excessive number of df? MacCallum et al. (1996) provide a table for selected df, that result in sample sizes between N=782 (df=10) and N=132 (df=100).
Timo
can we use the same procedures for finding the sample size for multi-group CFA?
The code can be adapted to be used for multi-sample SEM (see MacCallum et al., 2006, p26). However, the syntax on this page does not implement this, so far.
Timo
I get the following error message:
Warning # 92
>The limit of MXWARNS warnings in this data pass has been printed. Further
>warnings have been suppressed.
Degrees of freedom (df): 1034
Significance level (Alpha): .05
RMSEA (H0): .05
RMSEA (H1): .01
Required N for test of close of fit (McCallum et al., 1996):99950.00
could you help me?
thanks.
marialuisa
You misspecified the two RMSEAs for H0 and H1 - RMSEA (H1) must be larger than RMSEA (H0).
Timo
Using Kim's calculations for sample size estimation, is the sample size required to achieve adequate power for RMSEA equivalent to what would be required for CFI, when using the two goodness of fit indices with the same model and data set = or shoudl I also estimate the sample size required for CFI?
thank you
Roberta
For a discussion of this see Kim (2005) on page 383. He states that different fit indices probably will produce different magnitudes of proposed sample sizes as all fit indices are measuring different levels of misspecifications.
So to answer your question: it seems advisable to calculate the required sample size for each fit index used.
Timo
Is an adequate POWER enough to justify sample size or do I also have to adhere to rules of thumbs such as that N must be at least 10 x the number of observed variables.
I would say, this is a matter of opinion. There was a rather extended discussion on that topic in 2007 in Personality and Individual Differences, which reflects the points of view of different SEM-experts (e.g. by Barret, Bentler and Goffin).
Personally, I try to consider a combination of both - these power estimations as well as rules of thumbs (e.g. N of at least 200 etc.).
thanks!
Why it does not work?
I try to calculate the power or the required sample size of my data in SPSS with above spss syntax however it did not work.
It starts with blow mistakes and reapets it...
compute #df = 119. /* Degrees of freedom */
>Error # 100. Command name: compute
>This command is not permitted before the beginning of file definition
>commands.
>This command not executed.
compute #alpha = 0.05. /* Significance level */
As stated in the SPSS comments, you have to create at least one dummy variable before running the syntax. SPSS won´t run "compute"-commands (even on temporary variables) on an empty dataset.
Timo
Post new comment