Post-hoc comparisons for Friedman test
As a non-parametric test of mean differences for dependent samples SPSS implements the Friedman test. However, SPSS does not provide post-hoc tests to decide which groups are significantly different from each other. Schaich and Hamerle (1984) as well as Conover (1971, 1980) propose such post-hoc tests that are based upon the mean rank differences of the groups. These procedures are detailed in Bortz, Lienert and Boehnke (2000, pp. 275).
Note: Since version 18 SPSS provides comparable post-hoc tests by default.
Literature
- Bortz, J., Lienert, G. & Boehnke, K. (2000). Verteilungsfreie Methoden in der Biostatistik. Berlin: Springer. ISBN: 3540675906.
- Conover, W. J. (1971, 1980). Practical nonparametric statistics. New York: Wiley. ISBN: 0471168513.
- Schaich, E. & Hamerle, A. (1984). Verteilungsfreie statistische Prüfverfahren. Berlin: Springer. ISBN: 3540137769.
Download
Post-hoc comparison for Friedman test (SPSS syntax file)
Post-hoc comparison for Friedman-Test (SPSS data file)
English
Deutsch
Thank you so much. this was a huge help.
Hi Timo,
Thanks for posting this! I've had a look at it, and there's something I can't quite make sense of. I'm running SPSS 17.0. I've ran Friedman on my data (7 variables, 87 p's), which was significant, then ran your posthoc, as well as Wilcoxon paired analyses on each possible pair (21 in total) to compare results. The Wilcoxon paired analyses found only two pairs to be not significant (after .05/21 Bonferroni adjustment), but your analysis found one pair not significant using the Conover critical rank, and 12 pairs not significant using the Schaich & Hamerle critical rank. Some of these non-significant results in your analysis are coming up as extremely significant (p<.000000001) in the Wilcoxon analysis. I'm not sure why I'm finding such a vast difference in what is and isn't significant in to apparently similar analyses. Any suggestions?
Hi,
sorry I have missed your post. Unfortunately, I don't have an answer for you. Generally, the Conover approach is less conservative than the Schaich & Hamerle approach. You are more likely to get significant results with the former. I am not aware of simulations comparing these alternatives. So I cannot recommend one or the other.
Timo
Hallo,
I have got a similar problem. If I use the formula from Schaich&Hamerle (1984) or Conover I get different significant results than SPSS.
Furthermore I am confused because the SPSS-Help shows a formula for this test with a test statistic (z)with the standardized normal distribution whereas Schaich&Hamerle suggest a Chi²-test statistic.
Can anyone explain me how to explain these difference?
Or can you suggest any literature which shows what kind of formula or test SPSS uses fot the multiple comparions (how can I calculte the statistic comparative value)?
Thanks a lot for your help.
Saran
Hi,
SPSS uses a different post-hoc test than those by Conover or Schaich & Hamerle. I don't have a reference for the test implemented in SPSS. But your find the exact formula in the SPSS help - search for "Pairwise Multiple Comparisons (nonparametric tests algorithms)". SPSS simply calculates the mean rank difference and computes a z-standardization. On the other hand, the Schaich & Hamerle approach is based on the Chi2-test statistic from the Friedman test.
Timo
Timo
Hi Timo,
I am using the latest version of SPSS that contains the built-in post-hoc analysis for the Friedman test.
I was wondering if you would be able to specify what type of post-hoc test this is exactly (ie. is there a name for it).
Also, is this test an unplanned comparisons of mean ranks?
Unfortunately I do not have access to any of the books cited above and I need the details for a section of my thesis.
Any information you could provide would be greatly appreciated.
Thanks!
RS
Hi,
I am not aware of a specific name for this type of test.
> Also, is this test an unplanned comparisons of mean ranks?
Yes.
Timo
Thanks! This helps a fair bit.
There is also an R implementation for it here:
http://www.r-statistics.com/2010/02/post-hoc-analysis-for-friedmans-test...
Thanks for providing the link.
Timo
Hi Timo, thanks for the script.. seems to work perfectly
I just had one question which is more about the statistical asepcets. The critical values by Hamerle or Conover seem to be quite different: for example 2,30 vs. 1,10 respectively. Which one do you prefer/reccomend using? because the Conover values tend to be lower, so if I "need" to demonstrate significant differences I'd probably use that one!
Thanks for your help
Sorry, but I don´t have a recommendation. I don´t have any study at hand comparing these two procedures.
Timo
Hi Timo,
Thank you for sharing this very useful information :)
I've have now done several Friedman test with post hoc on my data. However, I'm a little confused about how I should interpret the results of the post hoc test, i.e. pairwise comparisons.
The thing is, that in some of them there are significant pairwise comparisons and in others there are none at all. So, I'm wondering, when there are no significant pairwise comparisons, how can I then found out, which samples are causing the significant result of the Friedman test in the first place?
I'm sorry, if it's a stupid question, but I'm not yet so hardcore in statistics, which is why I really appreciate blogs like this.
Kind regards,
Caroline
Hi,
well, the point of post-hoc tests is to identify the samples that differ significantly from each other. If, for example, you compare four groups (A, B, C, D) and your Friedman test reaches significance, it demonstrates that at least two of the four samples are signficantly different from each other. If in the second step your post-hoc tests are signficant for e.g. the comparision B vs. D and not significant for B vs. C, then you know that B and D differ significanly from each other but B and C do not.
Does this answer your question?
Timo
Hi Timo,
Thanks a lot for your fast reply!
However, my problem is more that when I do a Friedman test, which gives a significant outcome (P = 0.001) and hereafter look at the pairwise comparisons from the post hoc test, there are no pairs that differ significantly from each other (no yellow fields). In other words, most pairs have the value 1.000 (in the Adj. Sig column) and some pairs a lower value (0.094, 0.146, 0.192 and 0.606) but none are in fact really significant at the 0.05 alpha level.
How can that be? I mean, how can the Friedman test give a significant result, if no pairwise comparisons differ significantly from each other?
Thank again!
Caroline
Hi,
this might be a problem of a lower power of the post-hoc tests (I don´t really know) and could also be a consequence of the adjustment of the significance level to account for an increase in alpha error (particularly if you compare a large number of samples). You probably need a larger sample size to get a significant result.
Timo
I see...
The unadjusted significant values are significant, so do you think I could report them or should I state in my report that the sample sizes are too small to conduct a valid post hoc test?
Caroline
Well, generally I would report the adjusted signficance values. But you might include the unadjusted values as well for the important comparisons and inteprete them cautiously.
Timo
Thank you Timo. I will do that.
It's great that someone like you can help when your own statistics professor can't! :)
Cheers,
Caroline
Hi Timo,
Thanks for this info! I was hoping to look at your example data set, though, but when I click on the link, the page with the SPSS syntax comes up. Any way you could post the data, so I can make sure I'm setting up my code correctly? When I try right now I do get output (ranking lists), but I'm not sure how to interpret it, and there are no associated X2 or p values (and so I'm not really clear that the script is working correctly for me).
As an aside, I tried to run a Friedman ANOVA with post hoc analysis in SPSS 18, but can't seem to run the analysis - I keep getting an error that says that I must enter more than 2 fields with continuous data... yet I have entered those fields. Just wondering if this is a known bug that there is an upgrade for, or if I'm doing something wrong?
Thanks for your help!
Hi Tonya,
thanks for the information. I just fixed the link to the SPSS demo file.
@interpretation: You have to compare the rank differences in the table with the critical value by Schaich and Hamerle (1984) or Conover (1971). If your differences are higher than the critical value, the respective difference is significant.
@post-hoc in SPSS: See the reply by Maggie - you have to define the variables as 'Scale'.
Timo
I also have the same problem: 'I tried to run a Friedman ANOVA with post hoc analysis in SPSS 18, but can't seem to run the analysis - I keep getting an error that says that I must enter more than 2 fields with continuous data... yet I have entered those fields. '
Would love to know what I am doing wrong!
Thanks for all the helpful information on your site.
Just worked out that the data has to be identified as 'scale' rather than 'ordinal' for this to work.
Hi Timo,
Thank you so much for the detailed instructions on how to get the Friedman test and post-hoc comparisons through "point-and-click" with PASW 18. I don't think I would have found it by myself, but now I got the results I needed and they look very clear.
Hi Timo,
I have run a Friedman on PASW 18 and have the pairwise comparisons (thank you for all your instructions!).
My question is, how do I report the pairwise comparisons: are they planned or post hoc, does the test have a name (e.g., is it Wilcoxon signed-ranks?), and, what is the test statistic(e.g., χ2)?
I appreciate your help, thanks!
Halle
Hi,
the tests are simple pairwise comparisons (traditional post-hoc tests). The tests were proposed by Conover and Schaich/Hamerle (see the references for more details). There are no special name for them.
The test statistic for the test by Conover is X2-distributed and the one by Schaich and Hamerle is a t-statistic.
Timo
Hi Timo,
I have the same question about the post hoc pairwise comparison. Searching the help section of SPSS did not allow me to find exactly which test statistic to report for these post hoc tests. For each pair of variables, I can a "test statistic", a "Standard Error", and "Standard test statistic", a "significance", and an "adjusted significance".
I know I should report the test statistic, but should I write, for example, t = -1.23, SE = .31 p < .01 (i.e., a t statistic?)
Thanks,
Marie-Hélène
Hi,
I think the test uses a z-standardized variable. So instead of t you should use z=-1.23, p<.01. However the test statistic you provided (-1.23) corresponds to a p-value of .19. Make sure you report the standardized(!) test statistic (third column in the SPSS output): -1.23 / .31 = 3.97.
Timo
Hi Timo,
I was looking on the internet for a post hoc analyses for a friedman test and found your solution.
However I cannot run the syntax.
First I get a warning: missing command terminator.
My second quesions is: which modifications should i make to run the syntax with my own data?
Thanks Martijn
> First I get a warning: missing command terminator.
You´ve got a missing period '.'in one line.
> which modifications should i make to run the syntax with my own data?
Modify the two passages titled "Configuration 1" and "Configuration 2" in the syntax.
Timo
Sorry, only just noticed that they are at the beginning of the page.
Hi Timo,
thanks for sharing this online! It's a great help!! Unfortuantely I seem to be unable to get the script work for me ... Would you mind to have a look at my modified syntax file and push me towards the right direction??
Any help would be much appreciated,
Olaf.
Hi,
thanks for sharing this, would never have found this hidden post-hoc tests otherwise..
But some question remain, what test are conducted? Is there any correction applied? Like Bonferroni? What's the difference between Sig. and Adj.Sig.? Does Adj mean adjusted and that's the corrected value to use?
Sorry for all the questions, In felt so good with parametric tests and noww everything looks so different with non-parametric tests...
Sonja
----------
Hi,
danke für den Tipp, wie soll man die post-hoc tests denn finden wenn man das nicht gelesen hat..
Trotzdem noch ein paar Fragen: Was sind es denn genau für post-hoc Tests? Ist irgendeine Korrektur für den paarweisen Vergleich schon mit drin? So was wie Bonferroni? Und was ist der Unterschied zwischen Sig. und Adj. Sig.? Steht Adj. für Adjusted/Angepasst und das ist der Wert den man hernehmen muss um Signifikanz zu bestimmen?
Sorry für all die Fragen, jetzt hab ich zwar die ganzen parametrischen Tests und Ergebnisse versatnden aber für nicht-parametrische ist ja alles wieder anders...
Sonja
Hi Sonja,
as to the sort of test - the post-hoc tests are based on the standardized mean rank differences of the two samples. The exact formula is provided in the SPSS help under "Pairwise Multiple Comparisons (nonparametric tests algorithms)".
The column labeled "Sig." lists the uncorrected significance values. As you noted correctly, you should not use these as the multiple comparisons lead to an alpha cumulation. The column labeled "Adj. Sig" are the Bonferroni adjusted significance labels, that are calculated as p*K*(K−1)/2 with p as the uncorrected significance level and K as the number of comparisons. Use these values for you analysis.
Timo
Thanks Timo,
It worked!
Hi, Timo
You wrote that in SPSS version 18 post hoc for friedman is available? I installed the 18 version today (called PASW 18?) but could not find any possible way to do post doc? Am I looking at the wrong place?
Thanks
I forgot, of course there is no SPSS 18. Since it has been acquired by IBM, it is called PASW (but it is still the same software).
Timo
Hi Marcus,
the posthoc test is well hidden in SPSS 18:
1) Choose "Analyze" -> "Nonparametric tests" -> "Related samples"
2) Choose "Settings" and select "Friedman´s 2-way ANOVA by ranks" in the bottom, then select the option "Multiple comparisons".
3) In the output window, open the "Model Viewer" by double-clicking on the table.
4) Choose "View" -> "Pairwise comparisons" and voilá...
I hope this helps,
Timo
habe es heute noch einmal probiert und es klappte doch. Danke Stef
Hallo Timo,
meine Stichprobengröße (varcnt) beträgt 17, die sample size 16 Tiere. Diese Werte plus die Pfadangabe habe ich im syntax file angepasst, jedoch zeigt der output unzählige fehler.
Des Weiteren zeigt der Editor in Zeile 42-43 die worte compute in rot an. Ich nehme, an damit hat es ein Problem. Was mache ich nur falsch, bzw. was muss ich ändern?
Viele Grüße und 1000 Dank im Vorraus.
P.S.Ich habe die SPSS Version 18 und fand nach langem Suchen keine automatisch angebotenen post hocs. Kurz um ich bin am Ende meiner Nerven.
Hallo Stef,
so eine Ferndiagnose ist schwierig. Schick mir doch die Daten und das Syntaxfile per Email. Dann sehe ich es mir an.
Den Posthoc Test findest du in SPSS 18 unter:
1) Analyze -> Nonparametric tests -> Related samples
2) Unter "Settings" wähle rechts unten "Friedman´s 2-way ANOVA by ranks" und die Option "Multiple comparisons" aus.
3)Im Output öffnet ein Doppelklick auf die Tabelle den "Model Viewer". Unter "View" -> "Pairwise comparisons" findest du dann die Posthoc Vergleiche.
Usability ist nicht gerade eine Stärke von SPSS... :-)
Timo
Hi Timo,
after modifying the file and variable names the script runs without errors, but it doesn't produce any output. What am I doing wrong?
valjah
Hi,
I first got nothing in the output w/o error messages..
Then I tried only "integers" in the data (since only ordinal data counts) and it works!
Best,
Ron
Well, if there are no errors, than it does not seem to be a problem with the script itself but rather your SPSS settings. Check your SPSS options to display all commands in the output. Unfortunately, I don´t have any other suggestions...
Timo
i can't for the life of me seem to run this! it seems like there's lots of errors, even after modifying...
Thank you so much Timo. After editing the syntax for my dataset the posthoc tests work. Great stuff!!!
I am surprised by the totally useless file in german. I suggest next time to write it with cuneiform characters. Thanks for NOTHING.
You could have simply asked for translated instructions. I'd be happy to provide them.
But I guess, it's easier to rant about free stuff, you find on the internet, than being a little bit more constructive.
Timo
Hi Timo
Is it possible to get a copy of your syntax in english please? I would like to see what results it gives.
Natalie
I've sent them by email to you.
Timo
Kommentar hinzufügen