In this section we turn to Max & Onghena’s (“Some issues
in the statistical analysis of completely randomized and repeated measures
designs for speech, language and hearing research”, JSLHR 42, 261-270, 1999) second critique of common practice in doing ANOVAs:
the fact that we’ve been ignoring the problem of possible sphericity violations in repeated measures designs. StatView does not deal with this problem, and thus
it presents a limitation on using StatView to do your repeated measures analyses.
Sphericity
ANOVA assumes homogeneity of variance across
conditions but is robust against violations of this assumption IF the data
are uncorrelated. Heterogeneity of variances in correlated data gives
a positive bias in F tests (the critical value of the F is too low). Sphericity
means that data are uncorrelated; violations of sphericity require strict
homogeneity of variance. If there are only 2 levels of a factor, this
problem of correlation doesn't arise; but if there are 3 or more levels you
have to guard against it. StatView does not correct for sphericity violations
in RM analyses. Thus you can only reliably use StatView for RM designs
in which each factor has only two levels. If a design has more than
2 levels in any factor, then you should not use StatView. For example,
StatView's own sample Wine file, with 7 levels in its 1 factor, should not
be analyzed with StatView!
It
can be seen that StatView does not correct for sphericity violations by
comparing the output from StatView with the sample outputs in Max & Onghena’s
article. Use the file m&o-2.xls
with StatView’s RM analysis, as described above, and compare the result
with Table 1 from the article (below). It is
the same as the first row in the table, the “ANOVA with uncorrected df” analysis. This row is included by Max & Onghena to illustrate
the WRONG result (in that it overestimates the significance).
Some
options and partial solutions
Option
0: Of course if you run the test in StatView and there’s
no significant effect, then you’re ok – we’re talking here about over-estimating
significance and getting a false positive; if you get no effect even with
such a method, then you know there really is no effect, and there’s no need
to go to special lengths to continue to fail to get an effect.
Option
1: Test for sphericity
violations, i.e. test for correlations among variables.
StatView provides Bartlett’s test of sphericity.
A resulting high chi-square value with a low p value is BAD, but if
the data are uncorrelated you’re probably OK. (However,
Max & Onghena are dubious about the integrity of such tests in the first
place, so even this isn’t clear. We’ll see proof
that this is a waste of time with our later sample analysis.) But in any event note that StatView’s test doesn’t
do anything as a result of a significant correlation, or tell you what to
do to correct for the violation. All the test
does is tell you if there’s a problem (maybe).
how to do Bartlett’s test in StatView (though generally
a waste of time):
Data
cannot be in a compact variable: you need all the levels sitting there like
variables (the use of the term “variable” here is somewhat misleading). Under Analyze – Correlations, select Bartlett’s test;
drag all the levels of your repeated measure into the box.
Do this with Wine Tasting file (don’t make a compact variable). Data are correlated, i.e. analysis above was invalid.
Option
2: Use StatView’s MANOVA
instead of ANOVA; a significant result from this test is valid; but this procedure
is very conservative, i.e. unlikely to show significant differences.
how to do MANOVA in StatView:
First,
try a regular MANOVA (simultaneous analysis of 2 dependent variables): Open
their Exercise file, and under Analyze – ANOVA, select MANOVA. Ignore “independent variable”, drag the variables
into “dependent variables” and “factors” as appropriate.
This gves all the separate ANOVA tables, plus the other stuff.
Now
try a MANOVA as a substitute for ANOVA with a repeated measure:
Do some analysis so that you
have the Analysis Browser open (to the left) -- e.g. the file
3-level-compact -- do RM ANOVA (select View - Variable
Browser if it’s not already there); then under Analysis Browser
- ANOVA, double-click MANOVA Tables (you can’t use the regular menu for
this as it defaults to factorial design; but the browser gives you the choice);
be sure to select Repeated Measures design; AND use the Variable Browser
to declare your compact variable as dependent (note that you can do this
after the analysis fails to run – it will run after you do this)
You
use the Wilks’ Lambda result from MANOVA Tables (though here all are same).
If
you do this analysis on the Max & Onghena data, you’ll see that you
do get the “MANOVA (Wilk’s lambda)” result in row 3 of their table. Note that the difference is not even close to significant
this way – this is a very conservative test!
Another catch to this analysis is that it requires more subjects than
there are repeated measures (e.g. if you have 5 factors in your experiment
then you need at least 6 subjects).
Option
3: use SPSS
– it allows you to correct the df according to the degree of correlation
in the data – see next
section
