Effect size for Analysis of Variance (ANOVA)

October 31, 2010 at 5:00 pm 14 comments

If you’re reading this post, I’ll assume you have at least some prior knowledge of statistics in Psychology. Besides, you can’t possibly know what an ANOVA is unless you’ve had some form of statistics/research methods tuition.

This guide probably not suitable for anybody who is not at degree level of Psychology. Sorry, but not all posts can benefit everybody, and I know research methods is a difficult module at University. Thanks for your understanding!

Recap of effect size.

Effect size, in a nutshell, is a value which allows you to see how much your independent variable (IV) has affected the dependent variable (DV) in an experimental study. In other words, it looks at how much variance in your DV was a result of the IV. You can only calculate an effect size after conducting an appropriate statistical test for significance. This post will look at effect size with ANOVA (ANalysis Of VAriance), which is not the same as other tests (like a t-test). When using effect size with ANOVA, we use η² (Eta squared), rather than Cohen’s d with a t-test, for example.

Before looking at how to work out effect size, it might be worth looking at Cohen’s (1988) guidelines. According to him:

  • Small: 0.01
  • Medium: 0.059
  • Large: 0.138

So if you end up with η² = 0.45, you can assume the effect size is very large. It also means that 45% of the change in the DV can be accounted for by the IV.

Effect size for a between groups ANOVA

Calculating effect size for between groups designs is much easier than for within groups. The formula looks like this:

η² = Treatment Sum of Squares
Total Sum of Squares

So if we consider the output of a between groups ANOVA (using SPSS/PASW):
(Sorry, I’ve had to pinch this from a lecturer’s slideshow because my SPSS is playing up…)

Looking at the table above, we need the second column (Sum of Squares).
The treatment sum of squares is the first row: Between Groups (31.444)
The total sum of squares is the final row: Total (63.111)

Therefore:

η² = 31.444
63.111

η² =  0.498

This would be deemed by Cohen’s guidelines as a very large effect size; 49.8% of the variance was caused by the IV (treatment).

Effect size for a within subjects ANOVA

The formula is slightly more complicated here, as you have to work out the total Sum of Squares yourself:

Total Sum of Squares = Treatment Sum of Squares + Error Sum of Squares + Error (between subjects) Sum of Squares.

Then, you’d use the formula as normal.

η² = Treatment Sum of Squares
Total Sum of Squares

Let’s look at an example:
(Again, output ‘borrowed’ from my lecture slides as PASW is being mean!)

So, the total Sum of Squares, which we have to calculate, is as follows:

31.444 (top table, SPEED 1) + 21.889 (top table, Error(SPEED1)) + 9.778 (Bottom table, Error) = 63.111

As you can see, this value is the same as the last example with between groups – so it works!

Just enter the total in the formula as before:

η² = 31.444 =  0.498
63.111

Again, 49.8% of the variance in the DV is due to the IV.

And that’s all there is to it!

Just remember to consider the design of the study – is it between groups or within subjects?

Thanks for reading, I hope this helps!

Sam Eddy.

About these ads

Entry filed under: Statistics & Research Methods. Tags: , .

Colours in Consumer Psychology. Theories of Cognitive Development: Lev Vygotsky.

14 Comments Add your own

  • 1. xigu  |  February 12, 2011 at 11:34 pm

    Thank you so much for this wonderful information about how to calculate effect size for ANOVA. You have made it simple and easy to understand. Thanks again!

    Reply
  • 2. Kayleigh  |  April 2, 2011 at 8:20 pm

    Thank you so much for this! This has been so much help!

    Reply
    • 3. Sam Eddy  |  April 2, 2011 at 10:44 pm

      I’m so glad that this has helped! Thanks for your comment; it motivates me to write more when I know people are benefiting from my writing. Good luck with your studies, Sam.

      Reply
  • 4. James  |  November 6, 2011 at 7:00 pm

    ANOVA = ANalysis Of VAriance (not VARiables). Sorry to nit-pick! Also, a one-way ANOVA produces the same result as a t-test of independent samples, does it not? Are effect sizes in such a case equivalent?

    Reply
    • 5. Sam Eddy  |  November 6, 2011 at 9:27 pm

      Oh wow, I have no idea how I missed that. I knew it was variance – I was being careless obviously. I’ll correct that in a moment.
      T-tests are totally different, as they measure the difference between the mean of two groups. An ANOVA, as the name implies, is looking at the difference between variance in two or more groups. Follow up tests will usually involve conducting a t-test, but as such the effect size is difference. Eta squared (or η²) is for ANOVA, whereas for t-tests you will need to use Cohen’s d.

      Hope that helps,
      Sam.

      Reply
  • 6. James  |  November 7, 2011 at 12:21 am

    I’m not sure how different they are in fact: see http://sportsci.org/resource/stats/ttest.html
    Especially: “So t tests are just a special case of ANOVA: if you analyze the means of two groups by ANOVA, you get the same results as doing it with a t test”. ANOVA of course is the only approach to use when dealing with more than two groups, but otherwise…

    Reply
    • 7. Sam Eddy  |  November 7, 2011 at 12:37 pm

      This is true, an ANOVA can be used to measure the means – as the article implies it might be more appropriate to name it “ANOVASMAD”. However, although the models may be similar, they are ultimately two different tests which are used for different things. As you probably know, different effect size/power tables are used to calculate ANOVA scores which leads to different sample sizes etc. It’s not accurate to categorise them together, although yes, an ANOVA is a more powerful way to test means by implementing variance.

      Sam.

      Reply
  • 8. Phil  |  November 9, 2011 at 5:12 am

    If you have multiple variables in your w/in subjects ANOVA, would you just then add up all the SS’ + all the errors + b/w groups error to get your SStotal?

    Reply
  • 9. Terri  |  March 5, 2012 at 7:53 pm

    I know you refer to Cohen, 1988 when you give the value of a high, medium, and large effect size. But they seem to be off. I have only seen these:

    Here is a table of suggested values for low, medium and high effects (Cohen, 1988). These values should not be taken as absolutes and should interpreted within the context of your research program. The values for large effects are frequently exceeded in practice with values Cohen’s d greater than 1.0 not uncommon. However, using very large effect sizes in prospective power analysis is probably not a good idea as it could lead to under powered studies.

    small medum large
    t-test for means d .20 .50 .80
    t-test for corr r .10 .30 .50
    F-test for regress f2 .02 .15 .35
    F-test for anova f .10 .25 .40
    chi-square w .10 .30 .50

    Reply
    • 10. Sam Eddy  |  March 14, 2012 at 5:16 pm

      Thanks for adding this useful comment.
      I have only added the guidelines as provided by the statistics module of my university. I’m unsure where you got those figures (from the original article?), but I will certainly look into it.

      Reply
  • 11. alexandra  |  June 6, 2012 at 8:18 pm

    How would you calculate effect sizes from a Mixed-Design ANOVA output?

    Reply
    • 12. ruthcumming  |  August 16, 2012 at 12:58 pm

      That’s my question too! Any ideas on an answer?

      Reply
  • 13. Brian Chisenga,UNISA STUDENT  |  August 11, 2012 at 4:53 am

    The information is very helpful in research.Thank you.

    Reply
  • 14. Charles Silber  |  September 16, 2012 at 6:21 pm

    Thank you so much. Effect size is used in multi level analysis

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Trackback this post  |  Subscribe to the comments via RSS Feed


Welcome to PsychoHawks

Like the new logo? ;)

To subscribe, simply enter a valid e-mail address! You'll be updated as soon as posts are released, and gain access to exclusive subscriber only content!

Join 198 other followers

The Archive

Sam’s Twitter

Make a donation.

By making a donation, you can help the development of the blog. This will keep it free, and help me move it from WordPress to a real domain. Every little helps!

Thanks to all the wonderful readers.

  • 712,932 views and counting!

Follow

Get every new post delivered to your Inbox.

Join 198 other followers

%d bloggers like this: