As you are all very fully aware, we’ve spent the last 2 weeks doing about t-tests, which has been positively riveting. We’ve looked at between subjects t-tests and within groups t-tests, and so I thought I’d base my first blog of 2012 on the two different designs and which is better. Now try not to explode with excitement.

Firstly I’ll talk about the independent measures design. This is when each condition uses different participants. Because of this, our results don’t suffer from order effects, such as fatigue and boredom.

However, this design can lead to validity problems such as maturation (the effects of a treatment, for example), the participant gets used to being tested, or participants on one condition may talk to participants on another about the experiment.

The biggest downfall of IMD is that it suffers greatly from individual differences. No two people are the same; some people may find the task(s) in the experiment easy, and some may find it more difficult. For example, if it was a memory task, someone with poor memory would struggle more than someone with a reasonably good memory. Also, if the person tested is a psychology student, they may be able to figure out what the researcher is testing for, as opposed to someone who has not studied psychology, and so this could lead to demand characteristics.

So IMD doesn’t sound all that great, so let’s look at the alternative. Repeated measures design uses the same participants in all conditions, and so this removes individual differences as a potential confounding variable. Also, because the same participants are used in all the conditions then this means fewer participants are required, which can save on time and money.

However, RMD has its fair share of problems; it may not be possible to test all participants twice or more. Unlike IMD, because participants are in each condition, they could start to suffer from the order effects that IMD manages to avoid. But this is not the end of the world; these can be minimised using counterbalancing.

These order effects occur when people behave differently because of the order of the conditions, for example, performance may be enhanced by a practice effect, or it could be made worse from fatigue and boredom. Counterbalancing consists of some participants doing the conditions in one order, and others doing them in a different order. This randomises the order effects.

After looking at both designs, it is clear to see that repeated measures design comes out on top. Although both designs have their strengths and weaknesses, it is only repeated measures design that can counter their biggest weakness.

Homework for My TA – 10/02/12 « psucd3(17:33:15) :[…] 3) https://scarlettrose23.wordpress.com/2012/02/05/independent-measures-design-vs-repeated-measures-desi… […]

Homework for TA « vanilla85(16:29:18) :[…] https://scarlettrose23.wordpress.com/2012/02/05/independent-measures-design-vs-repeated-measures-desi… […]

vanilla85(16:18:53) :I think that there is no better or worse measure design. I believe that both are suitable for different studies. I agree that both designs have its own strengths and weaknesses but your post seems to be a bit biased towards repeated measure designs. You even mentioned ‘maturation’ as a weakness of an independent measure design. However, maturation is a factor typical for a repeated measures design (Gravetter & Horzano, 2009). Other factors, which were not mentioned, but are the disadvantages of repeated measures designs are:

• history (participants’ scores influenced by outside events in one treatment differently than in another treatment),

• instrumentation (change of measuring instrument that occur over time),

• regression (extreme scores tend to move toward the mean).

I would like to add that individual characteristics which are a problem occurring in the independent measures can be reduced by using a matched-subjects design.

Sources:

Gravetter, F. J., & Forzano, L. B., (2009). Research Methods for Behavioral Sciences. Wadsworth: Belmont.

cfredlevy(14:58:27) :I agree with your conclusion and there are a couple more factors that have been discussed in journals to explain the benefits of repeated measures. Maxwell and Delaney (2004) suggested that precision of scores are one of the greatest benefits. Other than counterbalancing, as you discussed, there is the benefit of individuals acting as their own control and therefore ignoring error between comparisons (Stevens, 2002). Maxwell and Delaney also highlight the benefit of the need for less participants. They suggested for a p=0.05 128 are needed for between and only 32 for within. This would save time and money, have a reduced error and still be able to keep validity/significance high. However repeated measures isn’t entirely without error due to potential carry over effects or fatigue (Tanguma,1999) but you wouldn’t expect these to be larger than error due to individual differences. Overall I agree that repeated measures have more benefits such as control over error, sample size & precision in experimental research.

http://www.aabri.com/manuscripts/08123.pdf <–review including below references

Tanguma, J. (1999). Repeated measures: A primer. In B. Thompson (Ed.), Advances in social

science methodology (Vol. 5, pp. 233 – 250). Stanford, CT: JAI Press.

Maxwell, S. E., & Delany, H. D. (2004). Designing experiments and analyzing data: A model

comparison perspective (2nd ed.). Mahwah, NJ: Lawrence Erlbaum Associated,

Publishers.

psucd3(18:08:10) :Counterbalancing is a good way to combat the repeated-measures design flaw (Gravetter & Wallnau, 2009). However if you have 4 different treatments (ABCD) then the number of counterbalances needed increase dramatically (24 different combinations). Which again will increase the number of participants needed, therefore Independent measures design is not that bad. Yes you need more participants (Gravetter & Wallnau, 2009), but there are 21,735 estimated (2008) people in Bangor alone (http://en.wikipedia.org/wiki/Bangor,_Gwynedd), all of these people could become participants. However, keeping one participant for 4 different experiment conditions could take hours of their time and lead to attrition (participants dropping out; Gravetter & Wallnau, 2009). Therefore an independent measures design is more appropriate when you increase the number of treatment conditions.

References

Gravetter, F. J., &Wallnau, L. B. (2009). Statistics for the behavioral sciences (8th ed.). Belmont, CA: Wadsworth.

Homework for my T.A. « danshephard(23:23:51) :[…] https://scarlettrose23.wordpress.com/2012/02/05/independent-measures-design-vs-repeated-measures-desi… […]

danshephard(23:15:28) :Great Blog.

Counterbalancing is a great advantage if it is used in a repeated measures design. It reduces the effects of order effects by distributing them across both conditions. The result of this is that they are similar for both groups and so balance the results out. This makes a repeated measures design seem far superior to an independent measures design. However repeated measures designs are significantly affected when participants drop out of the study, this is especially troublesome even though there are ways to reduce these effects Little, R.J. (1995) .There is another research design called a matched pairs design uses an independent measures design but the way in which individuals are allocated into a group is changed so that the results are less affected by individual differences. Participants are allocated too one of the groups by matching individuals on a scale such as intelligence or personality. A participant is then matched with another participant who has the same score these participants are then put in separate groups. This process helps ensure that the difference between the individuals within each of the groups is reduced. This means the results are more likely to be due to differences in the conditions rather than individual differences. Therefore this research design has some of the advantages of a repeated measures design but also all of the advantages of an independent measures design.

http://www.simplypsychology.org/experimental-designs.html

http://www.jstor.org/pss/2291350

Anonymous(23:12:24) :Great Blog.

Counterbalancing is a great advantage if it is used in a repeated measures design. It reduces the effects of order effects by distributing them across both conditions. The result of this is that they are similar for both groups and so balance the results out. This makes a repeated measures design seem far superior to an independent measures design. However repeated measures designs are significantly affected when participants drop out of the study, this is especially troublesome even though there are ways to reduce these effects Little, R.J. (1995) .There is another research design called a matched pairs design uses an independent measures design but the way in which individuals are allocated into a group is changed so that the results are less affected by individual differences. Participants are allocated too one of the groups by matching individuals on a scale such as intelligence or personality. A participant is then matched with another participant who has the same score these participants are then put in separate groups. This process helps ensure that the difference between the individuals within each of the groups is reduced. This means the results are more likely to be due to differences in the conditions rather than individual differences. Therefore this research design has some of the advantages of a repeated measures design but also all of the advantages of an independent measures design.

http://www.simplypsychology.org/experimental-designs.html

http://www.jstor.org/pss/2291350

Homework for my TA week 3 « My Statistical Blog(20:39:46) :[…] 2)https://scarlettrose23.wordpress.com/2012/02/05/independent-measures-design-vs-repeated-measures-desi… […]

Psucc0(19:25:04) :I have to agree with your conclusion that repeated measures comes out on top since it can overcome its weakness fairly easily and it is the most commonly used method in testing. (http://www-users.cs.umn.edu/~ludford/Stat_Guide/Paired_t.htm)

Using repeated measures also means that your variance will be less too since you don’t get the error caused by individual differences. As you pointed out the number of participants needed is less and this will also keep error and ‘noise’ to a minimum.

The only downside is one that applies to all t-tests is that you can’t re run t-tests on each individual group because this increases the error and through that the chance that you may make a type one or two error but that’s what ANOVAS are for!

limerickgirl20(21:54:10) :Hey great blog! But is repeated measures more reliable since it eliminates individual differences?! As you said their is no “perfect” measure that can make everything positive but there are forms that can help achieve these positives and maintain them. Although t-tests are relevant repeated measures designs tend to have more reliability!

Homework for my TA, blog comments week 2/3. « thewonderfulworldofstats(13:37:41) :[…] https://scarlettrose23.wordpress.com/2012/02/05/independent-measures-design-vs-repeated-measures-desi… […]

thewonderfulworldofstats(12:45:47) :As you have clearly pointed out, all forms of analysis methods have their good points and their bad points. It would be great if there were some form of method that we could use, where there were nothing but positives so that we weren’t faced with any problems when analysing our data. So imagine this, there would be no maturation, no practice effects. NOTHING! Everything would be perfect.

However, you have already pointed out that this is not the case, with this method.

So repeated measures eliminates the prospect of individual differences causing error because we use the same participants in each condition, unless of course you have a Jekyll and Hyde in your sample, then this may cause some problems Anyway because it reduces error, we are more likely to get a larger t value, which is all good (http://www.une.edu.au/WebStat/unit_materials/c6_common_statistical_tests/paired_sample_t.html).

T-tests are great until we want to test more than two conditions, but as you strongly concluded the repeated measures design is great for two conditions as it reduces the error, that we are likely to get in an independent t-test.