Meta-analysis

Meta-analysis was designed as a method of reducing the threats to validity that often arise as a result of small sample sizes. When sample sizes used for a particular experiment are too small, it becomes possible for errors to enter the data and cause it to become skewed or biased. Meta-analysis involves the survey and investigation of data from a number of related studies. Such analysis is usually advantageous in its ability to produce more accurate data.

One of the problems that arise when conducting a review of studies comes from the methods chosen to analyze data. The usual methods of integrating research that has been previously done often prove unable to cope with the growing amounts of research with which some researchers have to deal. Meta-analysis helps eliminate this problem. It also delves into the quality of the research being evaluated, in order to reduce the problem of citing research without proper examination of the conclusions and the methods used to reach these. It also prescribes methods for researchers to weigh adequately all the evidence whether it is for or against their own preconceived ideas or preferences, thereby reducing the bias of research.

Problems with internal validity arise as a result of such practices as non-randomization, small sample size, discontinuation of the studies by participants (drop-out), the occurrence of significant historical events during a study, lack of control groups, and the problem of extreme results versus the regression effect toward the mean (Losh, 2002). In order to improve the internal validity of research, meta-analysis covers a wide array of studies that serve to combat each of these problems in the following ways.

Because meta-analysis deals with a large number of individual studies, problems regarding small sample size can be diminished as the number of participants within the study now becomes the aggregate of all those who participated in the individual studies. As a result, meta-analyses “have more power to detect small but clinically significant effects” (Davies & Crombie, n.d.). Biases in the data that arise from non-randomization and problems with lack of control groups can also be diminished because of the practices of meta-analysis experts in choosing carefully which studies to include in their research. When conducting this type of research, it becomes crucial to choose primary research that is “a complete, unbiased collection of original, high-quality studies that examine the same […] question” (Davies & Crombie, n.d.). Researchers who adhere to this practice scrutinize the methodologies of the different studies and remove those that contain major control and randomization flaws.

The large number of studies used in meta-analysis also combats the problems or biases that may arise from such phenomena as regression toward the mean. When studies are done (or tests taken) it is often the case that a small percentage of participants score exceptionally high or low. It is often the case, too, that when/if retakes of these studies are done, these same exceptional scorers either increase or decrease their scores, taking them closer to the mean. With a large body of studies taken in meta-analysis, the effects of these exceptions and regressions can evened out, so that the study gives a more accurate and statistically valid picture of the problem/issue being examined.

As external validity is related to the ability to generalize results across populations, though similar studies must be chosen for meta-analyses, the researcher may be careful to include ones that contain a wide variety of subject types. This will reduce the effects of population sensitization (familiarity with the processes of the test) as well as the likelihood of certain subject types to be (artificially) more inclined to one outcome or another based on the demographic of that particular group. The more inclusive the criteria for the participants, the more widely generalizable will the meta-analytic study become (Davies & Crombie, n.d.).
References

Davies, H.T.O. & I. K. Crombie. (n.d.). “What is meta-analysis?” Evidence-based Medicine.      Howard Medical Communications. 1(8).

Losh, S. C. (2002). “Quasi-experiments, internal validity, and experiments II.” Methods of         Educational Research.  Florida State University. Retrieved on January 29, 2007 from http://edf5481-01.fa02.fsu.edu/Guide4.html