This is Volume 2 of a recurring series on interpreting research, meant to help educators better interpret the research out there on practices and interventions, so that we can all be more informed. Check out Volume 1 if you didn’t read it, which is on the anatomy of a research study and the different parts of a research article.
Now that you know the different sections of a research article, let’s talk about the different types of articles you might read, and the ways in which the experiments are designed in research articles.
There are two major types of articles you might read. The first is an analysis of a certain practice or intervention, in which the author(s) completes a study of the intervention/practice and report the results. The other major type of article is a meta-analysis, in which the author takes the research that others have completed, synthesizes the information and results from those studies into a larger analysis, and reports on the effectiveness of the intervention when all studies are combined. A meta-analysis is very useful for those of us searching for evidence-based practices because a meta-analysis has done the searching for us and synthesized all of that information for us. A meta-analysis will tell us if the practice has potential to work or not.
Let’s focus on the studies that are not meta-analyses, which is the majority of articles we will read. When reading these we have to think about a few things: Did the researchers include all of the information they should have? Is the sample size sufficient? What were the results, especially the effect size, if applicable? (See my next blog for more on effect sizes.) And, how was the research designed? I will focus on the last question for the rest of this entry.
Research design is important for us to determine if the intervention is something that actually has potential or not. The three major designs are called experimental, quasi-experimental, and single-subject.
Experimental studies are those that use random assignment. For example, in an experimental study if the researchers has 50 students that meet the criteria for their study, they would first randomly assign them to either an experimental group that receives the intervention or practice, or to a control group that does not receive the practice or intervention. Performance of students in each group is measured before and after the intervention to see what type of growth occurs and to compare the growth between the two groups. Random assignment helps to ensure that the two groups are similar in composition. One of the challenges of random assignment is that these experiments can happen in schools with children. Parents and educators may not be very happy about hearing that their child or student was placed in the control group and did not recieve the intervention when others did. Because of this researchers often use quasi-experimental designs in schools.
Quasi-experimental studies are those that do not use random assignment but still have a control group and an experimental group. For example, if there are two second grade classes, 2A and 2B, 2A receives the intervention and 2B does not making 2A the experimental group and 2B the control. In this case we would again compare performance before and after the experiment and compare growth. The major difference between quasi-experimental and experimental studies is the randomness factor, which is a disadvantage for the quasi-experimental designs since it is possible that there are differences between the two classrooms outside of the intervention that may impact growth.
A third type of research study is the single-subject study. These studies are exactly what they sound like, i.e., studies which focus on individual students in our case. In these studies the researchers will use an intervention or practice with a student and compare growth from before and after the practice or intervention. The methods and timing used will be written in detail, and will describe how long the intervention lasted and how it was delivered. Some single-subject studies will take baseline data, then implement an intervention, take post-intervention data and then stop the intervention. The process will then be repeated to see how student performance is affected with each delivery and removal of the intervention.
A meta-analysis should describe which types of studies it looked at and which qualified to be included in the meta-analysis. If the meta-analysis does not do this you should be cautious of the findings.
These are the three major types of studies you will see. When reading any study make sure you understand the design and that the study was done in one of these ways. Also, when you can, find a meta-analysis, they’ve already done the work for you!