Xun Ge, Ph.D., University of North Texas, USA
Kausalai (Kay) Wijekumar, Ph.D.,Texas A&M University, USA
Dirk Ifenthaler, Ph.D., University of Mannheim, Germany and Curtin University, Australia
Seng Chee Tan, Ph.D., Nanyang Technological University, Singapore
Replication of research is a critical scientific process for validating and confirming findings, solidifying theories, and building robust knowledge in a field (Christensen et al., 2022). It transforms single observations into accepted scientific knowledge (Schmidt, 2017), allowing researchers to challenge existing concepts and, more importantly, discover new solutions to ongoing problems. However, the scarcity of replication studies in the social sciences has become a growing concern, and this issue is particularly concerning in educational research, especially in educational technology (Christensen et al., 2022; Makel & Plucker, 2014; Perry, 2022; Schmidt, 2017; Schneider, 2019; What Works Clearinghouse, 2022). Several factors may have contributed to the lack of replication studies: journal biases that favor novel studies with more significant impact than replication studies (Makel & Plucker, 2014), and the inherent challenges of conducting experiments in real-world educational settings, which are often messy and unpredictable. The absence of replication studies can result in unreliable findings influencing decisions made by educators, administrators, and policymakers, ultimately making a negative impact in education.
In recent years, researchers in education, like those in areas of social sciences, have increasingly acknowledged the importance of replication studies (Plucker & Makel, 2021). Various efforts have been made to encourage more focus on replication, such as discussions within professional organizations like the National Technology Leadership Summit (NTLS) and the Society for Information Technology in Teacher Education (SITE), and editorials calling for replication research (Christensen et al., 2022). Journals such as the Journal of Research on Educational Effectiveness (JREE) now actively welcome replication studies, including those with null or negative results. Similarly, the U.S. Department of Education’s Institute of Education Sciences and the Educational Innovation and Research Programs have called for replication studies in their funding announcements. Replication and original studies are also required to be registered in research databases like https://sreereg.icpsr.umich.edu/sreereg/. Despite these attempts, more effort should be put forward to promote and publish replication studies in an academic culture that prioritizes original research and novel findings. Therefore, this special issue aims to further encourage replication research, specifically in the field of educational technology.
Replication Studies - Definitions and Frameworks
According to Schmidt (2009; 2017), replication studies are methodological tools that, through repetition, can transform observations into established knowledge (Schmidt, 2009).
Replication can be categorized into four main types on a continuum ranging from the highest level of replication to the least stringent replication (Darley, 2000; Schmidt, 2009)
-
Direct (Strict) Replication involves reproducing a study using the same methods, procedures, and instruments with a highly similar participant sample, ensuring the new study closely mirrors the original.
-
Operational Replication mainly tests for validity, including some level of reiteration of the conceptual framework, research plan and design, instrumentation, and so on.
-
Conceptual Replication uses different methods, populations, contexts, or analyses to test the hypothesis of the original study and confirm its findings, with a focus on the original study findings and theory.
Comparable to Darley’s and Schmidt’s categories described above, Christensen et al. (2022) offer a framework for replication that spans reproduction, reiteration, and refocus, specifically used for guiding replication research in the context of educational technology. These frameworks offer useful lenses for educational researchers to categorize replication studies.
We invite proposals for replication studies in the fields of instructional design, learning sciences, and educational technologies. Proposals should focus on one of the following types of replication studies: strict, partial, operational, or conceptual (or similar categories), over a period of time. The research proposal should provide a brief description of an original/existing study, based on which you are going to carry out the replication study, followed by your plan to conduct the replication study. Proposals (up to 1000 words) should include the following:
If your research results in a series of replications, subsequent replication studies can be featured in a future issue.
Christensen, R., Hodges, C. B., & Spector, J. M. (20221). A framework for classifying replication studies in educational technologies research. Technology, Knowledge and Learning, 27(4), 1021–1038. https://doi.org/10.1007/s10758-021-09532-3
Darley, W. K. (2000). Status of replication studies in marketing: A validation and extension. Marketing Management Journal, 10(2), 121–132.
Makel, M. C., & Plucker, J. A. (2014). Facts are more important than novelty: Replication in the education sciences. Educational Researcher, 43(6), 304–316. https://doi.org/10.3102/ 0013189Å~14545513
Makel, M. C., Plucker, J. A., Freeman, J., Lombardi, A., Simonsen, B., & Coyne, M. (2016). Replication of special education research: Necessary but far too rare. Remedial and Special Education, 37(4), 205–212. https://doi.org/10.1177/0741932516646083
Makel, M. C., Smith, K. N., McBee, M. T., Peters, S. J., & Miller, E. M. (2019). A path to greater credibility: Large-scale collaborative education research. AERA Open, 5(4).https://doi.org/10.1177/2332858419891963
Schmidt, S. (2017). Replication. In M. C. Makel & J. Plucker (Eds.), Toward a more perfect psychology: improving trust, accuracy, and transparency in research (pp. 233–253). American Psychological Association. https://doi.org/10.1037/0000033-015
What Works Clearinghouse (2022). What Works Clearinghouse Standards Guide 5.0. https://ies.ed.gov/ncee/wwc/handbooks