Methodological and statistical components of technology-based intervention studies to improve reading skills: A systematized literature review
Abstract
In this review, we aimed to examine the methodological and statistical components of studies on the effects of technology-based interventions on reading skills. We conducted a systematized review and analyzed 21 studies to examine these components. The findings suggest that studies addressing the efficacy of technology-based reading interventions employ moderately strong methodological and statistical techniques, although improvements are necessary to determine their effect more clearly. Future studies should focus on employing random assignment of participants, counterbalancing testing procedures, and incorporating statistical methods that estimate unbiased effect sizes and uncertainty in the findings. Technology is flexible, powerful, and highly engaging for children, making it an ideal venue to explore and integrate with traditional intervention strategies.
Article Metrics Graph
##plugins.themes.bootstrap3.article.details##
computerized interventions, literature review, reading, technology-based intervention
American Psychological Association. (2020). Publication manual of the American Psychological Association. (7th ed.). American Psychological Association.
Aro, M. (2013). Literacy acquisition from cross-linguistic perspectives. In R. Malatesha Joshi & P. G. Aaron (Eds.), Handbook of orthography and literacy (pp. 531–550). Routledge.
https://doi.org/10.4324/9780203824719
Cameron, C., Turner, R. & Samaranayaka, A. (2021). Understanding confidence intervals and why they are so important. The New Zealand Medical Student Journal, 33(1), 42–43.
https://doi.org/10.57129/AGAG5939
Cazzell, S., Skinner, C. H., Ciancio, D., Aspiranti, K., Watson, T., Taylor, K., McCurdy, M., & Skinner, A. (2016). Evaluating a computer flash-card sight-word recognition intervention with self-determined response intervals in elementary students with intellectual disability. School Psychology Quarterly, 32(3), 367–378.
https://doi.org/10.1037/spq0000172
Chai, Z. (2017). Improving early reading skills in young children through an iPad app: small-group instruction and observa-tional learning. Rural Special Education Quarterly, 36(2), 101–111.
https://doi.org/10.1177/8756870517712491
Comaskey, E. M., Savage, R. S., & Abrami, P. (2009). A randomized efficacy study of web-based synthetic and analytic programmes among disadvantaged urban kindergarten children. Journal of Research in Reading, 32(1), 92–108.
https://doi.org/10.1111/j.1467-9817.2008.01383.x
De León-Casillas, C. E., Bermonti-Pérez, M., & Moreno-Torres, M. A. (2020). Guía metodológica para una revisión de literatura. Revista Salud y Conducta Humana, 7(1), 24-38.
De León-Casillas, C. E., & Moreno-Torres, M. A. (2020). Marco teórico para una revisión de literatura sistemática. Revista Salud y Conducta Humana, 7(1), 10–23.
Dimitrov, D. M., & P Rumrill, P. D. Jr. (2003). Pretest-posttest designs and measure-ment of change. Work, 20(1), 159-165.
Durlak, J. A. (2009). How to Select, Calculate, and Interpret Effect Sizes. Journal of Pediatric Psychology, 34(9), 917–928. https://doi.org/10.1093/jpepsy/jsp004
Ecalle, J., Kleinsz, N., & Magnan, A. (2013). Computer-assisted learning in young poor readers: The effect of grapho-syllabic training on the development of word reading and reading comprehend-sion. Computers in Human Behavior, 29(4), 1368–1376.
https://doi.org/10.1016/j.chb.2013.01.041
Ecalle, J., Magnan, A., & Calmus, C. (2009). Lasting effects on literacy skills with a computer-assisted learning using syllabic units in low-progress readers. Computers and Education, 52(3), 554–561. https://doi.org/10.1016/j.compedu.2008.10.010
Elfil, M. & Negida, A. (2017). Sampling methods in clinical research; an educational review. Emergency (Tehran, Iran), 5(1).
Fan, M., Antle, A. N., Hoskyn, M., & Neustaedter, C. (2018). A design case study of a tangible system supporting young English language learners. International Journal of Child-Computer Interaction, 18, 67–78.
https://doi.org/10.1016/j.ijcci.2018.08.001
Graf, C., Wager, E., Alyson, B., Fiack, S., Diane Scott-Lichter, D. & Robinson, A. (2007). Best practice guidelines on publication ethics: A publisher’s perspective. International Journal of Clinical Practice, 61(152), 1-26.
https://doi.org/10.1111/j.1742-1241.2006.01230.x
Green, B. N., Johnson, C. D. & Adams, A. (2006). Writing narrative literature reviews for peer-reviewed journals: Secrets of the trade. Clinical Update, 5(6), 101-117.
https://doi.org/10.1016/S0899-3467(07)60142-6
Gustafson, S., Fälth, L., Svensson, I., Tjus, T., & Heimann, M. (2011). Effects of three interventions on the reading skills of children with reading disabilities in grade 2. Journal of Learning Disabilities, 44(2), 123–135. https://doi.org/10.1177/0022219410391187
Hedges, L. V. (1981). Distribution theory for Glass' s estimator of effect size and related estimators. Journal of Educational Statistics, 6(2), 107–128. https://doi.org/10.2307/1164588
Higgins, J. P., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M. J., & Welch, V. A. (2008). Cochrane handbook for systematic reviews of interventions. En J. P. Higgins & S. Green (Eds.), Cochrane handbook for systematic reviews of interventions.
Howell, D. (2010). Statistical methods for psychology. (8th ed). Wadsworth Cengage Learning.
Institute of Medicine. (2011). Finding what works in health care: standards for systematic reviews. National Academies Press. https://doi.org/10.17226/13059
Karemaker, A., Pitchford, N. J., & O'Malley, C. (2010). Enhanced recognition of written words and enjoyment of reading in struggling beginner readers through whole-word multimedia software. Computers and Education, 54(1), 199–208. https://doi.org/10.1016/j.compedu.2009.07.018
Kathryn-Horne, J. (2017). Reading Comprehension: A computerized intervention with primary-age poor readers. Dyslexia, 23(1), 119-140.
Kirk, R. E. (2012). Experimental design: Procedures for the behavioral sciences (4th ed.). Sage Publications.
Kleinsz, N., Potocki, A., Ecalle, J., & Magnan, A. (2017). Profiles of French poor readers: Underlying difficulties and effects of computerized training pro-grams. Learning and Individual Differences, 57, 45–57.
https://doi.org/10.1016/j.lindif.2017.05.009
Kyle, F., Kujala, J., Richardson, U., Lyytinen, H., & Goswami, U. (2013). Assessing the effectiveness of two theoretically motivated computer-assisted reading interventions in the United Kingdom: GG Rime and GG Phoneme. Reading Research Quarterly, 48(1), 61–76.
https://doi.org/10.1002/rrq.038
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. PLoS Medicine, 6(7).
https://doi.org/10.1371/journal.pmed.1000100
Loannidis, J. P. A. (2005). Why most published research findings are false. PloS Medicine, 2(8), 696-701.
https://doi.org/10.1371/journal.pmed.0020124
Messer, D., & Nash, G. (2018). An evaluation of the effectiveness of a computer-assisted reading intervention. Journal of Research in Reading, 41(1), 140–158. https://doi.org/10.1111/1467-9817.12107
Mishra, P., Pandey, C. M., Singh, S., Gupta, A. Sahu, C. & Keshri, A. (2019). Descriptive statistics and normality tests for statistical data. Ann Card Anaesth, 22(1), 67-72.
https://doi.org/10.4103/aca.ACA_157_18
Montenegro‐Montero, A. & García‐Basteiro, A. L. (2019). Transparency and reproducibility: A step forward. Health Sciences Reports, 2(3).
https://doi.org/10.1002/hsr2.117
Moser, G. P., Morrison, T. G., & Wilcox, B. (2017). Supporting fourth-grade students' word identification using application software. Reading Psychology, 38(4), 349–368.
https://doi.org/10.1080/02702711.2016.1278414
Nuzzo, R. (2014). Statistical errors. Nature, 506(1), 150-153.
Norton, B. J. & Strube, M. J. (2001). Understanding Statistical Power. Journal of Orthopaedic & Sports Physical Therapy, 31(6), 307-515.
https://doi.org/10.2519/jospt.2001.31.6.307
O'Callaghan, P., McIvor, A., McVeigh, C., & Rushe, T. (2016). A randomized controlled trial of an early-intervention, computer-based literacy program to boost phonological skills in 4- to 6-year-old children. British Journal of Educational Psychology, 86(4), 546–558. https://doi.org/10.1111/bjep.12122
Pindiprolu, S., & Forbush, D. (2009). Evaluating the promise of computer-based reading interventions with students with reading difficulties. Journal on School Educational Technology, 4(3), 41–49. https://doi.org/10.26634/jsch.4.3.588
Potocki, A., Magnan, A., & Ecalle, J. (2015). Computerized trainings in four groups of struggling readers: Specific effects on word reading and comprehension. Research in Developmental Disabilities, 45–46, 83–92.
https://doi.org/10.1016/j.ridd.2015.07.016
Pourhoseingholi, A. M., Baghestani, A. R. & Vahedi, M. (2012). How to control con-founding effects by statistical analysis. Gastroenterology and Hepatology From Bed to Bench, 5(2), 79-83.
Reback, J., jbrockmendel, McKinney, W., Bossche, J. Van den, Augspurger, T., Cloud, P., Hawkins, S., gfyoung, Roeschke, M., Sinhrks, Klein, A., Petersen, T., Tratner, J., She, C., Ayd, W., Hoefler, P., Naveh, S., Garcia, M., Schendel, J., … Seabold, S. (2021). pandas-dev/pandas: Pandas 1.3.3. https://doi.org/10.5281/ZENODO.5501881
Rosas, R., Escobar, J. P., Ramírez, M. P., Meneses, A., & Guajardo, A. (2017). Impact of a computer-based intervention in Chilean children at risk of manifesting reading difficulties / Impacto de una intervención basada en ordenador en niños chilenos con riesgo de manifestar dificultades lectoras. Infancia y Aprendizaje, 40(1), 158–188.
https://doi.org/10.1080/02103702.2016.1263451
Saine, N. L., Lerkkanen, M. K., Ahonen, T., Tolvanen, A., & Lyytinen, H. (2010). Predicting word-level reading fluency outcomes in three contrastive groups: Remedial and computer-assisted remedial reading intervention, and mainstream instruction. Learning and Individual Differences, 20(5), 402–414. https://doi.org/10.1016/j.lindif.2010.06.004
Schmitt, K. L., Hurwitz, L. B., Sheridan Duel, L., & Nichols Linebarger, D. L. (2018). Learning through play: The impact of web-based games on early literacy development. Computers in Human Behavior, 81, 378–389.
https://doi.org/10.1016/j.chb.2017.12.036
Schmitt, A. J., McCallum, E., Hawkins, R. O., Stephenson, E., & Vicencio, K. (2019). The effects of two assistive technologies on reading comprehension accuracy and rate. Assistive Technology, 31(4), 220–230. https://doi.org/10.1080/10400435.2018.1431974
Slavin, R. E., Lake, C., Davis, S. & Madden, N. A. (2011). Effective programs for struggling readers: A best-evidence synthesis. Educational Research Review, 6(1), 1-26.
Snyder, H. (2019). Literature review as a research methodology: An overview and guidelines. Journal of Business Research, 104(1), 333-339.
https://doi.org/10.1016/j.jbusres.2019.07.039
Solheim, O. J., Frijters, J. C., Lundetræ, K., & Uppstad, P. H. (2018). Effectiveness of an early reading intervention in a semi-transparent orthography: A group randomised controlled trial. Learning and Instruction, 58, 65–79.
http://doi.org/10.1016/j.learninstruc.2018.05.004
Suresh, K. P. (2011). An overview of randomization techniques: An unbiased assessment of outcome in clinical research. Journal of Human Reproductive Sciences, 4(1), 8-11.
http://doi.org/10.4103/0974-1208.82352
The National Academies of Sciences, Engineering and Medicine. (2019). Reproducibility and replicability in science. The National Academies Press.
Turner, H. M. & Bernard, R. M. (2006). Calculating and synthesizing effect sizes. Contemporary Issues in Communication Science and Disorders, 33(1), 42-55.
Wood, C. L., Mustian, A. L., & Lo, Y. yu. (2013). Effects of supplemental computer-assisted reciprocal peer tutoring on kindergarteners' phoneme segmentation fluency. Education and Treatment of Children, 36(1), 33–48. https://doi.org/10.1353/etc.2013.0004
https://orcid.org/0000-0001-5228-2538