Portal de Periódicos da CAPES

Protocol for a Systematic Review: Provision of Information and Communications Technology (ICT) for Improving Academic Achievement and School Engagement in Students Aged 4‐18

2016; The Campbell Collaboration; Volume: 12; Issue: 1 Linguagem: Inglês

10.1002/cl2.163

1891-1803

Kristin Liabo, Laurenz Langer, Antonia Simon, Kathy‐ann Daniel‐Gittens, Alex Elwick, Janice Tripney,

Early Childhood Education and Development

Campbell Systematic ReviewsVolume 12, Issue 1 p. 1-49 PROTOCOLOpen Access Protocol for a Systematic Review: Provision of Information and Communications Technology (ICT) for Improving Academic Achievement and School Engagement in Students Aged 4-18 Kristin Liabo, Kristin LiaboSearch for more papers by this authorLaurenz Langer, Laurenz LangerSearch for more papers by this authorAntonia Simon, Antonia SimonSearch for more papers by this authorKathy-ann Daniel-Gittens, Kathy-ann Daniel-GittensSearch for more papers by this authorAlex Elwick, Alex ElwickSearch for more papers by this authorJanice Tripney, Janice TripneySearch for more papers by this author Kristin Liabo, Kristin LiaboSearch for more papers by this authorLaurenz Langer, Laurenz LangerSearch for more papers by this authorAntonia Simon, Antonia SimonSearch for more papers by this authorKathy-ann Daniel-Gittens, Kathy-ann Daniel-GittensSearch for more papers by this authorAlex Elwick, Alex ElwickSearch for more papers by this authorJanice Tripney, Janice TripneySearch for more papers by this author First published: 01 March 2016 https://doi.org/10.1002/CL2.163Citations: 2AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat BACKGROUND The Problem, Condition or Issue Improving educational attainment, while reducing social inequality, is one of the fundamental pillars of education policy across the world (Ananiadou & Claro, 2009; Voogt, Knezek, Cox, Knezek, & ten Brummelhuis, 2013). Information and communications technology (ICT) is often provided by schools or national education departments with the intention of improving attainment and addressing this inequality (SQW Ipsos Mori & London Knowledge Lab, 2011) and yet – despite the need to justify and prioritize sparse educational resources – this spending is often not based upon evidence of effectiveness. This review will focus on the different forms of ICT in education with the aim to identify which forms yield the most educational benefits for students. Policymakers and schools both require evidence to guide them on which ICT to invest in. It is imperative for teachers, education policy makers, and local planners to know whether and how money spent will impact on the learning of pupils who receive it. This is particularly important in the face of multiple marketing efforts from competing manufacturers and developers and increased demand for ICT in schools from parents and students. The ICT environment is changing rapidly, and provision, use and expectations of ICT are very different now than only 10 years ago. Technological advancements in the past 5 years mean that smart phones and computers are cheaper and therefore easier to access, both in terms of affordability and community access. This is reflected in ownership. For example, in the UK only 3% of children and young people can now be described as 'non-users' of the internet (Office for National Statistics, 2013). This is a very different picture from only ten years ago, and means that ICT provision is operating within very different structures from before then. With basic ICT equipment becoming cheaper, there is no longer evidence of a sharp digital divide in some countries, and instead there is talk of a gap in how ICT is used by those acquiring or receiving it (Liabo, Simon, & Nutt, 2013). Many countries have invested huge amounts of their education budgets into ICT over the last decade (Haelermans & Blank, 2012): in the US $5 billion was spent by schools on ICT in 2004 (Market Data Retrieval, 2004), while in the UK, secondary schools forecast their 2014/15 spending on ICT will be around $480 million in total (BESA, 2013). Increasingly, as ICT equipment becomes more affordable, some schools are purchasing ICT in order to improve students' attainment, for example by providing each student with a tablet or laptop, offering reduced-rate internet subscription or by ICT-immersion programmes, which embed all classrooms with communications technology and computers. Often these programmes are particularly focused on reaching out to pupils who are under-achieving or who are at a social disadvantage that is perceived to reduce their ability to obtain ICT privately (Lee Watson & Watson, 2011; Mouza, 2008) . This is also happening through work of charities, and across the world (Cristia, Ibarraran, Cueto, Santiago, & Severin, 2012; Finn, Kerman, & LeCornec, 2005; Meyers & Andresen, 2000). Changes in ICT provision and use are not uniform however. International comparison league tables published by the OECD (OECD, 2010) detail the use of ICT equipment. In 2011 on average 92% of 4th grade pupils had access to a computer at school (based upon a sample of 50 OECD member countries participating in the Trends in International Mathematics and Science Study (International Association for the Evaluation of Educational Achievement, 2011). However, for some of the poorer countries in the OECD (based upon GDP per capita (World Bank, 2014b) this figure was much lower: in Morocco only 69% of students had access to a computer, and in the Yemen this figure was just 28%. For many of the richer countries in the OECD at least 99% of pupils had access (e.g., Qatar and Norway). The pressures on schools to invest comes from multiple sources, and the pitfalls that exist in terms of purchasing/leasing ICT equipment make it harder for schools to invest confidently. BBC Radio 5 live and the BBC's Panorama programme both conducted investigations in 2012 which exposed the tactics ICT vendors use in order to sell/lease ICT equipment (BBC, 2012; Goldberg, 2012) . Meanwhile, parents and students are both increasingly demanding greater ICT usage in schools (Heinrich, 2012; Matthewson, 2014; Selwyn, Potter, & Cramer, 2010). These pressures mean that schools may not make decisions objectively nor based primarily upon evidence of the effectiveness of such approaches. This review will address these challenges, by reviewing the effectiveness of interventions that provide ICT. Furthermore, this review incorporates studies that have researched the perspectives of students and teachers, in order to illuminate what it is like to be given ICT equipment, for use in learning. The Intervention In this review we use ICT as a collective term for stationary computers, laptops and tablets, internet-connected or not. Our interest is in initiatives, interventions or programmes which have provided such ICT individually to students, with the intention of improving their educational outcomes. The provision might be confined to the school environment (for example laptops to be used at school only), to pupils' homes (for example stationary computers at home) or be flexible across all environments (for example laptops which are used both at home and at school). Provision of software as part of a wider hardware package will be included. The provision of ICT might be free, or it might be provided at a discount. We will include studies if they have combined such provision with components to enhance or influence use of ICT in teaching and learning. The aims of ICT use equally varies around several aspects of learning: to find information, to learn core subjects such as maths and literacy, to learn computer skills, to foster self-regulating learning skills, or to engage people who are physically placed outside of the school (Tondeur, Van Braak, & Valcke, 2007; Vanderlinde, Aasaert, & Van Braak, 2014). However, as stated above, in this review we focus on the provision of ICT equipment (stationary computers, laptops and tablets) to students and this must be a component of the intervention in order for an evaluation to be included. Our boundary for the intervention might be seen as strict and exclusive in a multifaceted ICT learning world. Equally, our extension to interventions which combine provision with teaching might be seen as providing too much variability. Research which considers the relationship between self-acquired ICT at home and academic achievement and engagement (Biagi & Loi, 2013; Vigdor & Ladd, 2010) is outside of the scope, as is provision of software only, since this kind of use depends on existing ownership of hardware. How the Intervention Might Work While improved academic achievement will be an aim for most educational establishments, the rationales for ICT provision can be distinguished along four lines. These are interlinked, but it is useful for the purposes of this review to clarify them. First, the provision of ICT is sometimes related to a concern about access to ICT in the student population. The school may find that some of their students do not have home access to ICT and therefore provide ICT so that their teaching and homework assignments can utilise ICT as a teaching and learning tool. Second, ICT provision can be seen as important for embedding ICT within the school's pedagogical approaches, to enhance instruction. Third, and closely related to the second rationale, ICT can be provided to enhance student motivation, by facilitating self-directed learning (Mouza, 2008). Primarily, therefore, ICT provision is usually implemented with the aim of impacting on students' learning activities, thereby enhancing their academic achievement (Haelermans & Blank, 2012; Mouza, 2008; Penuel, 2006). Fourth, ICT provision can be used as a marketing tool to enhance the number of parents and students choosing a school or to boost a school's/school authority's reputation. In this context ICT becomes part of the environment in which students learn, both individually for each student as a tool to use within the classroom and/or for schoolwork, and collectively as something all students use and engage with as part of their school environment. So while ICT provision gives opportunities for individual use initiated by the student, it also provides the opportunity for collective learning, and teacher-initiated computer-based instruction. For example, most schools now have specific rooms with stationary computers which teachers book in advance for special computer-based sessions. Having the computers follow students, rather than the other way around, has the potential to change how teachers use computers in their teaching, as well as changing how students use them, because they are available all the time, and because all students have them. Computers can therefore be integrated into learning instead of being a separate item, away from the day-to-day classroom environment (Penuel, 2006). In fact, some have argued that this is the only way to revolutionize learning and teaching (Fleischer, 2012). While this review is primarily interested in the effects from providing students with personal ICT equipment, an important intermediary variable impacting on outcomes from provision is likely to be teachers' ability or willingness to utilize this in their teaching. Furthermore, we might identify studies that have evaluated schemes where the provision of ICT is accompanied by teacher-level interventions. To complement this important variable, we are including studies which have researched ICT provision from the perspective of students and teachers. Figure 1 below emphasizes the two main types of ICT provision to be included in the review. The first type includes provision only, such as One Laptop Per Child, where schools give all students, or all students within one year group, a tablet or laptop computer. The second type includes provision alongside further teacher-level intervention such as training on how to use ICT in classroom instructions, booster ICT-lessons, or ICT immersion programmes to enhance the entire school ICT infrastructure. Any effects from either model is likely to be mediated by the school's and teachers' capacity for change (Tearle, 2004). Figure 1Open in figure viewerPowerPoint Logic model of ICT provision (adapted from Zief, Lauver, & Maynard 2006, cited in Anderson et al., 2011) In addition the model aims to reflect that ICT in learning might also influence, and be influenced by, the relation between teacher and learner (Livingstone, 2012). As shown in Figure 1, the provision of ICT is expected to spur on ICT-related activities which may or may not be conducive to the outcomes of interest to this review, namely academic achievement and school engagement. ICT has additional uses outside of formal education and homework tasks. Provision of ICT also increases access to social networking opportunities, web-based or electronic gaming, and it gives students the opportunity to pursue a wide range of interests which may or may not be relevant to the school curriculum. Social networking, gaming and information-seeking are not inherently negative or positive pursuits. Students may engage in these activities in a whole range of ways, finding a good balance, engaging in risky behaviors, expanding their horizons, meeting new friends or engaging in cyber-bullying. Such activities are intermediary outcomes from ICT provision, which are likely to impact on the outcomes of interest to this review, namely academic achievement and school engagement (third box in Figure 1). All school-based interventions will invariably impact teachers and students in some way, as they are the people inhabiting the environments which are being changed. In terms of ICT provision, students are the recipients of the intervention and outcomes are measured from their behavior and cognitive development. However, children may or may not use the provided equipment as intended by program initiators who are rarely children or young people themselves. The intermediary outcomes listed in Figure 1 relate to how children themselves perceive and use the ICT provided. These activities can be counted, but equally important are children's reasons for this behavior: their underlying perceptions of ICT provision. It is important to consider Figure 1 in relation to the wider micro structures such as family and individual characteristics, meso structures such as individual school characteristics, and macro structures such as ICT infrastructure and social class (Biagi & Loi, 2013). In terms of provision of ICT to students, its impact on learning within the school environment will be related to the school's capacity for change, in this context "the collective competence of a school to implement ICT in a way that is a lever for instructional change" (Vanderlinde & van Braak, 2010, p. 543). Figure 2 below, illustrates the wider structures in which ICT provision operates. Figure 2Open in figure viewerPowerPoint Conceptual framework This review will be using the logic model (Figure 1) when considering relevant studies for the review. The conceptual framework (Figure 2) informs our selection of variables for moderator analysis (see page 27) and will help us contextualize the findings of included studies. For example, there is some evidence indicating that social class and gender are both wider variables which will intervene across pathways of impact from ICT provision (Elwick, Liabo, Simon, & Nutt, 2013; Eynon, 2009). The included studies on teachers' and students' perspectives on ICT provision might further help illustrate these contextual influences. Why it is Important to do the Review There are many reasons why this review is important, including the costliness of providing ICT interventions, the policy and educational goals tied to ICT interventions and the murky picture relating to the overall effects of these interventions. Knowing which ICT yield the most educational benefits for students is a challenge for schools. A challenge exacerbated by the fact that they must contend with parents advocating urgently for ICT in schools, pupils requesting ICT services and vendors overwhelming them with skewed advertising data. Knowing whether to invest in particular ICT, and if so, what size of investment, is critical in the context of tight budgets. Schools and educational authorities find themselves in the middle of this dilemma, wrestling to find solutions. Children and young people aged 4-18 spend a large proportion of their lives in school where they learn and engage in both academic and social enterprises. School is therefore an essential influence on their intellectual and social development. When considering school-based ICT interventions, it is important to include pupils' perspectives for two main reasons. First, according to the United Nation's Convention of the Rights of the Child, children have a right to have their views heard in matters that concern them. Second, since the outcomes sought by ICT enhancement programmes are directly connected to children's outcomes, children are central to the intervention. There might be a popular assumption that ICT provision is only perceived positively by children and young people, but there is the possibility that they may feel situational pressures to use ICT, against their own inclinations. Or, if they do indeed value ICT provision highly, this ought to be an important factor for consideration alongside other outcomes. As a result, understanding and incorporating student perspectives is an integral part of the review in terms of the inclusion of studies capturing these, as well as an important outcome of the overall review. Finally, including children's views relates to the overall aim of considering ICT impact on school engagement, since views studies might further illuminate this particular aspect of ICT interventions. ICT intervention studies have evaluated whether ownership of ICT impacts educational attainment (Fairlie & London, 2011; Fairlie & Robinson, 2011). These studies examined the impact of ICT on educational attainment by providing students with free laptops. Other intervention studies provided computer skills training along with discounted or free computers and internet access (Finn et al., 2005; Tsikalas, Lee, & Newkirk, 2007). In addition, some studies considered whether the use of internet impacts educational achievement (Jackson et al., 2006), and whether the use of ICT in both teaching and learning impacts achievement (Low & Beverton, 2004). While the findings of these studies vary, rigorous and recent individual studies have suggested that there is reason to question the effect of ICT provision on achievement (Belo, Ferreira, & Telang, 2010; Cristia et al., 2012; Fairlie & London, 2011; Fairlie & Robinson, 2011). Despite an accumulation of results from individual ICT intervention studies, a lack of clarity remains on the effect of ICT provision. A systematic review from 2006 concluded in a narrative synthesis that one-to-one laptop programs appeared to impact positively on technology use, technology literacy and writing skills (Penuel, 2006). A rapid systematic review of ICT interventions found both that there are mixed results from most studies, and that the overall effect seemed to be negligible (Liabo et al., 2013). Liabo et al (2013) considered intervention studies in which ICT were provided to pupils as well as intervention studies which focused on self-acquired ICT equipment (mainly computers and internet access at home). We have not found any systematic reviews on pupils' views on ICT provision. A comprehensive systematic review of both ICT intervention studies and studies of young people's ICT views is therefore imperative for schools, educational authorities and all who work and invest in the education sector. This Campbell systematic review will build on the previous rapid review by Liabo et al (2013). It will conduct a more comprehensive literature search and focus on the provision of specific ICT and technology services programs. We expect the results of this review to be of practical use to policy makers, education planners, school heads and governors who make budgeting decisions and draw up strategies for school improvement. OBJECTIVES This review has two main objectives. The first objective is to address the question "What is the effectiveness of ICT provision on pupils' school engagement and achievement?" by systematically reviewing, assessing and synthesising the evidence from randomised and quasi-experimental evaluations of such interventions. A sub-objective of this is to consider separately, such provision targeted at socially disadvantaged students. The second objective is to address the question "What are children, young people, and teachers' views and experiences of ICT provision?" by systematically reviewing, assessing and synthesising the evidence from primary research (qualitative studies and surveys) which have addressed this question. METHODOLOGY Criteria for Including and Excluding Studies Types of Participants This review focuses on children aged 4-18 who are in compulsory education at elementary/primary level or high/secondary level, living in a high income country in Europe, North America or Australasia, as defined by the World Bank (World Bank, 2014a). This is because there is considerable difference in ICT access and availability between high-income countries and those of lower-middle income, and because the policy aims of introducing ICT in schools are likely to vary across very different socio-economic settings. The countries included are therefore: Australia, Austria, Belgium, Canada, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Iceland, Ireland, Italy, Luxembourg, Netherlands, New Zealand, Norway, Poland, Slovak Republic, Slovenia, Spain, Sweden, Switzerland, United Kingdom, and the United States. The defined age span reflects the ages of compulsory schooling in these countries. In regards to the sub-objective of objective number one, by social disadvantage we refer to students who meet the EU social inclusion indicator of relative poverty (60% or less of the median income1) or those eligible for free or partially subsidised school meals. Also included within our definition of socially disadvantaged will be pupils where multiple factors indicate social disadvantage, most notably living in a poor area with a low household income and with parents without higher education. Pupils characterised by living in foster or residential care will also be considered to be at social disadvantage and defined within this group. In regards to objective number two we will include studies which have asked teachers' views on ICT provision programmes, as well as those asking students. This review will not include studies on students who have special educational needs, or who have been excluded from school on the basis of their behaviour. A separate title for assistive technology for people with autistic spectrum disorder is registered in the Campbell Library (Nikopoulos, Nikopoulo-Smymi, & Dillenburger, 2012). ICT provision for people with special educational needs or behaviour problems is likely to serve purposes going beyond those implemented for all students. Types of interventions As described earlier in this protocol, this review focuses on interventions which provide students with free or discounted computers, including tablets, to be used at home, at school, or at a community centre. This can be programmes only providing computers, computers in combination with internet connection, or provided along with another learning package such as curriculum innovation or professional development (e.g. teacher training in instructional ICT use). This also includes ICT immersion programmes, which are a whole school approach to technology with an emphasis on ICT across the school environment and for individual use by students in their learning. Comparisons can be "no intervention," another type of intervention to improve achievement or school engagement, or comparisons between different kinds of ICT provision interventions at least one of which adhere to the intervention definition provided here. Excluded will be programmes that have implemented ICT for teachers to influence their teaching, such as interactive whiteboards or online teaching planning, unless these were accompanied by additional ICT provision to students. Also excluded will be instructional programmes, and online learning programmes, such as Mathletics, again unless they were provided alongside additional ICT provision as described above. In relation to objective 2, studies which have asked children, young people or teachers about their views, experiences or use of ICT in general, and not in connection with a relevant intervention programme (i.e., one that meets the above criteria for inclusion in this review) will be excluded. Types of study designs Objective no. 1: To address objective number one (systematic review of the effect of ICT provision), we are interested in quantitative effect sizes derived from study designs where one or more groups receiving the intervention are compared with one or more groups not receiving it. The method of allocation can be randomisation and quasi-randomisation. Randomisation here refers to when the randomisation was true, for example computer-generated. Quasi-randomisation refers to when the randomisation was by a particular variable, for example date of birth or toss of a coin. We will also include quasi-experimental studies where the group allocation was not random. For quasi-experimental studies to be included, they will have had to consider the equivalence between groups and, if not found, to have addressed this by use of a statistical measure (for example propensity score matching or use of statistical controls). For quasi-experimental studies, baseline equivalence will be determined by the magnitude of an effect size. However, consideration of equivalence must include at least one of the outcomes of interest to this review and we will only include quasi-experimental studies for which there was equivalence on pre-test for at least one of the primary outcomes listed in this review. Outcomes for which there was no equivalence at pre-test will not be included. Equivalence is also addressed in the critical appraisal tool (selection bias assessment for quasi-experimental studies). The unit of allocation can be individuals, where students within one school received free or discounted ICT and were compared with those in the same school who did not schools, or by clusters, where schools are allocated and individuals are therefore in a group due to their attending a particular school. The advantage of cluster randomisation is that it controls for contamination between individuals within the same setting. Examples of included study designs to assess the impact of ICT access and provision interventions are provided in Table 1 below. Table 1. Examples of effect study designs Study Sample size/response rate Population, intervention, comparison and outcomes (PICO) Length to follow-up Fairlie and Robinson (2011) Intervention: 559 students who did not have a home computer. Follow-up survey response rate of 78.7% Sample with high concentration of minority, immigrant and non-English speaking students. Recruited from middle and high schools, average age 13 (across grades 6 to 10). Ongoing and at end of the school year (one academic year). US study Comparison: 564 students who did not have a home computer. Follow-up survey response rate of 76.1%. Intervention: Random allocation of free computers with basic Microsoft software. Students could upgrade at own expense. No technical support provided. The authors claim that the reliance on administrative data eliminates concerns over attrition in terms of test outcomes. Follow-up N for these not stated. Comparison: Waiting-list control. Grade Point Averages (GPAs) and Standardised Testing and Reporting (STAR) test score results collected each spring for all Californian students. Survey questionnaire about homework and attitude to school. Study Sample size/response rate Population, intervention, comparison and outcomes (PICO) Length to follow-up Dunleavy and Heinecke (2008) Intervention: 300 students randomly allocated over 3 years (100 each year) and across 3 grades and 4 volunteer teachers at each grade. Students attending a middle school in an urban area, serving 972 students (grades 6 to 8). School history of poor academic achievement. 2 years. US study Comparison: Students in the same school and in the same grades, and their teachers. Intervention: 24-hour access to laptops during the school week (but not allowed to take laptops home over the weekend). Laptops were Apple iBooks with Microsoft Office software and Internet Explorer. In addition, textbook resources and online textbook access. Students carried laptops from class to class, but the storing/charging cart was housed in the teacher's room. Wireless classrooms. Comparison: No laptops but access to the same resources via the school's computer lab. Outcomes: Standardised achievement test scores on mathematics and science tests. Excluded designs for objective number one are: Before-after studies: comparing results from the intervention period with previous periods in the same setting—for example, looking at in-school variation by year considering academic results or engagement levels before the introduction of ICT provision and after. Objective no. 2: To address objective number two (systematic review of children and young people's views on ICT provision) we will include: - Case study and ethnographic designs facilitating qualitat