Research: improved teaching practices, student engagement, and learning outcomes

Research Paper Summary
The Manaiakalani Project Evaluation
What us the impact of the Manaiakalani 1:1 Project
on literacy teaching and learning?

Introduction: “The Hook from Heaven”

Manaiakalani (a Hawaiian word meaning “the hook from heaven”) is a multi-year e-learning and literacy project being run by a group of seven schools in a poor urban community in Auckland, New Zealand. The majority of pupils in these schools come from socioeconomically-depressed areas and Maori and Pasifika populations; demographic groups that are over-represented amongst New Zealand’s lower-achieving students. The project’s goal is to increase these students’ engagement and learning outcomes through collaborative e-learning and literacy practices centred around student publishing on blogs.

Research on the first three years of the project (2008-10) found that student achievement had risen and that the students were more engaged. However both teachers and students were struggling with inadequate access to computers. In 2011 the project gained a new impetus with the formation of the Manaiakalani Trust – a public-private partnership that enabled the community to structure commercial relationships with government and commercial-sector organizations. A community-wide wireless extension to the school networks was established and individual laptops, purchased by parents, were provided for 465 students in years 5-10 at eighteen selected classrooms. Teachers in these classes have already been working with the Google Apps suite of Web-based tools, including Google Docs, Blogger, Gmail, as well as Teacher Dashboard for Google Apps, a classroom management toolset developed by Hapara.

This paper reports on research on the impact of this second stage of the project on student engagement and the teaching and learning of literacy. This research is part of an ongoing longitudinal study.

Research Method
The study used mixed methods. It combined field research based on observation (videotaping and analysing teaching methods and student engagement), interviews with teachers, students, principals and the project manager, analysis of reflective journals kept by teachers, a bi-annual teacher survey and an annual student survey. Student blogs and class blogs were also analysed twice per year and monitored for evidence of progress. Quantitative data on student achievement in reading and writing was obtained twice yearly from New Zealand’s literacy assessment tool (e-asTTle) allowing longitudinal sampling and comparison of each cohort to the national student profile.

Findings
New teaching practices
After an initial period focused around the development new routines, most classes dispensed with paper, using the laptops as the sole tool for learning and presentation of work. Students created work using Google Docs, spreadsheets, blogs or presentations, saving it into subject files that they shared with teachers. The teachers used Teacher Dashboard to view this work and track each student’s learning history. They noted that it was important to teach students how and where to file documents and to undertake monitoring to ensure that the filing of work was being done consistently. Some teachers developed a routine of displaying the dashboard at the end of each lesson to ensure students had filed their work before leaving the room.

Changes in teaching and learning
The use of Google Docs, Google Sites and individual and class blogs by teachers also resulted in a number of deeper changes in pedagogy. Teachers reported that Google Docs allowed them to comment on students’ work in real time, identifying individual learning issues in a timely manner, and reducing the amount of marking that was required outside of the classroom:

I now have online and real-time conferences with students.

There is less marking to do as I can comment on a student’s document as they work.

My feedback to students has increased: I can now comment at the start, middle and end of tasks.

By the final trimester of 2011, ten out of the eighteen teachers were using Google sites to co-ordinate their class and/or subject work. Starting out as an online taskboard, the sites developed into a richer planning centre that was shared and sometimes co-constructed with the students. The sites combined information about learning intentions, group activities, instructions, exemplars, external links and follow-up work. Analysis of videotaped lessons showed that teachers who used the sites issued fewer instructions and focused their talk with students around learning matters rather than on managing or directing activities.

Changes in student engagement
Students were strongly motivated and excited to have their own laptop. Observations showed a high level of on-task work, while teachers reported less noise, increased ownership of learning, and more frequent task completion. Analysis of videos showed evidence of student self-management, independence and critical thinking (three of the key competencies in the New Zealand curriculum).

In addition, teachers noted that the use of Google Sites had made their students more independent. This was borne out by the video observations: Students in the classes where class Sites were used were less reliant on the teacher than those in the other classes. They completed more work either independently or with peer support, allowing them to move on to activities that extended their thinking.

Instead of finding out what to do from the teacher we look on Google Sites – it’s a lot easier for us and the teacher (student blog post)

I spend less time on organisation – what each group is doing and what to do when finished – it is all on the site (teacher)

Each student published work on a personal learning blog, averaging one post per fortnight. This represented a 400% increase on the previous posting rate, when students had to take turns posting on a class blog. The majority of posts on the individual blogs were presentations of schoolwork using text and photos, together with some movies and animations. Teachers noted a strong level of student engagement and motivation surrounding this work:

The impact of kids actually getting something published onto their blog is huge… when they do post and see it published they get so enthusiastic and want to do more

Children want to complete work – they even ask, “Can we have time to finish so I can get it on my blog?”

Changes in student achievement
The focus in monitoring student achievement in the study was not on learning outcomes per se, but on the “shift”, or improvement, in learning outcomes that was made by each cohort at regular intervals during the study, as measured by the national assessment tool, e-asTTLe. The significance of these shifts was ascertained by comparing them with the “expected shifts” for students at each year level, based on aggregated national statistics.

The improvements in writing achievement were significant:

Although all of the cohorts achieved below the expected national level in writing, the shifts in achievement during 2011 ranged from just above the expected shift to six times the expected shift. For Pasifika students the shifts at all year levels ranged from twice to seven times the expected shifts. The shifts in writing by Maori students were from slightly higher than the expected shift to eight times the expected shift:

In reading there was a range of shifts in achievement, from just below the expected shift to three times the expected shift. Boys in years 6,7 and 8 were found to have made twice the expected shift. Overall, the girls achieved slightly higher scores in reading than the boys at all year levels.

Summary
After one year, the second stage of the Manaiakalani Project, based around the use of individual laptops and Google Apps tools, resulted in significant improvements in student engagement and self-management, as well as strong positive shifts in writing achievement. Improvements in reading were also evident. The use of Google Sites and Teacher Dashboard led to new teaching practices that resulted in more effective use of classroom time. Teachers were able to focus on supporting learning rather than managing activities, using the tools to build shared responsibility for learning, monitor group and individual learning progress, and provide just-in-time one-on-one feedback to individual students.

Note: Only four of the seven schools in the study used the same version of the e-asTTLe assessment tool. This meant that student achievement data could only be compared across these schools. The assessment tool is being aligned across schools for the rest of the study.

Complete report can be downloaded from the Manaiakalani Project Web site.

For more information, please contact us.