Research Paper Summary

The Manaiakalani Project Evaluation
What is the impact of the Manaiakalani 1:1 Project on literacy teaching and learning?

Introduction: “The Hook from Heaven”

Manaiakalani (a Hawaiian word meaning “the hook from heaven”) is a multi-year e-learning and literacy project being run by a group of seven schools in a poor urban community in Auckland, New Zealand. The majority of pupils in these schools come from socioeconomically-depressed areas and Maori and Pasifika populations; demographic groups that are over-represented amongst New Zealand’s lower-achieving students. The project’s goal is to increase these students’ engagement and learning outcomes through collaborative e-learning and literacy practices centred around student publishing on blogs.

Research on the first three years of the project (2008-10) found that student achievement had risen and that the students were more engaged. However both teachers and students were struggling with inadequate access to computers. In 2011 the project gained a new impetus with the formation of the Manaiakalani Trust – a public-private partnership that enabled the community to structure commercial relationships with government and commercial-sector organizations. A community-wide wireless extension to the school networks was established and individual laptops, purchased by parents, were provided for 465 students in years 5-10 at eighteen selected classrooms. Teachers in these classes have already been working with the Google Apps suite of Web-based tools, including Google Docs, Blogger, Gmail, as well as Teacher Dashboard for Google Apps, a classroom management toolset developed by Hapara.

This paper reports on research on the impact of this second stage of the project on student engagement and the teaching and learning of literacy. This research is part of an ongoing longitudinal study.

Research Method

The study used mixed methods. It combined field research based on observation (videotaping and analysing teaching methods and student engagement), interviews with teachers, students, principals and the project manager, analysis of reflective journals kept by teachers, a bi-annual teacher survey and an annual student survey. Student blogs and class blogs were also analysed twice per year and monitored for evidence of progress. Quantitative data on student achievement in reading and writing was obtained twice yearly from New Zealand’s literacy assessment tool (e-asTTle) allowing longitudinal sampling and comparison of each cohort to the national student profile.

Findings

New teaching practices

After an initial period focused around the development new routines, most classes dispensed with paper, using the laptops as the sole tool for learning and presentation of work. Students created work using Google Docs, spreadsheets, blogs or presentations, saving it into subject files that they shared with teachers. The teachers used Teacher Dashboard to view this work and track each student’s learning history. They noted that it was important to teach students how and where to file documents and to undertake monitoring to ensure that the filing of work was being done consistently. Some teachers developed a routine of displaying the dashboard at the end of each lesson to ensure students had filed their work before leaving the room.

Changes in teaching and learning

The use of Google Docs, Google Sites and individual and class blogs by teachers also resulted in a number of deeper changes in pedagogy. Teachers reported that Google Docs allowed them to comment on students’ work in real time, identifying individual learning issues in a timely manner, and reducing the amount of marking that was required outside of the classroom:

I now have online and real-time conferences with students.

There is less marking to do as I can comment on a student’s document as they work.

My feedback to students has increased: I can now comment at the start, middle and end of tasks.

By the final trimester of 2011, ten out of the eighteen teachers were using Google sites to co-ordinate their class and/or subject work. Starting out as an online taskboard, the sites developed into a richer planning centre that was shared and sometimes co-constructed with the students. The sites combined information about learning intentions, group activities, instructions, exemplars, external links and follow-up work. Analysis of videotaped lessons showed that teachers who used the sites issued fewer instructions and focused their talk with students around learning matters rather than on managing or directing activities.

Changes in student engagement

Students were strongly motivated and excited to have their own laptop. Observations showed a high level of on-task work, while teachers reported less noise, increased ownership of learning, and more frequent task completion. Analysis of videos showed evidence of student self-management, independence and critical thinking (three of the key competencies in the New Zealand curriculum).

In addition, teachers noted that the use of Google Sites had made their students more independent. This was borne out by the video observations: Students in the classes where class Sites were used were less reliant on the teacher than those in the other classes. They completed more work either independently or with peer support, allowing them to move on to activities that extended their thinking.

Instead of finding out what to do from the teacher we look on Google Sites – it’s a lot easier for us and the teacher (student blog post)

I spend less time on organisation – what each group is doing and what to do when finished – it is all on the site (teacher)

Each student published work on a personal learning blog, averaging one post per fortnight. This represented a 400% increase on the previous posting rate, when students had to take turns posting on a class blog. The majority of posts on the individual blogs were presentations of schoolwork using text and photos, together with some movies and animations. Teachers noted a strong level of student engagement and motivation surrounding this work:

The impact of kids actually getting something published onto their blog is huge… when they do post and see it published they get so enthusiastic and want to do more

Children want to complete work – they even ask, “Can we have time to finish so I can get it on my blog?”

Changes in student achievement

The focus in monitoring student achievement in the study was not on learning outcomes per se, but on the “shift”, or improvement, in learning outcomes that was made by each cohort at regular intervals during the study, as measured by the national assessment tool, e-asTTLe. The significance of these shifts was ascertained by comparing them with the “expected shifts” for students at each year level, based on aggregated national statistics.

The improvements in writing achievement were significant:

Although all of the cohorts achieved below the expected national level in writing, the shifts in achievement during 2011 ranged from just above the expected shift to six times the expected shift. For Pasifika students the shifts at all year levels ranged from twice to seven times the expected shifts. The shifts in writing by Maori students were from slightly higher than the expected shift to eight times the expected shift:

In reading there was a range of shifts in achievement, from just below the expected shift to three times the expected shift. Boys in years 6,7 and 8 were found to have made twice the expected shift. Overall, the girls achieved slightly higher scores in reading than the boys at all year levels.

Summary

After one year, the second stage of the Manaiakalani Project, based around the use of individual laptops and Google Apps tools, resulted in significant improvements in student engagement and self-management, as well as strong positive shifts in writing achievement. Improvements in reading were also evident. The use of Google Sites and Teacher Dashboard led to new teaching practices that resulted in more effective use of classroom time. Teachers were able to focus on supporting learning rather than managing activities, using the tools to build shared responsibility for learning, monitor group and individual learning progress, and provide just-in-time one-on-one feedback to individual students.

Note: Only four of the seven schools in the study used the same version of the e-asTTLe assessment tool. This meant that student achievement data could only be compared across these schools. The assessment tool is being aligned across schools for the rest of the study.

Complete report can be downloaded from the Manaiakalani Project Web site.

For more information, please contact us.

If you have not seen it already, the Pt. England School in Auckland NZ put together a great resource site with loads of advice and policies they’ve used to manage their 500+ blogs over the last few years. This is a must-read.

Using Teacher Dashboard Blogger views

Teacher Dashboard presents two views into student blogs: a Posts view and a Comments view.

Posts view

The Posts view shown above displays the most recent posts in each student blog, showing the post title, status indicator (Draft), the author of the post, and the time elapsed since last update.
Clicking on the post title opens the blog and the post in a new window. Moving the mouse over the title link displays a popup with the content of the post, including images, videos, etc.

The Posts view makes it easy for the teacher to very quickly scan the student Blogger work.

Comments view

The Comments view displays the most recent comments left on each student’s blog, and shows the comment title, the ID of the post the comment was made on, the author, and time elapsed since the comment was made. Clicking on the comment title opens a popup window with the post / comments open. Moving the mouse over the comment title link displays a popup with the full text of the comment.

The Comments view makes it easy to quickly ascertain who is commenting on student blogs, whose blogs have not had comments in a while, and how students react to the comments left for them (i.e. are these conversation starters, etc.)

Setting up student blogs for use with Teacher Dashboard

For the moment, student blogs need to be set up manually; Google is working on this and we expect this will change in the near future.

There are a few (very important) things to keep in mind when setting up student blogs:

Always, always, always:

1. create the blogs using an institutional domain account (school.apps.owner account created for Teacher Dashboard)
2. use a consistent naming strategy along with a URL prefix that distinguishes between your student blogs vs. class blogs for example.
   The recommended naming strategy is: schoolprefix-blogtype-partialstudentemail, resulting in URL’s like “pte-sb-anne9fkj1″. Partial student email address can be the address except the last few characters (before the @) to ensure the blog URL’s are unique but not expose the student’s email address to the public
3. give teachers admin rights but make absolutely certain they have a demonstrated understanding of Blogger basics, particularly blog monitoring and moderation
4. give students author rights only

Teacher Dashboard assumes that each student portfolio blog is shared with one and only one student as author.

The student blog URL prefix (“pte-sb-” from above example) needs to be added to the Classes spreadsheet, in a “Student Blogs” column, for any class whose teacher should see the student blog in their Teacher Dashboard (i.e. generally for all classes). Once the spreadsheet is updated, reload it using Teacher Dashboard Console – owing to caching there may be a 15 to 30 minute delay in the blog views appearing on the Dashboard.

If the student blogs are already in place and no naming convention is in place, the value of “any” can be put in the “Student Blogs” spreadsheet column to allow Teacher Dashboard to simply display the first blog it finds author rights to for each student.

We are looking to open source our implementation of Hapara codes to allow other schools to easily generate safe and robust student email addresses outside of Teacher Dashboard.

What do we want from a student email address?

A student email address must:

1. Protect the student’s identity i.e. not disclose or allow disclosure of the full student name or other identifying details (like student ID).  For this reason we cannot use the student’s first and last name (i.e. james.neutron@myschool) or the school/national ID (i.e. 38292747@myschool), or direct derivations thereof (so an attacker with full knowledge of a single student’s details could not use these to “reverse engineer” personal details of another student)

2. Be personalized: fully “synthetic” codes are impersonal and impede collaboration  (i.e. S828273@myschool)

3. Be respectful: the ID should not arbitrarily cut the student’s name.  In many cultures the names have actual meaning, and mechanical abbreviations can result in unintended consequences and be perceived as culturally insensitive.  To avoid mechanical abbreviations, a separate field for PREFERRED name needs to be maintained and used to store a shorter (or just preferred) alternative)

4. Automatically generated based on student info held by the SIMS/SMS – it’s impractical to ask students to pick a unique “handle” to personalize their account (i.e. seargent_pepper@myschool)

5. Be unique over tenure and scope of the account (tenure may be 1 to 14 years, scope may be just one school, a school district, or nation-wide – see point 9 below).  This means that the generated ID has to include a synthetic element derived from a unique stable identifier – some aspect of student’s identity that is unlikely to change over their tenure (like national student ID or school student ID #).  For this reason using academic years or class names to distinguish students isn’t a good idea (i.e. jamesY11R15@myschool).  Student ID’s cannot be used directly owing to 1) and 2)

6. Be as short as possible:  students will be increasingly expected to use this login, potentially multiple times over a day.  For this reason, the “synthetic” component of the ID must be kept as short as possible.

7. Be memorable and easy to communicate to student, and should therefore avoid any special characters (_+.={}() etc) and for the synthetic component, avoid illegible letter combinations (ill1 0O, etc.)

8. Be appropriate: the ID should not spell out anything questionable.  This is generally not an issue with names, but can become a problem when names are combined with synthetic codes (i.e. jamesISSY@myschool)

   OPTIONALLY:

9. Be portable from school to school to allow students to retain their identity while switching domains.  If the scope of the ID must span schools, the synthetic component of the ID must be derived from a SIMS/SMS data element

Hapara Student ID Codes

The Hapara codes are designed to be personalized yet protect the student’s identity, be short yet highly unique, and reduce the likelihood of inappropriate text being generated:

anne8f4ht@

anne8f4ht

Name prefix consists of the first name or preferred name (if provided) to ensure the account is personal to each student. Prefered names can be used to shorten compound first names or where the legal name isn’t commonly used.

anne8f4ht

Synthetic code contains a subset of lowercase digits and letters.  The code excludes vowels and starts with a digit to reduce the likelehood of inappropriate text being generated, and excludes characters that could be easily misread/confused.   The algorithm factors in the student’s last name and uses non-reversible mathematical constructs to make it more difficult to reverse-engineer national or school ID numbers based on the email address. 

The length of the synthetic code can be adjusted to make the resulting ID more unique:

What data is needed to generate Hapara codes?

Data needs to be complete, accurate, consistent and and stable.  Our algorithm requires the following data fields:

• Legal last name
• Legal first name
• Preferred name
• Stable ID: national student identifier, or school ID

Note that a other name-related fields are not needed (prefix, suffix, middle initial).  These may still be needed to associate a full name with the account ID (if desired), but are not used in generating a student ID.

Completeness of student data is usually mandated by regulatory requirements.  Records missing the above data elements (with the exception of preferred name, which is only used when populated) cannot be used to generate a Hapara code.

Accuracy is often a reflection of how the data is sourced and verified, and administrative maturity of the school.  Best practice and increasingly, legal requirement, is to only use specific forms of legal documents to populate name fields.  If students (and parents) do not actually use the legal name, the preferred name field should be used.  Experience suggests that students and parents are not a very reliable source for accurate legal name data :)

Consistency in name data relates largely to capitalization and use of punctuation.  Legal names are rarely all uppercase, and automated conversion requires considerable caution.  Punctuation, especially hyphenation of compound names, is also fraught and may require original legal documents to be sighted to correct.  Hapara codes generator ignores punctuation and letter case to reduce the likelihood of ID change due to a cosmetic change.

Stability means that student names are only changed for a valid reason.  Changes to student name has increasing and significant ripple-effect as more and more systems are linked to SIMS/SMS data; many online systems may not cope with an account rename operation gracefully, possibly resulting in loss of student content.  At minimum, admin staff responsible for day-to-day management of SIMS/SMS should be made aware of the implications of even seemingly “cosmetic” updates; realistically, some formal policies should be set in place around how student identity data is accepted and verified by the school.

Teacher Dashboard account ID generation

The Teacher Dashboard Console, Configuration, Account Creation page can be used to configure how Teacher Dashboard deals with student ID’s.  You can select to use your own ID’s, or have TD generate Hapara codes of selected length for students.

Please note that if you select that Hapara codes are to be generated, you must provide the school or national identifier in the student data.  Records without the appropriate code will be generate warnings on load and will not be processed.

The cat’s out of the bag :) … it’s fantastic to see our early adopting classroom teachers and partners talking about Teacher Dashboard. It’s hugely rewarding to see the interest (and the look of relief on teachers’ faces!) when we demo the tools. If you haven’t seen it yet, Maria from Watchdog will include Teacher Dashboard during her Google session on Friday morning.

Teacher Dashboard is an add-on to Google Apps Education Edition that makes it easier for teachers to deal with their classroom Google Docs, Sites, Gmail and Blogger content.

“Teacher Dashboard eliminates the confusion and massive time drain involved in managing classroom online content. Just keeping track of Google Docs for 30 kids is a real killer, involving hours of effort and tears just navigating individual student folders, not to mention cajoling students to get their content in the right place.” – Russell Burt, Principal, Pt. England School

Teacher Dashboard gives teachers a bird’s-eye view of the entire classroom:

The screen above shows the teacher the last few documents worked on by a each student, broken down by subject, along with some additional information:

In the example above, it’s clear that Sandy has started her Marco Polo presentation, but hasn’t touched the Volcano Report for a week. The teacher can also readily see that Sandy is a Year 6 student (indicated by the number next to her name), and is nearly 11 years old. As this is a mixed-year class, Patrick is a Year 5 student and is nearly a year younger – information that will probably affect how their work is evaluated.

The tabs above reflect the subject areas being worked on in Room 20, as defined by the school or the teacher. Beyond Google Docs, Teacher Dashboard can provide a similar view across:

• Student’s personal Google Sites (a personal e-portfolio site maintained by each student)
• Contributions to a classroom site
• Individual student’s blogs (on Blogger)
• Classroom blog (on Blogger)
• Student’s Gmail inbox and sent mail

Where the underlying Student Management System provides access to the information, we are able to make even more functionality available:

In the example above, Teacher Dashboard displays subject-specific assessment information for each student ((Reading Age in this instance), and provides one-click access to each student’s SMS record.

To allow all this happen, Teacher Dashboard requires the Google Apps environment to be set up consistently – and while this is easy to do for a small group of students, it becomes a significant problem when dealing with multiple classrooms or a school. We address this problem through other tools our larger Managed Learning Environment for Google Apps, of which Teacher Dashboard is just one component.

Our Managed Learning Environment for Google Apps provides the ability to automatically synchronize your Google Apps student accounts and class groups with your Student Management System, to ensure that your students can use a single account and password to access all school resources (online and in-school), and that the classroom and student environment is consistent and set up automatically.

Once students have their individual accounts created, the tools can configure each student’s environment to suit the classroom requirements. This can include set up of personal or shared folders, creation of personal Google Sites or access rights to the class site, etc.

The student view of Google Docs show above illustrates the automatic sharing of student content (with the teacher, and the school admin accounts), and the folders that have been created for the student as part of account setup. The shared subject folders are configured at a per-classroom level and cannot be deleted by students.

The MLE/GA can also provide policy enforcement to ensure that student content is shared appropriately, or that teachers (and admins) are automatically notified if inappropriate content is found.

Teacher’s Dashboard and the MLE/GA are fully cloud-based solutions powered by Google. There is no hardware or software for you to install or manage, and once in-place, there is no specific maintenance required on your part.

Are you interested in the Teacher Dashboard and the Managed Learning Environment for Google Apps at your school? Let us know and we will put you in touch with a local implementation partner. It usually takes only a few days to get going and give your teachers some relief.

Teacher Dashboard was developed in partnership with and based on the Google Apps experiences of teachers at the Pt. England school in Auckland. If you’re interested in how the tools fit into a broader pedagogy and a community-wide urban development programme please see www.manaiakalani.org and manaiakalani.blogspot.com.

Here it is, our favorite reason to NOT go with Google Apps: Corporate Email Skills.

We believe that “getting prepared to use Corporate Email” is pretty much the antithesis of “getting prepared for your future”. The attitude so beautifully summarized in this Live@Edu tweet is precisely what got our education IT systems to where we are now, and a clear indication of what’s driving Live@Edu.

Attitude aside, arguing specific application features is not very useful- they are all fast-moving targets. What may actually be important to you is getting your data out, and ability to use tools like Android devices, iPads and the like. Or even Firefox. Before jumping into Live@Edu, find out what happens should you chose to migrate away from Microsoft, or pick one of the cool new ChromeOS netbooks for use in your classrooms.

Plug: we help educational institutions migrate to Google Apps, and build tools that make Google Apps easier to use in the classroom, and reduce admin effort for schools and school districts. Contact us to find out more!

Congratulations to the delivery team at Tait Radio and WaveAdept on great Google Apps implementation!

Really cool video overview of the process is available at http://www.waveadept.com/2010/tait-radio-goes-google-with-waveadept-video/.

Tait Radio is also a Hapara Access Manager customer: welcome to our community!

Innovative New Zealand cloud applications specialist, Cloudbreak, is renaming to Hapara and going global with its portfolio of cloud products and services that provide enhanced functionality for large enterprise deployments of Google Apps.

“Our evolution from a cloud consulting to cloud products business was in the plan from the beginning,” says Jan Zawadzki, CEO Hapara, “we encountered the same customer needs during a number of implementation projects and have sought to leverage the power of Google App Engine platform to extend the capability of Google Apps even further.”

Hapara is New Zealand’s most experienced Google Apps implementation company with over 20 Google and enterprise deployments under their belt – including the provision of specialist technical advice to New Zealand Post, and the largest Google Apps deployment in the country to Unitec’s 46,000 users.

“Hapara’s specialist expertise has helped us to leapfrog capabilities for our students,” says Unitec’s Director of Information Technology, Peter Winquist.

The company’s international go to market plans have been brought forward several months due to overwhelming interest in Access Manager for Google Apps, the first Hapara product to be released. Information on the product has gone viral and been picked up via the global grapevine of user group forums and social media channels. Hapara has been receiving daily enquiries from large organisations in North America, and Europe. The company will be announcing the global availability of Access Manager in the Google App Marketplace within the next few weeks.

Access Manager is an enterprise management add-on for Google Apps that simplifies the administration of large scale enterprise Google App deployments, and supplies the essential features required by security and network administrators. The product is purchased as a cloud service. Features include real-time password synchronization with systems like Active Directory and OpenLDAP, desktop single sign-on, two-factor authentication using mobile phones, integration with Facebook and the ability to customise and track access to login screens. The outcome for users is easier and more secure access to their mailbox using the same username and password they already use for logging into their enterprise’s network.

“What makes Access Manager unique is the way it leaves the enterprise in control over the management and storage of their own identity credentials,” says Mr. Zawadzki.

In just under 6 months since the initial release, Access Manager has gained over 45,000 users across 6 organisations including Unitec and a major New Zealand electronics company.

Access Manager is the first of several Hapara cloud applications to be released.

Tracy Voice, CIO at New Zealand Post (our client!),
talks about their experiences in migrating to Google Apps