The e-Feedback Evaluation Project

Assessment of language learning naturally presents some unique challenges for both teaching staff and learners.  Regular practice of both spoken and written language production is a vital part of language training and requires a significant amount of ongoing feedback to support the acquisition of competence in the subject.  In a distance learning context in particular, but similarly in any setting where feedback is provided asynchronously rather than face-to-face, providing meaningful feedback on spoken texts especially is challenging, often requiring spoken feedback to correct pronunciation and structuring errors.

There have been a number of exciting projects around audio feedback in recent years, including the Optimising Audio Feedback project at Aberystwyth University, Sounds Good at Leeds (both funded by JISC) and Audio Supported Enhanced Learning, a collaboration between the Universities of Bradford and Hertfordshire.  The focus of the eFeedback Evaluation Project (eFEP), however, is the impact of the combination of both spoken and written feedback on language learning.

The eFEP project is led by the Department of Languages at The Open University, an institution with unique experience in providing language training through distance learning, a large part of which involves teaching through both formative and summative assessment and feedback.  The OU has a mature and robust eTMA (electronic tutor marked assignment) system which supports assessment across the institution, and provides feedback either via MP3 files or marked-up MS Word documents, as appropriate for the individual assignment.  Each form of feedback is supplemented with an HTML form (an example of which can be seen on the poster submitted by the project to the programme’s startup meeting) containing administrative information, marks awarded and additional feedback.

The project will examine the ways in which students and tutors interact and engage with their feedback, identify common perceptions and issues, and recommend areas requiring further support and guidelines for good practice.  In order to examine the applicability of this feedback approach in traditional settings, the project will also look at the impact of audio feedback in Italian modules at the University of Manchester.

The insight into the use of audio feedback across a variety of environments, and the range of training and support materials to be produced, should make eFEP a valuable addition to our understanding of the value of audio feedback as well as offering clear practical guidance to those considering adopting it.

Online Coursework Management Evaluation

The University of Exeter has developed an entirely online end-to-end coursework management system which is the subject of the Online Coursework Management Evaluation (OCME) project funded by JISC as part of the Assessment and Feedback programme Strand B.

This system sees the integration of Moodle and Turnitin within the university’s Exeter Learning Environment (ELE).  Assignments are submitted through the ELE, assigned an originality score by Turnitin, then available for marking through GradeMark (a commercial online marking system within Turnitin) or MS Word markup.  Feedback is returned to students either via uploaded forms or bespoke feedback forms, and are made available for viewing by both individual students and the personal tutor assigned to support them.  Initially deployed through a small 2011 pilot project funded by HEFCE, the system is now available institution-wide, although for practical reasons this evaluation project will concentrate on working with smaller groups across various disciplines.

Exeter’s Moodle support is provided by the University of London Computer Centre, who are developing the interface between Moodle and Turnitin.  There is strong internal support for the system which will be maintained and further developed well beyond the lifetime of this one year project.  What the OCME project will provide is a series of reports and briefing papers which will explore the pedagogic, technological and institutional aspects to transforming practice, and guidelines for future implementers and for those considering introducing such transformative technologies within their own institutions.  The experiences and lessons learned from this project should be of value across the sector.

Evaluating the Benefits of Electronic Assessment Management

Examining the embedding of electronic assessment management (EAM) within both administrative and teaching and learning practice is the main focus of the Evaluating the Benefits of Electronic Assessment Management (EBEAM) project running at the University of Huddersfield as part of the JISC Assessment and Feedback programme Strand B.  This 18 month project will look at how Turnitin, incorporating GradeMark and eRater, addresses student, staff and institutional requirements for timely, invidiualised and focused feedback, reduced staff workloads and increasing reflection on practice, and cost-effective, scaleable and sustainable innovation.

The dual focus on administrative and pedagogic aspects is crucial for real uptake of any new technology or process.  By providing a supportive administrative and technological infrastructure, institutions can enable academic staff to fully realise the benefits of innovative systems and practice, and provide a significantly enhanced learning environment for students.  The dynamic interplay of these factors is vividly illustrated in the poster the project submitted for the programme kick off meeting.  The impact on student satisfaction, achievement and retention rates already apparent at Huddersfield reflects the success of such an approach.

Like the Evaluation of Assessment Diaries and GradeMark at the University of Glamorgan project, EBEAM is grounded in previous evaluation work investigating the benefits of Turnitin on staff and students.  As with other projects, the decision to adopt existing technologies incorporated through the institutional VLE (in this case, Blackboard) is a pragmatic choice, adopting known and proven technology rather than expending time and resources in developing yet more tools to do the same things.  Being able to pick up such tools as needed greatly increases institutional agility, and provides ready access to existing user groups and a wealth of shared practice.

EBEAM project staff also have a keen awareness of the need for meaningful and effective staff development to enable teaching staff to make full use of new technologies and achieve the integration of new approaches within their teaching practice, a theme covered in several posts on their excellent project blog.  The project will produce a wide range of development materials, including practically-focused toolkits, webinars and screencasts, which will be available through the project site and the JISC Design Studio.  In addition, they’re looking at ways of fully exploiting the extensive amount of data generated by these EAM systems to further enhance teaching and learning support as well as engaging administrative departments in discussions on topics such as data warehousing and change management.

The EBEAM project should provide an excellent study in the benefits of eassessment and of methods of integration that take a holistic approach to institutions and stakeholders.  I’m very much looking forward to seeing the outcomes of their work.