This system sees the integration of Moodle and Turnitin within the university’s Exeter Learning Environment (ELE).  Assignments are submitted through the ELE, assigned an originality score by Turnitin, then available for marking through GradeMark (a commercial online marking system within Turnitin) or MS Word markup.  Feedback is returned to students either via uploaded forms or bespoke feedback forms, and are made available for viewing by both individual students and the personal tutor assigned to support them.  Initially deployed through a small 2011 pilot project funded by HEFCE, the system is now available institution-wide, although for practical reasons this evaluation project will concentrate on working with smaller groups across various disciplines.

Exeter’s Moodle support is provided by the University of London Computer Centre, who are developing the interface between Moodle and Turnitin.  There is strong internal support for the system which will be maintained and further developed well beyond the lifetime of this one year project.  What the OCME project will provide is a series of reports and briefing papers which will explore the pedagogic, technological and institutional aspects to transforming practice, and guidelines for future implementers and for those considering introducing such transformative technologies within their own institutions.  The experiences and lessons learned from this project should be of value across the sector.

The dual focus on administrative and pedagogic aspects is crucial for real uptake of any new technology or process.  By providing a supportive administrative and technological infrastructure, institutions can enable academic staff to fully realise the benefits of innovative systems and practice, and provide a significantly enhanced learning environment for students.  The dynamic interplay of these factors is vividly illustrated in the poster the project submitted for the programme kick off meeting.  The impact on student satisfaction, achievement and retention rates already apparent at Huddersfield reflects the success of such an approach.

Like the Evaluation of Assessment Diaries and GradeMark at the University of Glamorgan project, EBEAM is grounded in previous evaluation work investigating the benefits of Turnitin on staff and students.  As with other projects, the decision to adopt existing technologies incorporated through the institutional VLE (in this case, Blackboard) is a pragmatic choice, adopting known and proven technology rather than expending time and resources in developing yet more tools to do the same things.  Being able to pick up such tools as needed greatly increases institutional agility, and provides ready access to existing user groups and a wealth of shared practice.

EBEAM project staff also have a keen awareness of the need for meaningful and effective staff development to enable teaching staff to make full use of new technologies and achieve the integration of new approaches within their teaching practice, a theme covered in several posts on their excellent project blog.  The project will produce a wide range of development materials, including practically-focused toolkits, webinars and screencasts, which will be available through the project site and the JISC Design Studio.  In addition, they’re looking at ways of fully exploiting the extensive amount of data generated by these EAM systems to further enhance teaching and learning support as well as engaging administrative departments in discussions on topics such as data warehousing and change management.

The EBEAM project should provide an excellent study in the benefits of eassessment and of methods of integration that take a holistic approach to institutions and stakeholders.  I’m very much looking forward to seeing the outcomes of their work.

Students studying on the Teaching Qualification (Further Education) programme at Dundee since the start of the 2010-11 session have been supported by a centralised tutor system that enables consistency and timeliness of feedback across the entire programme.  TQFE-tutor consists of a centralised email account, blog and microblogging site to which all tutors on the course have access.  Rather than students being assigned a personal tutor (who may have as little as 0.1 FTE allocated to the programme), support is provided by the entire team acting through the centralised account.  Students may email the TQFE-Tutor email address or post comments via the programme blog, with duty staff picking up queries and assignments as they arrive.  Programme announcements can be disseminated via the programme Twitter account, offering time and potentially cost savings.

As well as significantly increasing efficiency – students are guaranteed a response to any submission within two days, and usually receive one much faster – there are more subtle but equally important pedagogic benefits.  Feedback and advice provided to an individual can then be disseminated, suitably anonymised, to the rest of the current cohort via the TQFE-tutor blog; these entries also remain available for future years.  The accumulation of an effectively tagged bank of data supports independent learning while peer interaction and support enriches the learning process.  The use of Blackboard Safe Assign in place of paper submission has also helped streamline the assessment process and reduce administrative workloads.  Student achievement rates have risen, and the system may well contribute to increased retention.

You can follow the project’s progress over the next few months via their project blog, from which project outputs will also be available in due course.

Arising as a result of a Change academy project running from 2008-10, the use of assessment diaries for scheduling and planning assessment, and GradeMark for online marking, have been adopted across the institution to various extents within different schools and faculties.  This new JISC project will examine the reasons for variation in adoption and explore staff and student experiences of these technologies as well as exploring strategies for staff development to encourage wider uptake.

The assessment diary system is a very simple, but very elegant approach to dealing with the issue of assessment bunching, identified by Glamorgan students as a major issue undermining learning and assessment performance.  Initially the problem was addressed simply by widely sharing assessment due dates for all courses within staff Outlook calendars, but considerable development work since then has resulted in an MS Access database backend with a web interface in Blackboard.  Seamless integration with Blackboard through a building block provides a single point of access to this information which can be fully personalised by both students and staff.  This provides a highly visual way of understanding the rhythms of course workload and has been very successful in helping students manage their time and plan their work in relation to assessment deadlines.  Staff have also found that the ease of access to detailed information on course timings has facilitated dialogue amongst staff across a range of courses and reflect more effectively on the student learning experience when redesigning or rescheduling modules.  Those departments that were early adopters of the diaries have seen significant improvement in their National Student Survey scores for feedback, and the diaries are also believed to have positively impacted on student retention rates.

By contrast, GradeMark is a commercial online marking tool within Turnitin that is gathering increasing adoption in the UK.  Adopted across the University of Glamorgan since 2009, it is seen as addressing a number of student dissatisfactions with feedback, including timeliness, level of detail and, importantly, the development and maintenance of a meaningful dialogue between both students and teachers, and amongst teachers themselves.  Both students and staff have responded very positively to the tool in an initial evaluation of its impact, and the current project will be able to explore its impact in greater detail.

Project outputs will include a series of video interviews with students and both teaching and administrative staff which will be freely available as OERs on YouTube, together with Panopto recordings of staff development and training sessions for asynchronous viewing.  The team is also exploring the use of online avatars for staff development discussions and scenario roleplaying, an exciting approach that has worked very successfully at the University of Hong Kong and which I hope to follow up in a later post.

The project’s blog is well worth a read to see how the team go about their evaluation and some of the issues they encounter on their journey.

As part of a consortium providing a significant amount of European funded work based learning in Wales, Coleg Sir Gâr were keen to find a solution that would meet both European and Welsh Assembly Government requirements for hand written signatures as well as providing the elegance and efficiency of the online learner management and learner support systems colleges and tutors wished for.

The Secure Work-Based Learning Administration through Networked Infrastructure (SWANI) project, funded under the JISC Learning and Teaching Innovation Grants SWaNI FE programme, therefore set out to identify ways of addressing this tension and establish a pilot project as a proof of concept to form the basis of a long term solution.

After some research the project team settled on the Fastdox digital document system as offering exactly the combination of hand signed originals and timestamped digital copies necessary to meet the needs of all parties.

The documents to be signed are created in a MySQL database supported by a very user friendly and remotely accessible web interface.  These are then printed using the Fastdox software which applies a unique pattern of microscopic dots to the physical document to communicate with the digital pen.  The pen functions just like an ordinary pen, allowing trainers to sign the documents normally and therefore produce the required hand signed physical document, but the pen also stores all the written information, time stamped, for later downloading into the online learner management and auditing system: an excellent overview of the entire process is available from the product site itself and an exploration of how it was put into practice can be found on the project’s blog.  At between £4-500 for each pen and software package it represents a one-time investment that fulfils a long term requirement, requires little training for tutors to use and meets all the requirements the project set out to address – indeed, the biggest problem the project team ran into was the lack of standardisation in documents across WBL providers and changes to the document design part way through the project which required some revision.

With the pilot now coming to a close, the project team will be adding further information to the project website and undertaking a series of dissemination activities.  Their solution should be useful not only to FE colleges with similar funding and auditing requirements but for anyone looking for efficient and effective digital document management and tracking.

John Winkley of AlphaPlus Consultancy, who has been working with JISC as an expert consultant in the area of assessment, opened the meeting by introducing delegates to a number of funding opportunities in the domain that JISC will be releasing in the next few weeks.  These opportunities include at least two and up to four demonstrator projects, funded to build on and further develop outputs from earlier JISC toolkit activities, and two Invitations to Tender for desktop research studies.  These studies will look at advanced eassessment techniques, and at quality concerns around eassessment.  The demonstrator projects must be led by a HEFCE-funded institution, while the ITTs will be open to all bidders including Scottish insitutions, FE colleges with less than 400 HE students, and the private sector.  All work is due to be completed by March 2009, and will add considerably to JISC’s portfolio of work in this area.

One project which has benefited from JISC funding for part of its lifetime is the WebPA project based at Loughborough University.  Nic Wilkinson presented the successful peer assessment system to delegates, illustrating some of the reasons for its success at the recent IMS Learning Impact Awards in Austin, Texas.  One of the most signficant factors in the system’s ongoing success is the effort the project team have put into attracting and supporting a signficiant number of participating organisations that have now integrated the system into their own teaching practice.  It was also extremely interesting to learn how positively the students themselves have responded to the system, and their attitudes towards the anonymity of peer marks: the system awards each member of a group an aggregated mark derived from the individual scores awarded by their peers, and students are reported to not want to receive individual marks in order to avoid potential clashes outside the classroom.

After the break, Karim Derrick of TAGLearning discussed a proposed British Standard for managing the transmission of coursemarks and portfolios of digital evidence of coursework between schools and awarding bodies.  Based on TAG’s extensive experience in this area, the proposed standard includes ‘an XML schema for describing the relationship between components, options and exam specifications’ and a ‘universal translator’ API to support data exchange between the various systems used by exam centres and awarding bodies.  Although the current focus for this work is firmly on the schools sector, if adopted it’s not hard to see how it could be extended to support the universities admissions process and external marking at all levels, particularly in vocational courses where a single accrediting body has to deal with substantial amounts of data. 

Alan Paull of APS Ltd closed the morning with a lively journey round the admissions domain landscape and the DELIA project.  DELIA enables the sharing of enhanced learner information as part of the admissions process, enabling admissions officers to make more informed decisions when evaluating borderline applications.  This not only improves the quality of the admissions process, enabling a closer matching between applicants and course requirements, but can have a positive impact on subsequent retetention of such students.

The afternoon featured a special session on student retention, looking at a range of issues around the topic and attempting to capture requirements for work in the domain.  Simon Grant of JISC CETIS and the Centre for Recording Achievement led an interactive session that asked participants to consider self-assessment of suitability for courses and the different personas we adopt as our contexts change.  Simon also touched on some of the problems that arise when our different personas come into conflict, a situation which can be exacerbated by the widespread use of social networking services and individuals’ lack of awareness of the potential implications of forgoing privacy when using them. 

Helen Richardson, also of JISC CETIS and the Centre for Recording Achievement, closed the day by discussing some of the findings of the STAR project and the National Audit Office’s report on student retention.  The STAR project produced a detailed series of guidelines to help support students both before and during their university careers, including the use of technologies such as SMS messaging to aid this.

We’re grateful to all our presenters for sharing their work with us and for being so willing to respond to questions and comments from the audience, and to all those who attended on the day and helped to make it a success.

The agenda includes the usual mix of news and updates, project presentations and discussion sessions, plus a special themed requirements gathering session focused on the pressing issue of student retention.  Myles Danson of JISC opens the day with a heads-up on forthcoming Invitations to Tender in the assessment domain, a topic that is always of great interest.  Nicola Wilkinson of the WebPA project, based at Loughborough University, will introduce their Learning Impact Award-nominated system, while Alan Paull will discuss the University of Nottingham’s DELIA project on admissions.

The admissions process is also the focus of proposed BSI standardisation work for the transmission of digital evidence and assessment data between schools and awarding bodies to be presented by Karim Derrick of TAG Learning.

The afternoon will feature presentations and discussions on student retention aimed at gathering requirements, recommendations and priorities for future activities, led by our own Simon Grant and Helen Richardson and building on the work of the STAR project and the National Audit Office.

As always, the meeting is free to attend, with lunch and refreshments provided.  It’s open to all, and we just ask that you register in advance to secure your place.  We look forward to seeing you there!