QuestionMark award winners at LearnX Asia Pacific

Congratulations to all at QuestionMark, whose flagship product Perception was named ‘Best Assessment Tool’ at the LearnX Asia Pacific 2008 eLearning and Training Awards held in Melbourne, Australia a couple of weeks ago.  Described by the judges as ‘the best online testing and reporting solution available – stable and actually works’, the award reflects QMP’s maturity, significant share of the eassessment market and ongoing popularity both in the UK and beyond.

Those planning to attend the 2008 QuestionMark European Users Conference on 14-16 September in London still have a few days to register for a £50 reduction – early bird registrations close on 30th June.

SOLAR white paper – innovating assessment

A new white paper from the SQA, Innovating Assessment in Scotland, is now available for download.  Reporting on work undertaken by the Scottish Online Assessment Resources (SOLAR) project using BTL’s ContentProducer, the white paper discusses their methodology for developing and delivering eassessments, analyses benefits, barriers and drivers for the use of eassessment and explores future approaches to the use of large scale eassessment and substantial item banks within and beyond Higher National qualifications.

Design and Development of Digital Closed Questions

A significant new publication from the SURF project ‘Active Learning, Transparent Assessment’ is now available as a free pdf downloadand is recommended as an excellent study of eassessment design and development methodology.

Chapters include a detailed taxonomy of item types based on response structure, mapped to other taxonomies such as QTI v2.x and QuestionMark Perception; an in-depth study of multiple response items; an examination of the inadequacies of existing guidelines for question authors with proposals for an improved set of guidelines with reports on their use in their fifteen case studies; guidance on how to select and enact an appropriate subset of the new guidelines in particular contexts; methodology and task allocation for item and assessment design and development; and a detailed discussion on how the QTI specification can support the design process and innovation in assessment.  Many of the author names will be familiar as leaders in the domain, including amongst others Rob Hartog of Wageningen University (who also edits the book),Ignace Latour of Cito and Pierre Gorissen of Fontys University of Applied Science.

This book is an invaluable addition to the literature on eassessment, and will be of interest to everyone from subject matter experts and educational researchers to educational technologists, systems developers and even managers and administrators.

QTI 2.1 and Moodle

Found all by myself this time :-) was a series of tweets detailing Pierre Gorissen and Steve Lay’s successful integration of the ASDEL QTI Playr with Moodle, meaning that Pierre is now able to run QTI 2.1 assessments within a Moodle course.  This is a signficant step forward for making the specification more attractive to users, given the huge popularity and dynamic community that surrounds Moodle.  It might also make QTI a more attractive alternative to OpenMark within the Moodle community – there are some interesting comments to be found on QTI in the Moodle forums.

QTI 2.1 test authoring

Thanks also go to Adam for pointing out RM’s Test Authoring System which claims to be fully compliant with IMS QTI 2.1, making it one of the earliest commercial products to implement the revised specification.  I couldn’t find a demo to try out, but it is good to see the specification finally being implemented in this type of system and market sector.  Also on the website are also a couple of research reports on the impact of ICT in the classroom which are well worth reading.

QTI in Korea

Things may be quiet at the moment on the release of the final version of IMS QTI 2.1, but there’s been quite a bit of activity in the background around implementations and integration with other systems.

Keris, the Korea Education and Research Information Service (similar to the UK’s JISC), have been quite active within IMS and in February signed a formal memorandum of understanding with IMS to launch IMS Korea.  Amongst their activities is involvement with Teaching Mate, a commercial product which aims to support QTI export by the end of the year.  Also from Keris is a QTI 2.1 player that does import and export QTI 2.1 – it’s in Korean, but Adam cunningly pointed out that you can use the browser status bar to work out what each button actually does.

It’s worth having a look around the Keris site to see the extent to which they support ICT in education.  Of particular interest is EDUNET, the National Teaching and Learning Center established in 1996 which, amongst a range of other services, provides a large range of school-level teaching materials and ‘an online testing service to evaluate students’ achievements’.  Would teachers in the UK welcome a centrally provided eassessment service to support the government’s eassessment targets?

Assessment, Portfolio and Enterprise too

A recent joint meeting of the JISC CETIS Assessment, Enterprise and Portfolio SIGs drew a wide range of participants to discuss topics of interest to all three SIGs.  The morning sessions covered a range of topics that touched on all three domains, while the afternoon was given over to a special session on student retention.

John Winkley of AlphaPlus Consultancy, who has been working with JISC as an expert consultant in the area of assessment, opened the meeting by introducing delegates to a number of funding opportunities in the domain that JISC will be releasing in the next few weeks.  These opportunities include at least two and up to four demonstrator projects, funded to build on and further develop outputs from earlier JISC toolkit activities, and two Invitations to Tender for desktop research studies.  These studies will look at advanced eassessment techniques, and at quality concerns around eassessment.  The demonstrator projects must be led by a HEFCE-funded institution, while the ITTs will be open to all bidders including Scottish insitutions, FE colleges with less than 400 HE students, and the private sector.  All work is due to be completed by March 2009, and will add considerably to JISC’s portfolio of work in this area.

One project which has benefited from JISC funding for part of its lifetime is the WebPA project based at Loughborough UniversityNic Wilkinson presented the successful peer assessment system to delegates, illustrating some of the reasons for its success at the recent IMS Learning Impact Awards in Austin, Texas.  One of the most signficant factors in the system’s ongoing success is the effort the project team have put into attracting and supporting a signficiant number of participating organisations that have now integrated the system into their own teaching practice.  It was also extremely interesting to learn how positively the students themselves have responded to the system, and their attitudes towards the anonymity of peer marks: the system awards each member of a group an aggregated mark derived from the individual scores awarded by their peers, and students are reported to not want to receive individual marks in order to avoid potential clashes outside the classroom.

After the break, Karim Derrick of TAGLearning discussed a proposed British Standard for managing the transmission of coursemarks and portfolios of digital evidence of coursework between schools and awarding bodies.  Based on TAG’s extensive experience in this area, the proposed standard includes ‘an XML schema for describing the relationship between components, options and exam specifications’ and a ‘universal translator’ API to support data exchange between the various systems used by exam centres and awarding bodies.  Although the current focus for this work is firmly on the schools sector, if adopted it’s not hard to see how it could be extended to support the universities admissions process and external marking at all levels, particularly in vocational courses where a single accrediting body has to deal with substantial amounts of data. 

Alan Paull of APS Ltd closed the morning with a lively journey round the admissions domain landscape and the DELIA project.  DELIA enables the sharing of enhanced learner information as part of the admissions process, enabling admissions officers to make more informed decisions when evaluating borderline applications.  This not only improves the quality of the admissions process, enabling a closer matching between applicants and course requirements, but can have a positive impact on subsequent retetention of such students.

The afternoon featured a special session on student retention, looking at a range of issues around the topic and attempting to capture requirements for work in the domain.  Simon Grant of JISC CETIS and the Centre for Recording Achievement led an interactive session that asked participants to consider self-assessment of suitability for courses and the different personas we adopt as our contexts change.  Simon also touched on some of the problems that arise when our different personas come into conflict, a situation which can be exacerbated by the widespread use of social networking services and individuals’ lack of awareness of the potential implications of forgoing privacy when using them. 

Helen Richardson, also of JISC CETIS and the Centre for Recording Achievement, closed the day by discussing some of the findings of the STAR project and the National Audit Office’s report on student retention.  The STAR project produced a detailed series of guidelines to help support students both before and during their university careers, including the use of technologies such as SMS messaging to aid this.

We’re grateful to all our presenters for sharing their work with us and for being so willing to respond to questions and comments from the audience, and to all those who attended on the day and helped to make it a success.

Testing teachers’ key skills

The Training and Development Agency for Schools (TDA) will be working with BTL and AlphaPlus to develop content for the assessment of Qualified Teacher Status skills.  Practice material available online demonstrates the use of BTL’s Virtual Desktop, a Flash-based simulated desktop which can provide a secure, locked-down replication of a desktop environment complete with replicated web browser and email.  The live version also disables right mouse clicks and keyboard shortcuts.  You can read more about this work here, and it’s well worth trying out the practice material to see the simulated desktop in action and its use as a training and assessment environment.

Counting the benefits of eassessment

Another blog post from last month needs updating with the news that the Association of Accounting Technicians (AAT) have joined the SQA in working with BTL in the transition from paper based examination to eassessment.  AAT offer an range of free online training modules using Flash to supplement traditional training materials, have rolled out an eportfolio system and seem to be committed to extending their use of eassessment in the future following a successful pilot study in January this year. 

Assessment meets Enterprise meets Portfolio: three way SIG meeting ahead

The room’s booked, the agenda’s confirmed and lunch has been ordered, so it must be time for another SIG meeting.  This time, the Assessment SIG is joining up with the Enterprise and Portfolio SIGs on 22 May at the University of Strathclyde to look at issues that affect all three domains and areas of overlap between the domains. 

The agenda includes the usual mix of news and updates, project presentations and discussion sessions, plus a special themed requirements gathering session focused on the pressing issue of student retention.  Myles Danson of JISC opens the day with a heads-up on forthcoming Invitations to Tender in the assessment domain, a topic that is always of great interest.  Nicola Wilkinson of the WebPA project, based at Loughborough University, will introduce their Learning Impact Award-nominated system, while Alan Paull will discuss the University of Nottingham’s DELIA project on admissions.

The admissions process is also the focus of proposed BSI standardisation work for the transmission of digital evidence and assessment data between schools and awarding bodies to be presented by Karim Derrick of TAG Learning.

The afternoon will feature presentations and discussions on student retention aimed at gathering requirements, recommendations and priorities for future activities, led by our own Simon Grant and Helen Richardson and building on the work of the STAR project and the National Audit Office.

As always, the meeting is free to attend, with lunch and refreshments provided.  It’s open to all, and we just ask that you register in advance to secure your place.  We look forward to seeing you there!