Category Archives: cetis-systems

Assessment meets Enterprise meets Portfolio: three way SIG meeting ahead

Posted on by

The room’s booked, the agenda’s confirmed and lunch has been ordered, so it must be time for another SIG meeting.  This time, the Assessment SIG is joining up with the Enterprise and Portfolio SIGs on 22 May at the University of Strathclyde to look at issues that affect all three domains and areas of overlap between the domains. 

The agenda includes the usual mix of news and updates, project presentations and discussion sessions, plus a special themed requirements gathering session focused on the pressing issue of student retention.  Myles Danson of JISC opens the day with a heads-up on forthcoming Invitations to Tender in the assessment domain, a topic that is always of great interest.  Nicola Wilkinson of the WebPA project, based at Loughborough University, will introduce their Learning Impact Award-nominated system, while Alan Paull will discuss the University of Nottingham’s DELIA project on admissions.

The admissions process is also the focus of proposed BSI standardisation work for the transmission of digital evidence and assessment data between schools and awarding bodies to be presented by Karim Derrick of TAG Learning.

The afternoon will feature presentations and discussions on student retention aimed at gathering requirements, recommendations and priorities for future activities, led by our own Simon Grant and Helen Richardson and building on the work of the STAR project and the National Audit Office.

As always, the meeting is free to attend, with lunch and refreshments provided.  It’s open to all, and we just ask that you register in advance to secure your place.  We look forward to seeing you there!

SQA licence BTL’s Surpass Suite

Posted on by

BTL Group Ltd have just announced that the Scottish Qualifications Authority (SQA) have licenced their Surpass Suite for authoring, storing and delivering eassessments across a range of qualifications and subject areas.  This follows shortly after news that the SQA worked with BTL to convert a range of English for Speakers of Other Languages materials into interactive electronic versions available both on CD and as SCORM packages.

The SQA has taken an interest in the possibilities of online assessment for some time, with the first Standard Grade assessed online in a 2006 pilot and online access to results available to candidates.  Linn van der Zanden and Bobby Elliott have both presented on some of the SQA’s more innovative work at recent SIG meetings.

IMS release QTI v2.1 public draft 2 Addendum

Posted on by

IMS Question and Test Interoperability v2.1 public draft 2 was released almost two years ago.  Since then there have been a number of implementation activities, including the JISC Capital Programme projects demonstrated to the community last February.  Insights and lessons learned from these development activities have contributed to the QTI v2.1 public draft 2 Addendum now available through the specification webpage.  The Addendum incorporates ‘bug fixes and updates to some of the examples, the specification documents, and the XML schema’ and now offers the best version yet of the specification.

Feedback can as always be submitted through the ‘specification problem and suggestion reporting’ section of the IMS website (registration is required so I can’t link to it directly in this blog) or through CETIS.

Enjoy!

REAPing the benefits of transformation

Posted on by

Attendees at last September’s SIG meeting will remember Martin Hawksey’s lively presentation on the Re-Engineering Assessment Practices in Scottish Higher Education (REAP) project.  Funded by the Scottish Funding Council and supported by JISC, the project explored ways in which technology can be used to enhance and transform assessment practice in large first year university classes, resulting in enhanced learner skills, greater achievement rates, and deeper engagement.

A final report on the project is available, discussing a range of topics such as project achievements and lessons learned, preparing for, managing and coping with large scale organisational changes, the pedagogic principles underlying transformation and a study on the use of electronic voting systems (EVS) and the surprising impact they can make on learning and achievement.

The figures reported are impressive: one course saw mean pass marks rise from 51.1% to 57.4%, another’s examination failure rate dropped from 24% to 4.6%, while a third saw a 10.4% gain in mean examination marks; hundreds of hours of staff time were saved through reductions in lectures, tutorials and the use of online assessments while students actually spent more time ‘on task’, and the nature of staff-student contact became more supportive and facilitative.  Self-assessment and peer assessment gave students more responsibility for and ownership of their learning, to which students generally responded positively.

As the report suggests, ‘these findings suggest that these processes of transformation are a plausible prospect more generally in the HE sector’.

The Horizon Report 2008

Posted on by

The 2008 Horizon Report, fifth in the annual series, is now available online.  A collaboration between the New Media Consortium (NMC) and the EDUCAUSE Learning Initiative, the report examines six emerging technologies that the authors predict ‘will likely enter mainstream use in learning-focused organizations … over the next one to five years’.

The report focuses on six technologies or practices in particular: grassroots video and collaboration webs, predicted to enter the mainstream over the next year; mobile broadband and data mashups (two to three years); and collective intelligence and social operating systems (four to five years).  The emphasis is on educational applications of these technologies, with a range of example projects and products illustrating them in action.

Earlier Horizon reports and other publications can also be freely downloaded from the NMC site.

Assessment in 2008: looking forward

Posted on by

Gales are howling, trains in chaos, so it must be January and time to look ahead to what 2008 has in store…

The final release of QTI v2.1 should be out this spring, and it’ll be interesting to see what uptake is like.  This will be the most stable and mature version of the specification to date, supported by a long public draft stage and a number of implementations.  Angel Learning are a significant commercial early adopter, and other vendors are bound to be looking at their experiences and whether Angel’s embracing of the specification has an impact on their own customer demand for QTI 2.1. 

Other significant implementors of 2.1 are the JISC Capital Programme projects which will be concluding around March.  AQuRate offers an item authoring tool, Minibix provides support for a range of item banking functions while ASDEL is an assessment delivery engine which supports both standalone use and integration with a VLE.    These projects should deliver quality resources to the community which will provide a firm foundation for use of the specification.  There was a sneak preview of these projects at our last SIG meeting.

Talking of SIG meetings, dates for the next two meetings can now be confirmed. 

On 19 February there will be a joint meeting with the CETIS Educational Content SIG in Cambridge.  This meeting will cover a range of shared concerns such as new content related specifications such as Common Cartridge and Tools Interoperability, and innovative approaches to educational material and assessment.  Information about this meeting and online registration will be available very soon.  This will be preceded by a workshop hosted by the Capital Programme projects discussed above.

The focus shifts from assessment as content to assessment as process with another joint meeting on 1 May in Glasgow.  This meeting will be a joint meeting with the CETIS Portfolio and Enterprise SIGs and will offer an opportunity to explore some of the shared issues in these domains.  Again, information on the event will be available on the mailing lists, on this blog and on the website in due course.

Another event of note is the annual International Computer Assisted Assessment Conference on 8 and 9 July at Loughborough.  The call for papers is already out, with submissions due by 29 February.  As always, this should be a lively and important event in the CAA calendar.  Alt-C 2008, Rethinking the Digital Divide, will be held in Leeds on 9 – 11 September; again, the closing date for submissions is 29 February.  There’s also a regularly updated list of non-CETIS assessment related events on the wiki.

And what about the trends for eassessment in 2008?  The results of Sheila’s poll, with a strong emphasis on Web 2.0 technologies and possibilities, do seem to reflect to some extent the comments on the last meeting’s evaluation forms which suggested increasing interest in innovative technologies, signficant concern with transforming and enhancing the assessment experience and direct engagement with teaching and learning rather than the more abstract issues of standards and specifications for their own sake.  It will be interesting to see how the more ‘traditional’ XML-based QTI v2.1 fares in the light of the increasing popularity of mashups and web services in 2008.

Assessment SIG meeting, 26 September 2007

Posted on by

Academics and developers met in Glasgow recently to participate in the most recent Assessment SIG meeting. The very full agenda covered a range of topics, both technical and pedagogic, and presentations led to some lively discussions.

Myles Danson of JISC opened the day by presenting JISC’s views and priorities for eassessment, as well as pointing to some future work they will be undertaking in the domain.

Yongwu Miao of the Open University of the Netherlands discussed work undertaken by the TENCompetence Project, with a particular focus on the relationship between IMS QTI and IMS Learning Design and the work they have done in this area. Dick Bacon of the University of Surrey and the HEA discussed the relationship between different varieties or ‘dialects’ of QTI, exploring some of the implementation and interpretation issues that hinder or break interoperability between systems nominally implementing the same version of the specification. CAL Consultant Graham Smith pleased the audience with news that a new Java version of his QTI demonstrator will be available shortly with updated support for QTI 2.0 items, which should help in the identification and resolution of implementation problems.

Martin Hawksey of the University of Strathclyde presented the work of the Re-Engineering Assessment Practices project. With a focus on real world assessment experiences, including an impressive collection of case studies exploring the impact of transformation within assessment practices, the REAP project was of particular interest to participants. Also of great interest, and perhaps unsuprisingly sparking the greatest amount of debate, was the exploration of ‘Assessment 2.0′ presented by Bobby Elliott of the Scottish Qualifications Authority. Bobby looked at ways in which Web 2.0 technologies can be used to enhance and modernise assessment in ways which can engage and appeal to increasingly digitally literate learners.

The day also featured several demonstrations of tools under development. Niall Barr of NB Software demonstrated his current work, an assessment tool which utilises the IMS QTI, Content Packaging and Common Cartridge specifications, while Steve Bennett of the University of Hertfordshire demonstrated MCQFM, a JISC-funded tool which provides a simple text-based format for converting and editing items between formats. Two more JISC projects closed the day. AQuRate, presented by Alicia Campos and David Livingstone of Kingstone University, is an elegant item authoring tool while ASDEL, presented by Jon Hare of the University of Southampton, is an assessment delivery tool which builds on the R2Q2 project to provide a fuller test tool. A third project, Minibix (University of Cambridge) on item banking, is working closely with AQuRate and ASDEL.

Links to presentations (via slideshare), project websites and other information can all be found on our wiki: http://wiki.cetis.org.uk/JISC_CETIS_Assessment_SIG_meeting%2C_26_September_2007.