Category Archives: Standards and specifications

Under development: QTI-IPS

Posted on by

A couple of months ago, JISC released an Invitiation to Tender for a QTI v2.1 implementation and profiling support project.  A consortium of experts produced the successful bid, bringing together some of the leading experts on QTI in UK HE, and the project formally kicked off this week.  It concludes in mid-September this year.

The consortium is led by the University of Glasgow, and includes experts from the University of Edinburgh and Kingston University, contributions from the IMS QTI working group chairs and tool developers, independent consultants Sue Milne, Graham Smith and Dick Bacon, and input from us here at JISC CETIS.  QTI experts at the University of Southampton are advisors to the project.

A project blog has been set up  which will provide a central point for dissemination to the wider QTI community.  Information on how to get involved with the QTI interoperability testing process is also available there.

The project aims include:

  • Contributing to the definition of the main profile of QTI 2.1;
  • Implementation of the main profile in at least one existing open source test rendering/responding system;
  • Providing support in the use of QTI 2.1 and the conversion of other question and test formats to QTI 2.1 to those developing assessment tools and authoring questions;
  • Providing a publicly available reference implementation of the QTI main profile that will enable question and test item authors to test whether their material is valid, and how it renders and responds.

Follow the project blog for future developments!

Draft briefing paper on IMS Question and Test Interoperability v2.1 now available

Posted on by

With IMS Question and Test Interoperability v2.1 almost ready for final release, this draft briefing paper provides an introduction to the specification based on the most recent public draft available.  It covers the structure and purpose of the specification, and looks at the history and background to it as well as reasons for its adoption and some common concerns and criticisms.  A final version will be released with the final version of the specification; in the meantime, we hope this paper will provide a useful guide to this significantly improved specification.  It is likely to be of particular interest to IT managers, learning technologists and developers interested in online and electronic assessment and new to QTI.

Any comments, corrections or requests for additional content are very welcome, either by commenting here on this blog or by email.

X3D working group goes open

Posted on by

X3D logo

The Web3D Consortium has just announced that its standardisation activities will now be open to the public, enabling non-members to participate in development of the specification at all stages rather than just during the public review prior to final approval.   There is still the opportunity for private discussion limited to consortium members for those concerned about commercial or other factors, but the overall emphasis is clearly on making this as open as possible.

X3D is an open ISO standard for representing information about computer generated 3D environments and objects.  Unlike its predecessor, Virtual Reality Modelling (or Markup) Language (VRML), X3D features integration with HTML, and extends the range of effects supported.  X3D is supported by some high profile systems such as the Blender design tool and Sun’s Project Wonderland.

At a time when closed, proprietary players such as Linden Labs are seeing large numbers of layoffs, with inevitable concern from Second Life’s active education community about the potential loss of a huge amount of work and resources should this trend continue, adopting an open approach to development seems a very sensible decision.

IMS QTI implementation survey results and implications

Posted on by

IMS have now released (registration required) the results of the QTI 2.1 implementation survey launched in January, looking in great detail into current implementations of the specification.

Twenty-one implementations are covered by the responses, representing a wide range of approaches to implementation, and the actual responses are available for download for those interested.  The responses support the notion of a core set of basic features implemented by all respondents, with broader parts of the specification being implemented on a more individual basis.

The results are feeding in to the development of profiles for QTI 2.1:

  • Base QTI Profile, covering the features available in the most comprehensive implementations;
  • CC-QTI, which updates the functionality covered by the QTI 1.2.1 profile within Common Cartridge 1.0 and which will be integrated into a later version of CC.

Profiling work, including the CETIS QTI working group activities, and subject-specific activities such as profiles for maths are also discussed.

This information will be very valuable for developers of tools and content, and it’s great to see IMS making it available to the community.

Qyouti: MCQ testing with QTI and scanners

Posted on by

Developed in response to frustration at existing high stakes MCQ testing options, Qyouti combines IMS QTI and scanning technology to provide robust, inexpensive and flexible assessment and is now available for free download from the tool’s SourceForge site.

Jon Maber, who developed the tool for Leeds Metropolitan University, describes how the tool works:

Qyouti is software which takes an IMS QTI file containing questions, a class list and prints the questions on an ordinary colour laser printer with areas for the student responses to be made in pencil or pen. I.e. the responses are marked as crosses or ticks (or just about any other kind of mark) in boxes that are right next to the options in the question paper. Every page is bar-coded with the candidate’s name and ID so it is impossible to give the marks to the wrong person. At the end of the exam the papers are scanned with an ordinary desktop scanner.  Then Qyouti processes the scanned images and produces a list of candidates with their marks […]  Each individual script has metrics encoded on it using square barcodes and so there is potential for customising font and layout for candidates with visual impairment or dyslexia.  A proper statistical analysis is done on the question items too.

Jon is keen to find volunteers to test and help contribute to the further development of the tool, and is offering free staff training in the use of MCQs in return for significant contributions.  He can be contacted through his homepage.

IMS QTI v2.1 Implementation Survey

Posted on by

IMS invite developers to participate in an IMS Question and Test Interoperability (QTI) v2.1 implementation survey.

The survey is designed to do two things:

  • establish the ‘state of play’ with regard to QTI v2.1 tool capabilities
  • establish whether there is sufficient overlap in tool capabilities to define one or more profiles.

IMS will publish an anonymous summery of the survey outcomes to all participants.  Survey results will be strictly confidential and data from this research will be reported only in the aggregate.  Your information will be coded and will remain confidential, however if you wish you may supply your email address if you are willing to be contacted to follow up or for additional information.

The survey only poses questions about QTI v2.1 capabilities, which means that it is not relevant for developers of earlier versions of QTI.

The survey will only take around 20-30 minutes to complete.  Information about the QTI project group and results from this survey will be posted in the IMS QTI forum.

This is a great opportunity for QTI v2.1 developers to help steer the future of the specification.

Public draft consultation on standard for transfer of assessment data

Posted on by

You may remember a proposed standard for the transfer of qualification assessment data and evidence that was previously covered on this blog.

Work on this has been ongoing since then, and a draft standard is now available for public consultation and comment.  The public draft can be accessed via the BSI website, and comments may be submitted by following the instructions there.

All comments must be submitted by 30 November to be considered for the final version of the standard.  Depending on the nature and extent of comments received, the standard is likely to be released in the first quarter of 2010.

IMS withdraw QTI v2.1 draft specification

Posted on by

Over the last few days a new notice has appeared on the IMS Question and Test Interoperability webpage in place of the QTI v2.1 draft specification:

The IMS QTIv2.1 draft specification has been removed from the IMS website. Adequate feedback on the specification has not been received, and therefore, the specification has been put back into the IMS project group process for further work.

QTI v2.1 was under public review for more than 2 years and did not achieve sufficient implementation and feedback to warrant being voted on as a final specification. Therefore it has been withdrawn for further work by the IMS membership. IMS cannot continue to publish specifications that have not met the rigors of the IMS process.”

IMS GLC has convened a set of leading organizations to take the lead on this new work – which will be considered to be in the CM/DN draft phase in the IMS process.  Therefore, we are very encouraged and hopeful that a new version will be available in due time, possibly a QTI v2.2, along with the necessary conformance profiles. However, we cannot assume that it will be a linear evolution from QTI v2.1.

Until that time the only version of QTI that is fully endorsed by IMS GLC is v1.2.1, that is supported under the Common Cartridge Alliance: http://www.imsglobal.org/cc/alliance.html . While QTI version 2.0 has been voted on as a final specification by the IMS members, it’s deficiencies are well known and IMS does not recommend implementation of it.

This was clearly completely unexpected, not only for us at CETIS but also amongst a number of commercial and academic developers who have been working with the specification as can be seen by posts to the technical discussion list hosted by UCLES.  In particular, I’d encourage you to read Wilbert’s response on behalf of CETIS.

Concerns from the developer community addressed a number of the issues raised in IMS’s statement.  In response to the claim that ‘adequate feedback on the specification has not been received’, several commentators argued that this is because of the high standard of the specification; while the suggestion that ‘QTI v2.1 … did not achieve sufficient implementation … to warrant being voted on as a final specification’ sparked the addition of a number of implementations to Wikipedia’s QTI page.

There is agreement that work will progress on the basis of the public draft, so it is still perfectly possible that the outcome will be a mildly amended version of the public draft with some small profiles.

CETIS will be following this up, and will of course keep you all informed about progress.  In the meantime, we’d be very keen to hear any thoughts or comments you have, although I would encourage you to sign up for both the UCLES list and the official IMS QTI list to ensure your voice is heard as widely as possible; it would be most beneficial for the wider QTI community I feel for discussion to be focused in one place, i.e. the UCLES list.

First IMS European TestFest to come to UK

Posted on by

Developers of tools that implement IMS specifications will have an opportunity to participate in a formal IMS TestFest as part of the next IMS Quarterly Meeting, being held in Birmingham on 15-18 September.  Seven QTI v2.1 tools and products have already signed up for the event on Tuesday 16th September, and will also be covered in the open session on QTI the day before: RM, ASDEL, JAssess, QTI Constraints Editor, AQuRate, Onyx and the QTI Migration Tool. 

Amongst other items of interest on Wednesday 17th is the final of the regional Learning Impact competition – entries for which must be in by 25 August, so there’s still time to get your application in!  The meeting closes on the 18th with a summit on Interoperability Now and Next, covering a broad range of issues and featuring a number of high profile speakers.

QTI 2.1 and Moodle

Posted on by

Found all by myself this time :-) was a series of tweets detailing Pierre Gorissen and Steve Lay’s successful integration of the ASDEL QTI Playr with Moodle, meaning that Pierre is now able to run QTI 2.1 assessments within a Moodle course.  This is a signficant step forward for making the specification more attractive to users, given the huge popularity and dynamic community that surrounds Moodle.  It might also make QTI a more attractive alternative to OpenMark within the Moodle community – there are some interesting comments to be found on QTI in the Moodle forums.