The consortium is led by the University of Glasgow, and includes experts from the University of Edinburgh and Kingston University, contributions from the IMS QTI working group chairs and tool developers, independent consultants Sue Milne, Graham Smith and Dick Bacon, and input from us here at JISC CETIS.  QTI experts at the University of Southampton are advisors to the project.

A project blog has been set up  which will provide a central point for dissemination to the wider QTI community.  Information on how to get involved with the QTI interoperability testing process is also available there.

The project aims include:

• Contributing to the definition of the main profile of QTI 2.1;
• Implementation of the main profile in at least one existing open source test rendering/responding system;
• Providing support in the use of QTI 2.1 and the conversion of other question and test formats to QTI 2.1 to those developing assessment tools and authoring questions;
• Providing a publicly available reference implementation of the QTI main profile that will enable question and test item authors to test whether their material is valid, and how it renders and responds.

Follow the project blog for future developments!

Any comments, corrections or requests for additional content are very welcome, either by commenting here on this blog or by email.

The Web3D Consortium has just announced that its standardisation activities will now be open to the public, enabling non-members to participate in development of the specification at all stages rather than just during the public review prior to final approval.   There is still the opportunity for private discussion limited to consortium members for those concerned about commercial or other factors, but the overall emphasis is clearly on making this as open as possible.

X3D is an open ISO standard for representing information about computer generated 3D environments and objects.  Unlike its predecessor, Virtual Reality Modelling (or Markup) Language (VRML), X3D features integration with HTML, and extends the range of effects supported.  X3D is supported by some high profile systems such as the Blender design tool and Sun’s Project Wonderland.

At a time when closed, proprietary players such as Linden Labs are seeing large numbers of layoffs, with inevitable concern from Second Life’s active education community about the potential loss of a huge amount of work and resources should this trend continue, adopting an open approach to development seems a very sensible decision.

Twenty-one implementations are covered by the responses, representing a wide range of approaches to implementation, and the actual responses are available for download for those interested.  The responses support the notion of a core set of basic features implemented by all respondents, with broader parts of the specification being implemented on a more individual basis.

The results are feeding in to the development of profiles for QTI 2.1:

• Base QTI Profile, covering the features available in the most comprehensive implementations;
• CC-QTI, which updates the functionality covered by the QTI 1.2.1 profile within Common Cartridge 1.0 and which will be integrated into a later version of CC.

Profiling work, including the CETIS QTI working group activities, and subject-specific activities such as profiles for maths are also discussed.

This information will be very valuable for developers of tools and content, and it’s great to see IMS making it available to the community.

Jon Maber, who developed the tool for Leeds Metropolitan University, describes how the tool works:

Qyouti is software which takes an IMS QTI file containing questions, a class list and prints the questions on an ordinary colour laser printer with areas for the student responses to be made in pencil or pen. I.e. the responses are marked as crosses or ticks (or just about any other kind of mark) in boxes that are right next to the options in the question paper. Every page is bar-coded with the candidate’s name and ID so it is impossible to give the marks to the wrong person. At the end of the exam the papers are scanned with an ordinary desktop scanner.  Then Qyouti processes the scanned images and produces a list of candidates with their marks […]  Each individual script has metrics encoded on it using square barcodes and so there is potential for customising font and layout for candidates with visual impairment or dyslexia.  A proper statistical analysis is done on the question items too.

Jon is keen to find volunteers to test and help contribute to the further development of the tool, and is offering free staff training in the use of MCQs in return for significant contributions.  He can be contacted through his homepage.

The survey is designed to do two things:

• establish the ‘state of play’ with regard to QTI v2.1 tool capabilities
• establish whether there is sufficient overlap in tool capabilities to define one or more profiles.

IMS will publish an anonymous summery of the survey outcomes to all participants.  Survey results will be strictly confidential and data from this research will be reported only in the aggregate.  Your information will be coded and will remain confidential, however if you wish you may supply your email address if you are willing to be contacted to follow up or for additional information.

The survey only poses questions about QTI v2.1 capabilities, which means that it is not relevant for developers of earlier versions of QTI.

The survey will only take around 20-30 minutes to complete.  Information about the QTI project group and results from this survey will be posted in the IMS QTI forum.

This is a great opportunity for QTI v2.1 developers to help steer the future of the specification.

Work on this has been ongoing since then, and a draft standard is now available for public consultation and comment.  The public draft can be accessed via the BSI website, and comments may be submitted by following the instructions there.

All comments must be submitted by 30 November to be considered for the final version of the standard.  Depending on the nature and extent of comments received, the standard is likely to be released in the first quarter of 2010.

The IMS QTIv2.1 draft specification has been removed from the IMS website. Adequate feedback on the specification has not been received, and therefore, the specification has been put back into the IMS project group process for further work.

QTI v2.1 was under public review for more than 2 years and did not achieve sufficient implementation and feedback to warrant being voted on as a final specification. Therefore it has been withdrawn for further work by the IMS membership. IMS cannot continue to publish specifications that have not met the rigors of the IMS process.”

IMS GLC has convened a set of leading organizations to take the lead on this new work – which will be considered to be in the CM/DN draft phase in the IMS process.  Therefore, we are very encouraged and hopeful that a new version will be available in due time, possibly a QTI v2.2, along with the necessary conformance profiles. However, we cannot assume that it will be a linear evolution from QTI v2.1.

Until that time the only version of QTI that is fully endorsed by IMS GLC is v1.2.1, that is supported under the Common Cartridge Alliance: http://www.imsglobal.org/cc/alliance.html . While QTI version 2.0 has been voted on as a final specification by the IMS members, it’s deficiencies are well known and IMS does not recommend implementation of it.

This was clearly completely unexpected, not only for us at CETIS but also amongst a number of commercial and academic developers who have been working with the specification as can be seen by posts to the technical discussion list hosted by UCLES.  In particular, I’d encourage you to read Wilbert’s response on behalf of CETIS.

Concerns from the developer community addressed a number of the issues raised in IMS’s statement.  In response to the claim that ‘adequate feedback on the specification has not been received’, several commentators argued that this is because of the high standard of the specification; while the suggestion that ‘QTI v2.1 … did not achieve sufficient implementation … to warrant being voted on as a final specification’ sparked the addition of a number of implementations to Wikipedia’s QTI page.

There is agreement that work will progress on the basis of the public draft, so it is still perfectly possible that the outcome will be a mildly amended version of the public draft with some small profiles.

CETIS will be following this up, and will of course keep you all informed about progress.  In the meantime, we’d be very keen to hear any thoughts or comments you have, although I would encourage you to sign up for both the UCLES list and the official IMS QTI list to ensure your voice is heard as widely as possible; it would be most beneficial for the wider QTI community I feel for discussion to be focused in one place, i.e. the UCLES list.

Amongst other items of interest on Wednesday 17th is the final of the regional Learning Impact competition – entries for which must be in by 25 August, so there’s still time to get your application in!  The meeting closes on the 18th with a summit on Interoperability Now and Next, covering a broad range of issues and featuring a number of high profile speakers.