Under development: Sharing Higher Education Data

Meaningful work placements and graduate employability have always been an important part of university education and preparation for a professional future in certain disciplines, and are arguably even more so today in a climate of limited employment opportunities, with high university fees and loans positioning students as customers investing in their future careers.  Certain subject areas enjoy good relationships with industry, providing industrial placements to give students real-world experience in their future fields, while local companies benefit from the expertise and cutting edge knowledge these students can bring to the workplace.  Universities and colleges similarly benefit from this ongoing engagement with industry, ensuring their courses remain relevant and meaningful.

Shrinking university staff numbers have increased workloads, limiting the time staff have to spend assisting individual students in seeking suitable placements and opportunities for work-based learning.  In any case, reliance on university staff is not necessarily the best way in which students can prepare themselves for seeking suitable, fulfilling employment on graduation, or establish fruitful relationships with potential employers.

The Sharing Higher Education Data (SHED) project attempts to address these issues through the delivery of a ‘matchmaking’ service for students and employers, which will both facilitate communication between them and enable students to plan their learning paths in the light of the expectations and requirements of their chosen profession.  Sample case studies included in the student and employer information sheets about the service help illustrate the range of ways in which SHED can benefit both user groups while increasing interaction between academia and industry.

SHED uses the popular Mahara open source eportfolio tool to allow students to develop their profiles, and, vitally, provides them with strict control over what information is made publicly viewable by potential recruiters.  Students can also view common employer search terms within their particular field in order to better understand the employment market in that area and to support the review and revision of their profiles to enhance their employability.  The integration of the XCRI information model and specification (eXchanging Course Related Information) provides a common framework for describing and sharing course information, while Leap2A and InterOperability provide support for the sharing of eportfolio and competence information.

As a partnership between the Centre for International ePortfolio Development at the University of Nottingham and Derby College, SHED will also be able to demonstrate how the system can be used across a number of different institutions without compromising privacy while maximising opportunities for placement and project work and professional development.  Although small-scale and local to begin with, it is intended that the system be scalable to include many institutions, subject areas and locations, and provide both a valuable service for students and employers and insight into regional and national trends in industry and development.

Badges, identity and the $2million prize fund

You’ll almost certainly have noticed some of the excitement that’s suddenly erupted around the use of badges in education.  Perhaps you’ve heard that it’s the latest in a long line of ‘game changers for education’, maybe you’re even hoping for a slice of that $2million prize fund the HASTAC Initiative, Mozilla and the MacArthur Foundation are offering for work around their adoption and development through the Digital Media and Learning Badges for Lifelong Learning competition.  Supported by a number of significant entities, including Intel, Microsoft and various US Government departments, the competition offers up to $200k each for a number of projects around content and infrastructure for badges for lifelong learning, as well as an $80k award for a research project in ‘Badges, trophies and achievements: recognition and accreditation for informal and interest-driven learning’ together with two smaller doctoral student grants, and student and faculty prizes.  That’s a decent amount of cash available for – what?

This is all based around Mozilla’s Open Badges Initiative, which attempts to provide an innovative infrastructure to support the recognition of non-traditional learning and achievement for professional development and progress.  Drawing upon the widespread use of badges and achievements in gaming and the current trend for gamification, the project is described in gamified language, claiming that badges can help adopters ‘level up’ in their careers via the acquisition and display (sharing) of badges.  There’s a fair point being made here: gamers can develop a profile and express their individual identity as gamers through the display of achievements they earn as they play, which can then be shared ingame through the use of special titles or on appropriate fora through signatures and site profiles.  Achievements reflect the different interests a player has (their weighting on the Bartle scale for example) as well as their skill.  Within a fairly closed community such as a single game, a suite of games or a website, these achievements have significant value as the viewers are other gamers for whom the achievements have meaning and value.

LarsH on Stack Overflow’s response to the question ‘why are badges motivating?‘, asked over a year ago but still very relevant, sums this up eloquently:

We like other people to admire us.  As geeks we like others to admire us for our skills.  Badges/achievements stay visible in association with our online identity long-term, unlike individual questions and answers which quickly fade into obscurity.

If I play a game and get a great score, it’s nice, but it means little to others unless they have the context of what typical scores are for that game (and difficulty level etc).  Whereas an achievment is a little more compact of a summary of what you’ve accomplished.

Badges also give us a checklist whereby we can see how far we’ve come since we joined the web site – and how far we have to go in order to be average, or to be exceptional.’

LarsH’s comments were in the context of participation in an online community which awards badges for numbers of ‘helpful’ answers to questions and other contributions, but the underlying theme is the same for all contexts: the notion of building a persistent persona associated with achievements and success that endures beyond a single assessed instance (one play through a game, one helpful answer) which which it is specifically associated.  It creates a sense of status and implies competence and trustworthiness, which in turn can inspire others to emulate that behaviour in the hope of seeking similar recognition, or indicate that this is a trusted individual to ask for advice or guidance from.  Badges not only provide recognition of past contributions but also an implication that future contributions can also be trusted and an incentive to participate usefully.

Being able to capture and reflect this sometimes quite fine-grained information in other contexts would indeed have some advantages.  But as soon as these awards and achievements are looked at by someone outside their immediate context, they immediately lose a large part of their value, not because they’re worthless outside their original context but because the viewer lacks the expertise in the field to be able to trust that the badge reflects what it claims or to understand the implications of what it claims.  The value of the badge, therefore, isn’t inherent in the badge itself but in the assertions around it: that is was issued by a trustworthy party on reliable evidence to the specific individual who claims it.  A lot like, say, a traditional certificate for completing an accredited course, perhaps…

As Alex Reid (no, not that one) says, ‘passing a high stakes test to get a badge is no different than the system we already have’, and a lot of the problems around developing a trustworthy system are those that have already been faced by traditional awarders.  Comparisons to diploma mills swiftly emerged in the aftermath of the competition announcement, and it’s not difficult to see why: if anyone can issue a badge, how do we know that a badge reflects anything of merit?  Cathy Davidson’s vision of a world where employers hand out badges for ‘Great Collaborator!’ or ‘Ace Teacher!’ is nice (if far too cutesy for my tastes), but it’s not exactly hard to see how easily it could be abused.

At the heart of the badges initiative is the far older issue of identity management.  As our badge ‘backpack‘ is intended to gather badges awarded in a range of different contexts, how are we to be sure that they all belong to the one person?  As the example above of Alex Reid, American academic, versus Alex Reid, cage fighter, cross dresser, Celebrity Big Brother contestant and ex husband of Jordan, demonstrates, names are useless for this, particularly when the same person can be known by a number of different names, all equally meaningful to them in the same different contexts the backpack is intended to unify.  Email addresses have often been suggested as a way of identifying individuals, yet how many of us use a single address from birth to death?  In the US, social security numbers are far too sensitive to be used, while UK National Insurance numbers aren’t unique.  Similarly, how is a recruiter to know that a badge has been issued by a ‘respectable’ provider on the basis of actual performance rather than simply bought from a badge mill?  Unique identification of individuals and awarders, and accreditation of accreditors themselves, whether through a central registry or decentralised web of trust, is at the heart of making this work, and that’s not a small problem to solve.  With the momentum behind the OER movement growing and individuals having more reason and opportunity to undertake free ad hoc informal learning, being able to recognise and credit this is important.  As David Wiley notes, however, there’s a difference between a badge awarded simply for moving through a learning resource, and one awarded as an outcome of validated, quality assured assessment specifically designed to measure learning and achievement, and this needs to be fully engaged with for open or alternative credentialing to fulfill its potential.

There’s also a danger that badges and achievements can be used to legitimise bad or inadequate content by turning it into a Skinner box, where candidates will repeatedly undertake a set task in the expectation of eventually getting a reward, rather than because the task itself is engaging or they’re learning from it.  Borrowing from games can be good, but gamers can be very easily coaxed into undertaking the most mindless, tedious activities long after their initial value has been exhausted if the eventual reward is perceived as worth it.

Unlike, say, augmented reality or other supposed game changers, it’s not the underlying technology itself that has the potential to be transformative – after all, it basically boils down to a set of identity assertion and management problems to be solved with which the IDM people have been wrestling for a long time, plus image exchange and suitable metadata – but rather the cultural transformation it expresses, with the recognition that informal or hobbyist learning and expertise can be a part of our professional skillset.  Are badges the right way of doing this?  Perhaps; but what’s much more important is that the discussion is being had.  And that has to be a good thing.

Under development: eMargin

eMargin logo

When I was studying English at university, one of the more engaging and intriguing sites of discussion and debate was the margins of printed texts.  These are the ultimate asynchronous discussions, taking place over decades in some cases, rarely revisted by their participants once they’d left their comment on previous comments.  It was fascinating to encounter often very different perceptions on both primary and secondary texts, and they encouraged me to reflect on my own interpretations and arguments as well as articulating them in the form of comments added to those already there.  These serendipitous discoveries definitely enhanced my learning experience, providing the opportunity to discuss texts and solidify my understanding significantly beyond that provided by limited tutorial time and the very few other opportunities for debate available.  Similarly, I encouraged my students to write on their books to increase engagement with the texts they read and legitimise their interpretations and opinions, although that was often met with askance looks that clearly said, ‘sod that, I’m selling them later.’

So I was very interested to learn about the eMargin project, which is developing an online collaborative textual annotation resource as part of the JISC Learning and Teaching Innovation Grants funding round six.  The eMargin system allows a range of annotation activities for electronic editions of texts, encompassing notes and comments on individual sections, highlighting, underlining and so on, all personalisable to support different tastes and access requirements.  What takes this beyond the usual functionality offered by ebook readers is the ability to share these annotations with class-mates and students from other institutions, enabling their use as educational resources by design rather than chance.  Teachers are able to control the degree of exposure of annotations in line with institutional policies on student IPR, and the system may be developed further to allow students to control which comments they wish to share and which to keep private, allowing them to use the same system for personal study as well as class work.  By providing an easy means for sharing ideas, together with a wiki feature for building and capturing consensus, this system will be of value in all disciplines, not just English Literature where it is being developed.

The project team, Andrew Kehoe and Matt Gee of the Research and Development Unit for English Studies at Birmingham City University, are developing the system through a number of iterations in the light of feedback from teachers and learners, and engaging participants in other institutions and other disciplines to demonstrate its versatility.  The team is also exploring the possibility of integrating eMargin with VLEs, and its potential as an eassessment tool; it may also have value in tracking the development of learners’ ideas in order to reduce opportunities for plagiarism.

The project runs until the end of May 2012, when source code, user guides and an archive of textual annotations will be available via the project site.  You can also visit their FaceBook page.

QTI v2.1 briefing paper now available

An updated version of our QTI Briefing Paper is now available.  It provides an introduction to the specification’s structure and purpose, some details about the history of the specification and a discussion of the pros and cons of adoption.

This is an updated version of the draft document released in March, and will be replaced with a final version after the final release of QTI v2.1 by IMS.

eAssessment Scotland 2011 details now available

The website for this year’s eAssessment Scotland event is now up, and has a wealth of information about this popular, free two-day event.

Running on 25 – 26 August at the University of Dundee, this third conference features a packed programme including a range of workshops and seminars, poster dispays and exhibitions, keynotes from Steve Wheeler, Becka Colley and Donald Clark as well as the presentation of this year’s Scottish eAssessment Awards.

Proposals for poster presentations can be submitted until 1 August, while entry for the Scottish eAssessment Awards closes on 16 August.

Registration will open shortly.

ePortfolios, Y|N?

I retweeted a link to this post yesterday, and promptly found myself in the middle of a storm of debate about the validity and legitimacy of the points it raises.  As it’s not exactly a topic that lends itself to discussion in 140 character chunks, I thought I’d bring it here to see if people want to continue what turned out to be a pretty impassioned and heated discussion.

For my part, I think there are some good points made here.  While I think there’s a definite role for eportfolio technology in certain contexts, I’m not sold on the whole lifelong portfolios for lifelong learners rhetoric, and I don’t think it necessarily meets the needs or desires of learners or teachers.

My biggest issue is that there is a lack of distinction between a portfolio of work that is ultimately intended as an assessment resource to be externally viewed and evaluated, and a student’s body of work which he is supposed to reflect on and learn from.  The intrusion of workplace CPD into this space simply exacerbates this lack of focus and conflicting motivations.  While it may be possible for a single system to fully meet the technical requirements of these very different competing interests, I don’t think that’s necessarily the appropriate approach.  Learning is all about having the freedom and safety to fail, and about taking ownership of our successes and failures in order to grow as learners and as experts in the subject we’re studying.  Having authority over our own work is a fundamental part of that, and something that has to be handed over when that work is used for formal evaluation.

I don’t think we need specialised software in order to retain a record of our learning and progress.  A personal blog can be a powerful tool for reflection, a pen drive of files can be more portable and accessible than a dedicated tool, your youtube or vimeo or flickr channel is more than adequate for preserving your creations.  All of these have permanence beyond the duration of a course: although some institutions will allow continued access to institutional portfolio systems after a student has finished his course of study, it’s not a given and is always subject to change.  Using existing services ironically offers far more opportunity for true lifelong learning than a dedicated system.  And such distributed systems reflect the ways in which people reflect on and share their work outside the walls of the university.  I still have my ‘portfolio’ of my undergraduate work: the printed out essays I handed in with my lecturers’ comments written on them.  That was exactly what I needed as a learner, that’s exactly what I need now should I ever wish to reflect on that period.

For material to be used for assessment, yes, there is a need for secure and reliable storage systems and appropriate standards such as Leap2A and BS8518 to support the exchange of evidence, but the systems and processes should be appropriate to the subject and the material to be assessed rather than assessment being tailored to suit the available systems.

Many thanks to @drdjwalker, @dkernohan, @mweller, @markpower, @jamesclay, @ostephens, @jontrinder and @asimong for joining the discussion on Twitter.

Under development: VWVLE

The VWVLE project, or Supporting Education in Virtual Worlds with Virtual Learning Environments to give it its full name, has been funded as part of the JISC Learning and Teaching Innovation Grants round 5 to examine the wide range of emerging pedagogical opportunities offered through the integration of virtual worlds and web-based virtual learning environments.

Led by the University of the West of Scotland, with partners including Imperial College London, The Open University and the University of Ulster, the project builds on the considerable experience and expertise the project team have developed through their work on SLOODLE and the use of games for learning within virtual environments.  SLOODLE (Simulation Linked Object Oriented Dynamic Learning Environment) provides seamless integration between the virtual world Second Life and Moodle, the popular open source VLE.  Pilot courses will see students in engineering, computing and medicine explore aspects of the core question of how web-based virtual learning environments can effectively support learning and teaching in virtual worlds, particularly focusing on personalisation and reuse of content, and gaming in VWs, and demonstrating the applicability of such technologies across different institutional and disciplinary contexts.

A number of outputs will be produced, including guidance for practitioners, a range of extensions or plug-ins for Moodle/SLOODLE, and a guide to producing reusable content in virtual worlds which will attempt to address some of the issues that present a significant barrier to the easy and effective exchange of such resources.  The emphasis on the integration of VWs and games with educational systems such as VLEs will both highlight the pedagogic benefits of such integration and attempt to clarify and address the challenges of doing so.  By making explicit the range of technologies and support resources relied upon by educators working with VWs, and identifying and sharing good practice, the project can make a real impact on practice in this area and future activities.

ReLIVE11 call for abstracts open

The dates and call for abstracts for Researching Learning in Immersive Virtual Environments 2011: Creative Solutions for New Futures (ReLIVE11) have now been released. Running at the Open University in Milton Keynes on 21 and 22 September, the conference will explore issues around innovation in teaching and learning through immersive virtual worlds, building on the outcomes of and lessons learned since the previous ReLIVE conference in 2008.

The call for abstracts is now open, with submissions due to be submitted by 21 May 2011. Proposals will be considered within three core themes:

  • Concepts: conceptual and explanatory frameworks for research processes and outcomes;
  • Methods: opportunities and challenges around researching learning in immersive environments;
  • Implementations: the technical aspects and challenges in the implementation of VWs in learning, teaching and research.

The Implementations theme is being run in conjunction with us here at JISC CETIS, with Paul Hollins and myself leading our input to it. We’d love to see papers or workshops addressing issues such as the challenges of the integration of VEs with other institutional systems, the interoperability of content and avatars across different VWs, open v proprietary platforms and content, and the use of technical standards within VWs. This isn’t an exhaustive list, and any relevant proposals are welcome!

Registration for the conference opens on 21 June and it’s likely to be a popular and well-attended event. An inworld Virtual Festival on 20 September will similarly feature a range of workshops, symposia, poster sessions and other events. ReLIVE11 is also linked to the Virtual World Conference taking place in Second Life on 14 September.

Under development: QTI-IPS

A couple of months ago, JISC released an Invitiation to Tender for a QTI v2.1 implementation and profiling support project.  A consortium of experts produced the successful bid, bringing together some of the leading experts on QTI in UK HE, and the project formally kicked off this week.  It concludes in mid-September this year.

The consortium is led by the University of Glasgow, and includes experts from the University of Edinburgh and Kingston University, contributions from the IMS QTI working group chairs and tool developers, independent consultants Sue Milne, Graham Smith and Dick Bacon, and input from us here at JISC CETIS.  QTI experts at the University of Southampton are advisors to the project.

A project blog has been set up  which will provide a central point for dissemination to the wider QTI community.  Information on how to get involved with the QTI interoperability testing process is also available there.

The project aims include:

  • Contributing to the definition of the main profile of QTI 2.1;
  • Implementation of the main profile in at least one existing open source test rendering/responding system;
  • Providing support in the use of QTI 2.1 and the conversion of other question and test formats to QTI 2.1 to those developing assessment tools and authoring questions;
  • Providing a publicly available reference implementation of the QTI main profile that will enable question and test item authors to test whether their material is valid, and how it renders and responds.

Follow the project blog for future developments!

CAA Conference 2011 call for papers out now

The 2011 International Computer Assisted Assessment (CAA) Conference: Research into eAssessment will be held on 5 and 6 July 2011 at the DeVere Grand Harbour Hotel, Southampton.  Jointly hosted by the School of Electronics and Computer Science at the University of Southampton and the Institute of Educational Technology at the Open University, this is a two day research-led conference which aims to advance the understanding and application of information technology to the assessment process through rigorous peer-reviewed research.

The 2011 Call for Papers is now available, together with detailed guidance on conference themes and the proposal submission process.  The deadline for receipt of submissions through the conference’s EasyChair web interface is Friday 15 April 2011.

Papers from the 2010 conference and earlier are also available.