Category Archives: innovation

Badges, identity and the $2million prize fund

Posted on by

You’ll almost certainly have noticed some of the excitement that’s suddenly erupted around the use of badges in education.  Perhaps you’ve heard that it’s the latest in a long line of ‘game changers for education’, maybe you’re even hoping for a slice of that $2million prize fund the HASTAC Initiative, Mozilla and the MacArthur Foundation are offering for work around their adoption and development through the Digital Media and Learning Badges for Lifelong Learning competition.  Supported by a number of significant entities, including Intel, Microsoft and various US Government departments, the competition offers up to $200k each for a number of projects around content and infrastructure for badges for lifelong learning, as well as an $80k award for a research project in ‘Badges, trophies and achievements: recognition and accreditation for informal and interest-driven learning’ together with two smaller doctoral student grants, and student and faculty prizes.  That’s a decent amount of cash available for – what?

This is all based around Mozilla’s Open Badges Initiative, which attempts to provide an innovative infrastructure to support the recognition of non-traditional learning and achievement for professional development and progress.  Drawing upon the widespread use of badges and achievements in gaming and the current trend for gamification, the project is described in gamified language, claiming that badges can help adopters ‘level up’ in their careers via the acquisition and display (sharing) of badges.  There’s a fair point being made here: gamers can develop a profile and express their individual identity as gamers through the display of achievements they earn as they play, which can then be shared ingame through the use of special titles or on appropriate fora through signatures and site profiles.  Achievements reflect the different interests a player has (their weighting on the Bartle scale for example) as well as their skill.  Within a fairly closed community such as a single game, a suite of games or a website, these achievements have significant value as the viewers are other gamers for whom the achievements have meaning and value.

LarsH on Stack Overflow’s response to the question ‘why are badges motivating?‘, asked over a year ago but still very relevant, sums this up eloquently:

We like other people to admire us.  As geeks we like others to admire us for our skills.  Badges/achievements stay visible in association with our online identity long-term, unlike individual questions and answers which quickly fade into obscurity.

If I play a game and get a great score, it’s nice, but it means little to others unless they have the context of what typical scores are for that game (and difficulty level etc).  Whereas an achievment is a little more compact of a summary of what you’ve accomplished.

Badges also give us a checklist whereby we can see how far we’ve come since we joined the web site – and how far we have to go in order to be average, or to be exceptional.’

LarsH’s comments were in the context of participation in an online community which awards badges for numbers of ‘helpful’ answers to questions and other contributions, but the underlying theme is the same for all contexts: the notion of building a persistent persona associated with achievements and success that endures beyond a single assessed instance (one play through a game, one helpful answer) which which it is specifically associated.  It creates a sense of status and implies competence and trustworthiness, which in turn can inspire others to emulate that behaviour in the hope of seeking similar recognition, or indicate that this is a trusted individual to ask for advice or guidance from.  Badges not only provide recognition of past contributions but also an implication that future contributions can also be trusted and an incentive to participate usefully.

Being able to capture and reflect this sometimes quite fine-grained information in other contexts would indeed have some advantages.  But as soon as these awards and achievements are looked at by someone outside their immediate context, they immediately lose a large part of their value, not because they’re worthless outside their original context but because the viewer lacks the expertise in the field to be able to trust that the badge reflects what it claims or to understand the implications of what it claims.  The value of the badge, therefore, isn’t inherent in the badge itself but in the assertions around it: that is was issued by a trustworthy party on reliable evidence to the specific individual who claims it.  A lot like, say, a traditional certificate for completing an accredited course, perhaps…

As Alex Reid (no, not that one) says, ‘passing a high stakes test to get a badge is no different than the system we already have’, and a lot of the problems around developing a trustworthy system are those that have already been faced by traditional awarders.  Comparisons to diploma mills swiftly emerged in the aftermath of the competition announcement, and it’s not difficult to see why: if anyone can issue a badge, how do we know that a badge reflects anything of merit?  Cathy Davidson’s vision of a world where employers hand out badges for ‘Great Collaborator!’ or ‘Ace Teacher!’ is nice (if far too cutesy for my tastes), but it’s not exactly hard to see how easily it could be abused.

At the heart of the badges initiative is the far older issue of identity management.  As our badge ‘backpack‘ is intended to gather badges awarded in a range of different contexts, how are we to be sure that they all belong to the one person?  As the example above of Alex Reid, American academic, versus Alex Reid, cage fighter, cross dresser, Celebrity Big Brother contestant and ex husband of Jordan, demonstrates, names are useless for this, particularly when the same person can be known by a number of different names, all equally meaningful to them in the same different contexts the backpack is intended to unify.  Email addresses have often been suggested as a way of identifying individuals, yet how many of us use a single address from birth to death?  In the US, social security numbers are far too sensitive to be used, while UK National Insurance numbers aren’t unique.  Similarly, how is a recruiter to know that a badge has been issued by a ‘respectable’ provider on the basis of actual performance rather than simply bought from a badge mill?  Unique identification of individuals and awarders, and accreditation of accreditors themselves, whether through a central registry or decentralised web of trust, is at the heart of making this work, and that’s not a small problem to solve.  With the momentum behind the OER movement growing and individuals having more reason and opportunity to undertake free ad hoc informal learning, being able to recognise and credit this is important.  As David Wiley notes, however, there’s a difference between a badge awarded simply for moving through a learning resource, and one awarded as an outcome of validated, quality assured assessment specifically designed to measure learning and achievement, and this needs to be fully engaged with for open or alternative credentialing to fulfill its potential.

There’s also a danger that badges and achievements can be used to legitimise bad or inadequate content by turning it into a Skinner box, where candidates will repeatedly undertake a set task in the expectation of eventually getting a reward, rather than because the task itself is engaging or they’re learning from it.  Borrowing from games can be good, but gamers can be very easily coaxed into undertaking the most mindless, tedious activities long after their initial value has been exhausted if the eventual reward is perceived as worth it.

Unlike, say, augmented reality or other supposed game changers, it’s not the underlying technology itself that has the potential to be transformative – after all, it basically boils down to a set of identity assertion and management problems to be solved with which the IDM people have been wrestling for a long time, plus image exchange and suitable metadata – but rather the cultural transformation it expresses, with the recognition that informal or hobbyist learning and expertise can be a part of our professional skillset.  Are badges the right way of doing this?  Perhaps; but what’s much more important is that the discussion is being had.  And that has to be a good thing.

Under development: eMargin

Posted on by

eMargin logo

When I was studying English at university, one of the more engaging and intriguing sites of discussion and debate was the margins of printed texts.  These are the ultimate asynchronous discussions, taking place over decades in some cases, rarely revisted by their participants once they’d left their comment on previous comments.  It was fascinating to encounter often very different perceptions on both primary and secondary texts, and they encouraged me to reflect on my own interpretations and arguments as well as articulating them in the form of comments added to those already there.  These serendipitous discoveries definitely enhanced my learning experience, providing the opportunity to discuss texts and solidify my understanding significantly beyond that provided by limited tutorial time and the very few other opportunities for debate available.  Similarly, I encouraged my students to write on their books to increase engagement with the texts they read and legitimise their interpretations and opinions, although that was often met with askance looks that clearly said, ‘sod that, I’m selling them later.’

So I was very interested to learn about the eMargin project, which is developing an online collaborative textual annotation resource as part of the JISC Learning and Teaching Innovation Grants funding round six.  The eMargin system allows a range of annotation activities for electronic editions of texts, encompassing notes and comments on individual sections, highlighting, underlining and so on, all personalisable to support different tastes and access requirements.  What takes this beyond the usual functionality offered by ebook readers is the ability to share these annotations with class-mates and students from other institutions, enabling their use as educational resources by design rather than chance.  Teachers are able to control the degree of exposure of annotations in line with institutional policies on student IPR, and the system may be developed further to allow students to control which comments they wish to share and which to keep private, allowing them to use the same system for personal study as well as class work.  By providing an easy means for sharing ideas, together with a wiki feature for building and capturing consensus, this system will be of value in all disciplines, not just English Literature where it is being developed.

The project team, Andrew Kehoe and Matt Gee of the Research and Development Unit for English Studies at Birmingham City University, are developing the system through a number of iterations in the light of feedback from teachers and learners, and engaging participants in other institutions and other disciplines to demonstrate its versatility.  The team is also exploring the possibility of integrating eMargin with VLEs, and its potential as an eassessment tool; it may also have value in tracking the development of learners’ ideas in order to reduce opportunities for plagiarism.

The project runs until the end of May 2012, when source code, user guides and an archive of textual annotations will be available via the project site.  You can also visit their FaceBook page.

LTIG Round 6 call for funding now out (edited 21/2/11)

Posted on by

The JISC Learning and Teaching Innovation Grants programme funds a small number of projects each year to explore and support innovative approaches to teaching and learning.  These projects cover a vast range of subject areas, technologies and activities, from poetry to chemistry, LaTeX to Twitter, QR codes to the Wii, virtual worlds to augmented reality.  Over the next few weeks I’ll be blogging about a number of these projects, the innovative activities they’ve undertaken and the very wide range of technologies in use within this programme.

JISC have just released a call for the latest round of funding available within this programme.  The application process is designed to encourage speculative and innovative ideas, the first stage consisting of submission of an outline proposal rather than the traditional full bid.

The deadline for submission of proposals for the current call is noon on Monday 21 March 2011.  There’s also a briefing event being held at 2pm on Tuesday 22 February in Elluminate which potential applicants are strongly encouraged to attend.

Best of luck to all applicants!

Edit (21 Feb 11): Martin Hawksey offers some excellent tips on bidding through his JISC RSE Scotland North & East blog.

Assessment of games based learning

Posted on by

It was well worth the early start today to attend a fascinating webinar presented by Nicola Whitton of MMU on ‘assessment of game based learning’.  Part of the successful series of webinars hosted by the Transforming Assessment project funded by the Australian Learning and Teaching Council and based at the University of Adelaide, this was the second of two events focusing particularly on games in education.

While the previous seminar looked less at assessment and more generally at games and pseudo-games such as Second Life, Nicola’s talk drew a sharp distinction between play, play worlds and simulations.  Games don’t need to have awesome graphics or vast budgets to succeed: great learning designs may be gamelike without the author ever consciously intending to design a game.  Games might ‘mashup the real world and the game world’ in imaginative and creative ways, but

lecture theatres aren’t particularly effective in real life, so reproducing them in a world where you can fly just seems really strange

I feel as though I’m SL-bashing again, but I found it really refreshing to have someone state so clearly that no, just because something’s virtual doesn’t make it a game, its the nature of the interaction between learner and content that does.  It doesn’t make it a more or less legitimate learning tool either, but the distinction is important as they both have valuable but different things to offer, and represent very different learning models.

Nicola distinguished between the use of games as an assessment tool and the assessment of games based learning, an important distinction that often seems to be overlooked.

Assessment within games offers some valuable elements: it can be automated, repeatable, potentially integrated in the learning process, impartial.  External assessment, defined here as any non-game assessment activity, by contrast, is capable of greater creativity, more tutor control, but is also more time-intensive and can be unconsciously partial.  Higher levels of learning such as analysis and critical thinking are far more difficult to assess by any automated method, including games, as they attempt to use quantitative methods to assess qualitative outcomes.

Games often have a binary, win or lose outcome that doesn’t accurately reflect the subtleties of degrees of competence or ability, and which can be counterproductive to learning through play when used as assessment.  By using external assessment processes and disassociating game performance from course grade, games can provide a safe learning environment in which failure in the immediate game context can actually be invaluable for further learning and growth.

As with any other form of assessment, including pen and paper tests, expertise in the assessment format – in this case, gaming literacy – can significantly alter the outcome of the assessment.  As always, assessment must genuinely assess the intended learning outcomes and not, for example, the ability to navigate effortlessly through the game world (a major issue even for experienced gamers when it comes to Second Life) or familiarity with general gaming conventions.  This suggests that assessing game based learning within the game environment would be a preferred approach, but while teachers may find it relatively easy to integrate innovative approaches within their teaching practice, applying this to assessment, particularly higher-stakes assessment, can provoke hostility from higher authorities.  Nicola did, however, reference the SQA’s GamesSpace initiative, presented at a CETIS special event earlier this year, as an example of a national assessment authority embracing such technologies for a major qualification strand.  GamesSpace is particularly worth noting as it allows the assessment of process and not simply product and incorporates human rather than automated marking: the candidate uses an avatar to progress through a series of role-related tasks, priorities and activities which are recorded in a format identical to the pen and paper alternative for manual human marking.

Learners too may demonstrate some hostility towards games as teaching aids – but this resistance is something that has been observed in relation to other innovative approaches too. Anything that appears to trivialise learning or that can be interpreted as trying to make learners ‘not feel they’re learning‘ can provoke scepticism and resistance in learners.  It can be hard to get away from the priviliging of traditional models of teaching and learning, from the scholars seated at the feet of the master, from the three essays in three hours make-or-break finals paper.  When learners can see the value in using such approaches, they are generally very willing to engage with them – learners are in general pragmatic, strategic and outcomes-orientated, whether their teachers like it or not.  Interestingly, Nicola’s research has demonstrated that a ‘propensity to play games for fun is in no way related to an inclination to play games for learning.’  She also cast some healthy scepticism on the oft-quoted finding that women play puzzles while men play shooters, pointing out that these findings come from surveys completed by self-selecting groups and can’t be taken as gospel; as a women who’d far rather shoot pigs than click cows I find it good to have my preferences acknowledged :)

The two sessions offered very different views of a sometimes controvertial field, and regardless of personal opinion these varied perspectives were invaluable.  This excellent series of seminars will be continuing for the rest of this year and into 2011 and is well worth engaging with, as is the rest of the project’s extensive and highly informative site.

Gotta catch ‘em all

Posted on by

I might tell myself that I hate gaming achievement systems and see them as a cynical way of artificially extending the lifetime of content while simultaneously making one loathe it, but I’m still a sucker for a challenge, even if it involves learning how to use MS Word…

Ribbon Hero (yes, I know) is a Microsoft concept test designed to help players learn more about features of Word and improve their efficiency in using them.  Rather than being presented with a cringingly patronising video tutorial, a terse set of text instructions or that paperclip, the player is given a brief task to complete, with hints available if they get stuck.  Successfully completed challenges award a varying number of achievement points depending on whether it’s finished with or without hints, within a certain timescale or with the minimum number of steps.  And it’s startlingly effective.  I learned more about the features of Word in the half hour I spent achievement hunting than I’ve done in the more than dozen years I’ve been using it before today.  By incentivising ‘working it out for yourself’, Ribbon Hero also makes the player think far more about the processes and patterns of how Word works, genuinely improving their efficiency with other tasks outside those offered by the game itself.  I was surprised at just how effective this approach was, and wouldn’t be at all surprised to see similar game-based training systems used in other products.

Spotted via the June 2010 issue of PC Gamer.

2010 ALT Awards open for entries

Posted on by

The annual ALT awards recognise good and innovative practice and achievement in learning technology.  Entries for this year’s awards open this month in three categories: learning technology practitioners, learning and teaching resources and effective use of video.

  • Practitioners who feel they are ‘outstanding in the use of technology to support learning’ may enter the ALT Learning Technologist of the Year Award by Thursday 10 June.  The award is split in two streams depending on the status of the nominee.
  • The Jorum Learning and Teaching Competition which recognises ‘exciting, innovative learning and teaching resources’ will again be presented at ALT-C.
  • Information on the ALT/Epiguem Award for the Most Effective Use of Video will be available on the site shortly.

Awards will be presented at the ALT-C Gala Dinner in Nottingham.  Good luck to all who enter!

Transforming Assessment webinar series 2010

Posted on by

The Transforming Assessment project funded by the Australian Learning and Teaching Council and led by Professor Geoffrey Crisp of the University of Adelaide is examining the use of eassessment in online learning, particularly in the context of Web 2.0 and virtual world technologies.

A series of free public webinars has just been announced, starting with a session led by Geoffrey on Wednesday 12 May at 08:00h London time. Sessions will be held in Wimba and run approximately monthly. A number of speakers have already been confirmed, but the team are still interested in hearing from potential presenters.

More information on the seminar series can be found at http://transformingassessment.com/

The investment gap

Posted on by

I’ve just had a short chat with a lecturer who was taking a quick break from the class he’s in the middle of teaching.  He mentioned that he was using a SmartBoard, and his frustration at seeing what he could do with the technology if only he’d had proper training in how to do so – as he said, being one of over 100 people at a 20 minute demonstration 18 months ago just doesn’t count as training, no matter how much the powers that be might want it to.

He also reported a similar situation at another institution at which he teaches, which has invested a great deal of money in technology enhanced classrooms and none in training people how to use them.  As a result, the potential of these classrooms is completely unused, and lecturers are frustrated at their lack of knowledge of how to tap into it.

‘They don’t need to pay us to train!’ he said, suggesting that both institutions simply open the classrooms for an hour or two some evenings with a technologist there to help people out, and let small groups get actual hands-on practice with the technology.  It’s just not reasonable to expect a lecturer to have his or her first experience of using new technology be in front of a class of 50 students.

This lecturer’s enthusiasm for exploring new ways of teaching, and his vision of the things he’d like to be able to do if only he knew how, were inspiring and infectious, and it’s so frustrating to see such a clear example of why new technologies aren’t being made the most of.  The reluctance to invest in adequately training staff to use the new technology an institution has just spent heavily on seems like a terrible false economy.

A Word in Your Ear 2009

Posted on by

A Word in Your Ear 2009 – Audio Feedback is a one day conference on the use of audio for providing assessment feedback to university students being held at Sheffield Hallam University on Friday 18 December.  There has been some interesting work in this area in recently such as the JISC-funded Sounds Good project (Bob Rotheram who led that project is the keynote speaker at this event) and this event looks like an excellent opportunity to learn more about initiatives in this area.

JISC resources on MUVEs and gaming in education

Posted on by

JISC have released several new publications recently looking at ways in which multi-user virtual environments and alternative reality games can be used in education.

Alternate reality games for orientation, socialisation and induction by Nicola Whitton of Manchester Metropolitan University reports on the experiences of the ARGOSI project, with which our own Scott Wilson was involved.  The project aimed to support student induction in university and acquisition of required library and information skills using a range of resources such as character blogs and supporting websites.  Student participation in the activity was disappointing, although consistent with participation in such games in general, and the report is possibly most useful for its analysis of where things did not go right – for example, the this is not a game aesthetic  that is fundamental to ARG design may actually be rather inappropriate in a resource designed for students who are already in an unfamiliar and potentially challenging environment.  The lessons learned from this project, and the extensive resources produced by it, make this a very useful study.

Second Life is the undisputed MUVE leader in terms of uptake both within and beyond HE,  and three JISC publications look at how newcomers and the more experienced can develop their practice within the system.  Getting started with Second Life offers exactly what you’d expect, a guide to everything new users need to know from how to register and log in for the first time to some guidance on teaching and course design, some advice on how to address institutional concerns, and a few useful pointers to further reading.  One significant omission is the lack of a list of relevant educational sims (impermanent though they may be) and support systems such as the SLED and Virtual Worlds mailing lists – as the guide itself observes, loneliness and the inability to find interesting locations are two of the biggest factors underlying SL’s massive new user attrition rate.

Modelling of Second Life environments reports on the MOOSE project based at the University of Leicester, which looks more deeply at design and delivery issues around learning in MUVEs and identity and socialisation issues arising from the use of avatars in virtual worlds.

Finally, Open habitat: multi-user virtual environments for teaching and learning points to the Open Habitat magazine, an attractive report on how MUVEs were used with students of art and design and philosophy to understand the nature of virtual group interaction and community building.

All these reports provide valuable information and insights into using MUVEs and aspects of gaming in education, and help to demonstrate the increasing significance of both in current educational practice.