Evaluating Electronic Voting Systems for Enhancing Student Experience

The eighth project in Strand B (Evidence and Evaluation) of the JISC Assessment and Feedback Programme is Evaluating Electronic Voting Systems for Enhancing Student Experience (EEVS), based at the University of Hertfordshire.  This one year project is undertaking an extensive review of the use of electronic voting systems (EVS) in a range of schools across the institution, gathering testimony from both staff and students on their experiences, insights and identified issues and success factors.

Hertfordshire has invested substantially in assessment and feedback in recent years, with an extensive programme of innovations including the purchase of nearly four thousand EVS handsets for use in teaching in eight schools.  The initial response to their introduction, from both staff and students, has been very positive, with the system seen as improving both classroom interaction and staff and student workloads.

The EEVS project will produce a thorough longitudinal study of the impact of EVS, including audio and video interviews, reflective writing and interviews over the course of the academic year.  This long term view will enable the project team to examine key periods in the academic year such as students’ initial encounters with the system, the perceived value and impact on exam performance of interactive revision lectures, technological issues around introduction in new classroom environments, and so on.

The project will produce a number of outputs, including valuable evidence to the sector on the impact of such large scale implementation, detailed guidance on the installation and deployment of EVS and subject-specific case studies, as well as a series vox pop snapshots from teaching staff, students and support staff on their experiences of EVS.  You can follow their progress on their project blog.

Deterrents don’t deter?

A recent article in THES reports on research by Robert J. Youmans at California State University Northridge that found that

Students who are aware that their work will be checked by plagiarism-detection software are just as likely to cheat as those who are not.

Conventional wisdom – and intuition – suggests that the threat of discovery, and subsequent punishment, is an effective deterrent against plagiarism – indeed, one of the comments on the article points to another study that suggested that students’ awareness of the use of Turnitin on a course significantly reduced plagiarism.

It’s not always clear whether plagiarism is an intentional and cynical attempt to deceive, the result of bad time management and poor writing or referencing skills, or due to genuine lack of understanding of the concept of plagiarism or differing cultural norms around it.  The first category of student is the category most likely to resort to essay mills as a safer alternative where it’s made clear that plagiarism detection is in use, which suggests that the majority of students ‘caught’ by Turnitin and other text matching techniques when their use is advertised as a supposed deterrent are those whose main problem is not a desire to cheat but academic or personal factors.

Findings like this seem to strengthen the arguments in favour of using Turnitin formatively, as part of a student’s academic development and the essay writing process, rather than as a way of detecting problems once it’s too late to do anything about them and the student has entered the disciplinary process.  The use of plagiarism detection only after submission seems to be based on the assumption that plagiarism only occurs through a deliberate desire to cheat, and as I’ve argued before, positions all students as potential cheats rather than as developing academics who may be in need of guidance and support to achieve their potential.

InterACT: modelling feedback flow

The InterACT project at the University of Dundee, part of the JISC Assessment and Feedback Programme Strand A (institutional change) is working on enhancing feedback dialogue, reflection and feed-forward in a large postgraduate online distance learning course in medical education.

The course is unusual in that progress is heavily learner-driven: as students are working professionals they are able to enrol and submit assignments at any time they chose rather than according to a predetermined course timetable, and while this significantly increases the flexibility and accessibility of the course, this lack of external structure can impact, together with the higher attrition rates noted in online distance learning in general, on student progress and retention.

Assessment feedback has traditionally been offered at the end of each module of study, when assignments are submitted, which clearly limits the potential for reflection and learning from feedback.  The InterACT project will transform this model through the integration of technology to support a more dynamic, ongoing feed-forward process that actively encourages learners to reflect and act on feedback and builds dialogue between learners and tutors.

The project team have now released their first draft of their proposed new model of feedback, and are actively seeking comment from the wider community.  Dialogue between tutor and learner is focused around cover sheets appended to submitted work which encourage self-evaluation and reflection on assessment performance as well as making explicit the intention that past feedback should impact on future work.  The use of private blogs or wikis as a personal reflective space is intended to encourage this focus on the ongoing interplay of past and future performance.

Do get involved with the discussion, either via the blog post or through contacting the project team.

eAssessment Scotland 2012 call for posters, presentations and workshops

The call for posters, presentations and workshops for eAssessment Scotland 2012 – Feeding Back, Forming the Future is now available.  Proposals should be submitted by 1 May.

This annual conference has become a valuable part of the eassessment calendar, as can be seen by the rich and varied content in the archive of past events.  The conference is being held on 31 August at the University of Dundee, with an online conference running from 23 August – 6 September.  The conference will also host the annual Scottish eAssessment Awards, for which submissions open in early March.  Registration for both events will open in mid-March.

2011: a CETIS year in blogging

If you subscribe to any of our CETIS mailing lists you’ll probably be aware that each month I send out a newsletter summarising our blog posts and news stories over the previous month as well as information on our publications, events and sector funding opportunities.  As part of this I always include a Top Five posts section, highlighting the five most popular posts of the month – a really interesting look at what our audiences are actually interested in.  So with the new year now firmly in place, it seemed like the ideal time to take a look back at what you enjoyed reading – and we enjoyed writing – in 2011…

What you liked reading

The top 20 most read posts of 2011 were:

  1. UKOER 2: Dissemination protocols in use and Jorum representation (26 August 2011) John Robertson
  2. Mobile Web Apps: a briefing paper (2 March 2011) Mark Power
  3. A TAACCCTful mandate? OER, SCORM and the $2bn grant (25 January 2011) Lorna Campbell
  4. Weak Signals and Text Mining II – Text Mining Background and Application Ideas (12 May 2011) Adam Cooper
  5. W3C Opens UK & Ireland Office (19 April 2011) Mark Power
  6. Analysis and structure of competence (4 January 2011) Simon Grant
  7. British Standards in ICT for Learning Education and Training – What of it? (24 January 2011) Adam Cooper
  8. Playing with canvas and webgl (21 April 2011) David Sherlock
  9. eBooks in Education – Looking at Trends (10 March 2011) Adam Cooper
  10. Google custom search for UKOER (20 January 2011) Phil Barker
  11. JISC CETIS OER Technical Interest Group (6 January 2011 ) Lorna Campbell
  12. ÜberStudent, Edubuntu – A sign of what is to come? (8 February 2011) Adam Cooper
  13. JISC CETIS OER Technical Mini Projects Call (2 March 2011) Phil Barker
  14. Crib sheet for 2011 Educause Horizon Report (9 February 2011) Sheila MacNeill
  15. Weak Signals and Text Mining I – An Introduction to Weak Signals (12 May 2011) Adam Cooper
  16. From Design to implementation – DVLE programme Strand A Showcase (31 January 2011) Sheila MacNeill
  17. Considering OAI-PMH (21 January 2011) John Robertson
  18. The Learning Registry: “Social Networking for Metadata” (22 March 2011) Dan Rehak (othervoices)
  19. Using video to capture reflection and evidence (17 March 2011) Sheila MacNeill
  20. Google Apps for Education UK User Group (16 February 2011) Sheila MacNeill

This information was generated by AWStats for our blogs.cetis.org.uk domain, although we’ve recently begun using Google Analytics for tracking, as discussed in David’s excellent post on developing a web analytics strategy for a distributed organisation such as CETIS.

The majority of the most popular posts are from the early part of 2011.  While this is unsurprising – the longer a post has been up, the more chance there is for people to find it – it’s also quite reassuring that the information we’re posting is still relevant and of interest to people after its original appearance!

As stats are collected only for our self-hosted blogs, those that are hosted elsewhere are unfortunately missing.  We don’t have any figures for Scott Wilson’s excellent blog which is always very well worth a read, or for Mark Power’s after May 2011 when he moved to his own domain – again, very well worth keeping track of.

What we enjoyed writing

While some of our posts are obviously of wider interest to our community than others, it’s not necessarily the ones with the most hits that are our personal favourites of the year.  I asked my colleagues which was their favourite story they blogged in 2011 and why…

Adam Cooper: Mine is Preparing for a Thaw – Seven Questions to Make Sense of the Future because I wrote it in the car park of Leeds University and because I enjoyed doing the visualisation which is still hidden in the comments (DOH!)

Christina Smart: My favourite post was Business Adopts Archi Modelling Tool which was an interview with Phil Beauvoir. I’ve done a number of interviews this year, and I always enjoy an excuse to chat to people who are so enthusiastic about what they do. I’ve picked this one because Archi had a great year last year, and to see a JISC funded tool gaining traction outside HE is quite rare, and clearly something to celebrate. (although no zombies)

David Sherlock: I’m not very good at writing and find blogging quite stressful so I’d say my favourite links are more to do with the things I was playing with that I found interesting behind the scenes rather than the writing bit.  I’m going to go with Playing with canvas and webgl because I found using the canvas element to draw shapes was fun, it reminded me of of when coding was fun on my Commodore 64.

Li Yuan: Big Data and analytics in education and learning.  “Big Data” and “analytics” is one of the topics that the JISC observatory working group have agreed to further investigate and look at since they are being applied to all sectors, including government, health, business, etc. This blog post was just an introduction to the concept of Big Data and the implications in teaching, learning and administration in institutions, many aspects are worth further exploring, such as technical, pedagogical and organisational issues in relation to application of big data and analytics in education.

Lisa Corley: Well, i’m not a prolific blogger, but would probably choose What’s in a Word(le)? Lifelong Learning and Work Based Learner experiences… mainly because it brought together lots of work in the programme I had just finished supporting, and after reading all the final reports and summarising them it felt useful to have the summaries in the public domain rather than in some report hidden away somewhere. I also really liked doing the ‘visualisations’ as I think it helps to look at the information in a different way.

Lorna Campbell: Suppose it would have to be a A TAACCCTful mandate? OER, SCORM and the $2bn grant which was an attempt to cut through the crap and  provide a rational summary of a rather overheated situation.  *cough* It also happened to be nominated for the second annual “Downes Prize”.

John Robertson: Hmm, I have to admit that some of the posts from last year which I like the best are ones that never quite got properly started or finished. Perhaps partially because of that and partially because it was a “throw away” response to a tweet which ended up drawing together and developing some of my thinking about open ed.  There’s lots about it that I think is imperfect (e.g. using the word “manifesto”) but it got some things right and there was a certain serendipity to its creation which makes me smile: An OER manifesto in twenty minutes.

Phil Barker: Modern Art of Metadata.  Unexpected interest during a meeting of the advisory group of the Resource Discovery Task Force Vision Implementation Plan Management Framework (I kid you not).

Rowin Young: My favourite post is my look at the excitement that surrounded the Mozilla Open Badges Initiative after the announcement of a substantial prize fund for developments, Badges, identity and the $2million prize fund.  It touched on a number of areas that are of particular interest to me, including gaming achievement systems as both motivators and exploiters and the increasing trend for using elements from gaming in other contexts, identity management in both the technical and social aspects, assessment and accreditation.  Writing the post provided me with an opportunity to work out a lot of my thinking around the topic, and I really enjoyed working on it.

Scott Wilson: My personal fave is this one: Converting Chrome Installed Web Apps into W3C Widgets.  Not because its that great a post, but because of all the chaos that ensued at W3C and elsewhere. This got picked up by Opera, who used it to publicly berate Google and Mozilla about supporting open standards, which drew in Microsoft, and before the end of the month even Adobe had joined in. It actually led more or less directly to the “future of offline web apps” event which was a huge success, and so there may even be a positive outcome.

Sharon Perry: Although I’m not a prolific blogger (only 3 posts in 2011!), I did like the story about using crowdsourcing to highlight and help companies repair inaccessible websites (Crowdsourcing to Fix the Web). I think crowdsourcing is becoming a very important part of social interaction on the web.  Not only can it help solve larger problems by developing micro-solutions but it encourages people to interact and engage with the area concerned.  There is often no financial advantage for those who take part, but the pay-off is perhaps more intangible, i.e. a person who provides such support or help may in turn get that “feel good factor” and a greater sense of well-being for being involved in the greater good.  I suppose I’m also highlighting this story again because the “Fix the Web” cause is now running out of funds and is struggling to survive.  I hope it gets the funds it needs to continue and that it may act as an example to other social enterprises.  Long may crowdsourcing continue!

Sheila MacNeill: My favourite post of last year was called Betweenness Centrality – helping us understand our networks. There are a couple of reasons I’ve picked it. Firstly what started out as as a serendipitous twitter conversation introduced me to a new concept (betweenness centrality)  which I was able to reflect on in terms of CETIS and its networks.  It also helped me to begin to consolidate some thoughts around SNA (social network analysis) and in relation to CETIS as how we can visualise, share understand and build our networks.  Over the past year I’ve been experimenting with Storify as a way to re-publish tweets into coherent stories, and this post allowed me to combine this technique within a more contextualised post.  And finally, the original conversation help brighten up quite a dull bank holiday Monday and legitimately referencing zombies in a work related post was just too hard to resist.

Simon Grant: I nominate Grasping the future (which I had completely forgotten about) for several reasons.  First, it wasn’t something I was thinking about self-consciously and deliberately, but thoughts that came to me from interaction with other people in IEC. I think often that’s the best tradition in blogging: something that would probably not see the light of day were it not for a convenient public platform. Second, because the comments it attracted are really interesting and stimulating in their own right. And third, because re-reading it makes me think, yes, there is something there that I or we really should take forward, something waiting to grasp in the future.

Many thanks to all my colleagues for their contributions to this post, and to all our readers for engaging with, commenting on and sharing what we write – here’s to a great 2012!