It’s a shame I couldn’t make it, but it looks like I wasn’t needed!

thanks for your comment, as ever you raise excellent points. And tbh I don’t think any UK HE IT policies are up to doing what you outline. What I was trying to say was that they could if the right policies were in place. As they said about Steve Austin “we have the technology” but just now the policies need to get sorted out and people shouldn’t just take pot shots at our IT support staff.

Sheila

Are university IT departments prepared from both a policy and technology standpoint to run educational software developed by researchers on behalf of faculty at scales approaching entire departments and whole universities? (I suspect a few do this, but most aren’t prepared to do so. This is a lot of what’s going on right now in MOOCs.)

Do IT systems in the UK let a student, or x00,000 students, run arbitrary code on their servers and evaluate that code securely? (I’m guessing in a limited number of cases faculty and departments are supporting this, such as computer science departments. But I’m guessing this isn’t going on at the scale that MOOCs are, which might just make this a scale argument.)

Do VLE’s or question and test systems let faculty create parameterized items and offer those to individual students? (Some might, but I think that most don’t. To me this is one of the bigger differences in what we’re seeing in MOOCs.)

Can anyone easily view a course in university system? (This is as much policy as it is a technical limitation of the VLE that requires the viewer to jump through a lot of hoops or the faculty that wants to allow it.)

Brandon

Sheila

Interesting post, and I joined in the discussion with @dkernohan on Twitter at the time.

My initial response was that IT systems couldn’t cope with MOOC madness, but I suspect it’s a blend of both ‘systems’ and ‘policies’. At many institutions, VLE areas are created automatically on the back of central student record systems, etc. Further, often students have to be registered for the correct amount of credits to be pushed through to the VLE. How would this cope with MOOCs?

Whilst that is very much more likely a policy decision, I’m sure a lot of people working/managing online systems can recite those occasions where system x (read VLE/portal/assessment system/etc) collapsed under the load of students submitting work, etc. I’ve seen it at 3 HEIs in the last 8 years (probably less actually), and across systems that are hosted internally and externally.
When the user numbers increase to MOOC proportions (even if hardly any of them submit work), the systems will undoubtedly struggle IMO.

P

Julie Jacko

Sheila

And so, as @mhawksey points out above, the policies don’t cope. Not just IT policies, but also others. Yesterday I was called with a query which I won’t detail for privacy reasons, but it related to flexible provision which simply couldn’t cope with the demands of a quality system which is geared to N people doing a course for a year, rather than N/10 people completing the same course 10 times a year.

Online courses which still hold (compulsory) physical course committees and wonder why/complain that students don’t turn up.

I could go on…but the VLE itself is entirely capable of being used to organise learning resources.

Just on a factual note, we don’t have 100k online submissions a year yet, but we do have to cope with 800k assignments generally and we do handle all of the submission records and marks electronically. We are using the data i presented to help us to specify a complete electronic assessment management system. Also, the 8.3m figure was from Jan 2012. And the usage data is from a collection of sources, including @thestubbs.

Our IT systems are definitely not up to handling that lot, yet, but when they are we are hoping that our policies will be ready….

Sheila

“I have a bad feeling about this . . .”

Great link, thank you

Sheila

On a more serious note there’s an interesting response to MOOCs [xMOOCs in US] from the philosophy faculty at San Jose State University [who are tied into many MOOC efforts]: http://chronicle.com/article/The-Document-an-Open-Letter/138937/ .

Oh how we LOLed.

Sheila

We prefer MOOCscape. Looking forward to it!

Sheila

Thanks for this post. The questions you pose chime with some of the thoughts I’ve had through my experiences with a couple of cMOOCs and the online courses I’ve run.

I have enjoyed being able to pick and choose what I wanted to focus on in the MOOCs. While I have participated for my own interest and not to get a certificate or badge, I am aware that I have probably done quite a bit of learning, as I’m sure most people have, that is completely unrecorded. In both of the MOOCs there have also been topics which are not represented in my CV, current role etc that I might like to have some kind of recognition for.

One of the things that has occurred to me is that I might want to submit some ‘evidence’ after a MOOC has finished in relation to the activities in a particular section that I might want to gain some kind of recognition for.

I liked the approach by the School of Webcraft in their pilot of Open Badges using open assessments (the detailed information on the pilot seems to have moved but here is an overview of it from Erin Knight http://erinknight.com/post/3218758524/school-of-webcraft-badge-pilot). Essentially, a number of open web development challenges and corresponding assessments were set in Wikiversity to allow learners to take the challenges when they wanted and submit material for assessment. The learner’s evidence could then be assessed and voted for by peers or web developer ‘gurus’ from the web development community or by stealth when some pre-defined but unknown task was achieved through the process of creating something. Learners were then issued an Open Badge.

Directions were provided for the assessments linked to each badge, which included instructions of the assessment process, how they would be assessed (e.g. who would do the assessing, if it would be one or multiple assessors, the rubric that would be used, and the requirements needed to complete the assessment).

Seems to me to open up new ways of assessing learning in MOOCs, in a more responsive and learner-controlled way. The Mozilla badge backpack allows users to create groups of badges and there is the potential to show the evidence submitted for an assessment – although this is down to the issuer as to whether or not this is included (question over whether or not the learner could control that, as they could swap out evidence) so a significant element of user control is lost there regarding the evidence) – but it does mean that users can potentially pull together groupings of related evidence via their badges and choose what websites to present these on. The Mozilla Open Badges Infrastructure, systems for issuing badges and processes are still at very evolutionary stages but I’m intrigued by the possibility that they could offer a user-owned portfolio combining evidence and awards issued to recognise pieces of learning that the learner has chosen to undertake, when it suited them…

Grainne

Thanks for your comment. I think that most MOOC providers are actually ignoring the need for tools like eportfolios, I think that they could be key in understanding “success’ and indeed lifelong learning.

Sheila

It’s hard to compare MOOCs to online PG programmes as their models of teaching are so different. The later have structured (paid for)support which MOOCs don’t have. There are an increasing number of PG and UG courses that are allowing aspects of their programmes to be openly available, which is an interesting model, and a good way for a potential student to try a course.

Sheila]

What do you think about a MOOC compare to Online Postgraduate Programmes provided by the University?

Thanks for your comment. You’ve highlighted a key aspect of #edcmooc in terms of the demographic of the learners. I suspect many signed up just to see what a MOOC was and not really to get any formal accreditation.

I think that just now there is a mix of self reflection/sense of achievement from as you say following twitter links, commenting on blogs. But if MOOCs are really going to reach out to those who are out of education (and not your typical #edcmooc student) then we need a whole new set of metrics and imho lots of new support mechanisims – not just peer support, but you know proper teachers:-)

Best wishes

Sheila

Interesting post; in a one-size-fits-all Mooc, trying to measure learning and what the statement of accomplishment means is, as you have highlighted, very challenging indeed.

I participated on the #edcmooc too and a question came to my head: how many of the people who finished it were far beyond its baseline level? This was supposed to be an introductory leve 4 course – 1st year undergraduate – but approximately two-thirds of the participants were people with postgraduate qualifications, according to stats shared by one of the tutors in the second google hangout.

One may claim that each person could get something out of the #edcmooc, regardless of their level as they could engage in different activities and discussions, with varied levels of reflection. While this may be true, I also learn from twitter by getting directed and reading blogposts like this or other sources such as the Times Higher Education website etc. Many of us these days read some blogposts, short articles and leave our comments online, in a Mooc-like fashion; should we be given statements of accomplishment for that?

Happy Easter,
Timos

I think I need to write another post based on your comments:-)

Thanks all for your really helpful comments. Andy, I like the MOOC=Book metaphor and know that many people do form MOOC groups. I have a slightly bigger concern that if, as many claim, teenagers don’t need to go to Uni/college but can create their own equivalent learning paths via MOOCs that they may not really have the skills/support to cope with this kind of learning or the skills/support/space to show their learning in a contextualised way.

We’re all in early days and I would like to see some thought given to this wider issues rather than the apparent emphasis on the MOOC platforms/providers.

Thanks again

Sheila

For me, that structure of ETMOOC really worked. There were weeks I just couldn’t keep up for various reasons, and I felt no guilt about it. If the suggested blog topics were uninteresting to me, I just wrote about something else. And I was SO engaged in this course as a result–I focused on what I had time for and what was most meaningful to me.

I think a portfolio might have been useful for me in ETMOOC, much more so than a certificate or badge. I could then have recorded the specific things that I did, that I discovered, that were meaningful to me, etc. Instead, what I did was create a Storify board with links to blog posts, presentations, Tweets and more to tell the story of my participation in ETMOOC. It’s specific–it says exactly what I did and what I thought about, which is much more than a badge or certificate would say. I’d rather have a portfolio because Storify may not be around for a long time!

This is a long way of saying that I think portfolios would be more useful to document learning in MOOCs, and also that there can be value in MOOCs that don’t suggest or encourage “finishing” in some traditional way.

Sarah

Very insightful post. I can promise you if I ever start a MOOC it will allow people to take the badges with them I hear you say you are in?

We have to wait and see what the OU’s FutureLearn will offer and how do employers then view such MOOC badges/certificates etc. I am sure the employers are thinking if the universities are going to run these new breed of courses, they should find a way to accredit these and make part of learning that is formally assessed and awarded credits in the same way as other courses they run.

It will be a lot easier that way for MOOCers and Employers no?

regards,

Manish

For me, taking a MOOC is a bit like reading a book. I make a personal commitment to finish the book, I invest a certain amount in choosing it and reading the first few chapters and if I give up (which, actually, I’ve only done a handful of times in my whole life) I feel like I’ve failed. No-one else has to care one way or the other.

No one has to give me a certificate to say that I’ve finished reading the book – can you imagine! That said, I admit to liking the feeling of seeing a full set of ‘results’ obtained during the MOOC. But that is very much a personal satisfaction.

If I was taking a MOOC for professional reasons (and the one I’m taking currently (and possibly about to drop tbh) comes closest to that category in all the MOOCs I have taken) then I’d probably tell various people at work that I was doing the MOOC – but I wouldn’t consider the certificate of completion to be evidence of anything and success… well, I’m not actually sure what would constitute success if I’m honest. I wouldn’t expect anyone else to be interested in a certificate either.

If I was on an interview panel and a candidate told me about a MOOC that they’d taken I’d be interested… but only in the same way that I’d be interested about any other relevant activity they’d undertaken – I dunno, volunteering or Duke of Edinburgh or something like that.

But that’s one of the troubles with ‘formal’ education – people put too much emphasis on the ‘qualification’ and not enough on the more rounded aspects of what ‘being educated’ and ‘interested in stuff’ makes you.

FWIW, I’ve also taken MOOCs as part of a group – which is a bit like reading a book as part of a book group in that there are people around that you can talk to after (or during) the reading. But, again, if people drop out no harm is done, except that there’s less people to discuss things at the end (or as you go along).

Thanks for a really interesting blog post. I’m currently taking a final Masters module with the Open University, looking at Openness and Innovation in e-Learning. The current part of the module is being taught in an OpenLearn MOOC, alongside hundreds of other learners.

This is a real shift from the closed, ‘safe’ environment that the locked down module VLE/forums offer, and I must say I’m finding a real struggle to a) keep on top of all the posts and b) engage in the weekly activities.

I think my for my first MOOC experience it would have been great to have the choice to continue or to ‘drop-out’ but given that participating in the MOOC is a requirement of the module that isn’t an option.

I did, however, complete an activity that awarded a badge, and I must say having something tangible did make it feel, for me, a bit more worth the struggle…

Adam

S

This is something that I have also been considering (see towards the end of my post from yesterday, where I reference your tweet about Suzanne’s comment).
http://learntechgalway.blogspot.ie/2013/03/e-learning-and-digital-cultures-week-4.html

I still have to write up my overview of my edcmooc experience. Like you, I felt that any learning that occurred for me was via my blog and interacting with a very small number of peers on the course. In fact, when it came to my digital artefact – I just submitted my week 2 blogpost.

I also signed up for ocTEL, in a moment of madness. You’ll know from my blog how it’s going.

Sharon

Thanks for this, yes the self reflection has been crucial for me too, but again in terms of say 18 year olds – not sure that this strategy would work so well for them? Good luck with ocTEL – I’ve decided that would be a mooc to far for me just now but no doubt will get pulled in to the twitter conversation at some stage. Hope it goes well for you too and will look forward to your blog posts.

Sheila

Thanks for this – this is exactly where I am and where I am coming from too. Although I did not take part in OLDS MOOC, mainly because it was at the same time as EDC MOOC, I can see the benefit in learning how to manage my time in accordance to what is available, when it’s available, and I now know what to look for to better plan my MOOCing.

I also planned a blog post for each week and this was far more valuable to me than the final artefact, for which I also achieved the certification.

I’m only planning one more MOOC at the moment, the ALT ocTEL one – I’ hoping that if anyone can do a MOOC properly, it’ll be the Association for Learning Technology!

All the best, David.

I did go to a MiData meeting about 18 months ago and I think MyDex were there. It was even early day then. I do recall there being lots of “heated debate” particularly from Which? representatives who were particularly concerned with the legal , ethical and ownership issues from the consumer pov.

“Mydex are a British social-enterprise helping to make it easier and safer for individuals to hold, control, and re-use their personal information in effective and secure ways.”
http://data.gov.uk/library/mydex

I’ve had a quick look and it’s definitely early days for this project. Interestingly one of the sections is for educational data. The issue Mydex has now is getting another partners to provide data feeds, currently it looks like a lot of manual data entry right now

Thanks for commenting. I think part of the problem is managing expectations. A lot of the hype leads to preconceptions of doing something really new but to many it isn’t – particularly us ed techies However lots of the communication side of things in particular (twitter, google+ etc) is really new to people as is using it as an integral part of their learning.

But there is a lot of work around digital literacy, ple’s and portofolios which needs some serious thought.

Looking forward to catching up next week

Sheila

I’ve been involving myself in the various MOOC initiatives since Stephen Downes, George Siemens and Dave Cormier began propogating the concept. I’ve just completed the edcmooc exercise and will be continuing with the Harvard course on ‘The Ancient Greek Hero’ next week.
What do I think? Well, not a lot so far. I found the edcmooc exercise mildly thought provoking but limited as a learning experience and deeply entrenched in conventional pedagogy and use of resources. I got the impression that it was one of these scenarios where everyone vaguely thinks that it is a good idea, but in practice it just blunders around randomly bumping into elements of learning effectiveness largely by chance.
I shall persevere, however.I firmly believe in the future globalisation of learning through the internet, but only after we have cast off those enduring shackles of institutional education (…!)

Anyway, hope to see you at the Cetis conference next week.

Sharon

Sheila

Me too, thanks for this!

All the best, David

Thank you for taking the time to comment. Firstly this post was written with my tongue very firmly in my cheek and yes it does reflect the “cosy” working world which I am all to aware I am fortunate to be part of.

Far from distaining MOOCs, I’m embracing them and the fantastic opportunity they are giving me and thousands of others. I hope that this is apparent in my other posts about my experiences. I make no claim to be particularly well educated – mainly because I am all to aware that I am not. So I welcome your comments as a counter point to my post and other comments.

I do have concerns that some MOOCs are aimed at “people like me” and aren’t addressing some of the bigger challenges in education our societies faces which need as much investment in face to face teaching as in content development. But they are being seen as “the answer” to what many call our broken education systems.
Sheila

And by people like me let me clarify – with a lifelong love of learning in all it’s diverse forms, financially crushed by the global farce that’s been the world economy for the last five years, and so no longer able to invest in even the simplest educational tools (books!) and for whom the cost of online learning from traditional third level institutions is laughably out of reach – and trust me I’ve tried all the providers and all the funding options, short of streetwalking.

By contrast to people like you, and for you to have any grasp of my perception of people like you, you’ll need to know the route that led me here. Trawling bccampus.ca in the hope of finding some good news on the educational front, I stumble across a post regarding strategic planning, which includes correspondence with Sheila & David K of JISC, which is touted as so insightful and relevant that it must be shared with all.

It opened well, though it took on a puzzling tone, and having read it through, I went back to follow some of the links – which brought me here. Penny drops, light dawns. For insight read cynicism, for relevant read shared stance.

I have to admit to being somewhat dismayed that those who purport to be professionals in the field of the advancement of education across new boundaries, feel it appropriate to publicly snigger and deride an approach still in its infancy. I can only be supremely grateful that such naysayers didn’t manage to deter Gutenberg when he was working out the kinks on his press, or that first nameless individual who came up with the bright if somewhat bizarre idea of turning animal skins into writing surfaces – how you’d have enjoyed making a mockery of that!

But forgive me because it’s apparent that the root of the hilarity is that (God forbid!) if the MOOC approach actually takes root and flourishes, it strikes fear into your very heart doesn’t it? Because where would that leave the exorbitant DVLE offerings that currently fund your smug elitism? Without the ample coin that corporations, colleges and governments currently pay for your ‘insight’ I doubt you’d find it so humorous. Not that I think you have any real cause for fear, since there will always be those who can afford to pay and will do, if only to prove their ability to do so sets them above the masses.

How reassuring to know that the more things change, the more they really fundamentally stay the same.

As an aside, it might interest you to know that many people who engage in the MOOC courses you disdain, have no great interest in the constant sludge of social media. Of those that I’ve taken, twitter has been mentioned once and facebook not at all – for which I confess I am profoundly grateful, being no particular advocate of either. Perhaps if you focused on the content rather than the social whirl, even someone as exceptionally well educated as you purport to be, might actually learn something.

In a traditional classroom setting, you’d be the wittering gossip who’s more interested in discussing what to wear to the rave tonight than paying any attention to the physics problem the teacher is describing – derailing the learning experience for others by forcing the teacher to deal with your insubordination. But thankfully we’re not in that setting, which is why a teacher with thousands of interested students wastes no time on your snidery, you’re not even a blip on their radar.

I was truly heartened to discover MOOCs not just for the increasing range of courses on offer, but also, having dipped my toe, for the depth and quality of the content.

I recognise that nothing I say is likely to have any effect on your mindset, but my purpose is writing is simply to make you aware that there is another point of view, and that there are many people (outside of your cosy world) who benefit from MOOCs in more ways than are dreamt of in your philosphy.

MOOCs may not last forever, what does? But while they’re here, allow those of us without the luxuries of time or money to access traditional education to relish the freedom to learn without your scorn.

Thanks for your comments and yes hopefully we’ll see some more interesting use cases over the coming months.

Sheila

There is some interesting custom reporting work being done on Tin Can data, and I think the community will benefit from seeing practical use cases once it all comes together in the coming months.

When I hear questions such as “What does Humanity mean …” I’m reminded of the nominalist/essentialist debate in philosophy. Nominalists tend to read definitions from right to left, whereas essentialists read from left to right.

For example, an essentialist asks “what does this term X on the left really mean, what is ‘the correct’ definition on the right.” The nominalist, on the other hand, asks: “Since we have a number of interesting definitions on the right, let’s agree for the sake of this discussion what short hand abbreviations (terms) we would like to assign to these definitions.”

Historically, it was fairly easy to come to some agreement on what the term “human” ment because all flesh-and-blood humans were largely determined by their genetic code. Human babies grew up to become adult humans and not trees or some other life form.

However, as we gain mastery over nature we gain the ability to fundamentally transform ourselves and our species. Education as we know it today – communication through the spoken word, writing, printed books, television, and audio/video/mouse/keyboard computer interfaces – is just the tip of the iceberg. We are now beginning to gain mastery of our genetic code; we are finding better and better ways to scan and understand our brains; and, we are building better and better direct brain-computer interfaces.

Some futurists, such as Ray Kurzweil, believe that machines will attain a mastery of natural language in about fifteen years, they will far surpass unenhanced human levels of intelligence in about thirty years, and that enhanced humans will also merge with their machines to transcend biological evolution in about thirty years.

According to Kurzweil, humans enhanced with direct brain-computer interfaces will do “most of their thinking in the cloud.”

In your post you say: “I’m heartened by the fact that almost every project I know of emphasises the need for time to develop human relationships for technology to be a success and bring about change.”

I’m not sure how Kurzweil would respond, but he might say that enhanced humans will do most of their “relating to one another” in the cloud as well.

If Kurzweil is right, it would be an absurd understatment to say – “Oh yes, I think this might have a significant impact on Higher Education as we understand it today.”

Some economists and educators might say – this is all idle speculation; it has nothing to do with lowering unemployment today; it has nothing to do with lowering the cost and improving the quality of education today.

As for myself, I do think it’s time for economists to re-examine their long-held beliefs that workers displaced by technology will always be able to be retrained to find new jobs. And, I do think it’s time for educators to start asking themselves if education as we now know it will be able to help unenhanced humanity keep up with its race against the machine.

Finally, I would hope that educators will be able to help both those who want to become enhanced humans and those would would rather die first find understanding and compassion for one another. If both can do so, then I would say that both deserve to be known as Human.

Thanks for taking the time for commenting, and yes that’s exactly why I included the twitter comments:-) so thanks for responding via twitter too.

Thanks for sharing this. Yes I think context is always going to be crucial. We’re going to be sharing a couple more case studies over the next month or so which will look at some of these issues in more detail.

Sheila

Aside from the ethics of whether they were empowered to have a choice in whether it would stop or not….thoughts were that it partly depended on your relationship with your line manager, especially if it is seen as performance monitoring. Some felt they were happy for a system to do the monitoring and feedback but wanted to opt-in or out at a more granular level as to who sees that feedback and analysis and for how long.

They also felt that this would not be the same for every learning activity and the opt-in/opt-out would change depending on context too as well as relationships with line manager.

I guess it kind of translates to student / instructor / facilitator / institution.

Of course I can share how I did it. I basically used a free software package called Gephi and followed this tutorial from Tony Hirsthttp://blog.ouseful.info/2011/07/07/visualising-twitter-friend-connections-using-gephi-an-example-using-wireduk-friends-network/.

If you are in the mooc facebook group it is very simple as you have access to the data. Just install this app
https://apps.facebook.com/netvizz/ – and download the file and put that into Gephi then follow the tutorial. Make sure you get a .gdf file. You can choose the type of data either friends or interactions. The MOOC group is very big so you might want to start with your own personal network first. I’m very much a novice at this and it is a bit trial and error but Tony’s blog is a great resource too.

Sheila

Since joining this MOOC I’ve learnt the basics of using TAGS and have used it each week for the chats. I was wondering how you were able to analyse the Facebook group? Is a similar technique used? Are you able to share your secrets? I would love to be able to learn how to use such software.

Thanks for the blog, I much enjoyed your post.

Sheila

A question I have is about the use of MOOCs to teach something other than MOOCs and their implications! It seems to always develop as a discussion topic even on courses ostensibly on statistics, history, etc.

As for the assignment/credit. I’m not sure. I’d like to complete this MOOC certainly, but it’s a busy time and I suspect that’s true of many participants. The excitement and accessibility of MOOCs and other forms of online learning can sometimes obscure the fact that if these are really meant to be ‘courses’ then that means we have to put in effort as students and that means carving out a time commitment and not just surfing along. One of the issues then is for MOOC designers to come up with an appreciation of , and to notify learners about, just how much time needs to be spent on each topic and to seriously examine how realistic an expectation that might be for their intended target population. Of course, from experience in distance learning and learning design, there are ways of doing this, but many other MOOCs seem to have not given this much real consideration and simply bludgeon their way through lecture after lecture.

That’s another strength of #edcmooc I guess. The material is all up front, and it is clear that the ‘workload’ needn’t be too onerous, depending on how much effort you are prepared to put into the assignment and discussion components.

MOOC’s will eat themselves !

Thanks for your comment. I agree entirely, effective participation is new to us all right now. I think much more attention needs to be paid on support and helping learners to contextualise where and when they interact online, so that they can make effective contributions and not feel lost, overwhelmed and isolated.

Sheila

Thanks for your comment – I think there is potential for elements of some mooc course designs to be used very successfully, however I think that it will take a lot more research and recognition of the need for more investment in student support/staff development.

Sheila

I think that is actually the crux of the matter. I have seen educators look at how successfully discussion boards and video games (possibly now MOOCs) have been for recreational purposes (not educational) and think how can we synthesize those very “essences” that make it great but within an educational context – however, most of our students are not as self-motivated, disciplined and enthusiastic as the people on the MOOC courses. So, it is potentially going to be an interesting challenge (or else we will be going up yet another blind alley).

It could very well be the wrong question, I really don’t know any answers, it just was a question that came to mind.

Thanks for your comment and I’m glad I’ve been able to contribute to a good start to your Friday, and I’m glad your finding #edcmooc a utopian experience.

thanks for your comment. I suspect that there will be a bit more organisation of the final assessment, and yes you are right there are ways to do this. In this instance from a personal point of view, the final assessment isn’t that important for me (shock, horror!) – I’m really more about “the journey”. But I will be sharing more as we get nearer to it and my views of the assessment experience.

Sheila

I’d be concerned in this case about how effective the peer assessment is going to be in the course you’re taking.

> “The final assessment is the creation of a digital artefact which will be peer assessed. Contributing to online discussions is encouraged but not mandatory.”

It sounds as if the instructors are not setting up the conditions for it to be effective. The articles that have come out on the other experiments with peer assessment seem to say that it’s really important to think about grouping of students for review (by region, native language, mixture of expertise, etc.), the use of rubrics and examples for grading and to organize the reviews, the importance of expectation setting and lots of practice. We’ve had one of the Coursera faculty talk here about his experiences and the limitations of the platform and student expectation and skill. There are ways to build this into the course, a single end of term peer assessment does not sound like it’s a recipe for success. (And there’s another one we know well which we should chat with.)

As with many educational experiences, having the right structure and organization can immensely improve the experience and outcome.

Thanks for commenting – glad you too use the power of cake to get you through things:-) auto correct is my nemesis too. Doing my best to keep up the obsfucations

Sheila

I doubt it was meant as imposition if an American idea with the name, and I would have argued for something else.

The biggest problem? A Learner Bill of Rights that is not written by Learners? It feels a bit hollow in that regard.

If this is to be hacked, lets get it in the hands of Learners and get out of the way.

Will be really interseting to see where this takes up.

Sheila

I’d better write something.

TIA

Paul

I think the HE approach to learning desperately needs a shake up and its current state of aggresive defensiveness might be an indication that MOOCs are threatening something.

Thanks for you comments and I’m glad you found the post helpful.

Sheila

John, I reckon there should be a badge ” Professional Cloudworke”r – let me know when you succeed!

Thanks for this nice review! We are very excited about this and also think it is a significant output!

As it says in the text -the image is via xkcd – link when you click on it.

Sheila

Im doing my dissertation on recruiting prospective students and what info they would like. The image you have used is something Im looking for. Did you make that yourself? Would it be ok for me to publish it within my project, please let me know if you have other information. Thanks

S

No problem. Looking forward to catching up with the recording and joining the discussions.

S

Sheila

S

I am an avid reader of your blog. With this article you did about Learning Analytics, I thought that you can make use of this infographic that we just developed. You can check it out here: http://newsroom.opencolleges.edu.au/learning-analytics-infographic/.

Please do let me know what you think. And if you can feature it on your blog, I’d be really ecstatic!

Thanks.

Krisca C. Te
Open Colleges, Sydney

Absolutely! I think we all just need to (and are) get better at filtering (ou and in) and recognising that we can’t keep up with everything. The twitter backchannel has been a really important addition to conferences and post conference communication/networking, and I would hate to lose it. I just get tired of some people moaning about tweets instead of just ignoring them, and doing whatever it is they do.

S

Martin

I imagine an interface that allows you to see your own connections and see those of others. It might make it possible for learners to find patterns of behaviour that are more successful for them… both by assessing their own and seeing others.

Yes it is complicated (such an understatement!!) On reflection I guess I was drawn to the “twitter stuff” as it is now such an integral part of my professional practice. And I have had a growing interest in SNA for a while now.
What impact it actually had on my learning is hard to tell, but I do “learn” a lot from things I find on twitter, sometimes in semi serendipitous ways, and sometimes in a very connected (ie this is what my peers are thinking about just now) ways. I think that does resonate with your rhizomatic ideas.

How do I interact professionally?
How much am I being forced to do this?
Do i like to think outloud?
Do I prefer input on my thinking at the beginning, during or after i have completed my ‘work’?

I’m still not sure how to track the social networking stuff and not get confused as to what i’m tracking. There’s great potential there, i think, particularly in making a students work transparent to them…

Lots to think about

I did it as the course progressed and it was great that Martin picked up on things and released his search template.

S

A couple of question…were you monitoring this activity as the course progressed, or did you undertake this analysis after completing the course?

Thanks for your comment. Yes it will be interesting to see if we get some more analytics from the course team and their reflections on things.

I think there is more to think about in the course design element around rhizomatic learning too. I was thinking about that when we were discussing designin moocs. They still seemed a bit linear to me . . . and some visualisations could maybe help show how connections grow, break-away, die etc. But again it does all depend on the learner. When there are no formal assessments, you really need to be quite a sophisticated learner (as well as digitally literate) to actually articulate what you are experiencing. Something I need to work on a bit more . . .

I spent some time watching the twitter traffic this week and was struck by how much of the dynamism of the course was not captured in this stream of information.

I am curious if we will find out what other analytical resources the organizers tried to access and what they hope to do with it.

Still more work in making these archives more useful, can’t help but feel there are opportunities around rhizomatic or vicarious learning …

best wishes

Iain

Sheila

I like Rachel’s idea of making a goal to connect with 2-3 people a day – that’s what I did at all those many conferences I used to attend (and plan). I am going to approach this as though it is the first of a new type of conference I’m attending, and use some of the strategies that worked so well for me in the past.

I wasn’t planning on giving up, just trying to figure out a way in. All these ideas were very helpful.

yes it was great. Thanks for the link – I think that sums up how most people have feel at some point in a MOOC.

Sheila

I made a comment on the post, but it feels like a place that might need some more populating.

Yes working in different time zones a bit confusing:-) Just update post with video of chat. I found it worthwhile so would be up for doing another one – if we can work out suitable times:-)

Sheila

Sheila

I too find MOOC’s very isolating…the tweets have been coming fast though, so I think that’s why I have felt more enagaged…

Can’t make your hangout…as I am already supposed to be two other places online at a similar time (have yet to figure out how that will work…) But I wanted to write and say – I can relate – I think that is something that needs to be discussed, how you can feel so alone within a MOOC…and Why

I am on twitter for future chats…@verenanz

Thanks for the response and totally understand re times. I’m usually a bit brain dead by time twitter chats start so just thought this would be an alternative – maybe chat later.

Sheila

Sheila

It could well be – but I have a feeling that it there would be more US participants whatever the time of year. Would be interesting tho to compare if it runs again.

Sheila

I’m wondering, though, if your comment re. it being mostly US participants is related to the fact here in the UK, we’re right in the middle of the summer break (and everyone was glued to the telly yesterday!) whereas in the US, they’re already starting to head back; don’t their terms start somewhat earlier than English ones (both at School and University level)?

Thanks for your response, you raise some great points. I think to an extent MOOCs do deliberately put more responsibility on the learner, particularly the ones that hark to the original MOOCs from Downes, Siemens et al. They are for more sophisticated learners and despite my self deprecating remarks I know that I can network effectively and deal with troublesome knowledge. But for most undergrads or people returning to education I share your concerns. The massive be be appealing and provide enough people who want to take the final test (and so give a commercial return) but what about the masses who drop out? I can afford to be philosophical about my MOOC engagement and successes/failures as I’m not relying on them for anything in particular.

Gerry McGovern wrote along similar lines back in 2010:
http://giraffeforum.com/wordpress/2010/04/18/the-decline-of-the-homepage/

It prompted me to review stats on our university website and to be honest I couldn’t find evidence to really back up what he was saying:
http://usability-ed.blogspot.co.uk/2010/05/homepage-design-and-purpose-mcgovern-vs.html

S

Yes, this isn’t formal specification bindings, but as a first step to understand the principles behind linking data structures between systems, I still think this is a useful explanation.

Sheila

Sheila

I’ve blogged anything he’s done that I’ve felt relevant to HE web management at Edinburgh for the past 3 or so years:
http://usability-ed.blogspot.co.uk/search/label/Gerry%20McGovern

Sheila

There is a lot of statistics and analysis on the Government Digital Service blog about their use of the approach. It’s also being used in the US with some higher education websites. I’ll be writing more about this theme at #iwmw12 shortly.

I don’t know if Ferdinand is aware of Gerry’s Top Tasks, but aspects of his talk certainly rang the same bell.

https://docs.google.com/spreadsheet/ccc?key=0Ak-iDQSojJ9adFhkdGtqVXFIVXNWZ2dMN2UzNzN2VEE&pli=1#gid=116

Personally I like to have space to try things out that isn’t connected to any of my current online identities, if it goes well then I want to be able to claim that space as mine somehow. Canv.as is the only social network I’ve seen implement that but I think its a really interesting idea.

I think the idea that we have one identity that can be tied to a picture and name is crazy!

Sheila

I think web and social sharing analytics is a real double-edged sword! While its great to know which audiences you are reaching and how effective a web campaign has been its very easy for me to get stuck in a feedback loop where I see an action as having a positive effect and push more resources that way without really thinking about why. I think asking whom it is going to is more important than how many.

Social networks analytics are a minefield! Perhaps because it is so tied up with our identity.

Interesting about LinkedIn, I think Facebook is very much identity driven and it forces you to have one identity tied to a real name and picture which limits what people are willing to share on, do I want my friends knowing we have work projects named after peaches? Perhaps this is why they are more willing to share this kind of work on linkedin where there identity is tied to professional interests? Twitter has become much more interest driven recently and I guess it doesn’t matter what you share when your identity is a throwaway username.

I think its useful to remember that different social networks have very different demographics, in the past I’ve found fun bits I’ve done around games/code/technology seem to have been the most socially shared, but I guess thats sort of thing really turns the average Reddit reader on!

That should be the link working now – had used a shortened link so think that might be why there was an error. Thanks for letting me know.

Sheila

Thanks for the comments and glad that you got a sense of the event from the post – I should have added some pictures of the dolphin display at the conference dinner:-) Re visualisations and connections between end users and developers yes we do need more connections. As you pointed out in your recent post there is a new wave of analysists coming through – kind of like the early days of learning technologist – all with different backgrounds but hopefully similar aims in terms of improving teaching and learning

S

Thanks for the commnet and I agree not everyone was/is talking about the bottom line – but it was more the differences between our systems which struck me, and also the fact that the North American model seems to be something our polititicians are drawn to.

Your five headings and discussion beneath each give me a sense of cautious optimism. In each, there seem to be possible clouds on the horizon but also promise of fair winds.

For example, the comment about things being done *to* students is something that has niggled me when reading many accounts.

Similarly, while we almost-all like a nice visualisation and they are a definite improvement over tables of numbers, they are not necessarily informative. Can I learn anything that is useful? What does it mean, really? I think this is not just about skills to interpret; it is also about ensuring that the right things are being shown in the right way to allow us to use our visual reasoning? I think this implies we need a better connection between the end-users and the developers/analysts.

Cheers, Adam

I wrote up my thoughts (along with some of my session notes) on my blog — but didn’t really publicize them until now (http://www.mura.org/2012/05/trip-report-thoughts-coming-out-of-learning-analytics-2012/)

Some reactions…

Re: U.S. Colleges and universities and marketing budgets to get students into their classes and keep them there.

I think that’s one way of looking at the use of analytics, but a slightly more charitable view is that some colleges understand the investment that they make in every student they enroll (more than just their tuition/fees). So they are motivated to see the students succeed–and if analytics can help that, then it’s a win-win. It’s a case of where the institution goals–ensuring a positive return on the investment–and student goals of completing/graduating usually line up.

Re: Linda Baer and Don Norris (see http://www.mura.org/2012/05/trip-report-thoughts-coming-out-of-learning-analytics-2012/#norris)

Of the plenaries I saw (I missed the Tuesday afternoon and Wednesday ones), I liked their’s the best. I thought it was a clear presentation of the issues to consider.

– as long as you don’t use it to do Bad Things. The only way is ethics!! Great post, & really enjoyed your live-tweeting of the conference. Many thanks.

Thanks for the comments and yes agree – just because you might not know what to do with data shouldn’t preclude you from accessing it.

S

Thanks, that’s a really interesting and thought provoking overview of the conference. A couple of thoughts…

if students were given access to their data, would they actually be able to do anything with it?
Possibly not, but surely that doesn’t necessarily mean we should deny them access of even ownership?

also if we are collected data about students shouldn’t we be also collected similar data about teaching staff?
Absolutely! I suspect one way to really understand student data will be to compare it to teacher data.

One last thought, I agree that the ability to understand data is a fundamental aspect of digital literacy and one that we need to make much greater effort to develop. One day I want to be able to understand what all those pretty pictures mean!

Martin

Thanks for this, yes fostering more links with schools and using the library as a hub for this is definitely worth encouraging more. Yes staff training is crucial too, and to ensure that it develops and is encompassing to foster literate abilities.

Sheila

Sorry about delay in replying. Was in London and then knocking together a PPT for LILAC. Suggestions might include:
Some universities have wider access programmes which might include contacts with schools and local welfare organisations and involvement in these is worth considering.
Secondly pupils in years 5-6 in local schools should be encouraged to use the university library for dissertation etc work. There has also been formal work in linking IL skills training between school and HE (e.g. Moira Bent at Newcastle). University librarians should work with local school librarians on possible joint programmes.
There is also the question of training of teachers. We will not have information literate school pupils until we have information literate teachers. Trainee teachers should receive IL training. Strathclyde U would be a good place to start. The Dept of Info Studies could offer training.

Best wishes

John

Seriously though, I’ll be interested to see how this develops. We’re all still learning when it comes to the most effective use of this technology.

All the best,

Bill.

Got to catch up. Would be intersted on hearing more of your views on how we could join up the informal and formal sectors more?

Sheila

Bill told me about this last week. Hope blog attacts a lot of postings. Good summary of IL issues above. However my interests have been for quite some time more in non HE, workplace and informal learning and I am increasingly concerned about the relative disinterest among academics in an area which does actually take in the vast majority of the population. Academia – university students and staff and academic librarians seem to have a clear view about what is most important in IL R&D – themselves – time to turn the tanker round?
PS Mentioned blog to Christine but she is a bit busy.

John

Glad you found it useful, and for the further links. Rebecca has also linked to work Simon is involved in too.

Sheila

The follow-up session by Simon Buckingham Shum at the OU dovetails nicely with Gardner’s session outlining the idea of an Open Learning Analytics ecosystem where the community can deposit and reuse analytics data which should help research explore ways to surface things like positive intervention http://lak12.wikispaces.com/Recordings

[You can also see parallels to this with the recent Value and benefits of text mining report http://www.jisc.ac.uk/news/stories/2012/03/textmining.aspx

Martin

That’s odd – I thought everything was open to anyone. Here is the main website for the course http://lak12.mooc.ca/ – maybe navigating to the course recordings that way will work – it did for me.

Sheila

Totally agree and that’s kind of what I was getting at. We need to ensure people get the chance to be involved in the process of gathering and using data, not just presented with “stuff” other thing are useful. If we only measure part of the picture we could end up in the ‘teaching to data’ trap like the teaching to the test trap. I think initially we just have to let people join the conversation.

S

“Invite-Only Wiki
This wiki has restricted membership. Contact the wiki organizers for details.”

Any suggestions for getting access?

Thanks, Stephen.

It’s not what you’ve got, it’s what you do with it that counts… how do we support a moral framework around decision making?

Sheila

s

Thanks for you comments.

I can see that there does need to be some reductive processes involved, however I think what I (and indeed Gardner conveys far more eloquently ) is that there isn’t just one map and you can only see the directions needed for driving to you destination when you are walking. Or indeed you don’t have to take the most direct route – an indirect one might be much more valuable in terms of and educational experience.

Sheila

The terminology issue points at the question of audience: is it for managers, teachers or learners? I’m sure we’ll settle on a good term for the application of analytical methods to learning in general, but it’ll always be crucial to bear in mind who the audience is.

The question of reductionism is more fundamental: I think analytics has to be reductive to work. A map that faithfully reproduces the full complexity of reality is useless. A map must simplify so that people can find the interesting and rich parts they’re looking for.

In that sense, I think there might be a tendency to over-estimate what analytics can do; If there are no clear quantifiable dependent variables, then the techniques we refer to by the name of ‘analytics’ are the wrong tool for the problem at hand. Al we can hope for is that it points at that problem.

S

Dear talented researchers,
There is a really interesting, continually emerging research situation very close to home.
Whether participative or illustrative, your methods and findings could make a real difference to the experience and outcomes of the learning community that surrounds you.
The good news is that there’s probably a network of people in your institution willing to work with you to design and build research instruments.
A label of “Learning Analytics” may ring some alarm bells and probably calls to mind the Einstein quote “Everything that can be counted does not necessarily count; everything that counts cannot necessarily be counted” but that’s why this project needs you!

Thanks for your useful, timely and relevant comments. Tim I share you concerns re universities not moving fast enough. That’s one of the reason we developed our matrix, to see if it could be used raise some key issues and allow people to start thinking at a more holistic level.

Sheila

Sheila

Bill.

Sheila

Interesting point, yes you’re right we mustn’t forget the non digital aspects of society and education.

Sheila

Not that I don’t think digital participation is needed – I do. But we also need to continue to encourage and develop ‘good old fashioned’ Participation.

There is an online group in LinkedIn which you can join.

Sheila

Cheers, Bill

Cheers, Adam

Totally take your point about the data. In the process of doing this we unearthed a few more rogue institutions, so Wilbert has been updating, so I’m sure he’s delighted you have a few more to add to his list:-) But unless we actually start playing with the data, we won’t find out these things.

S

Going down a dedicated KML template would certainly be my recommendation. The PROD site already renders the same data in different formats (HTML, RDF) so I it wouldn’t take much to add KML (perhaps a link of the query results page).

It’s also worthwhile remembering that linked data is well … only as good as the data. I’m sure Wilbert will chuck an AA roadmap at me if I mention this again, but not all the project lead institutions currently automatically generate a set of co-ordinates. For example, if I query the ‘open education’ strand you get 51 results and if I do the same query with institution locations I get 29 (This is particularly an issue with the OER strand because some of the leads were Subject Centres which aren’t recorded in the JISC Monitor Unit data. One of the serendipitous outputs of the OER Visualisation project is a list of unresolved locations which Wilbert can add).

Martin

Yes of course

S

Thanks
Ruth Drysdale
JISC Exec

Thanks for this. I agree “open” is a reasonable term to describe that process, my point was that we need to make sure that we are truly democratic in our use of it so we don’t close people off before they appreciate what aspects of what they already do are “open” and then don’t get realise the full potential of open practice and OERs.

S

Apologies for links to 2 of my own blog posts here;)

I’ve heard mention of an O’Reilly book dealing with open source stats/data-science tools, but I’ve no direct experience. I think Tony Hirst mentioed it on his blog but I’m too lazy to go and look.

In the future, I hope there will be more institutional support for people like the WORDLE crowd; this seems like one aspect of digital literacy that is coming down the road quite fast.

Cheers, Adam

Oh yes, I think there is a whole new twist on transactional distance through the use of social networks. I *must* register for your course! would be great to keep talking about this.

Actually that crossed my mind too re the SIGs too.

S

” . . . innovation tends to be spread by people who are peripheral in communities”. I think this is a key point for an Innovation Support Centre. We don’t need to be heavily involved in communities to have an impact, but we need to be able to make the right connections.

I think CETIS’ role in making connections between communities is critical to our usefulness as an Innovation Support Centre. And in some ways I think this goes right back to the network of connections, communities and individuals we originally built up through the SIGs. The SIGs may have predated twitter but I bet if we could go back in time and analyse those networks we’d see similar levels of betweeness centrality. To be able to query the data to illustrate these connections is a real step forward though. Fascinating stuff.

I will be watching this as am interested in connectedness between projects – useful for synthesis activities…

You now owe me 6¢

In PostRank Analytics, which collects social shares and more, it’s interesting how they put a suggested $ value on engagement weighting comments, like/tweet and bookmarks differently. For example, in PostRank terms this comment is worth 10 ‘engagement points’ and the tweet I did 7 points. At a suggested $0.25/pt that’s $4.25 I’ve earned you

If you total all the social engagement actions recorded in the spreadsheet the project home pages have generated over $30k in ‘engagement value’. With the focus on value for money I wonder if this type of data will be used to evidence benefit.

If anyone is interested in how I generated the spreadsheet this post http://mashe.hawksey.info/2011/08/and-the-most-engaging-jisc-project-is/ has more detail.

Thanks for sharing my work

Martin

yes I am aware of MUSKET, and should have mentioned it in the blog but thanks for reminding me. Yes, think we really need to get some people together to see what we can do with all we’ve got.

Sheila

With a whole bunch of learning outcome statements, it ought to be possible to use this type of tool to come up with plausible and helpful matches to material that a course designer is working with. And with an increasing library of already analysed text, you’d get better and better help.

Although the MUSKET tools were originally designed for course description comparison, it seems there are endless potential applications – including lots in curriculum design, course advertising, APEL and others.

Yes, Tony has very kindly offered to do some more experimentation with some of our personal twitter networks too, and I’ll keep everyone updated with progress.

So I guess what someone needs to do is to reproduce the kind of automatic functionality you get with LinkedIn Maps and port it to other tools. I guess someone has done it for a price, but the challenge will be to do it for free…

That’s frustrating isn’t it – I hope you are spamming Gartner back now too

Sheila

I think there are elements of the framework there, particularly the SOA approach, but what they have built is very much an institutional (and an distinctively online institution at that) take on having flexible services.

Sheila

I have to confess that I haven’t read the Case Study yet. (That registration form -meh.) However it struck me that what you are describing sounds rather a lot like *whisper* the eLearning Framework. Are their similarities or am I deluded?

Also I absolutely agree about the importance of communicating failure, otherwise how can we learn form our mistakes and move forward in innovative ways?

Thanks for your comments – you might want to look at dipity – that has some more functionality like keyword searches in twitter and the ability for a user to add their own “events” – an example http://www.dipity.com/sheilmcn/JISC_CETIS_EVENTS_2/ (NB I’m using the free version so there are ads).

Sheila

Hopefully we will be running another “bash” later this year – so watch this space for more info

Sheila

Nadia

Thanks for sharing this. I have been looking for something like this for ages! I am going to test it immediately. And Simon, wouldn’t it be great if such a timeline could be generated automatically from the ePortfolio artefacts (or partly, as a basis that one could then reflect upon and improve)?

Nadia

Great post. Widget Bash looks a great event. Will you be running it next year?

Thanks,

Chris

Yes, there are a number of services which do this kind of aggregation but there is just something about the simplicity of this interface I particularly like.

Sheila

A quick note that as an illustration I’ve set up an account for the OER topic feed http://memolane.com/cetisoer

John

At UKOLN we are looking for a light-weight, easy-to-use service that could create an online events/wall planner type calendar thing. Memolane looks like a possible option. Am definitley going to investigate. Do you know of anything similar that is also worth looking at?

Marieke

Yes – loads of possibilities and project reporting a great example. My personal “memolane” is @ http://memolane.com/sheilmcn

I love your writing style. It’s imaginative and interesting. Liked your Moodle reading list block to wookie widget.

Thanks for taking the time to post.

Good points – and yes like all data collection we need to be clear about the why and the how. Thanks for the comments and the references.

Sheila

Thanks for the commments. Kind of horses for courses in terms of what works best for catching up with things. I use netvibes to aggregate blogs I regularly follow, but the paper.li does through up the serendipitous links relatively regularly.

Seriously, it does look like an interesting way to catch up on key stuff your twitter friends are talking about (I think it chooses stories based on the frequency of references in your f-stream? not sure). I’ve not used it as I’m a big google reader fan and tend to subscribe to blogs willy-nilly, so catch up with things that way.

James

Sheila

“They used to say if man could fly, he’d have wings. But he did fly. He discovered he had to. Do you wish that we didn’t invent writing, or that we hadn’t invented computers or the wb? That’s like saying that you wished that you still operated with scalpels and sewed your patients up with catgut like your great-great-great-great-grandfather used to. [insert name here] is right in pointing out the enormous danger potential in any technology as fantastically advanced as this. But I must point out that the possibilities, the potential for knowledge and advancement is equally great. Risk … risk is our business. That’s what this University is all about. That’s why we’re aboard her.”

http://www.downes.ca/cgi-bin/page.cgi?post=54807