Comments for Adam Cooper http://blogs.cetis.org.uk/adam Cetis Blogs Wed, 07 Jan 2015 09:19:39 +0000 hourly 1 http://wordpress.org/?v=4.1.22 Comment on On the Question of Validity in Learning Analytics by Our pick of Cetis posts 2014 | Christina Smart http://blogs.cetis.org.uk/adam/2014/10/17/on-the-question-of-validity-in-learning-analytics/#comment-97692 Wed, 07 Jan 2015 09:19:39 +0000 http://blogs.cetis.org.uk/adam/?p=830#comment-97692 […] On the Question of Validity in Learning Analytics […]

]]> Comment on On the Question of Validity in Learning Analytics by Adam Cooper http://blogs.cetis.org.uk/adam/2014/10/17/on-the-question-of-validity-in-learning-analytics/#comment-97258 Tue, 06 Jan 2015 10:26:25 +0000 http://blogs.cetis.org.uk/adam/?p=830#comment-97258 Since this is cross-posted from the LACE site, some comments and ping-backs are missing; I would like to draw attention to a “response” post from Timothy Harfield at:
http://timothyharfield.com/blog/2014/10/27/against-dumbing-down-a-response-to-adam-coopers-on-the-question-of-validity-in-learning-analytics/

]]> Comment on On the Question of Validity in Learning Analytics by Worth Reading” is a hand-picked weekly collection of new and not-so-new articles, ideas, events and other items for busy professionals in higher education that prefer to spend their reading wisely. | Acrobatiq http://blogs.cetis.org.uk/adam/2014/10/17/on-the-question-of-validity-in-learning-analytics/#comment-37748 Thu, 30 Oct 2014 22:01:11 +0000 http://blogs.cetis.org.uk/adam/?p=830#comment-37748 […] On the Question of Validity in Learning Analytics […]

]]> Comment on On the Question of Validity in Learning Analytics by Howard http://blogs.cetis.org.uk/adam/2014/10/17/on-the-question-of-validity-in-learning-analytics/#comment-33092 Mon, 27 Oct 2014 18:01:49 +0000 http://blogs.cetis.org.uk/adam/?p=830#comment-33092 A recent book “Validity in Educational and Psychological Assessment” (Newton & Shaw, 2014) that I’ve been reading has a historic view of test validity that may be relevant. Their take is that validity is now heading toward being included with overall practice evaluation that merges a research based approach into technical and social / ethical issues. Evaluation here includes making arguments about the achievement of measurement objectives, decision-making objectives and impact objectives.

]]> Comment on Exploratory Data Analysis by More Tukey Gems | OUseful.Info, the blog... http://blogs.cetis.org.uk/adam/2012/05/18/exploratory-data-analysis/#comment-27702 Fri, 24 Oct 2014 15:19:15 +0000 http://blogs.cetis.org.uk/adam/?p=542#comment-27702 […] a half quote by Adam Cooper in his SoLAR flare talk today, elucidated in his blog post Exploratory Data Analysis, I am lead to a talk by John Tukey – The Technical Tools of Statistics – read at the […]

]]> Comment on On the Question of Validity in Learning Analytics by Adam Cooper http://blogs.cetis.org.uk/adam/2014/10/17/on-the-question-of-validity-in-learning-analytics/#comment-22391 Tue, 21 Oct 2014 13:37:57 +0000 http://blogs.cetis.org.uk/adam/?p=830#comment-22391 Mike –
thanks for the comment. My remark “learning analytics undertaken without validity being accounted for would be ethically questionable” was a little bit of comment-baiting…

I certainly acknowledge that we need to allow experimentation and some degree of risk-taking, but I think we also need to take a nuanced view of what is effective, sometimes recognising that “hard facts” about effectiveness may be misleading or irrelevant, or be in need of revision. Sometimes “moving the needle” may be too stringent a requirement, or an inappropriate means of addressing change in a complex system; sometimes the qualitative effects of a LA initiative may justify it.

So, a less provocative formulation would be: “learning analytics undertaken at scale without considering what kind of evidence is appropriate, and what evidence actually exists.”

Cheers, Adam

]]> Comment on On the Question of Validity in Learning Analytics by Mike Sharkey http://blogs.cetis.org.uk/adam/2014/10/17/on-the-question-of-validity-in-learning-analytics/#comment-21023 Mon, 20 Oct 2014 18:41:39 +0000 http://blogs.cetis.org.uk/adam/?p=830#comment-21023 Good points here, Adam. We need to have more dialog about measuring the impact of analytics. That is, did it move the needle? As someone who recently changed from a practitioner to a vendor, I’ve seen this from multiple sides. I wouldn’t say that analytics without validity is ethically questionable. I just think it’s REALLY hard to undertake a project that proves efficacy. It takes many months to do and it involves a lot of setup to frame the test correctly. Since these tend to be big obstacles, you find folks relying on a measure like accuracy or using the output from one small pilot as their evidence. It’s not unethical. Just because something doesn’t stand up to a rigorous challenge, that doesn’t mean it’s bad. It just means you need to be extra careful.

On the topic of validity, I have executed a fairly rigorous experiment in the past (http://bluecanarydata.com/evidence). I’m also launching one with a new client this week. If all goes well and if the client allows, I’ll share the outcomes in 6 months (yes…it’s going to take that long to see if we’ve successfully moved the needle).

Thanks,

Mike Sharkey
Blue Canary
http://www.bluecanarydata.com

]]> Comment on Open Standards Board – First Recommendations Announced by UK Government Crosses the Rubicon with Open Document Formats | Adam Cooper http://blogs.cetis.org.uk/adam/2013/10/01/open-standards-board-first-recommendations-announced/#comment-3325 Mon, 28 Jul 2014 12:02:15 +0000 http://blogs.cetis.org.uk/adam/?p=694#comment-3325 […] by technical panels, and recommendation by the Open Standards Board. The first tranche, which I wrote about in October 2013, was rather prosaic in dealing with HTTP, URL, Unicode, and UTF-8, and these do not really affect […]

]]> Comment on Objects in this Mirror are Closer than they Appear: Linked Data and the Web of Concepts by About metadata & resource description (pt 2) | Phil's work blog http://blogs.cetis.org.uk/adam/2009/06/25/objects-in-this-mirror-are-closer-than-they-appear-linked-data-and-the-web-of-concepts/#comment-3288 Mon, 26 May 2014 13:51:18 +0000 http://blogs.cetis.org.uk/adam/?p=85#comment-3288 […] See also Adam’s post Objects in this Mirror are Closer than they Appear: Linked Data and the Web of Concepts. […]

]]> Comment on “Openness” – Ideology or Rationality? by A reflection for open education week | Phil's work blog http://blogs.cetis.org.uk/adam/2010/09/15/openness-ideology-or-rationality/#comment-3287 Mon, 26 May 2014 13:49:51 +0000 http://blogs.cetis.org.uk/adam/?p=217#comment-3287 […] technical specifications to one which put openness at the centre. This wasn’t done for any ideological reason, but because we had an aim, “share stuff”, and the open approach seemed the one that […]

]]>