Comments on: The logic of competence assessability http://blogs.cetis.org.uk/asimong/2011/08/31/competence-assessability/ Cetis blog Tue, 22 Aug 2017 13:13:28 +0000 hourly 1 http://wordpress.org/?v=4.1.22 By: Simon Grant http://blogs.cetis.org.uk/asimong/2011/08/31/competence-assessability/#comment-166 Mon, 05 Sep 2011 16:31:11 +0000 http://blogs.cetis.org.uk/asimong/?p=861#comment-166 Thanks, Alan!

Yes, with pop songs, I should have said “more or less successful” and specified that one can measure that in terms of sales. Certainly what anyone considers a “good” song is a matter of taste, though it would be strange to have a definition of a good pop songwriter whose songs were not popular, on some measure!

You say “personal preference, or, if you like, judgement…” but there is an important distinction which should not be lost. Some judgements have more inter-rater reliability, others less. Where there is good inter-rater reliability, it is reasonable to set standards — indeed, many NVQs rely heavily on this.

Similarly your next comment. One can have a competence where no one is sure exactly how it is carried out, but the outcomes are clear. I think it is quite reasonable to talk of competence in those cases, and implicitly allow for variation between individuals in how the outcome is achieved. Of course, if there is no inter-rater reliability in judgement of the outcomes, then I agree it would be futile to try to agree a competence standard.

The Hudson River example was in my mind, too! That’s why I wrote “Where the core is procedural conformance, handling unexpected problems may also feature”. The Hudson River pilot was, it seems to me, showing exceptional competence at handling an unexpected emergency.

We agree that there are limits to the applicability of competence statements, but we could be a little clearer about the boundaries.

]]>
By: Alan Paull http://blogs.cetis.org.uk/asimong/2011/08/31/competence-assessability/#comment-165 Mon, 05 Sep 2011 11:46:27 +0000 http://blogs.cetis.org.uk/asimong/?p=861#comment-165 “it is clear that there are better and worse pop songwriters, and better and worse pilots.”

The latter yes, the former may be an aesthetic judgement.

I think your selection of poetry and mathematics also exhibits this problem. You can make a case for the assessment of both poetry and mathematics in some technical way. However, ‘good poetry’ and even ‘good mathematics’ (which is a creative, not a procedural subject at high academic levels) may be more a matter of personal preference, or, if you like, judgement using experience and a range of probably undefinable human qualities.

We have this in the game design area all the time. I may design a game that hits all the appropriate buttons for a ‘good game’, in other words, it would meet all the competence criteria if we had them, but it may be a rubbish game in the real world.

If you’re dealing with a domain in which “nobody has a bloody clue”, then competence in the classic sense may be irrelevant.

I think that in many job roles there are ‘emergent properties’ that are at the heart of the role and therefore cannot be teased out by a basically reductionist approach. Airline pilot may be one of these. While you need to be able to perform your procedural role in day-to-day circumstances (‘means’), the really vital aspect is not to kill your passengers in a crisis (‘ends’). This means that much of ‘airline piloting’ is playing a part in a system (involving teams, not just individual competences) to reduce the likelihood of killing the passengers. A pilot is therefore assessed on professional judgement, not just how well he or she performs the procedural role. For example, I doubt that the US pilot who landed on the Hudson River was strictly following a set procedure at the time.

I think what I’m saying here is that many job roles are not easily susceptible to traditional competence statements.

]]>