InLOC: the reason for lack of posts

InLOC has been taking up more and more of my non-CETIS time, and along with other pressures, I have not got round to posting here for two whole months. Here are my good intentions…

First, I will try to tweet more, particularly with the hashtag #InLOC, which fortunately no one outside the InLOC projects seems to have used. If readers have anything relevant to say which touches InLOC concerns, please feel free to use that hashtag. I started this just a few days ago, as you can see with a Twitter search, about a newly published European document called “Rethinking education: investing in skills for better socio-economic outcomes“, which among other things takes forward the argument that we need “more focus on ‘learning outcomes’ – the knowledge, skills and competences that students acquire”. This is just the kind the boost we need to take InLOC into mainstream awareness. This is what I tweeted…

Via @philbarker New EC strategy “Rethinking Education” http://europa.eu/rapid/press-release_IP-12-1233_en.htm … < – useful for #InLOC

I am very industriously working at present with the InLOC team to get drafts of our deliverables ready in good time — we are tasked to draft three CWAs: “CEN Workshop Agreements”. As part of this, I will try to work in reading the Rethinking Education strategy, and to refer to any key arguments as part of the InLOC deliverables.

I have also been meaning to continue the “logic of competence” series, now started over two years ago, and bring readers up to date with the latest conclusions of InLOC.

Well, good intentions perhaps, but I hope that the act of writing this down publicly will help galvanise my efforts to produce the goods! I would also be most grateful for any prompts from anyone who follows this part of my work, and who would like to know more.

What is my work?

Is there a good term for my specialist area of work for CETIS? I’ve been trying out “technology for learner support”, but that doesn’t fully seem to fit the bill. If I try to explain, reflecting on 10 years (as of this month) involvement with CETIS, might readers be able to help me?

Back in 2002, CETIS (through the CRA) had a small team working with “LIPSIG”, the CETIS special interest group involved with Learner Information (the “LI” of “LIPSIG”). Except that “learner information” wasn’t a particularly good title. It was also about the technology (soon to be labelled “e-portfolio”) that gathered and managed certain kinds of information related to learners, including their learning, their skills – abilities – competence, their development, and their plans. It was therefore also about PDP — Personal Development Planning — and PDP was known even then by its published definition “a structured and supported process undertaken by an individual to reflect upon their own learning, performance and/or achievement and to plan for their personal, educational and career development”.

There’s that root word, support (appearing as “supported”), and PDP is clearly about an “individual” in the learner role. Portfolio tools were, and still are, thought of as supporting people: in their learning; with the knowledge and skills they may attain, and evidence of these through their performance; their development as people, including their learning and work roles.

If you search the web now for “learner support”, you may get many results about funding — OK, that is financial support. Narrowing the search down to “technology for learner support”, the JISC RSC site mentions enabling “learners to be supported with their own particular learning issues”, and this doesn’t obviously imply support for everyone, but rather for those people with “issues”.

As web search is not much help, let’s take a step back, and try to see this area in a wider perspective. Over my 10 years involvement with CETIS, I have gradually come to see CETIS work as being in three overlapping areas. I see educational (or learning) technology, and related interoperability standards, as being aimed at:

  • institutions, to help them manage teaching, learning, and other processes;
  • providers of learning resources, to help those resources be stored, indexed, and found when appropriate;
  • individual learners;
  • perhaps there should be a branch aimed at employers, but that doesn’t seem to have been salient in CETIS work up to now.

Relatively speaking, there have always seemed to be plenty of resources to back up CETIS work in the first two areas, perhaps because we are dealing with powerful organisations and large amounts of money. But, rather than get involved in those two areas, I have always been drawn to the third — to the learner — and I don’t think it’s difficult to understand why. When I was a teacher for a short while, I was interested not in educational adminstration or writing textbooks, but in helping individuals learn, grow and develop. Similar themes pervade my long term interests in psychology, psychotherapy, counselling; my PhD was about cognitive science; my university teaching was about human-computer interaction — all to do with understanding and supporting individuals, and much of it involving the use of technology.

The question is, what does CETIS do — what can anyone do — for individual learners, either with the technology, or with the interoperability standards that allow ICT systems to work together?

The CETIS starting point may have been about “learner information”, but who benefits from this information? Instead of focusing on learners’ needs, it is all too easy for institutions to understand “learner information” as information than enables institutions to manage and control the learners. Happily though, the group of e-portfolio systems developers frequenting what became the “Portfolio” SIG (including Pebble, CIEPD and others) were keen to emphasise control by learners, and when they came together over the initiative that became Leap2A, nearly six years ago, the focus on supporting learners and learning was clear.

So at least then CETIS had a clear line of work in the area of e-portfolio tools and related interoperability standards. That technology is aimed at supporting personal, and increasingly professional, development. Partly, this can be by supporting learners taking responsibility for tracking the outcomes of their own learning. Several generic skills or competences support their development as people, as well as their roles as professionals or learners. But also, the fact that learners enter information about their own learning and development on the portfolio (or whatever) system means that the information can easily be made available to mentors, peers, or whoever else may want to support them. This means that support from people is easier to arrange, and better informed, thus likely to be more effective. Thus, the technology supports learners and learning indirectly, as well as directly.

That’s one thing that the phrase “technology for learner support” may miss — support for the processes of other people supporting the learner.

Picking up my personal path … building on my involvement in PDP and portfolio technology, it became clear that current representations of information about skills and competence were not as effective as they could be in supporting, for instance, the transition from education to work. So it was, that I found myself involved in the area that is currently the main focus of my work, both for CETIS, and also on my own account, through the InLOC project. This relates to learners rather indirectly: InLOC is enabling the communication and reuse of definitions and descriptions of learning outcomes and competence information, and particularly structures of sets of such definitions — which have up to now escaped an effective and well-adopted standard representation. Providing this will mean that it will be much easier for educators and employers to refer to the same definitions; and that should make a big positive difference to learners being able to prepare themselves effectively for the demands of their chosen work; or perhaps enable them to choose courses that will lead to the kind of work they want. Easier, clearer and more accurate descriptions of abilities surely must support all processes relating to people acquiring and evidencing abilities, and making use of related evidence towards their jobs, their well-being, and maybe the well-being of others.

My most recent interests are evidenced in my last two blog posts — Critical friendship pointer and Follower guidance: concept and rationale — where I have been starting to grapple with yet more complex issues. People benefit from appropriate guidance, but it is unlikely there will ever be the resources to provide this guidance from “experts” to everyone — if that is even what we really wanted.

I see these issues also as part of the broad concern with helping people learn, grow and develop. To provide full support without information technology only looks possible in a society that is stable — where roles are fixed and everyone knows their place, and the place of others they relate to. In such a traditionalist society, anyone and everyone can play their part maintaining the “social order” — but, sadly, such a fixed social order does not allow people to strike out in their own new ways. In any case, that is not our modern (and “modernist”) society.

I’ve just been reading Herman Hesse’s “Journey to the East” — a short, allegorical work. (It has been reproduced online.) Interestingly, it describes symbolically the kind of processes that people might have to go through in the course of their journey to personal enlightenment. The description is in no way realistic. Any “League” such as Hesse described, dedicated to supporting people on their journey, or quest, would practically be able to support only very few at most. Hesse had no personal information technology.

Robert K. Greenleaf was inspired by Hesse’s book to develop his ideas on “Servant Leadership“. His book of that name was put together in 1977, still before the widespread use of personal information techology, and the recognition of its potential. This idea of servant leadership is also very clearly about supporting people on their journey; supporting their development, personally and professionally. What information would be relevant to this?

Providing technology to support peer-to-peer human processes seems a very promising approach to allowing everyone to find their own, unique and personal way. What I wrote about follower guidance is related to this end: to describe ways by which we can offer each other helpful mutual support to guide our personal journeys, in work as well as learning and potentially other areas of life. Is there a short name for this? How can technology support it?

My involvement with Unlike Minds reminds me that there is a more important, wider concept than personal learning, which needs supporting. We should be aspiring even more to support personal well-being. And one way of doing this is through supporting individuals with information relevant to the decisions they make that affect their personal well-being. This can easily be seen to include: what options there are; ideas on how to make decisions; what the consequences of those decision may be. It is an area which has been more than touched on under the heading “Information, Advice and Guidance”.

I mentioned the developmental models of William G Perry and Robert Kegan back in my post earlier this year on academic humility. An understanding of these aspects of personal development is an essential part of what I have come to see as needed. How can we support people’s movement through Perry’s “positions”, or Kegan’s “orders of consciousness”? Recognising where people are in this, developmental, dimension is vital to informing effective support in so many ways.

My professional interest, where I have a very particular contribution, is around the representation of the information connected with all these areas. That’s what we try to deal with for interoperability and standardisation. So what do we have here? A quick attempt at a round-up…

  • Information about people (learners).
  • Information about what they have learned (learning outcomes, knowledge, skill, competence).
  • Information that learners find useful for their learning and development.
  • Information about many subtler aspects of personal development.
  • Information relevant to people’s well-being, including
    • information about possible choices and their likely outcomes
    • information about individual decision-making styles and capabilities
    • and, as this is highly context-dependent, information about contexts as well.
  • Information about other people who could help them
    • information supporting how to find and relate to those people
    • information supporting those relationships and the support processes
    • and in particular, the kind of information that would promote a trusting and trusted relationship — to do with personal values.

I have the strong sense that this all should be related. But the field as a whole doesn’t seem have a name. I am clear that it is not just the same as the other two areas (in my mind at least) of CETIS work:

  • information of direct relevance to institutions
  • information of direct relevance to content providers.

Of course my own area of interest is also relevant to those other players. Personal well-being is vital to the “student experience”, and thus to student retention, as well as to success in learning. That is of great interest to institutions. Knowing about individuals is of great value to those wanting to sell all kinds of services to to them, but particularly services to do with learning and resources supporting learning.

But now I ask people to think: where there is an overlap between information that the learner has an interest in, and information about learners of interest to institutions and content providers, surely the information should be under the control of the individual, not of those organisations?

What is the sum of this information?

Can we name that information and reclaim it?

Again, can people help me name this field, so my area of work can be better understood and recognised?

If you can, you earn 10 years worth of thanks…

Follower guidance: concept and rationale

The idea that I am calling “follower guidance” is about how to relate with chosen others to promote good work, well being, personal growth and development, in an essentially peer-to-peer manner — it’s an alternative to “mentoring”.

Detailing this vision will prepare the ground for thinking about technology to support the relationships and the learning that results from them, which will fill the space left when traditional control hierarchies no longer work well.

The motivation for the idea

Where do people get their direction from? What or who guides someone, and how? How do people find their way, in life, in education, in a work career, etc.? How do people find a way to live a good and worthwhile life, with satisfying, fulfilling work and relationships? All big questions, addressed, as circumstances allow, by others involved in those people’s education, in their personal and professional development, in advice and guidance, coaching and mentoring; as well as by their family and friends.

In my previous post I set out some related challenges. Since then, I was reminded of these kinds of question by a blog post I saw via Venessa Miemis.

To put possible answers in context: in traditionalist societies I would expect people’s life paths to have relatively few options, and the task of orientation and navigation therefore to be relatively straightforward. People know their allotted place in society, and if they are happy with that, fine. But the appropriate place for this attitude is progressively shrinking back into the childhood years, as the world has ever more variety — and ever less certainty — available to adults. Experts often have more options to hand than their own internal decision making can easily process. Perhaps I can illustrate this from my own situation.

Take CETIS, where I currently have a 0.6 FTE contract. It’s a brilliant place to work, within the University of Bolton’s IEC, with so many people who seem somehow to combine expertise and generosity with passion for their own interesting areas or work. It has never felt like a hierarchical workplace, and staff there are expected to be largely self-determining as well as self-motivated. While some CETIS people work closely together, I do so less, because other staff at Bolton are not so interested in the learner-centred side of learning technology and interoperability. Working largely by myself, it is not so easy to decide on priorities for my own effort, and it would be hard for anyone else to give an informed opinion on where I would best devote my time. Happily, the norm is for things to work out, with what I sense as priorities being accepted by others as worthwhile. But what if … ? It’s not the norm in CETIS culture for anyone to be told that they must stop doing what they think is most worthwhile and instead do something less appealing.

Or take Unlike Minds (“UM”), with whom I am currently investigating collaboration, both for myself and for CETIS. UM is a “capability network” — essentially a non-hierarchical grouping of people with fascinatingly rich and diverse backgrounds and approaches, but similarities of situation and motivation. Here, the starting point is that everyone is assumed to be independent and professional (though some, like me, have some employment). It is a challenge to arrange for very busy independent associates to spend significant amounts of their own time “following” the work of other UMs. But if they did so, they might well be able to contribute to filling any orientation deficit of others, as they would in turn be helped if they wanted. I would expect that the more colleagues know about each other’s work, the more they can help focus motivation; the richer will be the collective UM culture; and the more effective UM will become as a capability network.

I mention just these two, because I have personal knowledge, but surely this must apply to so many new-style organisations and networks that shun being governed ultimately by the necessity to maximise profit. Often no one is in a position to direct work from “the top”, either because the management simply don’t have the deep specialist knowledge to work out what people should be doing, or because there is no governance that provides a “top” at all. The risk in all of these cases is of a lack of coordination and coherence. There is also a risk that individuals perform below their potential, because they are not getting enough informed and trusted feedback on their current activities. How many independent workers these days, no matter how supposedly expert, really have the knowledge to ensure even their own optimum decisions? Very few indeed, I guess, if for no other reason that there is too much relevant available knowledge to be on top of it all.

Then there is the danger of over-independent experts falling into the trap of false guru-hood. Without proper feedback, where followers gather largely in admiration, a talented person may have the illusion of being more correct than he or she really is. Conversely, without dedicated and trusted feedback, the highly talented who lack confidence can easily undervalue what they have to offer. The starting point of my previous post was the observation that people are not reliable judges of their own abilities or personality, and the mistakes can be made in either direction.

That is my broad-brush picture of the motivation, the rationale, or the requirement. So how can we address these needs?

The essence of follower guidance

I will refer to the person who is followed, and who receives the guidance, as the “mover”; the other person I will call the “follower guide”. Here are some suggestions about how such a system could work, and they all seem to me to fit together.

  • Follower guidance is not hierarchical. The norm is for everyone to play both roles: mover and guide. Otherwise the numbers don’t add up.
  • Each mover has more than one follower guide. In my own experience, it is much more persuasive to have two or three people tell you something than one alone. The optimal number for a balance between effort and quality (in each situation and for each person) may vary, but I think three might be about right in many cases. The follower guidance idea differs from co-counselling.
  • The mentor role is different. There is a role for someone like a mentor, but in a follower guidance culture they would not be delivering the guidance, but rather trying to arrange the best matching of movers with follower guides.
  • Arrangements are by mutual agreement. It is essential that the mover and follower guide both want to play their roles with each other. Reluctant participants are unlikely to work. Good matches may be helped through mentoring.
  • Follower guides start by following. Central to the idea is that follower guides know the movers well, at least in the area which they are following. Guidance suggestions will then be well-informed and more likely to be well received, growing trust.
  • Follower guides may select areas to follow. The mover needs to spell out the areas of work or life that may be followed; but follower guides cannot be expected to be interested in all of someone’s life and work — nor can a mover be expected to trust people equally in different areas.
  • Follower guides offer questions, suggestions and feedback naturally. Dialogue may be invited through questions or personal suggestions, whenever it seems best. Movers may or may not accept suggestions or address questions; but they are more likely to respond to ideas that come from more than one follower guide.
  • The medium of dialogue needs to be chosen. Positive reinforcement is naturally given openly, e.g. as a comment on a blog post, or a tweet. The media for questions and critical feedback needs to be judged more carefully, to maintain trust. This is one way in which follower guidance may differ from simple following.
  • Follower guides are committed. Movers should be able to rely on their follower guides for feedback and opinion when they need it. That means the follower guides have to stay up to date with the mover’s actions or outputs. This is only likely if they have a genuine interest in the area of the mover’s work they are following. This also will help build trust.
  • Time spent should not be burdensome. If following comes from genuine interest, the time spent should be a natural part of the follower guide’s work. In any case, one can follow quite a lot in, say, half an hour a week. If guidance is natural, spontaneous and gentle, it may be delivered very briefly.
  • Follower guides should not all be older or wiser. This may be appropriate for mentors, but there is value in ideas from all quarters, as recognised in the idea of 360° feedback. Anyway the numbers would not work out.
  • Values fit needs care. Trust will be more easily established the better the values fit. The more secure and confident a mover is, the more they may be able to benefit from feedback from follower guides outside their value set.
  • Trust needs to be built up over time and maintained. Mentoring may help people to trust and to be trustworthy. If trust is nevertheless lost, it is unlikely that a follower guidance relationship would continue.
  • The follower guidance practices should be followed and guided. How could this best be done? Perhaps a question for the cyberneticians?

What do you think about the importance of each one of these points? I’d like to know. And could you imagine practising either side of this kind of relationship? Who with? What would come easily, what would you enjoy, and what would be challenging?

Where does this take us?

This concept is too large to be easily digested at one sitting. I hope I have given enough motivation and outline of the general idea that readers get the sense of what I am trying to get at. I’ve outlined above the way I could see it working, but there is so much more detail to work out. Depending on the response to this post, I will take the ideas forward here or elsewhere.

I do think that this kind of envisioning plays a useful part in the life of CETIS and the IEC. Colleagues are most welcome to criticise the ideas, and link them up to other research. If there already is related practice somewhere, that would be good to know. If people see what I am getting at, they can offer alternative solutions to the challenges addressed. Then, we might think about the kinds of (learning or educational) technology that might support such practices, and the information that might be managed and communicated. We might be able to see links with existing technologies and practices.

In the terms of Robert Kegan, I’m pointing towards a challenge of “modern” life, not, as Kegan focuses more on, in the transition between traditionalist and modern, but rather a challenge inherent in the individualistic nature of current modernism. As Brian said (in Monty Python’s “Life of Brian”) “You’ve all got to work it out for yourselves.” “Don’t let anyone tell you what to do!” This advice can help people grow to a maturity of individualism, but can also hold people back from further growth, through what Kegan calls “deconstructive postmodernism” towards
“reconstructive postmodernism”.

Most significant to me would be the attempt to implement a system such as this that I could participate in myself. This would include my trusted follower guides coming back to me with comments on this post, of course … At the time of writing, thanks to Neil and Alan for commenting on the preceding post, and I very much appreciate those kinds of comment.

Critical friendship pointer

I picked up a tweet yesterday via Paul Chippendale from an HBR blog called “You Are (Probably) Wrong About You” by Heidi Grant Halvorson. This seems to me a useful tying together of several important things: (e-)portfolios, reflection, critical friendship, and how to run P2P organisations. She writes:

Who knows you best? Well, the research suggests that they do — other people’s assessment of your personality predicts your behavior, on average, better than your assessment does.
[…]
In his fascinating book Strangers to Ourselves, psychologist Timothy Wilson summarizes decades of research […] showing us just how much of what we do during every moment of every day […] is happening below our conscious awareness. Some of it we can notice if we engage in a little self-reflection, but much of it we simply cannot — it’s not directly accessible to us at all.

This might remind us first of the perennial problem of e-portfolios and reflection. People tend to reflect only in their own way in their own time, and this is not necessarily helpful for their personal development. It is not easy for practitioners to persuade people to use e-portfolio tools to reflect in a fruitful way. And when it comes to putting together a presentation of one’s abilities and qualities using an e-portfolio tool, the result is therefore not always realistic.

Often what is more effective is a personal one-to-one approach, where the person in the helping role might be called a mentor, a coach, a personal tutor, or something else. But here we run into the problem of the moment: resources. In many related fields, resource is being taken away from personal contact, with learners left to fend for themselves, given only a website to browse.

If only … we could create an effective peer-to-peer mentoring service. This approach has certainly been explored in many places, not least in Bolton, but I do not have personal experience of this, nor do I currently know of authoritative reviews of what is seen as genuinely effective. One might expect pitfalls of schemes under that name to include a formulaic approach; a lack of genuine insight into the “mentee”; and a reliance on older-to-younger mentoring, rather than a more strictly peer-to-peer approach. In Bolton it would appear to be still a minority practice, and the support is clearly given by more advanced to less advanced students. In this kind of setting, what is the chance of a peer mentor helping to correct someone’s misconceptions about their own abilities?

The term “critical friend” seems to me to address some of these potential deficiencies with peer mentors. If the people in question really are friends, if they know each other well and trust each other, surely there is more of a possibility of bringing up and challenging personal misconceptions, given the mutual desire and a supportive culture. The Wikipedia article provides helpful background. There are many other useful sources of ideas about this idea, also known as “critical colleague”, “critical companion” or “learning partner”, all pointing in the same general direction. The idea has taken root, even if it is not yet a well-known commonplace.

The critical friend concept is certainly inspiring, but how many people have colleagues who are both willing and well-positioned to act in this role? In my experience, friends seldom see the range of professional behaviour that one would want constructive critique of, and colleagues seem rather more able to offer positive suggestions in some areas than in others. The challenge seems partly in bringing such practice into the mainstream, where it does not seem odd, or too upsetting to a culture too weak for anything more strenuous than laissez-faire.

What I believe we need is more practiced and reported experimentation along the lines of benefiting from what colleagues are prepared naturally to do, not expecting everyone to have counselling skills, or a sufficient rapport with each other to be the person … well … that we would like them to be! And in any case there are potential problems with small closed groups of people, whether pairs or slightly larger, all commenting on each other’s performance. It could easily lead to a kind of “groupthink”.

My guess is that there is a robust peer-to-peer solution waiting to be more widely acknowledged, tested, and incorporated into work cultures. I have provisionally thought of it as “follower guidance“, but I will save writing more on that to later, and hope that people may comment in the meantime on how would you address the challenges of people mis-assessing their own abilities and qualities. Really, we need to have a culture that promotes good self-knowledge, not only to help personal and professional development, but also to serve as the bedrock of an effective P2P culture.

p.s. I have now written more on the follower guidance idea.

Developing a new approach to competence representation

InLOC is a European project organised to come up with a good way of communicating structures or frameworks of competence, learning outcomes etc. We’ve now produced our interim reports for consultation: the Information Model and the Guidelines. We welcome feedback from everyone, to ensure this becomes genuinely useful and not just another academic exercise.

The reason I’ve not written any blog posts for a few weeks is that so much of my energy has been going into InLOC, and for good reason. It has been a really exciting time working with the team to develop a better approach to representing these things. Many of us have been pushing in this direction for years, without ever quite getting there. Several projects have been nearby, including, last year, InteropAbility (JISC page; project wiki) and eCOTOOL (project web site; my Competence Model page) — I’ve blogged about these before, and we have built on ideas from both of them, as well as from several other sources: you may be surprised at the range and variety of “stakeholders” in this area that we have assembled within InLOC. Doing the thinking for the Logic of Competence series was of course useful background, but nor did it quite get there.

What I want to announce now is that we are looking for the widest possible feedback as further input to the project. It’s all too easy for people like us, familiar with interoperability specifications, simply to cook up a new one. It is far more of a challenge, as well as hugely more worthwhile and satisfying, to create something genuinely useful, which people will actually use. We have been looking at other groups’ work for several months now, and discussing the rich, varied, and sometimes confusing ideas going around the community. Now we have made our own initial synthesis, and handed in the “interim” draft agreements, it is an excellent time to carry forward the wide and deep consultation process. We want to discuss with people whether our InLOC format will work for them; whether they can adopt, use or recommend it (or whatever their role is to do with specifications; or, what improvements need to be made so that they are most likely to take it on for real.

By the end of November we are planning to have completed this intense consultation, and we hope to end up with the desired genuinely useful results.

There are several features of this model which may be innovative (or seem so until someone points out somewhere they have been done before!)

  1. Relationships aren’t just direct as in RDF — there is a separate class to contain the relationship information. This allows extra information, including a number, vital for defining levels.
  2. We distinguish the normal simple properties, with literal objects, which are treated as integral parts of whatever it is (including: identifier, title, description, dates, etc.) from what could be called “compound properties”. Compound properties, that have more than one part to their range, are a little like relationships, and we give them a special property class, allowing labels, and a number (like in relationships).
  3. We have arranged for the logical structure, including the relationships and compound properties, to be largely independent of the representation structure. This allows several variant approaches to structuring, including tree structures, flat structures, or Atom-like structures.

The outcome is something that is slightly reminiscent both of Atom itself, and of Topic Maps. Both are not so like RDF, which uses the simplest possible building blocks, but resulting in the need for harder-to-grasp constructs like blank nodes. The fact of being hard to grasp leads to people trying different ways of doing things, and possibly losing interoperability on the way. Both Atom and Topic Maps, in contrast, add a little more general purpose structure, which does make quite a lot of intuitive sense in both cases, and they have been used widely, apparently with little troublesome divergence.

Are we therefore, in InLOC, trying to feel our way towards a general-purpose way of representing substantial hierarchical structures of independently existing units, in a way that makes more intuitive sense that elementary approaches to representing hierarchies? General taxonomies are simply trying to represent the relationships between concepts, whereas in InLOC we are dealing with a field where, for many years, people have recognised that the structure is an important entity in its own right — so much so that it has seemed hard to treat the components of existing structures (or “frameworks”) as independent and reusable.

So, see what you think, and please tell me, or one of the team, what you do honestly think. And let’s discuss it. The relevant links are also available straight from the InLOC wiki home page. And if you are responsible for creating or maintaining structures of intended learning outcomes, skills, competences, competencies, etc., then you are more than welcome to try out our new approach, that we hope combines ease of understanding with the power to express just what you want to express in your “framework”, and that you will be persuaded to use it “for real”, perhaps when we have made the improvements that you need.

We envisage a future when many ICT tools can use the same structures of learning outcomes and competences, saving effort, opening up interoperability, and greatly increasing the possibilities for services to build on top of each other. But you probably don’t need reminding of the value of those goals. We’re just trying to help along the way.

Reviewing the future for Leap2

JISC commissioned a Leap2A review report (PDF), carried out early in 2012, that has now been published. It is available along with other relevant materials from the e-Portfolio interoperability JISC page. For anyone following the fortunes of Leap2A, it is highly worthwhile reading. Naturally, not all possible questions were answered (or asked), and I’d like to take up some of these, with implications for the future direction of Leap2 more generally.

The summary recommendations were as follows — these are very welcome!

  1. JISC should continue to engage with vendors in HE who have not yet implemented Leap2A.
  2. Engagement should focus on communities of practice that are using or are likely to use e-portfolios, and situations where e-portfolio data transfer is likely to have a strong business case.
  3. JISC should continue to support small-scale tightly focused developments that are likely to show immediate impact.
  4. JISC should consider the production of case studies from PebblePad and Mahara that demonstrate the business case in favour of Leap2A.
  5. JISC should consider the best way of encouraging system vendors to provide seamless import services.
  6. JISC should consider constructing a standardisation roadmap via an appropriate BSI or CEN route.

That tallies reasonably with the outcome of the meeting back in November last year, where we reckoned that Leap2A needs: more adoption; more evidence of utility; to be taken more into the professional world; good governance; more examples; and for the practitioner community to build around it models of lifelong development that will justify its existence.

Working backwards up the list for the Leap2A review report, recommendation 6 is one for the long term. It could perhaps be read in the context of the newly formed CETIS position on the recent Government Open Standards Consultation. There we note:

Established public standards bodies (such as ISO, BSI and CEN), while doing valuable work, have some aspects that would benefit from modernisation to bring them more into line with organisations such as W3C and OASIS.

The point then elaborated is that the community really needs open standards that are freely available as well as royalty-free and unencumbered. The de jure standards bodies normally still charge for copies of their standards, as part of their business model, which we see as outdated. If we can circumvent that issue, then BSI and CEN would become more attractive options.

It is the previous recommendation, number 5 in the list above, that I will focus on more, though. Here is the fuller version of that recommendation (appearing as paragraph 81).

One of the challenges identified in this review is to increase the usability of data exchange with the Leap2A specification, by removing the current necessity for separate export and import. This report RECOMMENDS that JISC considers the best way of encouraging system vendors to provide seamless data exchange services between their products, perhaps based on converging practice in the use of interoperability and discovery technologies (for example future use of RDF). It is recognised that this type of data exchange may require co-ordinated agreement on interoperability approaches across HEIs, FECs and vendors, so that e-portfolio data can be made available through web services, stressing ease of access to the learner community. In an era of increasing quantities of open and linked data, this recommendation seems timely. The current initiatives around courses information — XCRI-CAP, Key Information Sets (KIS) and HEAR — may suggest some suitable technical approaches, even though a large scale and expensive initiative is not recommended in the current financially constrained circumstances.

As an ideal, that makes perfect sense from the point of view of an institution transferring a learner’s portfolio information to another institution. However, seamless transfer is inherently limited by the compatibility (or lack of it) between the information stored in each system. There is also a different scenario, that has always been in people’s minds when working on Leap2A. It is that learners themselves may want to be able to download their own information, to keep for use, at an uncertain time in the future, in various ways that are not necessarily predictable by the institutions that have been hosting their information. In any case, the predominant culture in the e-portfolio community is that all the information should be learner-ownable, if not actually learner-owned. This is reflected in the report’s paragraph 22, dealing with current usage from PebblePad.

The implication of the Leap2A functionality is that data transfer is a process of several steps under the learner’s control, so the learner has to be well-motivated to carry it out. In addition Leap2A is one of several different import/export possibilities, and it may be less well understood than other options. It should perhaps be stressed here that PebblePad supports extensive data transfer methods other than Leap2A, including zip archives, native PebblePad transfers of whole or partial data between accounts, and similarly full or partial export to HTML.

This is followed up in the report’s paragraph 36, part of the “Challenges and Issues” section.

There also appears to be a gap in promoting the usefulness of data transfer specifically to students. For example in the Mahara and PebblePad e-portfolios there is an option to export to a Leap2A zip file or to a website/HTML, without any explanation of what Leap2A is or why it might be valuable to export to that format. With a recognisable HTML format as the other option, it is reasonable to assume that students will pick the format that they understand. Similarly it was suggested that students are most likely to export into the default format, which in more than one case is not the Leap2A specification.

The obvious way to create a simpler interface for learners is to have just one format for export. What could that format be? It should be noted first that separate files that are attached to or included with a portfolio will always remain separate. The issue is the format of the core data, which in normal Leap2A exports is represented by a file named “leap2a.xml”.

  1. It could be plain HTML, but in this case the case for Leap2A would be lost, as there is no easy way for plain HTML to be imported into another portfolio system without a complex and time-consuming process of choosing where each single piece of information should be put in the new system.
  2. It could be Leap2A as it is, but the question then would be, would this satisfy users’ needs? Users’ own requirements for the use of exports is not spelled out in the report, and it does not appear to have been systematically investigated anywhere, but it would be reasonable to expect that one use case would be that users want to display the information so that it can be cut and pasted elsewhere. Leap2A supports the display of media files within text, and formatting of text, only through the inclusion of XHTML within the content of entries, in just the same way as Atom does. It is not unreasonable to conclude that limiting exports to plain Leap2A would not fully serve user export needs, and therefore it is and will continue to be unreasonable to expect portfolio systems to limit users to Leap2A export only.
  3. If there were a format that fully met the requirements both for ease of viewing and cut-and-paste, and for relatively easy and straightforward importing to another portfolio system (comparable to Leap2A currently), it might then be reasonable to expect portfolio systems to have this as their only export format. Then, users would not have to choose, would not be confused, and the files which they could view easily and fully through a browser on their own computer system would also be able to be imported to another portfolio system to save the same time and effort that is currently saved through the use of Leap2A.

So, on to the question, what could that format be? What follows explains just what the options are for this, and how it would work.

The idea for microformats apparently originated in 2000. The first sentence of the Wikipedia article summarises nicely:

A microformat (sometimes abbreviated µF) is a web-based approach to semantic markup which seeks to re-use existing HTML/XHTML tags to convey metadata and other attributes in web pages and other contexts that support (X)HTML, such as RSS. This approach allows software to process information intended for end-users (such as contact information, geographic coordinates, calendar events, and the like) automatically.

In 2004, a more sophisticated approach to similar ends was proposed in RDFa. Wikipedia has “RDFa (or Resource Description Framework –in– attributes) is a W3C Recommendation that adds a set of attribute-level extensions to XHTML for embedding rich metadata within Web documents.”

In 2009 the WHATWG were developing Microdata towards its current form. The Microformats community sees Microdata as having grown out of Microformats ideas. Wikipedia writes “Microdata is a WHATWG HTML specification used to nest semantics within existing content on web pages. Search engines, web crawlers, and browsers can extract and process Microdata from a web page and use it to provide a richer browsing experience for users.”

Wikipedia quotes the Schema.org originators (launched on 2 June 2011 by Bing, Google and Yahoo!) as stating that it was launched to “create and support a common set of schemas for structured data markup on web pages”. It provides a hierarchical vocabulary, in some cases drawing on Microformats work, that can be used within the RDFa as well as Microdata formats.

Is it possible to represent Leap2A information in this kind of way? Initial exploratory work on Leap2R has suggested that it is indeed possible to identify a set of classes and properties that could be used more or less as they are with RDFa, or could be correlated with the schema.org hierarchy for use with Microdata. However, the solution needs detail adding and working through.

In principle, using RDFa or Microdata, any portfolio information could be output as HTML, with the extra information currently represented by Leap2A added into the HTML attributes, which is not directly displayed, and so does not interfere with human reading of the HTML. Thus, this kind of representation could fully serve all the purposes currently served by HTML export of Leap2A. It seems highly likely that practical ways of doing this can be devised that can convey the complete structure currently given by Leap2A. The requirements currently satisfied by Leap2A would be satisfied by this new format, which might perhaps be called “Leap2H5″, for Leap2 information in HTML5, or maybe alternatively “Leap2XR”, for Leap2 information in XHTML+RDFa (in place of Leap2A, meaning Leap2 information in Atom).

Thus, in principle it appears perfectly possible to have a single format that simultaneously does the job both of HTML and Leap2A, and so could serve as a plausible principal export and import format, removing that key obstacle identified in paragraph 36 of the Leap2A review report. The practical details may be worked out in due course.

There is another clear motivation in using schema.org metadata to mark up portfolio information. If a web page uses schema.org semantics, whether publicly displayed on a portfolio system or on a user’s own site, Google and others state that the major search engines will create rich snippets to appear under the search result, explaining the content of the page. This means, potentially, that portfolio presentations would be more easily recognised by, for instance, employers looking for potential employees. In time, it might also mean that the search process itself was made more accurate. If portfolio systems were to adopt export and import using schema.org in HTML, it could also be used for all display of portfolio information through their systems. This would open the way to effective export of small amounts of portfolio information simply by saving a web page displayed through normal e-portfolio system operation; and could also serve as an even more effective and straightforward method for transferring small amounts of portfolio information between systems.

Having recently floated this idea of agreeing Leap2 semantics in schema.org with European collaborators, it looks like gaining substantial support. This opens up yet another very promising possibility: existing European portfolio related formats could be harmonised through this new format, that is not biased towards any of the existing ones — as well as Leap2A, there is the Dutch NTA 2035 (derived from IMS ePortfolio), and also the Europass CV format. (There is more about this strand of unfunded work through MELOI.) All of these are currently expressed using XML, but none have yet grasped the potential of schema.org in HTML through microdata or RDFa. To restate the main point here, this means having the semantics of portfolio information embedded in machine-processable ways, without interfering with the human-readable HTML.

I don’t want to be over-optimistic, as currently money tends only to go towards initiatives with a clear business case, but I am hopeful that in the medium term, people will recognise that this is an exciting and powerful potential development. When any development of Leap2 gets funded, I’m suggesting that this is what to go for, and if anyone has spare resource to work on Leap2 in the meanwhile, this is what I recommend.

The logic of tourism as an analogy for competence

(20th in my logic of competence series.)

Modelling competence is too far removed from common experience to be intuitive. So I’ve been thinking of what analogy might help. How about the analogy of tourism? This may help particularly with understanding the duality between competence frameworks (like tourist itineraries) and competence concept definitions (like tourist destinations).

The analogy is helped by the fact that last week I was in Lisbon for the first time, at work (the CEN WS-LT and TC 353), but also more relevantly as a tourist. (If you don’t know Lisbon, think of examples to suit your own chosen place to visit, that you know better.) I’ll start with the aspects of the analogy that seem to be most straightforward, and go on to more subtle features.

First things first, then: a tourist itinerary includes a list of destinations. This can be formalised as a guided tour, or left informal as a “things you should see” list given by a friend who has been there. A destination can be in any number of itineraries, or none. An itinerary has to include some destinations, but in principle it doesn’t have any upper limits: it could be a very detailed itinerary that takes a year to properly acquaint a newcomer with the ins and outs of the city. Different itineraries for the same place may have more, or fewer, destinations within that place. They may or may not agree on the destinations included. If there were destinations included by the large majority of guides, another guide could select these as the “essential” Lisbon or wherever. In this case, perhaps that would include visiting the Belem tower; the Castle of St George; Sintra; experiencing Fado; sampling the local food, particularly fish dishes; and a ride on one of the funicular trams that climb the steep hills. Or maybe not, in each case. There again, you could debate whether Sintra should be included in a guide to Lisbon, or just mentioned as a day trip.

A small itinerary could be made for a single destination, if desired. Some guides may just point you to a museum or destination as a whole; others may give detailed suggestions for what you should see within that destination. A cursory guide might say that you should visit Sintra; a detailed one might say that you really must visit the Castle of the Moors in Sintra, as well as other particular places in Sintra. A very detailed guide might direct you to particular things to see in the Castle of the Moors itself.

It should be clear from the above discussion that a place to visit should not be confused with an itinerary for that place. Any real place has an unlimited number of possible itineraries for it. An itinerary for a city may include a museum; an itinerary for a museum may include a painting; there may sometimes even be guides to a painting that direct the viewer to particular features of that painting. The guide to the painting is not the painting; the guide to the museum is not the museum; the guide to the city is not the city.

There might also be guides that do not propose particular itineraries, but list many places you might go, and you select yourself. In these cases, some kind of categorisation might be used to help you select the places of interest to you. What period of history do they come from? Are they busy or quiet? What do they cost? How long do they take to visit? Or a guide with itineraries may also categorise attractions, and make them explicitly optional. Optionality might be particularly helpful in guided tours, so that people can leave out things of less interest.

If a set of guides covered several whole places, not just one, it may make comparisons across the different places. If you liked the Cathar castles in the South of France, you may like the Castle of the Moors in Sintra. Those who like stately homes, on the other hand, may be given other suggestions.

A guide to a destination may also contain more than an itinerary of included destinations within it. A guidebook may give historical or cultural background information, which goes beyond the description of the destinations. Guides may also propose a visit sequence, which is not inherent in the destinations.

The features I have described above are reasonably replicated in discussion of competence. A guide or itinerary corresponds to a competence framework; a destination corresponds to a competence concept. This is largely intended to throw further light on what I discussed in number 12 in this series, Representing the interplay between competence definitions and structures.

Differences

One difference is that tourist destinations have independent existence in the physical world, whereas competence concepts do not. It may therefore be easier to understand what is being referred to in a guide book, from a short description, than in a competence framework. Both guide book and competence framework may rely on context. When a guide book says “the entrance”, you know it means the entrance to the location you are reading about, or may be visiting.

Physical embodiment brings clarity and constraints. Smaller places may be located within larger places, and this is relatively clear. But it is less clear whether lesser competence concepts are part of greater competence concepts. What one can say (and this carries through from the tourism analogy) is that concepts are included in frameworks (or not), and that any concept may be detailed by (any number of) frameworks.

Competence frameworks and concepts are more dependent on the words used in description, and because a description necessarily chooses particular words, it is easy to confuse the concept with the framework if they use the same words. It is easy to use the words of a descriptive framework to describe a concept. It is not so common, though perfectly possible, to use the description of an itinerary as a description of a place. It is because of this greater dependence on words (compared with tourist guides) that it may be more necessary to clarify the context of a competence concept definition, in order to understand what it actually means.

Where the analogy with competence breaks down more seriously is that high stakes decisions rarely depend on exactly where someone has visited. But at a stretch of the imagination, they could: recruitment for a relief tour guide could depend on having visited all of a given set of destinations, and being able to answer questions about them. What high stakes promotes is the sense that a particular structure (as defined or adopted by the body controlling the high-stakes decisions) defines a particular competence concept. Despite that, I assert that the competence structure and the separate competence concept remain strictly separate kinds of thing.

Understanding the logic of competence through this analogy

The features of competence models that are illustrated here are these.

  • Competence frameworks or structures may include relevant competence concepts, as well as other material. (See &numero; 12.)
  • Competence concept definitions may be detailed by a framework structure for that competence concept. Nevertheless the structure does not fully define the concept. (See &numero; 12 and &numero; 13.)
  • Competence frameworks may include optional competences (as well as necessary or mandatory ones). (See &numero; 15 and &numero; 7.)
  • Both frameworks and concepts may be categorised. (See also &numero; 5.)
  • Frameworks may contain sub-frameworks (just as itineraries may contain sub-itineraries).
  • But frameworks don’t contain concepts in the same way: they just include them (or not).
  • A framework may be simply an unstructured list of defined concepts.

I hope that helps anyone to understand more of the logic of competence, and I hope that also helps InLOC colleagues come to consensus on the related matters.

More and less specificity in competence definitions

(19th in my logic of competence series.)

Descriptions of personal ability can serve either as claims, like “This is what I am good at …”, or as answers to questions like “What are you good at?” or “can you … ?” In conversations — whether informally, or formally as in a job interview — the claims, questions, and answers may be more or less specific. That is a necessary and natural feature of communication. It is the implications of this that I want to explore here, as they bear on my current work, in particular including the InLOC project.

This is a new theme in my logic of competence series. Since the previous post in that series, I had to focus on completing the eCOTOOL competence model and managing the initial phases of InLOC, which left little time for following up earlier thinking. But there were ideas clearly evident in my last post in this series (representing level relationships) and now is the time for followup and development. The terms introduced previously there can be linked to this new idea of specificity. Simply: binarily assessable concepts are ones that are defined specifically enough for a yes/no judgement about a person’s ability; rankably assessable concepts have an intermediate degree of specificity, and are complemented by level definitions; while unorderly assessable concepts are ones that are less specifically defined, requiring more specificity to be properly assessable. (See that previous post for explanation of those terms.) The least specific competence-related concepts are not properly assessable at all, but serve as tags or headings.

As well as giving weight and depth to this idea of specificity in competence definitions, in this post I want to explore the connection between competence definitions and answering questions, because I think this will help to explain the ideas, because it is relatively straightforward to understand that questions and answers can be more or less specific.

Since the previous post in the series, my terminology has shifted slightly. The goals of InLOC — Integrating Learning Outcomes and Competences — have made it plain that we need to deal equally with learning outcomes and with competence or ability concepts. So I include “learning outcomes” more liberally, always meaning intended learning outcomes.

Job interviews

Imagine you are interviewing someone for a job. To make it more interesting, let’s make it an informal one: perhaps a mutual business contact has introduced you to a promising person at a business event. Add a little pressure by imagining that you have just a few minutes to make up your mind whether you want to ask this person to go through a longer, formal process. How would you structure the interview, and what questions would you ask?

As I envisage the process, one would probably start off with quite general, less specific questions, and then go into more detail where appropriate, where it mattered. So, for instance, one might ask “are you a programmer?”, and if the answer was yes, go into more detail about languages, development environments, length of experience, type of experience, etc. etc. The useful detail in this case would depend entirely on the circumstances of the job. For a graduate to be recruited into a large company, what matters might be aptitude, as it would be likely that full training would be supplied (which you could perhaps see as a kind of technical “enculturation”). On the other hand, for a specialist to join a short-term high-stakes project, even small details might matter a lot, as learning time would probably be minimal.

In reality, most job interviews start, not from a blank sheet, but from the basis of a job advert, and an application form, or CV and covering letter. A job advert may specify requirements; an application form may contain specific questions for which answers are expected, but in the absence of an appliation form, a CV and covering letter needs to try to answer, concisely, some of the key questions that would be asked first in an informal, unprepared job interview. This naturally explains the universal advice that CVs should be designed specifically for each job application. What you say about yourself unprompted not only reveals that information itself, but also says much about what you expect the other person to reckon as significant or interesting.

So, in the job interview, we notice the natural importance of varying specificity in descriptions and questions about abilities and experience.

Recruitment

This then carries over to the wider recruitment process. Potential employers often formulate a list of what is required of prospective employees, in terms of which abilities and experience are essential or desirable, but the detail and specificity of each item will naturally vary. The evidence for a less specific requirement may be assessed at interview with some quick general questions, but a more exacting requirement may want harder evidence such as a qualification, certificate or testimonial from an expert witness.

For example, in a regulated world such as pesticides that I wrote about recently, an employer might well want a prospective employee to have obtained a relevant certificate or qualification, so that they can legally do their job. Even when a certificate is not a legal requirement, some are widely asked for. A prospective sales employee with a driving licence or an office employee with an ECDL might be preferred over one without, and it would be perfectly reasonable for an employer to insist that non-native speakers had obtained a given certified level of proficiency in the principle workplace language. In each case, because the certificate is awarded only to people who have passed a carefully controlled test, the test result serves to answer many quite specific questions about the holder’s abilities, as well as the potential legal fact of their being allowed to perform certain actions in regulated occupations.

Vocational qualifications often detail quite specifically what holders are able to do. This is clearly the intention of the Europass Certificate Supplement (ECS), and has been in the UK, through the system of National Vocational Qualifications, relying on National Occupational Standards. So we could expect that employers with specific learning outcome or competence requirements may specify that candidates should have particular vocational qualifications; but what about less specific requirements? My guess is that those employers who have little regard for vocational qualifications are just those whose requirements are less specific. Time was when many employers looked only for a “good degree”, which in the UK often meant a “2:1″, an upper second class. This was supposed to answer generic questions, as typically the specific subject of the degree was not specified. Now there is a growing emphasis on the detail of the degree transcript or Europass Diploma Supplement (EDS), from which a prospective employer can read at least assessment results, if not yet explicit details of learning outcomes or competences. There is also a increasing trend towards making explicit the intended learning outcomes of courses at all levels, so the course information might be more informative than the transcript of EDS.

Interestingly, the CVs of many technical workers contain highly unspecific lists of programming languages that the individual implicitly claims, stating nothing about the detailed abilities and experience. These lists answer only the most general questions, and serve effectively only to open a conversation about what the person’s actual experience and achievements have been in those programming languages. At least for human languages there is the increasingly used CEFR; there does not appear to be any such widely recognised framework for programming languages. Perhaps, in the case of programming languages, it would be clumsy and ineffective to give answers to more detailed questions, because the individual does not know what those detailed questions would be.

Specificity in frameworks

Frameworks seem to gravitate towards specificity. Given that some people want to know the answers to specific questions, this is quite reasonable; but where does that leave the expression of the less specific requirements? For examples of curriculum frameworks, there is probably nowhere better than the American Achievement Standards Network (ASN). Here, as in many other places, learning outcomes are defined only in one or two levels. The ASN transcribes documents faithfully, then among many other things marks the “indexing status” of the various components. For an arbitrary example, see Earth and Space Science, which is a topic heading and not “indexable”. The heading below just states what the topic is about, and is not “indexable”. It is below this that the content becomes “indexable”, with first some less specific statements about what should be achieved by the end of fourth grade, broken down into the smallest components such as Identify characteristics of soils, minerals, rocks, water, and the atmosphere. It looks like it is just the “indexable” resources that are intended to represent intended learning outcome definitions.

At fourth grade, this is clearly nothing to do with employment, but even so, identifying characteristics of soils etc. is something that students may or may not be able to do, and this is part of the less specifically defined (but still “indexable”) “understanding of the characteristics of earth materials”. It strikes me that the item about identifying characteristics would fit reasonably (in my scheme of the previous post) as a “rankably assessible” concept, and its parent item about understanding might be classified (in my scheme) as unorderly assessable.

How to represent varying specificity

Having pointed out some of the practical examples of varying specificity in definitions of learning outcome or competence, the important issue for work such as InLOC is to provide some way of representing, not only different levels of specificity, but also how they relate to one another.

An approach through considering questions and answers

Any concept that is related to learning outcomes or competence can provide the basis for questions of an individual. Some of these questions have yes/no answers; some invite answers on a scale; some invite a longer, less straightforward reply, or a short reply that invites further questions. A stated concept can be both the answer to a question, and the ground for further questions. So, to go back to some of the above examples, a CV might somewhere state “French” or “Java”. These might be answers to the questions “what languages have you studied?” or “what languages do you use?” They also invite further questions, such as “how well do you know …?”, or “how much have you used …, and in what contexts?”, or “how good are you at …?” – which, if there is an appropriate scale, could be reformulated as “what level is your ability in …?”

Questions could be found corresponding to the ASN examples as well. “Identify characteristics of soils, minerals, rocks, water, and the atmosphere” has the same format that allows “can you …?” or “I can …”. The less specific statement — “By the end of fourth grade, students will develop an understanding of the characteristics of earth materials,” — looks like it corresponds with questions more like “what do you understand about earth materials?”.

As well as “summative” questions, there are related questions that are used in other ways than assessment. “How confident are you of your ability in …?” and “is your ability in … adequate in your current situation?” both come to mind (stimulated by considerations in LUSID).

What I am suggesting here is that we can adapt some of the natural properties of questions and answers to fit definitions of competence and ability. So what properties do I have in mind? Here is a provisional and tentative list.

  • Questions can be classified as inviting one of four kinds of answer:
    1. yes or no;
    2. a value on a (predefined) scale;
    3. examples;
    4. an explanation that is more complex than a simple value.
  • These types of answer probably need little explanation – many examples can readily be imagined.
  • The same form of answer can relate to more than one question, but usually the answer will mean different things. To be fully and clearly understood, an answer should relate to just one question. Using the above example, “French” as the answer to “what languages have you studied?” means something substantially different from “French” as the answer to “what languages are you fluent in?”
  • A more specific question may imply answers to less specific questions. For example, “what programming languages have you used in software development?” implies answers such as “software development” to the question “what competences do you have in ICT?” Many such implied questions and answers can be formulated. What matters in a particular framework is the other answers in that particular framework that can be inferred.
  • An answer to a less specific question may invite further more specific questions.
    1. Conversely to the example just above, if the question “what competences do you have in ICT?” includes the answer “software development”, a good follow-up question might be “what programming languages have you used in software development?” Similar patterns could be seen for any technical specialty. Often, answers like this may be taken from a known list of options. There are only so many languages, both human and computer.
    2. Where an answer is a rankable concept, questions about the level of that ability are invited. For instance, the question “what foreign languages can you speak?”, answered with “French” and “Italian”, invites questions such as “what is your European Language Passport level of ability in spoken interaction in French?”
    3. Where an answer has been analysed into its component parts, questions about each component part make sense. For example, if the answer to “are you able to clear sites for tree planting?”, following the LANTRA Treework NOS (2009) was “yes”, that invites the narrower implied questions set out in that NOS, like “can you select appropriate clearance methods …?” or “do you understand the potential impacts of your work on the environment …?”
    4. Unless the question is fully specific, admitting only the answers yes and no, and even in that case many times, it is nearly always possible to ask further questions, and give further answers. But everyone’s interest in detail stops sooner or later. The place to stop asking more specific questions is when the answer does not significantly affect the outcome you are looking for. And that varies between different interested parties.
  • Questions may be equivalent to other questions in other frameworks. This will come out from the answers given. If the answers given by the same person in the same context are always the same for two questions, they are effectively equivalent. It is genuinely helpful to know this, as it means that one can save time not repeating questions.
  • Answers to some questions may imply answers to other questions in different frameworks, without being equivalent. The answers may contain, or be contained by, their counterparts. This is another way of linking together questions from different frameworks, and saving asking unnecessary extra questions.

That covers a view of how to represent varying specificity in questions and answers, but not yet frameworks as they are at present.

Back to frameworks as they are at present

At present, it is not common practice to set out frameworks of competence or ability in terms of questions and answers, but only in terms of the concepts themselves. But, to me, it helps understanding enormously to imagine the frameworks as frameworks of questions, and the learning outcome or competence concepts as potential answers. In practice, all you see in the frameworks is the answers to the implied questions.

Perhaps this has come about through a natural process of doing away with unnecessary detail. The overall question in occupational competence frameworks is, “are you competent to do this job?”, so it can go unstated, with the title of the job standing in for the question. The rest of the questions in the framework are just the detailed questions about the component parts of that competence (see Carroll and Boutall’s ideas of Functional Analysis in their Guide to Developing National Occupational Standards). The formulation with action verbs helps greatly in this approach. To take NOS examples from way back in the 3rd post in this series, the units themselves and the individual performance criteria share a similar structure. Less specifically, “set out and establish crops” relates both to the question “are you able to set out and establish crops” and the competence claim “I am able to set out and establish crops”. More specifically, “place equipment and materials in the correct location ready for use” can be prefixed with “are you able to …” for a question, or “I am able to …” as a claim. Where all the questions take a form that invites answers yes or no, one really does not need to represent the questions at all.

With a less uniform structure, one would need mentally to remove all the questions to get a recognisable framework; or conversely, to understand a framework in terms of questions, one needs to add in those implied questions. This is not as easy, and perhaps that is why I have been drawn to elaborating all those structuring relationships between concepts.

We are left in a place that is very close to where we were before in the previous post. At simplest, we have the individual learning outcome or competence definitions (which are the answers) and the frameworks, which show how the answers connect up, without explicitly mentioning the questions themselves. The relations between the concepts can be factored out, and presented either together in the framework, or separately together with the concepts that are related by those relations.

If the relationships are simply “broader” and “narrower”, things are pretty straightforward. But if we admit less specific concepts and questions, because the questions are not explicitly represented, the structure needs a more elaborate set of relationships. In particular, we have to make particular provision for rankable concepts and levels. I’ll leave detailing the structures we are left with for later.

Before that, I’d like to help towards better grasp of the ideas through the analogy with tourism.

Competence and regulation

Today I had a most helpful phone call with a kind lady from the Health and Safety Executive (HSE), and it has illuminated the area of the competence world, related to regulation, that I was very unclear about, so I thought I would try to share my increased understanding.

The EU often comes up with directives intended for the good of European citizens in general. In this case, as an example we are looking at Directive 2009/128/EC of 2009-10-21 “establishing a framework for Community action to achieve the sustainable use of pesticides”. Good that this one looks uncontroversial in principle – we don’t want people to use pesticides in an unregulated way, potentially polluting common air, water or ground (potentially without our even being aware of it), so I guess most people would support the principle of regulation here.

If you work your way down to Article 5 of this directive, you see:

Article 5
Training
1. Member States shall ensure that all professional users, distributors and advisors have access to appropriate training by bodies designated by the competent authorities. This shall consist of both initial and additional training to acquire and update knowledge as appropriate.

The training shall be designed to ensure that such users, distributors and advisors acquire sufficient knowledge regarding the subjects listed in Annex I, taking account of their different roles and responsibilities.

2. By 14 December 2013, Member States shall establish certification systems and designate the competent authorities responsible for their implementation. These certificates shall, as a minimum, provide evidence of sufficient knowledge of the subjects listed in Annex I acquired by professional users, distributors and advisors either by undergoing training or by other means.

(I will say nothing at all about what “competent” means as in “competent authority”. Maybe it is quite different.)

It goes on. So what is this Annex I? That is really significant for the purposes of knowledge, skill and competence. It’s worth perhaps repeating this in full, just to get the full flavour of one example of the language and way these things are set out.

Training subjects referred to in Article 5

  1. All relevant legislation regarding pesticides and their use.
  2. The existence and risks of illegal (counterfeit) plant protection products, and the methods to identify such products.
  3. The hazards and risks associated with pesticides, and how to identify and control them, in particular:
    1. risks to humans (operators, residents, bystanders, people entering treated areas and those handling or eating treated items) and how factors such as smoking exacerbate these risks;
    2. symptoms of pesticide poisoning and first aid measures;
    3. risks to non-target plants, beneficial insects, wildlife, biodiversity and the environment in general.
  4. Notions on integrated pest management strategies and techniques, integrated crop management strategies and tech­niques, organic farming principles, biological pest control methods, information on the general principles and crop or sector-specific guidelines for integrated pest management.
  5. Initiation to comparative assessment at user level to help professional users make the most appropriate choices on pesticides with the least side effects on human health, non-target organisms and the environment among all auth­orised products for a given pest problem, in a given situation.
  6. Measures to minimise risks to humans, non-target organisms and the environment: safe working practices for storing, handling and mixing pesticides, and disposing of empty packaging, other contaminated materials and surplus pesticides (including tank mixes), whether in concentrate or dilute form; recommended way to control operator exposure (personal protection equipment).
  7. Risk-based approaches which take into account the local water extraction variables such as climate, soil and crop types, and relieves.
  8. Procedures for preparing pesticide application equipment for work, including its calibration, and for its operation with minimum risks to the user, other humans, non-target animal and plant species, biodiversity and the environment, including water resources.
  9. Use of pesticide application equipment and its maintenance, and specific spraying techniques (e.g. low-volume spraying and low-drift nozzles), as well as the objectives of the technical check of sprayers in use and ways to improve spray quality. Specific risks linked to use of handheld pesticide application equipment or knapsack sprayers and the relevant risk management measures.
  10. Emergency action to protect human health, the environment including water resources in case of accidental spillage and contamination and extreme weather events that would result in pesticide leaching risks.
  11. Special care in protection areas established under Articles 6 and 7 of Directive 2000/60/EC.
  12. Health monitoring and access facilities to report on any incidents or suspected incidents.
  13. Record keeping of any use of pesticides, in accordance with the relevant legislation.

What we have here is a kind of syllabus, but in some ways quite a vague syllabus. It does not make it expressly clear what people have to be able to do as a result of training with this syllabus, as is now good practice for many learning outcomes, particularly vocational ones. So it falls to the Member States to interpret that, with the result that different Member States may do things differently, the resultant competences may not be the same, and there may well be considerable differences across Europe in how safe people actually are from the dangers the regulation was brought in to counter.

So the European directive works its way down through the system to national governments, and out comes something like The Plant Protection Products (Basic Conditions) Regulations 1997. In this case, though the area is similar, this UK legislation was obviously created long before the above European directive. Here we read:

1. It shall be the duty of all employers to ensure that persons in their employment who may be required during the course of their employment to use prescribed plant protection products are provided with such instruction, training and guidance as is necessary to enable those persons to comply with any requirements provided in and under these Regulations and the Plant Protection Products Regulations.

and later

3. No person in the course of a business or employment shall use a prescribed plant protection product, or give an instruction to others on the use of a prescribed plant protection product, unless that person—
(a) has received adequate instruction, training and guidance in the safe, efficient and humane use of prescribed plant protection products, and
(b) is competent for the duties which that person is called upon to perform.

and yet later

7.—(1) No person in the course of a commercial service shall use a prescribed plant protection product approved for agricultural use unless that person—
(a) has obtained a certificate of competence recognised by the Ministers; or
(b) uses that plant protection product under the direct and personal supervision of a person who holds such a certificate; or […]

The UK Regulations do not themselves define in detail what counts as “adequate instruction, training and guidance”, nor indeed “competent” and “competence”. This is where the HSE comes in, by approving as “adequate” the proposals of awarding bodies aimed at the certification of this training etc.

Do we get the general idea here? I hope so. But wait a minute …

One cannot help remarking on the differences between the language of the EU regulation and the language recommended, say, in the Europass Certificate Supplement, where it is clear that each skill or competence item should start with an action verb. Is it therefore a case of lack of effective communication between DGs? It looks likely, but I have no evidence.

Nor do I have an opinion about the merits of leaving definitions open (perhaps deliberately so) to give room for courts to establish case law and precedent.

But it would seem to me a good idea, when formulating this kind of regulation, at the same time to put together a well-structured framework of knowledge, skill and competence to define the required abilities of the people concerned. Not defining them clearly just means that the cost of defining them is multiplied through being borne by every Member State, resulting not only in divergence but in considerable administrative work that one could say was unnecessary. Multiply this across all the relevant European regulations. OK, admitted, I have little knowledge of the workings of such bureaucracy. Maybe there is a reason, but at present I am an unsatisfied citizen.

And this is one area where InLOC outputs could potentially play a role. It would be principally at a European level, though national governments could do something similar for any national regulations. Some central European body could define the required knowledge and ability, for each European regulation across all areas of public life, according to clear and sensible standard approaches that relate directly to learning outcomes, competence, training and assessment. The requirements could be published in InLOC format in all relevant languages. (That’s what InLOC is set up to facilitate). How to train and assess would still be up to training, assessment and awarding bodies, and there would still have to be structures and practices (probably with considerable national variation) within which this is controlled, and operating licences managed. But at least several stages would be removed from the process, which could be much quicker. The Commission could be seen to be more in touch with the grass roots. Procedures would look more transparent and fairer. Maybe, even, European regulations would be held in higher repute. That would be a nice outcome.

Badges for singers

We had a badges session last week at the CETIS conference (here are some summary slides and my slides on some requirements). I’d like to reflect on that, not directly (as Phil has done) but instead by looking forward on how a badge system for leisure activity might be put together.

In the discussion part of our conference session, we looked at two areas of potential application of badges. First, for formative assesssment in high-stakes fields (such as medicine); second, for communities of practice such as the ones CETIS facilitates, still in the general area of work. What we didn’t look at was badges for leisure or recreation. The Mozilla Open Badges working paper makes no distinction between badges for skills that are explicitly about work and for skills that are not obviously about work, so looking at leisure applications complements the conference discussion nicely, while providing an example to think through many of the main issues with badges.

The worked example that follows builds on my personal knowledge of one hobby area, but is meant to be illustrative of many. Please think of your own favourite leisure activities.

Motivation

On returning from the conference, on the very same day as the badges session, it happened to be a rehearsal evening for the small choir I currently sing with. So what more natural for me to think about than a badge system for singing. The sense of need for such a system has continued to grow on me. Many people can sing well enough to participate in a choir. While learners of classical instruments have “grade” examinations and certificates indicating the stages of mastery of an instrument, there is no commonly used equivalent for choral singing. Singing is quite a diverse activity, with many genres as well as many levels of ability. The process of established groups and singers getting together tends to be slow and subject to chance, but worse, forming a new group is quite difficult, unless there is some existing larger body (school, college, etc) all the singers belong to.

Badges for singers might possibly help in two different ways. First, badges can mark attainment. A system of attainment badges could help singers find groups and other singers of the right standard for them to enjoy singing with. It may be worthy, but not terribly exciting singing with a group at a lower level, and one can feel out of one’s depth or embarrassed singing with people a lot more accomplished. So, when a group looks for a new member, it could specify levels of any particular skills that were expected, as well as the type of music sung. This wouldn’t necessarily remove the need for an audition, but it would help the right kind of singer to consider the choir. Compared with current approaches, including singers hearing a choir performing and then asking about vacancies, or learning of openings through friends, a badge system could well speed things up. But perhaps the greatest benefit would be to singers trying to start new groups or choirs, where there is no existing group to hear about or to go to listen. Here, a badge system could make all the difference between it being practical to get a new group together, or failing.

Second, the badges could provide a structured set of goals that would help motivate singers to broaden and improve their abilities. This idea of motivating steps in a pathway is a strong theme in the Open Badges documentation. There must be singers at several levels who would enjoy and gain satisfaction from moving on, up a level maybe. In conjunction with groups setting out their badge requirements, badges in the various aspects of choral singing would at very least provide a framework within which people could more clearly see what they needed to gain experience of and practice, in order to join the kind of group they really want.

By the way, I recognise that not all singing groups are called “choirs”. Barbershop groups tend to be “choruses”, while very small groups are more often called “ensembles”; but for simplicity here I use the term “choir” to refer to any singing group.

Teachers / coaches

Structured goals lead on to the next area. If there were a clear set of badged achievements to aim for, then the agenda for coaches, tutors, et al. would be more transparent. This might not produce a great increase in demand for paid tuition (and therefore “economic” activity) but might well be helpful for amateur coaching. Whichever way, a clear set of smaller, more limited goals on tried and tested pathways would provide a time-honoured approach to achieving greater goals, with whatever amount of help from others that is needed.

Badge content

I’ve never been in charge of a choir for more than a few songs, but I do have enough experience to have a reasonable guess at what choirmasters and other singers want from people who want to join them. First, there are the core singing skills, and these might be broken down for example like this:

  • vocal range and volume (most easily classified as soprano / alto / tenor / bass)
  • clarity and diction
  • voice quality and expressiveness (easy to hear in others, but hard to measure)
  • ability to sing printed music at first sight (“sight-singing”)
  • attentiveness to and blend with other singers
  • ability to sing a part by oneself
  • speed at learning one’s part if necessary
  • responsiveness to direction during rehearsal and performance
  • specialist skills

It wouldn’t be too difficult to design a set of badges that expressed something like these abilities, but this is not the time to do that job, as such a structure needs to reflect a reasonable consensus involving key stakeholders.

Then there are other personal attributes, not directly related to singing, that are desirable in choir members, e.g.:

  • reliability of attendance at rehearsals and particularly performances
  • helpfulness to other choir members
  • diligence in preparation

Badges for these could look a little childish, but as a badge system for singing would be entirely voluntary, perhaps there would be no harm in defining them, for the benefit of junior choirs at least.

Does this cover everything? It may or may not cover most of what can be improved — those things that can be better or not so good — but there is one other area that is vital for the mutual satisfaction of choir and singer. Singers have tastes in music; choirs have repertoires or styles they focus on. To get an effective matching system, style and taste would have to be represented.

Assessing and awarding badges

So who would assess and award these badges? The examinations for musical instrument playing are managed by bodies such as the ABRSM (indeed including solo singing). These exams have a very long history, and are often recognised, e.g. for entry to musical education institutions. But choral singers are usually wanting to enjoy themselves, not gain qualifications so they can be professional musicians. They are unlikely to want to pay for exams for what is just a hobby. That leaves three obvious options: choirmasters, fellow singers, and oneself.

In any case, the ABRSM and similar bodies already have their own certificates and records. A badge system for them would probably be just a new presentation of what currently exists. The really interesting challenge is to consider how badges can work effectively without an official regulating body.

On deeper consideration, there really isn’t much to choose between choirmasters and fellow singers as the people who award choral singing badges. There is nothing to stop any singer being a choirmaster, anyway. There is not much incentive for people to misrepresent their choral singing skills: as noted before, it’s not much fun being in a choir of the wrong standard, nor in singing music one doesn’t like much. So, effectively, a badge system would have the job of making personal and choir standards clear.

There is an analogy here with language skills, which are closely related in any case. The Europass Language Passport is designed to be self-assessed, with people judging their own ability against a set of criteria that were originally defined by the Council of Europe. The levels — A1 to C2 — all have reasonably clear descriptors, and one sees people describing their language skills using these level labels increasingly often.

This is all very well if people can do this self-assessment accurately. The difficulty is that some of the vital aspects of choral singing are quite hard to assess by oneself. Listening critically to one’s own voice is not particularly easy when singing in a group. It might be easier if recording were more common, but again, most people are unfamiliar with the sound of their own voice, and may be uncomfortable listening to it.

On the other hand, we don’t want people, in an attempt to be “kind”, to award each other badges over-generously. We could hope that dividing up the skills into enough separate badges would mean that there would be some badges for everyone, and no one need be embarrassed by being “below average” in some ways. Everyone in a choir can have a choir membership badge, which says something about their acceptance and performance within the choir as a whole. Then perhaps all other choir members can vote anonymously about the levels which others have reached. Some algorithm could be agreed for when to award badges based on peer votes.

The next obvious thing would be to give badges to the choir as a whole. Choirs have reputations, and saying that one has sung in a particularly choir may mean something. This could be done in several ways, all involving some external input. Individual singers (and listeners) could compare the qualities of different choirs in similar genres. Choral competitions are another obvious source of expert judgement.

Setting up a badge system

The more detailed questions come to a head in the setting up of an actual badge system. The problem would not only be the ICT architecture (such as Mozilla Open Badges is a working prototype for) but also the organisational arrangements for creating the systems around badges for singers. Now, perhaps, we can see more clearly that the ICT side is relatively easy. This is something that we are very familiar with in CETIS. The technology is hardly ever the limiting factor — it is the human systems.

So here are some questions or issues (among possibly many more) that would need to be solved, not necessarily in this order.

  • Who would take on responsibility for this project as a whole? Setting up a badge system is naturally a project that needs to be managed.
  • Who hosts the information?
  • How is the decision made about what each badge will be, and how it is designed?
  • How would singers and choirs be motivated to sign up in the first place?
  • If a rule is set for how badges are to be awarded, how is this enforced, or at least checked?
  • Is Mozilla Open Badges sufficient technical infrastructure, and if not, who decides what is?
  • Could this system be set up on top of other existing systems? (Which, or what kind?)

Please comment with more issues that need to be solved. I’ll add them if they fit!

Business case

And how does such a system run, financially? The beneficiaries would primarily be choirs and singers, and perhaps indirectly people who enjoy listening to live choral music. Finding people or organisations in whose financial interests this would be seems difficult. So it would probably be useful for the system to run with minimal resources.

One option might be to offer this as a service to one or more membership organisations that collects fees from members, or alternatively, as an added service that has synergy with an existing paid-for service. However, the obvious approach of limiting the service to paid members would work against its viability in terms of numbers. In this case, the service would in effect be advertising promoting the organisation. Following the advertising theme, it might be seen as reasonable, for users who do not already pay membership, to receive adverts from sellers of music or organisers of musical events, which could provide an adequate income stream. The nice thing is that the kind of information that makes sense for individuals to enter, to improve the effectiveness of the system, could well be used to target adverts more effectively.

Would this be enough to make a business case? I hope so, as I would like to use this system!

Reflection

I hope that this example illustrates some of the many practical and possibly thorny issues that lie before a real working badge system can be implemented, and these issues are not primarily technical. What would be really useful would be to have a working technical infrastructure available so that at least some of the technical issues are dealt with in advance. As I wrote in comments on a previous post, I’m not convinced that Mozilla Open Badges does the job properly, but at least it is a signpost in the right direction.