E-portfolio Scotland

The Scottish e-portfolio scene seems to have comparatively many colleges, many of which use or are interested in Mahara. It may be even more promising than England for exploring company e-portfolio use, and we should try to ensure Scots are represented in any work on skills frameworks for e-portfolio tools.

That was the most interesting conclusion for me in a generally interesting day conference, e-Portfolio Scotland at Queen Margaret University on Friday (2010-09-10). I was given a plenary spot about Leap2A, and the audience responded well to my participative overtures — which is where I gathered this valuable information — and asked some intelligent questions. Mahara and PebblePad are well used, with Blackboard’s offering less so. Reassuringly, Leap2A came up in the presentations / demonstrations of Mahara and PebblePad, and in the final plenary by Gordon Joyes, so the audience would not be doubt about how central Leap2A is. (We just have to carry on following through and delivering!)

It was interesting to meet so many new faces. Apart from Gordon, there was Derrin Kent, and Susi Peacock on her home ground, but I didn’t know any of the others well. There seemed to be a roughly even split between HE and FE, with a very few from professions and schools. Perhaps I ought to spend more e-portfolio time in Scotland…

The vendors present included Calibrand, who I first met at the EIfEL conference this summer, and Taskstream, who have been represented in many e-portfolio conferences over several years. I suggested to the latter that they really need to take on Leap2A to get more into the UK market. A Manchester-based company, OneFile, sells a “Portfolio Assessment Solution” that I had not come across at all before, and their location has obvious potential for future discussion. But perhaps the most interesting vendor there, also giving a presentation, was Linda Steedman, MD of eCom Scotland. Their company has got beyond being a micro-business and offers an “Enterprise Skills Management” tool called SkillsLocker. I was impressed by her presentation, ranging across accreditation of prior learning, work-based learning, and what is now fashionably called “talent management” rather than HR. It seems they are well-connected, with AlphaPlus among others; also that they have done some valuable work cross mapping different skill definitions — I intend to follow this up.

Though perhaps not quite so central to JISC as those working in the HE sector, we still need to find some way of supporting the adoption of Leap2A-friendly portfolio tools in such commercially-based concerns. Work- and skills-based learning and training is a natural successor to HE-based PDP and skills development, and we really need to link in to it to make HE portfolio use more universally motivating.

One big remaining challenge was broadly acknowledged: dealing with these skill and competence representation issues that we do have on our agenda. The vision I was putting around, with no dissenting voices, was to decouple portfolio tools from any particular skills framework, and to have the frameworks published with proper URIs (in good Linked Data style). Then any tool should be able to work with any skills framework, and Leap2A information would include the relevant URIs. Though there remains the problem with HE that they tend to define skills at a different level to industry demands, FE is comparatively much closer to their employers, and they have common reference points in National Occupational Standards. So, among other things, any help we can get to persuade Sector Skills Councils to give proper URIs and structure to their NOSs will be most welcome, and maybe the Scottish e-portfolio community can help with this, and with defining the needed structures?

Developing Semantic-Web-friendly specifications

This serves a personal position statement for the CETIS Future of Interoperability Standards Meeting 2010-01-12

Why and how the Semantic Web

We want interoperability specifications and standards with a Semantic Web underlay,

  • because that is
    • the fundamental common denominator,
    • well-adapted to evolving systems,
    • good for reuse,
    • post-modern;
  • using and enabling a “linked data” strategy, with emphasis on:
    • URI-identified resources,
      • with types of resource that are widely agreed for a domain;
    • links between them,
      • using common DC-like relationships/properties/predicates;
  • but with no immediate need for RDF all at once…
    • for RDF, think more Turtle than RDF/XML;
    • any XML should be RDF friendly:
      • able to be clearly mapped and transformed to triples;
      • there may be blank nodes
        • which may be filled in one day;
    • can approach RDF via RDFa and/or GRDDL approaches;
    • may not need XML, as long what there is can be transformed to RDF;
  • using the DCMI Abstract Model as a reference point.

Where a community of practice exists

Where there is good existing practice with electronic tools, experience suggests that it is effective to start with an informal, community-driven specification initiative, and the community in question would be in the best position to decide if and when to offer the specification to a formal body for standardisation. Such an initiative could:

  • start from existing data;
  • inclusively unify current good practice.

This unification would involve:

  • establishing common conceptual models as groundwork (see below);
  • identifing elements that are close enough, and merging them;
  • retaining elements that are likely to be used in more than one system.

Good qualities for a target specification include:

  • the appropriate reuse of existing RDF-friendly specs;
  • ease of implementation;
  • graceful degradation for lesser-used features;
  • being able to be repurposed and reused in the same way that it reuses other specs.

Common conceptual models

Where there is as yet insufficient practice to fuel a specification effort by a community of practice, it is useful to get together as many people as are interested, from formal and informal groupings, and:

  • seek first to agree on a clear common conceptual model  where everyone’s point of view is represented, filling out hidden, elided concepts,
    • recognising that this involves all in development, as
    • it is a challenge to loosen up a conceptual scheme, so
    • people need support and the right context.

Such a conceptual modelling process can work well by being primed with personal discussions between those able to develop their conceptual models. It is essential to the viability of a common conceptual model that everyone with a significant variant opinion is drawn in to the process of working towards a common model. Each of these discussions needs to focus on mutual understanding and a mutual development of positions so that each position comes to include a partial model that is shared between the parties. This takes time – typically several hours, not a few minutes – but is very promising.

This deep communication and shared modelling process is certainly not well-adapted to formal committee procedure. Nor is it suitable for a collective process of a community of interest or of practice. But both formal and informal bodies can perfectly well encourage dialogues of this kind to happen, and seek to check whether they have in fact taken place sufficiently to provide the basis of a usable common model.

Clearly, some people find loosening their conceptual structures more difficult than others. Bodies, formal and informal, should ideally stress that this is necessary, provide encouragement (and perhaps even education or training) in how to do it, and finally discourage those who are unable or unwilling to do this from participating in these processes at all.

Information models, specifications and standards

After the agreement of a common conceptual model, information models can be based on it, as the basis for specifications and eventually standards. This does not mean that the whole conceptual model needs to be represented in any information model, nor even that complete parts of the conceptual model need to be. If no relevant information attaches to a particular concept in the conceptual model, it is quite reasonable to leave it out from a practical information model (resulting in what I have termed “elision”) as long as the conceptual model is kept in mind to refer back to.

Derivative information models should, rather:

  • feel comfortable to practitioners;
  • not be hard to implement;
  • but still be interoperable.

Notes and references

More competency

The CEN WS-LT Competency SIG discussions of a conceptual model for skill/competence/competency are still at the very interesting early stage where very many questions are open. What kind of model are we trying to reach, and how can we get to where we could get? Anything seems possible, including experiments with procedures and conventions to help towards consensus.

Tuesday, December 1st, Berlin — a rainy day in the Ambassador Hotel, talking with an esteemed bunch of people about modelling skill/competence/competency. I won’t go on about participants and agenda — these can be seen at http://sites.google.com/site/competencydriven/ It was all interesting stuff, conducted in a positive atmosphere of enquiry. I’ll write here about just the issues that struck me, which were quite enough…

How many kinds of model are there?

At the meeting, there seemed to be quite some uncertainty about what kind of model we might be trying to agree on. I don’t know about other people, but I discern two kinds of model:

  • a conceptual model attempting to represent how people understand entities and relationships in the world;
  • an information model that could be used for expressing and exchanging information of common interest.

A binding isn’t really a separate model, but an expression of an information model.

My position, which I know is shared by several others, is that to be effective, information models should be based on common conceptual models. The point here is that without an agreed conceptual model, it is all too easy to imagine that you are building an information model where the terms mean the same thing, and play the same role. This could lead to conflict when agreeing the information model, as different people’s ideas would be based on different conceptual models, which would be hard to reconcile, or even worse in the long term, troublesome ambiguity could become embedded in an information model. Not all ambiguity is troublesome, if the things you are being ambiguous about really share the same information model, but no doubt you can imagine what I mean.

Claims and requirements for competence

A long-term aim of many people is to match what is learned in education with what is required for employment — Luk Vervenne was as usual championing the employer point of view. After reflection on what we have at the moment, and incorporating some of Luk’s ideas in the common information model I’ve been putting together, I’d say we have enough there to make a start, at least, in detailing what a competency claim might be, and how that might relate to a competency requirement.

In outline, a full claim for a single separate competence component could have

  • the definition of that component (or just a title or brief description if no proper definition available)
  • any assessment relevant to that component, with result
  • any qualification or other status relevant to that component (which may imply assessment result)
  • a narrative filling the gap between qualifications or assessment and what is claimed
  • any relevant testimonials
  • a record of relevant experience requiring, or likely to lead to, that competence component
  • links to / location of any other relevant “raw” (i.e. unassessed) evidence

I’ll detail later a possible model of competency requirements, and detail how the two could fit together. And I have now put up the latest version of the big conceptual model as well. There is clearly also a consequent need to be clearer about the structure of assessments, and we’ll be working on that, probably both within CETIS and within the CEN WS-LT.

What about competencies in themselves?

Reflected in the meeting, there still seems to be plenty of disagreement about the detail that is possible in an information model of a competency. Lester Gilbert, for example put forward a model in which he distinguished, for a fully specified educational objective:

  • situation;
  • constraints;
  • learned capability;
  • subject matter content;
  • standard of performance;
  • tools.

The question here, surely, is, to what extent are these facets of a definition (a) common and shared (b) amenable to representation in a usefully machine-processable way?

Personally, I wouldn’t like to rule anything in or out before investigating more fully. At least this could be a systematic investigation, looking at current practice across a range of application areas, carefully comparing what is used in the different areas. I have little difficulty believing that for most if not all learning outcomes or competency definitions, you could write a piece of text to fit into each of Lester’s headings. What I am much more doubtful about is whether there is any scheme that would get us beyond human-readable text into the situation where we could do any automatic matching on these values. Even if there are potential solutions for some, like medical subject headings for the subject matter content, we would need these labels to be pretty repeatable and consistent in order for them to be used automatically. And, what would we do with things like “situation”? The very best I could imagine for situation would be a classification of the different situations that are encountered in the course of a particular occupation. In UK NOSs, these might be written in to the documentation, either explicitly or implicitly. Similar considerations would apply to Lester’s “tools” facet. This might be tractable in the longer term, but would require at least the creation of many domain-specific ontologies, and the linking of any particular definition to one of these domain ontologies.

I can also envisage, as I have been advocating for some time, that some competency definitions would have ontology-like links to other related definitions. These could be ones of equivalence, or the SKOS terms “broadMatch” and “narrowMatch”, in cases where the authorities maintaining the definitions believed that in all contexts, the relationship was applicable.

What about frameworks of skill, competence, etc.?

It surprised me a little that we didn’t actually get round to talking about this in Berlin. But on reflection, with so many other fundamental questions still on the table, perhaps it was only to be expected. Interestingly, so far, I have found more progress here in my participation with MedBiquitous than with the CEN WS-LT.

I’ll write more about this later, but just to trail the key ideas in my version of the MedBiquitous approach:

  • a framework has some metadata (DC is a good basis), a set of competency objects, and a map;
  • the map is a set of propositions about the individual competency objects, relating them to each other and to objects that are not part of the framework;
  • frameworks themselves can be linked to as constituent parts of a framework, just as individual competency objects;
  • it is specified whether to accept the relationships defined within the competency objects, and in particular any breakdown into parts.

The point here is that just about any competency definition could, in principle, be analysed into a set of lower-level skills or competencies. This would be a framework. Equally, most frameworks could be used as objectives in themselves, so playing the same role as an individual competency object, being part of a competency framework. If a framework is included, and marked for including its constituent parts, then all those constituent parts would become part of the framework, by inclusion rather than by direct naming. In this way, it would be easy to extend someone else’s framework rather than duplicating it all.

Need for innovations in process and convention

Perhaps the most interesting conclusion from my point of view was about how we could conduct the processes better. There is a temptation to see the process as a competition between models — this would assume that each model is fixed in advance, and that people can be objective about their own, as well as other people’s models. Probably neither of these assumptions is justified. Most people seem to accept the question as “how can a common conceptual model be made from these models?”, even though there may be little wisdom around on how to do this. There is also the half-way approach of “what common information elements can be discerned between these models?” that might come into play if the greater aim of unifying the conceptual models was relinquished.

From my point of view, this brings me back to two points that I have come to recognise only in recent months.

This meeting, for me, displayed some of the same pattern as many previous ones. I was interested in the models being put forward Luk, and Lester, and others, but it was all too easy not to fully understand them, not quite to reach the stage of recognising the insights from them that could be applied to the model I’m continuing to put together. I put this down to the fact that the meeting environment is not conducive to a deep mutual understanding. One can ask a question here and there, but the questions of others may be related more to their own models, not to the relationship of the model under discussion with one’s own. So, one gets the feeling at the end of the meeting that one hasn’t fully grasped the direction one should take one’s own model. Little growth and development results.

So I proposed in the meeting what I have not actually proposed in a meeting before, that we schedule as many one-to-one conceptual encounters as are needed to facilitate that mutual growth of models at least towards the mutual understanding that could allow a meaningful composite to be assembled, if not a fully constituted isomorphism. I don’t know if people will be bold enough to do this, but I’ll keep on suggesting it in different forums until someone does, because I want to know if it is really an effective strategy.

The other point that struck me again was about the highest-level ontology used. One of the criteria, to my mind, of a conceptual model being truly shared, is that people answer questions about the concepts in recognisably similar ways, or largely the same on a multiple choice basis. Some of those questions could easily relate to the essential nature of the concept in question. In the terms of my own top ontology, is the concept about the material world? Or is it a repeatable pattern, belonging to the world of perception and thought? Is it, rather a concept related to communication — an expression of some kind? Whether this is exactly the most helpful set of distinctions is not the main point — it is that some set of distinctions like this will surely help people to clarify what kind of concepts they are discussing and representing in a conceptual model, and thus help people towards that mutual understanding.

A similar, but less clear point seems to apply to relationships between concepts. Allowed free rein in writing a conceptual model, people seem to write all kinds of things in for the relationships between concepts. Some of them seems to tie things in knots — “is a model of” for instance. So maybe, as well as having clear types for concepts, maybe we could agree a limited vocabulary for permitted relationships. That would certainly help the process of mapping two concept maps to each other. There are also two related conventions I have used in my most recent conceptual model.

  1. Whole-part relationships are represented by having contained concepts, of varying types, represented as inside a containing concept. This is easy to do in CmapTools. Typically the containing concept represents a sub-system of some kind. These correspond to the UML links terminated by diamond shapes (open and filled).
  2. Relationships typically called “kind of” or “is a” correspond to the UML sub-class relationship, given with an open triangle terminator. As these should always be between concepts of the same essential type, these can be picked out easily by being a uniform colour for the minimized and detailed representations of the whole.

So, all in all, a very stimulating meeting. Watch this space for further installments as trailed above.

Consensus process and conceptual models

I was in Umeå last week — at the NORDLET conference — but also there were lots of ISO SC36 people coming for their meeting. Now SC36 is trying to put forward a “PDTR” — draft technical report — “ITLET – Conceptual reference model for competencies and related objects” which I have been interested in for a while, as it would be nice to use this opportunity to broaden consensus about competence and related terms. There is a diagram representing a current proposal, and this is part of a set of documents tracing the development of this work, which up to now I was not aware of.

You know the basic idea about the standardization process — there is meant to be established practice in the area which would benefit from standardization. But, as I am just discovering, there is another kind of ISO document, the Technical Report. This one is of “type 3, when the joint technical committee has collected data of a different kind from that which is normally published as an International Standard (“state of the art”, for example).” And so we could at least think over, what is the appropriate time to try to agree a conceptual model? What is the appropriate process?

My experience and intuition both tell me that it takes a lot of work to agree on a shared conceptual model. People almost always build up their own private conceptual models of any area they are involved with. Concepts are used in discussion among groups, and these concepts become important to the group members, as they are part of their language, and their means of communication. Put two groups together, attempting to merge their conceptual models, and you are likely to find people defending their corners. That’s not consensus process, and it could easily result in a botch of a conceptual model that satisfies no one at all.

But put two individuals together, free from the constraints of being answerable to a group, and a wholly different process can take place. Instead of communication being a prompt to defend known terms, it is naturally a motive to explore the other person’s terms and meanings. And the prize at the end of a deep exploratory dialogue of this kind could be a shared conceptual model — not necessarily where the facts in the model are agreed, but at least where the language is agreed in which it is possible to disagree comprehensibly. And it is just during these dialogues — again, in my experience — that each person’s conceptual model is able to grow.

It seems like a good idea, therefore, in any case of trying to agree a conceptual model, to base the consensus process around dialogues between pairs of people. The time needed can seem long. My (good) experience of talking with Simone Laughton — an SC36 member from Canada — suggests that it takes at least several hours to get a good enough understanding of someone else’s conceptual model, at least of the type that SC36 are trying to agree on, if one wants to create a shared conceptual model.

So, contrast two alternatives. Firstly, the existing committee-based processes, which seem to run up again and again into the difficulties of disagreement, perhaps because of the mechanisms suggested above. Secondly, deep pairwise dialogue between all the participants who feel they have a position on a conceptual model. The second option is front-loaded — that is, it seems to need more work up front — but my guess is that once that work had been done — once people have had the experience of the development of their own model through dialogue — the committee process would probably be very quick and painless. And, maybe, the process might even take less time overall, as well as being more likely to result in a genuinely shared model.

Worth a try?

Worth a try also, perhaps, in areas that might be of similar complexity, such as the attempt to agree a management or governance structure between people in the same organisation, or with a common interest?

LEAP2A in ALT newsletter

The LEAP2A specification – for e-portfolio portability and interoperability – means a lot to the developers who have worked on it, but can be a challenge to describe concisely to others who are less familiar with this technology. Recently it was suggested that I write a short article to do just that, and it appeared in Issue 16 of the ALT Newsletter, May 2009.

More PDP and e-portfolios – Reading

Yesterday I went to an interesting event in Reading (at the University) called “Future-proofing PDP and ePortfolios“. My role was only to answer questions in a Q&A session on interoperability, but as there were few technical people around it was called “Can I take away what I’ve put into our PDP system?”

Two really interesting points emerged.

  1. Many institutions feel stuck with Blackboard at present, even for their portfolio functionality. Generally, they are unhappy with this.
  2. There are a few interesting tools that work in Blackboard, and people are keen on using the wiki facility for building portfolio presentations.

The wiki tool in question is from Learning Objects. The general idea is that learners find wiki technology an easy way to write a presentation, and if that is what they want to do with a “portfolio”, it should work fine – as indeed any wiki technology. I don’t know how important the integration with the rest of the e-learning system would be.

But this in turn brings up the question, if wikis are used as a platform for constructing e-portfolio presentations, can we make them interoperable with other e-portfolio systems? It would be great if we could. I intend to ask around, and think around, this issue, and write more. The basic idea would be to get a new version of LEAP2 out – LEAP2R – that would be LEAP2 in RDFa – and then see if a wiki system can be tweaked to export and import XHTML+RDFa in LEAP2R format. We would of course also build transforms to convert between LEAP2A and LEAP2R.

Book finally available

My book, “Electronic Portfolios: Personal information, personal development and personal values” has recently been published, and is eventually available on Amazon UK etc. (or .fr or .de or .com)

The publishers have it in their catalogue.

I was very surprised by the high list price, which I have had no influence over. I would publish it for no more than half that price. Perhaps the publishers aren’t expecting all that many sales? But I hope that doesn’t stop people ordering it for their libraries. It is relevant to many different people, and the principles should be valid for a few years, so I’d say it’s worth having in any library where there are educators using e-portfolios, or developers developing them.

Skills frameworks, interoperability, portfolios, etc.

Last Thursday (2009-04-16) I went to a very interesting meeting in Leeds, specially arranged, at the Leeds Institute of Medical Education, between various interested parties, about their needs and ideas for interoperability with e-portfolio tools – but also about skills frameworks.

It was interesting particularly because it showed more evidence of a groundswell of willingness to work towards e-portfolio interoperability, and this has two aspects for the people gathered (6 including me). On the one hand, the ALPS CETL is working with MyKnowledgeMap (MKM) – a small commercial learning technology vendor based in York – on a project involving health and social care students in their 5 HEIs around Leeds. They are using the MKM portfolio tool, Multi-Port, but are aware of a need to have records which are portable between their system and others. It looks like being a fairly straightforward case of a vendor with a portfolio tool being drawn in to the LEAP2A fold on the back of the success we have had so far – without the need for extra funding. The outcome should be a classic interoperability win-win: learners will be able to export their records to PebblePad, Mahara, etc., and the MKM tool users will be able to import their records from the LEAP2A-implementing systems to kick-start their portfolio records there with the ALPS CETL or other MKM sites.

MKM tools, as suggested by the MKM name, do cover the representation of skills frameworks, and this forms a bridge between two threads to this meeting: first, the ALPS CETL work, and second, the more challenging area of medical education, where frameworks – of knowledge, skill or competence – abound and are pretty important for medical students and in the professional development of medical practitioners, and health professionals more generally.

In this more challenging side of the meeting, we discussed some of the issues surrounding skills frameworks in medical education – including the transfer of students at undergraduate level; the transfer between a medical school like Leeds and a teaching hospital, where the doctors may well soon be using the NHS Foundation Year e-portfolio tools in conjunction with their further training and development; and then on to professional life.

The development of LEAP2A has probably been helped greatly by not trying to do too much all at once. We haven’t yet fully dealt with how to integrate skills frameworks into e-portfolio information. At one very simple level we have covered it – if each skill definition has a URI, that can be referred to by an “ability” item in the LEAP2A. But at another level it is greatly challenging. Here in medical education we have not one, but several real-life scenarios calling for interoperable skills frameworks for use with portfolio tools. So how are we actually going to advise the people who want to create skills frameworks, about how to do this in a useful way? Their users, using their portfolio tools, want to carry forward the learning (against learning outcomes) and evidence (of competence) to another setting. They want the information to be ready to use, to save them repetition – potentially wasteful to the institution as well as the learner.

The answer necessarily goes beyond portfolio technology, and needs to tackle the issues which several people are currently working on: European projects like TENCompetence and ICOPER, where I have given presentations or written papers; older JISC project work I have been involved with (ioNW2, SPWS); and now the recently set up a CETIS team on competences.

Happily, it seems like we are all pushing at an open door. I am happy to be able to respond in my role as Learning Technology Advisor for e-portfolio technology, and point MKM towards the documentation on – and those with experience of implementing – LEAP2A. And the new competence team has been looking for a good prompt to hold an initial meeting. I imagine we might hold a meeting, perhaps around the beginning of July, focused on frameworks of skills, competence, knowledge, and their use together with curriculum learning outcomes, with assessment criteria, and with portfolio evidence? The Leeds people would be very willing to contribute. Then, perhaps JISC might offer a little extra funding (on the same lines as previous PIOP and XCRI projects) to get together a group of medical educators to implement LEAP2A and related skills frameworks together – in whatever way we all agree is good to take forward the skills framework developments.

LEAP2A progress

The Portfolio InterOperability Projects (PIOP) partners have been working hard on our LEAP2A spec for portfolio interoperability and portability, and at our meeting last week we ironed out some of the last things necessary to agree a good working specification, able to represent just about any information that is in common use in more than one e-portfolio system. The spec just allows for export/download and import/upload so far, rather than web services, but that will come later.

The spec has been developed with and by developers for developers, so it is relatively easy to implement, being based on Atom. Several people working on e-portfolio-related projects were at the JISC e-Learning Programme meeting yesterday (at Aston) and there was plenty of positive and encouraging comment around.

Anyone interested is very welcome to comment on where we have got to, and send me suggestions for improvement so that any other relevant system dealing with similar information can have the appropriate information also represented in LEAP2A, to enable interoperability with the rest of our established partners.