Category Archives: standardization

Standardization process – ISO SC36

Posted on by

In mid March I spent several days at the ISO SC36 meeting in Strasbourg, an experience which was … how can I say this … frustrating. Let me give some background, explain my frustration, then offer ideas about causes and possible remedies.

ISO/IEC JTC1 SC36 (official ISO page; own web site) is the International Organization for Standardization’s committee on ITLET — information technologies for learning education and training. It currently meets twice a year at points round the globe, and as it was meeting relatively close by, and was dealing with an item of considerable importance to my work (e-portfolio reference model) I chose to attend: not, however, for the full week and more, but just for the WG3 meetings, which spanned 4 days. These meetings are attended by representatives of national standards bodies — in our case, BSI.

What happens at these meetings is governed by procedural rules that participants explain are necessary, and to which they are resigned, rather than interpreting creatively. The problem is not just the process itself, which would not be too bad if everything ran according to plan, but the way in which it is applied inflexibly. Even if it is clear that a draft presented to a committee needs a lot of revision, participants have to stick to wading through as many comments as national bodies have provided. As each national body provides comments independently, many of the comments conflict. Comments can be general, technical, or editorial, and the committee has to find a resolution on each one, fitting in to just one of the few acceptable formulae. This might be fine if everything was going the way the procedure writers imagined, but when it isn’t, it can be excruciating. The only way I found to make it tolerable is to multi-task, and to do other work at the same time as the committee proceedings when the matter in hand was of no great importance (though this does depend on having a good Internet connection available).

The result is great inconvenience, and either an inefficient process where people are only giving part of their attention, or a frustrating waste of time if you try to give the whole of your attention. It’s not that these processes don’t need to be gone through — they do — but better ways must be possible, and surely need to be implemented. For instance, what if the editors received feedback from national bodies, and produced an integrated redraft, dealing as well as they can with all the comments? No physical meeting is needed for this. This inability to change the process to suit the actual situation seems to feed the whole procedural problem.

The consequence of inefficiency could be that busy people are less likely to engage with the process than they might otherwise have been. Or perhaps the people who are really influential in large organisations will just stay at home, and send some minion to do the negotiation for them. That would compromise the nature of any agreement that is able to be reached. Many of the people who do appear at such meetings are either academics, with their own research agenda, or independent or semi-independent professionals, who use the opportunity to network, and in the worst cases (happily not evident at all among the people I met) such people can even use the processes to advance their own business interests at the expense of others. This is hardly the ideal recipe for an effective, meaningful, significant standards-setting body.

Sometimes, committee output is “just a technical report”, with no official status as a standard. Now in CEN, the processes is usefully differentiated: in our area there is an informal Workshop (WS-LT) to discuss things and come to agreements, and a formal Technical Committee (TC353), made up of national body representatives, to take decisions on standardisation. But in ISO, there is no similar division of role.

It seems to me that, particularly in the current economic climate, the SC36 mode of operation may not be sustainable, as people come to apply some kind of cost-benefit analysis. If we want such standardisation bodies to continue in the future, I’d say we need to review the processes deeply, coming to a new understanding of what structures and practices are helpful towards what ends. (We can then ask those who care about those ends to finance the process.)

Of course the following speculation is not of itself going to make changes happen. However, it is just possible that conversations about the issues may help towards a consensus about how to move things forward. (Co-incidentally, see my private blog about the piece by Theodore Zeldin on the Pont de l’Europe in Strasbourg.)

First, I would move most of the ISO process away from face-to-face. We do need to get to know others personally, but this could be done more effectively through something more like an annual conference, with the majority of time devoted to networking, and some presentations of the live issues that most need discussion, in preparation for the consensus process.

Second, I would adapt the processes so that they are better tuned to producing durable consensus. How to do this is too large a topic to address here.

Third, I would put in several checks at different stages of the work to confirm that whatever was being discussed was genuinely of importance to significant stakeholders. When a work item failed such a test, it would be dismissed. This would probably reduce the workload very significantly.

Fourth, I would try (though I don’t know how) to ensure that all participants

  • properly understand consensus process
  • are committed to acting transparently
  • come to the proceedings with good will

Even if bodies like ISO don’t get round to it, it would be good for those who care to formalise some set of principles such as the ones I am suggesting above, resulting in what could be seen as agreed standards for standards bodies. If we had a list of criteria by which to judge standards bodies and standardisation process, we could agree to support and attend only bodies that conformed. This would apply not only to official “de jure” standardisation bodies, but also to the many other bodies (including all those we know in CETIS) that prepare and publish interoperability specifications.

If anyone knows of any such existing guidelines, I’d be grateful to learn of them.

CEN WS-LT 2011-01-17 Brussels

Posted on by

At this useful meeting, on the premises of European Schoolnet, the question of e-portfolio information was introduced, and other business carried forward. Many of the familiar people were there: though among those missed were Scott, Mark Stubbs, Cleo Sgouropoulou. Dan Rehak was absent, but Angelo Panar was there in his place. I turned out to be the only UK person. A bright new face was Eleni Kargioti, from Q&R, a Greek company contracted to CEDEFOP to do interesting things with their Europass web site.

Jan Pawlowski, in the chair, changed the agenda a bit to include a presentation from Eleni, and to include discussion on the EC’s Standardisation Work Programme in ICT. “Our” bit is Domain 7: eSkills and eLearning. We have an obvious interest in having EC policy matched with what we think is important. The idea this time was just to gently edit, not to change anything substantive. As the term “e-portfolio” has now been included under out interest in European Learner Mobility, I persuaded people for the corresponding inclusion of “Professional Development” under the “elearning” interest heading.

We did discuss the InLOC proposal (integrating learning outcomes and competences), which has been very delayed this year mainly due to confusion over the resignation of DIN from providing the workshop secretariat. It has just last week been finally resubmitted to CEN who are doing their bit and passing it on. This is a vital proposal that I have been jointly authoring, as it follows on from the earlier European Learner Mobility work that led to the EuroLMAI specifications. Now, the secretarial question has nearly been resolved, with AENOR (Spanish national body) offering to take on the workshop on condition that InLOC is funded.

I made a presentation about the need for standardisation in area of e-portfolio information. It’s on Slideshare. After a brief history, I point out the key motivations, which include the fact of the Korean-led ISO SC36 e-portfolio reference model work. I outline the Leap2A model, and give thoughts about how Leap2A and the NL profile of IMS eP might be brought together. Renewed CEDEFOP involvement makes this work even more timely.

I also drew attention to the forthcoming ISO SC36 meeting in Strasbourg in March. We need to get together with our usual suspect colleagues (including Erlend, Tore and Christian who all plan to be there) and focus relevant ideas.

Eleni Kargioti gave a very heartening presentation, reflecting the fact that CEDEFOP is back in touch. There is now great potential for collaboration over Europass in several ways. Perhaps CETIS should follow this up, develop a relationship with CEDEFOP perhaps through Q&R, and make the connections with the HEAR work, for example.

Joris Klerkx talked about the Interoperability of Registries work. As I know little about learning resources or repositories, it mainly went over my head, though it did look vaguely plausible. The slight issue here is that there may be competition between this work and e-portfolio information work for funding in the coming year’s work programme. So, from our side, we need to find who is really interested in e-portfolio work and make a convincing case by the end of February, to maximise the chance of benefiting from the opportunity for Leap2A being a key part of a future European Standard, just as XCRI provided perhaps the majority of ideas for MLO.

The next meeting of the Workshop is in Madrid, April 11, followed the next day by TC353. After that, we are investigating a summer meeting for the Workshop in conjunction with the EUNIS conference, which is in Dublin in June; the following WS-LT and TC353 meeting is still scheduled for October 13 in Sweden.

eCOTOOL

Posted on by

eCOTOOL – eCOmpetences TOOLs – is a 2-year European project in which Bolton / CETIS are collaborating principally through me. We are producing an information model for the Europass Certificate Supplement (ECS), applied to training in the agricultural sector.

The kick-off meeting was in Essen, December 14th to 16th, and this post is an attempt to summarise our agreed starting point.

Partners

The University of Duisberg-Essen (UDE) is leading the project, and the 8 partners besides UDE and ourselves are

  • BIBB, the German Federal Institute for Vocational Education and Training
  • MAICh, the Mediterranean Agronomic Institute of Chania, Crete
  • ELOT, the Greek national body for standardization
  • Agro-Know Technologies, a Greek “research-oriented enterprise”
  • UZEI, the Czech Institute of Agricultural Economics and Information
  • KGZS, the Chamber of Agriculture and Forestry of Slovenia
  • ISFOL, the Italian Institute for the Development of Vocational Training
  • KION SpA, a company developing information systems for Italian universities

ISFOL and BIBB include their respective National Europass Centres.

We know several of the people involved through previous work. Christian M. Stracke, the overall project director, has been involved in many European standards bodies and projects. Cleo Sgouropoulou (ELOT) and Simone Ravaioli (KION) are part of the core team working with me on the European Learner Mobility work, assisted by Christian, our Scott (also working on this project), Alessandra Biancolini (ISFOL) and others. To understand this project, it is useful to set it in the context of this and other related work.

What the project is doing

At the meeting, Christian Stracke described as the “story in brief” how eCOTOOL provides the missing ingredient to add to a mix of

to create a “Europass CS with Competence as Application Profile and XML.” What this means in practice is (I hope) explained in what follows.

What is the ECS, and how could it be used?

Well then, what is this Europass Certificate Supplement? Its cousin the Europass Diploma Supplement (EDS) is better known, as the EDS has much in common with transcripts offered to graduates, and the HEAR (Higher Education Achievement Report) that is expected to supersede the transcript in the UK. The ECS, in contrast, does not have details of the results of individual learners. Exactly the same certificate is given to all those who successfully complete the same course. It is not designed for the kind of academic study where learners get marks and grades depending on their exam results, but rather for the kind of training course where people pass or fail. Either you can do something, or you can’t, the view would be, and being awarded the Certificate says that you can, because you did the course. The key information held in the ECS is the detailing of what it is that someone can do after successful completion of the course, which is just what the EDS does not record. This is done in the ECS Section 3, called “Profile of skills and competences”. However, the paper ECS does not define any specified structure to this section, leaving it simply as a text box. This is adequate for human reading, but inadequate for precise automatic use of the information contained.

Despite the lack of official definition, a common practice has built up in ECS use to list perhaps 5 to 15 items in Section 3, each comprising one sentence starting with an action verb. The expectation seems to be that a list of skill and competence items is defined specifically for each ECS offered, which may be common to several courses across different providers. This at least offers a little more informal standardization, and perhaps increases the ease of translation, but still does not address automatic processing well.

But, surely, there is much that could be done with an ECS with more electronic detail and structure, particularly in that Section 3. Here are a few ideas, based on ones that came up at the meeting.

  • National Europass Centres would be able to manage the national collection of ECS documents, being able, for instance, to search for all ECSs that had a particular skill or competence line.
  • Learners and training providers would be able to search for courses that had ECSs containing particular skills or competences. Learners (or their employers) could use this to plan their training; training providers could look for competitors, or gaps in the training market.
  • ECS information could be downloaded into e-portfolio tools for use by learners, without the need to cut and paste; or possibly even presented through e-portfolio tools, without any need for an actual downloading or copying. This, in turn, would facilitate the creation of CVs or other presentations that were searchable by recruiters for particular skills or competences.

The other “ingredients” of eCOTOOL are interesting for their illustration of the kind of approach to be taken.

PAS 1093 – what’s that?

Translated into English, this “Publicly Available Standard” deals with “Human Resource Development with special consideration of Learning, Education and Training – Competence Modelling in Human Resource Development”. In the Foreword, it is described as “This Publicly Available Specification (PAS) is a Reference Framework for the development as well as for the structural comparison and evaluation of competence Modeling in Human Resource Development.”

What PAS 1093 does do is give a structure for HR staff to go through the process of documenting the competences that are relevant to them. “Reference Framework” is a very slippery label, and PAS 1093 may not do other things that might be associated with the label. Probably, the essential “framework” could be summarised in a just few pages, but what is also useful about the document is that it sets out some of the thinking behind managing competence information.

An English language version was distributed to project members. It is held at http://www.qed-info.de/downloads.

EQF: the European Qualifications Framework

Much of the documentation for the EQF can be found though its European Commission page.

One obvious aim of the project is to include EQF terminology where appropriate into the information available through ECS documents. Clearly, one place for this is in Section 5, “Official basis of the certificate”, but ideally there might be a way of tracking though individual skills and competences in Section 3 to a corresponding EQF level, if that is not the same as the overall level of the certificate. There are two essential aspects to EQF terminology:

  • the levels, 1 to 8;
  • the distinction between knowledge, skill and competence;

and we may also want to find a good way of representing which of these categories a particular line or definition falls into.

ECVET

The European Credit system for Vocational Education and Training (ECVET) is nicely introduced by that linked page on the European Commission site. ECVET intends to do for vocational education and training something like what the ECTS (the credit transfer system) does for university education, helping learners move between courses and between countries, both during and after their VET. However, implementation is not envisaged until 2012 and after.

ECVET depends partly on representing the intended learning outcomes of VET opportunities. Some countries have defined standards like the UK’s National Occupational Standards, which serve to underpin the UK’s National Vocational Qualifications. In principle, if similar national standards were established across Europe, and cross-related to each other, this might provide the right basis for ECVET to work.

The eCOTOOL project takes the agricultural sector as a test case for establishing what can be done. We will have to deal with considerable diversity, including such variations as the fact that in the UK, there are different NVQs, NOSs and Sector Skills Councils for agriculture (LANTRA) and food and drink (Improve), while in other countries they are taken together. To me, that makes quite a lot of sense, as some countries retain much farm-based production, whereas in the UK food raw materials have for a long time come from UK farms and abroad.

Relationship to my other work

I’ve had an interest in National Occupational Standards and representing competencies for several years now, partly though work done for XCRI and the ioNW2 project funded by JISC and run by GMSA, and partly what I have written for TENCompetence project workshops. Most recently, there are very strong connections with the European Learner Mobility work I mentioned above, the ICOPER project we are also now involved in, and all the skill and competence related discussions I have been involved in through our CETIS work on competences, and the related CEN WS-LT competency group. All of this feeds in to my role in eCOTOOL.

My role in eCOTOOL

Our main responsibility in eCOTOOL is for WP1, which is called “Application Profile Development of Europass CS”. In the kick-off meeting we had to explain that the term “application profile” really just meant an information model, but one that could well be created on the basis of other established specifications. Much as we did in the European Learner Mobility work, this will mean starting with the ECS structure as it is, and working out how best to represent that as elements in an information model. Obviously, we need to be able to represent existing plain document ECS examples, but we also want to look carefully at possible integration with other Europass documents (including the DS, which we have worked extensively on), and we want to bear in mind all the likely uses of this kind of competence information.

One particular issue that I will focus on first will be the relationship between the one-line versions of a skill or competence that currently appear in Section 3 of ECS documents, and the fuller definition of skills and competences that would not actually appear in an ECS document, but which are often implicit in course offerings. One of the issues that also impinges greatly on the portfolio interoperability work involving LEAP2A is how to represent structures of related skill and competence definitions. All this has to be done, of course, in a way that allows useful tools for applications that are really wanted by end users – in our case, particularly learners and training bodies in the area of agricultural VET.

I would (of course!) welcome any comments or correspondence, as it would be good to integrate as many good ideas as we can grasp.

Developing Semantic-Web-friendly specifications

Posted on by

This serves a personal position statement for the CETIS Future of Interoperability Standards Meeting 2010-01-12

Why and how the Semantic Web

We want interoperability specifications and standards with a Semantic Web underlay,

  • because that is
    • the fundamental common denominator,
    • well-adapted to evolving systems,
    • good for reuse,
    • post-modern;
  • using and enabling a “linked data” strategy, with emphasis on:
    • URI-identified resources,
      • with types of resource that are widely agreed for a domain;
    • links between them,
      • using common DC-like relationships/properties/predicates;
  • but with no immediate need for RDF all at once…
    • for RDF, think more Turtle than RDF/XML;
    • any XML should be RDF friendly:
      • able to be clearly mapped and transformed to triples;
      • there may be blank nodes
        • which may be filled in one day;
    • can approach RDF via RDFa and/or GRDDL approaches;
    • may not need XML, as long what there is can be transformed to RDF;
  • using the DCMI Abstract Model as a reference point.

Where a community of practice exists

Where there is good existing practice with electronic tools, experience suggests that it is effective to start with an informal, community-driven specification initiative, and the community in question would be in the best position to decide if and when to offer the specification to a formal body for standardisation. Such an initiative could:

  • start from existing data;
  • inclusively unify current good practice.

This unification would involve:

  • establishing common conceptual models as groundwork (see below);
  • identifing elements that are close enough, and merging them;
  • retaining elements that are likely to be used in more than one system.

Good qualities for a target specification include:

  • the appropriate reuse of existing RDF-friendly specs;
  • ease of implementation;
  • graceful degradation for lesser-used features;
  • being able to be repurposed and reused in the same way that it reuses other specs.

Common conceptual models

Where there is as yet insufficient practice to fuel a specification effort by a community of practice, it is useful to get together as many people as are interested, from formal and informal groupings, and:

  • seek first to agree on a clear common conceptual model  where everyone’s point of view is represented, filling out hidden, elided concepts,
    • recognising that this involves all in development, as
    • it is a challenge to loosen up a conceptual scheme, so
    • people need support and the right context.

Such a conceptual modelling process can work well by being primed with personal discussions between those able to develop their conceptual models. It is essential to the viability of a common conceptual model that everyone with a significant variant opinion is drawn in to the process of working towards a common model. Each of these discussions needs to focus on mutual understanding and a mutual development of positions so that each position comes to include a partial model that is shared between the parties. This takes time – typically several hours, not a few minutes – but is very promising.

This deep communication and shared modelling process is certainly not well-adapted to formal committee procedure. Nor is it suitable for a collective process of a community of interest or of practice. But both formal and informal bodies can perfectly well encourage dialogues of this kind to happen, and seek to check whether they have in fact taken place sufficiently to provide the basis of a usable common model.

Clearly, some people find loosening their conceptual structures more difficult than others. Bodies, formal and informal, should ideally stress that this is necessary, provide encouragement (and perhaps even education or training) in how to do it, and finally discourage those who are unable or unwilling to do this from participating in these processes at all.

Information models, specifications and standards

After the agreement of a common conceptual model, information models can be based on it, as the basis for specifications and eventually standards. This does not mean that the whole conceptual model needs to be represented in any information model, nor even that complete parts of the conceptual model need to be. If no relevant information attaches to a particular concept in the conceptual model, it is quite reasonable to leave it out from a practical information model (resulting in what I have termed “elision”) as long as the conceptual model is kept in mind to refer back to.

Derivative information models should, rather:

  • feel comfortable to practitioners;
  • not be hard to implement;
  • but still be interoperable.

Notes and references

More competency

Posted on by

The CEN WS-LT Competency SIG discussions of a conceptual model for skill/competence/competency are still at the very interesting early stage where very many questions are open. What kind of model are we trying to reach, and how can we get to where we could get? Anything seems possible, including experiments with procedures and conventions to help towards consensus.

Tuesday, December 1st, Berlin — a rainy day in the Ambassador Hotel, talking with an esteemed bunch of people about modelling skill/competence/competency. I won’t go on about participants and agenda — these can be seen at http://sites.google.com/site/competencydriven/ It was all interesting stuff, conducted in a positive atmosphere of enquiry. I’ll write here about just the issues that struck me, which were quite enough…

How many kinds of model are there?

At the meeting, there seemed to be quite some uncertainty about what kind of model we might be trying to agree on. I don’t know about other people, but I discern two kinds of model:

  • a conceptual model attempting to represent how people understand entities and relationships in the world;
  • an information model that could be used for expressing and exchanging information of common interest.

A binding isn’t really a separate model, but an expression of an information model.

My position, which I know is shared by several others, is that to be effective, information models should be based on common conceptual models. The point here is that without an agreed conceptual model, it is all too easy to imagine that you are building an information model where the terms mean the same thing, and play the same role. This could lead to conflict when agreeing the information model, as different people’s ideas would be based on different conceptual models, which would be hard to reconcile, or even worse in the long term, troublesome ambiguity could become embedded in an information model. Not all ambiguity is troublesome, if the things you are being ambiguous about really share the same information model, but no doubt you can imagine what I mean.

Claims and requirements for competence

A long-term aim of many people is to match what is learned in education with what is required for employment — Luk Vervenne was as usual championing the employer point of view. After reflection on what we have at the moment, and incorporating some of Luk’s ideas in the common information model I’ve been putting together, I’d say we have enough there to make a start, at least, in detailing what a competency claim might be, and how that might relate to a competency requirement.

In outline, a full claim for a single separate competence component could have

  • the definition of that component (or just a title or brief description if no proper definition available)
  • any assessment relevant to that component, with result
  • any qualification or other status relevant to that component (which may imply assessment result)
  • a narrative filling the gap between qualifications or assessment and what is claimed
  • any relevant testimonials
  • a record of relevant experience requiring, or likely to lead to, that competence component
  • links to / location of any other relevant “raw” (i.e. unassessed) evidence

I’ll detail later a possible model of competency requirements, and detail how the two could fit together. And I have now put up the latest version of the big conceptual model as well. There is clearly also a consequent need to be clearer about the structure of assessments, and we’ll be working on that, probably both within CETIS and within the CEN WS-LT.

What about competencies in themselves?

Reflected in the meeting, there still seems to be plenty of disagreement about the detail that is possible in an information model of a competency. Lester Gilbert, for example put forward a model in which he distinguished, for a fully specified educational objective:

  • situation;
  • constraints;
  • learned capability;
  • subject matter content;
  • standard of performance;
  • tools.

The question here, surely, is, to what extent are these facets of a definition (a) common and shared (b) amenable to representation in a usefully machine-processable way?

Personally, I wouldn’t like to rule anything in or out before investigating more fully. At least this could be a systematic investigation, looking at current practice across a range of application areas, carefully comparing what is used in the different areas. I have little difficulty believing that for most if not all learning outcomes or competency definitions, you could write a piece of text to fit into each of Lester’s headings. What I am much more doubtful about is whether there is any scheme that would get us beyond human-readable text into the situation where we could do any automatic matching on these values. Even if there are potential solutions for some, like medical subject headings for the subject matter content, we would need these labels to be pretty repeatable and consistent in order for them to be used automatically. And, what would we do with things like “situation”? The very best I could imagine for situation would be a classification of the different situations that are encountered in the course of a particular occupation. In UK NOSs, these might be written in to the documentation, either explicitly or implicitly. Similar considerations would apply to Lester’s “tools” facet. This might be tractable in the longer term, but would require at least the creation of many domain-specific ontologies, and the linking of any particular definition to one of these domain ontologies.

I can also envisage, as I have been advocating for some time, that some competency definitions would have ontology-like links to other related definitions. These could be ones of equivalence, or the SKOS terms “broadMatch” and “narrowMatch”, in cases where the authorities maintaining the definitions believed that in all contexts, the relationship was applicable.

What about frameworks of skill, competence, etc.?

It surprised me a little that we didn’t actually get round to talking about this in Berlin. But on reflection, with so many other fundamental questions still on the table, perhaps it was only to be expected. Interestingly, so far, I have found more progress here in my participation with MedBiquitous than with the CEN WS-LT.

I’ll write more about this later, but just to trail the key ideas in my version of the MedBiquitous approach:

  • a framework has some metadata (DC is a good basis), a set of competency objects, and a map;
  • the map is a set of propositions about the individual competency objects, relating them to each other and to objects that are not part of the framework;
  • frameworks themselves can be linked to as constituent parts of a framework, just as individual competency objects;
  • it is specified whether to accept the relationships defined within the competency objects, and in particular any breakdown into parts.

The point here is that just about any competency definition could, in principle, be analysed into a set of lower-level skills or competencies. This would be a framework. Equally, most frameworks could be used as objectives in themselves, so playing the same role as an individual competency object, being part of a competency framework. If a framework is included, and marked for including its constituent parts, then all those constituent parts would become part of the framework, by inclusion rather than by direct naming. In this way, it would be easy to extend someone else’s framework rather than duplicating it all.

Need for innovations in process and convention

Perhaps the most interesting conclusion from my point of view was about how we could conduct the processes better. There is a temptation to see the process as a competition between models — this would assume that each model is fixed in advance, and that people can be objective about their own, as well as other people’s models. Probably neither of these assumptions is justified. Most people seem to accept the question as “how can a common conceptual model be made from these models?”, even though there may be little wisdom around on how to do this. There is also the half-way approach of “what common information elements can be discerned between these models?” that might come into play if the greater aim of unifying the conceptual models was relinquished.

From my point of view, this brings me back to two points that I have come to recognise only in recent months.

This meeting, for me, displayed some of the same pattern as many previous ones. I was interested in the models being put forward Luk, and Lester, and others, but it was all too easy not to fully understand them, not quite to reach the stage of recognising the insights from them that could be applied to the model I’m continuing to put together. I put this down to the fact that the meeting environment is not conducive to a deep mutual understanding. One can ask a question here and there, but the questions of others may be related more to their own models, not to the relationship of the model under discussion with one’s own. So, one gets the feeling at the end of the meeting that one hasn’t fully grasped the direction one should take one’s own model. Little growth and development results.

So I proposed in the meeting what I have not actually proposed in a meeting before, that we schedule as many one-to-one conceptual encounters as are needed to facilitate that mutual growth of models at least towards the mutual understanding that could allow a meaningful composite to be assembled, if not a fully constituted isomorphism. I don’t know if people will be bold enough to do this, but I’ll keep on suggesting it in different forums until someone does, because I want to know if it is really an effective strategy.

The other point that struck me again was about the highest-level ontology used. One of the criteria, to my mind, of a conceptual model being truly shared, is that people answer questions about the concepts in recognisably similar ways, or largely the same on a multiple choice basis. Some of those questions could easily relate to the essential nature of the concept in question. In the terms of my own top ontology, is the concept about the material world? Or is it a repeatable pattern, belonging to the world of perception and thought? Is it, rather a concept related to communication — an expression of some kind? Whether this is exactly the most helpful set of distinctions is not the main point — it is that some set of distinctions like this will surely help people to clarify what kind of concepts they are discussing and representing in a conceptual model, and thus help people towards that mutual understanding.

A similar, but less clear point seems to apply to relationships between concepts. Allowed free rein in writing a conceptual model, people seem to write all kinds of things in for the relationships between concepts. Some of them seems to tie things in knots — “is a model of” for instance. So maybe, as well as having clear types for concepts, maybe we could agree a limited vocabulary for permitted relationships. That would certainly help the process of mapping two concept maps to each other. There are also two related conventions I have used in my most recent conceptual model.

  1. Whole-part relationships are represented by having contained concepts, of varying types, represented as inside a containing concept. This is easy to do in CmapTools. Typically the containing concept represents a sub-system of some kind. These correspond to the UML links terminated by diamond shapes (open and filled).
  2. Relationships typically called “kind of” or “is a” correspond to the UML sub-class relationship, given with an open triangle terminator. As these should always be between concepts of the same essential type, these can be picked out easily by being a uniform colour for the minimized and detailed representations of the whole.

So, all in all, a very stimulating meeting. Watch this space for further installments as trailed above.

Consensus process and conceptual models

Posted on by

I was in Umeå last week — at the NORDLET conference — but also there were lots of ISO SC36 people coming for their meeting. Now SC36 is trying to put forward a “PDTR” — draft technical report — “ITLET – Conceptual reference model for competencies and related objects” which I have been interested in for a while, as it would be nice to use this opportunity to broaden consensus about competence and related terms. There is a diagram representing a current proposal, and this is part of a set of documents tracing the development of this work, which up to now I was not aware of.

You know the basic idea about the standardization process — there is meant to be established practice in the area which would benefit from standardization. But, as I am just discovering, there is another kind of ISO document, the Technical Report. This one is of “type 3, when the joint technical committee has collected data of a different kind from that which is normally published as an International Standard (“state of the art”, for example).” And so we could at least think over, what is the appropriate time to try to agree a conceptual model? What is the appropriate process?

My experience and intuition both tell me that it takes a lot of work to agree on a shared conceptual model. People almost always build up their own private conceptual models of any area they are involved with. Concepts are used in discussion among groups, and these concepts become important to the group members, as they are part of their language, and their means of communication. Put two groups together, attempting to merge their conceptual models, and you are likely to find people defending their corners. That’s not consensus process, and it could easily result in a botch of a conceptual model that satisfies no one at all.

But put two individuals together, free from the constraints of being answerable to a group, and a wholly different process can take place. Instead of communication being a prompt to defend known terms, it is naturally a motive to explore the other person’s terms and meanings. And the prize at the end of a deep exploratory dialogue of this kind could be a shared conceptual model — not necessarily where the facts in the model are agreed, but at least where the language is agreed in which it is possible to disagree comprehensibly. And it is just during these dialogues — again, in my experience — that each person’s conceptual model is able to grow.

It seems like a good idea, therefore, in any case of trying to agree a conceptual model, to base the consensus process around dialogues between pairs of people. The time needed can seem long. My (good) experience of talking with Simone Laughton — an SC36 member from Canada — suggests that it takes at least several hours to get a good enough understanding of someone else’s conceptual model, at least of the type that SC36 are trying to agree on, if one wants to create a shared conceptual model.

So, contrast two alternatives. Firstly, the existing committee-based processes, which seem to run up again and again into the difficulties of disagreement, perhaps because of the mechanisms suggested above. Secondly, deep pairwise dialogue between all the participants who feel they have a position on a conceptual model. The second option is front-loaded — that is, it seems to need more work up front — but my guess is that once that work had been done — once people have had the experience of the development of their own model through dialogue — the committee process would probably be very quick and painless. And, maybe, the process might even take less time overall, as well as being more likely to result in a genuinely shared model.

Worth a try?

Worth a try also, perhaps, in areas that might be of similar complexity, such as the attempt to agree a management or governance structure between people in the same organisation, or with a common interest?