StandardConnection – an interesting NSDL project

Posted on by

StandardConnection:Mapping NSDL Educational Objects to Content Standards.
PHASE II: An Achievement Standards Network Architecture
Standard Connection is a NSDL service project.

http://www.thegateway.org/asn
http://projects.ischool.washington.edu/sasutton/NSDL/StandardConnection/
The short version:

Following on from a project examining automatically matching learning resource descriptions with curriculum components, StandardConnection Phase2 is developing a machine addressable registry and repository of content standards and tools to use it. A content standard may perhaps be equivalent to a national curriculum programme of study in the UK. For example, in fifth year Jimmy will study algebra, one of the modules will be on quadratic equations.

Among other things the tool would allow a user to compare and match syllabi in different school districts or for different exam boards.

The longer version:

During a trip to Seattle I had the opportunity to have a chat with Stuart about some of the projects he’s working on. In the course of our discussion he talked a lot about this current project and noted that, irrespective of all the interesting and useful projects he has worked on, this project will probably be the one that has the biggest impact in the real world. If the automatic tagging works it will certainly be great, but in my opinion what it has already done is quite significant.

The project aims to “define and create:

  1. A machine-addressable repository of state and national organization academic content standards;
  2. A machine-addressable registry of alignments that relate or align those standards to each other;
  3. A cataloging specification using internationally accepted technical standards for metadata tagging for describing and locating NSDL and other resources according to state and national organization academic content standards: and
  4. A set of protocols, schemas, and program interfaces that will enable these services to communicate and resolve queries against the repositories.”

As interesting as the investigation of automatically assigned metadata in phase 1 may have been, the really cool part about this new phase is that they have put together a system that supports machine addressable mappings between different content standards – and they have some 450 such standards already stored.

This supports users if they want to compare and match standards – to translate the example into British terms: how does Kelly’s Higher compare to the first year of Mike’s A-level? And in the longer term this means that teachers dealing with young people switching between schools which follow syllabi from different different courses of study can gain an overview of what they should have studied up to that point.

A real world use Stuart cited related to a difference between a regional and federal standards for a certain grade. In the US, most schools are certified by the state and their syllabi (standards) set by state or district. If they want to receive federal funding however, the pupils progress has to be assessed at different grades (say in their 4th and 8th year) by standardised testing against a federal standard. Looking at one subject area, schools in one district where consistently performing very well against the state standard but poorly against the federal one. The registry supported a detailed comparison of the standards and demonstrated that the teaching was fine – the standards were just out of sync. If the assessment had been held at the end of the 5th year the pupils would have covered the same ground in both standards – they just covered material in a different order. The level of detailed comparison required to carry this out was made significantly easier by the registry.

In a highly mobile society this sort of development should make it much easier for pupils moving between school systems and those trying to teach them.

As an aside: Stuart’s also working on NSDL Registry: Supporting Interoperable Metadata Distribution.

Automatic Metadata Generation

Posted on by

Discussions about automatic metadata generation seem to conflate a number of different processes. As I see it there are a number of separate but related areas of interest in the area of ‘automatic’ metadata generation; these are:

  • The automatic population of record values based on collection-level profiles
  • The automatic population of record values based on user profiles
  • The automatic population of record values derived from other created metadata values
  • The automatic generation of metadata from existing structured information about the asset
  • The automatic generation of metadata from the use of the asset itself
  • The automatic generation of metadata from the content of the asset itself

The automatic population of record values based on collection-level profiles (bulk transforms)

By this I mean the addition to records of values which are fixed attributes of the collection they are part of. The collection could be the complete set or records in the repository or any meaningful or useful subset. This automatic population of values could part of a bulk import process or part of a record export process (singly or in bulk). Such profiles are particularly important when considering the movement and interoperability of metadata – some metadata may be unnecessary in a local context but vital in a wider context.

For example, the university’s name could be added to the publisher details when a set of metadata about institutional minutes is exposed for harvest. Another example would be the automatic addition of educational level information to a set of learning objects designed by high school teachers for local use but which need to be tagged at that level as their metadata is available in aggregator services.

The automatic population of record values based on user or object profiles

This process uses a user or object template to provide automatic or default (i.e. editable) values to a metadata record. It is most likely to be part of the record creation process.

Arguably this is very similar to 1. I’ve separated them to try to draw out the distinction between what is a collection-based transformation and what is essentially a stored template.

For example, the system automatically adds the author’s details and institutional affiliation to a record when the user starts to create it. Or for example the system has a departmental profile for adding a thesis to the system which provides standard information about the department and the university.

The automatic population of record values derived from other created metadata values

This involves the direct derivation of metadata values from user created values. This process may involve the use of external services. This is distinct from a profile in that a look up / or web service action is required based on data entry.

For example, creating a metadata entry for (or link to) representation information based on the supplied file type or including the author’s email address based on their name.

The automatic generation of metadata from existing structured information about the asset

This involves utilizing the supplied metadata or other structured information with the asset to attempt to generate further metadata about an asset.

Where I feel moves into generation (3 still population as it’s a direct inference; 4 involves a step beyond this)

For example, mapping a paper to a subject area based on what a system can infer from publicly available information about an author, (e.g. where they work, the courses they are associated with, the research funding they’ve obtained, previous items they’ve submitted to the system). Another case would be the assignation of an audience or educational level to a learning object based on the same available information.

The automatic generation of metadata from the use of the asset itself

This covers a spectrum of techniques based on the use of the asset. Common approaches range from deriving an inference of quality from usage statistics, citations, or links to inferring metadata more directly from the analysis of user reviews and annotations, feedback, or tagging patterns.

For example, Google, Amazon recommender system, citation metrics (not entirely sure I’d put this here),

This is arguably not formal metadata (though the LOM allows for user annotations to be formally included), but it increasingly plays a role in the management and discovery of assets and is likely to continue to do so.

The automatic generation of metadata from the content of the asset itself

The Holy Grail of automatic metadata generation is often regarded as the ability to remove the need for manual metadata creation. Whether or not this is always desirable, there is a greater certainty that it should be possible to derive more information from the asset itself – especially when the asset is text-based. This process can involve trying to identify structured information contained with the asset, or more basically it can attempt to derive structured information from the entirety of the asset.

For example, tools which mine pdfs or scrape web pages in an attempt to identify the title, author, and references contained in a paper, or tools which count the words in document to try to match it to a subject term.

Repositories Research Team

Posted on by

The short introduction to my job is that I’m the JISCCETIS representative on JISC’s Repositories Research Team (RRT).

Within the team I have a focus on e-learning and provide a degree of support for the e-learning projects in the DRP Programme. As this programme comes to an end and we begin to work more with the Repositories and Preservation strand projects the team’s focus will shift to more research and synthesis and we will have fewer direct support commitments. In the ongoing research of the team I provide a perspective from the JISCCETIS community and will also be focusing on liasing with the eframework.
The team’s self-description from the RRT wiki is:

Repositories Research Team
As part of the Digital Repositories Programme, JISC have established the Repositories Research Team. The remit for the work of the research team is quite wide and includes helping projects find and exploit synergies across the programme and beyond, gathering scenarios and use cases from projects, liaising with other national and international repositories activities, including liaison with the e-Framework, synthesizing project and programme outcomes, and engaging with interoperability standards activity and repository architectures.

The Repositories Research Team is a collaboration between UKOLN and CETIS. UKOLN have worked previously on repositories in a number of contexts including ePrints UK, the Open Archives Forum and Delos, and CETIS, the Centre for Educational Technology Interoperability Standards, has considerable experience in supporting the development of digital repositories for e-learning.
http://www.ukoln.ac.uk/repositories/digirep/index/JISC_Digital_Repository_Wiki

Hello world! – what is a work blog?

Posted on by

A first post; Welcome to my blog.

I’ve avoided blogging so far in life, but it now seems to be part of my job.

I’m not sure if I’m relieved or confused by this. I’m perhaps relieved as it gives me an excuse to write reflectively without having to justify why I should inflict my musings on the world. I’m confused because what is a work blog?

There seem to be two types of blog: filter blogs and journal blogs. I suspect that most work blogs tend to be filter blogs – collections of briefly annotated links. Journal blogs being more reflective and often more personal are less suited to the work environment. That said, if I think about the blogs I read, it’s the work blogs that blend the two types that I find myself coming back to. But if I look again at who blogs in this manner most of the people have established reputations – they blog the way they speak or present.

I’m not sure what sort of blog I’ll end up writing as part of the point of my work blog is to talk about what I’m doing. Oh well, I’ll figure it out as I go.

I’ll introduce myself and my job a little more thoroughly in my next post.