I am the aforementioned Pat.

Working from the topics listed in the paper / from my experience with xpert

Item identification
——————-

Xpert works on the basis that the URL for each item is unique (well it best be) so this is the “key” in database terms. This is how we distinguish between one item and another.

Item updates
————

If you’re harvesting URLs, I don’t think there is an issue with this. I would assume that all feeds are very much alive and continually changing or being modified.

Xpert empties all its metadata each night and reharvests each time, so it takes a new copy of the metadata each time. So if the metadata has been updated, xpert will reflect that.

I would agree with Jenny that any specific information is going to be confusing to end users, and possibly limit the ability to submit the same feed to other repositories.

Item deletions
————–
As xpert deletes its metadata every night, an item being taken from the feed will still be in the system (although we record when it was last found in a feed) but will be less likely to be found in a search result.

We’ve not explicitly had a problem with this though

Missing items
————-
I’m covering this later in a new section, but feeds can contain a tag on how often they should be harvested.

Polling period
————–
See above, but we harvest once a day. Each day brings about 10 or so new items.

Feed formats / Metadata formats
——————————-
DC seems by far the most widely used. I would worry more than people will stick to it and be consistent with data inputted.

Repository Required Metadata Profile
————————————
Xpert doesn’t have a minimum setting, other than it needs to have a link. Items with less metadata will just be found less often, but I wouldn’t not harvest just because metadata was sparse – people may still find it.

Licensing Content
—————–
You can have an overall license per feed, or information per item. I am not sure why subsequent release is an issue because they are already released?

Links or resource download
————————–
We only take links, no downloading occurs.

Other issues
————

Who is the end user? – I think it’s acceptable to have an RSS feed for harvesters and an RSS feed for people. Most feeds contain all their site’s content (I think only a few limit) and so are well suited to being harvested without any worries over missing items due to harvest frequency. That to me is better than trying to make one feed for all.

It might be logical that if we are making a second RSS for harvesting we might use some other technology instead.

Feeds that aren’t valid – Of all 60 feeds xpert takes, 5 aren’t valid xml, and approximately 20 aren’t valid RSS. The harvesting service has to be flexible to deal with this. It also needs to like lots of character sets, as a lot of content isn’t in English

Format of presentation – I think a link should go to a page. Some xpert links prompt for download, and some go off to a scorm package – which when accessed outside a scorm client / service looks absolutely awful to an end user (and would almost certainly put them off using it). It’s not an ideal situation.

Quality of metadata – It’s often very thin indeed

Author format – preferred way of displaying author

Making metadata consistent – one problem we have in xpert is the different forms of attribution – some organisations claim DC:Creator as them, some allow that for the individual creator. This makes presenting meaningful, consistent data to someone searching the data awkward.

DC nodes – Personally, i think they are a bit weak, and more categories are needed for more relevant information (level, accessibility, duration).

OAI-PMH SWORD etc are big technical barriers for many people who have resources to expose – anyone can make a feed. Along with some good guidelines on how to best use the feeds, this surely presents a good opportunity for Jorum, especially while significant collections such as OER Recommeder continue to use feeds, and people find it easy to use?

First let’s look at RSS feeds. In their favour we have:

– They are commonly used and familiar to everyone

– Many repositories can expose resources through RSS feeds

– They offer “continuous update”

However, The implementation suggested for JorumOpen negates many of these benefits:

– The standard form of an RSS feed is not suitable for ingest into JorumOpen because it requires suitable metadata and licence information to be added to the feed. This would require everyone who creates feeds to ensure they are enhanced for JorumOpen and will require clarity on what metadata formats are required and what minimum fields are needed for acceptance into JorumOpen. How are people to know if their feeds do not meet these minimum standards?

– Who can submit feeds? Presumably resources in feeds can be submitted on behalf of others. The metadata will need to make clear who are the owners of resources.

– The “continuous update” is not implemented as stated by Laura Shaw “The feed isn’t continually polled for new content (and obviously no functionality for deletes/updates within a feed)”

– “Items within a feed are not auto classified within Jorum.” Even if the classification information is provided within the metadata?

If no other alternatives are available then RSS feeds are perhaps a suitable quick solution.

However, given the availability of OAI-PMH and SWORD it could be quite straightforward to harvest metadata from one repository in the required metadata format (and to set up periodic updates) and to deposit these metadata records using SWORD, ensuring that all the metadata from the original resource is preserved – including classification metadata and ownership information.

Licence information is always likely to be more problematic and as a CC licence will need to be selected for JorumOpen.

The paper concludes that the OAI-PMH metadata harvesting tool is a better option for JorumOpen. I’d strongly agree. I realise it is technically more challenging to offer an OAI interface, but there are many opensource libraries available which you can plug in to java or php apps.

There’s no well-maintained LOM extension for RSS which makes it a risky implementation if you want your feeds to pass an external validation too. OAI would also allow you to have a custom xml format for import into Jorum if you felt it necessary.

I was concerned by the authors suggestions for item update identification. As he states, checksums etc will not work for website notification harvesting. But adding custom text like “jorum:update” to an RSS feed will annoy all the other users of my feed and pollute my data in their search results. Shouldn’t we just recommend people use the existing date tags appropriately and compare a date stored in the respository with the date on the feed?

Deletion of an asset is a tricky area. OAI would let you mark an item deleted. I’m more worried by the suggestion that a feed has a finite amount of items in it, and I can see you are too as many of the other problems listed in the drawback stem from this assumption. There’s no such restriction in the RSS standard and no expectation of a limited list in other OER aggregators like OERRecommender who expect you to list everything you’ve got. Have you talked to the developers of that tool to see how they handle updates, deletions, missing items etc? I know it was knocked together pretty quickly, but I expect they’ll have thought of these things. I can put you in touch if you need an email as Joel doesn’t work for USU COSL any more I don’t think.

One final thought – if you get everyone to offer a single feed format into JorumOpen, then one advantage of JorumOpen can be to offer all the other feed formats so that all the UKOER projects can be aggregated elsewhere.