Category Archives: CETIS Website

PROD’s Progress

Posted on by

Apart from the previous post about the OpenID implementation it has been a while since I’ve written about PROD so here is the “vision” and some details of what’s happening with the project.

Before we go on I’ve written an FAQ on the PROD wiki which you are all advised to have a look at…

The PROD Vision:

PROD is a dynamic directory of JISC projects providing an easy-to-use way to locate projects and get a view of their current status and activity. Through integration with the Standards Catalogue and e-Framework it will also provide an overview of interoperability standards used by projects and their rationale for doing so.

PROD draws information on projects from a number of sources including the JISC website, individual project sites and project RSS feeds. We have also developed import mechanisms for legacy spreadsheets and catalogues.

The data in prod can be exported in standard formats (including RSS, ATOM, DOAP and CSV) to facilitate re-use in other catalogues.

Progress report

People oriented activities:

We are currently looking at how this data can facilitate integration with efforts at OSSwatch and with the JISC PIM system. We had a meeting in London to discuss how we can leverage doap across the different systems to exchange data and avoid duplication of effort. Present included Ross Gardler from OSSwatch with SIMAL, Yvonne Howard and Dave Millard from Southampton with their e-Framework Knowledge Base, Neil Chue Hong from OMII in Edinburgh, and Simone Spencer who is heading up the JISC PIM. It was pretty satisfying to feel we all agreed that with a bit of work on our respective DOAP implementations we would be able share core project data and thus concentrate on the more individual value-adding aspects of our projects.

Here in Bolton we are holding a workshop tomorrow on how we plan to use PROD internally to help us with the process of ”technical audits” of projects and how we can go about integrating PROD with the other JISC CETIS web offerings.

Ongoing development work:

DOAP, RSS & CSV export for collections of projects through the browse/query interface. We’re also thinking about making widgets to embed this in other places (like the main JISC CETIS site – or your own personal iGoogle or Dashboard if you like!)

OpenID associations for existing users – this is part of the general OpenID implementation across JISC CETIS sites. Currently it works to enable commenting.

Selectively elevated privileges for project staff and programme managers. This will happen automatically through existing data where available, we will also put in a “claim” button to users to assert a relationship to a project where a connection is not already held.

General review of data held, sanitisation particularly around people, organisations, themes. This will include a manual trawl for project sites, feeds etc where they haven’t been auto-discovered. Administrative interfaces may also see some improvement.

Integration with Standards Catalogue. Users (CETIS staff, projects, etc) will be able to associate projects with relevant standards and comment on the rationale for their use or implementation. The standards catalogue bit is working fine now.

Integration with main JISC CETIS sites – highlighting relevant projects within domain pages and other CETIS output (blogs, e-learning focus etc). This activity will be of particular relevance to ongoing comms work including the “technology & standards briefings”.

Highlights of completed development work to date:
(Roughly in order of implementation)

  • Core data model
  • Core interface
  • Old directory import
  • JISC spreadsheet import
  • DOAP export
  • Search interface
  • Funding status indicators
  • AJAX editing (administrators only at the moment)
  • JISC web-scraper
  • RSS feed-scraper
  • Data-sanitisation utilities (for admins)
  • Activity indicators
  • Comments
  • Browse & querying interface
  • OpenID authentication (for commenting)

Down and dirty with OpenID

Posted on by

I’ve spent the last few hours (after getting home from a swift pint in the pub admittedly) having one of those satisfying coding experiences where the dots just start joining up… I took the very nicely written OpenIDenabled PHP library and bolted it on to the authentication routines for PROD.

The technical principles behind OpenID are simple enough: the user tells your application their openid URL, the app asks the relevant provider if everything is ok, the provider comes back and tells the app a whole bunch of stuff saying that the user is kosher (or halal or whatever it says in their profile).

The latest version of the toolkit made this a breeze – coming as it does with working examples and very well documented code. Most of the work was putting in a few new hooks in my authentication script to catch both ends of the transaction, copying and pasting some code from the example scripts to create the consumer object and set it flying and finally catching the response at the end and telling my application that the user is now logged in.

As with most quick work there is still quite a bit tidying up to do – particularly around how I associate existing users in the LDAP directory with their OpenIDs… At the moment I’m just not bothering. Useful error messages would probably be a good idea too! Testing it with a few different providers is also a must.

One gotcha I discovered was that at some point the exact recipe for doing Delegation must have changed and that the library is more fussy about this than other implementations I’ve seen and used. When testing using my own domain’s delegation which I’ve had set up for years it was consistently failing. This is not good news as there are probably thousands of people who still have it set up exactly as I did…

Another (Ubuntu specific) issue was that it was failing to authenticate against yahoo’s service because I was missing some bits of openssl… This was fixed with a quick sudo apt-get install openssl ca-certificates

Now I’ve had a few brushes in recent months with OpenID mainly around the web provision for the XCRI project – where we got OpenID working across WordPress, Mediawiki, and (through some rather cheap hacking) BBpress. It was however reliant on plugins for said apps and never really a very satisfactory experience – generating a long string of complaints from users getting very variable results depending on which provider they were using. Upgrading any particular component of the site seemed to just lead to more chaos.

Sadly I think that these variable experiences do rather detract from the potential that OpenID has to help us all better manage our online identities. That and the insistence of so many “providers” like Yahoo! and WordPress.com that they are just that, providers and not consumers. I’ve already got about 6 OpenIDs on the go without really realising – useful for testing but the exact opposite of the single authentication service goal. Tsk tsk.

Anyway… Now that I’ve actually tackled the problem at a slightly deeper level I’m feeling confident that over time we can not only iron out XCRI’s woes but also introduce OpenID across the JISC CETIS (and IEC) services in a reasonably robust way. The future looks rosy, the sky is blue, thunderclouds? What thunderclouds?

LDAP Disaster

Posted on by

On Monday afternoon I updated various packages on our Fedora Core 5 server using yum. This has in the past caused one or two little tragedies. Really I should know better and do such updates over the weekend but of course I went ahead all gung-ho.

The vital mission critical thing that died this time was the OpenLDAP server which runs authentication across all the CETIS sites. No-one could get in to edit the wikis or blogs or a whole bunch of other services which is pretty disastrous really.

I scratched my brain for all of Tuesday and even a few hours on Monday night – trying to figure out what had happened. Basically it seemed that all the data in the openldap database had disappeared. I could connect to the server but it was unable to list the nodes of the directory. I tried a few command-line diagnostic tools. slapcat produced absolutely no output slapd_db_recover happily recovered something but made no difference whatsoever. Doing an ldapsearch (which should dump the whole dataset) did the following:

[root@arwen ldap]# ldapsearch -x
# extended LDIF
#
# LDAPv3
# base with scope subtree
# filter: (objectclass=*)
# requesting: ALL
#

# search result
search: 2
result: 32 No such object

I started off thinking that my config files were knackered – so I pawed over ldap.conf and slapd.conf for hours – and nothing changed. I did notice that there was an /etc/ldap.conf as well as an /etc/openldap/ldap.conf. I compared the two and removed the one loose in /etc as it seemed wrong. Didn’t help.

Next I got drawn down a big red-herring as I noticed messages in the logs when starting slapd:

Jun 13 12:01:32 arwen slapd[18004]: sql_select option missing
Jun 13 12:01:32 arwen slapd[18004]: auxpropfunc error no mechanism available
Jun 13 12:01:32 arwen slapd[18004]: auxpropfunc error invalid parameter supplied

Several sources claimed that this was to do with permission problems and SASL – but it turned out that it was completely unrelated to my actual problem and could be safely ignored. Again I wasted loads of time reading about SASL and chmodding files everywhere. I suppose it might become important were I ever to decide to actually use SASL with the directory.

So where had my data gone? This morning while on a conference call I was idly noodling through the database files in /var/lib/ldap and noticed a directory called rpmorig which I hadn’t really been through. I looked and I saw and I suddenly realised that there were a lot more .bdb files in there than there were in the parent directory and that they were full of data. The penny dropped. yum had kindly backed up all my data into this directory and replaced the working files with fresh empty ones. I moved the rpmorig directory into the place of /var/lib/ldap, restarted slapd and behold EVERYTHING WORKS AGAIN.

I curse whoever put together that yum package.

CETIS-Redux

Posted on by

A year has passed since I started thinking about the redesign of the CETIS website, and inevitably now that the whole thing is creaking into some semblance of what Scott and I had originally intended it has been time to go back to basics and re-examine what we thought we were doing, why we are doing it, whether it is working and what on earth we are going to do next.

There are a few processes going on; Mark, Sharon and Adam have been conducting a review of the community wiki aspects of the web presence and the e-learning focus team are considering where to go next with their magazine-style site with a view to merging it together with the JISC-CETIS page. The Communications team has been discussing the whole show from start to finish and back again.

My thoughts

1) You are in a twisty turny maze

There is a tendancy for people to get lost in the site. Removing some navigational elements (especially in the wiki) and applying some others (breadcrumbs, menus etc) in a coherent way will make a difference I hope – but the main proposal is as follows:

Merge the www.cetis page, the jisc.cetis page and the elearning focus site into one coherent magaziney all-singing portal.

Front page mockup v3

(old versions: v1 v2)

The mockup (v3) shows the main elements we have identified – with a monthly editorial, regularly changing “features”, and constantly changing “news”. It also gives prominence to the SIGs with a prominent bar on the left hand side…

To start off with, the news and features would actually be drawn from the blogs as the current aggregation is – only the editiorial process will be stepped up – with lead-ins and article filtering-selection done by the focus team. As is done with e-learning focus, articles may also be commissioned by external writers.

V2 has a horizontal-slice approach; Banners | Navigation | Editorial | News etc (3 streams) | Other stuff (4 streams)

V1 is an earlier attempt – and is more like the aggregation as it stands at the moment.

2) Re-work the SIG entry points

We made a decision quite early on that the SIGs would simply have a protected wiki page each to serve as their main site – giving them total flexibility to do whatever they wanted. Of course this approach led to inconsistency and an extra learning curve for staff. The result was certainly not easy to follow for the outside observer and generally quite unsatisfactory to my mind.

sig-page.pdf
I have in fact started working on it

So the plan would be coherent entry points combining the main details of the sig, good quality linkage with the events system and drawing in content from the main aggregation and wiki (by tag naturally). Crucially the co-ordinators need customisable space to do with what they wish whether it be posting up some powerpoints or pointing to some interesting resources online somewhere – I need another pass of Mark and Sharon’s work as well as a round of discussion with co-ordinators to figure out exactly what really needs to be done.

3) The project tracker

Already discussed somewhat in my post about doap and ohloh I have a chunk of time set aside for re-working the project tracking system so we can reliably keep tabs on what is going on with JISC projects. Again this needs to be nicely integrated.

Example Project page

One last thing

There is one last thing. I propose removing all traces of rounded corners in favour of the infinitely superior square corners. Trendy design at it’s best don’t you think ;)