Student Retention: Engagement Analytics at University of Derby

Photo of a brass compassThe University of Derby has been scoping early indicators (engagement analytics) for spotting students at risk of withdrawing in their SETL (Student Experience Traffic Lighting) project.

Challenges

Institutions generally hold a vast array of data about students, often in different systems which are not always interoperable. The data challenges experienced in this project include:

  • predisposing factors, such as responsibility as a carer, means that students are more likely to withdraw from their studies, however it’s not always possible to capture this type of data as it’s not generally held in any IT system
  • there is little interoperability between different data systems; for example, the data required to populate an engagement dashboard is held in at least seven different systems at the University
  • each student is an individual who brings with them individual challenges to succeeding at and engaging with higher education

Benefits

Scoping out the type of data to be included in a dashboard of core engagement data:

  • means that staff would be able to view a student’s level of engagement with the institution; for example, linking data on absences, access to the library and the VLE (Virtual Learning Environment) could help a tutor see if the student was still engaging in the course, even if they were absent due to illness
  • has produced a change of thinking in the way students at risk of withdrawing will be supported at the University; i.e. it will be more proactive than reactive
  • has helped staff identify key points in the student lifecycle where students are most likely to be at risk of withdrawal.

Recommendations

Engagement analytics goes beyond the hard data recorded in learning analytics, because:

  • it’s dangerous to make decisions about student engagement based solely on a set of data, as understanding the context of the data is important and developing the relationship between the tutor and student is essential
  • both staff and students find it useful to have their own customisable engagement dashboards
  • soft data that can’t always be found on institutional systems should also be recorded and considered.

Further Information

If you would like to find out more about this project, the following resources may help:

Engaging Alumni: ALERT at Brunel University

Photo of graduates wearing mortar boards and gownsBrunel University’s “ALERT (Alumni Growth and Engagement across New Technologies)” project has been exploring the use of social networking technologies to provide e-mentoring between alumni and undergraduates as a means to improve employability.

Challenges

The main challenges relate to the difference in the use of social networking technologies by alumni and students:

  • students are hesitant to join LinkedIn and are more likely to belong to Facebook
  • alumni in employment are more likely (95% of the Brunel sample) to belong to LinkedIn and prefer to use LinkedIn rather than Facebook for e-mentoring
  • the uptake of professional business-oriented services is low amongst undergraduates
  • alumni were not mentioned as a source of advice (employability or career development) during the customer journey mapping exercise.

Benefits

Strengthening ties between alumni and their institution by encouraging engagement with current students can help:

  • provide a sense of belonging and reinforce positive feelings for both alumni and students
  • build mutually beneficial relationships, which improves business and community engagement
  • improve student skills and employability; for example demonstrating the value of creating a LinkedIn profile can raise student awareness around self-reflection and self-marketing
  • bring together staff from across the University
  • improve the student experience.

Recommendations

Alumni engagement has a lot to offer in terms of knowledge exchange, relationship building, skills development etc. In order to build the relationship between students and alumni, the project discovered that:

  • engaging alumni should begin as early as possible in the student lifecycle, e.g. by inviting alumni to talk to prospective students
  • it should be made clear to participants how they can contribute and gain from services, such as LinkedIn
  • specific LinkedIn groups (e.g. subject-specific groups) should be established rather than a single university entity, as students prefer to belong to a particular group
  • LinkedIn can be seen as a safe place that supports both students and alumni and this “electronic handshake” can help to build trust within relationships.

Further Information

If you would like to find out more about this project, the following resources may help:

Disenchanted with the world? Try RM for that warm and glowing feeling

Daisy with multi-coloured petals and dewdropsBar a few loose ends, the JISC funded RM (Relationship Management) Programme has now come to an end for the all the project teams involved. We’re just in the process of pulling all the project case studies together and will be producing a Compendium of Best Practice in RM over the next few months, which will showcase some of the key successes of the Programme.

I’ve been supporting the RM Programme since 2009 and feel privileged to be allowed to follow the ups and downs of all the projects involved. Although I’ve been acting as a neutral interface between JISC and the funded projects, to some extent I have been wearing a semi-JISC hat, yet all the project teams have been willing to share their “warts and all” experiences, sometimes privately, sometimes publicly. It’s not always easy to build up a relationship where project teams feel comfortable doing that. Projects have told me things that they wouldn’t necessarily feel comfortable telling their funders (although Myles Danson and Simon Whittemore, the JISC RM Programme Managers are lovely to work with) and this has allowed me to build up a good mental picture of RM across the whole Programme.

I have to say that I have great admiration for all the project teams involved. They’ve worked extremely hard to really try and make a difference to the students and customers in their institutions, and that hard work has paid off. I wish them all success as they continue to improve all aspects of relationship management in their institutions.

Team of people sitting together Aside from improving the customer experience (student or business), for me one of the key successes across the whole of the RM Programme has been that people are now talking to each other: staff talking to students about what they need, staff talking to staff in other departments, and people sharing their experiences with people in other institutions. Relationships have been built. Communication channels have been opened. This is really what relationship management is all about. It’s about breaking down barriers, understanding another person’s point of view, sharing experiences (good and bad) and coming together to try and make things work a bit better for everyone.

Now that the Programme has ended, I will miss following the project work as many of the teams continue their efforts beyond the funding period, as well as missing contact with the individuals involved. From a personal point of view, it’s been a joy and pleasure working with the RM projects, getting to know the people in the project teams, following project successes and failures and seeing everyone safely through to the end. It’s this warm and fuzzy feeling that makes it all worth it. It’s what building relationships (and relationship management) is all about.

Engaging Alumni: AstonConnect+ at Aston University

Photo of graduates wearing mortar boards and gownsThe JISC funded Relationship Management Programme is coming to an end and so I thought that I’d try and do a quick summary of each of the project Case Studies as they come in. I hope that this will provide you with a taster of the work that’s been done by the projects, which we’ll then summarise into a Compendium of Good Practice.

The first Case Study to land on my desk is Aston’s University’s “AstonConnect+ Supporting Mutually Beneficial Alumni Engagement“.

Project Outcomes

The project transformed the current engagement process between students and alumni by:

  • Enhancing the University’s Alumni Community website
  • Developing an international knowledge exchange hub by facilitating three special interest groups in Marketing, Entrepreneurship, and Leadership using LinkedIn
  • Engaging a new group of stakeholders (certificate and diploma students are now included on the Rasier’s Edge database)
  • Establishing an alumni:final year mentoring programme
  • Creating an online searchable archive of the University’s public lectures
  • Creating a showcase of alumni talking about their chosen career with top tips to help final year students transition into work
  • Creating a showcase of successful and up-and-coming alumni
  • Integrating existing systems.

Challenges

There were some unexpected issues (almost all of which were resolved), such as:

  • Maintaining password protection of some content, because of licensing agreements
  • More mentors than mentees for the alumni:final year student programme
  • Low take-up rates for the alumni:alumni programme (so it won’t be pursued at this time)
  • Public lectures had to be edited or reformatted to make them suitable for YouTube
  • Not all alumni who were interested in talking about their career were able to get to campus for filming or were reticent about being filmed
  • Lack of in-house resource for the website redevelopment.

Benefits

Although it is perhaps too soon to record any metrics, the following benefits have been identified:

  • More efficient processes as a result of integrating existing systems
  • Improved ease of use when adding content to the website leading tosavings in time and resources
  • Alumni:final year student mentoring programme now embedded as part of the University’s mainstream activity
  • Creation of a bank of knowledge (alumni talking about their chosen career) to support the student transition into work
  • Bringing together staff from across the University
  • Improving business and community engagement
  • Improving student skills and employability.

Further Information

If you would like to find out more about this project, the following resources may help:

Using Data to Improve Student Retention: More Questions than Answers?

The “Issues around Using Data to Improve Student Retention” session at the JISC CETIS Conference 2012 examined some of the issues around using data to predict students at risk of failure and provide some examples of possible solutions to ensure that the institution does not fail such students.

The session was divided into five mini-sessions run by some of the JISC Relationship Management Programme projects. Here is an overview of some of the solutions being trialled by the projects and the challenges they face.

Loughborough University’s Pedestal for Progression project is using Co-Tutor to collect attendance data. Student attendance at lectures and personal tutor meetings is recorded, so it’s possible to see at a glance whether a student is turning up or if they dislike Monday mornings. The system gives automatic flags and provides a personal tutor with a red, amber or green visual of attendance (traffic lighting). As a result of introducing registers, attendance has gone up from 65.54% in 2004/5 to 70.26% in 2010/11.

Roehampton University’s fulCRM project is being run in parallel with attendance data recording within the Psychology Department. Fingerprint readers have been installed in all lecture theatres to record student attendance. Although there have been debates by staff about the “big brother” aspects of this approach, it would appear that students aree less worried as many are already used to doing this in school and in any case, fingerprint attendance seems to make students think that it (and they) really count and that someone is taking an interest in them. As the Department is fairly small, data was being kept on an Excel spreadsheet and a traffic light system used. The fulCRM project is now in the process of pulling automated data feeds from the attendance monitoring system into a student performance module.

Using a traffic light system, 18 students were identified as “red” just after the first semester (mostly male or part-time students with personal problems, or who had to travel long distances, etc) so extra support was put in for these students. In 2010, 8.3% of at risk students terminated and 19.8% didn’t have enough credits to go to Year 2. In 2011, since the traffic lighting system was put in place, this was reduced to 5.2% of at risk students terminating and 9.34% without enough credits to go on to Year 2. Of the 19 students with red flags, 16 have now gone on to Year 2 following retakes and extra support (only 3 actually left). In the past, a lot of students would have just drifted off course.

During the course of their project, the Southampton Student Dashboard at the University of Southampton project team has met with resistance from data owners, service providers, ethics authorities and faculty administrators, who cite a number of reasons why various types of data (from photos to student grades) can’t be used. They currently have a simple dashboard that shows picture of tutees, directory info, whether coursework has been handed in, and attendance. This particularly helps staff who need to let the Borders Agency know that foreign students have been seen and have attended classes. However, most information isn’t available in the rest of the University because of data protection, which is seen as protecting staff from doing any work or taking any risks.

The University of Derby’s SETL (Student Engagement Traffic Lighting) project is also using a traffic lighting approach and has been looking at some of the softer areas off engagement analytics. The team has produced a dartboard diagram showing the primary (e.g. attendance monitoring), secondary (e.g. sickness) and tertiary (e.g. student complaints) indicators of risk. They have also produced a “Withdrawal Calendar” to ascertain whether there are any key dates for withdrawal – there are. At Derby, a Progress Board meets twice a year to decide whether students are meeting the academic requirements and can progress on their course. These key times when students are likely to withdraw.

The ESCAPES (Enhancing Student Centred Administration for Placement ExperienceS) project at the University of Nottingham is exploring how an ePortfolio can enhance student engagement whilst students are on placement. Previous project work has already shown that both students on placement and staff found that ePortfolio tools, such as Mahara, have helped them to keep in touch with each other. Students have also valued the feedback aspect as it helps them to feel more motivated.

Some of the challenges, questions, and issues that have arisen during these projects can be grouped into the following areas:

Technical

  • Using the relationship management systems, such as Co-Tutor, may result in a bigger workload for staff, which may then manifest as a lack of engagement with the system.
  • There may not be any integration with other systems, such as e-mail, so staff have to copy and paste any e-mail from students into the system, or it may be difficult to exchange data between systems.
  • Not all data may be captured electronically but may still be paper-based.
  • It can be expensive to roll out pilot data monitoring solutions across the whole institution.

Human Aspects

  • Where interactions are recorded, such as personal tutor meetings, staff may feel that they have less freedom about where that interaction takes place, i.e. it may now have to take place in the tutor’s office rather than a neutral space such as a café.
  • Institutions need to make it clear to students that such systems are there to help them.
  • Should students be forced to attend classes?
  • The behaviour of departments who hold such data can be difficult. Data owners may be distributed. People hold their data very close to their chests and don’t want to share. Perhaps the most difficult bit is managing the soft human interface.

Data Privacy/Ethics

  • Who should be allowed to see staff data?
  • Who monitors the monitors?
  • Can use of data in this way be seen as “Big Brotherish”?
  • How quickly should staff intervene if a student is “red-flagged”?
  • Is it ethical to pre-load a system with at-risk demographics, e.g. part-time, male students?
  • What directory information should be available to everyone on a university intranet? Should photos be included (it can be useful for staff to add a name to a face)? What information shouldn’t be included?
  • Should grade history be confidential?
  • Students already share their data informally on Facebook, so why should the institution get involved?
  • Is it OK to use student data for research?
  • Who are the stakeholders involved in holding the data? What about contracts with work based learning students?
  • Sensitive handling of data and how it’s being collected is necessary.
  • Who should access what data? Should whoever needs the data be able to access it? What is legitimate? Data access needs to be considered before any opt-in box is checked. Explicit permission is needed from the student. Should anyone have access to depersonalised data as long as it’s properly depersonalised?
  • What about control of access for student? Student might be overwhelmed with information if everyone has access to the student’s data.
  • What about access by parents or other interested parties?
  • What is the “right data” to collect and analyse? For example, a student might have patchy attendance but may interact very well with the tutor and have a good assessment. Whoever acts on the data provided needs to know the student well.
  • What are the softer engagement analytics? What aspects of the student lifecycle can be harnessed? How do we capture this soft data? It needs to be about the social life as well as the academic life.
  • How early do we need the data and what would those data sources be? The data might come too late, i.e. the predisposing factors may already be in place before the student starts.

The session brought a lot of issues to light, along with some real privacy/ethical concerns, but it also highlighted that even a small change (such as keeping a register) can make a difference to student retention. However, we need to remember that students are not just collections of bits and bytes to be analysed and examined for trends. We are human with all the foibles, idiosyncrasies and circumstances that make us unique.

Come and join us!

Paper cut-outs of people in a circle

Are you interested in improving your BCE CRM (Business and Community Engagement Customer Relationship Management) processes to help increase revenue streams? Do you want to find out about SLRM (Student Lifecycle Relationship Management) and putting the student at the heart of the process?

If so, come and join our RM in HE/FE Community of Practice discussion list, which is open to anyone who wants like to share their experiences of improving their relationship management in the tertiary education sector. We’d love you to ask questions, comment, suggest resources etc, or just follow the conversation.

We’ll be sharing our findings from the current JISC Relationship Management Programme, which is looking at three different areas:

  • Good Practice in CRM – delivering a comprehensive online handbook of good practice in CRM processes for HE and FE.
  • Student progression, retention and non-completion – using service desing to improve the quality of the student experience.
  • Alumni engagement – using web technologies to support mutally beneficial alumni engagement.

We have a range of resources available at both the JISC CETIS RMSAS (Relationship Management Support, Analysis and Synthesis) project website and at the Just Enough RM (Relationship Management (a dynamic resource to try and help anyone starting out on the RM path). We also have documents on using Service Design in HE and FE and an overview to RM in HE and FE to get you started.

So what are you waiting for? Come and join the RM in HE and FE Community – we’d love to see you there!

Relationship Management in UK HE and FE Report Now Available

Photo of a handshake

Relationship management is becoming increasingly important in the tertiary education sector as education institutions try to meet the challenges of funding cuts and increased student and community expectations. Customer relationships, if handled effectively, will bring benefits to both the organisation and the sector as a whole and it is in this area that JISC developed the Relationship Management Programme.

The first phase of the JISC Relationship Management Programme ran from July 2009 to April 2010. The Programme, supported by the JISC CETIS RMSAS (Relationship Management Support, Analysis and Synthesis) project, was divided into two strands and was :

  • BCE CRM (Business and Community Engagement Customer Relationship Management) Strand – aimed to improve business processes and to pilot and extend the BCE CRM SAF (Self-Analysis Framework). Twelve universities and one FE college used the SAF to examine factors affecting the people and processes that could affect the implementation or uptake of CRM (both as an approach and as a technology). BCE CRM includes employers and other external customers, who may have the potential to help the sector navigate through the current choppy waters of tertiary education sector funding. Good customer relationship management is vital in order to maintain and develop such relationships.
  • SLRM (Student Lifecycle Relationship Management) – focussed on improving the student experience by putting the student at the heart of the process. Six universities and one FE college trialled service design techniques at different stages of the student lifecycle in order to identify areas for improvement. As students clearly exhibit certain customer attributes, such as paying for a service and expecting higher levels of choice, quality and experience, it therefore seems appropriate to apply such commercial techniques, in order to improve the student experience, the institution’s efficiency and retention.

Whilst the two strands can be viewed as focusing on two different types of institutional stakeholder – external business contacts in the case of the BCE CRM strand and students in the SLRM strand – many of the issues regarding the way in which the relationship is managed by the institution are similar. For example:

  • BCE CRM
    • Ensure that an effective CRM strategy is in place, and that is disseminated to and understood by staff.
    • Use a framework to help your institution ask fundamental questions about the people, processes and systems currently in place, prior to making any decisions regarding improvements or attempting to purchase or implement a technical CRM system, because this will go a long way to help avoid potential pitfalls and dangerous assumptions.
    • Strong commitment from senior management is vital if CRM is to succeed.
  • SLRM
    • The institution should not assume that it knows what students want, need and expect.
    • Service improvements do not have to cover the whole service, e.g. enrolment, in one go – small adjustments can be made that can actually make a huge difference to the student experience.
    • Improving the effectiveness of a process can also improve efficiencies.

The current funding situation means that institutions need to become more cost-effective. Therefore, making the most of the systems already in place, improving processes, and ensuring that the student or BCE customer has a valuable experience may help achieve this goal.

This findings from this Programme are now available in PDF format: Relationship Management in UK Higher and Further Education – An Overview (Perry, S., Corley, L., and Hollins, P 2011). Phase 2 of the JISC Relationship Management Programme is now in full swing.

New JISC Relationship Management Programme Projects Up and Running

The next phase of the JISC Relationship Management Programme has just got underway and will be supported again by JISC CETIS. It follows on from the previous Programme which ran two strands: BCE CRM (Business Community Engagement Customer Relationship Management) Process Improvement (13 projects) and SLRM (Student Lifecycle Relationship Management) Pilots (7 projects).

This latest round of projects, which will run until July 2012, is split into three different strands:
* Strand 1: Good practice in CRM (Customer Relationship Management) – projects yet to be confirmed. A project to deliver a comprehensive online handbook of good practice in CRM processes for HE (Higher Education) and FE (Further Education).
* Strand 2: Student progression, retention and non-completion (8 projects). Demonstrator projects delivering service innovations that improve the quality of the student experience, specifically to enhance progression and retention to minimise non-completion.
* Strand 3: Alumni engagement (7 projects). Demonstrator projects developing and validating innovative processes and harnessing web technologies to support mutally beneficial alumni engagement.

The projects in Strands 2 and 3 follow on from the SLRM projects in the last Programme, but focus on different stages of the student lifecycle.

If you are interested in the progress of the projects, you can find further information from the JISC CETIS RMSAS (Relationship Management Support, Analysis and Synthesis) website or follow the Twitter tag: #rminhe.

If the last projects from the last Programme are anything to go by, the level of expertise in relationship management in the institutions involved will increase considerably, so we wish all the projects in this latest Programme every success.

Relationship Management: So Long, and Thanks for All the Hamsters

The final meeting for the JISC Relationship Management Programme was held in the beautiful surroundings of York St John University a couple of weeks ago. Simon Whittemore, JISC Programme Manager, has written an excellent summary of the event – The Pleasures and Pains of Managing Relationships and Changing the Habits of Information Hamsters – and as it’s such a good overview, I’ll try not to repeat anything here.

From a personal point of view, it was lovely to finally meet all the project teams and in some respects it’s a shame that it’s all coming to an end, as I’ve really enjoyed chatting to everyone and following project progress over the past 10 months or so. Because there were 20 projects in the Programme, it was decided to put them into groups, so they could each create a collaborative presentation for the final meeting. I know all the project teams have worked really hard and I must confess to feeling a bit like the producer of play on its opening night, who just wants all the efforts of that hard work to shine through.

Notes and presentations from the York meeting are now available, and I have tried to summarise some of the key issues for the CRM (Customer Relationship Management) and SLRM (Student Lifecycle Relationship Management) strands of the Programme.

The three key issues I see for CRM are:
* Buy-in from senior management is essential to ensure that the CRM approach is successful.
* Cultural change is inevitable but is challenging to manage.
* Data (around contacts and the customer relationship) must be shared, yet this is often a stumbling block. For example, some people are very happy to view other’s data, but not share their own (information hamsters?).

For SLRM, the issues are slightly different:
* The student must be put at the centre of the process.
* What the student wants is not necessarily what the institution thinks the student wants.
* Service design techniques can help to identify failpoints in the student experience and areas for improvement.

There is some work to do now on synthesising the findings across the Programme, but it’s been such a pleasure to work with everyone, that I will miss following their progress. So good luck – I wish you all every success with continuing your CRM or SLRM approaches in your institution.

Modelling – How Do You Know When to Stop?

I attended JISC CETIS’ Introduction to Modelling workshop in Birmingham last week to try and gain an understanding of issues and what one needs to consider, when attempting it for the first time.

So why bother with modelling? What value does it bring? Models are a way of communicating and sharing experiences. They are very visual and may have some narrative, but with some slight clarification of terminology used, they should be understandable by most stakeholders. This is key. Otherwise, how else can the whole system be shown to stakeholders who may only know a small part?

Whilst the workshop looked at both hard and soft models, it was the softer side that caught my attention. Hard models, such as UML (Unified Modelling Language) or BPM (Business Process Modelling) are ideal for defining technical specifications and describing business intelligence. Soft models, as one would expect, tend to be quite woolly and may include portfolios of evidence (documents, observational notes, video diaries, etc), scenarios and personas, and use SSM (Soft Sytems Methodologies).

There were two practical exercises that we all attempted. One of which was to produce a soft systems diagram showing how to respond to a JISC call for funding. I’d never done any modelling before, but something that had seemed so simple in the introduction, was unbelivably difficult when we sat down to try it for ourselves! We didn’t need any whizz bang technology – just a handful of coloured markers, multi-coloured post-its and a large sheet of paper. Here’s a model that one of the groups came up with.

Attempt at a Soft Systems Model

Attempt at a Soft Systems Model

Using diagrams like this can help tell a story by setting the scene (scenario) and describing the personas (not usually a real person but a ficticious description of a particular role that person might do) and the ways they interact. The main focus of a soft systems model is on the people or actors in the system.

People often have difficulties knowing how far they should go when modelling and when they should stop. However, the presenters were all unanimous in answering this question: the actual purpose for doing the modelling should set the boundaries for how far/deep one goes with it. But, as a newbie, the real key for me was that one should model just enough to achieve one’s aims (otherwise one could end up modelling for years!).

If you’d like to find out a little more about the event, there are notes and presentations from the workshop now available online. You might also be interested in the JISC Innovation Base (a repository of models for the Higher Education domain, which includes both formal and informal models) and the Agile Modeling Website (Agile Modeling is a practice-based methodology for effective modeling and documentation of software-based systems).

Happy modelling!