problems with lecture recordings?

People said our lecture recording wasn’t working properly. So we checked  a thousand recordings. This is what we found:

(Numbers given are out of one thousand and there is a percentage summary table at the end.) 

Empty room or blank recording.  98 instances where the room was booked and a recording scheduled, but no lecture took place. 

On discussion with Timetabling this was not unexpected, it is a legacy issue not related to the recording service.  Reasons for this could include a course changing or being cancelled and the room bookings not being removed from the Timetabling system.  A room change within one working day would not be able to filter through the services in time.   Timetabling had no way to know a course had been cancelled unless course staff told them. The other disadvantage of this is that the room booking remains in place and no one else can book it. 

We discounted this in the final summary. 

Microphone not used.   110 times.  This is a user error and makes the recording unusable and obviously impossible to provide captions/transcripts for.
 
Recording stopped in room near start.  35 times.  This is a user action and makes the recording unusable.
 
Recording paused.  2 times.  Using the pause button in the room at the start, making the recording unusable.
 
Microphone temporarily muted.  24 times.  This would be done by the user in the room and would make part of the recording unusable.
 
Audience microphone not used.  48 times.  This can be for a variety of reasons, including logistical ones.  It would result in questions/answers from the room not being heard and would make part of the recording unusable.
 
Microphones not used by all speakers.  4 times.  Forgetting to hand over the microphone or the other presenter(s) not wishing to use a microphone could be reasons.  In these cases, part of the recording would not be usable.
 
Audio quality poor, audio unclear or static.  15 occurrences.  These are technical faults and have been reported and investigated.
 

 

Results Summary. 

This seems to indicate that user error was far more likely to be an issue with the recordings than technical issues with the room. 

Summary of non-technical or timetabling issues. User based issues.   Number  % 
Mic not used count  110  10.7 
Recording stopped or paused.  37  3.6 
Partial audio issue from muting, audience or other speakers.  76  7.4 
Total checked  1028   
Total problems  223  21.7 

 

 

SADIE: Scoping AI Developments in Edtech at Edinburgh University

I wrote a a while back about the start of our SADIE project looking at AI in the third party systems we provide for our students and staff at University of Edinburgh.

Educational technology (EdTech) services have not been immune to the excitement and rushing wave of AI adoption. It is important for learning technologists in central services to understand the risks of new features being rolled out by our existing technology partners and retain the ability to assess and choose which ones we switch on for use by our community.

It is a fast moving space:

  • An AI detection feature was added to the similarity checking service Turnitin in April 2023.
  • Various AI helper tools have been added to our virtual learning environment Learn since July 2023.
  • Wooclap added an AI wizard to generate multiple choice or open questions in November 2023.

All these services are under the control of the University to enable (or not) and are all currently switched off at Edinburgh due to risks identified.

The biggest barrier to adoption for AI tools is likely to be clear assurances from suppliers on the compliance of their AI features with University policy and legal obligations. We need a common process  which will allow us to be consistent in the evaluation and adoption of AI tools and features. 

The processes we will now use have been developed carefully by senior learning technologists with expertise in providing our central systems. For the most part they are an extended reworking of existing processes for the introduction of new non-AI features into services. As ever we need to take into account the workloads of learning technologists and ensure the processes we develop should not be much of a burden on service teams to adopt these and extend them to AI features. 

Since assessing risks of AI tools will soon become a regular or routine part of business as usual, it is important that decisions on the enabling of AI features are transparent to users. The  Edinburgh AI Innovations Service Release Tracker, and the wider SADIE SharePoint site will give the rationale behind the approach adopted and decisions made. It will also provide advice on the risks of using a tool even if it has been made available. 

The adoption of AI tools and features will likely require a review of University policies, potentially including but not limited to the OER, Lecture Recording, Virtual Classroom and Learning Analytics policies, to take account of the risks identified as part of this project. 

The Scoping AI Developments in EdTech at Edinburgh (SADIE) project was set up to standardise an approach for service teams to test and evaluate the utility and suitability of the AI tools and features being made available in the centrally supported EdTech services. The approach developed looked at the risks of adopting a particular feature and calls upon the expertise of learning technologists within the Schools, as well as that of the service managers in Information Services, in evaluating them. 

We will be monitoring progress closely.

Delivering a University Web Strategy

Cool graphic designed by our cool LTW graphic design service

The University Web Strategy (https://edin.ac/web-strategy-2018) identified a programme of activities, including the development of new web publishing tools or web publishing platform.  

“There is a fragmentation of technology, working methods and standards, which leads to uneven and, in some cases, broken user journeys.” – University Web Strategy 

 The strategy stated that “it is important to recognise that technology is not a solution but an enabler. Whether delivered centrally or locally, there is a clear need to empower our staff by providing them with the intelligence, tools, standards and resources to attract and engage users.” 

Building a New Platform

 Our Web Publishing Platform (now named EdWeb 2) project’s aim was to create a web publishing platform for our publishing community within a strong web governance framework. Therefore, this project should not be viewed solely in isolation but within the wider context of the web strategy. The goal was to create a platform that could be continually improved, meet the immediate needs of our existing editorial base but also meet the needs of Schools and business units that had never used the central service for web publishing before.  

The web strategy identified the following common approaches:  

  • Improve the quality of our web estate and online channels through the adoption of an inclusive and supportive governance model  
  • Enhance the accessibility and security of our web estate by establishing and evolving University-wide standards  
  • Enhance solution and content quality by improving the digital skills of web publishers and practitioners; establishing a common understanding of web roles and capabilities; and delivering web publishing and management tools 

“The strategy does not set out to answer the exclusive needs of a business area or address a single specific University activity. As a pan-institutional strategy, it establishes a framework for the use of web technologies, both centrally and locally, to achieve business goals.” – University Web Strategy

To address these common approaches the web publishing platform aimed to deliver a product that was flexible and geared for iterative improvement to support ongoing business and user demands.  

The delivered solution needed to support central and local innovation, allowing website teams in schools to make use of and extend the product appropriately, aligned with agreed University standards and guidelines.   

This project has delivered the platform and associated processes to create the web publishing services that will support this strategy. 

This includes a high-quality, future-proof and long-term sustainable University web publishing platform, aligned web publishing services, training and a support model, all backed up by a team with the requisite knowledge and skills.  

Successes 

Planning 

  • Platform training programme with the planning, development and roll out in very close collaboration with our  Digital Skills team was very successful, and receiving excellent feedback (Phase 2 – Deliverable 1.3) 
  • Major achievement to deliver a fit-for-purpose, web platform, fulfilling the targets set regarding flexibility, migrating over 60 sites live despite some of the complex issues faced by the project.  (Phase 1 – Deliverable 1.3) 
  • Early and ongoing engagement with the service users, and the wider web publishing community, was essential to ensure the web platform developed in a meaningful way to address their external audience engagement needs. (Phase 1 – Deliverable 1.2, 7.1) 
  • The project has delivered a solution in line with the original University Web Strategy in terms of the flexibility offered and that the offering should appeal to Schools and business units that have never used the central service at all for their web publishing. For example the School of Chemistry now has a live site on EdWeb 2.  (Phase 2 – Objective 4, Deliverable 4.1) 

Developing for our Wider Community   

The University of Edinburgh is a highly devolved organisation in terms of physical locations, organisational structure, decision-making and budgets, purposes, goals and audiences.  

Aside from the central team, our community extends across the organisation, from system administrators and developers to UX and user-centred design practitioners, content creators and marketers, each operating with significant autonomy and responsibility for their own sites. We had to take this into consideration as early as possible in any decision-making process, and to emphasise the importance of user centred design, backed up by user research, and using the experience and expertise of staff around the University.   

The new platform had to work for existing users of our CMS but also support the needs of our devolved community of developers and administrators who are responsible for building, developing and managing our wider web estate, from colleges and schools to individual research projects.  

We chose to continue using Drupal as this platform and surrounding processes support highly distributed development, allowing code contribution back to a managed common modular code base.  

Project Approach  

The project was approached in two main phases, with the ultimate goal to have introduced the new web publishing platform and having all relevant EdWeb content and sites migrated to the new platform by November 2022 (the initial deadline for Drupal 7 end-of-life).  

The deadline of completing the migration needed to be moved due to the extremely complex nature of the websites built in EdWeb, and the differences between the two platforms, as well as ongoing resourcing issues and is currently set for November 2024.  

The project involved the application of Agile methodologies using the Scrum framework and has tried to use a user-centred approach to service design wherever possible.  

“User-centered design is based on the understanding of a user, their demands, priorities and experiences and when used, is known to lead to an increased product usefulness and usability as it delivers satisfaction to the user. “ – Wikipedia User-centered design 

 

hoarding behaviour online

Hoarding and storage of old stuff indefinitely is a challenging user behaviour for several of our platforms. Cloud storage costs the university money.   Users may be unaware of the need to align with data protection rules and to reduce our impact on the environment, every little bit helps and we have a huge amount of cloud storage being used. Our first batch of deletions resulted in the total number of courses in Learn being reduced from 91k to 65k courses.

Our media platfrom has 69,000 items on it which have never been played.

Understanding digital sustainability is a key knowledge set for the future.

Evaluating VLE change

The University of Edinburgh’s Learn Ultra upgrade aimed to enhance the virtual learning environment (VLE) to better support the diverse student body and align with other strategic initiatives and objectives. 

The Learn platform hosts over 6,000 courses with an average of 39,000 daily logins from students engaged in on-campus, online, and hybrid studies. 

The Learn Ultra upgrade project oversaw the successful upgrade of the University’s VLE from Learn Original to Learn Ultra. It focused on improving usability and accessibility based on feedback from students and faculty, aiming to create a more user-friendly and inclusive learning environment. 

This report presents an evaluation of the key decisions made by the central Project Team that have led to the successful delivery of the upgrade project. 

The evaluation combines qualitative and quantitative data sources to provide a comprehensive analysis of the delivery of the Learn Ultra upgrade project: 

  • A campus-wide student survey that gathered responses from 391 undergraduate students on their experiences with Learn Ultra. 
  • Semi-structured interviews with key stakeholders, including senior stakeholders, learning technologists, teaching office staff, academic colleagues, and relationship managers. 
  • Focus groups with the project’s implementation and user groups. 
  • Profession-specific focus groups with learning technologists, teaching office staff, and academic colleagues. 
  • Secondary analysis of existing data sources, including user experience (UX) data, Early Adopter feedback, training feedback, and internal project reports.  

 The evaluation identifies eight key overarching decisions made by the Project Team that have led to the success delivery: 

  • Upgrading the existing VLE, rather than procuring a new provider. 
  • Treating the upgrade as a Change project. 
  • Implementing a pluralistic governance framework. 
  • Extensive multimodal communications and engagement. 
  • Establishing the Early Adopter Programme. 
  • Creating an extensive training programme. 
  • Focusing on a user-centred upgrade approach. 
  • Enhanced focus on accessibility. 

Additionally, the evaluation identifies three themes that were found to be important to successful local upgrades: 

  • Pro-active local Change approaches and project ownership. 
  • Effective relationship-building and collaboration. 
  • Tailored local training and ongoing support. 

Our evaluation report makes a number of recommendations to the University for future projects. You can read more on our internal Learn Ultra Evaluation (sharepoint.com)

summer migrations

In LTW we meet in person twice yearly for LTW All Staff meetings and our summer gathering this year takes place next week  at the recently opened Bessie Watson Lecture Theatre at the Outreach Centre. There will presentations from colleagues, group exercises and snacks.

Every 6 months I  ask each of the LTW Heads to send me their list of team achievements, so if you think they might have missed any, now is the time to remind them.

As I read through their lists this time I am struck by how much time we spend on procurements, replacements and migrations as technology changes. Some of our funding comes from capital pots, which might usually be used for buildings. But technology changes much faster than buildings and we have a rolling 5-10 year plan to replace platforms and technologies as ( or before) they go out of date.

It takes  an enormous amount of work it takes to move from one platform to another.

If colleagues suggest we should get a new VLE, or a new portal or a new media asset management platform it is a huge amount of work and sometimes it feels like there is very little gain. Migrations and replacement projects seem often to be replacing like with like. So it is important to be able to identify the benefits which we will see, improvements in managing, keeping up to data and mitigation against risk. Risks in LTW are risks for the whole institution.  If we don’t have up to date robust systems, learning teaching and the student experience will suffer.

Never underestimate how much work a procurement, replacement or migration can be. But no one will thank you for it. It is the hidden labour behind the fancy new tools colleagues and students demand.

I have spoken much about the upgrade and migration work required for Learn Ultra.

We’ve have also moved away from QMP on-Premise to the Cloud – Karen H estimates this was the longest upgrade project we’ve ever had.  Early next year for complete decommissioning of the on-premise system and then we’ll have our celebration.

Our largest migration on going is a huge move of the entire University website (1.5 million pages) from Drupal 7 to Drupal 10.   Of those 1.5 million pages I’d estimate around five of them were the same, so the work to automate this lift and shift at scale while building a new platform in flight has been a huge undertaking. Perhaps we were naive fools even to try.

We have new colleagues in our website migration project team. we have worked hard to find creative technical solutions and to keep colleagues with us through the move. We introduced more resource and optimised our processes and engagement.  Current migration count is 75 completed, 86 still to go, almost 50%. EdWeb to Web Publishing Platform migrations | Website and Communications

By the next LTW All Staff in December, all the migrations will be completed. And Stratos and I are looking for a date for the ‘end of migrations’ party.

That will be a well-deserved celebration.

who learns online and who attends class?

We have been looking at the data  about students attending skills courses.

At University of Edinburgh students have a choice of whether to work through online materials in LinkedIn Learning at their own pace or to attend synchronous teacher-led sessions with our Digital Skills Training team, in person or via webinars.

The data show that  students from the College of Science and Engineering use LinkedIn Learning at a higher proportion than they were represented in the overall population, and at nearly double the rate of those attending Digital Skills Training.

Whereas students from College of Medicine and Veterinary Medicine were more likely to take Digital Skills Training than use LinkedIn Learning.

Students from College of Arts, Humanities and Social Sciences  were slightly more likely to attend Digital Skills Training than use LinkedIn Learning, but the difference was not as significant as for the other Colleges.

 

 

Short Courses Platform

Our new Short Courses Platform has met its first major milestone.

We have 18 early adopter courses and over 250 learners enrolled and using the new learning environment.

This allowed us to establish and test the basic platform configuration including notifications, basic learner/course set up, as well as the courses templates, training and guidance.

This is the first step in moving the University’s extensive credit short courses portfolio to the new Short Courses Platform.

The learners on our new platform will not have access to our closed Library collections, so all the courses will use open access materials on their resources and reading lists.

I am very pleased.

International Women’s Day 2024-Bessie Watson

My honour to open the Bessie Watson Lecture room on International Women’s day.  Beautiful ribbons in green, white, purple and red!  Not yet 100 years. Thank you to Lauren Johnston-Smith Lesley Greer, Katie Grieve Karen Howie Ellen Groen, Susanne Knowles for organising and Gillian Kidd for the fabulous artwork. These women have got the skills we value.

This is now becoming a theme for us to name rooms on-campus after inspirational women. There are so many rooms, buildings and roads named after men in this university it is great to be able to add some women’s names in there too.

We are featuring Bessie as the woman were are naming for today, but we have previously named Brenda Moon, Irene Young, Mary Somerville, Grace Hopper, Eleanor Ormerod, Marjorie Rackstraw, Annie Hutton Numbers, Xia Peisu and Charlotte Murchison

Bessie was born nearby, in the Vennel, which is now a very popular Instagram-able spot with great views up to the castle. She is famous for being a bag-pipe playing suffragette, but she’s also an alumna of Edinburgh. She studied French here and went on to a career as a music teacher and a modern languages teacher at Broughton High school, She married and lived  in Trinity and everyday for the rest of her life she played bagpipes at 11 am, which her neighbours obviously appreciated.  She died at 92.

But before that, when she was wee, played the bagpipes from a very young age. Her parents got her playing in the hope that it would strengthen her lungs against tuberculosis- an early example of proscribe culture.

Bessie and her mother were members of the Women’s Social and Political Union and that’s how she got her most famous gigs. She was invited to play the pipes in the famous procession down Princes Street, in 1909 she was 9. She also played lead the Scottish contingent at the Great Suffragette pageant in London in 1911. The procession was 5 miles long.

Those of you who study the history of the suffrage marches will know that they were difficult to organise because people had different ideas about who should go first and in what order, and which groups were more established, and in America, whether white and black women would march together or separately

In Edinburgh they decided to avoid all that and march in alphabetical order. Christabel Pankhurst was the star speaker at the march and afterwards she gave Bessie a brooch to commemorate the occasion- it was a brooch depicting queen Boudica,

Bessie continued to be actively involved in suffrage, wearing purple, green and white hair ribbons to school. And she piped outside Calton Gaol to raise the spirits of the suffragettes imprisoned there, who were being force-fed. The women were on hunger strike, and force-feeding of women at Calton jail started in February 1914, so around 110 years ago. Most of the force-feeding actually went on in Dundee and in London and She played the pipes on the platform of Waverley station as trains carrying convicted suffragettes departed to Holloway Prison in London.

It’s worth remembering that some of the convictions were for destroying property, damaging paintings of the King, blowing things up, it is hard to know how the campaign would have progressed were it not for the start of the First World War. One of the places they attempted to blow up was the royal observatory up near our campus at King’s Buildings. You can still see part of the bomb in the visitor centre.

Edinburgh University has connections to a number of Suffragettes, and a few years ago for the vote 100 campaign, colleagues and students worked together to create a histropedia timeline on Wikipedia,

The suffragettes and suffragists were campaigning for women’s rights, sex-based rights, struggling to get the right to do things like vote, access to education, the right have a bank account,  ability to get a mortgage in your own name , to not have to leave your job when you got married.

The right for all women to vote was not secured in the UK until 1928.  So we are not yet at 100 years.

The Representation of the People (Equal Franchise) Act 1928 It is sometimes referred to as the Fifth Reform Act.[2][3] The 1928 Act widened suffrage by giving women electoral equality with men. It gave the vote to all women over 21 years old, regardless of property ownership. Prior to this act only women over 30 who met minimum property qualifications could vote.

About 30% of my current staff in LTW are under 30, so the women in our office wouldn’t have been able to vote.

This was Tory reform and one of the reasons that Bessie later gave her Boudica brooch to Margaret Thatcher, the first woman to become Prime Minister. Some of those rights were still not in place until well into the 1970s, and still are not in many countries in the world.

I am privileged to have grown up in a time when I was able to go to university, get a job and pay my own mortgage. It wasn’t until the sex discrimination act 1975 after that banks were required to treat women equally and women were able to get mortgages in their own name without a male guarantor. Still women suffer a structural pay gap.  there is no English region where a single woman on median earnings can afford to rent or buy an averagely priced house.

But now we are allowed to have an education, and we can work in universities and so we do.  This year for the first time, LTW is equally filled with men and women and the pay gap is as small as I can possibly get it.

Our Learning Spaces teams teams’ fit out these rooms, and all the other central teaching spaces on campus- that’s 400 rooms, which a combined collection of around 10,000 pieces of kit.

if you were here last year, I made a tenuous connection between Charlotte’s work finding fossils in chalk cliff and classroom chalk,

This year I am going to make a similarly tenuous connection between playing the bagpipes and being heard.

Microphones and catch boxes.  Not only that presenter should be their microphones, but that students should not be shy in requesting a microphone if they want to speak, we have catch boxes available in every room and they are for students to use. Let your voice be heard.

 

Building a new learning platform for University of Edinburgh

So many short courses
So many short courses

It’s not often you get to start from scratch putting together a new learning platform for a University. Most learning technologists and digital leaders have experienced the procurement of a replacement VLE, or a migration or upgrade.  It is rare that we get to work with partners to design from the start, thinking about the new relationships you can make with your learners if you do it right.

Bringing courses from across the University together on a single platform with a consistent learner experience will require both technical and business changes to processes, training and best practice. The Short Courses Platform will be delivered through a phased rollout where we develop capability, test with early adopters and then scale the platform and service. 

The plan is being finalised and key dates will be published  when they are available. For now we are:

  • Holding workshops throughout March 2024 to co-design the new processes and specifications with the steering groups.
  • Establishing platform based roles/permissions, SSO and configuration to support the initial early adopter courses.
  • Working with a small set of agreed early adopter courses and tutors, from the Centre for Open Learning (COL), to develop support and guidance and trial the initial course templates and learner experience for courses running from Summer ’24. 
  • Developing the University’s new short course platform web catalogue including the course search and course description pages. 
  • Collating the short course inventory to understand when courses may move to the Short Courses Platform.  

The introduction of the platform, and supporting service, is the latest step in the University’s Digital Estate Strategy and aims to provide accessible and appropriate teaching and learning experiences for non-credited short courses. It is the start of a new relationship with Edinburgh learners who are not matriculated ‘students’ and who bring a new set of expectations.

Matriculated learners on campus and online will continue to learn via our Learn Ultra VLE, and staff development courses will be delivered on our corporate L&D platform. MOOC learners will still find us on EdX, Futurelearn and Coursera. But this new platform will provide a new home of CPD, PPD, Exec Ed, microcredentials, Data upskilling, lifelong learning, workplace learning, B2B and adult learning.

The Vision for Change 

The vision for the Short Courses Platform is that it will:  

  • Encourage wider access to, and continued learning with, the University through consistent learner experiences and the ability to promote further study.
  • Increase diversity in our university learning community through increased visibility of courses and the expansion of adult education.
  • Improve management information, strategic overview and reporting on non-matriculated learners and non-credited courses.
  • Streamline the learner journey, directing them to the systems and services which are licenced and resourced specifically for non-matriculated, short-course learners.
  • Enable process and system efficiencies by replacing end of life systems and delivering a platform designed specifically for non-credited learners.

This project aligns with Strategy 2030 (Opens in a new window). Key areas from the strategy that this project supports:

  • Social and Civic Responsibility – widening participation in higher education and supporting inclusion.
  • Teaching and Learning – encourage a culture of lifelong learning, greater focus on focuses on experience, employability. 
  • People – bring together people from a wide range of backgrounds and experience, both close to home and across the globe.
  • Research – as a research institution, many of the University’s short courses extend the impact of research taking place by bringing outputs and findings direct to learners across the world.

In January 2024, we started working with Instructure, and their delivery partners Drieam, to design and configure the system alongside establishing the service processes, migrating courses and drafting guidance and training.

Steering Groups with representatives from across the University will support the Board and guide the implementation. Visit Project Governance for more information and details of the Board and Steering Group members.