Report from the Next Generation Learning Management System Workshop 2020
urn:nbn:de:0009-5-53233
Abstract
What should the future of Learning Management Systems look like? In August 2020, 88 participants from Austria, Germany and Switzerland approached this question from various angles at the Next Generation LMS workshop, offered jointly by CampusSource, the research focus project D2L2 at the FernUni Hagen, and ETH Zurich. We report on the contents of the workshop, its outcomes and offer preliminary postulates as a guidance for future development in the field. Learning and Course Management faces internal and external challenges: users raise a broad and deep range of expectations while an increasingly connected life stimulates competition with outside players.
Keywords: e-learning; learning management system; artificial intelligence; higher education; life-long learning; learning objects; user data; privacy
On August 19th and 20th, 2020, the “Next Generation Learning Management System (LMS)” workshop took place with 88 participants from Austria, Germany, and Switzerland. The overarching question was, “what should an LMS look like in the age of Social Media, Cloud Computing, Machine Learning, Cyber Warfare, Life-Long Learners, Competency-Oriented Learning, and increasing diversity of students?” The question was intentionally buzzword-loaded, yet it was deemed to properly reflect the fact that post-secondary education will be subject to radical paradigm shifts over the next decade.
This shift will have been accelerated by the Corona-crisis, which already brought about irreversible changes – the same changes that forced this workshop online using Zoom and Miro as collaboration tools. Not only did the crisis bring about a new openness to and dependence on the application of online learning technologies, but also a new appreciation for the value of face-to-face instruction and the interplay between the physical and virtual realm. The demise and creation of whole industry segments highlighted the already anticipated necessity for life-long learning and retooling in adaptation to an ever more rapidly changing employment landscape. Many institutions of higher education, particularly outside the German-speaking regions, are reconsidering their role and sources of income in this new era, as purely residential versions of education are called into question.
The first day of the workshop was dedicated to presentations, which provided concepts and ideas for the following day. In the morning of the second day, participants worked in groups of up to six on two tasks: first, a collection of components of an (desired) LMS and second, devise, from the perspective of different user roles, so-called “user stories” that describe what a system should accomplish – an informal approach borrowed from agile software development. The user stories were then sorted by roles and handed back to the shuffled groups in the afternoon. Their task then was to write up a coherent scenario for each assigned role as an “epic story”. Results were presented in conclusion of the second day.
The impulse talks touched on a variety of innovative pedagogies with a strong focus on technologies. Several talks discussed architectures which would glue together different applications; in one view, this involved the establishment of a network of LMSs and other systems, while others talked about the LMS as gateway to other systems or open, flexible core architectures and the composition of systems and services in a “post-LMS” era.
The majority of talks put the learner into the center: engaging the learner through gamification, sustaining flow, fostering self-regulation, and personalizing the experience through data-driven adaptation of learning paths. Along the way, the learner is given opportunity to check on his or her learning progress toward learning goals through adaptive, formative assessment and high-information feedback. Learning goals were not limited to overt curricula, but included the acquisition of cross-disciplinary competencies. The learning journey would be life-long and seamless between institutions, and it would be barrier-free for a diverse group of learners in spite of challenges, handicaps, and limitations.
Learners were not seen in isolation: the systems would bridge the spatial and temporal chasm between presence and telepresence, joining conventional and digital spaces. They would foster collaboration between learners through automatically suggested or learner-initiated study groups and custom-fit sets of collaboration tools. In some scenarios, additional actors would appear: artificial-intelligence agents would function as users in the system.
The spectrum of enabling technologies included machine-learning techniques, geolocation awareness, web-service architectures, recommender systems, sophisticated video streaming, as well as various standards and standardization efforts. The papers associated with these topics are published in this issue of eleed.
Several shortcomings of the current generation of systems were mentioned throughout:
-
varying degree of maturity,
-
lack of synchronization capabilities with administrative systems and among components of the learning environment,
-
lack of support for specialized pedagogical scenarios and workflows,
-
pre-defined processes too rigid,
-
information overload,
-
inconsistent user experience,
-
data silos,
-
too course-centric,
-
solitary and isolated from other systems,
-
hardly any content is publicly available,
-
no common infrastructure,
-
lack of central digital space, and
-
missing search engine for content.
Future systems should be:
-
stable,
-
cloud-based,
-
offer interfaces to other systems and allow composition,
-
offer legal certainty,
-
reciprocity-based privacy,
-
all-embracing single sign-on,
-
safe environment with respect to privacy and copyright,
-
social network (between peers and student – teacher), and
-
a place for life-long learning.
The second day of the workshop was dedicated to the characterization of a hypothetical Learning Management System by gathering requirements and interests of various stakeholders.
A user story in agile software development should be formulated in the language of the customer or user and follow a particular template: “As a (user role), I can (action) to (achieve goal).” These stories, albeit simple in their structure, carry a lot of information and enable a semi-structured approach. After workshop participants compiled online “Post-Its” with notes from the talks and user roles, they were asked to write down such stories. The 88 participants generated 340 stories in one hour – perhaps a symptom of the wide spectrum of expectations for a next-generation system. Figure 1 shows an excerpt; the rigid structure of the template was not always strictly followed. Many of the user stories would be considered “epics” in agile development, i.e. they are high-level stories, requiring further concretization.
|
As an alumna, I can remain in the system to maintain contacts and view further training offers. As a student advisor, I can access content that a student has made available to me to advise students. As a prospective student, I can easily integrate previous achievements into the LMS in order to have them easily recognized. As a student, I can see on my personalized start page exactly what is important to me and what is not. As a student, I can check my level of knowledge (at any time) and know whether I am on target. As a student, I can use my LMS well and with all functionalities on my smartphone, so that I don't necessarily need a laptop. As a student, I can take courses based on the skills I need to achieve in my studies, so that I can organize my studies more effectively. As a student, I can see what other students have used to get help on what a promising approach might look like. As a student, I can keep control of my data so that I can access it at any time and take it with me into other systems. As a student, I can optionally get recommendations on the learning path to design my own learning path. As a student, I can use single sign-on to access all components/features/tools with one login. As a student, I can learn at my own speed, so that I don't get bored and don't get left behind. As a student, I can access the LMS even after my studies to organize my lifelong learning. As a student, I can access external materials (e.g. OERs) via the LMS to find additional content suitable for my course. As a student, I can contact my recommended study buddies to work on common difficulties or topics at the same time or to network locally. As a student, I can independently define group and study rooms to be able to exchange ideas with other students, teachers (and external students?) in a self-determined way. As a student, I can keep an ePortfolio, which I can re-import at other universities / continuing-education institutions (or there is a central storage place for it), in order to be able to prove which competences I have acquired (over many years). As a student, I can release my usage data for analysis and processing so that I can also benefit from the use of the usage data of other students. As a student, I can accomplish tasks to get feedback on my learning progress. As a student, I can access content that uses various sensory channels to adapt my learning experience to my individual didactic and physical needs. As a student, I can make individual privacy settings to be able to decide individually which data I make available. As a student, I can use the LMS without restrictions on different devices and mobile devices in order to access content barrier-free and independent of location. As a student, I have access to AI-based learning support in order to individually adapt and optimize my learning process, supported by an automatic evaluation of my learning behavior. As a system administrator, I can integrate external tools without much effort in order to provide them visually and integration-wise indistinguishable from internal tools (App-Store-like). As system administrator, I can anonymize user content when course participants leave the course in order to be able to use the course (e.g. forums / wikis) over several semesters. As a system administrator, I can access a hosted system as a client so that I do not have to take care of the operation of the instance myself. As a system administrator, I can give other systems fine-grained access in order to manage users and data more easily. As a system administrator, I can easily set up interfaces and plugins to include more content and functions. As a system administrator, I can maintain subsystems so that I don't have to take the entire system offline. As a system administrator, I can use a group-based system in order to manage my roles more easily. |
Figure 1. Examples of developed user stories
While user stories tend toward the specific in order to foster implementation in software, the goal of the workshop was to collaboratively build a coherent vision. The stories were therefore arranged by user role and each group of participants was assigned stories of two roles and asked to draw them together into a coherent usage scenario. By that the typical agile process was reversed and the concept of epic stories was interpreted even more freely to mean scenarios of users interacting with the system. Figure 2 shows two examples of these epic stories, one for an actual user teaching a course, and one for an AI-agent in the system. Other stories were collected for system administrators, librarians, students, and teaching assistants.
|
The AI agent called "HAL 9000" was installed at the university. HAL's main task is to collect data from the students, to evaluate it and to set actions on it. HAL receives the data from the learning management system. Its tasks are to match the typification of certain types of learners and similar types of learners. It does this to report strongly deviating student behavior to teachers in order to be able to offer active support. HAL must also be able to extract the strengths/weaknesses of the students from the user data and thus make recommendations to the students as to which learning content and media types are best suited for optimal learning success. For example, HAL suggests that certain students are more likely to consume the lecture recordings, while others are provided with interactive scripts. At the same time, HAL also provides literature recommendations so that students can delve deeper into the subject matter. HAL also gives students intelligent feedback on homework, etc. For example, suggestions for spelling correction for final papers, as well as active advice to students - e.g. for exam preparation. HAL provides background information on the curriculum in the form of a question-and-answer catalog to help teachers to optimally tailor courses to learning objectives. HAL also has all dates and workloads under control and can therefore provide students with feedback on the next possible deadlines, exam dates, etc. |
|
Kora would like to plan a course and get didactical advice from an AI assistant to avoid gross errors. On the basis of her information and formulated requirements, he suggests different digital learning scenarios to her and briefly presents them with videos or other information material. For this purpose, the AI suggests using appropriate didactic templates for courses. She is guided by the assistant through such a course creation, where she can select functions to support the desired learning methods in her upcoming course. As a further source of inspiration, Kora can also look at courses with similar topics of her colleagues and, if necessary, exchange or collaborate with them by means of OER materials. If she still misses functions, she can search for them in an app store connected to the LMS and, if they meet her requirements, install them easily. Central appointments or deadlines are automatically transferred to the students' personal desks. She can set different levels of privacy for her course. For example, she can open it only for the group or her institute/faculty or, to enable interdisciplinary work for her entire university or, for an open transfer, also for external cooperation partners. After creating the course, Kora can rely on the LMS to adapt the content and functions for different channels of perception and offer them to the students (subtitling of videos, creation of text documents from auditory content and vice versa, and more). Kora started her course last week. The Campus Management System directed all participants in the correct role in the course. In the progress display it recognizes the current processing status of its students. She recognizes that the workflow is not running smoothly for all students. In her haste bar she reads the students' comments and identifies where she needs to make the task description more understandable. She uses the hashtags to point out the most common problems to the LMS. Via the Omnichannel-Forum she gives hints which the students receive on their cell phones. In addition to the hashtags, an integrated AI evaluates the logs and displays the length of stay and the learning map. Kora uses this evaluation to support its students according to their needs. The AI supports Kora. Based on the comments of the studies, the AI learns the most common problems and provides proactive support. Kora exams digitally and is very happy that it can use a separate exam environment within the LMS, which prevents cheating as much as possible. She can draw on a collection of examination tasks in her subject area, which she can adapt if necessary. To ensure that the exam differs sufficiently from the exam of the previous semesters, the system only makes suggestions for exam modules that have not been used for at least x exam cycles. Tasks from the last x semesters are locked or marked with a corresponding warning message. Kora is concerned about whether she has prepared the content correctly and whether she is taking good care of the learners. The LMS points out to you when exercises are answered incorrectly by many participants. She can now go through this content again with the learners. |
Figure 2: Epic stories for an AI-agent and a lecturer in a next-generation system
When Course Management Systems first appeared on the landscape two decades ago, there were no Social Media, ubiquitous Internet, smart phones, cloud services, or online collaboration tools of significance. At the time, it was possible to create a monolithic web application that combined rudimentary communications, content management, assignment management, gradebook, and quizzing functionality. Most instructors were happy to upload syllabi and PDF-documents, as well as to email the whole class. With the emergence of Learning Management Systems, the learning process shifted into view, and meanwhile well-established practices like formative assessment and immediate feedback were fueled by this shift.
Today, user experiences and expectations differ vastly between what is possible inside Course Management Systems and most anywhere else in a user’s digital life. Tools that are hardwired into a single system that is supposed to do everything can hardly compete with what readily available specialized tools can offer in terms of usability, features, and polish. In addition, unlike most CMS, these tools can easily be used in any circumstances and on any device.
This may be one of the reasons that many instructors use Course Management Systems only for the most basic of functions, at times simply to fulfill university requirements, and instead make extensive use of external tools – much to the dismay of students who have to juggle a slew of different websites, identities and logins.
On the other hand, current and future students grew up in a connected world and cultivated their own digital environments and expectations and often cannot relate to why they should abandon the familiar in favor of some cumbersome technology. They evade the provisioned learning environments and instead construct their own from building blocks outside the realm of learning technology, thereby thwarting purpose and intent of Learning Management Systems.
Teachers and learners alike embrace current technologies and want to employ them. However, they want to freely design teaching and learning processes, interactions and spaces, implement them by composition of available services and are prepared to fill gaps with their own contributions. A rigid environment, that oftentimes represents the least common denominator between competing and conflicting expectations and requirements, does not cater well to these needs.
This was also reflected during the workshop: the expectation of a wide array of functionality, however, in some way integrated in an overarching architecture that glues it together and manages identities. More than once, the idea of an “app store” was mentioned, from which instructors can assemble not only the content, but also the functionality for their course from more than one provider.
Another limitation of classical Course Management Systems is the “course container” – both users and content exist within the container of the course that is being managed. As was emphasized during the workshop, the concepts of a life-long learner and multi-institutional study pathways are incompatible with a course container – and so is the necessity to sustainably offer online educational experiences across courses and semesters.
Learners, teachers and resources exist outside any CMS container and independently of one another. They come together in courses, in which they are exposed to each other in a safe environment that facilitates interactions between them. This stands in contrast to how the corresponding entities are represented in typical Course Management Systems.
Creating new content for online courses is overwhelming, so in a culture where sharing is well-established through the publication of scientific results, the idea of Open Educational Resources (OERs) has been around almost as long as Course Management Systems have. Unfortunately, OERs never quite lived up to the hype – even when MOOCs appeared to finally push them into the limelight, it fairly quickly became clear that MOOCs are not really “open” in terms of reusable and remixable content.
There are several possible reasons for the limited dissemination of OERs, some of which may be:
-
Locating content: OERs are distributed over a plethora of repositories, and their associated metadata does not allow to effectively search, as level, context, and prerequisites are not catalogued to the required level. Since OERs need to be downloaded from repositories, there is currently no way to automatically gather these metadata from usage context.
-
Evaluating content: Short of peer-review, which introduces a bottleneck for the incorporation of new content into repositories, as well as hesitancy on the part of potential content contributors, there are no efficient quality control mechanisms for OERs. Again, not being able to extract usage and learning analytics data from actual usage in learning scenarios prevents accumulation of knowledge about the content.
-
Incorporating content: OERs are oftentimes embedded into larger content structures, having branding and navigation elements. Their granularity thus frequently does not allow for modular incorporation into online curricula. Also, for some instructors, downloading content with all dependencies and then reuploading it into a Course Management System may be a “last mile” hurdle.
-
Supporting interactive content: Interactive content frequently needs server-side functionality. While there are some interoperability standards, they oftentimes only support the least common denominator of the currently available Course Management Systems and bring along another set of complexities. This strongly limits the level of interactivity that reusable content resources can have.
On the other hand, the workshop underlined the importance of OERs for the future of learning, as they appeared in a number of the user and epic stories. The next generation of LMSs thus needs to overcome the above hurdles – particularly the need for assistance with locating appropriate content appeared in several stories.
Almost anytime during the workshop that data analytics, artificial intelligence, machine learning, and data-driven decision making were mentioned, it was followed by a disclaimer that – of course – privacy and data security regulations need to be respected. This, in turn, seemed insurmountable. Also, the need for “erasing data” was mentioned as if that were realistically possible in redundant, multiple backed-up systems with interlinked data sources, especially if that same data would be needed again once a user reenters degree or certification programs at the same or another institution.
It seems that most online platforms are successful because they make the user consent to all kinds of data usage; while that practice might be acceptable for voluntary activities, it is unacceptable for public institutions of higher education. It also seems that at any given time, laws about data usage are years behind what is actually happening in online platforms, which could mean that a data usage that is acceptable today may not be acceptable anymore tomorrow. Any next generation LMS needs to provide a way for the users to control and own their data.
It became very apparent during the workshop that data privacy and copyright regulations introduce a high degree of uncertainty in users, manifesting either in fear of potential consequences in cases of accidental oversights or a general “all open”-attitude, ignoring any consequences until they actually arise. Future systems need to ensure that regulations are adhered to and reduce uncertainty e.g., by transparently informing users about implications of specific actions. Diminishing this uncertainty will build trust and might lead to more openness across boundaries.
For content to become more reusable and remixable, it needs to turn into a package of content and functionality. Such a package is able to provide its own environment to display itself in different contexts, interact with the user, process submissions, and exchange data with all parties involved. These containerized applications may be anything from simple parsers to AI-agents and even components of what are considered Course Management Systems today (,e.g. a containerized Moodle might travel alongside content that requires it).
This choice also makes participating systems more easily maintainable and extensible, as app-like functionality would be published in the same way as content – see Section 5.1.
Users need to bring their own data with them, which can exist in “data pods”. Among other data these pods capture educational experiences the user had participated in. They exist in encrypted form, and the user is able to lock and unlock specific parts of these pods for other experiences.
In its most advanced form, these data could include certified credentials, like whole degrees. But also fine-grained interaction data is relevant: users can benefit from their accumulated interaction data throughout their life-long educational experiences – most notably through the use of artificial intelligence.
Even though today, the use of Artificial Intelligence (AI) in education is limited and not always well-advised, the workshop made it clear that many stakeholders see future potential in this technology, along the lines of the epic story for “HAL” in Figure 2.
There is an apparent conflict between users’ control over their data (Section 5.5) and the need for AI to be data-driven; an important architectural concept may be that on the one hand, users have their own “private” AI-agent for personalized decisions, but on the other hand, data is gathered from all users in depersonalized form – the private AI-agent is operating on the depersonalized data from other users.
The workshop identified expectations far beyond the original intent of thinking about the next generation of an LMS – instead, it revealed what might best be considered an ecosystem in which learner (and their data) and content (and its functionality) come together.
This will require an open-platform which hosts learning venues, in which users, other users, and content temporarily come together. It does not manage a course, instead it enables interaction between these entities, some of those interactions tagged as part of courses.
The workshop was one of the first steps in a continuing dialogue. While first prototypes are already being built at institutions represented in CampusSource, it “takes a village” to bring about a paradigm change in learning management. Thus, the plan is to follow up with future workshops and special interest groups, and thereby join forces.