Barriers to a wider Implementation of LMS in Higher Education: a Swedish case study, 2006-2011
urn:nbn:de:0009-5-33715
Abstract
This article investigates barriers to a wider utilization of a Learning Management System (LMS). The study aims to identify the reasons why some tools in the LMS are rarely used, in spite of assertions that the learning experience and students’ performance can be improved by interaction and collaboration, facilitated by the LMS. Lecturers’ perceptions about the use of LMSs over the last four years at the School of Engineering, University of Borås were investigated. Seventeen lecturers who were interviewed in 2006 were interviewed again in 2011. The lecturers’ still use the LMS primarily for distribution of documents and course administration. The results indicate that their attitudes have not changed significantly. The apparent reluctance to utilize interactive features in the LMS is analyzed, by looking at the expected impact on the lecturers’ work situation.
The author argues that the main barrier to a wider utilization of LMS is the lecturers’ fear of additional demands on their time. Hence, if educational institutions want a wider utilization of LMS, some kind of incentives for lecturers are needed, in addition to support and training.
Keywords: E-learning, Learning Management Systems, LMS, Engineering Education, Educational Technology.
A Learning Management Systems (LMS) is a 'toolbox' of programs intended to support learning, teaching and course administration (Britain & Liber, 1999; Seeger & Åström, 2005; Sigrén & Holmqvist, 2005; Bongalos, et al, 2006; Dutton, et al., 2004).
A lot of optimistic predictions about Learning Management Systems (LMS) as an aid to raise the quality of education and to promote learner-centered pedagogical approaches have been published and many institutions in higher education have invested considerable efforts and money to use LMSs in their educational practice (Brill & Galloway, 2007; Browne, Jenkins & Walker, 2006; Czerniewicz & Brown, 2009; Marshall, 2004; VEL4VET, 2010).
The common features in a LMS can be divided into the following four groups: Tools for course administration are tools that are used to administrate and monitor the course, for example, to disseminate or report the students’ grades. Tools for distribution are the tools that allow teachers to upload documents that students can access. It may be text material, but it may also be any kind of media files. This is a one-way, teacher-to-learner distribution of information. Tools for communication enable information to travel/be disseminated teacher-to-student and vice versa or student-to-student. E-mail is the most common example. Tools for interaction are discussion boards, wikis and other tools that elicit reflection and promote feedback. These tools are of particular interest since they facilitate student activity and cooperation, hence may enhance learning (Garrote Jurado & Pettersson, 2011).
Many researchers have asserted that collaborative exchanges about topics in a course may have a substantial impact on the learning experience (Xie & Ke, 2011). Others have suggested that tools for interaction, such as Online Asynchronous Discussion (OAD) can enhance students’ performance by promoting social interaction and the creation of a community of learners (Britain & Liber, 1999; Irwin & Berge, 2006; Blin & Munro, 2008; Bush & Mott, 2009; Findik & Ozkan, 2010; Ubell, 2000; Wilson & Stacey, 2004; Ladyshewsky & Gardner, 2008).
The School of Engineering at the University of Borås (hereafter UB) currently offers 19 study programs on Diploma, BSc, MSc and PhD levels in Engineering. In 2010, there were 2311 new students enrolled, 1125 of them were full-time students. There were approximately 70 employees.
From 1999 until 2007, the LMS WebCT was available to lecturers at the School of Engineering. The pattern of use of the LMS at the School of Engineering was investigated by the author of this paper, first in 2004 (Garrote, 2006) then again in 2009-2010 (Garrote Jurado & Pettersson, 2011). In 2004 at least some tools in the LMS were utilized in over 100 courses (ca 60% of all courses). The use of WebCT was voluntary, but support and basic training was provided. In 2008 a new LMS (Ping Pong) was introduced as a common LMS for UB. Since 2008, Ping Pong is the only LMS supported by UB. During its implementation the lecturers were offered lessons, workshops and consultations on request.
The results, obtained by going through archived files from all courses demonstrated that the LMS, both in 2004 and 2011, was used primarily to facilitate the teachers' work by replacing earlier methods of document distribution. Similar patterns appear in many surveys of institutions of higher education (Bongalos, et al., 2006; Dutton, et al., 2004; Phillips, 2006; Selwyn, 2007).
In 2006 a survey was conducted by interviews with 22 lecturers at the School of Engineering to find out why many features in the LMS were not utilized to a larger extent (Garrote Jurado & Pettersson, 2007). The hypothesis that lecturers would hesitate to adopt new technology for fear of its complexity or undesirable impact on education was not substantiated. The conclusion was that institutions of higher education could, and should, make a serious effort and make LMSs a standard tool in the educational practice. Only then would students and teachers reap the full benefits of it utilization.
In 2011 it was decided to make a new survey of the lecturers’ attitudes about LMS, since the pattern of use had not evolved as expected. This investigation included all lecturers at the School of Engineering at UB who had worked with the LMS WebCT and since 2008 with the LMS Ping Pong. Seventeen lecturers met the conditions, they had participated in the investigation in 2006 and they all agreed to be interviewed again. Fourteen interviews were conducted face-to-face and three by telephone.
Out of the 22 lecturers that participated in the investigation in 2006 four people had left their positions and one had switched to an administrative post.
This article provides information about barriers to a wider utilization of LMSs in higher education. Results obtained in 2006 and 2011 are used to investigate to what extent the lecturers’ attitudes can explain the pattern of use of a LMS. The specific reluctance to use interactive tools in the LMS is analyzed.
The investigation covers three areas of interest: Firstly, the lecturers’ opinions about the implementation process and support for the new LMS. Secondly, if, and to what extent, the lecturers’ expectations about the future impact of the LMS from 2006 have been fulfilled. Thirdly, changes in the lecturers’ attitudes about the use of the LMS between 2006 and 2011.
The investigation was conducted at an engineering school in Sweden where both lecturers and students have ample Internet and computer access but where the use of most of the tools in the LMS is not mandated. A similar situation is seen at many institutions of higher education in developed countries (Carvalho, Areal & Silva, 2011; Limniou & Smith, 2010; Zhou & Xu, 2007) and hence the results should be widely applicable.
Seventeen lecturers at the School of Engineering that participated in an investigation in 2006 were interviewed again in 2011 using a similar interview form consisting of yes/no questions, multiple-choice questions with a five-grade Likert scale and open-ended questions (see appendix 1). The same procedures were followed as in the investigation in 2006, when responses were recorded in written notes. At the end of each interview, the respondents had the opportunity to review their responses, which resulted in some changes and additional comments. When the interview was conducted by telephone the interview form were also sent to the respondent by e-mail.
Answers from the interviews in 2006 were collected from the original notes. At the time the lecturers were divided in three groups, depending on the respondent’s level of experience with the LMS. In 2011 all respondents were considered to be experienced users. The quoted comments in the result section are from the interviews in 2011, given either as answers to the open-ended questions or as spontaneous comments during the interviews. All answers are translated from Swedish by the author.
To facilitate the interpretation of the answers, the responses are sorted in three groups in the result section, reflecting three areas of interest.
In Figure 1 the responses to the question ‘What parts of the LMS have you used in your courses?’ are given.
Respondents had to choose from the following categories; a: Tools for distribution. b: Tools for course management. c: Communication tools. d: Tools for interaction. As an introduction to this question, the lecturers were informed about which tools of the LMS belong to each category and why, according to the investigation at the School of Engineering in 2011.
Figure 1: The use of the LMS Ping Pong after its introduction
All seventeen respondents had used the LMS to distribute material. Thirteen had used the LMS for the purpose of course management and some of the communication tools had been used by eleven respondents. The tools for interaction had only been used by six of the lecturers. These answers are consistent with the findings by Garrote Jurado & Pettersson (2011): The tools intended to enable and promote student interaction and cooperation are used far less than tools that facilitate teacher-centered practices at the School of Engineering.
In Figure 2 the responses to the question ‘The needs and wants of the lecturers were sufficiently taken into consideration during the implementation process of the new LMS?’ are shown.
Figure 2: The needs of the lecturers during the introduction of the new LMS
Only one respondent agrees in full with the statement, while two of the respondents were clearly dissatisfied. Judging from reactions during the interviews, the lecturers had not utilized all the offered support.
In Figure 3 the responses to the question: ‘The integration of the LMS in teaching and learning is facilitated by good pedagogical support?’ are shown.
Figure 3: Perceptions about the pedagogical support
Since Ping Pong was introduced, the lecturers have been offered support, in the shape of lessons, workshops and consultations on request. Several comments suggest that the lecturers lack motivation to learn more about the pedagogical application of the LMS and in the interviews there was no apparent demand for pedagogical support; ”…We have been offered several half day workshops in Ping Pong, but it's hard to find time for it.” Another lecturer said:”If you are interested in using a LMS then you should learn to use it better. I do not see the need for me. Why get an education if you do not need it.”
In Figure 4 the responses to the statement: ‘You want more education and support to be able to use all components of a LMS?’ in the 2006 investigation are compared with responses to the statement: ‘You were given sufficient working time, education and support to start using the different tools in the new LMS at your work?’ in 2011.
Figure 4: The introduction of the new LMS
The responses indicate that much of the needs and wants of the lecturers were met during the introduction of Ping Pong. However, some lecturers pointed out that they had not enough time to familiarize themselves with the LMS. In the answers to the open-ended questions many respondents expressed their interest in learning more about the LMS, but also show a strong reluctance to utilizing any tool that may increase their workload.
A typical comment was “... I want to know more about the program, it could facilitate my work....some parts of the LMS are time-consuming to use even if you would like to use them”. Another lecturer said “…I would like more opportunities for reflection about the use…..time to learn not only to send documents, but to use other tools that enable interaction between students”
In Figure 5 the responses to the statement: ‘The use of a LMS has significantly helped many students to meet the demands of the education?’ in 2011 is compared with responses to the statement: ‘A LMS can increase study performance’ in the 2006 investigation.
Figure 5: The perceived impact of LMS
The responses show that the expectations the lecturers had in 2006 have not quite been fulfilled. On the other hand, the interviewed lecturers agreed that the LMS is a convenient tool for students to access documents.
In Figure 6 the responses to the statement: ‘The LMS has significantly facilitated the work of the lecturers?’ in 2011 is compared with responses to the statement: ‘A LMS can facilitate the lecturer's work considerably?’ in the 2006 investigation.
Figure 6: The LMS impact on the lectures work
The LMS facilitates the lecturers’ work, but judging from the responses not as much as the lecturers expected in 2006. One lecturer said;“…I have not noticed any appreciable time saving. Therefore I do not think it will be used more in the future”.
In Figure 7 the lecturers’ opinions about the system in terms of pedagogical support in 2006 and 2011 are shown: ‘If you have questions about the LMS you can always get help’ in the study of 2006 and 2011 are compared.
Figure 7: Opinions about the pedagogical support system
In both surveys, the responses are spread from “Agree in full” to “do not agree at all”. Some lecturers want more education and support;”…To properly use an LMS you need more training.”, “…Teachers must learn to use a LMS for more than to distribute documents.” Apparently not so many lecturers are satisfied with the access to the pedagogical support in 2011 as in 2006.
Others are hesitant;”…I am self-taught; sometimes I get in touch with the institution's support person to ask if something is allowed to do and get recommendations. The central pedagogical and support unit of the University of Borås has a number of workshops but there is no work time set aside to attend”, ”…I think the LMS Ping Pong have severe limitations to be used more extensively in teaching”.
In Figure 8 the lecturers’ opinions about the system for technical support system in 2006 and 2011 are shown.
Figure 8: Opinions about technical support system
In 2011 most lecturers seem confident that technical support is available. However, there are still some comments indicating dissatisfaction with the support. As one respondent put it;”…The support person at the School of Engineering works only part time. Often when you get problems no one is here and therefore the LMS cannot be used more extensively.’
In Figure 9 the lecturers’ expectations about the future impact of LMS use in 2006 and 2011 are compared.
Figure 9: The expectations about future impact of LMS use in the future
The optimism of 2006 has been replaced by a far more reluctant attitude in 2011. Still, the lecturers regard the LMS as a useful tool in their practice. A certain curiosity about the possibility to use more tools in the LMS in the future could be seen; “...I think there are more tools that would fit into my teaching. For example, tools for communication and interaction.”, “…An LMS can be a great addition to [traditional] teaching”.
In Figure 10 the lecturer’s expectations about future impact on students’ performance in 2006 and 2011 are compared.
Figure 10: The expectations about the future impact of LMS and students’ performance
The responses show the same pattern as above (Figure 7): high expectations in 2006 and a cautious, albeit still positive, attitude in 2011. A number of comments indicate that the main difference is the lecturers’ perception of the effort it takes to utilize the interactive tools; “…I would have liked to use more interaction tools with students during the course, but lack of time and the problem that students do not want to get involved, means that the LMS is used only as a distribution tool.”
In Figure 11 the lecturers’ assessment of the work climate at the school of Engineering in 2006 and 2011 are compared.
Figure 11: The climate at the School of Engineering
Since all teachers were expected to use the LMS in 2011, at least some parts of it, it is natural that the use of the LMS was discussed more frequently. Still, the results for 2011 indicate that lack of peer support and/or peer recognition may be a barrier to a more extensive use of the LMS.
In Figure 12, the lecturers’ perceptions of the institutional policy about the use of LMS in 2006 and 2011 are compared.
Figure 12: The institutional policy about the use of LMS
The difference shows that the lecturers are encouraged to use the LMS more in 2011, but their comments suggests that they do not feel much pressure to apply a wider range of features. One lecturer said;“[…I am not aware of any…] policy about how teachers should use the LMS. I use the LMS in all my courses, but solely for the distribution of documents”.
The LMS is well-established as a teaching aid, although only a narrow set of tools are used in most courses. During the interviews nothing indicated that respondents use social media as an alternative to the LMS. The most common use of the LMS is for course evaluation and distribution of documents to students. The lecturers appear to be reasonably satisfied with the technical support offered by the School of Engineering during and after the introduction of the new LMS in 2008. But they appear sceptical about the pedagogical support and not really motivated to integrate more features into their practice.
When the situations in 2006 and 2011 are compared, there are no significant differences in the way the LMS is used or in the lecturers’ attitudes toward the use of LMS. The common opinion among the lecturers is that a LMS may facilitate their work considerably, but the beneficial effect on student learning is not obvious, except for the convenient access to course material.
So, access to computers, a LMS and support are not sufficient to make lecturers utilize a wider range of tools in their work. In particular, little is done to promote students´ interaction and collaboration by means of interactive tools in the LMS.
When the lecturers’ expectations concerning the future impact of the LMS on the education are compared (Figure 9 and 10) it is clear that the lecturers are not as optimistic in 2011 as they were in 2006. Still, most of them believe that a LMS can have a positive impact on education and their work situation. Hence the reluctance to utilize tools for interaction cannot be explained by doubts about their potential pedagogical value.
A feature in the LMS that saves the lecturers’ time and effort, without adversely affecting the students’ performance, will probably be used eventually. Still, it may take some time and effort for a lecturer to start using a new tool. The “perceived ease of use” describes how hard it is to get started. A good start depends on the software quality, but also on the availability of good technical and pedagogical support. Whether available technology will be used depends on the balance between the perceived ease of use and the expected outcome (Al-Senaidi, Lin & Poirot, 2009; Wang & Wang, 2009).
To explain the difference in utilization between the tools for interaction and the other parts of the LMS we must look at the expected impact on the lecturers´ work situation. The impact of technology on the work situation can be divided into two parts; as change in workload and change in outcome, i.e. students’ performance. In Figure 13 the situation is visualized as a seesaw.
Figure 13: The balance between expected impact on workload and expected impact on sudents´performance (drawing by the author)
On the left-hand side is the perceived impact on the lecturers’ workload: if a tool is likely to save time and effort we move closer to the middle and the chance increases that the scale will turn and the tool be utilized. With better support, education and user-friendly programs, the perceived ease of use will increase. Another factor is the students’ computer literacy and prior experience of the feature in question; if the students are familiar with a tool it is probably easier for the lecturer.
On the right-hand side in Figure 13, the expected outcomes are considered. The impact on students’ performance can be positive or negative, giving better or worse results. Tools with perceived positive effects on students’ performance and/or learning experience are likely to be utilized as long as no extra effort is needed from the lecturer. A tool that is expected to require more time and effort from the lecturer may still be utilized, but only if the expected effects on students’ performance and/or learning experience outweigh the increased demands on the lecturer.
Since the decision to use tools in a LMS usually is left to the lecturers, it is easy to see that it will take a lot of positive expectations about learning outcomes from tools for interaction to balance a fear of extra workload.
With this in mind we can now explain the big difference in utilization of the four groups of tools in the LMS.
The tools for course administration are special since their application is usually decided by the educational institution rather than individual lecturers. Lecturers usually are requested to keep track of, and report on test results, students’ attendance etc. Such information may be handled with the LMS or other programs.
The tools for distribution of documents can be expected to lessen the lecturers’ workload by replacing paper copies, provided that the students have sufficient access to computers and the internet. This saves money and improves the students’ access to course material. In a similar way, the tools for communication are convenient and possibly time-saving as long as the students have sufficient access to computers and the internet. Those tools are used by most lecturers in this investigation.
The tools for interaction are primarily intended to improve the quality of education by facilitating student activity and cooperation, not to save time or effort for lecturers. On the contrary, it has been asserted that lecturers may have to spend time and effort to monitor and guide the use of tools for interaction (i.e. discussion forum) in a course (Murphy, 2004; Salmon, 2000). There are studies indicating that when students have experienced collaborative learning by means of a LMS they want to increase the use of tools for interaction (Limniou & Smith, 2010). In a recent study Heirdsfield et al. (2011) noted that students were more positive towards the use of interactive features than teachers.
It should be noted that demands from the students’ are not taken into consideration in Figure 13. Students’ requests may have an impact on the lecturer’s choice of tools, but only if the students are familiar with the features in question. In the interviews some comments indicated that the students’ lack of experience is an obstacle to start using more features in the LMS; “To make it work better the students have to learn to use it”, “I would have liked to use more interaction tools with students during the course, but lack of time and the problem that students do not want to get involved, means that the LMS is used only as a distribution tool”.
It is possible that some lecturers are not motivated to utilize tools for interaction. Traditional modes of teaching assume that the teacher knows whatever the students have to learn and information goes from the teacher and textbooks to the students. Education outcomes are usually measured by exams, testing the students’ knowledge within specific courses. In this mode of teaching, students’ interactions are not considered to be of any great use or interest.
In contrast, students’ interactions are important in constructivist modes of teaching (Duffy & Jonassen, 1992). Those methods are based on the idea that individuals construct their own knowledge from information, experience and interaction with other people or the outside world. From this perspective, memorizing information is only one step in the creation of knowledge; it is only by using and processing it that a person can transform memorized information into functional knowledge.
The interviewed lecturers emphasize lack of time as the major barrier to the use of tools for interaction. To give more attention to the students and society’s long-term interest lecturers would have to overcome their fear of extra demands on their time and look at the aims and purposes of education beyond the scope of single courses in their practice since a lot of the beneficial effects are long-term and interdisciplinary.
Some kind of incentives are probably needed to overcome the lecturers’ fear that the use of interactive tools will increase their workload (Llamas-Nistal, Caeiro-Rodriguez & Castro, 2011). Such incentives might be buy out time; special projects available for interested teachers who want to redesign their courses using interactive LMS tools; changes to criteria for promotion or wage increases; teaching and learning awards for excellence in the use of LMS or increased publicity for role models who demonstrate excellence.
It is not clear if the use of tools for interaction will actually increase the lecturers’ workload, except perhaps during the initiation phase. If the tools are used routinely, lecturers may save time, for example students may require less individual tutoring and the need for re-exams may decrease.
It has been argued before that staff training and support will not suffice to change lecturers’ practices and that a major change in the way computer technology is utilized can only take place as a result of major changes in the educational process (Blin & Munro, 2008; Selwyn, 2007).
Lecturers frequently use a LMS to distribute documents, send messages and for administration of courses. In contrast, tools intended to facilitate interaction, collaboration and the creation of communities of learners, are used sparingly. Lecturers decide which features to utilize, depending on the expected impact on their workload.
Tools for interaction may have a significant, beneficial impact on education by promoting and facilitating learner-centered procedures. But a lot of the beneficial effects are long-term and interdisciplinary, outside the frame defined by time and subject for specific courses. As a result the lecturers’ fear of additional workload will in most cases outweigh the potential benefits for the students. Hence, some kind of incentives for lecturers are needed if educational institutions want to improve the students’ learning experience by a wider utilization of Learning Management Systems. Future research is needed to determine if, and to what extent such incentives can stimulate lecturers to routinely utilize tools for interaction.
The author thanks Dr. Christine Räisänen, Dr. Michael Christie and Tomas Pettersson for their advice in the writing of this article.
1 What parts of the LMS have you used in your courses? (Figure 1)
(Information were given about which tools of the LMS belong to each category and why)
A: Tools for distribution.
B: Tools for course management.
C: Communication tools.
D: Tools for interaction
The statements 3-12 were assessed on a 5-grade Likert scale. On the scale 1 equals “Agree in full” and 5 equals “Do not agree at all”.
The impact of a LMS at the School of Engineering:
2. The use of a LMS has significantly helped many students to meet the demands of the education? (Figure 5)
3. The LMS has significantly facilitated the work of the lecturers? (Figure 6)
The introduction of the LMS Ping-Pong at the School of Engineering:
4. You were given sufficient working time, education and support to start using the different tools in the LMS at your work? (Figure 4)
5. The needs and wants of the lecturers were sufficiently taken into consideration during the implementation process of the new LMS? (Figure 2)
6. There is good help to get if there is a problem with the LMS? (Figure 8)
7. If you have questions about the LMS you can always get help? (Figure 7)
The climate at the School of Engineering:
8. There is a discussion going on among the lecturers on how to use the LMS? (Figure 11)
9. Within the institution the lecturers are encouraged to use LMS? (Figure 12)
10. The integration of the LMS in teaching and learning is facilitated by good pedagogical support. (Figure 3)
The future of LMS in engineering education:
11. A LMS can facilitate the lecturer’s work considerably? (Figure 9)
12. A LMS can increase study performance? (Figure 10)
The final questions, 13 and 14, were open-ended; the answers were recorded by key words.
13. How do you think a LMS should be used to facilitate the educational process the most in the future? (What parts and under what circumstances).
14. Are there any changes you would like to see in the policy and support system for the LMS at the school of engineering and/or the University of Borås?
Note: During the interviews the questions were in Swedish.
Al-Senaidi, S.; Lin, L.; Poirot, J.: Barriers to adopting technology for teaching and learning in Oman. In: Computers & Education, 53(3), 2009, pp. 575-590. doi: 10.1016/j.compedu.2009.03.015 (last check 2012-08-23)
Blin, F.; Munro, M.: Why hasn't technology disrupted academics' teaching practices? Understanding resistance to change through the lens of activity theory. In: Comput. Educ., 50(2), 2008, pp. 475-490.
Bongalos, Y. Q.; Bulaon, D. D. R.; de Celedonio, L. P.; de Guzman, A. B.; Ogarte, C. J. F.: University teachers' experiences in courseware development. In: British Journal of Educational Technology, 37, 2006, pp. 695-704.
Brill, J. M.; Galloway, C.: Perils and promises: University instructors’ integration of technology in classroom-based practices. In: British Journal of Educational Technology, 38(1), 2007, pp. 95–105.
Britain, S.; Liber, O.: A Framework for Pedagogical Evaluation of Virtual Learning Environments (143 Reports: Research; 160 Tests/Questionnaires). United Kingdom. 1999. http://www.jtap.ac.uk/reports/htm/jtap-041.html [verified 7 February 2012 at http://www.leeds.ac.uk/educol/documents/00001237.htm (last check 2012-08-23)]
Browne, T.; Jenkins, M.; Walker, R.: A longitudinal perspective regarding the use of VLEs by higher education institutions in the United Kingdom. In: Interactive Learning Environments, 14(2), 2006, pp. 177-192. doi: 10.1080/10494820600852795 (last check 2012-08-23)
Bush, M. D.; Mott, J. D.: The name assigned to the document by the author. This field may also contain sub-titles, series names, and report numbers.The Transformation of Learning with Technology: "Learner-Centricity, Content and Tool Malleability, and Network Effects". In: Educational Technology Publications, 49(2), 2009, pp. 3-20.
Carvalho, A.; Areal, N.; Silva, J.: Students' perceptions of Blackboard and Moodle in a Portuguese university. In: British Journal of Educational Technology, 42(5), 2011, pp. 824-841.
Czerniewicz, L.; Brown, C.: A study of the relationship between institutional policy, organisational culture and e-learning use in four South African universities. [Article]. In: Computers & Education, 53(1), 2009, pp. 121-131.
Duffy, T. M.; Jonassen, D. (Eds.): Constructivism and the technology of instruction: A conversation. Lawrence Erlbaum Associates, Hillsdale NJ, 1992.
Dutton, W. H.; Cheong, P.; Park, N.: The Social Shaping of a Virtual Learning Environment. In: Electronic Journal of e-Learning, 2(2), 2004, pp. 1-12.
Findik, D.; Ozkan, S.: Identifying success factors for WBLMS use by instructors of engineering departments. Paper presented at the Frontiers in Education Conference (FIE), 2010 IEEE, 27-30 Oct. 2010
Garrote Jurado, R.; Pettersson, T.: Lecturers' Attitudes about the Use of Learning Management Systems in Engineering Education: A Swedish Case Study. In: Australasian Journal of Educational Technology, 23(3), 2007, pp. 327-349. http://www.ascilite.org.au/ajet/ajet23/garrote.html (last check 2012-08-23)
Garrote Jurado, R.; Pettersson, T.: The use of learning management systems: A Longitudinal Case Study. In: eleed, 8(1), 2011 . http://eleed.campussource.de/archive/8/3145 (last check 2012-08-23)
Garrote, R.: The use of learning management systems in Engineering Education: A Swedish case study. In: Christie, M. F. (Ed.): Shifting Perspectives in Engineering Education: Chalmers Strategic Effort on Learning and Teaching (C-SELT) Chalmers University of Technology. pp. 213-226, 2006.
Heirdsfield, A. M.; Walker, S.; Tambyah, M. M.; Beutel, D. A.: Blackboard as an online learning environment : what do teacher education students and staff think? In: Australian Journal of Teacher Education, 36(7), 2011, pp. 1-16.
Honkaranta, A. M. a. A. M.: Bridging the Gap between Advanced Distributed Teaching and the Use of Learning Management Systems in the University Context. Paper presented at the Seventh IEEE International Conference on Advanced Learning Technologies (ICALT 2007), 2007.
Irwin, C.; Berge, Z.: Socialization in the Online Classroom. In: e-Journal of Instructional Science and Technology (e-JIST), 9(1). 2006. http://www.ascilite.org.au/ajet/e-jist/docs/vol9_no1/papers/full_papers/irwin_berge.htm (last check 2012-08-23)
Ladyshewsky, R. K.; Gardner, P.: Peer assisted learning and blogging: A strategy to promote reflective practice during clinical fieldwork. In: Australasian Journal of Educational Technology, 24(3), 2008, pp. 241-257.
Limniou, M.; Smith, M.: Teachers’ and students’ perspectives on teaching and learning through virtual learning environments. In: European Journal of Engineering Education, 35(6), 2010, pp. 645-653. doi: 10.1080/03043797.2010.505279 (last check 2012-08-23)
Llamas-Nistal, M.; Caeiro-Rodriguez, M.; Castro, M.: Use of E-Learning Functionalities and Standards: The Spanish Case. In: Education, IEEE Transactions on, 54(4), 2011, pp. 540-549.
Marshall, S. J.: Leading and managing the development of e-learning environments: An issue of comfort or discomfort? Paper presented at the ASCILITE 2004. 2004 . http://www.ascilite.org.au/conferences/perth04/procs/marshall-keynote.html (last check 2012-08-23)
Murphy, E.: Recognising and Promoting Collaboration in an Online Asynchronous Discussion. In: British Journal of Educational Technology, v35 n4, 2004, pp. 421-431.
Phillips, R.: Tools used in Learning Management Systems:Analysis of WebCT usage logs. Paper presented at the Ascilite. 2006. http://www.ascilite.org.au/conferences/sydney06/proceeding/pdf_papers/p208.pdf (last check 2012-08-23)
Salmon, G.: A model for CMC in education and training. E-moderating. The key to teaching and learning online. London: Kogan Page, 2000.
Seeger, M. A.; Åström, A.: Distansutbildning via lärplattform: en överlevnadsstrategi? Uppfattningar inom Sveriges naturbruksgymnasier. In: Distance Education and Learning Management Systems: a strategy for survival? Beliefs among the Agricultural Colleges of Sweden, 2005.
Selwyn, N.: The use of computer technology in university teaching and learning: a critical perspective. In: Journal of Computer Assisted Learning, 23(2), 2007, pp. 83-94.
Sigrén, P.; Holmqvist, H.: Syntes och analys av tidigare kravspecifikationer för upphandling av LMS inom den svenska högskolan 2000 – 2004. Härnösand: Myndigheten för Sveriges nätuniversitet, 2005.
Ubell, R.: Engineers turn to e-learning. Spectrum, IEEE, 37(10), 2000, pp. 59-63.
Wang, W.-T.; Wang, C.-C.: An empirical study of instructor adoption of web-based learning systems. In: Computers & Education, 53(3), 2009, pp. 761-774. doi: 10.1016/j.compedu.2009.02.021
VEL4VET: Innovative Learning Platform for VET. 2010. http://www.vles4vet.eu/VLEs4VET_Research_Report.pdf (not available on 2012-08-23), alternate URL: http://www.ieducate.eu/admin/upload_data/Resources/3-VLEs4VET_Research_Report.pdf (last check 2012-08-23)
Wilson, G.; Stacey, E.: Online interaction impacts on learning: Teaching the teachers to teach online. In: Australasian Journal of Educational Technology, 20(1), 2004, pp. 33-48.
Xie, K.; Ke, F.: The role of students' motivation in peer-moderated asynchronous online discussions. In: British Journal of Educational Technology, 42(6), 2011, pp. 916-930.
Zhou, G.; Xu, J.: Adoption of educational technology ten years after setting strategic goals: A Canadian university case. In: Australasian Journal of Educational Technology, 23(4), 2007, pp. 508-528.