HTML

Introduction

The fit between computer support and the educational contexts in which it is used is a major success factor (cf. Kerres 1999; Janneck/Strauss 2002). This applies especially to computer supported cooperative learning (CSCL) because CSCL systems do not primarily support the design and presentation of learning content, but are rather used as variable cooperation means for multiple didactical purposes. (cf. Wessner/Pfister 2001; Reinmann-Rothmeier 2003). To prevent ambiguity as a result of flexibility and the multitude of use possibilities, CSCL applications need to be matched carefully with their respective context of use. Our basic assumption is that the successful use of CSCL applications does not only depend on the software design, but mainly on accompanying didactical measures in their context of use (cf. Friedrich et al. 2000)

This paper exemplarily analyses the fit of the CSCL system CommSy in an open learning scenario by means of log file analyses. We derive implications for the use of CSCL systems and its evaluation by means of log file analyses. In the following section we characterize the foundations of our work: The didactical concept of open learning scenarios, their support through the CSCL system CommSy, and also different fields of action for using software support in open learning scenarios. Subsequently, we demonstrate the results of log file analysis concerning the use of CommSy in open learning scenarios at the department of Informatics at the University of Hamburg. We relate our findings to our prior experiences and recommendations concerning the didactical use of CommSy. We conclude with a summary of our results.

CommSy as Software Support in Open Learning Scenarios

The following description of using software support in what we call open learning scenarios draws on both educational theory and our experiences in several courses at the Universities of Hamburg and Tuebingen from 2001 to 2003 (cf. Strauss et al. 2003; Janneck et al. 2003). Altogether more than 100 Students took part in these courses. All courses were structured according to the principles outlined below, and CommSy was used as software support. All courses were evaluated by means of several methods: The students were asked to assess the course and the use of CommSy in questionnaires. Additionally, we conducted focus groups with participants of the courses. Furthermore, the anonymized log files of the CommSy use were evaluated.

Open learning scenarios

Open learning scenarios are based on the method for project-based teaching and learning. We use the following four principles didactical guidelines (cf. Janneck et al. 2003; Janneck/Krause 2004):

Student-centered learning: In an open learning scenario we do not insist on predefined learning targets regarding the content of students’ work. That applies to the choice of topics that students wish to work on as well as to the organization of their learning process and the presentation of their interim and final results. Instead we ask for drafts, progress reports, and final reports and encourage continuing mutual exchange among the teams. We expect open learning processes to be successful when students can develop and work on their own learning interests. This usually means that the chosen topics are of high practical and current interest.

Self-organized team work: Students form teams of three or four people to work on their self-chosen learning interests and tasks. They organize their work independently and have to account for their work results. In interim workshops and final presentations they introduce their work to us and to their fellow students.

Documentation and publishing: We encourage students to publish their work in appropriate ways. This comprises a final presentation open to the university members or even the general public, electronic publishing via the World Wide Web or submitting a conference paper. Thus, we encourage students to materialize a final product.

Role models: Our role as teachers changes fundamentally when we give up the traditional model of conveying knowledge in favor of facilitating and moderating the students’ own learning processes. We accompany and maybe guide students on their ”research journeys” but we decide neither on the means of travel nor on the destination. That also means that students have to give up their more passive role and take on responsibility for their route planning instead of relying on experienced tour guides.

Self-determined software use: The intensity of software use should be up to the students themselves and is subject to change. Teachers should not dominate the system usage but initiate the participants’ reflections on different ways of usage. In order to be a collective support the use purposes for a software system need to be negotiated among the participants.

Our concept of open learning scenarios enriches traditional teaching especially by allowing students to develop and express their own academic and learning interests and learn to organize team work, which is an important social skill. Furthermore, software support is explicitly introduced as methodical element. The evaluation of our courses showed that in spite of students’ former negative experiences with team work and some insecurity at the beginning of the course participants produced high-quality working results and rated open learning scenarios positively (cf. Strauss et al. 2003; Janneck et al. 2003).

CommSy

As software support we used the groupware system CommSy (http://www.commsy.de). CommSy stands for Community System and consists of two mutually related parts (cf. Jackewitz et al. 2004):

Project Rooms support learning groups of approximately 10 to 30 members in multiple ways: News and events can be announced, specific topics can be discussed in discussion forums, each member has a personal homepage to present him/herself to the group, and annotations can be made for every item in a Project Room. Working material can be collected in a simple reference manager and put in context by attaching it to any other item (e.g. an announced event). A group editor is available for cooperative writing of HTML documents.

The Community Room is an archive "in process" designed to support teaching and learning individuals over a longer period of time. Therefore, it offers a listing of courses and extracurricular activities in current and earlier semesters, complete with information on the responsible persons, appending Project Rooms and archived study material. Users can store and find learning material of any kind. It is possible to select material for specific purposes, to add comments to a material or to rearrange it. The Community Room does not only support information retrieval but also serves as a publishing medium. Students and faculty may produce results relevant for a broader discourse and structure the material in order to fulfill different needs, thus contributing to the context of the future usage of the archive.

The two parts of CommSy are related as follows: Learning groups within the broader community may take relevant material from the Community Room into their Project Room in order to discuss it more intensively, improve it and put it back to the Community Room after finishing their project. We thus consider Project Rooms and the Community Room as mutually related media designed to support project-based learning in university education.

CommSy incorporates the following three design principles (cf. Jackewitz et al. 2004):

Easily manageable basic technology: For users, CommSy is accessible with any web browser. All development technologies are publicly available: MySQL database, APACHE web server, PHP4 script language.
Easy individual usage: CommSy offers an adequate range of functions motivated by pedagogical aims. Users are not confronted with an excess of functions like they can be found in other groupware systems. The different functions all have a similar dialogue structure. Once it is understood, it serves for all functions. CommSy does without fancy layout features and thus enables a comfortable usage even with slow Internet access.
Responsible usage in cooperation: Project Rooms are exclusively accessible to members of a certain group, who need to register before entering. There are no anonymous postings. There is no concept of roles implemented into the software. All settings are equally visible to all members. Each member of a Project Room is allowed to read any entries and to use any function. Especially, there is no distinction between students and teachers. Thus, communication takes place on a common basis.

Facilitating and Using CommSy in Open Learning Scenarios

Typically CSCL applications can be used very flexibly in a variety of didactical settings. To avoid misconceptions or ambiguities, its concrete usage in a certain context should be negotiated among all participants. To do so we identify three areas of activity: (1) shaping the context of use, (2) supporting initial use and (3) supporting continuous use.

(1) Shaping the context of use comprises measures that make the software available to the students. These are mostly tasks to be accomplished by the teachers. First of all, the application needs to be installed and configured. Since we used a CommSy server offered by an external service provider for our courses, efforts were low in this regard (cf. Bleek et al. 2003). Adapting a Project Room for a certain context comprises choosing the desired range of functions, colors, and a nickname for the Project Room. Furthermore, some initial content – e.g. introductory reading – should be provided prior to the beginning of the course: an empty Project Room stifles the students’ motivation of use (cf. Bleek et al. 2000; Hinze/Blakowski 2002). Some measures in shaping the context of use have to be carried out continuously, e.g. administrating the system, managing user accounts or providing access to the Internet in case the students lack online opportunities.

(2) Supporting initial use at the beginning of the course aims at encouraging students to appropriate and use the software independently. We gradually decrease our support as students learn to deal with the software themselves (cf. Bleek et al. 2000; Bremer 2002). To do so, we assign several tasks with increasing degrees of difficulty, e.g. to enter an email address and telephone number or to upload an assignment. These tasks themselves are usually well-known to students. The challenge is to carry them out using the software. Thus, we foster skills in computer use and media competency among students (cf. Bleek et al. 2000; Feeken et al. 2002). We consider as highly important that a mutual understanding concerning the software use arises among the participants. Teachers and students should agree on expectations and obligations that arise from the use of CSCL application in the course (cf. Arnold et al. 2002; Hinze/Blakowski 2002).

(3) Supporting continuous use: Once the students are familiar with the software they need to be motivated and supported to make use of the software for their own purposes and goals. In this regard, teachers need to be aware that their own usage of the software serves as a continuous example for their students. To encourage use, teachers could follow up on comments in a discussion forum during class, or make use of materials the students uploaded. Also, the Project Room can be used to document the students’ work in a continuous way and to present intermediary as well as final results. Furthermore it is promising to initiate a reflection upon the system usage. Importantly, we accept if the participants choose to use other communication media or modes of cooperation than those we recommended: It is essential, however, that they reflect upon their choices (cf. Janneck et al. 2003).

Using Log File Analysis to Evaluate Didactical Fit

To confirm our recommendations concerning the match between software and didactical context empirically, we evaluated the use of CommSy in open learning scenarios using a variety of methods including questionnaires, focus groups and log file analysis. In this paper we focus on the use of log file analysis to evaluate the fit of CSCL applications and the didactical context in which they are used. To reach profound insights into the merits of this methodology we relate log file data to qualitative data from our focus groups.

A detailed account of our research methodology is given by Strauss/Pape (2004). An introduction to evaluation research is given by e.g. Bortz/Döring (2002) and Wottawa/Thierau (1998).

Log file analysis is of interest in online research because it allows a quasi-observation of users’ actions: By recording the users’ actions in the system, objective data such as quantity, point in time, continuity of use, preferred paths of navigation etc. can be extracted. Individual users may be observed during a certain timeframe as well as cross-sectional data about certain groups of users may be collected to find out e.g. which functionality of a software or what elements of a website are mainly used.

Usually server side logs are being evaluated, registering requests of all clients that used a certain service. In order to collect more general information about the actions of certain users client side logs, such as the browser history, are used in some studies. Döring (2003) gives an overview of possible applications of log file analysis in Internet research.

Grob et al. (2004) argue that the following data – which can be obtained by log file analysis – is useful for the evaluation of e-learning applications:

Frequency of use of e-learning applications,

associations between requested e-learning resources,

identification of student groups with similar usage patterns,

identification of dominant e-learning sequences and learning paths.

Whereas Grob et al. (2004) consider e-learning infrastructures as a whole, we focus on matching e-learning software (especially CSCL applications) with concrete didactical contexts. Therefore, our research questions address the following additional points:

Identification of foci and occasions of use,

comparison of different user types (e.g. regular vs. sporadic users),

identification of use patterns.

Comparing different user types is of interest regarding didactical fit in order to analyze whether differences in usage frequency are actually related to differences in use patterns (e.g., do regular and sporadic users nevertheless perceive similar occasions for use?). By the means of recording patterns and motives of use, we aim to find out what measures can be carried out by teachers to motivate students.

The log data we collected contained the anonymized user account name, the http request as well as its timestamp. Using this data we were able to reconstruct and to evaluate every single user´s click while accessing the system. The students consented to this procedure. To avoid having to work directly with the database we developed a tool featuring a web interface to analyze the data.

In the following sections we present the results of analyzing the log data of a Project Room that was used to support a course entitled ”Computer Supported Cooperative Learning (CSCL)” in the winter semester of 2001/2002 at the Department for Informatics of the University of Hamburg. 16 students participated in the course, which was organized by two teachers. Analyzing other similar courses resulted in almost identical conclusions.

Foci and occasions of use

Figure 1 shows the requests of students and teachers accessing their Project Room during the semester on a weekly basis.

Figure 1 - Weekly use comparing teachers and students

Prior to the beginning of the semester the teachers’ curve shows heavy activities and reaches its maximum during calendar weeks 45 and 46 (the first two weeks after the start of the semester). The participants’ curve shows its maximum in calendar week 46, which reflects the important phase of choosing a study topic and forming working teams. In this phase teachers were notably active in the Project Room, uploading relevant study literature and helping with the team-building process. They also proposed a method for brainstorming on possible study topics in the Project Room.

The next maximum of use intensity can be found in calendar week 49, prior to the first interim presentation where the teams where supposed to introduce their research topic and working schedules as well as present first results. During the Christmas holidays (calendar weeks 52 and 1) the Project Room was scarcely used. However, activities revived substantially afterwards without any teachers’ intervention. Prior to the final presentation the participants’ curve increases in calendar week 6. Not so the teachers’ curve: They only start to show increased activity towards the end of the semester to make documentations and results available in the Project Room. The participants, making use of the materials provided, produce a slightly increasing curve during the calendar week 8.

The curve progression clearly illustrates the teachers’ effects during the course: They provide initial input to motivate students to take advantage of the media they offered (e.g. the Project Room) and to support them while choosing a research topic and forming study teams. During the students’ process of working more and more independently the teachers retreat in their use of the Project Room. The students are working and making use of the software tool autonomously.

Figure 2 shows the frequency distribution regarding the weekdays on which teachers and students made use of the Project Room, showing that teachers mainly used the system on Wednesdays and Thursdays, having scheduled the preparation of the course on these days. They hardly used the Project Room on weekends.

Students, however, heavily used the Project Room on Mondays, when class was scheduled and they were given time to work autonomously in their teams. Unlike their teachers they also used the system intensively on the weekend, especially on Sundays, possibly preparing themselves for class on Monday.

Figure 2 - Frequency distribution regarding weekdays

Analyzing foci and occasions of use we draw the following implications for matching CommSy use and didactical context:

The analysis of log data confirms our recommendation to link the use of CommSy systematically with events occurring during the course: The ups and downs of the use curve can be closely related to the progression of the course. Examples are the use of the Project Room to support the team building process, to prepare presentations, or during the Christmas holidays.

Conversely, the foci and occasions of use elaborated above can serve as examples of how to use CommSy or other CSCL applications in other contexts. Phases of conceivably intense usage are especially appropriate to carry out didactical interventions (e.g. certain days of the week or periods of time during the semester).

The log file analysis furthermore confirms our notion of teachers as facilitators of learning (cf. Rogers 1969): They invested a lot of effort to set up the Project Room in the beginning and were always present in the background. In order to give the students the opportunity to develop their work independently, their intensity of attendance decreased over time.

Types of users

Table 1 shows that on average the teachers produced more requests than the students. At average the teachers were about three times as active. Since the use of CommSy as software support for this course was generally evaluated as successful, we imply that the teachers’ intensive commitment within the Project Room contributes to a productive CommSy use.

		requests
	N	total	average per person
students	16	13.526	845
teachers	2	5.332	2.666

Table 1 - Numbers of requests of teachers and students

Considering the frequency of use, three types of users can be identified (figure 3):

Frequent users (S1-S5): 1000 to 2000 requests

Average users (S6-S11): 600 to 1000 requests

Sporadic users (S12-S16): less that 600 requests

Obviously there are great differences concerning the frequency of use among the students. Whereas the most active users produced almost 2000 requests, the least active person did not reach the level of 100 requests. The focus group discussion carried out for course evaluation, however, revealed that in some teams only one student at a time was logged on the system and made entries for the whole team. These were usually also students who already knew the software well and enjoyed working with the Project Room. This accounts for some of the differences between frequent and sporadic users

Figure 3 - Students’ (S1-S16) requests

Having identified three groups of usage frequency, we analyzed their use patterns. Figure 4 shows the average number of requests by members of the respective group on a weekly basis. One can see that the curve progression is very similar. Thus, we conclude that in spite of the differences in their frequency of use, the different types of users perceived similar occasions and motives for use and acted accordingly. Even though the sporadic users did not take advantage of the Project Room very intensively, they were motivated to use the system by the same measures and occasions as the frequent users. Furthermore, even the sporadic users never stopped working during the semester.

Figure 4 - Requests of the different user types a weekly basis

In addition, we point to a phenomenon that can be observed during calendar week 48: The frequent users’ curve shows its local maximum prior to the other two groups. This can be ascribed to the fact that the active users appreciate new contributions earlier and spread them to their fellow students, thus decreasing the latter’s pressure to inform themselves.

As an implication for matching software use and context, we emphasize that differences in frequency of use among the students have to be accounted for. A mutual negotiation of expectations concerning software use is helpful in dealing with the situation. We recommend to pay particular attention to the occasions that motivate students to use the system and to mind that they do not stop working completely.

Patterns of use

Results of the focus group discussion carried out for course evaluation show that the different study teams made different use of the functionality (rubrics) of the Project Room. We contrast these findings with log file data: Table 2 shows the number of requests of the four study teams. On the basis of this data three very active groups (2,3,4) and one clearly less active group (1) can be identified.

		requests
	N	total per working group	average per group member
working group 1	4	1.438	360
working group 2	4	3.985	996
working group 3	4	4.420	1.105
working group 4	4	3.683	921

Table 2 - Number of requests of each study team

For further analysis we compare the two most active groups (2 and 3). To analyze their use patterns we illustrate their respective frequency of use related to the different rubrics of the Project Room (figure 5).

The rubrics are labelled as follows: ”Discussions” relates to the discussion forums supporting asynchronous communication, which were also used by some teams to structure materials they used and objectify their working process. The rubric ”Groups” gives an overview of all teams within the Project Room and their respective members and working materials. In the rubric ”Materials” all working materials, relevant literature, links to websites etc. are stored. ”News” and ”Dates” that are interesting and relevant for the Project Room members can be announced in the accordant rubrics. Finally, every member can create a small homepage with personal data and a picture in the rubric ”People”.

All teams used the rubric ”Discussions” the most frequently, showing its central role within the Project Room. Members communicate via the discussion forum but also use it to embed their working materials and other entries within the discussion threads, thus structuring the contents of the Project Room. Furthermore users need to click on every entry in the forum, thus creating a high number of requests.

Figure 5 - Use of rubrics

Differences between teams are mainly in the use of the rubrics ”Discussions” and ”People”. This is concurrent with focus group results showing that members of team No. 3 used the discussion forums to archive the materials they edited together and to document their work. Members of team No. 2, on the other hand, reported that access to personal data like email addresses and telephone numbers to get in contact with each other became very important when nearing the completion of their work. Both usage patterns are reflected in the frequency of use of the accordant areas.

Implications for matching CommSy and didactical context can be drawn as follows: The different functionalities offered by the system are used in manifold ways by the different student teams, so that distinctive usage patterns can be identified depending on the team and work structures. Allowing and reflecting on differences regarding frequency and patterns of use within the course is an important element in training social skills and media competency.

Conclusion

We hypothesized that the successful use of CSCL applications depends on accompanying didactical measures rather than the software itself. We therefore introduced open learning scenarios as didactical concept with CommSy as adequate software support. To match the use of CommSy with its context of use, we presented three areas of activity: shaping the context of use, and supporting initial and continuous use.

Drawing on log file analysis we identified occasions, types, and patterns of use and were able to draw implications for fitting CommSy into didactical contexts that can be generalized to other CSCL applications as well. We summon our findings as follows:

Shaping the context of use, that means installation and configuration of the application and providing initial content to inspire interest in the students, is a prerogative for successful system use. Analyzing the frequency of use shows that the teachers’ high activity at the beginning of the course is matched by intensive student use. At no other time during the course we found hints for a similar importance of teachers’ activity.

Supporting initial use aims at acquainting the students with the system and establishing rules for usage, e.g. using the system to prepare for class or prior to presentations. Our analysis shows that in the case presented the teachers succeeded in doing so. In open learning scenarios we pay special attention to the team building process and the choice of topics to work on. Repositories to store proposals and ideas that the students come up with are helpful in this regard. They can be used both physically – e.g. on flipcharts – and virtually, e.g. in a discussion forum in the project room.

Supporting continuous use confronts teachers with the task of motivating the students to use the software autonomously. Teachers need to be careful not to dominate the system use or to become the sole source of information, but should encourage students to help each other and supply their working materials for their fellow students as well (cf. Friedrich et al. 2000). Our analysis shows that teachers should be present in the system throughout the course and support students when needed but can gradually decrease their activities as students work more and more independently. In doing so, study teams also develop different patterns of usage, which should be acceptable to teachers.

We also elaborated in this paper how log file analysis can be used to evaluate CSCL applications and their fit with the respective use context. We consider log file analysis as a suitable method to collect and interpret objective data of users’ actions on a large scale. Log data serves to analyze frequency, foci, occasions and patterns of use and to compare them related to different user types and groups. Statements and observations from individual users collected elsewhere (e.g. in interviews) can be tested for another group or a larger scale of users.

Besides the number of requests that was utilized in our analysis other kinds of log data, e.g. paths of navigation or length of use, can be analyzed for a closer look at users’ actions. However, in our view, this is significantly more complex and time-consuming compared to recording the number of requests without producing a substantial gain in insights concerning the match between software use and didactical context. For HCI research, more detailed forms of log file analysis are nevertheless highly recommendable.

One generally needs to take into account that log data can be ”distorted” through different usage habits or system (e.g. browser) configurations. For example, it is possible that one user is less purposive when browsing the system, thus producing many clicks and requests, whereas another user reaches the same result in a straightforward and much faster way, thus appearing as the ”less active” regarding the number of requests.

Log file analyses also produce more meaningful results when combined with other kinds of data, e.g. from interviews, observations, or questionnaires. For example, our analysis showed that foci and occasions of use could only be interpreted on the basis of information about how the course was structured and when certain events (e.g. presentations, holidays) took place. We thus recommend a triangulation of research methods to evaluate the use of CSCL applications (cf. Strauss/Pape 2004).

References

Arnold, P.; Kilian, L.; Thillosen, A.: “So lonely!?” – Online-Betreuung als kritischer Erfolgsfaktor beim telematischen Studieren. In: Campus 2002 (eds Bachmann, G.; Haefeli, O.; Kindt M.) Münster u.a.: Waxmann, 2002, 334-344.

Bleek, W.-G.; Wolff, B.; Kielas, W.; Malon, K.; Otto, T.: Vorgehen zur Einführung von Community Systemen in Lerngemeinschaften. In: Virtuelle Organisation und Neue Medien 2000 (eds Engelien, M.; Neumann, D.) Lohmar/Köln: Josef Eul, 2000, 97-113.

Bleek, W-G.; Jackewitz, I.; Pape, B.: Matching Needs - Application Service Providing for Asynchronous Learning Networks. In: Proceedings of the 36th Annual Hawaii International Conference on System Sciences 2002 (ed Sprague, R. H.) Los Alamitos, CA u.a.: IEEE, 2003.

Bortz, J.; Döring, N.: Forschungsmethoden und Evaluation für Human- und Sozialwissenschaftler, 3. edn. Berlin u.a.: Springer, 2002.

Bremer, C.: Virtuelle Konferenzen. In: Virtuelle Organisation und Neue Medien 2002. Workshop GeNeMe2002. Gemeinschaften in Neuen Medien (eds Engelien, M.; Homann, J.) Lohmar/Köln: Josef Eul, 2002, 553-575.

Döring, N.: Sozialpsychologie des Internet. Die Bedeutung des Internet für Kommunikationsprozesse, Identitäten, soziale Beziehungen und Gruppen, 2. edn. Göttingen u.a.: Hogrefe, 2003.

Feeken, H.; Kleinschmidt, A.; Zawaki, O.: Das Konzept der Online-Betreuung im Projekt “Ökonomische Bildung online”. In: Campus 2002 (eds Bachmann, G.; Haefeli, O.; Kindt M.) Münster u.a.: Waxmann, 2002, 345-355.

Friedrich, H.F.; Hesse, F.W.; Ferber, S.; Heins, J.: Evaluation einer Strategie zur Moderation virtueller Seminar. In: Virtueller Campuss ’99 (eds Krahn, H.; Wedekind, J.) Münster u.a.: Waxmann, 2000, 127-137.

Grob, H. L;, Bensberg, F.; Kaderali, F.: Controlling Open Source Intermediaries – a Web Log Mining Approach. In: Proceedings of the 26th Int. Conf. Information Technology Interfaces ITI 2004, Sagreb (Croatia), 2004, 233-242.

Hinze, U.; Blakowski, G.: Anforderungen an die Betreuung im Onlinelernen: Ergebnisse einer qualitativen Inhaltsanalyse im Rahmen der VFH. In: Campus 2002. (eds Bachmann, G.; Haefeli, O.; Kindt M.) Münster u.a.: Waxmann, 2002, 323-333.

Jackewitz, I.; Janneck, M.; Strauss, M.; CommSy – Softwareunterstützung für Wissensprojekte. In: Wissensprojekte – Gemeinschaftliches Lernen aus didaktischer, softwaretechnischer und organisatorischer Sicht (eds Pape, B.; Krause, D.; Oberquelle, H.) Münster u.a.: Waxmann, 2004, 186-202.

Janneck, M.; Strauss, M.: Design-Prozesse für computerunterstütztes kooperatives Lernen. In: Informatik bewegt. Informatik 2002 – 32. Jahrestagung der Gesellschaft für Informatik e.V. (eds Schubert, S.; Reusch, B.; Jesse, N.) Bonn: Köllen, 2002, 237-238.

Janneck, M.; Krause, D.; Pape, B.; Strauss, M.: Medienunterstützung in offenen Seminaren – am Beispiel des CSCL-Systems CommSy. In: DeLFI 2003 – Tagungsband der 1. e-Learning Fachtagung Informatik (eds Bode, A.; Desel, J.; Rathmayer, S.; Wessner M.) Bonn: Gesellschaft für Informatik e.V., 2003, 47‑56.

Janneck, M.; Krause, D.: Einladung zur Nachahmung: Offene Lernveranstaltungen mit Medienunterstützung. In: Wissensprojekte – Gemeinschaftliches Lernen aus didaktischer, softwaretechnischer und organisatorischer Sicht (eds Pape, B.; Krause, D.; Oberquelle, H.) Münster u.a.: Waxmann, 2004, 74-89.

Kerres, M.: Didaktische Konzeption mulitmedialer und telemedialer Lernumgebungen. HMD – Praxis der Wirtschaftsinformatik 205, 1999, 9-21.

Reinmann-Rothmeier, G.: Didaktische Innovation durch Blended Learning. Bern u.a.: Hans Huber, 2003.

Strauss, M.; Pape, B.; Adam, F.; Klein, M.; Reinecke, L.: CommSy-Evaluationsbericht 2003: Softwareunterstützung für selbstständiges und kooperatives Lernen. Universität Hamburg Fachbereich Informatik: Bericht FBI-HH-B-251/03, 2003.

Strauss, M.; Pape, B.: Eine methodische Expedition zur formativen Evaluation kooperativer Lernplattformen. In: Wissensprojekte – Gemeinschaftliches Lernen aus didaktischer, softwaretechnischer und organisatorischer Sicht (eds Pape, B.; Krause, D.; Oberquelle, H.) Münster u.a.: Waxmann, 2004, 373-388.

Wessner, M.; Pfister, H.: Kooperatives Lehren und Lernen. In: CSCW-Kompendium - Lehr- und Handbuch zum computerunterstützten kooperativen Arbeiten (eds Schwabe, G.; Streitz, N.; Unland, R.) Berlin, Heidelberg u.a.: Springer, 2001, 251-263.

Wottawa, H.; Thierau, H.: Lehrbuch Evaluation, 2. edn. Bern u.a.: Huber, 1998.

Matching Software and Context
in Open Learning Scenarios

The Use of Log File Analysis to Evaluate Didactical Fit