Digital Library technology offers many effective ways to handle document content. Access and delivery of documents becomes more and more digital and decentralized, and new user groups can benefit from library services. This is true for textual documents. However, technological and scientific progress contribute to increasing availability of non-textual documents, which are worthy of library-oriented treatment. Examples include digitization efforts in Cultural Heritage, production of scientific film, recording of orchestral performances, as well as masses of primary research data produced in the natural sciences. All of these non-textual documents, while being potentially relevant for library-oriented service, are more difficult to accommodate in a Digital Library system than their textual counterparts. Main challenges in supporting non-textual documents include questions of document representation, indexing and content-based accessing, and document presentation. Specifically, content-based access in non-textual documents is a difficult problem as appropriate methods usually are application dependent and nontrivial to implement.

From the field of multimedia databases and multimedia visualization, many promising approaches have been proposed. But even if relevant document domains, use cases, and accommodation strategies have been identified, the problem of deploying such approaches within the operational context of a library operator needs to be solved. Probado aims at designing, developing and deploying Digital Library functionality for non-textual documents for a selection of use cases. At the same time, the project aims to propose a general reference architecture and protocol for consolidation of distributed non-textual document repositories of heterogeneous document types.

In the scope of the Probado Leistungszentrum, funded by the German Research Foundation (DFG) over five years from 2006 to 2011, we developed a Digital Library system to deal with these challenges. In particular we focused on two important domains – 3D architectural models and classical music – and put the system into production use in collaboration with the respective integration partners Technische Informationsbibliothek Hannover (TIB) and Bayerische Staatsbibliothek (BSB).

Existing Digital Library systems include Fedora [10], Greenstone [15], DLib [4] and Variations [7]. Fedora, Greenstone, and DLib support building Digital Libraries for textual documents; support for multimedia documents relies on metadata annotations according to specific standards such as MPEG-7. In Probado, the goal is to index and access non-textual documents specifically by content-based approaches. Therefore, the aforementioned systems are not directly applicable to our approach.

Fig. 1. The Probado 3D Webinterface for query formulation and result presentation [1]: Faceted search (left); Query with spatial configuration of rooms (middle); Building with Room Connectivity Graph (right).

Probado 3D is a part of the Probado framework [3] designed to support 3D documents, with the focus on the architectural domain. Momentarily, the index of the Probado 3D repository set up at TIB contains about 8000 building, construction unit and object models. Contributors are either students of architecture and component manufacturers or the models are part of architectural CAD application libraries or public databases for architectural CAD models. After uploading models and a minimum set of metadata, the files are processed like depicted in [2]. Indexing of architectural 3D models is a prerequisite for content-based query by example and document browsing. By creating a concise object description, the similarity between two 3D models can be computed. In a query-by-example scenario where the user either uploads a 3D component model or uses a sketch-based Probado 3D interface to generate one, the search for similar objects in the database is conducted using global shape features which are easy to compute and guarantee fast response times of the query engine. For browsing the Probado 3D repository based on shape similarity, we comprehensively characterize the components using high quality local shape descriptors and special distance measures tailored to the requirements of architectural 3D models [13]. While low-level semantic features are an effective means to characterize the geometric shape of an object, they are not well-suited to describe the structure of building models which is mainly defined by the topology of rooms and floors. To overcome this drawback, we introduced the concept of Room Connectivity Graphs [14] [12]. These graphs are especially designed to capture the topology of buildings. By that, users can search for building models that contain a certain spatial configuration of rooms. For the definition of such configurations we provide a graphical user interface. Additionally, the search can be further constrained regarding e.g. the area of certain rooms.

Probado Music is a multimodal digital library system that allows worldwide access to various music material [5]. Besides streaming and presenting digital music documents (in particular scanned sheet music, audio recordings, and lyrics), Probado Music employs current techniques from the field of music information retrieval to offer enhanced browsing, navigation, and search functionalities [6, 8, 9, 11]. The frontend for document retrieval and access is depicted in Figure 2. Besides classic metadata search, content-based search techniques are incorporated as well. Thus, score-, audio-, or lyrics-extracts can be used as queries. After selecting an entry from the hit list, all documents containing the according piece of music are prepared for presentation. To this end, the Probado Music system offers three distinct visualizations (score, audio, and lyrics) between which the user can freely navigate. As further feature, the current playback position in the selected audio recordings is highlighted in all visualizations (i.e., the current measure in the score and the currently audible word) through the application of music synchronization techniques. As further result of this synchronization, the score or the lyrics can be used for navigation in an audio recording. Additionally, if several audio interpretation are available, the user can switch between them while maintaining the musical position.

Fig. 2. The Probado Music webinterface for query formulation, result presentation, and document access.

In the context of the project Probado, the Bayerische Staatsbibliothek digitized a corpus of approx. 72; 000 pages of sheet music. In addition, 17 hours of public domain audio material were collected. Given such a large document collection, its proper organization and management are of utmost importance to allow for searching and accessing documents conveniently. In Probado Music, a full-fledged preprocessing workflow was defined and implemented. The developed management system offers convenient input masks for metadata annotation and includes several semi-automatic methods for content-based indexing, mapping, and linking of the music documents.

The goal for Probado was to develop and implement a service for providing contentbases access to non-textual documents, and put this service into production-use in scientific libraries. In particular, a semi-automatic indexing process was focused on, to deal with the ever-increasing amount of available documents.

Within the scope of Probado we reached these goals. Particularly we deployed such service-capabilities with our collaboration partner Technische Informationsbibliothek, Hannover for digital, 3D architectural models. For digitized, classical music, content-based query-services were developed with the Bayerische Staatsbibliothek in Munich.

Furthermore, to spread the use of Probado, a user-workshop for other interested library partners was held at the TIB Hannover. Though Probado was implemented with architectural models and classical music in mind, the system and the technology are extensible to other domains, such as image, video or primary research-data repositories.

1. Berndt, R., Blümel, I., Krottmaier, H., Wessel, R., Schreck, T.: Demonstration of user interfaces for querying in 3d architectural content in PROBADO3D. In: 13th European Conference on Digital Libraries (2009), demonstration Paper

2. Berndt, R., Blümel, I., Wessel, R.: Probado3d - towards an automatic multimedia indexing workflow for architectural 3d models. In: ELPUB 2010 - Publishing in the networked world: transforming the nature of communication. pp. 79–88 (2010)

3. Berndt, R., Krottmaier, H., Havemann, S., Schreck, T.: The PROBADO-framework: Content-based queries for non-textual documents. In: ELPUB2009 Conference on Electronic Publishing (2009)

4. Castelli, D., Pagano, P.: Opendlib: A dl service system. In: ECDL (2002)

5. Damm, D., Fremerey, C., Thomas, V., Clausen, M.: A Digital Library Framework for Heterogeneous Music Collections - from Document Acquisition to Cross-Modal Interaction. International Journal on Digital Libraries: Special Issue on Music Digital Libraries (to appear) (2012)

6. Damm, D., Kurth, F., Fremerey, C., Clausen, M.: A Concept for Using Combined Multimodal Queries in Digital Music Libraries. In: 13th European Conference on Digital Libraries (2009)

7. Dunn, J.W., Byrd, D., Notess, M., Scherle, R.: Variations2: Retrieving and using music in an academic setting. Communications of the ACM 49 (2006)

8. Fremerey, C., Clausen, M., Ewert, S., Müller, M.: Sheet Music-Audio Identification. In: Proceedings of the 10th International Conference on Music Information Retrieval (ISMIR 2009) (October 2009)

9. Fremerey, C., Müller, M., Clausen, M.: Handling Repeats and Jumps in Score- Performance Synchronization. In: Proceedings of the International Conference on Music Information Retrieval (ISMIR). pp. 243–248. Utrecht, Netherlands (2010)

10. Lagoze, C., Payette, S., Shin, E., Wilper, C.: Fedora: an architecture for complex objects and their relationships. Int. J. Digit. Libr. 6, 124–138 (2006)

11. Thomas, V., Wagner, C., Clausen, M.: OCR based post processing of OMR for the recovery of transposing instruments in complex orchestral scores. In: Proceedings of the 12th International Conference on Music Information Retrieval (ISMIR 2011) (to appear) (2011)

12. Wessel, R., Blümel, I., Klein, R.: The room connectivity graph: Shape retrieval in the architectural domain. In: Skala, V. (ed.) The 16-th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision’2008. UNION Agency-Science Press (Feb 2008)

13. Wessel, R., Klein, R.: Learning the compositional structure of man-made objects for 3d shape retrieval. In: EUROGRAPHICS 2010 Workshop on 3D Object Retrieval. pp. 39–46 (May 2010)

14. Wessel, R., Ochmann, S., Vock, R., Blümel, I., Klein, R.: Efficient retrieval of 3d building models using embeddings of attributed subgraphs. In: Poster Proceedings of ACM Conference on Information and Knowledge Management (CIKM 2011) (Oct 2011)

15. Witten, I.H., Mcnab, R.J., Boddie, S.J., Bainbridge, D.: Greenstone: A comprehensive open-source digital library software system. In: Proceedings of the Fifth ACM International Conference on Digital Libraries (2000)