CAMACEL - Collaborative and Automated Multimedia Annotation for Customized E-Learning

A growing number of audiovisual recordings, often enhanced by presentation-based content, transforms lectures, seminars, and meetings into potentially ubiquitous learning-objects. However, their value remains confined to a very limited set of applications as long as these objects are not searchable. As manual metadata allocation is not scalable in regard to the rapidly growing number of recordings, automated processes and collaborative scenarios are needed. Here, we propose automated content-search, automated speech recognition, and collaborative tagging for enriching recordings towards a basis for customized learning.

Assets of audiovisual and content-endorsed lectures are growing in size as content is produced on a more and more regular basis due to the increasing availability of a digital production and distribution infrastructure.

Designated advantages are limited, however, in some respect: Access towards data is systematic and orderly only, using descriptive keywords. Manual metadata allocation, being labour intensive as it is, is limited to a straightforward set of information, therefore content-based approaches towards the data are being restricted to a limited set of keywords. Moreover, metadata is static and describing the object as a whole, so users can only search the data for keywords assigned to the multimedia content as an entity. The content, on the contrary, is isochronal, i.e. evolving over time. Therefore, a keyword describing the final chapter of a presentation will result in the user having to watch (or scroll through) a vast amount of content not relevant to him/her. Finally, multimedia objects tend to foster a passive approach to learning, i.e. the consumption of recorded lectures. There are very limited ways of actively adopting learning objects or stimulating students to work with the recordings.

Automated recording systems today are able to capture content from the presenter's computer and synchronize it with audio and video recorded. In order to provide more intelligent approaches to these objects, metadata has to become isochronic as well. Any development towards more manageable and valuable data repositories, especially from the students' perspective, therefore would have to consider the following aspects:

• automated annotation based on content (i.e. presentation), OCR
• automated speech recognition ASR
• manual annotation by authors/experts with special authoring tools
• collaborative annotation by users/students (social tagging)

By making use of the logical and conceptual dependency structures of the audiovisual documents, the provided search results will extend the single list of sequentially ordered documents - ordered by some measure of relevance - provided by today's search engines. With the dependency structures made explicit a well suited visualization of the entire knowledge provided by the annotated audiovisual data will enable efficient domain navigation. According to the personal preferences of the user a content-based semantic search engine will present an overview of interdependent audiovisual documents related to the requested topic. Thus, enabling the user to arrange a customized lecture that consists of single interrelated audiovisual segments according to his personal information needs.


Scenarios
Customized Learning Scenarios will combine extracts from different recordings depending on the user's query, experience, and rights: Instead of consuming lectures as an entity, individualized and problem-based approaches towards a subject-matter become feasible by assembling extracts from different recordings.

Medical Recordings are produced at extraordinary expenses without adequate opportunities of usage. By providing tools for annotation, these recordings will become essential part of patient documentations.

Knowledge Pool: With companies being eager for collecting knowledge produced in their own ranks, recording presentations will be the first step to open another source for knowledge pools. Automatically metadating and manually annotating them, will be second, more important one to make this source a more profitable one.

MPEG7
automated metadata annotation
search engine technology
semantic search
multimedia information retrieval
intelligent multimedia repositories
e-learning
customized learning
multimedia enhanced learning
enhanced lectures
social tagging
collaborative annotation
social networking
web 2.0
multimedia annotation (MPEG7)

• multimedia information retrieval (feature extraction, similarity search)
• search engine technology (focused crawling, indexing, ranking)
• speaker independent speech recognition (term spotting)
• speaker independent speech recognition (German, English, French)
• semantic search (ontologies, ontology learning, ontology mapping, ontology alignment)
• e-learning, distance learning, customized learning, tele lecturing

• develop a similarity based search for video (multimedia) data
• develop a sufficient speaker independent speech recognition for multimedia indexing
• develop an MPEG 4 scene editor / encoder for multimedia lecture recordings
• provide/create content for educational multimedia data and medical multimedia documentation
• design interface to access isochronal data
• develop e-learning scenarios for social tagging of multimedia objects
• libraries with interest in handling of multimedia objects (MPEG-7, Dublin Core, DOI)

• libraries with digital multimedia archives and interest in developments towards MPEG-7, DOI etc.
• hospitals that provide medical multimedia content
• universities and other educational institutions that provide educational multimedia content
• institutions with experience in the work sought by partners (research)
• SME with interest in development of multimedia knowledge pools according to work programme

Yes