An automatic multimedia content summarization system for video recommendation

Multimedia-based learning has recently become a promising instructional resource. According to dual-coding theory, the human brain deals with imagery representation better than verbal representation (Clark & Paivio, 1991; Paivio, 1971). Many studies report that multimedia content is often more useful for learning and teaching than traditional text-based learning (Mackey & Ho, 2008; Mayer & Moreno, 2002; Rose, 2003). There are many types of multimedia learning materials, but video is the most representative and popular one. This is because video integrates many multimedia resources, such as text, images, sound, and speech. Based on the theory of constructivism, video provides a context wherein learners can construct their own knowledge (Brown, Collins, & Duguid, 1989). Several studies demonstrate that video is a suitable material for context-based learning (Choi & Johnson, 2005; Choi & Johnson, 2007). In addition, video learning is an effective way of providing motivation, keeping attention, and giving satisfaction to the learner (Choi & Johnson, 2005; Choi & Johnson, 2007; Mackey & Ho, 2008). However, there are some limitations that exist in video learning. For example, choosing a suitable video for instructors and learners from the rapidly growing number of videos can be a problem. A similar problem arises when recommending videos to learners and instructors. Moreover, when learners get a great deal of information in a short period of time, it causes cognitive overload (Pass & van Merrienboer, 1994; Sweller & Chandler, 1994; Sweller, van Merrienboer & Pass, 1998). The usefulness of video learning is obvious, but it is necessary to enhance the recommendation mechanism for learners in order to facilitate multimedia learning.

Unlike texts, watching videos requires much more time since video content is usually displayed linearly. Summarization is used to preserve the most informative parts of the source content. Therefore, video summarization is essential for enabling the learner to skim through video content. With the rapid growth of the video industry, acquiring the appropriate video from a huge database is a difficult task. Adaptive video recommendation could be a way to deal with this situation, as this system is specifically designed to help learners filter information. In essence, the combination of summarization and recommendation is helpful in reducing cognitive overload with regard to video learning. Traditionally, these tasks (i.e., content summarization and recommendation in e-learning) are done manually, which is a very demanding and time-consuming process. Consequently, there is a strong demand for an automatic video summarization and recommendation system.

Recently, recommendation systems have been applied to some products and information databases by making adaptive suggestions based on previous examples of a user's preference (Melville, Mooney & Nagarajan, 2002; Mooney & Roy, 2000; Wang, Tsai, Lee & Chiu, 2007). A recommendation system for learning material can provide objects easily and efficiently thereby enhancing learning activities. Without recommendation mechanisms, learners would spend more time selecting suitable learning objects and less time involved in the actual activity of learning. Several studies have shown (Tsai, Chiu, Lee & Wang, 2006; Wang et al., 2007) that automatic recommendation mechanisms that refer to learner profiles can promote the accuracy of learning object recommendation. Nevertheless, these recommendation mechanisms are only suitable for structured or semi-structured data (Popescul, Ungar, Pennock & Lawrence, 2001; Tsai et al., 2006; Wang et al., 2007). In other words, these systems may not work well with raw videos and raw texts. On the other hand, recommendation systems are also applied to movie or TV recommendations (Alspector, Kolcz & Karunaithi, 1998; Basu, Hirsh & Cohen, 1998; Cotter & Smith, 2000; Melville et al., 2002). However, these studies tend to apply very limited sets of features such as the movie title, the director, keywords, and actors, as well as like-minded user ratings. The feature that is lacking in all of these studies is the inclusion of speech content, which contains a substantial amount of information relating to the video itself. In other words, the studies mentioned above ignore important content within learning materials.

Automatic summarization is an important research topic, especially in relation to automatic text-based and video-based summarization. Text-based summarization research, such as the Document Understanding Conference (DUC) (http://duc.nist.gov/), aims at extracting important sentences from source documents. These techniques focus on generating summaries from news-like articles (i.e., newspaper and newswire data) (Dang, 2006; Dang, 2007) which are usually shorter and more coherent than video stories. Moreover, a video story usually contains multiple subtopics. On the contrary, video-based summarization research, such as the TRECVid workshop (http://wwwnlpir.nist.gov/projects/t01v/), aims at extracting key-frames and shots from source videos (Over, Ianeva, Kraaij & Smeaton, 2005; Over, Ianeva, Kraaij & Smeaton, 2006), offering a sketch that contains a description of an object (such as color, shape, or motion) (Liu & Li, 2002; Milrad, Rossmanith & Scholz, 2005; Over et al., 2005; Over et al., 2006). This technique is often used in surveillance systems (Osadchy & Keren, 2004; Piriou, Bouthemy & Yao, 2006) and medical videos (Fasquel, Agnus, Moreau, Soler & Marescaux, 2006). Nevertheless, these types of summarization may be not useful for learners due to the neglect of video content. Furthermore, the traditional video-based summarization is not generally used for educational purposes.