Black-Box Testing of Practical Movie Recommendation Systems: a Comparative Study

Namhee Lee1, Jason J. Jung2, 3, Ali Selamat2 and Dosam Hwang3

  1. School of Business Administration
    Sogang University, Seoul, Korea
  2. Software Engineering Research Group (SERG), Knowledge Economy Research Alliance and Faculty of Computing
    Universiti Teknologi Malaysia, 81310 Johor, Malaysia
  3. Department of Computer Engineering
    Yeungnam University, Gyeongsan, Korea


Many practical recommendation systems have been studied, and also the services based on such recommendation systems have been opened in real world. The main research questions of this work are i) how these recommendation services provide users with useful information, and ii) how different the results from the systems are from each other. In this paper, we propose a black-box evaluation framework of the practical recommendation services. Thus, we have designed user modeling process for generating synthesized user models as the inputs for the recommendation services. User models (i.e., a set of user ratings) have been synthesized to discriminate the recommendation results. Given a set of practical recommendation systems, the proposed black-box testing scheme has been applied by comparing recommendation results. Particularly, we focus on investigating whether the services consider attribute selection.

Key words

Social networks; Recommendation systems; Black-box testing; Comparative study

Digital Object Identifier (DOI)

Publication information

Volume 11, Issue 1 (January 2014)
Year of Publication: 2014
ISSN: 1820-0214 (Print) 2406-1018 (Online)
Publisher: ComSIS Consortium

Full text

DownloadAvailable in PDF
Portable Document Format

How to cite

Lee, N., Jung, J. J., Selamat, A., Hwang, D.: Black-Box Testing of Practical Movie Recommendation Systems: a Comparative Study. Computer Science and Information Systems, Vol. 11, No. 1, 241–249. (2014),