Towards Privacy-Preserving Evaluation for Information Retrieval Models over Industry Data Sets

Peilin Yang and Hui Fang. Towards Privacy-Preserving Evaluation for Information Retrieval Models Over Industry Data Sets. In Proceedings of the 13th Asia Information Retrieval Societies Conference (AIRS'2017). Springer International Publishing, Jeju Island, South Korea, 210-221.

@InProceedings{10.1007/978-3-319-70145-5_16,
 author={Yang, Peilin
 and Zhou, Mianwei
 and Chang, Yi
 and Zhai, Chengxiang
 and Fang, Hui},
 editor={Sung, Won-Kyung
 and Jung, Hanmin
 and Xu, Shuo
 and Chinnasarn, Krisana
 and Sumiya, Kazutoshi
 and Lee, Jeonghoon
 and Dou, Zhicheng
 and Yang, Grace Hui
 and Ha, Young-Guk
 and Lee, Seungbock},
 title={Towards Privacy-Preserving Evaluation for Information Retrieval Models Over Industry Data Sets},
 booktitle={Information Retrieval Technology},
 year={2017},
 publisher={Springer International Publishing},
 address={Cham},
 pages={210--221},
 abstract={The development of Information Retrieval (IR) techniques heavily depends on empirical studies over real world data collections. Unfortunately, those real world data sets are often unavailable to researchers due to privacy concerns. In fact, the lack of publicly available industry data sets has become a serious bottleneck hindering IR research. To address this problem, we propose to bridge the gap between academic research and industry data sets through a privacy-preserving evaluation platform. The novelty of the platform lies in its ``data-centric'' mechanism, where the data sit on a secure server and IR algorithms to be evaluated would be uploaded to the server. The platform will run the codes of the algorithms and return the evaluation results. Preliminary experiments with retrieval models reveal interesting new observations and insights about state of the art retrieval models, demonstrating the value of an industry data set.}, isbn={978-3-319-70145-5}
 }