Skip Abstract Section
Abstract
The paper reviews methods used in the fashion compatibility recommendation domain. We select methods based on reproducibility, explainability, and novelty aspects and then organize them chronologically and thematically. We presented general characteristics of publicly available datasets that are related to the fashion compatibility recommendation task. Finally, we analyzed the representation bias of datasets, fashion-based algorithms’ sustainability, and explainable model assessment. The paper describes practical problem explanations, methodologies, and published datasets that may serve as an inspiration for further research. The proposed structure of the survey organizes knowledge in the fashion recommendation domain and will be beneficial for those who want to learn the topic from scratch, expand their knowledge, or find a new field for research. Furthermore, the information included in this paper could contribute to developing an effective and ethical fashion-based recommendation system.
- Kenan E Ak, Joo Hwee Lim, Jo Yew Tham, and Ashraf A Kassim. 2018. Efficient multi-attribute similarity learning towards attribute-based fashion search. In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, Lake Tahoe, NV, USA, 1671–1679.Google Scholar
Cross Ref
- Debopriyo Banerjee, Lucky Dhakad, Harsh Maheshwari, Muthusamy Chelliah, Niloy Ganguly, and Arnab Bhattacharya. 2022. Recommendation of Compatible Outfits Conditioned on Style. In European Conference on Information Retrieval. Springer, Springer, Stavanger, Norway, 35–50.Google Scholar
- Aurore Bardey, Madison Booth, Giuliana Heger, and Jonas Larsson. 2022. Finding yourself in your wardrobe: An exploratory study of lived experiences with a capsule wardrobe. International Journal of Market Research 64, 1 (2022), 113–131.Google ScholarCross Ref
- Sean Bell and Kavita Bala. 2015. Learning visual similarity for product design with convolutional neural networks. ACM transactions on graphics (TOG) 34, 4 (2015), 1–10.Google Scholar
- Roberto Cahuantzi, Xinye Chen, and Stefan Güttel. 2021. A comparison of LSTM and GRU networks for learning symbolic sequences. arXiv preprint arXiv:2107.02248 1, 02248 (2021).Google Scholar
- Suthee Chaidaroon, Yi Fang, Min Xie, and Alessandro Magnani. 2019. Neural compatibility ranking for text-based fashion matching. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, New York, NY, United States, 1229–1232.Google ScholarDigital Library
- Huiyuan Chen, Yusan Lin, Fei Wang, and Hao Yang. 2021. Tops, bottoms, and shoes: building capsule wardrobes via cross-attention tensor network. In Fifteenth ACM Conference on Recommender Systems. Association for Computing Machinery, New York, NY, United States, 453–462.Google ScholarDigital Library
- Wen Chen, Pipei Huang, Jiaming Xu, Xin Guo, Cheng Guo, Fei Sun, Chao Li, Andreas Pfadler, Huan Zhao, and Binqiang Zhao. 2019. POG: personalized outfit generation for fashion recommendation at Alibaba iFashion. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. Association for Computing Machinery, New York, NY, United States, 2662–2670.Google ScholarDigital Library
- Ching-I Cheng and Damon Shing-Min Liu. 2007. A decision making framework for dressing consultant. In 2007 IEEE Symposium on Computational Intelligence in Multi-Criteria Decision-Making. IEEE, IEEE, Honolulu, HI, USA, 267–271.Google ScholarCross Ref
- Davide Chicco. 2021. Siamese neural networks: An overview. Artificial Neural Networks 1 (2021), 73–94.Google ScholarCross Ref
- Antonia Creswell, Tom White, Vincent Dumoulin, Kai Arulkumaran, Biswa Sengupta, and Anil A Bharath. 2018. Generative adversarial networks: An overview. IEEE signal processing magazine 35, 1 (2018), 53–65.Google Scholar
- Guillem Cucurull, Perouz Taslakian, and David Vazquez. 2019. Context-aware visual compatibility prediction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. IEEE, Long Beach, CA, USA, 12617–12626.Google ScholarCross Ref
- Zeyu Cui, Zekun Li, Shu Wu, Xiao-Yu Zhang, and Liang Wang. 2019. Dressing as a whole: Outfit compatibility learning based on node-wise graph neural networks. In The World Wide Web Conference. Association for Computing Machinery, New York, NY, United States, 307–317.Google ScholarDigital Library
- Yashar Deldjoo, Fatemeh Nazary, Arnau Ramisa, Julian Mcauley, Giovanni Pellegrini, Alejandro Bellogin, and Tommaso Di Noia. 2022. A review of modern fashion recommender systems. arXiv preprint arXiv:2202.02757 1, 02757 (2022).Google Scholar
- Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition. IEEE, IEEE, Miami, FL, USA, 248–255.Google ScholarCross Ref
- Xue Dong, Xuemeng Song, Fuli Feng, Peiguang Jing, Xin-Shun Xu, and Liqiang Nie. 2019. Personalized capsule wardrobe creation with garment and user modeling. In Proceedings of the 27th ACM International Conference on Multimedia. Association for Computing Machinery, New York, NY, United States, 302–310.Google ScholarDigital Library
- Xue Dong, Xuemeng Song, Na Zheng, Jianlong Wu, Hongjun Dai, and Liqiang Nie. 2022. TryonCM2: Try-on-Enhanced Fashion Compatibility Modeling Framework. IEEE Transactions on Neural Networks and Learning Systems 0, 0(2022), 1–12.Google Scholar
- Guangyu Gao, Liling Liu, Li Wang, and Yihang Zhang. 2019. Fashion clothes matching scheme based on Siamese Network and AutoEncoder. Multimedia Systems 25, 6 (2019), 593–602.Google ScholarCross Ref
- Wei Gong and Laila Khalid. 2021. Aesthetics, Personalization and Recommendation: A survey on Deep Learning in Fashion. arXiv preprint arXiv:2101.08301 1, 08301 (2021).Google Scholar
- Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao. 2021. Knowledge distillation: A survey. International Journal of Computer Vision 129, 6 (2021), 1789–1819.Google ScholarDigital Library
- Xiaoling Gu, Fei Gao, Min Tan, and Pai Peng. 2020. Fashion analysis and understanding with artificial intelligence. Information Processing & Management 57, 5 (2020), 102276.Google ScholarCross Ref
- Weili Guan, Haokun Wen, Xuemeng Song, Chung-Hsing Yeh, Xiaojun Chang, and Liqiang Nie. 2021. Multimodal compatibility modeling via exploring the consistent and complementary correlations. In Proceedings of the 29th ACM International Conference on Multimedia. Association for Computing Machinery, New YorkNYUnited States, 2299–2307.Google ScholarDigital Library
- Xianjing Han, Xuemeng Song, Yiyang Yao, Xin-Shun Xu, and Liqiang Nie. 2019. Neural compatibility modeling with probabilistic knowledge distillation. IEEE Transactions on Image Processing 29 (2019), 871–882.Google ScholarDigital Library
- Xintong Han, Zuxuan Wu, Yu-Gang Jiang, and Larry S Davis. 2017. Learning fashion compatibility with bidirectional lstms. In Proceedings of the 25th ACM international conference on Multimedia. Association for Computing Machinery, New York, NY, United States, 1078–1086.Google ScholarDigital Library
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, USA, 770–778.Google ScholarCross Ref
- Shintami Chusnul Hidayati, Cheng-Chun Hsu, Yu-Ting Chang, Kai-Lung Hua, Jianlong Fu, and Wen-Huang Cheng. 2018. What dress fits me best? Fashion recommendation on the clothing style for personal body shape. In Proceedings of the 26th ACM international conference on Multimedia. Association for Computing Machinery, New York, NY, United States, 438–446.Google Scholar
- Yuxin Hou, Eleonora Vig, Michael Donoser, and Loris Bazzani. 2021. Learning attribute-driven disentangled representations for interactive fashion retrieval. In Proceedings of the IEEE/CVF International Conference on Computer Vision. IEEE, Montreal, Canada, 12147–12157.Google ScholarCross Ref
- Yang Hu, Xi Yi, and Larry S Davis. 2015. Collaborative fashion recommendation: A functional tensor factorization approach. In Proceedings of the 23rd ACM international conference on Multimedia. IEEE, Montreal, Canada, 129–138.Google ScholarDigital Library
- Kazuya Kawakami. 2008. Supervised sequence labelling with recurrent neural networks. Ph. D. Dissertation. Technical University of Munich.Google Scholar
- Donghyun Kim, Kuniaki Saito, Samarth Mishra, Stan Sclaroff, Kate Saenko, and Bryan A Plummer. 2021. Self-supervised visual attribute learning for fashion compatibility. In Proceedings of the IEEE/CVF International Conference on Computer Vision. IEEE, Montreal, Canada, 1057–1066.Google ScholarCross Ref
- Seongjae Kim, Jinseok Seol, Holim Lim, and Sang-goo Lee. 2021. False Negative Distillation and Contrastive Learning for Personalized Outfit Recommendation. arXiv preprint arXiv:2110.06483 1, 06483 (2021).Google Scholar
- Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25 (2012).Google Scholar
- Zhanghui Kuang, Yiming Gao, Guanbin Li, Ping Luo, Yimin Chen, Liang Lin, and Wayne Zhang. 2019. Fashion retrieval via graph reasoning networks on a similarity pyramid. In Proceedings of the IEEE/CVF International Conference on Computer Vision. IEEE, Seoul, Korea (South), 3066–3075.Google ScholarCross Ref
- Katrien Laenen and Marie-Francine Moens. 2020. A comparative study of outfit recommendation methods with a focus on attention-based fusion. Information Processing & Management 57, 6 (2020), 102316.Google ScholarCross Ref
- Jui-Hsin Lai, Bo Wu, Xin Wang, Dan Zeng, Tao Mei, and Jingen Liu. 2019. Theme-matters: fashion compatibility learning via theme attention. arXiv preprint arXiv:1912.06227 1, 06227 (2019).Google Scholar
- Xingchen Li, Xiang Wang, Xiangnan He, Long Chen, Jun Xiao, and Tat-Seng Chua. 2020. Hierarchical fashion graph network for personalized outfit recommendation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, New York, NY, United States, 159–168.Google ScholarDigital Library
- Yuncheng Li, Liangliang Cao, Jiang Zhu, and Jiebo Luo. 2017. Mining fashion outfit composition using an end-to-end deep learning approach on set data. IEEE Transactions on Multimedia 19, 8 (2017), 1946–1955.Google ScholarDigital Library
- Zekun Li, Zeyu Cui, Shu Wu, Xiaoyu Zhang, and Liang Wang. 2019. Semi-supervised compatibility learning across categories for clothing matching. In 2019 IEEE International Conference on Multimedia and Expo (ICME). IEEE, Association for Computing Machinery, New York, NY, United States, 484–489.Google ScholarCross Ref
- Min Lin, Qiang Chen, and Shuicheng Yan. 2013. Network in network. arXiv preprint arXiv:1312.4400 1, 1 (2013).Google Scholar
- Yusan Lin, Maryam Moosaei, and Hao Yang. 2020. OutfitNet: Fashion outfit recommendation with attention-based multiple instance learning. In Proceedings of The Web Conference 2020. Association for Computing Machinery, New York, NY, United States, 77–87.Google ScholarDigital Library
- Yujie Lin, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Jun Ma, and Maarten De Rijke. 2019. Explainable outfit recommendation with joint outfit matching and comment generation. IEEE Transactions on Knowledge and Data Engineering 32, 8(2019), 1502–1516.Google ScholarCross Ref
- Yen-Liang Lin, Son Tran, and Larry S Davis. 2020. Fashion outfit complementary item retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Seattle, WA, USA, 3311–3319.Google ScholarCross Ref
- Jinhuan Liu, Xuemeng Song, Zhumin Chen, and Jun Ma. 2019. Neural fashion experts: I know how to make the complementary clothing matching. Neurocomputing 359(2019), 249–263.Google ScholarDigital Library
- Linlin Liu, Haijun Zhang, Yuzhu Ji, and QM Jonathan Wu. 2019. Toward AI fashion design: An Attribute-GAN model for clothing match. Neurocomputing 341(2019), 156–167.Google ScholarDigital Library
- Linlin Liu, Haijun Zhang, and Dongliang Zhou. 2021. Clothing generation by multi-modal embedding: A compatibility matrix-regularized GAN model. Image and Vision Computing 107 (2021), 104097.Google ScholarCross Ref
- Zhi Lu, Yang Hu, Yan Chen, and Bing Zeng. 2021. Personalized outfit recommendation with learnable anchors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Nashville, TN, USA, 12722–12731.Google ScholarCross Ref
- Zhi Lu, Yang Hu, Yunchao Jiang, Yan Chen, and Bing Zeng. 2019. Learning binary code for personalized fashion recommendation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Seattle, WA, USA, 10562–10570.Google ScholarCross Ref
- Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. 2015. Image-based recommendations on styles and substitutes. In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval. Association for Computing Machinery, New York, NY, United States, 43–52.Google ScholarDigital Library
- Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 1, 1 (2013).Google Scholar
- Seyed Omid Mohammadi and Ahmad Kalhor. 2021. Smart fashion: a review of AI applications in the Fashion & Apparel Industry. arXiv preprint arXiv:2111.00905 1, 00905 (2021).Google Scholar
- Maryam Moosaei, Yusan Lin, Ablaikhan Akhazhanov, Huiyuan Chen, Fei Wang, and Hao Yang. 2022. OutfitGAN: Learning Compatible Items for Generative Fashion Outfits. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, New Orleans, United States, 2273–2277.Google ScholarCross Ref
- Takuma Nakamura and Ryosuke Goto. 2018. Outfit generation and style extraction via bidirectional lstm and autoencoder. arXiv preprint arXiv:1807.03133 1, 03133 (2018).Google Scholar
- Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). EMNLP, Doha, Qatar, 1532–1543.Google ScholarCross Ref
- Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2012. BPR: Bayesian personalized ranking from implicit feedback. arXiv preprint arXiv:1205.2618 1 (2012), 1–14.Google Scholar
- Ludger Rüschendorf. 1985. The Wasserstein distance and approximation theorems. Probability Theory and Related Fields 70, 1 (1985), 117–129.Google ScholarCross Ref
- Dikshant Sagar, Jatin Garg, Prarthana Kansal, Sejal Bhalla, Rajiv Ratn Shah, and Yi Yu. 2020. Pai-bpr: Personalized outfit recommendation scheme with attribute-wise interpretability. In 2020 IEEE Sixth International Conference on Multimedia Big Data (BigMM). IEEE, IEEE, New Delhi, India, 221–230.Google ScholarCross Ref
- Hiroyuki Shinnou, Masayuki Asahara, Kanako Komiya, and Minoru Sasaki. 2017. nwjc2vec: Word embedding data constructed from ninjal web japanese corpus. Journal of Natural Language Processing 24, 5 (2017), 705–720.Google ScholarCross Ref
- Xuemeng Song, Fuli Feng, Xianjing Han, Xin Yang, Wei Liu, and Liqiang Nie. 2018. Neural compatibility modeling with attentive knowledge distillation. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. Association for Computing Machinery, New York, NY, United States, 5–14.Google ScholarDigital Library
- Xuemeng Song, Fuli Feng, Jinhuan Liu, Zekun Li, Liqiang Nie, and Jun Ma. 2017. Neurostylist: Neural compatibility modeling for clothing matching. In Proceedings of the 25th ACM international conference on Multimedia. Association for Computing Machinery, New York, NY, United States, 753–761.Google ScholarDigital Library
- Xuemeng Song, Xianjing Han, Yunkai Li, Jingyuan Chen, Xin-Shun Xu, and Liqiang Nie. 2019. GP-BPR: Personalized compatibility modeling for clothing matching. In Proceedings of the 27th ACM International Conference on Multimedia. Association for Computing Machinery, New York, NY, United States, 320–328.Google ScholarDigital Library
- Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander A Alemi. 2017. Inception-v4, inception-resnet and the impact of residual connections on learning. In Thirty-first AAAI conference on artificial intelligence. AAAI, San Francisco, California, USA, 4278–4284.Google ScholarCross Ref
- Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, Boston, MA, USA, 1–9.Google ScholarCross Ref
- Tan, Reuben, and Vasileva, Mariya I and Saenko, Kate and Plummer, Bryan A. 2019. Learning similarity conditions without explicit supervision. In Proceedings of the IEEE/CVF International Conference on Computer Vision. IEEE, Seoul, Korea, 10373–10382.Google Scholar
- Mariya I Vasileva, Bryan A Plummer, Krishna Dusad, Shreya Rajpal, Ranjitha Kumar, and David Forsyth. 2018. Learning type-aware embeddings for fashion compatibility. In Proceedings of the European conference on computer vision (ECCV). Springer, Munich, Germany, 390–405.Google ScholarDigital Library
- Andreas Veit, Serge Belongie, and Theofanis Karaletsos. 2017. Conditional similarity networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, Honolulu, HI, USA, 830–838.Google ScholarCross Ref
- Andreas Veit, Balazs Kovacs, Sean Bell, Julian McAuley, Kavita Bala, and Serge Belongie. 2015. Learning visual clothing style with heterogeneous dyadic co-occurrences. In Proceedings of the IEEE International Conference on Computer Vision. IEEE, Santiago, Chile, 4642–4650.Google ScholarDigital Library
- Jianfeng Wang, Xiaochun Cheng, Ruomei Wang, and Shaohui Liu. 2021. Learning outfit compatibility with graph attention network and visual-semantic embedding. In 2021 IEEE International Conference on Multimedia and Expo (ICME). IEEE, IEEE, Shenzhen, China, 1–6.Google ScholarCross Ref
- Zhen Wang and Hongyan Quan. 2019. Fashion outfit composition combining sequential learning and deep aesthetic network. In 2019 International Joint Conference on Neural Networks (IJCNN). IEEE, IEEE, Budapest, Hungary, 1–7.Google ScholarCross Ref
- Wang, Xin, and Wu, Bo and Zhong, Yueqi. 2019. Outfit compatibility prediction and diagnosis with multi-layered comparison network. In Proceedings of the 27th ACM International Conference on Multimedia. Association for Computing Machinery, New York, NY, United States, 329–337.Google Scholar
- Jane E Workman and Cathryn M Studak. 2006. Fashion consumers and fashion problem recognition style. International Journal of Consumer Studies 30, 1 (2006), 75–84.Google ScholarCross Ref
- Xun Yang, Yunshan Ma, Lizi Liao, Meng Wang, and Tat-Seng Chua. 2019. TransNFCM: Translation-based neural fashion compatibility modeling. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. AAAI, Honolulu, Hawaii, USA, 403–410.Google ScholarDigital Library
- Xin Yang, Xuemeng Song, Fuli Feng, Haokun Wen, Ling-Yu Duan, and Liqiang Nie. 2021. Attribute-wise explainable fashion compatibility modeling. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 17, 1 (2021), 1–21.Google ScholarDigital Library
- Xuewen Yang, Dongliang Xie, Xin Wang, Jiangbo Yuan, Wanying Ding, and Pengyun Yan. 2020. Learning tuple compatibility for conditional outfit recommendation. In Proceedings of the 28th ACM International Conference on Multimedia. Association for Computing Machinery, New York, NY, United States, 2636–2644.Google ScholarDigital Library
- Ruiping Yin, Kan Li, Jie Lu, and Guangquan Zhang. 2019. Enhancing fashion recommendation with visual compatibility relationship. In The world wide web conference. Association for Computing Machinery, New York, NY, United States, 3434–3440.Google Scholar
- Huijing Zhan, Jie Lin, Kenan Emir Ak, Boxin Shi, Ling-Yu Duan, and Alex C Kot. 2021. Attentive Attribute-Aware Fashion Knowledge Graph for Outfit Preference Prediction. IEEE Transactions on Multimedia 24 (2021), 819–831.Google ScholarCross Ref
- Heming Zhang, Xuewen Yang, Jianchao Tan, Chi-Hao Wu, Jue Wang, and C-C Jay Kuo. 2020. Learning Color Compatibility in Fashion Outfits. arXiv preprint arXiv:2007.02388 01 (2020).Google Scholar
- Ye Zhang and Byron Wallace. 2015. A sensitivity analysis of (and practitioners’ guide to) convolutional neural networks for sentence classification. arXiv preprint arXiv:1510.03820 1, 1 (2015).Google Scholar
- Na Zheng, Xuemeng Song, Qingying Niu, Xue Dong, Yibing Zhan, and Liqiang Nie. 2021. Collocation and try-on network: Whether an outfit is compatible. In Proceedings of the 29th ACM International Conference on Multimedia. Association for Computing Machinery, New York, NY, United States, 309–317.Google ScholarDigital Library
Index Terms
- A review of explainable fashion compatibility modeling methods
Recommendations
Computational Technologies for Fashion Recommendation: A Survey
Fashion recommendation is a key research field in computational fashion research and has attracted considerable interest in the computer vision, multimedia, and information retrieval communities in recent years. Due to the great demand for applications, ...
Fashion Meets Computer Vision: A Survey
Fashion is the way we present ourselves to the world and has become one of the world’s largest industries. Fashion, mainly conveyed by vision, has thus attracted much attention from computer vision researchers in recent years. Given the rapid ...
Learning Fashion Compatibility with Bidirectional LSTMs
MM '17: Proceedings of the 25th ACM international conference on MultimediaThe ubiquity of online fashion shopping demands effective recommendation services for customers. In this paper, we study two types of fashion recommendation: (i) suggesting an item that matches existing components in a set to form a stylish outfit (a ...
Comments