Skip Abstract Section
Skip Supplemental Material SectionAbstract
Over the past decade, three-dimensional (3D) graphics have become highly detailed to mimic the real world, exploding their size and complexity. Certain applications and device constraints necessitate their simplification and/or lossy compression, which can degrade their visual quality. Thus, to ensure the best Quality of Experience, it is important to evaluate the visual quality to accurately drive the compression and find the right compromise between visual quality and data size. In this work, we focus on subjective and objective quality assessment of textured 3D meshes. We first establish a large-scale dataset, which includes 55 source models quantitatively characterized in terms of geometric, color, and semantic complexity, and corrupted by combinations of five types of compression-based distortions applied on the geometry, texture mapping, and texture image of the meshes. This dataset contains over 343k distorted stimuli. We propose an approach to select a challenging subset of 3,000 stimuli for which we collected 148,929 quality judgments from over 4,500 participants in a large-scale crowdsourced subjective experiment. Leveraging our subject-rated dataset, a learning-based quality metric for 3D graphics was proposed. Our metric demonstrates state-of-the-art results on our dataset of textured meshes and on a dataset of distorted meshes with vertex colors. Finally, we present an application of our metric and dataset to explore the influence of distortion interactions and content characteristics on the perceived quality of compressed textured meshes.
Supplemental Material
- . 2020. Perceptual characterization of 3D graphical contents based on attention complexity measures. In Proceedings of the 1st Workshop on Quality of Experience (QoE) in Visual Multimedia Applications (QoEVMA’20). 31–36. Google Scholar
Digital Library
- . 2020. No-reference mesh visual quality assessment via ensemble of convolutional neural networks and compact multi-linear pooling. Pattern Recogn. 100 (2020).Google ScholarDigital Library
- . 2017. A convolutional neural network framework for blind mesh visual quality assessment. In Proceedings of the IEEE International Conference on Image Processing (ICIP’17). 755–759. Google ScholarDigital Library
- . 2017. On the performance of metrics to predict quality in point cloud representations. In Applications of Digital Image Processing XL, (Ed.), Vol. 10396. International Society for Optics and Photonics, SPIE, 282–297.Google Scholar
- . 2018. Point cloud quality assessment metric based on angular similarity. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME’18). 1–6. Google ScholarCross Ref
- . 2017. Towards subjective quality assessment of point cloud imaging in augmented reality. In Proceedings of the IEEE 19th International Workshop on Multimedia Signal Processing (MMSP’17). 1–6. Google ScholarCross Ref
- . 2019. A comprehensive study of the rate-distortion performance in MPEG point cloud compression. APSIPA Trans. Signal Info. Process. 8 (2019), e27.Google ScholarCross Ref
- . 2020. PointXR: A toolbox for visualization and subjective evaluation of point clouds in virtual reality. In Proceedings of the 12th International Conference on Quality of Multimedia Experience (QoMEX’20). 1–6. Google ScholarCross Ref
- . 2002. MESH: Measuring errors between surfaces using the Hausdorff distance. In Proceedings of the IEEE International Conference on Multimedia and Expo. Google ScholarCross Ref
- . 2018. Deep neural networks for no-reference and full-reference image quality assessment. IEEE Trans. Image Process. 27, 1 (2018), 206–219. Google ScholarCross Ref
- . 2016. Progressive compression of arbitrary textured meshes. Comput. Graph. Forum 35, 7 (
Oct. 2016), 475–484. Google ScholarDigital Library - . 2018. Three-dimensional mesh quality metric with reference based on a support vector regression model. J. Electr. Imag. 27, 4 (2018), 1–9. Google ScholarCross Ref
- . 2018. Subjective visual quality assessment of immersive 3D media compressed by open-source static 3D mesh codecs. In Proceedings of the 25th International Conference on MultiMedia Modeling (MMM’18). 1–12.Google Scholar
- . 2008. MeshLab: An open-source mesh processing tool. In Proceedings of the Eurographics Italian Chapter Conference. The Eurographics Association. Google ScholarCross Ref
- . 2007. Watermarked 3D mesh quality assessment. IEEE Trans. Multimedia 9 (2007), 247–256. Google ScholarDigital Library
- . 2009. Reduced-reference metric design for objective perceptual quality assessment in wireless imaging. Signal Process.: Image Commun. 24, 7 (2009), 525–547. Google ScholarDigital Library
- . 2017. DeepSim: Deep similarity for image quality assessment. Neurocomputing 257 (2017), 104–114.
Machine Learning and Signal Processing for Big Multimedia Analysis . Google ScholarCross Ref - . 1998. Simplifying surfaces with color and texture using quadric error metrics. In Proceedings of the Conference on Visualization. 263–269. Google Scholar
- . 2016. Massive online crowdsourced study of subjective and objective picture quality. IEEE Trans. Image Process. 25, 1 (2016), 372–387. Google ScholarDigital Library
- . 2016. Subjective and objective visual quality assessment of textured 3D meshes. ACM Trans. Appl. Percept. 14 (
10 2016), 1–20. Google ScholarDigital Library - . 2020. Quality evaluation of 3D objects in mixed reality for different lighting conditions. Electr. Imag. 2020 (
Jan. 2020). Google ScholarCross Ref - . 2021. Towards a colored point cloud quality assessment method using colored texture and curvature projection. In Proceedings of the IEEE International Conference on Image Processing (ICIP’21). 1444–1448. Google ScholarCross Ref
- . 2014. Best practices for QoE crowdtesting: QoE assessment with crowdsourcing. IEEE Trans. Multimedia 16, 2 (2014), 541–558. Google ScholarDigital Library
- . 2018. Multimodal unsupervised image-to-image translation. In Proceedings of the European Conference on Computer Vision (ECCV’18). 179–196. Google ScholarDigital Library
- . 2012. Methodology for the subjective assessment of the quality of television pictures BT Series Broadcasting service. Technical report. International Telecommunication Union, Geneva, Switzerland.Google Scholar
- . 2008. Subjective video quality assessment methods for multimedia applications. Technical report. International Telecommunication Union, Geneva, Switzerland.Google Scholar
- . 2017. Subjective and objective quality evaluation of compressed point clouds. In Proceedings of the IEEE 19th International Workshop on Multimedia Signal Processing (MMSP’17). 1–6. Google ScholarCross Ref
- . 2019. Point cloud rendering after coding: Impacts on subjective and objective quality. Retrieved from https://arXiv:1912.09137.Google Scholar
- . 2020. Mahalanobis based point to distribution metric for point cloud geometry quality evaluation. IEEE Signal Process. Lett. 27 (2020), 1350–1354. Google ScholarCross Ref
- . 2018. Influence of number of stimuli for subjective speech quality assessment in crowdsourcing. In Proceedings of the 10th International Conference on Quality of Multimedia Experience (QoMEX’18) (2018), 1–6. Google ScholarCross Ref
- . 2014. Convolutional neural networks for no-reference image quality assessment. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1733–1740. Google ScholarDigital Library
- . 1995. A distortion measure for blocking artifacts in images based on human visual sensitivity. IEEE Trans. Image Process. 4, 6 (1995), 713–724. Google ScholarDigital Library
- . 2016. On the accuracy of objective image and video quality models: New methodology for performance evaluation. In Proceedings of the 8th International Conference on Quality of Multimedia Experience (QoMEX) (2016), 1–6.Google Scholar
- . 2009. A local roughness measure for 3D meshes and its application to visual masking. ACM Trans. Appl. Percept. 5, 4, Article
21 (2009), 23 pages. Google ScholarDigital Library - . 2011. A multiscale metric for 3D mesh visual quality assessment. Comput. Graph. Forum 30, 5 (2011), 1427–1437.Google ScholarCross Ref
- . 2016. On the efficiency of image metrics for evaluating the visual quality of 3D models. IEEE Trans. Visual. Comput. Graph. 22, 8 (2016), 1987–1999. Google ScholarDigital Library
- . 2021. Benchmarking of objective quality metrics for point cloud compression. In Proceedings of the 23rd IEEE International Workshop on Multimedia Signal Processing (MMSP’21). 1–6.Google ScholarCross Ref
- . 2021b. Reduced reference perceptual quality model with application to rate control for video-based point cloud compression. IEEE Trans. Image Process. 30 (2021), 6623–6636. Google ScholarDigital Library
- . 2021c. PQA-Net: Deep no reference point cloud quality assessment via multi-view projection. IEEE Trans. Circ. Syst. Video Technol. 31, 12 (2021), 4645–4660. Google ScholarCross Ref
- . 2021a. Point cloud quality assessment: Dataset construction and learning-based no-reference approach. Retrieved from https://arXiv:2012.11895.Google Scholar
- . 2020. Real-world textured things: A repository of textured models generated with modern photo-reconstruction tools. Retrieved from https://arXiv:2004.14753.Google Scholar
- . 2011. HDR-VDP-2: A calibrated visual metric for visibility and quality predictions in all luminance conditions. ACM Trans. Graph. 30, 4, Article
40 (July 2011), 14 pages. Google ScholarDigital Library - . 2012. Comparison of four subjective methods for image quality assessment. Comput. Graph. Forum 31, 8 (
Dec. 2012), 2478–2491. http://doi.wiley.com/10.1111/j.1467-8659.2012.03188.xGoogle ScholarCross Ref - . 2017. Design, implementation, and evaluation of a point cloud codec for tele-immersive video. IEEE Trans. Circ. Syst. Video Technol. 27, 4 (
Apr. 2017), 828–842.Google ScholarDigital Library - . 2019. PC-MSDM: A quality metric for 3D point clouds. In Proceedings of the 11th International Conference on Quality of Multimedia Experience (QoMEX’19). 1–3. Google ScholarCross Ref
- . 2020. PCQM: A full-reference quality metric for colored 3D point clouds. In Proceedings of the 12th International Conference on Quality of Multimedia Experience (QoMEX’20). 1–6. Google ScholarCross Ref
- . 2021a. Exploring crowdsourcing for subjective quality assessment of 3D graphics. In Proceedings of the IEEE International Workshop on Multimedia Signal Processing (MMSP’21).Google Scholar
- . 2021b. Visual quality of 3D meshes with diffuse colors in virtual reality: Subjective and objective evaluation. IEEE Trans. Visual. Comput. Graph. 27, 3 (2021), 2202–2219. Google ScholarCross Ref
- . 2020. Comparison of subjective methods for quality assessment of 3D graphics in virtual reality. ACM Trans. Appl. Percept. 18, 1 (
Dec. 2020), 1–23. Google ScholarDigital Library - . 2017. 3D blind mesh quality assessment index. In Proceedings of the IS&T International Symposium on Electronic Imaging.Google ScholarCross Ref
- . 2005. Quality metric for approximating subjective evaluation of 3D objects. IEEE Trans. Multimedia 7, 2 (
Apr. 2005), 269–279. Google ScholarDigital Library - . 2020. Quality evaluation of static point clouds encoded using MPEG codecs. In Proceedings of the IEEE International Conference on Image Processing (ICIP’20). 3428–3432. Google ScholarCross Ref
- . 2014. Color-image quality assessment: From prediction to optimization. IEEE Trans. Image Process. 23, 3 (2014), 1366–1378. Google ScholarDigital Library
- . 2021. A deep perceptual metric for 3D point clouds. Electr. Imag. 2021, 9 (2021), 2571–2577. Google ScholarCross Ref
- . 2015. When the crowd challenges the lab: Lessons learnt from subjective studies on image aesthetic appeal. In Proceedings of the 4th International Workshop on Crowdsourcing for Multimedia. 33–38. Google ScholarDigital Library
- . 2021. Teaching data-driven video processing via crowdsourced data collection. In Proceedings of Eurographics’21: Education Papers. Google ScholarCross Ref
- . 2001. Are image quality metrics adequate to evaluate the quality of geometric objects? InProceedings of SPIE: The International Society for Optical Engineering. Google ScholarCross Ref
- . 2011. Perceptual models of viewpoint preference. ACM Trans. Graph. 30, 5 (
Oct. 2011).Google ScholarDigital Library - . 2006. Image information and visual quality. IEEE Trans. Image Process. 15, 2 (
Feb. 2006), 430–444. Google ScholarDigital Library - . 2019. Perceptual quality assessment of 3D point clouds. In Proceedings of the IEEE International Conference on Image Processing (ICIP’19). 3182–3186.Google ScholarCross Ref
- . 2020. Comparing the quality of highly realistic digital humans in 3DoF and 6DoF: A volumetric video case study. In Proceedings of the IEEE Conference on Virtual Reality and 3D User Interfaces (VR’20). IEEE, 127–136.Google ScholarCross Ref
- . 2018. NIMA: Neural image assessment. IEEE Trans. Image Process. 27, 8 (2018), 3998–4011. Google ScholarCross Ref
- . 2021. Point Cloud Projection and Multi-Scale Feature Fusion Network Based Blind Quality Assessment for Colored Point Clouds. New York, NY, 5266–5272. Google Scholar
- . 2020. Why are deep representations good perceptual quality features? Computer Vision – ECCV 2020 (2020), 445–461. Google ScholarDigital Library
- . 2008. Batex3: Bit allocation for progressive transmission of textured 3D models. IEEE Trans. Circ. Syst. Video Technol. 18, 1 (2008), 23–35.Google ScholarDigital Library
- . 2017. Geometric distortion metrics for point cloud compression. In Proceedings of the IEEE International Conference on Image Processing (ICIP’17). 3460–3464. Google ScholarDigital Library
- . 2014. A curvature-tensor-based perceptual quality metric for 3D triangular meshes. Mach. Graph. Vision 23, 1 (2014), 59–82.Google ScholarCross Ref
- . 2015. Perceptual quality assessment of 3D dynamic meshes: Subjective and objective studies. Signal Process.: Image Commun. 31, 2 (
Feb. 2015), 185–204. Google ScholarDigital Library - . 2018. A novel methodology for quality assessment of voxelized point clouds. In Applications of Digital Image Processing XLI, (Ed.), Vol. 10752. International Society for Optics and Photonics, SPIE, 174–190.Google Scholar
- . 2017. Visual quality assessment of 3D models: On the influence of light-material interaction. ACM Trans. Appl. Percept. 15, 1 (2017). Google ScholarDigital Library
- . 2012. Dihedral angle mesh error: A fast perception correlated distortion measure for fixed connectivity triangle meshes. Comput. Graph. Forum 31, 5 (2012).Google ScholarDigital Library
- . 2020. A color-based objective quality metric for point cloud contents. In Proceedings of the 12th International Conference on Quality of Multimedia Experience (QoMEX’20). 1–6. Google ScholarCross Ref
- . 2022. On the impact of VR assessment on the Quality of Experience of Highly Realistic Digital Humans. Retrieved from https://arXiv:2201.07701.Google Scholar
- . 2010. Report on the validation of video quality models for high definition video content. Technical report. Video Quality Experts Group.Google Scholar
- . 2012. A fast roughness-based approach to the assessment of 3D mesh visual quality. Comput. Graph. 36, 7 (2012), 808–818.Google ScholarDigital Library
- . 2004. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 13, 4 (2004), 600–612.Google ScholarDigital Library
- . 2001. Measuring and predicting visual fidelity. ACM Siggraph (2001), 213–220. Google Scholar
- . 2020. End-to-end blind image quality prediction with cascaded deep neural network. IEEE Trans. Image Process. 29 (2020), 7414–7426. Google ScholarDigital Library
- . 2021. Subjective quality database and objective study of compressed point clouds with 6DoF head-mounted display. IEEE Trans. Circ. Syst. Video Technol. 31, 12 (2021), 4630–4644.Google ScholarCross Ref
- . 2018. Pose guided human video generation. In Proceedings of the European Conference on Computer Vision (ECCV’18). 204–219. Google ScholarDigital Library
- . 2020. Predicting the perceptual quality of point cloud: A 3D-to-2D projection-based exploration. IEEE Trans. Multimedia 23 (2020), 3877–3891.Google Scholar
- . 2020. Inferring point cloud quality via graph similarity. IEEE Trans. Pattern Anal. Mach. Intell. 44, 6 (2020), 3015–3029. Google ScholarCross Ref
- . 2020. A machine learning framework for full-reference 3D shape quality assessment. Visual Comput. 36, 1 (2020), 127–139. Google ScholarDigital Library
- . 2013. Image complexity and spatial information. In Proceedings of the 5th International Workshop on Quality of Multimedia Experience (QoMEX’13). 12–17. Google ScholarCross Ref
- . 2019. Subjective and objective quality assessment for volumetric video compression. Electr. Imag. 2019, 10 (2019), 323–1.Google Scholar
- . 2020. Textured mesh vs coloured point cloud: A subjective study for volumetric video compression. In Proceedings of the 12th International Conference on Quality of Multimedia Experience (QoMEX’20). IEEE, 1–6.Google ScholarCross Ref
- . 2018. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’18). 586–595. Google ScholarCross Ref
- . 2021. MS-GraphSIM: Inferring Point Cloud Quality via Multiscale Graph Similarity. ACM, New York, NY, 1230–1238. Google ScholarDigital Library
- . 2010. Quantitative analysis of discrete 3D geometrical detail levels based on perceptual metric. Comput. Graph. 34, 1 (2010), 55–65. Google ScholarDigital Library
Index Terms
- Textured Mesh Quality Assessment: Large-scale Dataset and Deep Learning-based Quality Metric
Recommendations
Perceptual Quality Metrics for 3D Meshes: Towards an Optimal Multi-attribute Computational Model
SMC '13: Proceedings of the 2013 IEEE International Conference on Systems, Man, and Cybernetics3D graphical data, commonly represented using triangular meshes, are deployed in a wide range of application processes including compression, filtering, watermarking, and simplification. These processes often introduce geometric distortions which affect ...
Subjective and Objective Visual Quality Assessment of Textured 3D Meshes
Objective visual quality assessment of 3D models is a fundamental issue in computer graphics. Quality assessment metrics may allow a wide range of processes to be guided and evaluated, such as level of detail creation, compression, filtering, and so on. ...
A machine learning framework for full-reference 3D shape quality assessment
AbstractTo decide whether the perceived quality of a mesh is influenced by a certain modification such as compression or simplification, a metric for estimating the visual quality of 3D meshes is required. Today, machine learning and deep learning ...
Comments