Abstract
Deep learning methods have demonstrated remarkable success in recognizing geochemical anomalies for mineral exploration. Typically, these methods identify anomalies by reconstructing the geochemical background, which is marked by long-distance spatial variability, giving rise to long-range spatial dependencies within geochemical signals. However, current deep learning models for geochemical anomaly recognition face limitations in capturing intricate long-range spatial dependencies. Additionally, concerns emerge from the uncertainty associated with preprocessing in existing deep learning models, which involve generating interpolated images and topological graphs to represent the spatial structure of geochemical samples. In this paper, we present a novel end-to-end method for geochemical anomaly extraction based on the Transformer model. Our model utilizes self-attention mechanism to adequately capture both global and local interconnections among geochemical samples from a holistic perspective, enabling the reconstruction of geochemical background. Moreover, the self-attention mechanism allows the Transformer model to directly input free-form geochemical samples, eliminating the uncertainty associated with the employment of prior interpolation or graph generation typically required for geochemical samples. To align geochemical data with Transformer's architecture, we tailor a specialized data organization integrating learnable positional encoding and data masking. This enables the ingestion of entire geochemical data into the Transformer for anomaly recognition. Capitalizing on the flexibility afforded by the attention mechanism, we devise a contrastive loss for training, establishing a self-supervised learning scheme that enhances model generalizability for anomaly recognition. The proposed method is utilized to recognize geochemical anomalies related to Au mineralization in the northwest Jiaodong Peninsula, Eastern China. By comparison with anomalies identified by models of graph attention network and geographically weighted regression, it is demonstrated that the proposed method is more effective and geologically sound in identifying mineralization-associated anomalies. This superior performance in geochemical anomaly recognition is attributed to its ability to capture long-range dependencies within geochemical data.
Similar content being viewed by others
References
Afzal, P., Khakzad, A., Moarefvand, P., Omran, N. R., Esfandiari, B., & Alghalandis, Y. F. (2010). Geochemical anomaly separation by multifractal modeling in Kahang (Gor Gor) porphyry system, Central Iran. Journal of Geochemical Exploration, 104(1), 34–46.
Afzal, P., Harati, H., Alghalandis, Y. F., & Yasrebi, A. B. (2013). Application of spectrum–area fractal model to identify of geochemical anomalies based on soil data in Kahang porphyry-type Cu deposit, Iran. Geochemistry, 73(4), 533–543.
Agterberg, F. P., & Bonham-Carter, G. F. (2005). Measuring the performance of mineral-potential maps. Natural Resources Research, 14(1), 1–17.
Aitchison, J. (1982). The statistical analysis of compositional data. Journal of the Royal Statistical Society: Series B (Methodological), 44(2), 139–160.
Albanese, S., De Vivo, B., Lima, A., & Cicchella, D. (2007). Geochemical background and baseline values of toxic elements in stream sediments of Campania region (Italy). Journal of Geochemical Exploration, 93(1), 21–34.
Allègre, C. J., & Lewin, E. (1995). Scaling laws and geochemical distributions. Earth and Planetary Science Letters, 132(1), 1–13.
Beal, J., Kim, E., Tzeng, E., Park, D. H., Zhai, A., & Kislyuk, D. (2020). Toward transformer-based object detection. arXiv preprintarXiv:2012.09958.
Bölviken, B., Stokke, P. R., Feder, J., & Jössang, T. (1992). The fractal nature of geochemical landscapes. Journal of Geochemical Exploration, 43(2), 91–109.
Cameron, E. M., Hamilton, S. M., Leybourne, M. I., Hall, G. E. M., & McClenaghan, M. B. (2004). Finding deeply buried deposits using geochemistry. Geochemistry: Exploration, Environment, Analysis, 4(1), 7–32.
Chen, J., Mao, X., Deng, H., Liu, Z., & Wang, Q. (2020a). Three-dimensional modelling of alteration zones based on geochemical exploration data: An interpretable machine-learning approach via generalized additive models. Applied Geochemistry, 123, 104781.
Chen, L., Guan, Q., Feng, B., Yue, H., Wang, J., & Zhang, F. (2019). A multi-convolutional autoencoder approach to multivariate geochemical anomaly recognition. Minerals, 9(5), 270.
Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020). A simple framework for contrastive learning of visual representations. In International conference on machine learning, 1597–1607.
Chen, Y., & Wu, W. (2016). A prospecting cost-benefit strategy for mineral potential mapping based on ROC curve analysis. Ore Geology Reviews, 74, 26–38.
Chen, Y., & Wu, W. (2017). Mapping mineral prospectivity by using one-class support vector machine to identify multivariate geological anomalies from digital geological survey data. Australian Journal of Earth Sciences, 64(5), 639–651.
Chen, Y., Lu, L., & Li, X. (2014). Application of continuous restricted Boltzmann machine to identify multivariate geochemical anomaly. Journal of Geochemical Exploration, 140, 56–63.
Chen, Z., Cheng, Q., Chen, J., & Xie, S. (2007). A novel iterative approach for mapping local singularities from geochemical data. Nonlinear Processes in Geophysics, 14(3), 317–324.
Cheng, Q. (1999). Spatial and scaling modelling for geochemical anomaly separation. Journal of Geochemical Exploration, 65(3), 175–194.
Cheng, Q. (2007). Mapping singularities with stream sediment geochemical data for prediction of undiscovered mineral deposits in Gejiu, Yunnan Province, China. Ore Geology Reviews, 32(1), 314–324.
Cheng, Q. (2012). Singularity theory and methods for mapping geochemical anomalies caused by buried sources and for predicting undiscovered mineral deposits in covered areas. Journal of Geochemical Exploration, 122, 55–70.
Cheng, Q., Agterberg, F. P., & Ballantyne, S. B. (1994). The separation of geochemical anomalies from background by fractal methods. Journal of Geochemical Exploration, 51(2), 109–130.
Cheng, Q., Xu, Y., & Grunsky, E. (2000). Integrated spatial and spectrum method for geochemical anomaly separation. Natural Resources Research, 9(1), 43–52.
Cohen, D. R., Kelley, D. L., Anand, R., & Coker, W. B. (2010). Major advances in exploration geochemistry, 1998–2007. Geochemistry: Exploration, Environment, Analysis, 10(1), 3–16.
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q. V., & Salakhutdinov, R. (2019). Transformer-XL: Attentive language models beyond a fixed-length context. arXiv preprintarXiv:1901.02860.
Deng, J., Yang, L.-Q., Groves, D. I., Zhang, L., Qiu, K.-F., & Wang, Q.-F. (2020). An integrated mineral system model for the gold deposits of the giant Jiaodong province, eastern China. Earth-Science Reviews, 208, 103274.
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: Human language technologies, 1, 4171–4186.
Doherty, S., Tighe, M. K., Milan, L. A., Johannessen, B., Mitchell, V., Hamilton, J., Jonston, S. G., & Wilson, S. C. (2021). Long-range spatial variability in sediment associations and solid-phase speciation of antimony and arsenic in a mining-impacted river system. Applied Geochemistry, 135, 105112.
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., & Houlsby, N. (2020). An image is worth 16×16 words: Transformers for image recognition at scale. arXiv preprintarXiv:2010.11929.
Egozcue, J. J., Pawlowsky-Glahn, V., Mateu-Figueras, G., & Barcelo-Vidal, C. (2003). Isometric logratio transformations for compositional data analysis. Mathematical Geology, 35(3), 279–300.
Engel, N., Belagiannis, V., & Dietmayer, K. (2021). Point Transformer. IEEE Access, 9, 134826–134840.
Fan, H. R., Zhai, M. G., Xie, Y. H., & Yang, J. H. (2003). Ore-forming fluids associated with granite-hosted gold mineralization at the Sanshandao deposit, Jiaodong gold province, China. Mineralium Deposita, 38(6), 739–750.
Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognition Letters, 27(8), 861–874.
Filzmoser, P., & Hron, K. (2009). Correlation analysis for compositional data. Mathematical Geosciences, 41(8), 905–919.
Fletcher, W. K. (1997). Stream sediment geochemistry in today’s exploration world. In Proceedings of exploration, 97, 249–260.
Gałuszka, A. (2007). A review of geochemical background concepts and an example using data from Poland. Environmental geology, 52, 861–870.
Goldfarb, R. J., & Santosh, M. (2014). The dilemma of the Jiaodong gold deposits: Are they unique? Geoscience Frontiers, 5(2), 139–153.
Griffith, D. A., & Chun, Y. (2012). Spatial Statistics and Geostatistics: Theory and Applications for Geographic Information Science and Technology. Spatial Statistics and Geostatistics, 1–200.
Grunsky, E. C., & Agterberg, F. P. (1988). Spatial and multivariate analysis of geochemical data from metavolcanic rocks in the Ben Nevis area. Ontario. Mathematical Geology, 20(7), 825–861.
Guan, Q., Ren, S., Chen, L., Feng, B., & Yao, Y. (2021). A spatial-compositional feature fusion convolutional autoencoder for multivariate geochemical anomaly recognition. Computers & Geosciences, 156, 104890.
Guan, Q., Ren, S., Chen, L., Yao, Y., Hu, Y., Wang, R., & Chen, W. (2022). Recognizing multivariate geochemical anomalies related to mineralization by using deep unsupervised graph learning. Natural Resources Research, 31(5), 2225–2245.
Guo, P., Santosh, M., & Li, S. (2013). Geodynamics of gold metallogeny in the Shandong Province, NE China: An integrated geological, geophysical and geochemical perspective. Gondwana Research, 24(3), 1172–1202.
Han, K., Wang, Y., Chen, H., Chen, X., Guo, J., Liu, Z., & Tao, D. (2023). A survey on vision transformer. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(1), 87–110.
Hawkes, H. E., & Webb, J. S. (1963). Geochemistry in mineral exploration. In Soil Science, 95(4), 283.
Inácio, M., Pereira, V., & Pinto, M. (2008). The soil geochemical atlas of Portugal: overview and applications. Journal of Geochemical Exploration, 98(1–2), 22–33.
Jimenez-Espinosa, R., Sousa, A. J., & Chica-Olmo, M. (1993). Identification of geochemical anomalies using principal component analysis and factorial kriging analysis. Journal of Geochemical Exploration, 46(3), 245–256.
Jing, L., & Tian, Y. (2021). Self-supervised visual feature learning with deep neural networks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(11), 4037–4058.
Li, T., Zuo, R., Xiong, Y., & Peng, Y. (2021). Random-drop data augmentation of deep convolutional neural network for mineral prospectivity mapping. Natural Resources Research, 30(1), 27–38.
Liu, C., Yang, H., Fu, J., & Qian, X. (2022). Learning trajectory-aware transformer for video super-resolution. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 5687–5696
Liu, X., Zhang, F., Hou, Z., Mian, L., Wang, Z., Zhang, J., & Tang, J. (2023). Self-supervised learning: Generative or contrastive. IEEE Transactions On knowledge and Data Engineering, 35(1), 857–876.
Liu, Z., Mao, X., Jedemann, A., Bayless, R. C., Deng, H., Chen, J., & Xiao, K. (2021a). Evolution of pyrite compositions at the Sizhuang Gold Deposit, Jiaodong Peninsula, Eastern China: Implications for the genesis of Jiaodong-type orogenic gold mineralization. Minerals, 11(4), 344.
Liu, Z., Mao, X., Wang, F., Tang, L., Chen, G., Chen, J., & Deng, H. (2021b). Deciphering anomalous Ag enrichment recorded by galena in Dayingezhuang Au(-Ag) deposit, Jiaodong Peninsula, Eastern China. Transactions of Nonferrous Metals Society of China, 31(12), 3831–3846.
Liu, Z., Hollings, P., Mao, X., Lawley, C. J. M., Yang, B., & Tang, L. (2021c). Metal remobilization from country rocks into the Jiaodong-type orogenic gold systems, Eastern China: New constraints from scheelite and galena isotope results at the Xiadian and Majiayao gold deposits. Ore Geology Reviews, 134, 104126.
Luo, Z., Xiong, Y., & Zuo, R. (2020). Recognition of geochemical anomalies using a deep variational autoencoder network. Applied Geochemistry, 104710.
Luo, Z., Zuo, R., Xiong, Y., & Wang, X. (2021). Detection of geochemical anomalies related to mineralization using the GANomaly network. Applied Geochemistry, 131, 105043.
Ma, Y. Z., & Ma, Y. Z. (2019). Multiscale heterogeneities in reservoir geology and petrophysical properties. Quantitative Geosciences: Data Analytics, Geostatistics, Reservoir Characterization and Modeling, pp. 175–200.
Mao, X., Ren, J., Liu, Z., Chen, J., Tang, L., Deng, H., & Liu, C. (2019). Three-dimensional prospectivity modeling of the Jiaojia-type gold deposit, Jiaodong Peninsula, Eastern China: A case study of the Dayingezhuang deposit. Journal of Geochemical Exploration, 203, 27–44.
Matschullat, J., Ottenstein, R., & Reimann, C. (2000). Geochemical background–can we calculate it? Environmental geology, 39, 990–1000.
Moeini, H., & Torab, F. M. (2017). Comparing compositional multivariate outliers with autoencoder networks in anomaly detection at Hamich exploration area, east of Iran. Journal of Geochemical Exploration, 180, 15–23.
Mosammam, A. M. (2013). Geostatistics: Modeling spatial uncertainty, second edition. Journal of Applied Statistics, 40(4), 923–923.
Pourgholam, M. M., Afzal, P., Yasrebi, A. B., Gholinejad, M., & Wetherelt, A. (2021). Detection of geochemical anomalies using a fractal-wavelet model in Ipack area, Central Iran. Journal of Geochemical Exploration, 220, 106675.
Regenauer-Lieb, K., Veveakis, M., Poulet, T., Wellmann, F., Karrech, A., Liu, J., & Trefry, M. (2013). Multiscale coupling and multiphysics approaches in earth sciences: Theory. Journal of Coupled Systems and Multiscale Dynamics, 1(1), 49–73.
Reimann, C., & de Caritat, P. (2017). Establishing geochemical background variation and threshold values for 59 elements in Australian surface soil. Science of the Total Environment, 578, 633–648.
Reimann, C., & Garrett, R. G. (2005). Geochemical background—concept and reality. Science of the Total Environment, 350(1–3), 12–27.
Reimann, C., Kashulina, G., de Caritat, P., & Niskavaara, H. (2001). Multi-element, multi-medium regional geochemistry in the European Arctic: Element concentration, variation and correlation. Applied Geochemistry, 16(7–8), 759–780.
Sinclair, A. J. (1974). Selection of threshold values in geochemical data using probability graphs. Journal of Geochemical Exploration, 3(2), 129–149.
Slukovskii, Z., Medvedev, M., & Siroezhko, E. (2020). Long-range transport of heavy metals as a factor of the formation of the geochemistry of sediments in the southwest of the Republic of Karelia, Russia. Journal of Elementology, 25(1).
Song, M. C., Li, S. Z., Santosh, M., Zhao, S., Yu, S., Yi, P. H., & Zhou, M. L. (2015). Types, characteristics and metallogenesis of gold deposits in the Jiaodong Peninsula, Eastern North China Craton. Ore Geology Reviews, 65, 612–625.
Tian, M., Wang, X., Nie, L., & Zhang, C. (2018). Recognition of geochemical anomalies based on geographically weighted regression: A case study across the boundary areas of China and Mongolia. Journal of Geochemical Exploration, 190, 381–389.
Torshizian, H., Afzal, P., Rahbar, K., Yasrebi, A. B., Wetherelt, A., & Fyzollahhi, N. (2021). Application of modified wavelet and fractal modeling for detection of geochemical anomaly. Geochemistry, 81(4), 125800.
Tripathi, V. S. (1979). Factor analysis in geochemical exploration. Journal of Geochemical Exploration, 11(3), 263–275.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30.
Wang, H., & Zuo, R. (2015). A comparative study of trend surface analysis and spectrum–area multifractal model to identify geochemical anomalies. Journal of Geochemical Exploration, 155, 84–90.
Wang, W., Zhao, J., Cheng, Q., & Liu, J. (2012). Tectonic–geochemical exploration modeling for characterizing geo-anomalies in southeastern Yunnan district, China. Journal of Geochemical Exploration, 122, 71–80.
Xie, X., Mu, X., & Ren, T. (1997). Geochemical mapping in China. Journal of Geochemical Exploration, 60(1), 99–113.
Xiong, Y., & Zuo, R. (2016). Recognition of geochemical anomalies using a deep autoencoder network. Computers & Geosciences, 86, 75–82.
Xiong, Y., & Zuo, R. (2022). Robust feature extraction for geochemical anomaly recognition using a stacked convolutional denoising autoencoder. Mathematical Geosciences, 54(3), 623–644.
Xiong, Y., Zuo, R., Luo, Z., & Wang, X. (2021). A physically constrained variational autoencoder for geochemical pattern recognition. Mathematical Geosciences.
Xu, Y., & Zuo, R. (2023). An interpretable graph attention network for mineral prospectivity mapping. Mathematical Geosciences.
Yang, B., Wang, H., Han, J., Zhou, W., Liu, G., Yan, Q., & Wang, Y. (2014). Oriental prognosis of deep orebodies based on multiple abnormal information in Hedong gold deposit, northwest Jiaodong. Earth Science Frontiers, 21(5), 221–226.
Yang, L. Q., Deng, J., Wang, Z. L., Guo, L. N., Li, R. H., Groves, D. I., & Zhao, H. (2016). Relationships between gold and pyrite at the Xincheng Gold Deposit, Jiaodong Peninsula, China: Implications for gold source and deposition in a brittle epizonal environment. Economic Geology, 111(1), 105–126.
Yang, L., Zhao, R., Wang, Q., Liu, X., & Carranza, E. J. M. (2018). Fault geometry and fluid-rock reaction: Combined controls on mineralization in the Xinli gold deposit, Jiaodong Peninsula, China. Journal of Structural Geology, 111, 14–26.
Yousefi, M., Carranza, E. J. M., & Kamkar-Rouhani, A. (2013). Weighted drainage catchment basin mapping of geochemical anomalies using stream sediment data for mineral potential modeling. Journal of Geochemical Exploration, 128, 88–96.
Yu, S., Deng, H., Liu, Z., Chen, J., Gu, X., Li, J., & Mao, X. (2022). Identifying multivariate geochemical anomalies via tensor dictionary learning over spatial-elemental dimensionalities. Computers & Geosciences, 165, 105153.
Zhang, C., Zuo, R., & Xiong, Y. (2021). Detection of the multivariate geochemical anomalies associated with mineralization using a deep convolutional neural network and a pixel-pair feature method. Applied Geochemistry, 130, 104994.
Zhao, H., Jiang, L., Jia, J., Torr, P. H., & Koltun, V. (2021). Point transformer. In Proceedings of the IEEE/CVF international conference on computer vision, 16259–16268.
Zuo, R., & Xiong, Y. (2018). Big data analytics of identifying geochemical anomalies supported by machine learning methods. Natural Resources Research, 27(1), 5–13.
Zuo, R., & Xiong, Y. (2020). Geodata science and geochemical mapping. Journal of Geochemical Exploration, 209, 106431.
Zuo, R., & Xu, Y. (2023). Graph deep learning model for mapping mineral prospectivity. Mathematical Geosciences, 55(1), 1–21.
Zuo, R., Carranza, E. J. M., & Wang, J. (2016). Spatial analysis and visualization of exploration geochemical data. Earth-Science Reviews, 158, 9–18.
Zuo, R., Xiong, Y., Wang, J., & Carranza, E. J. M. (2019). Deep learning and its application in geochemical mapping. Earth-Science Reviews, 192, 1–14.
Acknowledgments
This research was funded by National Key R&D Program of China (No. 2023YFC2906403), National Natural Science Foundation of China (Nos. 42272344, 41972309, 42030809), Science and Technology Innovation Program of Hunan Province (2021RC4055), and Open Topic of Hunan Engineering Research Center of Geographic Information Security and Application (HNGISA2003006).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Yu, S., Deng, H., Liu, Z. et al. Identification of Geochemical Anomalies Using an End-to-End Transformer. Nat Resour Res 33, 973–994 (2024). https://doi.org/10.1007/s11053-024-10334-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11053-024-10334-4