Skip Abstract Section
Abstract
Tiny Machine Learning (TinyML) is an emerging technology proposed by the scientific community for developing autonomous and secure devices that can gather, process, and provide results without transferring data to external entities. The technology aims to democratize AI by making it available to more sectors and contribute to the digital revolution of intelligent devices. In this work, a classification of the most common optimization techniques for Neural Network compression is conducted. Additionally, a review of the development boards and TinyML software is presented. Furthermore, the work provides educational resources, a classification of the technology applications, and future directions and concludes with the challenges and considerations.
- [n.d.]. TinyML in Publications - Dimensions. https://app.dimensions.ai/discover/publication?search_mode=content&search_text=TinyML&search_type=kws&search_field=full_searchGoogle Scholar
- Nasir Abbas, Yan Zhang, Amir Taherkordi, and Tor Skeie. 2017. Mobile edge computing: A survey. IEEE Internet of Things Journal 5, 1 (2017), 450–465.Google ScholarCross Ref
- Jashaswimalya Acharjee and Suman Deb. 2021. Cartoonize Images using TinyML Strategies with Transfer Learning. In 2021 International Conference on Innovation and Intelligence for Informatics, Computing, and Technologies (3ICT). IEEE, 411–417.Google ScholarCross Ref
- Adafruit 2021. Adafruit EdgeBadge - TensorFlow Lite for Microcontrollers. https://www.adafruit.com/product/4400 Retrieved December 30, 2021 fromGoogle Scholar
- Francesco Alongi, Nicolo Ghielmetti, Danilo Pau, Federico Terraneo, and William Fornaciari. 2020. Tiny Neural Networks for Environmental Predictions: an integrated approach with Miosix. In 2020 IEEE International Conference on Smart Computing (SMARTCOMP). IEEE, 350–355.Google ScholarCross Ref
- Amazon 2022. Amazon Alexa. https://www.amazon.com/b?ie=UTF8&node=21576558011 Retrieved January 8, 2022 fromGoogle Scholar
- AmbiQmicro 2019. Apollo3 Blue Datasheet. https://cdn.sparkfun.com/assets/learn_tutorials/9/0/9/Apollo3_Blue_MCU_Data_Sheet_v0_9_1.pdf Retrieved January 11, 2022 fromGoogle Scholar
- Pedro Andrade, Ivanovitch Silva, Gabriel Signoretti, Marianne Silva, João Dias, Lucas Marques, and Daniel G Costa. 2021. An Unsupervised TinyML Approach Applied for Pavement Anomalies Detection Under the Internet of Intelligent Vehicles. In 2021 IEEE International Workshop on Metrology for Industry 4.0 & IoT (MetroInd4. 0&IoT). IEEE, 642–647.Google Scholar
- Sajid Anwar, Kyuyeon Hwang, and Wonyong Sung. 2017. Structured pruning of deep convolutional neural networks. ACM Journal on Emerging Technologies in Computing Systems (JETC) 13, 3(2017), 1–18.Google ScholarDigital Library
- Apache TVM 2020. Apache TVM. https://tvm.apache.org/ Retrieved January 11, 2022 fromGoogle Scholar
- Apple 2022. HomePod mini. https://www.apple.com/homepod-mini/ Retrieved January 8, 2022 fromGoogle Scholar
- Arducam 2021. Arducam Pico4ML TinyML Dev Kit. https://www.arducam.com/docs/pico/arducam-pico4mltinymldevkit/ Retrieved December 30, 2021 fromGoogle Scholar
- ArduCam 2022. ArduCam 0.3MP: OV7675. https://www.arducam.com/products/camera-breakout-board/0-3mp-ov7675/ Retrieved January 8, 2022 fromGoogle Scholar
- Arduino 2021. Arduino Nano 33 BLE Sense. https://store-usa.arduino.cc/products/arduino-nano-33-ble-sense Retrieved December 30, 2021 fromGoogle Scholar
- Arduino 2021. Portenta H7. https://www.arduino.cc/pro/hardware/product/portenta-h7 Retrieved December 30, 2021 fromGoogle Scholar
- Arduino 2022. Arduino. https://www.arduino.cc/ Retrieved January 8, 2022 fromGoogle Scholar
- Arduino Project Hub 2019. Cough Detection with TinyML on Arduino. https://create.arduino.cc/projecthub/edge-impulse/cough-detection-with-tinyml-on-arduino-417f37 Retrieved December 19, 2021 fromGoogle Scholar
- Arduino Project Hub 2021. TinyML-Language Detector-Based on Edge Impulse and Arduino © MIT. https://create.arduino.cc/projecthub/enzo2/tinyml-language-detector-based-on-edge-impulse-arduino-f5cfa8?ref=part&ref_id=107215&offset=0 Retrieved January 11, 2022 fromGoogle Scholar
- Arduino Project Hub 2021. TinyML: Speech Commands Detection. https://create.arduino.cc/projecthub/ugotan/tinyml-speech-commands-detection-a3b51b?ref=part&ref_id=107215&offset=1 Retrieved January 11, 2022 fromGoogle Scholar
- Arm 2022. Arm. https://www.arm.com/ Retrieved January 8, 2022 fromGoogle Scholar
- Aruba Networks 2019. Secure Wi-Fi For Healthcare Applications. https://www.arubanetworks.com/assets/wp/WP Healthcare WLAN.pdf Retrieved October 23, 2021 fromGoogle Scholar
- Shaojie Bai, J Zico Kolter, and Vladlen Koltun. 2019. Deep equilibrium models. arXiv preprint arXiv:1909.01377(2019).Google Scholar
- Colby Banbury, Chuteng Zhou, Igor Fedorov, Ramon Matas, Urmish Thakker, Dibakar Gope, Vijay Janapa Reddi, Matthew Mattina, and Paul Whatmough. 2021. Micronets: Neural network architectures for deploying tinyml applications on commodity microcontrollers. Proceedings of Machine Learning and Systems 3 (2021).Google Scholar
- Colby R Banbury, Vijay Janapa Reddi, Max Lam, William Fu, Amin Fazel, Jeremy Holleman, Xinyuan Huang, Robert Hurtado, David Kanter, Anton Lokhmotov, et al. 2020. Benchmarking TinyML systems: Challenges and direction. arXiv preprint arXiv:2003.04821(2020).Google Scholar
- Mandrita Banerjee, Junghee Lee, and Kim-Kwang Raymond Choo. 2018. A blockchain future for internet of things security: a position paper. Digital Communications and Networks 4, 3 (Aug. 2018), 149–160. https://doi.org/10.1016/j.dcan.2017.10.006Google ScholarCross Ref
- Ron Banner, Yury Nahshan, Elad Hoffer, and Daniel Soudry. 2018. Aciq: Analytical clipping for integer quantization of neural networks. (2018).Google Scholar
- Paul Palomero Bernardo, Christoph Gerum, Adrian Frischknecht, Konstantin Lübeck, and Oliver Bringmann. 2020. Ultratrail: A configurable ultralow-power tc-resnet ai accelerator for efficient keyword spotting. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 39, 11(2020), 4240–4251.Google ScholarCross Ref
- Sizhen Bian and Paul Lukowicz. 2021. Capacitive Sensing Based On-board Hand Gesture Recognition with TinyML. In Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable Computers. 4–5.Google Scholar
- Oliver Bringmann, Wolfgang Ecker, Ingo Feldner, Adrian Frischknecht, Christoph Gerum, Timo Hämäläinen, Muhammad Abdullah Hanif, Michael J Klaiber, Daniel Mueller-Gritschneder, Paul Palomero Bernardo, et al. 2021. Automated HW/SW Co-design for Edge AI: State, Challenges and Steps Ahead: Special Session Paper. In 2021 International Conference on Hardware/Software Codesign and System Synthesis (CODES+ ISSS). IEEE, 11–20.Google Scholar
- Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165(2020).Google Scholar
- Alessio Burrello, Angelo Garofalo, Nazareno Bruschi, Giuseppe Tagliavini, Davide Rossi, and Francesco Conti. 2021. Dory: Automatic end-to-end deployment of real-world dnns on low-cost iot mcus. IEEE Trans. Comput. (2021).Google Scholar
- Paul M Cabacungan, Carlos M Oppus, Nerissa G Cabacungan, John Paul A Mamaradlo, Paul Ryan A Santiago, Neil Angelo M Mercado, E Vincent S Faustino, and Gregory L Tangonan. 2021. Design and Development of A-vent: A Low-Cost Ventilator with Cost-Effective Mobile Cloud Caching and Embedded Machine Learning. In 2021 IEEE Region 10 Symposium (TENSYMP). IEEE, 1–8.Google Scholar
- Léopold Cambier, Anahita Bhiwandiwalla, Ting Gong, Mehran Nekuii, Oguz H Elibol, and Hanlin Tang. 2020. Shifted and squeezed 8-bit floating point format for low-precision training of deep neural networks. arXiv preprint arXiv:2001.05674(2020).Google Scholar
- Erick Cantú-Paz. 2003. Pruning neural networks with distribution estimation algorithms. In Genetic and Evolutionary Computation Conference. Springer, 790–800.Google ScholarCross Ref
- Varun Chandola, Arindam Banerjee, and Vipin Kumar. 2009. Anomaly detection: A survey. ACM computing surveys (CSUR) 41, 3 (2009), 1–58.Google ScholarDigital Library
- Wenlin Chen, James Wilson, Stephen Tyree, Kilian Weinberger, and Yixin Chen. 2015. Compressing neural networks with the hashing trick. In International conference on machine learning. PMLR, 2285–2294.Google Scholar
- Yu-Hsin Chen, Tushar Krishna, Joel S Emer, and Vivienne Sze. 2016. Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks. IEEE journal of solid-state circuits 52, 1 (2016), 127–138.Google ScholarDigital Library
- Yu-Hsin Chen, Tien-Ju Yang, Joel Emer, and Vivienne Sze. 2019. Eyeriss v2: A flexible accelerator for emerging deep neural networks on mobile devices. IEEE Journal on Emerging and Selected Topics in Circuits and Systems 9, 2(2019), 292–308.Google ScholarCross Ref
- Ping Chi, Shuangchen Li, Cong Xu, Tao Zhang, Jishen Zhao, Yongpan Liu, Yu Wang, and Yuan Xie. 2016. Prime: A novel processing-in-memory architecture for neural network computation in reram-based main memory. ACM SIGARCH Computer Architecture News 44, 3 (2016), 27–39.Google ScholarDigital Library
- Konstantinos Christidis and Michael Devetsikiotis. 2016. Blockchains and Smart Contracts for the Internet of Things. IEEE Access 4(2016), 2292–2303. https://doi.org/10.1109/ACCESS.2016.2566339Google ScholarCross Ref
- Codelabs 2020. AI Speech Recognition with TensorFlow Lite for Microcontrollers and SparkFun Edge. https://codelabs.developers.google.com/codelabs/sparkfun-tensorflow/#0 Retrieved January 3, 2022 fromGoogle Scholar
- Coursera 2022. Computer Vision with Embedded Machine Learning. https://www.coursera.org/learn/computer-vision-with-em bedded-machine-learning Retrieved January 3, 2022 fromGoogle Scholar
- Coursera 2022. Coursera. https://www.coursera.org/ Retrieved January 8, 2022 fromGoogle Scholar
- Coursera 2022. Introduction to Embedded Machine Learning. https://www.coursera.org/learn/introduction-to-embedde d-machine-learning Retrieved January 3, 2022 fromGoogle Scholar
- Giulia Crocioni, Giambattista Gruosso, Danilo Pau, Davide Denaro, Luigi Zambrano, and Giuseppe Di Giore. 2021. Characterization of Neural Networks Automatically Mapped on Automotive-grade Microcontrollers. arXiv preprint arXiv:2103.00201(2021).Google Scholar
- Raj Dabre and Atsushi Fujita. 2019. Recurrent stacking of layers for compact neural machine translation models. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 6292–6299.Google ScholarDigital Library
- Bin Dai, Chen Zhu, Baining Guo, and David Wipf. 2018. Compressing neural networks using the variational information bottleneck. In International Conference on Machine Learning. PMLR, 1135–1144.Google Scholar
- DaleGia 2020. The Hacky Super Loop Arduino Nano 33 BLE Sense Example You Have Been Waiting For. https://dalegi.com/2020/06/09/the-hacky-super-loop-arduino-nano-33-ble-sense-example-you-have-been-waiting-for/ Retrieved January 11, 2022 fromGoogle Scholar
- Robert David, Jared Duke, Advait Jain, Vijay Janapa Reddi, Nat Jeffries, Jian Li, Nick Kreeger, Ian Nappier, Meghna Natraj, Tiezhen Wang, et al. 2021. TensorFlow Lite Micro: Embedded Machine Learning for TinyML Systems. Proceedings of Machine Learning and Systems 3 (2021).Google Scholar
- Miguel de Prado, Manuele Rusci, Alessandro Capotondi, Romain Donze, Luca Benini, and Nuria Pazos. 2021. Robustifying the Deployment of tinyML Models for Autonomous mini-vehicles. Sensors 21, 4 (2021), 1339.Google ScholarCross Ref
- Matthias Delange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Greg Slabaugh, and Tinne Tuytelaars. 2021. A continual learning survey: Defying forgetting in classification tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence (2021).Google Scholar
- Misha Denil, Babak Shakibi, Laurent Dinh, Marc’Aurelio Ranzato, and Nando De Freitas. 2013. Predicting parameters in deep learning. arXiv preprint arXiv:1306.0543(2013).Google Scholar
- Tim Dettmers. 2015. 8-bit approximations for parallelism in deep learning. arXiv preprint arXiv:1511.04561(2015).Google Scholar
- Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805(2018).Google Scholar
- Hiroshi Doyu, Roberto Morabito, and Martina Brachmann. 2021. A tinymlaas ecosystem for machine learning in iot: Overview and research challenges. In 2021 International Symposium on VLSI Design, Automation and Test (VLSI-DAT). IEEE, 1–5.Google ScholarCross Ref
- Bardienus P Duisterhof, Srivatsan Krishnan, Jonathan J Cruz, Colby R Banbury, William Fu, Aleksandra Faust, Guido CHE de Croon, and Vijay Janapa Reddi. 2019. Learning to seek: Autonomous source seeking with deep reinforcement learning onboard a nano drone microcontroller. arXiv preprint arXiv:1909.11236(2019).Google Scholar
- Abir Dutta and Shri Kant. 2021. Implementation of Cyber Threat Intelligence Platform on Internet of Things (IoT) using TinyML Approach for Deceiving Cyber Invasion. In 2021 International Conference on Electrical, Computer, Communications and Mechatronics Engineering (ICECCME). IEEE, 1–6.Google ScholarCross Ref
- Lachit Dutta and Swapna Bharali. 2021. TinyML Meets IoT: A Comprehensive Survey. Internet of Things 16(2021), 100461.Google ScholarCross Ref
- Edge Impulse 2022. Announcing Intro to Embedded Machine Learning on Coursera. https://www.edgeimpulse.com/blog/announcing-intro-to-embedded-machine-learning-on-coursera Retrieved January 8, 2022 fromGoogle Scholar
- Edge Impulse 2022. Edge Impulse. https://www.edgeimpulse.com/ Retrieved January 8, 2022 fromGoogle Scholar
- edX 2022. edX. https://www.edx.org Retrieved January 8, 2022 fromGoogle Scholar
- edX 2022. The Future of ML is Tiny and Bright. https://www.edx.org/professional-certificate/harvardx- tiny-machine-learning Retrieved January 3, 2022 fromGoogle Scholar
- EEworldONLINE 2019. Part 1: Microcontrollers and Microprocessors Continue Rapid Market Growth. https://www.eeworldonline.com/part-1-microcontrollers-and-microprocessors-continue-rapid-market-growth/ Retrieved January 8, 2022 fromGoogle Scholar
- David Eigen, Jason Rolfe, Rob Fergus, and Yann LeCun. 2013. Understanding deep architectures using a recursive convolutional network. arXiv preprint arXiv:1312.1847(2013).Google Scholar
- Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. 2019. Neural architecture search: A survey. The Journal of Machine Learning Research 20, 1 (2019), 1997–2017.Google ScholarDigital Library
- A.P. Engelbrecht. 2001. A new pruning heuristic based on variance analysis of sensitivity information. IEEE Transactions on Neural Networks 12, 6 (2001), 1386–1399. https://doi.org/10.1109/72.963775Google ScholarDigital Library
- Espressif 2021. ESP32-DevKitC. https://www.espressif.com/en/products/devkits/esp32-devkitc/overview Retrieved December 30, 2021 fromGoogle Scholar
- Farah Fahim, Benjamin Hawks, Christian Herwig, James Hirschauer, Sergo Jindariani, Nhan Tran, Luca P Carloni, Giuseppe Di Guglielmo, Philip Harris, Jeffrey Krupa, et al. 2021. hls4ml: An Open-Source Codesign Workflow to Empower Scientific Low-Power Machine Learning Devices. arXiv preprint arXiv:2103.05579(2021).Google Scholar
- Angela Fan, Pierre Stock, Benjamin Graham, Edouard Grave, Rémi Gribonval, Herve Jegou, and Armand Joulin. 2020. Training with quantization noise for extreme model compression. arXiv preprint arXiv:2004.07320(2020).Google Scholar
- Igor Fedorov, Ryan P Adams, Matthew Mattina, and Paul N Whatmough. 2019. Sparse: Sparse architecture search for cnns on resource-constrained microcontrollers. arXiv preprint arXiv:1905.12107(2019).Google Scholar
- Igor Fedorov, Marko Stamenovic, Carl Jensen, Li-Chia Yang, Ari Mandell, Yiming Gan, Matthew Mattina, and Paul N Whatmough. 2020. TinyLSTMs: Efficient neural speech enhancement for hearing aids. arXiv preprint arXiv:2005.11138(2020).Google Scholar
- Eric Flamand, Davide Rossi, Francesco Conti, Igor Loi, Antonio Pullini, Florent Rotenberg, and Luca Benini. 2018. GAP-8: A RISC-V SoC for AI at the Edge of the IoT. In 2018 IEEE 29th International Conference on Application-specific Systems, Architectures and Processors (ASAP). IEEE, 1–4.Google ScholarCross Ref
- Jonathan Frankle and Michael Carbin. 2018. The lottery ticket hypothesis: Finding sparse, trainable neural networks. arXiv preprint arXiv:1803.03635(2018).Google Scholar
- Daichi Fujiki, Xiaowei Wang, Arun Subramaniyan, and Reetuparna Das. 2021. In-/Near-Memory Computing. Synthesis Lectures on Computer Architecture 16, 2 (2021), 1–140.Google ScholarCross Ref
- Sukhpal Singh Gill, Minxian Xu, Carlo Ottaviani, Panos Patros, Rami Bahsoon, Arash Shaghaghi, Muhammed Golec, Vlado Stankovski, Huaming Wu, Ajith Abraham, Manmeet Singh, Harshit Mehta, Soumya K. Ghosh, Thar Baker, Ajith Kumar Parlikad, Hanan Lutfiyya, Salil S. Kanhere, Rizos Sakellariou, Schahram Dustdar, Omer Rana, Ivona Brandic, and Steve Uhlig. 2022. AI for next generation computing: Emerging trends and future directions. Internet of Things 19(Aug. 2022), 100514. https://doi.org/10.1016/j.iot.2022.100514Google ScholarCross Ref
- Marco Giordano, Philipp Mayer, and Michele Magno. 2020. A Battery-Free Long-Range Wireless Smart Camera for Face Detection. In Proceedings of the 8th International Workshop on Energy Harvesting and Energy-Neutral Sensing Systems. 29–35.Google ScholarDigital Library
- GitHub 2020. TinyML Example: Anomaly Detection. https://github.com/ShawnHymel/tinyml-example-anomaly-detection Retrieved January 3, 2022 fromGoogle Scholar
- GitHub 2021. CurrentSense-TinyML. https://github.com/Santandersecurityresearch/CurrentSense-TinyML Retrieved January 3, 2022 fromGoogle Scholar
- GitHub 2021. Embedded Learning Library. https://github.com/microsoft/ELL Retrieved January 8, 2022 fromGoogle Scholar
- GitHub 2021. TinyML Study Group. https://github.com/scaledown-team/study-group Retrieved January 3, 2022 fromGoogle Scholar
- GitHub 2021. uTensor. https://github.com/uTensor/uTensor Retrieved January 8, 2022 fromGoogle Scholar
- Google 2022. Google. https://www.google.com/ Retrieved January 8, 2022 fromGoogle Scholar
- Google 2022. Google Nest. https://support.google.com/googlenest/answer/7130274?hl=en Retrieved January 8, 2022 fromGoogle Scholar
- Qiushan Guo, Zhipeng Yu, Yichao Wu, Ding Liang, Haoyu Qin, and Junjie Yan. 2019. Dynamic recursive neural network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5147–5156.Google ScholarCross Ref
- Suyog Gupta, Ankur Agrawal, Kailash Gopalakrishnan, and Pritish Narayanan. 2015. Deep learning with limited numerical precision. In International conference on machine learning. PMLR, 1737–1746.Google Scholar
- Philipp Gysel, Jon Pimentel, Mohammad Motamedi, and Soheil Ghiasi. 2018. Ristretto: A framework for empirical study of resource-efficient inference in convolutional neural networks. IEEE transactions on neural networks and learning systems 29, 11(2018), 5784–5789.Google Scholar
- Hackster.io 2020. Easy TinyML on ESP32 and Arduino. https://www.hackster.io/news/easy-tinyml-on-esp32-and-arduino-a9dbc509f26c Retrieved January 3, 2022 fromGoogle Scholar
- Hackster.io 2020. Handwriting Recognition. https://www.hackster.io/naveenbskumar/handwriting-recognition-7583e3 Retrieved January 3, 2022 fromGoogle Scholar
- Hackster.io 2021. TapLock - A bike lock with machine learning. https://www.hackster.io/taplock/taplock-a-bike-lock-with-machine-learning-85641c Retrieved January 3, 2022 fromGoogle Scholar
- Masafumi Hagiwara. 1993. Removal of hidden units and weights for back propagation networks. In Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan), Vol. 1. IEEE, 351–354.Google ScholarCross Ref
- Rahman Hajian, S ZakeriKia, SH Erfani, and M Mirabi. 2020. SHAPARAK: Scalable healthcare authentication protocol with attack-resilience and anonymous key-agreement. Computer Networks 183(2020), 107567.Google ScholarCross Ref
- Hong-Gui Han and Jun-Fei Qiao. 2013. A structure optimisation algorithm for feedforward neural network construction. Neurocomputing 99(2013), 347–357.Google ScholarDigital Library
- Harvard 2022. Harvard University. https://www.harvard.edu/ Retrieved January 8, 2022 fromGoogle Scholar
- Babak Hassibi, David G Stork, and Gregory J Wolff. 1993. Optimal brain surgeon and general network pruning. In IEEE international conference on neural networks. IEEE, 293–299.Google ScholarCross Ref
- Yihui He, Ji Lin, Zhijian Liu, Hanrui Wang, Li-Jia Li, and Song Han. 2018. Amc: Automl for model compression and acceleration on mobile devices. In Proceedings of the European conference on computer vision (ECCV). 784–800.Google ScholarDigital Library
- Lennart Heim, Andreas Biri, Zhongnan Qu, and Lothar Thiele. 2021. Measuring what Really Matters: Optimizing Neural Networks for TinyML. arXiv preprint arXiv:2104.10645(2021).Google Scholar
- Himax 2021. Himax WE-I. https://www.himax.com.tw/products/intelligent-sensing/always-on-smart-sensing/ Retrieved December 30, 2021 fromGoogle Scholar
- Lu Hou, Quanming Yao, and James T Kwok. 2016. Loss-aware binarization of deep networks. arXiv preprint arXiv:1611.01600(2016).Google Scholar
- Jie Hu, Li Shen, and Gang Sun. 2018. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7132–7141.Google ScholarCross Ref
- Yiming Hu, Siyang Sun, Jianquan Li, Xingang Wang, and Qingyi Gu. 2018. A novel channel pruning method for deep neural network compression. arXiv preprint arXiv:1805.11394(2018).Google Scholar
- IC Insights 2020. MCUs Expected to Make Modest Comeback After 2020 Drop. https://www.icinsights.com/news/bulletins/MCUs-Expected-To-Make-Modest-Comeback-After-2020-Drop--/ Retrieved January 8, 2022 fromGoogle Scholar
- Gian Marco Iodice. 2022. TinyML Cookbook combine artificial intelligence and ultra-low-power embedded devices to make... the world smarter.Packt Publishing Limited, S.l.OCLC: 1309923814.Google Scholar
- Rahul Jain, Vijay Bhaskar Semwal, and Praveen Kaushik. 2021. Deep ensemble learning approach for lower extremity activities recognition using wearable sensors. Expert Systems (2021), e12743.Google Scholar
- Nikhil Jangamreddy. 2019. A Survey on Specialised Hardware for Machine Learning. (2019).Google Scholar
- Weiwen Jiang, Lei Yang, Edwin Hsing-Mean Sha, Qingfeng Zhuge, Shouzhen Gu, Sakyasingha Dasgupta, Yiyu Shi, and Jingtong Hu. 2020. Hardware/software co-exploration of neural architectures. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 39, 12(2020), 4805–4815.Google ScholarCross Ref
- Raj Kamal, Aabha Jain, and Manojkumar Vilasrao Deshpande. 2022. Architectural Design for Inspection of Machine Objects Using Small DNNs as TinyML for Machine Vision of Defects and Faults in the Manufacturing Processes. In Smart Systems: Innovations in Computing. Springer, 377–387.Google Scholar
- Ehud D Karnin. 1990. A simple procedure for pruning back-propagation trained neural networks. IEEE transactions on neural networks 1, 2 (1990), 239–242.Google ScholarDigital Library
- Wazir Zada Khan, Ejaz Ahmed, Saqib Hakak, Ibrar Yaqoob, and Arif Ahmed. 2019. Edge computing: A survey. Future Generation Computer Systems 97 (2019), 219–235.Google ScholarDigital Library
- Jiwon Kim, Jung Kwon Lee, and Kyoung Mu Lee. 2016. Deeply-recursive convolutional network for image super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1637–1645.Google ScholarCross Ref
- Okan Köpüklü, Maryam Babaee, Stefan Hörmann, and Gerhard Rigoll. 2019. Convolutional neural networks with layer reuse. In 2019 IEEE International Conference on Image Processing (ICIP). IEEE, 345–349.Google ScholarCross Ref
- Raghuraman Krishnamoorthi. 2018. Quantizing deep convolutional networks for efficient inference: A whitepaper. arXiv preprint arXiv:1806.08342(2018).Google Scholar
- Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25 (2012), 1097–1105.Google Scholar
- Nir Kshetri. 2017. Blockchain’s roles in strengthening cybersecurity and protecting privacy. Telecommunications Policy 41, 10 (Nov. 2017), 1027–1038. https://doi.org/10.1016/j.telpol.2017.09.003Google ScholarCross Ref
- Jisu Kwon and Daejin Park. 2021. Hardware/Software Co-Design for TinyML Voice-Recognition Application on Resource Frugal Edge Devices. Applied Sciences 11, 22 (2021), 11073.Google ScholarCross Ref
- Shashikant Vitthalrao Lahade, Srikanth Namuduri, Himanshu Upadhyay, and Shekhar Bhansali. 2020. Alcohol Sensor Calibration on the Edge Using Tiny Machine Learning (Tiny-ML) Hardware. In ECS Meeting Abstracts. IOP Publishing, 1848.Google Scholar
- Liangzhen Lai, Naveen Suda, and Vikas Chandra. 2018. Cmsis-nn: Efficient neural network kernels for arm cortex-m cpus. arXiv preprint arXiv:1801.06601(2018).Google Scholar
- Maximilian Lam, Sharad Chitlangia, Srivatsan Krishnan, Zishen Wan, Gabriel Barth-Maron, Aleksandra Faust, and Vijay Janapa Reddi. 2019. Quantized reinforcement learning (quarl). arXiv preprint arXiv:1910.01055(2019).Google Scholar
- Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019. Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942(2019).Google Scholar
- Yann LeCun, John S Denker, and Sara A Solla. 1990. Optimal brain damage. In Advances in neural information processing systems. 598–605.Google Scholar
- Namhoon Lee, Thalaiyasingam Ajanthan, and Philip HS Torr. 2018. Snip: Single-shot network pruning based on connection sensitivity. arXiv preprint arXiv:1810.02340(2018).Google Scholar
- Davide Nadalini Alessandro Capotondi Francesco Conti Leonardo Ravaglia, Manuele Rusci and Luca Benini. 2021. A TinyML Platform for On-Device Continual Learning With Quantized Latent Replays. IEEE Journal on Emerging and Selected Topics in Circuits and Systems (2021).Google Scholar
- Jiajun Li, Guihai Yan, Wenyan Lu, Shuhao Jiang, Shijun Gong, Jingya Wu, and Xiaowei Li. 2018. SmartShuttle: Optimizing off-chip memory accesses for deep learning accelerators. In 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE). IEEE, 343–348.Google Scholar
- Shaohui Lin, Rongrong Ji, Yuchao Li, Yongjian Wu, Feiyue Huang, and Baochang Zhang. 2018. Accelerating Convolutional Networks via Global & Dynamic Filter Pruning.. In IJCAI, Vol. 2. 8.Google Scholar
- Shaohui Lin, Rongrong Ji, Chenqian Yan, Baochang Zhang, Liujuan Cao, Qixiang Ye, Feiyue Huang, and David Doermann. 2019. Towards optimal structured cnn pruning via generative adversarial learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2790–2799.Google ScholarCross Ref
- Hanxiao Liu, Karen Simonyan, and Yiming Yang. 2018. Darts: Differentiable architecture search. arXiv preprint arXiv:1806.09055(2018).Google Scholar
- Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692(2019).Google Scholar
- Zhuang Liu, Mingjie Sun, Tinghui Zhou, Gao Huang, and Trevor Darrell. 2018. Rethinking the value of network pruning. arXiv preprint arXiv:1810.05270(2018).Google Scholar
- Mansoureh Lord. 2021. TinyML, Anomaly Detection. Ph.D. Dissertation. California State University, Northridge.Google Scholar
- Christos Louizos, Karen Ullrich, and Max Welling. 2017. Bayesian compression for deep learning. arXiv preprint arXiv:1705.08665(2017).Google Scholar
- Tom H Luan, Longxiang Gao, Zhi Li, Yang Xiang, Guiyi Wei, and Limin Sun. 2015. Fog computing: Focusing on mobile users at the edge. arXiv preprint arXiv:1502.01815(2015).Google Scholar
- Zheda Mai, Ruiwen Li, Jihwan Jeong, David Quispe, Hyunwoo Kim, and Scott Sanner. 2022. Online continual learning in image classification: An empirical survey. Neurocomputing 469(2022), 28–51.Google ScholarDigital Library
- Arun Mallya and Svetlana Lazebnik. 2018. Piggyback: Adding multiple tasks to a single, fixed network by learning to mask. arXiv preprint arXiv:1801.06519 6, 8 (2018).Google Scholar
- Yuyi Mao, Changsheng You, Jun Zhang, Kaibin Huang, and Khaled B Letaief. 2017. A survey on mobile edge computing: The communication perspective. IEEE Communications Surveys & Tutorials 19, 4 (2017), 2322–2358.Google ScholarCross Ref
- Muhammad Ali Mazidi, Janice Gillispie Mazidi, and Rolin D McKinlay. 2006. The 8051 microcontroller and embedded systems: using Assembly and C. Vol. 626. Pearson/Prentice Hall.Google Scholar
- John T McCoy and Lidia Auret. 2019. Machine learning applications in minerals processing: A review. Minerals Engineering 132(2019), 95–109.Google ScholarCross Ref
- Marci Meingast, Tanya Roosta, and Shankar Sastry. 2006. Security and privacy issues with health care information technology. In 2006 International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE, 5453–5458.Google ScholarCross Ref
- Tal Melamed. 2018. An active man-in-the-middle attack on bluetooth smart devices. Safety and Security Studies (2018) 15 (2018), 2018.Google Scholar
- Naveen Mellempudi, Sudarshan Srinivasan, Dipankar Das, and Bharat Kaul. 2019. Mixed precision training with 8-bit floating point. arXiv preprint arXiv:1905.12334(2019).Google Scholar
- Paul Merolla, Rathinakumar Appuswamy, John Arthur, Steve K Esser, and Dharmendra Modha. 2016. Deep neural networks are robust to weight binarization and other non-linear distortions. arXiv preprint arXiv:1606.01981(2016).Google Scholar
- Szymon Migacz. 2017. 8-bit inference with tensorrt. In GPU technology conference, Vol. 2. 5.Google Scholar
- MLCommons 2021. MLCommons aims to accelerate machine learning innovation to benefit everyone. https://www.tensorflow.org/lite Retrieved January 11, 2022 fromGoogle Scholar
- Dmitry Molchanov, Arsenii Ashukha, and Dmitry Vetrov. 2017. Variational Dropout Sparsifies Deep Neural Networks. In Proceedings of the 34th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 70), Doina Precup and Yee Whye Teh (Eds.). PMLR, 2498–2507. https://proceedings.mlr.press/v70/molchanov17a.htmlGoogle Scholar
- Kathryn Montgomery, Jeff Chester, and Katharina Kopp. 2018. Health wearables: ensuring fairness, preventing discrimination, and promoting equity in an emerging Internet-of-Things environment. Journal of Information Policy 8 (2018), 34–77.Google ScholarCross Ref
- Michael C Mozer and Paul Smolensky. 1989. Skeletonization: A technique for trimming the fat from a network via relevance assessment. In Advances in neural information processing systems. 107–115.Google Scholar
- Onur Mutlu, Saugata Ghose, Juan Gómez-Luna, and Rachata Ausavarungnirun. 2020. A modern primer on processing in memory. arXiv preprint arXiv:2012.03112(2020).Google Scholar
- Pramod L Narasimha, Walter H Delashmit, Michael T Manry, Jiang Li, and Francisco Maldonado. 2008. An integrated growing-pruning method for feedforward network training. Neurocomputing 71, 13-15 (2008), 2831–2847.Google ScholarDigital Library
- James O’ Neill. 2020. An overview of neural network compression. arXiv preprint arXiv:2006.03669(2020).Google Scholar
- Justin Nguyen, Reese Grimsley, and Bob Iannucci. 2021. TrafficNNode: Low Power Vehicle Sensing Platform for Smart Cities. In 2021 IEEE International Conference on Smart Internet of Things (SmartIoT). IEEE, 278–282.Google Scholar
- Steven J Nowlan and Geoffrey E Hinton. 1992. Simplifying neural networks by soft weight-sharing. Neural computation 4, 4 (1992), 473–493.Google Scholar
- Asaf Noy, Niv Nayman, Tal Ridnik, Nadav Zamir, Sivan Doveh, Itamar Friedman, Raja Giryes, and Lihi Zelnik. 2020. Asap: Architecture search, anneal and prune. In International Conference on Artificial Intelligence and Statistics. PMLR, 493–503.Google Scholar
- NVIDIA 2021. Jetson Nano Developer Kit. https://developer.nvidia.com/embedded/jetson-nano-developer-kit Retrieved December 30, 2021 fromGoogle Scholar
- NXP 2022. eIQ® ML Software Development Environment. https://www.nxp.com/design/software/development-software/eiq-ml-development-environment:EIQ Retrieved January 11, 2022 fromGoogle Scholar
- Samson Otieno Ooko, Didacienne Mukanyiligira, Jean Pierre Munyampundu, and Jimmy Nsenga. 2021. Synthetic Exhaled Breath Data-Based Edge AI Model for the Prediction of Chronic Obstructive Pulmonary Disease. In 2021 International Conference on Computing and Communications Applications and Technologies (I3CAT). IEEE, 1–6.Google Scholar
- Charalampos Orfanidis, Rayén Bel Haj Hassen, Armando Kwiek, Xenofon Fafoutis, and Martin Jacobsson. 2021. A Discreet Wearable Long-Range Emergency System Based on Embedded Machine Learning. In 2021 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops). IEEE, 182–187.Google ScholarCross Ref
- Francesco Paissan, Alberto Ancilotto, and Elisabetta Farella. 2021. PhiNets: a scalable backbone for low-power AI at the edge. arXiv preprint arXiv:2110.00337(2021).Google Scholar
- Zhengyuan Pang, Lifeng Sun, Zhi Wang, Erfang Tian, and Shiqiang Yang. 2015. A survey of cloudlet based mobile computing. In 2015 International Conference on Cloud Computing and Big Data (CCBD). IEEE, 268–275.Google ScholarDigital Library
- Eunhyeok Park, Sungjoo Yoo, and Peter Vajda. 2018. Value-aware quantization for training and inference of neural networks. In Proceedings of the European Conference on Computer Vision (ECCV). 580–595.Google ScholarDigital Library
- Niki Parmar, Ashish Vaswani, Jakob Uszkoreit, Lukasz Kaiser, Noam Shazeer, Alexander Ku, and Dustin Tran. 2018. Image transformer. In International Conference on Machine Learning. PMLR, 4055–4064.Google Scholar
- Aditya Jyoti Paul, Puranjay Mohan, and Stuti Sehgal. 2020. Rethinking generalization in american sign language prediction for edge devices with extremely low memory footprint. In 2020 IEEE Recent Advances in Intelligent Computational Systems (RAICS). IEEE, 147–152.Google Scholar
- Hongwu Peng, Shaoyi Huang, Tong Geng, Ang Li, Weiwen Jiang, Hang Liu, Shusen Wang, and Caiwen Ding. 2021. Accelerating Transformer-based Deep Learning Models on FPGAs using Column Balanced Block Pruning. In 2021 22nd International Symposium on Quality Electronic Design (ISQED). IEEE, 142–148.Google ScholarCross Ref
- Andrew Perrin. 2019. Digital gap between rural and nonrural America persists. Pew Research Center (2019).Google Scholar
- Bryan A Plummer, Nikoli Dryden, Julius Frost, Torsten Hoefler, and Kate Saenko. 2020. Shapeshifter networks: Cross-layer parameter sharing for scalable and effective deep learning. arXiv e-prints (2020), arXiv–2006.Google Scholar
- Panjie Qi, Edwin Hsing-Mean Sha, Qingfeng Zhuge, Hongwu Peng, Shaoyi Huang, Zhenglun Kong, Yuhong Song, and Bingbing Li. 2021. Accelerating Framework of Transformer by Hardware Design and Model Compression Co-Optimization. In 2021 IEEE/ACM International Conference On Computer Aided Design (ICCAD). IEEE, 1–9.Google ScholarDigital Library
- Sajay Rai, Philip Chukwuma, and Richard Cozart. 2016. Security and Auditing of Smart Devices: Managing Proliferation of Confidential Data on Corporate and BYOD Devices. Auerbach Publications.Google Scholar
- Raspberry Pi 2021. Raspberry Pi. https://www.raspberrypi.org/ Retrieved January 8, 2022 fromGoogle Scholar
- Raspberry Pi 2021. Raspberry Pi 4. https://www.raspberrypi.com/products/raspberry-pi-4-model-b/ Retrieved December 30, 2021 fromGoogle Scholar
- Raspberry Pi 2021. Raspberry Pi Pico. https://www.raspberrypi.com/products/raspberry-pi-pico/ Retrieved December 30, 2021 fromGoogle Scholar
- Leonardo Ravaglia, Manuele Rusci, Davide Nadalini, Alessandro Capotondi, Francesco Conti, and Luca Benini. 2021. A TinyML Platform for On-Device Continual Learning With Quantized Latent Replays. IEEE Journal on Emerging and Selected Topics in Circuits and Systems 11, 4(2021), 789–802.Google ScholarCross Ref
- Wamiq Raza, Anas Osman, Francesco Ferrini, and Francesco De Natale. 2021. Energy-Efficient Inference on the Edge Exploiting TinyML Capabilities for UAVs. Drones 5, 4 (2021), 127.Google ScholarCross Ref
- Vijay Janapa Reddi, Brian Plancher, Susan Kennedy, Laurence Moroney, Pete Warden, Anant Agarwal, Colby Banbury, Massimo Banzi, Matthew Bennett, Benjamin Brown, et al. 2021. Widening Access to Applied Machine Learning with TinyML. arXiv preprint arXiv:2106.04008(2021).Google Scholar
- Russell Reed. 1993. Pruning algorithms-a survey. IEEE transactions on Neural Networks 4, 5 (1993), 740–747.Google ScholarDigital Library
- Haoyu Ren, Darko Anicic, and Thomas Runkler. 2021. TinyOL: TinyML with Online-Learning on Microcontrollers. arXiv preprint arXiv:2103.08295(2021).Google Scholar
- Francesco Restuccia and Tommaso Melodia. 2020. Deepwierl: Bringing deep reinforcement learning to the internet of self-adaptive things. In IEEE INFOCOM 2020-IEEE Conference on Computer Communications. IEEE, 844–853.Google ScholarDigital Library
- A Navaas Roshan, B Gokulapriyan, C Siddarth, and Priyanka Kokil. 2021. Adaptive Traffic Control With TinyML. In 2021 Sixth International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET). IEEE, 451–455.Google Scholar
- Firdose Saeik, Marios Avgeris, Dimitrios Spatharakis, Nina Santi, Dimitrios Dechouniotis, John Violos, Aris Leivadeas, Nikolaos Athanasopoulos, Nathalie Mitton, and Symeon Papavassiliou. 2021. Task offloading in Edge and Cloud Computing: A survey on mathematical, artificial intelligence and control theory solutions. Computer Networks 195(Aug. 2021), 108177. https://doi.org/10.1016/j.comnet.2021.108177Google ScholarCross Ref
- Ramon Sanchez-Iborra and Antonio F Skarmeta. 2020. Tinyml-enabled frugal smart objects: Challenges and opportunities. IEEE Circuits and Systems Magazine 20, 3 (2020), 4–18.Google ScholarCross Ref
- Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108(2019).Google Scholar
- Pedro Savarese and Michael Maire. 2019. Learning implicitly recurrent CNNs through parameter sharing. arXiv preprint arXiv:1902.09701(2019).Google Scholar
- Scaledown 2022. Scaledown. https://scaledown-team.github.io/ Retrieved January 11, 2022 fromGoogle Scholar
- Seedstudio 2021. Everything About TinyML – Basics, Courses, Projects & More! https://www.seeedstudio.com/blog/2021/06/14/everything-about-tinyml-basics-courses-projects-more/ Retrieved January 3, 2022 fromGoogle Scholar
- Seedstudio 2021. Wio Terminal - Hello World of AI, an Azure certified device to get started with IoT and TinyML. https://www.seeedstudio.com/wio-terminal-tinyml.html Retrieved January 3, 2022 fromGoogle Scholar
- Seeedstudio 2021. Seeeduino-XIAO. https://www.seeedstudio.com/Seeeduino-XIAO-Arduino-Microcontroller-SAMD21-Cortex-M0+-p-4426.html Retrieved December 30, 2021 fromGoogle Scholar
- Seeedstudio 2021. Wio Terminal. https://www.seeedstudio.com/Wio-Terminal-p-4509.html Retrieved December 30, 2021 fromGoogle Scholar
- Suranga Seneviratne, Yining Hu, Tham Nguyen, Guohao Lan, Sara Khalifa, Kanchana Thilakarathna, Mahbub Hassan, and Aruna Seneviratne. 2017. A survey of wearable devices and challenges. IEEE Communications Surveys & Tutorials 19, 4 (2017), 2573–2620.Google ScholarCross Ref
- Rudy Setiono and Wee Kheng Leow. 2000. Pruned neural networks for regression. In Pacific Rim International Conference on Artificial Intelligence. Springer, 500–509.Google ScholarCross Ref
- Muhammad Shafique, Theocharis Theocharides, Vijay Janapa Reddy, and Boris Murmann. 2021. TinyML: Current Progress, Research Challenges, and Future Roadmap. In 2021 58th ACM/IEEE Design Automation Conference (DAC). IEEE, 1303–1306.Google ScholarDigital Library
- Qinfeng Shi, James Petterson, Gideon Dror, John Langford, Alex Smola, and SVN Vishwanathan. 2009. Hash kernels for structured data.Journal of Machine Learning Research 10, 11 (2009).Google Scholar
- Weisong Shi, Jie Cao, Quan Zhang, Youhuizi Li, and Lanyu Xu. 2016. Edge computing: Vision and challenges. IEEE internet of things journal 3, 5 (2016), 637–646.Google Scholar
- Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. 2019. Megatron-lm: Training multi-billion parameter language models using model parallelism. arXiv preprint arXiv:1909.08053(2019).Google Scholar
- Ajay Shrestha and Ausif Mahmood. 2019. Review of Deep Learning Algorithms and Architectures. IEEE Access 7(2019), 53040–53065. https://doi.org/10.1109/ACCESS.2019.2912200Google ScholarCross Ref
- Shachar Siboni, Asaf Shabtai, Nils O Tippenhauer, Jemin Lee, and Yuval Elovici. 2016. Advanced security testbed framework for wearable IoT devices. ACM Transactions on Internet Technology (TOIT) 16, 4 (2016), 1–25.Google ScholarDigital Library
- Yuhong Song, Weiwen Jiang, Bingbing Li, Panjie Qi, Qingfeng Zhuge, Edwin Hsing-Mean Sha, Sakyasingha Dasgupta, Yiyu Shi, and Caiwen Ding. 2021. Dancing along Battery: Enabling Transformer with Run-time Reconfigurability on Mobile Devices. arXiv preprint arXiv:2102.06336(2021).Google Scholar
- Sony 2021. Spresense. https://developer.sony.com/develop/spresense/ Retrieved December 30, 2021 fromGoogle Scholar
- Stanislava Soro. 2021. TinyML for Ubiquitous Edge AI. arXiv preprint arXiv:2102.01255(2021).Google Scholar
- SparkFun 2021. SparkFun Edge Development Board - Apollo3 Blue. https://www.sparkfun.com/products/15170 Retrieved December 30, 2021 fromGoogle Scholar
- STM32L4 Discovery kit 2021. STM32L4 Discovery kit. https://www.st.com/en/evaluation-tools/b-l475e-iot01a.html#overview&secondary=st_all-features_sec-nav-tab Retrieved December 30, 2021 fromGoogle Scholar
- Pierre Stock, Armand Joulin, Rémi Gribonval, Benjamin Graham, and Hervé Jégou. 2019. And the bit goes down: Revisiting the quantization of neural networks. arXiv preprint arXiv:1907.05686(2019).Google Scholar
- Filip Svoboda, David Nunes, Milad Alizadeh, Russel Daries, Rui Luo, Akhil Mathur, Sourav Bhattacharya, Jorge Sa Silva, and Nicholas Donald Lane. 2020. Resource Efficient Deep Reinforcement Learning for Acutely Constrained TinyML Devices. In Research Symposium on Tiny Machine Learning.Google Scholar
- SYNTIANT 2019. The Speed and Power Advantage of a Purpose-Built Neural Compute Engine. https://www.syntiant.com/post/keyword-spotting-power-comparison Retrieved January 11, 2022 fromGoogle Scholar
- Syntiant 2021. Syntiant Tiny Machine Learning Development Board. https://www.syntiant.com/tinyml Retrieved December 30, 2021 fromGoogle Scholar
- Enrico Tabanelli, Giuseppe Tagliavini, and Luca Benini. 2021. DNN is not all you need: Parallelizing Non-Neural ML Algorithms on Ultra-Low-Power IoT Processors. arXiv preprint arXiv:2107.09448(2021).Google Scholar
- Ying Tai, Jian Yang, and Xiaoming Liu. 2017. Image super-resolution via deep recursive residual network. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3147–3155.Google ScholarCross Ref
- Paul J. Taylor, Tooska Dargahi, Ali Dehghantanha, Reza M. Parizi, and Kim-Kwang Raymond Choo. 2020. A systematic literature review of blockchain cyber security. Digital Communications and Networks 6, 2 (May 2020), 147–156. https://doi.org/10.1016/j.dcan.2019.01.005Google ScholarCross Ref
- TensorFlow Blog 2021. Building a TinyML Application with TF Micro and SensiML. https://blog.tensorflow.org/2021/05/building-tinyml-application-with-tf-micro-and-sensiml.html Retrieved January 3, 2022 fromGoogle Scholar
- Lucas Theis, Iryna Korshunova, Alykhan Tejani, and Ferenc Huszár. 2018. Faster gaze prediction with dense networks and Fisher pruning. CoRR abs/1801.05787(2018). arXiv:1801.05787 http://arxiv.org/abs/1801.05787Google Scholar
- TinyML 2022. TinyML Foundation. https://www.tinyml.org/ Retrieved January 8, 2022 fromGoogle Scholar
- TinyML Book 2019. TinyML Machine Learning with TensorFlow Lite on Arduino and Ultra-Low-Power Microcontrollers. https://tinymlbook.com/ Retrieved January 3, 2022 fromGoogle Scholar
- Kristof T’Jonck, Chandrakanth R Kancharla, Jens Vankeirsbilck, Hans Hallez, Jeroen Boydens, and Bozheng Pang. 2021. Real-Time Activity Tracking using TinyML to Support Elderly Care. In 2021 XXX International Scientific Conference Electronics (ET). IEEE, 1–6.Google ScholarCross Ref
- Vasileios Tsoukas, Anargyros Gkogkidis, Aikaterini Kampa, Georgios Spathoulas, and Athanasios Kakarountas. 2022. Enhancing Food Supply Chain Security through the Use of Blockchain and TinyML. Information 13, 5 (May 2022), 213. https://doi.org/10.3390/info13050213 Number: 5 Publisher: Multidisciplinary Digital Publishing Institute.Google ScholarCross Ref
- Juanjuan Tu, Yongzhao Zhan, and Fei Han. 2010. A neural network pruning method optimized with PSO algorithm. In 2010 Second International Conference on Computer Modeling and Simulation, Vol. 3. IEEE, 257–259.Google ScholarDigital Library
- Karen Ullrich, Edward Meeds, and Max Welling. 2017. Soft weight-sharing for neural network compression. arXiv preprint arXiv:1702.04008(2017).Google Scholar
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998–6008.Google Scholar
- Federico Venturini, Federico Mason, Francesco Pase, Federico Chiariotti, Alberto Testolin, Andrea Zanella, and Michele Zorzi. 2020. Distributed reinforcement learning for flexible UAV swarm control with transfer learning capabilities. In Proceedings of the 6th ACM Workshop on Micro Aerial Vehicle Networks, Systems, and Applications. 1–6.Google ScholarDigital Library
- Ishan Karunanayake Visal Rajapakse and Nadeem Ahmed. 2023. Intelligence at the Extreme Edge: A Survey on Reformable TinyML. ACM Comput. Surv. 55, 13s(2023).Google Scholar
- Miljan Vuletic, Vladimir Mujagic, Nikola Milojevic, and Debmalya Biswas. [n.d.]. Edge AI Framework for Healthcare Applications. ([n. d.]).Google Scholar
- Chandrasekar Vuppalapati, Anitha Ilapakurti, Karthik Chillara, Sharat Kedari, and Vanaja Mamidi. 2020. Automating Tiny ML Intelligent Sensors DevOPS Using Microsoft Azure. In 2020 IEEE International Conference on Big Data (Big Data). IEEE, 2375–2384.Google Scholar
- Chandrasekar Vuppalapati, Anitha Ilapakurti, Sharat Kedari, Jaya Vuppalapati, Santosh Kedari, and Raja Vuppalapati. 2020. Democratization of AI, Albeit Constrained IoT Devices & Tiny ML, for Creating a Sustainable Food Future. In 2020 3rd International Conference on Information and Computer Technologies (ICICT). IEEE, 525–530.Google ScholarCross Ref
- Chandrasekar Vuppalapati, Anitha Ilapakurti, Sharat Kedari, Raja Vuppalapati, Jaya Vuppalapati, and Santosh Kedari. 2021. Crossing the Artificial Intelligence (AI) Chasm, Albeit Using Constrained IoT Edges and Tiny ML, for Creating a Sustainable Food Future. In Proceedings of Fifth International Congress on Information and Communication Technology. Springer, 540–553.Google ScholarCross Ref
- Christopher A Walsh. 2013. Peter Huttenlocher (1931–2013). Nature 502, 7470 (2013), 172–172.Google Scholar
- Kuan Wang, Zhijian Liu, Yujun Lin, Ji Lin, and Song Han. 2019. Haq: Hardware-aware automated quantization with mixed precision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8612–8620.Google ScholarCross Ref
- Shengling Wang, Rongfang Bie, Feng Zhao, Nan Zhang, Xiuzhen Cheng, and Hyeong-Ah Choi. 2016. Security in wearable communications. IEEE Network 30, 5 (2016), 61–67.Google ScholarCross Ref
- Pete Warden and Daniel Situnayake. 2019. TinyML. O’Reilly Media, Incorporated.Google Scholar
- Kilian Weinberger, Anirban Dasgupta, John Langford, Alex Smola, and Josh Attenberg. 2009. Feature hashing for large scale multitask learning. In Proceedings of the 26th annual international conference on machine learning. 1113–1120.Google ScholarDigital Library
- Wei Wen, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li. 2016. Learning structured sparsity in deep neural networks. Advances in neural information processing systems 29 (2016), 2074–2082.Google Scholar
- Darrell Whitley, Timothy Starkweather, and Christopher Bogart. 1990. Genetic algorithms and neural networks: Optimizing connections and connectivity. Parallel computing 14, 3 (1990), 347–361.Google Scholar
- Alexander Wong, Mahmoud Famouri, Maya Pavlova, and Siddharth Surana. 2020. Tinyspeech: Attention condensers for deep speech recognition neural networks on edge devices. arXiv preprint arXiv:2008.04245(2020).Google Scholar
- Tong Xiao, Yinqiao Li, Jingbo Zhu, Zhengtao Yu, and Tongran Liu. 2019. Sharing attention weights for fast transformer. arXiv preprint arXiv:1906.11024(2019).Google Scholar
- Lei Yang, Weiwen Jiang, Weichen Liu, HM Edwin, Yiyu Shi, and Jingtong Hu. 2020. Co-exploring neural architecture and network-on-chip design for real-time artificial intelligence. In 2020 25th Asia and South Pacific Design Automation Conference (ASP-DAC). IEEE, 85–90.Google ScholarDigital Library
- Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. Advances in neural information processing systems 32 (2019).Google Scholar
- Wei Yu, Fan Liang, Xiaofei He, William Grant Hatcher, Chao Lu, Jie Lin, and Xinyu Yang. 2017. A survey on the edge computing for the Internet of Things. IEEE access 6(2017), 6900–6919.Google Scholar
- Dejiao Zhang, Haozhu Wang, Mario Figueiredo, and Laura Balzano. 2018. Learning to share: Simultaneous parameter tying and sparsification in deep learning. In International Conference on Learning Representations.Google Scholar
- Yundong Zhang, Naveen Suda, Liangzhen Lai, and Vikas Chandra. 2017. Hello edge: Keyword spotting on microcontrollers. arXiv preprint arXiv:1711.07128(2017).Google Scholar
- Yulun Zhang, Yapeng Tian, Yu Kong, Bineng Zhong, and Yun Fu. 2018. Residual dense network for image super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2472–2481.Google ScholarCross Ref
- Andy Zhou, Rikky Muller, and Jan Rabaey. 2021. Memory-Efficient, Limb Position-Aware Hand Gesture Recognition using Hyperdimensional Computing. arXiv preprint arXiv:2103.05267(2021).Google Scholar
- Aojun Zhou, Anbang Yao, Kuan Wang, and Yurong Chen. 2018. Explicit loss-error-aware quantization for low-bit deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 9426–9435.Google ScholarCross Ref
- Shuchang Zhou, Yuxin Wu, Zekun Ni, Xinyu Zhou, He Wen, and Yuheng Zou. 2016. Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients. arXiv preprint arXiv:1606.06160(2016).Google Scholar
- Xingyu Zhou, Zhuangwei Kang, Robert Canady, Shunxing Bao, Daniel Allen Balasubramanian, and Aniruddha Gokhale. 2021. Exploring Cloud Assisted Tiny Machine Learning Application Patterns for PHM Scenarios. In Annual Conference of the PHM Society, Vol. 13.Google Scholar
Index Terms
- A Review on the emerging technology of TinyML
Recommendations
A Review of Machine Learning and TinyML in Healthcare
PCI '21: Proceedings of the 25th Pan-Hellenic Conference on InformaticsHealthcare is the field that can benefit from the large amount of raw data generated from portable and wearable devices. This data must be sent to the Cloud for processing due to the computationally intensive nature of current state-of-the-art ...
The Potential of Emerging Technology for Social Change
SIGMIS-CPR '16: Proceedings of the 2016 ACM SIGMIS Conference on Computers and People ResearchThe purpose of this panel is to explore the potential for emerging technology to engender social change. Technology is transformative, and has already seen innovative application in business and industry, and of course in relation to society and social ...
Revolutionizing Intelligent Transportation Systems with Cellular Vehicle-to-Everything (C-V2X) technology: Current trends, use cases, emerging technologies, standardization bodies, industry analytics and future directions
AbstractThe emergence of Cellular Vehicle-to-Everything communication (C-V2X) has brought significant advancements in the field of Intelligent Transportation Systems. This review presents an in-depth analysis of the benefits offered by C-V2X technology, ...
Comments