Maintained by Difan Deng and Marius Lindauer.
The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.
Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.
Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.
2022
1889.
Xu, Shuhan; Ren, Yudan; Tao, Zeyang; Song, Limei; He, Xiaowei
In: eNeuro, 2022.
@article{XuENEURO.0200-22.2022,
title = {Hierarchical Individual Naturalistic Functional Brain Networks with Group Consistency uncovered by a Two-Stage NAS-Volumetric Sparse DBN Framework},
author = {Shuhan Xu and Yudan Ren and Zeyang Tao and Limei Song and Xiaowei He},
url = {https://www.eneuro.org/content/early/2022/08/19/ENEURO.0200-22.2022},
doi = {10.1523/ENEURO.0200-22.2022},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {eNeuro},
publisher = {Society for Neuroscience},
abstract = {The functional magnetic resonance imaging under naturalistic paradigm (NfMRI) showed great advantages in identifying complex and interactive functional brain networks due to its dynamics and multimodal information. In recent years, various deep learning models, such as deep convolutional autoencoder (DCAE), deep belief network (DBN) and volumetric sparse deep belief network (vsDBN), can obtain hierarchical functional brain networks (FBN) and temporal features from fMRI data. Among them, the vsDBN model revealed a good capability in identifying hierarchical FBNs by modelling fMRI volume images. However, due to the high dimensionality of fMRI volumes and the diverse training parameters of deep learning methods, especially the network architecture that is the most critical parameter for uncovering the hierarchical organization of human brain function, researchers still face challenges in designing an appropriate deep learning framework with automatic network architecture optimization to model volumetric NfMRI. In addition, most of the existing deep learning models ignore the group-wise consistency and inter-subject variation properties embedded in NfMRI volumes. To solve these problems, we proposed a two-stage neural architecture search and vs DBN model (two-stage NAS-vsDBN model) to identify the hierarchical human brain spatio-temporal features possessing both group-consistency and individual-uniqueness under naturalistic condition. Moreover, our model defined reliable network structure for modelling volumetric NfMRI data via NAS framework, and the group-level and individual-level FBNs and associated temporal features exhibited great consistency. In general, our method well identified the hierarchical temporal and spatial features of the brain function and revealed the crucial properties of neural processes under natural viewing condition.Significance StatementIn this paper, we proposed and applied a novel analytical strategy – a two-stage NAS-vsDBN model to identify both group-level and individual-level spatio-temporal features at multi-scales from volumetric NfMRI data. The proposed PSO-based NAS framework can find optimal neural structure for both group-wise and individual-level vs-DBN models. Furthermore, with well-established correspondence between two stages of vsDBN models, our model can effectively detect group-level FBNs that reveal the consistency in neural processes across subjects and individual-level FBNs that maintain the subject specific variability, verifying the inherent property of brain function under naturalistic condition.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The functional magnetic resonance imaging under naturalistic paradigm (NfMRI) showed great advantages in identifying complex and interactive functional brain networks due to its dynamics and multimodal information. In recent years, various deep learning models, such as deep convolutional autoencoder (DCAE), deep belief network (DBN) and volumetric sparse deep belief network (vsDBN), can obtain hierarchical functional brain networks (FBN) and temporal features from fMRI data. Among them, the vsDBN model revealed a good capability in identifying hierarchical FBNs by modelling fMRI volume images. However, due to the high dimensionality of fMRI volumes and the diverse training parameters of deep learning methods, especially the network architecture that is the most critical parameter for uncovering the hierarchical organization of human brain function, researchers still face challenges in designing an appropriate deep learning framework with automatic network architecture optimization to model volumetric NfMRI. In addition, most of the existing deep learning models ignore the group-wise consistency and inter-subject variation properties embedded in NfMRI volumes. To solve these problems, we proposed a two-stage neural architecture search and vs DBN model (two-stage NAS-vsDBN model) to identify the hierarchical human brain spatio-temporal features possessing both group-consistency and individual-uniqueness under naturalistic condition. Moreover, our model defined reliable network structure for modelling volumetric NfMRI data via NAS framework, and the group-level and individual-level FBNs and associated temporal features exhibited great consistency. In general, our method well identified the hierarchical temporal and spatial features of the brain function and revealed the crucial properties of neural processes under natural viewing condition.Significance StatementIn this paper, we proposed and applied a novel analytical strategy – a two-stage NAS-vsDBN model to identify both group-level and individual-level spatio-temporal features at multi-scales from volumetric NfMRI data. The proposed PSO-based NAS framework can find optimal neural structure for both group-wise and individual-level vs-DBN models. Furthermore, with well-established correspondence between two stages of vsDBN models, our model can effectively detect group-level FBNs that reveal the consistency in neural processes across subjects and individual-level FBNs that maintain the subject specific variability, verifying the inherent property of brain function under naturalistic condition.
1888.
Li, Nan; Ma, Lianbo; Yu, Guo; Xue, Bing; Zhang, Mengjie; Jin, Yaochu
Survey on Evolutionary Deep Learning: Principles, Algorithms, Applications and Open Issues Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2208-10658,
title = {Survey on Evolutionary Deep Learning: Principles, Algorithms, Applications and Open Issues},
author = {Nan Li and Lianbo Ma and Guo Yu and Bing Xue and Mengjie Zhang and Yaochu Jin},
url = {https://doi.org/10.48550/arXiv.2208.10658},
doi = {10.48550/arXiv.2208.10658},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2208.10658},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1887.
Ghosh, Arjun; Jana, Nanda Dulal; Mallik, Saurav; Zhao, Zhongming
Designing optimal convolutional neural network architecture using differential evolution algorithm Journal Article
In: Patterns, vol. 3, no. 9, pp. 100567, 2022, ISSN: 2666-3899.
@article{GHOSH2022100567,
title = {Designing optimal convolutional neural network architecture using differential evolution algorithm},
author = {Arjun Ghosh and Nanda Dulal Jana and Saurav Mallik and Zhongming Zhao},
url = {https://www.sciencedirect.com/science/article/pii/S2666389922001787},
doi = {https://doi.org/10.1016/j.patter.2022.100567},
issn = {2666-3899},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Patterns},
volume = {3},
number = {9},
pages = {100567},
abstract = {Summary
Convolutional neural networks (CNNs) are deep learning models used widely for solving various tasks like computer vision and speech recognition. CNNs are developed manually based on problem-specific domain knowledge and tricky settings, which are laborious, time consuming, and challenging. To solve these, our study develops an improved differential evolution of convolutional neural network (IDECNN) algorithm to design CNN layer architectures for image classification. Variable-length encoding is utilized to represent the flexible layer architecture of a CNN model in IDECNN. An efficient heuristic mechanism is proposed in IDECNN to evolve CNN architecture through mutation and crossover to prevent premature convergence during the evolutionary process. Eight well-known imaging datasets were utilized. The results showed that IDECNN could design suitable architecture compared with 20 existing CNN models. Finally, CNN architectures are applied to pneumonia and coronavirus disease 2019 (COVID-19) X-ray biomedical image data. The results demonstrated the usefulness of the proposed approach to generate a suitable CNN model.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Summary
Convolutional neural networks (CNNs) are deep learning models used widely for solving various tasks like computer vision and speech recognition. CNNs are developed manually based on problem-specific domain knowledge and tricky settings, which are laborious, time consuming, and challenging. To solve these, our study develops an improved differential evolution of convolutional neural network (IDECNN) algorithm to design CNN layer architectures for image classification. Variable-length encoding is utilized to represent the flexible layer architecture of a CNN model in IDECNN. An efficient heuristic mechanism is proposed in IDECNN to evolve CNN architecture through mutation and crossover to prevent premature convergence during the evolutionary process. Eight well-known imaging datasets were utilized. The results showed that IDECNN could design suitable architecture compared with 20 existing CNN models. Finally, CNN architectures are applied to pneumonia and coronavirus disease 2019 (COVID-19) X-ray biomedical image data. The results demonstrated the usefulness of the proposed approach to generate a suitable CNN model.
Convolutional neural networks (CNNs) are deep learning models used widely for solving various tasks like computer vision and speech recognition. CNNs are developed manually based on problem-specific domain knowledge and tricky settings, which are laborious, time consuming, and challenging. To solve these, our study develops an improved differential evolution of convolutional neural network (IDECNN) algorithm to design CNN layer architectures for image classification. Variable-length encoding is utilized to represent the flexible layer architecture of a CNN model in IDECNN. An efficient heuristic mechanism is proposed in IDECNN to evolve CNN architecture through mutation and crossover to prevent premature convergence during the evolutionary process. Eight well-known imaging datasets were utilized. The results showed that IDECNN could design suitable architecture compared with 20 existing CNN models. Finally, CNN architectures are applied to pneumonia and coronavirus disease 2019 (COVID-19) X-ray biomedical image data. The results demonstrated the usefulness of the proposed approach to generate a suitable CNN model.
1886.
Wang, Zihan; Wan, Chengcheng; Chen, Yuting; Lin, Ziyi; Jiang, He; Qiao, Lei
Hierarchical Memory-Constrained Operator Scheduling of Neural Architecture Search Networks Proceedings Article
In: Proceedings of the 59th ACM/IEEE Design Automation Conference, pp. 493–498, Association for Computing Machinery, San Francisco, California, 2022, ISBN: 9781450391429.
@inproceedings{10.1145/3489517.3530472,
title = {Hierarchical Memory-Constrained Operator Scheduling of Neural Architecture Search Networks},
author = {Zihan Wang and Chengcheng Wan and Yuting Chen and Ziyi Lin and He Jiang and Lei Qiao},
url = {https://doi.org/10.1145/3489517.3530472},
doi = {10.1145/3489517.3530472},
isbn = {9781450391429},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Proceedings of the 59th ACM/IEEE Design Automation Conference},
pages = {493–498},
publisher = {Association for Computing Machinery},
address = {San Francisco, California},
series = {DAC '22},
abstract = {Neural Architecture Search (NAS) is widely used in industry, searching for neural networks meeting task requirements. Meanwhile, it faces a challenge in scheduling networks satisfying memory constraints. This paper proposes HMCOS that performs hierarchical memory-constrained operator scheduling of NAS networks: given a network, HMCOS constructs a hierarchical computation graph and employs an iterative scheduling algorithm to progressively reduce peak memory footprints. We evaluate HMCOS against RPO and Serenity (two popular scheduling techniques). The results show that HMCOS outperforms existing techniques in supporting more NAS networks, reducing 8.7~42.4% of peak memory footprints, and achieving 137--283x of speedups in scheduling.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Neural Architecture Search (NAS) is widely used in industry, searching for neural networks meeting task requirements. Meanwhile, it faces a challenge in scheduling networks satisfying memory constraints. This paper proposes HMCOS that performs hierarchical memory-constrained operator scheduling of NAS networks: given a network, HMCOS constructs a hierarchical computation graph and employs an iterative scheduling algorithm to progressively reduce peak memory footprints. We evaluate HMCOS against RPO and Serenity (two popular scheduling techniques). The results show that HMCOS outperforms existing techniques in supporting more NAS networks, reducing 8.7~42.4% of peak memory footprints, and achieving 137--283x of speedups in scheduling.
1885.
Negi, Shubham; Chakraborty, Indranil; Ankit, Aayush; Roy, Kaushik
NAX: Neural Architecture and Memristive Xbar Based Accelerator Co-Design Proceedings Article
In: Proceedings of the 59th ACM/IEEE Design Automation Conference, pp. 451–456, Association for Computing Machinery, San Francisco, California, 2022, ISBN: 9781450391429.
@inproceedings{10.1145/3489517.3530476,
title = {NAX: Neural Architecture and Memristive Xbar Based Accelerator Co-Design},
author = {Shubham Negi and Indranil Chakraborty and Aayush Ankit and Kaushik Roy},
url = {https://doi.org/10.1145/3489517.3530476},
doi = {10.1145/3489517.3530476},
isbn = {9781450391429},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Proceedings of the 59th ACM/IEEE Design Automation Conference},
pages = {451–456},
publisher = {Association for Computing Machinery},
address = {San Francisco, California},
series = {DAC '22},
abstract = {Neural Architecture Search (NAS) has provided the ability to design efficient deep neural network (DNN) catered towards different hardwares like GPUs, CPUs etc. However, integrating NAS with Memristive Crossbar Array (MCA) based In-Memory Computing (IMC) accelerator remains an open problem. The hardware efficiency (energy, latency and area) as well as application accuracy (considering device and circuit non-idealities) of DNNs mapped to such hardware are co-dependent on network parameters such as kernel size, depth etc. and hardware architecture parameters such as crossbar size and the precision of analog-to-digital converters. Co-optimization of both network and hardware parameters presents a challenging search space comprising of different kernel sizes mapped to varying crossbar sizes. To that effect, we propose NAX - an efficient neural architecture search engine that co-designs neural network and IMC based hardware architecture. NAX explores the aforementioned search space to determine kernel and corresponding crossbar sizes for each DNN layer to achieve optimal tradeoffs between hardware efficiency and application accuracy. For CIFAR-10 and Tiny ImageNet, our models achieve 0.9% and 18.57% higher accuracy at 30% and -10.47% lower EDAP (energy-delay-area product), compared to baseline ResNet-20 and ResNet-18 models, respectively.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Neural Architecture Search (NAS) has provided the ability to design efficient deep neural network (DNN) catered towards different hardwares like GPUs, CPUs etc. However, integrating NAS with Memristive Crossbar Array (MCA) based In-Memory Computing (IMC) accelerator remains an open problem. The hardware efficiency (energy, latency and area) as well as application accuracy (considering device and circuit non-idealities) of DNNs mapped to such hardware are co-dependent on network parameters such as kernel size, depth etc. and hardware architecture parameters such as crossbar size and the precision of analog-to-digital converters. Co-optimization of both network and hardware parameters presents a challenging search space comprising of different kernel sizes mapped to varying crossbar sizes. To that effect, we propose NAX - an efficient neural architecture search engine that co-designs neural network and IMC based hardware architecture. NAX explores the aforementioned search space to determine kernel and corresponding crossbar sizes for each DNN layer to achieve optimal tradeoffs between hardware efficiency and application accuracy. For CIFAR-10 and Tiny ImageNet, our models achieve 0.9% and 18.57% higher accuracy at 30% and -10.47% lower EDAP (energy-delay-area product), compared to baseline ResNet-20 and ResNet-18 models, respectively.
1884.
Luo, Xiangzhong; Liu, Di; Kong, Hao; Huai, Shuo; Chen, Hui; Liu, Weichen
You Only Search Once: On Lightweight Differentiable Architecture Search for Resource-Constrained Embedded Platforms Proceedings Article
In: Proceedings of the 59th ACM/IEEE Design Automation Conference, pp. 475–480, Association for Computing Machinery, San Francisco, California, 2022, ISBN: 9781450391429.
@inproceedings{10.1145/3489517.3530488,
title = {You Only Search Once: On Lightweight Differentiable Architecture Search for Resource-Constrained Embedded Platforms},
author = {Xiangzhong Luo and Di Liu and Hao Kong and Shuo Huai and Hui Chen and Weichen Liu},
url = {https://doi.org/10.1145/3489517.3530488},
doi = {10.1145/3489517.3530488},
isbn = {9781450391429},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Proceedings of the 59th ACM/IEEE Design Automation Conference},
pages = {475–480},
publisher = {Association for Computing Machinery},
address = {San Francisco, California},
series = {DAC '22},
abstract = {Benefiting from the search efficiency, differentiable neural architecture search (NAS) has evolved as the most dominant alternative to automatically design competitive deep neural networks (DNNs). We note that DNNs must be executed under strictly hard performance constraints in real-world scenarios, for example, the runtime latency on autonomous vehicles. However, to obtain the architecture that meets the given performance constraint, previous hardware-aware differentiable NAS methods have to repeat a plethora of search runs to manually tune the hyper-parameters by trial and error, and thus the total design cost increases proportionally. To resolve this, we introduce a lightweight hardware-aware differentiable NAS framework dubbed LightNAS, striving to find the required architecture that satisfies various performance constraints through a one-time search (i.e., you only search once). Extensive experiments are conducted to show the superiority of LightNAS over previous state-of-the-art methods. Related codes will be released at https://github.com/stepbuystep/LightNAS.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Benefiting from the search efficiency, differentiable neural architecture search (NAS) has evolved as the most dominant alternative to automatically design competitive deep neural networks (DNNs). We note that DNNs must be executed under strictly hard performance constraints in real-world scenarios, for example, the runtime latency on autonomous vehicles. However, to obtain the architecture that meets the given performance constraint, previous hardware-aware differentiable NAS methods have to repeat a plethora of search runs to manually tune the hyper-parameters by trial and error, and thus the total design cost increases proportionally. To resolve this, we introduce a lightweight hardware-aware differentiable NAS framework dubbed LightNAS, striving to find the required architecture that satisfies various performance constraints through a one-time search (i.e., you only search once). Extensive experiments are conducted to show the superiority of LightNAS over previous state-of-the-art methods. Related codes will be released at https://github.com/stepbuystep/LightNAS.
1883.
Jääsaari, Elias; Ma, Michelle; Talwalkar, Ameet; Chen, Tianqi
SONAR: Joint Architecture and System Optimization Search Technical Report
2022.
@techreport{jaasaari2022sonar,
title = {SONAR: Joint Architecture and System Optimization Search},
author = {Elias Jääsaari and Michelle Ma and Ameet Talwalkar and Tianqi Chen},
url = {https://arxiv.org/abs/2208.12218},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {arXiv preprint arXiv:2208.12218},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1882.
Hong, Deokki; Choi, Kanghyun; Lee, Hye Yoon; Yu, Joonsang; Park, Noseong; Kim, Youngsok; Lee, Jinho
Enabling Hard Constraints in Differentiable Neural Network and Accelerator Co-Exploration Proceedings Article
In: Proceedings of the 59th ACM/IEEE Design Automation Conference, pp. 589–594, Association for Computing Machinery, San Francisco, California, 2022, ISBN: 9781450391429.
@inproceedings{10.1145/3489517.3530507,
title = {Enabling Hard Constraints in Differentiable Neural Network and Accelerator Co-Exploration},
author = {Deokki Hong and Kanghyun Choi and Hye Yoon Lee and Joonsang Yu and Noseong Park and Youngsok Kim and Jinho Lee},
url = {https://doi.org/10.1145/3489517.3530507},
doi = {10.1145/3489517.3530507},
isbn = {9781450391429},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Proceedings of the 59th ACM/IEEE Design Automation Conference},
pages = {589–594},
publisher = {Association for Computing Machinery},
address = {San Francisco, California},
series = {DAC '22},
abstract = {Co-exploration of an optimal neural architecture and its hardware accelerator is an approach of rising interest which addresses the computational cost problem, especially in low-profile systems. The large co-exploration space is often handled by adopting the idea of differentiable neural architecture search. However, despite the superior search efficiency of the differentiable co-exploration, it faces a critical challenge of not being able to systematically satisfy hard constraints such as frame rate. To handle the hard constraint problem of differentiable co-exploration, we propose HDX, which searches for hard-constrained solutions without compromising the global design objectives. By manipulating the gradients in the interest of the given hard constraint, high-quality solutions satisfying the constraint can be obtained.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Co-exploration of an optimal neural architecture and its hardware accelerator is an approach of rising interest which addresses the computational cost problem, especially in low-profile systems. The large co-exploration space is often handled by adopting the idea of differentiable neural architecture search. However, despite the superior search efficiency of the differentiable co-exploration, it faces a critical challenge of not being able to systematically satisfy hard constraints such as frame rate. To handle the hard constraint problem of differentiable co-exploration, we propose HDX, which searches for hard-constrained solutions without compromising the global design objectives. By manipulating the gradients in the interest of the given hard constraint, high-quality solutions satisfying the constraint can be obtained.
1881.
Ghosh, Arjun; Jana, Nanda Dulal; Mallik, Saurav; Zhao, Zhongming
Designing optimal convolutional neural network architecture using differential evolution algorithm Journal Article
In: Patterns, vol. 3, no. 9, pp. 100567, 2022, ISSN: 2666-3899.
@article{GHOSH2022100567b,
title = {Designing optimal convolutional neural network architecture using differential evolution algorithm},
author = {Arjun Ghosh and Nanda Dulal Jana and Saurav Mallik and Zhongming Zhao},
url = {https://www.sciencedirect.com/science/article/pii/S2666389922001787},
doi = {https://doi.org/10.1016/j.patter.2022.100567},
issn = {2666-3899},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Patterns},
volume = {3},
number = {9},
pages = {100567},
abstract = {Summary
Convolutional neural networks (CNNs) are deep learning models used widely for solving various tasks like computer vision and speech recognition. CNNs are developed manually based on problem-specific domain knowledge and tricky settings, which are laborious, time consuming, and challenging. To solve these, our study develops an improved differential evolution of convolutional neural network (IDECNN) algorithm to design CNN layer architectures for image classification. Variable-length encoding is utilized to represent the flexible layer architecture of a CNN model in IDECNN. An efficient heuristic mechanism is proposed in IDECNN to evolve CNN architecture through mutation and crossover to prevent premature convergence during the evolutionary process. Eight well-known imaging datasets were utilized. The results showed that IDECNN could design suitable architecture compared with 20 existing CNN models. Finally, CNN architectures are applied to pneumonia and coronavirus disease 2019 (COVID-19) X-ray biomedical image data. The results demonstrated the usefulness of the proposed approach to generate a suitable CNN model.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Summary
Convolutional neural networks (CNNs) are deep learning models used widely for solving various tasks like computer vision and speech recognition. CNNs are developed manually based on problem-specific domain knowledge and tricky settings, which are laborious, time consuming, and challenging. To solve these, our study develops an improved differential evolution of convolutional neural network (IDECNN) algorithm to design CNN layer architectures for image classification. Variable-length encoding is utilized to represent the flexible layer architecture of a CNN model in IDECNN. An efficient heuristic mechanism is proposed in IDECNN to evolve CNN architecture through mutation and crossover to prevent premature convergence during the evolutionary process. Eight well-known imaging datasets were utilized. The results showed that IDECNN could design suitable architecture compared with 20 existing CNN models. Finally, CNN architectures are applied to pneumonia and coronavirus disease 2019 (COVID-19) X-ray biomedical image data. The results demonstrated the usefulness of the proposed approach to generate a suitable CNN model.
Convolutional neural networks (CNNs) are deep learning models used widely for solving various tasks like computer vision and speech recognition. CNNs are developed manually based on problem-specific domain knowledge and tricky settings, which are laborious, time consuming, and challenging. To solve these, our study develops an improved differential evolution of convolutional neural network (IDECNN) algorithm to design CNN layer architectures for image classification. Variable-length encoding is utilized to represent the flexible layer architecture of a CNN model in IDECNN. An efficient heuristic mechanism is proposed in IDECNN to evolve CNN architecture through mutation and crossover to prevent premature convergence during the evolutionary process. Eight well-known imaging datasets were utilized. The results showed that IDECNN could design suitable architecture compared with 20 existing CNN models. Finally, CNN architectures are applied to pneumonia and coronavirus disease 2019 (COVID-19) X-ray biomedical image data. The results demonstrated the usefulness of the proposed approach to generate a suitable CNN model.
1880.
Chen, Yantong; Bian, Shichang; Liu, Yang; Zhang, Zhongling
A novel method based on neural architecture search for Diptera insect classification on embedded devices Journal Article
In: Ecological Informatics, vol. 71, pp. 101791, 2022, ISSN: 1574-9541.
@article{CHEN2022101791,
title = {A novel method based on neural architecture search for Diptera insect classification on embedded devices},
author = {Yantong Chen and Shichang Bian and Yang Liu and Zhongling Zhang},
url = {https://www.sciencedirect.com/science/article/pii/S1574954122002412},
doi = {https://doi.org/10.1016/j.ecoinf.2022.101791},
issn = {1574-9541},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Ecological Informatics},
volume = {71},
pages = {101791},
abstract = {Diptera insects have the characteristics of spreading diseases and destroying forests. There are similarities among different species, which makes it difficult to identify a Diptera insect. Most traditional convolutional neural networks have large parameters and high recognition latency. Therefore, they are not suitable for deploying models on embedded devices for classification and recognition. This paper proposes an improved neural architecture based on differentiable search method. First, we designed a network search cell by adding the feature output of the previous layer to each search cell. Second, we added the attention module to the search space to expand the searchable range. At the same time, we used methods such as model quantization and limiting the ReLU function to the ReLU6 function to reduce computer resource consumption. Finally, the network model was transplanted to the NVIDIA Jetson Xavier NX embedded development platform to verify the network performance so that the neural architecture search could be organically combined with the embedded development platform. The experimental results show that the designed neural architecture achieves 98.9% accuracy on the Diptera insect dataset with a latency of 8.4 ms. It has important practical significance for the recognition of Diptera insects in embedded devices.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Diptera insects have the characteristics of spreading diseases and destroying forests. There are similarities among different species, which makes it difficult to identify a Diptera insect. Most traditional convolutional neural networks have large parameters and high recognition latency. Therefore, they are not suitable for deploying models on embedded devices for classification and recognition. This paper proposes an improved neural architecture based on differentiable search method. First, we designed a network search cell by adding the feature output of the previous layer to each search cell. Second, we added the attention module to the search space to expand the searchable range. At the same time, we used methods such as model quantization and limiting the ReLU function to the ReLU6 function to reduce computer resource consumption. Finally, the network model was transplanted to the NVIDIA Jetson Xavier NX embedded development platform to verify the network performance so that the neural architecture search could be organically combined with the embedded development platform. The experimental results show that the designed neural architecture achieves 98.9% accuracy on the Diptera insect dataset with a latency of 8.4 ms. It has important practical significance for the recognition of Diptera insects in embedded devices.
1879.
Vikashini, Vetha; Salam, Hanan; Nasir, Jauwairia; Bruno, Barbara; Celiktutan, Oya
Personalized Productive Engagement Recognition in Robot-Mediated Collaborative Learning Proceedings Article
In: 2022.
@inproceedings{Vikashini:296044,
title = {Personalized Productive Engagement Recognition in Robot-Mediated Collaborative Learning},
author = {Vetha Vikashini and Hanan Salam and Jauwairia Nasir and Barbara Bruno and Oya Celiktutan},
url = {http://infoscience.epfl.ch/record/296044},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
abstract = {In this paper, we propose and compare personalized models for Productive Engagement (PE) recognition. PE is defined as the level of engagement that maximizes learning. Previously, in the context of robot-mediated collaborative learning, a framework of productive engagement was developed by utilizing multimodal data of 32 dyads and learning profiles, namely, Expressive Explorers (EE), Calm Tinkerers (CT), and Silent Wanderers (SW) were identified which categorize learners according to their learning gain. Within the same framework, a PE score was constructed in a non-supervised manner for real-time evaluation. Here, we use these profiles and the PE score within an AutoML deep learning framework to personalize PE models. We investigate two approaches for this purpose: (1) Single-task Deep Neural Architecture Search (ST-NAS), and (2) Multitask NAS (MT-NAS). In the former approach, personalized models for each learner profile are learned from multimodal features and compared to non-personalized models. In the MT-NAS approach, we investigate whether jointly classifying the learners' profiles with the engagement score through multi-task learning would serve as an implicit personalization of PE. Moreover, we compare the predictive power of two types of features: incremental and non-incremental features. Non-incremental features correspond to features computed from the participant's behaviours in fixed time windows. Incremental features are computed by accounting to the behaviour from the beginning of the learning activity till the time window where productive engagement is observed. Our experimental results show that (1) personalized models improve the recognition performance with respect to non-personalized models when training models for the gainer vs. non-gainer groups, (2) multitask NAS (implicit personalization) also outperforms non-personalized models, (3) the speech modality has high contribution towards prediction, and (4) non-incremental features outperform the incremental ones overall.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
In this paper, we propose and compare personalized models for Productive Engagement (PE) recognition. PE is defined as the level of engagement that maximizes learning. Previously, in the context of robot-mediated collaborative learning, a framework of productive engagement was developed by utilizing multimodal data of 32 dyads and learning profiles, namely, Expressive Explorers (EE), Calm Tinkerers (CT), and Silent Wanderers (SW) were identified which categorize learners according to their learning gain. Within the same framework, a PE score was constructed in a non-supervised manner for real-time evaluation. Here, we use these profiles and the PE score within an AutoML deep learning framework to personalize PE models. We investigate two approaches for this purpose: (1) Single-task Deep Neural Architecture Search (ST-NAS), and (2) Multitask NAS (MT-NAS). In the former approach, personalized models for each learner profile are learned from multimodal features and compared to non-personalized models. In the MT-NAS approach, we investigate whether jointly classifying the learners' profiles with the engagement score through multi-task learning would serve as an implicit personalization of PE. Moreover, we compare the predictive power of two types of features: incremental and non-incremental features. Non-incremental features correspond to features computed from the participant's behaviours in fixed time windows. Incremental features are computed by accounting to the behaviour from the beginning of the learning activity till the time window where productive engagement is observed. Our experimental results show that (1) personalized models improve the recognition performance with respect to non-personalized models when training models for the gainer vs. non-gainer groups, (2) multitask NAS (implicit personalization) also outperforms non-personalized models, (3) the speech modality has high contribution towards prediction, and (4) non-incremental features outperform the incremental ones overall.
1878.
Shimizu, Shoma; Nishio, Takayuki; Saito, Shota; Hirose, Yoichi; Chen, Yen-Hsiu; Shirakawa, Shinichi
Neural Architecture Search for Improving Latency-Accuracy Trade-off in Split Computing Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2208-13968,
title = {Neural Architecture Search for Improving Latency-Accuracy Trade-off in Split Computing},
author = {Shoma Shimizu and Takayuki Nishio and Shota Saito and Yoichi Hirose and Yen-Hsiu Chen and Shinichi Shirakawa},
url = {https://doi.org/10.48550/arXiv.2208.13968},
doi = {10.48550/arXiv.2208.13968},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2208.13968},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1877.
Yakovlev, Konstantin D.; Grebenkova, Olga S.; Bakhteev, Oleg Y.; Strijov, Vadim V.
Neural Architecture Search with Structure Complexity Control Proceedings Article
In: Burnaev, Evgeny; Ignatov, Dmitry I.; Ivanov, Sergei; Khachay, Michael; Koltsova, Olessia; Kutuzov, Andrei; Kuznetsov, Sergei O.; Loukachevitch, Natalia; Napoli, Amedeo; Panchenko, Alexander; Pardalos, Panos M.; Saramäki, Jari; Savchenko, Andrey V.; Tsymbalov, Evgenii; Tutubalina, Elena (Ed.): Recent Trends in Analysis of Images, Social Networks and Texts, pp. 207–219, Springer International Publishing, Cham, 2022, ISBN: 978-3-031-15168-2.
@inproceedings{10.1007/978-3-031-15168-2_17,
title = {Neural Architecture Search with Structure Complexity Control},
author = {Konstantin D. Yakovlev and Olga S. Grebenkova and Oleg Y. Bakhteev and Vadim V. Strijov},
editor = {Evgeny Burnaev and Dmitry I. Ignatov and Sergei Ivanov and Michael Khachay and Olessia Koltsova and Andrei Kutuzov and Sergei O. Kuznetsov and Natalia Loukachevitch and Amedeo Napoli and Alexander Panchenko and Panos M. Pardalos and Jari Saramäki and Andrey V. Savchenko and Evgenii Tsymbalov and Elena Tutubalina},
isbn = {978-3-031-15168-2},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Recent Trends in Analysis of Images, Social Networks and Texts},
pages = {207--219},
publisher = {Springer International Publishing},
address = {Cham},
abstract = {The paper investigates the problem of deep learning model selection. We propose a method of a neural architecture search with respect to the desired model complexity called DARTS-CC. An amount of parameters in the model is considered as a model complexity. The proposed method is based on a differential architecture search algorithm (DARTS). Instead of optimizing structural parameters of the architecture, we consider them as a function depending on the complexity parameter. It enables us to obtain multiple architectures at one optimization procedure and select the architecture based on our computation budget. To evaluate the performance of the proposed algorithm, we conduct experiments on the Fashion-MNIST and CIFAR-10 datasets and compare the resulting architecture with architectures obtained by other neural architecture search methods.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
The paper investigates the problem of deep learning model selection. We propose a method of a neural architecture search with respect to the desired model complexity called DARTS-CC. An amount of parameters in the model is considered as a model complexity. The proposed method is based on a differential architecture search algorithm (DARTS). Instead of optimizing structural parameters of the architecture, we consider them as a function depending on the complexity parameter. It enables us to obtain multiple architectures at one optimization procedure and select the architecture based on our computation budget. To evaluate the performance of the proposed algorithm, we conduct experiments on the Fashion-MNIST and CIFAR-10 datasets and compare the resulting architecture with architectures obtained by other neural architecture search methods.
1876.
Xue, Boyang; Hu, Shoukang; Xu, Junhao; Geng, Mengzhe; Liu, Xunying; Meng, Helen
Bayesian Neural Network Language Modeling for Speech Recognition Journal Article
In: IEEE ACM Trans. Audio Speech Lang. Process., vol. 30, pp. 2900–2917, 2022.
@article{DBLP:journals/taslp/XueHXGLM22,
title = {Bayesian Neural Network Language Modeling for Speech Recognition},
author = {Boyang Xue and Shoukang Hu and Junhao Xu and Mengzhe Geng and Xunying Liu and Helen Meng},
url = {https://doi.org/10.1109/TASLP.2022.3203891},
doi = {10.1109/TASLP.2022.3203891},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE ACM Trans. Audio Speech Lang. Process.},
volume = {30},
pages = {2900--2917},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1875.
Yin, Benshun; Chen, Zhiyong; Tao, Meixia
Dynamic Data Collection and Neural Architecture Search for Wireless Edge Intelligence Systems Journal Article
In: IEEE Transactions on Wireless Communications, pp. 1-1, 2022.
@article{9861242,
title = {Dynamic Data Collection and Neural Architecture Search for Wireless Edge Intelligence Systems},
author = {Benshun Yin and Zhiyong Chen and Meixia Tao},
url = {https://ieeexplore.ieee.org/abstract/document/9861242},
doi = {10.1109/TWC.2022.3197809},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Wireless Communications},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1874.
Han, Zhu; Hong, Danfeng; Gao, Lianru; Roy, Swalpa Kumar; Zhang, Bing; Chanussot, Jocelyn
Reinforcement Learning for Neural Architecture Search in Hyperspectral Unmixing Journal Article
In: IEEE Geoscience and Remote Sensing Letters, vol. 19, pp. 1-5, 2022.
@article{9865216,
title = {Reinforcement Learning for Neural Architecture Search in Hyperspectral Unmixing},
author = {Zhu Han and Danfeng Hong and Lianru Gao and Swalpa Kumar Roy and Bing Zhang and Jocelyn Chanussot},
url = {https://ieeexplore.ieee.org/abstract/document/9865216},
doi = {10.1109/LGRS.2022.3199583},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Geoscience and Remote Sensing Letters},
volume = {19},
pages = {1-5},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1873.
Xu, Peng; Wang, Ke; Hassan, Mohammad Mehedi; Chen, Chien-Ming; Lin, Weiguo; Hassan, Md. Rafiul; Fortino, Giancarlo
Adversarial Robustness in Graph-Based Neural Architecture Search for Edge AI Transportation Systems Journal Article
In: IEEE Transactions on Intelligent Transportation Systems, pp. 1-10, 2022.
@article{9868259,
title = {Adversarial Robustness in Graph-Based Neural Architecture Search for Edge AI Transportation Systems},
author = {Peng Xu and Ke Wang and Mohammad Mehedi Hassan and Chien-Ming Chen and Weiguo Lin and Md. Rafiul Hassan and Giancarlo Fortino},
url = {https://ieeexplore.ieee.org/abstract/document/9868259},
doi = {10.1109/TITS.2022.3197713},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Intelligent Transportation Systems},
pages = {1-10},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1872.
Shvetsov, Egor; Osin, Dmitry; Zaytsev, Alexey; Koryakovskiy, Ivan; Buchnev, Valentin; Trofimov, Ilya; Burnaev, Evgeny
QuantNAS for super resolution: searching for efficient quantization-friendly architectures against quantization noise Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2208-14839,
title = {QuantNAS for super resolution: searching for efficient quantization-friendly architectures against quantization noise},
author = {Egor Shvetsov and Dmitry Osin and Alexey Zaytsev and Ivan Koryakovskiy and Valentin Buchnev and Ilya Trofimov and Evgeny Burnaev},
url = {https://doi.org/10.48550/arXiv.2208.14839},
doi = {10.48550/arXiv.2208.14839},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2208.14839},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1871.
Cotrim, Lucas P.; Barreira, Rodrigo A.; Santos, Ismael H. F.; Gomi, Edson S.; Costa, Anna Helena Reali; Tannuri, Eduardo Aoun
Neural Network Meta-Models for FPSO Motion Prediction From Environmental Data With Different Platform Loads Journal Article
In: IEEE Access, vol. 10, pp. 86558–86577, 2022.
@article{DBLP:journals/access/CotrimBSGCT22,
title = {Neural Network Meta-Models for FPSO Motion Prediction From Environmental Data With Different Platform Loads},
author = {Lucas P. Cotrim and Rodrigo A. Barreira and Ismael H. F. Santos and Edson S. Gomi and Anna Helena Reali Costa and Eduardo Aoun Tannuri},
url = {https://doi.org/10.1109/ACCESS.2022.3199009},
doi = {10.1109/ACCESS.2022.3199009},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Access},
volume = {10},
pages = {86558--86577},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1870.
Zhang, Li Lyna; Homma, Youkow; Wang, Yujing; Wu, Min; Yang, Mao; Zhang, Ruofei; Cao, Ting; Shen, Wei
SwiftPruner: Reinforced Evolutionary Pruning for Efficient Ad Relevance Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2209-00625,
title = {SwiftPruner: Reinforced Evolutionary Pruning for Efficient Ad Relevance},
author = {Li Lyna Zhang and Youkow Homma and Yujing Wang and Min Wu and Mao Yang and Ruofei Zhang and Ting Cao and Wei Shen},
url = {https://doi.org/10.48550/arXiv.2209.00625},
doi = {10.48550/arXiv.2209.00625},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2209.00625},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1869.
Liang, Jingkang; Liao, Yixiao; Chen, Zhuyun; Lin, Huibin; Jin, Gang; Gryllias, Konstantinos; Li, Weihua
Intelligent fault diagnosis of rotating machinery using lightweight network with modified tree-structured parzen estimators Journal Article
In: IET Collaborative Intelligent Manufacturing, vol. 4, no. 3, pp. 194-207, 2022.
@article{https://doi.org/10.1049/cim2.12055,
title = {Intelligent fault diagnosis of rotating machinery using lightweight network with modified tree-structured parzen estimators},
author = {Jingkang Liang and Yixiao Liao and Zhuyun Chen and Huibin Lin and Gang Jin and Konstantinos Gryllias and Weihua Li},
url = {https://ietresearch.onlinelibrary.wiley.com/doi/abs/10.1049/cim2.12055},
doi = {https://doi.org/10.1049/cim2.12055},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IET Collaborative Intelligent Manufacturing},
volume = {4},
number = {3},
pages = {194-207},
abstract = {Abstract Deep learning-based methods have been widely used in the field of rotating machinery fault diagnosis. It is of practical significance to improve the calculation speed of the model on the premise of ensuring accuracy, so as to realise real-time fault diagnosis. However, designing an efficient and lightweight fault diagnosis network requires expert knowledge to determine the network structure and adjust the hyperparameters of the network, which is time-consuming and laborious. In order to design fault diagnosis networks considering both time and accuracy effortlessly, a novel lightweight network with modified tree-structured parzen estimators (LN-MT) is proposed for intelligent fault diagnosis of rotating machinery. Firstly, a lightweight framework based on global average pooling and group convolution is proposed, and a hyperparameter optimisation (HPO) method based on Bayesian optimisation called tree-structured parzen estimator is utilised to automatically search the optimal hyperparameters for the fault diagnosis task. The objective of the HPO algorithm is the weighting of accuracy and calculating time, so as to find models that balance both time and accuracy. The results of comparison experiments indicate that LN-MT can achieve superior fault diagnosis accuracies with few trainable parameters and less calculating time.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Abstract Deep learning-based methods have been widely used in the field of rotating machinery fault diagnosis. It is of practical significance to improve the calculation speed of the model on the premise of ensuring accuracy, so as to realise real-time fault diagnosis. However, designing an efficient and lightweight fault diagnosis network requires expert knowledge to determine the network structure and adjust the hyperparameters of the network, which is time-consuming and laborious. In order to design fault diagnosis networks considering both time and accuracy effortlessly, a novel lightweight network with modified tree-structured parzen estimators (LN-MT) is proposed for intelligent fault diagnosis of rotating machinery. Firstly, a lightweight framework based on global average pooling and group convolution is proposed, and a hyperparameter optimisation (HPO) method based on Bayesian optimisation called tree-structured parzen estimator is utilised to automatically search the optimal hyperparameters for the fault diagnosis task. The objective of the HPO algorithm is the weighting of accuracy and calculating time, so as to find models that balance both time and accuracy. The results of comparison experiments indicate that LN-MT can achieve superior fault diagnosis accuracies with few trainable parameters and less calculating time.
1868.
Mandal, Murari; Meedimale, Yashwanth Reddy; Reddy, M. Satish Kumar; Vipparthi, Santosh Kumar
Neural Architecture Search for Image Dehazing Journal Article
In: IEEE Transactions on Artificial Intelligence, pp. 1-11, 2022.
@article{9878218,
title = {Neural Architecture Search for Image Dehazing},
author = {Murari Mandal and Yashwanth Reddy Meedimale and M. Satish Kumar Reddy and Santosh Kumar Vipparthi},
url = {https://ieeexplore.ieee.org/abstract/document/9878218},
doi = {10.1109/TAI.2022.3204732},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Artificial Intelligence},
pages = {1-11},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1867.
Zoljodi, Ali; Loni, Mohammad; Abadijou, Sadegh; Alibeigi, Mina; Daneshtalab, Masoud
3DLaneNAS: Neural Architecture Search for Accurate and Light-Weight 3D Lane Detection Proceedings Article
In: Pimenidis, Elias; Angelov, Plamen; Jayne, Chrisina; Papaleonidas, Antonios; Aydin, Mehmet (Ed.): Ärtificial Neural Networks and Machine Learning -- ICANN 2022", pp. 404–415, Springer International Publishing, Cham, 2022, ISBN: 978-3-031-15919-0.
@inproceedings{10.1007/978-3-031-15919-0_34,
title = {3DLaneNAS: Neural Architecture Search for Accurate and Light-Weight 3D Lane Detection},
author = {Ali Zoljodi and Mohammad Loni and Sadegh Abadijou and Mina Alibeigi and Masoud Daneshtalab},
editor = {Elias Pimenidis and Plamen Angelov and Chrisina Jayne and Antonios Papaleonidas and Mehmet Aydin},
isbn = {978-3-031-15919-0},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Ärtificial Neural Networks and Machine Learning -- ICANN 2022"},
pages = {404--415},
publisher = {Springer International Publishing},
address = {Cham},
abstract = {Lane detection is one of the most fundamental tasks for autonomous driving. It plays a crucial role in the lateral control and the precise localization of autonomous vehicles. Monocular 3D lane detection methods provide state-of-the-art results for estimating the position of lanes in 3D world coordinates using only the information obtained from the front-view camera. Recent advances in Neural Architecture Search (NAS) facilitate automated optimization of various computer vision tasks. NAS can automatically optimize monocular 3D lane detection methods to enhance the extraction and combination of visual features, consequently reducing computation loads and increasing accuracy. This paper proposes 3DLaneNAS, a multi-objective method that enhances the accuracy of monocular 3D lane detection for both short- and long-distance scenarios while at the same time providing a fair amount of hardware acceleration. 3DLaneNAS utilizes a new multi-objective energy function to optimize the architecture of feature extraction and feature fusion modules simultaneously. Moreover, a transfer learning mechanism is used to improve the convergence of the search process. Experimental results reveal that 3DLaneNAS yields a minimum of 5.2% higher accuracy and $$backslashapprox $$≈1.33$$backslashtimes $$texttimeslower latency over competing methods on the synthetic-3D-lanes dataset. Code is at https://github.com/alizoljodi/3DLaneNAS},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Lane detection is one of the most fundamental tasks for autonomous driving. It plays a crucial role in the lateral control and the precise localization of autonomous vehicles. Monocular 3D lane detection methods provide state-of-the-art results for estimating the position of lanes in 3D world coordinates using only the information obtained from the front-view camera. Recent advances in Neural Architecture Search (NAS) facilitate automated optimization of various computer vision tasks. NAS can automatically optimize monocular 3D lane detection methods to enhance the extraction and combination of visual features, consequently reducing computation loads and increasing accuracy. This paper proposes 3DLaneNAS, a multi-objective method that enhances the accuracy of monocular 3D lane detection for both short- and long-distance scenarios while at the same time providing a fair amount of hardware acceleration. 3DLaneNAS utilizes a new multi-objective energy function to optimize the architecture of feature extraction and feature fusion modules simultaneously. Moreover, a transfer learning mechanism is used to improve the convergence of the search process. Experimental results reveal that 3DLaneNAS yields a minimum of 5.2% higher accuracy and $$backslashapprox $$≈1.33$$backslashtimes $$texttimeslower latency over competing methods on the synthetic-3D-lanes dataset. Code is at https://github.com/alizoljodi/3DLaneNAS
1866.
Zhang, Pingjian; Liu, Yuankai
NAS4FBP: Facial Beauty Prediction Based on Neural Architecture Search Proceedings Article
In: Pimenidis, Elias; Angelov, Plamen; Jayne, Chrisina; Papaleonidas, Antonios; Aydin, Mehmet (Ed.): Ärtificial Neural Networks and Machine Learning -- ICANN 2022", pp. 225–236, Springer Nature Switzerland, Cham, 2022, ISBN: 978-3-031-15934-3.
@inproceedings{10.1007/978-3-031-15934-3_19,
title = {NAS4FBP: Facial Beauty Prediction Based on Neural Architecture Search},
author = {Pingjian Zhang and Yuankai Liu},
editor = {Elias Pimenidis and Plamen Angelov and Chrisina Jayne and Antonios Papaleonidas and Mehmet Aydin},
isbn = {978-3-031-15934-3},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Ärtificial Neural Networks and Machine Learning -- ICANN 2022"},
pages = {225--236},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {Facial Beauty Prediction (FBP) is an important task in image processing, which simulates human perception of facial beauty. In related studies, most methods are based on canonical convolutional backbones. However, can the canonical backbones perform best in FBP? To tackle this problem, we propose a NAS4FBP framework, which adopts a multi-task neural architecture search strategy to auto determine the backbone structure. In our multi-task learning scheme, we propose HBLoss to better reveal the nature of facial aesthetic hierarchy. In addition, we introduce a new pre-processing method to enhance the data diversity and propose a non-local spatial attention module, to further improve the model performance. Our model achieves 0.9387 PC on the SCUT-FBP5500 benchmark dataset, surpassing other related models and reaching a new state-of-the-art.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Facial Beauty Prediction (FBP) is an important task in image processing, which simulates human perception of facial beauty. In related studies, most methods are based on canonical convolutional backbones. However, can the canonical backbones perform best in FBP? To tackle this problem, we propose a NAS4FBP framework, which adopts a multi-task neural architecture search strategy to auto determine the backbone structure. In our multi-task learning scheme, we propose HBLoss to better reveal the nature of facial aesthetic hierarchy. In addition, we introduce a new pre-processing method to enhance the data diversity and propose a non-local spatial attention module, to further improve the model performance. Our model achieves 0.9387 PC on the SCUT-FBP5500 benchmark dataset, surpassing other related models and reaching a new state-of-the-art.
1865.
Silva, André Ramos Fernandes Da; Pavelski, Lucas Marcondes; Júnior, Luiz Alberto Queiroz Cordovil; Gomes, Paulo Henrique De Oliveira; Azevedo, Layane Menezes; Junior, Francisco Erivaldo Fernandes
An evolutionary search algorithm for efficient ResNet-based architectures: a case study on gender recognition Proceedings Article
In: 2022 IEEE Congress on Evolutionary Computation (CEC), pp. 1-10, 2022.
@inproceedings{9870434,
title = {An evolutionary search algorithm for efficient ResNet-based architectures: a case study on gender recognition},
author = {André Ramos Fernandes Da Silva and Lucas Marcondes Pavelski and Luiz Alberto Queiroz Cordovil Júnior and Paulo Henrique De Oliveira Gomes and Layane Menezes Azevedo and Francisco Erivaldo Fernandes Junior},
url = {https://ieeexplore.ieee.org/abstract/document/9870434},
doi = {10.1109/CEC55065.2022.9870434},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {2022 IEEE Congress on Evolutionary Computation (CEC)},
pages = {1-10},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
1864.
Ivanovic, Milos; Simic, Visnja
Efficient evolutionary optimization using predictive auto-scaling in containerized environment Journal Article
In: Applied Soft Computing, vol. 129, pp. 109610, 2022, ISSN: 1568-4946.
@article{IVANOVIC2022109610,
title = {Efficient evolutionary optimization using predictive auto-scaling in containerized environment},
author = {Milos Ivanovic and Visnja Simic},
url = {https://www.sciencedirect.com/science/article/pii/S1568494622006597},
doi = {https://doi.org/10.1016/j.asoc.2022.109610},
issn = {1568-4946},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Applied Soft Computing},
volume = {129},
pages = {109610},
abstract = {Solving complex real-world optimization problems is a computationally demanding task. To solve it efficiently and effectively, one must possess expert knowledge in various fields (problem domain knowledge, optimization, parallel and distributed computing) and appropriate expensive software and hardware resources. In this regard, we present a cloud-native, container-based distributed optimization framework that enables efficient and cost-effective optimization over platforms such as Amazon ECS/EKS, Azure AKS, and on-premise Kubernetes. The solution consists of dozens of microservices scaled out using a specially developed PETAS Auto-scaler based on predictive analytics. Existing schedulers, whether Kubernetes or commercial, do not take into account the specifics of optimization based on evolutionary algorithms. Therefore, their performance is not optimal in terms of results’ delivery time and cloud infrastructure costs. The proposed PETAS Auto-scaler elastically maintains an adequate number of worker pods following the exact pace dictated by the demands of the optimization process. We evaluate the proposed framework’s performance using two real-world computationally demanding optimizations. The first use case belongs to the manufacturing domain and involves optimization of the transportation pallets for train parts. The second use case belongs to the field of automated machine learning and includes neural architecture search and hyperparameter optimization. The results indicate an IaaS cost savings of up to 49% can be achieved, with almost unchanged result delivery time.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Solving complex real-world optimization problems is a computationally demanding task. To solve it efficiently and effectively, one must possess expert knowledge in various fields (problem domain knowledge, optimization, parallel and distributed computing) and appropriate expensive software and hardware resources. In this regard, we present a cloud-native, container-based distributed optimization framework that enables efficient and cost-effective optimization over platforms such as Amazon ECS/EKS, Azure AKS, and on-premise Kubernetes. The solution consists of dozens of microservices scaled out using a specially developed PETAS Auto-scaler based on predictive analytics. Existing schedulers, whether Kubernetes or commercial, do not take into account the specifics of optimization based on evolutionary algorithms. Therefore, their performance is not optimal in terms of results’ delivery time and cloud infrastructure costs. The proposed PETAS Auto-scaler elastically maintains an adequate number of worker pods following the exact pace dictated by the demands of the optimization process. We evaluate the proposed framework’s performance using two real-world computationally demanding optimizations. The first use case belongs to the manufacturing domain and involves optimization of the transportation pallets for train parts. The second use case belongs to the field of automated machine learning and includes neural architecture search and hyperparameter optimization. The results indicate an IaaS cost savings of up to 49% can be achieved, with almost unchanged result delivery time.
1863.
Huang, Hai; Zhang, Zhikun; Shen, Yun; Backes, Michael; Li, Qi; Zhang, Yang
On the Privacy Risks of Cell-Based NAS Architectures Proceedings Article
In: 022 ACM SIGSAC Conference on Computer and Communications Security, 2022.
@inproceedings{DBLP:journals/corr/abs-2209-01688,
title = {On the Privacy Risks of Cell-Based NAS Architectures},
author = {Hai Huang and Zhikun Zhang and Yun Shen and Michael Backes and Qi Li and Yang Zhang},
url = {https://doi.org/10.48550/arXiv.2209.01688},
doi = {10.48550/arXiv.2209.01688},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {022 ACM SIGSAC Conference on Computer and Communications Security},
journal = {CoRR},
volume = {abs/2209.01688},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
1862.
Rakaraddi, Appan; Lam, Siew Kei; Pratama, Mahardhika; Carvalho, Marcus
Reinforced Continual Learning for Graphs Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2209-01556,
title = {Reinforced Continual Learning for Graphs},
author = {Appan Rakaraddi and Siew Kei Lam and Mahardhika Pratama and Marcus Carvalho},
url = {https://doi.org/10.48550/arXiv.2209.01556},
doi = {10.48550/arXiv.2209.01556},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2209.01556},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1861.
Wang, Lanfei; Xie, Lingxi; Zhao, Kaili; Guo, Jun; Tian, Qi
Regularized Differentiable Architecture Search Journal Article
In: IEEE Embedded Systems Letters, pp. 1-1, 2022.
@article{9878264,
title = {Regularized Differentiable Architecture Search},
author = {Lanfei Wang and Lingxi Xie and Kaili Zhao and Jun Guo and Qi Tian},
url = {https://ieeexplore.ieee.org/abstract/document/9878264},
doi = {10.1109/LES.2022.3204856},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Embedded Systems Letters},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1860.
Al-Sabri, Raeed; Gao, Jianliang; Chen, Jiamin; Oloulade, Babatounde Moctard; Lyu, Tengfei
Multi-View Graph Neural Architecture Search for Biomedical Entity and Relation Extraction Journal Article
In: IEEE/ACM Transactions on Computational Biology and Bioinformatics, pp. 1-13, 2022.
@article{9881878,
title = {Multi-View Graph Neural Architecture Search for Biomedical Entity and Relation Extraction},
author = {Raeed Al-Sabri and Jianliang Gao and Jiamin Chen and Babatounde Moctard Oloulade and Tengfei Lyu},
url = {https://ieeexplore.ieee.org/abstract/document/9881878},
doi = {10.1109/TCBB.2022.3205113},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE/ACM Transactions on Computational Biology and Bioinformatics},
pages = {1-13},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1859.
Wu, Binyi; Waschneck, Bernd; Mayr, Christian
Neural Architecture Search for Low-Precision Neural Networks Proceedings Article
In: Pimenidis, Elias; Angelov, Plamen; Jayne, Chrisina; Papaleonidas, Antonios; Aydin, Mehmet (Ed.): Ärtificial Neural Networks and Machine Learning -- ICANN 2022", pp. 743–755, Springer Nature Switzerland, Cham, 2022, ISBN: 978-3-031-15937-4.
@inproceedings{10.1007/978-3-031-15937-4_62,
title = {Neural Architecture Search for Low-Precision Neural Networks},
author = {Binyi Wu and Bernd Waschneck and Christian Mayr},
editor = {Elias Pimenidis and Plamen Angelov and Chrisina Jayne and Antonios Papaleonidas and Mehmet Aydin},
isbn = {978-3-031-15937-4},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Ärtificial Neural Networks and Machine Learning -- ICANN 2022"},
pages = {743--755},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {In our work, we extend the search space of the differentiable Neural Architecture Search (NAS) by adding bitwidth. The extended NAS algorithm is performed directly with low-precision from scratch without the proxy of full-precision. With our low-precision NAS, we can search for low- and mixed-precision network architectures of Convolutional Neural Networks (CNNs) under specific constraints, such as power consumption. Experiments on the ImageNet dataset demonstrate the effectiveness of our method, where the searched models achieve better accuracy (up to 1.2 percentage point) with smaller model sizes (up to $$27backslash%$$27%smaller) and lower power consumption (up to $$27backslash%$$27%lower) compared to the state-of-art methods. In our low-precision NAS, sharing of convolution is developed to speed up training and decrease memory consumption. Compared to the FBNet-V2 implementation, our solution reduces training time and memory cost by nearly 3$$backslashtimes $$texttimesand 2$$backslashtimes $$texttimes, respectively. Furthermore, we adapt the NAS to train the entire supernet instead of a subnet in each iteration to address the insufficient training issue. Besides, we also propose the forward-and-backward scaling method, which addresses the issue by eliminating the vanishing of the forward activations and backward gradients.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
In our work, we extend the search space of the differentiable Neural Architecture Search (NAS) by adding bitwidth. The extended NAS algorithm is performed directly with low-precision from scratch without the proxy of full-precision. With our low-precision NAS, we can search for low- and mixed-precision network architectures of Convolutional Neural Networks (CNNs) under specific constraints, such as power consumption. Experiments on the ImageNet dataset demonstrate the effectiveness of our method, where the searched models achieve better accuracy (up to 1.2 percentage point) with smaller model sizes (up to $$27backslash%$$27%smaller) and lower power consumption (up to $$27backslash%$$27%lower) compared to the state-of-art methods. In our low-precision NAS, sharing of convolution is developed to speed up training and decrease memory consumption. Compared to the FBNet-V2 implementation, our solution reduces training time and memory cost by nearly 3$$backslashtimes $$texttimesand 2$$backslashtimes $$texttimes, respectively. Furthermore, we adapt the NAS to train the entire supernet instead of a subnet in each iteration to address the insufficient training issue. Besides, we also propose the forward-and-backward scaling method, which addresses the issue by eliminating the vanishing of the forward activations and backward gradients.
1858.
Wang, Yu; Li, Yansheng; Chen, Wei; Li, Yunzhou; Dang, Bo
DNAS: Decoupling Neural Architecture Search for High-Resolution Remote Sensing Image Semantic Segmentation Journal Article
In: Remote Sensing, vol. 14, no. 16, pp. 3864, 2022, (Copyright - © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License; Last updated - 2022-08-25).
@article{nokey,
title = {DNAS: Decoupling Neural Architecture Search for High-Resolution Remote Sensing Image Semantic Segmentation},
author = {Yu Wang and Yansheng Li and Wei Chen and Yunzhou Li and Bo Dang},
url = {https://www.proquest.com/scholarly-journals/dnas-decoupling-neural-architecture-search-high/docview/2706285738/se-2},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Remote Sensing},
volume = {14},
number = {16},
pages = {3864},
abstract = {Deep learning methods, especially deep convolutional neural networks (DCNNs), have been widely used in high-resolution remote sensing image (HRSI) semantic segmentation. In literature, most successful DCNNs are artificially designed through a large number of experiments, which often consume lots of time and depend on rich domain knowledge. Recently, neural architecture search (NAS), as a direction for automatically designing network architectures, has achieved great success in different kinds of computer vision tasks. For HRSI semantic segmentation, NAS faces two major challenges: (1) The task’s high complexity degree, which is caused by the pixel-by-pixel prediction demand in semantic segmentation, leads to a rapid expansion of the search space; (2) HRSI semantic segmentation often needs to exploit long-range dependency (i.e., a large spatial context), which means the NAS technique requires a lot of display memory in the optimization process and can be tough to converge. With the aforementioned considerations in mind, we propose a new decoupling NAS (DNAS) framework to automatically design the network architecture for HRSI semantic segmentation. In DNAS, a hierarchical search space with three levels is recommended: path-level, connection-level, and cell-level. To adapt to this hierarchical search space, we devised a new decoupling search optimization strategy to decrease the memory occupation. More specifically, the search optimization strategy consists of three stages: (1) a light super-net (i.e., the specific search space) in the path-level space is trained to get the optimal path coding; (2) we endowed the optimal path with various cross-layer connections and it is trained to obtain the connection coding; (3) the super-net, which is initialized by path coding and connection coding, is populated with kinds of concrete cell operators and the optimal cell operators are finally determined. It is worth noting that the well-designed search space can cover various network candidates and the optimization process can be done efficiently. Extensive experiments on the publicly open GID and FU datasets showed that our DNAS outperformed the state-of-the-art methods, including artificial networks and NAS methods.},
note = {Copyright - © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License; Last updated - 2022-08-25},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Deep learning methods, especially deep convolutional neural networks (DCNNs), have been widely used in high-resolution remote sensing image (HRSI) semantic segmentation. In literature, most successful DCNNs are artificially designed through a large number of experiments, which often consume lots of time and depend on rich domain knowledge. Recently, neural architecture search (NAS), as a direction for automatically designing network architectures, has achieved great success in different kinds of computer vision tasks. For HRSI semantic segmentation, NAS faces two major challenges: (1) The task’s high complexity degree, which is caused by the pixel-by-pixel prediction demand in semantic segmentation, leads to a rapid expansion of the search space; (2) HRSI semantic segmentation often needs to exploit long-range dependency (i.e., a large spatial context), which means the NAS technique requires a lot of display memory in the optimization process and can be tough to converge. With the aforementioned considerations in mind, we propose a new decoupling NAS (DNAS) framework to automatically design the network architecture for HRSI semantic segmentation. In DNAS, a hierarchical search space with three levels is recommended: path-level, connection-level, and cell-level. To adapt to this hierarchical search space, we devised a new decoupling search optimization strategy to decrease the memory occupation. More specifically, the search optimization strategy consists of three stages: (1) a light super-net (i.e., the specific search space) in the path-level space is trained to get the optimal path coding; (2) we endowed the optimal path with various cross-layer connections and it is trained to obtain the connection coding; (3) the super-net, which is initialized by path coding and connection coding, is populated with kinds of concrete cell operators and the optimal cell operators are finally determined. It is worth noting that the well-designed search space can cover various network candidates and the optimization process can be done efficiently. Extensive experiments on the publicly open GID and FU datasets showed that our DNAS outperformed the state-of-the-art methods, including artificial networks and NAS methods.
1857.
Falanti, Andrea; Lomurno, Eugenio; Samele, Stefano; Ardagna, Danilo; Matteucci, Matteo
POPNASv2: An Efficient Multi-Objective Neural Architecture Search Technique Proceedings Article
In: International Joint Conference on Neural Networks, IJCNN 2022, Padua, Italy, July 18-23, 2022, pp. 1–8, IEEE, 2022.
@inproceedings{DBLP:conf/ijcnn/FalantiLSAM22,
title = {POPNASv2: An Efficient Multi-Objective Neural Architecture Search Technique},
author = {Andrea Falanti and Eugenio Lomurno and Stefano Samele and Danilo Ardagna and Matteo Matteucci},
url = {https://doi.org/10.1109/IJCNN55064.2022.9892073},
doi = {10.1109/IJCNN55064.2022.9892073},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {International Joint Conference on Neural Networks, IJCNN 2022, Padua,
Italy, July 18-23, 2022},
pages = {1--8},
publisher = {IEEE},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
1856.
He, Chunmao; Zhang, Lingyun; Huang, Songqing; Zhang, Pingjian
A Differentiable Architecture Search Approach for Few-Shot Image Classification Proceedings Article
In: Pimenidis, Elias; Angelov, Plamen; Jayne, Chrisina; Papaleonidas, Antonios; Aydin, Mehmet (Ed.): Ärtificial Neural Networks and Machine Learning -- ICANN 2022", pp. 521–532, Springer Nature Switzerland, Cham, 2022, ISBN: 978-3-031-15937-4.
@inproceedings{10.1007/978-3-031-15937-4_44,
title = {A Differentiable Architecture Search Approach for Few-Shot Image Classification},
author = {Chunmao He and Lingyun Zhang and Songqing Huang and Pingjian Zhang},
editor = {Elias Pimenidis and Plamen Angelov and Chrisina Jayne and Antonios Papaleonidas and Mehmet Aydin},
isbn = {978-3-031-15937-4},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Ärtificial Neural Networks and Machine Learning -- ICANN 2022"},
pages = {521--532},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {Few-shot image classification is to learn models to distinguish between unseen categories, even though only a few labeled samples are involved in the training process. To alleviate the over-fitting problem caused by insufficient samples, researchers typically utilize artificially designed simple convolutional neural networks to extract features. However, the feature extraction capability of these networks is not strong enough to extract abstract semantic features, which will affect subsequent feature processing and significantly degrade performance when transferred to other datasets. This paper aims to design a general feature extraction network for few-shot image classification by improving the differentiable architecture search process. We propose a search space regularization method based on DropBlock and an early-stopping strategy based on pooling operation. Through the end-to-end search on the few-shot image dataset CUB, we obtain a light-weighted model FSLNet with excellent generalization ability. In addition, we propose a spatial pyramid self-attention mechanism to optimize the feature expression capability of FSLNet. Experiments show that the FSLNet searched in this paper achieves significant performance. The optimized FSLNet reaches state-of-the-art accuracy on the standard few-shot image classification datasets and in a cross-domain setting.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Few-shot image classification is to learn models to distinguish between unseen categories, even though only a few labeled samples are involved in the training process. To alleviate the over-fitting problem caused by insufficient samples, researchers typically utilize artificially designed simple convolutional neural networks to extract features. However, the feature extraction capability of these networks is not strong enough to extract abstract semantic features, which will affect subsequent feature processing and significantly degrade performance when transferred to other datasets. This paper aims to design a general feature extraction network for few-shot image classification by improving the differentiable architecture search process. We propose a search space regularization method based on DropBlock and an early-stopping strategy based on pooling operation. Through the end-to-end search on the few-shot image dataset CUB, we obtain a light-weighted model FSLNet with excellent generalization ability. In addition, we propose a spatial pyramid self-attention mechanism to optimize the feature expression capability of FSLNet. Experiments show that the FSLNet searched in this paper achieves significant performance. The optimized FSLNet reaches state-of-the-art accuracy on the standard few-shot image classification datasets and in a cross-domain setting.
1855.
Zhao, Yunfeng; Ferguson, Stuart; Zhou, Huiyu; Rafferty, Karen
Representing Camera Response Function by a Single Latent Variable and Fully Connected Neural Network Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2209-03624,
title = {Representing Camera Response Function by a Single Latent Variable and Fully Connected Neural Network},
author = {Yunfeng Zhao and Stuart Ferguson and Huiyu Zhou and Karen Rafferty},
url = {https://doi.org/10.48550/arXiv.2209.03624},
doi = {10.48550/arXiv.2209.03624},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2209.03624},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1854.
Gerum, Christoph; Frischknecht, Adrian; Hald, Tobias; Bernardo, Paul Palomero; Lübeck, Konstantin; Bringmann, Oliver
Hardware Accelerator and Neural Network Co-Optimization for Ultra-Low-Power Audio Processing Devices Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2209-03807,
title = {Hardware Accelerator and Neural Network Co-Optimization for Ultra-Low-Power Audio Processing Devices},
author = {Christoph Gerum and Adrian Frischknecht and Tobias Hald and Paul Palomero Bernardo and Konstantin Lübeck and Oliver Bringmann},
url = {https://doi.org/10.48550/arXiv.2209.03807},
doi = {10.48550/arXiv.2209.03807},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2209.03807},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1853.
Qin, Zidi; Liu, Yang; He, Qing; Ao, Xiang
Explainable Graph-based Fraud Detection via Neural Meta-graph Search Proceedings Article
In: CIKM2022, 2022.
@inproceedings{qin2022explainable,
title = {Explainable Graph-based Fraud Detection via Neural Meta-graph Search},
author = {Zidi Qin and Yang Liu and Qing He and Xiang Ao},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {CIKM2022},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
1852.
Zhang, Hui; Yao, Quanming; Kwok, James T.; Bai, Xiang
Searching a High Performance Feature Extractor for Text Recognition Network Journal Article
In: IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1-15, 2022.
@article{9887897,
title = {Searching a High Performance Feature Extractor for Text Recognition Network},
author = {Hui Zhang and Quanming Yao and James T. Kwok and Xiang Bai},
doi = {10.1109/TPAMI.2022.3205748},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
pages = {1-15},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1851.
Kim, Do-Guk; Lee, Heung-Chang
Proxyless Neural Architecture Adaptation at Once Journal Article
In: IEEE Access, vol. 10, pp. 99745–99753, 2022.
@article{DBLP:journals/access/KimL22f,
title = {Proxyless Neural Architecture Adaptation at Once},
author = {Do-Guk Kim and Heung-Chang Lee},
url = {https://doi.org/10.1109/ACCESS.2022.3206765},
doi = {10.1109/ACCESS.2022.3206765},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Access},
volume = {10},
pages = {99745--99753},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1850.
He, Xin; Ying, Guohao; Zhang, Jiyong; Chu, Xiaowen
Evolutionary Multi-objective Architecture Search Framework: Application to COVID-19 3D CT Classification Proceedings Article
In: Wang, Linwei; Dou, Qi; Fletcher, P. Thomas; Speidel, Stefanie; Li, Shuo (Ed.): Medical Image Computing and Computer Assisted Intervention -- MICCAI 2022, pp. 560–570, Springer Nature Switzerland, Cham, 2022, ISBN: 978-3-031-16431-6.
@inproceedings{10.1007/978-3-031-16431-6_53,
title = {Evolutionary Multi-objective Architecture Search Framework: Application to COVID-19 3D CT Classification},
author = {Xin He and Guohao Ying and Jiyong Zhang and Xiaowen Chu},
editor = {Linwei Wang and Qi Dou and P. Thomas Fletcher and Stefanie Speidel and Shuo Li},
isbn = {978-3-031-16431-6},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2022},
pages = {560--570},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {The COVID-19 pandemic has threatened global health. Many studies have applied deep convolutional neural networks (CNN) to recognize COVID-19 based on chest 3D computed tomography (CT). Recent works show that no model generalizes well across CT datasets from different countries, and manually designing models for specific datasets requires expertise; thus, neural architecture search (NAS) that aims to search models automatically has become an attractive solution. To reduce the search cost on large 3D CT datasets, most NAS-based works use the weight-sharing (WS) strategy to make all models share weights within a supernet; however, WS inevitably incurs search instability, leading to inaccurate model estimation. In this work, we propose an efficient Evolutionary Multi-objective ARchitecture Search (EMARS) framework. We propose a new objective, namely potential, which can help exploit promising models to indirectly reduce the number of models involved in weights training, thus alleviating search instability. We demonstrate that under objectives of accuracy and potential, EMARS can balance exploitation and exploration, i.e., reducing search time and finding better models. Our searched models are small and perform better than prior works on three public COVID-19 3D CT datasets.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
The COVID-19 pandemic has threatened global health. Many studies have applied deep convolutional neural networks (CNN) to recognize COVID-19 based on chest 3D computed tomography (CT). Recent works show that no model generalizes well across CT datasets from different countries, and manually designing models for specific datasets requires expertise; thus, neural architecture search (NAS) that aims to search models automatically has become an attractive solution. To reduce the search cost on large 3D CT datasets, most NAS-based works use the weight-sharing (WS) strategy to make all models share weights within a supernet; however, WS inevitably incurs search instability, leading to inaccurate model estimation. In this work, we propose an efficient Evolutionary Multi-objective ARchitecture Search (EMARS) framework. We propose a new objective, namely potential, which can help exploit promising models to indirectly reduce the number of models involved in weights training, thus alleviating search instability. We demonstrate that under objectives of accuracy and potential, EMARS can balance exploitation and exploration, i.e., reducing search time and finding better models. Our searched models are small and perform better than prior works on three public COVID-19 3D CT datasets.
1849.
Akhauri, Yash; Munoz, J. Pablo; Jain, Nilesh; Iyer, Ravi
Evolving Zero Cost Proxies For Neural Architecture Scoring Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2209-07413,
title = {Evolving Zero Cost Proxies For Neural Architecture Scoring},
author = {Yash Akhauri and J. Pablo Munoz and Nilesh Jain and Ravi Iyer},
url = {https://doi.org/10.48550/arXiv.2209.07413},
doi = {10.48550/arXiv.2209.07413},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2209.07413},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1848.
Hakim, Tal
NAAP-440 Dataset and Baseline for Neural Architecture Accuracy Prediction Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2209-06626,
title = {NAAP-440 Dataset and Baseline for Neural Architecture Accuracy Prediction},
author = {Tal Hakim},
url = {https://doi.org/10.48550/arXiv.2209.06626},
doi = {10.48550/arXiv.2209.06626},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2209.06626},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1847.
Zhu, Zhenyu; Liu, Fanghui; Chrysos, Grigorios G.; Cevher, Volkan
Generalization Properties of NAS under Activation and Skip Connection Search Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2209-07238,
title = {Generalization Properties of NAS under Activation and Skip Connection Search},
author = {Zhenyu Zhu and Fanghui Liu and Grigorios G. Chrysos and Volkan Cevher},
url = {https://doi.org/10.48550/arXiv.2209.07238},
doi = {10.48550/arXiv.2209.07238},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2209.07238},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1846.
Hu, Xiaobin; Shen, Ruolin; Luo, Donghao; Tai, Ying; Wang, Chengjie; Menze, Bjoern H.
AutoGAN-Synthesizer: Neural Architecture Search for Cross-Modality MRI Synthesis Proceedings Article
In: Wang, Linwei; Dou, Qi; Fletcher, P. Thomas; Speidel, Stefanie; Li, Shuo (Ed.): Medical Image Computing and Computer Assisted Intervention -- MICCAI 2022, pp. 397–409, Springer Nature Switzerland, Cham, 2022, ISBN: 978-3-031-16446-0.
@inproceedings{10.1007/978-3-031-16446-0_38,
title = {AutoGAN-Synthesizer: Neural Architecture Search for Cross-Modality MRI Synthesis},
author = {Xiaobin Hu and Ruolin Shen and Donghao Luo and Ying Tai and Chengjie Wang and Bjoern H. Menze},
editor = {Linwei Wang and Qi Dou and P. Thomas Fletcher and Stefanie Speidel and Shuo Li},
url = {https://link.springer.com/chapter/10.1007/978-3-031-16446-0_38},
isbn = {978-3-031-16446-0},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Medical Image Computing and Computer Assisted Intervention -- MICCAI 2022},
pages = {397--409},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {Considering the difficulty to obtain complete multi-modality MRI scans in some real-world data acquisition situations, synthesizing MRI data is a highly relevant and important topic to complement diagnosis information in clinical practice. In this study, we present a novel MRI synthesizer, called AutoGAN-Synthesizer, which automatically discovers generative networks for cross-modality MRI synthesis. Our AutoGAN-Synthesizer adopts gradient-based search strategies to explore the generator architecture by determining how to fuse multi-resolution features and utilizes GAN-based perceptual searching losses to handle the trade-off between model complexity and performance. Our AutoGAN-Synthesizer can search for a remarkable and light-weight architecture with 6.31 Mb parameters only occupying 12 GPU hours. Moreover, to incorporate richer prior knowledge for MRI synthesis, we derive K-space features containing the low- and high-spatial frequency information and incorporate such features into our model. To our best knowledge, this is the first work to explore AutoML for cross-modality MRI synthesis, and our approach is also capable of tailoring networks given either different multiple modalities or just a single modality as input. Extensive experiments show that our AutoGAN-Synthesizer outperforms the state-of-the-art MRI synthesis methods both quantitatively and qualitatively. The code are available at https://github.com/HUuxiaobin/AutoGAN-Synthesizer.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Considering the difficulty to obtain complete multi-modality MRI scans in some real-world data acquisition situations, synthesizing MRI data is a highly relevant and important topic to complement diagnosis information in clinical practice. In this study, we present a novel MRI synthesizer, called AutoGAN-Synthesizer, which automatically discovers generative networks for cross-modality MRI synthesis. Our AutoGAN-Synthesizer adopts gradient-based search strategies to explore the generator architecture by determining how to fuse multi-resolution features and utilizes GAN-based perceptual searching losses to handle the trade-off between model complexity and performance. Our AutoGAN-Synthesizer can search for a remarkable and light-weight architecture with 6.31 Mb parameters only occupying 12 GPU hours. Moreover, to incorporate richer prior knowledge for MRI synthesis, we derive K-space features containing the low- and high-spatial frequency information and incorporate such features into our model. To our best knowledge, this is the first work to explore AutoML for cross-modality MRI synthesis, and our approach is also capable of tailoring networks given either different multiple modalities or just a single modality as input. Extensive experiments show that our AutoGAN-Synthesizer outperforms the state-of-the-art MRI synthesis methods both quantitatively and qualitatively. The code are available at https://github.com/HUuxiaobin/AutoGAN-Synthesizer.
1845.
Luo, Xiangzhong; Liu, Di; Kong, Hao; Huai, Shuo; Chen, Hui; Liu, Weichen
LightNAS: On Lightweight and Scalable Neural Architecture Search for Embedded Platforms Journal Article
In: IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, pp. 1-1, 2022.
@article{9896156,
title = {LightNAS: On Lightweight and Scalable Neural Architecture Search for Embedded Platforms},
author = {Xiangzhong Luo and Di Liu and Hao Kong and Shuo Huai and Hui Chen and Weichen Liu},
url = {https://ieeexplore.ieee.org/abstract/document/9896156},
doi = {10.1109/TCAD.2022.3208187},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1844.
Peng, Hongwu; Zhou, Shanglin; Luo, Yukui; Duan, Shijin; Xu, Nuo; Ran, Ran; Huang, Shaoyi; Wang, Chenghong; Geng, Tong; Li, Ang; Wen, Wujie; Xu, Xiaolin; Ding, Caiwen
PolyMPCNet: Towards ReLU-free Neural Architecture Search in Two-party Computation Based Private Inference Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2209-09424,
title = {PolyMPCNet: Towards ReLU-free Neural Architecture Search in Two-party Computation Based Private Inference},
author = {Hongwu Peng and Shanglin Zhou and Yukui Luo and Shijin Duan and Nuo Xu and Ran Ran and Shaoyi Huang and Chenghong Wang and Tong Geng and Ang Li and Wujie Wen and Xiaolin Xu and Caiwen Ding},
url = {https://doi.org/10.48550/arXiv.2209.09424},
doi = {10.48550/arXiv.2209.09424},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2209.09424},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1843.
Minglong, Xue; Jian, Li; Qingli, Luo
Height Estimation from a Single SAR Image by Depth-Aware Proxyless Neural Architecture Search Proceedings Article
In: 2022 7th International Conference on Signal and Image Processing (ICSIP), pp. 632-636, 2022.
@inproceedings{9886737,
title = {Height Estimation from a Single SAR Image by Depth-Aware Proxyless Neural Architecture Search},
author = {Xue Minglong and Li Jian and Luo Qingli},
doi = {10.1109/ICSIP55141.2022.9886737},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {2022 7th International Conference on Signal and Image Processing (ICSIP)},
pages = {632-636},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
1842.
Phan, Quan Minh; Luong, Ngoc Hoang
TF-MOPNAS: Training-free Multi-objective Pruning-Based Neural Architecture Search Proceedings Article
In: Nguyen, Ngoc Thanh; Manolopoulos, Yannis; Chbeir, Richard; Kozierkiewicz, Adrianna; Trawiński, Bogdan (Ed.): Computational Collective Intelligence, pp. 297–310, Springer International Publishing, Cham, 2022, ISBN: 978-3-031-16014-1.
@inproceedings{10.1007/978-3-031-16014-1_24,
title = {TF-MOPNAS: Training-free Multi-objective Pruning-Based Neural Architecture Search},
author = {Quan Minh Phan and Ngoc Hoang Luong},
editor = {Ngoc Thanh Nguyen and Yannis Manolopoulos and Richard Chbeir and Adrianna Kozierkiewicz and Bogdan Trawiński},
isbn = {978-3-031-16014-1},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Computational Collective Intelligence},
pages = {297--310},
publisher = {Springer International Publishing},
address = {Cham},
abstract = {Pruning-based neural architecture search (NAS) methods are effective approaches in finding network architectures that have high performance with low complexity. However, current methods only yield a single final architecture instead of an approximation Pareto set, which is typically the desirable result of solving multi-objective problems. Furthermore, the network performance evaluation in NAS involves the computationally expensive network training process, and the search cost thus considerably increases because numerous architectures are evaluated during an NAS run. Using computational resource efficiently, therefore, is an essential problem that needs to be considered. Recent studies have attempted to address this resource issue by replacing the network accuracy metric in NAS optimization objectives with so-called training-free performance metrics, which can be calculated without requiring any training epoch. In this paper, we propose a training-free multi-objective pruning-based neural architecture search (TF-MOPNAS) framework that produces competitive trade-off fronts for multi-objective NAS with a trivial cost by using the Synaptic Flow metric. We test our proposed method on multi-objective NAS problems created on a wide range of well-known NAS benchmarks, i.e., NAS-Bench-101, NAS-Bench-1shot1, and NAS-Bench-201. Experimental results indicate that our method can figure out trade-off fronts that have the equivalent quality to the ones found by state-of-the-art NAS methods but with much less computation resource. The code is available at: https://github.com/ELO-Lab/TF-MOPNAS.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Pruning-based neural architecture search (NAS) methods are effective approaches in finding network architectures that have high performance with low complexity. However, current methods only yield a single final architecture instead of an approximation Pareto set, which is typically the desirable result of solving multi-objective problems. Furthermore, the network performance evaluation in NAS involves the computationally expensive network training process, and the search cost thus considerably increases because numerous architectures are evaluated during an NAS run. Using computational resource efficiently, therefore, is an essential problem that needs to be considered. Recent studies have attempted to address this resource issue by replacing the network accuracy metric in NAS optimization objectives with so-called training-free performance metrics, which can be calculated without requiring any training epoch. In this paper, we propose a training-free multi-objective pruning-based neural architecture search (TF-MOPNAS) framework that produces competitive trade-off fronts for multi-objective NAS with a trivial cost by using the Synaptic Flow metric. We test our proposed method on multi-objective NAS problems created on a wide range of well-known NAS benchmarks, i.e., NAS-Bench-101, NAS-Bench-1shot1, and NAS-Bench-201. Experimental results indicate that our method can figure out trade-off fronts that have the equivalent quality to the ones found by state-of-the-art NAS methods but with much less computation resource. The code is available at: https://github.com/ELO-Lab/TF-MOPNAS.
1841.
Eslami, Saeedeh; Monsefi, Reza; Akbari, Mohammad
Learning a Unified Latent Space for NAS: Toward Leveraging Structural and Symbolic Information Journal Article
In: IEEE Access, vol. 10, pp. 102945–102956, 2022.
@article{DBLP:journals/access/EslamiMA22,
title = {Learning a Unified Latent Space for NAS: Toward Leveraging Structural and Symbolic Information},
author = {Saeedeh Eslami and Reza Monsefi and Mohammad Akbari},
url = {https://doi.org/10.1109/ACCESS.2022.3208591},
doi = {10.1109/ACCESS.2022.3208591},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Access},
volume = {10},
pages = {102945--102956},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1840.
Louati, Hassen; Louati, Ali; Bechikh, Slim; Said, Lamjed Ben
Design and Compression Study for Convolutional Neural Networks Based on Evolutionary Optimization for Thoracic X-Ray Image Classification Proceedings Article
In: Nguyen, Ngoc Thanh; Manolopoulos, Yannis; Chbeir, Richard; Kozierkiewicz, Adrianna; Trawiński, Bogdan (Ed.): Computational Collective Intelligence, pp. 283–296, Springer International Publishing, Cham, 2022, ISBN: 978-3-031-16014-1.
@inproceedings{10.1007/978-3-031-16014-1_23,
title = {Design and Compression Study for Convolutional Neural Networks Based on Evolutionary Optimization for Thoracic X-Ray Image Classification},
author = {Hassen Louati and Ali Louati and Slim Bechikh and Lamjed Ben Said},
editor = {Ngoc Thanh Nguyen and Yannis Manolopoulos and Richard Chbeir and Adrianna Kozierkiewicz and Bogdan Trawiński},
isbn = {978-3-031-16014-1},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Computational Collective Intelligence},
pages = {283--296},
publisher = {Springer International Publishing},
address = {Cham},
abstract = {Computer Vision has lately shown progress in addressing a variety of complex health care difficulties and has the potential to aid in the battle against certain lung illnesses, including COVID-19. Indeed, chest X-rays are one of the most commonly performed radiological techniques for diagnosing a range of lung diseases. Therefore, deep learning researchers have suggested that computer-aided diagnostic systems be built using deep learning methods. In fact, there are several CNN structures described in the literature. However, there are no guidelines for designing and compressing a specific architecture for a specific purpose; thus, such design remains highly subjective and heavily dependent on data scientists' knowledge and expertise. While deep convolutional neural networks have lately shown their ability to perform well in classification and dimension reduction tasks, the challenge of parameter selection is critical for these networks. However, since a CNN has a high number of parameters, its implementation in storage devices is difficult. This is due to the fact that the search space grows exponentially in size as the number of layers increases, and the large number of parameters necessitates extensive computation and storage, making it impractical for use on low-capacity devices. Motivated by these observations, we propose an automated method for CNN design and compression based on an evolutionary algorithm (EA) for X-Ray image classification that is capable of classifying radiography images and detecting possible chest abnormalities and infections, including COVID-19.Our evolutionary method is validated through a series of comparative experiments against relevant state-of-the-art architectures.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Computer Vision has lately shown progress in addressing a variety of complex health care difficulties and has the potential to aid in the battle against certain lung illnesses, including COVID-19. Indeed, chest X-rays are one of the most commonly performed radiological techniques for diagnosing a range of lung diseases. Therefore, deep learning researchers have suggested that computer-aided diagnostic systems be built using deep learning methods. In fact, there are several CNN structures described in the literature. However, there are no guidelines for designing and compressing a specific architecture for a specific purpose; thus, such design remains highly subjective and heavily dependent on data scientists' knowledge and expertise. While deep convolutional neural networks have lately shown their ability to perform well in classification and dimension reduction tasks, the challenge of parameter selection is critical for these networks. However, since a CNN has a high number of parameters, its implementation in storage devices is difficult. This is due to the fact that the search space grows exponentially in size as the number of layers increases, and the large number of parameters necessitates extensive computation and storage, making it impractical for use on low-capacity devices. Motivated by these observations, we propose an automated method for CNN design and compression based on an evolutionary algorithm (EA) for X-Ray image classification that is capable of classifying radiography images and detecting possible chest abnormalities and infections, including COVID-19.Our evolutionary method is validated through a series of comparative experiments against relevant state-of-the-art architectures.