Maintained by Difan Deng and Marius Lindauer.
The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.
Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.
Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.
2022
2230.
Zhang, Xuelin; Cheng, Xuelian; Zhang, Donghao; Bonnington, Paul; Ge, Zongyuan
Learning Network Architecture for Open-set Recognition Proceedings Article
In: AAAI, 2022.
@inproceedings{ZhangOpenset2022,
title = {Learning Network Architecture for Open-set Recognition},
author = {Xuelin Zhang and Xuelian Cheng and Donghao Zhang and Paul Bonnington and Zongyuan Ge},
url = {https://www.aaai.org/AAAI22Papers/AAAI-12763.ZhangX.pdf},
year = {2022},
date = {2022-02-01},
urldate = {2022-02-01},
booktitle = {AAAI},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2229.
Hu, Zhonggang; Bao, Wenxing; Qu, Kewen; Liang, Hongbo
Image-based neural architecture automatic search method for hyperspectral image classification Journal Article
In: J. of Applied Remote Sensing, vol. 16, 2022.
@article{Hu2022,
title = {Image-based neural architecture automatic search method for hyperspectral image classification},
author = {Zhonggang Hu and Wenxing Bao and Kewen Qu and Hongbo Liang
},
url = {https://www.spiedigitallibrary.org/journals/journal-of-applied-remote-sensing/volume-16/issue-01/016501/Image-based-neural-architecture-automatic-search-method-for-hyperspectral-image/10.1117/1.JRS.16.016501.full?tab=ArticleLinkFigureTable},
year = {2022},
date = {2022-01-06},
journal = {J. of Applied Remote Sensing},
volume = {16},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2228.
Diao, Huabin; Li, Gongyan; Xu, Shaoyun; Hao, Yuexing
PA-NAS: Partial operation activation for memory-efficient architecture search Journal Article
In: Applied Intelligence, 2022.
@article{DiaoPANAS2022,
title = {PA-NAS: Partial operation activation for memory-efficient architecture search},
author = {Huabin Diao and Gongyan Li and Shaoyun Xu and Yuexing Hao},
url = {https://link.springer.com/article/10.1007/s10489-021-02961-8},
year = {2022},
date = {2022-01-05},
urldate = {2022-01-05},
journal = {Applied Intelligence},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2227.
Klos, Andreas; Rosenbaum, Marius; Schiffmann, Wolfram
Neural Architecture Search based on Genetic Algorithm and Deployed in a Bare-Metal Kubernetes Cluster Journal Article
In: International Journal of Networking and Computing, vol. 12, no. 1, 2022.
@article{Klos2022,
title = {Neural Architecture Search based on Genetic Algorithm and Deployed in a Bare-Metal Kubernetes Cluster},
author = {Andreas Klos and Marius Rosenbaum and Wolfram Schiffmann},
url = {http://www.ijnc.org/index.php/ijnc/article/view/276},
year = {2022},
date = {2022-01-03},
urldate = {2022-01-03},
journal = {International Journal of Networking and Computing},
volume = {12},
number = {1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2226.
Fei, Ke; Li, Qi; Zhu, Congcong
Non-technical losses detection using missing values’ pattern and neural architecture search Journal Article
In: International Journal of Electrical Power & Energy Systems, vol. 134, pp. 107410, 2022, ISSN: 0142-0615.
@article{FEI2022107410,
title = {Non-technical losses detection using missing values’ pattern and neural architecture search},
author = {Ke Fei and Qi Li and Congcong Zhu},
url = {https://www.sciencedirect.com/science/article/pii/S0142061521006499},
doi = {https://doi.org/10.1016/j.ijepes.2021.107410},
issn = {0142-0615},
year = {2022},
date = {2022-01-01},
journal = {International Journal of Electrical Power & Energy Systems},
volume = {134},
pages = {107410},
abstract = {The fast growth of non-technical loss (NTL) has gradually become one of the main concerns for distribution network operators (DNOs). Electricity theft which constitutes the main part of NTL not only brings losses to the DNOs, but also reduces the quality of the supply. A traditional detection method relies on utility workers’ experience and consumes a large amount of manpower. Thanks to the emerging of advanced metering infrastructure (AMI), utility companies can now collect detailed data reflecting consumers’ electricity usage, which enabled algorithms-based non-technical loss detection. The current data-based methods focus on the characteristics of electricity consumption thereby less efficient when dealing with rapidly changed electricity theft techniques. This article introduced a new data set, the location information of missing values, to improve the accuracy of non-technical loss detection. The relationship between missing values and electricity theft techniques is analyzed and a neural network model is built through neural architecture search (NAS). The improved model achieved an excellent Area Under Curve (AUC) value around 0.926 which verified the close link between missing values and electricity theft techniques. The nature of neural architecture search allows automatic model update which makes it a user-friendly tool even for engineers without any neural network expertise. A case study was carried out in which the missing value pattern was analyzed through Density-Based Spatial Clustering of Applications with Noise (DBSCAN) clustering algorithm.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The fast growth of non-technical loss (NTL) has gradually become one of the main concerns for distribution network operators (DNOs). Electricity theft which constitutes the main part of NTL not only brings losses to the DNOs, but also reduces the quality of the supply. A traditional detection method relies on utility workers’ experience and consumes a large amount of manpower. Thanks to the emerging of advanced metering infrastructure (AMI), utility companies can now collect detailed data reflecting consumers’ electricity usage, which enabled algorithms-based non-technical loss detection. The current data-based methods focus on the characteristics of electricity consumption thereby less efficient when dealing with rapidly changed electricity theft techniques. This article introduced a new data set, the location information of missing values, to improve the accuracy of non-technical loss detection. The relationship between missing values and electricity theft techniques is analyzed and a neural network model is built through neural architecture search (NAS). The improved model achieved an excellent Area Under Curve (AUC) value around 0.926 which verified the close link between missing values and electricity theft techniques. The nature of neural architecture search allows automatic model update which makes it a user-friendly tool even for engineers without any neural network expertise. A case study was carried out in which the missing value pattern was analyzed through Density-Based Spatial Clustering of Applications with Noise (DBSCAN) clustering algorithm.
2225.
Nistor, Sergiu Cosmin; Czibula, Gabriela
IntelliSwAS: Optimizing deep neural network architectures using a particle swarm-based approach Journal Article
In: Expert Systems with Applications, vol. 187, pp. 115945, 2022, ISSN: 0957-4174.
@article{NISTOR2022115945,
title = {IntelliSwAS: Optimizing deep neural network architectures using a particle swarm-based approach},
author = {Sergiu Cosmin Nistor and Gabriela Czibula},
url = {https://www.sciencedirect.com/science/article/pii/S0957417421012987},
doi = {https://doi.org/10.1016/j.eswa.2021.115945},
issn = {0957-4174},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Expert Systems with Applications},
volume = {187},
pages = {115945},
abstract = {Deep learning is in a continuous evolution and many domains benefit from this substantial progress in the development of intelligent solutions. While this progress has been swift, there are more and more applications and the specific requirements of each application domain entails much work done by researchers to adapt existing models or create new ones. This traditional approach has produced many successful designs, but recently, automated methods of finding neural network architectures emerged. We introduce IntelliSwAS approach for optimizing deep neural network architectures for a classification or regression task. Image classification was selected as the task, but IntelliSwAS is generic enough such that it could be successfully applied on other classification or regression problems. A particle swarm-based optimization algorithm is proposed for the automatic search for convolutional neural network architectures. The search technique is enhanced by a machine learning model (DAGRNN) which we designed for predicting the quality of the network architectures and thus increasing the performance of the algorithm. The proposed model is able to process data structured as directed acyclic graphs in general and we applied it specifically on network architectures. The network architecture that we discovered using IntelliSwAS surpassed 89.8% of the competing image classification models that we considered for comparison.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Deep learning is in a continuous evolution and many domains benefit from this substantial progress in the development of intelligent solutions. While this progress has been swift, there are more and more applications and the specific requirements of each application domain entails much work done by researchers to adapt existing models or create new ones. This traditional approach has produced many successful designs, but recently, automated methods of finding neural network architectures emerged. We introduce IntelliSwAS approach for optimizing deep neural network architectures for a classification or regression task. Image classification was selected as the task, but IntelliSwAS is generic enough such that it could be successfully applied on other classification or regression problems. A particle swarm-based optimization algorithm is proposed for the automatic search for convolutional neural network architectures. The search technique is enhanced by a machine learning model (DAGRNN) which we designed for predicting the quality of the network architectures and thus increasing the performance of the algorithm. The proposed model is able to process data structured as directed acyclic graphs in general and we applied it specifically on network architectures. The network architecture that we discovered using IntelliSwAS surpassed 89.8% of the competing image classification models that we considered for comparison.
2224.
Chen, Haoze; Zhang, Zhijie; Jin, Mingwu; Wang, Fengxiang
Prediction of dMRI signals with neural architecture search Journal Article
In: Journal of Neuroscience Methods, vol. 365, pp. 109389, 2022, ISSN: 0165-0270.
@article{CHEN2022109389,
title = {Prediction of dMRI signals with neural architecture search},
author = {Haoze Chen and Zhijie Zhang and Mingwu Jin and Fengxiang Wang},
url = {https://www.sciencedirect.com/science/article/pii/S0165027021003241},
doi = {https://doi.org/10.1016/j.jneumeth.2021.109389},
issn = {0165-0270},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Journal of Neuroscience Methods},
volume = {365},
pages = {109389},
abstract = {Background
There is growing interest in the neuroscience community in estimating and mapping microscopic properties of brain tissue non-invasively using magnetic resonance measurements. Machine learning methods are actively investigated to predict the signals measured in diffusion magnetic resonance imaging (dMRI).
New method
We applied the neural architecture search (NAS) to train a recurrent neural network to generate a multilayer perceptron to predict the dMRI data of unknown signals based on the different acquisition parameters and training data. The search space of NAS is the number of neurons in each layer of the multilayer perceptron network. To our best knowledge, this is the first time to apply NAS to solve the dMRI signal prediction problem.
Results
The experimental results demonstrate that the proposed NAS method can achieve fast training and predict dMRI signals accurately. For dMRI signals with four acquisition strategies of double diffusion encoding (DDE), double oscillating diffusion encoding (DODE), multi-shell and DSI-like pulsed gradient spin-echo (PGSE), the mean squared errors of the multilayer perceptron network designed by NAS are 0.0043, 0.0034, 0.0147 and 0.0199, respectively.
Comparison with existing method(s)
We also compared NAS with other machine learning prediction methods, such as support vector regression (SVR), decision tree (DT) and random forest (RF), k-nearest neighbors (KNN), adaboost regressor (AR), gradient boosting regressor (GBR) and extra-trees regressor (ET). NAS achieved the better prediction performance in most cases.
Conclusion
In this study, NAS was developed for the prediction of dMRI signals and could become an effective prediction tool.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Background
There is growing interest in the neuroscience community in estimating and mapping microscopic properties of brain tissue non-invasively using magnetic resonance measurements. Machine learning methods are actively investigated to predict the signals measured in diffusion magnetic resonance imaging (dMRI).
New method
We applied the neural architecture search (NAS) to train a recurrent neural network to generate a multilayer perceptron to predict the dMRI data of unknown signals based on the different acquisition parameters and training data. The search space of NAS is the number of neurons in each layer of the multilayer perceptron network. To our best knowledge, this is the first time to apply NAS to solve the dMRI signal prediction problem.
Results
The experimental results demonstrate that the proposed NAS method can achieve fast training and predict dMRI signals accurately. For dMRI signals with four acquisition strategies of double diffusion encoding (DDE), double oscillating diffusion encoding (DODE), multi-shell and DSI-like pulsed gradient spin-echo (PGSE), the mean squared errors of the multilayer perceptron network designed by NAS are 0.0043, 0.0034, 0.0147 and 0.0199, respectively.
Comparison with existing method(s)
We also compared NAS with other machine learning prediction methods, such as support vector regression (SVR), decision tree (DT) and random forest (RF), k-nearest neighbors (KNN), adaboost regressor (AR), gradient boosting regressor (GBR) and extra-trees regressor (ET). NAS achieved the better prediction performance in most cases.
Conclusion
In this study, NAS was developed for the prediction of dMRI signals and could become an effective prediction tool.
There is growing interest in the neuroscience community in estimating and mapping microscopic properties of brain tissue non-invasively using magnetic resonance measurements. Machine learning methods are actively investigated to predict the signals measured in diffusion magnetic resonance imaging (dMRI).
New method
We applied the neural architecture search (NAS) to train a recurrent neural network to generate a multilayer perceptron to predict the dMRI data of unknown signals based on the different acquisition parameters and training data. The search space of NAS is the number of neurons in each layer of the multilayer perceptron network. To our best knowledge, this is the first time to apply NAS to solve the dMRI signal prediction problem.
Results
The experimental results demonstrate that the proposed NAS method can achieve fast training and predict dMRI signals accurately. For dMRI signals with four acquisition strategies of double diffusion encoding (DDE), double oscillating diffusion encoding (DODE), multi-shell and DSI-like pulsed gradient spin-echo (PGSE), the mean squared errors of the multilayer perceptron network designed by NAS are 0.0043, 0.0034, 0.0147 and 0.0199, respectively.
Comparison with existing method(s)
We also compared NAS with other machine learning prediction methods, such as support vector regression (SVR), decision tree (DT) and random forest (RF), k-nearest neighbors (KNN), adaboost regressor (AR), gradient boosting regressor (GBR) and extra-trees regressor (ET). NAS achieved the better prediction performance in most cases.
Conclusion
In this study, NAS was developed for the prediction of dMRI signals and could become an effective prediction tool.
2223.
Cheng, Guoan; Matsune, Ai; Du, Hao; Liu, XinZhi; Zhan, Shu
Exploring more diverse network architectures for single image super-resolution Journal Article
In: Knowledge-Based Systems, vol. 235, pp. 107648, 2022, ISSN: 0950-7051.
@article{CHENG2022107648,
title = {Exploring more diverse network architectures for single image super-resolution},
author = {Guoan Cheng and Ai Matsune and Hao Du and XinZhi Liu and Shu Zhan},
url = {https://www.sciencedirect.com/science/article/pii/S0950705121009102},
doi = {https://doi.org/10.1016/j.knosys.2021.107648},
issn = {0950-7051},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Knowledge-Based Systems},
volume = {235},
pages = {107648},
abstract = {We propose a plug-and-play neural architecture search (NAS) method to explore diverse architectures for single image super-resolution (SISR). Unlike current NAS-based methods with the single path setting and pipeline setting, our proposed method achieves the trade-off between diverse network architectures and search cost. Our proposed method formulates the task in a differentiable manner, which inherits the architecture parameter optimization method from Discrete Stochastic Neural Architecture Search (DSNAS). Besides the straightforward searching of operations, we also search each node in a cell for the activation function, from-node, and skip-connection node, which diverse the searched architecture topologies. The individually searching of skip-connection node avoids skip-connection excessive phenomenon. Moreover, to alleviate the influence of inconsistent architecture between training and testing periods, we introduce random variables into the architecture parameter as regularization. Benchmark experiments show our state-of-the-art performance under specific parameters and FLOPs constraints. Compared with other NAS-based SISR methods, our proposed methods achieve better performance with less searching time and resources. The superior results further demonstrate the effectiveness of our proposed NAS methods.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
We propose a plug-and-play neural architecture search (NAS) method to explore diverse architectures for single image super-resolution (SISR). Unlike current NAS-based methods with the single path setting and pipeline setting, our proposed method achieves the trade-off between diverse network architectures and search cost. Our proposed method formulates the task in a differentiable manner, which inherits the architecture parameter optimization method from Discrete Stochastic Neural Architecture Search (DSNAS). Besides the straightforward searching of operations, we also search each node in a cell for the activation function, from-node, and skip-connection node, which diverse the searched architecture topologies. The individually searching of skip-connection node avoids skip-connection excessive phenomenon. Moreover, to alleviate the influence of inconsistent architecture between training and testing periods, we introduce random variables into the architecture parameter as regularization. Benchmark experiments show our state-of-the-art performance under specific parameters and FLOPs constraints. Compared with other NAS-based SISR methods, our proposed methods achieve better performance with less searching time and resources. The superior results further demonstrate the effectiveness of our proposed NAS methods.
2222.
Grahn, Lea; Rachner, Jonas; Göppert, Amon; Saeed, Sazvan; Schmitt, Robert H.
Framework for Potential Analysis by Approximating Line-Less Assembly Systems with AutoML Proceedings Article
In: Ändersen, Ann-Louise; Andersen, Rasmus; Brunoe, Thomas Ditlev; Larsen, Maria Stoettrup Schioenning; Nielsen, Kjeld; Napoleone, Alessia; Kjeldgaard, Stefan" (Ed.): Towards Sustainable Customization: Bridging Smart Products and Manufacturing Systems, pp. 423–430, Springer International Publishing, Cham, 2022, ISBN: 978-3-030-90700-6.
@inproceedings{10.1007/978-3-030-90700-6_48,
title = {Framework for Potential Analysis by Approximating Line-Less Assembly Systems with AutoML},
author = {Lea Grahn and Jonas Rachner and Amon Göppert and Sazvan Saeed and Robert H. Schmitt},
editor = {Ann-Louise Ändersen and Rasmus Andersen and Thomas Ditlev Brunoe and Maria Stoettrup Schioenning Larsen and Kjeld Nielsen and Alessia Napoleone and Stefan" Kjeldgaard},
url = {https://link.springer.com/chapter/10.1007/978-3-030-90700-6_48},
isbn = {978-3-030-90700-6},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Towards Sustainable Customization: Bridging Smart Products and Manufacturing Systems},
pages = {423--430},
publisher = {Springer International Publishing},
address = {Cham},
abstract = {Ässembly systems are required to be more flexible due to increasing product variety. This is accomplished by breaking up the prevailing rigid linking of assembly stations in classic line configurations to line-less assembly systems (LAS). For a quantified potential analysis, it is necessary to assess the dependency of performance indicators on input parameters in a large number of production scenarios. Existing methods are either experience-based or computationally expensive due to full-factorial experiment plans. Therefore, the contribution of this paper is threefold. First, a seamlessly automated scenario analysis tool is developed, simulating a large set of assembly scenarios using a discrete-event simulation. Second, an artificial neural network (ANN) based approximation pipeline of the scenario analysis is implemented. An integrated AutoML pipeline for hyperparameter optimization (HPO) and Neural Architecture Search (NAS) allows for a faster potential approximation with sufficiently accurate prediction. Last, an integrated decision support system including the mentioned components is defined. It includes a priori planning of the overall system and allows the assessment of adaptions in reaction to the current system status."},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Ässembly systems are required to be more flexible due to increasing product variety. This is accomplished by breaking up the prevailing rigid linking of assembly stations in classic line configurations to line-less assembly systems (LAS). For a quantified potential analysis, it is necessary to assess the dependency of performance indicators on input parameters in a large number of production scenarios. Existing methods are either experience-based or computationally expensive due to full-factorial experiment plans. Therefore, the contribution of this paper is threefold. First, a seamlessly automated scenario analysis tool is developed, simulating a large set of assembly scenarios using a discrete-event simulation. Second, an artificial neural network (ANN) based approximation pipeline of the scenario analysis is implemented. An integrated AutoML pipeline for hyperparameter optimization (HPO) and Neural Architecture Search (NAS) allows for a faster potential approximation with sufficiently accurate prediction. Last, an integrated decision support system including the mentioned components is defined. It includes a priori planning of the overall system and allows the assessment of adaptions in reaction to the current system status."
2221.
Li, Xudong; Zheng, Jianhua; Li, Mingtao; Ma, Wenzhen; Hu, Yang
One-shot neural architecture search for fault diagnosis using vibration signals Journal Article
In: Expert Systems with Applications, vol. 190, pp. 116027, 2022, ISSN: 0957-4174.
@article{LI2022116027,
title = {One-shot neural architecture search for fault diagnosis using vibration signals},
author = {Xudong Li and Jianhua Zheng and Mingtao Li and Wenzhen Ma and Yang Hu},
url = {https://www.sciencedirect.com/science/article/pii/S0957417421013737},
doi = {https://doi.org/10.1016/j.eswa.2021.116027},
issn = {0957-4174},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Expert Systems with Applications},
volume = {190},
pages = {116027},
abstract = {Machine learning method has been widely applied in industrial fault diagnosis, especially the deep learning method. In the field of industrial fault diagnosis, deep learning is mostly used to extract features of vibration signals to achieve end-to-end fault diagnosis systems. Due to the complexity and variety of actual industrial datasets, some deep learning models are designed to be complicated. However, designing neural network architectures requires rich professional knowledge, experience, and a large number of experiments, increasing the difficulty of developing deep learning models. Fortunately, Neural Architecture Search (NAS), a branch of Automated Machine Learning (AutoML), is developing rapidly. Given a search space, NAS can search for networks that perform better than manually designed. In this paper, a one-shot NAS method for fault diagnosis is proposed. The one-shot model is a supernet that contains all candidate networks in a given search space. The supernet is trained to evaluate the actual performance of candidate networks by measuring the difference between its output probability and the true labels. According to the prediction of supernet, the networks with excellent performance can be searched, using some common search methods such as random search or evolutionary algorithm. Finally, the searched network is trained by reusing the weights of the supernet. To evaluate the proposed method, two search spaces are designed, ResNet and Inception search spaces, to search on PHM 2009 Data Challenge gearbox dataset. The state-of-the-art results are obtained, and accuracies of searched ResNet-A and Inception-A are 84.11% and 83.81%, which are 3.29% and 10.88% higher than Reinforcement Learning based NAS.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Machine learning method has been widely applied in industrial fault diagnosis, especially the deep learning method. In the field of industrial fault diagnosis, deep learning is mostly used to extract features of vibration signals to achieve end-to-end fault diagnosis systems. Due to the complexity and variety of actual industrial datasets, some deep learning models are designed to be complicated. However, designing neural network architectures requires rich professional knowledge, experience, and a large number of experiments, increasing the difficulty of developing deep learning models. Fortunately, Neural Architecture Search (NAS), a branch of Automated Machine Learning (AutoML), is developing rapidly. Given a search space, NAS can search for networks that perform better than manually designed. In this paper, a one-shot NAS method for fault diagnosis is proposed. The one-shot model is a supernet that contains all candidate networks in a given search space. The supernet is trained to evaluate the actual performance of candidate networks by measuring the difference between its output probability and the true labels. According to the prediction of supernet, the networks with excellent performance can be searched, using some common search methods such as random search or evolutionary algorithm. Finally, the searched network is trained by reusing the weights of the supernet. To evaluate the proposed method, two search spaces are designed, ResNet and Inception search spaces, to search on PHM 2009 Data Challenge gearbox dataset. The state-of-the-art results are obtained, and accuracies of searched ResNet-A and Inception-A are 84.11% and 83.81%, which are 3.29% and 10.88% higher than Reinforcement Learning based NAS.
2220.
Xue, Chao; Hu, Mengting; Huang, Xueqi; Li, Chun-Guang
Automated search space and search strategy selection for AutoML Journal Article
In: Pattern Recognition, vol. 124, pp. 108474, 2022, ISSN: 0031-3203.
@article{XUE2022108474,
title = {Automated search space and search strategy selection for AutoML},
author = {Chao Xue and Mengting Hu and Xueqi Huang and Chun-Guang Li},
url = {https://www.sciencedirect.com/science/article/pii/S0031320321006506},
doi = {https://doi.org/10.1016/j.patcog.2021.108474},
issn = {0031-3203},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Pattern Recognition},
volume = {124},
pages = {108474},
abstract = {Existing works on Automated Machine Learning (AutoML) are mainly based on predefined search space. This paper seeks synergetic automation of two ingredients, i.e., search space and search strategies. Specifically, we formulate the automation of search space and search strategies as a combinatorial optimization problem. Our empirical study on many architecture benchmarks shows that identifying the suitable search space exerts more effect than choosing a sophisticated search strategy. Motivated by this, we attempt to leverage a machine learning method to solve the discrete optimization problem, and thus develop a Layered Architecture Search Tree (LArST) approach to synergize these two components. In addition, we use a probe model-based method to extract dataset-wise features, i.e., meta-features, which is able to facilitate the estimation of proper search space and search strategy for a given task. Experimental results show the efficacy of our approach under different search mechanisms and various datasets and hardware platforms.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Existing works on Automated Machine Learning (AutoML) are mainly based on predefined search space. This paper seeks synergetic automation of two ingredients, i.e., search space and search strategies. Specifically, we formulate the automation of search space and search strategies as a combinatorial optimization problem. Our empirical study on many architecture benchmarks shows that identifying the suitable search space exerts more effect than choosing a sophisticated search strategy. Motivated by this, we attempt to leverage a machine learning method to solve the discrete optimization problem, and thus develop a Layered Architecture Search Tree (LArST) approach to synergize these two components. In addition, we use a probe model-based method to extract dataset-wise features, i.e., meta-features, which is able to facilitate the estimation of proper search space and search strategy for a given task. Experimental results show the efficacy of our approach under different search mechanisms and various datasets and hardware platforms.
2219.
Oloulade, Babatounde Moctard; Gao, Jianliang; Chen, Jiamin; Lyu, Tengfei; Al-Sabri, Raeed
Graph neural architecture search: A survey Journal Article
In: Tsinghua Science and Technology, vol. 27, no. 4, pp. 692-708, 2022.
@article{9645440,
title = {Graph neural architecture search: A survey},
author = {Babatounde Moctard Oloulade and Jianliang Gao and Jiamin Chen and Tengfei Lyu and Raeed Al-Sabri},
url = {https://ieeexplore.ieee.org/document/9645440},
doi = {10.26599/TST.2021.9010057},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Tsinghua Science and Technology},
volume = {27},
number = {4},
pages = {692-708},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2218.
Baymurzina, Dilyara; Golikov, Eugene; Burtsev, Mikhail
A review of neural architecture search Journal Article
In: Neurocomputing, vol. 474, pp. 82-93, 2022, ISSN: 0925-2312.
@article{BAYMURZINA202282,
title = {A review of neural architecture search},
author = {Dilyara Baymurzina and Eugene Golikov and Mikhail Burtsev},
url = {https://www.sciencedirect.com/science/article/pii/S0925231221018439},
doi = {https://doi.org/10.1016/j.neucom.2021.12.014},
issn = {0925-2312},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Neurocomputing},
volume = {474},
pages = {82-93},
abstract = {Despite the impressive progress in neural network architecture design, improving the performance of the existing state-of-the-art models has become increasingly challenging. For this reason, the paradigm for neural architecture design is shifting from being expert-driven to almost fully automated. An emerging body of research related to such machine-aided design is called a Neural Architecture Search (NAS). This paper reviews the recent works on NAS and highlights several crucial concepts and problems of this field.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Despite the impressive progress in neural network architecture design, improving the performance of the existing state-of-the-art models has become increasingly challenging. For this reason, the paradigm for neural architecture design is shifting from being expert-driven to almost fully automated. An emerging body of research related to such machine-aided design is called a Neural Architecture Search (NAS). This paper reviews the recent works on NAS and highlights several crucial concepts and problems of this field.
2217.
Fan, Zhenkun; Hu, Guosheng; Sun, Xin; Wang, Gaige; Dong, Junyu; Su, Chi
Self-attention neural architecture search for semantic image segmentation Journal Article
In: Knowledge-Based Systems, vol. 239, pp. 107968, 2022, ISSN: 0950-7051.
@article{FAN2022107968,
title = {Self-attention neural architecture search for semantic image segmentation},
author = {Zhenkun Fan and Guosheng Hu and Xin Sun and Gaige Wang and Junyu Dong and Chi Su},
url = {https://www.sciencedirect.com/science/article/pii/S0950705121010947},
doi = {https://doi.org/10.1016/j.knosys.2021.107968},
issn = {0950-7051},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Knowledge-Based Systems},
volume = {239},
pages = {107968},
abstract = {Self-attention can capture long-distance dependencies and is widely used in semantic segmentation. Existing methods mainly use two kinds of self-attentions, i.e., spatial attention and channel attention, which can capture the relations in HW dimension (image plane, height and width) and C dimension (channels), respectively. Very little research investigates self-attention along other dimensions, which can potentially improve the segmentation performance. In this work, we investigate the self-attentions along all the possible dimensions H,W,C,HW,HC,CW,HWC. Then we explore the aggregation of all the possible self-attentions. We apply the neural architecture search (NAS) technique to achieve optimal aggregation. Specifically, we carefully design (1) the search space and (2) the optimization method. For (1), we introduce a building block, a basic self-attention search unit (BSU), which can model self-attentions along all the dimensions. And the search space contains within-BSU and cross-BSU operations. In addition, we propose an attention-map splitting method, which can reduce the computations by 1/3. For (2), we apply an efficient differentiable optimization method to search the optimal aggregation. We conduct extensive experiments on Cityscapes and ADE20K datasets. The results show the effectiveness of the proposed method, and we achieve very competitive performance against state-of-the-art methods.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Self-attention can capture long-distance dependencies and is widely used in semantic segmentation. Existing methods mainly use two kinds of self-attentions, i.e., spatial attention and channel attention, which can capture the relations in HW dimension (image plane, height and width) and C dimension (channels), respectively. Very little research investigates self-attention along other dimensions, which can potentially improve the segmentation performance. In this work, we investigate the self-attentions along all the possible dimensions H,W,C,HW,HC,CW,HWC. Then we explore the aggregation of all the possible self-attentions. We apply the neural architecture search (NAS) technique to achieve optimal aggregation. Specifically, we carefully design (1) the search space and (2) the optimization method. For (1), we introduce a building block, a basic self-attention search unit (BSU), which can model self-attentions along all the dimensions. And the search space contains within-BSU and cross-BSU operations. In addition, we propose an attention-map splitting method, which can reduce the computations by 1/3. For (2), we apply an efficient differentiable optimization method to search the optimal aggregation. We conduct extensive experiments on Cityscapes and ADE20K datasets. The results show the effectiveness of the proposed method, and we achieve very competitive performance against state-of-the-art methods.
2216.
Liu, Shengfeng; Ge, Fangfei; Zhao, Lin; Wang, Tianfu; Ni, Dong; Liu, Tianming
NAS-optimized topology-preserving transfer learning for differentiating cortical folding patterns Journal Article
In: Medical Image Analysis, vol. 77, pp. 102316, 2022, ISSN: 1361-8415.
@article{LIU2022102316,
title = {NAS-optimized topology-preserving transfer learning for differentiating cortical folding patterns},
author = {Shengfeng Liu and Fangfei Ge and Lin Zhao and Tianfu Wang and Dong Ni and Tianming Liu},
url = {https://www.sciencedirect.com/science/article/pii/S1361841521003613},
doi = {https://doi.org/10.1016/j.media.2021.102316},
issn = {1361-8415},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Medical Image Analysis},
volume = {77},
pages = {102316},
abstract = {Increasing evidence suggests that cortical folding patterns of human cerebral cortex manifest overt structural and functional differences. However, for interpretability, few studies leverage advanced techniques (e.g., deep learning) to investigate the difference among cortical folds, resulting in more differences yet to be extensively explored. To this end, we proposed an effective topology-preserving transfer learning framework to differentiate cortical fMRI time series extracted from cortical folds. Our framework consists of three main parts: (1) Neural architecture search (NAS), which is used to devise a well-performing network structure based on an initialized hand-designed super-graph in an image dataset; (2) Topology-preserving transfer, which takes the model searched by NAS as the source network, keeping the topological connectivity in the network unchanged, while transforming all 2D operations including convolution and pooling into 1D, therefore resulting in a topology-preserving network, named TPNAS-Net; (3) Classification and correlation analysis, which involves using the TPNAS-Net to classify 1D cortical fMRI time series for each individual brain, and performing a group difference analysis between autism spectrum disorder (ASD) and healthy control (HC) and correlation analysis with clinical information (i.e., age). Extensive experiments on two ASD datasets obtain consistent results, demonstrating that the TPNAS-Net not only discriminates cortical folding patterns at high classification accuracy, but also captures subtle differences between ASD and HC (p-value = 0.042). In addition, we discover that there is a positive correlation between the classification accuracy and age in ASD (r = 0.39},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Increasing evidence suggests that cortical folding patterns of human cerebral cortex manifest overt structural and functional differences. However, for interpretability, few studies leverage advanced techniques (e.g., deep learning) to investigate the difference among cortical folds, resulting in more differences yet to be extensively explored. To this end, we proposed an effective topology-preserving transfer learning framework to differentiate cortical fMRI time series extracted from cortical folds. Our framework consists of three main parts: (1) Neural architecture search (NAS), which is used to devise a well-performing network structure based on an initialized hand-designed super-graph in an image dataset; (2) Topology-preserving transfer, which takes the model searched by NAS as the source network, keeping the topological connectivity in the network unchanged, while transforming all 2D operations including convolution and pooling into 1D, therefore resulting in a topology-preserving network, named TPNAS-Net; (3) Classification and correlation analysis, which involves using the TPNAS-Net to classify 1D cortical fMRI time series for each individual brain, and performing a group difference analysis between autism spectrum disorder (ASD) and healthy control (HC) and correlation analysis with clinical information (i.e., age). Extensive experiments on two ASD datasets obtain consistent results, demonstrating that the TPNAS-Net not only discriminates cortical folding patterns at high classification accuracy, but also captures subtle differences between ASD and HC (p-value = 0.042). In addition, we discover that there is a positive correlation between the classification accuracy and age in ASD (r = 0.39
2215.
Sivangi, Kaushik Bhargav; Dasari, Chandra Mohan; Amilpur, Santhosh; Bhukya, Raju
NoAS-DS: Neural optimal architecture search for detection of diverse DNA signals Journal Article
In: Neural Networks, vol. 147, pp. 63-71, 2022, ISSN: 0893-6080.
@article{SIVANGI202263,
title = {NoAS-DS: Neural optimal architecture search for detection of diverse DNA signals},
author = {Kaushik Bhargav Sivangi and Chandra Mohan Dasari and Santhosh Amilpur and Raju Bhukya},
url = {https://www.sciencedirect.com/science/article/pii/S0893608021004822},
doi = {https://doi.org/10.1016/j.neunet.2021.12.009},
issn = {0893-6080},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Neural Networks},
volume = {147},
pages = {63-71},
abstract = {Neural network architectures are high-performing variable models that can solve many learning tasks. Designing architectures manually require substantial time and also prior knowledge and expertise to develop a high-accuracy model. Most of the architecture search methods are developed over the task of image classification resulting in the building of complex architectures intended for large data inputs such as images. Motivated by the applications of DNA computing in Neural Architecture Search (NAS), we propose NoAS-DS which is specifically built for the architecture search of sequence-based classification tasks. Furthermore, NoAS-DS is applied to the task of predicting binding sites. Unlike other methods that implement only Convolution layers, NoAS-DS, specifically combines Convolution and LSTM layers that helps in the process of automatic architecture building. This hybrid approach helped in achieving high accuracy results on TFBS and RBP datasets which outperformed other models in TF-DNA binding prediction tasks. The best architectures generated by the proposed model can be applied to other DNA datasets of similar nature using transfer learning technique that demonstrates its generalization capability. This greatly reduces the effort required to build new architectures for other prediction tasks.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Neural network architectures are high-performing variable models that can solve many learning tasks. Designing architectures manually require substantial time and also prior knowledge and expertise to develop a high-accuracy model. Most of the architecture search methods are developed over the task of image classification resulting in the building of complex architectures intended for large data inputs such as images. Motivated by the applications of DNA computing in Neural Architecture Search (NAS), we propose NoAS-DS which is specifically built for the architecture search of sequence-based classification tasks. Furthermore, NoAS-DS is applied to the task of predicting binding sites. Unlike other methods that implement only Convolution layers, NoAS-DS, specifically combines Convolution and LSTM layers that helps in the process of automatic architecture building. This hybrid approach helped in achieving high accuracy results on TFBS and RBP datasets which outperformed other models in TF-DNA binding prediction tasks. The best architectures generated by the proposed model can be applied to other DNA datasets of similar nature using transfer learning technique that demonstrates its generalization capability. This greatly reduces the effort required to build new architectures for other prediction tasks.
2214.
Zhang, Yang; Zong, Ruohan; Kou, Ziyi; Shang, Lanyu; Wang, Dong
On streaming disaster damage assessment in social sensing: A crowd-driven dynamic neural architecture searching approach Journal Article
In: Knowledge-Based Systems, vol. 239, pp. 107984, 2022, ISSN: 0950-7051.
@article{ZHANG2022107984,
title = {On streaming disaster damage assessment in social sensing: A crowd-driven dynamic neural architecture searching approach},
author = {Yang Zhang and Ruohan Zong and Ziyi Kou and Lanyu Shang and Dong Wang},
url = {https://www.sciencedirect.com/science/article/pii/S0950705121011023},
doi = {https://doi.org/10.1016/j.knosys.2021.107984},
issn = {0950-7051},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Knowledge-Based Systems},
volume = {239},
pages = {107984},
abstract = {Motivated by the recent advances in Internet and communication techniques and the proliferation of online social media, social sensing has emerged as a new sensing paradigm to obtain timely observations of the physical world from “human sensors”. In this study, we focus on an emerging application in social sensing – streaming disaster damage assessment (DDA), which aims to automatically assess the damage severity of affected areas in a disaster event on the fly by leveraging the streaming imagery data about the disaster on social media. In particular, we study a dynamic optimal neural architecture searching (NAS) problem in streaming DDA applications. Our goal is to dynamically determine the optimal neural network architecture that accurately estimates the damage severity for each newly arrived image in the stream by leveraging human intelligence from the crowdsourcing systems. The present study is motivated by the observation that the neural network architectures in current DDA solutions are mainly designed by artificial intelligence (AI) experts, which often leads to non-negligible costs and errors given the dynamic nature of the streaming DDA applications and the lack of real-time annotations of the massive social media data inputs. Two critical technical challenges exist in solving our problem: (i) it is non-trivial to dynamically identify the optimal neural network architecture for each image on the fly without knowing its ground-truth label a priori; (ii) it is challenging to effectively leverage the imperfect crowd intelligence to correctly identify the optimal neural network architecture for each image. To address the above challenges, we developed CD-NAS, a dynamic crowd-AI collaborative NAS framework that carefully explores the human intelligence from crowdsourcing systems to solve the dynamic optimal NAS problem and optimize the performance of streaming DDA applications. The evaluation results from a real-world streaming DDA application show that CD-NAS consistently outperforms the state-of-the-art AI and NAS baselines by achieving the highest disaster damage assessment accuracy while maintaining the lowest computational cost.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Motivated by the recent advances in Internet and communication techniques and the proliferation of online social media, social sensing has emerged as a new sensing paradigm to obtain timely observations of the physical world from “human sensors”. In this study, we focus on an emerging application in social sensing – streaming disaster damage assessment (DDA), which aims to automatically assess the damage severity of affected areas in a disaster event on the fly by leveraging the streaming imagery data about the disaster on social media. In particular, we study a dynamic optimal neural architecture searching (NAS) problem in streaming DDA applications. Our goal is to dynamically determine the optimal neural network architecture that accurately estimates the damage severity for each newly arrived image in the stream by leveraging human intelligence from the crowdsourcing systems. The present study is motivated by the observation that the neural network architectures in current DDA solutions are mainly designed by artificial intelligence (AI) experts, which often leads to non-negligible costs and errors given the dynamic nature of the streaming DDA applications and the lack of real-time annotations of the massive social media data inputs. Two critical technical challenges exist in solving our problem: (i) it is non-trivial to dynamically identify the optimal neural network architecture for each image on the fly without knowing its ground-truth label a priori; (ii) it is challenging to effectively leverage the imperfect crowd intelligence to correctly identify the optimal neural network architecture for each image. To address the above challenges, we developed CD-NAS, a dynamic crowd-AI collaborative NAS framework that carefully explores the human intelligence from crowdsourcing systems to solve the dynamic optimal NAS problem and optimize the performance of streaming DDA applications. The evaluation results from a real-world streaming DDA application show that CD-NAS consistently outperforms the state-of-the-art AI and NAS baselines by achieving the highest disaster damage assessment accuracy while maintaining the lowest computational cost.
2213.
Chen, Dong; Shen, Hao; Shen, Yuchen
PT-NAS: Designing efficient keypoint-based object detectors for desktop CPU platforms Journal Article
In: Neurocomputing, vol. 476, pp. 38-52, 2022, ISSN: 0925-2312.
@article{CHEN202238,
title = {PT-NAS: Designing efficient keypoint-based object detectors for desktop CPU platforms},
author = {Dong Chen and Hao Shen and Yuchen Shen},
url = {https://www.sciencedirect.com/science/article/pii/S0925231221019238},
doi = {https://doi.org/10.1016/j.neucom.2021.12.067},
issn = {0925-2312},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Neurocomputing},
volume = {476},
pages = {38-52},
abstract = {Recently, keypoint-based object detectors have attracted widespread attention, due to their novel structure and excellent performance. However, in terms of their design, there are still two limitations: 1) Most keypoint-based methods are designed for GPU platforms, which makes them inefficient on desktop CPU platforms. 2) Existing works still rely heavily on manual design and prior knowledge. To this end, this work aims to offer a practical solution for designing CPU-efficient key-point detectors. First, we present a set of practical design guidelines by comparing different detection architectures. Following the proposed guidelines, we further develop a progressive three-phase network architecture search (PT-NAS) to achieve the automated design of detection architectures. Benefiting from our hierarchical search space and novel search pipeline, our PT-NAS not only achieves higher search efficiency, but also satisfies the practicality of CPU platforms. On the MS-COCO benchmark, we utilize our PT-NAS to generate several key-point detectors for fast inference on desktop CPUs. Finally, comprehensive comparison experiments prove that the proposed PT-NAS can produce new state-of-the-art keypoint-based detectors for CPU platforms.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Recently, keypoint-based object detectors have attracted widespread attention, due to their novel structure and excellent performance. However, in terms of their design, there are still two limitations: 1) Most keypoint-based methods are designed for GPU platforms, which makes them inefficient on desktop CPU platforms. 2) Existing works still rely heavily on manual design and prior knowledge. To this end, this work aims to offer a practical solution for designing CPU-efficient key-point detectors. First, we present a set of practical design guidelines by comparing different detection architectures. Following the proposed guidelines, we further develop a progressive three-phase network architecture search (PT-NAS) to achieve the automated design of detection architectures. Benefiting from our hierarchical search space and novel search pipeline, our PT-NAS not only achieves higher search efficiency, but also satisfies the practicality of CPU platforms. On the MS-COCO benchmark, we utilize our PT-NAS to generate several key-point detectors for fast inference on desktop CPUs. Finally, comprehensive comparison experiments prove that the proposed PT-NAS can produce new state-of-the-art keypoint-based detectors for CPU platforms.
2212.
Zhang, Zhen; Liu, Shanghao; Zhang, Yang; Chen, Wenbo
RS-DARTS: A Convolutional Neural Architecture Search for Remote Sensing Image Scene Classification Journal Article
In: Remote Sensing, vol. 14, no. 1, 2022, ISSN: 2072-4292.
@article{rs14010141,
title = {RS-DARTS: A Convolutional Neural Architecture Search for Remote Sensing Image Scene Classification},
author = {Zhen Zhang and Shanghao Liu and Yang Zhang and Wenbo Chen},
url = {https://www.mdpi.com/2072-4292/14/1/141},
doi = {10.3390/rs14010141},
issn = {2072-4292},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Remote Sensing},
volume = {14},
number = {1},
abstract = {Due to the superiority of convolutional neural networks, many deep learning methods have been used in image classification. The enormous difference between natural images and remote sensing images makes it difficult to directly utilize or modify existing CNN models for remote sensing scene classification tasks. In this article, a new paradigm is proposed that can automatically design a suitable CNN architecture for scene classification. A more efficient search framework, RS-DARTS, is adopted to find the optimal network architecture. This framework has two phases. In the search phase, some new strategies are presented, making the calculation process smoother, and better distinguishing the optimal and other operations. In addition, we added noise to suppress skip connections in order to close the gap between trained and validation processing and ensure classification accuracy. Moreover, a small part of the neural network is sampled to reduce the redundancy in exploring the network space and speed up the search processing. In the evaluation phase, the optimal cell architecture is stacked to construct the final network. Extensive experiments demonstrated the validity of the search strategy and the impressive classification performance of RS-DARTS on four public benchmark datasets. The proposed method showed more effectiveness than the manually designed CNN model and other methods of neural architecture search. Especially, in terms of search cost, RS-DARTS consumed less time than other NAS methods.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Due to the superiority of convolutional neural networks, many deep learning methods have been used in image classification. The enormous difference between natural images and remote sensing images makes it difficult to directly utilize or modify existing CNN models for remote sensing scene classification tasks. In this article, a new paradigm is proposed that can automatically design a suitable CNN architecture for scene classification. A more efficient search framework, RS-DARTS, is adopted to find the optimal network architecture. This framework has two phases. In the search phase, some new strategies are presented, making the calculation process smoother, and better distinguishing the optimal and other operations. In addition, we added noise to suppress skip connections in order to close the gap between trained and validation processing and ensure classification accuracy. Moreover, a small part of the neural network is sampled to reduce the redundancy in exploring the network space and speed up the search processing. In the evaluation phase, the optimal cell architecture is stacked to construct the final network. Extensive experiments demonstrated the validity of the search strategy and the impressive classification performance of RS-DARTS on four public benchmark datasets. The proposed method showed more effectiveness than the manually designed CNN model and other methods of neural architecture search. Especially, in terms of search cost, RS-DARTS consumed less time than other NAS methods.
2211.
Reiser, Daniel; Reichel, Peter; Pechmann, Stefan; Mallah, Maen; Oppelt, Maximilian; Hagelauer, Amelie; Breiling, Marco; Fey, Dietmar; Reichenbach, Marc
A Framework for Ultra Low-Power Hardware Accelerators Using NNs for Embedded Time Series Classification Journal Article
In: Journal of Low Power Electronics and Applications, vol. 12, no. 1, 2022, ISSN: 2079-9268.
@article{jlpea12010002,
title = {A Framework for Ultra Low-Power Hardware Accelerators Using NNs for Embedded Time Series Classification},
author = {Daniel Reiser and Peter Reichel and Stefan Pechmann and Maen Mallah and Maximilian Oppelt and Amelie Hagelauer and Marco Breiling and Dietmar Fey and Marc Reichenbach},
url = {https://www.mdpi.com/2079-9268/12/1/2},
doi = {10.3390/jlpea12010002},
issn = {2079-9268},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Journal of Low Power Electronics and Applications},
volume = {12},
number = {1},
abstract = {In embedded applications that use neural networks (NNs) for classification tasks, it is important to not only minimize the power consumption of the NN calculation, but of the whole system. Optimization approaches for individual parts exist, such as quantization of the NN or analog calculation of arithmetic operations. However, there is no holistic approach for a complete embedded system design that is generic enough in the design process to be used for different applications, but specific in the hardware implementation to waste no energy for a given application. Therefore, we present a novel framework that allows an end-to-end ASIC implementation of a low-power hardware for time series classification using NNs. This includes a neural architecture search (NAS), which optimizes the NN configuration for accuracy and energy efficiency at the same time. This optimization targets a custom designed hardware architecture that is derived from the key properties of time series classification tasks. Additionally, a hardware generation tool is used that creates a complete system from the definition of the NN. This system uses local multi-level RRAM memory as weight and bias storage to avoid external memory access. Exploiting the non-volatility of these devices, such a system can use a power-down mode to save significant energy during the data acquisition process. Detection of atrial fibrillation (AFib) in electrocardiogram (ECG) data is used as an example for evaluation of the framework. It is shown that a reduction of more than 95% of the energy consumption compared to state-of-the-art solutions is achieved.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
In embedded applications that use neural networks (NNs) for classification tasks, it is important to not only minimize the power consumption of the NN calculation, but of the whole system. Optimization approaches for individual parts exist, such as quantization of the NN or analog calculation of arithmetic operations. However, there is no holistic approach for a complete embedded system design that is generic enough in the design process to be used for different applications, but specific in the hardware implementation to waste no energy for a given application. Therefore, we present a novel framework that allows an end-to-end ASIC implementation of a low-power hardware for time series classification using NNs. This includes a neural architecture search (NAS), which optimizes the NN configuration for accuracy and energy efficiency at the same time. This optimization targets a custom designed hardware architecture that is derived from the key properties of time series classification tasks. Additionally, a hardware generation tool is used that creates a complete system from the definition of the NN. This system uses local multi-level RRAM memory as weight and bias storage to avoid external memory access. Exploiting the non-volatility of these devices, such a system can use a power-down mode to save significant energy during the data acquisition process. Detection of atrial fibrillation (AFib) in electrocardiogram (ECG) data is used as an example for evaluation of the framework. It is shown that a reduction of more than 95% of the energy consumption compared to state-of-the-art solutions is achieved.
2210.
Ding, Yadong; Wu, Yu; Huang, Chengyue; Tang, Siliang; Wu, Fei; Yang, Yi; Zhu, Wenwu; Zhuang, Yueting
NAP: Neural Architecture search with Pruning Journal Article
In: Neurocomputing, 2022, ISSN: 0925-2312.
@article{DING2022,
title = {NAP: Neural Architecture search with Pruning},
author = {Yadong Ding and Yu Wu and Chengyue Huang and Siliang Tang and Fei Wu and Yi Yang and Wenwu Zhu and Yueting Zhuang},
url = {https://www.sciencedirect.com/science/article/pii/S0925231221018361},
doi = {https://doi.org/10.1016/j.neucom.2021.12.002},
issn = {0925-2312},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Neurocomputing},
abstract = {There has been continuously increasing attention attracted by Neural Architecture Search (NAS). Due to its computational efficiency, gradient-based NAS methods like DARTS have become the most popular framework for NAS tasks. Nevertheless, as the search iterates, the derived model in previous NAS frameworks becomes dominated by skip-connects, causing the performance downfall. In this work, we present a novel approach to alleviate this issue, named Neural Architecture search with Pruning (NAP). Unlike prior differentiable architecture search works, our approach draws the idea from network pruning. We first train an over-parameterized network, including all candidate operations. Then we propose a criterion to prune the network. Based on a newly designed relaxation of architecture representation, NAP can derive the most potent model by removing trivial and redundant edges from the whole network topology. Experiments show the effectiveness of our proposed approach. Specifically, the model searched by NAP achieves state-of-the-art performances (2.48% test error) on CIFAR-10. We transfer the model to ImageNet and obtains a 25.1% test error with only 5.0M parameters, which is on par with modern NAS methods.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
There has been continuously increasing attention attracted by Neural Architecture Search (NAS). Due to its computational efficiency, gradient-based NAS methods like DARTS have become the most popular framework for NAS tasks. Nevertheless, as the search iterates, the derived model in previous NAS frameworks becomes dominated by skip-connects, causing the performance downfall. In this work, we present a novel approach to alleviate this issue, named Neural Architecture search with Pruning (NAP). Unlike prior differentiable architecture search works, our approach draws the idea from network pruning. We first train an over-parameterized network, including all candidate operations. Then we propose a criterion to prune the network. Based on a newly designed relaxation of architecture representation, NAP can derive the most potent model by removing trivial and redundant edges from the whole network topology. Experiments show the effectiveness of our proposed approach. Specifically, the model searched by NAP achieves state-of-the-art performances (2.48% test error) on CIFAR-10. We transfer the model to ImageNet and obtains a 25.1% test error with only 5.0M parameters, which is on par with modern NAS methods.
2209.
Zhou, Kaixiong; Liu, Zirui; Duan, Keyu; Hu, Xia
Graph Neural Networks: AutoML Book Chapter
In: pp. 371-389, 2022.
@inbook{Zhou2022,
title = {Graph Neural Networks: AutoML},
author = {Kaixiong Zhou and Zirui Liu and Keyu Duan and Xia Hu},
url = {https://link.springer.com/chapter/10.1007/978-981-16-6054-2_17},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
pages = {371-389},
keywords = {},
pubstate = {published},
tppubtype = {inbook}
}
2208.
Rorabaugh, Ariel Keller; Caíno-Lores, Silvina; Johnston, Travis; Taufer, Michela
High frequency accuracy and loss data of random neural networks trained on image datasets Journal Article
In: Data in Brief, vol. 40, pp. 107780, 2022, ISSN: 2352-3409.
@article{RORABAUGH2022107780,
title = {High frequency accuracy and loss data of random neural networks trained on image datasets},
author = {Ariel Keller Rorabaugh and Silvina Caíno-Lores and Travis Johnston and Michela Taufer},
url = {https://www.sciencedirect.com/science/article/pii/S2352340921010544},
doi = {https://doi.org/10.1016/j.dib.2021.107780},
issn = {2352-3409},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Data in Brief},
volume = {40},
pages = {107780},
abstract = {Neural Networks (NNs) are increasingly used across scientific domains to extract knowledge from experimental or computational data. An NN is composed of natural or artificial neurons that serve as simple processing units and are interconnected into a model architecture; it acquires knowledge from the environment through a learning process and stores this knowledge in its connections. The learning process is conducted by training. During NN training, the learning process can be tracked by periodically validating the NN and calculating its fitness. The resulting sequence of fitness values (i.e., validation accuracy or validation loss) is called the NN learning curve. The development of tools for NN design requires knowledge of diverse NNs and their complete learning curves. Generally, only final fully-trained fitness values for highly accurate NNs are made available to the community, hampering efforts to develop tools for NN design and leaving unaddressed aspects such as explaining the generation of an NN and reproducing its learning process. Our dataset fills this gap by fully recording the structure, metadata, and complete learning curves for a wide variety of random NNs throughout their training. Our dataset captures the lifespan of 6000 NNs throughout generation, training, and validation stages. It consists of a suite of 6000 tables, each table representing the lifespan of one NN. We generate each NN with randomized parameter values and train it for 40 epochs on one of three diverse image datasets (i.e., CIFAR-100, FashionMNIST, SVHN). We calculate and record each NN’s fitness with high frequency—every half epoch—to capture the evolution of the training and validation process. As a result, for each NN, we record the generated parameter values describing the structure of that NN, the image dataset on which the NN trained, and all loss and accuracy values for the NN every half epoch. We put our dataset to the service of researchers studying NN performance and its evolution throughout training and validation. Statistical methods can be applied to our dataset to analyze the shape of learning curves in diverse NNs, and the relationship between an NN’s structure and its fitness. Additionally, the structural data and metadata that we record enable the reconstruction and reproducibility of the associated NN.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Neural Networks (NNs) are increasingly used across scientific domains to extract knowledge from experimental or computational data. An NN is composed of natural or artificial neurons that serve as simple processing units and are interconnected into a model architecture; it acquires knowledge from the environment through a learning process and stores this knowledge in its connections. The learning process is conducted by training. During NN training, the learning process can be tracked by periodically validating the NN and calculating its fitness. The resulting sequence of fitness values (i.e., validation accuracy or validation loss) is called the NN learning curve. The development of tools for NN design requires knowledge of diverse NNs and their complete learning curves. Generally, only final fully-trained fitness values for highly accurate NNs are made available to the community, hampering efforts to develop tools for NN design and leaving unaddressed aspects such as explaining the generation of an NN and reproducing its learning process. Our dataset fills this gap by fully recording the structure, metadata, and complete learning curves for a wide variety of random NNs throughout their training. Our dataset captures the lifespan of 6000 NNs throughout generation, training, and validation stages. It consists of a suite of 6000 tables, each table representing the lifespan of one NN. We generate each NN with randomized parameter values and train it for 40 epochs on one of three diverse image datasets (i.e., CIFAR-100, FashionMNIST, SVHN). We calculate and record each NN’s fitness with high frequency—every half epoch—to capture the evolution of the training and validation process. As a result, for each NN, we record the generated parameter values describing the structure of that NN, the image dataset on which the NN trained, and all loss and accuracy values for the NN every half epoch. We put our dataset to the service of researchers studying NN performance and its evolution throughout training and validation. Statistical methods can be applied to our dataset to analyze the shape of learning curves in diverse NNs, and the relationship between an NN’s structure and its fitness. Additionally, the structural data and metadata that we record enable the reconstruction and reproducibility of the associated NN.
2207.
Wang, Xin; Zhang, Ziwei; Zhu, Wenwu
Automated Graph Machine Learning: Approaches, Libraries and Directions Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2201-01288,
title = {Automated Graph Machine Learning: Approaches, Libraries and Directions},
author = {Xin Wang and Ziwei Zhang and Wenwu Zhu},
url = {https://arxiv.org/abs/2201.01288},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2201.01288},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2206.
Rorabaugh, Ariel Keller; Caino-Lores, Silvina; Johnston, Travis; Taufer, Michela
Building High-throughput Neural Architecture Search Workflows via a Decoupled Fitness Prediction Engine Journal Article
In: IEEE Transactions on Parallel and Distributed Systems, pp. 1-1, 2022.
@article{9674227,
title = {Building High-throughput Neural Architecture Search Workflows via a Decoupled Fitness Prediction Engine},
author = {Ariel Keller Rorabaugh and Silvina Caino-Lores and Travis Johnston and Michela Taufer},
url = {https://ieeexplore.ieee.org/abstract/document/9674227},
doi = {10.1109/TPDS.2022.3140681},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Parallel and Distributed Systems},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2205.
Zhan, Cheng; Zhang, Licheng; Zhao, Xin; Lee, Chang-Chun; Huang, Shujiao
Neural Architecture Search for Inversion Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2201-01772,
title = {Neural Architecture Search for Inversion},
author = {Cheng Zhan and Licheng Zhang and Xin Zhao and Chang-Chun Lee and Shujiao Huang},
url = {https://arxiv.org/abs/2201.01772},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2201.01772},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2204.
Zhou, Jian; Zheng, Lianyu; Wang, Yiwei; Wang, Cheng; Gao, Robert X.
Automated model generation for machinery fault diagnosis based on reinforcement learning and neural architecture search Journal Article
In: IEEE Transactions on Instrumentation and Measurement, pp. 1-1, 2022.
@article{9673794,
title = {Automated model generation for machinery fault diagnosis based on reinforcement learning and neural architecture search},
author = {Jian Zhou and Lianyu Zheng and Yiwei Wang and Cheng Wang and Robert X. Gao},
url = {https://ieeexplore.ieee.org/abstract/document/9673794},
doi = {10.1109/TIM.2022.3141166},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Instrumentation and Measurement},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2203.
Zhang, Haoyu; Jin, Yaochu; Jin, Yaochu; Hao, Kuangrong
Evolutionary Search for Complete Neural Network Architectures with Partial Weight Sharing Journal Article
In: IEEE Transactions on Evolutionary Computation, pp. 1-1, 2022.
@article{9672175,
title = {Evolutionary Search for Complete Neural Network Architectures with Partial Weight Sharing},
author = {Haoyu Zhang and Yaochu Jin and Yaochu Jin and Kuangrong Hao},
url = {https://ieeexplore.ieee.org/abstract/document/9672175},
doi = {10.1109/TEVC.2022.3140855},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Evolutionary Computation},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2202.
Yipeng, Lv; Wenbing, Lv; Kaixuan, Han; Wentao, Tao; Ling, Zheng; Shizhuang, Weng; Lingsheng, Huang
In: Food Control, pp. 108819, 2022, ISSN: 0956-7135.
@article{YIPENG2022108819,
title = {Determination of wheat kernels damaged by Fusarium head blight using monochromatic images of effective wavelengths from hyperspectral imaging coupled with an architecture self-search deep network},
author = {Lv Yipeng and Lv Wenbing and Han Kaixuan and Tao Wentao and Zheng Ling and Weng Shizhuang and Huang Lingsheng},
url = {https://www.sciencedirect.com/science/article/pii/S0956713522000123},
doi = {https://doi.org/10.1016/j.foodcont.2022.108819},
issn = {0956-7135},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Food Control},
pages = {108819},
abstract = {Wheat kernels damaged by Fusarium head blight (FHB) lose moisture, protein and starch and carry dangerous toxins. Classification of degree of damage can be helpful to customise the use of wheat kernels, reduce financial losses and ensure grain safety. In this study, hyperspectral imaging (HSI) and deep learning network were explored to determine sound, mildly, moderately and severely damaged wheat kernels. Effective wavelengths (EWs) were selected from the reflectance spectroscopy of HSI images by ReliefF, uninformative variable elimination, random frog and shuffled frog leaping algorithm, and the monochromatic images of different combinations of EWs were adopted to develop the classification models combined with an architecture self-search deep network (ASSDN). ASSDN and the images composed of 941, 876 and 732 nm achieved the best determination with average accuracy of 100% and 98.31% in the training and prediction sets, respectively, outperforming other images or methods. And the average area under the curve of 0.9985 indicated its excellent robustness. Using images of sporadic wavelengths, the computation and operation complexity were evidently reduced, and a simple and custom-built instrument can be easily designed for practical recognition of FHB-damaged wheat kernels. Meanwhile, ASSDN can generate and optimise the high-performance classification network by itself, which is user–friendly and largely expands application potential of deep network.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Wheat kernels damaged by Fusarium head blight (FHB) lose moisture, protein and starch and carry dangerous toxins. Classification of degree of damage can be helpful to customise the use of wheat kernels, reduce financial losses and ensure grain safety. In this study, hyperspectral imaging (HSI) and deep learning network were explored to determine sound, mildly, moderately and severely damaged wheat kernels. Effective wavelengths (EWs) were selected from the reflectance spectroscopy of HSI images by ReliefF, uninformative variable elimination, random frog and shuffled frog leaping algorithm, and the monochromatic images of different combinations of EWs were adopted to develop the classification models combined with an architecture self-search deep network (ASSDN). ASSDN and the images composed of 941, 876 and 732 nm achieved the best determination with average accuracy of 100% and 98.31% in the training and prediction sets, respectively, outperforming other images or methods. And the average area under the curve of 0.9985 indicated its excellent robustness. Using images of sporadic wavelengths, the computation and operation complexity were evidently reduced, and a simple and custom-built instrument can be easily designed for practical recognition of FHB-damaged wheat kernels. Meanwhile, ASSDN can generate and optimise the high-performance classification network by itself, which is user–friendly and largely expands application potential of deep network.
2201.
Wu, Meng-Ting; Lin, Hung-I; Tsai, Chun-Wei
A Training-Free Genetic Neural Architecture Search Proceedings Article
In: Proceedings of the 2021 ACM International Conference on Intelligent Computing and Its Emerging Applications, pp. 65–70, Association for Computing Machinery, Jinan, China, 2022, ISBN: 9781450391603.
@inproceedings{10.1145/3491396.3506510,
title = {A Training-Free Genetic Neural Architecture Search},
author = {Meng-Ting Wu and Hung-I Lin and Chun-Wei Tsai},
url = {https://doi.org/10.1145/3491396.3506510},
doi = {10.1145/3491396.3506510},
isbn = {9781450391603},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Proceedings of the 2021 ACM International Conference on Intelligent Computing and Its Emerging Applications},
pages = {65–70},
publisher = {Association for Computing Machinery},
address = {Jinan, China},
series = {ACM ICEA '21},
abstract = {The so-called neural architecture search (NAS) provides an alternative way to construct a "good neural architecture," which would normally outperform hand-made architectures, for solving complex problems without domain knowledge. However, a critical issue for most of the NAS techniques is in that it is computationally very expensive because several complete/partial training processes are involved in evaluating the goodness of a neural architecture during the process of NAS. To mitigate this problem for evaluating a single neural architecture found by the search algorithm of NAS, we present an efficient NAS in this study, called genetic algorithm and noise immunity for neural architecture search without training (GA-NINASWOT). The genetic algorithm (GA) in the proposed algorithm is used to search for high potential neural architectures while a modified scoring method based on the neural architecture search without training (NASWOT) is used to replace the training process of each neural architecture found by the GA for measuring its quality. To evaluate the performance of GA-NINASWOT, we compared it with several state-of-the-art NAS techniques, which include weight-sharing methods, non-weight-sharing methods, and NASWOT. Simulation results show that GA-NINASWOT outperforms all the other state-of-the-art weight-sharing methods and NASWOT compared in this study in terms of the accuracy and computational time. Moreover, GA-NINASWOT gives a result that is comparable to those found by the non-weight-sharing methods while reducing 99% of the search time.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
The so-called neural architecture search (NAS) provides an alternative way to construct a "good neural architecture," which would normally outperform hand-made architectures, for solving complex problems without domain knowledge. However, a critical issue for most of the NAS techniques is in that it is computationally very expensive because several complete/partial training processes are involved in evaluating the goodness of a neural architecture during the process of NAS. To mitigate this problem for evaluating a single neural architecture found by the search algorithm of NAS, we present an efficient NAS in this study, called genetic algorithm and noise immunity for neural architecture search without training (GA-NINASWOT). The genetic algorithm (GA) in the proposed algorithm is used to search for high potential neural architectures while a modified scoring method based on the neural architecture search without training (NASWOT) is used to replace the training process of each neural architecture found by the GA for measuring its quality. To evaluate the performance of GA-NINASWOT, we compared it with several state-of-the-art NAS techniques, which include weight-sharing methods, non-weight-sharing methods, and NASWOT. Simulation results show that GA-NINASWOT outperforms all the other state-of-the-art weight-sharing methods and NASWOT compared in this study in terms of the accuracy and computational time. Moreover, GA-NINASWOT gives a result that is comparable to those found by the non-weight-sharing methods while reducing 99% of the search time.
2200.
Chen, Yi-Ping; Tsai, Chun-Wei
An Effective Neural Architecture Optimization Algorithm for CNN Based on Search Economics Proceedings Article
In: Proceedings of the 2021 ACM International Conference on Intelligent Computing and Its Emerging Applications, pp. 40–45, Association for Computing Machinery, Jinan, China, 2022, ISBN: 9781450391603.
@inproceedings{10.1145/3491396.3506505,
title = {An Effective Neural Architecture Optimization Algorithm for CNN Based on Search Economics},
author = {Yi-Ping Chen and Chun-Wei Tsai},
url = {https://doi.org/10.1145/3491396.3506505},
doi = {10.1145/3491396.3506505},
isbn = {9781450391603},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Proceedings of the 2021 ACM International Conference on Intelligent Computing and Its Emerging Applications},
pages = {40–45},
publisher = {Association for Computing Machinery},
address = {Jinan, China},
series = {ACM ICEA '21},
abstract = {Developing a good convolutional neural network (CNN) architecture manually by trial and error is typically extremely time-consuming and requires a lot of effort. That is why several recent studies have attempted to develop ways to automatically construct a suitable CNN architecture. In this study, an effective neural architecture search (NAS) algorithm based on a novel metaheuristic algorithm, search economics (SE), is presented for CNN to improve the accuracy of image classification. The basic idea of the proposed algorithm is to use the "expected value" instead of the öbjective value" to evaluate a set of searched solutions, i.e., neural architectures in this case. As such, the searches of the proposed algorithm will trend to high potential regions in the solution space. Simulation results show that the proposed algorithm outperforms genetic algorithm-based NAS algorithm in terms of the accuracy, especially for complex image classification problems.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Developing a good convolutional neural network (CNN) architecture manually by trial and error is typically extremely time-consuming and requires a lot of effort. That is why several recent studies have attempted to develop ways to automatically construct a suitable CNN architecture. In this study, an effective neural architecture search (NAS) algorithm based on a novel metaheuristic algorithm, search economics (SE), is presented for CNN to improve the accuracy of image classification. The basic idea of the proposed algorithm is to use the "expected value" instead of the öbjective value" to evaluate a set of searched solutions, i.e., neural architectures in this case. As such, the searches of the proposed algorithm will trend to high potential regions in the solution space. Simulation results show that the proposed algorithm outperforms genetic algorithm-based NAS algorithm in terms of the accuracy, especially for complex image classification problems.
2199.
Pochelu, Pierrick; Petiton, Serge G.; Conche, Bruno
A Deep Neural Networks Ensemble Workflow from Hyperparameter Search to Inference Leveraging GPU Clusters Proceedings Article
In: International Conference on High Performance Computing in Asia-Pacific Region, pp. 61–71, Association for Computing Machinery, Virtual Event, Japan, 2022, ISBN: 9781450384988.
@inproceedings{10.1145/3492805.3492819,
title = {A Deep Neural Networks Ensemble Workflow from Hyperparameter Search to Inference Leveraging GPU Clusters},
author = {Pierrick Pochelu and Serge G. Petiton and Bruno Conche},
url = {https://doi.org/10.1145/3492805.3492819},
doi = {10.1145/3492805.3492819},
isbn = {9781450384988},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {International Conference on High Performance Computing in Asia-Pacific Region},
pages = {61–71},
publisher = {Association for Computing Machinery},
address = {Virtual Event, Japan},
series = {HPCAsia2022},
abstract = {Automated Machine Learning with ensembling (or AutoML with ensembling) seeks to automatically build ensembles of Deep Neural Networks (DNNs) to achieve qualitative predictions. Ensemble of DNNs are well known to avoid over-fitting but they are memory and time consuming approaches. Therefore, an ideal AutoML would produce in one single run time different ensembles regarding accuracy and inference speed. While previous works on AutoML focus to search for the best model to maximize its generalization ability, we rather propose a new AutoML to build a larger library of accurate and diverse individual models to then construct ensembles. First, our extensive benchmarks show asynchronous Hyperband is an efficient and robust way to build a large number of diverse models to combine them. Then, a new ensemble selection method based on a multi-objective greedy algorithm is proposed to generate accurate ensembles by controlling their computing cost. Finally, we propose a novel algorithm to optimize the inference of the DNNs ensemble in a GPU cluster based on allocation optimization. The produced AutoML with ensemble method shows robust results on two datasets using efficiently GPU clusters during both the training phase and the inference phase.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Automated Machine Learning with ensembling (or AutoML with ensembling) seeks to automatically build ensembles of Deep Neural Networks (DNNs) to achieve qualitative predictions. Ensemble of DNNs are well known to avoid over-fitting but they are memory and time consuming approaches. Therefore, an ideal AutoML would produce in one single run time different ensembles regarding accuracy and inference speed. While previous works on AutoML focus to search for the best model to maximize its generalization ability, we rather propose a new AutoML to build a larger library of accurate and diverse individual models to then construct ensembles. First, our extensive benchmarks show asynchronous Hyperband is an efficient and robust way to build a large number of diverse models to combine them. Then, a new ensemble selection method based on a multi-objective greedy algorithm is proposed to generate accurate ensembles by controlling their computing cost. Finally, we propose a novel algorithm to optimize the inference of the DNNs ensemble in a GPU cluster based on allocation optimization. The produced AutoML with ensemble method shows robust results on two datasets using efficiently GPU clusters during both the training phase and the inference phase.
2198.
Durbha, Krishna Srikar; Amuru, Saidhiraj
AutoML Models for Wireless Signals Classification and their effectiveness against Adversarial Attacks Proceedings Article
In: 2022 14th International Conference on COMmunication Systems NETworkS (COMSNETS), pp. 265-269, 2022.
@inproceedings{9668448,
title = {AutoML Models for Wireless Signals Classification and their effectiveness against Adversarial Attacks},
author = {Krishna Srikar Durbha and Saidhiraj Amuru},
url = {https://ieeexplore.ieee.org/abstract/document/9668448},
doi = {10.1109/COMSNETS53615.2022.9668448},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {2022 14th International Conference on COMmunication Systems NETworkS (COMSNETS)},
pages = {265-269},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2197.
Jiang, Chunheng; Pedapati, Tejaswini; Chen, Pin-Yu; Sun, Yizhou; Gao, Jianxi
Neural Capacitance: A New Perspective of Neural Network Selection via Edge Dynamics Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2201-04194,
title = {Neural Capacitance: A New Perspective of Neural Network Selection via Edge Dynamics},
author = {Chunheng Jiang and Tejaswini Pedapati and Pin-Yu Chen and Yizhou Sun and Jianxi Gao},
url = {https://arxiv.org/abs/2201.04194},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2201.04194},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2196.
Cai, Han; Lin, Ji; Han, Song
Chapter 4 - Efficient methods for deep learning Book Section
In: Davies, E. R.; Turk, Matthew A. (Ed.): Advanced Methods and Deep Learning in Computer Vision, pp. 159-190, Academic Press, 2022, ISBN: 978-0-12-822109-9.
@incollection{CAI2022159,
title = {Chapter 4 - Efficient methods for deep learning},
author = {Han Cai and Ji Lin and Song Han},
editor = {E. R. Davies and Matthew A. Turk},
url = {https://www.sciencedirect.com/science/article/pii/B9780128221099000138},
doi = {https://doi.org/10.1016/B978-0-12-822109-9.00013-8},
isbn = {978-0-12-822109-9},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Advanced Methods and Deep Learning in Computer Vision},
pages = {159-190},
publisher = {Academic Press},
series = {Computer Vision and Pattern Recognition},
abstract = {Deep neural networks (DNNs) have achieved unprecedented success in computer vision. However, their superior performance comes at the considerable cost of computational complexity, which greatly hinders their applications in many resource-constrained devices, such as mobile phones and Internet of Things (IoT) devices. Therefore, methods and techniques that can lift the efficiency bottleneck while preserving the high accuracy of DNNs are in great demand to enable numerous edge AI applications. This chapter provides an overview of efficient deep learning methods. We start from introducing popular model compression methods, including pruning, factorization, and quantization. We then describe compact model design techniques including efficient convolution layers and representative efficient CNN architectures. Finally, to reduce the large design cost of these manual solutions, we discuss the AutoML framework for each of them, such as neural architecture search (NAS) and automated pruning and quantization.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
Deep neural networks (DNNs) have achieved unprecedented success in computer vision. However, their superior performance comes at the considerable cost of computational complexity, which greatly hinders their applications in many resource-constrained devices, such as mobile phones and Internet of Things (IoT) devices. Therefore, methods and techniques that can lift the efficiency bottleneck while preserving the high accuracy of DNNs are in great demand to enable numerous edge AI applications. This chapter provides an overview of efficient deep learning methods. We start from introducing popular model compression methods, including pruning, factorization, and quantization. We then describe compact model design techniques including efficient convolution layers and representative efficient CNN architectures. Finally, to reduce the large design cost of these manual solutions, we discuss the AutoML framework for each of them, such as neural architecture search (NAS) and automated pruning and quantization.
2195.
Tan, Huixin; Lai, Jiewei; Liu, Yunbi; Song, Yuzhang; Wang, Jinliang; Chen, Mingyang; Yan, Yong; Zhong, Liming; Feng, Qianjin; Yang, Wei
Neural architecture search for real-time quality assessment of wearable multi-lead ECG on mobile devices Journal Article
In: Biomedical Signal Processing and Control, vol. 74, pp. 103495, 2022, ISSN: 1746-8094.
@article{TAN2022103495,
title = {Neural architecture search for real-time quality assessment of wearable multi-lead ECG on mobile devices},
author = {Huixin Tan and Jiewei Lai and Yunbi Liu and Yuzhang Song and Jinliang Wang and Mingyang Chen and Yong Yan and Liming Zhong and Qianjin Feng and Wei Yang},
url = {https://www.sciencedirect.com/science/article/pii/S1746809422000179},
doi = {https://doi.org/10.1016/j.bspc.2022.103495},
issn = {1746-8094},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Biomedical Signal Processing and Control},
volume = {74},
pages = {103495},
abstract = {Users need to pay for the medical services of the wearable ECG signals. The complicated interferences of wearable ECG may cause invalid medical diagnosis. To save medical expenses and reduce the workload of cardiologists, unacceptable wearable ECG should be filtered out on mobile devices in real time. Therefore, a robust and lightweight quality assessment model is essential. Existing convolutional neural network models are more robust than hand-crafted features based models, but most are too cumbersome to run on mobile devices. In this study, we utilize neural architecture search algorithm to search a network with a few parameters and low latency automatically for real-time quality assessment of wearable ECG data. To reach this goal, we construct a searching space with lightweight network blocks and add the hardware latency loss. We also propose a novel and effective strategy, any-lead, to assess the quality of all leads using a single model. We evaluated our method on a large-scale (10,709 signals) wearable 12-lead ECG dataset and a public dataset named Physionet Cinc Challenge 2011. The parameters and FLOPs of the architecture we searched were about 66.76 K and 36.44 M. Our model achieved excellent performance on the aforementioned datasets, with AUC of 98.32% and 97.64%, F1 scores of 94.36% and 93.52%, MCC of 84.74% and 83.22%, respectively, and with the inference time on an Android emulator of about 78 ms. Extensive experimental results demonstrate the effectiveness of our method in assessing the quality of all leads of wearable ECG data on mobile devices in real time.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Users need to pay for the medical services of the wearable ECG signals. The complicated interferences of wearable ECG may cause invalid medical diagnosis. To save medical expenses and reduce the workload of cardiologists, unacceptable wearable ECG should be filtered out on mobile devices in real time. Therefore, a robust and lightweight quality assessment model is essential. Existing convolutional neural network models are more robust than hand-crafted features based models, but most are too cumbersome to run on mobile devices. In this study, we utilize neural architecture search algorithm to search a network with a few parameters and low latency automatically for real-time quality assessment of wearable ECG data. To reach this goal, we construct a searching space with lightweight network blocks and add the hardware latency loss. We also propose a novel and effective strategy, any-lead, to assess the quality of all leads using a single model. We evaluated our method on a large-scale (10,709 signals) wearable 12-lead ECG dataset and a public dataset named Physionet Cinc Challenge 2011. The parameters and FLOPs of the architecture we searched were about 66.76 K and 36.44 M. Our model achieved excellent performance on the aforementioned datasets, with AUC of 98.32% and 97.64%, F1 scores of 94.36% and 93.52%, MCC of 84.74% and 83.22%, respectively, and with the inference time on an Android emulator of about 78 ms. Extensive experimental results demonstrate the effectiveness of our method in assessing the quality of all leads of wearable ECG data on mobile devices in real time.
2194.
Traoré, Kalifou René; Camero, Andrés; Zhu, Xiao Xiang
Landscape of Neural Architecture Search across sensors: how much do they differ ? Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2201-06321,
title = {Landscape of Neural Architecture Search across sensors: how much do they differ ?},
author = {Kalifou René Traoré and Andrés Camero and Xiao Xiang Zhu},
url = {https://arxiv.org/abs/2201.06321},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2201.06321},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2193.
Fedorov, Igor; Navarro, Ramon Matas; Tann, Hokchhay; Zhou, Chuteng; Mattina, Matthew; Whatmough, Paul N.
UDC: Unified DNAS for Compressible TinyML Models Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2201-05842,
title = {UDC: Unified DNAS for Compressible TinyML Models},
author = {Igor Fedorov and Ramon Matas Navarro and Hokchhay Tann and Chuteng Zhou and Matthew Mattina and Paul N. Whatmough},
url = {https://arxiv.org/abs/2201.05842},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2201.05842},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2192.
Liu, Xin; Zhao, Jianwei; Li, Jie; Cao, Bin; Lv, Zhihan
Federated Neural Architecture Search for Medical Data Security Journal Article
In: IEEE Transactions on Industrial Informatics, pp. 1-1, 2022.
@article{9684972,
title = {Federated Neural Architecture Search for Medical Data Security},
author = {Xin Liu and Jianwei Zhao and Jie Li and Bin Cao and Zhihan Lv},
url = {https://ieeexplore.ieee.org/abstract/document/9684972},
doi = {10.1109/TII.2022.3144016},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Industrial Informatics},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2191.
Guo, Qingbei; Wu, Xiao-Jun; Kittler, Josef; Feng, Zhiquan
Differentiable neural architecture learning for efficient neural networks Journal Article
In: Pattern Recognition, vol. 126, pp. 108448, 2022, ISSN: 0031-3203.
@article{GUO2022108448,
title = {Differentiable neural architecture learning for efficient neural networks},
author = {Qingbei Guo and Xiao-Jun Wu and Josef Kittler and Zhiquan Feng},
url = {https://www.sciencedirect.com/science/article/pii/S0031320321006245},
doi = {https://doi.org/10.1016/j.patcog.2021.108448},
issn = {0031-3203},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Pattern Recognition},
volume = {126},
pages = {108448},
abstract = {Efficient neural networks has received ever-increasing attention with the evolution of convolutional neural networks (CNNs), especially involving their deployment on embedded and mobile platforms. One of the biggest problems to obtaining such efficient neural networks is efficiency, even recent differentiable neural architecture search (DNAS) requires to sample a small number of candidate neural architectures for the selection of the optimal neural architecture. To address this computational efficiency issue, we introduce a novel architecture parameterization based on scaled sigmoid function, and propose a general Differentiable Neural Architecture Learning (DNAL) method to obtain efficient neural networks without the need to evaluate candidate neural networks. Specifically, for stochastic supernets as well as conventional CNNs, we build a new channel-wise module layer with the architecture components controlled by a scaled sigmoid function. We train these neural network models from scratch. The network optimization is decoupled into the weight optimization and the architecture optimization, which avoids the interaction between the two types of parameters and alleviates the vanishing gradient problem. We address the non-convex optimization problem of efficient neural networks by the continuous scaled sigmoid method instead of the common softmax method. Extensive experiments demonstrate our DNAL method delivers superior performance in terms of efficiency, and adapts to conventional CNNs (e.g., VGG16 and ResNet50), lightweight CNNs (e.g., MobileNetV2) and stochastic supernets (e.g., ProxylessNAS). The optimal neural networks learned by DNAL surpass those produced by the state-of-the-art methods on the benchmark CIFAR-10 and ImageNet-1K dataset in accuracy, model size and computational complexity. Our source code is available at https://github.com/QingbeiGuo/DNAL.git.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Efficient neural networks has received ever-increasing attention with the evolution of convolutional neural networks (CNNs), especially involving their deployment on embedded and mobile platforms. One of the biggest problems to obtaining such efficient neural networks is efficiency, even recent differentiable neural architecture search (DNAS) requires to sample a small number of candidate neural architectures for the selection of the optimal neural architecture. To address this computational efficiency issue, we introduce a novel architecture parameterization based on scaled sigmoid function, and propose a general Differentiable Neural Architecture Learning (DNAL) method to obtain efficient neural networks without the need to evaluate candidate neural networks. Specifically, for stochastic supernets as well as conventional CNNs, we build a new channel-wise module layer with the architecture components controlled by a scaled sigmoid function. We train these neural network models from scratch. The network optimization is decoupled into the weight optimization and the architecture optimization, which avoids the interaction between the two types of parameters and alleviates the vanishing gradient problem. We address the non-convex optimization problem of efficient neural networks by the continuous scaled sigmoid method instead of the common softmax method. Extensive experiments demonstrate our DNAL method delivers superior performance in terms of efficiency, and adapts to conventional CNNs (e.g., VGG16 and ResNet50), lightweight CNNs (e.g., MobileNetV2) and stochastic supernets (e.g., ProxylessNAS). The optimal neural networks learned by DNAL surpass those produced by the state-of-the-art methods on the benchmark CIFAR-10 and ImageNet-1K dataset in accuracy, model size and computational complexity. Our source code is available at https://github.com/QingbeiGuo/DNAL.git.
2190.
Lin, Qiuzhen; Fang, Zhixiong; Chen, Yi; Tan, Kay Chen; Li, Yun
Evolutionary Architectural Search for Generative Adversarial Networks Journal Article
In: IEEE Transactions on Emerging Topics in Computational Intelligence, pp. 1-12, 2022.
@article{9686068,
title = {Evolutionary Architectural Search for Generative Adversarial Networks},
author = {Qiuzhen Lin and Zhixiong Fang and Yi Chen and Kay Chen Tan and Yun Li},
url = {https://ieeexplore.ieee.org/abstract/document/9686068},
doi = {10.1109/TETCI.2021.3137377},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Emerging Topics in Computational Intelligence},
pages = {1-12},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2189.
Wang, Yabin; Ma, Zhiheng; Wei, Xing; Zheng, Shuai; Wang, Yaowei; Hong, Xiaopeng
ECCNAS: Efficient Crowd Counting Neural Architecture Search Journal Article
In: ACM Trans. Multimedia Comput. Commun. Appl., vol. 18, no. 1s, 2022, ISSN: 1551-6857.
@article{10.1145/3465455,
title = {ECCNAS: Efficient Crowd Counting Neural Architecture Search},
author = {Yabin Wang and Zhiheng Ma and Xing Wei and Shuai Zheng and Yaowei Wang and Xiaopeng Hong},
url = {https://doi.org/10.1145/3465455},
doi = {10.1145/3465455},
issn = {1551-6857},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {ACM Trans. Multimedia Comput. Commun. Appl.},
volume = {18},
number = {1s},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
abstract = {Recent solutions to crowd counting problems have already achieved promising performance across various benchmarks. However, applying these approaches to real-world applications is still challenging, because they are computation intensive and lack the flexibility to meet various resource budgets. In this article, we propose an efficient crowd counting neural architecture search (ECCNAS) framework to search efficient crowd counting network structures, which can fill this research gap. A novel search from pre-trained strategy enables our cross-task NAS to explore the significantly large and flexible search space with less search time and get more proper network structures. Moreover, our well-designed search space can intrinsically provide candidate neural network structures with high performance and efficiency. In order to search network structures according to hardwares with different computational performance, we develop a novel latency cost estimation algorithm in our ECCNAS. Experiments show our searched models get an excellent trade-off between computational complexity and accuracy and have the potential to deploy in practical scenarios with various resource budgets. We reduce the computational cost, in terms of multiply-and-accumulate (MACs), by up to 96% with comparable accuracy. And we further designed experiments to validate the efficiency and the stability improvement of our proposed search from pre-trained strategy.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Recent solutions to crowd counting problems have already achieved promising performance across various benchmarks. However, applying these approaches to real-world applications is still challenging, because they are computation intensive and lack the flexibility to meet various resource budgets. In this article, we propose an efficient crowd counting neural architecture search (ECCNAS) framework to search efficient crowd counting network structures, which can fill this research gap. A novel search from pre-trained strategy enables our cross-task NAS to explore the significantly large and flexible search space with less search time and get more proper network structures. Moreover, our well-designed search space can intrinsically provide candidate neural network structures with high performance and efficiency. In order to search network structures according to hardwares with different computational performance, we develop a novel latency cost estimation algorithm in our ECCNAS. Experiments show our searched models get an excellent trade-off between computational complexity and accuracy and have the potential to deploy in practical scenarios with various resource budgets. We reduce the computational cost, in terms of multiply-and-accumulate (MACs), by up to 96% with comparable accuracy. And we further designed experiments to validate the efficiency and the stability improvement of our proposed search from pre-trained strategy.
2188.
Kim, Youngeun; Li, Yuhang; Park, Hyoungseob; Venkatesha, Yeshwanth; Panda, Priyadarshini
Neural Architecture Search for Spiking Neural Networks Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2201-10355,
title = {Neural Architecture Search for Spiking Neural Networks},
author = {Youngeun Kim and Yuhang Li and Hyoungseob Park and Yeshwanth Venkatesha and Priyadarshini Panda},
url = {https://arxiv.org/abs/2201.10355},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2201.10355},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2187.
Rho, Daniel; Park, Jinhyeok; Ko, Jong Hwan
NAS-VAD: Neural Architecture Search for Voice Activity Detection Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2201-09032,
title = {NAS-VAD: Neural Architecture Search for Voice Activity Detection},
author = {Daniel Rho and Jinhyeok Park and Jong Hwan Ko},
url = {https://arxiv.org/abs/2201.09032},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2201.09032},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2186.
Shu, Yao; Dai, Zhongxiang; Wu, Zhaoxuan; Low, Bryan Kian Hsiang
Unifying and Boosting Gradient-Based Training-Free Neural Architecture Search Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2201-09785,
title = {Unifying and Boosting Gradient-Based Training-Free Neural Architecture Search},
author = {Yao Shu and Zhongxiang Dai and Zhaoxuan Wu and Bryan Kian Hsiang Low},
url = {https://arxiv.org/abs/2201.09785},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2201.09785},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2185.
Chen, Mingzhe; Xiao, Xi; Zhang, Bin; Liu, Xinyu; Lu, Runiu
Neural Architecture Searching for Facial Attributes-based Depression Recognition Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2201-09799,
title = {Neural Architecture Searching for Facial Attributes-based Depression Recognition},
author = {Mingzhe Chen and Xi Xiao and Bin Zhang and Xinyu Liu and Runiu Lu},
url = {https://arxiv.org/abs/2201.09799},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2201.09799},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2184.
Mao, Pengli; Lin, Yan; Xue, Song; Zhang, Baochang
Remaining Useful Life Estimation of Aircraft Engines Using Differentiable Architecture Search Journal Article
In: Mathematics, vol. 10, no. 3, 2022, ISSN: 2227-7390.
@article{math10030352,
title = {Remaining Useful Life Estimation of Aircraft Engines Using Differentiable Architecture Search},
author = {Pengli Mao and Yan Lin and Song Xue and Baochang Zhang},
url = {https://www.mdpi.com/2227-7390/10/3/352},
doi = {10.3390/math10030352},
issn = {2227-7390},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Mathematics},
volume = {10},
number = {3},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2183.
Zhang, Xiaofan; Zhou, Zongwei; Chen, Deming; Wang, Yu Emma
AutoDistill: an End-to-End Framework to Explore and Distill Hardware-Efficient Language Models Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2201-08539,
title = {AutoDistill: an End-to-End Framework to Explore and Distill Hardware-Efficient Language Models},
author = {Xiaofan Zhang and Zongwei Zhou and Deming Chen and Yu Emma Wang},
url = {https://arxiv.org/abs/2201.08539},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2201.08539},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2182.
Litany, Or; Maron, Haggai; Acuna, David; Kautz, Jan; Chechik, Gal; Fidler, Sanja
Federated Learning with Heterogeneous Architectures using Graph HyperNetworks Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2201-08459,
title = {Federated Learning with Heterogeneous Architectures using Graph HyperNetworks},
author = {Or Litany and Haggai Maron and David Acuna and Jan Kautz and Gal Chechik and Sanja Fidler},
url = {https://arxiv.org/abs/2201.08459},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2201.08459},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2181.
Hong, Zhenhou; Wang, Jianzong; Qu, Xiaoyang; Zhao, Chendong; Liu, Jie; Xiao, Jing
Neural Architecture Search as Self-assessor in Semi-supervised Learning Proceedings Article
In: Liao, Xiangke; Zhao, Wei; Chen, Enhong; Xiao, Nong; Wang, Li; Gao, Yang; Shi, Yinghuan; Wang, Changdong; Huang, Dan (Ed.): Big Data, pp. 95–107, Springer Singapore, Singapore, 2022, ISBN: 978-981-16-9709-8.
@inproceedings{10.1007/978-981-16-9709-8_7,
title = {Neural Architecture Search as Self-assessor in Semi-supervised Learning},
author = {Zhenhou Hong and Jianzong Wang and Xiaoyang Qu and Chendong Zhao and Jie Liu and Jing Xiao},
editor = {Xiangke Liao and Wei Zhao and Enhong Chen and Nong Xiao and Li Wang and Yang Gao and Yinghuan Shi and Changdong Wang and Dan Huang},
url = {https://link.springer.com/chapter/10.1007/978-981-16-9709-8_7},
isbn = {978-981-16-9709-8},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Big Data},
pages = {95--107},
publisher = {Springer Singapore},
address = {Singapore},
abstract = {Neural Architecture Search (NAS) forms powerful automatic learning, which has helped achieve remarkable performance in several applications in recent years. Previous research focused on NAS in standard supervised learning to explore its performance, requiring labeled data. In this paper, our goal is to examine the implementation of NAS with large amounts of unlabeled data. We propose the NAS as a self-assessor, called NAS-SA, by adding the consistency method and prior knowledge. We design an adaptive search strategy, a balanced search space, and a multi-object optimization to generate a robust and efficient small model in NAS-SA. The image and text classification tasks proved that our NAS-SA method had achieved the best performance.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Neural Architecture Search (NAS) forms powerful automatic learning, which has helped achieve remarkable performance in several applications in recent years. Previous research focused on NAS in standard supervised learning to explore its performance, requiring labeled data. In this paper, our goal is to examine the implementation of NAS with large amounts of unlabeled data. We propose the NAS as a self-assessor, called NAS-SA, by adding the consistency method and prior knowledge. We design an adaptive search strategy, a balanced search space, and a multi-object optimization to generate a robust and efficient small model in NAS-SA. The image and text classification tasks proved that our NAS-SA method had achieved the best performance.