AutoML | Literature on Neural Architecture Search

@article{YANG2021,

title = {Ultra-lightweight CNN design based on neural architecture search and knowledge distillation: A novel method to build the automatic recognition model of space target ISAR images},

author = {Hong Yang and Ya-sheng Zhang and Can-bin Yin and Wen-zhe Ding},

url = {https://www.sciencedirect.com/science/article/pii/S2214914721000763},

doi = {https://doi.org/10.1016/j.dt.2021.04.014},

issn = {2214-9147},

year  = {2021},

date = {2021-01-01},

journal = {Defence Technology},

abstract = {In this paper, a novel method of ultra-lightweight convolution neural network (CNN) design based on neural architecture search (NAS) and knowledge distillation (KD) is proposed. It can realize the automatic construction of the space target inverse synthetic aperture radar (ISAR) image recognition model with ultra-lightweight and high accuracy. This method introduces the NAS method into the radar image recognition for the first time, which solves the time-consuming and labor-consuming problems in the artificial design of the space target ISAR image automatic recognition model (STIIARM). On this basis, the NAS model's knowledge is transferred to the student model with lower computational complexity by the flow of the solution procedure (FSP) distillation method. Thus, the decline of recognition accuracy caused by the direct compression of model structural parameters can be effectively avoided, and the ultra-lightweight STIIARM can be obtained. In the method, the Inverted Linear Bottleneck (ILB) and Inverted Residual Block (IRB) are firstly taken as each block's basic structure in CNN. And the expansion ratio, output filter size, number of IRBs, and convolution kernel size are set as the search parameters to construct a hierarchical decomposition search space. Then, the recognition accuracy and computational complexity are taken as the objective function and constraint conditions, respectively, and the global optimization model of the CNN architecture search is established. Next, the simulated annealing (SA) algorithm is used as the search strategy to search out the lightweight and high accuracy STIIARM directly. After that, based on the three principles of similar block structure, the same corresponding channel number, and the minimum computational complexity, the more lightweight student model is designed, and the FSP matrix pairing between the NAS model and student model is completed. Finally, by minimizing the loss between the FSP matrix pairs of the NAS model and student model, the student model's weight adjustment is completed. Thus the ultra-lightweight and high accuracy STIIARM is obtained. The proposed method's effectiveness is verified by the simulation experiments on the ISAR image dataset of five types of space targets.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

Close

In this paper, a novel method of ultra-lightweight convolution neural network (CNN) design based on neural architecture search (NAS) and knowledge distillation (KD) is proposed. It can realize the automatic construction of the space target inverse synthetic aperture radar (ISAR) image recognition model with ultra-lightweight and high accuracy. This method introduces the NAS method into the radar image recognition for the first time, which solves the time-consuming and labor-consuming problems in the artificial design of the space target ISAR image automatic recognition model (STIIARM). On this basis, the NAS model's knowledge is transferred to the student model with lower computational complexity by the flow of the solution procedure (FSP) distillation method. Thus, the decline of recognition accuracy caused by the direct compression of model structural parameters can be effectively avoided, and the ultra-lightweight STIIARM can be obtained. In the method, the Inverted Linear Bottleneck (ILB) and Inverted Residual Block (IRB) are firstly taken as each block's basic structure in CNN. And the expansion ratio, output filter size, number of IRBs, and convolution kernel size are set as the search parameters to construct a hierarchical decomposition search space. Then, the recognition accuracy and computational complexity are taken as the objective function and constraint conditions, respectively, and the global optimization model of the CNN architecture search is established. Next, the simulated annealing (SA) algorithm is used as the search strategy to search out the lightweight and high accuracy STIIARM directly. After that, based on the three principles of similar block structure, the same corresponding channel number, and the minimum computational complexity, the more lightweight student model is designed, and the FSP matrix pairing between the NAS model and student model is completed. Finally, by minimizing the loss between the FSP matrix pairs of the NAS model and student model, the student model's weight adjustment is completed. Thus the ultra-lightweight and high accuracy STIIARM is obtained. The proposed method's effectiveness is verified by the simulation experiments on the ISAR image dataset of five types of space targets.

Close

1412.

Zhou, Zheng; Li, Tianfu; Zhang, Zilong; Zhao, Zhibin; Sun, Chuang; Yan, Ruqiang; Chen, Xuefeng

Bayesian Differentiable Architecture Search for Efficient Domain Matching Fault Diagnosis Journal Article

In: IEEE Transactions on Instrumentation and Measurement, vol. 70, pp. 1-11, 2021.

Links | BibTeX

1411.

Zhang, Lingling; Wang, Shaowei; Chang, Xiaojun; Liu, Jun; Ge, Zongyuan; Zheng, Qinghua

Auto-FSL: Searching the Attribute Consistent Network for Few-Shot Learning Journal Article

In: IEEE Transactions on Circuits and Systems for Video Technology, pp. 1-1, 2021.

Links | BibTeX

1410.

Bruggemann, David; Kanakis, Menelaos; Obukhov, Anton; Georgoulis, Stamatios; Gool, Luc Van

Exploring Relational Context for Multi-Task Dense Prediction Technical Report

2021.

Links | BibTeX

1409.

Valsesia, Diego; Fracastoro, Giulia; Magli, Enrico

Don’t Stack Layers In Graph Neural Networks, Wire Them Randomly Proceedings Article

In: ICLR 2021 Workshop on Geometrical and Topological Representation Learning, 2021.

Links | BibTeX

1408.

-, Julia Guerrero; Hauns, Sven; Izquierdo, Sergio; Miotto, Guilherme; Schrodi, Simon; Biedenkapp, Andre; Elsken, Thomas; Deng, Difan; Lindauer, Marius; Hutter, Frank

Bag of Baselines for Multi-objective Joint Neural Architecture Search and Hyperparameter Optimization Proceedings Article

In: 8th ICML Workshop on Automated Machine Learning, 2021.

Links | BibTeX

1407.

Lv, Jindi; Ye, Qing; Sun, Yanan; Zhao, Juan; Lv, Jiancheng

Heart-Darts: Classification of Heartbeats Using Differentiable Architecture Search Technical Report

2021.

Links | BibTeX

1406.

Mo, Hyunho; Custode, Leonardo Lucio; Iacca, Giovanni

Evolutionary neural architecture search for remaining useful life prediction Journal Article

In: Applied Soft Computing, vol. 108, pp. 107474, 2021, ISSN: 1568-4946.

Abstract | Links | BibTeX

@article{MO2021107474,

title = {Evolutionary neural architecture search for remaining useful life prediction},

author = {Hyunho Mo and Leonardo Lucio Custode and Giovanni Iacca},

url = {https://www.sciencedirect.com/science/article/pii/S1568494621003975},

doi = {https://doi.org/10.1016/j.asoc.2021.107474},

issn = {1568-4946},

year  = {2021},

date = {2021-01-01},

journal = {Applied Soft Computing},

volume = {108},

pages = {107474},

abstract = {With the advent of Industry 4.0, making accurate predictions of the remaining useful life (RUL) of industrial components has become a crucial aspect in predictive maintenance (PdM). To this aim, various Deep Neural Network (DNN) models have been proposed in the recent literature. However, while the architectures of these models have a large impact on their performance, they are usually determined empirically. To exclude the time-consuming process and the unnecessary computational cost of manually engineering these models, we present a Neural Architecture Search (NAS) technique based on an Evolutionary Algorithm (EA) applied to optimize the architecture of a DNN used to predict the RUL. The EA explores the combinatorial parameter space of a multi-head Convolutional Neural Network with Long Short Term Memory (CNN-LSTM) to search for the best architecture. In particular, our method requires minimum computational resources by making use of an early stopping policy and a history of the evaluated architectures. We dub the proposed method ENAS-PdM. To our knowledge, this is the first work where an EA-based NAS is used to optimize a CNN-LSTM architecture in the field of PdM. In our experiments, we use the well-established Commercial Modular Aero-Propulsion System Simulation (C-MAPSS) dataset from NASA. Compared to the current state-of-the-art, our method obtains better results in terms of two different metrics, RMSE and Score, when aggregating across all the C-MAPSS sub-datasets. Without aggregation, we achieve lower RMSE in 3 out of 4 sub-datasets. Our experimental results verify that the proposed method is a reliable tool for obtaining state-of-the-art RUL predictions and as such it can have a strong impact in several industrial applications, especially those with limited available computing power.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

Close

1405.

Chen, Haoze; Zhang, Zhijie; Zhao, Chenyang; Liu, Jiaqi; Yin, Wuliang; Li, Yanfeng; Wang, Fengxiang; Li, Chao; Lin, Zhenyu

Depth Classification of Defects Based on Neural Architecture Search Journal Article

In: IEEE Access, vol. 9, pp. 73424-73432, 2021.

Links | BibTeX