AutoML | Literature on Neural Architecture Search

In: Castillo, Pedro A.; Laredo, Juan Luis Jim'enez (Ed.): Applications of Evolutionary Computation: 24th International Conference, EvoApplications 2021, Held as Part of EvoStar 2021, Virtual Event, April 7--9, 2021, Proceedings, pp. 291, Springer Nature 2021.

Links | BibTeX

1466.

White, Colin; Zela, Arber; Ru, Binxin; Liu, Yang; Hutter, Frank

How Powerful are Performance Predictors in Neural Architecture Search? Technical Report

2021.

Links | BibTeX

1465.

Ren, Pengzhen; Xiao, Gang; Chang, Xiaojun; Xiao, Yun; Li, Zhihui; Chen, Xiaojiang

NAS-TC: Neural Architecture Search on Temporal Convolutions for Complex Action Recognition Technical Report

2021.

Links | BibTeX

1464.

Dong, Heyou; Chen, Dan; Zhang, Lei; Ke, Hengjin; Li, Xiaoli

Subject Sensitive EEG Discrimination with Fast Reconstructable CNN Driven by Reinforcement Learning: A Case Study of ASD Evaluation Journal Article

In: Neurocomputing, 2021, ISSN: 0925-2312.

Abstract | Links | BibTeX

@article{DONG2021,

title = {Subject Sensitive EEG Discrimination with Fast Reconstructable CNN Driven by Reinforcement Learning: A Case Study of ASD Evaluation},

author = {Heyou Dong and Dan Chen and Lei Zhang and Hengjin Ke and Xiaoli Li},

url = {https://www.sciencedirect.com/science/article/pii/S0925231221005270},

doi = {https://doi.org/10.1016/j.neucom.2021.04.009},

issn = {0925-2312},

year  = {2021},

date = {2021-01-01},

journal = {Neurocomputing},

abstract = {Recent Electroencephalogram (EEG) analysis in connection with brain disorders has been tremendously benefiting from the (Deep) Neural Network technology in neuroscience research and neuro-engineering practices. However, the performance of existing hand-crafted models, such as the stability, has largely been refrained. This is the case especially in the paradigms that sensitive to the individuality of subjects and the non-stationarity of cognitive dynamics, such as Autism Spectrum Disorder (ASD) evaluation. Aiming at this problem, this study develops a Q-Learning method to enable fast reconstruction of Convolutional Neural Network (CNN) thus to support EEG discrimination adapting to the individuality of subjects under examination. The proposed method first generates a CNN model with the structure and hyper-parameters determined (i.e., Neural Architecture Search) by the customized Q-Learning algorithm, where the CNN model is treated as a discrete system to be optimized. With the sharp shift of subjects, the Q-Learning algorithm reconstructs the CNN model to reach optimization reusing the tacit knowledge learned from the previous trials. A case study has been performed to check the proposed method versus state-of-the-art counterparts based on resting-state EEG collected from 175 ASD-suspicious children with a diverse geological distribution. The observations in the case study indicate that: 1) the method outperforms the counterparts with an individual/sample accuracy of 92.63%/83.23% achieved; 2) the method can quickly reconstruct the CNN model with the group of subjects shifting from one region to another to maintain an encouraging performance while the counterparts without reconstruction may drop by about 12%.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

Close

1463.

Shi, Xian; Zhou, Pan; Chen, Wei; Xie, Lei

Darts-Conformer: Towards Efficient Gradient-Based Neural Architecture Search For End-to-End ASR Technical Report

2021.

Links | BibTeX

1462.

Ganepola, Vayangi Vishmi Vishara; Wirasingha, Torin

Automating Generative Adversarial Networks using Neural Architecture Search: A Review Proceedings Article

In: 2021 International Conference on Emerging Smart Computing and Informatics (ESCI), pp. 577-582, 2021.

Links | BibTeX

1461.

Liu, Hongying; Xu, Derong; Zhu, Tianwen; Shang, Fanhua; Liu, Yuanyuan; Lu, Jianhua; Yang, Ri

Graph Convolutional Networks by Architecture Search for PolSAR Image Classification Journal Article

In: Remote Sensing, vol. 13, no. 7, 2021, ISSN: 2072-4292.

Abstract | Links | BibTeX

@article{rs13071404,

title = {Graph Convolutional Networks by Architecture Search for PolSAR Image Classification},

author = {Hongying Liu and Derong Xu and Tianwen Zhu and Fanhua Shang and Yuanyuan Liu and Jianhua Lu and Ri Yang},

url = {https://www.mdpi.com/2072-4292/13/7/1404},

doi = {10.3390/rs13071404},

issn = {2072-4292},

year  = {2021},

date = {2021-01-01},

journal = {Remote Sensing},

volume = {13},

number = {7},

abstract = {Classification of polarimetric synthetic aperture radar (PolSAR) images has achieved good results due to the excellent fitting ability of neural networks with a large number of training samples. However, the performance of most convolutional neural networks (CNNs) degrades dramatically when only a few labeled training samples are available. As one well-known class of semi-supervised learning methods, graph convolutional networks (GCNs) have gained much attention recently to address the classification problem with only a few labeled samples. As the number of layers grows in the network, the parameters dramatically increase. It is challenging to determine an optimal architecture manually. In this paper, we propose a neural architecture search method based GCN (ASGCN) for the classification of PolSAR images. We construct a novel graph whose nodes combines both the physical features and spatial relations between pixels or samples to represent the image. Then we build a new searching space whose components are empirically selected from some graph neural networks for architecture search and develop the differentiable architecture search method to construction our ASGCN. Moreover, to address the training of large-scale images, we present a new weighted mini-batch algorithm to reduce the computing memory consumption and ensure the balance of sample distribution, and also analyze and compare with other similar training strategies. Experiments on several real-world PolSAR datasets show that our method has improved the overall accuracy as much as 3.76% than state-of-the-art methods.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

Close

1460.

Ge, Wanying; Panariello, Michele; Patino, Jose; Todisco, Massimiliano; Evans, Nicholas W D

Partially-Connected Differentiable Architecture Search for Deepfake and Spoofing Detection Technical Report

2021.

Links | BibTeX

1459.

Yu, Kaicheng; é, Ren; Salzmann, Mathieu

Landmark Regularization: Ranking Guided Super-Net Training in Neural Architecture Search Technical Report

2021.

Links | BibTeX

1458.

Liu, Yukun; Li, Ta; Zhang, Pengyuan; Yan, Yonghong

Improved Conformer-based End-to-End Speech Recognition Using Neural Architecture Search Technical Report

2021.

Links | BibTeX

1457.

Wang, Chunnan; Chen, Bozhou; Li, Geng; Wang, Hongzhi

FL-AGCNS: Federated Learning Framework for Automatic Graph Convolutional Network Search Technical Report

2021.

Links | BibTeX

1456.

Meng, Tianjian; Chen, Xiaohan; Jiang, Yifan; Wang, Zhangyang

A Design Space Study for LISTA and Beyond Proceedings Article

In: International Conference on Learning Representations, 2021.

Links | BibTeX

1455.

Liu, Aoming; Huang, Zehao; Huang, Zhiwu; Wang, Naiyan

Direct Differentiable Augmentation Search Technical Report

2021.

Links | BibTeX

1454.

Tashiro, Kyosuke; Takeda, Koji; Tanaka, Kanji; Tomoe, Hiroki

TaylorMade VDD: Domain-adaptive Visual Defect Detector for High-mix Low-volume Production of Non-convex Cylindrical Metal Objects Technical Report

2021.

Links | BibTeX

1453.

Peter, David; Roth, Wolfgang; Pernkopf, Franz

End-to-end Keyword Spotting using Neural Architecture Search and Quantization Technical Report

2021.

Links | BibTeX

1452.

Mundt, Martin; Pliushch, Iuliia; Ramesh, Visvanathan

Neural Architecture Search of Deep Priors: Towards Continual Learning without Catastrophic Interference Technical Report

2021.

Links | BibTeX

1451.

Zhao, Huan; Yao, Quanming; -, Wei

Search to aggregate neighborhood for graph neural network Technical Report

2021.

Links | BibTeX

1450.

Xue, Chao; Wang, Xiaoxing; Yan, Junchi; Hu, Yonggang; Yang, Xiaokang; Sun, Kewei

Rethinking Bi-Level Optimization in Neural Architecture Search: A Gibbs Sampling Perspective Proceedings Article

In: AAAI 2021, 2021.

Links | BibTeX

1449.

Yin, Yihang; Huang, Siyu; Zhang, Xiang; Dou, Dejing

BM-NAS: Bilevel Multimodal Neural Architecture Search Technical Report

2021.

Links | BibTeX

1448.

Ahn, Joon Young; Cho, Nam Ik

Neural Architecture Search for Image Super-Resolution Using Densely Constructed Search Space: DeCoNAS Technical Report

2021.

Links | BibTeX

1447.

Wang, Jiahao; Shu, Han; Xia, Weihao; Yang, Yujiu; Wang, Yunhe

Coarse-to-Fine Searching for Efficient Generative Adversarial Networks Technical Report

2021.

Links | BibTeX

1446.

Yan, Bin; Peng, Houwen; Wu, Kan; Wang, Dong; Fu, Jianlong; Lu, Huchuan

LightTrack: Finding Lightweight Neural Networks for Object Tracking via One-Shot Architecture Search Proceedings Article

In: CVPR2021, 2021.

Links | BibTeX

1445.

Chen, Minghao; Peng, Houwen; Fu, Jianlong; Ling, Haibin

One-Shot Neural Ensemble Architecture Search by Diversity-Guided Search Space Shrinking Proceedings Article

In: CVPR2021, 2021.

Links | BibTeX

1444.

Verma, Monu; Reddy, Satish Kumar M; Meedimale, Yashwanth Reddy; Mandal, Murari; Vipparthi, Santosh Kumar

AutoMER: Spatiotemporal Neural Architecture Search for Microexpression Recognition Journal Article

In: IEEE Transactions on Neural Networks and Learning Systems, pp. 1-13, 2021.

Links | BibTeX

1443.

Wu, Haiwei; Zhou, Jiantao

IID-Net: Image Inpainting Detection Network via Neural Architecture Search and Attention Journal Article

In: IEEE Transactions on Circuits and Systems for Video Technology, pp. 1-1, 2021.

Links | BibTeX

1442.

Song, Seok Bin; Nam, Jung Woo; Kim, Jin Heon

NAS-PPG: PPG based heart rate estimation using Neural Architecture Search Journal Article

In: IEEE Sensors Journal, pp. 1-1, 2021.

Links | BibTeX

1441.

Bodin, Erik; Tomasi, Federico; Dai, Zhenwen

Making Differentiable Architecture Search less local Technical Report

2021.

Links | BibTeX

1440.

-, Min; -, Hao; -, Min; -, Yu; -, Hsien; -, Yi; -, Hung; Jou, Kevin

Network Space Search for Pareto-Efficient Spaces Technical Report

2021.

Links | BibTeX

1439.

Duggal, Rahul; Zhou, Hao; Yang, Shuo; Xiong, Yuanjun; Xia, Wei; Tu, Zhuowen; Soatto, Stefano

Compatibility-aware Heterogeneous Visual Search Technical Report

2021.

Links | BibTeX

1438.

Di, Shimin; Yao, Quanming; Chen, Lei

Searching to Sparsify Tensor Decomposition for N-ary Relational Data Technical Report

2021.

Links | BibTeX

1437.

Di, Shimin; Yao, Quanming; Zhang, Yongqi; Chen, Lei

Efficient Relation-aware Scoring Function Search for Knowledge Graph Embedding Technical Report

2021.

Links | BibTeX

1436.

Clemens, John

MLDS: A Dataset for Weight-Space Analysis of Neural Networks Technical Report

2021.

Links | BibTeX

1435.

Pasini, Massimiliano Lupo; Yin, Junqi; Li, Ying Wai; Eisenbach, Markus

A scalable algorithm for the optimization of neural network architectures Journal Article

In: Parallel Computing, vol. 104-105, pp. 102788, 2021, ISSN: 0167-8191.

Abstract | Links | BibTeX

1434.

Yang, Zhao; Sun, Qingshuang

Efficient Resource-aware Neural Architecture Search with Dynamic Adaptive Network Sampling Proceedings Article

In: 2021 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1-5, 2021.

Links | BibTeX

1433.

Amer, Mohammed; á, Tom; Liao, Iman Yi

Balancing Accuracy and Latency in Multipath Neural Networks Technical Report

2021.

Links | BibTeX

1432.

Lei, Zhou; Yang, Kangkang; Jiang, Kai; Chen, Shengbo

KDAS-ReID: Architecture Search for Person Re-Identification via Distilled Knowledge with Dynamic Temperature Journal Article

In: Algorithms, vol. 14, no. 5, 2021, ISSN: 1999-4893.

Abstract | Links | BibTeX

1431.

Laube, Kevin Alexander; Zell, Andreas

Inter-choice dependent super-network weights Technical Report

2021.

Links | BibTeX

1430.

Chen, Oscal Tzyh-Chiang; Zhang, Yu Cheng; Chang, Yu-Xuan; Chang, Yu-Lung

Iterative Multiple-Path One-Shot NAS for the Optimized Performance Proceedings Article

In: 2021 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1-5, 2021.

Links | BibTeX

1429.

Risso, Matteo; Burrello, Alessio; Pagliari, Daniele Jahier; Benatti, Simone; Macii, Enrico; Benini, Luca; Poncino, Massimo

Robust and Energy-efficient PPG-based Heart-Rate Monitoring Proceedings Article

In: 2021 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1-5, 2021.

Links | BibTeX

1428.

Byun, Younghoon; Lee, Youngjoo

Rapid Design Space Exploration of Near-Optimal Memory-Reduced DCNN Architecture using Multiple Model Compression Techniques Proceedings Article

In: 2021 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1-5, 2021.

Links | BibTeX

1427.

Oymak, Samet; Li, Mingchen; Soltanolkotabi, Mahdi

Generalization Guarantees for Neural Architecture Search with Train-Validation Split Technical Report

2021.

Links | BibTeX

1426.

Yan, Bin; Peng, Houwen; Wu, Kan; Wang, Dong; Fu, Jianlong; Lu, Huchuan

LightTrack: Finding Lightweight Neural Networks for Object Tracking via One-Shot Architecture Search Technical Report

2021.

Links | BibTeX

1425.

Yang, Hong; Zhang, Ya-sheng; Yin, Can-bin; Ding, Wen-zhe

Ultra-lightweight CNN design based on neural architecture search and knowledge distillation: A novel method to build the automatic recognition model of space target ISAR images Journal Article

In: Defence Technology, 2021, ISSN: 2214-9147.

Abstract | Links | BibTeX

@article{YANG2021,

title = {Ultra-lightweight CNN design based on neural architecture search and knowledge distillation: A novel method to build the automatic recognition model of space target ISAR images},

author = {Hong Yang and Ya-sheng Zhang and Can-bin Yin and Wen-zhe Ding},

url = {https://www.sciencedirect.com/science/article/pii/S2214914721000763},

doi = {https://doi.org/10.1016/j.dt.2021.04.014},

issn = {2214-9147},

year  = {2021},

date = {2021-01-01},

journal = {Defence Technology},

abstract = {In this paper, a novel method of ultra-lightweight convolution neural network (CNN) design based on neural architecture search (NAS) and knowledge distillation (KD) is proposed. It can realize the automatic construction of the space target inverse synthetic aperture radar (ISAR) image recognition model with ultra-lightweight and high accuracy. This method introduces the NAS method into the radar image recognition for the first time, which solves the time-consuming and labor-consuming problems in the artificial design of the space target ISAR image automatic recognition model (STIIARM). On this basis, the NAS model's knowledge is transferred to the student model with lower computational complexity by the flow of the solution procedure (FSP) distillation method. Thus, the decline of recognition accuracy caused by the direct compression of model structural parameters can be effectively avoided, and the ultra-lightweight STIIARM can be obtained. In the method, the Inverted Linear Bottleneck (ILB) and Inverted Residual Block (IRB) are firstly taken as each block's basic structure in CNN. And the expansion ratio, output filter size, number of IRBs, and convolution kernel size are set as the search parameters to construct a hierarchical decomposition search space. Then, the recognition accuracy and computational complexity are taken as the objective function and constraint conditions, respectively, and the global optimization model of the CNN architecture search is established. Next, the simulated annealing (SA) algorithm is used as the search strategy to search out the lightweight and high accuracy STIIARM directly. After that, based on the three principles of similar block structure, the same corresponding channel number, and the minimum computational complexity, the more lightweight student model is designed, and the FSP matrix pairing between the NAS model and student model is completed. Finally, by minimizing the loss between the FSP matrix pairs of the NAS model and student model, the student model's weight adjustment is completed. Thus the ultra-lightweight and high accuracy STIIARM is obtained. The proposed method's effectiveness is verified by the simulation experiments on the ISAR image dataset of five types of space targets.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

Close

In this paper, a novel method of ultra-lightweight convolution neural network (CNN) design based on neural architecture search (NAS) and knowledge distillation (KD) is proposed. It can realize the automatic construction of the space target inverse synthetic aperture radar (ISAR) image recognition model with ultra-lightweight and high accuracy. This method introduces the NAS method into the radar image recognition for the first time, which solves the time-consuming and labor-consuming problems in the artificial design of the space target ISAR image automatic recognition model (STIIARM). On this basis, the NAS model's knowledge is transferred to the student model with lower computational complexity by the flow of the solution procedure (FSP) distillation method. Thus, the decline of recognition accuracy caused by the direct compression of model structural parameters can be effectively avoided, and the ultra-lightweight STIIARM can be obtained. In the method, the Inverted Linear Bottleneck (ILB) and Inverted Residual Block (IRB) are firstly taken as each block's basic structure in CNN. And the expansion ratio, output filter size, number of IRBs, and convolution kernel size are set as the search parameters to construct a hierarchical decomposition search space. Then, the recognition accuracy and computational complexity are taken as the objective function and constraint conditions, respectively, and the global optimization model of the CNN architecture search is established. Next, the simulated annealing (SA) algorithm is used as the search strategy to search out the lightweight and high accuracy STIIARM directly. After that, based on the three principles of similar block structure, the same corresponding channel number, and the minimum computational complexity, the more lightweight student model is designed, and the FSP matrix pairing between the NAS model and student model is completed. Finally, by minimizing the loss between the FSP matrix pairs of the NAS model and student model, the student model's weight adjustment is completed. Thus the ultra-lightweight and high accuracy STIIARM is obtained. The proposed method's effectiveness is verified by the simulation experiments on the ISAR image dataset of five types of space targets.

Close