AutoML | Literature on Neural Architecture Search

@article{BALDEONCALISTO2021102154,

title = {EMONAS-Net: Efficient multiobjective neural architecture search using surrogate-assisted evolutionary algorithm for 3D medical image segmentation},

author = {Maria Baldeon Calisto and Susana K. Lai-Yuen},

url = {https://www.sciencedirect.com/science/article/pii/S0933365721001470},

doi = {https://doi.org/10.1016/j.artmed.2021.102154},

issn = {0933-3657},

year  = {2021},

date = {2021-01-01},

urldate = {2021-01-01},

journal = {Artificial Intelligence in Medicine},

volume = {119},

pages = {102154},

abstract = {Deep learning plays a critical role in medical image segmentation. Nevertheless, manually designing a neural network for a specific segmentation problem is a very difficult and time-consuming task due to the massive hyperparameter search space, long training time and large volumetric data. Therefore, most designed networks are highly complex, task specific and over-parametrized. Recently, multiobjective neural architecture search (NAS) methods have been proposed to automate the design of accurate and efficient segmentation architectures. However, they only search for either the micro- or macro-structure of the architecture, do not use the information produced during the optimization process to increase the efficiency of the search, or do not consider the volumetric nature of medical images. In this work, we present EMONAS-Net, an Efficient MultiObjective NAS framework for 3D medical image segmentation that optimizes both the segmentation accuracy and size of the network. EMONAS-Net has two key components, a novel search space that considers the configuration of the micro- and macro-structure of the architecture and a Surrogate-assisted Multiobjective Evolutionary based Algorithm (SaMEA algorithm) that efficiently searches for the best hyperparameter values. The SaMEA algorithm uses the information collected during the initial generations of the evolutionary process to identify the most promising subproblems and select the best performing hyperparameter values during mutation to improve the convergence speed. Furthermore, a Random Forest surrogate model is incorporated to accelerate the fitness evaluation of the candidate architectures. EMONAS-Net is tested on the tasks of prostate segmentation from the MICCAI PROMISE12 challenge, hippocampus segmentation from the Medical Segmentation Decathlon challenge, and cardiac segmentation from the MICCAI ACDC challenge. In all the benchmarks, the proposed framework finds architectures that perform better or comparable with competing state-of-the-art NAS methods while being considerably smaller and reducing the architecture search time by more than 50%.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

Close

Deep learning plays a critical role in medical image segmentation. Nevertheless, manually designing a neural network for a specific segmentation problem is a very difficult and time-consuming task due to the massive hyperparameter search space, long training time and large volumetric data. Therefore, most designed networks are highly complex, task specific and over-parametrized. Recently, multiobjective neural architecture search (NAS) methods have been proposed to automate the design of accurate and efficient segmentation architectures. However, they only search for either the micro- or macro-structure of the architecture, do not use the information produced during the optimization process to increase the efficiency of the search, or do not consider the volumetric nature of medical images. In this work, we present EMONAS-Net, an Efficient MultiObjective NAS framework for 3D medical image segmentation that optimizes both the segmentation accuracy and size of the network. EMONAS-Net has two key components, a novel search space that considers the configuration of the micro- and macro-structure of the architecture and a Surrogate-assisted Multiobjective Evolutionary based Algorithm (SaMEA algorithm) that efficiently searches for the best hyperparameter values. The SaMEA algorithm uses the information collected during the initial generations of the evolutionary process to identify the most promising subproblems and select the best performing hyperparameter values during mutation to improve the convergence speed. Furthermore, a Random Forest surrogate model is incorporated to accelerate the fitness evaluation of the candidate architectures. EMONAS-Net is tested on the tasks of prostate segmentation from the MICCAI PROMISE12 challenge, hippocampus segmentation from the Medical Segmentation Decathlon challenge, and cardiac segmentation from the MICCAI ACDC challenge. In all the benchmarks, the proposed framework finds architectures that perform better or comparable with competing state-of-the-art NAS methods while being considerably smaller and reducing the architecture search time by more than 50%.

Close

1256.

Boutros, Fadi; Siebke, Patrick; Klemt, Marcel; Damer, Naser; Kirchbuchner, Florian; Kuijper, Arjan

PocketNet: Extreme Lightweight Face Recognition Network using Neural Architecture Search and Multi-Step Knowledge Distillation Technical Report

2021.

Links | BibTeX

1255.

Chitty-Venkata, Krishna Teja; Somani, Arun K.

Array-Aware Neural Architecture Search Proceedings Article

In: 2021 IEEE 32nd International Conference on Application-specific Systems, Architectures and Processors (ASAP), pp. 125-132, 2021.

Links | BibTeX

1254.

Zhan, Zheng; Gong, Yifan; Zhao, Pu; Yuan, Geng; Niu, Wei; Wu, Yushu; Zhang, Tianyun; Jayaweera, Malith; Kaeli, David R.; Ren, Bin; Lin, Xue; Wang, Yanzhi

Achieving on-Mobile Real-Time Super-Resolution with Neural Architecture and Pruning Search Technical Report

2021.

Links | BibTeX

1253.

Wei, Lanning; Zhao, Huan; Yao, Quanming; He, Zhiqiang

Pooling Architecture Search for Graph Classification Technical Report

2021.

Links | BibTeX

1252.

Heuillet, Alexandre; Tabia, Hedi; Arioui, Hichem; Youcef-Toumi, Kamal

D-DARTS: Distributed Differentiable Architecture Search Technical Report

2021.

Links | BibTeX

1251.

Hsieh, Jun-Wei; Chang, Ming-Ching; Chen, Ping-Yang; Santra, Santanu; Chou, Cheng-Han; Huang, Chih-Sheng

MS-DARTS: Mean-Shift Based Differentiable Architecture Search Technical Report

2021.

Links | BibTeX

1250.

Chitty-Venkata, Krishna Teja; Somani, Arun K.; Kothandaraman, Sreenivas

Searching Architecture and Precision for U-net based Image Restoration Tasks Proceedings Article

In: 2021 IEEE International Conference on Image Processing (ICIP), pp. 1989-1993, 2021.

Links | BibTeX

1249.

Traore, Kalifou Rene; Camero, Andrés; Zhu, Xiao Xiang

Lessons from the Clustering Analysis of a Search Space: A Centroid-based Approach to Initializing NAS Technical Report

2021.

Links | BibTeX

1248.

Shi, Pengfei; Zhao, Zhongxin; Fan, Xinnan; Yan, Xijun; Yan, Wei; Xin, Yuanxue

Remote Sensing Image Object Detection Based on Angle Classification Journal Article

In: IEEE Access, vol. 9, pp. 118696-118707, 2021.

Links | BibTeX

1247.

Lin, Jun-Liang; Sung, Yi-Lin; Hong, Cheng-Yao; Lee, Han-Hung; Liu, Tyng-Luh

The Maximum a Posterior Estimation of Darts Proceedings Article

In: 2021 IEEE International Conference on Image Processing (ICIP), pp. 419-423, 2021.

Links | BibTeX

1246.

Huynh, Lam; Nguyen, Phong; Matas, Jiri; Rahtu, Esa; Heikkilä, Janne

Lightweight Monocular Depth with a Novel Neural Architecture Search Method Technical Report

2021.

Links | BibTeX

1245.

Chen, Wuyang; Gong, Xinyu; Wei, Yunchao; Shi, Humphrey; Yan, Zhicheng; Yang, Yi; Wang, Zhangyang

Understanding and Accelerating Neural Architecture Search with Training-Free and Theory-Grounded Metrics Technical Report

2021.

Links | BibTeX

1244.

Zhang, Yang; Zong, Ruohan; Kou, Ziyi; Shang, Lanyu; Wang, Dong

A Crowd-driven Dynamic Neural Architecture Searching Approach to Quality-aware Streaming Disaster Damage Assessment Technical Report

2021.

Links | BibTeX

1243.

Wang, Huiqun; Yang, Ruijie; Huang, Di; Wang, Yunhong

iDARTS: Improving DARTS by Node Normalization and Decorrelation Discretization Technical Report

2021.

Links | BibTeX

1242.

Lyu, Bo; Wen, Shiping; Shi, Kaibo; Huang, Tingwen

Multiobjective Reinforcement Learning-Based Neural Architecture Search for Efficient Portrait Parsing Journal Article

In: IEEE Transactions on Cybernetics, pp. 1-12, 2021.

Links | BibTeX

1241.

Xu, Jin; Tan, Xu; Song, Kaitao; Luo, Renqian; Leng, Yichong; Qin, Tao; Liu, Tie-Yan; Li, Jian

Analyzing and Mitigating Interference in Neural Architecture Search Technical Report

2021.

Links | BibTeX

1240.

Du, Mingyang; He, Xikai; Cai, Xiaohao; Bi, Daping

Balanced Neural Architecture Search and Its Application in Specific Emitter Identification Journal Article

In: IEEE Transactions on Signal Processing, vol. 69, pp. 5051-5065, 2021.

Links | BibTeX

1239.

Huynh, Lam; Nguyen, Tri; Nguyen, Thu; Pirttikangas, Susanna; Siirtola, Pekka

StressNAS: Affect State and Stress Detection Using Neural Architecture Search Proceedings Article

In: Doryab, Afsaneh; Lv, Qin; Beigl, Michael (Ed.): UbiComp/ISWC '21: 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and 2021 ACM International Symposium on Wearable Computers, Virtual Event, September 21-25, 2021, pp. 121–125, ACM, 2021.

Links | BibTeX

1238.

Liu, Zuhao; Wang, Huan; Gao, Yibo; Shi, Shunchen

Automatic Attention Learning Using Neural Architecture Search for Detection of Cardiac Abnormality in 12-Lead ECG Journal Article

In: IEEE Transactions on Instrumentation and Measurement, vol. 70, pp. 1-12, 2021.

Links | BibTeX

1237.

Liu, Zhijian; Tang, Haotian; Zhao, Shengyu; Shao, Kevin; Han, Song

PVNAS: 3D Neural Architecture Search with Point-Voxel Convolution Journal Article

In: IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1-1, 2021.

Links | BibTeX

1236.

Li, Yuening; Chen, Zhengzhang; Zha, Daochen; Zhou, Kaixiong; Jin, Haifeng; Chen, Haifeng; Hu, Xia

Automated Anomaly Detection via Curiosity-Guided Search and Self-Imitation Learning Journal Article

In: IEEE Transactions on Neural Networks and Learning Systems, pp. 1-13, 2021.

Links | BibTeX

1235.

Nie, Xing; Liu, Yongcheng; Chen, Shaohong; Chang, Jianlong; Huo, Chunlei; Meng, Gaofeng; Tian, Qi; Hu, Weiming; Pan, Chunhong

Differentiable Convolution Search for Point Cloud Processing Technical Report

2021.

Links | BibTeX

1234.

Xue, Song; Zhao, Bo; Chen, Hanlin; Wang, Runqi; Zhang, Baochang

UCB-ENAS based on Reinforcement Learning Proceedings Article

In: 2021 IEEE 16th Conference on Industrial Electronics and Applications (ICIEA), pp. 2008-2013, 2021.

Links | BibTeX

1233.

Yu, Kaicheng; Ranftl, Rene; Salzmann, Mathieu

An Analysis of Super-Net Heuristics in Weight-Sharing NAS Journal Article

In: IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1-1, 2021.

Links | BibTeX

1232.

Gong, Xinyu; Wang, Heng; Shou, Zheng; Feiszli, Matt; Wang, Zhangyang; Yan, Zhicheng

Searching for Two-Stream Models in Multivariate Space for Video Recognition Technical Report

2021.

Links | BibTeX

1231.

Deng, Jiajun; Gutierrez, Fabian Ritter; Hu, Shoukang; Geng, Mengzhe; Xie, Xurong; Ye, Zi; Liu, Shansong; Yu, Jianwei; Liu, Xunying; Meng, Helen

Bayesian Parametric and Architectural Domain Adaptation of LF-MMI Trained TDNNs for Elderly and Dysarthric Speech Recognition Proceedings Article

In: Proc. Interspeech 2021, pp. 4818–4822, 2021.

Links | BibTeX

1230.

Shu, Yao; Cai, Shaofeng; Dai, Zhongxiang; Ooi, Beng Chin; Low, Bryan Kian Hsiang

NASI: Label- and Data-agnostic Neural Architecture Search at Initialization Technical Report

2021.

Links | BibTeX

1229.

Umamageswari, A.; Bharathiraja, N.; Irene, D. Shiny

A Novel Fuzzy C-Means based Chameleon Swarm Algorithm for Segmentation and Progressive Neural Architecture Search for Plant Disease Classification Journal Article

In: ICT Express, 2021, ISSN: 2405-9595.

Abstract | Links | BibTeX

1228.

Liu, Jinyuan; Wu, Yuhui; Huang, Zhanbo; Liu, Risheng; Fan, Xin

SMoA: Searching a Modality-Oriented Architecture for Infrared and Visible Image Fusion Journal Article

In: IEEE Signal Processing Letters, vol. 28, pp. 1818-1822, 2021.

Links | BibTeX

1227.

Liao, Yi-Lun; Karaman, Sertac; Sze, Vivienne

Searching for Efficient Multi-Stage Vision Transformers Technical Report

2021.

Links | BibTeX

1226.

Jlassi, Sinda; Jdey, Imen; Ltifi, Hela

Bayesian Hyperparameter Optimization of Deep Neural Network Algorithms Based on Ant Colony Optimization Proceedings Article

In: Lladós, Josep; Lopresti, Daniel; Uchida, Seiichi (Ed.): Document Analysis and Recognition -- ICDAR 2021, pp. 585–594, Springer International Publishing, Cham, 2021, ISBN: 978-3-030-86334-0.

Abstract | BibTeX

1225.

Cai, Shaofei; Li, Liang; Han, Xinzhe; Zha, Zhengjun; Huang, Qingming

Edge-featured Graph Neural Architecture Search Technical Report

2021.

Links | BibTeX

1224.

Shu, Yao; Chen, Yizhou; Dai, Zhongxiang; Low, Bryan Kian Hsiang

Going Beyond Neural Architecture Search with Sampling-based Neural Ensemble Search Technical Report

2021.

Links | BibTeX

1223.

Bai, Haoyue; Zhou, Fengwei; Hong, Lanqing; Ye, Nanyang; Chan, S. -H. Gary; Li, Zhenguo

NAS-OoD: Neural Architecture Search for Out-of-Distribution Generalization Technical Report

2021.

Links | BibTeX

1222.

Letunovskiy, Alexey; Korviakov, Vladimir; Polovnikov, Vladimir; Kargapoltseva, Anastasiia; Mazurenko, Ivan; Xiong, Yepan

ISyNet: Convolutional Neural Networks design for AI accelerator Technical Report

2021.

Links | BibTeX

1221.

Shi, Xiaohu; Guo, Hongyan; Wu, Chunguo; Liang, Yanchun; Chang, Zhiyong

A block-encoding method for evolving neural network architecture Journal Article

In: Int. J. Bio Inspired Comput., vol. 18, no. 1, pp. 27–37, 2021.

Links | BibTeX

1220.

Rawat, Ambrish; Sinn, Mathieu; Buesser, Beat

Automated Robustness with Adversarial Training as a Post-Processing Step Technical Report

2021.

Links | BibTeX

1219.

Yu, Ke; Li, Zexian; Peng, Yue; Loy, Chen Change; Gu, Jinwei

ReconfigISP: Reconfigurable Camera Image Processing Pipeline Technical Report

2021.

Links | BibTeX

1218.

Yang, Ji; Sang, Lu; Cremers, Daniel

Dive into Layers: Neural Network Capacity Bounding using Algebraic Geometry Technical Report

2021.

Links | BibTeX

1217.

Yang, Fan; Qiao, Yanan; Huang, Cheng; Wang, Shan; Wang, Xiao

An Automatic Credit Scoring Strategy (ACSS) using memetic evolutionary algorithm and neural architecture search Journal Article

In: Applied Soft Computing, vol. 113, pp. 107871, 2021, ISSN: 1568-4946.

Abstract | Links | BibTeX

@article{YANG2021107871,

title = {An Automatic Credit Scoring Strategy (ACSS) using memetic evolutionary algorithm and neural architecture search},

author = {Fan Yang and Yanan Qiao and Cheng Huang and Shan Wang and Xiao Wang},

url = {https://www.sciencedirect.com/science/article/pii/S1568494621007936},

doi = {https://doi.org/10.1016/j.asoc.2021.107871},

issn = {1568-4946},

year  = {2021},

date = {2021-01-01},

urldate = {2021-01-01},

journal = {Applied Soft Computing},

volume = {113},

pages = {107871},

abstract = {Credit scoring is playing an increasingly critical role with the rising number of lending operations for micro and small enterprises as well as individuals. A large number of research is primarily based on the combination and construction methods of credit scoring models by the experts. However, different credit data have distinct requirements for the models, and how to automatically search and construct credit scoring models according to credit data has become essential, that is the main concern of this paper. In response to the current challenges for credit scoring research, we proposed an Automatic Credit Scoring Strategy (ACSS), designed a credit assessment platform which includes data import, classification model automatic search, feature selection, hyperparameter optimization, data mining, classification output and other modules. Aiming at the problem of substantial imbalance in credit data, we propose an improved SMOTE algorithm that is capable of generating supplementary data for the lack of minority in credit data, thereby making the credit data distribution well balanced. As for classification model selection, features engineering, parameter optimization and other parts, we further incorporate automatic search ways to reduce manual interaction. We utilize public and self-owned credit data sets to conduct experiments and compare them with the latest credit assessment methods. Extensive experiments have demonstrated that our ACSS method achieves relatively noticeable performance improvements using the German credit dataset, the Taiwan credit dataset and the personal credit dataset for credit scoring. The best results achieved by our proposed method are 0.98, 0.99, 0.895 and 0.901 for MAE, RMSE, accuracy and precision respectively. In addition, the experimental results also show that our proposed improved SMOTE algorithm contributes to the credit scoring performance enhancement. The experimental findings suggest that our proposed ACSS balances automation and accuracy, which can be implemented to the consumption industry to enhance the reliability of credit assessments.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

Close

Credit scoring is playing an increasingly critical role with the rising number of lending operations for micro and small enterprises as well as individuals. A large number of research is primarily based on the combination and construction methods of credit scoring models by the experts. However, different credit data have distinct requirements for the models, and how to automatically search and construct credit scoring models according to credit data has become essential, that is the main concern of this paper. In response to the current challenges for credit scoring research, we proposed an Automatic Credit Scoring Strategy (ACSS), designed a credit assessment platform which includes data import, classification model automatic search, feature selection, hyperparameter optimization, data mining, classification output and other modules. Aiming at the problem of substantial imbalance in credit data, we propose an improved SMOTE algorithm that is capable of generating supplementary data for the lack of minority in credit data, thereby making the credit data distribution well balanced. As for classification model selection, features engineering, parameter optimization and other parts, we further incorporate automatic search ways to reduce manual interaction. We utilize public and self-owned credit data sets to conduct experiments and compare them with the latest credit assessment methods. Extensive experiments have demonstrated that our ACSS method achieves relatively noticeable performance improvements using the German credit dataset, the Taiwan credit dataset and the personal credit dataset for credit scoring. The best results achieved by our proposed method are 0.98, 0.99, 0.895 and 0.901 for MAE, RMSE, accuracy and precision respectively. In addition, the experimental results also show that our proposed improved SMOTE algorithm contributes to the credit scoring performance enhancement. The experimental findings suggest that our proposed ACSS balances automation and accuracy, which can be implemented to the consumption industry to enhance the reliability of credit assessments.

Close

1216.

Zhang, Mingyang; Yu, Xinyi; Rong, Jingtao; Ou, Linlin; Gao, Feng

RepNAS: Searching for Efficient Re-parameterizing Blocks Technical Report

2021.

Links | BibTeX