AutoML.org

Freiburg-Hannover-Tübingen

Literature on Neural Architecture Search

Maintained by Difan Deng and Marius Lindauer.

The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.

Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.

Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.

3251 entries « ‹ 41 of 66 › »

2021

1251.

Tang, Sheyang; Hosseini, Mahdi S.; Chen, Lina; Varma, Sonal; Rowsell, Corwyn; Damaskinos, Savvas; Plataniotis, Konstantinos N.; Wang, Zhou

Probeable DARTS with Application to Computational Pathology Technical Report

2021.

1250.

Zhao, Pengyu; Xiao, Kecheng; Zhang, Yuanxing; Bian, Kaigui; Yan, Wei

AMEIR: Automatic Behavior Modeling, Interaction Exploration and MLP Investigation in the Recommender System Proceedings Article

In: Zhou, Zhi-Hua (Ed.): Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI 2021, Virtual Event / Montreal, Canada, 19-27 August 2021, pp. 2104–2110, ijcai.org, 2021.

1249.

Lyu, Bo; Yang, Yin; Wen, Shiping; Huang, Tingwen; Li, Ke

Neural Architecture Search for Portrait Parsing Journal Article

In: IEEE Transactions on Neural Networks and Learning Systems, pp. 1-10, 2021.

1248.

Liu, Chia-Hsiang; Han, Yu-Shin; Sung, Yuan-Yao; Lee, Yi; Chiang, Hung-Yueh; Wu, Kai-Chiang

FOX-NAS: Fast, On-device and Explainable Neural Architecture Search Technical Report

2021.

1247.

Kim, Youngkee; Yun, Won Joon; Lee, Youn Kyu; Jung, Soyi; Kim, Joongheon

Trends in Neural Architecture Search: Towards the Acceleration of Search Technical Report

2021.

1246.

Ganepola, Vayangi Vishmi Vishara; Wirasingha, Torin

Generative Adversarial Networks Using Neural Architecture Search for Semantic Image Segmentation Proceedings Article

In: 2021 IEEE 4th International Conference on Big Data and Artificial Intelligence (BDAI), pp. 236-241, 2021.

1245.

Wang, Ruochen; Chen, Xiangning; Cheng, Minhao; Tang, Xiaocheng; Hsieh, Cho-Jui

RANK-NOSH: Efficient Predictor-Based Architecture Search via Non-Uniform Successive Halving Proceedings Article

In: ICCV 2021, 2021.

1244.

Hou, Pengfei; Jin, Ying; Chen, Yukang

Single-DARTS: Towards Stable Architecture Search Technical Report

2021.

1243.

Fan, Yi; Zhou, Quoqiang; Zhang, Weifeng; Bao, Shudi; Shen, Jun

Determining learning direction via multi-controller model for stably searching generative adversarial networks Journal Article

In: Neurocomputing, vol. 464, pp. 37-47, 2021, ISSN: 0925-2312.

Abstract | Links | BibTeX

1242.

Peng, Jiefeng; Zhang, Jiqi; Li, Changlin; Wang, Guangrun; Liang, Xiaodan; Lin, Liang

Pi-NAS: Improving Neural Architecture Search by Reducing Supernet Training Consistency Shift Technical Report

2021.

1241.

Calisto, Maria Baldeon; Lai-Yuen, Susana K.

EMONAS-Net: Efficient multiobjective neural architecture search using surrogate-assisted evolutionary algorithm for 3D medical image segmentation Journal Article

In: Artificial Intelligence in Medicine, vol. 119, pp. 102154, 2021, ISSN: 0933-3657.

Abstract | Links | BibTeX

@article{BALDEONCALISTO2021102154,

title = {EMONAS-Net: Efficient multiobjective neural architecture search using surrogate-assisted evolutionary algorithm for 3D medical image segmentation},

author = {Maria Baldeon Calisto and Susana K. Lai-Yuen},

url = {https://www.sciencedirect.com/science/article/pii/S0933365721001470},

doi = {https://doi.org/10.1016/j.artmed.2021.102154},

issn = {0933-3657},

year  = {2021},

date = {2021-01-01},

urldate = {2021-01-01},

journal = {Artificial Intelligence in Medicine},

volume = {119},

pages = {102154},

abstract = {Deep learning plays a critical role in medical image segmentation. Nevertheless, manually designing a neural network for a specific segmentation problem is a very difficult and time-consuming task due to the massive hyperparameter search space, long training time and large volumetric data. Therefore, most designed networks are highly complex, task specific and over-parametrized. Recently, multiobjective neural architecture search (NAS) methods have been proposed to automate the design of accurate and efficient segmentation architectures. However, they only search for either the micro- or macro-structure of the architecture, do not use the information produced during the optimization process to increase the efficiency of the search, or do not consider the volumetric nature of medical images. In this work, we present EMONAS-Net, an Efficient MultiObjective NAS framework for 3D medical image segmentation that optimizes both the segmentation accuracy and size of the network. EMONAS-Net has two key components, a novel search space that considers the configuration of the micro- and macro-structure of the architecture and a Surrogate-assisted Multiobjective Evolutionary based Algorithm (SaMEA algorithm) that efficiently searches for the best hyperparameter values. The SaMEA algorithm uses the information collected during the initial generations of the evolutionary process to identify the most promising subproblems and select the best performing hyperparameter values during mutation to improve the convergence speed. Furthermore, a Random Forest surrogate model is incorporated to accelerate the fitness evaluation of the candidate architectures. EMONAS-Net is tested on the tasks of prostate segmentation from the MICCAI PROMISE12 challenge, hippocampus segmentation from the Medical Segmentation Decathlon challenge, and cardiac segmentation from the MICCAI ACDC challenge. In all the benchmarks, the proposed framework finds architectures that perform better or comparable with competing state-of-the-art NAS methods while being considerably smaller and reducing the architecture search time by more than 50%.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

Deep learning plays a critical role in medical image segmentation. Nevertheless, manually designing a neural network for a specific segmentation problem is a very difficult and time-consuming task due to the massive hyperparameter search space, long training time and large volumetric data. Therefore, most designed networks are highly complex, task specific and over-parametrized. Recently, multiobjective neural architecture search (NAS) methods have been proposed to automate the design of accurate and efficient segmentation architectures. However, they only search for either the micro- or macro-structure of the architecture, do not use the information produced during the optimization process to increase the efficiency of the search, or do not consider the volumetric nature of medical images. In this work, we present EMONAS-Net, an Efficient MultiObjective NAS framework for 3D medical image segmentation that optimizes both the segmentation accuracy and size of the network. EMONAS-Net has two key components, a novel search space that considers the configuration of the micro- and macro-structure of the architecture and a Surrogate-assisted Multiobjective Evolutionary based Algorithm (SaMEA algorithm) that efficiently searches for the best hyperparameter values. The SaMEA algorithm uses the information collected during the initial generations of the evolutionary process to identify the most promising subproblems and select the best performing hyperparameter values during mutation to improve the convergence speed. Furthermore, a Random Forest surrogate model is incorporated to accelerate the fitness evaluation of the candidate architectures. EMONAS-Net is tested on the tasks of prostate segmentation from the MICCAI PROMISE12 challenge, hippocampus segmentation from the Medical Segmentation Decathlon challenge, and cardiac segmentation from the MICCAI ACDC challenge. In all the benchmarks, the proposed framework finds architectures that perform better or comparable with competing state-of-the-art NAS methods while being considerably smaller and reducing the architecture search time by more than 50%.

1240.

Boutros, Fadi; Siebke, Patrick; Klemt, Marcel; Damer, Naser; Kirchbuchner, Florian; Kuijper, Arjan

PocketNet: Extreme Lightweight Face Recognition Network using Neural Architecture Search and Multi-Step Knowledge Distillation Technical Report

2021.

1239.

Chitty-Venkata, Krishna Teja; Somani, Arun K.

Array-Aware Neural Architecture Search Proceedings Article

In: 2021 IEEE 32nd International Conference on Application-specific Systems, Architectures and Processors (ASAP), pp. 125-132, 2021.

1238.

Zhan, Zheng; Gong, Yifan; Zhao, Pu; Yuan, Geng; Niu, Wei; Wu, Yushu; Zhang, Tianyun; Jayaweera, Malith; Kaeli, David R.; Ren, Bin; Lin, Xue; Wang, Yanzhi

Achieving on-Mobile Real-Time Super-Resolution with Neural Architecture and Pruning Search Technical Report

2021.

1237.

Wei, Lanning; Zhao, Huan; Yao, Quanming; He, Zhiqiang

Pooling Architecture Search for Graph Classification Technical Report

2021.

1236.

Heuillet, Alexandre; Tabia, Hedi; Arioui, Hichem; Youcef-Toumi, Kamal

D-DARTS: Distributed Differentiable Architecture Search Technical Report

2021.

1235.

Hsieh, Jun-Wei; Chang, Ming-Ching; Chen, Ping-Yang; Santra, Santanu; Chou, Cheng-Han; Huang, Chih-Sheng

MS-DARTS: Mean-Shift Based Differentiable Architecture Search Technical Report

2021.

1234.

Chitty-Venkata, Krishna Teja; Somani, Arun K.; Kothandaraman, Sreenivas

Searching Architecture and Precision for U-net based Image Restoration Tasks Proceedings Article

In: 2021 IEEE International Conference on Image Processing (ICIP), pp. 1989-1993, 2021.

1233.

Traore, Kalifou Rene; Camero, Andrés; Zhu, Xiao Xiang

Lessons from the Clustering Analysis of a Search Space: A Centroid-based Approach to Initializing NAS Technical Report

2021.

1232.

Shi, Pengfei; Zhao, Zhongxin; Fan, Xinnan; Yan, Xijun; Yan, Wei; Xin, Yuanxue

Remote Sensing Image Object Detection Based on Angle Classification Journal Article

In: IEEE Access, vol. 9, pp. 118696-118707, 2021.

1231.

Lin, Jun-Liang; Sung, Yi-Lin; Hong, Cheng-Yao; Lee, Han-Hung; Liu, Tyng-Luh

The Maximum a Posterior Estimation of Darts Proceedings Article

In: 2021 IEEE International Conference on Image Processing (ICIP), pp. 419-423, 2021.

1230.

Huynh, Lam; Nguyen, Phong; Matas, Jiri; Rahtu, Esa; Heikkilä, Janne

Lightweight Monocular Depth with a Novel Neural Architecture Search Method Technical Report

2021.

1229.

Chen, Wuyang; Gong, Xinyu; Wei, Yunchao; Shi, Humphrey; Yan, Zhicheng; Yang, Yi; Wang, Zhangyang

Understanding and Accelerating Neural Architecture Search with Training-Free and Theory-Grounded Metrics Technical Report

2021.

1228.

Zhang, Yang; Zong, Ruohan; Kou, Ziyi; Shang, Lanyu; Wang, Dong

A Crowd-driven Dynamic Neural Architecture Searching Approach to Quality-aware Streaming Disaster Damage Assessment Technical Report

2021.

1227.

Wang, Huiqun; Yang, Ruijie; Huang, Di; Wang, Yunhong

iDARTS: Improving DARTS by Node Normalization and Decorrelation Discretization Technical Report

2021.

1226.

Lyu, Bo; Wen, Shiping; Shi, Kaibo; Huang, Tingwen

Multiobjective Reinforcement Learning-Based Neural Architecture Search for Efficient Portrait Parsing Journal Article

In: IEEE Transactions on Cybernetics, pp. 1-12, 2021.

1225.

Xu, Jin; Tan, Xu; Song, Kaitao; Luo, Renqian; Leng, Yichong; Qin, Tao; Liu, Tie-Yan; Li, Jian

Analyzing and Mitigating Interference in Neural Architecture Search Technical Report

2021.

1224.

Du, Mingyang; He, Xikai; Cai, Xiaohao; Bi, Daping

Balanced Neural Architecture Search and Its Application in Specific Emitter Identification Journal Article

In: IEEE Transactions on Signal Processing, vol. 69, pp. 5051-5065, 2021.

1223.

Huynh, Lam; Nguyen, Tri; Nguyen, Thu; Pirttikangas, Susanna; Siirtola, Pekka

StressNAS: Affect State and Stress Detection Using Neural Architecture Search Proceedings Article

In: Doryab, Afsaneh; Lv, Qin; Beigl, Michael (Ed.): UbiComp/ISWC '21: 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and 2021 ACM International Symposium on Wearable Computers, Virtual Event, September 21-25, 2021, pp. 121–125, ACM, 2021.

1222.

Liu, Zuhao; Wang, Huan; Gao, Yibo; Shi, Shunchen

Automatic Attention Learning Using Neural Architecture Search for Detection of Cardiac Abnormality in 12-Lead ECG Journal Article

In: IEEE Transactions on Instrumentation and Measurement, vol. 70, pp. 1-12, 2021.

1221.

Liu, Zhijian; Tang, Haotian; Zhao, Shengyu; Shao, Kevin; Han, Song

PVNAS: 3D Neural Architecture Search with Point-Voxel Convolution Journal Article

In: IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1-1, 2021.

1220.

Li, Yuening; Chen, Zhengzhang; Zha, Daochen; Zhou, Kaixiong; Jin, Haifeng; Chen, Haifeng; Hu, Xia

Automated Anomaly Detection via Curiosity-Guided Search and Self-Imitation Learning Journal Article

In: IEEE Transactions on Neural Networks and Learning Systems, pp. 1-13, 2021.

1219.

Nie, Xing; Liu, Yongcheng; Chen, Shaohong; Chang, Jianlong; Huo, Chunlei; Meng, Gaofeng; Tian, Qi; Hu, Weiming; Pan, Chunhong

Differentiable Convolution Search for Point Cloud Processing Technical Report

2021.

1218.

Xue, Song; Zhao, Bo; Chen, Hanlin; Wang, Runqi; Zhang, Baochang

UCB-ENAS based on Reinforcement Learning Proceedings Article

In: 2021 IEEE 16th Conference on Industrial Electronics and Applications (ICIEA), pp. 2008-2013, 2021.

1217.

Yu, Kaicheng; Ranftl, Rene; Salzmann, Mathieu

An Analysis of Super-Net Heuristics in Weight-Sharing NAS Journal Article

In: IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1-1, 2021.

1216.

Gong, Xinyu; Wang, Heng; Shou, Zheng; Feiszli, Matt; Wang, Zhangyang; Yan, Zhicheng

Searching for Two-Stream Models in Multivariate Space for Video Recognition Technical Report

2021.

1215.

Deng, Jiajun; Gutierrez, Fabian Ritter; Hu, Shoukang; Geng, Mengzhe; Xie, Xurong; Ye, Zi; Liu, Shansong; Yu, Jianwei; Liu, Xunying; Meng, Helen

Bayesian Parametric and Architectural Domain Adaptation of LF-MMI Trained TDNNs for Elderly and Dysarthric Speech Recognition Proceedings Article

In: Proc. Interspeech 2021, pp. 4818–4822, 2021.

1214.

Shu, Yao; Cai, Shaofeng; Dai, Zhongxiang; Ooi, Beng Chin; Low, Bryan Kian Hsiang

NASI: Label- and Data-agnostic Neural Architecture Search at Initialization Technical Report

2021.

1213.

Umamageswari, A.; Bharathiraja, N.; Irene, D. Shiny

A Novel Fuzzy C-Means based Chameleon Swarm Algorithm for Segmentation and Progressive Neural Architecture Search for Plant Disease Classification Journal Article

In: ICT Express, 2021, ISSN: 2405-9595.

Abstract | Links | BibTeX

1212.

Liu, Jinyuan; Wu, Yuhui; Huang, Zhanbo; Liu, Risheng; Fan, Xin

SMoA: Searching a Modality-Oriented Architecture for Infrared and Visible Image Fusion Journal Article

In: IEEE Signal Processing Letters, vol. 28, pp. 1818-1822, 2021.

1211.

Liao, Yi-Lun; Karaman, Sertac; Sze, Vivienne

Searching for Efficient Multi-Stage Vision Transformers Technical Report

2021.

1210.

Jlassi, Sinda; Jdey, Imen; Ltifi, Hela

Bayesian Hyperparameter Optimization of Deep Neural Network Algorithms Based on Ant Colony Optimization Proceedings Article

In: Lladós, Josep; Lopresti, Daniel; Uchida, Seiichi (Ed.): Document Analysis and Recognition -- ICDAR 2021, pp. 585–594, Springer International Publishing, Cham, 2021, ISBN: 978-3-030-86334-0.

Abstract | BibTeX

1209.

Cai, Shaofei; Li, Liang; Han, Xinzhe; Zha, Zhengjun; Huang, Qingming

Edge-featured Graph Neural Architecture Search Technical Report

2021.

1208.

Shu, Yao; Chen, Yizhou; Dai, Zhongxiang; Low, Bryan Kian Hsiang

Going Beyond Neural Architecture Search with Sampling-based Neural Ensemble Search Technical Report

2021.

1207.

Bai, Haoyue; Zhou, Fengwei; Hong, Lanqing; Ye, Nanyang; Chan, S. -H. Gary; Li, Zhenguo

NAS-OoD: Neural Architecture Search for Out-of-Distribution Generalization Technical Report

2021.

1206.

Letunovskiy, Alexey; Korviakov, Vladimir; Polovnikov, Vladimir; Kargapoltseva, Anastasiia; Mazurenko, Ivan; Xiong, Yepan

ISyNet: Convolutional Neural Networks design for AI accelerator Technical Report

2021.

1205.

Shi, Xiaohu; Guo, Hongyan; Wu, Chunguo; Liang, Yanchun; Chang, Zhiyong

A block-encoding method for evolving neural network architecture Journal Article

In: Int. J. Bio Inspired Comput., vol. 18, no. 1, pp. 27–37, 2021.

1204.

Rawat, Ambrish; Sinn, Mathieu; Buesser, Beat

Automated Robustness with Adversarial Training as a Post-Processing Step Technical Report

2021.

1203.

Yu, Ke; Li, Zexian; Peng, Yue; Loy, Chen Change; Gu, Jinwei

ReconfigISP: Reconfigurable Camera Image Processing Pipeline Technical Report

2021.

1202.

Yang, Ji; Sang, Lu; Cremers, Daniel

Dive into Layers: Neural Network Capacity Bounding using Algebraic Geometry Technical Report

2021.

3251 entries « ‹ 41 of 66 › »