AutoML.org

Freiburg-Hannover-Tübingen

Literature on Neural Architecture Search

Maintained by Difan Deng and Marius Lindauer.

The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.

Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.

Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.

3980 entries « ‹ 50 of 80 › »

2021

1530.

Nayman, Niv; Aflalo, Yonathan; Noy, Asaf; Zelnik-Manor, Lihi

HardCoRe-NAS: Hard Constrained diffeRentiable Neural Architecture Search Technical Report

2021.

1529.

Nguyen, Nam; Chang, Morris J

Contrastive Self-supervised Neural Architecture Search Miscellaneous

2021.

1528.

Guo, Yong; Zheng, Yin; Tan, Mingkui; Chen, Qi; Li, Zhipeng; Chen, Jian; Zhao, Peilin; Huang, Junzhou

Towards Accurate and Compact Architectures via Neural Architecture Transformer Technical Report

2021.

1527.

Wu, Junru; Dai, Xiyang; Chen, Dongdong; Chen, Yinpeng; Liu, Mengchen; Yu, Ye; Wang, Zhangyang; Liu, Zicheng; Chen, Mei; Yuan, Lu

Weak NAS Predictors Are All You Need Technical Report

2021.

1526.

Zhang, Zijun; Cofer, Evan M.; Troyanskaya, Olga G.

AMBIENT: Accelerated Convolutional Neural Network Architecture Search for Regulatory Genomics Proceedings Article

In: Machine Learning in Computational Biology (MLCB 2020), 2021.

1525.

Chen, J; Jiang, Y; Huang, Z; Guo, X; Wu, B; Sun, L; Wu, T

Fine-Grained Detection of Driver Distraction Based on Neural Architecture Search Journal Article

In: IEEE Transactions on Intelligent Transportation Systems, pp. 1-19, 2021.

1524.

Zhang, T; Lei, C; Zhang, Z; Meng, X -B; Chen, C L P

AS-NAS: Adaptive Scalable Neural Architecture Search with Reinforced Evolutionary Algorithm for Deep Learning Journal Article

In: IEEE Transactions on Evolutionary Computation, pp. 1-1, 2021.

1523.

Xie, Guoyang; Wang, Jinbao; Yu, Guo; Zheng, Feng; Jin, Yaochu

Tiny Adversarial Mulit-Objective Oneshot Neural Architecture Search Technical Report

2021.

1522.

Le, Cat P; Soltani, Mohammadreza; Ravier, Robert; Tarokh, Vahid

Neural Architecture Search From Task Similarity Measure Technical Report

2021.

1521.

Guo, Yong; Chen, Yaofo; Zheng, Yin; Chen, Qi; Zhao, Peilin; Chen, Jian; Huang, Junzhou; Tan, Mingkui

Pareto-Frontier-aware Neural Architecture Generation for Diverse Budgets Technical Report

2021.

1520.

Jeong, Wonyong; Lee, Hayeon; Park, Gun; Hyung, Eunyoung; Baek, Jinheon; Hwang, Sung Ju

Task-Adaptive Neural Network Retrieval with Meta-Contrastive Learning Technical Report

2021.

1519.

Zhang, Ziwei; Wang, Xin; Zhu, Wenwu

Automated Machine Learning on Graphs: A Survey Technical Report

2021.

1518.

Liu, Luyan; Wen, Zhiwei; Liu, Songwei; Zhou, Hong-Yu; Zhu, Hongwei; Xie, Weicheng; Shen, Linlin; Ma, Kai; Zheng, Yefeng

MixSearch: Searching for Domain Generalized Medical Image Segmentation Architectures Technical Report

2021.

1517.

Guo, Qingbei; Wu, Xiao-Jun; Kittler, Josef; Feng, Zhiquan

Differentiable Neural Architecture Learning for Efficient Neural Network Design Technical Report

2021.

1516.

Javeri, Indrajeet Y; Toutiaee, Mohammadhossein; Arpinar, Ismailcem B; Miller, Tom W; Miller, John A

Improving Neural Networks for Time Series Forecasting using Data Augmentation and AutoML Technical Report

2021.

1515.

Chen, Yaofo; Guo, Yong; Chen, Qi; Li, Minli; Wang, Yaowei; Zeng, Wei; Tan, Mingkui

Contrastive Neural Architecture Search with Neural Architecture Comparators Proceedings Article

In: CVPR2021, 2021.

1514.

Zhang, Kaiyu; Chen, Jinglong; He, Shuilong; Xu, Enyong; Li, Fudong; Zhou, Zitong

Differentiable neural architecture search augmented with pruning and multi-objective optimization for time-efficient intelligent fault diagnosis of machinery Journal Article

In: Mechanical Systems and Signal Processing, vol. 158, pp. 107773, 2021, ISSN: 0888-3270.

Abstract | Links | BibTeX

@article{ZHANG2021107773,

title = {Differentiable neural architecture search augmented with pruning and multi-objective optimization for time-efficient intelligent fault diagnosis of machinery},

author = {Kaiyu Zhang and Jinglong Chen and Shuilong He and Enyong Xu and Fudong Li and Zitong Zhou},

url = {https://www.sciencedirect.com/science/article/pii/S0888327021001680},

doi = {https://doi.org/10.1016/j.ymssp.2021.107773},

issn = {0888-3270},

year  = {2021},

date = {2021-01-01},

journal = {Mechanical Systems and Signal Processing},

volume = {158},

pages = {107773},

abstract = {Intelligent fault diagnosis, which is mainly based on neural network, has been widely used in machinery monitoring. Although such deep learning methods are effective, the new architectures are mainly handcrafted by series of experiments that require ample time and substantial efforts. To automate process of building neural networks and save designing time, a novel differentiable neural architecture search method is proposed. By gradually reducing candidate operations while retaining trained parameters during pruning, computation consumed by each stage of neural architecture search is decreased, which accelerates search process. To improve inferential efficiency of subnetworks, specially designed penalty terms are introduced into the objective function for searching optimal numbers of layers and nodes, which can reduce complexity of subnetworks and save calculation time of signal analysis. In addition, exclusive competition between candidate operations is broken by changing discretization and selection methods of operations, which provides a basis for channel fusion. Effectiveness of the proposed method is verified by two datasets. Experiments show that this method can generate subnetworks of lower complexity and less computational cost than other state-of-art neural architecture search techniques, while achieving competitive result.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

1513.

Liang, Tingting; Wang, Yongtao; Tang, Zhi; Hu, Guosheng; Ling, Haibin

OPANAS: One-Shot Path Aggregation Network Architecture Search for Object Detection Technical Report

2021.

1512.

Cao, Shengcao; Wang, Xiaofang; Kitani, Kris

Efficient Model Performance Estimation via Feature Histories Technical Report

2021.

1511.

Zhang, Shi-Xin; Hsieh, Chang-Yu; Zhang, Shengyu; Yao, Hong

Neural Predictor based Quantum Architecture Search Technical Report

2021.

1510.

Liu, Xiaobo; Zhang, Chaochao; Cai, Zhihua; Yang, Jianfeng; Zhou, Zhilang; Gong, Xin

Continuous Particle Swarm Optimization-Based Deep Learning Architecture Search for Hyperspectral Image Classification Journal Article

In: Remote Sensing, vol. 13, no. 6, 2021, ISSN: 2072-4292.

Abstract | Links | BibTeX

1509.

Ban, Hao; Xie, Pengtao

Interleaving Learning, with Application to Neural Architecture Search Technical Report

2021.

1508.

Choi, Kwanghee; Choe, Minyoung; Lee, Hyelee

Pretraining Neural Architecture Search Controllers with Locality-based Self-Supervised Learning Technical Report

2021.

1507.

Gracheva, Ekaterina

Trainless Model Performance Estimation for Neural Architecture Search Technical Report

2021.

1506.

Wu, Xuan; Zhang, Xiuyi; Jia, Linhan; Chen, Liang; Liang, Yanchun; Zhou, You; Wu, Chunguo

Neural Architecture Search based on Cartesian Genetic Programming Coding Method Technical Report

2021.

1505.

Luo, Xiangzhong; Liu, Di; Huai, Shuo; Liu, Weichen

HSCoNAS: Hardware-Software Co-Design of Efficient DNNs via Neural Architecture Search Technical Report

2021.

1504.

Wang, Ruochen; Cheng, Minhao; Chen, Xiangning; Tang, Xiaocheng; Hsieh, Cho-Jui

Rethinking Architecture Selection in Differentiable NAS Proceedings Article

In: International Conference on Learning Representations, 2021.

1503.

-, Sian; -, Wei

Searching by Generating: Flexible and Efficient One-Shot NAS with Architecture Generator Technical Report

2021.

1502.

Lakhmiri, Dounia; é, S

Use of static surrogates in hyperparameter optimization Technical Report

2021.

1501.

Inkawhich, Matthew; Inkawhich, Nathan; Davis, Eric; Li, Hai; Chen, Yiran

The Untapped Potential of Off-the-Shelf Convolutional Neural Networks Technical Report

2021.

1500.

Chen, Zhiqiang; Xu, Ting-Bing; Liao, Weijian; Li, Zhengcheng; Li, Jinpeng; Liu, Cheng-Lin; He, Huiguang

SNAP: Shaping neural architectures progressively via information density criterion Journal Article

In: Pattern Recognition, vol. 116, pp. 107923, 2021, ISSN: 0031-3203.

Abstract | Links | BibTeX

@article{CHEN2021107923,

title = {SNAP: Shaping neural architectures progressively via information density criterion},

author = {Zhiqiang Chen and Ting-Bing Xu and Weijian Liao and Zhengcheng Li and Jinpeng Li and Cheng-Lin Liu and Huiguang He},

url = {https://www.sciencedirect.com/science/article/pii/S0031320321001102},

doi = {https://doi.org/10.1016/j.patcog.2021.107923},

issn = {0031-3203},

year  = {2021},

date = {2021-01-01},

journal = {Pattern Recognition},

volume = {116},

pages = {107923},

abstract = {Excellent neural network architecture is built on the specific target task and device. As the target task or device is different, the neural architecture we need will be different, too. Rather than redesigning or searching a brand new one, adjusting the existing architecture automatically is an alternative yet efficient way. To this end, we propose a method to Shape the existing Neural Architectures Progressively (SNAP) to adapt the target task and device better. Inspired by the streamline of water drop shaped by air resistance, we define an information density criterion (play the role of resistance) to drive the network architecture reducing the size of the part with the lowest information density. Iteratively, a more adaptive architecture will be obtained progressively in a greedy way. Theoretically, we prove that the greedy strategy is reasonable and can shape a better architecture. Because of the small adjustment of architecture each time, new architecture can inherit the parameters in old architecture to avoid retraining it from scratch. So the proposed method is very efficient in no need of high computation cost. Experimental results show that proposed method can effectively improve the given network by adjusting its architecture. And it can generate different architectures for different tasks and devices to adapt them well. Compared with search-based auto-generated neural architectures, our approach can achieve comparable or even better performance in no need of tremendous computation resources.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

1499.

Li, Chaojian; Yu, Zhongzhi; Fu, Yonggan; Zhang, Yongan; Zhao, Yang; You, Haoran; Yu, Qixuan; Wang, Yue; Lin, Yingyan

HW-NAS-Bench: Hardware-Aware Neural Architecture Search Benchmark Proceedings Article

In: ICLR 2021, 2021.

1498.

Pyeon, Myeongjang; Moon, Jihwan; Hahn, Taeyoung; Kim, Gunhee

SEDONA: Search for Decoupled Neural Networks toward Greedy Block-wise Learning Proceedings Article

In: International Conference on Learning Representations, 2021.

1497.

Zhou, Allan; Knowles, Tom; Finn, Chelsea

Meta-learning Symmetries by Reparameterization Proceedings Article

In: International Conference on Learning Representations, 2021.

1496.

Mehrotra, Abhinav; Ramos, Alberto Gil C P; Bhattacharya, Sourav; Ł,; Vipperla, Ravichander; Chau, Thomas; Abdelfattah, Mohamed S; Ishtiaq, Samin; Lane, Nicholas Donald

NAS-Bench-ASR: Reproducible Neural Architecture Search for Speech Recognition Proceedings Article

In: International Conference on Learning Representations, 2021.

1495.

Zhou, Daquan; Jin, Xiaojie; Lian, Xiaochen; Yang, Linjie; Xue, Yujing; Hou, Qibin; Feng, Jiashi

AutoSpace: Neural Architecture Search with Less Human Interference Technical Report

2021.

1494.

Li, Changlin; Tang, Tao; Wang, Guangrun; Peng, Jiefeng; Wang, Bing; Liang, Xiaodan; Chang, Xiaojun

BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search Technical Report

2021.

1493.

Abed, H; Gyires-Tóth, B

Efficient Neural Architecture Search for Long Short-Term Memory Networks Proceedings Article

In: 2021 IEEE 19th World Symposium on Applied Machine Intelligence and Informatics (SAMI), pp. 000287-000292, 2021.

1492.

Ai, Dige; Zhang, Hong

GNAS: A Generalized Neural Network Architecture Search Framework Technical Report

2021.

1491.

Zhang, Haichao; Hao, Kuangrong; Gao, Lei; -, Xue; Wei, Bing

Enhanced Gradient for Differentiable Architecture Search Technical Report

2021.

1490.

Su, Xiu; Huang, Tao; Li, Yanxi; You, Shan; Wang, Fei; Qian, Chen; Zhang, Changshui; Xu, Chang

Prioritized Architecture Sampling with Monto-Carlo Tree Search Technical Report

2021.

1489.

Wang, Hongbo; Pan, Xingyu; Fan, Lingyan; Zhao, Shuofeng

Steganalysis of convolutional neural network based on neural architecture search Journal Article

In: Multimedia Systems, 2021.

1488.

Lu, Zexin; Xia, Wenjun; Huang, Yongqiang; Shan, Hongming; Chen, Hu; Zhou, Jiliu; Zhang, Yi

MANAS: Multi-Scale and Multi-Level Neural Architecture Search for Low-Dose CT Denoising Journal Article

In: CoRR, vol. abs/2103.12995, 2021.

1487.

"Yang Zhao" "Hai Min" "Yan-Xiang Chen Wei Jia" "Wei Xia",

2D and 3D Palmprint and Palm Vein Recognition Based on Neural Architecture Search Journal Article

In: International Journal of Automation and Computing, no. IJAC-2021-01-01, pp. 1, 2021, ISSN: 1476-8186.

1486.

Lin, Yunfeng; Li, Guilin; Zhang, Xing; Zhang, Weinan; Chen, BO; Tang, Ruiming; Li, Zhenguo; Feng, Jiashi; Yu, YOng

ModularNAS: Towards Modularized and Reusable Neural Architecture Search Proceedings Article

In: Proceedings of Machine Learning and Systems 3 pre-proceedings (MLSys 2021), 2021.

1485.

Wang, Rui; Wei, Zhihua; Ji, Shouling; Hong, Zhen

EfficientTDNN: Efficient Architecture Search for Speaker Recognition in the Wild Technical Report

2021.

1484.

Ottelander, Tom Den; Dushatskiy, Arkadiy; Virgolin, Marco; Bosman, Peter A N

Local Search is a Remarkably Strong Baseline for Neural Architecture Search Proceedings Article

In: Ishibuchi, Hisao; Zhang, Qingfu; Cheng, Ran; Li, Ke; Li, Hui; Wang, Handing; Zhou, Aimin (Ed.): Evolutionary Multi-Criterion Optimization - 11th International Conference, EMO 2021, Shenzhen, China, March 28-31, 2021, Proceedings, pp. 465–479, Springer, 2021.

1483.

Ding, Mingyu; Huo, Yuqi; Lu, Haoyu; Yang, Linjie; Wang, Zhe; Lu, Zhiwu; Wang, Jingdong; Luo, Ping

Learning Versatile Neural Architectures by Propagating Network Codes Technical Report

2021.

1482.

Lankford, Séamus

Open-Source Neural Architecture Search with Ensemble and Pre-trained Networks Journal Article

In: International Journal of Modeling and Optimization, vol. 11, no. 2, pp. 33-41, 2021.

1481.

Li, Yanxi; Wen, Zean; Yunhe Wang,; Xu, Chang

One-shot Graph Neural Architecture Search with Dynamic Search Space Proceedings Article

In: AAAI 2021, 2021.

3980 entries « ‹ 50 of 80 › »