AutoML.org

Freiburg-Hannover-Tübingen

Literature on Neural Architecture Search

Maintained by Difan Deng and Marius Lindauer.

The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.

Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.

Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.

4564 entries « ‹ 90 of 92 › »

2018

114.

Schrimpf, Martin; Merity, Stephen; Bradbury, James; Socher, Richard

A Flexible Approach to Automated RNN Architecture Generation Proceedings Article

In: 2018.

113.

Suganuma, Masanori; Shirakawa, Shinichi; Nagao, Tomoharu

A Genetic Programming Approach to Designing Convolutional Neural Network Architectures Proceedings Article

In: pp. 5369-5373, 2018.

112.

Zhong, Zhao; Yan, Junjie; Wu, Wei; Shao, Jing; Liu, Cheng-Lin

Practical Block-Wise Neural Network Architecture Generation Proceedings Article

In: pp. 2423-2432, 2018.

111.

Baker, Bowen; Gupta, Otkrist; Raskar, Ramesh; Naik, Nikhil

Accelerating Neural Architecture Search using Performance Prediction Proceedings Article

In: 2018.

110.

Liu, Hanxiao; Simonyan, Karen; Vinyals, Oriol; Fernando, Chrisantha; Kavukcuoglu, Koray

Hierarchical Representations for Efficient Architecture Search Proceedings Article

In: 2018.

109.

Zoph, Barret; Vasudevan, Vijay; Shlens, Jonathon; Le, Quoc V

Learning Transferable Architectures for Scalable Image Recognition Proceedings Article

In: pp. 8697-8710, 2018.

108.

Elsken, Thomas; Metzen, Jan Hendrik; Hutter, Frank

Simple and efficient architecture search for Convolutional Neural Networks Proceedings Article

In: 2018.

107.

Hazan, Elad; Klivans, Adam R; Yuan, Yang

Hyperparameter optimization - a spectral approach Proceedings Article

In: 2018.

106.

Brock, Andrew; Lim, Theodore; Ritchie, James M; Weston, Nick

SMASH - One-Shot Model Architecture Search through HyperNetworks Proceedings Article

In: 2018.

105.

Cai, Han; Chen, Tianyao; Zhang, Weinan; Yu, Yong; Wang, Jun

Efficient Architecture Search by Network Transformation Proceedings Article

In: pp. 2787-2794, 2018.

104.

Chen, Liang-Chieh; Collins, Maxwell D.; Zhu, Yukun; Papandreou, George; Zoph, Barret; Schroff, Florian; Adam, Hartwig; Shlens, Jonathon

Searching for Efficient Multi-Scale Architectures for Dense Image Prediction Proceedings Article

In: Bengio, Samy; Wallach, Hanna M.; Larochelle, Hugo; Grauman, Kristen; Cesa-Bianchi, Nicolò; Garnett, Roman (Ed.): Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pp. 8713–8724, 2018.

2017

103.

Chang, Hao

Skin cancer reorganization and classification with deep neural network Technical Report

2017.

102.

Suganuma, Masanori; Shirakawa, Shinichi; Nagao, Tomoharu

A Genetic Programming Approach to Designing Convolutional Neural Network Architectures Technical Report

2017.

101.

Chang, Hao

Skin cancer reorganization and classification with deep neural network Technical Report

2017.

100.

Kani, Nagoor J; Elsheikh, Ahmed H

DR-RNN: A deep residual recurrent neural network for model reduction Technical Report

2017.

99.

Sun, Yanan; Xue, Bing; Zhang, Mengjie

A Particle Swarm Optimization-based Flexible Convolutional Auto-Encoder for Image Classification Technical Report

2017.

98.

Negrinho, Renato; Gordon, Geoffrey J

DeepArchitect - Automatically Designing and Training Deep Architectures Technical Report

2017.

97.

Real, Esteban; Moore, Sherry; Selle, Andrew; Saxena, Saurabh; Suematsu, Yutaka Leon; Tan, Jie; Le, Quoc V; Kurakin, Alexey

Large-Scale Evolution of Image Classifiers Proceedings Article

In: pp. 2902-2911, 2017.

96.

Bello, Irwan; Zoph, Barret; Vasudevan, Vijay; Le, Quoc V

Neural Optimizer Search with Reinforcement Learning Proceedings Article

In: pp. 459-468, 2017.

95.

Wistuba, Martin

Finding Competitive Network Architectures Within a Day Using UCT Technical Report

2017.

94.

Cortes, Corinna; Gonzalvo, Xavier; Kuznetsov, Vitaly; Mohri, Mehryar; Yang, Scott

AdaNet - Adaptive Structural Learning of Artificial Neural Networks Proceedings Article

In: pp. 874-883, 2017.

93.

Baker, Bowen; Gupta, Otkrist; Naik, Nikhil; Raskar, Ramesh

Designing Neural Network Architectures using Reinforcement Learning Proceedings Article

In: 2017.

92.

Klein, Aaron; Falkner, Stefan; Springenberg, Jost Tobias; Hutter, Frank

Learning Curve Prediction with Bayesian Neural Networks Proceedings Article

In: 2017.

91.

Li, Lisha; Jamieson, Kevin G; DeSalvo, Giulia; Rostamizadeh, Afshin; Talwalkar, Ameet

Hyperband - A Novel Bandit-Based Approach to Hyperparameter Optimization Journal Article

In: vol. 18, pp. 185:1-185:52, 2017.

2016

90.

Vu, Ngoc Thang; Adel, Heike; Gupta, Pankaj; Schütze, Hinrich

Combining Recurrent and Convolutional Neural Networks for Relation Classification Technical Report

2016.

89.

Chen, Long; Zhang, Hanwang; Xiao, Jun; Nie, Liqiang; Shao, Jian; Liu, Wei; Chua, Tat-Seng

SCA-CNN: Spatial and Channel-wise Attention in Convolutional Networks for Image Captioning Technical Report

2016.

88.

Smithson, Sean C; Yang, Guang; Gross, Warren J; Meyer, Brett H

Neural networks designing neural networks - multi-objective hyper-parameter optimization Proceedings Article

In: pp. 104, 2016.

87.

Saxena, Shreyas; Verbeek, Jakob

Convolutional Neural Fabrics Proceedings Article

In: pp. 4053-4061, 2016.

86.

Loshchilov, Ilya; Hutter, Frank

CMA-ES for Hyperparameter Optimization of Deep Neural Networks Technical Report

2016.

85.

Chen, Tianqi; Goodfellow, Ian J; Shlens, Jonathon

Net2Net - Accelerating Learning via Knowledge Transfer Proceedings Article

In: 2016.

84.

Chen, Justin

Combinatorially Generated Piecewise Activation Functions Technical Report

2016.

2015

83.

Kulish, Vadym V; Malyi, Oleksandr I; Persson, Clas; Wu, Ping

Evaluation of Phosphorene as Anode Material for Na-ion Batteries from First Principles Technical Report

2015.

82.

Young, Steven R; Rose, Derek C; Karnowski, Thomas P; Lim, Seung-Hwan; Patton, Robert M

Optimizing deep learning hyper-parameters through an evolutionary algorithm Proceedings Article

In: pp. 4:1-4:5, 2015.

2013

81.

Lopez, Pablo; Fernandez, David; Marin-Perez, Rafael; Jara, Antonio J; Gomez-Skarmeta, Antonio F

Scalable Oriented-Service Architecture for Heterogeneous and Ubiquitous IoT Domains Technical Report

2013.

80.

Fister, Iztok; Jr., Iztok Fister; Yang, Xin-She; Brest, Janez

A comprehensive review of firefly algorithms Technical Report

2013.

2012

79.

Anas, M; Javaid, N; Mahmood, A; Raza, S M; Qasim, U; Khan, Z A

Minimizing Electricity Theft using Smart Meters in AMI Technical Report

2012.

78.

Snoek, Jasper; Larochelle, Hugo; Adams, Ryan P

Practical Bayesian Optimization of Machine Learning Algorithms Proceedings Article

In: pp. 2960-2968, 2012.

2009

77.

Jaffres-Runser, Katia; Gorce, Jean-Marie; Comaniciu, Cristina

A multiobjective Tabu framework for the optimization and evaluation of wireless systems Technical Report

2009.

76.

Stanley, Kenneth O; D'Ambrosio, David B; Gauci, Jason

A Hypercube-Based Encoding for Evolving Large-Scale Neural Networks Journal Article

In: vol. 15, no. 2, pp. 185-212, 2009.

2008

75.

Floreano, Dario; Dürr, Peter; Mattiussi, Claudio

Neuroevolution - from architectures to learning Journal Article

In: vol. 1, no. 1, pp. 47-62, 2008.

2005

74.

Czarnowski, Ireneusz; Jedrzejowicz, Piotr

An Agent-Based PLA for the Cascade Correlation Learning Architecture Proceedings Article

In: pp. 197-202, 2005.

2004

73.

Roy, N K; Potter, Walter D; Landau, D P

Designing Polymer Blends Using Neural Networks, Genetic Algorithms, and Markov Chains Journal Article

In: vol. 20, no. 3, pp. 215-229, 2004.

1994

72.

Angeline, Peter J; Saunders, Gregory M; Pollack, Jordan B

An evolutionary algorithm that constructs recurrent neural networks Journal Article

In: vol. 5, no. 1, pp. 54-65, 1994.

1990

71.

Kitano, Hiroaki

Designing Neural Networks Using Genetic Algorithms with Graph Generation System Journal Article

In: vol. 4, no. 4, 1990.

1988

70.

Tenorio, Manoel Fernando; Lee, Wei-Tsih

Self Organizing Neural Networks for the Identification Problem Proceedings Article

In: pp. 57-64, 1988.

0000

69.

Waris, Faisal; Reynolds, Robert G.; Lee, Joonho

Evolving Deep Neural Networks with Cultural Algorithms for Real-Time Industrial Applications Journal Article

In: International Journal of Semantic Computing, vol. 0, no. 0, pp. 1-32, 0000.

Abstract | Links | BibTeX

@article{doi:10.1142/S1793351X22500027,

title = {Evolving Deep Neural Networks with Cultural Algorithms for Real-Time Industrial Applications},

author = {Faisal Waris and Robert G. Reynolds and Joonho Lee},

url = {https://doi.org/10.1142/S1793351X22500027},

doi = {10.1142/S1793351X22500027},

year  = {0000},

date = {0000-01-01},

urldate = {0000-01-01},

journal = {International Journal of Semantic Computing},

volume = {0},

number = {0},

pages = {1-32},

abstract = {The goal of this paper is to investigate the applicability of evolutionary algorithms to the design of real-time industrial controllers. Present-day “deep learning” (DL) is firmly established as a useful tool for addressing many practical problems. This has spurred the development of neural architecture search (NAS) methods in order to automate the model search activity. CATNeuro is a NAS algorithm based on the graph evolution concept devised by Neuroevolution of Augmenting Topologies (NEAT) but propelled by cultural algorithm (CA) as the evolutionary driver. The CA is a network-based, stochastic optimization framework inspired by problem solving in human cultures. Knowledge distribution (KD) across the network of graph models is a key to problem solving success in CAT systems. Two alternative mechanisms for KD across the network are employed. One supports cooperation (CATNeuro) in the network and the other competition (WM). To test the viability of each configuration prior to use in the industrial setting, they were applied to the design of a real-time controller for a two-dimensional fighting game. While both were able to beat the AI program that came with the fighting game, the cooperative method performed statistically better. As a result, it was used to track the motion of a trailer (in lateral and vertical directions) using a camera mounted on the tractor vehicle towing the trailer. In this second real-time application (trailer motion), the CATNeuro configuration was compared to the original NEAT (elitist) method of evolution. CATNeuro is found to perform statistically better than NEAT in many aspects of the design including model training loss, model parameter size, and overall model structure consistency. In both scenarios, the performance improvements were attributed to the increased model diversity due to the interaction of CA knowledge sources both cooperatively and competitively.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

68.

Simsek, Ozlem Imik; Alagoz, Baris Baykant

Optimal architecture artificial neural network model design with exploitative alpha gray wolf optimization for soft calibration of CO concentration measurements in electronic nose applications Journal Article

In: Transactions of the Institute of Measurement and Control, vol. 0, no. 0, pp. 01423312221119648, 0000.

Abstract | Links | BibTeX

@article{doi:10.1177/01423312221119648,

title = {Optimal architecture artificial neural network model design with exploitative alpha gray wolf optimization for soft calibration of CO concentration measurements in electronic nose applications},

author = {Ozlem Imik Simsek and Baris Baykant Alagoz},

url = {https://doi.org/10.1177/01423312221119648},

doi = {10.1177/01423312221119648},

year  = {0000},

date = {0000-01-01},

urldate = {0000-01-01},

journal = {Transactions of the Institute of Measurement and Control},

volume = {0},

number = {0},

pages = {01423312221119648},

abstract = {The low-cost and small size solid-state sensor arrays are suitable to implement a wide-area electronic nose (e-nose) for real-time air quality monitoring. However, accuracy of these low-cost sensors is not adequate for precise measurements of pollutant concentrations. Artificial neural network (ANN) estimation models are used for the soft calibration of low-cost sensor array measurements and significantly improve the accuracy of low-cost multi-sensor measurements. However, optimality of neural architecture affects the performance of ANN estimation models, and optimization of the ANN architecture for a training data set is essential to improve data-driven modeling performance of ANNs to reach optimal neural complexity and improved generalization. In this study, an optimal architecture ANN estimator design scheme is suggested to improve the estimation performance of ANN models for e-nose applications. To this end, a gray wolf optimization (GWO) algorithm is modified, and an exploitative alpha gray wolf optimization (EA-GWO) algorithm is suggested. This modification enhances local exploitation skill of the best alpha gray wolf search agent, and thus allows the fine-tuning of ANN architectures by minimizing a multi-objective cost function that implements mean error search policy. Experimental study demonstrates the effectiveness of optimal architecture ANN models to estimate CO concentration from the low-cost multi-sensor data.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

67.

Soniya,; Singh, Lotika; Paul, Sandeep

Hybrid evolutionary network architecture search (HyENAS) for convolution class of deep neural networks with applications Journal Article

In: Expert Systems, vol. n/a, no. n/a, pp. e12690, 0000.

Abstract | Links | BibTeX

66.

Zhang, Bao Feng; Zhou, Guo Qiang

Control the number of skip-connects to improve robustness of the NAS algorithm Journal Article

In: IET Computer Vision, vol. n/a, no. n/a, 0000.

Abstract | Links | BibTeX

65.

Shashirangana, Jithmi; Padmasiri, Heshan; Meedeniya, Dulani; Perera, Charith; Nayak, Soumya R; Nayak, Janmenjoy; Vimal, Shanmuganthan; Kadry, Seifidine

License plate recognition using neural architecture search for edge devices Journal Article

In: International Journal of Intelligent Systems, vol. n/a, no. n/a, 0000.

Abstract | Links | BibTeX

4564 entries « ‹ 90 of 92 › »