Maintained by Difan Deng and Marius Lindauer.
The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.
Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.
Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.
2017
89.
Kani, Nagoor J; Elsheikh, Ahmed H
DR-RNN: A deep residual recurrent neural network for model reduction Technical Report
2017.
@techreport{Kani2017_duk,
title = {DR-RNN: A deep residual recurrent neural network for model reduction},
author = {Nagoor J Kani and Ahmed H Elsheikh},
url = {https://arxiv.org/abs/2004.10928},
year = {2017},
date = {2017-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
88.
Sun, Yanan; Xue, Bing; Zhang, Mengjie
A Particle Swarm Optimization-based Flexible Convolutional Auto-Encoder for Image Classification Technical Report
2017.
@techreport{Sun2017_mvr,
title = {A Particle Swarm Optimization-based Flexible Convolutional Auto-Encoder for Image Classification},
author = {Yanan Sun and Bing Xue and Mengjie Zhang},
url = {https://arxiv.org/abs/1712.05042},
year = {2017},
date = {2017-01-01},
volume = {abs/1712.05042},
key = {journals/corr/abs-1712-05042},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
87.
Negrinho, Renato; Gordon, Geoffrey J
DeepArchitect - Automatically Designing and Training Deep Architectures Technical Report
2017.
@techreport{Negrinho2017_fur,
title = {DeepArchitect - Automatically Designing and Training Deep Architectures},
author = {Renato Negrinho and Geoffrey J Gordon},
url = {https://arxiv.org/abs/1704.08792},
year = {2017},
date = {2017-01-01},
volume = {abs/1704.08792},
key = {journals/corr/NegrinhoG17},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
86.
Real, Esteban; Moore, Sherry; Selle, Andrew; Saxena, Saurabh; Suematsu, Yutaka Leon; Tan, Jie; Le, Quoc V; Kurakin, Alexey
Large-Scale Evolution of Image Classifiers Proceedings Article
In: pp. 2902-2911, 2017.
@inproceedings{Real2017_qek,
title = {Large-Scale Evolution of Image Classifiers},
author = {Esteban Real and Sherry Moore and Andrew Selle and Saurabh Saxena and Yutaka Leon Suematsu and Jie Tan and Quoc V Le and Alexey Kurakin},
url = {https://arxiv.org/abs/1703.01041},
year = {2017},
date = {2017-01-01},
pages = {2902-2911},
key = {conf/icml/RealMSSSTLK17},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
85.
Bello, Irwan; Zoph, Barret; Vasudevan, Vijay; Le, Quoc V
Neural Optimizer Search with Reinforcement Learning Proceedings Article
In: pp. 459-468, 2017.
@inproceedings{Bello2017_pnz,
title = {Neural Optimizer Search with Reinforcement Learning},
author = {Irwan Bello and Barret Zoph and Vijay Vasudevan and Quoc V Le},
url = {https://arxiv.org/abs/1709.07417},
year = {2017},
date = {2017-01-01},
pages = {459-468},
key = {conf/icml/BelloZVL17},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
84.
Wistuba, Martin
Finding Competitive Network Architectures Within a Day Using UCT Technical Report
2017.
@techreport{Wistuba2017_xtv,
title = {Finding Competitive Network Architectures Within a Day Using UCT},
author = {Martin Wistuba},
url = {https://arxiv.org/abs/1712.07420},
year = {2017},
date = {2017-01-01},
volume = {abs/1712.07420},
key = {journals/corr/abs-1712-07420},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
83.
Cortes, Corinna; Gonzalvo, Xavier; Kuznetsov, Vitaly; Mohri, Mehryar; Yang, Scott
AdaNet - Adaptive Structural Learning of Artificial Neural Networks Proceedings Article
In: pp. 874-883, 2017.
@inproceedings{Cortes2017_tpp,
title = {AdaNet - Adaptive Structural Learning of Artificial Neural Networks},
author = {Corinna Cortes and Xavier Gonzalvo and Vitaly Kuznetsov and Mehryar Mohri and Scott Yang},
url = {https://arxiv.org/abs/1607.01097},
year = {2017},
date = {2017-01-01},
pages = {874-883},
key = {conf/icml/CortesGKMY17},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
82.
Baker, Bowen; Gupta, Otkrist; Naik, Nikhil; Raskar, Ramesh
Designing Neural Network Architectures using Reinforcement Learning Proceedings Article
In: 2017.
@inproceedings{Baker2017_auf,
title = {Designing Neural Network Architectures using Reinforcement Learning},
author = {Bowen Baker and Otkrist Gupta and Nikhil Naik and Ramesh Raskar},
url = {https://arxiv.org/abs/1611.02167},
year = {2017},
date = {2017-01-01},
key = {conf/iclr/BakerGNR17},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
81.
Klein, Aaron; Falkner, Stefan; Springenberg, Jost Tobias; Hutter, Frank
Learning Curve Prediction with Bayesian Neural Networks Proceedings Article
In: 2017.
@inproceedings{Klein2017_ryl,
title = {Learning Curve Prediction with Bayesian Neural Networks},
author = {Aaron Klein and Stefan Falkner and Jost Tobias Springenberg and Frank Hutter},
url = {http://ml.informatik.uni-freiburg.de/papers/17-ICLR-LCNet.pdf},
year = {2017},
date = {2017-01-01},
key = {conf/iclr/KleinFSH17},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
80.
Li, Lisha; Jamieson, Kevin G; DeSalvo, Giulia; Rostamizadeh, Afshin; Talwalkar, Ameet
Hyperband - A Novel Bandit-Based Approach to Hyperparameter Optimization Journal Article
In: vol. 18, pp. 185:1-185:52, 2017.
@article{Li2017_nng,
title = {Hyperband - A Novel Bandit-Based Approach to Hyperparameter Optimization},
author = {Lisha Li and Kevin G Jamieson and Giulia DeSalvo and Afshin Rostamizadeh and Ameet Talwalkar},
url = {https://arxiv.org/abs/1603.06560},
year = {2017},
date = {2017-01-01},
volume = {18},
pages = {185:1-185:52},
key = {journals/jmlr/LiJDRT17},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2016
79.
Vu, Ngoc Thang; Adel, Heike; Gupta, Pankaj; Schütze, Hinrich
Combining Recurrent and Convolutional Neural Networks for Relation Classification Technical Report
2016.
@techreport{Vu2016_scr,
title = {Combining Recurrent and Convolutional Neural Networks for Relation
Classification},
author = {Ngoc Thang Vu and Heike Adel and Pankaj Gupta and Hinrich Schütze},
url = {https://scholarworks.rit.edu/cgi/viewcontent.cgi?article=11785&context=theses},
year = {2016},
date = {2016-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
78.
Chen, Long; Zhang, Hanwang; Xiao, Jun; Nie, Liqiang; Shao, Jian; Liu, Wei; Chua, Tat-Seng
SCA-CNN: Spatial and Channel-wise Attention in Convolutional Networks for Image Captioning Technical Report
2016.
@techreport{Chen2016_gwr,
title = {SCA-CNN: Spatial and Channel-wise Attention in Convolutional Networks
for Image Captioning},
author = {Long Chen and Hanwang Zhang and Jun Xiao and Liqiang Nie and Jian Shao and Wei Liu and Tat-Seng Chua},
url = {https://ieeexplore.ieee.org/abstract/document/9102879},
year = {2016},
date = {2016-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
77.
Smithson, Sean C; Yang, Guang; Gross, Warren J; Meyer, Brett H
Neural networks designing neural networks - multi-objective hyper-parameter optimization Proceedings Article
In: pp. 104, 2016.
@inproceedings{Smithson2016_ykh,
title = {Neural networks designing neural networks - multi-objective hyper-parameter optimization},
author = {Sean C Smithson and Guang Yang and Warren J Gross and Brett H Meyer},
url = {https://arxiv.org/abs/1611.02120},
doi = {10.1145/2966986.2967058},
year = {2016},
date = {2016-01-01},
pages = {104},
key = {conf/iccad/SmithsonYGM16},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
76.
Saxena, Shreyas; Verbeek, Jakob
Convolutional Neural Fabrics Proceedings Article
In: pp. 4053-4061, 2016.
@inproceedings{Saxena2016_neg,
title = {Convolutional Neural Fabrics},
author = {Shreyas Saxena and Jakob Verbeek},
url = {https://arxiv.org/abs/1606.02492},
year = {2016},
date = {2016-01-01},
pages = {4053-4061},
key = {conf/nips/SaxenaV16},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
75.
Loshchilov, Ilya; Hutter, Frank
CMA-ES for Hyperparameter Optimization of Deep Neural Networks Technical Report
2016.
@techreport{Loshchilov2016_ems,
title = {CMA-ES for Hyperparameter Optimization of Deep Neural Networks},
author = {Ilya Loshchilov and Frank Hutter},
url = {https://arxiv.org/abs/1604.07269},
year = {2016},
date = {2016-01-01},
volume = {abs/1604.07269},
key = {journals/corr/LoshchilovH16},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
74.
Chen, Tianqi; Goodfellow, Ian J; Shlens, Jonathon
Net2Net - Accelerating Learning via Knowledge Transfer Proceedings Article
In: 2016.
@inproceedings{Chen2016_uzf,
title = {Net2Net - Accelerating Learning via Knowledge Transfer},
author = {Tianqi Chen and Ian J Goodfellow and Jonathon Shlens},
url = {https://arxiv.org/abs/1511.05641},
year = {2016},
date = {2016-01-01},
key = {journals/corr/ChenGS15},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
73.
Chen, Justin
Combinatorially Generated Piecewise Activation Functions Technical Report
2016.
@techreport{Chen2016_okg,
title = {Combinatorially Generated Piecewise Activation Functions},
author = {Justin Chen},
url = {http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf},
year = {2016},
date = {2016-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2015
72.
Kulish, Vadym V; Malyi, Oleksandr I; Persson, Clas; Wu, Ping
Evaluation of Phosphorene as Anode Material for Na-ion Batteries from First Principles Technical Report
2015.
@techreport{Kulish2015_twu,
title = {Evaluation of Phosphorene as Anode Material for Na-ion Batteries from
First Principles},
author = {Vadym V Kulish and Oleksandr I Malyi and Clas Persson and Ping Wu},
url = {https://arxiv.org/abs/1912.12522},
year = {2015},
date = {2015-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
71.
Young, Steven R; Rose, Derek C; Karnowski, Thomas P; Lim, Seung-Hwan; Patton, Robert M
Optimizing deep learning hyper-parameters through an evolutionary algorithm Proceedings Article
In: pp. 4:1-4:5, 2015.
@inproceedings{Young2015_rur,
title = {Optimizing deep learning hyper-parameters through an evolutionary algorithm},
author = {Steven R Young and Derek C Rose and Thomas P Karnowski and Seung-Hwan Lim and Robert M Patton},
url = {https://dl.acm.org/citation.cfm?id=2834896},
doi = {10.1145/2834892.2834896},
year = {2015},
date = {2015-01-01},
pages = {4:1-4:5},
key = {conf/sc/YoungRKLP15},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2013
70.
Lopez, Pablo; Fernandez, David; Marin-Perez, Rafael; Jara, Antonio J; Gomez-Skarmeta, Antonio F
Scalable Oriented-Service Architecture for Heterogeneous and Ubiquitous IoT Domains Technical Report
2013.
@techreport{Lopez2013_yux,
title = {Scalable Oriented-Service Architecture for Heterogeneous and Ubiquitous
IoT Domains},
author = {Pablo Lopez and David Fernandez and Rafael Marin-Perez and Antonio J Jara and Antonio F Gomez-Skarmeta},
url = {https://ieeexplore.ieee.org/abstract/document/9269354},
year = {2013},
date = {2013-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
69.
Fister, Iztok; Jr., Iztok Fister; Yang, Xin-She; Brest, Janez
A comprehensive review of firefly algorithms Technical Report
2013.
@techreport{Fister2013_noy,
title = {A comprehensive review of firefly algorithms},
author = {Iztok Fister and Iztok Fister Jr. and Xin-She Yang and Janez Brest},
url = {https://link.springer.com/chapter/10.1007/978-981-15-0306-1_5},
year = {2013},
date = {2013-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2012
68.
Anas, M; Javaid, N; Mahmood, A; Raza, S M; Qasim, U; Khan, Z A
Minimizing Electricity Theft using Smart Meters in AMI Technical Report
2012.
@techreport{Anas2012_lze,
title = {Minimizing Electricity Theft using Smart Meters in AMI},
author = {M Anas and N Javaid and A Mahmood and S M Raza and U Qasim and Z A Khan},
url = {https://ieeexplore.ieee.org/abstract/document/9275605},
year = {2012},
date = {2012-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
67.
Snoek, Jasper; Larochelle, Hugo; Adams, Ryan P
Practical Bayesian Optimization of Machine Learning Algorithms Proceedings Article
In: pp. 2960-2968, 2012.
@inproceedings{Snoek2012_oyf,
title = {Practical Bayesian Optimization of Machine Learning Algorithms},
author = {Jasper Snoek and Hugo Larochelle and Ryan P Adams},
url = {https://papers.nips.cc/paper/4522-practical-bayesian-optimization-of-machine-learning-algorithms.pdf},
year = {2012},
date = {2012-01-01},
pages = {2960-2968},
key = {conf/nips/SnoekLA12},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2009
66.
Jaffres-Runser, Katia; Gorce, Jean-Marie; Comaniciu, Cristina
A multiobjective Tabu framework for the optimization and evaluation of wireless systems Technical Report
2009.
@techreport{Jaffres-Runser2009_vyd,
title = {A multiobjective Tabu framework for the optimization and evaluation of
wireless systems},
author = {Katia Jaffres-Runser and Jean-Marie Gorce and Cristina Comaniciu},
url = {https://ieeexplore.ieee.org/abstract/document/9127493},
year = {2009},
date = {2009-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
65.
Stanley, Kenneth O; D'Ambrosio, David B; Gauci, Jason
A Hypercube-Based Encoding for Evolving Large-Scale Neural Networks Journal Article
In: vol. 15, no. 2, pp. 185-212, 2009.
@article{Stanley2009_mid,
title = {A Hypercube-Based Encoding for Evolving Large-Scale Neural Networks},
author = {Kenneth O Stanley and David B D'Ambrosio and Jason Gauci},
url = {https://ieeexplore.ieee.org/document/6792316/},
doi = {10.1162/ARTL.2009.15.2.15202},
year = {2009},
date = {2009-01-01},
volume = {15},
number = {2},
pages = {185-212},
key = {journals/alife/StanleyDG09},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2008
64.
Floreano, Dario; Dürr, Peter; Mattiussi, Claudio
Neuroevolution - from architectures to learning Journal Article
In: vol. 1, no. 1, pp. 47-62, 2008.
@article{Floreano2008_wpi,
title = {Neuroevolution - from architectures to learning},
author = {Dario Floreano and Peter Dürr and Claudio Mattiussi},
url = {https://link.springer.com/article/10.1007/s12065-007-0002-4},
doi = {10.1007/S12065-007-0002-4},
year = {2008},
date = {2008-01-01},
volume = {1},
number = {1},
pages = {47-62},
key = {journals/evi/FloreanoDM08},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2005
63.
Czarnowski, Ireneusz; Jedrzejowicz, Piotr
An Agent-Based PLA for the Cascade Correlation Learning Architecture Proceedings Article
In: pp. 197-202, 2005.
@inproceedings{Czarnowski2005_quf,
title = {An Agent-Based PLA for the Cascade Correlation Learning Architecture},
author = {Ireneusz Czarnowski and Piotr Jedrzejowicz},
url = {https://papers.nips.cc/paper/207-the-cascade-correlation-learning-architecture},
doi = {10.1007/11550907_32},
year = {2005},
date = {2005-01-01},
pages = {197-202},
key = {conf/icann/CzarnowskiJ05},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2004
62.
Roy, N K; Potter, Walter D; Landau, D P
Designing Polymer Blends Using Neural Networks, Genetic Algorithms, and Markov Chains Journal Article
In: vol. 20, no. 3, pp. 215-229, 2004.
@article{Roy2004_app,
title = {Designing Polymer Blends Using Neural Networks, Genetic Algorithms, and Markov Chains},
author = {N K Roy and Walter D Potter and D P Landau},
url = {https://dl.acm.org/citation.cfm?id=94034},
doi = {10.1023/B:APIN.0000021414.50728.34},
year = {2004},
date = {2004-01-01},
volume = {20},
number = {3},
pages = {215-229},
key = {journals/apin/RoyPL04},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1994
61.
Angeline, Peter J; Saunders, Gregory M; Pollack, Jordan B
An evolutionary algorithm that constructs recurrent neural networks Journal Article
In: vol. 5, no. 1, pp. 54-65, 1994.
@article{Angeline1994_qts,
title = {An evolutionary algorithm that constructs recurrent neural networks},
author = {Peter J Angeline and Gregory M Saunders and Jordan B Pollack},
url = {https://ieeexplore.ieee.org/document/265960/},
doi = {10.1109/72.265960},
year = {1994},
date = {1994-01-01},
volume = {5},
number = {1},
pages = {54-65},
key = {journals/tnn/AngelineSP94},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1990
60.
Kitano, Hiroaki
Designing Neural Networks Using Genetic Algorithms with Graph Generation System Journal Article
In: vol. 4, no. 4, 1990.
@article{Kitano1990_uli,
title = {Designing Neural Networks Using Genetic Algorithms with Graph Generation System},
author = {Hiroaki Kitano},
url = {http://www.complex-systems.com/abstracts/v04_i04_a06/},
year = {1990},
date = {1990-01-01},
volume = {4},
number = {4},
key = {journals/compsys/Kitano90},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1988
59.
Tenorio, Manoel Fernando; Lee, Wei-Tsih
Self Organizing Neural Networks for the Identification Problem Proceedings Article
In: pp. 57-64, 1988.
@inproceedings{Tenorio1988_rex,
title = {Self Organizing Neural Networks for the Identification Problem},
author = {Manoel Fernando Tenorio and Wei-Tsih Lee},
url = {https://papers.nips.cc/paper/149-self-organizing-neural-networks-for-the-identification-problem},
year = {1988},
date = {1988-01-01},
pages = {57-64},
key = {conf/nips/TenorioL88},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
0000
58.
Waris, Faisal; Reynolds, Robert G.; Lee, Joonho
Evolving Deep Neural Networks with Cultural Algorithms for Real-Time Industrial Applications Journal Article
In: International Journal of Semantic Computing, vol. 0, no. 0, pp. 1-32, 0000.
@article{doi:10.1142/S1793351X22500027,
title = {Evolving Deep Neural Networks with Cultural Algorithms for Real-Time Industrial Applications},
author = {Faisal Waris and Robert G. Reynolds and Joonho Lee},
url = {https://doi.org/10.1142/S1793351X22500027},
doi = {10.1142/S1793351X22500027},
year = {0000},
date = {0000-01-01},
urldate = {0000-01-01},
journal = {International Journal of Semantic Computing},
volume = {0},
number = {0},
pages = {1-32},
abstract = {The goal of this paper is to investigate the applicability of evolutionary algorithms to the design of real-time industrial controllers. Present-day “deep learning” (DL) is firmly established as a useful tool for addressing many practical problems. This has spurred the development of neural architecture search (NAS) methods in order to automate the model search activity. CATNeuro is a NAS algorithm based on the graph evolution concept devised by Neuroevolution of Augmenting Topologies (NEAT) but propelled by cultural algorithm (CA) as the evolutionary driver. The CA is a network-based, stochastic optimization framework inspired by problem solving in human cultures. Knowledge distribution (KD) across the network of graph models is a key to problem solving success in CAT systems. Two alternative mechanisms for KD across the network are employed. One supports cooperation (CATNeuro) in the network and the other competition (WM). To test the viability of each configuration prior to use in the industrial setting, they were applied to the design of a real-time controller for a two-dimensional fighting game. While both were able to beat the AI program that came with the fighting game, the cooperative method performed statistically better. As a result, it was used to track the motion of a trailer (in lateral and vertical directions) using a camera mounted on the tractor vehicle towing the trailer. In this second real-time application (trailer motion), the CATNeuro configuration was compared to the original NEAT (elitist) method of evolution. CATNeuro is found to perform statistically better than NEAT in many aspects of the design including model training loss, model parameter size, and overall model structure consistency. In both scenarios, the performance improvements were attributed to the increased model diversity due to the interaction of CA knowledge sources both cooperatively and competitively.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The goal of this paper is to investigate the applicability of evolutionary algorithms to the design of real-time industrial controllers. Present-day “deep learning” (DL) is firmly established as a useful tool for addressing many practical problems. This has spurred the development of neural architecture search (NAS) methods in order to automate the model search activity. CATNeuro is a NAS algorithm based on the graph evolution concept devised by Neuroevolution of Augmenting Topologies (NEAT) but propelled by cultural algorithm (CA) as the evolutionary driver. The CA is a network-based, stochastic optimization framework inspired by problem solving in human cultures. Knowledge distribution (KD) across the network of graph models is a key to problem solving success in CAT systems. Two alternative mechanisms for KD across the network are employed. One supports cooperation (CATNeuro) in the network and the other competition (WM). To test the viability of each configuration prior to use in the industrial setting, they were applied to the design of a real-time controller for a two-dimensional fighting game. While both were able to beat the AI program that came with the fighting game, the cooperative method performed statistically better. As a result, it was used to track the motion of a trailer (in lateral and vertical directions) using a camera mounted on the tractor vehicle towing the trailer. In this second real-time application (trailer motion), the CATNeuro configuration was compared to the original NEAT (elitist) method of evolution. CATNeuro is found to perform statistically better than NEAT in many aspects of the design including model training loss, model parameter size, and overall model structure consistency. In both scenarios, the performance improvements were attributed to the increased model diversity due to the interaction of CA knowledge sources both cooperatively and competitively.
57.
Simsek, Ozlem Imik; Alagoz, Baris Baykant
In: Transactions of the Institute of Measurement and Control, vol. 0, no. 0, pp. 01423312221119648, 0000.
@article{doi:10.1177/01423312221119648,
title = {Optimal architecture artificial neural network model design with exploitative alpha gray wolf optimization for soft calibration of CO concentration measurements in electronic nose applications},
author = {Ozlem Imik Simsek and Baris Baykant Alagoz},
url = {https://doi.org/10.1177/01423312221119648},
doi = {10.1177/01423312221119648},
year = {0000},
date = {0000-01-01},
urldate = {0000-01-01},
journal = {Transactions of the Institute of Measurement and Control},
volume = {0},
number = {0},
pages = {01423312221119648},
abstract = {The low-cost and small size solid-state sensor arrays are suitable to implement a wide-area electronic nose (e-nose) for real-time air quality monitoring. However, accuracy of these low-cost sensors is not adequate for precise measurements of pollutant concentrations. Artificial neural network (ANN) estimation models are used for the soft calibration of low-cost sensor array measurements and significantly improve the accuracy of low-cost multi-sensor measurements. However, optimality of neural architecture affects the performance of ANN estimation models, and optimization of the ANN architecture for a training data set is essential to improve data-driven modeling performance of ANNs to reach optimal neural complexity and improved generalization. In this study, an optimal architecture ANN estimator design scheme is suggested to improve the estimation performance of ANN models for e-nose applications. To this end, a gray wolf optimization (GWO) algorithm is modified, and an exploitative alpha gray wolf optimization (EA-GWO) algorithm is suggested. This modification enhances local exploitation skill of the best alpha gray wolf search agent, and thus allows the fine-tuning of ANN architectures by minimizing a multi-objective cost function that implements mean error search policy. Experimental study demonstrates the effectiveness of optimal architecture ANN models to estimate CO concentration from the low-cost multi-sensor data.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The low-cost and small size solid-state sensor arrays are suitable to implement a wide-area electronic nose (e-nose) for real-time air quality monitoring. However, accuracy of these low-cost sensors is not adequate for precise measurements of pollutant concentrations. Artificial neural network (ANN) estimation models are used for the soft calibration of low-cost sensor array measurements and significantly improve the accuracy of low-cost multi-sensor measurements. However, optimality of neural architecture affects the performance of ANN estimation models, and optimization of the ANN architecture for a training data set is essential to improve data-driven modeling performance of ANNs to reach optimal neural complexity and improved generalization. In this study, an optimal architecture ANN estimator design scheme is suggested to improve the estimation performance of ANN models for e-nose applications. To this end, a gray wolf optimization (GWO) algorithm is modified, and an exploitative alpha gray wolf optimization (EA-GWO) algorithm is suggested. This modification enhances local exploitation skill of the best alpha gray wolf search agent, and thus allows the fine-tuning of ANN architectures by minimizing a multi-objective cost function that implements mean error search policy. Experimental study demonstrates the effectiveness of optimal architecture ANN models to estimate CO concentration from the low-cost multi-sensor data.
56.
Soniya,; Singh, Lotika; Paul, Sandeep
Hybrid evolutionary network architecture search (HyENAS) for convolution class of deep neural networks with applications Journal Article
In: Expert Systems, vol. n/a, no. n/a, pp. e12690, 0000.
@article{https://doi.org/10.1111/exsy.12690,
title = {Hybrid evolutionary network architecture search (HyENAS) for convolution class of deep neural networks with applications},
author = {Soniya and Lotika Singh and Sandeep Paul},
url = {https://onlinelibrary.wiley.com/doi/abs/10.1111/exsy.12690},
doi = {https://doi.org/10.1111/exsy.12690},
journal = {Expert Systems},
volume = {n/a},
number = {n/a},
pages = {e12690},
abstract = {Abstract Convolutional Neural Networks (CNNs) and its variants are increasingly used across wide domain of applications achieving high performance measures. For high performance, application specific CNN architecture is required, hence the need for network architecture search (NAS) becomes essential. This paper proposes a hybrid evolutionary approach for network architecture search (HyENAS), and targets convolution class of neural networks. One of the significant contribution of this technique is to completely evolve the high performance network by simultaneously finding network structures and their corresponding parameters. An elegant string representation has been proposed which efficiently represents the network. The concept of sparse block evolving requisite layer wise features for dense network is deployed. This permits the network to dynamically evolve for a specific application. In comparison to the other state-of-art methods, the high performance of the proposed HyENAS approach is demonstrated across various benchmark data sets belonging to the domain of malariology, oncology, neurology, ophthalmology, and genomics. Further, to deploy the proposed model on lower hardware specification devices, another salient feature of the HyENAS technique is to seamlessly sift out the simpler network architecture with comparable accuracy.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Abstract Convolutional Neural Networks (CNNs) and its variants are increasingly used across wide domain of applications achieving high performance measures. For high performance, application specific CNN architecture is required, hence the need for network architecture search (NAS) becomes essential. This paper proposes a hybrid evolutionary approach for network architecture search (HyENAS), and targets convolution class of neural networks. One of the significant contribution of this technique is to completely evolve the high performance network by simultaneously finding network structures and their corresponding parameters. An elegant string representation has been proposed which efficiently represents the network. The concept of sparse block evolving requisite layer wise features for dense network is deployed. This permits the network to dynamically evolve for a specific application. In comparison to the other state-of-art methods, the high performance of the proposed HyENAS approach is demonstrated across various benchmark data sets belonging to the domain of malariology, oncology, neurology, ophthalmology, and genomics. Further, to deploy the proposed model on lower hardware specification devices, another salient feature of the HyENAS technique is to seamlessly sift out the simpler network architecture with comparable accuracy.
55.
Zhang, Bao Feng; Zhou, Guo Qiang
Control the number of skip-connects to improve robustness of the NAS algorithm Journal Article
In: IET Computer Vision, vol. n/a, no. n/a, 0000.
@article{https://doi.org/10.1049/cvi2.12036,
title = {Control the number of skip-connects to improve robustness of the NAS algorithm},
author = {Bao Feng Zhang and Guo Qiang Zhou},
url = {https://ietresearch.onlinelibrary.wiley.com/doi/abs/10.1049/cvi2.12036},
doi = {https://doi.org/10.1049/cvi2.12036},
journal = {IET Computer Vision},
volume = {n/a},
number = {n/a},
abstract = {Abstract Recently, the gradient-based neural architecture search has made remarkable progress with the characteristics of high efficiency and fast convergence. However, two common problems in the gradient-based NAS algorithms are found. First, with the increase in the raining time, the NAS algorithm tends to skip-connect operation, leading to performance degradation and instability results. Second, another problem is no reasonable allocation of computing resources on valuable candidate network models. The above two points lead to the difficulty in searching the optimal sub-network and poor stability. To address them, the trick of pre-training the super-net is applied, so that each operation has an equal opportunity to develop its strength, which provides a fair competition condition for the convergence of the architecture parameters. In addition, a skip-controller is proposed to ensure each sampled sub-network with an appropriate number of skip-connects. The experiments were performed on three mainstream datasets CIFAR-10, CIFAR-100 and ImageNet, in which the improved method achieves comparable results with higher accuracy and stronger robustness.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Abstract Recently, the gradient-based neural architecture search has made remarkable progress with the characteristics of high efficiency and fast convergence. However, two common problems in the gradient-based NAS algorithms are found. First, with the increase in the raining time, the NAS algorithm tends to skip-connect operation, leading to performance degradation and instability results. Second, another problem is no reasonable allocation of computing resources on valuable candidate network models. The above two points lead to the difficulty in searching the optimal sub-network and poor stability. To address them, the trick of pre-training the super-net is applied, so that each operation has an equal opportunity to develop its strength, which provides a fair competition condition for the convergence of the architecture parameters. In addition, a skip-controller is proposed to ensure each sampled sub-network with an appropriate number of skip-connects. The experiments were performed on three mainstream datasets CIFAR-10, CIFAR-100 and ImageNet, in which the improved method achieves comparable results with higher accuracy and stronger robustness.
54.
Shashirangana, Jithmi; Padmasiri, Heshan; Meedeniya, Dulani; Perera, Charith; Nayak, Soumya R; Nayak, Janmenjoy; Vimal, Shanmuganthan; Kadry, Seifidine
License plate recognition using neural architecture search for edge devices Journal Article
In: International Journal of Intelligent Systems, vol. n/a, no. n/a, 0000.
@article{https://doi.org/10.1002/int.22471,
title = {License plate recognition using neural architecture search for edge devices},
author = {Jithmi Shashirangana and Heshan Padmasiri and Dulani Meedeniya and Charith Perera and Soumya R Nayak and Janmenjoy Nayak and Shanmuganthan Vimal and Seifidine Kadry},
url = {https://onlinelibrary.wiley.com/doi/abs/10.1002/int.22471},
doi = {https://doi.org/10.1002/int.22471},
journal = {International Journal of Intelligent Systems},
volume = {n/a},
number = {n/a},
abstract = {Abstract The mutually beneficial blend of artificial intelligence with internet of things has been enabling many industries to develop smart information processing solutions. The implementation of technology enhanced industrial intelligence systems is challenging with the environmental conditions, resource constraints and safety concerns. With the era of smart homes and cities, domains like automated license plate recognition (ALPR) are exploring automate tasks such as traffic management and fraud detection. This paper proposes an optimized decision support solution for ALPR that works purely on edge devices at night-time. Although ALPR is a frequently addressed research problem in the domain of intelligent systems, still they are generally computationally intensive and unable to run on edge devices with limited resources. Therefore, as a novel approach, we consider the complex aspects related to deploying lightweight yet efficient and fast ALPR models on embedded devices. The usability of the proposed models is assessed in real-world with a proof-of-concept hardware design and achieved competitive results to the state-of-the-art ALPR solutions that run on server-grade hardware with intensive resources.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Abstract The mutually beneficial blend of artificial intelligence with internet of things has been enabling many industries to develop smart information processing solutions. The implementation of technology enhanced industrial intelligence systems is challenging with the environmental conditions, resource constraints and safety concerns. With the era of smart homes and cities, domains like automated license plate recognition (ALPR) are exploring automate tasks such as traffic management and fraud detection. This paper proposes an optimized decision support solution for ALPR that works purely on edge devices at night-time. Although ALPR is a frequently addressed research problem in the domain of intelligent systems, still they are generally computationally intensive and unable to run on edge devices with limited resources. Therefore, as a novel approach, we consider the complex aspects related to deploying lightweight yet efficient and fast ALPR models on embedded devices. The usability of the proposed models is assessed in real-world with a proof-of-concept hardware design and achieved competitive results to the state-of-the-art ALPR solutions that run on server-grade hardware with intensive resources.
53.
Wang, Xingbin; Zhao, Boyan; Hou, Rui; Awad, Amro; Tian, Zhihong; Meng, Dan
NASGuard: A Novel Accelerator Architecture for Robust Neural Architecture Search (NAS) Networks Proceedings Article
In: 2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA), 0000.
@inproceedings{WangISCA2021,
title = {NASGuard: A Novel Accelerator Architecture for Robust Neural Architecture Search (NAS) Networks},
author = {Xingbin Wang and Boyan Zhao and Rui Hou and Amro Awad and Zhihong Tian and Dan Meng},
url = {https://conferences.computer.org/iscapub/pdfs/ISCA2021-4ghucdBnCWYB7ES2Pe4YdT/333300a776/333300a776.pdf},
booktitle = {2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA)},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
52.
Zhang, Jianwei; Li, Dong; Wang, Lituan; Zhang, Lei
One-Shot Neural Architecture Search by Dynamically Pruning Supernet in Hierarchical Order Journal Article
In: International Journal of Neural Systems, 0000, (PMID: 34128778).
@article{doi:10.1142/S0129065721500295,
title = {One-Shot Neural Architecture Search by Dynamically Pruning Supernet in Hierarchical Order},
author = {Jianwei Zhang and Dong Li and Lituan Wang and Lei Zhang},
url = {https://doi.org/10.1142/S0129065721500295},
doi = {10.1142/S0129065721500295},
journal = {International Journal of Neural Systems},
abstract = {Neural Architecture Search (NAS), which aims at automatically designing neural architectures, recently draw a growing research interest. Different from conventional NAS methods, in which a large number of neural architectures need to be trained for evaluation, the one-shot NAS methods only have to train one supernet which synthesizes all the possible candidate architectures. As a result, the search efficiency could be significantly improved by sharing the supernet’s weights during the candidate architectures’ evaluation. This strategy could greatly speed up the search process but suffer a challenge that the evaluation based on sharing weights is not predictive enough. Recently, pruning the supernet during the search has been proven to be an efficient way to alleviate this problem. However, the pruning direction in complex-structured search space remains unexplored. In this paper, we revisited the role of path dropout strategy, which drops the neural operations instead of the neurons, in supernet training, and several interesting characters of the supernet trained with dropout are found. Based on the observations, a Hierarchically-Ordered Pruning Neural Architecture Search (HOPNAS) algorithm is proposed by dynamically pruning the supernet with a proper pruning direction. Experimental results indicate that our method is competitive with state-of-the-art approaches on CIFAR10 and ImageNet.},
note = {PMID: 34128778},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Neural Architecture Search (NAS), which aims at automatically designing neural architectures, recently draw a growing research interest. Different from conventional NAS methods, in which a large number of neural architectures need to be trained for evaluation, the one-shot NAS methods only have to train one supernet which synthesizes all the possible candidate architectures. As a result, the search efficiency could be significantly improved by sharing the supernet’s weights during the candidate architectures’ evaluation. This strategy could greatly speed up the search process but suffer a challenge that the evaluation based on sharing weights is not predictive enough. Recently, pruning the supernet during the search has been proven to be an efficient way to alleviate this problem. However, the pruning direction in complex-structured search space remains unexplored. In this paper, we revisited the role of path dropout strategy, which drops the neural operations instead of the neurons, in supernet training, and several interesting characters of the supernet trained with dropout are found. Based on the observations, a Hierarchically-Ordered Pruning Neural Architecture Search (HOPNAS) algorithm is proposed by dynamically pruning the supernet with a proper pruning direction. Experimental results indicate that our method is competitive with state-of-the-art approaches on CIFAR10 and ImageNet.
51.
Li, Xiao; Lei, Lin; Kuang, Gangyao
Multi-Modal Fusion Architecture Search for Land over Classification using Heterogeneous Remote Sensing Images Technical Report
0000.
@techreport{Li2021,
title = {Multi-Modal Fusion Architecture Search for Land over Classification using Heterogeneous Remote Sensing Images },
author = {Xiao Li and Lin Lei and Gangyao Kuang},
url = {https://www.researchgate.net/profile/Xiao-Li-120/publication/353236680_MULTI-MODAL_FUSION_ARCHITECTURE_SEARCH_FOR_LAND_COVER_CLASSIFICATION_USING_HETEROGENEOUS_REMOTE_SENSING_IMAGES/links/60eeae6316f9f31300802de4/MULTI-MODAL-FUSION-ARCHITECTURE-SEARCH-FOR-LAND-COVER-CLASSIFICATION-USING-HETEROGENEOUS-REMOTE-SENSING-IMAGES.pdf},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
50.
Sapra, Dolly; Pimentel, Andy D.
Designing convolutional neural networks with constrained evolutionary piecemeal training Journal Article
In: Applied Intelligence , 0000.
@article{Sapra2021,
title = {Designing convolutional neural networks with constrained evolutionary piecemeal training},
author = {Dolly Sapra and Andy D. Pimentel },
url = {https://link.springer.com/article/10.1007/s10489-021-02679-7},
journal = {Applied Intelligence },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
49.
Artin, Javad; Valizadeh, Amin; Ahmadi, Mohsen; Kumar, Sathish A. P.; Sharifi, Abbas
In: Complexity, 0000.
@article{Artin21,
title = {Presentation of a Novel Method for Prediction of Traffic with Climate Condition Based on Ensemble Learning of Neural Architecture Search (NAS) and Linear Regression},
author = {Javad Artin and Amin Valizadeh and Mohsen Ahmadi and Sathish A. P. Kumar and Abbas Sharifi },
url = {https://doi.org/10.1155/2021/8500572},
journal = {Complexity},
abstract = {Traffic prediction is critical to expanding a smart city and country because it improves urban planning and traffic management. This prediction is very challenging due to the multifactorial and random nature of traffic. This study presented a method based on ensemble learning to predict urban traffic congestion based on weather criteria. We used the NAS algorithm, which in the output based on heuristic methods creates an optimal model concerning input data. We had 400 data, which included the characteristics of the day’s weather, including six features: absolute humidity, dew point, visibility, wind speed, cloud height, and temperature, which in the final column is the urban traffic congestion target. We have analyzed linear regression with the results obtained in the project; this method was more efficient than other regression models. This method had an error of 0.00002 in terms of MSE criteria and SVR, random forest, and MLP methods, which have error values of 0.01033, 0.00003, and 0.0011, respectively. According to the MAE criterion, this method has a value of 0.0039. The other methods have obtained values of 0.0850, 0.0045, and 0.027, respectively, which show that our proposed model has a minor error than other methods and has been able to outpace the other models.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Traffic prediction is critical to expanding a smart city and country because it improves urban planning and traffic management. This prediction is very challenging due to the multifactorial and random nature of traffic. This study presented a method based on ensemble learning to predict urban traffic congestion based on weather criteria. We used the NAS algorithm, which in the output based on heuristic methods creates an optimal model concerning input data. We had 400 data, which included the characteristics of the day’s weather, including six features: absolute humidity, dew point, visibility, wind speed, cloud height, and temperature, which in the final column is the urban traffic congestion target. We have analyzed linear regression with the results obtained in the project; this method was more efficient than other regression models. This method had an error of 0.00002 in terms of MSE criteria and SVR, random forest, and MLP methods, which have error values of 0.01033, 0.00003, and 0.0011, respectively. According to the MAE criterion, this method has a value of 0.0039. The other methods have obtained values of 0.0850, 0.0045, and 0.027, respectively, which show that our proposed model has a minor error than other methods and has been able to outpace the other models.
48.
Gong, Yunhong; Sun, Yanan; Peng, Dezhong; Chen, Peng; Yan, Zhongtai; Yang, Ke
Analyze COVID-19 CT images based on evolutionary algorithm with dynamic searching space Journal Article
In: Complex & Intelligent Systems , 0000.
@article{Gong2021,
title = {Analyze COVID-19 CT images based on evolutionary algorithm with dynamic searching space},
author = {Yunhong Gong and Yanan Sun and Dezhong Peng and Peng Chen and Zhongtai Yan and Ke Yang },
url = {https://link.springer.com/article/10.1007/s40747-021-00513-8},
journal = {Complex & Intelligent Systems },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
47.
Cheng, Hsin-Pai
Efficient and Generalizable Neural Architecture Search for Visual Recognition PhD Thesis
0000.
@phdthesis{ChengPhD2021,
title = {Efficient and Generalizable Neural Architecture Search for Visual Recognition},
author = {Hsin-Pai Cheng},
url = {https://dukespace.lib.duke.edu/dspace/bitstream/handle/10161/23808/Cheng_duke_0066D_16412.pdf},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
46.
Zhu, Xunyu; Li, Jian; Liu, Yong; Liao, Jun; Wang, Weiping
Operation-level Progressive Differentiable Architecture Search Technical Report
0000.
@techreport{ZhuDARTS2021,
title = {Operation-level Progressive Differentiable Architecture Search},
author = {Xunyu Zhu and Jian Li and Yong Liu and Jun Liao and Weiping Wang},
url = {https://gsai.ruc.edu.cn/uploads/20210924/52c916158c2b3d29015ca71d85484c27.pdf},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
45.
Huang, Sian-Yao; Chu, Wei-Ta
OSNASLib: One-Shot NAS Library Proceedings Article
In: ICCV 2021 Workshop on Neural Architectures: Past, Present and Future, 0000.
@inproceedings{HuangISNASLib2021,
title = {OSNASLib: One-Shot NAS Library},
author = {Sian-Yao Huang and Wei-Ta Chu},
url = {https://neural-architecture-ppf.github.io/papers/00010.pdf},
booktitle = {ICCV 2021 Workshop on Neural Architectures: Past, Present and Future},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
44.
Madhu, G.; Bharadwaj, B. Lalith; Boddeda, Rohit; Vardhan, Sai; Kautish, K. Sandeep; Alnowibet, Khalid; Alrasheedi, Adel F.; Mohamed, Ali Wagdy
Deep Stacked Ensemble Learning Model for COVID-19 Classification Technical Report
0000.
@techreport{Madhu2021,
title = {Deep Stacked Ensemble Learning Model for COVID-19 Classification},
author = {G. Madhu and B. Lalith Bharadwaj and Rohit Boddeda and Sai Vardhan and K. Sandeep Kautish and Khalid Alnowibet and Adel F. Alrasheedi and Ali Wagdy Mohamed},
url = {https://www.researchgate.net/profile/B-Lalith-Bharadwaj/publication/355180470_Deep_Stacked_Ensemble_Learning_Model_for_COVID-19_Classification/links/6164cb470bf51d4817768880/Deep-Stacked-Ensemble-Learning-Model-for-COVID-19-Classification.pdf},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
43.
Kandukuri, Nikhil; Sakhtivel, Sangeetha; Xie, Pengtao
Neural Architecture Search For Skin Cancer Detection Technical Report
0000.
@techreport{Kandukuri2021,
title = {Neural Architecture Search For Skin Cancer Detection},
author = {Nikhil Kandukuri and Sangeetha Sakhtivel and Pengtao Xie},
url = {https://assets.researchsquare.com/files/rs-953342/v1_covered.pdf?c=1633976839},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
42.
Jiang, Yingying; Gan, Zhuoxin; Lin, Ke; A, Yong
AttNAS: Searching Attentions for Lightweight Semantic Segmentation Proceedings Article
In: British Machine Vision Conference (BMVC) 2021, 0000.
@inproceedings{Jiang2021,
title = {AttNAS: Searching Attentions for Lightweight Semantic Segmentation},
author = {Yingying Jiang and Zhuoxin Gan and Ke Lin and Yong A},
url = {https://www.bmvc2021-virtualconference.com/assets/papers/0575.pdf},
booktitle = {British Machine Vision Conference (BMVC) 2021},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
41.
Loni, Mohammad; Mousavi, Hamid; Riazati, Mohammad; Daneshtalab, Masoud; Sjödin, Mikael
TAS : Ternarized Neural Architecture Search for Resource-Constrained Edge Devices Proceedings Article
In: Design, Automation and Test in Europe ConferenceDesign, Automation and Test in Europe Conference (DATE) 2022, ANTWERP, BELGIUM :, 0000.
@inproceedings{Loni1620831,
title = {TAS : Ternarized Neural Architecture Search for Resource-Constrained Edge Devices},
author = {Mohammad Loni and Hamid Mousavi and Mohammad Riazati and Masoud Daneshtalab and Mikael Sjödin},
url = {https://www.diva-portal.org/smash/record.jsf?pid=diva2%3A1620831&dswid=-1720},
booktitle = {Design, Automation and Test in Europe ConferenceDesign, Automation and Test in Europe Conference (DATE) 2022, ANTWERP, BELGIUM :},
institution = {Shahid Bahonar University of Kerman, Iran},
abstract = {Ternary Neural Networks (TNNs) compress network weights and activation functions into 2-bit representation resulting in remarkable network compression and energy efficiency. However, there remains a significant gap in accuracy between TNNs and full-precision counterparts. Recent advances in Neural Architectures Search (NAS) promise opportunities in automated optimization for various deep learning tasks. Unfortunately, this area is unexplored for optimizing TNNs. This paper proposes TAS, a framework that drastically reduces the accuracy gap between TNNs and their full-precision counterparts by integrating quantization into the network design. We experienced that directly applying NAS to the ternary domain provides accuracy degradation as the search settings are customized for full-precision networks. To address this problem, we propose (i) a new cell template for ternary networks with maximum gradient propagation; and (ii) a novel learnable quantizer that adaptively relaxes the ternarization mechanism from the distribution of the weights and activation functions. Experimental results reveal that TAS delivers 2.64% higher accuracy and 2.8x memory saving over competing methods with the same bit-width resolution on the CIFAR-10 dataset. These results suggest that TAS is an effective method that paves the way for the efficient design of the next generation of quantized neural networks.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Ternary Neural Networks (TNNs) compress network weights and activation functions into 2-bit representation resulting in remarkable network compression and energy efficiency. However, there remains a significant gap in accuracy between TNNs and full-precision counterparts. Recent advances in Neural Architectures Search (NAS) promise opportunities in automated optimization for various deep learning tasks. Unfortunately, this area is unexplored for optimizing TNNs. This paper proposes TAS, a framework that drastically reduces the accuracy gap between TNNs and their full-precision counterparts by integrating quantization into the network design. We experienced that directly applying NAS to the ternary domain provides accuracy degradation as the search settings are customized for full-precision networks. To address this problem, we propose (i) a new cell template for ternary networks with maximum gradient propagation; and (ii) a novel learnable quantizer that adaptively relaxes the ternarization mechanism from the distribution of the weights and activation functions. Experimental results reveal that TAS delivers 2.64% higher accuracy and 2.8x memory saving over competing methods with the same bit-width resolution on the CIFAR-10 dataset. These results suggest that TAS is an effective method that paves the way for the efficient design of the next generation of quantized neural networks.
40.
Ma, Zhiyuan; Yu, Wenting; Zhang, Peng; Huang, Zhi; Lin, Anni; Xia, Yan
LPI Radar Waveform Recognition Based on Neural Architecture Search Journal Article
In: Computational Intelligence and Neuroscience, vol. 2022, 0000.
@article{Ma2022,
title = {LPI Radar Waveform Recognition Based on Neural Architecture Search},
author = {Zhiyuan Ma and Wenting Yu and Peng Zhang and Zhi Huang and Anni Lin and Yan Xia},
url = {https://doi.org/10.1155/2022/4628481},
journal = {Computational Intelligence and Neuroscience},
volume = {2022},
keywords = {},
pubstate = {published},
tppubtype = {article}
}