Maintained by Difan Deng and Marius Lindauer.
The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.
Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.
Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.
2022
2036.
Ren, Yankun; Li, Longfei; Yang, Xinxing; Zhou, Jun
AutoTransformer: Automatic Transformer Architecture Design For Time Series Classification Proceedings Article
In: Advances in Knowledge Discovery and Data Mining: 26th Pacific-Asia Conference, PAKDD 2022, Chengdu, China, May 16–19, 2022, Proceedings, Part I, pp. 143–155, Springer-Verlag, Chengdu, China, 2022, ISBN: 978-3-031-05932-2.
@inproceedings{10.1007/978-3-031-05933-9_12,
title = {AutoTransformer: Automatic Transformer Architecture Design For Time Series Classification},
author = {Yankun Ren and Longfei Li and Xinxing Yang and Jun Zhou},
url = {https://doi.org/10.1007/978-3-031-05933-9_12},
doi = {10.1007/978-3-031-05933-9_12},
isbn = {978-3-031-05932-2},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Advances in Knowledge Discovery and Data Mining: 26th Pacific-Asia Conference, PAKDD 2022, Chengdu, China, May 16–19, 2022, Proceedings, Part I},
pages = {143–155},
publisher = {Springer-Verlag},
address = {Chengdu, China},
abstract = {Time series classification (TSC) aims to assign labels to time series. Deep learning methods, such as InceptionTime and Transformer, achieve promising performances in TSC. Although deep learning methods do not require manually crafted features, they do require careful manual design of the network structure. The design of architectures heavily relies on researchers’ prior knowledge and experience. Due to the limitations of human’s knowledge, the designed architecture may not be optimal on the dataset of interest. To automate and optimize the architecture design, we propose a data-driven TSC network architecture design method called AutoTransformer. AutoTransformer designs the suitable network architecture automatically depending on the target TSC dataset. Inspired by the overall architecture of Transformer, we first propose a novel search space tailored for TSC. The search space includes a variety of substructures that are capable of extracting global and local features from time series. Then, with the help of neural architecture search (NAS) technique, a suitable network architecture for the target TSC dataset can be found from the search space. Experimental results show that AutoTransformer finds proper architectures on different TSC datasets and outperforms state-of-the-art methods on the UCR archive. Ablation studies verify the effectiveness of the proposed search space.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Time series classification (TSC) aims to assign labels to time series. Deep learning methods, such as InceptionTime and Transformer, achieve promising performances in TSC. Although deep learning methods do not require manually crafted features, they do require careful manual design of the network structure. The design of architectures heavily relies on researchers’ prior knowledge and experience. Due to the limitations of human’s knowledge, the designed architecture may not be optimal on the dataset of interest. To automate and optimize the architecture design, we propose a data-driven TSC network architecture design method called AutoTransformer. AutoTransformer designs the suitable network architecture automatically depending on the target TSC dataset. Inspired by the overall architecture of Transformer, we first propose a novel search space tailored for TSC. The search space includes a variety of substructures that are capable of extracting global and local features from time series. Then, with the help of neural architecture search (NAS) technique, a suitable network architecture for the target TSC dataset can be found from the search space. Experimental results show that AutoTransformer finds proper architectures on different TSC datasets and outperforms state-of-the-art methods on the UCR archive. Ablation studies verify the effectiveness of the proposed search space.
2035.
Deng, Difan; Karl, Florian; Hutter, Frank; Bischl, Bernd; Lindauer, Marius
Efficient Automated Deep Learning for Time Series Forecasting Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2205-05511,
title = {Efficient Automated Deep Learning for Time Series Forecasting},
author = {Difan Deng and Florian Karl and Frank Hutter and Bernd Bischl and Marius Lindauer},
url = {https://doi.org/10.48550/arXiv.2205.05511},
doi = {10.48550/arXiv.2205.05511},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2205.05511},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2034.
Du, Mengge; Chen, Yuntian; Zhang, Dongxiao
AutoKE: An automatic knowledge embedding framework for scientific machine learning Journal Article
In: CoRR, vol. abs/2205.05390, 2022.
@article{DBLP:journals/corr/abs-2205-05390,
title = {AutoKE: An automatic knowledge embedding framework for scientific machine learning},
author = {Mengge Du and Yuntian Chen and Dongxiao Zhang},
url = {https://doi.org/10.48550/arXiv.2205.05390},
doi = {10.48550/arXiv.2205.05390},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2205.05390},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2033.
Kim, Do-Guk; Lee, Heung-Chang
Proxyless Neural Architecture Adaptation for Supervised Learning and Self-Supervised Learning Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2205-07168,
title = {Proxyless Neural Architecture Adaptation for Supervised Learning and Self-Supervised Learning},
author = {Do-Guk Kim and Heung-Chang Lee},
url = {https://doi.org/10.48550/arXiv.2205.07168},
doi = {10.48550/arXiv.2205.07168},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2205.07168},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2032.
Sun, Jialiang; Zheng, Xiaohu; Yao, Wen; Zhang, Xiaoya; Zhou, Weien
Heat Source Layout Optimization Using Automatic Deep Learning Surrogate Model and Multimodal Neighborhood Search Algorithm Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2205-07812,
title = {Heat Source Layout Optimization Using Automatic Deep Learning Surrogate Model and Multimodal Neighborhood Search Algorithm},
author = {Jialiang Sun and Xiaohu Zheng and Wen Yao and Xiaoya Zhang and Weien Zhou},
url = {https://doi.org/10.48550/arXiv.2205.07812},
doi = {10.48550/arXiv.2205.07812},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2205.07812},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2031.
Belciug, Smaranda
Learning deep neural networks' architectures using differential evolution. Case study: Medical imaging processing Journal Article
In: Computers in Biology and Medicine, vol. 146, pp. 105623, 2022, ISSN: 0010-4825.
@article{BELCIUG2022105623,
title = {Learning deep neural networks' architectures using differential evolution. Case study: Medical imaging processing},
author = {Smaranda Belciug},
url = {https://www.sciencedirect.com/science/article/pii/S0010482522004152},
doi = {https://doi.org/10.1016/j.compbiomed.2022.105623},
issn = {0010-4825},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Computers in Biology and Medicine},
volume = {146},
pages = {105623},
abstract = {The COVID-19 pandemic has changed the way we practice medicine. Cancer patient and obstetric care landscapes have been distorted. Delaying cancer diagnosis or maternal-fetal monitoring increased the number of preventable deaths or pregnancy complications. One solution is using Artificial Intelligence to help the medical personnel establish the diagnosis in a faster and more accurate manner. Deep learning is the state-of-the-art solution for image classification. Researchers manually design the structure of fix deep learning neural networks structures and afterwards verify their performance. The goal of this paper is to propose a potential method for learning deep network architectures automatically. As the number of networks architectures increases exponentially with the number of convolutional layers in the network, we propose a differential evolution algorithm to traverse the search space. At first, we propose a way to encode the network structure as a candidate solution of fixed-length integer array, followed by the initialization of differential evolution method. A set of random individuals is generated, followed by mutation, recombination, and selection. At each generation the individuals with the poorest loss values are eliminated and replaced with more competitive individuals. The model has been tested on three cancer datasets containing MRI scans and histopathological images and two maternal-fetal screening ultrasound images. The novel proposed method has been compared and statistically benchmarked to four state-of-the-art deep learning networks: VGG16, ResNet50, Inception V3, and DenseNet169. The experimental results showed that the model is competitive to other state-of-the-art models, obtaining accuracies between 78.73% and 99.50% depending on the dataset it had been applied on.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The COVID-19 pandemic has changed the way we practice medicine. Cancer patient and obstetric care landscapes have been distorted. Delaying cancer diagnosis or maternal-fetal monitoring increased the number of preventable deaths or pregnancy complications. One solution is using Artificial Intelligence to help the medical personnel establish the diagnosis in a faster and more accurate manner. Deep learning is the state-of-the-art solution for image classification. Researchers manually design the structure of fix deep learning neural networks structures and afterwards verify their performance. The goal of this paper is to propose a potential method for learning deep network architectures automatically. As the number of networks architectures increases exponentially with the number of convolutional layers in the network, we propose a differential evolution algorithm to traverse the search space. At first, we propose a way to encode the network structure as a candidate solution of fixed-length integer array, followed by the initialization of differential evolution method. A set of random individuals is generated, followed by mutation, recombination, and selection. At each generation the individuals with the poorest loss values are eliminated and replaced with more competitive individuals. The model has been tested on three cancer datasets containing MRI scans and histopathological images and two maternal-fetal screening ultrasound images. The novel proposed method has been compared and statistically benchmarked to four state-of-the-art deep learning networks: VGG16, ResNet50, Inception V3, and DenseNet169. The experimental results showed that the model is competitive to other state-of-the-art models, obtaining accuracies between 78.73% and 99.50% depending on the dataset it had been applied on.
2030.
Frey, Nathan; Soklaski, Ryan; Axelrod, Simon; Samsi, Siddharth; Gomez-Bombarelli, Rafael; Coley, Connor; Gadepally, Vijay
Neural Scaling of Deep Chemical Models Journal Article
In: ChemRxiv, 2022.
@article{frey_soklaski_axelrod_samsi_gomez-bombarelli_coley_gadepally_2022,
title = {Neural Scaling of Deep Chemical Models},
author = {Nathan Frey and Ryan Soklaski and Simon Axelrod and Siddharth Samsi and Rafael Gomez-Bombarelli and Connor Coley and Vijay Gadepally},
doi = {10.26434/chemrxiv-2022-3s512},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {ChemRxiv},
publisher = {Cambridge Open Engage},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2029.
Miriyala, Srinivas Soumitri; Pujari, Keerthi NagaSree; Naik, Sakshi; Mitra, Kishalay
Evolutionary neural architecture search for surrogate models to enable optimization of industrial continuous crystallization process Journal Article
In: Powder Technology, vol. 405, pp. 117527, 2022, ISSN: 0032-5910.
@article{MIRIYALA2022117527,
title = {Evolutionary neural architecture search for surrogate models to enable optimization of industrial continuous crystallization process},
author = {Srinivas Soumitri Miriyala and Keerthi NagaSree Pujari and Sakshi Naik and Kishalay Mitra},
url = {https://www.sciencedirect.com/science/article/pii/S0032591022004211},
doi = {https://doi.org/10.1016/j.powtec.2022.117527},
issn = {0032-5910},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Powder Technology},
volume = {405},
pages = {117527},
abstract = {Optimal performance of the crystallization process is of utmost importance for industries handling bulk commodity chemicals to pharmaceuticals. Such an optimization exercise becomes extremely time expensive as the mathematical models mimicking such complex processes involve the solution of Integro-Differential Population Balance Equations using High Resolution Finite Volume Methods. In order to build a fast and robust data based alternative model, a surrogate assisted approach using Artificial Neural Networks has been proposed here. To overcome the heuristics-based estimation of the hyper-parameters in ANNs, we aim to contribute a novel Neural Architecture Search strategy for the auto-tuning of hyper-parameters integrated with sample size determination techniques. While solving a multi-objective optimization of crystallization process ensuring maximum productivity, the results from surrogates are compared with those of a high-fidelity physics driven model, which reports five order of magnitude speed improvement without sacrificing much on accuracy.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Optimal performance of the crystallization process is of utmost importance for industries handling bulk commodity chemicals to pharmaceuticals. Such an optimization exercise becomes extremely time expensive as the mathematical models mimicking such complex processes involve the solution of Integro-Differential Population Balance Equations using High Resolution Finite Volume Methods. In order to build a fast and robust data based alternative model, a surrogate assisted approach using Artificial Neural Networks has been proposed here. To overcome the heuristics-based estimation of the hyper-parameters in ANNs, we aim to contribute a novel Neural Architecture Search strategy for the auto-tuning of hyper-parameters integrated with sample size determination techniques. While solving a multi-objective optimization of crystallization process ensuring maximum productivity, the results from surrogates are compared with those of a high-fidelity physics driven model, which reports five order of magnitude speed improvement without sacrificing much on accuracy.
2028.
Lee, Joo-Hyun; Chang, Joon-Hyuk; Yang, Jae-Mo; Moon, Han-Gil
NAS-TasNet: Neural Architecture Search for Time-Domain Speech Separation Journal Article
In: IEEE Access, vol. 10, pp. 56031-56043, 2022.
@article{9777717,
title = {NAS-TasNet: Neural Architecture Search for Time-Domain Speech Separation},
author = {Joo-Hyun Lee and Joon-Hyuk Chang and Jae-Mo Yang and Han-Gil Moon},
url = {https://ieeexplore.ieee.org/abstract/document/9777717},
doi = {10.1109/ACCESS.2022.3176003},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Access},
volume = {10},
pages = {56031-56043},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2027.
Qian, Qi; Sang, Qingbing
No-reference image quality assessment based on automatic machine learning Journal Article
In: ITM Web Conf., vol. 45, pp. 01034, 2022.
@article{refId0b,
title = {No-reference image quality assessment based on automatic machine learning},
author = {Qi Qian and Qingbing Sang},
url = {https://doi.org/10.1051/itmconf/20224501034},
doi = {10.1051/itmconf/20224501034},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {ITM Web Conf.},
volume = {45},
pages = {01034},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2026.
Ogundokun, Roseline Oluwaseun; Misra, Sanjay; Douglas, Mychal; Damaševičius, Robertas; Maskeliūnas, Rytis
Medical Internet-of-Things Based Breast Cancer Diagnosis Using Hyperparameter-Optimized Neural Networks Journal Article
In: Future Internet, vol. 14, no. 5, 2022, ISSN: 1999-5903.
@article{fi14050153,
title = {Medical Internet-of-Things Based Breast Cancer Diagnosis Using Hyperparameter-Optimized Neural Networks},
author = {Roseline Oluwaseun Ogundokun and Sanjay Misra and Mychal Douglas and Robertas Damaševičius and Rytis Maskeliūnas},
url = {https://www.mdpi.com/1999-5903/14/5/153},
doi = {10.3390/fi14050153},
issn = {1999-5903},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Future Internet},
volume = {14},
number = {5},
abstract = {In today’s healthcare setting, the accurate and timely diagnosis of breast cancer is critical for recovery and treatment in the early stages. In recent years, the Internet of Things (IoT) has experienced a transformation that allows the analysis of real-time and historical data using artificial intelligence (AI) and machine learning (ML) approaches. Medical IoT combines medical devices and AI applications with healthcare infrastructure to support medical diagnostics. The current state-of-the-art approach fails to diagnose breast cancer in its initial period, resulting in the death of most women. As a result, medical professionals and researchers are faced with a tremendous problem in early breast cancer detection. We propose a medical IoT-based diagnostic system that competently identifies malignant and benign people in an IoT environment to resolve the difficulty of identifying early-stage breast cancer. The artificial neural network (ANN) and convolutional neural network (CNN) with hyperparameter optimization are used for malignant vs. benign classification, while the Support Vector Machine (SVM) and Multilayer Perceptron (MLP) were utilized as baseline classifiers for comparison. Hyperparameters are important for machine learning algorithms since they directly control the behaviors of training algorithms and have a significant effect on the performance of machine learning models. We employ a particle swarm optimization (PSO) feature selection approach to select more satisfactory features from the breast cancer dataset to enhance the classification performance using MLP and SVM, while grid-based search was used to find the best combination of the hyperparameters of the CNN and ANN models. The Wisconsin Diagnostic Breast Cancer (WDBC) dataset was used to test the proposed approach. The proposed model got a classification accuracy of 98.5% using CNN, and 99.2% using ANN.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
In today’s healthcare setting, the accurate and timely diagnosis of breast cancer is critical for recovery and treatment in the early stages. In recent years, the Internet of Things (IoT) has experienced a transformation that allows the analysis of real-time and historical data using artificial intelligence (AI) and machine learning (ML) approaches. Medical IoT combines medical devices and AI applications with healthcare infrastructure to support medical diagnostics. The current state-of-the-art approach fails to diagnose breast cancer in its initial period, resulting in the death of most women. As a result, medical professionals and researchers are faced with a tremendous problem in early breast cancer detection. We propose a medical IoT-based diagnostic system that competently identifies malignant and benign people in an IoT environment to resolve the difficulty of identifying early-stage breast cancer. The artificial neural network (ANN) and convolutional neural network (CNN) with hyperparameter optimization are used for malignant vs. benign classification, while the Support Vector Machine (SVM) and Multilayer Perceptron (MLP) were utilized as baseline classifiers for comparison. Hyperparameters are important for machine learning algorithms since they directly control the behaviors of training algorithms and have a significant effect on the performance of machine learning models. We employ a particle swarm optimization (PSO) feature selection approach to select more satisfactory features from the breast cancer dataset to enhance the classification performance using MLP and SVM, while grid-based search was used to find the best combination of the hyperparameters of the CNN and ANN models. The Wisconsin Diagnostic Breast Cancer (WDBC) dataset was used to test the proposed approach. The proposed model got a classification accuracy of 98.5% using CNN, and 99.2% using ANN.
2025.
Xu, Ship Peng; Wang, Ke; Hassan, Md. Rafiul; Hassan, Mohammad Mehedi; Chen, Chien-Ming
An Interpretive Perspective: Adversarial Trojaning Attack on Neural-Architecture-Search Enabled Edge AI Systems Journal Article
In: IEEE Transactions on Industrial Informatics, pp. 1-1, 2022.
@article{9780600,
title = {An Interpretive Perspective: Adversarial Trojaning Attack on Neural-Architecture-Search Enabled Edge AI Systems},
author = {Ship Peng Xu and Ke Wang and Md. Rafiul Hassan and Mohammad Mehedi Hassan and Chien-Ming Chen},
url = {https://ieeexplore.ieee.org/abstract/document/9780600},
doi = {10.1109/TII.2022.3177442},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Industrial Informatics},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2024.
Cummings, Daniel; Sarah, Anthony; Sridhar, Sharath Nittur; Szankin, Maciej; Muñoz, Juan Pablo; Sundaresan, Sairam
A Hardware-Aware Framework for Accelerating Neural Architecture Search Across Modalities Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2205-10358,
title = {A Hardware-Aware Framework for Accelerating Neural Architecture Search Across Modalities},
author = {Daniel Cummings and Anthony Sarah and Sharath Nittur Sridhar and Maciej Szankin and Juan Pablo Muñoz and Sairam Sundaresan},
url = {https://doi.org/10.48550/arXiv.2205.10358},
doi = {10.48550/arXiv.2205.10358},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2205.10358},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2023.
Li, Yanyu; Zhao, Pu; Yuan, Geng; Lin, Xue; Wang, Yanzhi; Chen, Xin
Pruning-as-Search: Efficient Neural Architecture Search via Channel Pruning and Structural Reparameterization Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2206-01198,
title = {Pruning-as-Search: Efficient Neural Architecture Search via Channel Pruning and Structural Reparameterization},
author = {Yanyu Li and Pu Zhao and Geng Yuan and Xue Lin and Yanzhi Wang and Xin Chen},
url = {https://doi.org/10.48550/arXiv.2206.01198},
doi = {10.48550/arXiv.2206.01198},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2206.01198},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2022.
Smith, James Seale; Seymour, Zachary; Chiu, Han-Pang
Incremental Learning with Differentiable Architecture and Forgetting Search Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2205-09875,
title = {Incremental Learning with Differentiable Architecture and Forgetting Search},
author = {James Seale Smith and Zachary Seymour and Han-Pang Chiu},
url = {https://doi.org/10.48550/arXiv.2205.09875},
doi = {10.48550/arXiv.2205.09875},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2205.09875},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2021.
Tuli, Shikhar; Dedhia, Bhishma; Tuli, Shreshth; Jha, Niraj K.
FlexiBERT: Are Current Transformer Architectures too Homogeneous and Rigid? Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2205-11656,
title = {FlexiBERT: Are Current Transformer Architectures too Homogeneous and Rigid?},
author = {Shikhar Tuli and Bhishma Dedhia and Shreshth Tuli and Niraj K. Jha},
url = {https://doi.org/10.48550/arXiv.2205.11656},
doi = {10.48550/arXiv.2205.11656},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2205.11656},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2020.
Risso, Matteo; Burrello, Alessio; Conti, Francesco; Lamberti, Lorenzo; Chen, Yukai; Benini, Luca; Macii, Enrico; Poncino, Massimo; Pagliari, Daniele Jahier
Lightweight Neural Architecture Search for Temporal Convolutional Networks at the Edge Journal Article
In: IEEE Transactions on Computers, pp. 1-1, 2022.
@article{9782512,
title = {Lightweight Neural Architecture Search for Temporal Convolutional Networks at the Edge},
author = {Matteo Risso and Alessio Burrello and Francesco Conti and Lorenzo Lamberti and Yukai Chen and Luca Benini and Enrico Macii and Massimo Poncino and Daniele Jahier Pagliari},
url = {https://ieeexplore.ieee.org/abstract/document/9782512},
doi = {10.1109/TC.2022.3177955},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Computers},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2019.
Gao, Yang; Zhang, Peng; Yang, Hong; Zhou, Chuan; Tian, Zhihong; Hu, Yue; Li, Zhao; Zhou, Jingren
GraphNAS++: Distributed Architecture Search for Graph Neural Networks Journal Article
In: IEEE Transactions on Knowledge and Data Engineering, pp. 1-1, 2022.
@article{9782531,
title = {GraphNAS++: Distributed Architecture Search for Graph Neural Networks},
author = {Yang Gao and Peng Zhang and Hong Yang and Chuan Zhou and Zhihong Tian and Yue Hu and Zhao Li and Jingren Zhou},
url = {https://ieeexplore.ieee.org/abstract/document/9782531},
doi = {10.1109/TKDE.2022.3178153},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Knowledge and Data Engineering},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2018.
Zhang, Zeyang; Chen, Yidong; Zhou, Changle
Self-Growing Binary Activation Network: A Novel Deep Learning Model With Dynamic Architecture Journal Article
In: IEEE Transactions on Neural Networks and Learning Systems, pp. 1-10, 2022.
@article{9783448,
title = {Self-Growing Binary Activation Network: A Novel Deep Learning Model With Dynamic Architecture},
author = {Zeyang Zhang and Yidong Chen and Changle Zhou},
url = {https://ieeexplore.ieee.org/abstract/document/9783448},
doi = {10.1109/TNNLS.2022.3176027},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Neural Networks and Learning Systems},
pages = {1-10},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2017.
Abbasi, Saad; Wong, Alexander; Shafiee, Mohammad Javad
MAPLE-X: Latency Prediction with Explicit Microprocessor Prior Knowledge Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2205-12660,
title = {MAPLE-X: Latency Prediction with Explicit Microprocessor Prior Knowledge},
author = {Saad Abbasi and Alexander Wong and Mohammad Javad Shafiee},
url = {https://doi.org/10.48550/arXiv.2205.12660},
doi = {10.48550/arXiv.2205.12660},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2205.12660},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2016.
Paulo, Maurício; Turnes, Javier Noa; Happ, P.; Ferreira, Matheus; Marques, Haroldo; Feitosa, Raul
HOW FAR SHOULD I LOOK? A NEURAL ARCHITECTURE SEARCH STRATEGY FOR SEMANTIC SEGMENTATION OF REMOTE SENSING IMAGES Journal Article
In: ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. V-3-2022, pp. 17-23, 2022.
@article{MauricioISPRS2022,
title = {HOW FAR SHOULD I LOOK? A NEURAL ARCHITECTURE SEARCH STRATEGY FOR SEMANTIC SEGMENTATION OF REMOTE SENSING IMAGES},
author = {Maurício Paulo and Javier Noa Turnes and P. Happ and Matheus Ferreira and Haroldo Marques and Raul Feitosa},
doi = {10.5194/isprs-annals-V-3-2022-17-2022},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences},
volume = {V-3-2022},
pages = {17-23},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2015.
Lopes, Vasco; Santos, Miguel; Degardin, Bruno; Alexandre, Luís A.
Efficient Guided Evolution for Neural Architecture Search Proceedings Article
In: GECCO2022, 2022.
@inproceedings{LopesGECCO2022,
title = {Efficient Guided Evolution for Neural Architecture Search},
author = {Vasco Lopes and Miguel Santos and Bruno Degardin and Luís A. Alexandre},
url = {http://www.di.ubi.pt/~lfbaa/pubs/GECCO2022.pdf},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {GECCO2022},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2014.
Wang, Yicheng; Han, Xiaotian; Chang, Chia-Yuan; Zha, Daochen; Braga-Neto, Ulisses; Hu, Xia
Auto-PINN: Understanding and Optimizing Physics-Informed Neural Architecture Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2205-13748,
title = {Auto-PINN: Understanding and Optimizing Physics-Informed Neural Architecture},
author = {Yicheng Wang and Xiaotian Han and Chia-Yuan Chang and Daochen Zha and Ulisses Braga-Neto and Xia Hu},
url = {https://doi.org/10.48550/arXiv.2205.13748},
doi = {10.48550/arXiv.2205.13748},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2205.13748},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2013.
Li, Yanyu; Zhao, Pu; Yuan, Geng; Lin, Xue; Wang, Yanzhi; Chen, Xin
Pruning-as-Search: Efficient Neural Architecture Search via Channel Pruning and Structural Reparameterization Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2206-01198b,
title = {Pruning-as-Search: Efficient Neural Architecture Search via Channel Pruning and Structural Reparameterization},
author = {Yanyu Li and Pu Zhao and Geng Yuan and Xue Lin and Yanzhi Wang and Xin Chen},
url = {https://doi.org/10.48550/arXiv.2206.01198},
doi = {10.48550/arXiv.2206.01198},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2206.01198},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2012.
Risso, Matteo; Burrello, Alessio; Benini, Luca; Macii, Enrico; Poncino, Massimo; Pagliari, Daniele Jahier
Multi-Complexity-Loss DNAS for Energy-Efficient and Memory-Constrained Deep Neural Networks Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2206-00302,
title = {Multi-Complexity-Loss DNAS for Energy-Efficient and Memory-Constrained Deep Neural Networks},
author = {Matteo Risso and Alessio Burrello and Luca Benini and Enrico Macii and Massimo Poncino and Daniele Jahier Pagliari},
url = {https://doi.org/10.48550/arXiv.2206.00302},
doi = {10.48550/arXiv.2206.00302},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2206.00302},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2011.
Li, Yanyu; Yuan, Geng; Wen, Yang; Hu, Eric; Evangelidis, Georgios; Tulyakov, Sergey; Wang, Yanzhi; Ren, Jian
EfficientFormer: Vision Transformers at MobileNet Speed Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2206-01191,
title = {EfficientFormer: Vision Transformers at MobileNet Speed},
author = {Yanyu Li and Geng Yuan and Yang Wen and Eric Hu and Georgios Evangelidis and Sergey Tulyakov and Yanzhi Wang and Jian Ren},
url = {https://doi.org/10.48550/arXiv.2206.01191},
doi = {10.48550/arXiv.2206.01191},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2206.01191},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2010.
Zheng, Xiawu; Fei, Xiang; Zhang, Lei; Wu, Chenglin; Chao, Fei; Liu, Jianzhuang; Zeng, Wei; Tian, Yonghong; Ji, Rongrong
Neural Architecture Search With Representation Mutual Information Proceedings Article
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11912–11921, 2022.
@inproceedings{zheng2022neural,
title = {Neural Architecture Search With Representation Mutual Information},
author = {Xiawu Zheng and Xiang Fei and Lei Zhang and Chenglin Wu and Fei Chao and Jianzhuang Liu and Wei Zeng and Yonghong Tian and Rongrong Ji},
url = {https://openaccess.thecvf.com/content/CVPR2022/papers/Zheng_Neural_Architecture_Search_With_Representation_Mutual_Information_CVPR_2022_paper.pdf},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages = {11912--11921},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2009.
Pan, Junyi; Sun, Chong; Zhou, Yizhou; Zhang, Ying; Li, Chen
Distribution Consistent Neural Architecture Search Proceedings Article
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10884–10893, 2022.
@inproceedings{pan2022distribution,
title = {Distribution Consistent Neural Architecture Search},
author = {Junyi Pan and Chong Sun and Yizhou Zhou and Ying Zhang and Chen Li},
url = {https://openaccess.thecvf.com/content/CVPR2022/papers/Pan_Distribution_Consistent_Neural_Architecture_Search_CVPR_2022_paper.pdf},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages = {10884--10893},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2008.
Xie, Pengtao; Du, Xuefeng
Performance-Aware Mutual Knowledge Distillation for Improving Neural Architecture Search Proceedings Article
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11922–11932, 2022.
@inproceedings{xie2022performance,
title = {Performance-Aware Mutual Knowledge Distillation for Improving Neural Architecture Search},
author = {Pengtao Xie and Xuefeng Du},
url = {https://openaccess.thecvf.com/content/CVPR2022/papers/Xie_Performance-Aware_Mutual_Knowledge_Distillation_for_Improving_Neural_Architecture_Search_CVPR_2022_paper.pdf},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages = {11922--11932},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2007.
Xu, Kepeng; He, Gang
DNAS: A Decoupled Global Neural Architecture Search Method Proceedings Article
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1979–1985, 2022.
@inproceedings{xu2022dnas,
title = {DNAS: A Decoupled Global Neural Architecture Search Method},
author = {Kepeng Xu and Gang He},
url = {https://openaccess.thecvf.com/content/CVPR2022W/NAS/papers/Xu_DNASA_Decoupled_Global_Neural_Architecture_Search_Method_CVPRW_2022_paper.pdf},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages = {1979--1985},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2006.
Shang, Ronghua; Zhu, Songling; Ren, Jinhong; Liu, Hangcheng; Jiao, Licheng
Evolutionary neural architecture search based on evaluation correction and functional units Journal Article
In: Knowledge-Based Systems, vol. 251, pp. 109206, 2022, ISSN: 0950-7051.
@article{SHANG2022109206,
title = {Evolutionary neural architecture search based on evaluation correction and functional units},
author = {Ronghua Shang and Songling Zhu and Jinhong Ren and Hangcheng Liu and Licheng Jiao},
url = {https://www.sciencedirect.com/science/article/pii/S0950705122006001},
doi = {https://doi.org/10.1016/j.knosys.2022.109206},
issn = {0950-7051},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Knowledge-Based Systems},
volume = {251},
pages = {109206},
abstract = {Neural architecture search (NAS) has been a great success in the automated design of deep neural networks. However, neural architecture search using evolutionary algorithms is challenging due to the diverse structure of neural networks and the difficulty in performance evaluation. To this end, this paper proposes an evolutionary neural architecture search algorithm (called EF-ENAS) based on evaluation corrections and functional units. First, a mating selection operation based on evaluation correction is developed, which can help EF-ENAS discriminate high-performance network architectures and reduce the harmful effects of low fidelity accuracy evaluation methods. Then, a functional unit-based network architecture crossover operation is designed, which divides the neural network into different functional units for crossover and protects valuable network architectures from destruction. Finally, the idea of species protection is introduced into the traditional environmental selection operation and a species protection-based environmental selection operation is designed, which can improve the diversity of network architectures in a population. The EF-ENAS is tested on ten benchmark datasets with varying complexities. In addition, the proposed algorithm is compared with 44 state-of-the-art algorithms, including DARTS, EvoCNN, CNN-GA, AE-CNN, etc. The experimental results show that the proposed algorithm11The code of EF-ENAS is available at https://github.com/codesl173/EF-ENAS. can automatically design neural networks and perform better.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Neural architecture search (NAS) has been a great success in the automated design of deep neural networks. However, neural architecture search using evolutionary algorithms is challenging due to the diverse structure of neural networks and the difficulty in performance evaluation. To this end, this paper proposes an evolutionary neural architecture search algorithm (called EF-ENAS) based on evaluation corrections and functional units. First, a mating selection operation based on evaluation correction is developed, which can help EF-ENAS discriminate high-performance network architectures and reduce the harmful effects of low fidelity accuracy evaluation methods. Then, a functional unit-based network architecture crossover operation is designed, which divides the neural network into different functional units for crossover and protects valuable network architectures from destruction. Finally, the idea of species protection is introduced into the traditional environmental selection operation and a species protection-based environmental selection operation is designed, which can improve the diversity of network architectures in a population. The EF-ENAS is tested on ten benchmark datasets with varying complexities. In addition, the proposed algorithm is compared with 44 state-of-the-art algorithms, including DARTS, EvoCNN, CNN-GA, AE-CNN, etc. The experimental results show that the proposed algorithm11The code of EF-ENAS is available at https://github.com/codesl173/EF-ENAS. can automatically design neural networks and perform better.
2005.
Liu, Shiqing; Zhang, Haoyu; Jin, Yaochu
A Survey on Surrogate-assisted Efficient Neural Architecture Search Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2206-01520,
title = {A Survey on Surrogate-assisted Efficient Neural Architecture Search},
author = {Shiqing Liu and Haoyu Zhang and Yaochu Jin},
url = {https://doi.org/10.48550/arXiv.2206.01520},
doi = {10.48550/arXiv.2206.01520},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2206.01520},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2004.
Kim, Youngkee; Jung, Soyi; Choi, Minseok; Kim, Joongheon
Search Space Adaptation for Differentiable Neural Architecture Search in Image Classification Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2206-02098,
title = {Search Space Adaptation for Differentiable Neural Architecture Search in Image Classification},
author = {Youngkee Kim and Soyi Jung and Minseok Choi and Joongheon Kim},
url = {https://doi.org/10.48550/arXiv.2206.02098},
doi = {10.48550/arXiv.2206.02098},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2206.02098},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2003.
Li, Wenshuo; Chen, Xinghao; Bai, Jinyu; Ning, Xuefei; Wang, Yunhe
Searching for Energy-Efficient Hybrid Adder-Convolution Neural Networks Proceedings Article
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1943–1952, 2022.
@inproceedings{li2022searching,
title = {Searching for Energy-Efficient Hybrid Adder-Convolution Neural Networks},
author = {Wenshuo Li and Xinghao Chen and Jinyu Bai and Xuefei Ning and Yunhe Wang},
url = {https://openaccess.thecvf.com/content/CVPR2022W/NAS/papers/Li_Searching_for_Energy-Efficient_Hybrid_Adder-Convolution_Neural_Networks_CVPRW_2022_paper.pdf},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages = {1943--1952},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2002.
Ferjani, Imen; Hidri, Minyar Sassi; Frihida, Ali
SiNoptiC: swarm intelligence optimisation of convolutional neural network architectures for text classification Journal Article
In: International Journal of Computer Applications in Technology, vol. 68, no. 1, pp. 82-100, 2022.
@article{doi:10.1504/IJCAT.2022.123237,
title = {SiNoptiC: swarm intelligence optimisation of convolutional neural network architectures for text classification},
author = {Imen Ferjani and Minyar Sassi Hidri and Ali Frihida},
url = {https://www.inderscienceonline.com/doi/abs/10.1504/IJCAT.2022.123237},
doi = {10.1504/IJCAT.2022.123237},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {International Journal of Computer Applications in Technology},
volume = {68},
number = {1},
pages = {82-100},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2001.
Liu, Jinyuan; Wu, Yuhui; Wu, Guanyao; Liu, Risheng; Fan, Xin
Learn to Search a Lightweight Architecture for Target-aware Infrared and Visible Image Fusion Journal Article
In: IEEE Signal Processing Letters, pp. 1-5, 2022.
@article{9789723,
title = {Learn to Search a Lightweight Architecture for Target-aware Infrared and Visible Image Fusion},
author = {Jinyuan Liu and Yuhui Wu and Guanyao Wu and Risheng Liu and Xin Fan},
url = {https://ieeexplore.ieee.org/abstract/document/9789723},
doi = {10.1109/LSP.2022.3180672},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Signal Processing Letters},
pages = {1-5},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2000.
Li, Zhuowei; Gao, Yibo; Zha, Zhenzhou; Hu, Zhiqiang; Xia, Qing; Zhang, Shaoting; Metaxas, Dimitris N.
Towards Self-supervised and Weight-preserving Neural Architecture Search Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2206-04125,
title = {Towards Self-supervised and Weight-preserving Neural Architecture Search},
author = {Zhuowei Li and Yibo Gao and Zhenzhou Zha and Zhiqiang Hu and Qing Xia and Shaoting Zhang and Dimitris N. Metaxas},
url = {https://doi.org/10.48550/arXiv.2206.04125},
doi = {10.48550/arXiv.2206.04125},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2206.04125},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1999.
Du, Yipeng; Liu, Jian; Wang, Xiang; Wang, Peng
SSVEP based Emotion Recognition for IoT via Multiobjective Neural Architecture Search Journal Article
In: IEEE Internet of Things Journal, pp. 1-1, 2022.
@article{9793561,
title = {SSVEP based Emotion Recognition for IoT via Multiobjective Neural Architecture Search},
author = {Yipeng Du and Jian Liu and Xiang Wang and Peng Wang},
url = {https://ieeexplore.ieee.org/abstract/document/9793561},
doi = {10.1109/JIOT.2022.3180215},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Internet of Things Journal},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1998.
Zhang, Yuanhan; Zhou, Kaiyang; Liu, Ziwei
Neural Prompt Search Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2206-04673,
title = {Neural Prompt Search},
author = {Yuanhan Zhang and Kaiyang Zhou and Ziwei Liu},
url = {https://doi.org/10.48550/arXiv.2206.04673},
doi = {10.48550/arXiv.2206.04673},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2206.04673},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1997.
Huang, Mingqiang; Liu, Yucen; Man, Changhai; Li, Kai; Cheng, Quan; Mao, Wei; Yu, Hao
A High Performance Multi-Bit-Width Booth Vector Systolic Accelerator for NAS Optimized Deep Learning Neural Networks Journal Article
In: IEEE Transactions on Circuits and Systems I: Regular Papers, pp. 1-13, 2022.
@article{9793397,
title = {A High Performance Multi-Bit-Width Booth Vector Systolic Accelerator for NAS Optimized Deep Learning Neural Networks},
author = {Mingqiang Huang and Yucen Liu and Changhai Man and Kai Li and Quan Cheng and Wei Mao and Hao Yu},
url = {https://ieeexplore.ieee.org/abstract/document/9793397},
doi = {10.1109/TCSI.2022.3178474},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Circuits and Systems I: Regular Papers},
pages = {1-13},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1996.
Xue, Xizhe; Zhang, Haokui; Fang, Bei; Bai, Zongwen; Li, Ying
Grafting Transformer on Automatically Designed Convolutional Neural Network for Hyperspectral Image Classification Journal Article
In: IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1-16, 2022.
@article{9791305,
title = {Grafting Transformer on Automatically Designed Convolutional Neural Network for Hyperspectral Image Classification},
author = {Xizhe Xue and Haokui Zhang and Bei Fang and Zongwen Bai and Ying Li},
url = {https://ieeexplore.ieee.org/abstract/document/9791305},
doi = {10.1109/TGRS.2022.3180685},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {IEEE Transactions on Geoscience and Remote Sensing},
volume = {60},
pages = {1-16},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
1995.
Aradhya, Abhay M. S.; Ashfahani, Andri; Angelina, Fienny; Pratama, Mahardhika; Mello, Rodrigo Fernandes; Sundaram, Suresh
Autonomous CNN (AutoCNN): A data-driven approach to network architecture determination Journal Article
In: Information Sciences, vol. 607, pp. 638-653, 2022, ISSN: 0020-0255.
@article{ARADHYA2022638,
title = {Autonomous CNN (AutoCNN): A data-driven approach to network architecture determination},
author = {Abhay M. S. Aradhya and Andri Ashfahani and Fienny Angelina and Mahardhika Pratama and Rodrigo Fernandes Mello and Suresh Sundaram},
url = {https://www.sciencedirect.com/science/article/pii/S0020025522005370},
doi = {https://doi.org/10.1016/j.ins.2022.05.100},
issn = {0020-0255},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Information Sciences},
volume = {607},
pages = {638-653},
abstract = {Designing a Convolutional Neural Networks (CNN) is a complex task and requires expert knowledge to optimize the performance and network architecture. In this paper, a novel data-driven approach is proposed to determine the architecture of CNN models. The proposed Autonomous Convolutional Neural Networks (AutoCNNThe executable code and original numerical results can be downloaded from (https://tinyurl.com/AutoCNN)) algorithm introduces data driven strategies for addition of new convolutional layers, pruning of redundant filters and training cycle optimization. AutoCNN is evaluated using MNIST, MNIST-rot-back-image, Fashion MNIST and the ADHD200 datasets to measure the performance on small datasets with varied feature distributions. The results indicate that AutoCNN optimizes the CNN network architecture and helps maximise the classification performance. The data-driven network determination approach introduced in this paper was found to not only provides competitive performance similar to existing evolutionary computation based network determination algorithms in literature, but was found to be an effective optimization tool to improve the performance of existing CNN architectures. Further, the AutoCNN was found to highly immune to noise in the dataset and has proven to be effective method to transfer knowledge between related datasets. Therefore, the AutoCNN is a highly versatile CNN architecture determination tool that has a wide range of applications in the field of autonomous driving, medical image analysis, image enhancement, camera based security monitoring and image based fault detection.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Designing a Convolutional Neural Networks (CNN) is a complex task and requires expert knowledge to optimize the performance and network architecture. In this paper, a novel data-driven approach is proposed to determine the architecture of CNN models. The proposed Autonomous Convolutional Neural Networks (AutoCNNThe executable code and original numerical results can be downloaded from (https://tinyurl.com/AutoCNN)) algorithm introduces data driven strategies for addition of new convolutional layers, pruning of redundant filters and training cycle optimization. AutoCNN is evaluated using MNIST, MNIST-rot-back-image, Fashion MNIST and the ADHD200 datasets to measure the performance on small datasets with varied feature distributions. The results indicate that AutoCNN optimizes the CNN network architecture and helps maximise the classification performance. The data-driven network determination approach introduced in this paper was found to not only provides competitive performance similar to existing evolutionary computation based network determination algorithms in literature, but was found to be an effective optimization tool to improve the performance of existing CNN architectures. Further, the AutoCNN was found to highly immune to noise in the dataset and has proven to be effective method to transfer knowledge between related datasets. Therefore, the AutoCNN is a highly versatile CNN architecture determination tool that has a wide range of applications in the field of autonomous driving, medical image analysis, image enhancement, camera based security monitoring and image based fault detection.
1994.
Shahawy, Mohamed; Benkhelifa, Elhadj; White, David
A Review on Plastic Artificial Neural Networks: Exploring the Intersection between Neural Architecture Search and Continual Learning Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2206-05625,
title = {A Review on Plastic Artificial Neural Networks: Exploring the Intersection between Neural Architecture Search and Continual Learning},
author = {Mohamed Shahawy and Elhadj Benkhelifa and David White},
url = {https://doi.org/10.48550/arXiv.2206.05625},
doi = {10.48550/arXiv.2206.05625},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2206.05625},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1993.
Franken, G. G. H.; Singh, Prabhant; Vanschoren, Joaquin
EmProx: Neural Network Performance Estimation For Neural Architecture Search Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2206-05972,
title = {EmProx: Neural Network Performance Estimation For Neural Architecture Search},
author = {G. G. H. Franken and Prabhant Singh and Joaquin Vanschoren},
url = {https://doi.org/10.48550/arXiv.2206.05972},
doi = {10.48550/arXiv.2206.05972},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2206.05972},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1992.
Lu, Qing; Xu, Xiaowei; Dong, Shunjie; Hao, Cong; Yang, Lei; Zhuo, Cheng; Shi, Yiyu
RT-DNAS: Real-time Constrained Differentiable Neural Architecture Search for 3D Cardiac Cine MRI Segmentation Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2206-04682,
title = {RT-DNAS: Real-time Constrained Differentiable Neural Architecture Search for 3D Cardiac Cine MRI Segmentation},
author = {Qing Lu and Xiaowei Xu and Shunjie Dong and Cong Hao and Lei Yang and Cheng Zhuo and Yiyu Shi},
url = {https://doi.org/10.48550/arXiv.2206.04682},
doi = {10.48550/arXiv.2206.04682},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2206.04682},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1991.
Liu, Jiawei; Zhang, Kaiyu; Hu, Weitai; Yang, Qing
Improve Ranking Correlation of Super-net through Training Scheme from One-shot NAS to Few-shot NAS Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2206-05896,
title = {Improve Ranking Correlation of Super-net through Training Scheme from One-shot NAS to Few-shot NAS},
author = {Jiawei Liu and Kaiyu Zhang and Weitai Hu and Qing Yang},
url = {https://doi.org/10.48550/arXiv.2206.05896},
doi = {10.48550/arXiv.2206.05896},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2206.05896},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1990.
Eslahi, Samira Vafay; Tao, Jian; Ji, Jim
ERNAS: An Evolutionary Neural Architecture Search for Magnetic Resonance Image Reconstructions Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2206-07280,
title = {ERNAS: An Evolutionary Neural Architecture Search for Magnetic Resonance Image Reconstructions},
author = {Samira Vafay Eslahi and Jian Tao and Jim Ji},
url = {https://doi.org/10.48550/arXiv.2206.07280},
doi = {10.48550/arXiv.2206.07280},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2206.07280},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1989.
Yang, Yingguang; Yang, Renyu; Li, Yangyang; Cui, Kai; Yang, Zhiqin; Wang, Yue; Xu, Jie; Xie, Haiyong
RoSGAS: Adaptive Social Bot Detection with Reinforced Self-Supervised GNN Architecture Search Technical Report
2022.
@techreport{DBLP:journals/corr/abs-2206-06757,
title = {RoSGAS: Adaptive Social Bot Detection with Reinforced Self-Supervised GNN Architecture Search},
author = {Yingguang Yang and Renyu Yang and Yangyang Li and Kai Cui and Zhiqin Yang and Yue Wang and Jie Xu and Haiyong Xie},
url = {https://doi.org/10.48550/arXiv.2206.06757},
doi = {10.48550/arXiv.2206.06757},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2206.06757},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
1988.
Ren, Xuhong; Chen, Jianlang; Juefei-Xu, Felix; Xue, Wanli; Guo, Qing; Ma, Lei; Zhao, Jianjun; Chen, Shengyong
DARTSRepair: Core-failure-set guided DARTS for network robustness to common corruptions Journal Article
In: Pattern Recognition, vol. 131, pp. 108864, 2022, ISSN: 0031-3203.
@article{REN2022108864,
title = {DARTSRepair: Core-failure-set guided DARTS for network robustness to common corruptions},
author = {Xuhong Ren and Jianlang Chen and Felix Juefei-Xu and Wanli Xue and Qing Guo and Lei Ma and Jianjun Zhao and Shengyong Chen},
url = {https://www.sciencedirect.com/science/article/pii/S0031320322003454},
doi = {https://doi.org/10.1016/j.patcog.2022.108864},
issn = {0031-3203},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Pattern Recognition},
volume = {131},
pages = {108864},
abstract = {Network architecture search (NAS), in particular the differentiable architecture search (DARTS) method, has shown a great power to learn excellent model architectures on the specific dataset of interest. In contrast to using a fixed dataset, in this work, we focus on a different but important scenario for NAS: how to refine a deployed network’s model architecture to enhance its robustness with the guidance of a few collected and misclassified examples that are degraded by some real-world unknown corruptions having a specific pattern (e.g., noise, blur, etc..). To this end, we first conduct an empirical study to validate that the model architectures can be definitely related to the corruption patterns. Surprisingly, by just adding a few corrupted and misclassified examples (e.g., 103 examples) to the clean training dataset (e.g., 5.0×104 examples), we can refine the model architecture and enhance the robustness significantly. To make it more practical, the key problem, i.e., how to select the proper failure examples for the effective NAS guidance, should be carefully investigated. Then, we propose a novel core-failure-set guided DARTS that embeds a K-center-greedy algorithm for DARTS to select suitable corrupted failure examples to refine the model architecture. We use our method for DARTS-refined DNNs on the clean as well as 15 corruptions with the guidance of four specific real-world corruptions. Compared with the state-of-the-art NAS as well as data-augmentation-based enhancement methods, our final method can achieve higher accuracy on both corrupted datasets and the original clean dataset. On some of the corruption patterns, we can achieve as high as over 45% absolute accuracy improvements.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Network architecture search (NAS), in particular the differentiable architecture search (DARTS) method, has shown a great power to learn excellent model architectures on the specific dataset of interest. In contrast to using a fixed dataset, in this work, we focus on a different but important scenario for NAS: how to refine a deployed network’s model architecture to enhance its robustness with the guidance of a few collected and misclassified examples that are degraded by some real-world unknown corruptions having a specific pattern (e.g., noise, blur, etc..). To this end, we first conduct an empirical study to validate that the model architectures can be definitely related to the corruption patterns. Surprisingly, by just adding a few corrupted and misclassified examples (e.g., 103 examples) to the clean training dataset (e.g., 5.0×104 examples), we can refine the model architecture and enhance the robustness significantly. To make it more practical, the key problem, i.e., how to select the proper failure examples for the effective NAS guidance, should be carefully investigated. Then, we propose a novel core-failure-set guided DARTS that embeds a K-center-greedy algorithm for DARTS to select suitable corrupted failure examples to refine the model architecture. We use our method for DARTS-refined DNNs on the clean as well as 15 corruptions with the guidance of four specific real-world corruptions. Compared with the state-of-the-art NAS as well as data-augmentation-based enhancement methods, our final method can achieve higher accuracy on both corrupted datasets and the original clean dataset. On some of the corruption patterns, we can achieve as high as over 45% absolute accuracy improvements.
1987.
Dervisi, Foteini; Kyriakides, George; Margaritis, Konstantinos
Evaluating Acceleration Techniques for Genetic Neural Architecture Search Proceedings Article
In: Iliadis, Lazaros; Jayne, Chrisina; Tefas, Anastasios; Pimenidis, Elias (Ed.): Engineering Applications of Neural Networks, pp. 3–14, Springer International Publishing, Cham, 2022, ISBN: 978-3-031-08223-8.
@inproceedings{10.1007/978-3-031-08223-8_1,
title = {Evaluating Acceleration Techniques for Genetic Neural Architecture Search},
author = {Foteini Dervisi and George Kyriakides and Konstantinos Margaritis},
editor = {Lazaros Iliadis and Chrisina Jayne and Anastasios Tefas and Elias Pimenidis},
isbn = {978-3-031-08223-8},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Engineering Applications of Neural Networks},
pages = {3--14},
publisher = {Springer International Publishing},
address = {Cham},
abstract = {The increase in the available data and computational power has led to the rapid evolution of the field of deep learning over the last few years. However, the success of deep learning methods relies on making appropriate neural architecture choices, which is not a straightforward task and usually requires a time-consuming trial-and-error procedure. Neural architecture search is the process of automating the design of neural network architectures capable of performing well on specific tasks. It is a field that has emerged in order to address the problem of designing efficient neural architectures and is gaining popularity due to the rapid evolution of deep learning, which has led to an increasing need for the discovery of high-performing neural architectures. This paper focuses on evolutionary neural architecture search, which is an efficient but also time-consuming and computationally expensive neural architecture search approach, and aims to pave the way for speeding up such algorithms by assessing the effect of acceleration methods on the overall performance of the neural architecture search procedure as well as on the produced architectures.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
The increase in the available data and computational power has led to the rapid evolution of the field of deep learning over the last few years. However, the success of deep learning methods relies on making appropriate neural architecture choices, which is not a straightforward task and usually requires a time-consuming trial-and-error procedure. Neural architecture search is the process of automating the design of neural network architectures capable of performing well on specific tasks. It is a field that has emerged in order to address the problem of designing efficient neural architectures and is gaining popularity due to the rapid evolution of deep learning, which has led to an increasing need for the discovery of high-performing neural architectures. This paper focuses on evolutionary neural architecture search, which is an efficient but also time-consuming and computationally expensive neural architecture search approach, and aims to pave the way for speeding up such algorithms by assessing the effect of acceleration methods on the overall performance of the neural architecture search procedure as well as on the produced architectures.