AutoML.org

Freiburg-Hannover-Tübingen

Literature on Neural Architecture Search

Maintained by Difan Deng and Marius Lindauer.

The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.

Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.

Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.

2755 entries « ‹ 1 of 56 › »

2023

2755.

Zhao, Junbo; Ning, Xuefei; Liu, Enshu; Ru, Binxin; Zhou, Zixuan; Zhao, Tianchen; Chen, Chen; Zhang, Jiajin; Liao, Qingmin; Wang, Yu

Dynamic Ensemble of Low-fidelity Experts: Mitigating NAS “Cold-Start” Technical Report

2023.

2754.

Zhang, Baochang; Xu, Sheng; Lin, Mingbao; Wang, Tiancheng; Doermann, David

Binary Neural Networks: Algorithms, Architectures, and Applications Book

2023.

2753.

Lukasik, Jovita

Topology Learning for Prediction, Generation, and Robustness in Neural Architecture Search PhD Thesis

2023.

2752.

ZiWen, Dou; YuQi, Li; Dong, Ye

FasterMDE: A real-time monocular depth estimation search method that balances accuracy and speed on the edge Journal Article

In: Applied Intelligence, 2023.

2751.

Hosseini, Ramtin; Zhang, Li; Garg, Bhanu; Xie, Pengtao

Fair and Accurate Decision Making through Group-Aware Learning Conference

ICML 2023, 2023.

2750.

Wang, Wei; Wang, Xianpeng; Song, Xiangman

Maximal sparse convex surrogate-assisted evolutionary convolutional neural architecture search for image segmentation Journal Article

In: Complex & Intelligent Systems , 2023.

2749.

Yang, Yi; Wei, Jiaxuan; Yu, Zhixuan; Zhang, Ruisheng

A trustworthy neural architecture search framework for pneumonia image classification utilizing blockchain technology Journal Article

In: The Journal of Supercomputing, 2023.

2748.

Liu, Ruihua; Nan, Haoyu; Zou, Yangyang; Xie, Ting

AS-3DFCN: Automatically Seeking 3DFCN-Based Brain Tumor Segmentation Conference

Cognitive Computation, 2023.

2747.

Thomson, Sarah L; Ochoa, Gabriela; Veerapen, Nadarajen; Michalak, Krzysztof

Channel Configuration for Neural Architecture: Insights from the Search Space Proceedings Article

In: The Genetic and Evolutionary Computation Conference (GECCO) 2023, 2023.

2746.

Zhang, Anqi; Zhao, Shengmei

Evolutionary-based searching method for quantum circuit architecture Journal Article

In: Quantum Information Processing , 2023.

2745.

Ren, Yaodanjun; Chen, Chen; Qi, Zhengwei

HaeNAS: Hardware-Aware Efficient Neural Architecture Search via Zero-Cost Proxy Miscellaneous

2023.

2744.

(Ed.)

SpeedLimit: Neural Architecture Search for Quantized Transformer Models Collection

2023.

2743.

(Ed.)

QuantumDARTS: Differentiable Quantum Architecture Search for Variational Quantum Algorithms Collection

2023.

2742.

(Ed.)

Data Driven Designing of Convolutional Neural Networks Architectures for Image Classification. Collection

2023.

2741.

Wang, Runqi; Yang, Linlin; Chen, Hanlin; Wang, Wei; Doermann, David; Zhang, Baochang

Anti-Bandit for Neural Architecture Search Journal Article

In: International Journal of Computer Vision , 2023.

2740.

Feng, Yen-Wei; Jiang, Bing-Ru; Lin, Albert

NeuroEvolution-based network architecture evolution in semicon- ductor manufacturing Technical Report

2023.

2739.

(Ed.)

Rethinking Dilated Convolution for Real-time Semantic Segmentation Collection

2023.

2738.

Wang, Yu; Drgoňa, Ján; Zhang, Jiaxin; Suryanarayana, Karthik Somayaji Nanjangud; Schram, Malachi; Liu, Frank; Li, Peng

AutoNF: Automated Architecture Optimization of Normalizing Flows with Unconstrained Continuous Relaxation Admitting Optimal Discrete Solution Journal Article

In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 8, pp. 10244-10252, 2023.

2737.

Lu, Shun; Hu, Yu; Wang, Peihao; Han, Yan; Tan, Jianchao; Li, Jixiang; Yang, Sen; Liu, Ji

PINAT: A Permutation INvariance Augmented Transformer for NAS Predictor Journal Article

In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 7, pp. 8957-8965, 2023.

2736.

Ma, Dongning; Zhao, Pengfei; Jiao, Xun

PerfHD: Efficient ViT Architecture Performance Ranking Using Hyperdimensional Computing Proceedings Article

In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 2229-2236, 2023.

2735.

Hendrickx, Lotte; Symons, Arne; Ranst, Wiebe Van; Verhelst, Marian; Goedemé, Toon

Hardware-Aware NAS by Genetic Optimisation With a Design Space Exploration Simulator Proceedings Article

In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 2274-2282, 2023.

2734.

-, Kwang

On Impact of Network Architecture for Deep Learning PhD Thesis

2023.

2733.

Pijackova, Kristyna; Nejedly, Petr; Kremen, Vaclav; Plesinger, Filip; Mivalt, Filip; Lepkova, Kamila; Pail, Martin; Jurak, Pavel; Worrell, Gregory; Brazdil, Milan; Klimes, Petr

Genetic algorithm designed for optimization of neural network architectures for intracranial EEG recordings analysis Journal Article

In: Journal of Neural Engineering, vol. 20, no. 3, pp. 036034, 2023.

Abstract | Links | BibTeX

@article{Pijackova_2023,

title = {Genetic algorithm designed for optimization of neural network architectures for intracranial EEG recordings analysis},

author = {Kristyna Pijackova and Petr Nejedly and Vaclav Kremen and Filip Plesinger and Filip Mivalt and Kamila Lepkova and Martin Pail and Pavel Jurak and Gregory Worrell and Milan Brazdil and Petr Klimes},

url = {https://dx.doi.org/10.1088/1741-2552/acdc54},

doi = {10.1088/1741-2552/acdc54},

year  = {2023},

date = {2023-06-01},

urldate = {2023-06-01},

journal = {Journal of Neural Engineering},

volume = {20},

number = {3},

pages = {036034},

publisher = {IOP Publishing},

abstract = {Objective. The current practices of designing neural networks rely heavily on subjective judgment and heuristic steps, often dictated by the level of expertise possessed by architecture designers. To alleviate these challenges and streamline the design process, we propose an automatic method, a novel approach to enhance the optimization of neural network architectures for processing intracranial electroencephalogram (iEEG) data. Approach. We present a genetic algorithm, which optimizes neural network architecture and signal pre-processing parameters for iEEG classification. Main results. Our method improved the macro F1 score of the state-of-the-art model in two independent datasets, from St. Anne’s University Hospital (Brno, Czech Republic) and Mayo Clinic (Rochester, MN, USA), from 0.9076 to 0.9673 and from 0.9222 to 0.9400 respectively. Significance. By incorporating principles of evolutionary optimization, our approach reduces the reliance on human intuition and empirical guesswork in architecture design, thus promoting more efficient and effective neural network models. The proposed method achieved significantly improved results when compared to the state-of-the-art benchmark model (McNemar’s test, p ≪ 0.01). The results indicate that neural network architectures designed through machine-based optimization outperform those crafted using the subjective heuristic approach of a human expert. Furthermore, we show that well-designed data preprocessing significantly affects the models’ performance.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

2732.

Theodorakopoulos, Daphne; Manss, Christoph; Stahl, Frederic; Lindauer, Marius

GREEN-AUTOML FOR PLASTIC LITTER DETECTION Conference

ICLR 2023 workshop, Tackling Climate Change with Machine Learning, 2023.

2731.

P, Pushpaveni H; Yadav, Anil Kumar; Yadav, Santosh Kumar; Adhikari, Sulav Narayan

Detection of Skin Cancer using Neural Architecture Search with Model Quantization Journal Article

In: International Journal of Advances in Engineering Architecture Science and Technology, vol. 1, iss. 2, 2023.

2730.

Deng, Shaojiang; Fan, Chao; Li, Yantao; Qin, Huafeng; El-Yacoubi, Mounim A.; Zhou, Gang

GRU-based Neural Architecture Search for Finger-Vein Identification Technical Report

2023.

2729.

(Ed.)

AstroNet: When Astrocyte Meets Artificial Neural Network Collection

2023.

2728.

(Ed.)

DisWOT: Student Architecture Search for Distillation WithOut Training Collection

2023.

2727.

(Ed.)

MDL-NAS: A Joint Multi-domain Learning Framework for Vision Transformer Collection

2023.

2726.

Thuan, Nguyen Duc; Dong, Trinh Phuong; Nguyen, Hue Thi; Hoang, Hong Si

Efficient bearing fault diagnosis with neural network search and parameter quantization based on vibration and temperature Journal Article

In: Engineering Research Express, vol. 5, no. 2, 2023.

2725.

Huang, Junhao; Xue, Bing; Sun, Yanan; Zhang, Mengjie; Yen, Gary G

Split-Level Evolutionary Neural Architecture Search With Elite Weight Inheritance. Journal Article

In: IEEE transactions on neural networks and learning systems, 2023.

2724.

Li, Yifan; Liu, Jing

A survey: evolutionary deep learning Journal Article

In: Soft Computing , 2023.

2723.

Gao, Jianliang; Al-Sabri, Raeed; Oloulade, Babatounde Moctard; Chen, Jiamin; Lyu, Tengfei; Wu, Zhenpeng

GM2NAS: multitask multiview graph neural architecture search Journal Article

In: Knowledge and Information Systems , 2023.

2722.

Loni, Mohammad; Mohan, Aditya; Asadi, Mehdi; Lindauer, Marius

Learning Activation Functions for Sparse Neural Networks Technical Report

2023.

2721.

(Ed.)

Searching sharing relationship for instance segmentation decoder Collection

2023.

2720.

Yang, Tiejun; He, Qing; Huang, Lin

OM-NAS: pigmented skin lesion image classification based on a neural architecture search Journal Article

In: Biomed. Opt. Express, vol. 14, no. 5, pp. 2153–2165, 2023.

Abstract | Links | BibTeX

2719.

Ouertatani, Houssem; Maxim, Cristian; Niar, Smail; Talbi, El-Ghazali

Bayesian optimization for NAS with pretrained deep ensembles Proceedings Article

In: International Conference in Optimization and Learning (OLA),, 2023.

2718.

Yao, Dixi

Solving Heterogeneity via Personalized Federated Learning Technical Report

2023.

2717.

Li, Yantao; Luo, Jiaxing; Deng, Shaojiang; Zhou, Gang

SearchAuth: Neural Architecture Search Based Continuous Authentication Using Auto Augmentation Search Journal Article

In: ACM Trans. Sen. Netw., 2023, ISSN: 1550-4859, (Just Accepted).

Abstract | Links | BibTeX

@article{10.1145/3599727,

title = {SearchAuth: Neural Architecture Search Based Continuous Authentication Using Auto Augmentation Search},

author = {Yantao Li and Jiaxing Luo and Shaojiang Deng and Gang Zhou},

url = {https://doi.org/10.1145/3599727},

doi = {10.1145/3599727},

issn = {1550-4859},

year  = {2023},

date = {2023-05-01},

urldate = {2023-05-01},

journal = {ACM Trans. Sen. Netw.},

publisher = {Association for Computing Machinery},

address = {New York, NY, USA},

abstract = {Mobile devices have been playing significant roles in our daily lives, which has made device security and privacy protection extremely important. These mobile devices storing user sensitive and private information, therefore, need rigorous user authentication mechanisms. In this paper, we present SearchAuth, a novel continuous authentication system on smartphones exploiting a neural architecture search (NAS) to find an optimal network architecture and an auto augmentation search (AAS) to more effectively train the optimal network along with the best data augmentation policies, by leveraging the accelerometer, gyroscope and magnetometer on smartphones to capture users’ behavioral patterns. Specifically, SearchAuth consists of three stages, i.e. the offline stage, registration stage, and authentication stage. In the offline stage, we utilize the NAS on sensor data of the accelerometer, gyroscope and magnetometer to find an optimal network architecture based on the designed search space. With the optimal network architecture, namely NAS-based model, the AAS automatically optimizes the augmentation of the input data for more effectively training the model that is for feature extraction. In the registration stage, we use the trained NAS-based model to learn and extract deep features from the legitimate user’s data, and train the LOF classifier with 55 features selected by the PCA. In the authentication stage, with the well-trained NAS-based model and LOF classifier, SearchAuth identifies the current user as a legitimate user or an impostor when the user starts operating a smartphone. Based on our dataset, we evaluate the performance of the proposed SearchAuth, and the experimental results demonstrate that SearchAuth surpasses the representative authentication schemes by achieving the best accuracy of 93.95%, F1-score of 94.30%, and EER of 5.30% on the LOF classifier with dataset size of 100.},

note = {Just Accepted},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

2716.

Zhang, Xindong

Towards lightweight and efficient network design for image super-resolution PhD Thesis

2023.

2715.

Zhang, Zhenman; Xue, Yu; Slowik, Adam; Yuan, Ziming

Sle-CNN: a novel convolutional neural network for sleep stage classification Proceedings Article

In: 2023.

2714.

Tragoudaras, Antonios; Antoniadis, Charalampos; Massoud, Yehia

TinyML for EEG Decoding on Microcontrollers Technical Report

2023.

2713.

Chatzimichailidis, Avraam

ALTERNATIVE OPTIMIZATION METHODS FOR TRAINING OF LARGE DEEP NEURAL NETWORKS PhD Thesis

2023.

2712.

Mohan, Rohit; Elsken, Thomas; Zela, Arber; Metze, Jan Hendrik; Staffler, Benedikt; Brox, Thomas; Valada, Abhinav; Hutter, Frank

Neural Architecture Search for Dense Prediction Tasks in Computer Vision Journal Article

In: International Journal of Computer Vision (2023), 2023.

2711.

Wang, Zhixian; Wang, Guoqing; Yang, Yang

NAS-StegNet: Lightweight Image Steganography Networks via Neural Architecture Search Proceedings Article

In: International Conference on Neural Information Processing 22, 2023.

2710.

Shin, Jie Yong; Kang, Dae-Ki

A Higher Performing DARTS Model for CIFAR-10 Proceedings Article

In: -, Hsuan (Ed.): IHCI 2022: Intelligent Human Computer Interaction , 2023.

2709.

Roberts, Nicholas; Liang, Yingyu; Sala, Frederic

UNDERSTANDING NEURAL ARCHITECTURE SEARCH BY ITS ARCHITECTURE PARAMETERS Technical Report

2023.

2708.

Ruan, Diwang; Han, Jinzhao; Yan, Jianping; Gühmann, Clemens

Light convolutional neural network by neural architecture search and model pruning for bearing fault diagnosis and remaining useful life prediction Journal Article

In: 2023.

2707.

Jin, Guangyin; Yan, Huan; Li, Fuxian; Li, Yong; Huang, Jincai

Dual Graph Convolution Architecture Search for Travel Time Estimation Journal Article

In: ACM Trans. Intell. Syst. Technol., 2023, ISSN: 2157-6904, (Just Accepted).

Abstract | Links | BibTeX

@article{10.1145/3591361,

title = {Dual Graph Convolution Architecture Search for Travel Time Estimation},

author = {Guangyin Jin and Huan Yan and Fuxian Li and Yong Li and Jincai Huang},

url = {https://doi.org/10.1145/3591361},

doi = {10.1145/3591361},

issn = {2157-6904},

year  = {2023},

date = {2023-04-01},

urldate = {2023-04-01},

journal = {ACM Trans. Intell. Syst. Technol.},

publisher = {Association for Computing Machinery},

address = {New York, NY, USA},

abstract = {Travel time estimation (TTE) is a crucial task in intelligent transportation systems, which has been widely used in navigation and route planning. In recent years, several deep learning frameworks have been proposed to capture the dynamic features of road segments or intersections for travel time estimation. However, most existing works do not consider the joint features of the intersections and road segments. Moreover, most deep neural networks for TTE are designed based on empirical knowledge. Since the independent and joint features of intersections and road segments commonly vary with different datasets, the empirical deterministic neural architectures have limited adaptability to different scenarios. To tackle the above problems, we propose a novel automated deep learning framework, namely Automated Spatio-Temporal Dual Graph Convolutional Networks (Auto-STDGCN), for travel time estimation. Specifically, we propose to construct the node-wise graph and edge-wise graph to characterize the spatio-temporal features of intersections and road segments respectively. In order to capture the joint spatio-temporal correlations of the dual graphs, a hierarchical neural architecture search approach is introduced, whose search space is composed of internal and external search space. In the internal search space, spatial graph convolution and temporal convolution operations are adopted to capture the respective spatio-temporal correlations of the dual graphs. Further, we design the external search space including the node-wise and edge-wise graph convolution operations from the internal architecture search to capture the interaction patterns between the intersections and road segments. We evaluate our proposed model Auto-STDGCN on three real-world datasets, which demonstrates that our model is significantly superior to the state-of-art methods. In addition, we also conduct case studies to visualize and explain the neural architectures learned by our model.},

note = {Just Accepted},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

2706.

Sun, Jialiang; Zheng, Xiaohu; Yao, Wen; Zhang, Xiaoya; Zhou, Weien; Chen, Xiaoqian

Heat source layout optimization using automatic deep learning surrogate and multimodal neighborhood search algorithm Journal Article

In: Annals of Operations Research , 2023.

2755 entries « ‹ 1 of 56 › »