Maintained by Difan Deng and Marius Lindauer.
The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.
Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.
Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.
0000
38.
Wenfeng Feng and, Xin Zhang; Song, Qiushuang; Sun, Guoying
Incoherence of Deep Isotropic Neural Networks increase their performance on Image Classification Technical Manual
0000.
@manual{nokey,
title = {Incoherence of Deep Isotropic Neural Networks increase their performance on Image Classification},
author = { Wenfeng Feng and, Xin Zhang and Qiushuang Song and Guoying Sun
},
url = {https://www.preprints.org/manuscript/202210.0092/v1},
keywords = {},
pubstate = {published},
tppubtype = {manual}
}
37.
Malkova, Aleksandra; Amini, Massih-Reza; Denis, Benoit; Villien, Christophe
Radio Map Reconstruction with Deep Neural Networks in a Weakly Labeled Learning Context with use of Heterogeneous Side Information Technical Manual
0000.
@manual{Malkova2022,
title = {Radio Map Reconstruction with Deep Neural Networks in a Weakly Labeled Learning Context with use of Heterogeneous Side Information},
author = {Aleksandra Malkova and Massih-Reza Amini and Benoit Denis and Christophe Villien},
url = {https://hal.archives-ouvertes.fr/hal-03823629/document},
keywords = {},
pubstate = {published},
tppubtype = {manual}
}
36.
Aboalam, Kawther; Neuswirth, Christoph; Pernau, Florian; Schiebel, Stefan; Spaethe, Fabian; Strohrmann, Manfred
Image Processing and Neural Network Optimization Methods for Automatic Visual Inspection Technical Manual
0000.
@manual{Aboalam2022,
title = {Image Processing and Neural Network Optimization Methods for Automatic Visual Inspection},
author = {Kawther Aboalam and Christoph Neuswirth and Florian Pernau and Stefan Schiebel and Fabian Spaethe and Manfred Strohrmann},
url = {https://www.researchgate.net/profile/Christoph-Reich/publication/364343172_Artificial_Intelligence_--_Applications_in_Medicine_and_Manufacturing_--_The_Upper_Rhine_Artificial_Intelligence_Symposium_UR-AI_2022/links/634cfa3476e39959d6c8bfb2/Artificial-Intelligence--Applications-in-Medicine-and-Manufacturing--The-Upper-Rhine-Artificial-Intelligence-Symposium-UR-AI-2022.pdf#page=33},
keywords = {},
pubstate = {published},
tppubtype = {manual}
}
35.
Mishra, Vidyanand; Kane, Lalit
A survey of designing convolutional neural network using evolutionary algorithms Journal Article
In: Artificial Intelligence Review, 0000.
@article{Mishra-AIR2022,
title = {A survey of designing convolutional neural network using evolutionary algorithms},
author = {Vidyanand Mishra and Lalit Kane
},
url = {https://link.springer.com/article/10.1007/s10462-022-10303-4},
journal = { Artificial Intelligence Review},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
34.
Sun, Yanan; Yen, Gary G.; Zhang, Mengjie
Evolutionary Deep Neural Architecture Search: Fundamentals, Methods, and recent AdvanceYa Book
0000.
@book{SunEDA22,
title = {Evolutionary Deep Neural Architecture Search: Fundamentals, Methods, and recent AdvanceYa},
author = {Yanan Sun and Gary G. Yen and Mengjie Zhang},
url = {https://books.google.de/books?hl=de&lr=&id=2RWbEAAAQBAJ&oi=fnd&pg=PR5&dq=%22neural+architecture+search%22&ots=yjnrR-vqyW&sig=0KFGVSnhQWTc1sQmWWewvmeuGqw#v=onepage&q=%22neural%20architecture%20search%22&f=false},
keywords = {},
pubstate = {published},
tppubtype = {book}
}
33.
Park, Minje
Proxy Data Generation for Fast and Efficient Neural Architecture Search Journal Article
In: Journal of Electrical Engineering & Technology 2022, 0000.
@article{ParkJEET22,
title = {Proxy Data Generation for Fast and Efficient Neural Architecture Search},
author = {
Minje Park
},
url = {https://link.springer.com/article/10.1007/s42835-022-01321-x},
journal = {Journal of Electrical Engineering & Technology 2022},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
32.
Singh, Yeshwant; Biswas, Anupam
In: Expert Systems, vol. n/a, no. n/a, pp. e13241, 0000.
@article{https://doi.org/10.1111/exsy.13241,
title = {Lightweight convolutional neural network architecture design for music genre classification using evolutionary stochastic hyperparameter selection},
author = {Yeshwant Singh and Anupam Biswas},
url = {https://onlinelibrary.wiley.com/doi/abs/10.1111/exsy.13241},
doi = {https://doi.org/10.1111/exsy.13241},
journal = {Expert Systems},
volume = {n/a},
number = {n/a},
pages = {e13241},
abstract = {Abstract Convolutional neural networks (CNNs) have succeeded in various domains, including music information retrieval (MIR). Music genre classification (MGC) is one such task in the MIR that has gained attention over the years because of the massive increase in online music content. Accurate indexing and automatic classification of these large volumes of music content require high computational resources, which pose a significant challenge to building a lightweight system. CNNs are a popular deep learning-based choice for building systems for MGC. However, finding an optimal CNN architecture for MGC requires domain knowledge both in CNN architecture design and music. We present MGA-CNN, a genetic algorithm-based approach with a novel stochastic hyperparameter selection for finding an optimal lightweight CNN-based architecture for the MGC task. The proposed approach is unique in automating the CNN architecture design for the MGC task. MGA-CNN is evaluated on three widely used music datasets and compared with seven peer rivals, which include three automatic CNN architecture design approaches and four manually designed popular CNN architectures. The experimental results show that MGA-CNN surpasses the peer approaches in terms of classification accuracy, parameter numbers, and execution time. The optimal architectures generated by MGA-CNN also achieve classification accuracy comparable to the manually designed CNN architectures while spending fewer computing resources.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Abstract Convolutional neural networks (CNNs) have succeeded in various domains, including music information retrieval (MIR). Music genre classification (MGC) is one such task in the MIR that has gained attention over the years because of the massive increase in online music content. Accurate indexing and automatic classification of these large volumes of music content require high computational resources, which pose a significant challenge to building a lightweight system. CNNs are a popular deep learning-based choice for building systems for MGC. However, finding an optimal CNN architecture for MGC requires domain knowledge both in CNN architecture design and music. We present MGA-CNN, a genetic algorithm-based approach with a novel stochastic hyperparameter selection for finding an optimal lightweight CNN-based architecture for the MGC task. The proposed approach is unique in automating the CNN architecture design for the MGC task. MGA-CNN is evaluated on three widely used music datasets and compared with seven peer rivals, which include three automatic CNN architecture design approaches and four manually designed popular CNN architectures. The experimental results show that MGA-CNN surpasses the peer approaches in terms of classification accuracy, parameter numbers, and execution time. The optimal architectures generated by MGA-CNN also achieve classification accuracy comparable to the manually designed CNN architectures while spending fewer computing resources.
31.
Gupta, Pritha; Drees, Jan Peter; Hüllermeier, Eyke
Automated Side-Channel Attacks using Black-Box Neural Architecture Search Technical Report
0000.
@techreport{Gupta22,
title = {Automated Side-Channel Attacks using Black-Box Neural Architecture Search},
author = {Pritha Gupta and Jan Peter Drees and Eyke Hüllermeier},
url = {https://eprint.iacr.org/2023/093.pdf},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
30.
Pandelea, Vlad; Ragusa, Edoardo; Gastaldo, Paolo; Cambria, Erik
SELECTING LANGUAGE MODELS FEATURES VIA SOFTWARE-HARDWARE CO-DESIGN Miscellaneous
0000.
@misc{Pandelea23,
title = {SELECTING LANGUAGE MODELS FEATURES VIA SOFTWARE-HARDWARE CO-DESIGN},
author = {Vlad Pandelea and Edoardo Ragusa and Paolo Gastaldo and Erik Cambria },
url = {https://w.sentic.net/selecting-language-models-features-via-software-hardware-co-design.pdf},
keywords = {},
pubstate = {published},
tppubtype = {misc}
}
29.
Huynh, Lam
FROM 3D SENSING TO DENSE PREDICTION PhD Thesis
0000.
@phdthesis{HuynhPhD23,
title = {FROM 3D SENSING TO DENSE PREDICTION},
author = {Lam Huynh},
url = {http://jultika.oulu.fi/files/isbn9789526235165.pdf},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
28.
Cho, Minsu
Deep Learning Model Design Algorithms for High-Performing Plaintext and Ciphertext Inference PhD Thesis
0000.
@phdthesis{ChoPHD23,
title = {Deep Learning Model Design Algorithms for High-Performing Plaintext and Ciphertext Inference},
author = {Minsu Cho},
url = {https://www.proquest.com/docview/2767241424?pq-origsite=gscholar&fromopenview=true},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
27.
Zhou, Dongzhan
Designing Deep Model and Training Paradigm for Object Perception PhD Thesis
0000.
@phdthesis{ZhouPhD2023,
title = {Designing Deep Model and Training Paradigm for Object Perception},
author = {Zhou, Dongzhan
},
url = {https://ses.library.usyd.edu.au/handle/2123/31055},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
26.
Shariatzadeh, Seyed Mahdi; Fathy, Mahmood; Berangi, Reza
Improving the accuracy and speed of fast template-matching algorithms by neural architecture search Journal Article
In: Expert Systems, vol. n/a, no. n/a, pp. e13358, 0000.
@article{https://doi.org/10.1111/exsy.13358,
title = {Improving the accuracy and speed of fast template-matching algorithms by neural architecture search},
author = {Seyed Mahdi Shariatzadeh and Mahmood Fathy and Reza Berangi},
url = {https://onlinelibrary.wiley.com/doi/abs/10.1111/exsy.13358},
doi = {https://doi.org/10.1111/exsy.13358},
journal = {Expert Systems},
volume = {n/a},
number = {n/a},
pages = {e13358},
abstract = {Abstract Neural architecture search can be used to find convolutional neural architectures that are precise and robust while enjoying enough speed for industrial image processing applications. In this paper, our goal is to achieve optimal convolutional neural networks (CNNs) for multiple-templates matching for applications such as licence plates detection (LPD). We perform an iterative local neural architecture search for the models with minimum validation error as well as low computational cost from our search space of about 32 billion models. We describe the findings of the experience and discuss the specifications of the final optimal architectures. About 20-times error reduction and 6-times computational complexity reduction is achieved over our engineered neural architecture after about 500 neural architecture evaluation (in about 10 h). The typical speed of our final model is comparable to classic template matching algorithms while performing more robust and multiple-template matching with different scales.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Abstract Neural architecture search can be used to find convolutional neural architectures that are precise and robust while enjoying enough speed for industrial image processing applications. In this paper, our goal is to achieve optimal convolutional neural networks (CNNs) for multiple-templates matching for applications such as licence plates detection (LPD). We perform an iterative local neural architecture search for the models with minimum validation error as well as low computational cost from our search space of about 32 billion models. We describe the findings of the experience and discuss the specifications of the final optimal architectures. About 20-times error reduction and 6-times computational complexity reduction is achieved over our engineered neural architecture after about 500 neural architecture evaluation (in about 10 h). The typical speed of our final model is comparable to classic template matching algorithms while performing more robust and multiple-template matching with different scales.
25.
Yang, Yongjia; Zhan, Jinyu; Jiang, Wei; Jiang, Yucheng; Yu, Antai
Neural architecture search for resource constrained hardware devices: A survey Journal Article
In: IET Cyber-Physical Systems: Theory & Applications, vol. n/a, no. n/a, 0000.
@article{https://doi.org/10.1049/cps2.12058,
title = {Neural architecture search for resource constrained hardware devices: A survey},
author = {Yongjia Yang and Jinyu Zhan and Wei Jiang and Yucheng Jiang and Antai Yu},
url = {https://ietresearch.onlinelibrary.wiley.com/doi/abs/10.1049/cps2.12058},
doi = {https://doi.org/10.1049/cps2.12058},
journal = {IET Cyber-Physical Systems: Theory & Applications},
volume = {n/a},
number = {n/a},
abstract = {Abstract With the emergence of powerful and low-energy Internet of Things devices, deep learning computing is increasingly applied to resource-constrained edge devices. However, the mismatch between hardware devices with low computing capacity and the increasing complexity of Deep Neural Network models, as well as the growing real-time requirements, bring challenges to the design and deployment of deep learning models. For example, autonomous driving technologies rely on real-time object detection of the environment, which cannot tolerate the extra latency of sending data to the cloud, processing and then sending the results back to edge devices. Many studies aim to find innovative ways to reduce the size of deep learning models, the number of Floating-point Operations per Second, and the time overhead of inference. Neural Architecture Search (NAS) makes it possible to automatically generate efficient neural network models. The authors summarise the existing NAS methods on resource-constrained devices and categorise them according to single-objective or multi-objective optimisation. We review the search space, the search algorithm and the constraints of NAS on hardware devices. We also explore the challenges and open problems of hardware NAS.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Abstract With the emergence of powerful and low-energy Internet of Things devices, deep learning computing is increasingly applied to resource-constrained edge devices. However, the mismatch between hardware devices with low computing capacity and the increasing complexity of Deep Neural Network models, as well as the growing real-time requirements, bring challenges to the design and deployment of deep learning models. For example, autonomous driving technologies rely on real-time object detection of the environment, which cannot tolerate the extra latency of sending data to the cloud, processing and then sending the results back to edge devices. Many studies aim to find innovative ways to reduce the size of deep learning models, the number of Floating-point Operations per Second, and the time overhead of inference. Neural Architecture Search (NAS) makes it possible to automatically generate efficient neural network models. The authors summarise the existing NAS methods on resource-constrained devices and categorise them according to single-objective or multi-objective optimisation. We review the search space, the search algorithm and the constraints of NAS on hardware devices. We also explore the challenges and open problems of hardware NAS.
24.
Yan, Longhao; Wu, Qingyu; Li, Xi; Xie, Chenchen; Zhou, Xilin; Li, Yuqi; Shi, Daijing; Yu, Lianfeng; Zhang, Teng; Tao, Yaoyu; Yan, Bonan; Zhong, Min; Song, Zhitang; Yang, Yuchao; Huang, Ru
In: Advanced Functional Materials, vol. n/a, no. n/a, pp. 2300458, 0000.
@article{https://doi.org/10.1002/adfm.202300458,
title = {Neural Architecture Search with In-Memory Multiply–Accumulate and In-Memory Rank Based on Coating Layer Optimized C-Doped Ge2Sb2Te5 Phase Change Memory},
author = {Longhao Yan and Qingyu Wu and Xi Li and Chenchen Xie and Xilin Zhou and Yuqi Li and Daijing Shi and Lianfeng Yu and Teng Zhang and Yaoyu Tao and Bonan Yan and Min Zhong and Zhitang Song and Yuchao Yang and Ru Huang},
url = {https://onlinelibrary.wiley.com/doi/abs/10.1002/adfm.202300458},
doi = {https://doi.org/10.1002/adfm.202300458},
journal = {Advanced Functional Materials},
volume = {n/a},
number = {n/a},
pages = {2300458},
abstract = {Abstract Neural architecture search (NAS), as a subfield of automated machine learning, can design neural network models with better performance than manual design. However, the energy and time consumptions of conventional software-based NAS are huge, hindering its development and applications. Herein, 4 Mb phase change memory (PCM) chips are first fabricated that enable two key in-memory computing operations—in-memory multiply-accumulate (MAC) and in-memory rank for efficient NAS. The impacts of the coating layer material are systematically analyzed for the blade-type heating electrode on the device uniformity and in turn NAS performance. The random weights in the searched network architecture can be fine-tuned in the last stage. With 512 × 512 arrays based on 40 nm CMOS process, the PCM-based NAS has achieved 25–53× smaller model size and better performance than manually designed networks and improved the energy and time efficiency by 4779× and 123×, respectively, compared with NAS running on graphic processing unit (GPU). This work can expand the hardware accelerated in-memory operators, and significantly extend the applications of in-memory computing enabled by nonvolatile memory in advanced machine learning tasks.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Abstract Neural architecture search (NAS), as a subfield of automated machine learning, can design neural network models with better performance than manual design. However, the energy and time consumptions of conventional software-based NAS are huge, hindering its development and applications. Herein, 4 Mb phase change memory (PCM) chips are first fabricated that enable two key in-memory computing operations—in-memory multiply-accumulate (MAC) and in-memory rank for efficient NAS. The impacts of the coating layer material are systematically analyzed for the blade-type heating electrode on the device uniformity and in turn NAS performance. The random weights in the searched network architecture can be fine-tuned in the last stage. With 512 × 512 arrays based on 40 nm CMOS process, the PCM-based NAS has achieved 25–53× smaller model size and better performance than manually designed networks and improved the energy and time efficiency by 4779× and 123×, respectively, compared with NAS running on graphic processing unit (GPU). This work can expand the hardware accelerated in-memory operators, and significantly extend the applications of in-memory computing enabled by nonvolatile memory in advanced machine learning tasks.
23.
Addad, Youva; ad Frédéric Jurie, Alexis Lechervy
Multi-Exit Resource-Efficient Neural Architecture for Image Classification with Optimized Fusion Block Technical Report
0000.
@techreport{Addad-hal23a,
title = {Multi-Exit Resource-Efficient Neural Architecture for Image Classification with Optimized Fusion Block},
author = {Youva Addad and Alexis Lechervy ad Frédéric Jurie},
url = {https://hal.science/hal-04181149/document},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
22.
Tomczak, Nathaniel; Kuppannagari, Sanmukh
Automated Indexing Of TEM Diffraction Patterns Using Machine Learning Technical Report
0000.
@techreport{Tomczak-ieee-hpec23a,
title = {Automated Indexing Of TEM Diffraction Patterns Using Machine Learning},
author = {Nathaniel Tomczak and Sanmukh Kuppannagari},
url = {https://ieee-hpec.org/wp-content/uploads/2023/09/143.pdf},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
21.
Taccioli, Tommaso; Ragusa, Edoardo; Pomili, Tania; Gastaldo, Paolo; Pompa, Pier Paolo
Semi-quantitative determination of thiocyanate in saliva through colorimetric assays: design of CNN architecture via input-aware NAS Journal Article
In: IEEE SENSORS JOURNAL, , 0000.
@article{TaccioliSC23a,
title = {Semi-quantitative determination of thiocyanate in saliva through colorimetric assays: design of CNN architecture via input-aware NAS},
author = {Tommaso Taccioli and Edoardo Ragusa and Tania Pomili and Paolo Gastaldo and Pier Paolo Pompa},
url = {https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=10295395},
doi = {10.1109/JSEN.2023.3325545},
journal = {IEEE SENSORS JOURNAL, },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
20.
Ahmed, Mohammed; Du, Hongbo; AlZoubi, Alaa
In: Ultrasonic Imaging, vol. 0, no. 0, pp. 01617346231208709, 0000, (PMID: 37981781).
@article{doi:10.1177/01617346231208709,
title = {ENAS-B: Combining ENAS With Bayesian Optimization for Automatic Design of Optimal CNN Architectures for Breast Lesion Classification From Ultrasound Images},
author = {Mohammed Ahmed and Hongbo Du and Alaa AlZoubi},
url = {https://doi.org/10.1177/01617346231208709},
doi = {10.1177/01617346231208709},
journal = {Ultrasonic Imaging},
volume = {0},
number = {0},
pages = {01617346231208709},
abstract = {Efficient Neural Architecture Search (ENAS) is a recent development in searching for optimal cell structures for Convolutional Neural Network (CNN) design. It has been successfully used in various applications including ultrasound image classification for breast lesions. However, the existing ENAS approach only optimizes cell structures rather than the whole CNN architecture nor its trainable hyperparameters. This paper presents a novel framework for automatic design of CNN architectures by combining strengths of ENAS and Bayesian Optimization in two-folds. Firstly, we use ENAS to search for optimal normal and reduction cells. Secondly, with the optimal cells and a suitable hyperparameter search space, we adopt Bayesian Optimization to find the optimal depth of the network and optimal configuration of the trainable hyperparameters. To test the validity of the proposed framework, a dataset of 1522 breast lesion ultrasound images is used for the searching and modeling. We then evaluate the robustness of the proposed approach by testing the optimized CNN model on three external datasets consisting of 727 benign and 506 malignant lesion images. We further compare the CNN model with the default ENAS-based CNN model, and then with CNN models based on the state-of-the-art architectures. The results (error rate of no more than 20.6% on internal tests and 17.3% on average of external tests) show that the proposed framework generates robust and light CNN models.},
note = {PMID: 37981781},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Efficient Neural Architecture Search (ENAS) is a recent development in searching for optimal cell structures for Convolutional Neural Network (CNN) design. It has been successfully used in various applications including ultrasound image classification for breast lesions. However, the existing ENAS approach only optimizes cell structures rather than the whole CNN architecture nor its trainable hyperparameters. This paper presents a novel framework for automatic design of CNN architectures by combining strengths of ENAS and Bayesian Optimization in two-folds. Firstly, we use ENAS to search for optimal normal and reduction cells. Secondly, with the optimal cells and a suitable hyperparameter search space, we adopt Bayesian Optimization to find the optimal depth of the network and optimal configuration of the trainable hyperparameters. To test the validity of the proposed framework, a dataset of 1522 breast lesion ultrasound images is used for the searching and modeling. We then evaluate the robustness of the proposed approach by testing the optimized CNN model on three external datasets consisting of 727 benign and 506 malignant lesion images. We further compare the CNN model with the default ENAS-based CNN model, and then with CNN models based on the state-of-the-art architectures. The results (error rate of no more than 20.6% on internal tests and 17.3% on average of external tests) show that the proposed framework generates robust and light CNN models.
19.
Herron, Emily J.
0000.
@phdthesis{Herron-phd23a,
title = {Generalized Differentiable Neural Architecture Search with Performance and Stability ImprovementsPerformance and Stability Improvements},
author = {Emily J. Herron},
url = {https://trace.tennessee.edu/cgi/viewcontent.cgi?article=10188&context=utk_graddiss},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
18.
Chen, Xiangning
Advancing Automated Machine Learning: Neural Architectures and Optimization Algorithms PhD Thesis
0000.
@phdthesis{Chen-phd23a,
title = {Advancing Automated Machine Learning: Neural Architectures and Optimization Algorithms},
author = {Chen, Xiangning},
url = {https://escholarship.org/content/qt2f40c1w4/qt2f40c1w4.pdf},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
17.
Xiao, Songyi; Wang, Wenjun
Ranking-based architecture generation for surrogate-assisted neural architecture search Journal Article
In: Concurrency and Computation: Practice and Experience, vol. n/a, no. n/a, pp. e8051, 0000.
@article{https://doi.org/10.1002/cpe.8051,
title = {Ranking-based architecture generation for surrogate-assisted neural architecture search},
author = {Songyi Xiao and Wenjun Wang},
url = {https://onlinelibrary.wiley.com/doi/abs/10.1002/cpe.8051},
doi = {https://doi.org/10.1002/cpe.8051},
journal = {Concurrency and Computation: Practice and Experience},
volume = {n/a},
number = {n/a},
pages = {e8051},
abstract = {Abstract Architectures generation optimization has been received a lot of attention in neural architecture search (NAS) since its efficiency in generating architecture. By learning the architecture representation through unsupervised learning and constructing a latent space, the prediction process of predictors is simplified, leading to improved efficiency in architecture search. However, searching for architectures with top performance in complex and large NAS search spaces remains challenging. In this paper, an approach that combined a ranker and generative model is proposed to address this challenge through regularizing the latent space and identifying architectures with top rankings. We introduce the ranking error to gradually regulate the training of the generative model, making it easier to identify architecture representations in the latent space. Additionally, a surrogate-assisted evolutionary search method that utilized neural network Bayesian optimization is proposed to efficiently explore promising architectures in the latent space. We demonstrate the benefits of our approach in optimizing architectures with top rankings, and our method outperforms state-of-the-art techniques on various NAS benchmarks. The code is available at https://github.com/outofstyle/RAGS-NAS.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Abstract Architectures generation optimization has been received a lot of attention in neural architecture search (NAS) since its efficiency in generating architecture. By learning the architecture representation through unsupervised learning and constructing a latent space, the prediction process of predictors is simplified, leading to improved efficiency in architecture search. However, searching for architectures with top performance in complex and large NAS search spaces remains challenging. In this paper, an approach that combined a ranker and generative model is proposed to address this challenge through regularizing the latent space and identifying architectures with top rankings. We introduce the ranking error to gradually regulate the training of the generative model, making it easier to identify architecture representations in the latent space. Additionally, a surrogate-assisted evolutionary search method that utilized neural network Bayesian optimization is proposed to efficiently explore promising architectures in the latent space. We demonstrate the benefits of our approach in optimizing architectures with top rankings, and our method outperforms state-of-the-art techniques on various NAS benchmarks. The code is available at https://github.com/outofstyle/RAGS-NAS.
16.
Avval, Sasan Salmani Pour; Yaghoubi, Vahid; Eskue, Nathan D.; Groves, Roger M.
Systematic Review on Neural Architecture Search Technical Report
0000.
@techreport{Avval-24a,
title = {Systematic Review on Neural Architecture Search},
author = {Sasan Salmani Pour Avval and Vahid Yaghoubi and Nathan D. Eskue and Roger M. Groves},
url = {https://www.researchsquare.com/article/rs-4085293/v1},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
15.
Yao, Yang; Wang, Xin; Qin, Yijian; Zhang, Ziwei; Zhu, Wenwu; Mei, Hong
Customized Cross-device Neural Architecture Search with Images Miscellaneous
0000.
@misc{Yao2024,
title = {Customized Cross-device Neural Architecture Search with Images},
author = {Yang Yao and Xin Wang and Yijian Qin and Ziwei Zhang and Wenwu Zhu and Hong Mei},
url = {http://mn.cs.tsinghua.edu.cn/xinwang/PDF/papers/2024_Customized%20Cross-device%20Neural%20Architecture%20Search%20with%20Images.pdf},
keywords = {},
pubstate = {published},
tppubtype = {misc}
}
14.
Changwei, Ping Haoyu Ma Yongjie Song
Evolutionary Neural Architecture Search for Traffic Sign Recognition Journal Article
In: 光电子快报, 0000.
@article{nokey,
title = {Evolutionary Neural Architecture Search for Traffic Sign Recognition},
author = {Ping Haoyu Ma Yongjie Song Changwei},
url = {http://www.oelett.net/gdzkb/article/abstract/2024067},
journal = {光电子快报},
publisher = {the Editorial Department of Optoelectronics Letters},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
13.
Hosseinzadeh, Hamed
Neural Architecture Search for Adaptive Neural Network Structures: Comparative Analysis of Layer and Neuron Adjustments Miscellaneous
0000.
@misc{Hosseinzadeh-misc241,
title = {Neural Architecture Search for Adaptive Neural Network Structures: Comparative Analysis of Layer and Neuron Adjustments},
author = {Hamed Hosseinzadeh},
url = {https://www.researchsquare.com/article/rs-4909959/v1},
keywords = {},
pubstate = {published},
tppubtype = {misc}
}
12.
Deevi, Sri Aditya; Mishra, Asish Kumar; Mishra, Deepak; Kumar, L Ravi; Kumar, G V P Bharat; Bhagavan, G Murali Krishna
Efficient Self-Supervised Neural Architecture Search Booklet
EasyChair Preprint 14726, 0000.
@booklet{EasyChair:14726,
title = {Efficient Self-Supervised Neural Architecture Search},
author = {Sri Aditya Deevi and Asish Kumar Mishra and Deepak Mishra and L Ravi Kumar and G V P Bharat Kumar and G Murali Krishna Bhagavan},
url = {https://easychair.org/publications/preprint/6hNw},
howpublished = {EasyChair Preprint 14726},
month = {00},
keywords = {},
pubstate = {published},
tppubtype = {booklet}
}
11.
(Ed.)
An Efficient Neural Architecture Search Model for Medical Image Classification Collection
0000.
@collection{xie-esann24a,
title = {An Efficient Neural Architecture Search Model for Medical Image Classification},
author = {Lunchen Xie and Eugenio Lomurno and Matteo Gambella and Danilo Ardagna and Manuel Roveri and Matteo Matteucci and and Qingjiang Shi},
url = {https://www.esann.org/sites/default/files/proceedings/2024/ES2024-119.pdf},
booktitle = {European Symposium on Artificial Neural Networks 2024},
keywords = {},
pubstate = {published},
tppubtype = {collection}
}
10.
Banerjee, Jishnu
Autonomous Task-Tiling and Deep Neural Architecture Search for Intermittent Systems PhD Thesis
0000.
@phdthesis{nokey,
title = {Autonomous Task-Tiling and Deep Neural Architecture Search for Intermittent Systems},
author = {Banerjee, Jishnu},
url = {https://www.proquest.com/openview/0709fcf82e103976505463c37c325677/1?pq-origsite=gscholar&cbl=18750&diss=y},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
9.
(Ed.)
Educational Evolutionary Neural Architecture Search for Time Series Prediction Collection
0000.
@collection{nokey,
title = {Educational Evolutionary Neural Architecture Search for Time Series Prediction},
author = {Martha Isabel Escalona-Llaguno and Sergio M. Sarmiento-Rosales},
url = {https://www.scitepress.org/Papers/2024/129489/129489.pdf},
doi = {10.5220/0012948900003837},
booktitle = {Proceedings of the 16th International Joint Conference on Computational Intelligence (IJCCI 2024)},
keywords = {},
pubstate = {published},
tppubtype = {collection}
}
8.
Kim, Youngeun
Algorithmic Approaches for Empowering Spike-based Machine Intelligence PhD Thesis
0000.
@phdthesis{kim-phds25a,
title = {Algorithmic Approaches for Empowering Spike-based Machine Intelligence},
author = {Youngeun Kim
},
url = {https://www.proquest.com/docview/3164060689?pq-origsite=gscholar&fromopenview=true&sourcetype=Dissertations%20&%20Theses},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
7.
García, Cosijopii García
Multi-objective Evolutionary Algorithms for the optimization of Deep Neural Network Architectures PhD Thesis
0000.
@phdthesis{nokey,
title = {Multi-objective Evolutionary Algorithms for the optimization of Deep Neural Network Architectures},
author = {Cosijopii García García},
url = {https://inaoe.repositorioinstitucional.mx/jspui/bitstream/1009/2656/1/GARC%C3%8DAGC_DCC.pdf},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
6.
Rafiq, Shehla; Assad, Assif
RNAS-sgRNA: Recurrent Neural Architecture Search for Detection of On-Target Effects in Single Guide RNA Journal Article
In: Journal of Computational Biology, vol. 0, no. 0, pp. null, 0000, (PMID: 40501348).
@article{doi:10.1089/cmb.2025.0031,
title = {RNAS-sgRNA: Recurrent Neural Architecture Search for Detection of On-Target Effects in Single Guide RNA},
author = {Shehla Rafiq and Assif Assad},
url = {https://doi.org/10.1089/cmb.2025.0031},
doi = {10.1089/cmb.2025.0031},
journal = {Journal of Computational Biology},
volume = {0},
number = {0},
pages = {null},
abstract = {Clustered Regularly Interspaced Short Palindromic Repeats (CRISPR)/Cas9 is a leading genomic editing tool, but its effectiveness is limited by considerable heterogeneity in target efficiency among different single guide RNAs (sgRNA). This study presents RNAS-sgRNA, a hybrid model that integrates neural architecture search (NAS) with recurrent neural networks (RNN) to evaluate the on-target efficacy of CRISPR/Cas9 sgRNA. The RNAS-sgRNA model automates architectural discovery, improving sgRNA sequence categorization without considerable manual adjustment. The NAS component improves the RNN architecture, which analyzes sgRNA sequences represented as binary matrices and produces a classification score. Upon evaluation across several datasets, RNAS-sgRNA exhibits substantial performance enhancements with multiple cell lines, comparing its area under the receiver operating characteristic curve (AUROC) performance to the baseline CRISPRpred(SEQ) and DeepCRISPR models. RNAS-sgRNA demonstrated substantial improvements in AUROC performance in several cell lines compared with existing models. Notable improvements include enhancements of 8.62% for HCT116, 121.57% for HEK293T, 13.40% for HeLa, and 20.78% for HL60 cell lines, resulting in an overall improvement of 13.46%. Compared with DeepCRISPR, the model achieved additional AUROC gains in all cell lines tested, with an average improvement of 14.74%. The study also highlighted the ability of the model to deliver superior performance on smaller datasets through transfer learning, underscoring its potential applications in personalized medicine and genetic research. RNAS-sgRNA introduces a novel integration of NAS with RNN to evaluate the efficacy of CRISPR/Cas9 sgRNA. Unlike traditional methods that require significant manual adjustments, this model automates architectural discovery, optimizing the RNN structure for sgRNA sequence analysis. Furthermore, the application of transfer learning to fine-tune the pretrained model on small cell-line datasets represents a pioneering approach in the domain. The model’s demonstrated ability to significantly outperform existing algorithms, including CRISPRpred(SEQ) and DeepCRISPR, across multiple cell lines highlights its innovative contribution to genomic editing research and personalized medicine.},
note = {PMID: 40501348},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Clustered Regularly Interspaced Short Palindromic Repeats (CRISPR)/Cas9 is a leading genomic editing tool, but its effectiveness is limited by considerable heterogeneity in target efficiency among different single guide RNAs (sgRNA). This study presents RNAS-sgRNA, a hybrid model that integrates neural architecture search (NAS) with recurrent neural networks (RNN) to evaluate the on-target efficacy of CRISPR/Cas9 sgRNA. The RNAS-sgRNA model automates architectural discovery, improving sgRNA sequence categorization without considerable manual adjustment. The NAS component improves the RNN architecture, which analyzes sgRNA sequences represented as binary matrices and produces a classification score. Upon evaluation across several datasets, RNAS-sgRNA exhibits substantial performance enhancements with multiple cell lines, comparing its area under the receiver operating characteristic curve (AUROC) performance to the baseline CRISPRpred(SEQ) and DeepCRISPR models. RNAS-sgRNA demonstrated substantial improvements in AUROC performance in several cell lines compared with existing models. Notable improvements include enhancements of 8.62% for HCT116, 121.57% for HEK293T, 13.40% for HeLa, and 20.78% for HL60 cell lines, resulting in an overall improvement of 13.46%. Compared with DeepCRISPR, the model achieved additional AUROC gains in all cell lines tested, with an average improvement of 14.74%. The study also highlighted the ability of the model to deliver superior performance on smaller datasets through transfer learning, underscoring its potential applications in personalized medicine and genetic research. RNAS-sgRNA introduces a novel integration of NAS with RNN to evaluate the efficacy of CRISPR/Cas9 sgRNA. Unlike traditional methods that require significant manual adjustments, this model automates architectural discovery, optimizing the RNN structure for sgRNA sequence analysis. Furthermore, the application of transfer learning to fine-tune the pretrained model on small cell-line datasets represents a pioneering approach in the domain. The model’s demonstrated ability to significantly outperform existing algorithms, including CRISPRpred(SEQ) and DeepCRISPR, across multiple cell lines highlights its innovative contribution to genomic editing research and personalized medicine.
5.
More, Shraddha Subhash; Bansode, Rajesh
FCN-YOLOS: An Effective Deep-Learning Model for Real-Time Object Detection Journal Article
In: Journal of Field Robotics, vol. n/a, no. n/a, 0000.
@article{https://doi.org/10.1002/rob.70001,
title = {FCN-YOLOS: An Effective Deep-Learning Model for Real-Time Object Detection},
author = {Shraddha Subhash More and Rajesh Bansode},
url = {https://onlinelibrary.wiley.com/doi/abs/10.1002/rob.70001},
doi = {https://doi.org/10.1002/rob.70001},
journal = {Journal of Field Robotics},
volume = {n/a},
number = {n/a},
abstract = {ABSTRACT Real-time object recognition is a significant field of research with numerous applications, including object tracking, video surveillance, and autonomous driving. This identifies the smallest bounding boxes that encompass the objects of interest within the input images. Nevertheless, these approaches face challenges, like limited support for quantization and suboptimal trade in achieving accurate object detection. To address these issues, a novel approach called Faster region-based Convoluted Non-monopolize search You Only Look Once neural architecture Search (FCN-YOLOS) is introduced for object detection. This approach merges the advanced feature abstraction abilities of Faster R-CNN with the efficient object recognition strengths of YOLOv8, enhanced by NAS optimization. YOLOv8 is employed for its rapid and accurate real-time detection of abandoned items, while Faster R-CNN contributes sophisticated feature extraction by utilizing statistical, grid, and Histogram of Oriented Optical Flow (HOOF) features to improve object representation and classification. Additionally, NAS optimizes hyperparameters by balancing exploration and exploitation, which helps minimize the loss function, reduce overfitting, and enhance generalization. This results in exceptional real-time object detection performance within the FCN-YOLOS framework. The proposed technique has demonstrated a maximum image of approximately 99%, 96.3%, 94.9%, and 95.2% concerning brightness realization compared to existing methods for accuracy, recall, precision, and F1 score, respectively. These outcomes highlight its extensive applicability across diverse object detection contexts, rendering it a compelling option for both academic and industrial research. Overall, the proposed approach for object recognition techniques in feature extraction and hyperparameter adjustments further improves evaluation in terms of efficiency and object detection accuracy.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
ABSTRACT Real-time object recognition is a significant field of research with numerous applications, including object tracking, video surveillance, and autonomous driving. This identifies the smallest bounding boxes that encompass the objects of interest within the input images. Nevertheless, these approaches face challenges, like limited support for quantization and suboptimal trade in achieving accurate object detection. To address these issues, a novel approach called Faster region-based Convoluted Non-monopolize search You Only Look Once neural architecture Search (FCN-YOLOS) is introduced for object detection. This approach merges the advanced feature abstraction abilities of Faster R-CNN with the efficient object recognition strengths of YOLOv8, enhanced by NAS optimization. YOLOv8 is employed for its rapid and accurate real-time detection of abandoned items, while Faster R-CNN contributes sophisticated feature extraction by utilizing statistical, grid, and Histogram of Oriented Optical Flow (HOOF) features to improve object representation and classification. Additionally, NAS optimizes hyperparameters by balancing exploration and exploitation, which helps minimize the loss function, reduce overfitting, and enhance generalization. This results in exceptional real-time object detection performance within the FCN-YOLOS framework. The proposed technique has demonstrated a maximum image of approximately 99%, 96.3%, 94.9%, and 95.2% concerning brightness realization compared to existing methods for accuracy, recall, precision, and F1 score, respectively. These outcomes highlight its extensive applicability across diverse object detection contexts, rendering it a compelling option for both academic and industrial research. Overall, the proposed approach for object recognition techniques in feature extraction and hyperparameter adjustments further improves evaluation in terms of efficiency and object detection accuracy.
4.
Dantas, Pierre V.; Cordeiro, Lucas C.; Junior, Waldir S. S.
A Review of State-of-the-Art Techniques for Large Language Model Compression Journal Article
In: Complex & Intelligent Systems, 0000, ISSN: 2198-6053.
@article{39081e27cda444049c39fff371ed2d60,
title = {A Review of State-of-the-Art Techniques for Large Language Model Compression},
author = {Pierre V. Dantas and Lucas C. Cordeiro and Waldir S. S. Junior},
url = {https://research.manchester.ac.uk/en/publications/a-review-of-state-of-the-art-techniques-for-large-language-model-},
issn = {2198-6053},
journal = {Complex & Intelligent Systems},
publisher = {Springer Nature},
abstract = {The rapid advancement of large language models (LLMs) has driven significant progress in natural language processing (NLP) and related domains. However, their deployment remains constrained by challenges related to computation, memory, and energy efficiency – particularly in real-world applications. This work presents a comprehensive review of state-of-the-art compression techniques, including pruning, quantization, knowledge distillation, and neural architecture search (NAS), which collectively aim to reduce model size, enhance inference speed, and lower energy consumption while maintaining performance. A robust evaluation framework is introduced, incorporating traditional metrics, such as accuracy and perplexity (PPL), alongside advanced criteria including latency-accuracy trade-offs, parameter efficiency, multi-objective Pareto optimization,and fairness considerations. This study further highlights trends and challenges, such as fairness-aware compression, robustness against adversarial attacks, and hardware-specific optimizations. Additionally, NAS-driven strategies are explored as a means to design task-aware, hardware-adaptive architectures that enhance LLM compression efficiency. Hybrid and adaptive methods are also examined to dynamically optimize computational efficiency across diverse deployment scenarios. This work not only synthesizes recent advancements and identifies open problems but also proposes a structured research roadmap to guide thedevelopment of efficient, scalable, and equitable LLMs. By bridging the gap between compression research and real-world deployment, this study offers actionable insights for optimizing LLMs across a range of environments, including mobile devices and large-scale cloud infrastructures.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The rapid advancement of large language models (LLMs) has driven significant progress in natural language processing (NLP) and related domains. However, their deployment remains constrained by challenges related to computation, memory, and energy efficiency – particularly in real-world applications. This work presents a comprehensive review of state-of-the-art compression techniques, including pruning, quantization, knowledge distillation, and neural architecture search (NAS), which collectively aim to reduce model size, enhance inference speed, and lower energy consumption while maintaining performance. A robust evaluation framework is introduced, incorporating traditional metrics, such as accuracy and perplexity (PPL), alongside advanced criteria including latency-accuracy trade-offs, parameter efficiency, multi-objective Pareto optimization,and fairness considerations. This study further highlights trends and challenges, such as fairness-aware compression, robustness against adversarial attacks, and hardware-specific optimizations. Additionally, NAS-driven strategies are explored as a means to design task-aware, hardware-adaptive architectures that enhance LLM compression efficiency. Hybrid and adaptive methods are also examined to dynamically optimize computational efficiency across diverse deployment scenarios. This work not only synthesizes recent advancements and identifies open problems but also proposes a structured research roadmap to guide thedevelopment of efficient, scalable, and equitable LLMs. By bridging the gap between compression research and real-world deployment, this study offers actionable insights for optimizing LLMs across a range of environments, including mobile devices and large-scale cloud infrastructures.
3.
Cassimon, Amber
0000.
@phdthesis{Cassimon-25a,
title = { Building efficient neural networks using scalable and transferable neural architecture search strategies },
author = { Cassimon, Amber },
url = {https://repository.uantwerpen.be/desktop/irua},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
2.
Yu, Jiaying; Luo, Haoqiu; Dai, Xinfa; Wu, Yong; Cong, Peijin; Zhou, Junlong
A Multi-Level Lightweight Framework for Deep Neural Networks on Resource-Constrained Edge Devices Journal Article
In: Journal of Circuits, Systems and Computers, vol. 0, no. ja, pp. null, 0000.
@article{doi:10.1142/S0218126626500283,
title = {A Multi-Level Lightweight Framework for Deep Neural Networks on Resource-Constrained Edge Devices},
author = {Jiaying Yu and Haoqiu Luo and Xinfa Dai and Yong Wu and Peijin Cong and Junlong Zhou},
url = {https://doi.org/10.1142/S0218126626500283},
doi = {10.1142/S0218126626500283},
journal = {Journal of Circuits, Systems and Computers},
volume = {0},
number = {ja},
pages = {null},
abstract = {Deep neural networks (DNNs) are widely used in Artificial Intelligence (AI) applications due to their powerful representational capacity. However, deploying DNNs on edge devices faces severe challenges, including limited computational resources, restricted memory capacity, and heterogeneous processor architectures, which often make conventional DNN models infeasible. In this paper, we propose a novel multi-level model lightweighting framework designed to address these challenges. The framework integrates three key techniques: Neural Architecture Search (NAS), Knowledge Distillation (KD), and layer-adaptive quantization. NAS is employed to automatically discover an efficient network architecture under hardware constraints, enabling structural compression. Subsequently, KD transfers knowledge from the original model to the compressed model, mitigating the accuracy loss resulting from compression. Finally, a layer-adaptive quantization strategy is implemented, assigning distinct bit-widths to different layers based on their data distribution characteristics, thereby enhancing storage and computational efficiency without sacrificing performance. Experimental evaluations on edge devices with multiple different processor architectures demonstrate the efficacy of the framework, achieving a model compression rate ranging from 15% to 70% with less than a 1% reduction in accuracy.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Deep neural networks (DNNs) are widely used in Artificial Intelligence (AI) applications due to their powerful representational capacity. However, deploying DNNs on edge devices faces severe challenges, including limited computational resources, restricted memory capacity, and heterogeneous processor architectures, which often make conventional DNN models infeasible. In this paper, we propose a novel multi-level model lightweighting framework designed to address these challenges. The framework integrates three key techniques: Neural Architecture Search (NAS), Knowledge Distillation (KD), and layer-adaptive quantization. NAS is employed to automatically discover an efficient network architecture under hardware constraints, enabling structural compression. Subsequently, KD transfers knowledge from the original model to the compressed model, mitigating the accuracy loss resulting from compression. Finally, a layer-adaptive quantization strategy is implemented, assigning distinct bit-widths to different layers based on their data distribution characteristics, thereby enhancing storage and computational efficiency without sacrificing performance. Experimental evaluations on edge devices with multiple different processor architectures demonstrate the efficacy of the framework, achieving a model compression rate ranging from 15% to 70% with less than a 1% reduction in accuracy.
1.
Wang, Haishuai; Gao, Yang; Zheng, Xin; Zhang, Peng; Bu, Jiajun; Yu, Philip S.
Graph neural architecture search with large language models Journal Article
In: Science China Information Sciences , 0000.
@article{nokey,
title = {Graph neural architecture search with large language models},
author = {
Haishuai Wang and Yang Gao and Xin Zheng and Peng Zhang and Jiajun Bu and Philip S. Yu
},
url = {https://link.springer.com/article/10.1007/s11432-024-4539-1},
journal = {Science China Information Sciences },
keywords = {},
pubstate = {published},
tppubtype = {article}
}