Maintained by Difan Deng and Marius Lindauer.
The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.
Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.
Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.
2023
2576.
Liang, Jason; Shahrzad, Hormoz; Miikkulainen, Risto
Asynchronous Evolution of Deep Neural Network Architectures Technical Report
2023.
@techreport{liang2023asynchronous,
title = {Asynchronous Evolution of Deep Neural Network Architectures},
author = {Jason Liang and Hormoz Shahrzad and Risto Miikkulainen},
url = {https://arxiv.org/pdf/2308.04102},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2575.
Patcharabumrung, Praiwan; Jewajinda, Yutana; Praditwong, Kata
Effects of Genetic Operators on Neural Architecture Search Using Multi-Objective Genetic Algorithm Proceedings Article
In: 2023 20th International Joint Conference on Computer Science and Software Engineering (JCSSE), pp. 61-66, 2023.
@inproceedings{10201969,
title = {Effects of Genetic Operators on Neural Architecture Search Using Multi-Objective Genetic Algorithm},
author = {Praiwan Patcharabumrung and Yutana Jewajinda and Kata Praditwong},
url = {https://ieeexplore.ieee.org/abstract/document/10201969},
doi = {10.1109/JCSSE58229.2023.10201969},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {2023 20th International Joint Conference on Computer Science and Software Engineering (JCSSE)},
pages = {61-66},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2574.
Ozaeta, Mark Anthony A.; Fajardo, Arnel C.; Brazas, Felimon P.; Cantal, Jed Allan M.
Seagrass Classification Using Differentiable Architecture Search Proceedings Article
In: 2023 20th International Joint Conference on Computer Science and Software Engineering (JCSSE), pp. 123-128, 2023.
@inproceedings{10202072,
title = {Seagrass Classification Using Differentiable Architecture Search},
author = {Mark Anthony A. Ozaeta and Arnel C. Fajardo and Felimon P. Brazas and Jed Allan M. Cantal},
url = {https://ieeexplore.ieee.org/abstract/document/10202072},
doi = {10.1109/JCSSE58229.2023.10202072},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {2023 20th International Joint Conference on Computer Science and Software Engineering (JCSSE)},
pages = {123-128},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2573.
Dong, Wenqian; Kestor, Gokcen; Li, Dong
Auto-HPCnet: An Automatic Framework to Build Neural Network-based Surrogate for High-Performance Computing Applications Proceedings Article
In: Butt, Ali Raza; Mi, Ningfang; Chard, Kyle (Ed.): Proceedings of the 32nd International Symposium on High-Performance Parallel and Distributed Computing, HPDC 2023, Orlando, FL, USA, June 16-23, 2023, pp. 31–44, ACM, 2023.
@inproceedings{DBLP:conf/hpdc/DongK023,
title = {Auto-HPCnet: An Automatic Framework to Build Neural Network-based Surrogate for High-Performance Computing Applications},
author = {Wenqian Dong and Gokcen Kestor and Dong Li},
editor = {Ali Raza Butt and Ningfang Mi and Kyle Chard},
url = {https://doi.org/10.1145/3588195.3592985},
doi = {10.1145/3588195.3592985},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Proceedings of the 32nd International Symposium on High-Performance
Parallel and Distributed Computing, HPDC 2023, Orlando, FL, USA,
June 16-23, 2023},
pages = {31–44},
publisher = {ACM},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2572.
Maashi, Mashael S.; Alamro, Hayam; Mohsen, Heba; Negm, Noha; Mohammed, Gouse Pasha; Ahmed, Noura Abdelaziz; Ibrahim, Sara Saadeldeen; Alsaid, Mohamed Ibrahim
Modeling of Reptile Search Algorithm With Deep Learning Approach for Copy Move Image Forgery Detection Journal Article
In: IEEE Access, vol. 11, pp. 87297–87304, 2023.
@article{DBLP:journals/access/MaashiAMNMAIA23,
title = {Modeling of Reptile Search Algorithm With Deep Learning Approach for Copy Move Image Forgery Detection},
author = {Mashael S. Maashi and Hayam Alamro and Heba Mohsen and Noha Negm and Gouse Pasha Mohammed and Noura Abdelaziz Ahmed and Sara Saadeldeen Ibrahim and Mohamed Ibrahim Alsaid},
url = {https://doi.org/10.1109/ACCESS.2023.3304237},
doi = {10.1109/ACCESS.2023.3304237},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Access},
volume = {11},
pages = {87297–87304},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2571.
Huang, Yuxuan; Zhang, Xixi; Wang, Yu; Jiao, Donglai; Gui, Guan; Ohtsuki, Tomoaki
NASEI: Neural Architecture Search-Based Specific Emitter Identification Method Proceedings Article
In: 2023 IEEE 97th Vehicular Technology Conference (VTC2023-Spring), pp. 1-5, 2023.
@inproceedings{10199409,
title = {NASEI: Neural Architecture Search-Based Specific Emitter Identification Method},
author = {Yuxuan Huang and Xixi Zhang and Yu Wang and Donglai Jiao and Guan Gui and Tomoaki Ohtsuki},
url = {https://ieeexplore.ieee.org/abstract/document/10199409},
doi = {10.1109/VTC2023-Spring57618.2023.10199409},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {2023 IEEE 97th Vehicular Technology Conference (VTC2023-Spring)},
pages = {1-5},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2570.
Kapoor, Rahul; Pillay, Nelishia
Iterative Structure-Based Genetic Programming for Neural Architecture Search Proceedings Article
In: Proceedings of the Companion Conference on Genetic and Evolutionary Computation, pp. 595–598, Association for Computing Machinery, Lisbon, Portugal, 2023, ISBN: 9798400701207.
@inproceedings{10.1145/3583133.3590759,
title = {Iterative Structure-Based Genetic Programming for Neural Architecture Search},
author = {Rahul Kapoor and Nelishia Pillay},
url = {https://doi.org/10.1145/3583133.3590759},
doi = {10.1145/3583133.3590759},
isbn = {9798400701207},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Proceedings of the Companion Conference on Genetic and Evolutionary Computation},
pages = {595–598},
publisher = {Association for Computing Machinery},
address = {Lisbon, Portugal},
series = {GECCO '23 Companion},
abstract = {In this paper we present an iterative structure-based genetic programming algorithm for neural architecture search. Canonical genetic programming uses a fitness function to determine where to move the search to in the program space. This research investigates using the structure of the syntax trees, representing different areas of the program space, in addition to the fitness function to direct the search. The structure is used to avoid areas of the search that previously led to local optima both globally (exploration) and locally (exploitation). The proposed approach is evaluated for image classification and video shorts creation. The iterative structure-based approach was found to produce better results then canonical genetic programming for both problem domains, with a slight reduction in computational cost. The approach also produced better results than genetic algorithms which are traditionally used for neural architecture search.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
In this paper we present an iterative structure-based genetic programming algorithm for neural architecture search. Canonical genetic programming uses a fitness function to determine where to move the search to in the program space. This research investigates using the structure of the syntax trees, representing different areas of the program space, in addition to the fitness function to direct the search. The structure is used to avoid areas of the search that previously led to local optima both globally (exploration) and locally (exploitation). The proposed approach is evaluated for image classification and video shorts creation. The iterative structure-based approach was found to produce better results then canonical genetic programming for both problem domains, with a slight reduction in computational cost. The approach also produced better results than genetic algorithms which are traditionally used for neural architecture search.
2569.
Klein, Aaron; Golebiowski, Jacek; Ma, Xingchen; Perrone, Valerio; Archambeau, Cédric
Structural pruning of large language models via neural architecture search Proceedings Article
In: AutoML Conference 2023, 2023.
@inproceedings{Klein2023,
title = {Structural pruning of large language models via neural architecture search},
author = {Aaron Klein and Jacek Golebiowski and Xingchen Ma and Valerio Perrone and Cédric Archambeau},
url = {https://www.amazon.science/publications/structural-pruning-of-large-language-models-via-neural-architecture-search},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {AutoML Conference 2023},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2568.
Huang, Junhao; Xue, Bing; Sun, Yanan; Zhang, Mengjie
Multi-Objective Evolutionary Search of Compact Convolutional Neural Networks with Training-Free Estimation Proceedings Article
In: Proceedings of the Companion Conference on Genetic and Evolutionary Computation, pp. 655–658, Association for Computing Machinery, Lisbon, Portugal, 2023, ISBN: 9798400701207.
@inproceedings{10.1145/3583133.3590535,
title = {Multi-Objective Evolutionary Search of Compact Convolutional Neural Networks with Training-Free Estimation},
author = {Junhao Huang and Bing Xue and Yanan Sun and Mengjie Zhang},
url = {https://doi.org/10.1145/3583133.3590535},
doi = {10.1145/3583133.3590535},
isbn = {9798400701207},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Proceedings of the Companion Conference on Genetic and Evolutionary Computation},
pages = {655–658},
publisher = {Association for Computing Machinery},
address = {Lisbon, Portugal},
series = {GECCO '23 Companion},
abstract = {With the increasing demand of deploying convolutional neural networks (CNNs) on resource-constrained devices, designing high-performance and lightweight architectures has become a main challenge for neural architecture search (NAS). This paper develops an evolutionary multi-objective optimization framework to explore CNNs with different compactness in a flexible way. A multi-scale convolutional module is developed to enhance the feature learning capability. To further improve the architecture search efficiency, a low-cost metric based on neural tangent kernel is leveraged to estimate the trainability of CNNs instead of performing an expensive training process. Experiments are carried out on CIFAR-10 and CIFAR-100, to verify the effectiveness of the proposed method. Compared with the state-of-the-art algorithms, the proposed method discovers architectures with a smaller number of parameters and competitive classification performance using only up to 0.2 GPU days, showing a better trade-off between accuracy and model complexity.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
With the increasing demand of deploying convolutional neural networks (CNNs) on resource-constrained devices, designing high-performance and lightweight architectures has become a main challenge for neural architecture search (NAS). This paper develops an evolutionary multi-objective optimization framework to explore CNNs with different compactness in a flexible way. A multi-scale convolutional module is developed to enhance the feature learning capability. To further improve the architecture search efficiency, a low-cost metric based on neural tangent kernel is leveraged to estimate the trainability of CNNs instead of performing an expensive training process. Experiments are carried out on CIFAR-10 and CIFAR-100, to verify the effectiveness of the proposed method. Compared with the state-of-the-art algorithms, the proposed method discovers architectures with a smaller number of parameters and competitive classification performance using only up to 0.2 GPU days, showing a better trade-off between accuracy and model complexity.
2567.
Yan, Xueming; Huang, Han; Jin, Yaochu; Chen, Liang; Liang, Zhanning; Hao, Zhifeng
Neural Architecture Search via Multi-Hashing Embedding and Graph Tensor Networks for Multilingual Text Classification Journal Article
In: IEEE Transactions on Emerging Topics in Computational Intelligence, pp. 1-14, 2023.
@article{10218728,
title = {Neural Architecture Search via Multi-Hashing Embedding and Graph Tensor Networks for Multilingual Text Classification},
author = {Xueming Yan and Han Huang and Yaochu Jin and Liang Chen and Zhanning Liang and Zhifeng Hao},
url = {https://ieeexplore.ieee.org/abstract/document/10218728},
doi = {10.1109/TETCI.2023.3301774},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Emerging Topics in Computational Intelligence},
pages = {1-14},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2566.
Cheng, Quan; Huang, Mingqiang; Man, Changhai; Shen, Ao; Dai, Liuyao; Yu, Hao; Hashimoto, Masanori
Reliability Exploration of System-on-Chip With Multi-Bit-Width Accelerator for Multi-Precision Deep Neural Networks Journal Article
In: IEEE Transactions on Circuits and Systems I: Regular Papers, pp. 1-14, 2023.
@article{10220121,
title = {Reliability Exploration of System-on-Chip With Multi-Bit-Width Accelerator for Multi-Precision Deep Neural Networks},
author = {Quan Cheng and Mingqiang Huang and Changhai Man and Ao Shen and Liuyao Dai and Hao Yu and Masanori Hashimoto},
doi = {10.1109/TCSI.2023.3300899},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Circuits and Systems I: Regular Papers},
pages = {1-14},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2565.
Burghoff, Julian; Rottmann, Matthias; Conta, Jill; Schoenen, Sebastian; Witte, Andreas; Gottschalk, Hanno
ResBuilder: Automated Learning of Depth with Residual Structures Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2308-08504,
title = {ResBuilder: Automated Learning of Depth with Residual Structures},
author = {Julian Burghoff and Matthias Rottmann and Jill Conta and Sebastian Schoenen and Andreas Witte and Hanno Gottschalk},
url = {https://doi.org/10.48550/arXiv.2308.08504},
doi = {10.48550/arXiv.2308.08504},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2308.08504},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2564.
Kuş, Zeki; Kiraz, Berna; Göksu, Tuğçe Koçak; Aydın, Musa; Özkan, Esra; Vural, Atay; Kiraz, Alper; Can, Burhanettin
Differential evolution-based neural architecture search for brain vessel segmentation Journal Article
In: Engineering Science and Technology, an International Journal, vol. 46, pp. 101502, 2023, ISSN: 2215-0986.
@article{KUS2023101502,
title = {Differential evolution-based neural architecture search for brain vessel segmentation},
author = {Zeki Kuş and Berna Kiraz and Tuğçe Koçak Göksu and Musa Aydın and Esra Özkan and Atay Vural and Alper Kiraz and Burhanettin Can},
url = {https://www.sciencedirect.com/science/article/pii/S2215098623001805},
doi = {https://doi.org/10.1016/j.jestch.2023.101502},
issn = {2215-0986},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Engineering Science and Technology, an International Journal},
volume = {46},
pages = {101502},
abstract = {Brain vasculature analysis is critical in developing novel treatment targets for neurodegenerative diseases. Such an accurate analysis cannot be performed manually but requires a semi-automated or fully-automated approach. Deep learning methods have recently proven indispensable for the automated segmentation and analysis of medical images. However, optimizing a deep learning network architecture is another challenge. Manually selecting deep learning network architectures and tuning their hyper-parameters requires a lot of expertise and effort. To solve this problem, neural architecture search (NAS) approaches that explore more efficient network architectures with high segmentation performance have been proposed in the literature. This study introduces differential evolution-based NAS approaches in which a novel search space is proposed for brain vessel segmentation. We select two architectures that are frequently used for medical image segmentation, i.e. U-Net and Attention U-Net, as baselines for NAS optimizations. The conventional differential evolution and the opposition-based differential evolution with novel search space are employed as search methods in NAS. Furthermore, we perform ablation studies and evaluate the effects of specific loss functions, model pruning, threshold selection and generalization performance on the proposed models. The experiments are conducted on two datasets providing 335 single-channel 8-bit gray-scale images. These datasets are a public volumetric cerebrovascular system dataset (vesseINN) and our own dataset called KUVESG. The proposed NAS approaches, namely UNAS-Net and Attention UNAS-Net architectures, yield better segmentation performance in terms of different segmentation metrics. More specifically, UNAS-Net with differential evolution reveals high dice score/sensitivity values of 79.57/81.48, respectively. Moreover, they provide shorter inference times by a factor of 9.15 than the baseline methods.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Brain vasculature analysis is critical in developing novel treatment targets for neurodegenerative diseases. Such an accurate analysis cannot be performed manually but requires a semi-automated or fully-automated approach. Deep learning methods have recently proven indispensable for the automated segmentation and analysis of medical images. However, optimizing a deep learning network architecture is another challenge. Manually selecting deep learning network architectures and tuning their hyper-parameters requires a lot of expertise and effort. To solve this problem, neural architecture search (NAS) approaches that explore more efficient network architectures with high segmentation performance have been proposed in the literature. This study introduces differential evolution-based NAS approaches in which a novel search space is proposed for brain vessel segmentation. We select two architectures that are frequently used for medical image segmentation, i.e. U-Net and Attention U-Net, as baselines for NAS optimizations. The conventional differential evolution and the opposition-based differential evolution with novel search space are employed as search methods in NAS. Furthermore, we perform ablation studies and evaluate the effects of specific loss functions, model pruning, threshold selection and generalization performance on the proposed models. The experiments are conducted on two datasets providing 335 single-channel 8-bit gray-scale images. These datasets are a public volumetric cerebrovascular system dataset (vesseINN) and our own dataset called KUVESG. The proposed NAS approaches, namely UNAS-Net and Attention UNAS-Net architectures, yield better segmentation performance in terms of different segmentation metrics. More specifically, UNAS-Net with differential evolution reveals high dice score/sensitivity values of 79.57/81.48, respectively. Moreover, they provide shorter inference times by a factor of 9.15 than the baseline methods.
2563.
Wang, Li; Xie, Tao; Zhang, Xinyu; Jiang, Zhiqiang; Yang, Linqi; Zhang, Haoming; Li, Xiaoyu; Ren, Yilong; Yu, Haiyang; Li, Jun; Liu, Huaping
Auto-Points: Automatic Learning for Point Cloud Analysis with Neural Architecture Search Journal Article
In: IEEE Transactions on Multimedia, pp. 1-16, 2023.
@article{10223431,
title = {Auto-Points: Automatic Learning for Point Cloud Analysis with Neural Architecture Search},
author = {Li Wang and Tao Xie and Xinyu Zhang and Zhiqiang Jiang and Linqi Yang and Haoming Zhang and Xiaoyu Li and Yilong Ren and Haiyang Yu and Jun Li and Huaping Liu},
doi = {10.1109/TMM.2023.3304892},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Multimedia},
pages = {1-16},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2562.
Aach, Marcel; Inanc, Eray; Sarma, Rakesh; Riedel, Morris; Lintermann, Andreas
Optimal Resource Allocation for Early Stopping-based Neural Architecture Search Methods Proceedings Article
In: AutoML Conference 2023, 2023.
@inproceedings{<LineBreak>aach2023optimal,
title = {Optimal Resource Allocation for Early Stopping-based Neural Architecture Search Methods},
author = {Marcel Aach and Eray Inanc and Rakesh Sarma and Morris Riedel and Andreas Lintermann},
url = {https://openreview.net/forum?id=lmtNt–6dw},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {AutoML Conference 2023},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2561.
Xue, Yu; Lu, Changchang; Neri, Ferrante; Qin, Jiafeng
Improved Differentiable Architecture Search With Multi-Stage Progressive Partial Channel Connections Journal Article
In: IEEE Transactions on Emerging Topics in Computational Intelligence, pp. 1-12, 2023.
@article{10223437,
title = {Improved Differentiable Architecture Search With Multi-Stage Progressive Partial Channel Connections},
author = {Yu Xue and Changchang Lu and Ferrante Neri and Jiafeng Qin},
doi = {10.1109/TETCI.2023.3301395},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Emerging Topics in Computational Intelligence},
pages = {1-12},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2560.
Zhang, Xixi; Chen, Xiaofeng; Wang, Yu; Gui, Guan; Adebisi, Bamidele; Sari, Hikmet; Adachi, Fumiyuki
Lightweight Automatic Modulation Classification via Progressive Differentiable Architecture Search Journal Article
In: IEEE Transactions on Cognitive Communications and Networking, pp. 1-1, 2023.
@article{10224342,
title = {Lightweight Automatic Modulation Classification via Progressive Differentiable Architecture Search},
author = {Xixi Zhang and Xiaofeng Chen and Yu Wang and Guan Gui and Bamidele Adebisi and Hikmet Sari and Fumiyuki Adachi},
doi = {10.1109/TCCN.2023.3306391},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Cognitive Communications and Networking},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2559.
Cheng, Ke; Xi, Ning; Liu, Ximeng; Zhu, Xinghui; Gao, Haichang; Zhang, Zhiwei; Shen, Yulong
Private Inference for Deep Neural Networks: A Secure, Adaptive, and Efficient Realization Journal Article
In: IEEE Transactions on Computers, pp. 1-13, 2023.
@article{10224651,
title = {Private Inference for Deep Neural Networks: A Secure, Adaptive, and Efficient Realization},
author = {Ke Cheng and Ning Xi and Ximeng Liu and Xinghui Zhu and Haichang Gao and Zhiwei Zhang and Yulong Shen},
doi = {10.1109/TC.2023.3305754},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Computers},
pages = {1-13},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2558.
Chowdhury, Anjir Ahmed; Mahmud, S. M. Hasan; Hoque, Khadija Kubra Shahjalal; Ahmed, Kawsar; Bui, Francis M.; Lio, Pietro; Moni, Mohammad Ali; Al-Zahrani, Fahad Ahmed
StackFBAs: Detection of fetal brain abnormalities using CNN with stacking strategy from MRI images Journal Article
In: Journal of King Saud University - Computer and Information Sciences, vol. 35, no. 8, pp. 101647, 2023, ISSN: 1319-1578.
@article{CHOWDHURY2023101647b,
title = {StackFBAs: Detection of fetal brain abnormalities using CNN with stacking strategy from MRI images},
author = {Anjir Ahmed Chowdhury and S. M. Hasan Mahmud and Khadija Kubra Shahjalal Hoque and Kawsar Ahmed and Francis M. Bui and Pietro Lio and Mohammad Ali Moni and Fahad Ahmed Al-Zahrani},
url = {https://www.sciencedirect.com/science/article/pii/S131915782300201X},
doi = {https://doi.org/10.1016/j.jksuci.2023.101647},
issn = {1319-1578},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Journal of King Saud University - Computer and Information Sciences},
volume = {35},
number = {8},
pages = {101647},
abstract = {Predicting fetal brain abnormalities (FBAs) is an urgent global problem, as nearly three of every thousand women are pregnant with neurological abnormalities. Therefore, early detection of FBAs using deep learning (DL) can help to enhance the planning and quality of diagnosis and treatment for pregnant women. Most of the research papers focused on brain abnormalities of newborns and premature infants, but fewer studies concentrated on fetuses. This study proposed a deep learning-CNN-based framework named StackFBAs that utilized the stacking strategy to classify fetus brain abnormalities more accurately using MRI images at an early stage. We considered the Greedy-based Neural architecture search (NAS) method to identify the best CNN architectures to solve this problem utilizing brain MRI images. A total of 94 CNN architectures were generated from the NAS method, and the best 5 CNN models were selected to build the baseline models. Subsequently, the probabilistic scores of these baseline models were combined to construct the final meta-model (KNN) utilizing the stacking strategy. The experimental results demonstrated that StackFBAs outperform pre-trained CNN Models (e.g., VGG16, VGG19, ResNet50, DenseNet121, and ResNet152) with transfer learning (TL) and existing models with the 5-fold cross-validation tests. StackFBAs achieved an overall accuracy of 80%, an F1-score of 78%, 76% sensitivity, and a specificity of 78%. Moreover, we employed the federated learning technique that protects sensitive fetal MRI data, combines results, and finds common patterns from many users, making the model more robust for the privacy and security of user-sensitive data. We believe that our novel framework could be used as a helpful tool for detecting brain abnormalities at an early stage.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Predicting fetal brain abnormalities (FBAs) is an urgent global problem, as nearly three of every thousand women are pregnant with neurological abnormalities. Therefore, early detection of FBAs using deep learning (DL) can help to enhance the planning and quality of diagnosis and treatment for pregnant women. Most of the research papers focused on brain abnormalities of newborns and premature infants, but fewer studies concentrated on fetuses. This study proposed a deep learning-CNN-based framework named StackFBAs that utilized the stacking strategy to classify fetus brain abnormalities more accurately using MRI images at an early stage. We considered the Greedy-based Neural architecture search (NAS) method to identify the best CNN architectures to solve this problem utilizing brain MRI images. A total of 94 CNN architectures were generated from the NAS method, and the best 5 CNN models were selected to build the baseline models. Subsequently, the probabilistic scores of these baseline models were combined to construct the final meta-model (KNN) utilizing the stacking strategy. The experimental results demonstrated that StackFBAs outperform pre-trained CNN Models (e.g., VGG16, VGG19, ResNet50, DenseNet121, and ResNet152) with transfer learning (TL) and existing models with the 5-fold cross-validation tests. StackFBAs achieved an overall accuracy of 80%, an F1-score of 78%, 76% sensitivity, and a specificity of 78%. Moreover, we employed the federated learning technique that protects sensitive fetal MRI data, combines results, and finds common patterns from many users, making the model more robust for the privacy and security of user-sensitive data. We believe that our novel framework could be used as a helpful tool for detecting brain abnormalities at an early stage.
2557.
Chen, Wuyang; Huang, Wei; Wang, Zhangyang
“No Free Lunch” in Neural Architectures? A Joint Analysis of Expressivity, Convergence, and Generalization Proceedings Article
In: AutoML Conference 2023, 2023.
@inproceedings{<LineBreak>chen2023no,
title = {“No Free Lunch” in Neural Architectures? A Joint Analysis of Expressivity, Convergence, and Generalization},
author = {Wuyang Chen and Wei Huang and Zhangyang Wang},
url = {https://openreview.net/forum?id=EMys3eIDJ2},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {AutoML Conference 2023},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2556.
Aichberger, Lukas; Klambauer, Günter
ELENAS: Elementary Neural Architecture Search Proceedings Article
In: AutoML Conference 2023, 2023.
@inproceedings{<LineBreak>aichberger2023elenas,
title = {ELENAS: Elementary Neural Architecture Search},
author = {Lukas Aichberger and Günter Klambauer},
url = {https://openreview.net/forum?id=1tZY0La5GFRp},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {AutoML Conference 2023},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2555.
Chen, Shiming; Chen, Shihuang; Hou, Wenjin; Ding, Weiping; You, Xinge
EGANS: Evolutionary Generative Adversarial Network Search for Zero-Shot Learning Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2308-09915,
title = {EGANS: Evolutionary Generative Adversarial Network Search for Zero-Shot Learning},
author = {Shiming Chen and Shihuang Chen and Wenjin Hou and Weiping Ding and Xinge You},
url = {https://doi.org/10.48550/arXiv.2308.09915},
doi = {10.48550/arXiv.2308.09915},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2308.09915},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2554.
Meyer-Lee, Gabriel; Cheney, Nick
On the selection of neural architectures from a supernet Proceedings Article
In: AutoML Conference 2023, 2023.
@inproceedings{<LineBreak>meyer-lee2023on,
title = {On the selection of neural architectures from a supernet},
author = {Gabriel Meyer-Lee and Nick Cheney},
url = {https://openreview.net/forum?id=XM_v85teqN},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {AutoML Conference 2023},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2553.
Lukasik, Jovita; Geiping, Jonas; Moeller, Michael; Keuper, Margret
Differentiable Architecture Search: a One-Shot Method? Proceedings Article
In: AutoML Conference 2023, 2023.
@inproceedings{<LineBreak>lukasik2023differentiable,
title = {Differentiable Architecture Search: a One-Shot Method?},
author = {Jovita Lukasik and Jonas Geiping and Michael Moeller and Margret Keuper},
url = {https://openreview.net/forum?id=LV-5kHj-uV5},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {AutoML Conference 2023},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2552.
Roshtkhari, Mehraveh Javan; Toews, Matthew; Pedersoli, Marco
Balanced Mixture of Supernets for Learning the CNN Pooling Architecture Proceedings Article
In: AutoML Conference 2023, 2023.
@inproceedings{<LineBreak>roshtkhari2023balanced,
title = {Balanced Mixture of Supernets for Learning the CNN Pooling Architecture},
author = {Mehraveh Javan Roshtkhari and Matthew Toews and Marco Pedersoli},
url = {https://openreview.net/forum?id=8-8k3okjpY},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {AutoML Conference 2023},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2551.
Dimanov, Daniel; Singleton, Colin; Rostami, Shahin; Balaguer-Ballester, Emili
MEOW - Multi-Objective Evolutionary Weapon Detection Proceedings Article
In: AutoML Conference 2023, 2023.
@inproceedings{<LineBreak>dimanov2023meow,
title = {MEOW - Multi-Objective Evolutionary Weapon Detection},
author = {Daniel Dimanov and Colin Singleton and Shahin Rostami and Emili Balaguer-Ballester},
url = {https://openreview.net/forum?id=Eyzx7rDo-JNh},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {AutoML Conference 2023},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2550.
Carmichael, Zachariah J; Moon, Tim; Jacobs, Sam Ade
Learning Debuggable Models Through Multi-Objective NAS Proceedings Article
In: AutoML Conference 2023, 2023.
@inproceedings{<LineBreak>carmichael2023learning,
title = {Learning Debuggable Models Through Multi-Objective NAS},
author = {Zachariah J Carmichael and Tim Moon and Sam Ade Jacobs},
url = {https://openreview.net/forum?id=AwL9ZZOPVlN},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {AutoML Conference 2023},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2549.
Yoshihama, Yutaka; Yadani, Kenichi; Isobe, Shota
Hardware-Aware Zero-Shot Neural Architecture Search Proceedings Article
In: 2023 18th International Conference on Machine Vision and Applications (MVA), pp. 1-5, 2023.
@inproceedings{10216205,
title = {Hardware-Aware Zero-Shot Neural Architecture Search},
author = {Yutaka Yoshihama and Kenichi Yadani and Shota Isobe},
url = {https://ieeexplore.ieee.org/abstract/document/10216205},
doi = {10.23919/MVA57639.2023.10216205},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {2023 18th International Conference on Machine Vision and Applications (MVA)},
pages = {1-5},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2548.
Zhang, Ruohan; Jiao, Licheng; Wang, Dan; Liu, Fang; Liu, Xu; Yang, Shuyuan
A Fast Evolutionary Knowledge Transfer Search for Multiscale Deep Neural Architecture Journal Article
In: IEEE Transactions on Neural Networks and Learning Systems, pp. 1-15, 2023.
@article{10227743,
title = {A Fast Evolutionary Knowledge Transfer Search for Multiscale Deep Neural Architecture},
author = {Ruohan Zhang and Licheng Jiao and Dan Wang and Fang Liu and Xu Liu and Shuyuan Yang},
url = {https://ieeexplore.ieee.org/abstract/document/10227743},
doi = {10.1109/TNNLS.2023.3304291},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Neural Networks and Learning Systems},
pages = {1-15},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2547.
Zhang, Rui; Gao, Mei-Rong; Zhang, Peng-Yun; Zhang, Yong-Mei; Fu, Liu-Hu; Chai, Yan-Feng
Research on an ultrasonic detection method for weld defects based on neural network architecture search Journal Article
In: Measurement, vol. 221, pp. 113483, 2023, ISSN: 0263-2241.
@article{ZHANG2023113483,
title = {Research on an ultrasonic detection method for weld defects based on neural network architecture search},
author = {Rui Zhang and Mei-Rong Gao and Peng-Yun Zhang and Yong-Mei Zhang and Liu-Hu Fu and Yan-Feng Chai},
url = {https://www.sciencedirect.com/science/article/pii/S0263224123010473},
doi = {https://doi.org/10.1016/j.measurement.2023.113483},
issn = {0263-2241},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Measurement},
volume = {221},
pages = {113483},
abstract = {In order to further reduce the subjectivity of network design and improve the ability of model feature extraction, an ultrasonic detection method for weld defects based on neural network architecture search is proposed. Through the designed multi-level and multi-branch search space and an untrained architecture search and evaluation method, an efficient defect classification network was automatically constructed to complete the task of weld defect classification. Experiments were carried out on a self-constructed data set, and compared with the manually designed model, the classification accuracy of defect types reached 95.26% when the number of parameters was only 7.3 M. Compared with the model constructed using neural network architecture search, the proposed method can reduce the searching time to 8.29% of the baseline model while weighing multiple conflicting objectives, which proved the efficiency and effectiveness of the proposed method.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
In order to further reduce the subjectivity of network design and improve the ability of model feature extraction, an ultrasonic detection method for weld defects based on neural network architecture search is proposed. Through the designed multi-level and multi-branch search space and an untrained architecture search and evaluation method, an efficient defect classification network was automatically constructed to complete the task of weld defect classification. Experiments were carried out on a self-constructed data set, and compared with the manually designed model, the classification accuracy of defect types reached 95.26% when the number of parameters was only 7.3 M. Compared with the model constructed using neural network architecture search, the proposed method can reduce the searching time to 8.29% of the baseline model while weighing multiple conflicting objectives, which proved the efficiency and effectiveness of the proposed method.
2546.
Luo, Erjing; Huang, Haitong; Liu, Cheng; Li, Guoyu; Yang, Bing; Wang, Ying; Li, Huawei; Li, Xiaowei
DeepBurning-MixQ: An Open Source Mixed-Precision Neural Network Accelerator Design Framework for FPGAs Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2308-11334,
title = {DeepBurning-MixQ: An Open Source Mixed-Precision Neural Network Accelerator Design Framework for FPGAs},
author = {Erjing Luo and Haitong Huang and Cheng Liu and Guoyu Li and Bing Yang and Ying Wang and Huawei Li and Xiaowei Li},
url = {https://doi.org/10.48550/arXiv.2308.11334},
doi = {10.48550/arXiv.2308.11334},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2308.11334},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2545.
Deng, Xinchi; Shi, Han; Huang, Runhui; Li, Changlin; Xu, Hang; Han, Jianhua; Kwok, James T.; Zhao, Shen; Zhang, Wei; Liang, Xiaodan
GrowCLIP: Data-aware Automatic Model Growing for Large-scale Contrastive Language-Image Pre-training Journal Article
In: CoRR, vol. abs/2308.11331, 2023.
@article{DBLP:journals/corr/abs-2308-11331,
title = {GrowCLIP: Data-aware Automatic Model Growing for Large-scale Contrastive Language-Image Pre-training},
author = {Xinchi Deng and Han Shi and Runhui Huang and Changlin Li and Hang Xu and Jianhua Han and James T. Kwok and Shen Zhao and Wei Zhang and Xiaodan Liang},
url = {https://doi.org/10.48550/arXiv.2308.11331},
doi = {10.48550/arXiv.2308.11331},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2308.11331},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2544.
Wu, Jiong; Fan, Yong
HNAS-Reg: Hierarchical Neural Architecture Search for Deformable Medical Image Registration Proceedings Article
In: 20th IEEE International Symposium on Biomedical Imaging, ISBI 2023, Cartagena, Colombia, April 18-21, 2023, pp. 1–4, IEEE, 2023.
@inproceedings{DBLP:conf/isbi/WuF23,
title = {HNAS-Reg: Hierarchical Neural Architecture Search for Deformable Medical Image Registration},
author = {Jiong Wu and Yong Fan},
url = {https://doi.org/10.1109/ISBI53787.2023.10230534},
doi = {10.1109/ISBI53787.2023.10230534},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {20th IEEE International Symposium on Biomedical Imaging, ISBI
2023, Cartagena, Colombia, April 18-21, 2023},
pages = {1–4},
publisher = {IEEE},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2543.
Zhang, Dongran; Luo, Gang; Li, Jun
Traffic Spatial-Temporal Prediction Based on Neural Architecture Search Proceedings Article
In: Proceedings of the 18th International Symposium on Spatial and Temporal Data, pp. 21–30, Association for Computing Machinery, Calgary, AB, Canada, 2023, ISBN: 9798400708992.
@inproceedings{10.1145/3609956.3609962,
title = {Traffic Spatial-Temporal Prediction Based on Neural Architecture Search},
author = {Dongran Zhang and Gang Luo and Jun Li},
url = {https://doi.org/10.1145/3609956.3609962},
doi = {10.1145/3609956.3609962},
isbn = {9798400708992},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Proceedings of the 18th International Symposium on Spatial and Temporal Data},
pages = {21–30},
publisher = {Association for Computing Machinery},
address = {Calgary, AB, Canada},
series = {SSTD '23},
abstract = {Traffic spatial-temporal prediction is essential for intelligent transportation systems. However, the current approach relies heavily on expert knowledge and time-consuming manual modeling. Neural architecture search can build models adaptively, but it is rarely used for traffic spatial-temporal prediction, nor is it designed specifically for traffic spatial-temporal feature. In response to the above problems, we propose neural architecture search spatial-temporal prediction (NASST), which is a method to automatically generate a traffic spatial-temporal prediction network by performing a differentiable neural network architecture search in an optimized search space. First, we adopt a differentiable neural architecture search method to continuously relax the discrete traffic spatial-temporal prediction model architecture search, and adopt a fusion strategy of comprehensive concatenate and addition (CA) to achieve efficient neural architecture search. Second, we optimize the search space and introduce a series of classic traffic spatial-temporal feature extraction modules, which are more in line with the architectural requirements of traffic spatial-temporal prediction network. Finally, our model is validated on two public traffic datasets and achieves the best predictions. Compared with traditional manual modeling methods, our method can realize the automatic search of high-precision predictive model architectures, which improves the modeling efficiency.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Traffic spatial-temporal prediction is essential for intelligent transportation systems. However, the current approach relies heavily on expert knowledge and time-consuming manual modeling. Neural architecture search can build models adaptively, but it is rarely used for traffic spatial-temporal prediction, nor is it designed specifically for traffic spatial-temporal feature. In response to the above problems, we propose neural architecture search spatial-temporal prediction (NASST), which is a method to automatically generate a traffic spatial-temporal prediction network by performing a differentiable neural network architecture search in an optimized search space. First, we adopt a differentiable neural architecture search method to continuously relax the discrete traffic spatial-temporal prediction model architecture search, and adopt a fusion strategy of comprehensive concatenate and addition (CA) to achieve efficient neural architecture search. Second, we optimize the search space and introduce a series of classic traffic spatial-temporal feature extraction modules, which are more in line with the architectural requirements of traffic spatial-temporal prediction network. Finally, our model is validated on two public traffic datasets and achieves the best predictions. Compared with traditional manual modeling methods, our method can realize the automatic search of high-precision predictive model architectures, which improves the modeling efficiency.
2542.
Meyer-Lee, Gabriel; Cheney, Nick
Evaluating supernets for neural architecture search Proceedings Article
In: AutoML Conference 2023 (Workshop), 2023.
@inproceedings{<LineBreak>meyer-lee2023evaluating,
title = {Evaluating supernets for neural architecture search},
author = {Gabriel Meyer-Lee and Nick Cheney},
url = {https://openreview.net/forum?id=13BSG9cwvu},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {AutoML Conference 2023 (Workshop)},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2541.
Wei-ming, LI Chun-chun YANG Tie-jun [DENG
Object detection for nameplate based on neural architecture search Journal Article
In: Journal of Graphics, vol. 44, no. 4, pp. 718-727, 2023.
@article{DENGWei-ming_718,
title = {Object detection for nameplate based on neural architecture search},
author = {LI Chun-chun YANG Tie-jun [DENG Wei-ming},
doi = {http://www.txxb.com.cn/EN/10.11996/JG.j.2095-302X.2023040718},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Journal of Graphics},
volume = {44},
number = {4},
pages = {718-727},
publisher = {Journal of Graphics},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2540.
Fu, Jian; Wang, Qifeng
Study of DNN Network Architecture Search for Robot Vision Proceedings Article
In: 2023 International Conference on Advanced Robotics and Mechatronics (ICARM), pp. 366-372, 2023.
@inproceedings{10218405,
title = {Study of DNN Network Architecture Search for Robot Vision},
author = {Jian Fu and Qifeng Wang},
doi = {10.1109/ICARM58088.2023.10218405},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {2023 International Conference on Advanced Robotics and Mechatronics (ICARM)},
pages = {366-372},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2539.
Xiang, Lichuan; Dudziak, Łukasz; Mehrotra, Abhinav; Abdelfattah, Mohamed S; Lane, Nicholas Donald; Wen, Hongkai
Generating Neural Network Architectures with Conditional Graph Normalizing Flows Proceedings Article
In: AutoML Conference 2023 (Workshop), 2023.
@inproceedings{<LineBreak>xiang2023generating,
title = {Generating Neural Network Architectures with Conditional Graph Normalizing Flows},
author = {Lichuan Xiang and Łukasz Dudziak and Abhinav Mehrotra and Mohamed S Abdelfattah and Nicholas Donald Lane and Hongkai Wen},
url = {https://openreview.net/forum?id=pwilwGCwPQ},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {AutoML Conference 2023 (Workshop)},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2538.
Ding, Chenchen; Ren, Hongwei; Guo, Zhiru; Bi, Minjie; Man, Changhai; Wang, Tingting; Li, Shuwei; Luo, Shaobo; Zhang, Rumin; Yu, Hao
TT-LCD: Tensorized-Transformer based Loop Closure Detection for Robotic Visual SLAM on Edge Proceedings Article
In: 2023 International Conference on Advanced Robotics and Mechatronics (ICARM), pp. 166-172, 2023.
@inproceedings{10218828,
title = {TT-LCD: Tensorized-Transformer based Loop Closure Detection for Robotic Visual SLAM on Edge},
author = {Chenchen Ding and Hongwei Ren and Zhiru Guo and Minjie Bi and Changhai Man and Tingting Wang and Shuwei Li and Shaobo Luo and Rumin Zhang and Hao Yu},
doi = {10.1109/ICARM58088.2023.10218828},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {2023 International Conference on Advanced Robotics and Mechatronics (ICARM)},
pages = {166-172},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2537.
Wu, Ruoyou; Li, Cheng; Zou, Juan; Wang, Shanshan
Generalizable Learning Reconstruction for Accelerating MR Imaging via Federated Neural Architecture Search Technical Report
2023.
@techreport{wu2023generalizable,
title = {Generalizable Learning Reconstruction for Accelerating MR Imaging via Federated Neural Architecture Search},
author = {Ruoyou Wu and Cheng Li and Juan Zou and Shanshan Wang},
url = {https://arxiv.org/abs/2308.13995},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2536.
Kuş, Zeki; Kiraz, Berna
BaDENAS: Bayesian Based Neural Architecture Search for Retinal Vessel Segmentation Proceedings Article
In: 2023 31st Signal Processing and Communications Applications Conference (SIU), pp. 1-4, 2023.
@inproceedings{10223862,
title = {BaDENAS: Bayesian Based Neural Architecture Search for Retinal Vessel Segmentation},
author = {Zeki Kuş and Berna Kiraz},
url = {https://ieeexplore.ieee.org/abstract/document/10223862},
doi = {10.1109/SIU59756.2023.10223862},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {2023 31st Signal Processing and Communications Applications Conference (SIU)},
pages = {1-4},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2535.
Selg, Hardi; Jenihhin, Maksim; Ellervee, Peeter; Raik, Jaan
ML-Based Online Design Error Localization for RISC-V Implementations Proceedings Article
In: 2023 IEEE 29th International Symposium on On-Line Testing and Robust System Design (IOLTS), pp. 1-7, 2023.
@inproceedings{10224864,
title = {ML-Based Online Design Error Localization for RISC-V Implementations},
author = {Hardi Selg and Maksim Jenihhin and Peeter Ellervee and Jaan Raik},
url = {https://ieeexplore.ieee.org/abstract/document/10224864},
doi = {10.1109/IOLTS59296.2023.10224864},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {2023 IEEE 29th International Symposium on On-Line Testing and Robust System Design (IOLTS)},
pages = {1-7},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2534.
Sasaki, Yuya
Efficient and Explainable Graph Neural Architecture Search via Monte-Carlo Tree Search Technical Report
2023.
@techreport{sasaki2023efficient,
title = {Efficient and Explainable Graph Neural Architecture Search via Monte-Carlo Tree Search},
author = {Yuya Sasaki},
url = {https://arxiv.org/abs/2308.15734},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2533.
Sridhar, Sharath Nittur; Kundu, Souvik; Sundaresan, Sairam; Szankin, Maciej; Sarah, Anthony
InstaTune: Instantaneous Neural Architecture Search During Fine-Tuning Technical Report
2023.
@techreport{sridhar2023instatune,
title = {InstaTune: Instantaneous Neural Architecture Search During Fine-Tuning},
author = {Sharath Nittur Sridhar and Souvik Kundu and Sairam Sundaresan and Maciej Szankin and Anthony Sarah},
url = {https://arxiv.org/abs/2308.15609},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2532.
Gao, Jianliang; He, Changlong; Chen, Jiamin; Li, Qiutong; Wang, Yili
Decoupled Graph Neural Architecture Search with Variable Propagation Operation and Appropriate Depth Proceedings Article
In: Proceedings of the 35th International Conference on Scientific and Statistical Database Management, Association for Computing Machinery, Los Angeles, CA, USA, 2023, ISBN: 9798400707469.
@inproceedings{10.1145/3603719.3603729,
title = {Decoupled Graph Neural Architecture Search with Variable Propagation Operation and Appropriate Depth},
author = {Jianliang Gao and Changlong He and Jiamin Chen and Qiutong Li and Yili Wang},
url = {https://doi.org/10.1145/3603719.3603729},
doi = {10.1145/3603719.3603729},
isbn = {9798400707469},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Proceedings of the 35th International Conference on Scientific and Statistical Database Management},
publisher = {Association for Computing Machinery},
address = {Los Angeles, CA, USA},
series = {SSDBM '23},
abstract = {To alleviate the over-smoothing problem caused by deep graph neural networks, decoupled graph neural networks (DGNNs) are proposed. DGNNs decouple the graph neural network into two atomic operations, the propagation (P) operation and the transformation (T) operation. Since manually designing the architecture of DGNNs is a time-consuming and expert-dependent process, the DF-GNAS method is designed, which can automatically construct the architecture of DGNNs with fixed propagation operation and deep layers. The propagation operation is a key process for DGNNs to aggregate graph structure information. However, DF-GNAS automatically designs DGNN architecture using fixed propagation operation for different graph structures will cause performance loss. Meanwhile, DF-GNAS designs deep DGNNs for graphs with simple distributions, which may lead to overfitting problems. To solve the above challenges, we propose the Decoupled Graph Neural Architecture Search with Variable Propagation Operation and Appropriate Depth (DGNAS-PD) method. In DGNAS-PD, we design a DGNN operation space with variable efficient propagation operations in order to better aggregate information on different graph structures. We build an effective genetic search strategy to adaptively design appropriate DGNN depths instead of deep DGNNs for the graph with simple distributions in DGNAS-PD. The experiments on five real-world graphs show that DGNAS-PD outperforms state-of-art baseline methods.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
To alleviate the over-smoothing problem caused by deep graph neural networks, decoupled graph neural networks (DGNNs) are proposed. DGNNs decouple the graph neural network into two atomic operations, the propagation (P) operation and the transformation (T) operation. Since manually designing the architecture of DGNNs is a time-consuming and expert-dependent process, the DF-GNAS method is designed, which can automatically construct the architecture of DGNNs with fixed propagation operation and deep layers. The propagation operation is a key process for DGNNs to aggregate graph structure information. However, DF-GNAS automatically designs DGNN architecture using fixed propagation operation for different graph structures will cause performance loss. Meanwhile, DF-GNAS designs deep DGNNs for graphs with simple distributions, which may lead to overfitting problems. To solve the above challenges, we propose the Decoupled Graph Neural Architecture Search with Variable Propagation Operation and Appropriate Depth (DGNAS-PD) method. In DGNAS-PD, we design a DGNN operation space with variable efficient propagation operations in order to better aggregate information on different graph structures. We build an effective genetic search strategy to adaptively design appropriate DGNN depths instead of deep DGNNs for the graph with simple distributions in DGNAS-PD. The experiments on five real-world graphs show that DGNAS-PD outperforms state-of-art baseline methods.
2531.
Li, Jialin; Cao, Xuan; Chen, Renxiang; Zhang, Xia; Huang, Xianzhen; Qu, Yongzhi
Graph neural network architecture search for rotating machinery fault diagnosis based on reinforcement learning Journal Article
In: Mechanical Systems and Signal Processing, vol. 202, pp. 110701, 2023, ISSN: 0888-3270.
@article{LI2023110701,
title = {Graph neural network architecture search for rotating machinery fault diagnosis based on reinforcement learning},
author = {Jialin Li and Xuan Cao and Renxiang Chen and Xia Zhang and Xianzhen Huang and Yongzhi Qu},
url = {https://www.sciencedirect.com/science/article/pii/S088832702300609X},
doi = {https://doi.org/10.1016/j.ymssp.2023.110701},
issn = {0888-3270},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Mechanical Systems and Signal Processing},
volume = {202},
pages = {110701},
abstract = {In order to improve the accuracy of fault diagnosis, researchers are constantly trying to develop new diagnostic models. However, limited by the inherent thinking of human beings, it has always been difficult to build a pioneering architecture for rotating machinery fault diagnosis. In order to solve this problem, this paper uses reinforcement learning algorithm based on adjacency matrix to carry out network architecture search (NAS) of rotating machinery fault diagnosis model. A reinforcement learning agent for deep deterministic policy gradient (DDPG) is developed based on actor–critic neural networks. The observation state of reinforcement learning is used to develop the graph neural network (GNN) diagnosis model, and the diagnosis accuracy is fed back to the agent as a reward for updating the reinforcement learning parameters. The MFPT bearing fault datasets and the developed gear pitting fault experimental data are used to validate the proposed network architecture search method based on reinforcement learning (RL-NAS). The proposed method is proved to be practical and effective in various aspects such as fault diagnosis ability, search space, search efficiency and multi-working condition performance.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
In order to improve the accuracy of fault diagnosis, researchers are constantly trying to develop new diagnostic models. However, limited by the inherent thinking of human beings, it has always been difficult to build a pioneering architecture for rotating machinery fault diagnosis. In order to solve this problem, this paper uses reinforcement learning algorithm based on adjacency matrix to carry out network architecture search (NAS) of rotating machinery fault diagnosis model. A reinforcement learning agent for deep deterministic policy gradient (DDPG) is developed based on actor–critic neural networks. The observation state of reinforcement learning is used to develop the graph neural network (GNN) diagnosis model, and the diagnosis accuracy is fed back to the agent as a reward for updating the reinforcement learning parameters. The MFPT bearing fault datasets and the developed gear pitting fault experimental data are used to validate the proposed network architecture search method based on reinforcement learning (RL-NAS). The proposed method is proved to be practical and effective in various aspects such as fault diagnosis ability, search space, search efficiency and multi-working condition performance.
2530.
Dai, Haixing
Brain-inspired Approaches for Advancing Artificial Intelligence PhD Thesis
University of Georgia, 2023.
@phdthesis{DaiHaixing2023BAfA,
title = {Brain-inspired Approaches for Advancing Artificial Intelligence},
author = {Haixing Dai},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
pages = {197},
school = {University of Georgia},
abstract = {Deep learning has experienced rapid growth and garnered significant attention in recent decades. Simultaneously, neuroscience has remained a challenging and enigmatic field of study. Inspired by the structure and function of the brain, researchers have developed increasingly powerful and sophisticated deep learning models that have achieved remarkable performance in various domains, including computer vision, natural language processing, and medical image analysis. These brain-inspired models have revolutionized the field of artificial intelligence, enabling breakthroughs in tasks such as image recognition, language understanding, and disease diagnosis. In turn, the application of these advanced deep learning models has provided valuable insights into the inner workings of the human brain, revealing temporal and spatial functional brain networks. The symbiotic relationship between artificial intelligence and neuroscience is evident, as they continuously inform and complement each other's progress.This dissertation presents novel frameworks that integrate deep learning and knowledge from brain science. This research aims to gain insights into the brain and refine deep learning models through brain-inspired principles. The dissertation first discusses how deep learning has been applied to study the brain, focusing on areas such as modeling cortical folding patterns, hierarchical brain structures, and spatial-temporal brain networks. It then discusses how artificial neural networks have drawn inspiration from the brain, using examples like convolutional neural networks, attention mechanisms, and language models. The dissertation’s main contributions are several computational frameworks integrating brain-inspired insights. These include a graph representation neural architecture search method to optimize recurrent neural networks for analyzing spatiotemporal brain networks, a hierarchical semantic tree concept whitening model to disentangle concept representations for image classification, a twin-transformer framework to study gyri and sulci in the cortex, a core-periphery guided vision transformer, and methods leveraging language models to generate data and analyze health narratives. Overall, this dissertation explores how we can understand the brain better using deep learning and ultimately build more efficient, robust, and interpretable artificial neural networks inspired by the brain.},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
Deep learning has experienced rapid growth and garnered significant attention in recent decades. Simultaneously, neuroscience has remained a challenging and enigmatic field of study. Inspired by the structure and function of the brain, researchers have developed increasingly powerful and sophisticated deep learning models that have achieved remarkable performance in various domains, including computer vision, natural language processing, and medical image analysis. These brain-inspired models have revolutionized the field of artificial intelligence, enabling breakthroughs in tasks such as image recognition, language understanding, and disease diagnosis. In turn, the application of these advanced deep learning models has provided valuable insights into the inner workings of the human brain, revealing temporal and spatial functional brain networks. The symbiotic relationship between artificial intelligence and neuroscience is evident, as they continuously inform and complement each other's progress.This dissertation presents novel frameworks that integrate deep learning and knowledge from brain science. This research aims to gain insights into the brain and refine deep learning models through brain-inspired principles. The dissertation first discusses how deep learning has been applied to study the brain, focusing on areas such as modeling cortical folding patterns, hierarchical brain structures, and spatial-temporal brain networks. It then discusses how artificial neural networks have drawn inspiration from the brain, using examples like convolutional neural networks, attention mechanisms, and language models. The dissertation’s main contributions are several computational frameworks integrating brain-inspired insights. These include a graph representation neural architecture search method to optimize recurrent neural networks for analyzing spatiotemporal brain networks, a hierarchical semantic tree concept whitening model to disentangle concept representations for image classification, a twin-transformer framework to study gyri and sulci in the cortex, a core-periphery guided vision transformer, and methods leveraging language models to generate data and analyze health narratives. Overall, this dissertation explores how we can understand the brain better using deep learning and ultimately build more efficient, robust, and interpretable artificial neural networks inspired by the brain.
2529.
Shariatzadeh, Seyed Mahdi; Fathy, Mahmood; Berangi, Reza
Multi-objective single-shot neural architecture search via efficient convolutional filters Journal Article
In: Electronics Letters, vol. 59, no. 17, pp. e12939, 2023.
@article{https://doi.org/10.1049/ell2.12939,
title = {Multi-objective single-shot neural architecture search via efficient convolutional filters},
author = {Seyed Mahdi Shariatzadeh and Mahmood Fathy and Reza Berangi},
url = {https://ietresearch.onlinelibrary.wiley.com/doi/abs/10.1049/ell2.12939},
doi = {https://doi.org/10.1049/ell2.12939},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Electronics Letters},
volume = {59},
number = {17},
pages = {e12939},
abstract = {Abstract This paper presents a novel approach for fast neural architecture search (NAS) in Convolutional Neural Networks (CNNs) for end-to-end License Plate Recognition (LPR). The authors propose a one-shot schema that considers the efficiency of different convolutional filters to create a search space for more efficient architectures on vector processing cores. The authors’ approach utilizes a super-network for LPR using Connectionist-Temporal-Cost (CTC) and ranks the importance of filters to generate a fine-grain list of architectures. These architectures are evaluated in a multi-objective manner, resulting in several Pareto-optimal architectures with different computational costs and validation errors. Rather than using a single complicated building block for all layers, the authors’ method allows each stage to select a custom building block with fewer or more operations. The authors show that their super-network is flexible to calculate filters of any required size and stride in each stage while keeping it efficient by the structural pruning. The authors’ experiments, which were performed on Iranian LPR, demonstrate that this method produces a variety of fast and efficient CNNs. Furthermore, the authors discuss the potential of this method for use in other areas of CNN application.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Abstract This paper presents a novel approach for fast neural architecture search (NAS) in Convolutional Neural Networks (CNNs) for end-to-end License Plate Recognition (LPR). The authors propose a one-shot schema that considers the efficiency of different convolutional filters to create a search space for more efficient architectures on vector processing cores. The authors’ approach utilizes a super-network for LPR using Connectionist-Temporal-Cost (CTC) and ranks the importance of filters to generate a fine-grain list of architectures. These architectures are evaluated in a multi-objective manner, resulting in several Pareto-optimal architectures with different computational costs and validation errors. Rather than using a single complicated building block for all layers, the authors’ method allows each stage to select a custom building block with fewer or more operations. The authors show that their super-network is flexible to calculate filters of any required size and stride in each stage while keeping it efficient by the structural pruning. The authors’ experiments, which were performed on Iranian LPR, demonstrate that this method produces a variety of fast and efficient CNNs. Furthermore, the authors discuss the potential of this method for use in other areas of CNN application.
2528.
Le, Minh; Nguyen, Nhan; Luong, Ngoc Hoang
Efficacy of Neural Prediction-Based NAS for Zero-Shot NAS Paradigm Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2308-16775,
title = {Efficacy of Neural Prediction-Based NAS for Zero-Shot NAS Paradigm},
author = {Minh Le and Nhan Nguyen and Ngoc Hoang Luong},
url = {https://doi.org/10.48550/arXiv.2308.16775},
doi = {10.48550/arXiv.2308.16775},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2308.16775},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2527.
Odema, Mohanad; Faruque, Mohammad Abdullah Al
PrivyNAS: Privacy-Aware Neural Architecture Search for Split Computing in Edge-Cloud Systems Journal Article
In: IEEE Internet of Things Journal, pp. 1-1, 2023.
@article{10239258,
title = {PrivyNAS: Privacy-Aware Neural Architecture Search for Split Computing in Edge-Cloud Systems},
author = {Mohanad Odema and Mohammad Abdullah Al Faruque},
url = {https://ieeexplore.ieee.org/document/10239258},
doi = {10.1109/JIOT.2023.3311761},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Internet of Things Journal},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}