Maintained by Difan Deng and Marius Lindauer.
The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.
Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.
Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.
2023
2731.
Zhang, Zijun; Lamson, Adam R.; Shelley, Michael; Troyanskaya, Olga G.
Interpretable neural architecture search and transfer learning for understanding sequence dependent enzymatic reactions Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2305-11917,
title = {Interpretable neural architecture search and transfer learning for understanding sequence dependent enzymatic reactions},
author = {Zijun Zhang and Adam R. Lamson and Michael Shelley and Olga G. Troyanskaya},
url = {https://doi.org/10.48550/arXiv.2305.11917},
doi = {10.48550/arXiv.2305.11917},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2305.11917},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2730.
Liu, Zhu; Liu, Jinyuan; Wu, Guanyao; Fan, Xin; Liu, Risheng
Embracing Compact and Robust Architectures for Multi-Exposure Image Fusion Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2305-12236,
title = {Embracing Compact and Robust Architectures for Multi-Exposure Image Fusion},
author = {Zhu Liu and Jinyuan Liu and Guanyao Wu and Xin Fan and Risheng Liu},
url = {https://doi.org/10.48550/arXiv.2305.12236},
doi = {10.48550/arXiv.2305.12236},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2305.12236},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2729.
Zhang, Ya-Lin; Zhou, Jun; Ren, Yankun; Zhang, Yue; Yang, Xinxing; Li, Meng; Shi, Qitao; Li, Longfei
ALT: An Automatic System for Long Tail Scenario Modeling Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2305-11390,
title = {ALT: An Automatic System for Long Tail Scenario Modeling},
author = {Ya-Lin Zhang and Jun Zhou and Yankun Ren and Yue Zhang and Xinxing Yang and Meng Li and Qitao Shi and Longfei Li},
url = {https://doi.org/10.48550/arXiv.2305.11390},
doi = {10.48550/arXiv.2305.11390},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2305.11390},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2728.
Xu, Peng; Zhang, Lin; Liu, Xuanzhou; Sun, Jiaqi; Zhao, Yue; Yang, Haiqing; Yu, Bei
Do Not Train It: A Linear Neural Architecture Search of Graph Neural Networks Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2305-14065,
title = {Do Not Train It: A Linear Neural Architecture Search of Graph Neural Networks},
author = {Peng Xu and Lin Zhang and Xuanzhou Liu and Jiaqi Sun and Yue Zhao and Haiqing Yang and Bei Yu},
url = {https://doi.org/10.48550/arXiv.2305.14065},
doi = {10.48550/arXiv.2305.14065},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2305.14065},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2727.
Ye, Zhen; Xue, Wei; Tan, Xu; Liu, Qifeng; Guo, Yike
NAS-FM: Neural Architecture Search for Tunable and Interpretable Sound Synthesis based on Frequency Modulation Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2305-12868,
title = {NAS-FM: Neural Architecture Search for Tunable and Interpretable Sound Synthesis based on Frequency Modulation},
author = {Zhen Ye and Wei Xue and Xu Tan and Qifeng Liu and Yike Guo},
url = {https://doi.org/10.48550/arXiv.2305.12868},
doi = {10.48550/arXiv.2305.12868},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2305.12868},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2726.
Yin, Kangyong; Huang, Zhechen; Qiu, Suo; Wang, Zhi; Tao, Fengbo; Liang, Wei; Huang, Haosheng
An action recognition method based on neural architecture search Proceedings Article
In: Wang, Hongzhi; Kong, Xiangjie (Ed.): International Conference on Internet of Things and Machine Learning (IoTML 2022), pp. 126401Q, International Society for Optics and Photonics SPIE, 2023.
@inproceedings{10.1117/12.2673723,
title = {An action recognition method based on neural architecture search},
author = {Kangyong Yin and Zhechen Huang and Suo Qiu and Zhi Wang and Fengbo Tao and Wei Liang and Haosheng Huang},
editor = {Hongzhi Wang and Xiangjie Kong},
url = {https://doi.org/10.1117/12.2673723},
doi = {10.1117/12.2673723},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {International Conference on Internet of Things and Machine Learning (IoTML 2022)},
volume = {12640},
pages = {126401Q},
publisher = {SPIE},
organization = {International Society for Optics and Photonics},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2725.
Rajapakshe, Thejan; Rana, Rajib; Khalifa, Sara; Sisman, Berrak; Schuller, Björn W.
Improving Speech Emotion Recognition Performance using Differentiable Architecture Search Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2305-14402,
title = {Improving Speech Emotion Recognition Performance using Differentiable Architecture Search},
author = {Thejan Rajapakshe and Rajib Rana and Sara Khalifa and Berrak Sisman and Björn W. Schuller},
url = {https://doi.org/10.48550/arXiv.2305.14402},
doi = {10.48550/arXiv.2305.14402},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2305.14402},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2724.
Davy, N.; Waheed, U. Bin; Koeshidayatullah, A.; El-Husseiny, A.
Automated Deep Learning (AutoDL) for Facies Prediction: Implementation and Strategy Journal Article
In: vol. 2023, no. 1, pp. 1-5, 2023, ISSN: 2214-4609.
@article{eage:/content/papers/10.3997/2214-4609.202310455,
title = {Automated Deep Learning (AutoDL) for Facies Prediction: Implementation and Strategy},
author = {N. Davy and U. Bin Waheed and A. Koeshidayatullah and A. El-Husseiny},
url = {https://www.earthdoc.org/content/papers/10.3997/2214-4609.202310455},
doi = {https://doi.org/10.3997/2214-4609.202310455},
issn = {2214-4609},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
volume = {2023},
number = {1},
pages = {1-5},
publisher = {Ëuropean Association of Geoscientists & Engineers},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2723.
Deutel, Mark; Kontes, Georgios D.; Mutschler, Christopher; Teich, Jürgen
Augmented Random Search for Multi-Objective Bayesian Optimization of Neural Networks Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2305-14109,
title = {Augmented Random Search for Multi-Objective Bayesian Optimization of Neural Networks},
author = {Mark Deutel and Georgios D. Kontes and Christopher Mutschler and Jürgen Teich},
url = {https://doi.org/10.48550/arXiv.2305.14109},
doi = {10.48550/arXiv.2305.14109},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2305.14109},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2722.
Wei, Wei; Zhao, Shuyi; Xu, Songzheng; Zhang, Lei; Zhang, Yanning
Semi-Supervised Neural Architecture Search for Hyperspectral Imagery Classification Method With Dynamic Feature Clustering Journal Article
In: IEEE Transactions on Geoscience and Remote Sensing, vol. 61, pp. 1-14, 2023.
@article{10132869,
title = {Semi-Supervised Neural Architecture Search for Hyperspectral Imagery Classification Method With Dynamic Feature Clustering},
author = {Wei Wei and Shuyi Zhao and Songzheng Xu and Lei Zhang and Yanning Zhang},
url = {https://ieeexplore.ieee.org/abstract/document/10132869},
doi = {10.1109/TGRS.2023.3279437},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Geoscience and Remote Sensing},
volume = {61},
pages = {1-14},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2721.
Zhang, Jiuling; Ding, Zhiming
Small Temperature is All You Need for Differentiable Architecture Search Proceedings Article
In: Kashima, Hisashi; Ide, Tsuyoshi; Peng, Wen-Chih (Ed.): Ädvances in Knowledge Discovery and Data Mining", pp. 303–315, Springer Nature Switzerland, Cham, 2023, ISBN: 978-3-031-33374-3.
@inproceedings{10.1007/978-3-031-33374-3_24,
title = {Small Temperature is All You Need for Differentiable Architecture Search},
author = {Jiuling Zhang and Zhiming Ding},
editor = {Hisashi Kashima and Tsuyoshi Ide and Wen-Chih Peng},
isbn = {978-3-031-33374-3},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Ädvances in Knowledge Discovery and Data Mining"},
pages = {303–315},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {Differentiable architecture search (DARTS) yields highly efficient gradient-based neural architecture search (NAS) by relaxing the discrete operation selection to optimize continuous architecture parameters that maps NAS from the discrete optimization to a continuous problem. DARTS then remaps the relaxed supernet back to the discrete space by one-off post-search pruning to obtain the final architecture (finalnet). Some emerging works argue that this remap is inherently prone to mismatch the network between training and evaluation which leads to performance discrepancy and even model collapse in extreme cases. We propose to close the gap between the relaxed supernet in training and the pruned finalnet in evaluation through utilizing small temperature to sparsify the continuous distribution in the training phase. To this end, we first formulate sparse-noisy softmax to get around gradient saturation. We then propose an exponential temperature schedule to better control the outbound distribution and elaborate an entropy-based adaptive scheme to finally achieve the enhancement. We conduct extensive experiments to verify the efficiency and efficacy of our method.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Differentiable architecture search (DARTS) yields highly efficient gradient-based neural architecture search (NAS) by relaxing the discrete operation selection to optimize continuous architecture parameters that maps NAS from the discrete optimization to a continuous problem. DARTS then remaps the relaxed supernet back to the discrete space by one-off post-search pruning to obtain the final architecture (finalnet). Some emerging works argue that this remap is inherently prone to mismatch the network between training and evaluation which leads to performance discrepancy and even model collapse in extreme cases. We propose to close the gap between the relaxed supernet in training and the pruned finalnet in evaluation through utilizing small temperature to sparsify the continuous distribution in the training phase. To this end, we first formulate sparse-noisy softmax to get around gradient saturation. We then propose an exponential temperature schedule to better control the outbound distribution and elaborate an entropy-based adaptive scheme to finally achieve the enhancement. We conduct extensive experiments to verify the efficiency and efficacy of our method.
2720.
Zhang, Peng; Zhang, Pengjv; Wu, Haoran; Tang, Jinsong
Automatic deep learning for automatic target detection of sonar images Proceedings Article
In: Ba, Shuhong; Zhou, Fan (Ed.): Third International Conference on Machine Learning and Computer Application (ICMLCA 2022), pp. 1263634, International Society for Optics and Photonics SPIE, 2023.
@inproceedings{10.1117/12.2675409,
title = {Automatic deep learning for automatic target detection of sonar images},
author = {Peng Zhang and Pengjv Zhang and Haoran Wu and Jinsong Tang},
editor = {Shuhong Ba and Fan Zhou},
url = {https://doi.org/10.1117/12.2675409},
doi = {10.1117/12.2675409},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Third International Conference on Machine Learning and Computer Application (ICMLCA 2022)},
volume = {12636},
pages = {1263634},
publisher = {SPIE},
organization = {International Society for Optics and Photonics},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2719.
Huang, Jia-Cheng; Zeng, Guo-Qiang; Geng, Guang-Gang; Weng, Jian; Lu, Kang-Di; Zhang, Yu
In: Computers & Security, vol. 132, pp. 103310, 2023, ISSN: 0167-4048.
@article{HUANG2023103310,
title = {Differential evolution-based convolutional neural networks: An automatic architecture design method for intrusion detection in industrial control systems},
author = {Jia-Cheng Huang and Guo-Qiang Zeng and Guang-Gang Geng and Jian Weng and Kang-Di Lu and Yu Zhang},
url = {https://www.sciencedirect.com/science/article/pii/S0167404823002201},
doi = {https://doi.org/10.1016/j.cose.2023.103310},
issn = {0167-4048},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Computers & Security},
volume = {132},
pages = {103310},
abstract = {Industrial control systems (ICSs) are facing serious and evolving security threats because of a variety of malicious attacks. Deep learning-based intrusion detection systems (IDSs) have been widely considered as one of promising security solutions for ICSs, but these deep neural networks for IDSs in ICSs have been designed manually, which are extremely dependent on expert experience with numerous model parameters. This paper makes the first attempt to develop an automatic architecture design method of convolutional neural networks (CNNs) based on differential evolution (abbreviated as DE-CNN) for the intrusion detection issue in ICSs. The first phase of the proposed DE-CNN is the off-line architecture optimization of the CNNs constructed by three basic units such as ResNetBlockUnit, DenseNetBlockUnit, and PoolingUnit, including encoding the architecture parameters of a CNN as a population, evaluating the fitness of the population by the validation accuracy and the number of CNN model parameters, implementing the evolutionary process including mutation and crossover operations, and selecting the best individual from the population. Then, the optimal CNN model obtained by the off-line optimization of DE-CNN is deployed for the online IDSs. The experimental results on two intrusion detection datasets in ICSs including SWaT and WADI have demonstrated the superiority of the proposed DE-CNN to the state-of-the-art manually-designed and neuroevolution-based methods under both unsupervised and supervised learning in terms of Precision, Recall, F1-Score and the number of the CNN model parameters.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Industrial control systems (ICSs) are facing serious and evolving security threats because of a variety of malicious attacks. Deep learning-based intrusion detection systems (IDSs) have been widely considered as one of promising security solutions for ICSs, but these deep neural networks for IDSs in ICSs have been designed manually, which are extremely dependent on expert experience with numerous model parameters. This paper makes the first attempt to develop an automatic architecture design method of convolutional neural networks (CNNs) based on differential evolution (abbreviated as DE-CNN) for the intrusion detection issue in ICSs. The first phase of the proposed DE-CNN is the off-line architecture optimization of the CNNs constructed by three basic units such as ResNetBlockUnit, DenseNetBlockUnit, and PoolingUnit, including encoding the architecture parameters of a CNN as a population, evaluating the fitness of the population by the validation accuracy and the number of CNN model parameters, implementing the evolutionary process including mutation and crossover operations, and selecting the best individual from the population. Then, the optimal CNN model obtained by the off-line optimization of DE-CNN is deployed for the online IDSs. The experimental results on two intrusion detection datasets in ICSs including SWaT and WADI have demonstrated the superiority of the proposed DE-CNN to the state-of-the-art manually-designed and neuroevolution-based methods under both unsupervised and supervised learning in terms of Precision, Recall, F1-Score and the number of the CNN model parameters.
2718.
Wang, Zigeng; Li, Bingbing; Xiao, Xia; Zhang, Tianyun; Bragin, Mikhail A.; Yan, Bing; Ding, Caiwen; Rajasekaran, Sanguthevar
Automatic Subnetwork Search Through Dynamic Differentiable Neuron Pruning Proceedings Article
In: 2023 24th International Symposium on Quality Electronic Design (ISQED), pp. 1-6, 2023.
@inproceedings{10129379,
title = {Automatic Subnetwork Search Through Dynamic Differentiable Neuron Pruning},
author = {Zigeng Wang and Bingbing Li and Xia Xiao and Tianyun Zhang and Mikhail A. Bragin and Bing Yan and Caiwen Ding and Sanguthevar Rajasekaran},
url = {https://ieeexplore.ieee.org/abstract/document/10129379},
doi = {10.1109/ISQED57927.2023.10129379},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {2023 24th International Symposium on Quality Electronic Design (ISQED)},
pages = {1-6},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2717.
Jin, Xiu; Zhang, Tong; Wang, Lianglong; Luo, Qing; Li, Shaowen; Zhang, Xiaodan; Lu, Jie; Rao, Yuan
Using an optimised neural architecture search for predicting the quantum yield of photosynthesis of winter wheat Journal Article
In: Biosystems Engineering, vol. 230, pp. 442-457, 2023, ISSN: 1537-5110.
@article{JIN2023442,
title = {Using an optimised neural architecture search for predicting the quantum yield of photosynthesis of winter wheat},
author = {Xiu Jin and Tong Zhang and Lianglong Wang and Qing Luo and Shaowen Li and Xiaodan Zhang and Jie Lu and Yuan Rao},
url = {https://www.sciencedirect.com/science/article/pii/S1537511023000971},
doi = {https://doi.org/10.1016/j.biosystemseng.2023.04.015},
issn = {1537-5110},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Biosystems Engineering},
volume = {230},
pages = {442-457},
abstract = {The quantum yield (QY) values of crops are important for assessing photosynthetic efficiency. Our hypothesis is that high-throughput prediction models of QY values can be established based on vegetation indices (VIs). The objectives of this research were to predict QY values based on VIs with high correlations and to compare the performances of classical machine learning with that of B-NAS methods. Compared with support vector regression (SVR) and other traditional prediction methods, in this study, a novel prediction model based on an optimised neural architecture search (B-NAS) was proposed, and achieved comparatively better results. In addition, an improved architecture was obtained and compared for ten different search architectures on the experimental dataset. The experimental results showed that the neural network model constructed based on the B-NAS method predicted QY values outperformed the models obtained using SVR and Adaboost methods in terms of all considered performance metrics (R2, RMSE, RPD, Bias). Meanwhile, the proposed model meets the criteria for a class A model. The final B-NAS model was tested on an independent validation set, obtaining similar performance metrics and an accuracy of 92% for post-prediction classification into three equal interval QY classes. An analysis of the neural architecture obtained by B-NAS showed that a four-layer network architecture had certain advantages. As well, the importance of the preprocessing and dropout layers is discussed.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The quantum yield (QY) values of crops are important for assessing photosynthetic efficiency. Our hypothesis is that high-throughput prediction models of QY values can be established based on vegetation indices (VIs). The objectives of this research were to predict QY values based on VIs with high correlations and to compare the performances of classical machine learning with that of B-NAS methods. Compared with support vector regression (SVR) and other traditional prediction methods, in this study, a novel prediction model based on an optimised neural architecture search (B-NAS) was proposed, and achieved comparatively better results. In addition, an improved architecture was obtained and compared for ten different search architectures on the experimental dataset. The experimental results showed that the neural network model constructed based on the B-NAS method predicted QY values outperformed the models obtained using SVR and Adaboost methods in terms of all considered performance metrics (R2, RMSE, RPD, Bias). Meanwhile, the proposed model meets the criteria for a class A model. The final B-NAS model was tested on an independent validation set, obtaining similar performance metrics and an accuracy of 92% for post-prediction classification into three equal interval QY classes. An analysis of the neural architecture obtained by B-NAS showed that a four-layer network architecture had certain advantages. As well, the importance of the preprocessing and dropout layers is discussed.
2716.
Chen, Tianyi; Liang, Luming; Ding, Tianyu; Zharkov, Ilya
Towards Automatic Neural Architecture Search within General Super-Networks Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2305-18030,
title = {Towards Automatic Neural Architecture Search within General Super-Networks},
author = {Tianyi Chen and Luming Liang and Tianyu Ding and Ilya Zharkov},
url = {https://doi.org/10.48550/arXiv.2305.18030},
doi = {10.48550/arXiv.2305.18030},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2305.18030},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2715.
Lawton, Neal; Kumar, Anoop; Thattai, Govind; Galstyan, Aram; Steeg, Greg Ver
Neural Architecture Search for Parameter-Efficient Fine-tuning of Large Pre-trained Language Models Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2305-16597,
title = {Neural Architecture Search for Parameter-Efficient Fine-tuning of Large Pre-trained Language Models},
author = {Neal Lawton and Anoop Kumar and Govind Thattai and Aram Galstyan and Greg Ver Steeg},
url = {https://doi.org/10.48550/arXiv.2305.16597},
doi = {10.48550/arXiv.2305.16597},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2305.16597},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2714.
Huang, Zhe; Li, Yudian
FSD: Fully-Specialized Detector via Neural Architecture Search Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2305-16649,
title = {FSD: Fully-Specialized Detector via Neural Architecture Search},
author = {Zhe Huang and Yudian Li},
url = {https://doi.org/10.48550/arXiv.2305.16649},
doi = {10.48550/arXiv.2305.16649},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2305.16649},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2713.
An, Sohyun; Lee, Hayeon; Jo, Jaehyeong; Lee, Seanie; Hwang, Sung Ju
DiffusionNAG: Task-guided Neural Architecture Generation with Diffusion Models Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2305-16943,
title = {DiffusionNAG: Task-guided Neural Architecture Generation with Diffusion Models},
author = {Sohyun An and Hayeon Lee and Jaehyeong Jo and Seanie Lee and Sung Ju Hwang},
url = {https://doi.org/10.48550/arXiv.2305.16943},
doi = {10.48550/arXiv.2305.16943},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2305.16943},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2712.
Jiang, Zhiying; Liu, Risheng; Yang, Shuzhou; Zhang, Zengxi; Fan, Xin
Contrastive Learning Based Recursive Dynamic Multi-Scale Network for Image Deraining Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2305-18092,
title = {Contrastive Learning Based Recursive Dynamic Multi-Scale Network for Image Deraining},
author = {Zhiying Jiang and Risheng Liu and Shuzhou Yang and Zengxi Zhang and Xin Fan},
url = {https://doi.org/10.48550/arXiv.2305.18092},
doi = {10.48550/arXiv.2305.18092},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2305.18092},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2711.
Pietron, Marcin; Zurek, Dominik; Faber, Kamil; Corizzo, Roberto
AD-NEV: A Scalable Multi-level Neuroevolution Framework for Multivariate Anomaly Detection Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2305-16497,
title = {AD-NEV: A Scalable Multi-level Neuroevolution Framework for Multivariate Anomaly Detection},
author = {Marcin Pietron and Dominik Zurek and Kamil Faber and Roberto Corizzo},
url = {https://doi.org/10.48550/arXiv.2305.16497},
doi = {10.48550/arXiv.2305.16497},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2305.16497},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2710.
Silva, Ada Cristina França; Cortes, Omar Andres Carmona
GANASUNet: An Efficient Convolutional Neural Architecture for Segmenting Iron Ore Images Proceedings Article
In: Äbraham, Ajith; Pllana, Sabri; Casalino, Gabriella; Ma, Kun; Bajaj, Anu" (Ed.): Ïntelligent Systems Design and Applications", pp. 281–291, Springer Nature Switzerland, Cham, 2023, ISBN: 978-3-031-35510-3.
@inproceedings{10.1007/978-3-031-35510-3_27,
title = {GANASUNet: An Efficient Convolutional Neural Architecture for Segmenting Iron Ore Images},
author = {Ada Cristina França Silva and Omar Andres Carmona Cortes},
editor = {Ajith Äbraham and Sabri Pllana and Gabriella Casalino and Kun Ma and Anu" Bajaj},
isbn = {978-3-031-35510-3},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Ïntelligent Systems Design and Applications"},
pages = {281–291},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {Ïron ore segmentation has a challenge in segmenting different types of ores in the same area; the detection and segmentation of iron ore are used to analyze the material quality and optimize the plant processing. This paper presents an UNet-based Convolutional Neural Network (CNN) optimized by a technique so-called Neural Architecture Search(NAS) to segment fine iron ore regions. The images were collected from an iron ore plant, in which it was possible to obtain a dataset composed of 688 images and their label segmentation. The results of the optimized architecture show that the UNet-based architecture achieved a result of 80% of Intersect Over Union(IoU) against UNet without optimization with 75% and DeepLabV3+ with 78%, respectively."},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Ïron ore segmentation has a challenge in segmenting different types of ores in the same area; the detection and segmentation of iron ore are used to analyze the material quality and optimize the plant processing. This paper presents an UNet-based Convolutional Neural Network (CNN) optimized by a technique so-called Neural Architecture Search(NAS) to segment fine iron ore regions. The images were collected from an iron ore plant, in which it was possible to obtain a dataset composed of 688 images and their label segmentation. The results of the optimized architecture show that the UNet-based architecture achieved a result of 80% of Intersect Over Union(IoU) against UNet without optimization with 75% and DeepLabV3+ with 78%, respectively."
2709.
Guo, Hongwei; Zhuang, Xiaoying; Alajlan, Naif; Rabczuk, Timon
Physics-informed deep learning for melting heat transfer analysis with model-based transfer learning Journal Article
In: Computers & Mathematics with Applications, vol. 143, pp. 303-317, 2023, ISSN: 0898-1221.
@article{GUO2023303,
title = {Physics-informed deep learning for melting heat transfer analysis with model-based transfer learning},
author = {Hongwei Guo and Xiaoying Zhuang and Naif Alajlan and Timon Rabczuk},
url = {https://www.sciencedirect.com/science/article/pii/S0898122123002122},
doi = {https://doi.org/10.1016/j.camwa.2023.05.014},
issn = {0898-1221},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Computers & Mathematics with Applications},
volume = {143},
pages = {303-317},
abstract = {We present an adaptive deep collocation method (DCM) based on physics-informed deep learning for the melting heat transfer analysis of a non-Newtonian (Sisko) fluid over a moving surface with nonlinear thermal radiation. Fitted neural network search (NAS) and model based transfer learning (TL) are developed to improve model computational efficiency and accuracy. The governing equations for this boundary-layer flow problem are derived using Buongiornoś and a nonlinear thermal radiation model. Next, similarity transformations are introduced to reduce the governing equations into coupled nonlinear ordinary differential equations (ODEs) subjected to asymptotic infinity boundary conditions. By incorporating physics constraints into the neural networks, we employ the proposed deep learning model to solve the coupled ODEs. The imposition of infinity boundary conditions is carried out by adding an inequality constraint to the loss function, with infinity added to the hyper-parameters of the neural network, which is updated dynamically in the optimization process. The effects of various dimensionless parameters on three profiles (velocity, temperature, concentration) are investigated. Finally, we demonstrate the performance and accuracy of the adaptive DCM with transfer learning through several numerical examples, which can be the promising surrogate model to solve boundary layer problems.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
We present an adaptive deep collocation method (DCM) based on physics-informed deep learning for the melting heat transfer analysis of a non-Newtonian (Sisko) fluid over a moving surface with nonlinear thermal radiation. Fitted neural network search (NAS) and model based transfer learning (TL) are developed to improve model computational efficiency and accuracy. The governing equations for this boundary-layer flow problem are derived using Buongiornoś and a nonlinear thermal radiation model. Next, similarity transformations are introduced to reduce the governing equations into coupled nonlinear ordinary differential equations (ODEs) subjected to asymptotic infinity boundary conditions. By incorporating physics constraints into the neural networks, we employ the proposed deep learning model to solve the coupled ODEs. The imposition of infinity boundary conditions is carried out by adding an inequality constraint to the loss function, with infinity added to the hyper-parameters of the neural network, which is updated dynamically in the optimization process. The effects of various dimensionless parameters on three profiles (velocity, temperature, concentration) are investigated. Finally, we demonstrate the performance and accuracy of the adaptive DCM with transfer learning through several numerical examples, which can be the promising surrogate model to solve boundary layer problems.
2708.
Serianni, Aaron; Kalita, Jugal
Training-free Neural Architecture Search for RNNs and Transformers Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2306-00288,
title = {Training-free Neural Architecture Search for RNNs and Transformers},
author = {Aaron Serianni and Jugal Kalita},
url = {https://doi.org/10.48550/arXiv.2306.00288},
doi = {10.48550/arXiv.2306.00288},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2306.00288},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2707.
Chen, Zihao; Yang, Fan; Shang, Li; Zeng, Xuan
Automated and Agile Design of Layout Hotspot Detector via Neural Architecture Search Proceedings Article
In: 2023 Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 1-6, 2023.
@inproceedings{10137142,
title = {Automated and Agile Design of Layout Hotspot Detector via Neural Architecture Search},
author = {Zihao Chen and Fan Yang and Li Shang and Xuan Zeng},
doi = {10.23919/DATE56975.2023.10137142},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {2023 Design, Automation & Test in Europe Conference & Exhibition (DATE)},
pages = {1-6},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2706.
Waterlaat, Nick; Vogel, Sebastian; Rodriguez, Hiram Rayo Torres; Sanberg, Willem; Daalderop, Gerardo
Quantization-Aware Neural Architecture Search with Hyperparameter Optimization for Industrial Predictive Maintenance Applications Proceedings Article
In: 2023 Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 1-2, 2023.
@inproceedings{10137073,
title = {Quantization-Aware Neural Architecture Search with Hyperparameter Optimization for Industrial Predictive Maintenance Applications},
author = {Nick Waterlaat and Sebastian Vogel and Hiram Rayo Torres Rodriguez and Willem Sanberg and Gerardo Daalderop},
url = {https://ieeexplore.ieee.org/abstract/document/10137073},
doi = {10.23919/DATE56975.2023.10137073},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {2023 Design, Automation & Test in Europe Conference & Exhibition (DATE)},
pages = {1-2},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2705.
Zhao, Shixin; Qu, Songyun; Wang, Ying; Han, Yinhe
ENASA: Towards Edge Neural Architecture Search based on CIM acceleration Proceedings Article
In: 2023 Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 1-2, 2023.
@inproceedings{10137157,
title = {ENASA: Towards Edge Neural Architecture Search based on CIM acceleration},
author = {Shixin Zhao and Songyun Qu and Ying Wang and Yinhe Han},
doi = {10.23919/DATE56975.2023.10137157},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {2023 Design, Automation & Test in Europe Conference & Exhibition (DATE)},
pages = {1-2},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2704.
Pan, Hu; Hu, Yuan; Yang, Ying; Zhang, Xu
Graphormer-NAS-X: A Novel Graphormer-Based Neural Architecture Search Method Proceedings Article
In: Park, Ji Su; Yang, Laurence T.; Pan, Yi; Park, Jong Hyuk (Ed.): Ädvances in Computer Science and Ubiquitous Computing", pp. 45–51, Springer Nature Singapore, Singapore, 2023, ISBN: 978-981-99-1252-0.
@inproceedings{10.1007/978-981-99-1252-0_6,
title = {Graphormer-NAS-X: A Novel Graphormer-Based Neural Architecture Search Method},
author = {Hu Pan and Yuan Hu and Ying Yang and Xu Zhang},
editor = {Ji Su Park and Laurence T. Yang and Yi Pan and Jong Hyuk Park},
isbn = {978-981-99-1252-0},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Ädvances in Computer Science and Ubiquitous Computing"},
pages = {45–51},
publisher = {Springer Nature Singapore},
address = {Singapore},
abstract = {Neural architecture search (NAS) aims to search for the best neural network model for various tasks. In this paper, an improved Graphormer is used as an encoder to learn the embedding of the neural architecture, called Graphormer-NAS-X, where X is related to the specific search algorithm. Specifically, the centrality encoding is improved by adding an eigenvector centrality that can capture the importance of adjacent nodes. And the position encoding is enhanced by using the shortest distance of the directed graph, which can better reflect neural architectureś single-directional flow of computational information. The experiments are evaluated on two public datasets with two commonly used search algorithms. Graphormer-NAS-X outperforms most existing NAS algorithms, which indicates Graphormer benefits the downstream search.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Neural architecture search (NAS) aims to search for the best neural network model for various tasks. In this paper, an improved Graphormer is used as an encoder to learn the embedding of the neural architecture, called Graphormer-NAS-X, where X is related to the specific search algorithm. Specifically, the centrality encoding is improved by adding an eigenvector centrality that can capture the importance of adjacent nodes. And the position encoding is enhanced by using the shortest distance of the directed graph, which can better reflect neural architectureś single-directional flow of computational information. The experiments are evaluated on two public datasets with two commonly used search algorithms. Graphormer-NAS-X outperforms most existing NAS algorithms, which indicates Graphormer benefits the downstream search.
2703.
Chang, Hung-Yang; Mozafari, Seyyed Hasan; Clark, James J.; Meyer, Brett H.; Gross, Warren J.
High-Throughput Edge Inference for BERT Models via Neural Architecture Search and Pipeline Proceedings Article
In: Proceedings of the Great Lakes Symposium on VLSI 2023, pp. 455–459, Association for Computing Machinery, Knoxville, TN, USA, 2023, ISBN: 9798400701252.
@inproceedings{10.1145/3583781.3590302,
title = {High-Throughput Edge Inference for BERT Models via Neural Architecture Search and Pipeline},
author = {Hung-Yang Chang and Seyyed Hasan Mozafari and James J. Clark and Brett H. Meyer and Warren J. Gross},
url = {https://doi.org/10.1145/3583781.3590302},
doi = {10.1145/3583781.3590302},
isbn = {9798400701252},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Proceedings of the Great Lakes Symposium on VLSI 2023},
pages = {455–459},
publisher = {Association for Computing Machinery},
address = {Knoxville, TN, USA},
series = {GLSVLSI '23},
abstract = {There has been growing interest in improving the BERT inference throughput on resource-constrained edge devices for a satisfactory user experience. One methodology is to employ heterogeneous computing, which utilizes multiple processing elements to accelerate inference. Another methodology is to deploy Neural Architecture Search (NAS) to find optimal solutions in accuracy-throughput design space. In this paper, for the first time, we incorporate NAS with pipelining for BERT models. We show that performing NAS with pipelining achieves on average 53% higher throughput, compared to NAS with a homogeneous system.Additionally, we propose a NAS algorithm that incorporates hardware performance feedback to accelerate the NAS process. Our proposed NAS algorithm speeds up the search process by ~4x, and 5.5x on the design space of the BERT and CNNs, respectively. Also, by exploring the accuracy-throughput design space of BERT models, we demonstrate that performing pipelining then NAS (Pipeline-then-NAS) can lead to solutions with up to 9x higher inference throughput, compared to running homogeneous inference on the BERT-base model, with only a 1.3% decrease in accuracy.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
There has been growing interest in improving the BERT inference throughput on resource-constrained edge devices for a satisfactory user experience. One methodology is to employ heterogeneous computing, which utilizes multiple processing elements to accelerate inference. Another methodology is to deploy Neural Architecture Search (NAS) to find optimal solutions in accuracy-throughput design space. In this paper, for the first time, we incorporate NAS with pipelining for BERT models. We show that performing NAS with pipelining achieves on average 53% higher throughput, compared to NAS with a homogeneous system.Additionally, we propose a NAS algorithm that incorporates hardware performance feedback to accelerate the NAS process. Our proposed NAS algorithm speeds up the search process by ~4x, and 5.5x on the design space of the BERT and CNNs, respectively. Also, by exploring the accuracy-throughput design space of BERT models, we demonstrate that performing pipelining then NAS (Pipeline-then-NAS) can lead to solutions with up to 9x higher inference throughput, compared to running homogeneous inference on the BERT-base model, with only a 1.3% decrease in accuracy.
2702.
Che, Kaiwei; Zhou, Zhaokun; Ma, Zhengyu; Fang, Wei; Chen, Yanqi; Shen, Shuaijie; Yuan, Li; Tian, Yonghong
Auto-Spikformer: Spikformer Architecture Search Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2306-00807,
title = {Auto-Spikformer: Spikformer Architecture Search},
author = {Kaiwei Che and Zhaokun Zhou and Zhengyu Ma and Wei Fang and Yanqi Chen and Shuaijie Shen and Li Yuan and Yonghong Tian},
url = {https://doi.org/10.48550/arXiv.2306.00807},
doi = {10.48550/arXiv.2306.00807},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2306.00807},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2701.
Lou, Wenqi; Qian, Jiaming; Gong, Lei; Wang, Xuan; Wang, Chao; Zhou, Xuehai
NAF: Deeper Network/Accelerator Co-Exploration for Customizing CNNs on FPGA Proceedings Article
In: 2023 Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 1-6, 2023.
@inproceedings{10137094,
title = {NAF: Deeper Network/Accelerator Co-Exploration for Customizing CNNs on FPGA},
author = {Wenqi Lou and Jiaming Qian and Lei Gong and Xuan Wang and Chao Wang and Xuehai Zhou},
doi = {10.23919/DATE56975.2023.10137094},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {2023 Design, Automation & Test in Europe Conference & Exhibition (DATE)},
pages = {1-6},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2700.
Dou, Ziwen; Ye, Dong; Wang, Boya
AutoSegEdge: Searching for the edge device real-time semantic segmentation based on multi-task learning Journal Article
In: Image and Vision Computing, vol. 136, pp. 104719, 2023, ISSN: 0262-8856.
@article{DOU2023104719,
title = {AutoSegEdge: Searching for the edge device real-time semantic segmentation based on multi-task learning},
author = {Ziwen Dou and Dong Ye and Boya Wang},
url = {https://www.sciencedirect.com/science/article/pii/S0262885623000938},
doi = {https://doi.org/10.1016/j.imavis.2023.104719},
issn = {0262-8856},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Image and Vision Computing},
volume = {136},
pages = {104719},
abstract = {Real-time semantic segmentation is a challenging task for resource-constrained edge devices. We propose AutoSegEdge, based on Neural Architecture Search (NAS), a semantic segmentation approach that runs on edge devices in real-time. Besides accuracy, we employ FLOPs and latency on the target edge devices as search constraints. Our work is probably one of the first attempts to translate multi-objectives NAS into Multi-Task Learning. Be inspired by Multi-Task Learning, we regard the sub-objective in multi-objective NAS as a learning task in Multi-Task Learning. The total loss function of the multi-objective NAS is deconstructed into the weighted sum of the sub-objective loss function. However, the conflict among the sub-objective will cause the searched networks to “architecture collapse.” To avoid the multi-objectives NAS falls into “architecture collapse.” Based on uncertainty, this paper proposes a method to learn the weights of sub-objective loss functions automatically. AutoSegEdge was discovered from an efficient cell-level search space that integrates multi-resolution branches. Additionally, AutoSegEdge employs knowledge distillation to further boost accuracy. Finally, we accelerated AutoSegEdge using NVIDIAś TensorRT and deployed it on the Nvidia Jetson NX. Experiments demonstrate that multi-objectives NAS only requires 1.5 GPU days to obtain the best result on a single Nvidia Tesla V100 GPU. On the Cityscapes dataset, AutoSegEdge achieved an mIoU of 70.3% with 16.6 FPS on the Nvidia Jetson NX (and 194.54 FPS on an Nvidia Tesla V100 GPU) at the original resolution (1024 × 2048) using TensorRT. Our method is 2–3× faster than existing state-of-the-art real-time methods while maintaining competitive accuracy. We also conducted robustness experiments to analyze our method and modules. The code is available: https://github.com/douziwenhit/AutoSeg_edge.git.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Real-time semantic segmentation is a challenging task for resource-constrained edge devices. We propose AutoSegEdge, based on Neural Architecture Search (NAS), a semantic segmentation approach that runs on edge devices in real-time. Besides accuracy, we employ FLOPs and latency on the target edge devices as search constraints. Our work is probably one of the first attempts to translate multi-objectives NAS into Multi-Task Learning. Be inspired by Multi-Task Learning, we regard the sub-objective in multi-objective NAS as a learning task in Multi-Task Learning. The total loss function of the multi-objective NAS is deconstructed into the weighted sum of the sub-objective loss function. However, the conflict among the sub-objective will cause the searched networks to “architecture collapse.” To avoid the multi-objectives NAS falls into “architecture collapse.” Based on uncertainty, this paper proposes a method to learn the weights of sub-objective loss functions automatically. AutoSegEdge was discovered from an efficient cell-level search space that integrates multi-resolution branches. Additionally, AutoSegEdge employs knowledge distillation to further boost accuracy. Finally, we accelerated AutoSegEdge using NVIDIAś TensorRT and deployed it on the Nvidia Jetson NX. Experiments demonstrate that multi-objectives NAS only requires 1.5 GPU days to obtain the best result on a single Nvidia Tesla V100 GPU. On the Cityscapes dataset, AutoSegEdge achieved an mIoU of 70.3% with 16.6 FPS on the Nvidia Jetson NX (and 194.54 FPS on an Nvidia Tesla V100 GPU) at the original resolution (1024 × 2048) using TensorRT. Our method is 2–3× faster than existing state-of-the-art real-time methods while maintaining competitive accuracy. We also conducted robustness experiments to analyze our method and modules. The code is available: https://github.com/douziwenhit/AutoSeg_edge.git.
2699.
Pierros, Ioannis; Kouloumpris, Eleftherios; Zaikis, Dimitrios; Vlahavas, Ioannis
Retail Demand Forecasting for 1 Million Products Proceedings Article
In: Äbraham, Ajith; Pllana, Sabri; Casalino, Gabriella; Ma, Kun; Bajaj, Anu" (Ed.): Ïntelligent Systems Design and Applications", pp. 467–479, Springer Nature Switzerland, Cham, 2023, ISBN: 978-3-031-27440-4.
@inproceedings{10.1007/978-3-031-27440-4_45,
title = {Retail Demand Forecasting for 1 Million Products},
author = {Ioannis Pierros and Eleftherios Kouloumpris and Dimitrios Zaikis and Ioannis Vlahavas},
editor = {Ajith Äbraham and Sabri Pllana and Gabriella Casalino and Kun Ma and Anu" Bajaj},
url = {https://link.springer.com/chapter/10.1007/978-3-031-27440-4_45#citeas},
isbn = {978-3-031-27440-4},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Ïntelligent Systems Design and Applications"},
pages = {467–479},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {Retail chains without proper demand forecasting tools are susceptible to significant financial losses by missing out on sales due to stocked-out products or having to throw out expired products because they were overstocked. Extensive research has been carried out, comparing different forecasting methodologies and models, examining the influence of different factors, and highlighting the significance of intermittent forecasting. However, these approaches often struggle to scale up and crumble when dealing with larger retail chains. In this paper, we analyze the real case of a big retail chain with 300 stores, 200 product groups per store and over 1 million products in total. We propose an architecture made up of multiple Neural Network models that can generate forecasts in a timely manner, taking into account calendar features, promotions and the interactions between competing products. It produces daily predictions in under 3 h and retrains weekly the models whose performance deteriorates in 12 h, using an AutoML component to explore deeper and larger architectures. It is a critical component of the companyś Order Management System, achieving a Root Mean Squared Error of 4.48 units across each horizon that was defined by the company.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Retail chains without proper demand forecasting tools are susceptible to significant financial losses by missing out on sales due to stocked-out products or having to throw out expired products because they were overstocked. Extensive research has been carried out, comparing different forecasting methodologies and models, examining the influence of different factors, and highlighting the significance of intermittent forecasting. However, these approaches often struggle to scale up and crumble when dealing with larger retail chains. In this paper, we analyze the real case of a big retail chain with 300 stores, 200 product groups per store and over 1 million products in total. We propose an architecture made up of multiple Neural Network models that can generate forecasts in a timely manner, taking into account calendar features, promotions and the interactions between competing products. It produces daily predictions in under 3 h and retrains weekly the models whose performance deteriorates in 12 h, using an AutoML component to explore deeper and larger architectures. It is a critical component of the companyś Order Management System, achieving a Root Mean Squared Error of 4.48 units across each horizon that was defined by the company.
2698.
Akhauri, Yash; Abdelfattah, Mohamed S.
Multi-Predict: Few Shot Predictors For Efficient Neural Architecture Search Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2306-02459,
title = {Multi-Predict: Few Shot Predictors For Efficient Neural Architecture Search},
author = {Yash Akhauri and Mohamed S. Abdelfattah},
url = {https://doi.org/10.48550/arXiv.2306.02459},
doi = {10.48550/arXiv.2306.02459},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2306.02459},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2697.
Nasir, Muhammad Umair; Earle, Sam; Togelius, Julian; James, Steven; Cleghorn, Christopher W.
LLMatic: Neural Architecture Search via Large Language Models and Quality-Diversity Optimization Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2306-01102,
title = {LLMatic: Neural Architecture Search via Large Language Models and Quality-Diversity Optimization},
author = {Muhammad Umair Nasir and Sam Earle and Julian Togelius and Steven James and Christopher W. Cleghorn},
url = {https://doi.org/10.48550/arXiv.2306.01102},
doi = {10.48550/arXiv.2306.01102},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2306.01102},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2696.
Zhang, Xiaoyun; Ping, Xieyi; Zhang, Jianwei
Deep Active Learning with Structured Neural Depth Search Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2306-02808,
title = {Deep Active Learning with Structured Neural Depth Search},
author = {Xiaoyun Zhang and Xieyi Ping and Jianwei Zhang},
url = {https://doi.org/10.48550/arXiv.2306.02808},
doi = {10.48550/arXiv.2306.02808},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2306.02808},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2695.
Hafiz, Faizal; Broekaert, Jan; Torre, Davide La; Swain, Akshya
Co-evolution of neural architectures and features for stock market forecasting: A multi-objective decision perspective Journal Article
In: Decision Support Systems, pp. 114015, 2023, ISSN: 0167-9236.
@article{HAFIZ2023114015,
title = {Co-evolution of neural architectures and features for stock market forecasting: A multi-objective decision perspective},
author = {Faizal Hafiz and Jan Broekaert and Davide La Torre and Akshya Swain},
url = {https://www.sciencedirect.com/science/article/pii/S0167923623000908},
doi = {https://doi.org/10.1016/j.dss.2023.114015},
issn = {0167-9236},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Decision Support Systems},
pages = {114015},
abstract = {In a multi-objective setting, a portfolio manager’s highly consequential decisions can benefit from assessing alternative forecasting models of stock index movement. The present investigation proposes a new approach to identify a set of non-dominated neural network models for further selection by the decision-maker. A new co-evolution approach is proposed to simultaneously select the features and topology of neural networks (collectively referred to as neural architecture), where the features are viewed from a topological perspective as input neurons. Further, the co-evolution is posed as a multi-criteria problem to evolve sparse and efficacious neural architectures. The well-known dominance and decomposition based multi-objective evolutionary algorithms are augmented with a non-geometric crossover operator to diversify and balance the search for neural architectures across conflicting criteria. Moreover, the co-evolution is augmented to accommodate the data-based implications of distinct market behaviors prior to and during the ongoing COVID-19 pandemic. A detailed comparative evaluation is carried out with the conventional sequential approach of feature selection followed by neural topology design, as well as a scalarized co-evolution approach. The results on three market indices (NASDAQ, NYSE, and S&P500) in pre- and peri-COVID time windows convincingly demonstrate that the proposed co-evolution approach can evolve a set of non-dominated neural forecasting models with better generalization capabilities.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
In a multi-objective setting, a portfolio manager’s highly consequential decisions can benefit from assessing alternative forecasting models of stock index movement. The present investigation proposes a new approach to identify a set of non-dominated neural network models for further selection by the decision-maker. A new co-evolution approach is proposed to simultaneously select the features and topology of neural networks (collectively referred to as neural architecture), where the features are viewed from a topological perspective as input neurons. Further, the co-evolution is posed as a multi-criteria problem to evolve sparse and efficacious neural architectures. The well-known dominance and decomposition based multi-objective evolutionary algorithms are augmented with a non-geometric crossover operator to diversify and balance the search for neural architectures across conflicting criteria. Moreover, the co-evolution is augmented to accommodate the data-based implications of distinct market behaviors prior to and during the ongoing COVID-19 pandemic. A detailed comparative evaluation is carried out with the conventional sequential approach of feature selection followed by neural topology design, as well as a scalarized co-evolution approach. The results on three market indices (NASDAQ, NYSE, and S&P500) in pre- and peri-COVID time windows convincingly demonstrate that the proposed co-evolution approach can evolve a set of non-dominated neural forecasting models with better generalization capabilities.
2694.
Ye, Shaozhuang; Wang, Tuo; Ding, Mingyue; Zhang, Xuming
F-DARTS: Foveated Differentiable Architecture Search Based Multimodal Medical Image Fusion Journal Article
In: IEEE Transactions on Medical Imaging, pp. 1-1, 2023.
@article{10145413,
title = {F-DARTS: Foveated Differentiable Architecture Search Based Multimodal Medical Image Fusion},
author = {Shaozhuang Ye and Tuo Wang and Mingyue Ding and Xuming Zhang},
url = {https://ieeexplore.ieee.org/abstract/document/10145413},
doi = {10.1109/TMI.2023.3283517},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Medical Imaging},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2693.
Liu, Yang; Liu, Jing
A surrogate evolutionary neural architecture search algorithm for graph neural networks Journal Article
In: Applied Soft Computing, vol. 144, pp. 110485, 2023, ISSN: 1568-4946.
@article{LIU2023110485,
title = {A surrogate evolutionary neural architecture search algorithm for graph neural networks},
author = {Yang Liu and Jing Liu},
url = {https://www.sciencedirect.com/science/article/pii/S1568494623005033},
doi = {https://doi.org/10.1016/j.asoc.2023.110485},
issn = {1568-4946},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Applied Soft Computing},
volume = {144},
pages = {110485},
abstract = {Due to the unique construction module and design of graph neural networks (GNNs), neural architecture search (NAS) methods specifically for GNNs have become a promising research hotspot in recent years. Among the existing methods, one class of methods microscopically searches for the constituent components of network layers. However, most of them ignore the topology connections between network layers or the feature fusion strategies. Another class of methods, called differentiable architecture search methods, has the advantage of searching topology connections and feature fusion strategies. However, constrained by the requirement of predefining all candidate operations, these methods can only sample a limited number of network layers. In this paper, we propose a surrogate evolutionary graph neural architecture search (GNAS) algorithm whose search space contains not only the microscopic network layer components but also topology connections and feature fusion strategies (called CTFGNAS). The GNN sampled in CTFGNAS is represented by a simple one-dimensional vector and does not fix the network depth. To address the problem that traditional crossover and mutation operators applied to GNAS may produce illegal solutions, we design a repair operation to guarantee the legitimacy of the solutions. The network depth is also increased with a large probability in the mutation operation to alleviate the oversmoothing problem. In addition, to cope with the challenge of computational resources due to the increased search space, we form a surrogate model with three classical regression models, where only a small number of solutions are truly evaluated for their fitness, and the remaining large number of solutions are predicted for their fitness by the surrogate model. Finally, experiments are executed on six widely used real-world datasets. The experimental results illustrate that CTFGNAS obtains more effective results than the state-of-the-art handcrafted GNNs and GNAS methods on all datasets. CTFGNAS is now available on the following website: https://github.com/chnyliu/CTFGNAS.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Due to the unique construction module and design of graph neural networks (GNNs), neural architecture search (NAS) methods specifically for GNNs have become a promising research hotspot in recent years. Among the existing methods, one class of methods microscopically searches for the constituent components of network layers. However, most of them ignore the topology connections between network layers or the feature fusion strategies. Another class of methods, called differentiable architecture search methods, has the advantage of searching topology connections and feature fusion strategies. However, constrained by the requirement of predefining all candidate operations, these methods can only sample a limited number of network layers. In this paper, we propose a surrogate evolutionary graph neural architecture search (GNAS) algorithm whose search space contains not only the microscopic network layer components but also topology connections and feature fusion strategies (called CTFGNAS). The GNN sampled in CTFGNAS is represented by a simple one-dimensional vector and does not fix the network depth. To address the problem that traditional crossover and mutation operators applied to GNAS may produce illegal solutions, we design a repair operation to guarantee the legitimacy of the solutions. The network depth is also increased with a large probability in the mutation operation to alleviate the oversmoothing problem. In addition, to cope with the challenge of computational resources due to the increased search space, we form a surrogate model with three classical regression models, where only a small number of solutions are truly evaluated for their fitness, and the remaining large number of solutions are predicted for their fitness by the surrogate model. Finally, experiments are executed on six widely used real-world datasets. The experimental results illustrate that CTFGNAS obtains more effective results than the state-of-the-art handcrafted GNNs and GNAS methods on all datasets. CTFGNAS is now available on the following website: https://github.com/chnyliu/CTFGNAS.
2692.
Zhang, Qi; Peng, Yuanxi; Zhang, Zhiwen; Li, Teng
Semantic Segmentation of Spectral LiDAR Point Clouds Based on Neural Architecture Search Journal Article
In: IEEE Transactions on Geoscience and Remote Sensing, vol. 61, pp. 1-11, 2023.
@article{10147790,
title = {Semantic Segmentation of Spectral LiDAR Point Clouds Based on Neural Architecture Search},
author = {Qi Zhang and Yuanxi Peng and Zhiwen Zhang and Teng Li},
doi = {10.1109/TGRS.2023.3284995},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Geoscience and Remote Sensing},
volume = {61},
pages = {1-11},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2691.
Jawahar, Ganesh; Yang, Haichuan; Xiong, Yunyang; Liu, Zechun; Wang, Dilin; Sun, Fei; Li, Meng; Pappu, Aasish; Oguz, Barlas; Abdul-Mageed, Muhammad; Lakshmanan, Laks V. S.; Krishnamoorthi, Raghuraman; Chandra, Vikas
Mixture-of-Supernets: Improving Weight-Sharing Supernet Training with Architecture-Routed Mixture-of-Experts Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2306-04845,
title = {Mixture-of-Supernets: Improving Weight-Sharing Supernet Training with Architecture-Routed Mixture-of-Experts},
author = {Ganesh Jawahar and Haichuan Yang and Yunyang Xiong and Zechun Liu and Dilin Wang and Fei Sun and Meng Li and Aasish Pappu and Barlas Oguz and Muhammad Abdul-Mageed and Laks V. S. Lakshmanan and Raghuraman Krishnamoorthi and Vikas Chandra},
url = {https://doi.org/10.48550/arXiv.2306.04845},
doi = {10.48550/arXiv.2306.04845},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2306.04845},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2690.
Ha, Hyeonjeong; Kim, Minseon; Hwang, Sung Ju
Generalizable Lightweight Proxy for Robust NAS against Diverse Perturbations Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2306-05031,
title = {Generalizable Lightweight Proxy for Robust NAS against Diverse Perturbations},
author = {Hyeonjeong Ha and Minseon Kim and Sung Ju Hwang},
url = {https://doi.org/10.48550/arXiv.2306.05031},
doi = {10.48550/arXiv.2306.05031},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2306.05031},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2689.
Deng, Yuwen; Kang, Wang; Xing, Wei W.
Differentiable Multi-Fidelity Fusion: Efficient Learning of Physics Simulations with Neural Architecture Search and Transfer Learning Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2306-06904,
title = {Differentiable Multi-Fidelity Fusion: Efficient Learning of Physics Simulations with Neural Architecture Search and Transfer Learning},
author = {Yuwen Deng and Wang Kang and Wei W. Xing},
url = {https://doi.org/10.48550/arXiv.2306.06904},
doi = {10.48550/arXiv.2306.06904},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2306.06904},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2688.
Jidney, Tasmia Tahmida; Biswas, Angona; Nasim, Md Abdullah Al; Hossain, Ismail; Alam, Md Jahangir; Talukder, Sajedul; Hossain, Mofazzal; Ullah, Md. Azim
AutoML Systems For Medical Imaging Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2306-04750,
title = {AutoML Systems For Medical Imaging},
author = {Tasmia Tahmida Jidney and Angona Biswas and Md Abdullah Al Nasim and Ismail Hossain and Md Jahangir Alam and Sajedul Talukder and Mofazzal Hossain and Md. Azim Ullah},
url = {https://doi.org/10.48550/arXiv.2306.04750},
doi = {10.48550/arXiv.2306.04750},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2306.04750},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2687.
Salinas-Guerra, Rocío; Mejía-Dios, Jesús-Adolfo; Mezura-Montes, Efrén; Márquez-Grajales, Aldo
An Evolutionary Bilevel Optimization Approach for Neuroevolution Book Chapter
In: Castillo, Oscar; Melin, Patricia (Ed.): Hybrid Intelligent Systems Based on Extensions of Fuzzy Logic, Neural Networks and Metaheuristics, pp. 395–423, Springer Nature Switzerland, Cham, 2023, ISBN: 978-3-031-28999-6.
@inbook{Salinas-Guerra2023,
title = {An Evolutionary Bilevel Optimization Approach for Neuroevolution},
author = {Rocío Salinas-Guerra and Jesús-Adolfo Mejía-Dios and Efrén Mezura-Montes and Aldo Márquez-Grajales},
editor = {Oscar Castillo and Patricia Melin},
url = {https://doi.org/10.1007/978-3-031-28999-6_25},
doi = {10.1007/978-3-031-28999-6_25},
isbn = {978-3-031-28999-6},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Hybrid Intelligent Systems Based on Extensions of Fuzzy Logic, Neural Networks and Metaheuristics},
pages = {395–423},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {Convolutional neural networks (CNN) have been extensively studied and achieved significant progress on a variety of computer vision tasks in recent years. However, the design of their architectures remains challenging due to the computational cost and the number of parameters used. Neuroevolution has offered various evolutionary algorithms to provide a suitable option for designing CNNs. Moreover, modeling such a design as a bilevel optimization problem has recently attracted the interest of researchers and practitioners because it can be seen as a hierarchical task. This work precisely addresses it as a bilevel optimization problem. Unlike existing approaches, the upper level minimizes the complexity of the network (described by the number of its parameters), while the lower level optimizes the topology of the network structure for maximum accuracy. The results suggest that based on user preferences, the bilevel optimization approach can report neural architecture with higher accuracy values or simpler convolutional neural networks.},
keywords = {},
pubstate = {published},
tppubtype = {inbook}
}
Convolutional neural networks (CNN) have been extensively studied and achieved significant progress on a variety of computer vision tasks in recent years. However, the design of their architectures remains challenging due to the computational cost and the number of parameters used. Neuroevolution has offered various evolutionary algorithms to provide a suitable option for designing CNNs. Moreover, modeling such a design as a bilevel optimization problem has recently attracted the interest of researchers and practitioners because it can be seen as a hierarchical task. This work precisely addresses it as a bilevel optimization problem. Unlike existing approaches, the upper level minimizes the complexity of the network (described by the number of its parameters), while the lower level optimizes the topology of the network structure for maximum accuracy. The results suggest that based on user preferences, the bilevel optimization approach can report neural architecture with higher accuracy values or simpler convolutional neural networks.
2686.
Chauhan, Vinod Kumar; Zhou, Jiandong; Lu, Ping; Molaei, Soheila; Clifton, David A.
A Brief Review of Hypernetworks in Deep Learning Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2306-06955,
title = {A Brief Review of Hypernetworks in Deep Learning},
author = {Vinod Kumar Chauhan and Jiandong Zhou and Ping Lu and Soheila Molaei and David A. Clifton},
url = {https://doi.org/10.48550/arXiv.2306.06955},
doi = {10.48550/arXiv.2306.06955},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2306.06955},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2685.
Richey, Blake; Clay, Mitchell; Grecos, Christos; Shirvaikar, Mukul
Evolution of hardware-aware neural architecture search (NAS) on the edge Proceedings Article
In: Kehtarnavaz, Nasser; Shirvaikar, Mukul V. (Ed.): Real-Time Image Processing and Deep Learning 2023, pp. 125280A, International Society for Optics and Photonics SPIE, 2023.
@inproceedings{10.1117/12.2664894,
title = {Evolution of hardware-aware neural architecture search (NAS) on the edge},
author = {Blake Richey and Mitchell Clay and Christos Grecos and Mukul Shirvaikar},
editor = {Nasser Kehtarnavaz and Mukul V. Shirvaikar},
url = {https://doi.org/10.1117/12.2664894},
doi = {10.1117/12.2664894},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Real-Time Image Processing and Deep Learning 2023},
volume = {12528},
pages = {125280A},
publisher = {SPIE},
organization = {International Society for Optics and Photonics},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2684.
Lei, Tongfei; Hu, Jiabei; Riaz, Saleem
An innovative approach based on meta-learning for real-time modal fault diagnosis with small sample learning Journal Article
In: Frontiers in Physics, vol. 11, 2023, ISSN: 2296-424X.
@article{10.3389/fphy.2023.1207381,
title = {An innovative approach based on meta-learning for real-time modal fault diagnosis with small sample learning},
author = {Tongfei Lei and Jiabei Hu and Saleem Riaz},
url = {https://www.frontiersin.org/articles/10.3389/fphy.2023.1207381},
doi = {10.3389/fphy.2023.1207381},
issn = {2296-424X},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Frontiers in Physics},
volume = {11},
abstract = {The actual multimodal process data usually exhibit non-linear time correlation and non-Gaussian distribution accompanied by new modes. Existing fault diagnosis methods have difficulty adapting to the complex nature of new modalities and are unable to train models based on small samples. Therefore, this paper proposes a new modal fault diagnosis method based on meta-learning (ML) and neural architecture search (NAS), MetaNAS. Specifically, the best performing network model of the existing modal is first automatically obtained using NAS, and then, the fault diagnosis model design is learned from the NAS of the existing model using ML. Finally, when generating new modalities, the gradient is updated based on the learned design experience, i.e., new modal fault diagnosis models are quickly generated under small sample conditions. The effectiveness and feasibility of the proposed method are fully verified by the numerical system and simulation experiments of the Tennessee Eastman (TE) chemical process. As a primary goal, the abstract should render the general significance and conceptual advance of the work clearly accessible to a broad readership. References should not be cited in the abstract. Leave the Abstract empty if your article does not require one–please see the “Article types” on every Frontiers journal page for full details.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The actual multimodal process data usually exhibit non-linear time correlation and non-Gaussian distribution accompanied by new modes. Existing fault diagnosis methods have difficulty adapting to the complex nature of new modalities and are unable to train models based on small samples. Therefore, this paper proposes a new modal fault diagnosis method based on meta-learning (ML) and neural architecture search (NAS), MetaNAS. Specifically, the best performing network model of the existing modal is first automatically obtained using NAS, and then, the fault diagnosis model design is learned from the NAS of the existing model using ML. Finally, when generating new modalities, the gradient is updated based on the learned design experience, i.e., new modal fault diagnosis models are quickly generated under small sample conditions. The effectiveness and feasibility of the proposed method are fully verified by the numerical system and simulation experiments of the Tennessee Eastman (TE) chemical process. As a primary goal, the abstract should render the general significance and conceptual advance of the work clearly accessible to a broad readership. References should not be cited in the abstract. Leave the Abstract empty if your article does not require one–please see the “Article types” on every Frontiers journal page for full details.
2683.
Kang, Beom Woo; Wohn, Junho; Lee, Seongju; Park, Sunghyun; Noh, Yung-Kyun; Park, Yongjun
Synchronization-Aware NAS for an Efficient Collaborative Inference on Mobile Platforms Proceedings Article
In: Proceedings of the 24th ACM SIGPLAN/SIGBED International Conference on Languages, Compilers, and Tools for Embedded Systems, pp. 13–25, Association for Computing Machinery, Orlando, FL, USA, 2023, ISBN: 9798400701740.
@inproceedings{10.1145/3589610.3596284,
title = {Synchronization-Aware NAS for an Efficient Collaborative Inference on Mobile Platforms},
author = {Beom Woo Kang and Junho Wohn and Seongju Lee and Sunghyun Park and Yung-Kyun Noh and Yongjun Park},
url = {https://doi.org/10.1145/3589610.3596284},
doi = {10.1145/3589610.3596284},
isbn = {9798400701740},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Proceedings of the 24th ACM SIGPLAN/SIGBED International Conference on Languages, Compilers, and Tools for Embedded Systems},
pages = {13–25},
publisher = {Association for Computing Machinery},
address = {Orlando, FL, USA},
series = {LCTES 2023},
abstract = {Previous neural architecture search (NAS) approaches for mobile platforms have achieved great success in designing a slim-but-accurate neural network that is generally well-matched to a single computing unit such as a CPU or GPU. However, as recent mobile devices consist of multiple heterogeneous computing units, the next main challenge is to maximize both accuracy and efficiency by fully utilizing multiple available resources. We propose an ensemble-like approach with intermediate feature aggregations, namely synchronizations, for active collaboration between individual models on a mobile device. A main challenge is to determine the optimal synchronization strategies for achieving both performance and efficiency. To this end, we propose SyncNAS to automate the exploration of synchronization strategies for collaborative neural architectures that maximize utilization of heterogeneous computing units on a target device. We introduce a novel search space for synchronization strategy and apply Monte Carlo tree search (MCTS) algorithm to improve the sampling efficiency and reduce the search cost. On ImageNet, our collaborative model based on MobileNetV2 achieves 2.7% top-1 accuracy improvement within the baseline latency budget. Under the reduced target latency down to half, our model maintains higher accuracy than its baseline model, owing to the enhanced utilization and collaboration. As an impact of MCTS, SyncNAS reduces its search cost by up to 21x in searching for the optimal strategy.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Previous neural architecture search (NAS) approaches for mobile platforms have achieved great success in designing a slim-but-accurate neural network that is generally well-matched to a single computing unit such as a CPU or GPU. However, as recent mobile devices consist of multiple heterogeneous computing units, the next main challenge is to maximize both accuracy and efficiency by fully utilizing multiple available resources. We propose an ensemble-like approach with intermediate feature aggregations, namely synchronizations, for active collaboration between individual models on a mobile device. A main challenge is to determine the optimal synchronization strategies for achieving both performance and efficiency. To this end, we propose SyncNAS to automate the exploration of synchronization strategies for collaborative neural architectures that maximize utilization of heterogeneous computing units on a target device. We introduce a novel search space for synchronization strategy and apply Monte Carlo tree search (MCTS) algorithm to improve the sampling efficiency and reduce the search cost. On ImageNet, our collaborative model based on MobileNetV2 achieves 2.7% top-1 accuracy improvement within the baseline latency budget. Under the reduced target latency down to half, our model maintains higher accuracy than its baseline model, owing to the enhanced utilization and collaboration. As an impact of MCTS, SyncNAS reduces its search cost by up to 21x in searching for the optimal strategy.
2682.
Qin, Shixi; Zhang, Zixun; Jiang, Yuncheng; Cui, Shuguang; Cheng, Shenghui; Li, Zhen
NG-NAS: Node growth neural architecture search for 3D medical image segmentation Journal Article
In: Computerized Medical Imaging and Graphics, vol. 108, pp. 102268, 2023, ISSN: 0895-6111.
@article{QIN2023102268,
title = {NG-NAS: Node growth neural architecture search for 3D medical image segmentation},
author = {Shixi Qin and Zixun Zhang and Yuncheng Jiang and Shuguang Cui and Shenghui Cheng and Zhen Li},
url = {https://www.sciencedirect.com/science/article/pii/S0895611123000861},
doi = {https://doi.org/10.1016/j.compmedimag.2023.102268},
issn = {0895-6111},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Computerized Medical Imaging and Graphics},
volume = {108},
pages = {102268},
abstract = {Neural architecture search (NAS) has been applied to design proper 3D networks for medical image segmentation. In order to reduce the computation cost in NAS, researchers tend to adopt weight sharing mechanism to search architectures in a supernet. However, recent studies state that the searched architecture rankings may not be accurate with weight sharing mechanism because the training situations are inconsistent between the searching and training phases. In addition, some NAS algorithms design inflexible supernets that only search operators in a pre-defined backbone and ignore the importance of network topology, which limits the performance of searched architecture. To avoid weight sharing mechanism which may lead to inaccurate results and to comprehensively search network topology and operators, we propose a novel NAS algorithm called NG-NAS. Following the previous studies, we consider the segmentation network as a U-shape structure composed of a set of nodes. Instead of searching from the supernet with a limited search space, our NG-NAS starts from a simple architecture with only 5 nodes, and greedily grows the best candidate node until meeting the constraint. We design 2 kinds of node generations to form various network topological structures and prepare 4 candidate operators for each node. To efficiently evaluate candidate node generations, we use NAS without training strategies. We evaluate our method on several public 3D medical image segmentation benchmarks and achieve state-of-the-art performance, demonstrating the effectiveness of the searched architecture and our NG-NAS. Concretely, our method achieves an average Dice score of 85.11 on MSD liver, 65.70 on MSD brain, and 87.59 in BTCV, which performs much better than the previous SOTA methods.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Neural architecture search (NAS) has been applied to design proper 3D networks for medical image segmentation. In order to reduce the computation cost in NAS, researchers tend to adopt weight sharing mechanism to search architectures in a supernet. However, recent studies state that the searched architecture rankings may not be accurate with weight sharing mechanism because the training situations are inconsistent between the searching and training phases. In addition, some NAS algorithms design inflexible supernets that only search operators in a pre-defined backbone and ignore the importance of network topology, which limits the performance of searched architecture. To avoid weight sharing mechanism which may lead to inaccurate results and to comprehensively search network topology and operators, we propose a novel NAS algorithm called NG-NAS. Following the previous studies, we consider the segmentation network as a U-shape structure composed of a set of nodes. Instead of searching from the supernet with a limited search space, our NG-NAS starts from a simple architecture with only 5 nodes, and greedily grows the best candidate node until meeting the constraint. We design 2 kinds of node generations to form various network topological structures and prepare 4 candidate operators for each node. To efficiently evaluate candidate node generations, we use NAS without training strategies. We evaluate our method on several public 3D medical image segmentation benchmarks and achieve state-of-the-art performance, demonstrating the effectiveness of the searched architecture and our NG-NAS. Concretely, our method achieves an average Dice score of 85.11 on MSD liver, 65.70 on MSD brain, and 87.59 in BTCV, which performs much better than the previous SOTA methods.