Maintained by Difan Deng and Marius Lindauer.
The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.
Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.
Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.
2023
2886.
Huang, Lan; Sun, Shiqi; Zeng, Jia; Wang, Wencong; Pang, Wei; Wang, Kangping
U-DARTS: Uniform-space differentiable architecture search Journal Article
In: Information Sciences, vol. 628, pp. 339-349, 2023, ISSN: 0020-0255.
@article{HUANG2023339,
title = {U-DARTS: Uniform-space differentiable architecture search},
author = {Lan Huang and Shiqi Sun and Jia Zeng and Wencong Wang and Wei Pang and Kangping Wang},
url = {https://www.sciencedirect.com/science/article/pii/S002002552300141X},
doi = {https://doi.org/10.1016/j.ins.2023.01.129},
issn = {0020-0255},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Information Sciences},
volume = {628},
pages = {339-349},
abstract = {Differentiable architecture search (DARTS) is an effective neural architecture search algorithm based on gradient descent. However, there are two limitations in DARTS. First, a small proxy search space is exploited due to memory and computational resource constraints. Second, too many simple operations are preferred, which leads to the network deterioration. In this paper, we propose a uniform-space differentiable architecture search, named U-DARTS, to address the above problems. In one hand, the search space is redesigned to enable the search and evaluation of the architectures in the same space, and the new search space couples with a sampling and parameter sharing strategy to reduce resource overheads. This means that various cell structures are explored directly rather than cells with same structure are stacked to compose the network. In another hand, a regularization method, which takes the depth and the complexity of the operations into account, is proposed to prevent network deterioration. Our experiments show that U-DARTS is able to find excellent architectures. Specifically, we achieve an error rate of 2.59% with 3.3M parameters on CIFAR-10. The code is released in https://github.com/Sun-Shiqi/U-DARTS.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Differentiable architecture search (DARTS) is an effective neural architecture search algorithm based on gradient descent. However, there are two limitations in DARTS. First, a small proxy search space is exploited due to memory and computational resource constraints. Second, too many simple operations are preferred, which leads to the network deterioration. In this paper, we propose a uniform-space differentiable architecture search, named U-DARTS, to address the above problems. In one hand, the search space is redesigned to enable the search and evaluation of the architectures in the same space, and the new search space couples with a sampling and parameter sharing strategy to reduce resource overheads. This means that various cell structures are explored directly rather than cells with same structure are stacked to compose the network. In another hand, a regularization method, which takes the depth and the complexity of the operations into account, is proposed to prevent network deterioration. Our experiments show that U-DARTS is able to find excellent architectures. Specifically, we achieve an error rate of 2.59% with 3.3M parameters on CIFAR-10. The code is released in https://github.com/Sun-Shiqi/U-DARTS.
2885.
Ji, Zhanghexuan; Guo, Dazhou; Wang, Puyang; Yan, Ke; Lu, Le; Xu, Minfeng; Zhou, Jingren; Wang, Qifeng; Ge, Jia; Gao, Mingchen; Ye, Xianghua; Jin, Dakai
Continual Segment: Towards a Single, Unified and Accessible Continual Segmentation Model of 143 Whole-body Organs in CT Scans Technical Report
2023.
@techreport{https://doi.org/10.48550/arxiv.2302.00162,
title = {Continual Segment: Towards a Single, Unified and Accessible Continual Segmentation Model of 143 Whole-body Organs in CT Scans},
author = {Zhanghexuan Ji and Dazhou Guo and Puyang Wang and Ke Yan and Le Lu and Minfeng Xu and Jingren Zhou and Qifeng Wang and Jia Ge and Mingchen Gao and Xianghua Ye and Dakai Jin},
url = {https://arxiv.org/abs/2302.00162},
doi = {10.48550/ARXIV.2302.00162},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
publisher = {arXiv},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2884.
Kang, Jeon-Seong; Kang, JinKyu; Kim, Jung-Jun; Jeon, Kwang-Woo; Chung, Hyun-Joon; Park, Byung-Hoon
Neural Architecture Search Survey: A Computer Vision Perspective Journal Article
In: Sensors, vol. 23, no. 3, 2023, ISSN: 1424-8220.
@article{s23031713,
title = {Neural Architecture Search Survey: A Computer Vision Perspective},
author = {Jeon-Seong Kang and JinKyu Kang and Jung-Jun Kim and Kwang-Woo Jeon and Hyun-Joon Chung and Byung-Hoon Park},
url = {https://www.mdpi.com/1424-8220/23/3/1713},
doi = {10.3390/s23031713},
issn = {1424-8220},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Sensors},
volume = {23},
number = {3},
abstract = {In recent years, deep learning (DL) has been widely studied using various methods across the globe, especially with respect to training methods and network structures, proving highly effective in a wide range of tasks and applications, including image, speech, and text recognition. One important aspect of this advancement is involved in the effort of designing and upgrading neural architectures, which has been consistently attempted thus far. However, designing such architectures requires the combined knowledge and know-how of experts from each relevant discipline and a series of trial-and-error steps. In this light, automated neural architecture search (NAS) methods are increasingly at the center of attention; this paper aimed at summarizing the basic concepts of NAS while providing an overview of recent studies on the applications of NAS. It is worth noting that most previous survey studies on NAS have been focused on perspectives of hardware or search strategies. To the best knowledge of the present authors, this study is the first to look at NAS from a computer vision perspective. In the present study, computer vision areas were categorized by task, and recent trends found in each study on NAS were analyzed in detail.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
In recent years, deep learning (DL) has been widely studied using various methods across the globe, especially with respect to training methods and network structures, proving highly effective in a wide range of tasks and applications, including image, speech, and text recognition. One important aspect of this advancement is involved in the effort of designing and upgrading neural architectures, which has been consistently attempted thus far. However, designing such architectures requires the combined knowledge and know-how of experts from each relevant discipline and a series of trial-and-error steps. In this light, automated neural architecture search (NAS) methods are increasingly at the center of attention; this paper aimed at summarizing the basic concepts of NAS while providing an overview of recent studies on the applications of NAS. It is worth noting that most previous survey studies on NAS have been focused on perspectives of hardware or search strategies. To the best knowledge of the present authors, this study is the first to look at NAS from a computer vision perspective. In the present study, computer vision areas were categorized by task, and recent trends found in each study on NAS were analyzed in detail.
2883.
Rampavan, Medipelly; Ijjina, Earnest Paul
Genetic brake-net: Deep learning based brake light detection for collision avoidance using genetic algorithm Journal Article
In: Knowledge-Based Systems, pp. 110338, 2023, ISSN: 0950-7051.
@article{RAMPAVAN2023110338,
title = {Genetic brake-net: Deep learning based brake light detection for collision avoidance using genetic algorithm},
author = {Medipelly Rampavan and Earnest Paul Ijjina},
url = {https://www.sciencedirect.com/science/article/pii/S0950705123000886},
doi = {https://doi.org/10.1016/j.knosys.2023.110338},
issn = {0950-7051},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Knowledge-Based Systems},
pages = {110338},
abstract = {Automobiles are the primary means of transportation and increased traffic leads to the emphasis on techniques for safe transportation. Vehicle brake light detection is essential to avoid collisions among vehicles. Even though motorcycles are a common mode of transportation in many developing countries, little research has been done on motorcycle brake light detection. The effectiveness of Deep Neural Network (DNN) models has led to their adoption in different domains. The efficiency of the manually designed DNN architecture is dependent on the expert’s insight on optimality, which may not lead to an optimal model. Recently, Neural Architecture Search (NAS) has emerged as a method for automatically generating a task-specific backbone for object detection and classification tasks. In this work, we propose a genetic algorithm based NAS approach to construct a Mask R-CNN based object detection model. We designed the search space to include the architecture of the backbone in Mask R-CNN along with attributes used in training the object detection model. Genetic algorithm is used to explore the search space to find the optimal backbone architecture and training attributes. We achieved a mean accuracy of 97.14% and 89.44% for detecting brake light status for two-wheelers (on NITW-MBS dataset) and four-wheelers (on CaltechGraz dataset) respectively. The experimental study suggests that the architecture obtained using the proposed approach exhibits superior performance compared to existing models.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Automobiles are the primary means of transportation and increased traffic leads to the emphasis on techniques for safe transportation. Vehicle brake light detection is essential to avoid collisions among vehicles. Even though motorcycles are a common mode of transportation in many developing countries, little research has been done on motorcycle brake light detection. The effectiveness of Deep Neural Network (DNN) models has led to their adoption in different domains. The efficiency of the manually designed DNN architecture is dependent on the expert’s insight on optimality, which may not lead to an optimal model. Recently, Neural Architecture Search (NAS) has emerged as a method for automatically generating a task-specific backbone for object detection and classification tasks. In this work, we propose a genetic algorithm based NAS approach to construct a Mask R-CNN based object detection model. We designed the search space to include the architecture of the backbone in Mask R-CNN along with attributes used in training the object detection model. Genetic algorithm is used to explore the search space to find the optimal backbone architecture and training attributes. We achieved a mean accuracy of 97.14% and 89.44% for detecting brake light status for two-wheelers (on NITW-MBS dataset) and four-wheelers (on CaltechGraz dataset) respectively. The experimental study suggests that the architecture obtained using the proposed approach exhibits superior performance compared to existing models.
2882.
Sarti, Simone; Lomurno, Eugenio; Falanti, Andrea; Matteucci, Matteo
Enhancing Once-For-All: A Study on Parallel Blocks, Skip Connections and Early Exits Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2302-01888,
title = {Enhancing Once-For-All: A Study on Parallel Blocks, Skip Connections and Early Exits},
author = {Simone Sarti and Eugenio Lomurno and Andrea Falanti and Matteo Matteucci},
url = {https://doi.org/10.48550/arXiv.2302.01888},
doi = {10.48550/arXiv.2302.01888},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2302.01888},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2881.
Kulbach, Cedric Peter Charles
Adaptive Automated Machine Learning PhD Thesis
Karlsruher Institut für Technologie (KIT), 2023.
@phdthesis{Kulbach2023_1000155322,
title = {Adaptive Automated Machine Learning},
author = {Cedric Peter Charles Kulbach},
url = {https://publikationen.bibliothek.kit.edu/1000155322},
doi = {10.5445/IR/1000155322},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
publisher = {Karlsruher Institut für Technologie (KIT)},
school = {Karlsruher Institut für Technologie (KIT)},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
2880.
Wang, Chao; Jiao, Licheng; Zhao, Jiaxuan; Li, Lingling; Liu, Xu; Liu, Fang; Yang, Shuyuan
Bi-level Multi-objective Evolutionary Learning: A Case Study on Multi-task Graph Neural Topology Search Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2302-02565,
title = {Bi-level Multi-objective Evolutionary Learning: A Case Study on Multi-task Graph Neural Topology Search},
author = {Chao Wang and Licheng Jiao and Jiaxuan Zhao and Lingling Li and Xu Liu and Fang Liu and Shuyuan Yang},
url = {https://doi.org/10.48550/arXiv.2302.02565},
doi = {10.48550/arXiv.2302.02565},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2302.02565},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2879.
Gao, Yang; Zhang, Peng; Zhou, Chuan; Yang, Hong; Li, Zhao; Hu, Yue; Yu, Philip S.
HGNAS++: Efficient Architecture Search for Heterogeneous Graph Neural Networks Journal Article
In: IEEE Transactions on Knowledge and Data Engineering, pp. 1-14, 2023.
@article{10040227,
title = {HGNAS++: Efficient Architecture Search for Heterogeneous Graph Neural Networks},
author = {Yang Gao and Peng Zhang and Chuan Zhou and Hong Yang and Zhao Li and Yue Hu and Philip S. Yu},
doi = {10.1109/TKDE.2023.3239842},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Knowledge and Data Engineering},
pages = {1-14},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2878.
Zhou, Yuan; Hao, Jieke; Huo, Shuwei; Wang, Boyu; Ge, Leijiao; Kung, Sun-Yuan
Automatic Metric Search for Few-Shot Learning Journal Article
In: IEEE Transactions on Neural Networks and Learning Systems, pp. 1-12, 2023.
@article{10040944,
title = {Automatic Metric Search for Few-Shot Learning},
author = {Yuan Zhou and Jieke Hao and Shuwei Huo and Boyu Wang and Leijiao Ge and Sun-Yuan Kung},
doi = {10.1109/TNNLS.2023.3238729},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Neural Networks and Learning Systems},
pages = {1-12},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2877.
Äkiva-Hochman, Ruth; Finder, Shahaf E.; Turek, Javier S.; Treister, Eran"
Searching for N:M Fine-grained Sparsity of Weights and Activations in Neural Networks Proceedings Article
In: Karlinsky, Leonid; Michaeli, Tomer; Nishino, Ko (Ed.): Computer Vision -- ECCV 2022 Workshops, pp. 130–143, Springer Nature Switzerland, Cham, 2023, ISBN: 978-3-031-25082-8.
@inproceedings{10.1007/978-3-031-25082-8_9,
title = {Searching for N:M Fine-grained Sparsity of Weights and Activations in Neural Networks},
author = {Ruth Äkiva-Hochman and Shahaf E. Finder and Javier S. Turek and Eran" Treister},
editor = {Leonid Karlinsky and Tomer Michaeli and Ko Nishino},
isbn = {978-3-031-25082-8},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Computer Vision -- ECCV 2022 Workshops},
pages = {130--143},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {Sparsity in deep neural networks has been extensively studied to compress and accelerate models for environments with limited resources. The general approach of pruning aims at enforcing sparsity on the obtained model, with minimal accuracy loss, but with a sparsity structure that enables acceleration on hardware. The sparsity can be enforced on either the weights or activations of the network, and existing works tend to focus on either one for the entire network. In this paper, we suggest a strategy based on Neural Architecture Search (NAS) to sparsify both activations and weights throughout the network, while utilizing the recent approach of N:M fine-grained structured sparsity that enables practical acceleration on dedicated GPUs. We show that a combination of weight and activation pruning is superior to each option separately. Furthermore, during the training, the choice between pruning the weights of activations can be motivated by practical inference costs (e.g., memory bandwidth). We demonstrate the efficiency of the approach on several image classification datasets.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Sparsity in deep neural networks has been extensively studied to compress and accelerate models for environments with limited resources. The general approach of pruning aims at enforcing sparsity on the obtained model, with minimal accuracy loss, but with a sparsity structure that enables acceleration on hardware. The sparsity can be enforced on either the weights or activations of the network, and existing works tend to focus on either one for the entire network. In this paper, we suggest a strategy based on Neural Architecture Search (NAS) to sparsify both activations and weights throughout the network, while utilizing the recent approach of N:M fine-grained structured sparsity that enables practical acceleration on dedicated GPUs. We show that a combination of weight and activation pruning is superior to each option separately. Furthermore, during the training, the choice between pruning the weights of activations can be motivated by practical inference costs (e.g., memory bandwidth). We demonstrate the efficiency of the approach on several image classification datasets.
2876.
Ünal, Hamit Taner; Başçiftçi, Fatih
Neural Logic Circuits: An evolutionary neural architecture that can learn and generalize Journal Article
In: Knowledge-Based Systems, vol. 265, pp. 110379, 2023, ISSN: 0950-7051.
@article{UNAL2023110379,
title = {Neural Logic Circuits: An evolutionary neural architecture that can learn and generalize},
author = {Hamit Taner Ünal and Fatih Başçiftçi},
url = {https://www.sciencedirect.com/science/article/pii/S0950705123001296},
doi = {https://doi.org/10.1016/j.knosys.2023.110379},
issn = {0950-7051},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Knowledge-Based Systems},
volume = {265},
pages = {110379},
abstract = {We introduce Neural Logic Circuits (NLC), an evolutionary, weightless, and learnable neural architecture loosely inspired by the neuroplasticity of the brain. This new paradigm achieves learning by evolution of its architecture through reorganization of augmenting synaptic connections and generation of artificial neurons functioning as logic gates. These neural units mimic biological nerve cells stimulated by binary input signals and emit excitatory or inhibitory pulses, thus executing the “all-or-none” character of their natural counterparts. Unlike Artificial Neural Networks (ANN), our model achieves generalization ability without intensive weight training and dedicates computational resources solely to building network architecture with optimal connectivity. We evaluated our model on well-known binary classification datasets using advanced performance metrics and compared results with modern and competitive machine learning algorithms. Extensive experimental data reveal remarkable superiority of our initial model, called NLCv1, on all test instances, achieving outstanding results for implementation of this new paradigm.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
We introduce Neural Logic Circuits (NLC), an evolutionary, weightless, and learnable neural architecture loosely inspired by the neuroplasticity of the brain. This new paradigm achieves learning by evolution of its architecture through reorganization of augmenting synaptic connections and generation of artificial neurons functioning as logic gates. These neural units mimic biological nerve cells stimulated by binary input signals and emit excitatory or inhibitory pulses, thus executing the “all-or-none” character of their natural counterparts. Unlike Artificial Neural Networks (ANN), our model achieves generalization ability without intensive weight training and dedicates computational resources solely to building network architecture with optimal connectivity. We evaluated our model on well-known binary classification datasets using advanced performance metrics and compared results with modern and competitive machine learning algorithms. Extensive experimental data reveal remarkable superiority of our initial model, called NLCv1, on all test instances, achieving outstanding results for implementation of this new paradigm.
2875.
Jin, Guangyin; Sha, Hengyu; Xi, Zhexu; Huang, Jincai
Urban hotspot forecasting via automated spatio-temporal information fusion Journal Article
In: Applied Soft Computing, vol. 136, pp. 110087, 2023, ISSN: 1568-4946.
@article{JIN2023110087,
title = {Urban hotspot forecasting via automated spatio-temporal information fusion},
author = {Guangyin Jin and Hengyu Sha and Zhexu Xi and Jincai Huang},
url = {https://www.sciencedirect.com/science/article/pii/S1568494623001059},
doi = {https://doi.org/10.1016/j.asoc.2023.110087},
issn = {1568-4946},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Applied Soft Computing},
volume = {136},
pages = {110087},
abstract = {Urban hotspot forecasting is one of the most important tasks for resource scheduling and security in future smart cities. Most previous works employed fixed neural architectures based on many complicated spatial and temporal learning modules. However, designing appropriate neural architectures is challenging for urban hotspot forecasting. One reason is that there is currently no adequate support system for how to fuse multi-scale spatio-temporal information rationally by integrating different spatial and temporal learning modules. Another one is that the empirical fixed neural architecture is difficult to adapt to different data scenarios from different domains or cities. To address the above problems, we propose a novel framework based on neural architecture search for urban hotspot forecasting, namely Automated Spatio-Temporal Information Fusion Neural Network (ASTIF-Net). In the search space of our ASTIF-Net, normal convolution and graph convolution operations are adopted to capture spatial geographic neighborhood dependencies and spatial semantic neighborhood dependencies, and different types of temporal convolution operations are adopted to capture short-term and long-term temporal dependencies. In addition to combining spatio-temporal learning operations from different scales, ASTIF-Net can also search appropriate fusion methods for aggregating multi-scale spatio-temporal hidden information. We conduct extensive experiments to evaluate ASTIF-Net on three real-world urban hotspot datasets from different domains to demonstrate that our proposed model can obtain effective neural architectures and achieve superior performance (about 5%∼10% improvements) compared with the existing state-of-art baselines.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Urban hotspot forecasting is one of the most important tasks for resource scheduling and security in future smart cities. Most previous works employed fixed neural architectures based on many complicated spatial and temporal learning modules. However, designing appropriate neural architectures is challenging for urban hotspot forecasting. One reason is that there is currently no adequate support system for how to fuse multi-scale spatio-temporal information rationally by integrating different spatial and temporal learning modules. Another one is that the empirical fixed neural architecture is difficult to adapt to different data scenarios from different domains or cities. To address the above problems, we propose a novel framework based on neural architecture search for urban hotspot forecasting, namely Automated Spatio-Temporal Information Fusion Neural Network (ASTIF-Net). In the search space of our ASTIF-Net, normal convolution and graph convolution operations are adopted to capture spatial geographic neighborhood dependencies and spatial semantic neighborhood dependencies, and different types of temporal convolution operations are adopted to capture short-term and long-term temporal dependencies. In addition to combining spatio-temporal learning operations from different scales, ASTIF-Net can also search appropriate fusion methods for aggregating multi-scale spatio-temporal hidden information. We conduct extensive experiments to evaluate ASTIF-Net on three real-world urban hotspot datasets from different domains to demonstrate that our proposed model can obtain effective neural architectures and achieve superior performance (about 5%∼10% improvements) compared with the existing state-of-art baselines.
2874.
Huang, Mingqiang; Liu, Yucen; Huang, Sixiao; Li, Kai; Wu, Qiuping; Yu, Hao
Multi-Bit-Width CNN Accelerator with Systolic-in-Systolic Dataflow and Single DSP Multiple Multiplication Scheme Proceedings Article
In: Proceedings of the 2023 ACM/SIGDA International Symposium on Field Programmable Gate Arrays, pp. 229, Association for Computing Machinery, Monterey, CA, USA, 2023, ISBN: 9781450394178.
@inproceedings{10.1145/3543622.3573209,
title = {Multi-Bit-Width CNN Accelerator with Systolic-in-Systolic Dataflow and Single DSP Multiple Multiplication Scheme},
author = {Mingqiang Huang and Yucen Liu and Sixiao Huang and Kai Li and Qiuping Wu and Hao Yu},
url = {https://doi.org/10.1145/3543622.3573209},
doi = {10.1145/3543622.3573209},
isbn = {9781450394178},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Proceedings of the 2023 ACM/SIGDA International Symposium on Field Programmable Gate Arrays},
pages = {229},
publisher = {Association for Computing Machinery},
address = {Monterey, CA, USA},
series = {FPGA '23},
abstract = {Multi-bit-width neural network enlightens a promising method for high performance yet energy efficient edge computing due to its balance between software algorithm accuracy and hardware efficiency. To date, FPGA has been one of the core hardware platforms for deploying various neural networks. However, it is still difficult to fully make use of the dedicated digital signal processing (DSP) blocks in FPGA for accelerating the multi-bit-width network. In this work, we develop state-of-the-art multi-bit-width convolutional neural network accelerator with novel systolic-in-systolic type of dataflow and single DSP multiple multiplication (SDMM) INT2/4/8 execution scheme. Multi-level optimizations have also been adopted to further improve the performance, including group-vector systolic array for maximizing the circuit efficiency as well as minimizing the systolic delay, and differential neural architecture search (NAS) method for the high accuracy multi-bit-width network generation. The proposed accelerator has been practically deployed on Xilinx ZCU102 with accelerating NAS optimized VGG16 and Resnet18 networks as case studies. Average performance on accelerating the convolutional layer in VGG16 and Resnet18 is 1289GOPs and 1155GOPs, respectively. Throughput for running the full multi-bit-width VGG16 network is 870.73 GOPS at 250MHz, which has exceeded all of previous CNN accelerators on the same platform.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Multi-bit-width neural network enlightens a promising method for high performance yet energy efficient edge computing due to its balance between software algorithm accuracy and hardware efficiency. To date, FPGA has been one of the core hardware platforms for deploying various neural networks. However, it is still difficult to fully make use of the dedicated digital signal processing (DSP) blocks in FPGA for accelerating the multi-bit-width network. In this work, we develop state-of-the-art multi-bit-width convolutional neural network accelerator with novel systolic-in-systolic type of dataflow and single DSP multiple multiplication (SDMM) INT2/4/8 execution scheme. Multi-level optimizations have also been adopted to further improve the performance, including group-vector systolic array for maximizing the circuit efficiency as well as minimizing the systolic delay, and differential neural architecture search (NAS) method for the high accuracy multi-bit-width network generation. The proposed accelerator has been practically deployed on Xilinx ZCU102 with accelerating NAS optimized VGG16 and Resnet18 networks as case studies. Average performance on accelerating the convolutional layer in VGG16 and Resnet18 is 1289GOPs and 1155GOPs, respectively. Throughput for running the full multi-bit-width VGG16 network is 870.73 GOPS at 250MHz, which has exceeded all of previous CNN accelerators on the same platform.
2873.
Wang, Xueying; Li, Guangli; Ma, Xiu; Feng, Xiaobing
Facilitating hardware-aware neural architecture search with learning-based predictive models Journal Article
In: Journal of Systems Architecture, vol. 137, pp. 102838, 2023, ISSN: 1383-7621.
@article{WANG2023102838,
title = {Facilitating hardware-aware neural architecture search with learning-based predictive models},
author = {Xueying Wang and Guangli Li and Xiu Ma and Xiaobing Feng},
url = {https://www.sciencedirect.com/science/article/pii/S1383762123000176},
doi = {https://doi.org/10.1016/j.sysarc.2023.102838},
issn = {1383-7621},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Journal of Systems Architecture},
volume = {137},
pages = {102838},
abstract = {Neural architecture search (NAS), which automatically explores the efficient model design, has achieved ground-breaking advances in recent years. To achieve the optimal model latency on deployment platforms, a performance tuning process is usually needed to select reasonable parameters and implementations for each neural network operator. As the tuning process is time-consuming, it is impractical for tuning each candidate architecture generated in the search procedure. Recent NAS systems usually utilize theoretical metrics or rule-based heuristics on-device latency to approximately estimate the model performance. Nevertheless, we discovered that there is still a gap between the estimated latency and the optimal latency, potentially causing a sub-optimal solution for neural architecture search. This paper presents an accurate and efficient approach for estimating the practical model latency on target platforms, which employs lightweight learning-based predictive models (LBPMs) to assist to obtain the realistic deployment-time model latency with acceptable run-time overhead, thereby facilitating hardware-aware neural architecture search. We propose an LBPM-based NAS framework, LBPM-NAS, and evaluate it by searching model architectures for ImageNet classification and facial landmark localization tasks on various hardware platforms. Experimental results show that the LBPM-NAS achieves up to 2.4× performance boost compared with the baselines under the same-level accuracy.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Neural architecture search (NAS), which automatically explores the efficient model design, has achieved ground-breaking advances in recent years. To achieve the optimal model latency on deployment platforms, a performance tuning process is usually needed to select reasonable parameters and implementations for each neural network operator. As the tuning process is time-consuming, it is impractical for tuning each candidate architecture generated in the search procedure. Recent NAS systems usually utilize theoretical metrics or rule-based heuristics on-device latency to approximately estimate the model performance. Nevertheless, we discovered that there is still a gap between the estimated latency and the optimal latency, potentially causing a sub-optimal solution for neural architecture search. This paper presents an accurate and efficient approach for estimating the practical model latency on target platforms, which employs lightweight learning-based predictive models (LBPMs) to assist to obtain the realistic deployment-time model latency with acceptable run-time overhead, thereby facilitating hardware-aware neural architecture search. We propose an LBPM-based NAS framework, LBPM-NAS, and evaluate it by searching model architectures for ImageNet classification and facial landmark localization tasks on various hardware platforms. Experimental results show that the LBPM-NAS achieves up to 2.4× performance boost compared with the baselines under the same-level accuracy.
2872.
Chatzianastasis, Michail; Ilias, Loukas; Askounis, Dimitris; Vazirgiannis, Michalis
Neural Architecture Search with Multimodal Fusion Methods for Diagnosing Dementia Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2302-05894,
title = {Neural Architecture Search with Multimodal Fusion Methods for Diagnosing Dementia},
author = {Michail Chatzianastasis and Loukas Ilias and Dimitris Askounis and Michalis Vazirgiannis},
url = {https://doi.org/10.48550/arXiv.2302.05894},
doi = {10.48550/arXiv.2302.05894},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2302.05894},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2871.
Zhu, Xunyu; Li, Jian; Liu, Yong; Wang, Weiping
Improving Differentiable Architecture Search via Self-Distillation Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2302-05629,
title = {Improving Differentiable Architecture Search via Self-Distillation},
author = {Xunyu Zhu and Jian Li and Yong Liu and Weiping Wang},
url = {https://doi.org/10.48550/arXiv.2302.05629},
doi = {10.48550/arXiv.2302.05629},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2302.05629},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2870.
Wang, Zhe; Yang, Fangfang; Xu, Qiang; Wang, Yongjian; Yan, Hong; Xie, Min
Capacity estimation of lithium-ion batteries based on data aggregation and feature fusion via graph neural network Journal Article
In: Applied Energy, vol. 336, pp. 120808, 2023, ISSN: 0306-2619.
@article{WANG2023120808,
title = {Capacity estimation of lithium-ion batteries based on data aggregation and feature fusion via graph neural network},
author = {Zhe Wang and Fangfang Yang and Qiang Xu and Yongjian Wang and Hong Yan and Min Xie},
url = {https://www.sciencedirect.com/science/article/pii/S0306261923001721},
doi = {https://doi.org/10.1016/j.apenergy.2023.120808},
issn = {0306-2619},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Applied Energy},
volume = {336},
pages = {120808},
abstract = {Lithium-ion batteries in electrical devices face inevitable degradation along with the long-term usage. The accompanying battery capacity estimation is crucial for battery health management. However, the hand-crafted feature engineering in traditional methods and complicated network design followed by the laborious trial in data-driven methods hinder efficient capacity estimation. In this work, the battery measurements from different sensors are organized as the graph structure and comprehensively utilized based on graph neural network. The feature fusion is further designed to enhance the network capacity. The specific data aggregation and feature fusion operations are selected by neural architecture search, which relieves the network design and increases the adaptability. Two public datasets are adopted to verify the effectiveness of the proposed scheme. Additional discussions are conducted to emphasize the capability of the graph neural network and the necessity of architecture searching. The comparison analysis and the performance under noisy environment further demonstrate the superiority of proposed scheme.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Lithium-ion batteries in electrical devices face inevitable degradation along with the long-term usage. The accompanying battery capacity estimation is crucial for battery health management. However, the hand-crafted feature engineering in traditional methods and complicated network design followed by the laborious trial in data-driven methods hinder efficient capacity estimation. In this work, the battery measurements from different sensors are organized as the graph structure and comprehensively utilized based on graph neural network. The feature fusion is further designed to enhance the network capacity. The specific data aggregation and feature fusion operations are selected by neural architecture search, which relieves the network design and increases the adaptability. Two public datasets are adopted to verify the effectiveness of the proposed scheme. Additional discussions are conducted to emphasize the capability of the graph neural network and the necessity of architecture searching. The comparison analysis and the performance under noisy environment further demonstrate the superiority of proposed scheme.
2869.
Ismail, Walaa N.; Alsalamah, Hessah A.; Hassan, Mohammad Mehedi; Mohamed, Ebtesam
AUTO-HAR: An adaptive human activity recognition framework using an automated CNN architecture design Journal Article
In: Heliyon, vol. 9, no. 2, pp. e13636, 2023, ISSN: 2405-8440.
@article{ISMAIL2023e13636,
title = {AUTO-HAR: An adaptive human activity recognition framework using an automated CNN architecture design},
author = {Walaa N. Ismail and Hessah A. Alsalamah and Mohammad Mehedi Hassan and Ebtesam Mohamed},
url = {https://www.sciencedirect.com/science/article/pii/S2405844023008435},
doi = {https://doi.org/10.1016/j.heliyon.2023.e13636},
issn = {2405-8440},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Heliyon},
volume = {9},
number = {2},
pages = {e13636},
abstract = {Convolutional neural networks (CNNs) have demonstrated exceptional results in the analysis of time- series data when used for Human Activity Recognition (HAR). The manual design of such neural architectures is an error-prone and time-consuming process. The search for optimal CNN architectures is considered a revolution in the design of neural networks. By means of Neural Architecture Search (NAS), network architectures can be designed and optimized automatically. Thus, the optimal CNN architecture representation can be found automatically because of its ability to overcome the limitations of human experience and thinking modes. Evolution algorithms, which are derived from evolutionary mechanisms such as natural selection and genetics, have been widely employed to develop and optimize NAS because they can handle a blackbox optimization process for designing appropriate solution representations and search paradigms without explicit mathematical formulations or gradient information. The Genetic optimization algorithm (GA) is widely used to find optimal or near-optimal solutions for difficult problems. Considering these characteristics, an efficient human activity recognition architecture (AUTO-HAR) is presented in this study. Using the evolutionary GA to select the optimal CNN architecture, the current study proposes a novel encoding schema structure and a novel search space with a much broader range of operations to effectively search for the best architectures for HAR tasks. In addition, the proposed search space provides a reasonable degree of depth because it does not limit the maximum length of the devised task architecture. To test the effectiveness of the proposed framework for HAR tasks, three datasets were utilized: UCI-HAR, Opportunity, and DAPHNET. Based on the results of this study, it has been found that the proposed method can efficiently recognize human activity with an average accuracy of 98.5% (∓1.1), 98.3%, and 99.14% (∓0.8) for UCI-HAR, Opportunity, and DAPHNET, respectively.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Convolutional neural networks (CNNs) have demonstrated exceptional results in the analysis of time- series data when used for Human Activity Recognition (HAR). The manual design of such neural architectures is an error-prone and time-consuming process. The search for optimal CNN architectures is considered a revolution in the design of neural networks. By means of Neural Architecture Search (NAS), network architectures can be designed and optimized automatically. Thus, the optimal CNN architecture representation can be found automatically because of its ability to overcome the limitations of human experience and thinking modes. Evolution algorithms, which are derived from evolutionary mechanisms such as natural selection and genetics, have been widely employed to develop and optimize NAS because they can handle a blackbox optimization process for designing appropriate solution representations and search paradigms without explicit mathematical formulations or gradient information. The Genetic optimization algorithm (GA) is widely used to find optimal or near-optimal solutions for difficult problems. Considering these characteristics, an efficient human activity recognition architecture (AUTO-HAR) is presented in this study. Using the evolutionary GA to select the optimal CNN architecture, the current study proposes a novel encoding schema structure and a novel search space with a much broader range of operations to effectively search for the best architectures for HAR tasks. In addition, the proposed search space provides a reasonable degree of depth because it does not limit the maximum length of the devised task architecture. To test the effectiveness of the proposed framework for HAR tasks, three datasets were utilized: UCI-HAR, Opportunity, and DAPHNET. Based on the results of this study, it has been found that the proposed method can efficiently recognize human activity with an average accuracy of 98.5% (∓1.1), 98.3%, and 99.14% (∓0.8) for UCI-HAR, Opportunity, and DAPHNET, respectively.
2868.
Gillard, Ryan; Jonany, Stephen; Miao, Yingjie; Munn, Michael; Souza, Connal; Dungay, Jonathan; Liang, Chen; So, David R.; Le, Quoc V.; Real, Esteban
Unified Functional Hashing in Automatic Machine Learning Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2302-05433,
title = {Unified Functional Hashing in Automatic Machine Learning},
author = {Ryan Gillard and Stephen Jonany and Yingjie Miao and Michael Munn and Connal Souza and Jonathan Dungay and Chen Liang and David R. So and Quoc V. Le and Esteban Real},
url = {https://doi.org/10.48550/arXiv.2302.05433},
doi = {10.48550/arXiv.2302.05433},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2302.05433},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2867.
Romero, David W.; Zeghidour, Neil
DNArch: Learning Convolutional Neural Architectures by Backpropagation Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2302-05400,
title = {DNArch: Learning Convolutional Neural Architectures by Backpropagation},
author = {David W. Romero and Neil Zeghidour},
url = {https://doi.org/10.48550/arXiv.2302.05400},
doi = {10.48550/arXiv.2302.05400},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2302.05400},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2866.
Zhang, Jinxia; Chen, Xinyi; Wei, Haikun; Zhang, Kanjian
A lightweight network for photovoltaic cell defect detection in electroluminescence images based on neural architecture search and knowledge distillation Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2302-07455,
title = {A lightweight network for photovoltaic cell defect detection in electroluminescence images based on neural architecture search and knowledge distillation},
author = {Jinxia Zhang and Xinyi Chen and Haikun Wei and Kanjian Zhang},
url = {https://doi.org/10.48550/arXiv.2302.07455},
doi = {10.48550/arXiv.2302.07455},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2302.07455},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2865.
Yuan, Gonglin; Wang, Bin; Xue, Bing; Zhang, Mengjie
Particle Swarm Optimization for Efficiently Evolving Deep Convolutional Neural Networks Using an Autoencoder-based Encoding Strategy Journal Article
In: IEEE Transactions on Evolutionary Computation, pp. 1-1, 2023.
@article{10045029,
title = {Particle Swarm Optimization for Efficiently Evolving Deep Convolutional Neural Networks Using an Autoencoder-based Encoding Strategy},
author = {Gonglin Yuan and Bin Wang and Bing Xue and Mengjie Zhang},
doi = {10.1109/TEVC.2023.3245322},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Evolutionary Computation},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2864.
Bhattacharjee, Abhiroop; Moitra, Abhishek; Panda, Priyadarshini
XploreNAS: Explore Adversarially Robust & Hardware-efficient Neural Architectures for Non-ideal Xbars Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2302-07769,
title = {XploreNAS: Explore Adversarially Robust & Hardware-efficient Neural Architectures for Non-ideal Xbars},
author = {Abhiroop Bhattacharjee and Abhishek Moitra and Priyadarshini Panda},
url = {https://doi.org/10.48550/arXiv.2302.07769},
doi = {10.48550/arXiv.2302.07769},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2302.07769},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2863.
Cheng, Guangliang; Sun, Peng; Xu, Ting-Bing; Lyu, Shuchang; Lin, Peiwen
Local-to-Global Information Communication for Real-Time Semantic Segmentation Network Search Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2302-08481,
title = {Local-to-Global Information Communication for Real-Time Semantic Segmentation Network Search},
author = {Guangliang Cheng and Peng Sun and Ting-Bing Xu and Shuchang Lyu and Peiwen Lin},
url = {https://doi.org/10.48550/arXiv.2302.08481},
doi = {10.48550/arXiv.2302.08481},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2302.08481},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2862.
Mohammadrezaei, Parsa; Aminan, Mohammad; Soltanian, Mohammad; Borna, Keivan
Improving CNN-based solutions for emotion recognition using evolutionary algorithms Journal Article
In: Results in Applied Mathematics, vol. 18, pp. 100360, 2023, ISSN: 2590-0374.
@article{MOHAMMADREZAEI2023100360,
title = {Improving CNN-based solutions for emotion recognition using evolutionary algorithms},
author = {Parsa Mohammadrezaei and Mohammad Aminan and Mohammad Soltanian and Keivan Borna},
url = {https://www.sciencedirect.com/science/article/pii/S2590037423000067},
doi = {https://doi.org/10.1016/j.rinam.2023.100360},
issn = {2590-0374},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Results in Applied Mathematics},
volume = {18},
pages = {100360},
abstract = {AI-based approaches, especially deep learning have made remarkable achievements in Speech Emotion Recognition (SER). Needless to say, Convolutional Neural Networks (CNNs) have been the backbone of many of these solutions. Although the use of CNNs have resulted in high performing models, building them needs domain knowledge and direct human intervention. The same issue arises while improving a model. To solve this problem, we use techniques that were firstly introduced in Neural Architecture Search (NAS) and use a genetic process to search for models with improved accuracy. More specifically, we insert blocks with dynamic structures in between the layers of an already existing model and then use genetic operations (i.e. selection, mutation, and crossover) to find the best performing structures. To validate our method, we use this algorithm to improve architectures by searching on the Berlin Database of Emotional Speech (EMODB). The experimental results show at least 1.7% performance improvement in terms of Accuracy on EMODB test set.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
AI-based approaches, especially deep learning have made remarkable achievements in Speech Emotion Recognition (SER). Needless to say, Convolutional Neural Networks (CNNs) have been the backbone of many of these solutions. Although the use of CNNs have resulted in high performing models, building them needs domain knowledge and direct human intervention. The same issue arises while improving a model. To solve this problem, we use techniques that were firstly introduced in Neural Architecture Search (NAS) and use a genetic process to search for models with improved accuracy. More specifically, we insert blocks with dynamic structures in between the layers of an already existing model and then use genetic operations (i.e. selection, mutation, and crossover) to find the best performing structures. To validate our method, we use this algorithm to improve architectures by searching on the Berlin Database of Emotional Speech (EMODB). The experimental results show at least 1.7% performance improvement in terms of Accuracy on EMODB test set.
2861.
Gao, Si; Zheng, Chengjian; Zhang, Xiaofeng; Liu, Shaoli; Wu, Biao; Lu, Kaidi; Zhang, Diankai; Wang, Ning
RCBSR: Re-parameterization Convolution Block for Super-Resolution Proceedings Article
In: Karlinsky, Leonid; Michaeli, Tomer; Nishino, Ko (Ed.): Computer Vision -- ECCV 2022 Workshops, pp. 540–548, Springer Nature Switzerland, Cham, 2023, ISBN: 978-3-031-25063-7.
@inproceedings{10.1007/978-3-031-25063-7_33,
title = {RCBSR: Re-parameterization Convolution Block for Super-Resolution},
author = {Si Gao and Chengjian Zheng and Xiaofeng Zhang and Shaoli Liu and Biao Wu and Kaidi Lu and Diankai Zhang and Ning Wang},
editor = {Leonid Karlinsky and Tomer Michaeli and Ko Nishino},
isbn = {978-3-031-25063-7},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Computer Vision -- ECCV 2022 Workshops},
pages = {540--548},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {Super resolution(SR) with high efficiency and low power consumption is highly demanded in the actual application scenes. In this paper, We designed a super light-weight SR network with strong feature expression. The network we proposed is named RCBSR. Based on the novel technique of re-parameterization, we adopt a block with multiple paths structure in the training stage and merge multiple paths structure into one single 3$$backslashtimes $$texttimes3 convolution in the inference stage. And then the neural architecture search(NAS) method is adopted to determine amounts of block M and amounts of channel C. Finally, the proposed SR network achieves a fairly good result of PSNR(27.52 dB) with power consumption(0.1 W@30 fps) on the MediaTek Dimensity 9000 platform in the challenge testing stage.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Super resolution(SR) with high efficiency and low power consumption is highly demanded in the actual application scenes. In this paper, We designed a super light-weight SR network with strong feature expression. The network we proposed is named RCBSR. Based on the novel technique of re-parameterization, we adopt a block with multiple paths structure in the training stage and merge multiple paths structure into one single 3$$backslashtimes $$texttimes3 convolution in the inference stage. And then the neural architecture search(NAS) method is adopted to determine amounts of block M and amounts of channel C. Finally, the proposed SR network achieves a fairly good result of PSNR(27.52 dB) with power consumption(0.1 W@30 fps) on the MediaTek Dimensity 9000 platform in the challenge testing stage.
2860.
Maulik, Romit; Egele, Romain; Raghavan, Krishnan; Balaprakash, Prasanna
Quantifying uncertainty for deep learning based forecasting and flow-reconstruction using neural architecture search ensembles Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2302-09748,
title = {Quantifying uncertainty for deep learning based forecasting and flow-reconstruction using neural architecture search ensembles},
author = {Romit Maulik and Romain Egele and Krishnan Raghavan and Prasanna Balaprakash},
url = {https://doi.org/10.48550/arXiv.2302.09748},
doi = {10.48550/arXiv.2302.09748},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2302.09748},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2859.
Wei, Lanning; He, Zhiqiang; Zhao, Huan; Yao, Quanming
Search to Capture Long-range Dependency with Stacking GNNs for Graph Classification Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2302-08671,
title = {Search to Capture Long-range Dependency with Stacking GNNs for Graph Classification},
author = {Lanning Wei and Zhiqiang He and Huan Zhao and Quanming Yao},
url = {https://doi.org/10.48550/arXiv.2302.08671},
doi = {10.48550/arXiv.2302.08671},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2302.08671},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2858.
Han, Fred X.; Mills, Keith G.; Chudak, Fabian; Riahi, Parsa; Salameh, Mohammad; Zhang, Jialin; Lu, Wei; Jui, Shangling; Niu, Di
A General-Purpose Transferable Predictor for Neural Architecture Search Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2302-10835,
title = {A General-Purpose Transferable Predictor for Neural Architecture Search},
author = {Fred X. Han and Keith G. Mills and Fabian Chudak and Parsa Riahi and Mohammad Salameh and Jialin Zhang and Wei Lu and Shangling Jui and Di Niu},
url = {https://doi.org/10.48550/arXiv.2302.10835},
doi = {10.48550/arXiv.2302.10835},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2302.10835},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2857.
Lyu, Zimeng; Ororbia, Alexander; Desell, Travis
Online Evolutionary Neural Architecture Search for Multivariate Non-Stationary Time Series Forecasting Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2302-10347,
title = {Online Evolutionary Neural Architecture Search for Multivariate Non-Stationary Time Series Forecasting},
author = {Zimeng Lyu and Alexander Ororbia and Travis Desell},
url = {https://doi.org/10.48550/arXiv.2302.10347},
doi = {10.48550/arXiv.2302.10347},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2302.10347},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2856.
Kuş, Zeki; Aydin, Musa; Kiraz, Berna; Can, Burhanettin
Neural Architecture Search Using Metaheuristics for Automated Cell Segmentation Proceedings Article
In: Gaspero, Luca Di; Festa, Paola; Nakib, Amir; Pavone, Mario (Ed.): Metaheuristics, pp. 158–171, Springer International Publishing, Cham, 2023, ISBN: 978-3-031-26504-4.
@inproceedings{10.1007/978-3-031-26504-4_12,
title = {Neural Architecture Search Using Metaheuristics for Automated Cell Segmentation},
author = {Zeki Kuş and Musa Aydin and Berna Kiraz and Burhanettin Can},
editor = {Luca Di Gaspero and Paola Festa and Amir Nakib and Mario Pavone},
url = {https://link.springer.com/chapter/10.1007/978-3-031-26504-4_12},
isbn = {978-3-031-26504-4},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Metaheuristics},
pages = {158--171},
publisher = {Springer International Publishing},
address = {Cham},
abstract = {Deep neural networks give successful results for segmentation of medical images. The need for optimizing many hyper-parameters presents itself as a significant limitation hampering the effectiveness of deep neural network based segmentation task. Manual selection of these hyper-parameters is not feasible as the search space increases. At the same time, these generated networks are problem-specific. Recently, studies that perform segmentation of medical images using Neural Architecture Search (NAS) have been proposed. However, these studies significantly limit the possible network structures and search space. In this study, we proposed a structure called UNAS-Net that brings together the advantages of successful NAS studies and is more flexible in terms of the networks that can be created. The UNAS-Net structure has been optimized using metaheuristics including Differential Evolution (DE) and Local Search (LS), and the generated networks have been tested on Optofil and Cell Nuclei data sets. When the results are examined, it is seen that the networks produced by the heuristic methods improve the performance of the U-Net structure in terms of both segmentation performance and computational complexity. As a result, the proposed structure can be used when the automatic generation of neural networks that provide fast inference as well as successful segmentation performance is desired.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Deep neural networks give successful results for segmentation of medical images. The need for optimizing many hyper-parameters presents itself as a significant limitation hampering the effectiveness of deep neural network based segmentation task. Manual selection of these hyper-parameters is not feasible as the search space increases. At the same time, these generated networks are problem-specific. Recently, studies that perform segmentation of medical images using Neural Architecture Search (NAS) have been proposed. However, these studies significantly limit the possible network structures and search space. In this study, we proposed a structure called UNAS-Net that brings together the advantages of successful NAS studies and is more flexible in terms of the networks that can be created. The UNAS-Net structure has been optimized using metaheuristics including Differential Evolution (DE) and Local Search (LS), and the generated networks have been tested on Optofil and Cell Nuclei data sets. When the results are examined, it is seen that the networks produced by the heuristic methods improve the performance of the U-Net structure in terms of both segmentation performance and computational complexity. As a result, the proposed structure can be used when the automatic generation of neural networks that provide fast inference as well as successful segmentation performance is desired.
2855.
Gülcü, Ayla; Kuş, Zeki
Neural Architecture Search Using Differential Evolution in MAML Framework for Few-Shot Classification Problems Proceedings Article
In: Gaspero, Luca Di; Festa, Paola; Nakib, Amir; Pavone, Mario (Ed.): Metaheuristics, pp. 143–157, Springer International Publishing, Cham, 2023, ISBN: 978-3-031-26504-4.
@inproceedings{10.1007/978-3-031-26504-4_11,
title = {Neural Architecture Search Using Differential Evolution in MAML Framework for Few-Shot Classification Problems},
author = {Ayla Gülcü and Zeki Kuş},
editor = {Luca Di Gaspero and Paola Festa and Amir Nakib and Mario Pavone},
isbn = {978-3-031-26504-4},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Metaheuristics},
pages = {143--157},
publisher = {Springer International Publishing},
address = {Cham},
abstract = {Model-Agnostic Meta-Learning (MAML) algorithm is an optimization based meta-learning algorithm which aims to find a good initial state of the neural network that can then be adapted to any novel task using a few optimization steps. In this study, we take MAML with a simple four-block convolution architecture as our baseline, and try to improve its few-shot classification performance by using an architecture generated automatically through the neural architecture search process. We use differential evolution algorithm as the search strategy for searching over cells within a predefined search space. We have performed our experiments using two well-known few-shot classification datasets, miniImageNet and FC100 dataset. For each of those datasets, the performance of the original MAML is compared to the performance of our MAML-NAS model under both 1-shot 5-way and 5-shot 5-way settings. The results reveal that MAML-NAS results in better or at least comparable accuracy values for both of the datasets in all settings. More importantly, this performance is achieved by much simpler architectures, that is architectures requiring less floating-point operations.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Model-Agnostic Meta-Learning (MAML) algorithm is an optimization based meta-learning algorithm which aims to find a good initial state of the neural network that can then be adapted to any novel task using a few optimization steps. In this study, we take MAML with a simple four-block convolution architecture as our baseline, and try to improve its few-shot classification performance by using an architecture generated automatically through the neural architecture search process. We use differential evolution algorithm as the search strategy for searching over cells within a predefined search space. We have performed our experiments using two well-known few-shot classification datasets, miniImageNet and FC100 dataset. For each of those datasets, the performance of the original MAML is compared to the performance of our MAML-NAS model under both 1-shot 5-way and 5-shot 5-way settings. The results reveal that MAML-NAS results in better or at least comparable accuracy values for both of the datasets in all settings. More importantly, this performance is achieved by much simpler architectures, that is architectures requiring less floating-point operations.
2854.
Muneer, V; Biju, G M; Bhattacharya, Avik
Optimal Machine Learning based Controller for Shunt Active Power Filter by Auto Machine Learning Journal Article
In: IEEE Journal of Emerging and Selected Topics in Power Electronics, pp. 1-1, 2023.
@article{10049454,
title = {Optimal Machine Learning based Controller for Shunt Active Power Filter by Auto Machine Learning},
author = {V Muneer and G M Biju and Avik Bhattacharya},
url = {https://ieeexplore.ieee.org/abstract/document/10049454},
doi = {10.1109/JESTPE.2023.3244605},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Journal of Emerging and Selected Topics in Power Electronics},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2853.
Zhan, Lin; Fan, Jiayuan; Ye, Peng; Cao, Jianjian
A2S-NAS: Asymmetric Spectral-Spatial Neural Architecture Search For Hyperspectral Image Classification Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2302-11868,
title = {A2S-NAS: Asymmetric Spectral-Spatial Neural Architecture Search For Hyperspectral Image Classification},
author = {Lin Zhan and Jiayuan Fan and Peng Ye and Jianjian Cao},
url = {https://doi.org/10.48550/arXiv.2302.11868},
doi = {10.48550/arXiv.2302.11868},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2302.11868},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2852.
Zhan, Lin; Fan, Jiayuan; Ye, Peng; Cao, Jianjian
A2S-NAS: Asymmetric Spectral-Spatial Neural Architecture Search For Hyperspectral Image Classification Journal Article
In: CoRR, vol. abs/2302.11868, 2023.
@article{DBLP:journals/corr/abs-2302-11868b,
title = {A2S-NAS: Asymmetric Spectral-Spatial Neural Architecture Search For Hyperspectral Image Classification},
author = {Lin Zhan and Jiayuan Fan and Peng Ye and Jianjian Cao},
url = {https://ieeexplore.ieee.org/abstract/document/10052655},
doi = {10.48550/arXiv.2302.11868},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2302.11868},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2851.
Lu, Xiaotong; Dong, Weisheng; Li, Xin; Wu, Jinjian; Li, Leida; Shi, Guangming
Adaptive Search-and-Training for Robust and Efficient Network Pruning Journal Article
In: IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1-14, 2023.
@article{10052756,
title = {Adaptive Search-and-Training for Robust and Efficient Network Pruning},
author = {Xiaotong Lu and Weisheng Dong and Xin Li and Jinjian Wu and Leida Li and Guangming Shi},
url = {https://ieeexplore.ieee.org/abstract/document/10052756},
doi = {10.1109/TPAMI.2023.3248612},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
pages = {1-14},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2850.
Bataineh, Ali Al; Kaur, Devinder; Al-khassaweneh, Mahmood; Al-sharoa, Esraa
Automated CNN Architectural Design: A Simple and Efficient Methodology for Computer Vision Tasks Journal Article
In: Mathematics, vol. 11, no. 5, pp. 1-17, 2023.
@article{RePEc:gam:jmathe:v:11:y:2023:i:5:p:1141-:d:1079919,
title = {Automated CNN Architectural Design: A Simple and Efficient Methodology for Computer Vision Tasks},
author = {Ali Al Bataineh and Devinder Kaur and Mahmood Al-khassaweneh and Esraa Al-sharoa},
url = {https://www.mdpi.com/2227-7390/11/5/1141},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Mathematics},
volume = {11},
number = {5},
pages = {1-17},
abstract = {Convolutional neural networks (CNN) have transformed the field of computer vision by enabling the automatic extraction of features, obviating the need for manual feature engineering. Despite their success, identifying an optimal architecture for a particular task can be a time-consuming and challenging process due to the vast space of possible network designs. To address this, we propose a novel neural architecture search (NAS) framework that utilizes the clonal selection algorithm (CSA) to automatically design high-quality CNN architectures for image classification problems. Our approach uses an integer vector representation to encode CNN architectures and hyperparameters, combined with a truncated Gaussian mutation scheme that enables efficient exploration of the search space. We evaluated the proposed method on six challenging EMNIST benchmark datasets for handwritten digit recognition, and our results demonstrate that it outperforms nearly all existing approaches. In addition, our approach produces state-of-the-art performance while having fewer trainable parameters than other methods, making it low-cost, simple, and reusable for application to multiple datasets.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Convolutional neural networks (CNN) have transformed the field of computer vision by enabling the automatic extraction of features, obviating the need for manual feature engineering. Despite their success, identifying an optimal architecture for a particular task can be a time-consuming and challenging process due to the vast space of possible network designs. To address this, we propose a novel neural architecture search (NAS) framework that utilizes the clonal selection algorithm (CSA) to automatically design high-quality CNN architectures for image classification problems. Our approach uses an integer vector representation to encode CNN architectures and hyperparameters, combined with a truncated Gaussian mutation scheme that enables efficient exploration of the search space. We evaluated the proposed method on six challenging EMNIST benchmark datasets for handwritten digit recognition, and our results demonstrate that it outperforms nearly all existing approaches. In addition, our approach produces state-of-the-art performance while having fewer trainable parameters than other methods, making it low-cost, simple, and reusable for application to multiple datasets.
2849.
Zheng, Shenghe; Wang, Hongzhi; Mu, Tianyu
DCLP: Neural Architecture Predictor with Curriculum Contrastive Learning Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2302-13020,
title = {DCLP: Neural Architecture Predictor with Curriculum Contrastive Learning},
author = {Shenghe Zheng and Hongzhi Wang and Tianyu Mu},
url = {https://doi.org/10.48550/arXiv.2302.13020},
doi = {10.48550/arXiv.2302.13020},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2302.13020},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2848.
Zheng, Xin; Zhang, Miao; Chen, Chunyang; Zhang, Qin; Zhou, Chuan; Pan, Shirui
Auto-HeG: Automated Graph Neural Network on Heterophilic Graphs Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2302-12357,
title = {Auto-HeG: Automated Graph Neural Network on Heterophilic Graphs},
author = {Xin Zheng and Miao Zhang and Chunyang Chen and Qin Zhang and Chuan Zhou and Shirui Pan},
url = {https://doi.org/10.48550/arXiv.2302.12357},
doi = {10.48550/arXiv.2302.12357},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2302.12357},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2847.
Chen, Angelica; Dohan, David M.; So, David R.
EvoPrompting: Language Models for Code-Level Neural Architecture Search Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2302-14838,
title = {EvoPrompting: Language Models for Code-Level Neural Architecture Search},
author = {Angelica Chen and David M. Dohan and David R. So},
url = {https://doi.org/10.48550/arXiv.2302.14838},
doi = {10.48550/arXiv.2302.14838},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2302.14838},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2846.
He, Yang; Xiao, Lingao
Structured Pruning for Deep Convolutional Neural Networks: A survey Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2303-00566,
title = {Structured Pruning for Deep Convolutional Neural Networks: A survey},
author = {Yang He and Lingao Xiao},
url = {https://doi.org/10.48550/arXiv.2303.00566},
doi = {10.48550/arXiv.2303.00566},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2303.00566},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2845.
Rajesh, Chilukamari; Kumar, Sushil
Äutomatic Retinal Vessel Segmentation Using BTLBO Proceedings Article
In: Thakur, Manoj; Agnihotri, Samar; Rajpurohit, Bharat Singh; Pant, Millie; Deep, Kusum; Nagar, Atulya K. (Ed.): Soft Computing for Problem Solving, pp. 189–200, Springer Nature Singapore, Singapore, 2023, ISBN: 978-981-19-6525-8.
@inproceedings{10.1007/978-981-19-6525-8_15,
title = {Äutomatic Retinal Vessel Segmentation Using BTLBO},
author = {Chilukamari Rajesh and Sushil Kumar},
editor = {Manoj Thakur and Samar Agnihotri and Bharat Singh Rajpurohit and Millie Pant and Kusum Deep and Atulya K. Nagar},
url = {https://link.springer.com/chapter/10.1007/978-981-19-6525-8_15},
isbn = {978-981-19-6525-8},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Soft Computing for Problem Solving},
pages = {189--200},
publisher = {Springer Nature Singapore},
address = {Singapore},
abstract = {The accuracy of retinal vessel segmentation (RVS) is crucial in assisting physicians in the ophthalmology diagnosis or other systemic diseases. However, manual segmentation needs a high level of knowledge, time-consuming, complex, and prone to errors. As a result, automatic vessel segmentation is required, which might be a significant technological breakthrough in the medical field. We proposed a novel strategy in this paper, that uses neural architecture search (NAS) to optimize a U-net architecture using a binary teaching learning-based optimization (BTLBO) evolutionary algorithm for RVS to increase vessel segmentation performance and reduce the workload of manually developing deep networks with limited computing resources. We used a publicly available DRIVE dataset to examine the proposed approach and showed that the discovered model generated by the proposed approach outperforms existing methods.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
The accuracy of retinal vessel segmentation (RVS) is crucial in assisting physicians in the ophthalmology diagnosis or other systemic diseases. However, manual segmentation needs a high level of knowledge, time-consuming, complex, and prone to errors. As a result, automatic vessel segmentation is required, which might be a significant technological breakthrough in the medical field. We proposed a novel strategy in this paper, that uses neural architecture search (NAS) to optimize a U-net architecture using a binary teaching learning-based optimization (BTLBO) evolutionary algorithm for RVS to increase vessel segmentation performance and reduce the workload of manually developing deep networks with limited computing resources. We used a publicly available DRIVE dataset to examine the proposed approach and showed that the discovered model generated by the proposed approach outperforms existing methods.
2844.
Wang, Chunnan; Chen, Bozhou; Li, Geng; Wang, Hongzhi
Automated Graph Neural Network Search Under Federated Learning Framework Journal Article
In: IEEE Transactions on Knowledge and Data Engineering, pp. 1-13, 2023.
@article{10056291,
title = {Automated Graph Neural Network Search Under Federated Learning Framework},
author = {Chunnan Wang and Bozhou Chen and Geng Li and Hongzhi Wang},
url = {https://ieeexplore.ieee.org/abstract/document/10056291},
doi = {10.1109/TKDE.2023.3250264},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Knowledge and Data Engineering},
pages = {1-13},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2843.
Eisenbach, Markus; Lübberstedt, Jannik; Aganian, Dustin; Gross, Horst-Michael
A Little Bit Attention Is All You Need for Person Re-Identification Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2302-14574,
title = {A Little Bit Attention Is All You Need for Person Re-Identification},
author = {Markus Eisenbach and Jannik Lübberstedt and Dustin Aganian and Horst-Michael Gross},
url = {https://doi.org/10.48550/arXiv.2302.14574},
doi = {10.48550/arXiv.2302.14574},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2302.14574},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2842.
Hasana, Md. Mehedi; Ibrahim, Muhammad; Ali, Md. Sawkat
Speeding Up EfficientNet: Selecting Update Blocks of Convolutional Neural Networks using Genetic Algorithm in Transfer Learning Technical Report
2023.
@techreport{DBLP:journals/corr/abs-2303-00261,
title = {Speeding Up EfficientNet: Selecting Update Blocks of Convolutional Neural Networks using Genetic Algorithm in Transfer Learning},
author = {Md. Mehedi Hasana and Muhammad Ibrahim and Md. Sawkat Ali},
url = {https://doi.org/10.48550/arXiv.2303.00261},
doi = {10.48550/arXiv.2303.00261},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2303.00261},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
2841.
Lin, Shih-Ping; Wang, Sheng-De
SGAS-es: Avoiding Performance Collapse by Sequential Greedy Architecture Search with the Early Stopping Indicator Proceedings Article
In: Ärai, Kohei" (Ed.): Ädvances in Information and Communication", pp. 135–154, Springer Nature Switzerland, Cham, 2023, ISBN: 978-3-031-28073-3.
@inproceedings{10.1007/978-3-031-28073-3_10,
title = {SGAS-es: Avoiding Performance Collapse by Sequential Greedy Architecture Search with the Early Stopping Indicator},
author = {Shih-Ping Lin and Sheng-De Wang},
editor = {Kohei" Ärai},
url = {https://link.springer.com/chapter/10.1007/978-3-031-28073-3_10},
isbn = {978-3-031-28073-3},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Ädvances in Information and Communication"},
pages = {135--154},
publisher = {Springer Nature Switzerland},
address = {Cham},
abstract = {Sequential Greedy Architecture Search (SGAS) reduces the discretization loss of Differentiable Architecture Search (DARTS). However, we observed that SGAS may lead to unstable searched results as DARTS. We referred to this problem as the cascade performance collapse issue. Therefore, we proposed Sequential Greedy Architecture Search with the Early Stopping Indicator (SGAS-es). We adopted the early stopping mechanism in each phase of SGAS to stabilize searched results and further improve the searching ability. The early stopping mechanism is based on the relation among Flat Minima, the largest eigenvalue of the Hessian matrix of the loss function, and performance collapse. We devised a mathematical derivation to show the relation between Flat Minima and the largest eigenvalue. The moving averaged largest eigenvalue is used as an early stopping indicator. Finally, we used NAS-Bench-201 and Fashion-MNIST to confirm the performance and stability of SGAS-es. Moreover, we used EMNIST-Balanced to verify the transferability of searched results. These experiments show that SGAS-es is a robust method and can derive the architecture with good performance and transferability.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Sequential Greedy Architecture Search (SGAS) reduces the discretization loss of Differentiable Architecture Search (DARTS). However, we observed that SGAS may lead to unstable searched results as DARTS. We referred to this problem as the cascade performance collapse issue. Therefore, we proposed Sequential Greedy Architecture Search with the Early Stopping Indicator (SGAS-es). We adopted the early stopping mechanism in each phase of SGAS to stabilize searched results and further improve the searching ability. The early stopping mechanism is based on the relation among Flat Minima, the largest eigenvalue of the Hessian matrix of the loss function, and performance collapse. We devised a mathematical derivation to show the relation between Flat Minima and the largest eigenvalue. The moving averaged largest eigenvalue is used as an early stopping indicator. Finally, we used NAS-Bench-201 and Fashion-MNIST to confirm the performance and stability of SGAS-es. Moreover, we used EMNIST-Balanced to verify the transferability of searched results. These experiments show that SGAS-es is a robust method and can derive the architecture with good performance and transferability.
2840.
Lu, Shun; Hu, Yu; Yang, Longxing; Sun, Zihao; Mei, Jilin; Tan, Jianchao; Song, Chengru
PA&DA: Jointly Sampling PAth and DAta for Consistent NAS Conference
CVPR2023, vol. abs/2302.14772, 2023.
@conference{DBLP:journals/corr/abs-2302-14772,
title = {PA&DA: Jointly Sampling PAth and DAta for Consistent NAS},
author = {Shun Lu and Yu Hu and Longxing Yang and Zihao Sun and Jilin Mei and Jianchao Tan and Chengru Song},
url = {https://doi.org/10.48550/arXiv.2302.14772},
doi = {10.48550/arXiv.2302.14772},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {CVPR2023},
journal = {CoRR},
volume = {abs/2302.14772},
keywords = {},
pubstate = {published},
tppubtype = {conference}
}
2839.
Njor, Emil; Madsen, Jan; Fafoutis, Xenofon
Data Aware Neural Architecture Search Proceedings Article
In: Proceedings of tinyML Research Symposium, 2023, (tinyML Research Symposium’23 ; Conference date: 27-03-2023 Through 27-03-2023).
@inproceedings{40cb879dbb9a4fd9bd92ad27b617056f,
title = {Data Aware Neural Architecture Search},
author = {Emil Njor and Jan Madsen and Xenofon Fafoutis},
url = {https://orbit.dtu.dk/en/publications/data-aware-neural-architecture-search},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {Proceedings of tinyML Research Symposium},
abstract = {Neural Architecture Search (NAS) is a popular tool for automatically generating Neural Network (NN) architectures. In early NAS works, these tools typically optimized NN architectures for a single metric, such as accuracy. However, in the case of resource constrained Machine Learning, one single metric is not enough to evaluate a NN architecture. For example, a NN model achieving a high accuracy is not useful if it does not fit inside the flash memory of a given system. Therefore, recent works on NAS for resource constrained systems have investigated various approaches to optimize for multiple metrics. In this paper, we propose that, on top of these approaches, it could be beneficial for NAS optimization of resource constrained systems to also consider input data granularity. We name such a system “Data Aware NAS”, and we provide experimental evidence of its benefits by comparing it to traditional NAS.},
note = {tinyML Research Symposium’23 ; Conference date: 27-03-2023 Through 27-03-2023},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Neural Architecture Search (NAS) is a popular tool for automatically generating Neural Network (NN) architectures. In early NAS works, these tools typically optimized NN architectures for a single metric, such as accuracy. However, in the case of resource constrained Machine Learning, one single metric is not enough to evaluate a NN architecture. For example, a NN model achieving a high accuracy is not useful if it does not fit inside the flash memory of a given system. Therefore, recent works on NAS for resource constrained systems have investigated various approaches to optimize for multiple metrics. In this paper, we propose that, on top of these approaches, it could be beneficial for NAS optimization of resource constrained systems to also consider input data granularity. We name such a system “Data Aware NAS”, and we provide experimental evidence of its benefits by comparing it to traditional NAS.
2838.
Wang, Kun; Han, Ling; Li, Liangzhi
A decoupled search deep network framework for high-resolution remote sensing image classification Journal Article
In: Remote Sensing Letters, vol. 14, no. 3, pp. 243-253, 2023.
@article{doi:10.1080/2150704X.2023.2185110,
title = {A decoupled search deep network framework for high-resolution remote sensing image classification},
author = {Kun Wang and Ling Han and Liangzhi Li},
url = {https://doi.org/10.1080/2150704X.2023.2185110},
doi = {10.1080/2150704X.2023.2185110},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {Remote Sensing Letters},
volume = {14},
number = {3},
pages = {243-253},
publisher = {Taylor & Francis},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
2837.
Xue, Yu; Chen, Chen; Słowik, Adam
Neural Architecture Search Based on A Multi-objective Evolutionary Algorithm With Probability Stack Journal Article
In: IEEE Transactions on Evolutionary Computation, pp. 1-1, 2023.
@article{10059145,
title = {Neural Architecture Search Based on A Multi-objective Evolutionary Algorithm With Probability Stack},
author = {Yu Xue and Chen Chen and Adam Słowik},
doi = {10.1109/TEVC.2023.3252612},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {IEEE Transactions on Evolutionary Computation},
pages = {1-1},
keywords = {},
pubstate = {published},
tppubtype = {article}
}