Maintained by Difan Deng and Marius Lindauer.
The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know.
Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization. We would therefore like to share some best practices for empirical evaluations of NAS methods, which we believe will facilitate sustained and measurable progress in the field. If you are interested in a teaser, please read our blog post or directly jump to our checklist.
Transformers have gained increasing popularity in different domains. For a comprehensive list of papers focusing on Neural Architecture Search for Transformer-Based spaces, the awesome-transformer-search repo is all you need.
2023
3081.
Park, Soohyun; Son, Seok Bin; Lee, Youn Kyu; Jung, Soyi; Kim, Joongheon
Two-stage architectural fine-tuning for neural architecture search in efficient transfer learning Journal Article
In: ELECTRONICS LETTERS , vol. 59, no. 24, 2023.
@article{Park-el23a,
title = {Two-stage architectural fine-tuning for neural architecture search in efficient transfer learning},
author = {Soohyun Park and Seok Bin Son and Youn Kyu Lee and Soyi Jung and Joongheon Kim},
url = {https://ietresearch.onlinelibrary.wiley.com/doi/pdfdirect/10.1049/ell2.13066},
year = {2023},
date = {2023-12-01},
urldate = {2023-12-01},
journal = {ELECTRONICS LETTERS },
volume = {59},
number = {24},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3080.
Saluky, Saluky; Nugraha, Gusti Baskara; Supangkat, Suhono Harso
Enhancing Abandoned Object Detection with Dual Background Models and Yolo-NAS Journal Article
In: International Journal of Intelligent Systems and Applications in Engineering, vol. 12, no. 2, pp. 547–554, 2023.
@article{Saluky_Nugraha_Supangkat_2023,
title = {Enhancing Abandoned Object Detection with Dual Background Models and Yolo-NAS},
author = {Saluky Saluky and Gusti Baskara Nugraha and Suhono Harso Supangkat},
url = {https://ijisae.org/index.php/IJISAE/article/view/4298},
year = {2023},
date = {2023-12-01},
urldate = {2023-12-01},
journal = {International Journal of Intelligent Systems and Applications in Engineering},
volume = {12},
number = {2},
pages = {547–554},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3079.
Bellodi, E.; Bertozzi, D.; Bizzarri, A.; Favalli, M.; Fraccaroli, M.; Zese, R.
Efficient Resource-Aware Neural Architecture Search with a Neuro-Symbolic Approach Proceedings Article
In: 2023 IEEE 16th International Symposium on Embedded Multicore/Many-core Systems-on-Chip (MCSoC), pp. 171-178, IEEE Computer Society, Los Alamitos, CA, USA, 2023.
@inproceedings{10387830,
title = {Efficient Resource-Aware Neural Architecture Search with a Neuro-Symbolic Approach},
author = {E. Bellodi and D. Bertozzi and A. Bizzarri and M. Favalli and M. Fraccaroli and R. Zese},
url = {https://doi.ieeecomputersociety.org/10.1109/MCSoC60832.2023.00034},
doi = {10.1109/MCSoC60832.2023.00034},
year = {2023},
date = {2023-12-01},
urldate = {2023-12-01},
booktitle = {2023 IEEE 16th International Symposium on Embedded Multicore/Many-core Systems-on-Chip (MCSoC)},
pages = {171-178},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
abstract = {Hardware-aware Neural Architectural Search (NAS) is gaining momentum to enable the deployment of deep learning on edge devices with limited computing capabilities. Incorporating device-related objectives such as affordable floating point operations, latency, power, memory usage, etc. into the optimization process makes searching for the most efficient neural architecture more complicated, since both model accuracy and hardware cost should guide the search. The main concern with most state-of-the-art hardware-aware NAS strategies is that they propose for evaluation also trivially infeasible network models for the capabilities of the hardware platform at hand. Moreover, previously generated models are frequently not exploited to intelligently generate new ones, leading to prohibitive computational costs for practical relevance. This paper aims to boost the computational efficiency of hardware-aware NAS by means of a neuro-symbolic framework revolving around a Probabilistic Inductive Logic Programming module to define and exploit a set of symbolic rules. This component learns and refines the probabilities associated with the rules, allowing the framework to adapt and improve over time, thus quickly narrowing down the search space toward the most promising neural architectures.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Hardware-aware Neural Architectural Search (NAS) is gaining momentum to enable the deployment of deep learning on edge devices with limited computing capabilities. Incorporating device-related objectives such as affordable floating point operations, latency, power, memory usage, etc. into the optimization process makes searching for the most efficient neural architecture more complicated, since both model accuracy and hardware cost should guide the search. The main concern with most state-of-the-art hardware-aware NAS strategies is that they propose for evaluation also trivially infeasible network models for the capabilities of the hardware platform at hand. Moreover, previously generated models are frequently not exploited to intelligently generate new ones, leading to prohibitive computational costs for practical relevance. This paper aims to boost the computational efficiency of hardware-aware NAS by means of a neuro-symbolic framework revolving around a Probabilistic Inductive Logic Programming module to define and exploit a set of symbolic rules. This component learns and refines the probabilities associated with the rules, allowing the framework to adapt and improve over time, thus quickly narrowing down the search space toward the most promising neural architectures.
3078.
Kapoor, A.; Soans, R.; Dixit, S.; Ns, P.; Singh, B.; Das, M.
NASEREX: Optimizing Early Exits via AutoML for Scalable Efficient Inference in Big Image Streams Proceedings Article
In: 2023 IEEE International Conference on Big Data (BigData), pp. 5266-5271, IEEE Computer Society, Los Alamitos, CA, USA, 2023.
@inproceedings{10386502,
title = {NASEREX: Optimizing Early Exits via AutoML for Scalable Efficient Inference in Big Image Streams},
author = {A. Kapoor and R. Soans and S. Dixit and P. Ns and B. Singh and M. Das},
url = {https://doi.ieeecomputersociety.org/10.1109/BigData59044.2023.10386502},
doi = {10.1109/BigData59044.2023.10386502},
year = {2023},
date = {2023-12-01},
urldate = {2023-12-01},
booktitle = {2023 IEEE International Conference on Big Data (BigData)},
pages = {5266-5271},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
abstract = {We investigate the problem of smart operational efficiency, at scale, in Machine Learning models for Big Data streams, in context of embedded AI applications, by learning optimal early exits. Embedded AI applications that employ deep neural models depend on efficient model inference at scale, especially on resource-constrained hardware. Recent vision/text/audio models are computationally complex with huge parameter spaces and input samples typically pass through multiple layers, each with large tensor computations, to produce valid outputs. Generally, in most real scenarios, AI applications deal with big data streams, such as streams of audio signals, static images and/or high resolution video frames. Deep ML models powering such applications have to continuously perform inference on such big data streams for varied tasks such as noise suppression, face detection, gait estimation and so on. Ensuring efficiency is challenging, even with model compression techniques since they reduce model size but often fail to achieve scalable inference efficiency over continuous streams. Early exits enable adaptive inference by extracting valid outputs from any pre-final layer of a deep model which significantly boosts efficiency at scale since many of the input instances need not be processed at all the layers of a deep model, especially for big streams. Suitable early exit structure design (number + positions) is a difficult but crucial aspect in improving efficiency without any loss in predictive performance, especially in context of big streams. Naive manual early exit design that does not consider the hardware capacity or data stream characteristics is counterproductive. We propose NASEREX framework that leverages Neural architecture Search (NAS) with a novel saliency-constrained search space and exit decision metric to learn suitable early exit structure to augment Deep Neural models for scalable efficient inference on big image streams. Optimized exit-augmented models perform $approx 2.5 times$ faster having $approx 4 times$ aggregated lower effective FLOPs, with no significant accuracy loss.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We investigate the problem of smart operational efficiency, at scale, in Machine Learning models for Big Data streams, in context of embedded AI applications, by learning optimal early exits. Embedded AI applications that employ deep neural models depend on efficient model inference at scale, especially on resource-constrained hardware. Recent vision/text/audio models are computationally complex with huge parameter spaces and input samples typically pass through multiple layers, each with large tensor computations, to produce valid outputs. Generally, in most real scenarios, AI applications deal with big data streams, such as streams of audio signals, static images and/or high resolution video frames. Deep ML models powering such applications have to continuously perform inference on such big data streams for varied tasks such as noise suppression, face detection, gait estimation and so on. Ensuring efficiency is challenging, even with model compression techniques since they reduce model size but often fail to achieve scalable inference efficiency over continuous streams. Early exits enable adaptive inference by extracting valid outputs from any pre-final layer of a deep model which significantly boosts efficiency at scale since many of the input instances need not be processed at all the layers of a deep model, especially for big streams. Suitable early exit structure design (number + positions) is a difficult but crucial aspect in improving efficiency without any loss in predictive performance, especially in context of big streams. Naive manual early exit design that does not consider the hardware capacity or data stream characteristics is counterproductive. We propose NASEREX framework that leverages Neural architecture Search (NAS) with a novel saliency-constrained search space and exit decision metric to learn suitable early exit structure to augment Deep Neural models for scalable efficient inference on big image streams. Optimized exit-augmented models perform $approx 2.5 times$ faster having $approx 4 times$ aggregated lower effective FLOPs, with no significant accuracy loss.
3077.
Heuillet, Alexandre
Exploring deep neural network differentiable architecture design PhD Thesis
Université Paris-Saclay, 2023.
@phdthesis{heuillet:tel-04420933,
title = {Exploring deep neural network differentiable architecture design},
author = {Alexandre Heuillet},
url = {https://hal.science/tel-04420933},
year = {2023},
date = {2023-12-01},
urldate = {2023-12-01},
number = {2023UPASG069},
school = {Université Paris-Saclay},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
3076.
Chen, Hui; Li, Nannan; Chen, Rong
Ni-DehazeNet: representation learning via bilevel optimized architecture search for nighttime dehazing Journal Article
In: The Visual Computer, 2023.
@article{Chen-vc23a,
title = {Ni-DehazeNet: representation learning via bilevel optimized architecture search for nighttime dehazing},
author = {
Hui Chen and Nannan Li and Rong Chen
},
url = {https://link.springer.com/article/10.1007/s00371-023-03159-4},
year = {2023},
date = {2023-11-28},
journal = {The Visual Computer},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3075.
Lyu, Zonglei; Yu, Tong; Pan, Fuxi; Zhang, Yilin; Luo, Jia; Zhang, Dan; Chen, Yiren; Zhang, Bo; Li, Guangyao
A survey of model compression strategies for object detection Journal Article
In: Multimedia Tools and Applications , 2023.
@article{nokey,
title = {A survey of model compression strategies for object detection},
author = {
Zonglei Lyu and Tong Yu and Fuxi Pan and Yilin Zhang and Jia Luo and Dan Zhang and Yiren Chen and Bo Zhang and Guangyao Li
},
url = {https://link.springer.com/article/10.1007/s11042-023-17192-x},
year = {2023},
date = {2023-11-02},
urldate = {2023-11-02},
journal = {Multimedia Tools and Applications },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3074.
POYSER, MATTHEW
2023.
@phdthesis{POYSER-phd2023,
title = {Minimizing Computational Resources for Deep Machine Learning: A Compression and Neural Architecture Search Perspective for Image Classification and Object Detection},
author = {POYSER, MATTHEW},
url = {http://etheses.dur.ac.uk/15207/1/main.pdf},
year = {2023},
date = {2023-11-01},
urldate = {2023-11-01},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
3073.
Liu, Shiya
Energy-efficient Neuromorphic Computing for Resource-constrained Internet of Things Devices PhD Thesis
2023.
@phdthesis{Liu-phd23a,
title = {Energy-efficient Neuromorphic Computing for Resource-constrained Internet of Things Devices},
author = {Liu, Shiya},
url = {https://vtechworks.lib.vt.edu/handle/10919/116629},
year = {2023},
date = {2023-11-01},
urldate = {2023-11-01},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
3072.
Xie, Tao; Zhang, Haoming; Yang, Linqi; Wang, Ke; Dai, Kun; Li, Ruifeng; Zhao, Lijun
Point-NAS: A Novel Neural Architecture Search Framework for Point Cloud Analysis Journal Article
In: IEEE transactions on image processing : a publication of the IEEE Signal Processing Society, vol. PP, 2023, ISSN: 1057-7149.
@article{PMID:37963007,
title = {Point-NAS: A Novel Neural Architecture Search Framework for Point Cloud Analysis},
author = {Tao Xie and Haoming Zhang and Linqi Yang and Ke Wang and Kun Dai and Ruifeng Li and Lijun Zhao},
url = {https://doi.org/10.1109/TIP.2023.3331223},
doi = {10.1109/tip.2023.3331223},
issn = {1057-7149},
year = {2023},
date = {2023-11-01},
urldate = {2023-11-01},
journal = {IEEE transactions on image processing : a publication of the IEEE Signal Processing Society},
volume = {PP},
abstract = {Recently, point-based networks have exhibited extraordinary potential for 3D point cloud processing. However, owing to the meticulous design of both parameters and hyperparameters inside the network, constructing a promising network for each point cloud task can be an expensive endeavor. In this work, we develop a novel one-shot search framework called Point-NAS to automatically determine optimum architectures for various point cloud tasks. Specifically, we design an elastic feature extraction (EFE) module that serves as a basic unit for architecture search, which expands seamlessly alongside both the width and depth of the network for efficient feature extraction. Based on the EFE module, we devise a searching space, which is encoded into a supernet to provide a wide number of latent network structures for a particular point cloud task. To fully optimize the weights of the supernet, we propose a weight coupling sandwich rule that samples the largest, smallest, and multiple medium models at each iteration and fuses their gradients to update the supernet. Furthermore, we present a united gradient adjustment algorithm that mitigates gradient conflict induced by distinct gradient directions of sampled models and supernet, thus expediting the convergence of the supernet and assuring that it can be comprehensively trained. Pursuant to the provided techniques, the trained supernet enables a multitude of subnets to be incredibly well-optimized. Finally, we conduct an evolutionary search for the supernet under resource constraints to find promising architectures for different tasks. Experimentally, the searched Point-NAS with weights inherited from the supernet realizes outstanding results across a variety of benchmarks. i.e., 94.2% and 88.9% overall accuracy under ModelNet40 and ScanObjectNN, 68.6% mIoU under S3DIS, 63.6% and 69.3% mAP@0.25 under SUN RGB-D and ScanNet V2 datasets.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Recently, point-based networks have exhibited extraordinary potential for 3D point cloud processing. However, owing to the meticulous design of both parameters and hyperparameters inside the network, constructing a promising network for each point cloud task can be an expensive endeavor. In this work, we develop a novel one-shot search framework called Point-NAS to automatically determine optimum architectures for various point cloud tasks. Specifically, we design an elastic feature extraction (EFE) module that serves as a basic unit for architecture search, which expands seamlessly alongside both the width and depth of the network for efficient feature extraction. Based on the EFE module, we devise a searching space, which is encoded into a supernet to provide a wide number of latent network structures for a particular point cloud task. To fully optimize the weights of the supernet, we propose a weight coupling sandwich rule that samples the largest, smallest, and multiple medium models at each iteration and fuses their gradients to update the supernet. Furthermore, we present a united gradient adjustment algorithm that mitigates gradient conflict induced by distinct gradient directions of sampled models and supernet, thus expediting the convergence of the supernet and assuring that it can be comprehensively trained. Pursuant to the provided techniques, the trained supernet enables a multitude of subnets to be incredibly well-optimized. Finally, we conduct an evolutionary search for the supernet under resource constraints to find promising architectures for different tasks. Experimentally, the searched Point-NAS with weights inherited from the supernet realizes outstanding results across a variety of benchmarks. i.e., 94.2% and 88.9% overall accuracy under ModelNet40 and ScanObjectNN, 68.6% mIoU under S3DIS, 63.6% and 69.3% mAP@0.25 under SUN RGB-D and ScanNet V2 datasets.
3071.
Dimanov, Daniel
2023.
@phdthesis{Dimanov-phd23a,
title = {Efficient Multi-Objective NeuroEvolution in Computer Vision and Applications for Threat Identification},
author = {Daniel Dimanov},
url = {http://eprints.bournemouth.ac.uk/39138/1/DIMANOV%2C%20Daniel_Ph.D._2023.pdf},
year = {2023},
date = {2023-11-01},
urldate = {2023-11-01},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
3070.
Li, Yueyang; Wu, Zhouejie; Shen, Junfei; Zhang, Qican
In: Opt. Express, vol. 31, no. 24, pp. 40803–40823, 2023.
@article{Li:23,
title = {Real-time 3D shape measurement of dynamic scenes using fringe projection profilometry: lightweight NAS-optimized dual frequency deep learning approach},
author = {Yueyang Li and Zhouejie Wu and Junfei Shen and Qican Zhang},
url = {https://opg.optica.org/oe/abstract.cfm?URI=oe-31-24-40803},
doi = {10.1364/OE.506343},
year = {2023},
date = {2023-11-01},
urldate = {2023-11-01},
journal = {Opt. Express},
volume = {31},
number = {24},
pages = {40803–40823},
publisher = {Optica Publishing Group},
abstract = {Achieving real-time and high-accuracy 3D reconstruction of dynamic scenes is a fundamental challenge in many fields, including online monitoring, augmented reality, and so on. On one hand, traditional methods, such as Fourier transform profilometry (FTP) and phase-shifting profilometry (PSP), are struggling to balance measuring efficiency and accuracy. On the other hand, deep learning-based approaches, which offer the potential for improved accuracy, are hindered by large parameter amounts and complex structures less amenable to real-time requirements. To solve this problem, we proposed a network architecture search (NAS)-based method for real-time processing and 3D measurement of dynamic scenes with rate equivalent to single-shot. A NAS-optimized lightweight neural network was designed for efficient phase demodulation, while an improved dual-frequency strategy was employed coordinately for flexible absolute phase unwrapping. The experiment results demonstrate that our method can effectively perform 3D reconstruction with a reconstruction speed of 58fps, and realize high-accuracy measurement of dynamic scenes based on deep learning for what we believe to be the first time with the average RMS error of about 0.08 mm.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Achieving real-time and high-accuracy 3D reconstruction of dynamic scenes is a fundamental challenge in many fields, including online monitoring, augmented reality, and so on. On one hand, traditional methods, such as Fourier transform profilometry (FTP) and phase-shifting profilometry (PSP), are struggling to balance measuring efficiency and accuracy. On the other hand, deep learning-based approaches, which offer the potential for improved accuracy, are hindered by large parameter amounts and complex structures less amenable to real-time requirements. To solve this problem, we proposed a network architecture search (NAS)-based method for real-time processing and 3D measurement of dynamic scenes with rate equivalent to single-shot. A NAS-optimized lightweight neural network was designed for efficient phase demodulation, while an improved dual-frequency strategy was employed coordinately for flexible absolute phase unwrapping. The experiment results demonstrate that our method can effectively perform 3D reconstruction with a reconstruction speed of 58fps, and realize high-accuracy measurement of dynamic scenes based on deep learning for what we believe to be the first time with the average RMS error of about 0.08 mm.
3069.
Xie, G.; Li, Q.; Shi, Z.; Fang, H.; Ji, S.; Jiang, Y.; Yuan, Z.; Ma, L.; Xu, M.
Generating Neural Networks for Diverse Networking Classification Tasks via Hardware-Aware Neural Architecture Search Journal Article
In: IEEE Transactions on Computers, no. 01, pp. 1-14, 2023, ISSN: 1557-9956.
@article{10323250,
title = {Generating Neural Networks for Diverse Networking Classification Tasks via Hardware-Aware Neural Architecture Search},
author = {G. Xie and Q. Li and Z. Shi and H. Fang and S. Ji and Y. Jiang and Z. Yuan and L. Ma and M. Xu},
url = {https://www.computer.org/csdl/journal/tc/5555/01/10323250/1SewSv3Y4VO},
doi = {10.1109/TC.2023.3333253},
issn = {1557-9956},
year = {2023},
date = {2023-11-01},
urldate = {5555-11-01},
journal = {IEEE Transactions on Computers},
number = {01},
pages = {1-14},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
abstract = {Neural networks (NNs) are widely used in classification-based networking analysis to help traffic transmission and system security. However, there are heterogeneous network devices (e.g., switches and routers) in a network. Manually customizing NNs with specific device requirements (e.g., max allowed running latency) can be time-consuming and labor-intensive. Furthermore, the diverse data characteristics of different networking classification tasks add to the burden of NN customization. This paper introduces Loong, a neural architecture search (NAS) based system that automatically generates NNs for various networking tasks and devices. Loong includes a neural operation embedding module, which embeds candidate neural operations into the layer to be designed. Then, the layer-wise training is used to generate a task-specific NN layer by layer. This layer-wise scheme simultaneously trains and selects candidate neural operations using gradient feedback. Finally, only the important operations are selected to form the layer, maximizing accuracy. By incorporating multiple objectives, including deployment memory and running latency of devices, into the training and selection of NNs, Loong is able to customize NNs for heterogeneous network devices. Experiments show that Loong’s NNs outperform 13 manual-designed and NAS-based NNs, with a 4.11% improvement in F1-score. Additionally, Loong’s NNs achieve faster (7.92X) speeds on commodity devices.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Neural networks (NNs) are widely used in classification-based networking analysis to help traffic transmission and system security. However, there are heterogeneous network devices (e.g., switches and routers) in a network. Manually customizing NNs with specific device requirements (e.g., max allowed running latency) can be time-consuming and labor-intensive. Furthermore, the diverse data characteristics of different networking classification tasks add to the burden of NN customization. This paper introduces Loong, a neural architecture search (NAS) based system that automatically generates NNs for various networking tasks and devices. Loong includes a neural operation embedding module, which embeds candidate neural operations into the layer to be designed. Then, the layer-wise training is used to generate a task-specific NN layer by layer. This layer-wise scheme simultaneously trains and selects candidate neural operations using gradient feedback. Finally, only the important operations are selected to form the layer, maximizing accuracy. By incorporating multiple objectives, including deployment memory and running latency of devices, into the training and selection of NNs, Loong is able to customize NNs for heterogeneous network devices. Experiments show that Loong’s NNs outperform 13 manual-designed and NAS-based NNs, with a 4.11% improvement in F1-score. Additionally, Loong’s NNs achieve faster (7.92X) speeds on commodity devices.
3068.
Wang, G.; Li, C.; Yuan, L.; Peng, J.; Xian, X.; Liang, X.; Chang, X.; Lin, L.
DNA Family: Boosting Weight-Sharing NAS with Block-Wise Supervisions Journal Article
In: IEEE Transactions on Pattern Analysis & Machine Intelligence, no. 01, pp. 1-18, 2023, ISSN: 1939-3539.
@article{10324326,
title = {DNA Family: Boosting Weight-Sharing NAS with Block-Wise Supervisions},
author = {G. Wang and C. Li and L. Yuan and J. Peng and X. Xian and X. Liang and X. Chang and L. Lin},
url = {https://www.computer.org/csdl/journal/tp/5555/01/10324326/1SgbFgBOI7u},
doi = {10.1109/TPAMI.2023.3335261},
issn = {1939-3539},
year = {2023},
date = {2023-11-01},
urldate = {5555-11-01},
journal = {IEEE Transactions on Pattern Analysis & Machine Intelligence},
number = {01},
pages = {1-18},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
abstract = {Neural Architecture Search (NAS), aiming at automatically designing neural architectures by machines, has been considered a key step toward automatic machine learning. One notable NAS branch is the weight-sharing NAS, which significantly improves search efficiency and allows NAS algorithms to run on ordinary computers. Despite receiving high expectations, this category of methods suffers from low search effectiveness. By employing a generalization boundedness tool, we demonstrate that the devil behind this drawback is the untrustworthy architecture rating with the oversized search space of the possible architectures. Addressing this problem, we modularize a large search space into blocks with small search spaces and develop a family of models with the distilling neural architecture (DNA) techniques. These proposed models, namely a DNA family, are capable of resolving multiple dilemmas of the weight-sharing NAS, such as scalability, efficiency, and multi-modal compatibility. Our proposed DNA models can rate all architecture candidates, as opposed to previous works that can only access a sub- search space using heuristic algorithms. Moreover, under a certain computational complexity constraint, our method can seek architectures with different depths and widths. Extensive experimental evaluations show that our models achieve state-of-the-art top-1 accuracy of 78.9% and 83.6% on ImageNet for a mobile convolutional network and a small vision transformer, respectively. Additionally, we provide in-depth empirical analysis and insights into neural architecture ratings. Codes available: https://github.com/changlin31/DNA.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Neural Architecture Search (NAS), aiming at automatically designing neural architectures by machines, has been considered a key step toward automatic machine learning. One notable NAS branch is the weight-sharing NAS, which significantly improves search efficiency and allows NAS algorithms to run on ordinary computers. Despite receiving high expectations, this category of methods suffers from low search effectiveness. By employing a generalization boundedness tool, we demonstrate that the devil behind this drawback is the untrustworthy architecture rating with the oversized search space of the possible architectures. Addressing this problem, we modularize a large search space into blocks with small search spaces and develop a family of models with the distilling neural architecture (DNA) techniques. These proposed models, namely a DNA family, are capable of resolving multiple dilemmas of the weight-sharing NAS, such as scalability, efficiency, and multi-modal compatibility. Our proposed DNA models can rate all architecture candidates, as opposed to previous works that can only access a sub- search space using heuristic algorithms. Moreover, under a certain computational complexity constraint, our method can seek architectures with different depths and widths. Extensive experimental evaluations show that our models achieve state-of-the-art top-1 accuracy of 78.9% and 83.6% on ImageNet for a mobile convolutional network and a small vision transformer, respectively. Additionally, we provide in-depth empirical analysis and insights into neural architecture ratings. Codes available: https://github.com/changlin31/DNA.
3067.
He, Xin; Chu, Xiaowen
MedPipe: End-to-End Joint Search of Data Augmentation and Neural Architecture for 3D Medical Image Classification Journal Article
In: 2023.
@article{He_2023,
title = {MedPipe: End-to-End Joint Search of Data Augmentation and Neural Architecture for 3D Medical Image Classification},
author = {Xin He and Xiaowen Chu},
url = {http://dx.doi.org/10.36227/techrxiv.19513780.v2},
doi = {10.36227/techrxiv.19513780.v2},
year = {2023},
date = {2023-11-01},
urldate = {2023-11-01},
publisher = {Institute of Electrical and Electronics Engineers (IEEE)},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3066.
Kim, Bosung; Lee, Seulki
On-NAS: On-Device Neural Architecture Search on Memory-Constrained Intelligent Embedded Systems Bachelor Thesis
2023.
@bachelorthesis{Kim-sensys23a,
title = {On-NAS: On-Device Neural Architecture Search on Memory-Constrained Intelligent Embedded Systems},
author = {Bosung Kim and Seulki Lee},
url = {https://aigs.unist.ac.kr/filebox/item/1917192674_8d43d5a0_On-NAS_SenSys2023.pdf},
year = {2023},
date = {2023-11-01},
urldate = {2023-11-01},
journal = {ACM Conference on Embedded Networked Sensor Systems (SenSys ’23},
keywords = {},
pubstate = {published},
tppubtype = {bachelorthesis}
}
3065.
Cai, Zhiqiang; Chen, Jialin; Xu, Ke; Wang, Lingli
Recognizing Good Variational Quantum Circuits with Monte Carlo Tree Search Technical Report
2023.
@techreport{Cai-rs23a,
title = {Recognizing Good Variational Quantum Circuits with Monte Carlo Tree Search},
author = {Zhiqiang Cai and Jialin Chen and Ke Xu and Lingli Wang},
url = {https://www.researchsquare.com/article/rs-3490986/v1},
year = {2023},
date = {2023-10-27},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
3064.
Xu, Zheng; Jain, Deepak Kumar; Shamsolmoali, Pourya; Goli, Alireza; Neelakandan, Subramani; Jain, Amar
Slime Mold optimization with hybrid deep learning enabled crowd-counting approach in video surveillance Journal Article
In: Machine Learning and Big Data Analytics for IoT Security and Privacy (SPIoT 2022), 2023.
@article{Xu-SPIoT22a,
title = {Slime Mold optimization with hybrid deep learning enabled crowd-counting approach in video surveillance},
author = {
Zheng Xu and Deepak Kumar Jain and Pourya Shamsolmoali and Alireza Goli and Subramani Neelakandan and Amar Jain
},
url = {https://link.springer.com/article/10.1007/s00521-023-09083-x},
year = {2023},
date = {2023-10-26},
journal = {Machine Learning and Big Data Analytics for IoT Security and Privacy (SPIoT 2022)},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3063.
Yang, Lei; Mei, Sen; Liang, Pan; Li, Yan; Ma, Ling; Gao, Jianbo; Jiang, Huiqin
A 3D prediction model for benign or malignant of pulmonary nodules based on neural architecture search Journal Article
In: Signal, Image and Video Processing , 2023.
@article{Yang-SIV23a,
title = {A 3D prediction model for benign or malignant of pulmonary nodules based on neural architecture search},
author = {
Lei Yang and Sen Mei and Pan Liang and Yan Li and Ling Ma and Jianbo Gao and Huiqin Jiang
},
url = {https://link.springer.com/article/10.1007/s11760-023-02807-5},
year = {2023},
date = {2023-10-18},
journal = {Signal, Image and Video Processing },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3062.
Wu, Zhenpeng; Chen, Jiamin; Al-Sabri, Raeed; Oloulade, Babatounde Moctard; Gao, Jianliang
Adaptive graph contrastive learning with joint optimization of data augmentation and graph encoder Journal Article
In: Knowledge and Information Systems , 2023.
@article{Wu-KIS23a,
title = {Adaptive graph contrastive learning with joint optimization of data augmentation and graph encoder},
author = {
Zhenpeng Wu and Jiamin Chen and Raeed Al-Sabri and Babatounde Moctard Oloulade and Jianliang Gao
},
url = {https://link.springer.com/article/10.1007/s10115-023-01979-3},
year = {2023},
date = {2023-10-12},
urldate = {2023-10-12},
journal = {Knowledge and Information Systems },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3061.
Ragusa, Edoardo; Dosen, Straginja; Zunino, Rodolfo; Gastaldo, Paolo
Affordance Segmentation Using Tiny Networks for Sensing Systems in Wearable Robotic Devices Journal Article
In: IEEE SENSORS JOURNAL, , 2023.
@article{Ragusa-ieeesensorsjournal,
title = {Affordance Segmentation Using Tiny Networks for Sensing Systems in Wearable Robotic Devices},
author = {Edoardo Ragusa and Straginja Dosen and Rodolfo Zunino and Paolo Gastaldo},
url = {https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=10235885&tag=1},
year = {2023},
date = {2023-10-01},
journal = {IEEE SENSORS JOURNAL, },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3060.
(Ed.)
Designing a New Search Space for Multivariate Time-Series Neural Architecture Search Collection
2023.
@collection{MacKinnon-ecmlaaltd23a,
title = {Designing a New Search Space for Multivariate Time-Series Neural Architecture Search},
author = {Christopher MacKinnon and Robert Atkinson},
url = {https://ecml-aaltd.github.io/aaltd2023/papers/Designing_a_New_Search_Space_for_Multivariate_Time_Series_Neural_Architecture_Search___AALTD__ECML_PKDD_%20(32).pdf},
year = {2023},
date = {2023-10-01},
urldate = {2023-10-01},
booktitle = {8th Workshop on Advanced Analytics and Learning on Temporal Data (AALTD 2023), ECML 2023},
keywords = {},
pubstate = {published},
tppubtype = {collection}
}
3059.
García, Jesús Leopoldo Llano; Monroy, Raúl; Hernández, Víctor Adrián Sosa
An Experimental Protocol for Neural Architecture Search in Super-Resolution Proceedings Article
In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pp. 4139-4146, 2023.
@inproceedings{Garcia_2023_ICCV,
title = {An Experimental Protocol for Neural Architecture Search in Super-Resolution},
author = {Jesús Leopoldo Llano García and Raúl Monroy and Víctor Adrián Sosa Hernández},
url = {https://openaccess.thecvf.com/content/ICCV2023W/LXCV/html/Garcia_An_Experimental_Protocol_for_Neural_Architecture_Search_in_Super-Resolution_ICCVW_2023_paper.html},
year = {2023},
date = {2023-10-01},
urldate = {2023-10-01},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops},
pages = {4139-4146},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
3058.
Soro, Bedionita; Song, Chong
Enhancing Differentiable Architecture Search: A Study on Small Number of Cell Blocks in the Search Stage, and Important Branches-Based Cells Selection Proceedings Article
In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pp. 1253-1261, 2023.
@inproceedings{Soro_2023_ICCV,
title = {Enhancing Differentiable Architecture Search: A Study on Small Number of Cell Blocks in the Search Stage, and Important Branches-Based Cells Selection},
author = {Bedionita Soro and Chong Song},
url = {https://openaccess.thecvf.com/content/ICCV2023W/RCV/papers/Soro_Enhancing_Differentiable_Architecture_Search_A_Study_on_Small_Number_of_ICCVW_2023_paper.pdf},
year = {2023},
date = {2023-10-01},
urldate = {2023-10-01},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops},
pages = {1253-1261},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
3057.
Rosales, Rafael; Munoz, Pablo; Paulitsch, Michael
Assessing the Impact of Diversity on the Resilience of Deep Learning Ensembles: A Comparative Study on Model Architecture, Output, Activation, and Attribution Proceedings Article
In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pp. 4406-4416, 2023.
@inproceedings{Rosales_2023_ICCV,
title = {Assessing the Impact of Diversity on the Resilience of Deep Learning Ensembles: A Comparative Study on Model Architecture, Output, Activation, and Attribution},
author = {Rafael Rosales and Pablo Munoz and Michael Paulitsch},
url = {https://openaccess.thecvf.com/content/ICCV2023W/OODCV/html/Rosales_Assessing_the_Impact_of_Diversity_on_the_Resilience_of_Deep_ICCVW_2023_paper.html},
year = {2023},
date = {2023-10-01},
urldate = {2023-10-01},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops},
pages = {4406-4416},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
3056.
Bhardwaj, Kartikeya; Cheng, Hsin-Pai; Priyadarshi, Sweta; Li, Zhuojin
ZiCo-BC: A Bias Corrected Zero-Shot NAS for Vision Tasks Proceedings Article
In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pp. 1353-1357, 2023.
@inproceedings{Bhardwaj_2023_ICCV,
title = {ZiCo-BC: A Bias Corrected Zero-Shot NAS for Vision Tasks},
author = {Kartikeya Bhardwaj and Hsin-Pai Cheng and Sweta Priyadarshi and Zhuojin Li},
url = {https://openaccess.thecvf.com/content/ICCV2023W/RCV/papers/Bhardwaj_ZiCo-BC_A_Bias_Corrected_Zero-Shot_NAS_for_Vision_Tasks_ICCVW_2023_paper.pdf},
year = {2023},
date = {2023-10-01},
urldate = {2023-10-01},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops},
pages = {1353-1357},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
3055.
Cavagnero, Niccolò; Robbiano, Luca; Pistilli, Francesca; Caputo, Barbara; Averta, Giuseppe
Entropic Score Metric: Decoupling Topology and Size in Training-Free NAS Proceedings Article
In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pp. 1459-1468, 2023.
@inproceedings{Cavagnero_2023_ICCV,
title = {Entropic Score Metric: Decoupling Topology and Size in Training-Free NAS},
author = {Niccolò Cavagnero and Luca Robbiano and Francesca Pistilli and Barbara Caputo and Giuseppe Averta},
url = {https://openaccess.thecvf.com/content/ICCV2023W/RCV/papers/Cavagnero_Entropic_Score_Metric_Decoupling_Topology_and_Size_in_Training-Free_NAS_ICCVW_2023_paper.pdf},
year = {2023},
date = {2023-10-01},
urldate = {2023-10-01},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops},
pages = {1459-1468},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
3054.
Sakuma, Yuiko; Ishii, Masato; Narihira, Takuya
DetOFA: Efficient Training of Once-for-All Networks for Object Detection Using Path Filter Proceedings Article
In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pp. 1333-1342, 2023.
@inproceedings{Sakuma_2023_ICCV,
title = {DetOFA: Efficient Training of Once-for-All Networks for Object Detection Using Path Filter},
author = {Yuiko Sakuma and Masato Ishii and Takuya Narihira},
url = {https://openaccess.thecvf.com/content/ICCV2023W/RCV/papers/Sakuma_DetOFA_Efficient_Training_of_Once-for-All_Networks_for_Object_Detection_Using_ICCVW_2023_paper.pdf},
year = {2023},
date = {2023-10-01},
urldate = {2023-10-01},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops},
pages = {1333-1342},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
3053.
Akinola, Solomon Oluwole; Qingguo, Wang; Olukanmi, Peter; Tshilidzi, Marwala
A Boosted Evolutionary Neural Architecture Search for Timeseries Forecasting with Application to South African COVID-19 Cases Journal Article
In: International Journal of Online and Biomedical Engineering (iJOE), vol. 19, no. 14, pp. pp. 107–130, 2023.
@article{Akinola_Qingguo_Olukanmi_Tshilidzi_2023,
title = {A Boosted Evolutionary Neural Architecture Search for Timeseries Forecasting with Application to South African COVID-19 Cases},
author = {Solomon Oluwole Akinola and Wang Qingguo and Peter Olukanmi and Marwala Tshilidzi},
url = {https://online-journals.org/index.php/i-joe/article/view/41291},
doi = {10.3991/ijoe.v19i14.41291},
year = {2023},
date = {2023-10-01},
urldate = {2023-10-01},
journal = {International Journal of Online and Biomedical Engineering (iJOE)},
volume = {19},
number = {14},
pages = {pp. 107–130},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3052.
Zhang, Lei; Chen, Zhiqian; Lu, Chang-Tien; Zhao, Liang
Fast and Adaptive Dynamics-on-Graphs to Dynamics-of-Graphs Translation Journal Article
In: Sec. Data Mining and Management , 2023.
@article{Zhang-secdmm23a,
title = { Fast and Adaptive Dynamics-on-Graphs to Dynamics-of-Graphs Translation },
author = { Lei Zhang and Zhiqian Chen and Chang-Tien Lu and Liang Zhao},
url = {https://www.frontiersin.org/articles/10.3389/fdata.2023.1274135/abstract},
doi = { 10.3389/fdata.2023.1274135},
year = {2023},
date = {2023-10-01},
urldate = {2023-10-01},
journal = {Sec. Data Mining and Management },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3051.
Hoang, Quang Minh
Practical Methods for Automated Algorithm Design in Machine Learning and Computational Biology PhD Thesis
2023.
@phdthesis{nokey,
title = {Practical Methods for Automated Algorithm Design in Machine Learning and Computational Biology},
author = { Quang Minh Hoang},
url = {http://reports-archive.adm.cs.cmu.edu/anon/2023/abstracts/23-139.html},
year = {2023},
date = {2023-10-01},
urldate = {2023-10-01},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
3050.
An, S.; Channing, G.; Schuman, C.; Taufer, M.
VINARCH: A Visual Analytics Interactive Tool for Neural Network Archaeology Proceedings Article
In: 2023 IEEE International Conference on Cluster Computing Workshops (CLUSTER Workshops), pp. 50-51, IEEE Computer Society, Los Alamitos, CA, USA, 2023.
@inproceedings{10321892,
title = {VINARCH: A Visual Analytics Interactive Tool for Neural Network Archaeology},
author = {S. An and G. Channing and C. Schuman and M. Taufer},
url = {https://doi.ieeecomputersociety.org/10.1109/CLUSTERWorkshops61457.2023.00020},
doi = {10.1109/CLUSTERWorkshops61457.2023.00020},
year = {2023},
date = {2023-10-01},
urldate = {2023-10-01},
booktitle = {2023 IEEE International Conference on Cluster Computing Workshops (CLUSTER Workshops)},
pages = {50-51},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
abstract = {Most neural networks (NNs) generated by neural architecture search (NAS) are discarded except for the final output to limit the memory usage on high performance computing (HPC) systems on which the search is performed. However, discarded NNs are vital for understanding the NAS structure’s evolution and reproducibility. We design a visual interactive tool for NN archaeology that explores the evolution of NAS structures, finds matching subsequences in the structures, and visualizes NN similarities across NAS outputs, including discarded NNs. We demonstrate the capabilities of our tool to discover and visualize matching subsequences on a dataset of NNs generated by NSGA-Net, a genetic NAS.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Most neural networks (NNs) generated by neural architecture search (NAS) are discarded except for the final output to limit the memory usage on high performance computing (HPC) systems on which the search is performed. However, discarded NNs are vital for understanding the NAS structure’s evolution and reproducibility. We design a visual interactive tool for NN archaeology that explores the evolution of NAS structures, finds matching subsequences in the structures, and visualizes NN similarities across NAS outputs, including discarded NNs. We demonstrate the capabilities of our tool to discover and visualize matching subsequences on a dataset of NNs generated by NSGA-Net, a genetic NAS.
3049.
Hu, Xuemei; Huang, Lan; Zeng, Jia; Wang, Kangping; Wang, Yan
EGFA-NAS: a neural architecture search method based on explosion gravitation field algorithm Journal Article
In: Complex & Intelligent Systems , 2023.
@article{hu-cis23a,
title = {EGFA-NAS: a neural architecture search method based on explosion gravitation field algorithm},
author = {
Xuemei Hu and Lan Huang and Jia Zeng and Kangping Wang and Yan Wang
},
url = {https://link.springer.com/article/10.1007/s40747-023-01230-0},
year = {2023},
date = {2023-09-30},
urldate = {2023-09-30},
journal = {Complex & Intelligent Systems },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3048.
Pan, Yang; Jin, Mingwu; Zhang, Shun-Rong; Wing, Simon; Deng, Yue
Neural Network Models for Ionospheric Electron Density Prediction: A Neural Architecture Search Study Journal Article
In: ESS Open Archive, 2023.
@article{Pan-ESS23a,
title = {Neural Network Models for Ionospheric Electron Density Prediction: A Neural Architecture Search Study},
author = {Yang Pan and Mingwu Jin and Shun-Rong Zhang and Simon Wing and Yue Deng},
url = {https://essopenarchive.org/users/281372/articles/669825-neural-network-models-for-ionospheric-electron-density-prediction-a-neural-architecture-search-study},
year = {2023},
date = {2023-09-30},
urldate = {2023-09-30},
journal = {ESS Open Archive},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3047.
O'Neill, Damien
2023.
@phdthesis{Neill-phd23a,
title = {Evolutionary Computation for the Optimisation of Skip-Connection Structures on Dense Convolutional Neural Networks},
author = {O'Neill, Damien},
url = {https://openaccess.wgtn.ac.nz/articles/thesis/Evolutionary_Computation_for_the_Optimisation_of_Skip-Connection_Structures_on_Dense_Convolutional_Neural_Networks/24210165},
year = {2023},
date = {2023-09-30},
urldate = {2023-09-30},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
3046.
(Ed.)
2023.
@collection{Zeng-icv23a,
title = {MPCViT: Searching for Accurate and Efficient MPC-Friendly Vision Transformer with Heterogeneous Attention},
author = {Wenxuan Zeng and Meng Li and Wenjie Xiong and Tong Tong and Wen-jie Lu and Jin Tan and Runsheng Wang and Ru Huang},
url = {https://openaccess.thecvf.com/content/ICCV2023/papers/Zeng_MPCViT_Searching_for_Accurate_and_Efficient_MPC-Friendly_Vision_Transformer_with_ICCV_2023_paper.pdf},
year = {2023},
date = {2023-09-30},
urldate = {2023-09-30},
booktitle = {ICCV2023},
keywords = {},
pubstate = {published},
tppubtype = {collection}
}
3045.
(Ed.)
Automated Knowledge Distillation via Monte Carlo Tree Search Collection
2023.
@collection{Li-iccv23ab,
title = {Automated Knowledge Distillation via Monte Carlo Tree Search},
author = {Lujun Li and Peijie Dong and Zimian Wei and Ya Yang},
url = {https://openaccess.thecvf.com/content/ICCV2023/papers/Li_Automated_Knowledge_Distillation_via_Monte_Carlo_Tree_Search_ICCV_2023_paper.pdf},
year = {2023},
date = {2023-09-30},
urldate = {2023-09-30},
booktitle = {ICCV 2023},
keywords = {},
pubstate = {published},
tppubtype = {collection}
}
3044.
Xu, Yang; Ma, Yongjie
Evolutionary neural architecture search combining multi-branch ConvNet and improved transformer Journal Article
In: 2023.
@article{Xu-ja23a,
title = {Evolutionary neural architecture search combining multi-branch ConvNet and improved transformer},
author = {Yang Xu and Yongjie Ma
},
url = {https://www.nature.com/articles/s41598-023-42931-3},
year = {2023},
date = {2023-09-22},
urldate = {2023-09-22},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3043.
(Ed.)
2023.
@collection{Lukasik-dagm,
title = {An Evaluation of Zero-Cost Proxies - from Neural Architecture Performance Prediction to Model Robustness},
author = {Jovita Lukasik and Michael Moeller and Margret Keuper},
url = {https://www.dagm-gcpr.de/fileadmin/dagm-gcpr/pictures/2023_Heidelberg/Paper_MainTrack/064.pdf},
year = {2023},
date = {2023-09-19},
urldate = {2023-09-19},
booktitle = {DAGM German Conference on Pattern Recognition},
keywords = {},
pubstate = {published},
tppubtype = {collection}
}
3042.
Rogers, Brendan; Noman, Nasimul; Chalup, Stephan; Moscato, Pablo
A comparative analysis of deep neural network architectures for sentence classification using genetic algorithm Journal Article
In: Evolutionary Intelligence, 2023.
@article{Rogers-ei23a,
title = {A comparative analysis of deep neural network architectures for sentence classification using genetic algorithm},
author = {Brendan Rogers and Nasimul Noman and Stephan Chalup and Pablo Moscato
},
url = {https://link.springer.com/article/10.1007/s12065-023-00874-8},
year = {2023},
date = {2023-09-08},
urldate = {2023-09-08},
journal = {Evolutionary Intelligence},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3041.
Tang, Yi-Jun; Yan, Ke; Zhang, Xingyi; Tian, Ye; Liu, Bin
Protein intrinsically disordered region prediction by combining neural architecture search and multi-objective genetic algorithm Journal Article
In: BMC Biology , 2023.
@article{Tang-bmc23a,
title = {Protein intrinsically disordered region prediction by combining neural architecture search and multi-objective genetic algorithm},
author = {Yi-Jun Tang and Ke Yan and Xingyi Zhang and Ye Tian and Bin Liu
},
url = {https://bmcbiol.biomedcentral.com/articles/10.1186/s12915-023-01672-5},
year = {2023},
date = {2023-09-07},
urldate = {2023-09-07},
journal = {BMC Biology },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3040.
Afif, Mouna; Ayachi, Riadh; Said, Yahia; Atri, Mohamed
An indoor scene recognition system based on deep learning evolutionary algorithms Journal Article
In: Soft Computing , 2023.
@article{Afif-sc23a,
title = {An indoor scene recognition system based on deep learning evolutionary algorithms},
author = {Mouna Afif and Riadh Ayachi and Yahia Said and Mohamed Atri
},
url = {https://link.springer.com/article/10.1007/s00500-023-09177-7},
year = {2023},
date = {2023-09-05},
urldate = {2023-09-05},
journal = {Soft Computing },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3039.
Zhang, Junfeng; Xie, Cheng; Yu, Beibei; Yang, Rui
Adaptive Hierarchical Knowledge Distillation from GNNs to MLPs Technical Report
2023.
@techreport{Zhang,
title = {Adaptive Hierarchical Knowledge Distillation from GNNs to MLPs},
author = {Junfeng Zhang and Cheng Xie and Beibei Yu and Rui Yang},
url = {https://www.researchsquare.com/article/rs-3258299/v1},
year = {2023},
date = {2023-09-01},
urldate = {2023-09-01},
keywords = {},
pubstate = {published},
tppubtype = {techreport}
}
3038.
Cheung, Ming
Learning from the Past: Fast NAS for Tasks and Datasets Journal Article
In: ACM Trans. Multimedia Comput. Commun. Appl., 2023, ISSN: 1551-6857, (Just Accepted).
@article{10.1145/3618000,
title = {Learning from the Past: Fast NAS for Tasks and Datasets},
author = {Ming Cheung},
url = {https://doi.org/10.1145/3618000},
doi = {10.1145/3618000},
issn = {1551-6857},
year = {2023},
date = {2023-09-01},
urldate = {2023-09-01},
journal = {ACM Trans. Multimedia Comput. Commun. Appl.},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
abstract = {Nowadays, with the advancement of technology, many retail companies require in-house data scientist teams to build machine learning tasks, such as user segmentation and item price prediction. These teams typically use a trial-and-error process to obtain a good model for a given dataset and machine learning task, which is time-consuming and requires expertise. On the other hand, the team may have built models for other tasks on different datasets. This paper proposes a framework to obtain a model architecture using the previous solved machine learning tasks and datasets. By analyzing real datasets with over 70,000 images from 11 online retail e-commerce websites, it is demonstrated that the performance of a model is related to the similarity among datasets, models, and machine learning tasks. A framework is hence proposed to obtain the model using the similarities among them. It was proven that the model was 26.6% better in accuracy, and using only 20% of the runtime while comparing to a auto network architecture search library, auto-keras, in predicting the attributes of fashion images. To the best of our knowledge, this is the first paper to obtain the best model based on the similarity among machine learning tasks, models, and datasets.},
note = {Just Accepted},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Nowadays, with the advancement of technology, many retail companies require in-house data scientist teams to build machine learning tasks, such as user segmentation and item price prediction. These teams typically use a trial-and-error process to obtain a good model for a given dataset and machine learning task, which is time-consuming and requires expertise. On the other hand, the team may have built models for other tasks on different datasets. This paper proposes a framework to obtain a model architecture using the previous solved machine learning tasks and datasets. By analyzing real datasets with over 70,000 images from 11 online retail e-commerce websites, it is demonstrated that the performance of a model is related to the similarity among datasets, models, and machine learning tasks. A framework is hence proposed to obtain the model using the similarities among them. It was proven that the model was 26.6% better in accuracy, and using only 20% of the runtime while comparing to a auto network architecture search library, auto-keras, in predicting the attributes of fashion images. To the best of our knowledge, this is the first paper to obtain the best model based on the similarity among machine learning tasks, models, and datasets.
3037.
Chitty-Venkata, Krishna Teja
Hardware-aware design, search, and optimization of deep neural networks Bachelor Thesis
2023.
@bachelorthesis{Venkata-phD23a,
title = {Hardware-aware design, search, and optimization of deep neural networks},
author = {Krishna Teja Chitty-Venkata},
url = {https://www.proquest.com/openview/4ab8b9b7b6c338ab4729cc0a11279c45/1?pq-origsite=gscholar&cbl=18750&diss=y},
year = {2023},
date = {2023-09-01},
urldate = {2023-09-01},
keywords = {},
pubstate = {published},
tppubtype = {bachelorthesis}
}
3036.
Yang, Zhao; Sun, Qingshuang
Energy-Efficient Personalized Federated Search with Graph for Edge Computing Journal Article
In: ACM Trans. Embed. Comput. Syst., vol. 22, no. 5s, 2023, ISSN: 1539-9087.
@article{10.1145/3609435,
title = {Energy-Efficient Personalized Federated Search with Graph for Edge Computing},
author = {Zhao Yang and Qingshuang Sun},
url = {https://doi.org/10.1145/3609435},
doi = {10.1145/3609435},
issn = {1539-9087},
year = {2023},
date = {2023-09-01},
urldate = {2023-09-01},
journal = {ACM Trans. Embed. Comput. Syst.},
volume = {22},
number = {5s},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
abstract = {Federated Learning (FL) is a popular method for privacy-preserving machine learning on edge devices. However, the heterogeneity of edge devices, including differences in system architecture, data, and co-running applications, can significantly impact the energy efficiency of FL. To address these issues, we propose an energy-efficient personalized federated search framework. This framework has three key components. Firstly, we search for partial models with high inference efficiency to reduce training energy consumption and the occurrence of stragglers in each round. Secondly, we build lightweight search controllers that control the model sampling and respond to runtime variances, mitigating new straggler issues caused by co-running applications. Finally, we design an adaptive search update strategy based on graph aggregation to improve personalized training convergence. Our framework reduces the energy consumption of the training process by lowering the training overhead of each round and speeding up the training convergence rate. Experimental results show that our approach achieves up to 5.02% accuracy and 3.45× energy efficiency improvements.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Federated Learning (FL) is a popular method for privacy-preserving machine learning on edge devices. However, the heterogeneity of edge devices, including differences in system architecture, data, and co-running applications, can significantly impact the energy efficiency of FL. To address these issues, we propose an energy-efficient personalized federated search framework. This framework has three key components. Firstly, we search for partial models with high inference efficiency to reduce training energy consumption and the occurrence of stragglers in each round. Secondly, we build lightweight search controllers that control the model sampling and respond to runtime variances, mitigating new straggler issues caused by co-running applications. Finally, we design an adaptive search update strategy based on graph aggregation to improve personalized training convergence. Our framework reduces the energy consumption of the training process by lowering the training overhead of each round and speeding up the training convergence rate. Experimental results show that our approach achieves up to 5.02% accuracy and 3.45× energy efficiency improvements.
3035.
Mousavi, Hamid; Loni, Mohammad; Alibeigi, Mina; Daneshtalab, Masoud
DASS: Differentiable Architecture Search for Sparse Neural Networks Journal Article
In: ACM Trans. Embed. Comput. Syst., vol. 22, no. 5s, 2023, ISSN: 1539-9087.
@article{10.1145/3609385,
title = {DASS: Differentiable Architecture Search for Sparse Neural Networks},
author = {Hamid Mousavi and Mohammad Loni and Mina Alibeigi and Masoud Daneshtalab},
url = {https://doi.org/10.1145/3609385},
doi = {10.1145/3609385},
issn = {1539-9087},
year = {2023},
date = {2023-09-01},
urldate = {2023-09-01},
journal = {ACM Trans. Embed. Comput. Syst.},
volume = {22},
number = {5s},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
abstract = {The deployment of Deep Neural Networks (DNNs) on edge devices is hindered by the substantial gap between performance requirements and available computational power. While recent research has made significant strides in developing pruning methods to build a sparse network for reducing the computing overhead of DNNs, there remains considerable accuracy loss, especially at high pruning ratios. We find that the architectures designed for dense networks by differentiable architecture search methods are ineffective when pruning mechanisms are applied to them. The main reason is that the current methods do not support sparse architectures in their search space and use a search objective that is made for dense networks and does not focus on sparsity.This paper proposes a new method to search for sparsity-friendly neural architectures. It is done by adding two new sparse operations to the search space and modifying the search objective. We propose two novel parametric SparseConv and SparseLinear operations in order to expand the search space to include sparse operations. In particular, these operations make a flexible search space due to using sparse parametric versions of linear and convolution operations. The proposed search objective lets us train the architecture based on the sparsity of the search space operations. Quantitative analyses demonstrate that architectures found through DASS outperform those used in the state-of-the-art sparse networks on the CIFAR-10 and ImageNet datasets. In terms of performance and hardware effectiveness, DASS increases the accuracy of the sparse version of MobileNet-v2 from 73.44% to 81.35% (+7.91% improvement) with a 3.87× faster inference time.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The deployment of Deep Neural Networks (DNNs) on edge devices is hindered by the substantial gap between performance requirements and available computational power. While recent research has made significant strides in developing pruning methods to build a sparse network for reducing the computing overhead of DNNs, there remains considerable accuracy loss, especially at high pruning ratios. We find that the architectures designed for dense networks by differentiable architecture search methods are ineffective when pruning mechanisms are applied to them. The main reason is that the current methods do not support sparse architectures in their search space and use a search objective that is made for dense networks and does not focus on sparsity.This paper proposes a new method to search for sparsity-friendly neural architectures. It is done by adding two new sparse operations to the search space and modifying the search objective. We propose two novel parametric SparseConv and SparseLinear operations in order to expand the search space to include sparse operations. In particular, these operations make a flexible search space due to using sparse parametric versions of linear and convolution operations. The proposed search objective lets us train the architecture based on the sparsity of the search space operations. Quantitative analyses demonstrate that architectures found through DASS outperform those used in the state-of-the-art sparse networks on the CIFAR-10 and ImageNet datasets. In terms of performance and hardware effectiveness, DASS increases the accuracy of the sparse version of MobileNet-v2 from 73.44% to 81.35% (+7.91% improvement) with a 3.87× faster inference time.
3034.
Gao, Jianliang; Oloulade, Babatounde Moctard; Al-Sabri, Raeed; Chen, Jiamin; Lyu, Tengfei; zhenpeng Wu,
Graph neural architecture prediction Journal Article
In: Knowledge and Information Systems , 2023.
@article{nokey,
title = {Graph neural architecture prediction},
author = {Jianliang Gao and Babatounde Moctard Oloulade and Raeed Al-Sabri and Jiamin Chen and Tengfei Lyu and zhenpeng Wu
},
url = {https://link.springer.com/article/10.1007/s10115-023-01968-6},
year = {2023},
date = {2023-08-31},
urldate = {2023-08-31},
journal = {Knowledge and Information Systems },
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3033.
Huang, Lin; Deng, Weiming; Li, Chunchun; Yang, Tiejun
Object detection for blind inspection of industrial products based on neural architecture search Journal Article
In: 2023.
@article{Huang-jim23a,
title = {Object detection for blind inspection of industrial products based on neural architecture search},
author = {
Lin Huang and Weiming Deng and Chunchun Li and Tiejun Yang
},
url = {https://link.springer.com/article/10.1007/s10845-023-02199-w},
year = {2023},
date = {2023-08-29},
urldate = {2023-08-29},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
3032.
Barbudo, Rafael; Ventura, Sebastián; Romero, José Raúl
Eight years of AutoML: categorisation, review and trends Journal Article
In: Knowledge and Information Systems, 2023.
@article{Barbudo-kis23a,
title = {Eight years of AutoML: categorisation, review and trends},
author = {
Rafael Barbudo and Sebastián Ventura and José Raúl Romero
},
url = {https://link.springer.com/article/10.1007/s10115-023-01935-1},
year = {2023},
date = {2023-08-08},
urldate = {2023-08-08},
journal = {Knowledge and Information Systems},
keywords = {},
pubstate = {published},
tppubtype = {article}
}