AutoML | Literature on Neural Architecture Search

@article{LIN2021279,

title = {Semantic segmentation of 3D indoor LiDAR point clouds through feature pyramid architecture search},

author = {Haojia Lin and Shangbin Wu and Yiping Chen and Wen Li and Zhipeng Luo and Yulan Guo and Cheng Wang and Jonathan Li},

url = {https://www.sciencedirect.com/science/article/pii/S0924271621001349},

doi = {https://doi.org/10.1016/j.isprsjprs.2021.05.009},

issn = {0924-2716},

year  = {2021},

date = {2021-01-01},

journal = {ISPRS Journal of Photogrammetry and Remote Sensing},

volume = {177},

pages = {279-290},

abstract = {Semantic segmentation of 3D Light Detection and Ranging (LiDAR) indoor point clouds using deep learning has been an active topic in recent years. However, most deep neural networks on point clouds conduct multi-level feature fusion via a simple U-shape architecture, which lacks enough capacity on both classification and localization in the segmentation task. In this paper, we propose a Neural Architecture Search (NAS) method to search a Feature Pyramid Network (FPN) module for 3D indoor point cloud semantic segmentation. Specifically, we aim to automatically find an effective feature pyramid architecture as a feature fusion neck in a designed novel pyramidal search space covering all information communication paths for multi-level features. The searched FPN module, named SFPN, contains the most important connections among all the potential paths to fuse representations at different levels. Our proposed SFPN is generic and effective as well as capable to be added to existing segmentation networks to augment the segmentation performance. Extensive experiments on ScanNet and S3DIS show that consistent and remarkable gains of segmentation performance can be achieved by different classical networks combined with SFPN. Specially, PointNet++-SFPN achieves mIoU gains of 7.8% on ScanNet v2 and 4.7% on S3DIS, and PointConv-SFPN achieves 4.5% and 3.7% improvement respectively on the above datasets.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

Close

1374.

Zhao, Yuekai; Dong, Li; Shen, Yelong; Zhang, Zhihua; Wei, Furu; Chen, Weizhu

Memory-Efficient Differentiable Transformer Architecture Search Technical Report

2021.

Links | BibTeX

1373.

Nader, Andrew; Azar, Danielle

Evolution of Activation Functions: An Empirical Investigation Technical Report

2021.

Links | BibTeX

1372.

Pan, Zijie; Zeng, Jiajin; Cheng, Riqiang; Yan, Hongyang; Li, Jin

PNAS: A privacy preserving framework for neural architecture search services Journal Article

In: Information Sciences, vol. 573, pp. 370-381, 2021, ISSN: 0020-0255.

Abstract | Links | BibTeX

1371.

Bungert, Leon; Roith, Tim; Tenbrinck, Daniel; Burger, Martin

Neural Architecture Search via Bregman Iterations Technical Report

2021.

Links | BibTeX

1370.

Mo, Tong; Liu, Bang

Encoder-Decoder Neural Architecture Optimization for Keyword Spotting Technical Report

2021.

Links | BibTeX

1369.

Miao, Yingjie; Song, Xingyou; Peng, Daiyi; Yue, Summer; Brevdo, Eugene; Faust, Aleksandra

RL-DARTS: Differentiable Architecture Search for Reinforcement Learning Technical Report

2021.

Links | BibTeX

1368.

Saito, Masahiko; Kishimoto, Tomoe; Kaneta, Yuya; Itoh, Taichi; Umeda, Yoshiaki; Tanaka, Junichi; Iiyama, Yutaro; Sawada, Ryu; Terashi, Koji

Event Classification with Multi-step Machine Learning Technical Report

2021.

Links | BibTeX

1367.

Guo, Xin; Yang, Jianlei; Zhou, Haoyi; Ye, Xucheng; Li, Jianxin

RoSearch: Search for Robust Student Architectures When Distilling Pre-trained Language Models Technical Report

2021.

Links | BibTeX

1366.

Dey, Debadeepta; Shah, Shital; é, S

FEAR: A Simple Lightweight Method to Rank Architectures Technical Report

2021.

Links | BibTeX

1365.

Na, Byunggook; Mok, Jisoo; Choe, Hyeokjun; Yoon, Sungroh

Accelerating Neural Architecture Search via Proxy Data Technical Report

2021.

Links | BibTeX

1364.

Yao, Yuansheng; Liu, Risheng; Zhang, Jiaao; Zhong, Wei; Fan, Xin; Luo, Zhongxuan

Hardware-Aware Low-Light Image Enhancement via One-Shot Neural Architecture Search with Shrinkage Sampling Proceedings Article

In: 2021 IEEE International Conference on Multimedia and Expo (ICME), pp. 1-6, 2021.

Links | BibTeX

1363.

Mao, Yuxu; Zhong, Guoqiang; Wang, Yanan; Deng, Zhaoyang

Differentiable Light-Weight Architecture Search Proceedings Article

In: 2021 IEEE International Conference on Multimedia and Expo (ICME), pp. 1-6, 2021.

Links | BibTeX

1362.

Yang, Binbin; Liang, Xiaodan; Zhong, Junhao; Peng, Jiefeng; Wang, Guangrun; Lin, Liang

Unifying Dynamic Optimizer Search and Network Architecture Search Proceedings Article

In: 2021 IEEE International Conference on Multimedia and Expo (ICME), pp. 1-6, 2021.

Links | BibTeX

1361.

Salinas, David; Perrone, Valerio; Cruchant, Olivier; é, C

A multi-objective perspective on jointly tuning hardware and hyperparameters Technical Report

2021.

Links | BibTeX

1360.

Xue, Xinwei; Meng, Xiangyu; Ma, Long; Wang, Yi; Liu, Risheng; Fan, Xin

Searching Frame-Recurrent Attentive Deformable Network for Real-Time Video Deraining Technical Report

2021.

Links | BibTeX

1359.

Eltotongy, Assem; Awad, Mohammed I; Maged, Shady A; Onsy, Ahmed

Fault Detection and Classification of Machinery Bearing Under Variable Operating Conditions Based on Wavelet Transform and CNN Proceedings Article

In: 2021 International Mobile, Intelligent, and Ubiquitous Computing Conference (MIUCC), pp. 117-123, 2021.

Links | BibTeX

1358.

Vargas-Hákim, Gustavo A; Mezura-Montes, Efrén; Acosta-Mesa, Héctor-Gabriel

A Review on Convolutional Neural Networks Encodings for Neuroevolution Journal Article

In: IEEE Transactions on Evolutionary Computation, pp. 1-1, 2021.

Links | BibTeX

1357.

Gu, Hongyang; Fu, Guangyuan; Wang, Xu; Zhu, Jun

Learning auto-scale representations for person re-identification Journal Article

In: Image and Vision Computing, pp. 104241, 2021, ISSN: 0262-8856.

Abstract | Links | BibTeX

@article{GU2021104241,

title = {Learning auto-scale representations for person re-identification},

author = {Hongyang Gu and Guangyuan Fu and Xu Wang and Jun Zhu},

url = {https://www.sciencedirect.com/science/article/pii/S0262885621001463},

doi = {https://doi.org/10.1016/j.imavis.2021.104241},

issn = {0262-8856},

year  = {2021},

date = {2021-01-01},

journal = {Image and Vision Computing},

pages = {104241},

abstract = {Person re-identification (ReID) is a hot topic in computer vision. The data in the ReID is often collected from the cameras with different views and is affected by other environmental factors, which poses a significant challenge to ReID. The omni-scales proposed by OSNet can extract discriminative feature representations, which shows that omni-scales are adequate for the task of the ReID. However, the OSNet is mainly based on a manually designed network architecture. In the OSnet, each block uses the same architecture and has only four scale feature representations. Inspired by neural network architecture search (NAS), we propose a method of auto-scale representations for ReID. Specifically, we first design the auto-scale block, mainly composed of the Lite 3 × 3 operations and the RCB 1 × 1 operations. The connection status among the Lite 3 × 3 operations is just our search space. Then we give our entire macro network architecture, the auto-scale network, which is mainly composed of 6 auto-scale blocks. Unlike other NAS-related work, each block in our search space does not need to share the same architecture but can maintain a different architecture. In the search process, we propose the entropy regularization, the validity regularization and the consistent regularization to alleviate the discretized gap, no valid path, and meaningless edges, respectively. Finally, we verify the effectiveness of the model we searched on four commonly used datasets. Our model can maintain the same 2.2 M parameters as OSNet but can achieve the performance of SOTA. The mAP on the Market1501 dataset can reach 88.7%.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

Close

1356.

Yang, Xiaojing; Guo, Min; Lyu, Qiongshuai; Ma, Miao

Detection and classification of damaged wheat kernels based on progressive neural architecture search Journal Article

In: Biosystems Engineering, vol. 208, pp. 176-185, 2021, ISSN: 1537-5110.

Abstract | Links | BibTeX

1355.

Hu, Yufei; Belkhir, Nacim; ú, Jes; Yao, Angela; Franchi, Gianni

Learning Deep Morphological Networks with Neural Architecture Search Technical Report

2021.

Links | BibTeX

1354.

Jie, Renlong; Gao, Junbin

Differentiable Neural Architecture Search with Morphism-based Transformable Backbone Architectures Technical Report

2021.

Links | BibTeX

1353.

Xiang, Lichuan; Dudziak, Lukasz; Abdelfattah, Mohamed S; Chau, Thomas C P; Lane, Nicholas D; Wen, Hongkai

Zero-Cost Proxies Meet Differentiable Architecture Search Technical Report

2021.

Links | BibTeX

1352.

Ding, Mingyu; Lian, Xiaochen; Yang, Linjie; Wang, Peng; Jin, Xiaojie; Lu, Zhiwu; Luo, Ping

HR-NAS: Searching Efficient High-Resolution Neural Architectures with Lightweight Transformers Proceedings Article

In: CVPR2021, 2021.

Links | BibTeX

1351.

Chen, Chuangtao; He, Zhimin; Li, Lvzhou; Zheng, Shenggen; Situ, Haozhen

Quantum Architecture Search with Meta-learning Technical Report

2021.

Links | BibTeX

1350.

Fu, Yonggan; Zhang, Yongan; Zhang, Yang; Cox, David; Lin, Yingyan

Auto-NBA: Efficient and Effective Search Over the Joint Space of Networks, Bitwidths, and Accelerators Technical Report

2021.

Links | BibTeX

1349.

Akhauri, Yash; Niranjan, Adithya; ñ, Pablo Mu J; Banerjee, Suvadeep; Davare, Abhijit; Cocchini, Pasquale; Sorokin, Anton A; Iyer, Ravi; Jain, Nilesh

RHNAS: Realizable Hardware and Neural Architecture Search Technical Report

2021.

Links | BibTeX

1348.

Garciarena, Unai; Santana, Roberto; Mendiburu, Alexander

Redefining Neural Architecture Search of Heterogeneous Multi-Network Models by Characterizing Variation Operators and Model Components Technical Report

2021.

Links | BibTeX

1347.

Li, Hanjun; Wu, Gaojie; -, Wei

Combined Depth Space based Architecture Search For Person Re-identification Book Section

In: CVPR2021, vol. abs/2104.04163, 2021.

Links | BibTeX

1346.

Lee, Hayeon; Lee, Sewoong; Chong, Song; Hwang, Sung Ju

HELP: Hardware-Adaptive Efficient Latency Predictor for NAS via Meta-Learning Technical Report

2021.

Links | BibTeX

1345.

Chen, Weiwei; Wang, Ying; Lin, Gangliang; Gao, Chengsi; Liu, Cheng; Zhang, Lei

CHaNAS: Coordinated Search for Network Architecture and Scheduling Policy Proceedings Article

In: Proceedings of the 22nd ACM SIGPLAN/SIGBED International Conference on Languages, Compilers, and Tools for Embedded Systems, pp. 42–53, Association for Computing Machinery, Virtual, Canada, 2021, ISBN: 9781450384728.

Abstract | Links | BibTeX

@inproceedings{10.1145/3461648.3463846,

title = {CHaNAS: Coordinated Search for Network Architecture and Scheduling Policy},

author = {Weiwei Chen and Ying Wang and Gangliang Lin and Chengsi Gao and Cheng Liu and Lei Zhang},

url = {https://doi.org/10.1145/3461648.3463846},

doi = {10.1145/3461648.3463846},

isbn = {9781450384728},

year  = {2021},

date = {2021-01-01},

booktitle = {Proceedings of the 22nd ACM SIGPLAN/SIGBED International Conference on Languages, Compilers, and Tools for Embedded Systems},

pages = {42–53},

publisher = {Association for Computing Machinery},

address = {Virtual, Canada},

series = {LCTES 2021},

abstract = {Automatically design an efficient DNN solution for a given deep learning task on the 

target hardware mainly decided by the neural network architecture and the schedule 

mapping strategy, where the two goals are closely coupled with each other to fully 

exploit the advantages of the underlying hardware. Prior hardware-aware Neural Architecture 

Search (NAS) methods mostly ignore the impacts of different scheduling policies (e.g., 

graph-level optimization, loop transformations, parallelization, etc.) on network 

candidates being evaluated in the search process. Thus, they may miss the true-optimal 

architecture that can only be discovered by trying-out different scheduling policies. 

This work proposes a NAS framework (CHaNAS) that searches for not only the network 

architecture but also the dedicated scheduling policy, as the optimal co-design solution 

on target hardware that fully exploits the advantages of the underlying hardware. 

We propose to use a block-based pre-scheduling methodology to reduce the co-design 

search space, and enable the automatic generation of the optimal co-design, including 

the network architecture and the tensor programs that practice the scheduling policy. 

We evaluate CHaNAS on Imagenet on different hardware back-ends against the state-of-the-art 

hardware-aware search method MobileNet-v3. Experimental results show that the co-design 

solutions obtained by ChaNAS show up to 1.6x, 1.9x, and 1.7x performance boost on 

NVIDIA P100 GPU, Intel Xeon 8163 CPU, and Samsung Note 10 Mobile, respectively, over 

the baselines of the same-level accuracy.},

keywords = {},

pubstate = {published},

tppubtype = {inproceedings}

}

Close

1344.

Manu, Daniel; Huang, Shaoyi; Ding, Caiwen; Yang, Lei

Co-Exploration of Graph Neural Network and Network-on-Chip Design Using AutoML Proceedings Article

In: Proceedings of the 2021 on Great Lakes Symposium on VLSI, pp. 175–180, Association for Computing Machinery, Virtual Event, USA, 2021, ISBN: 9781450383936.

Abstract | Links | BibTeX

@inproceedings{10.1145/3453688.3461741,

title = {Co-Exploration of Graph Neural Network and Network-on-Chip Design Using AutoML},

author = {Daniel Manu and Shaoyi Huang and Caiwen Ding and Lei Yang},

url = {https://doi.org/10.1145/3453688.3461741},

doi = {10.1145/3453688.3461741},

isbn = {9781450383936},

year  = {2021},

date = {2021-01-01},

booktitle = {Proceedings of the 2021 on Great Lakes Symposium on VLSI},

pages = {175–180},

publisher = {Association for Computing Machinery},

address = {Virtual Event, USA},

series = {GLSVLSI '21},

abstract = {Recently, Graph Neural Networks (GNNs) have exhibited high efficiency in several graph-based 

machine learning tasks. Compared with the neural networks for computer vision or speech 

tasks (e.g., Convolutional Neural Networks), GNNs have much higher requirements on 

communication due to the complicated graph structures; however, when applying GNNs 

for real-world applications, say in recommender systems (e.g. Uber Eats), it commonly 

has the real-time requirements. To deal with the tradeoff between the complicated 

architecture and the high-demand timing performance, both GNN architecture and hardware 

accelerator need to be optimized. Network-on-Chip (NoC), derived for efficiently managing 

the high-volume of communications, naturally becomes one of the top candidates to 

accelerate GNNs. However, there is a missing link between the optimize of GNN architecture 

and the NoC design.In this work, we present an AutoML-based framework GN-NAS, aiming 

at searching for the optimum GNN architecture, which can be suitable for the NoC accelerator. 

We devise a robust reinforcement learning based controller to validate the retained 

best GNN architectures, coupled with a parameter sharing approach, namely ParamShare, 

to improve search efficiency. Experimental results on four graph-based benchmark datasets, 

Cora, Citeseer, Pubmed and Protein-Protein Interaction show that the GNN architectures 

obtained by our framework outperform that of the state-of-the-art and baseline models, 

whilst reducing model size which makes them easy to deploy onto the NoC platform.},

keywords = {},

pubstate = {published},

tppubtype = {inproceedings}

}

Close

1343.

Li, Yuening; Chen, Zhengzhang; Zha, Daochen; Zhou, Kaixiong; Jin, Haifeng; Chen, Haifeng; Hu, Xia

AutoOD: Neural Architecture Search for Outlier Detection Proceedings Article

In: 2021 IEEE 37th International Conference on Data Engineering (ICDE), pp. 2117-2122, 2021.

Links | BibTeX

1342.

Azarmehr, Neda; Ye, Xujiong; Howard, James P; Lane, Elisabeth S; Labs, Robert; Shun-Shin, Matthew J; Cole, Graham D; Bidaut, Luc; Francis, Darrel P; Zolgharni, Massoud

Neural architecture search of echocardiography view classifiers Journal Article

In: Journal of Medical Imaging, vol. 8, no. 3, pp. 1 – 21, 2021.

Links | BibTeX

1341.

Maile, Kaitlin; Lecarpentier, Erwan; é, Herv; Wilson, Dennis G

On Constrained Optimization in Differentiable Neural Architecture Search Technical Report

2021.

Links | BibTeX

1340.

Santra, Santanu; Hsieh, Jun-Wei; Lin, Chi-Fang

Gradient Descent Effects on Differential Neural Architecture Search: A Survey Journal Article

In: IEEE Access, vol. 9, pp. 89602-89618, 2021.

Links | BibTeX

1339.

Zeinali, Behnam; Zhuang, Di; Chang, Morris J

ESAI: Efficient Split Artificial Intelligence via Early Exiting Using Neural Architecture Search Technical Report

2021.

Links | BibTeX

1338.

Eriksson, David; -, Pierce I; Daulton, Samuel; Xia, Peng; Shrivastava, Akshat; Babu, Arun; Zhao, Shicong; Aly, Ahmed; Venkatesh, Ganesh; Balandat, Maximilian

Latency-Aware Neural Architecture Search with Multi-Objective Bayesian Optimization Proceedings Article

In: 8th ICML Workshop on Automated Machine Learning (2021), 2021.

Links | BibTeX