AutoML | Literature on Neural Architecture Search

In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 279–288, Association for Computing Machinery, Virtual Event, Singapore, 2021, ISBN: 9781450383325.

Abstract | Links | BibTeX

@inproceedings{10.1145/3447548.3467447,

title = {DiffMG: Differentiable Meta Graph Search for Heterogeneous Graph Neural Networks},

author = {Yuhui Ding and Quanming Yao and Huan Zhao and Tong Zhang},

url = {https://doi.org/10.1145/3447548.3467447},

doi = {10.1145/3447548.3467447},

isbn = {9781450383325},

year  = {2021},

date = {2021-01-01},

urldate = {2021-01-01},

booktitle = {Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining},

pages = {279–288},

publisher = {Association for Computing Machinery},

address = {Virtual Event, Singapore},

series = {KDD '21},

abstract = {In this paper, we propose a novel framework to automatically utilize task-dependent 

semantic information which is encoded in heterogeneous information networks (HINs). 

Specifically, we search for a meta graph, which can capture more complex semantic 

relations than a meta path, to determine how graph neural networks (GNNs) propagate 

messages along different types of edges. We formalize the problem within the framework 

of neural architecture search (NAS) and then perform the search in a differentiable 

manner. We design an expressive search space in the form of a directed acyclic graph 

(DAG) to represent candidate meta graphs for a HIN, and we propose task-dependent 

type constraint to filter out those edge types along which message passing has no 

effect on the representations of nodes that are related to the downstream task. The 

size of the search space we define is huge, so we further propose a novel and efficient 

search algorithm to make the total search cost on a par with training a single GNN 

once. Compared with existing popular NAS algorithms, our proposed search algorithm 

improves the search efficiency. We conduct extensive experiments on different HINs 

and downstream tasks to evaluate our method, and experimental results show that our 

method can outperform state-of-the-art heterogeneous GNNs and also improves efficiency 

compared with those methods which can implicitly learn meta paths.},

keywords = {},

pubstate = {published},

tppubtype = {inproceedings}

}

Close

1272.

Li, Jihao; Weinmann, Martin; Sun, Xian; Diao, Wenhui; Feng, Yingchao; Fu, Kun

Random Topology and Random Multiscale Mapping: An Automated Design of Multiscale and Lightweight Neural Network for Remote-Sensing Image Recognition Journal Article

In: IEEE Transactions on Geoscience and Remote Sensing, pp. 1-17, 2021.

Links | BibTeX

1271.

Wang, Xin; Zhu, Wenwu

Automated Machine Learning on Graph Proceedings Article

In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 4082–4083, Association for Computing Machinery, Virtual Event, Singapore, 2021, ISBN: 9781450383325.

Abstract | Links | BibTeX

1270.

Geiping, Jonas; Lukasik, Jovita; Keuper, Margret; Moeller, Michael

DARTS for Inverse Problems: a Study on Hyperparameter Sensitivity Journal Article

In: CoRR, vol. abs/2108.05647, 2021.

Links | BibTeX

1269.

Razali, Mohd Norhisham; Moung, Ervin Gubin; Yahya, Farashazillah; Hou, Chong Joon; Hanapi, Rozita; Mohamed, Raihani; Hashem, Ibrahim Abakr Targio

Indigenous Food Recognition Model Based on Various Convolutional Neural Network Architectures for Gastronomic Tourism Business Analytics Journal Article

In: Inf., vol. 12, no. 8, pp. 322, 2021.

Links | BibTeX

1268.

Chen, Boyu; Li, Peixia; Li, Baopu; Lin, Chen; Li, Chuming; Sun, Ming; Yan, Junjie; Ouyang, Wanli

BN-NAS: Neural Architecture Search with Batch Normalization Technical Report

2021.

Links | BibTeX

1267.

Benmeziane, Hadjer; Maghraoui, Kaoutar El; Ouarnoughi, Hamza; Niar, Smaïl; Wistuba, Martin; Wang, Naigang

Hardware-Aware Neural Architecture Search: Survey and Taxonomy Proceedings Article

In: Zhou, Zhi-Hua (Ed.): Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI 2021, Virtual Event / Montreal, Canada, 19-27 August 2021, pp. 4322–4329, ijcai.org, 2021.

Links | BibTeX

1266.

Li, Yanxi; Yang, Zhaohui; Wang, Yunhe; Xu, Chang

Neural Architecture Dilation for Adversarial Robustness Technical Report

2021.

Links | BibTeX

1265.

Hosseini, Mahdi S.; Zhang, Jia Shu; Liu, Zhe; Fu, Andre; Su, Jingxuan; Tuli, Mathieu; Plataniotis, Konstantinos N.

CONet: Channel Optimization for Convolutional Neural Networks Technical Report

2021.

Links | BibTeX

1264.

Zhao, Changsheng; Hua, Ting; Shen, Yilin; Lou, Qian; Jin, Hongxia

Automatic Mixed-Precision Quantization Search of BERT Proceedings Article

In: Zhou, Zhi-Hua (Ed.): Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI 2021, Virtual Event / Montreal, Canada, 19-27 August 2021, pp. 3427–3433, ijcai.org, 2021.

Links | BibTeX

1263.

Tang, Sheyang; Hosseini, Mahdi S.; Chen, Lina; Varma, Sonal; Rowsell, Corwyn; Damaskinos, Savvas; Plataniotis, Konstantinos N.; Wang, Zhou

Probeable DARTS with Application to Computational Pathology Technical Report

2021.

Links | BibTeX

1262.

Zhao, Pengyu; Xiao, Kecheng; Zhang, Yuanxing; Bian, Kaigui; Yan, Wei

AMEIR: Automatic Behavior Modeling, Interaction Exploration and MLP Investigation in the Recommender System Proceedings Article

In: Zhou, Zhi-Hua (Ed.): Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI 2021, Virtual Event / Montreal, Canada, 19-27 August 2021, pp. 2104–2110, ijcai.org, 2021.

Links | BibTeX

1261.

Lyu, Bo; Yang, Yin; Wen, Shiping; Huang, Tingwen; Li, Ke

Neural Architecture Search for Portrait Parsing Journal Article

In: IEEE Transactions on Neural Networks and Learning Systems, pp. 1-10, 2021.

Links | BibTeX

1260.

Liu, Chia-Hsiang; Han, Yu-Shin; Sung, Yuan-Yao; Lee, Yi; Chiang, Hung-Yueh; Wu, Kai-Chiang

FOX-NAS: Fast, On-device and Explainable Neural Architecture Search Technical Report

2021.

Links | BibTeX

1259.

Kim, Youngkee; Yun, Won Joon; Lee, Youn Kyu; Jung, Soyi; Kim, Joongheon

Trends in Neural Architecture Search: Towards the Acceleration of Search Technical Report

2021.

Links | BibTeX

1258.

Ganepola, Vayangi Vishmi Vishara; Wirasingha, Torin

Generative Adversarial Networks Using Neural Architecture Search for Semantic Image Segmentation Proceedings Article

In: 2021 IEEE 4th International Conference on Big Data and Artificial Intelligence (BDAI), pp. 236-241, 2021.

Links | BibTeX

1257.

Wang, Ruochen; Chen, Xiangning; Cheng, Minhao; Tang, Xiaocheng; Hsieh, Cho-Jui

RANK-NOSH: Efficient Predictor-Based Architecture Search via Non-Uniform Successive Halving Proceedings Article

In: ICCV 2021, 2021.

Links | BibTeX

1256.

Hou, Pengfei; Jin, Ying; Chen, Yukang

Single-DARTS: Towards Stable Architecture Search Technical Report

2021.

Links | BibTeX

1255.

Fan, Yi; Zhou, Quoqiang; Zhang, Weifeng; Bao, Shudi; Shen, Jun

Determining learning direction via multi-controller model for stably searching generative adversarial networks Journal Article

In: Neurocomputing, vol. 464, pp. 37-47, 2021, ISSN: 0925-2312.

Abstract | Links | BibTeX

1254.

Peng, Jiefeng; Zhang, Jiqi; Li, Changlin; Wang, Guangrun; Liang, Xiaodan; Lin, Liang

Pi-NAS: Improving Neural Architecture Search by Reducing Supernet Training Consistency Shift Technical Report

2021.

Links | BibTeX

1253.

Calisto, Maria Baldeon; Lai-Yuen, Susana K.

EMONAS-Net: Efficient multiobjective neural architecture search using surrogate-assisted evolutionary algorithm for 3D medical image segmentation Journal Article

In: Artificial Intelligence in Medicine, vol. 119, pp. 102154, 2021, ISSN: 0933-3657.

Abstract | Links | BibTeX

@article{BALDEONCALISTO2021102154,

title = {EMONAS-Net: Efficient multiobjective neural architecture search using surrogate-assisted evolutionary algorithm for 3D medical image segmentation},

author = {Maria Baldeon Calisto and Susana K. Lai-Yuen},

url = {https://www.sciencedirect.com/science/article/pii/S0933365721001470},

doi = {https://doi.org/10.1016/j.artmed.2021.102154},

issn = {0933-3657},

year  = {2021},

date = {2021-01-01},

urldate = {2021-01-01},

journal = {Artificial Intelligence in Medicine},

volume = {119},

pages = {102154},

abstract = {Deep learning plays a critical role in medical image segmentation. Nevertheless, manually designing a neural network for a specific segmentation problem is a very difficult and time-consuming task due to the massive hyperparameter search space, long training time and large volumetric data. Therefore, most designed networks are highly complex, task specific and over-parametrized. Recently, multiobjective neural architecture search (NAS) methods have been proposed to automate the design of accurate and efficient segmentation architectures. However, they only search for either the micro- or macro-structure of the architecture, do not use the information produced during the optimization process to increase the efficiency of the search, or do not consider the volumetric nature of medical images. In this work, we present EMONAS-Net, an Efficient MultiObjective NAS framework for 3D medical image segmentation that optimizes both the segmentation accuracy and size of the network. EMONAS-Net has two key components, a novel search space that considers the configuration of the micro- and macro-structure of the architecture and a Surrogate-assisted Multiobjective Evolutionary based Algorithm (SaMEA algorithm) that efficiently searches for the best hyperparameter values. The SaMEA algorithm uses the information collected during the initial generations of the evolutionary process to identify the most promising subproblems and select the best performing hyperparameter values during mutation to improve the convergence speed. Furthermore, a Random Forest surrogate model is incorporated to accelerate the fitness evaluation of the candidate architectures. EMONAS-Net is tested on the tasks of prostate segmentation from the MICCAI PROMISE12 challenge, hippocampus segmentation from the Medical Segmentation Decathlon challenge, and cardiac segmentation from the MICCAI ACDC challenge. In all the benchmarks, the proposed framework finds architectures that perform better or comparable with competing state-of-the-art NAS methods while being considerably smaller and reducing the architecture search time by more than 50%.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

Close

Deep learning plays a critical role in medical image segmentation. Nevertheless, manually designing a neural network for a specific segmentation problem is a very difficult and time-consuming task due to the massive hyperparameter search space, long training time and large volumetric data. Therefore, most designed networks are highly complex, task specific and over-parametrized. Recently, multiobjective neural architecture search (NAS) methods have been proposed to automate the design of accurate and efficient segmentation architectures. However, they only search for either the micro- or macro-structure of the architecture, do not use the information produced during the optimization process to increase the efficiency of the search, or do not consider the volumetric nature of medical images. In this work, we present EMONAS-Net, an Efficient MultiObjective NAS framework for 3D medical image segmentation that optimizes both the segmentation accuracy and size of the network. EMONAS-Net has two key components, a novel search space that considers the configuration of the micro- and macro-structure of the architecture and a Surrogate-assisted Multiobjective Evolutionary based Algorithm (SaMEA algorithm) that efficiently searches for the best hyperparameter values. The SaMEA algorithm uses the information collected during the initial generations of the evolutionary process to identify the most promising subproblems and select the best performing hyperparameter values during mutation to improve the convergence speed. Furthermore, a Random Forest surrogate model is incorporated to accelerate the fitness evaluation of the candidate architectures. EMONAS-Net is tested on the tasks of prostate segmentation from the MICCAI PROMISE12 challenge, hippocampus segmentation from the Medical Segmentation Decathlon challenge, and cardiac segmentation from the MICCAI ACDC challenge. In all the benchmarks, the proposed framework finds architectures that perform better or comparable with competing state-of-the-art NAS methods while being considerably smaller and reducing the architecture search time by more than 50%.

Close

1252.

Boutros, Fadi; Siebke, Patrick; Klemt, Marcel; Damer, Naser; Kirchbuchner, Florian; Kuijper, Arjan

PocketNet: Extreme Lightweight Face Recognition Network using Neural Architecture Search and Multi-Step Knowledge Distillation Technical Report

2021.

Links | BibTeX

1251.

Chitty-Venkata, Krishna Teja; Somani, Arun K.

Array-Aware Neural Architecture Search Proceedings Article

In: 2021 IEEE 32nd International Conference on Application-specific Systems, Architectures and Processors (ASAP), pp. 125-132, 2021.

Links | BibTeX

1250.

Zhan, Zheng; Gong, Yifan; Zhao, Pu; Yuan, Geng; Niu, Wei; Wu, Yushu; Zhang, Tianyun; Jayaweera, Malith; Kaeli, David R.; Ren, Bin; Lin, Xue; Wang, Yanzhi

Achieving on-Mobile Real-Time Super-Resolution with Neural Architecture and Pruning Search Technical Report

2021.

Links | BibTeX