fANOVA is a tool for assessing the importance of an algorithm’s hyperparameters. It takes as input performance data gathered with different hyperparameter settings of the algorithm, fits a random forest to capture the relationship between hyperparameters and performance, and then applies functional ANOVA to assess how important each of the hyperparameters and each low-order interaction of hyperparameters is to performance.
Frank Hutter, Holger Hoos, and Kevin Leyton-Brown.
An Efficient Approach for Assessing Hyperparameter Importance [pdf] [pdf long version] [bib]
In: International Conference on Machine Learning (ICML’14).