SMAC

SMAC (sequential model-based algorithm configuration) is a versatile tool for optimizing algorithm parameters (or the parameters of some other process we can run automatically, or a function we can evaluate, such as a simulation).

SMAC has helped us speed up both local search and tree search algorithms by orders of magnitude on certain instance distributions. Recently, we have also found it to be very effective for the hyperparameter optimization of machine learning algorithms, scaling better to high dimensions and discrete input dimensions than other algorithms. Finally, the predictive models SMAC is based on can also capture and exploit important information about the model domain, such as which input variables are most important. We hope you find SMAC similarly useful. Ultimately, we hope that it  helps algorithm designers focus on tasks that are more scientifically valuable than parameter tuning. 

• SMAC in Java
• SMAC 3 in Python 
• DEPRECATED: SpySMAC (Extension of pysmac with evaluation module)
  • Instead, we recommend to use CAVE for analysing SMAC data.
• DEPRECATED: pySMAC (Python interface to Java SMAC )

References

• M. Lindauer and F. Hutter
  Warmstarting of Model-based Algorithm Configuration
  In: Proceedings of the AAAI’18
  Link to source code of warmstarted SMAC and log files of experiments
• Falkner, S. and Lindauer, M. and Hutter, F.
  SpySMAC: Automated Configuration and Performance Analysis of SAT Solvers
  In: Proceedings of the International Conference on Satisfiability Solving (SAT’15)
• Frank Hutter, Holger Hoos, and Kevin Leyton-Brown.
  An evaluation of sequential model-based optimization for expensive blackbox functions
  In GECCO 2013 Blackbox Optimization Benchmarking workshop (BBOB’13).
• Frank Hutter, Holger Hoos, and Kevin Leyton-Brown.
  Parallel Algorithm Configuration
  In: Learning and Intelligent Optimization (LION 6)
• Frank Hutter, Holger Hoos, and Kevin Leyton-Brown.
  Bayesian Optimization With Censored Response Data
  2011 NIPS workshop on Bayesian Optimization, Experimental Design, and Bandits.
• Frank Hutter, Holger Hoos, and Kevin Leyton-Brown.
  Sequential Model-Based Optimization for General Algorithm Configuration
  In LION-5, 2011.  Second best paper prize
• Frank Hutter, Holger Hoos, Kevin Leyton-Brown, and Kevin Murphy.
  Time-Bounded Sequential Parameter Optimization
  In LION4, 2010