Multi-objective Machine Learning Hyper-parameter Optimization: A Case Study in Dam Hazard Classification

Machine learning methods often require tuning hyperparameters to optimize their performance. Often these hyperparameters are selected based on prior assumptions or a single-objective optimization. However, these techniques fail to capture tradeoffs between type I (i.e., false positive) and type II (i.e., false negative) misclassifications. This presentation advances the process of hyperparameter optimization by analysing tradeoffs among multiple classification objectives. Our approach starts with feature selection, coupled with a machine learning classification model. We employ the BORG multiobjective evolutionary algorithm to explore different values of hyperparameters and identify tradeoffs among objectives describing misclassifications, including area under the receiver operating characteristic curve, precision, and accuracy. Such an approach is broadly applicable to environmental applications where type I and type II errors have differing consequences, empowering analysts to make informed choices of hyperparameter values when applying machine learning algorithms to real-world situations. The approach is demonstrated on the novel classification problem of dams deemed to have a high or not-high hazard potential (HP). A machine learning algorithm “learns” to classify existing dam hazard classifications based on features such as dam height, length, reservoir size, and downstream population. This is a problem where type I and type II errors could have dire implications because a dam with a high HP means that failure or misoperation would cause probable loss of human life. In this research, we develop a data-driven dam HP classification model, demonstrating its feasibility with National Inventory of Dams entries in the northeastern United States.