Machine Learning-Based Portfolio Optimization: A Comparative Analysis of Ensemble Learning Algorithms for Investment Decision Support in the Banking and Financial Sector

Abstract

Portfolio optimization remains one of the central challenges of quantitative investment management, requiring a balance between expected return, risk exposure, transaction costs, and regulatory constraints. Classical mean–variance approaches, while theoretically elegant, are known to be highly sensitive to estimation error in expected returns and covariance matrices, and they struggle to capture the non-linear, regime-dependent relationships that characterize modern financial markets. This paper presents a comparative analysis of ensemble machine learning algorithms  Random Forests, Gradient Boosting Machines (including XGBoost and LightGBM), AdaBoost, and Stacked Generalization  as return-and-risk forecasting engines embedded within a portfolio construction framework, with a specific focus on their applicability as investment decision support tools for banks and financial institutions. We describe a structured methodology encompassing feature engineering from market, fundamental, and macroeconomic data, model training and hyperparameter tuning, and integration of ensemble forecasts into a constrained mean-variance and risk-parity optimization layer. Using an illustrative backtesting framework over a multi-asset universe, we compare the algorithms on predictive accuracy, portfolio-level risk-adjusted performance, turnover, and computational cost, and discuss their interpretability and governance implications for regulated financial institutions. Results indicate that gradient boosting ensembles offer the strongest risk-adjusted performance and predictive accuracy, Random Forests provide a favorable balance of robustness and interpretability, and stacked ensembles yield modest incremental gains at increased computational and governance complexity. We conclude with practical recommendations for banks seeking to operationalize ensemble learning within investment decision support systems, along with a discussion of limitations and directions for future research.