Effectiveness Evaluation of High-Frequency Trading Machine Learning Prediction Models under Transaction Cost Constraints

DOI

10.2991/978-94-6239-830-6_25How to use a DOI?

Keywords

High-frequency trading; Limit order book; Mid-price prediction; Machine learning; Transaction costs

Abstract

This study investigates the difference between the statistical effectiveness and economic effectiveness of short-term mid-price direction prediction models under transaction cost constraints, based on the preprocessed FI-2010 limit order book (LOB) dataset. Four three-class prediction models, including Logistic Regression, eXtreme Gradient Boosting (XGBoost), Long Short-Term Memory (LSTM), and Convolutional Neural Network-Long Short-Term Memory (CNN-LSTM), were evaluated. Their predictions were mapped to long, neutral, and short trading signals, followed by simplified backtesting under different fixed transaction cost coefficients. Within a representative prediction window, CNN-LSTM achieved the highest Accuracy, Macro-F1, and Weighted-F1 scores, at 0.799, 0.751, and 0.793, respectively. However, with a 20-tick prediction window and a transaction cost coefficient of 0.001, the cumulative net return proxy of LSTM is 6.090, higher than that of CNN-LSTM (-4.606). Further analysis revealed that with a 100-tick prediction window, the cumulative net return proxy of all models is negative, and the correlation coefficient between Macro-F1 and the cumulative net return proxy is only about 0.21. The results show that stronger classification performance does not necessarily correspond to greater economic effectiveness. High-frequency trading prediction models should therefore be evaluated jointly in terms of classification performance, cumulative net return proxy, and turnover.

Copyright

© 2026 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - Hongtao Chen
PY  - 2026
DA  - 2026/11/14
TI  - Effectiveness Evaluation of High-Frequency Trading Machine Learning Prediction Models under Transaction Cost Constraints
BT  - Proceedings of the 6th International Conference on Internet Finance and Digital Economy (ICIFDE 2026)
PB  - Atlantis Press
SP  - 301
EP  - 311
SN  - 2352-5428
UR  - https://doi.org/10.2991/978-94-6239-830-6_25
DO  - 10.2991/978-94-6239-830-6_25
ID  - Chen2026
ER  -