Comparative Analysis of YOLOv12 and DepthAnythingV2 Inference Performance Using PyTorch, ONNX Runtime, and NCNN

DOI

10.2991/978-94-6239-828-3_7How to use a DOI?

Keywords

Inference frameworks; CPU deployment; ONNX Runtime; NCNN YOLOv12; DepthAnythingV2; Monocular depth estimation; Benchmarking

Abstract

CPU deployment requires inference frameworks that balance latency, throughput, memory, and accuracy. This paper presents a controlled comparison of PyTorch, ONNX Runtime, and NCNN on two representative workloads: real- time object detection with YOLOv12n and monocular depth estimation with DepthAnythingV2 (ViT-S). Unlike prior comparisons, we explicitly fix the number of CPU threads and pin execution to identical physical cores across all frameworks, removing a thread-allocation confound that otherwise inflates apparent speed differences. All latency figures are reported as mean standard deviation over 100 iterations after 20 warm-up runs. ONNX Runtime achieves the lowest latency on both tasks (e.g., 20.4 ms versus 31.0 ms for PyTorch and 46.6 ms for NCNN on detection at four threads). Object-detection accuracy is identical across all three frameworks, and ONNX Runtime reproduces PyTorch depth accuracy exactly; however, NCNN shows a substantial accuracy drop on the ViT-based depth model (𝛿₁ falling from 95.5% to 47.0%), which we trace to model conversion rather than measurement. A thread-scaling analysis further shows that unconstrained multi- threading causes CPU oversubscription with little or no latency benefit. These findings give practical guidance for CPU deployment and expose framework- specific limitations for transformer architectures.

Copyright

© 2026 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - Nada Nadhira Najwa Mazaya
AU  - Fitri Utaminingrum
AU  - Barlian Henryranu Prasetio
PY  - 2026
DA  - 2026/11/04
TI  - Comparative Analysis of YOLOv12 and DepthAnythingV2 Inference Performance Using PyTorch, ONNX Runtime, and NCNN
BT  - Proceedings of the 2026 11th International Conference on Sustainable Information Engineering and Technology (SIET 2026)
PB  - Atlantis Press
SP  - 67
EP  - 76
SN  - 2589-4900
UR  - https://doi.org/10.2991/978-94-6239-828-3_7
DO  - 10.2991/978-94-6239-828-3_7
ID  - Mazaya2026
ER  -