ICPE 2026 Paper on Benchmarking Tracing Agents

David Georg Reichelt, Shinhyung Yang, Marcel Hanson, and Wilhelm Hasselbring. 2026. “Benchmarking the Overhead of Distributed Tracing Agents.” In Proceedings of the 17th ACM/SPEC International Conference on Performance Engineering (ICPE ‘26). 147-161. doi: https://doi.org/10.1145/3777884.3797004

Abstract

Tracing is a fundamental technique for analyzing the runtime behavior of software systems. By recording the start and end times of method executions together with contextual metadata, tracing enables detailed performance analysis, architecture reconstruction, and program comprehension. However, such instrumentation inevitably introduces runtime overhead that can distort performance measurements and increase variability. Quantifying and comparing this overhead across tracing frameworks and configurations is therefore essential for selecting suitable tools and ensuring reliable performance evaluations. The overhead of different tracing frameworks and their configuration can be measured by the MooBench microbenchmark. In this work, we extend the MooBench microbenchmark to support the established Java tracing frameworks Elastic APM Agent, inspectIT, Kieker, OpenTelemetry, Pinpoint, Scouter, and SkyWalking. By executing MooBench with these agents, we find (1) significant differences in performance overhead, whereby the industry standard implementation of OpenTelemetry is comparably slow, while the Kieker agent has the lowest overhead among the functionally correct frameworks, (2) the agents of Pinpoint and Scouter do not store all records, making their behavior not fulfill the functional requirements, and (3) that avoidable overhead of some of the frameworks is created by extensive metadata gathering and needless copying of data.

Preprint

Available at: doi: https://icpe2026.spec.org/preprint/Benchmarking_the_Overhead_of_Distributed_Tracing_Agents_2.pdf