posted on 2021-11-04, 07:55authored byLubomír Bulej, François Farquet, Vojtěch Horký, Michele Tucci, Petr TůmaPetr Tůma
For the past four years, we have used several public Java benchmarks (DaCapo, ScalaBench, Renaissance) to track the performance changes introduced by the daily development changes of the Graal compiler.
The talk will outline how we tackle common measurement issues such as measurement scheduling and change detection, summarize observed parameters of the performance changes themselves, and then discuss factors that impact the usefulness of such testing for the compiler development process, for example (1) what changes are useful to report (or not), (2) what changes are missed by the public benchmarks, or (3) what aspects of the compiler behavior make such testing difficult (or easy).