Search Results

Background. Compilers are actively developed programs, with newer versions implicitly promising better performance. Aim. In this paper, we evaluate whether the cost of the expected improvements in optimisations in newer versions of the GCC C compiler pays off, both in terms of execution time and energy consumption. Method. To do so, we select 10 major versions of GCC (spanning GCC-4 to GCC-13), and four different optimization flags (-Og, -O2, -O3, and -Os) to compile and run benchmarks written in C from the SPEC CPU 2017 benchmarking suite, measuring both the energy and time of compilation and running of the benchmarks. A linear mixed-effects model mapping the GCC major version number to a numerical variable is then fitted to the data. Result. We find a statistically significant trend of decreasing running costs when optimized with -O2 and -O3, with an overall decrease in costs (for both energy and time) amounting to 13% for -O2 and 9% for -O3 overall (from GCC-4 to GCC-13). However, this is not without a cost, as there is also a statistically significant trend of increasing compilation cost for the optimization levels tested except -Og, with an increase amounting to 17% in energy and 15% in time with -O3 enabled, and 18% in energy and 16% in time with -O2 enabled. Some benchmarks benefit more than others, which we hypothesize is due to more advanced vectorization in newer compiler versions. Conclusion. GCC is producing lower-cost programs, at the expense of higher compilation cost. Still, the trade-off is well worth it as a single additional execution of the compiled program offsets the increased cost of compilation for both the run-time and energy cost.