Page 1 of 2
Link Time Optimizations
Posted: Fri Dec 12, 2025 9:22 am
by another_commander
Now that we have an upgraded build system for Windows, I am in the process of enabling LTO (link time optimizations) to the executables included with the installers. LTO offers a general improvement in performance of about 5-20% on average and is basically a ticket for better performance without any code changes. The tradeoff is noticeably longer link times and more complicated debugging, which is why I am thinking of enabling it only on installer builds. This way developers can continue using standard non-LTO builds which compile quickly and can be debugged without problem, while CI installers produced by github or final release binaries for which compile time is not a critical factor can get the optimization benefits for free.
But before I can enable this I need to be sure that the Linux builds can work with LTO too. Given that Linux builds have been using recent GCC versions, I would expect no issues but we have to be sure. So, any Linux user willing to assist, could you please add the -flto flag in your ADDITIONAL_CLFAGS, ADDITIONAL_OBJCFLAGS and ADDITIONAL_LDFLAGS in GNUmakefile (you may need to add the ADDITIONAL_LDLFAGS line manually) and try to build the game binary normally? You should be able to get an executable that works just like the original but be prepared to wait for a few minutes during the linking phase.
NOTE: Do not mix LTO with non-LTO object files, make sure you execute a make clean before starting the build just to be sure.
If all good we can go ahead, otherwise this will be a Windows feature only. Thanks in advance for anyone willing to provide feedback.
Re: Link Time Optimizations
Posted: Fri Dec 12, 2025 4:40 pm
by Lone_Wolf
GCC is being a bit silly again
Code: Select all
src/Core/NSUserDefaults+Override.m: In function ‘-[NSUserDefaults(Override) writeDictionary:toFile:]’:
src/Core/NSUserDefaults+Override.m:40:17: error: format not a string literal and no format arguments [-Werror=format-security]
40 | NSLog(@"Defaults database filename is empty when writing");
| ^
cc1obj: some warnings being treated as errors
Want me to file an issue ?
Re: Link Time Optimizations
Posted: Fri Dec 12, 2025 5:14 pm
by another_commander
No don't worry, just fixed it in master.
Re: Link Time Optimizations
Posted: Fri Dec 12, 2025 8:46 pm
by Commander_X
I gave it a try, the binary works[*].
Funny enough, it seems to be ignoring ESPEAK_DATA_PATH environment variable, so unless I'm doing a
Code: Select all
cd <whatever>/oolite.app; ./oolite
. it would bail complaining it cannot find "Resources/espeak-data/phonetab".
This happens both with -flto and without -flto flags, now -- seems to be a new thing?
Re: Link Time Optimizations
Posted: Fri Dec 12, 2025 8:56 pm
by Lone_Wolf
Either the modern build enables LTO or I've been building oolite with LTO without realising it for sometime already.
EDIT1: no problems running it
EDIT2 : the log is without adding -flto to GNUmakefile
Code: Select all
make -f GNUmakefile debug=no strip=yes
make[1]: Entering directory '/home/panoramix/Documents/Aur/pkgbuilds/oolite-git/src/oolite-git'
This is gnustep-make 2.9.3. Type 'make print-gnustep-make-help' for help.
Running in gnustep-make version 2 strict mode.
Compiling modern build
/usr/share/GNUstep/Makefiles/objc.make:30: objc.make is deprecated. Please use tool.make instead
cd .; \
/usr/share/GNUstep/Makefiles/mkinstalldirs ./obj.spk
Compiling modern build
/usr/share/GNUstep/Makefiles/objc.make:30: objc.make is deprecated. Please use tool.make instead
Making all for objc_program oolite...
Compiling modern build
/usr/share/GNUstep/Makefiles/objc.make:30: objc.make is deprecated. Please use tool.make instead
cd .; \
/usr/share/GNUstep/Makefiles/mkinstalldirs ./obj.spk/oolite.obj/
Compiling modern build
/usr/share/GNUstep/Makefiles/objc.make:30: objc.make is deprecated. Please use tool.make instead
gcc src/Core/legacy_random.c -c \
-MMD -MP -DGNUSTEP -DGNUSTEP_BASE_LIBRARY=1 -DGNU_RUNTIME=1 -DGNUSTEP_BASE_LIBRARY=1 -pthread -pthread -fPIC -Wall -DGSWARN -DGSDIAGNOSE -Wno-import -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=3 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -g -ffile-prefix-map=/build/gnustep-make/src=/usr/src/debug/gnustep-make -flto=auto -Wall -DLINUX -DNEED_STRLCPY `sdl-config --cflags` `nspr-config --cflags` -DOOLITE_MODERN_BUILD=1 -DOO_OXP_VERIFIER_ENABLED=1 -DOO_LOCALIZATION_TOOLS=1 -DDEBUG_GRAPHVIZ=1 -Ideps/Linux-deps/x86_64/mozilla/include -Isrc/SDL -Isrc/Core -Isrc/BSDCompat -Isrc/Core/Scripting -Isrc/Core/Materials -Isrc/Core/Entities -Isrc/Core/OXPVerifier -Isrc/Core/Debug -Isrc/Core/Tables -Isrc/Core/MiniZip -Ideps/Linux-deps/include -I. -I/home/panoramix/GNUstep/Library/Headers -I/usr/include \
-o obj.spk/oolite.obj/legacy_random.c.o
gcc src/BSDCompat/strlcpy.c -c \
-MMD -MP -DGNUSTEP -DGNUSTEP_BASE_LIBRARY=1 -DGNU_RUNTIME=1 -DGNUSTEP_BASE_LIBRARY=1 -pthread -pthread -fPIC -Wall -DGSWARN -DGSDIAGNOSE -Wno-import -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=3 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -g -ffile-prefix-map=/build/gnustep-make/src=/usr/src/debug/gnustep-make -flto=auto -Wall -DLINUX -DNEED_STRLCPY `sdl-config --cflags` `nspr-config --cflags` -DOOLITE_MODERN_BUILD=1 -DOO_OXP_VERIFIER_ENABLED=1 -DOO_LOCALIZATION_TOOLS=1 -DDEBUG_GRAPHVIZ=1 -Ideps/Linux-deps/x86_64/mozilla/include -Isrc/SDL -Isrc/Core -Isrc/BSDCompat -Isrc/Core/Scripting -Isrc/Core/Materials -Isrc/Core/Entities -Isrc/Core/OXPVerifier -Isrc/Core/Debug -Isrc/Core/Tables -Isrc/Core/MiniZip -Ideps/Linux-deps/include -I. -I/home/panoramix/GNUstep/Library/Headers -I/usr/include \
-o obj.spk/oolite.obj/strlcpy.c.o
full log at
http://0x0.st/KCJv.txt
Re: Link Time Optimizations
Posted: Fri Dec 12, 2025 9:17 pm
by Commander_X
Commander_X wrote: ↑Fri Dec 12, 2025 8:46 pm
[...]
Funny enough, it seems to be ignoring ESPEAK_DATA_PATH environment variable
[...]
This seems to be caused by the line 783 as github shows it, in Core/Universe.m:
Code: Select all
espeak_Initialize(AUDIO_OUTPUT_PLAYBACK, 100, "Resources", 0);
Replacing "Resources" with NULL (as I found it in a July this year version of the file), fixes the issue.
Re: Link Time Optimizations
Posted: Sat Dec 13, 2025 11:31 am
by another_commander
Lone_Wolf wrote: ↑Fri Dec 12, 2025 8:56 pm
Either the modern build enables LTO or I've been building oolite with LTO without realising it for sometime already.
Yup, you are already using LTO. Good.
Re: Link Time Optimizations
Posted: Sat Dec 13, 2025 11:33 am
by another_commander
Commander_X wrote: ↑Fri Dec 12, 2025 9:17 pm
Commander_X wrote: ↑Fri Dec 12, 2025 8:46 pm
[...]
Funny enough, it seems to be ignoring ESPEAK_DATA_PATH environment variable
[...]
This seems to be caused by the line 783 as github shows it, in Core/Universe.m:
Code: Select all
espeak_Initialize(AUDIO_OUTPUT_PLAYBACK, 100, "Resources", 0);
Replacing "Resources" with NULL (as I found it in a July this year version of the file), fixes the issue.
Can you please try this code and see if it works for you?
Code: Select all
int result = espeak_Initialize(AUDIO_OUTPUT_PLAYBACK, 100, "Resources", 0);
if (result < 0) espeak_Initialize(AUDIO_OUTPUT_PLAYBACK, 100, NULL, 0); // re-attempt init with legacy path parameter
Re: Link Time Optimizations
Posted: Sat Dec 13, 2025 5:53 pm
by Commander_X
another_commander wrote: ↑Sat Dec 13, 2025 11:33 am
[...]
Can you please try this code and see if it works for you?
Code: Select all
int result = espeak_Initialize(AUDIO_OUTPUT_PLAYBACK, 100, "Resources", 0);
if (result < 0) espeak_Initialize(AUDIO_OUTPUT_PLAYBACK, 100, NULL, 0); // re-attempt init with legacy path parameter
Just did, it seems to go out on the first line (doesn't get to the if line).
Re: Link Time Optimizations
Posted: Sat Dec 13, 2025 6:10 pm
by mcarans
another_commander wrote: ↑Fri Dec 12, 2025 9:22 am
Now that we have an upgraded build system for Windows, I am in the process of enabling LTO (link time optimizations) to the executables included with the installers. LTO offers a general improvement in performance of about 5-20% on average and is basically a ticket for better performance without any code changes. The tradeoff is noticeably longer link times and more complicated debugging, which is why I am thinking of enabling it only on installer builds. This way developers can continue using standard non-LTO builds which compile quickly and can be debugged without problem, while CI installers produced by github or final release binaries for which compile time is not a critical factor can get the optimization benefits for free.
But before I can enable this I need to be sure that the Linux builds can work with LTO too. Given that Linux builds have been using recent GCC versions, I would expect no issues but we have to be sure. So, any Linux user willing to assist, could you please add the -flto flag in your ADDITIONAL_CLFAGS, ADDITIONAL_OBJCFLAGS and ADDITIONAL_LDFLAGS in GNUmakefile (you may need to add the ADDITIONAL_LDLFAGS line manually) and try to build the game binary normally? You should be able to get an executable that works just like the original but be prepared to wait for a few minutes during the linking phase.
NOTE: Do not mix LTO with non-LTO object files, make sure you execute a make clean before starting the build just to be sure.
If all good we can go ahead, otherwise this will be a Windows feature only. Thanks in advance for anyone willing to provide feedback.
Nice optimisation - it works fine on the modern Linux Clang build.
Re: Link Time Optimizations
Posted: Sat Dec 13, 2025 6:12 pm
by another_commander
@Commander_X: OK, how about this one then:
Code: Select all
if (!SDL_getenv("ESPEAK_DATA_PATH")) espeak_Initialize(AUDIO_OUTPUT_PLAYBACK, 100, "Resources", 0);
else espeak_Initialize(AUDIO_OUTPUT_PLAYBACK, 100, NULL, 0);
Re: Link Time Optimizations
Posted: Sat Dec 13, 2025 6:23 pm
by mcarans
another_commander wrote: ↑Sat Dec 13, 2025 6:12 pm
@Commander_X: OK, how about this one then:
Code: Select all
if (!SDL_getenv("ESPEAK_DATA_PATH")) espeak_Initialize(AUDIO_OUTPUT_PLAYBACK, 100, "Resources", 0);
else espeak_Initialize(AUDIO_OUTPUT_PLAYBACK, 100, NULL, 0);
For the new Linux AppImage to work (see
viewtopic.php?p=303449#p303449), that would be:
Code: Select all
if (!SDL_getenv("ESPEAK_DATA_PATH"))
{
espeak_Initialize(AUDIO_OUTPUT_PLAYBACK, 100, [[ResourceManager builtInPath] UTF8String], 0);
}
else
{
espeak_Initialize(AUDIO_OUTPUT_PLAYBACK, 100, NULL, 0);
}
Re: Link Time Optimizations
Posted: Sat Dec 13, 2025 7:21 pm
by Commander_X
Considering for both suggestions I'm targeting the else branch, I tried mcarans non-hardcoding solution, and it works.
Hope it does, too, for environments where env variable is not set
Re: Link Time Optimizations
Posted: Sat Dec 13, 2025 8:16 pm
by mcarans
Commander_X wrote: ↑Sat Dec 13, 2025 7:21 pm
Considering for both suggestions I'm targeting the else branch, I tried mcarans non-hardcoding solution, and it works.
Hope it does, too, for environments where env variable is not set
I have tested the modern_linux branch with both the ESPEAK_DATA_PATH and -flto changes and it works fine. I have checked in those changes to the branch.
Re: Link Time Optimizations
Posted: Sun Dec 14, 2025 12:20 pm
by another_commander
All right, link time optimizations are now enabled for the deployment and test release installers for Linux and Windows*.
The snapshot installers have also been slightly re-configured. They now ship executables with debug information enabled and LTO disabled, so that debugging, if needed, can become easier. This makes the snapshot variant somewhat more useful, because until now the only difference between a test release and a snapshot was the watermark on the top right of the screen. If you want to concentrate on testing, you can use snapshot. If you want to just play with the best performance possible use deployment or test release.
* The modern build is required for LTO. Currently github builds Windows with the legacy environment and Linux with the modern configuration. As a result, only the Linux installers from github have LTO enabled, but you can still generate LTO builds for Windows if you build the installers yourself. This is expected to change soon, when we transition the github CI workflow to use the modern build configuration also for Windows.