Page 1 of 1
Debugging Oolite startup issues on fresh mingw64
Posted: Mon Aug 25, 2025 9:53 pm
by mcarans
I've been experimenting with building Oolite on a newly downloaded MSYS2 environment. I managed to get it to build and link with a load of effort. It begins to start up but then stops unfortunately.
In the logs, it gets as far as this:
09:44:16.628 [joystick.init]: Number of joysticks detected: 0
09:44:16.628 [rendering.opengl.version]: OpenGL renderer version: 4.6.0 ("4.6.0 NVIDIA 560.94"). Vendor: "NVIDIA Corporation". Renderer: "NVIDIA GeForce RTX 4060 Laptop GPU/PCIe/SSE2".
09:44:16.628 [rendering.opengl.extensions]: OpenGL extensions (404):
There are loads of OpenGL extensions listed after the above lines, but then nothing after that. What is the process for debugging failures like this? Is there a way to get much more detailed logging so I can track down where it is failing?
Re: Debugging Oolite startup issues
Posted: Tue Aug 26, 2025 2:37 am
by Commander_X
mcarans wrote: ↑Mon Aug 25, 2025 9:53 pm
I've been experimenting with building Oolite on a newly downloaded MSYS2 environment. [...]
MSYS2? Why would you try that, and not, say, cygwin64? MSYS uses also some POSIX wrappers around windows calls part of their msys-2.0.dll.
I'm using another_commander's mingw64 build bundle, and it works as expected. Not broken, don't fix.
Didn't try to spawn a "personal" setup in my (almost clean) more complete mingw64 environment, as patching and installing the supporting libraries would take quite a while.
My .02 Cr on your issue would be the hybrid build you might've managed. First of all a "clean" MSYS2 Oolite would need to be compiled against Linux make targets. Or clone the Linux targets and ajust for MSYS2 layout (e.g. library names, if autotools/GNUstep don't pick them up correctly).
Second, you'd need your own compiled (or installed) "deps" libraries targeted for MSYS2 (and the big challenge here would definitely be Mozilla's JS; I don't think it would be trivial to get it as a MSYS2 dll).
If you didn't go the clean MSYS2 route, and tried to supplement your libraries with those provided as "deps" in the tree (deps/Windows-deps) that's a no-no. Similarly with Cygwin(64), MSYS(2) shouldn't mix "native" windows dlls with its "msys-" branded dlls -- that's a receipt for disaster.
Re: Debugging Oolite startup issues on fresh mingw64
Posted: Tue Aug 26, 2025 3:17 am
by mcarans
Commander_X wrote: ↑Tue Aug 26, 2025 2:37 am
MSYS2? Why would you try that, and not, say, cygwin64? MSYS uses also some POSIX wrappers around windows calls part of their msys-2.0.dll.
I'm using another_commander's mingw64 build bundle, and it works as expected. Not broken, don't fix.
Didn't try to spawn a "personal" setup in my (almost clean) more complete mingw64 environment, as patching and installing the supporting libraries would take quite a while.
My .02 Cr on your issue would be the hybrid build you might've managed. First of all a "clean" MSYS2 Oolite would need to be compiled against Linux make targets. Or clone the Linux targets and ajust for MSYS2 layout (e.g. library names, if autotools/GNUstep don't pick them up correctly).
Second, you'd need your own compiled (or installed) "deps" libraries targeted for MSYS2 (and the big challenge here would definitely be Mozilla's JS; I don't think it would be trivial to get it as a MSYS2 dll).
If you didn't go the clean MSYS2 route, and tried to supplement your libraries with those provided as "deps" in the tree (deps/Windows-deps) that's a no-no. Similarly with Cygwin(64), MSYS(2) shouldn't mix "native" windows dlls with its "msys-" branded dlls -- that's a receipt for disaster.
I am using MSYS2 mingw64 environment so should be similar although much more up to date than the Windows development build.
I have applied all the SDL changes to latest master of SDL1. 2:
https://github.com/mcarans/SDL-1.2/tree/oolite_changes
Then I built it from source.
I also built gnustep 1.31.1 (using gcc). I might need to have a crack at building other libraries from source instead of using the ones in Windows deps. It would be great though to have some idea of which library is causing the failure so I can focus on the right library.
I think it would really help if there were some basic checks on Oolite startup to confirm important libraries like OpenGL, OpenAL, SDL, Javascript etc. can be called and hence rapidly narrow down the source of problems. Could such tests go in main.m before the application is started?
It gets much farther when running in gdb than when not, an idea I took from this post:
viewtopic.php?p=292025#p292025
Then I get these additional logs:
17:18:20.181 [rendering.opengl.shader.support]: Shaders are supported.
17:18:21.191 [searchPaths.dumpAll]: Resource paths:
Resources
../AddOns
../AddOns/Basic-debug.oxp
17:18:21.197 [MSAA.setup]: Multisample anti-aliasing not requested.
17:18:21.475 [shipData.load.begin]: Loading ship data.
17:18:24.540 [debugTCP.disconnect]: No connection to debug console: "Connection to debug console failed: 'No connection could be made because the target machine actively refused it.
' (outStream status: 7, inStream status: 7)."
17:18:24.540 [debugTCP.send.warning]: Error sending packet header, retrying.
17:18:24.566 [debugTCP.send.error]: The following packet could not be sent: {"Oolite version" = 1.91; "packet type" = "Request Connection"; "protocol version" = 65792; }
17:18:24.569 [debugTCP.disconnect]: No connection to debug console: "Connection to debug console failed: 'bad stream.' (outStream status: 0, inStream status: 0)."
17:18:24.569 [debugTCP.connect.failed]: Failed to connect to debug console at address 127.0.0.1:8563.
17:18:24.569 [display.initGL]: Requested a new surface of 1280 x 720, windowed.
17:18:24.612 [display.initGL]: Created a new surface of 1280 x 720, windowed.
17:18:24.694 [startup.complete]: ========== Loading complete in 6.60 seconds. ==========
Not sure if that provides any clues?
Re: Debugging Oolite startup issues on fresh mingw64
Posted: Tue Aug 26, 2025 5:45 am
by another_commander
mcarans wrote: ↑Mon Aug 25, 2025 9:53 pm
What is the process for debugging failures like this? Is there a way to get much more detailed logging so I can track down where it is failing?
Check for the files stderr.txt and stdout.txt in oolite.app while the game is running. Do they contain any messages? If the game terminates normally these will be auto-deleted, be aware of that in case you can't find them.
You can also try editing logcontrol.plist. At the top of the file there is a commented out
_override = yes;
line. Remove the comment and activate it, it will force printing of
all log message classes.
If all else fails you can always try the universally standard debug method: printfs in the code - or, in our case, NSLog(@"your message here") statements. They use the same format specifiers as printf. Sprinkle some of those after the OpenGL extensions have been printed out and see where NSLogs stop getting printed. With patience you can narrow it down to the exact line that fails. Sounds like torture? Sure! Been there, done that, got the t-shirt, used it as cleaning rag, threw it in the trash and then did it again. That's part of the price of admission if you want to set up a build environment for Oolite from scratch.
Which version did you build? Oolite uses 1.2.13 and the patch with the changes may not be entirely applicable for later SDL 1.2 versions. Maybe it works, but there are no guarantees and I believe that a lot of the patch changes are already present in more recent SDL 1.2. You may need to ensure tha all changes made are indeed necessary and do not duplicate existing functionality if you have built SDL later than 1.2.13.
I also built gnustep 1.31.1 (using gcc).
Did you include TLS support when you built it? It is absolutely critical that you have TLS in gnustep, or 99% of OXPs cannot be downloaded due to living in https domains.
I think it would really help if there were some basic checks on Oolite startup to confirm important libraries like OpenGL, OpenAL, SDL, Javascript etc. can be called and hence rapidly narrow down the source of problems. Could such tests go in main.m before the application is started?
What do you mean exactly by "getting called"? Windows already has checks when loading dlls, that they contain all the functions the application asks for during its startup. If an application contains calls to a function that's not there you will get an error message along the lines of "Entry point for function whatever could not be located in whatsitsname.dll" and the app doesn't start. As long as all functions can be linked dynamically at run time, the app starts. Now, if a function in a dll is buggy or has strange interactions with the OS, that's something that we can only find out at the time of executing that function.
Oolite already has plenty of sanity checks when loading up libraries like OpenAL, JS, etc. Not sure about gnustep itself, but I am not sure about how you would add checks for gnustep's health either.
Re: Debugging Oolite startup issues on fresh mingw64
Posted: Tue Aug 26, 2025 5:52 am
by another_commander
mcarans wrote: ↑Tue Aug 26, 2025 3:17 am
It gets much farther when running in gdb than when not, an idea I took from this post:
viewtopic.php?p=292025#p292025
Then I get these additional logs:
17:18:20.181 [rendering.opengl.shader.support]: Shaders are supported.
17:18:21.191 [searchPaths.dumpAll]: Resource paths:
Resources
../AddOns
../AddOns/Basic-debug.oxp
17:18:21.197 [MSAA.setup]: Multisample anti-aliasing not requested.
17:18:21.475 [shipData.load.begin]: Loading ship data.
17:18:24.540 [debugTCP.disconnect]: No connection to debug console: "Connection to debug console failed: 'No connection could be made because the target machine actively refused it.
' (outStream status: 7, inStream status: 7)."
17:18:24.540 [debugTCP.send.warning]: Error sending packet header, retrying.
17:18:24.566 [debugTCP.send.error]: The following packet could not be sent: {"Oolite version" = 1.91; "packet type" = "Request Connection"; "protocol version" = 65792; }
17:18:24.569 [debugTCP.disconnect]: No connection to debug console: "Connection to debug console failed: 'bad stream.' (outStream status: 0, inStream status: 0)."
17:18:24.569 [debugTCP.connect.failed]: Failed to connect to debug console at address 127.0.0.1:8563.
17:18:24.569 [display.initGL]: Requested a new surface of 1280 x 720, windowed.
17:18:24.612 [display.initGL]: Created a new surface of 1280 x 720, windowed.
17:18:24.694 [startup.complete]: ========== Loading complete in 6.60 seconds. ==========
Not sure if that provides any clues?
Running under gdb and failing when not under gdb usually relates to uninitialized variables. When a debugger is used everything gets initialized to either zero or some marker dummy value. The problem of course is a) confirming that indeed this is the reason and not something else and b) assuming that uninitialized objects is the reason, figuring out which object (and maybe in which library) is not initialized correctly.
Just to be sure, you do get the splash screen and the game window to open, right?
Re: Debugging Oolite startup issues on fresh mingw64
Posted: Tue Aug 26, 2025 6:31 am
by mcarans
another_commander wrote: ↑Tue Aug 26, 2025 5:52 am
Running under gdb and failing when not under gdb usually relates to uninitialized variables. When a debugger is used everything gets initialized to either zero or some marker dummy value. The problem of course is a) confirming that indeed this is the reason and not something else and b) assuming that uninitialized objects is the reason, figuring out which object (and maybe in which library) is not initialized correctly.
Just to be sure, you do get the splash screen and the game window to open, right?
Yes with gdb, I get the splash screen and game window, but not without gdb.
I applied the SDL changes manually by diffing the original 1.2.13 SDL code with a patched version then copying changes to the new master branch SDL version which required a bit of analysis of the code in some cases where the code was no longer the same.
Re: Debugging Oolite startup issues on fresh mingw64
Posted: Tue Aug 26, 2025 2:23 pm
by Commander_X
<still pursuing the possibility of a hybrid build>
Did you check that there is no msys-*.dll dependency for your Oollite.exe?
Re: Debugging Oolite startup issues on fresh mingw64
Posted: Tue Aug 26, 2025 9:37 pm
by mcarans
another_commander wrote: ↑Tue Aug 26, 2025 5:45 am
Did you include TLS support when you built it? It is absolutely critical that you have TLS in gnustep, or 99% of OXPs cannot be downloaded due to living in https domains.
I checked the gnustep-base build and the config.log says:
configure:13680: checking gnutls support
configure:13990: result: yes
...
HAVE_GNUTLS='1'
I presume that means it built with TLS support.
BTW, I think that my patched master SDL could be copied into the Windows development environment and built in there. If it worked including HDR, force feedback etc., then at least the SDL would be the newest 1.2 version available. It is easy to update this SDL fork because all that is needed is to click the button in GitHub to update the fork when updates to the original repo occur (unless there is a merge conflict with patched code).
Re: Debugging Oolite startup issues on fresh mingw64
Posted: Wed Aug 27, 2025 6:27 am
by mcarans
another_commander wrote: ↑Tue Aug 26, 2025 5:45 am
Check for the files stderr.txt and stdout.txt in oolite.app while the game is running. Do they contain any messages? If the game terminates normally these will be auto-deleted, be aware of that in case you can't find them.
You can also try editing logcontrol.plist. At the top of the file there is a commented out _override = yes;
line. Remove the comment and activate it, it will force printing of all log message classes.
If all else fails you can always try the universally standard debug method: printfs in the code - or, in our case, NSLog(@"your message here") statements. They use the same format specifiers as printf. Sprinkle some of those after the OpenGL extensions have been printed out and see where NSLogs stop getting printed. With patience you can narrow it down to the exact line that fails. Sounds like torture? Sure! Been there, done that, got the t-shirt, used it as cleaning rag, threw it in the trash and then did it again. That's part of the price of admission if you want to set up a build environment for Oolite from scratch.
Thanks for all your help so far. Running Oolite in GDB, I was able to start a new game and fly around. WIthout GDB, it stops before the splash screen. When it fails, stdout.txt and stderr.txt are 0 bytes. Editing logcontrol.plist did not add any new messages.
So I resorted to nslog. Here is where it failed:
https://github.com/OoliteProject/oolite ... ger.m#L266
NSLog(@"Stage 1");
NSDictionary *gpuConfig = [self lookUpPerGPUSettingsWithVersionString:versionStr extensionsString:extensionsStr];
NSLog(@"Stage 2");
Stage 1 was printed, but not Stage 2. Do you have any idea what the problem is and why when running in GDB, that line presumably runs fine since I can fly around?
Re: Debugging Oolite startup issues on fresh mingw64
Posted: Wed Aug 27, 2025 7:15 am
by phkb
mcarans wrote: ↑Wed Aug 27, 2025 6:27 am
Stage 1 was printed, but not Stage 2.
Can you add some additional NSlog entries to the
lookUpPerGPUSettingsWithVersionString
function (lines 788-825 of the same file)? How far through the function does it get.
Re: Debugging Oolite startup issues on fresh mingw64
Posted: Wed Aug 27, 2025 8:35 am
by mcarans
phkb wrote: ↑Wed Aug 27, 2025 7:15 am
mcarans wrote: ↑Wed Aug 27, 2025 6:27 am
Stage 1 was printed, but not Stage 2.
Can you add some additional NSlog entries to the
lookUpPerGPUSettingsWithVersionString
function (lines 788-825 of the same file)? How far through the function does it get.
It failed between stages 3 and 4:
https://github.com/OoliteProject/oolite ... ger.m#L809
expr = [match objectForKey:@"vendor"];
NSLog(@"Stage 3");
if (!CheckRegExps(vendor, expr)) continue;
NSLog(@"Stage 4");
Re: Debugging Oolite startup issues on fresh mingw64
Posted: Wed Aug 27, 2025 8:41 am
by phkb
mcarans wrote: ↑Wed Aug 27, 2025 8:35 am
It failed between stages 3 and 4
...and in the
CheckRegExps
function (lines 741-767)? Maybe output the content of the parameters in that function as well (string and regexps).
Re: Debugging Oolite startup issues on fresh mingw64
Posted: Thu Aug 28, 2025 5:53 am
by mcarans
phkb wrote: ↑Wed Aug 27, 2025 8:41 am
mcarans wrote: ↑Wed Aug 27, 2025 8:35 am
It failed between stages 3 and 4
...and in the
CheckRegExps
function (lines 741-767)? Maybe output the content of the parameters in that function as well (string and regexps).
I traced the call and eventually it reaches starting up the JavaScript engine. Perhaps unsurprisingly, this is where the problem occurs:
https://github.com/OoliteProject/oolite ... ine.m#L283
Code: Select all
NSLog(@"Stage 6");
_runtime = JS_NewRuntime(jsRuntimeInMiB * 1024L * 1024L);
NSLog(@"Stage 7");
Stage 6 is printed, not Stage 7.
Re: Debugging Oolite startup issues on fresh mingw64
Posted: Thu Aug 28, 2025 6:16 am
by another_commander
Oh, it looks like you may have to enter the hell of the JS library source to find the root cause...
Been there, not going back. Most likely you will need a JS lib built with debug symbols (actually best if you build everything (Oolite and any support library) with debug symbols) and set breakpoint at Stage 6, then try to get a backtrace and scan for any suspicious null pointers or anything that looks unusual.