Changing Oolite...

[another_commander]

But the real point is: Why do we store specific versions of a library within Github if we then still compile with external references? Should we not try to link towards the stuff we later distribute with the release?

Yes, that's what we are supposed to do, which is why you need to build against version 1.20 and not version 1.25. Your dev environment must match what we are distributing.

[hiran]

But the real point is: Why do we store specific versions of a library within Github if we then still compile with external references? Should we not try to link towards the stuff we later distribute with the release?

Yes, that's what we are supposed to do, which is why you need to build against version 1.20 and not version 1.25. Your dev environment must match what we are distributing.

So we need to either change the makefile (I am even better talking french) or at least find out how to call make so the compiler gets the right information. From LD_LIBRARY_PATH – or: How to get yourself into trouble! I see we need to look at compiler options -L and -rpath:

Code: Select all

cc -o myprog obj1.o ... objn.o -Wl,-rpath=/path/to/lib \
   -L/path/to/lib -lmylib

This page could also help: Creating relocatable Linux executables by setting RPATH with $ORIGIN

[another_commander]

I don't think it needs to be so overcomplicated. You only need to build against the versions of the .so files that we distribute with the installer.

[hiran]

I don't think it needs to be so overcomplicated. You only need to build against the versions of the .so files that we distribute with the installer.

We agree on the concept. But how to do it in practice?
I am getting lost between the Makefile gibberish, the GNUMakefile gibberish and the shell script installers/posix/make_installer.sh.

From my logs I can see that the use_deps=yes option that Commancer_X emphaiszed is used already, and it seems according to https://github.com/HiranChaudhuri/oolit ... kefile#L68 that should be sufficient. However I cannot see the commands invoking the compiler or the linker in the build output. And given that this directory contains the libs.

Looking at the 'should be' directory after my build reveals that it contains this:

Code: Select all

vagrant@OoliteBuilder:~/oolite/deps/Linux-deps/x86_64/lib_linker$ ls -l
total 4
lrwxrwxrwx 1 root    root     22 Jan 28 07:35 libSDL.so -> ../lib/libSDL-1.2.so.0
lrwxrwxrwx 1 root    root     21 Jan 28 07:35 libespeak.so -> ../lib/libespeak.so.1
lrwxrwxrwx 1 root    root     18 Jan 28 07:35 libogg.so -> ../lib/libogg.so.0
lrwxrwxrwx 1 root    root     21 Jan 28 07:35 libopenal.so -> ../lib/libopenal.so.1
lrwxrwxrwx 1 root    root     21 Jan 28 07:35 libpng.so -> ../lib/libpng14.so.14
lrwxrwxrwx 1 root    root     21 Jan 28 07:35 libvorbis.so -> ../lib/libvorbis.so.0
lrwxrwxrwx 1 root    root     25 Jan 28 07:35 libvorbisfile.so -> ../lib/libvorbisfile.so.3
lrwxrwxrwx 1 root    root     16 Jan 28 07:35 libz.so -> ../lib/libz.so.1
-rwxr-xr-x 1 vagrant vagrant 695 Jan 28 07:35 make_so_links.sh
vagrant@OoliteBuilder:~/oolite/deps/Linux-deps/x86_64/lib_linker$

Interestingly it does not contain libgnustep-base.so.1.20, which exists in ../lib. Why so? Could it mean there is something missing in https://github.com/OoliteProject/oolite ... o_links.sh ?

[another_commander]

I don't think there is anything missing. We have been generating releases with those makefiles and scripts for ages, so it's most likely something on your side. Also, GNUstep is a quite complex system and if I am not mistaken, the paths to the libraries linking against are set when its startup scripts execute. So when we link using -lgnustep-base, it is expected that the folders to go and look for the import libraries for linking are known. Maybe that is why there is no specific action involving gnustep-base inside the make_so_links script.

If you have installed and build using gnustep-base 1.25 but we package the 1.20 shared library, then obviously this will not produce a working executable. So the very first step should be to ensure that you build with the expected dependencies.

Proposed solution: apt-get-install gnustep-base 1.20 and attempt a project rebuild.

[another_commander]

However I cannot see the commands invoking the compiler or the linker in the build output

For this one, execute tha make command using messages=yes. This will generate verbose output.

Code: Select all

make -fMakefile pkg-posix-snapshot messages=yes

[hiran]

I don't think there is anything missing. We have been generating releases with those makefiles and scripts for ages, so it's most likely something on your side. Also, GNUstep is a quite complex system and if I am not mistaken, the paths to the libraries linking against are set when its startup scripts execute. So when we link using -lgnustep-base, it is expected that the folders to go and look for the import libraries for linking are known. Maybe that is why there is no specific action involving gnustep-base inside the make_so_links script.

If you have installed and build using gnustep-base 1.25 but we package the 1.20 shared library, then obviously this will not produce a working executable. So the very first step should be to ensure that you build with the expected dependencies.

I believe the assumption 'it worked for years therefore something is wrong on your side' is wrong. Times have changed, and I cannot setup the build system as it had been without good instructions how to do it.

Years ago libgnustep-base 1.20 was the latest and greatest version available. At that time it did not matter whether you build against the project-supplied version or the system-wide installed version. Well, now it matters since the system wide is no less than 1.25 and I do not see evidence that the linker is using the project supplied library.
I will try to investigate using your suggestion.

When compiling, some calls to external libraries are placed in the code (e.g. oolite.c -> oolite.o). At link time the external calls are resolved. If we were statically linking, the code from the library would be copied together with the compile output. For dynamic linking, we remain on calls to external libraries but the library names and method offsets are put in the binary (e.g. oolite.o -> oolite). So the linker was trying to find the library libgnustep-base on the link-library path (defined with -L), cannot find the project provided version 1.20 and therefore links against the system provided 1.25 library.

Now at runtime, when we transport the release to another machine, the oolite executable again has to be linked with version 1.25 of the library. The wrapper script defines the LD_LIBRARY_PATH with tells the dynamic linker to search for the project-provided 1.20 version, but the filename does not match so the dynamic linker goes to the systtem-provided library and again does not find a match (I have 1.26 installed on my Ubuntu 20). Here execution halts with a 'library not found'.
I will try to create the symlink, then compile again.

Proposed solution: apt-get-install gnustep-base 1.20 and attempt a project rebuild.

Package does not exist (any more):
https://packages.debian.org/search?suit ... ustep-base

[hiran]

So what I obtained now building with messages=yes is this call (which I believe to be the linker step:

Code: Select all

gcc  -rdynamic     -shared-libgcc  -pthread  -fexceptions -fgnu-runtime -o [color=#BF4040]<long list of object files>[/color]  -L/home/runner/GNUstep/Library/Libraries -L/usr/local/lib -L/usr/lib   -Ldeps/Linux-deps/x86_64/lib_linker -lGLU -lGL -lX11 -lSDL -lgnustep-base -Ldeps/Linux-deps/x86_64/mozilla -ljs_static -lopenal -lz -lvorbisfile -lpng `nspr-config --libs` -lstdc++  -lespeak    -lobjc  -lm

From that I take it as the path the linker is searching for libraries looks like

Code: Select all

-L/home/runner/GNUstep/Library/Libraries 
-L/usr/local/lib 
-L/usr/lib   
-Ldeps/Linux-deps/x86_64/lib_linker 
-Ldeps/Linux-deps/x86_64/mozilla 

Already in /usr/lib - the second location - I can find libgnustep-base.so pointing to 1.26 on Debian 10.11. Something which would explain why the library is not even searched in the thirs location.

I will next create symlinks into /usr/local/lib.

[another_commander]

Another idea: What if you distribute gnustep-base.so version 1.25 or whichever version it is you are building it with instead of 1.20? This would maybe cause the Windows and Linux builds to deviate (they are currently using the same gnustep version), but if the alternative is to not be able to build, it may be something worth considering.

[hiran]

Another idea: What if you distribute gnustep-base.so version 1.25 or whichever version it is you are building it with instead of 1.20? This would maybe cause the Windows and Linux builds to deviate (they are currently using the same gnustep version), but if the alternative is to not be able to build, it may be something worth considering.

Been there, done that. I had created a script to iterate over oolite and everything in an empty lib directory. Whatever ldd found to be missing the script would automatically copy in, and the next iteration would start.

This way I was able to add all the dependencies for Oolite except for glibc (which is kernel dependent) and get around the 'library not found' issue. Unfortunately, when transporting that to a run environment I just got a segmentation fault.
We can look into such a setup more closely.

[hiran]

So what I obtained now building with messages=yes is this call (which I believe to be the linker step:

Code: Select all

gcc  -rdynamic     -shared-libgcc  -pthread  -fexceptions -fgnu-runtime -o [color=#BF4040]<long list of object files>[/color]  -L/home/runner/GNUstep/Library/Libraries -L/usr/local/lib -L/usr/lib   -Ldeps/Linux-deps/x86_64/lib_linker -lGLU -lGL -lX11 -lSDL -lgnustep-base -Ldeps/Linux-deps/x86_64/mozilla -ljs_static -lopenal -lz -lvorbisfile -lpng `nspr-config --libs` -lstdc++  -lespeak    -lobjc  -lm

From that I take it as the path the linker is searching for libraries looks like

Code: Select all

-L/home/runner/GNUstep/Library/Libraries 
-L/usr/local/lib 
-L/usr/lib   
-Ldeps/Linux-deps/x86_64/lib_linker 
-Ldeps/Linux-deps/x86_64/mozilla 

Already in /usr/lib - the second location - I can find libgnustep-base.so pointing to 1.26 on Debian 10.11. Something which would explain why the library is not even searched in the thirs location.

I will next create symlinks into /usr/local/lib.

It seems like making one library accessible via /usr/local/lib helped as much as the linker reacted. However that lib has dependencies that could not be satisfied and the linker failed. I will next copy all libs to that directory.

[hiran]

So I copied all the project-supplied libs to /usr/local/lib as I did not know how to change the linker library path via the Makefiles.
Compilation really succeeded based on these libraries now, and in the end I could verify this:

Code: Select all

$ LD_LIBRARY_PATH=oolite.app/lib ldd oolite.app/oolite
	linux-vdso.so.1 (0x00007ffe4e36c000)
	libGLU.so.1 => /lib/x86_64-linux-gnu/libGLU.so.1 (0x00007f0331f13000)
	libGL.so.1 => /lib/x86_64-linux-gnu/libGL.so.1 (0x00007f0331e8b000)
	libX11.so.6 => /lib/x86_64-linux-gnu/libX11.so.6 (0x00007f0331d4e000)
	libSDL-1.2.so.0 => oolite.app/lib/libSDL-1.2.so.0 (0x00007f0331ab3000)
	libgnustep-base.so.1.20 => oolite.app/lib/libgnustep-base.so.1.20 (0x00007f03313e0000)
	libopenal.so.1 => oolite.app/lib/libopenal.so.1 (0x00007f033118e000)
	libz.so.1 => oolite.app/lib/libz.so.1 (0x00007f0330f76000)
	libvorbisfile.so.3 => oolite.app/lib/libvorbisfile.so.3 (0x00007f0330d6e000)
	libpng14.so.14 => oolite.app/lib/libpng14.so.14 (0x00007f0330b4a000)
	libplds4.so.0d => oolite.app/lib/libplds4.so.0d (0x00007f0330947000)
	libplc4.so.0d => oolite.app/lib/libplc4.so.0d (0x00007f0330743000)
	libnspr4.so.0d => oolite.app/lib/libnspr4.so.0d (0x00007f0330506000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f03304e3000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f03304dd000)
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f03302fb000)
	libespeak.so.1 => oolite.app/lib/libespeak.so.1 (0x00007f0330092000)
	libobjc.so.2 => oolite.app/lib/libobjc.so.2 (0x00007f032fe75000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f032fd24000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f032fd09000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f032fb17000)
	libGLdispatch.so.0 => /lib/x86_64-linux-gnu/libGLdispatch.so.0 (0x00007f032fa5f000)
	libGLX.so.0 => /lib/x86_64-linux-gnu/libGLX.so.0 (0x00007f032fa2b000)
	libxcb.so.1 => /lib/x86_64-linux-gnu/libxcb.so.1 (0x00007f032fa01000)
	libgmp.so.10 => oolite.app/lib/libgmp.so.10 (0x00007f032f787000)
	libgnutls.so.30 => oolite.app/lib/libgnutls.so.30 (0x00007f032f45f000)
	libgcrypt.so.20 => oolite.app/lib/libgcrypt.so.20 (0x00007f032f195000)
	libxml2.so.2 => /lib/x86_64-linux-gnu/libxml2.so.2 (0x00007f032efdb000)
	libffi.so.4 => oolite.app/lib/libffi.so.4 (0x00007f032edd3000)
	libnsl.so.1 => oolite.app/lib/libnsl.so.1 (0x00007f032ebb8000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f032ebad000)
	libvorbis.so.0 => oolite.app/lib/libvorbis.so.0 (0x00007f032e982000)
	libogg.so.0 => oolite.app/lib/libogg.so.0 (0x00007f032e77c000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f0332a91000)
	libportaudio.so.2 => oolite.app/lib/libportaudio.so.2 (0x00007f032e556000)
	libXau.so.6 => /lib/x86_64-linux-gnu/libXau.so.6 (0x00007f032e54e000)
	libXdmcp.so.6 => /lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007f032e546000)
	libnettle.so.6 => oolite.app/lib/libnettle.so.6 (0x00007f032e310000)
	libhogweed.so.4 => oolite.app/lib/libhogweed.so.4 (0x00007f032e0dc000)
	libgpg-error.so.0 => oolite.app/lib/libgpg-error.so.0 (0x00007f032deca000)
	libicuuc.so.66 => /lib/x86_64-linux-gnu/libicuuc.so.66 (0x00007f032dce2000)
	liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f032dcb9000)
	libasound.so.2 => /lib/x86_64-linux-gnu/libasound.so.2 (0x00007f032dbbe000)
	libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0 (0x00007f032dba4000)
	libicudata.so.66 => /lib/x86_64-linux-gnu/libicudata.so.66 (0x00007f032c0e3000)
$

However running it revals a core dump (no graphical window visible):

Code: Select all

$ ./oolite
+ ESPEAK_DATA_PATH=oolite.app/Resources
+ LD_LIBRARY_PATH=oolite.app/lib
+ oolite.app/oolite
./oolite: line 2: 419986 Aborted                 (core dumped) ESPEAK_DATA_PATH=oolite.app/Resources LD_LIBRARY_PATH=oolite.app/lib oolite.app/oolite
$

With that I am running out of ideas.

[another_commander]

You are linking with gnustep-base 1.20, but what about the gnustep headers in use? Are the headers still those from 1.25? The APIs between the two versions may differ and this could be a potential explanation for the core dump. If any gnustep headers are included, it would be good to check and ensure that they are the 1.20 ones. One thing you can try is run gdb on the oolite executable and get a backtrace when it crashes. If the crash happens on a gnustep call, the above could be the cause.

[hiran]

You are linking with gnustep-base 1.20, but what about the gnustep headers in use? Are the headers still those from 1.25? The APIs between the two versions may differ and this could be a potential explanation for the core dump. If any gnustep headers are included, it would be good to check and ensure that they are the 1.20 ones. One thing you can try is run gdb on the oolite executable and get a backtrace when it crashes. If the crash happens on a gnustep call, the above could be the cause.

I am neither aware that the GNUStep header files were provided in the project nor that the headers for v1.20 were on the system at all. So it is likely that during compile headers for 1.25 (or whatever is around) are used, and at link time we then take the 1.20 version.

How exactly would I apply gdb to find that out? Or would it make sense for you to get your hands on such a system?

I have created a vagrant/virtualbox/ansible setup that performs the whole job locally. If you have Vagrant and Virtualbox installed on your Windows it might work straight out of the box, and you could have the Linux based compilation at your fingertips.

[another_commander]

I am neither aware that the GNUStep header files were provided in the project nor that the headers for v1.20 were on the system at all. So it is likely that during compile headers for 1.25 (or whatever is around) are used, and at link time we then take the 1.20 version.

Indeed we don't provide the 1.20 headers. These are installed when you install gnustep in your build environment. In case it helps, here is a link to the source code of the gnustep-base distibution Oolite is based on: http://ftp.gnustep.org/pub/gnustep/core ... 0.1.tar.gz

How exactly would I apply gdb to find that out?

Getting a backtrace from gdb should be a matter of:
1. Setting up the environment variables and running GNUstep.sh, as if you were preparing to run the game normally.
2. From the folder containing the Oolite executable, typing gdb oolite
3. Typing run as soon as the gdb prompt appears.
4. Waiting for the game to crash.
5. Typing bt as soon as gdb has reported the crash.
6. Typing q to quit gdb and return to the shell.

Note that depending on the crash location and the build type (release, debug etc) you may get a more or a less meaningful backtrace. If you get an unhelpful backtrace one possible solution is toremove the strip=yes commands in the Makefile and repeat the process.

Here is a good online resource about everything gdb: https://sourceware.org/gdb/onlinedocs/g ... x.html#Top

I have created a vagrant/virtualbox/ansible setup that performs the whole job locally. If you have Vagrant and Virtualbox installed on your Windows it might work straight out of the box, and you could have the Linux based compilation at your fingertips.

Regrettably, I do not have Vagrant and Virtualbox and setting up a build environment for Linux is something that demands much nmore time than I have available to dedicate. The easy role of backseat driver, whenever an opportunity is given, is the best I can hope for these days.