The truth about Oolite's Linux builds...

[hiran]

will soon be uncovered.

We experienced compilation problems on Ubuntu 23 (Cas Tell's thread): viewtopic.php?t=21533
Our CI/CD chain on Github runs on the LTS version which consists of Ubuntu 22 LTS.

Just yesterday Ubuntu 24 LTS was released, and soon Github will integrate it into their infrastructure. At which point Oolite will be compiled on 24 LTS.
We will have to see if the problem discovered on Ubuntu 23 persists or everything just runs hunkidory.

Here is the page stating what is supported by Github: https://docs.github.com/en/actions/usin ... -resources

[hiran]

Here is another clue for the puzzle:
About half a year after it's release, I upgraded to Ubuntu 24 LTS. What can I say, the problem we had seen in Ubuntu 23 persists.

Code: Select all

$ ./oolite
./oolite.app/oolite: error while loading shared libraries: libgnustep-base.so.1.28: cannot open shared object file: No such file or directory

Erk. It looks like Oolite died with an error. When making an error
report, please copy + paste the log above into the report.

$

The Oolite version that is installed is 1.91.0.7646-241128-10e222e. The OoliteStarter version does not matter, as it shows the same error as reproduced on the console.

[MrFlibble]

Here is another clue for the puzzle:
About half a year after it's release, I upgraded to Ubuntu 24 LTS. What can I say, the problem we had seen in Ubuntu 23 persists.

Code: Select all

$ ./oolite
./oolite.app/oolite: error while loading shared libraries: libgnustep-base.so.1.28: cannot open shared object file: No such file or directory

Erk. It looks like Oolite died with an error. When making an error
report, please copy + paste the log above into the report.

$

The Oolite version that is installed is 1.91.0.7646-241128-10e222e. The OoliteStarter version does not matter, as it shows the same error as reproduced on the console.

I stumbled on this a while back. It seems the bundled libraries are way out of whack, and rely on the system libraries to be 'on point'. So basically, the bundled libraries are way out of date and of no use. I did manage to get it going on Mint 22 (which is basically the same problem), by dragging the lib binaries into the Oolite directories from a previous system's lib directories.

This needs fixing, and I'll gladly do all I can to help. I've not had much time of late, but this is an annoyance I'll drop stuff to help sort.

[Cholmondely]

I read through Cas Tell’s thread.

Does it all mean that one needs different versions of Oolite for different versions of Ubuntu?

[MrFlibble]

I read through Cas Tell’s thread.

Does it all mean that one needs different versions of Oolite for different versions of Ubuntu?

I don't know what thread you're referring to.

Very broadly speaking, one should not need a specific Oolite version for a particular linux distribution or version. The way it's built, as a "one size fits all", is fine for most distributions of around the same age as the build system.

The way most programs are distributed in the Linux world at the moment, and since the dawn of package managers, is effectively to have build instructions per distribution type, and build it on each distribution as a package for installation. This allows a more streamlined installer, which is more resource friendly. If we were to use github to automate this, it would be limited to a narrow range of distributions unless we were to start using either physical or virtual installations of other distributions and versions to act as build slaves. This is not impossible, just a lot of work to maintain, and requires resources outside github.

The way Oolite is being built as a vaguely universal installer, lends itself to the constraints of github, and works well as long a we bundle all the correct library versions. We currently do not. The bundle of libraries hasn't been preened for a while, and it just happened that for the last few years, the libraries on most common distributions would fill the gaps. Now things are moving on, so Oolite is not working without a fight on new distros.

Those who use a less common distribution have the option to build from source.

[hiran]

Oolite needs gnustep. And Ubuntu provides gnustep. So far so good. But note that on different versions of Ubuntu different versions of gnustep are provided:

Taken from https://packages.ubuntu.com/search?keyw ... chon=names:

Code: Select all

Paket libgnustep-base1.26
focal (20.04LTS) (libs): GNUstep Base library [universe]
1.26.0-7: amd64 arm64 armhf ppc64el riscv64 s390x
Paket libgnustep-base1.28
jammy (22.04LTS) (libs): GNUstep Base library [universe]
1.28.0-4build1: amd64 arm64 armhf ppc64el riscv64 s390x
Paket libgnustep-base1.29
noble (24.04LTS) (libs): GNUstep Base library [universe]
1.29.0-7ubuntu4: amd64 arm64 armhf ppc64el riscv64 s390x

Now the question is: are these libraries incompatible, or are they just slight evolutions that Oolite would even fail to notice? This can be obtained from
https://github.com/gnustep/libs-base/releases

My gut feeling is that the changes are invisible to Oolite. Now if that is the case, why is Oolite so picky about loading one specific version? Likely we need to review the wrapper scripts or the build process to find out.

[another_commander]

Firstly, we don't really know what changes in each gnustep version. Yes, there are changelogs etc. but one would need to study them in detail for each new gnustep version coming out to judge if and how it would affect us, which understandably is not something people want to do. And yes, we did have in the past gnustep versions that broke Oolite for Linux (1.19.3 if I recall correctly).

Secondly, Oolite just asks for the gnustep version it was linked against when it was built. That's where being picky comes from. If it was built against the 1.29 library then that's the one it will be looking for in order to run and if it finds any other version it will bail out.

The decision to lock the gnustep version to 1.20.1 and distribute Oolite with that had been taken in the past in order to:
1. Avoid all the above headaches and
2. Ensure that Windows and Linux behave in the exact same way as far as gnustep is concerned and that any bugs appearing would be attributed exclusively to our code and not gnustep's.

I would really prefer to investigate the possibility of remaining linked with 1.20.1, as controversial as it may sound for the Linux guys.

[MrFlibble]

I would really prefer to investigate the possibility of remaining linked with 1.20.1, as controversial as it may sound for the Linux guys.

While libgnustep-base.so.1.20 is bundled with Oolite, that's not what it appears to be built against, so when you upgrade from a system with 1.28 to one with 1.29, (e.g. Linuxmint 21.3 to 22) it all falls down. It's quite easy to copy /usr/lib/libgnustep-base.so.1.28 from an older system (if one still has one) onto a newer system in e.g. ~/GNUstep/Applications/Oolite/oolite-deps/lib/ to fix it.

I found that other libraries need copying across too.

This is why I suggest that the bundle of libraries in GNUstep/Applications/Oolite/oolite-deps needs checking. It's stale. It's not a problem if one is using a similar distro to the build distro, but unlikely so on other/newer/older distros. If all required libraries get copied there at build time, then surely the resulting installer would work on more platforms?

Or... maybe build static and avoid that too, at least for the installer, which is as far as I can tell intended to be as universal as possible. Not optimally efficient on the target system if it has some of the right libraries... but potentially simpler/cleaner.

Local builds sans installer should arguably, by default, remain dynamic for efficiency.

Apologies if that's a ramble. I need coffee.

[another_commander]

No, that's fine, there are some really good thoughts in there.

Obviously if the set of libraries distributed with the game is stale and they cannot be used in recent distros anymore, it would be a good time to update them to newer versions. What would be good in my opinion though, and this is just a recommendation from someone who is looking at the Linux situation from the outside, is to try to maintain as much as possible the distribution efficiency we had in the past. That means, find and test a set of libraries that works and lock it for official distribution. This way you would ensure stability and avoid the constant catch-up with new supporting library versions and new distros of Linux (which in itself is another huge can of worms from what I can gather).

Achieving such a goal would mean that we could even consider a new release in the future. Hope this is good enough incentive

[MrFlibble]

Obviously if the set of libraries distributed with the game is stale and they cannot be used in recent distros anymore...

They're not even the versions used by Oolite any more It's just by chance that recent distros have tended to have mostly the same versions. Most of us had to intentionally install some gnustep stuff to get Oolite to work.

Hence my main suggestion was that if the installer build process could detect, and copy, the used library binaries into the installer from the build system, that would help most folk run it on Linux regardless the distro/version. As a bonus, this would hopefully mean that as the build system on github inevitably gets updated, and as long as Oolite builds on those evolving systems, the installed Oolite will work on most Linux boxes without any extra effort. It may mean some bloat, but so would a static build, but at least it'd run on oddball distros too e.g. Alpine. The only pain point would be making sure the build worked on the latest github Linux distro of choice.. past that, it should all be smooth,

So as I see it, rather than trying to make a fixed set of libs work, we have to move with the times a bit to have github automated builds behave well, and to allow underlying libs to gain bug/security fixes. Anything that changes in a disruptive way either needs working around in the Oolite code, or worst case, that library being build from a selected version kept in the build tree.

I'll get back under my rock

[hiran]

Efficiency is actually not required to look at in this case. While relying on shared libraries, there is a trend to ship applications as snap, flatpak or appimage.

The content is a whole filesystem wrapped within an archive. Inside this filesystem there is the application, it's dependencies (the shared libs) and static resources. If Oolite were packaged fhis way some effort would be to allow it access to the user's home directory to access logs, savegames and OXPs.

Today's machines access the files inside that filesystem with not noticeable effort but shipping applications has become so much easier.

[hiran]

Firstly, we don't really know what changes in each gnustep version. Yes, there are changelogs etc. but one would need to study them in detail for each new gnustep version coming out to judge if and how it would affect us, which understandably is not something people want to do. And yes, we did have in the past gnustep versions that broke Oolite for Linux (1.19.3 if I recall correctly).

To be precise, the changelog for 1.29 states it was just a new version number. Can you believe that? Further versions comment on slight bugfixes, stability via unit tests and changes on the windows builds. Nothing of concern really. We should at least try before calling it a fail.

Secondly, Oolite just asks for the gnustep version it was linked against when it was built. That's where being picky comes from. If it was built against the 1.29 library then that's the one it will be looking for in order to run and if it finds any other version it will bail out.

Oolite works with 1.28 but fails when 1.29 is present. No, it works on 1.28 and fails if 1.28 is not present. 1.29 is not even considered. The linux dynamic linker can be smarter than that, but for some reason the Oolite build process overrides it. Have a look at
https://tldp.org/HOWTO/Program-Library- ... aries.html

From here I understand "The key to managing shared libraries is the separation of these names. Programs, when they internally list the shared libraries they need, should only list the soname they need."

So each library actually has three names. We may have mixed that up when linking Oolite, then again when shipping Oolite with the wrong lib and finally excluding the newer version in the OS when running Oolite.

The decision to lock the gnustep version to 1.20.1 and distribute Oolite with that had been taken in the past in order to:
1. Avoid all the above headaches and
2. Ensure that Windows and Linux behave in the exact same way as far as gnustep is concerned and that any bugs appearing would be attributed exclusively to our code and not gnustep's.

I would really prefer to investigate the possibility of remaining linked with 1.20.1, as controversial as it may sound for the Linux guys.

You did notice all the prereleases we built on Github require 1.28 as is shown in the error message above?

[MrFlibble]

I can't argue that Flatpak and chums aren't convenient. They truly are. I don't like them for a bunch of reasons, including bloat and detachment from the package manager, but they are convenient.

For Oolite I can see that such an approach (static bloat) is saner than attempting to get lots of distros to build packages. Convincing some distros to build packages would be nice. I'd like to see "apt install oolite-test" working, or rpm -i oolite etc. Maybe some kind Debian and/or Ubuntu devs will read this and take up the torch.

As I hinted before, GNUstep libs are not the only ones dangling. On my Mint 22 box I needed to copy in this lot from a Mint 21.3:

libgnustep-base.so.1.28
libicudata.so.70
libicui18n.so.70
libicuuc.so.70
libobjc.so.4

... then it all works quite nicely. I expect similar on the version of Ubuntu which LinuxMint 22 is based on (would look it up but gotta scarper).