Changing Oolite...

[hiran]

Regrettably, I do not have Vagrant and Virtualbox and setting up a build environment for Linux is something that demands much nmore time than I have available to dedicate. The easy role of backseat driver, whenever an opportunity is given, is the best I can hope for these days.

To recap:
I spent time on restoring Windows builds, which has worked based on your help.
I spent time on restoring Mac builds, which create a release but it crashes when invoked.
I spent time on restoring Linux builds, which work on the OS they were compiled on.
Since I used automation the complete setup of the build envrionment and the run of the build themselves can be done in a matter of minutes.

The problems are always in the build environment. I am getting the information about what is missing or what could be done in small slices, while it all seems we are poking in the dark. Now you say it will still take far more time than you can devote. How much more time? Are we close to the result or still discovering the top of the iceberg?

Therefore I get the feeling I need to give up at this point in time. Sadly that means Oolite lost the ability to further support Mac and Linux distributions.

[Cholmondely]

Question: where did these nightlies first come from?

Who created the first AppleMac nightlies? the first Linux nightlies? And, for that matter, the first Windows?

[Commander_X]

[...] Sadly that means Oolite lost the ability to further support Mac and Linux distributions.

The sad part about this is that both Mac and Linux are the distros that actually evolve at a higher rate on Oolite's front.
Mac, with its "walled garden" approach, is challenging the build process with several issues:
- Xcode specifics: UI and the way the "sliding" of the views in the main window is/will be supposed to work
- The fast deprecation of some "classics" (OpenGL, some *Step specifics)
- Debug.oxp-wise: python (2) was kept in a limbo for a while; latest macOS doesn't even ship a "native" python, unless you go XQuartz (the third party X-Window provider for macOS)
- Last but not the least: code-signing?! Unless a well-established code developer for macOS will take over, building and distributing open source using Xcode isn't any longer appealing for the casual coder. Switching to Oolite's code base for macOS to "brew" mode might be a better long term option.

Linux on the other hand:
- still "works" with the way the initial builds were setup (that is, the binaries still work; this is to some extent valid for macOS too)
- building Oolite on a recent Linux distribution will raise the issues hiran uncovered, topped by the fact that the tools that were used to "provide" for the binaries in the "deps" are long since obsoleted. E.g. libgnustep-base 1.20 in deps is "bundled" with an accompanying libobjc.so.2. My gcc 9.2.0 (yes, quite old) came with a libobjc.so.4. Unless someone uses for build a quite old version of gcc, using _all_ the deps won't do.
- there are "situations" you have to plan for:
- does the user have gnustep-base installed ? (that is you cannot not bundle _a_ gnustep-base, and expect the user to fetch it; e.g. Slackware has no clue of what you might be trying to ask it, no apt-get, no yum)
- does the user have a gcc (+objc) installed ?
- will the user's environment match "the build environment" least favorable expected configuration (i.e. will it be able to actually use the binaries in Oolite's runtime "deps" folder).

The Windows case is quite ... unique: you can really bag (zip) up an working build environment, and, because it's mainly a third party from a Microsoft perspective, have it "deployed" anywhere and (mainly because of its self-containment) will do its job.

[phkb]

If I remember correctly, the previous Linux builds were done on a Linux VM with a very specific load out, such that it would always link the correct files. Is that right, a_c?

[another_commander]

If I remember correctly, the previous Linux builds were done on a Linux VM with a very specific load out, such that it would always link the correct files. Is that right, a_c?

I believe you are right. I never really had a chance to peek under the hood, but the principle is correct as far as I can tell.

[another_commander]

I managed to find this: https://bb.oolite.space/viewtopic.ph ... 06#p206706

Getafix describes the issue hiran is facing right now and how he resolved it, which also answers phkb's question above.

Of course, there is still the problem of how to set all this up inside a github workflow, assuming that a self-contained linux dev environment can be generated.

[hiran]

I managed to find this: https://bb.oolite.space/viewtopic.ph ... 06#p206706

Getafix describes the issue hiran is facing right now and how he resolved it, which also answers phkb's question above.

Of course, there is still the problem of how to set all this up inside a github workflow, assuming that a self-contained linux dev environment can be generated.

Linux 8.04 -> that sounds like Ubuntu Hardy Heron, released 24APR2008 and unsupported since 12MAY2011.
It is still downloadable at http://old-releases.ubuntu.com/releases/hardy/ yet not available on Github.

To circumvent this, I have setup a Linux 8.04 development environment (vbox),
where I have compiled and installed from source, all the libraries distributed.
This environment is also used for the nightly build.

The quick shot might be to get hold of that Virtualbox from Getafix. The better approach would be to verify if we can also compile everything from scratch.

[hiran]

I managed to find this: https://bb.oolite.space/viewtopic.ph ... 06#p206706

Getafix describes the issue hiran is facing right now and how he resolved it, which also answers phkb's question above.

Of course, there is still the problem of how to set all this up inside a github workflow, assuming that a self-contained linux dev environment can be generated.

Just to remind that if you use pkg-posix Makefile options, the package will most probably NOT work.
Perhaps it will work on an environment that is pretty much the same as the machine used to build the package.

The problem is that the oolite binary is built using the linux environment libraries, however,
it is distributed with the libraries in deps/Linux-deps/x86[_64]/lib . Confused?

This is pure proof that the Oolite project is anything but self-contained. It does not provide everything that is required for runtime. Not even to talk about compile time. That is why we have this strong dependency of the build environment - something that also happened on Windows btw.

[hiran]

Is anyone able to ping Getafix and ask for the Virtualbox machine?

[hiran]

Someone did something right. I can confirm the latest build works on Ubuntu 20.04 LTS.



Interesting to see that the build uses some system libraries and some from it's own package - and it just works.

Code: Select all

$ LD_LIBRARY_PATH=../oolite-deps/lib ldd oolite
	linux-vdso.so.1 (0x00007ffc29bf6000)
	libGLU.so.1 => /lib/x86_64-linux-gnu/libGLU.so.1 (0x00007f1844102000)
	libGL.so.1 => /lib/x86_64-linux-gnu/libGL.so.1 (0x00007f184407a000)
	libX11.so.6 => /lib/x86_64-linux-gnu/libX11.so.6 (0x00007f1843f3d000)
	libSDL-1.2.so.0 => ../oolite-deps/lib/libSDL-1.2.so.0 (0x00007f1843ca2000)
	libgnustep-base.so.1.20 => ../oolite-deps/lib/libgnustep-base.so.1.20 (0x00007f18435cf000)
	libopenal.so.1 => ../oolite-deps/lib/libopenal.so.1 (0x00007f184337d000)
	libz.so.1 => ../oolite-deps/lib/libz.so.1 (0x00007f1843165000)
	libvorbisfile.so.3 => ../oolite-deps/lib/libvorbisfile.so.3 (0x00007f1842f5d000)
	libpng14.so.14 => ../oolite-deps/lib/libpng14.so.14 (0x00007f1842d39000)
	libplds4.so.0d => ../oolite-deps/lib/libplds4.so.0d (0x00007f1842b36000)
	libplc4.so.0d => ../oolite-deps/lib/libplc4.so.0d (0x00007f1842932000)
	libnspr4.so.0d => ../oolite-deps/lib/libnspr4.so.0d (0x00007f18426f5000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f18426d2000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f18426cc000)
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f18424ea000)
	libespeak.so.1 => ../oolite-deps/lib/libespeak.so.1 (0x00007f1842281000)
	libobjc.so.2 => ../oolite-deps/lib/libobjc.so.2 (0x00007f1842064000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f1841f13000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f1841ef8000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1841d06000)
	libGLdispatch.so.0 => /lib/x86_64-linux-gnu/libGLdispatch.so.0 (0x00007f1841c4e000)
	libGLX.so.0 => /lib/x86_64-linux-gnu/libGLX.so.0 (0x00007f1841c1a000)
	libxcb.so.1 => /lib/x86_64-linux-gnu/libxcb.so.1 (0x00007f1841bf0000)
	libgmp.so.10 => ../oolite-deps/lib/libgmp.so.10 (0x00007f1841976000)
	libgnutls.so.30 => ../oolite-deps/lib/libgnutls.so.30 (0x00007f184164e000)
	libgcrypt.so.20 => ../oolite-deps/lib/libgcrypt.so.20 (0x00007f1841384000)
	libxml2.so.2 => /lib/x86_64-linux-gnu/libxml2.so.2 (0x00007f18411ca000)
	libffi.so.4 => ../oolite-deps/lib/libffi.so.4 (0x00007f1840fc2000)
	libnsl.so.1 => ../oolite-deps/lib/libnsl.so.1 (0x00007f1840da7000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f1840d9c000)
	libvorbis.so.0 => ../oolite-deps/lib/libvorbis.so.0 (0x00007f1840b71000)
	libogg.so.0 => ../oolite-deps/lib/libogg.so.0 (0x00007f184096b000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f1844186000)
	libportaudio.so.2 => ../oolite-deps/lib/libportaudio.so.2 (0x00007f1840745000)
	libXau.so.6 => /lib/x86_64-linux-gnu/libXau.so.6 (0x00007f184073d000)
	libXdmcp.so.6 => /lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007f1840735000)
	libnettle.so.6 => ../oolite-deps/lib/libnettle.so.6 (0x00007f18404ff000)
	libhogweed.so.4 => ../oolite-deps/lib/libhogweed.so.4 (0x00007f18402cb000)
	libgpg-error.so.0 => ../oolite-deps/lib/libgpg-error.so.0 (0x00007f18400b9000)
	libicuuc.so.66 => /lib/x86_64-linux-gnu/libicuuc.so.66 (0x00007f183fed1000)
	liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f183fea8000)
	libasound.so.2 => /lib/x86_64-linux-gnu/libasound.so.2 (0x00007f183fdad000)
	libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0 (0x00007f183fd93000)
	libicudata.so.66 => /lib/x86_64-linux-gnu/libicudata.so.66 (0x00007f183e2d2000)
$

[another_commander]

Someone did something right.

That was Getafix trying to salvage the Linux builder VMs. This was just a test to ensure that a working build can still be generated with those VMs.

Stay on the line, we may have an update soon.

[hiran]

That was Getafix trying to salvage the Linux builder VMs. This was just a test to ensure that a working build can still be generated with those VMs.

Stay on the line, we may have an update soon.

Seems all is silent. Or is something still happening in the background?

[another_commander]

Seems all is silent. Or is something still happening in the background?

To be frank, I am not sure. I was expecting we would have had news by now but it looks like things got delayed for reasons I am not aware of.

[hiran]

Seems all is silent. Or is something still happening in the background?

To be frank, I am not sure. I was expecting we would have had news by now but it looks like things got delayed for reasons I am not aware of.

What are we going to do with that then?
It seems apart from Getafix there is noone able to build a working version of Oolite any more.
Should we not try to move this wisdom into Github actions so it can be built automatically, whenever we update code?

If Getafix is out of the game, would be just accept the fact and never change the code again?

[another_commander]

What are we going to do with that then?

One option is to do what Getafix did a few years ago, but with current and up to date compilers, libraries, etc.: Create a development environment in a Linux VM capable of building Oolite and all its dependencies and, whenever necessary, the dependencies of the dependencies, and drill it down low-level, to a point where any and all Linux distros can run the resulting binaries without failure. It is an extremely laborious and complicated task, but Getafix did all the hard work back in the day, proving that it is doable. So what we need here is someone with Linux knowledge and enough dedication to repeat the task all over again with modern tools.

The alternative option is to keep waiting in case Getafix manages to publish his VMs somewhere, at which point building distro-agnostic binaries on Linux becomes once again possible.

It seems apart from Getafix there is noone able to build a working version of Oolite any more.

This is somewhat inaccurate, I believe. Of course anyone can build a working version of Oolite on their own Linux system using the source code from github. You have already done this yourself, right? What noone apart from Getafix is able to do anymore, is build universal binaries for Linux which are not dependent on the distro they run on (this includes new release binaries, which admittedly is a problem if we want to keep delivering a single package for all distros). The issue here is not with building a version that works, it is with its specifically chosen distribution method. The solution to the problem is to find a Linux maintainer with enough willingness, available time and dedication to put the hard work.

Should we not try to move this wisdom into Github actions so it can be built automatically, whenever we update code?

If Getafix is out of the game, would be just accept the fact and never change the code again?

We do not have this wisdom unfortunately, until Getafix manages to publish his VMs. But this should not be a deterrent to anything. Automatic builds and continuous integration are great and very convenient for both monitoring the status of the project and delivering binaries that can be tested by people, but, strictly speaking, they are not critical. For more than a few years Oolite was being developed without these facilities and it was still perfectly possible to push development forward. Not having auto-builds on every commit does not mean that commits cannot - or should not - be made. It would just be trickier to create test builds. Don't get me wrong here, I would love to have continuous integration fully working on both Linux and Mac, but I do not see its potential absence as a total blocker.


It all comes down to this, really: As it happens with all open source projects, contributors come and go and sometimes this occurs with key contributors too. When a key contributor departs, either someone else jumps in and takes over with what information is available at the time, carrying the project forward, or the parts of the project the departed contributor was working on remain stagnant until new interest is expressed. It is just the nature of open source projects, so if there is anyone Linux-savvy out there who wants the title of maintainer, this is the time to step forward. Your first task is already discussed above. Same goes for the Mac port.