[Split] Distro vs oolite.org binaries

[herodotus]

I'm looking for a volunteer

I have very little time before May this year (I shouldn't really even be playing Oolite!) but after that I might be a bit more free. If there's still a discussion to be had about this then I might be up for looking into it... It would be nice to find a solution that balanced the needs of the Oolite devs with the requirements for getting it into the package repos.

[herodotus]

What is the answer then?

Bottom line: this *&^? is hard, and anyone who pretends otherwise is either lying, deluded, or hasn't spent any serious time developing large-scale software. From the perspective of the developer of a small app with limited deps, the policies of the distro maintainers can look rather fascist; but they're looking at the bigger picture of trying to put together a whole system that works reliably and predictably according to a bunch of clear conventions that users can learn and follow. They have to worry about security and stability as well. What happens when libpng 1.4 turns out to have a horrific buffer overrun vuln that allows an OXP author to get root access on your box and turn it into a bitcoin mining bot for the Russian government? Then centralised dependency management starts to look like a much better idea!

One thing that would help a lot is if the whole OSS community learned to do versioning properly with SemVer. Then at least it would be possible to reason about dependencies automatically. I sympathise a lot with your complaint:

Its the ones who constantly change the structure of these dependencies with complete disregard for any backwards or forward comparability

Although there are often reasons why these changes make sense from the bigger perspective, I'm not going to deny that there are plenty of OSS projects that break compatibility wilfully and carelessly without providing warning or support to app authors that depend on them (*cough* GNOME *cough*) Proper versioning would help and might also give developers pause for thought before they break backwards compatibility because they feel stupid putting out version 254.3.1 a year after the first release

But in the end there's no getting away from the fact that the vast majority of software is not developed in isolation. It runs as part of bigger systems that change and develop over time. Oolite has a lot more dependencies than the obvious library deps, for example. It depends on a toolchain, the linux binary format, the OS itself. Those things change as well, albeit slowly. As the developer of an app it would be nice not to have to think about that stuff, but it's unavoidable. Software that isn't kept up-to-date slowly rots until it eventually becomes impossible to run without a lot of archaeological contortions (emulators, special sandbox environments etc). The only real alternative is controlling the whole stack from hardware upwards for your app (i.e. 80s nintendo games!)

What I will say is that since moving to Arch and it's rolling releases, I've had a wider range of software available, that's more up-to-date (usually within a day or less of developer release) and more stable than any other linux distro I've ever tried, bar none. The focus on a single, most-up-to-date version of everything really works wonders. Yes, there are old versions of libraries available for software that needs them. I can install libpng 1.4 from the AUR if I really need to; hell libpng 1.2 is in the community repo (I guess some really popular bit of software depends on it still); but that's the exception rather than the rule, and the overall result of that policy, for me, has been extremely positive!

It's probably worth thinking back to what started all this discussion in the first place - non-compliant PNG files in OXPs. The oolite.org packaged libpng doesn't spew crap into the logs, but that's because it's quietly accepting non-compliant PNGs. I think we can all agree that the *ideal* solution here would be for the PNG files to actually be correct. In this instance, moving to a more up-to-date library is actually helping to bring to light potential problems with bad colour profiles. Perhaps not the biggest issue in the world, granted, but not something that would make me feel like the library developers are being difficult or unhelpful.

As I said in my previous message, if I have a bit more time later this year I'll try and put my time where my mouth is and see what I can do to help. In the meantime I'm going to ignore my own advice and install the oolite.org packages like Getafix told me to (never disobey the druid!)

[Getafix]

herodotus wrote:

...Software that isn't kept up-to-date slowly rots...

Something that has never stopped impressing me,
is this project being the recipient of that much love and appreciation,
and the worry for the factors that may accelerate the project's end of life.

Oolite will be progressing according to the skills and availability that will be there at any given time.

[Stormrider]

herodotus thanks for the thoughtful reples,

Bottom line: this *&^? is hard

That's certainly true and I have a small understanding of how complex the situation is. One of my more popular oxps has features based on code that broke the market system for all oxps created before the code change, most oxps got updated and are working fine but one that has licensing issues hasn't been touched. At least one user is so unhappy about this breakage that he 's posted more than once about it.

The oolite.org packaged libpng doesn't spew crap into the logs, but that's because it's quietly accepting non-compliant PNGs. I think we can all agree that the *ideal* solution here would be for the PNG files to actually be correct. In this instance, moving to a more up-to-date library is actually helping to bring to light potential problems with bad colour profiles.

Non-compliant with whom? What exactly is wrong with our pngs and how much work are oxp authors going to have to do to fix these problems?

The the next more up to date library will certainly blacklist such pngs and will break many fine oxps.

I'm just a carpenter, I struggle to grasp some of the most basic code that you don't even have to think about to implement properly and while I'm reasonably artistic I never tried to create textures for models, or even 3d virtual models for that matter. I've spent hundreds of hours working on textures and still failed to even come close to creating graphics at the same level as oolites fine developers. Now those pngs are not even good enough, even though they were created with tools that the linux developers provided me with at that time.
I just don't have the time to keep up with the development cycle I guess, I've got a few things I need to fix in one oxp and have some new features I'd like to include, but there is so much to do that I don't even know how to do yet, I doubt if its even realistic.

[herodotus]

Non-compliant with whom? What exactly is wrong with our pngs and how much work are oxp authors going to have to do to fix these problems?

Non-compliant with the PNG standard or one of the standards it depends on. Formats like PNG can be displayed as images on a screen because there are a bunch of agreed conventions about how to turn a sequence of bits into a set of coloured pixels on a screen. Now I confess that I am extremely far from being an expert in this area, but my understanding is that the warning in question is due to an embedded colour profile within the file that gives instructions designed to enable a PNG to be displayed accurately on correctly calibrated equipment. Apparently according to the Arch wiki:

The old profile uses a D50 whitepoint, where D65 is standard.

For a game like Oolite this is not very important, but for software like Gimp/Photoshop this sort of thing can be extremely significant because it can mean that colours may not be displayed as the artist intended them. Sadly, the one thing you can guarantee about standards is that they won't be followed; in this case some commonly used software has created something that doesn't follow the standard at some point. Some software, including older versions of libpng, have quietly accepted this common mistake and tried to "do the right thing", but it's clear that the maintainers of the library decided that it was better to issue a warning (which some downstream software then interprets as an error); perhaps because there's potential ambiguity about what "doing the right thing" actually is; or perhaps in a bid to pressure the maintainers of the erroneous software to fix it to comply with the standard. Maybe someone more intimately familiar with image standards and colour profiles can explain exactly what's going on here.

Anyway, it seems like the Arch wiki has a couple of easy solutions - either using GIMP or ImageMagick that will enable such PNGs to be fixed. If you have ImageMagick available then it should be easy enough to batch convert all the PNGs from the commandline.

[pritter]

I hope, it's still relevant: I managed to successfully build a flatpak'ed version of Oolite. It's not as beautiful as I had envisioned, but the build succeeds.

I haven't done any testing of the app itself so far, but I have an automated build for a flatpak repo. Before going down that path and releasing the results, I want to make sure the devs are okay with that. Can anyone point me in the right direction, whom to ask? The GitHub issue related to flatpak mentiones the forum, which is why I posted Here.

As I said, I have an blog post and a gitlab-repo ready to be released. I could also host the flatpak-repo.

[another_commander]

I hope, it's still relevant: I managed to successfully build a flatpak'ed version of Oolite. It's not as beautiful as I had envisioned, but the build succeeds.

I haven't done any testing of the app itself so far, but I have an automated build for a flatpak repo. Before going down that path and releasing the results, I want to make sure the devs are okay with that. Can anyone point me in the right direction, whom to ask? The GitHub issue related to flatpak mentiones the forum, which is why I posted Here.
 
As I said, I have an blog post and a gitlab-repo ready to be released. I could also host the flatpak-repo.

Hi and thanks for making the effort for a flatpak. I think that some more information may be required in order for us to understand how suitable that particular pak is. More specifically:

• What version of linraries were used to generate the build.
• Why it is not as beautiful as you thought. Is there some issue with it?
• It really needs to be tested extensively on a wide variety of distros and confirmed functional before it gets adopted.

Right now we have something that we are very satisfied with, so moving to a different distribution method needs to result in functionality at least equal to or better than the current one. I think that flatpak has great potential, but we need to confirm what is being proposed here.

[pritter]

Hi, thanks for the feedback. I guess I should provide some more details.

First of all, I have to say that I am very happy with the universal installer Oolite provides. I works very well, and I never had an issue. The only reason I looked into flatpak was that I want to switch to atomic workstation. In atomic workstation, /opt is not writable, so there is no possibility for a system wide installation, a per user installation in /home should be possible, but I did not test that.

So, to build the flatpak, I leverage the great work you already put into the installer. My original plan was to simply do an unattended installation inside the flatpak environment, which would probably satisfy the requirements to get the app into flathub. But /opt is not writable on flatpak-builder either. To make it work, I use sort of a hack: I do the unattended installation on the build root and just move everything into the flatpak.

So basically, it should work just like a normal installation with the installer provided by the oolite project, because I did not change anything (except remove the uninstall script and add a wrapper to start the app).

The next step would be to test whether everything works fine. I plan on doing that in the near future and then provide updates on the matter.

The following caveats apply:
- The current build is 64bit only
- A reasonable description should be added. Currently, I simply copied some of the text from the main homepage as description
- The icon is required to be 128x128, but I only managed to find the 48x48 version

If it would be ok, I'd publish my blog post with details, make the repo public and link both of them here. The community could then test this version and identify issues, which I'd then adress.

Do you think a build from source would be the better approach? I seriuosly don't know how to do that with the information provided on github.

[another_commander]

OK thanks, sounds all right. Something that would be really cool would be the ability to integrate flatpak builds also for the nightlies. I guess we'll have to provide the nightly scripts we use in order for this to be even attempted, maybe Getafix can give some input here.

As for building from source, this may turn out to be trickier than it looks. Right now we are building the game on Linux on ancient virtual machines and we've had some reports of builds on some newer Linux distros that fail to run properly, possibly due to incompatibilities introduced with gnustep 1.25. Maybe it would be best to stick to the provided binaries for now, at least until we've worked out how to get a stable build on one of those modern distros. Again, I think Getafix is da man here.

Regarding the 128x128 icon file, check oolite\installers\win32\Oolite.ico. Not sure if it would help much on Linux, but this is a multi size icon file and it contains also a 256x256 icon. Maybe this one can be scaled down to 128x128 and become usable?

[Getafix]

Hi pritter and welcome aboard!

Thanx for sharing your work here.
How can I get my hands on the flatpak you have made? (drooling penguin)

[pritter]

Thanx for sharing your work here.
How can I get my hands on the flatpak you have made? (drooling penguin)

No problem. I am going to create a new topic with all the links and some information on how to use/test. You can start with https://gitlab.com/paulritter/flatpak-oolite there should be build artefacts of the last automated build, but I did not test these myself yet. I have that planned for wednesday

[pritter]

OK thanks, sounds all right. Something that would be really cool would be the ability to integrate flatpak builds also for the nightlies. I guess we'll have to provide the nightly scripts we use in order for this to be even attempted, maybe Getafix can give some input here.

I studied the GitHub repo regarding the nightlies. If these are "normal" installers, it shouldn't be a problem to build flatpaks. The major benefit would be that users can install stable and nightly versions in parallel.

To automate the build of nightly flatpaks, I'd figure out a way to automatically get the URL of the nightly *.tgz because it seems to change every night. Maybe using a webhook that posts the URL to trigger the build.

Update
It seems to work, I will use a cron job and some regex-handling for now. Is there a release interval for the nightlies? Currently, I set it up to rebuild daily, but that does not seem necessary.