Use sharun + wrappe instead of appimage

[mcarans]

To avoid being locked to old dependencies with appimage or having to build every major dependency from source on old Ubuntu, I suggest moving to Sharun + Wrappe, 2 amazing new tools that allow producing a run anywhere executable without having to build on old Linux.

Sharun allows running dynamically linked ELF binaries everywhere (musl and glibc are supported): https://github.com/VHSgunzo/sharun
Wrappe can also work on Windows and Mac as a single executable without the need for an installer: https://github.com/VHSgunzo/wrappe

An alternative would be to use sharun with appimage as it can make truly portable appimages as described here: https://pkgforge-dev.github.io/Anylinux ... THESE.html

[hiran]

Seems there are in depth problems even when packaging AppImages? I never heard of the tools you mention but they surely exist for a reason.

Yet all this sounds like beating around the bush as noone wants to go for the real problem: upgrade Oolite for new/latest dependencies.

Flatpak vs AppImage leaves me undecided - they both have their beauties and downsided. Yet another package format? In that case I'd prefer to use sharun within the AppImage. And move on to create an AppImage that enhances on metadata to be better integratable with hubs and installers (see my other post).

[another_commander]

wrappe looks a bit like the Enigma Virtual Box executable packager I was using for the Monolith builds when the legacy build was the only option. I may have to look at updating my Enigma project to work with the new builds configuration but that is work for another time as currently the day doesn't have enough hours to accommodate it.

Tried downloading wrappe for Windows from their github releases and got stuck - what do I do with the file I downloaded? It doesn't seem to be an installer neither an executable, so now that I have a file called wrappe-x86_64 sitting on my hard disk, what next?

[phkb]

Tried downloading wrappe for Windows from their github releases and got stuck - what do I do with the file I downloaded? It doesn't seem to be an installer neither an executable, so now that I have a file called wrappe-x86_64 sitting on my hard disk, what next?

I think the files to use are at this location: https://github.com/Systemcluster/wrappe/releases, which is a different spot to where you go if you click on the "Releases" link on the GitHub project home page.

[another_commander]

Ah thanks! Much better now, downloading the new toy....

[another_commander]

OK, I played a bit with wrappe and generated a single 235MB packaged executable for Windows that can be used to launch the game.

The good:
- Extremely easy to use and generate a monolithic executable. Very simple command, straightforward arguments, did it first time no problem.

The bad:
- Decompresses packaged contents to disk. Enigma decompresses to memory, which means that it is faster. wrappe package contents end up in a folder %localappdata%/temp/oolite.app/<wrappe-created id>/<game files here>. This is not OK as it interferes with the expected folder structure of the game. If you have an AddOns folder with OXPs, you must move it inside the localappdata temp folder and at the same level as the wrappe-generated id folder. Otherwise it is impossible to find OXPs - or at least I wasn't able to find a way, I played with working directory options and nothing worked for me. In comparison, the monolithic executable generated by Enigma works just like a normal one - you don't have to move any AddOns folders anywhere to make it see them. Additionally, with wrappe, if you want to look at logs etc you need to know where to look inside the localappdata temp folder, which is not something that users are expected to.
- 12 engines on VirusTotal marked the resulting monolithic executable from wrappe as malware. It is a false alarm of course, but one of the engines that flagged it was Crowdstrike Falcon, which belongs to the serious group of AV products. AVAST also flagged it. The rest were engines like SecureAge (which flags pretty much everything just to be on the safe side), McAfee (which is McAfee) and so on. But 12 is a lot. Normally if I see more than 3 engines flagging something on VT and at least one of them is a serious one, I take a step back. This will be a problem on Windows if wrappe is used. For comparison, Enigma's executables are typically marked by 5 or so engines but none of the good ones has ever misidentified them as malware.

Verdict: For now I think it's probably best to stick to Enigma for monolith executables, which is a pity because wrappe is so nice to work with and could integrate with CI/CD pipelines a lot better. But it needs some more features (especially memory decompression) and some time for AVs to know about it and its generated exes. But it's good to have the option and maybe it works much better on Linux.

[mcarans]

OK, I played a bit with wrappe and generated a single 235MB packaged executable for Windows that can be used to launch the game.

The good:
- Extremely easy to use and generate a monolithic executable. Very simple command, straightforward arguments, did it first time no problem.

The bad:
- Decompresses packaged contents to disk. Enigma decompresses to memory, which means that it is faster. wrappe package contents end up in a folder %localappdata%/temp/oolite.app/<wrappe-created id>/<game files here>. This is not OK as it interferes with the expected folder structure of the game. If you have an AddOns folder with OXPs, you must move it inside the localappdata temp folder and at the same level as the wrappe-generated id folder. Otherwise it is impossible to find OXPs - or at least I wasn't able to find a way, I played with working directory options and nothing worked for me. In comparison, the monolithic executable generated by Enigma works just like a normal one - you don't have to move any AddOns folders anywhere to make it see them. Additionally, with wrappe, if you want to look at logs etc you need to know where to look inside the localappdata temp folder, which is not something that users are expected to.
- 12 engines on VirusTotal marked the resulting monolithic executable from wrappe as malware. It is a false alarm of course, but one of the engines that flagged it was Crowdstrike Falcon, which belongs to the serious group of AV products. AVAST also flagged it. The rest were engines like SecureAge (which flags pretty much everything just to be on the safe side), McAfee (which is McAfee) and so on. But 12 is a lot. Normally if I see more than 3 engines flagging something on VT and at least one of them is a serious one, I take a step back. This will be a problem on Windows if wrappe is used. For comparison, Enigma's executables are typically marked by 5 or so engines but none of the good ones has ever misidentified them as malware.

Verdict: For now I think it's probably best to stick to Enigma for monolith executables, which is a pity because wrappe is so nice to work with and could integrate with CI/CD pipelines a lot better. But it needs some more features (especially memory decompression) and some time for AVs to know about it and its generated exes. But it's good to have the option and maybe it works much better on Linux.

I think it will work quite like an AppImage which provides various env vars that are used by the launch script. Apparently wrappe provides WRAPPE_UNPACK_DIR and WRAPPE_LAUNCH_DIR so in theory I should be able to replace the AppImage env vars in the script with those, then set the game environment variables as before so Oolite finds the AddOns folder... Actually not quite, I've made an issue because I need something like WRAPPE_EXE or WRAPPE_EXE_DIR: https://github.com/VHSgunzo/sharun/issues/107.

Seems there are in depth problems even when packaging AppImages? I never heard of the tools you mention but they surely exist for a reason.

Yet all this sounds like beating around the bush as noone wants to go for the real problem: upgrade Oolite for new/latest dependencies.

Flatpak vs AppImage leaves me undecided - they both have their beauties and downsided. Yet another package format? In that case I'd prefer to use sharun within the AppImage. And move on to create an AppImage that enhances on metadata to be better integratable with hubs and installers (see my other post).

Yes AppImages must be built on old Linux (I'm currently using Ubuntu 22.04 in GH Actions), Upgrading the Oolite dependencies is a problem with standard AppImage as Ubuntu 22.04 has very old dependencies eg. SDL so all dependencies would need to be built from source to get modern versions.

However the AnyLinux AppImage that uses sharun avoids this problem allowing AppImages to be built on new Linux with up to date dependencies but run on old Linux. The only disadvantage of the AnyLinux AppImage is the fuse2 requirement. libfuse2 is no longer installed by default on the majority of modern Linux distributions because they have moved to fuse3. This means users must install fuse2 before they can use AppImages. However, I can see that they have fallbacks when fuse2 is not present for AnyLinux AppImages. The docs say:
"Use FUSE if available, else fallback to using namespaces and if that is not possible then we automatically extract to TMPDIR and run with post cleanup,"

With wrappe, there is nothing to install, but the format is less well known. In this issue, I ask what is the status of wrappe as its repo lacks recent activity: https://github.com/VHSgunzo/sharun/issues/107. I hope VHSgunzo will take it over and maintain it then it will really go places given their outstanding work on sharun. At the moment, AnyLinux AppImage seems like the best option.

[another_commander]

I think it will work quite like an AppImage which provides various env vars that are used by the launch script. Apparently wrappe provides WRAPPE_UNPACK_DIR and WRAPPE_LAUNCH_DIR so in theory I should be able to replace the AppImage env vars in the script with those, then set the game environment variables as before so Oolite finds the AddOns folder... Actually not quite, I've made an issue because I need something like WRAPPE_EXE or WRAPPE_EXE_DIR: https://github.com/VHSgunzo/sharun/issues/107.

I got wrappe to work somewhat better by taking the AddOns folder from its standard location in a typical Oolite installation and moving it one step up, essentially outside of the game folder. This avoids having to dig inside the Windows temp folders to place addons in, but it is not ideal either. The environment variables you asked for may help.

There is also another problem with it: Logs, GNUstep and its subfolders, saves, screens etc. are all generated inside the temp folders where it gets unpacked. We don't want that, we want to have them generated where we are launching from. We do have all the environment variables that can be set in order to point all these files to where we want them to, but the catch is that, to do this we need to launch from a batch file or similar so that those env vars can be set and this defies the purpose of a single exe file distribution. It would be helpful if there was a possibility to set up any env vars we want at runner launch. For the record, Enigma doesn't have such restrictions and all external files are generated exactly where we want them to without user intervention or env vars requirement. If you want, try downloading the latest monolith from the thread in Oolite PC and see how it works for yourself. It is an old 1.91 version but we don't care about that, it's just to see how it functions in comparison with the wrappe attempt.

The AV engines false alarms issue should not be underestimated either. It will certainly deter users from downloading an exe that gets flagged so much. Speaking of AV engines, yesterday I forgot to mention that Malwarebytes was also flagging the wrappe executable and that's another one of the serious and highly respected AV scanners.

I can totally see the potential in wrappe, but so far I am not fully convinced yet for using it on Windows. I may have to explore it a bit more and will do so as time permits.

[hiran]

Can we rule out wrappe output is flagged for a good reason?

Just to remind open source is quite safe but things can go the other way also:
https://en.wikipedia.org/wiki/XZ_Utils_backdoor

[another_commander]

Can we rule out wrappe output is flagged for a good reason?

It almost certainly gets flagged because it decompresses executables and other files to disk, then runs one of them (the main game exe). This is typical malware behaviour but in this case It is clearly a false alarm because that is exactly what those app virtualization programs are expected to do and almost all of the alarms are AI-heuristics-based. But still, the average user just gets scared when they see 12 red flags on VirusTotal.