Rethinking Oolite game folders

[mcarans]

With the introduction of the AppImage build, there has been some discussion on folders, particularly for OXPs.

cbr said "But i was looking for the Addons dir ... Ok with 'x' it was created in .oolite.

Instead of this hidden dir would it be possible to change this to Library/ApplicationSupport/Oolite next to the managed ones?
Why hide it when we want to edit?"

I implemented and tested this change locally yesterday, but looking at the folders Oolite uses, I began to think about a cleanup.

These are current folders:

Code: Select all

~/GNUstep/Library/ApplicationSupport/Oolite/ManagedAddOns/ (AddOns managed within Oolite)
~/.Oolite/AddOns/ or <Oolite installation folder>/AddOns/ (User addons but installation folder work with an AppImage)
~/.Oolite/Logs/ (application Logs)
~/.Oolite/.oolite-run (suppresses display of README)
~/oolite-saves/ (saved games)
~/oolite-saves/snapshots/ (snapshots)
~/GNUstep/Defaults/oolite.plist or OoliteDefaults.plist (game defaults)

I agree with cbr about the hidden folder, so why don't we just unhide it by removing the full stop and use it for everything ie. ~/Oolite

Then we could have a structure like this (Option 1):

Code: Select all

~/Oolite
    AddOns/    
        Managed/  (moved from ~/GNUstep/Library/ApplicationSupport/Oolite/ManagedAddOns/)        
        User/ (moved from ~/.Oolite/AddOns/) 
    Logs/ (moved from ~/.Oolite/Logs/)
    .oolite-run (moved from ~/.Oolite/.oolite-run)
    SavedGames/ (moved from ~/oolite-saves/)    
    Snapshots/ (moved from ~/oolite-saves/snapshots/)
    OoliteDefaults.plist (maybe move this too from ~/GNUstep/Defaults/OoliteDefaults.plist)

What do you think?
We should still allow in addition <Oolite installation folder>/AddOns/ or possibly rename slightly to <Oolite installation folder>/UserAddOns/

[Switeck]

~/Oolite
AddOns/
Managed/ (moved from ~/GNUstep/Library/ApplicationSupport/Oolite/ManagedAddOns/)
User/ (moved from ~/.Oolite/AddOns/)
...
We should still allow in addition <Oolite installation folder>/AddOns/ or possibly rename slightly to <Oolite installation folder>/UserAddOns/

It is a bit of an annoyance to have to drill down to the current managed addons folder, so I am in favor of eliminating a few levels there!

AddOns should handle user-managed addons in it even if they're not inside the User or Managed folders.
Also the AddOns folder presumably can handle addons in both .oxp and .oxz formats...for both Managed and User folders.

[mcarans]

It is a bit of an annoyance to have to drill down to the current managed addons folder, so I am in favor of eliminating a few levels there!

AddOns should handle user-managed addons in it even if they're not inside the User or Managed folders.
Also the AddOns folder presumably can handle addons in both .oxp and .oxz formats...for both Managed and User folders.

If the existing user addons folder currently handles both formats, then that won't change.

Could just get rid of the UserAddons and use Addons as the user addons folder (Option 2):

Code: Select all

~/Oolite
    AddOns/  (moved from ~/.Oolite/AddOns/)   
        Managed/  (moved from ~/GNUstep/Library/ApplicationSupport/Oolite/ManagedAddOns/)        
    Logs/ (moved from ~/.Oolite/Logs/)
    .oolite-run (moved from ~/.Oolite/.oolite-run)
    SavedGames/ (moved from ~/oolite-saves/)    
    Snapshots/ (moved from ~/oolite-saves/snapshots/)
    OoliteDefaults.plist (maybe move this too from ~/GNUstep/Defaults/OoliteDefaults.plist)

If you prefer less levels for Managed Addons - could moved both addon folders up (Option 3):

Code: Select all

~/Oolite
    ManagedAddOns/ (moved from ~/GNUstep/Library/ApplicationSupport/Oolite/ManagedAddOns/)           
    UserAddOns/ (moved from ~/.Oolite/AddOns/) 
    Logs/ (moved from ~/.Oolite/Logs/)
    .oolite-run (moved from ~/.Oolite/.oolite-run)
    SavedGames/ (moved from ~/oolite-saves/)    
    Snapshots/ (moved from ~/oolite-saves/snapshots/)
    OoliteDefaults.plist (maybe move this too from ~/GNUstep/Defaults/OoliteDefaults.plist)

[another_commander]

Just mentioning here that the Managed Add-ons folder was never meant to be easily accessible by users and its location was part of the original in-game expansion manager design.

We have mentioned many more times than I can count that users should not be drilling down to that folder, as making changes there can potentially destabilize the functionality of the expansion manager, unless they really, really know what they are doing. I would therefore suggest leaving that folder where it is.

But I will not object heavily if people want to take chances either

[Lone_Wolf]

Keep in mind that the trend on linux is (and has been for a long time) to decrease the amount of folders in $HOME and use dot folders for personalised things .

Most of the standards used in linux are maintained by freedesktop.org (named X desktop Group in the past) and this one is not an exception. The formal version of the XDG Basedir Specification is at https://specifications.freedesktop.org/basedir/latest/ .

A less formal overview can be found in archwiki at https://wiki.archlinux.org/title/XDG_Base_Directory .
You will notice that the top 4 all default to a specific folder starting with .

Also keep in mind that windows and mac have similar approaches.
For example XDG_CONFIG_HOME (linux) and %APPDATA% (windows) are often seen as equivalent .

oolite devs are free to design their own schema but that may hinder/block linux distros from adding official packages for oolite to their distro.

There also is a thread about getting oolite in the windows store. Will MS accept oolite in that store if it doesn't use standard locations ?

[another_commander]

oolite devs are free to design their own schema but that may hinder/block linux distros from adding official packages for oolite to their distro.

There also is a thread about getting oolite in the windows store. Will MS accept oolite in that store if it doesn't use standard locations ?

These are good points. The original plan for Oolite on Windows was that you can copy its entire folder and subfolders, transfer it over to another PC and it will just work. Using the MS-approved folder locations would most likely negate this. However, it is true that the Windows design philosophy for apps is not what it used to be when Oolite started and indeed getting it to the store might be a problem unless currently accepted folder locations for user data are used.

[Wildeblood]

Just mentioning here that the Managed Add-ons folder was never meant to be easily accessible by users and its location was part of the original in-game expansion manager design.

IIRC, I was the first to complain about the obscure location of ManagedAddOns, and the first to be fobbed off with a_c's "It's not meant to be easy to find."

So, I'll just chip in here to say, it's taken ten years, but I've come around to agreeing with him.

[cbr]

The original plan for Oolite on Windows was that you can copy its entire folder and subfolders, transfer it over to another PC and it will just work.

This is what i ( like to ) do with the Addons folder ( the one with oxp's/(oxz's?) i ( want to ) edit ).

The accessibility of the location of the managed addons folder is not important for me because i do not edit/make changes there

[Commander_X]

My 0.02 Cr -- although I mentioned the "unreliable" relative @"AddOns" in Linux, it is something I'm using, again, as someone not making use of the ManagedAddOns.
It is useful for multiple Oolite instances where the common ~/.Oolite/AddOns could become a drawback, or an unwanted presence. My case for this are SOTL, or SDL2 build I needed once, for which I had to create "somewhere else" AddOns folders and symlink them in ~/.Oolite.

[mcarans]

My 0.02 Cr -- although I mentioned the "unreliable" relative @"AddOns" in Linux, it is something I'm using, again, as someone not making use of the ManagedAddOns.
It is useful for multiple Oolite instances where the common ~/.Oolite/AddOns could become a drawback, or an unwanted presence. My case for this are SOTL, or SDL2 build I needed once, for which I had to create "somewhere else" AddOns folders and symlink them in ~/.Oolite.

This is what i ( like to ) do with the Addons folder ( the one with oxp's/(oxz's?) i ( want to ) edit ).

The accessibility of the location of the managed addons folder is not important for me because i do not edit/make changes there

For the AppImage, it is possible to add code so that it finds the folder you have copied it to (there is an environment variable APPIMAGE, for where the AppImage is located). I will add that in this PR for now while the discussion continues on a possible future folder structure.

[Switeck]

AddOns should handle user-managed addons in it even if they're not inside the User or Managed folders.
Also the AddOns folder presumably can handle addons in both .oxp and .oxz formats...for both Managed and User folders.

If the existing user addons folder currently handles both formats, then that won't change.

Could just get rid of the UserAddons and use Addons as the user addons folder

Another potential problem with Addons folder (at least on Windows...) is filename + pathname total character length...which is another reason to keep the path depth as short as possible.

The managed folder does not have near the same problem, because it should only contain .oxz files not .oxp subfolders with folders inside it with files with possibly 100+ character long names.
So if the design intention is to keep the managed Addons folder hidden from most users, so be it!
(not very important but I'll mention it anyway: I'm still a little annoyed when it overwrites an older version of the addon, because we've had some difficulty recovering past versions such as pre-shader ships and stations.)

[Lone_Wolf]

The original plan for Oolite on Windows was that you can copy its entire folder and subfolders, transfer it over to another PC and it will just work. Using the MS-approved folder locations would most likely negate this.

Some projects solve this by creating a separate portable version that can take its data from anywhere the user has acces to.

Another option is to add import/export options to oolite main menu.
The export option would put the gamedata in a form (zip-archive ? ) suitable for transporting/saving/backupping . Import would convert the archive to the format used by the game.

[Stormrider]

With the introduction of the AppImage build, there has been some discussion on folders, particularly for OXPs.

I have an AddOns folder in GNUstep/Applications/Oolite and one in GNUstep/Applications/Oolite-trunk.
I can have different expansions in each of these folders to customize my game differently for different versions of Oolite. The AddOns folder in .Oolite is accessible by both versiions so I don't have to have redundant copies of AddOns that I want to use for multiple versions of Oolite.
It seems to me that your suggestion to do away with the AddOns folder in .Oolite would eliminate this ability to have shared AddOns available for multiple versions of Oolite and it would certainly eliminate this ability for older versions of Oolite in relation to newer versions.
.Oolite/AddOns is also useful for those that install Oolite system wide instead of in the home folder so they don't have to enter their password every time they want to add or remove AddOns and it allows for different users to install their own customized expansion sets.

I understand the interest in cleaning up the scattered folders but it seems to me like part of the reason for this is because Oolite might be installed system wide and users without administrative access should be able to manage their own saves, snapshots and expansions.

I appreciate all that you are doing and I understand trying to get an appimage or flatpak version in order to deal with the issues of distro incompatibility but I'm not sure this folder structure is really a problem.
I'd really like to be able to download the trunk version of Oolite and simply run the update script to update to the latest nightly like we used to. The trunk version I have now is old (1.89) and I have a version i believe you built that is supposed to be 1.91 but the directory is just named Oolite and when I run it the log states that it is 1.90.

[mcarans]

With the introduction of the AppImage build, there has been some discussion on folders, particularly for OXPs.

I have an AddOns folder in GNUstep/Applications/Oolite and one in GNUstep/Applications/Oolite-trunk.
I can have different expansions in each of these folders to customize my game differently for different versions of Oolite. The AddOns folder in .Oolite is accessible by both versiions so I don't have to have redundant copies of AddOns that I want to use for multiple versions of Oolite.
It seems to me that your suggestion to do away with the AddOns folder in .Oolite would eliminate this ability to have shared AddOns available for multiple versions of Oolite and it would certainly eliminate this ability for older versions of Oolite in relation to newer versions.
.Oolite/AddOns is also useful for those that install Oolite system wide instead of in the home folder so they don't have to enter their password every time they want to add or remove AddOns and it allows for different users to install their own customized expansion sets.

I understand the interest in cleaning up the scattered folders but it seems to me like part of the reason for this is because Oolite might be installed system wide and users without administrative access should be able to manage their own saves, snapshots and expansions.

I appreciate all that you are doing and I understand trying to get an appimage or flatpak version in order to deal with the issues of distro incompatibility but I'm not sure this folder structure is really a problem.
I'd really like to be able to download the trunk version of Oolite and simply run the update script to update to the latest nightly like we used to. The trunk version I have now is old (1.89) and I have a version i believe you built that is supposed to be 1.91 but the directory is just named Oolite and when I run it the log states that it is 1.90.

The old setup copied the game files into ~/GNUstep/Applications/Oolite or ~/GNUstep/Applications/Oolite-trunk.

To achieve what you want, just copy the latest AppImage in ~/GNUstep/Applications/Oolite or GNUstep/Applications/Oolite-trunk. When you run the AppImage, it will look for an AddOns folder in the folder in which it resides and hence find your AddOns folder.

~/.Oolite/AddOns still works. I was imply proposing making it ~/Oolite/AddOns (ie. not hidden) and moving other things like saved games in there too rather than making multiple folders in the user's home directory.

[mcarans]

Keep in mind that the trend on linux is (and has been for a long time) to decrease the amount of folders in $HOME and use dot folders for personalised things .

Most of the standards used in linux are maintained by freedesktop.org (named X desktop Group in the past) and this one is not an exception. The formal version of the XDG Basedir Specification is at https://specifications.freedesktop.org/basedir/latest/ .

A less formal overview can be found in archwiki at https://wiki.archlinux.org/title/XDG_Base_Directory .
You will notice that the top 4 all default to a specific folder starting with .

Also keep in mind that windows and mac have similar approaches.
For example XDG_CONFIG_HOME (linux) and %APPDATA% (windows) are often seen as equivalent .

oolite devs are free to design their own schema but that may hinder/block linux distros from adding official packages for oolite to their distro.

There also is a thread about getting oolite in the windows store. Will MS accept oolite in that store if it doesn't use standard locations ?

Yes, you are right. On Linux, we should probably be using ~/.local/share/<app_name> for everything. I am in favour of following the standards rather than doing our own thing. BTW, I ran the game with flatpak and the game folders, GNUstep, oolite-saves and .Oolite were created in ~/.var/app/org.OoliteProject.oolite/

For Linux, using the standard would give:

Code: Select all

~/.local/share/Oolite
    AddOns/ (moved from ~/.Oolite/AddOns/) 
    Logs/ (moved from ~/.Oolite/Logs/)
    .oolite-run (moved from ~/.Oolite/.oolite-run)
    SavedGames/ (moved from ~/oolite-saves/)    
    Snapshots/ (moved from ~/oolite-saves/snapshots/)

For Windows ~/.local/share would be %LOCALAPPDATA%\ and for MacOS ~/Library/Application Support/. According to AI, this is "The 'Strict OS' Standard (Cleanest)".

(<Game installation folder>/AddOns would still be allowed for OXPs in addition.)

Are users meant to be able to find and modify OoliteDefaults.plist easily or is it meant to be somewhat hidden like ManagedAddOns?

Another less pure but easier to find alternative is ~/Documents/Oolite (Linux, MacOS) or %USERPROFILE%\Documents\Oolite (Windows) instead of ~/.local/share/Oolite. According to AI, this is "The 'User-Friendly' Standard (Most Common)...Most games place addons in the Documents folder because it is the easiest place for non-technical users to find without enabling 'Hidden Files.'"

What are your thoughts?