Oolite Bulletins

For OXPs the wiki contains some nice documentation how to create, how to pack it up and how to distribute the expansion that users create. But once that OXP archive is uploaded to some publicly available location there is still one more step to be done:

Register the OXP in the expansions manager.

This process seems undocumented right now, and I think it has room for improvement.
Could someone with more insight here create a wiki page that shows
a) what OXP developers need to do to submit their expansions for inclusion into the expansion manager
b) what is going behind the scenes (like running the OXP verifier, approval process, ...)

I was wrong in stating the process is undocumented. It is, and the documentation is actually really good at
https://bb.oolite.space/viewtopic.php?f=4&t=16688

From here I understand the discrepancies I discovered between Expansion Manager and the OXPs themselves is because publishing an OXP is a two step process:
1) upload the OXZ to the wiki
2) create the manifest and upload into the Oolite homepage

It seems when OXP authors release updates they obviously perform step 1 but often skip step 2.

Therefore I suggest to modify that process. But I will need some help doing so.

I presume your looking for an automated solution?

I think it would help greatly if for example all you had to do is upload the working oxp ( zipped or not ) and a proces would take care of the rest...

hiran wrote: ↑Tue Jun 08, 2021 6:47 am

It seems when OXP authors release updates they obviously perform step 1 but often skip step 2.

Although, in order for an update to appear on the download manager, it has to be updated as well. If there are discrepancies between the version number in the manifest of an OXP uploaded to wiki, and what the equivalent manifest on the DM says, you’ll get errors in Oolite when it tries to download it (I think this is right, but I haven’t tested it in a while).

cbr wrote: ↑Tue Jun 08, 2021 8:20 am

I presume your looking for an automated solution?

I think it would help greatly if for example all you had to do is upload the working oxp ( zipped or not ) and a proces would take care of the rest...

Congratulations, you nailed it. This approach would be a great simplification for OXP developers while at the same time ensuring we have an up to date OXP repository for the Expansion Manager.
I think that's what I will call it: OXP repository

phkb wrote: ↑Tue Jun 08, 2021 8:47 am

hiran wrote: ↑Tue Jun 08, 2021 6:47 am

It seems when OXP authors release updates they obviously perform step 1 but often skip step 2.

Although, in order for an update to appear on the download manager, it has to be updated as well. If there are discrepancies between the version number in the manifest of an OXP uploaded to wiki, and what the equivalent manifest on the DM says, you’ll get errors in Oolite when it tries to download it (I think this is right, but I haven’t tested it in a while).

You are dead right again. But that is what happens when you run package managers with outdated index files.
However the automated solution could help here. The problem you state comes when some OXP is overwritten with a newer version, so step 1 was executed. Regardless whether step 2 was performed or not - if the Expansion Manager is invoked with an outdated index, it will offer the old version to the user but download the newer version. Not what we want.

Now imagine the OXP repository would allow OXPs to be uploaded, but it does not allow overwriting files. Checking whether a file was uploaded already is quite easy.
Next, every file would be offered on a unique URL. This is not a problem as the URLs get autogenerated, and they get automatically fed into the index file.
Finally, an Expansion Manager downloads the list and offers the OXPs it finds. If the Expansion Manager runs on an outdated list, it would not offer the latest versions of the OXPs but those that it offers are definitely available. So the user would perceive a WYSIWYG. If more up to date OXPs shall be offered the user can at any time download a later version of the index.

So now we would have to think of when/how to proclaim an OXP/OXP version to prevent the repository is growing unlimited. Right now all registered OXPs take about 5 GB - and I am talking about storing mulpiple versions of the OXPs in parallel. But such setting could be configurable depending on the disk space we want to spend.

That said, I could definitely contribute creating such an application. But I would need help to know who could host that stuff as it needs to be integrated into the existing infrastructure of oolite.org, aegidian.org and alioth.net.
(wondering anyway why we have three domains for one open source project)

hiran wrote: ↑Tue Jun 08, 2021 8:21 pm

That said, I could definitely contribute creating such an application. But I would need help to know who could host that stuff as it needs to be integrated into the existing infrastructure of oolite.org, aegidian.org and alioth.net.
(wondering anyway why we have three domains for one open source project)

Just to say, if we make registering/publishing an OXP too rigorous, we will generate other problems.

Sothis & SothisTC. Sothis has no license, and the friendliest bulletin board this side of Riedquat chased away the author, KillerWolf with bayonets raised high. Spara tried approaching him, back in 2012/3 and got absolutely nowhere. We can put SothisTC (Spara's override oxp) up on the EM. But what do we do with Sothis?

Or Galactic Navy for which there is no license and the author has dissapeared. We can argue convincingly that we can carry on making GN available. And of course Montana's override will have a license. But what about GN? And it's a really important one. It permeates the wiki - just look at the Sector pages for the star systems. Or read the older ship pages. Or even look at dear old phkb's GalCop Galactic Registry which religiously lists the regional command posts for an OXP which has been out of date for years...

Cholmondely wrote: ↑Tue Jun 08, 2021 9:08 pm

Just to say, if we make registering/publishing an OXP too rigorous, we will generate other problems.

You are absolutely right.
But at the same time, having a one step approach is easier than the current model. Plus it shows problems that occur anyway (like a missing license) just in the beginning where publishing is the exiting part rather than in the end, when an author feels chased.

It neither adds work (for the OXP author) nor does it solve a legal issue. It just ensures we clarify the situation on time and before it gets a problem. And this has already happened multiple times, as you are pointing out with Sothis and Galactic Navy.

In other words: Sothis and Galactic Navy have already happened, and I cannot help recover that. But I can prevent future OXPs to end up in the same situation.

hiran wrote: ↑Tue Jun 08, 2021 8:21 pm

...wondering anyway why we have three domains for one open source project.

Never put all your basques in one exit!

Cody wrote: ↑Tue Jun 08, 2021 9:28 pm

hiran wrote: ↑Tue Jun 08, 2021 8:21 pm

...wondering anyway why we have three domains for one open source project.

Never put all your basques in one exit!

Well, at least a reason.

What do you think about the rest of this thread?

Cody wrote: ↑Tue Jun 08, 2021 9:28 pm

hiran wrote: ↑Tue Jun 08, 2021 8:21 pm

...wondering anyway why we have three domains for one open source project.

Never put all your basques in one exit!

Quite. Oolite is far too important for the continued existence of reality to have everything dependent on the one domain, especially when one or another tends to sunder...

hiran wrote: ↑Tue Jun 08, 2021 9:21 pm

Cholmondely wrote: ↑Tue Jun 08, 2021 9:08 pm

Just to say, if we make registering/publishing an OXP too rigorous, we will generate other problems.

You are absolutely right.
But at the same time, having a one step approach is easier than the current model. Plus it shows problems that occur anyway (like a missing license) just in the beginning where publishing is the exiting part rather than in the end, when an author feels chased.

It neither adds work (for the OXP author) nor does it solve a legal issue. It just ensures we clarify the situation on time and before it gets a problem. And this has already happened multiple times, as you are pointing out with Sothis and Galactic Navy.

In other words: Sothis and Galactic Navy have already happened, and I cannot help recover that. But I can prevent future OXPs to end up in the same situation.

But... neither Sothis or the Great and Glorious Galactic Navy exist on the EM... and if we put the overrides up on the EM what on earth do we do with the unlicensed originals?

Cholmondely wrote: ↑Tue Jun 08, 2021 10:18 pm

But... neither Sothis or the Great and Glorious Galactic Navy exist on the EM... and if we put the overrides up on the EM what on earth do we do with the unlicensed originals?

I could put the originals up, adding a manifest.plist is not a real modification of the code, specially with a note clarifying the still existing copyrights. The GalTech ships and Military missile were handled in this way.

Cholmondely wrote: ↑Tue Jun 08, 2021 10:18 pm

But... neither Sothis or the Great and Glorious Galactic Navy exist on the EM... and if we put the overrides up on the EM what on earth do we do with the unlicensed originals?

They are not on the expansion manager due to the license issue. Want to know what happened in other places in real life?

Although Linux is open source, SCO claimed to hold the copyright for some central parts of the Linux kernel.
https://www.eweek.com/servers/sco-lette ... -in-linux/
This became a major lawsuit for the Linux world. To circumvent this, the code parts SCO was referring to were recreated.

A similar thing is ongoing for https://reactos.org/: They aim to have a free Windows-compatible operating system but need to circumvent any copyright claims from Microsoft. So they spent years investigating APIs and behaviour to come up with their own implementation.

When I contributed code to the Apache Software Foundation, they would not even look at the pull request without the files holding a satisfactory copyright message.

So these situations are more common than you might think. If you want to be safe from future damage better reimplement the OXPs. Do not care to 'invalidate' what the original author had created - if he cared he could have declared a suitable license. And if you think reimplementing is not worth it then dump them on some lonely asteroid hurtling towards the next black hole...

hiran wrote: ↑Wed Jun 09, 2021 6:00 am

Cholmondely wrote: ↑Tue Jun 08, 2021 10:18 pm

But... neither Sothis or the Great and Glorious Galactic Navy exist on the EM... and if we put the overrides up on the EM what on earth do we do with the unlicensed originals?

They are not on the expansion manager due to the license issue. Want to know what happened in other places in real life?

Although Linux is open source, SCO claimed to hold the copyright for some central parts of the Linux kernel.
https://www.eweek.com/servers/sco-lette ... -in-linux/
This became a major lawsuit for the Linux world. To circumvent this, the code parts SCO was referring to were recreated.

A similar thing is ongoing for https://reactos.org/: They aim to have a free Windows-compatible operating system but need to circumvent any copyright claims from Microsoft. So they spent years investigating APIs and behaviour to come up with their own implementation.

When I contributed code to the Apache Software Foundation, they would not even look at the pull request without the files holding a satisfactory copyright message.

So these situations are more common than you might think. If you want to be safe from future damage better reimplement the OXPs. Do not care to 'invalidate' what the original author had created - if he cared he could have declared a suitable license. And if you think reimplementing is not worth it then dump them on some lonely asteroid hurtling towards the next black hole...

So what about if we tweak the wiki site to allow uploading oxp's as well, and put the original untweaked GN & Sothis there? Then we could allow downloading from the wiki pages without sending people off to extraneous sites which might sunder at any time? That way people are downloading the untweaked, unmodified original packages, as devised by the departed.

Or is this also dodgy?