Oolite Website Domain & Fixing the Expansions Manager

[timer]

Was just checking oolite.space, and on the home page, about halfway down, there's a link to "oolite.space/starting". However, each time I click on it it takes me to "oolite.org/starting", which is a 404 error. I can edit the url and successfully land on the right page, but I've tried on two systems (phone and Chromebook) and they both seem to be doing it. Is it just me?

Edit to add: this isn't the link in the side menu (which works), but the one in the body text.

IMHO this is problem of page code - this link is not trailing with "/" as other links, f.e. like "expansion packs" below.
If you edit url and add trailing "/" - it works.

[phkb]

IMHO this is problem of page code - this link is not trailing with "/" as other links, f.e. like "expansion packs" below.

Well, that's a bummer. Probably most people will click the link on the side, but I guess we'll have to wait until we can launch the new site for a final fix. Oh well. On to the next thing!

[hiran]

I agree Cloudflare looks like I have enough privileges and just don't know how to use them.

explore it! ))

Hesitating as I do not want to break stuff. Never touch a running system. But after all I had a first look around, yes.

In Github we do not have the same privileges. You can see and maintain the Cloudflare app whereas I cannot.

I got the rights from @AC and I see the "Add people" button - IMHO I can give you an Admin rights like mine, I'm sure you know GitHub better than me and won't break anything, and two admins are more reliable! Agree?

Sounds good, let's work on that together.

I cannot promise I never break anything. But it will not be intentional and I'd undo such a change ASAP.
Then we can sit and think what to do better. Even better if we sit and think before breaking stuff.

And your VPS is interesting as currently it hosts a copy of the website so that just the manifest editing stuff is performend on the old oolite.org server, but noone but you can bring in updates.

Ooh no! It seems you misunderstand! There is not a single line of code and no database on my proxy server! Only Nginx is installed on it. My proxy server is just a relay of requests from you to the old server - the whole point of its work is that it can change the Host header in the request - it replaces it on oolite.org and forwards it to the old original server IP, because the original server cannot accept a request that has a different (not oolite.org) Host in the header

Ah, my bad again. You had mentioend at some point. I'll update the diagram soon.

At this moment I try to use only two components for all architecture: GitHub + CloudFlare.

I like that approach. The less components the easier to control.

So, what about this? ) do?

Code: Select all

oolite.space     -> new site (CloudFlare Pages + GitHub oolite-web repo, branch "only-static") [production]
dev.oolite.space -> new site (CloudFlare Pages + GitHub oolite-web repo, branch "only-static-dev") [development]
api.oolite.space -> CloudFlare Workers
old.oolite.space -> my proxy -> old origin oolite.org server [temporary for editing manifest and for memories]
oolite.site - removed from using

That is pretty compatible with what I had in mind. I guess all the manifest processing would be on the Cloudflare worker then.
The think I like most is independence of the old to-be-decomissioned setup.

How long would it take to get there? Does it require you to finish the manifest editor?

[UK_Eliter]

Hello!

http://www.oolite.org/ seems to be down.

[Cody]

Yeah, it's been sold - see here. We're now using oolite.space instead.
You'll need to add a line to your .GNUstepDefaults file - see here.

[Cholmondely]

Hello!

http://www.oolite.org/ seems to be down.

...
You'll need to add a line to your .GNUstepDefaults file - see here.

Or just upgrade to Hiran's Oolite Starter (which also manages oxps with manifest.plists): https://github.com/HiranChaudhuri/Oolit ... r/releases

[timer]

Sounds good, let's work on that together. :-)

Done

Does it require you to finish the manifest editor?

No, we can use the old server for now via old.oolite.space/admin

How long would it take to get there?

I can try to do it tonight

At this moment already done
old.oolite.space
and addons.oolite.space is vhost on my proxy - the main manifest file will be available for download from my server (does not depend on the old server, but is updated from it if possible), until we make a new manifest management system

[timer]

well... new site is available on oolite.space
update of all dns servers cache can take about 48 hours.

dev.oolite.space is set up to only-static-dev branch - we can develop site and see results on separate domain and then merge to "production" only-static branch. For now, only-static needs to be merged to only-static-dev to keep them in sync.

All DNS records for oolite.site have been removed.

upd: set redirect from oolite.site to oolite.space

[timer]

Next task for me is move main manifest from my server to GitHub and exclude my server from serving url http://addons.oolite.space/api/1.0/overview
and I am not sure that I know how to do this for now...

upd1: ok, now main manifest in GitHub and it can be downloaded from it (after change DNS records).
I have to write a script that will automatically update the data in the git when a modified manifest file is received from the old server (when someone uses its editor), and then I can switch DNS for addons.oolite.space to CloudFlare Pages instead of my proxy.

[hiran]

well... new site is available on oolite.space
update of all dns servers cache can take about 48 hours.

dev.oolite.space is set up to only-static-dev branch - we can develop site and see results on separate domain and then merge to "production" only-static branch. For now, only-static needs to be merged to only-static-dev to keep them in sync.

All DNS records for oolite.site have been removed.

upd: set redirect from oolite.site to oolite.space

This is impressive. I could already see my latest changes on oolite.site.
Since I am more graphical oriented, could you help me draw some diagrams?

[hiran]

Next task for me is move main manifest from my server to GitHub and exclude my server from serving url http://addons.oolite.space/api/1.0/overview
and I am not sure that I know how to do this for now...

upd1: ok, now main manifest in GitHub and it can be downloaded from it (after change DNS records).
I have to write a script that will automatically update the data in the git when a modified manifest file is received from the old server (when someone uses its editor), and then I can switch DNS for addons.oolite.space to CloudFlare Pages instead of my proxy.

This is where I already offered help. Just let me know the scope: what should the script read, what should it write? Where does this input and output data reside? How shall the script be triggered? Probably I could have the script run whenever the data in the repo changes plus every once a week - as an example.

[timer]

Since I am more graphical oriented, could you help me draw some diagrams?

can u advise some online tool for this?

This is where I already offered help. Just let me know the scope: what should the script read, what should it write? Where does this input and output data reside? How shall the script be triggered? Probably I could have the script run whenever the data in the repo changes plus every once a week - as an example.

oh, it's not such a big problem, I'll write this script, at the moment I have only one small question - how to quickly and easily find out the current sha of a file in the git - this is required to replace it through the API.

If interested, the script algorithm is as follows (approximately):
1) once every 15 minutes (this is done by cron) download the manifest from the old server (so far this is the only data source)
2) compare its md5 with the previous one
3) if md5 is different - upload a new file to git by api (root/api/1.0/overview for OXP manager), then convert the file to json (I already wrote this tool in perl), and also upload this json file to git by api (root/data/oxp.json for site JS)

This will allow us to migrate the main manifest file to git without losing the ability to edit it using the old server. In parallel, we can develop other ways to update this file, and over time we will completely replace the functionality of the old server.

add update1: block "Latest Expansion Releases..." on main page is hardcoded - I will change it to auto-update from root/data/oxp.json a bit later.

I'm sorry, but I'm very tired and it's time for me to sleep (utc+5) - tomorrow I have a hard day.

see u later.

[hiran]

Since I am more graphical oriented, could you help me draw some diagrams?

can u advise some online tool for this?

I love https://plantuml.com/.

They have good documentation for each of the supported diagram types
https://plantuml.com/en/deployment-diagram

Plus they have an online editor that let's you grab simply the resulting picture.
https://www.plantuml.com/plantuml/uml/T ... fBtlQ3_WS0

oh, it's not such a big problem, I'll write this script, at the moment I have only one small question - how to quickly and easily find out the current sha of a file in the git - this is required to replace it through the API.

Do you mean you need to know the latest commit id (which may cause the job that pushes the data to Github Pages)?
Or would you like to know for an arbitrary file in the repository what that one's latest commit id is?
For the latter, you might want to use something like

Code: Select all

git log --max-count=1 --oneline <filename>

If interested, the script algorithm is as follows (approximately):
1) once every 15 minutes (this is done by cron) download the manifest from the old server (so far this is the only data source)
2) compare its md5 with the previous one
3) if md5 is different - upload a new file to git by api (root/api/1.0/overview for OXP manager), then convert the file to json (I already wrote this tool in perl), and also upload this json file to git by api (root/data/oxp.json for site JS)

You convert to json? Why so? The expansion manager (in-game as well as OoliteStarter) do read plist files. What would be the use of a new format?
Or do you have even more stuff in mind?

This will allow us to migrate the main manifest file to git without losing the ability to edit it using the old server. In parallel, we can develop other ways to update this file, and over time we will completely replace the functionality of the old server.

add update1: block "Latest Expansion Releases..." on main page is hardcoded - I will change it to auto-update from root/data/oxp.json a bit later.

Ah, that one might explain.
Ok, go for it.

[timer]

Do you mean you need to know the latest commit id (which may cause the job that pushes the data to Github Pages)?
Or would you like to know for an arbitrary file in the repository what that one's latest commit id is?
For the latter, you might want to use something like

Code: Select all

git log --max-count=1 --oneline <filename>

https://docs.github.com/en/rest/repos/c ... ry-content
see sha in Example response
IMHO in usual vps I can simply use get-repository-content to get sha, download overhead no matter.

already wrote this tool in perl), and also upload this json file to git by api (root/data/oxp.json for site JS)

You convert to json? Why so? The expansion manager (in-game as well as OoliteStarter) do read plist files. What would be the use of a new format?
Or do you have even more stuff in mind?

site js works with json (used root/data/oxp.json as data source) - render OXP table and other features.

[hiran]

already wrote this tool in perl), and also upload this json file to git by api (root/data/oxp.json for site JS)

You convert to json? Why so? The expansion manager (in-game as well as OoliteStarter) do read plist files. What would be the use of a new format?
Or do you have even more stuff in mind?

site js works with json (used root/data/oxp.json as data source) - render OXP table and other features.

Ok, I have code that can read a list of OXZ download URLs (a simple text file with one URL per line).
It downloads the OXZs and extracts their manifest.plist files.
This information is collected in a catalog, and the catalog can be emitted as plist, xml, json and html.

With that we have an easy to maintain catalog (that's what I call the file available at http://addons.oolite.space/api/1.0/overview/) as changes to the OXZs will directly reflect in the catalog - there is no duplicate data store. And we can include a test for a suitable license so at distribution time the author will have to markup his artefact accordingly, and there is no question what license it was or should be.

Further we'd verify the URLs to be valid. So if the hosting storage for an OXP stops existing or the OXP is removed for other reasons, it would no longer be in the catalog and thus no longer offered for download by Expansion Managers.

It then all boils down how we want to manage the list of URLs.