Oolite Website Domain & Fixing the Expansions Manager

[hiran]

However I do not believe it will become easier with the admin panel. It seems to me PHKB updates expansions by overwriting the online files.
In that case there is not even the need to go to an admin panel to enter the new version number.

I agree, but only if rescanning happens quite often and it is fast.

I want to clarify about the admin panel - I believe that its main purpose is to allow everyone to add their own extensions.
1) register on the site
2) add your URL
3) manage your URLs

IMHO this is difficult to do only through GitHub - we cannot give everyone write rights.

How about we watch for some time and see how much traffic we get? If it is too much we can still decide for the admin panel.

[timer]

How about we watch for some time and see how much traffic we get? If it is too much we can still decide for the admin panel.

we are in no hurry ))

[phkb]

I've got more questions!

I just tried using the download manager in Oolite, which was able to download a catalog (although it did say, briefly, that the file size was something ridiculous - a detail that didn't prevent the catalog downloading). Everything looks ok-ish (there obviously a lot of missing info URL's), but the "Asteroid Tweaks" OXP is flagged on my system as "Installed but no longer available for download". It's the only one flagged this way. OoliteStarter, however, says it's Online and Installable. I checked the download URL in the and the file is at the designated location. So not sure what is going on here. Unless we are not confident that the Oolite Expansion Manager is working at the moment.

I'm also still confused by the upload_date. You said:

But as the scanner is not just triggered on repo changes but also regularly, sooner or later the scanner will access the expansion. Metadata is extracted and put into the catalog, and the Last-Modified header is taken as the upload_date.

With the Manual Witchspace Alignment OXP, it is still showing as 1/1/1970 (ie no upload date), although it was picked up as a changed file in index.html. Or is another process at work that will update the upload date?

[hiran]

I just tried using the download manager in Oolite, which was able to download a catalog (although it did say, briefly, that the file size was something ridiculous - a detail that didn't prevent the catalog downloading). Everything looks ok-ish (there obviously a lot of missing info URL's), but the "Asteroid Tweaks" OXP is flagged on my system as "Installed but no longer available for download". It's the only one flagged this way. OoliteStarter, however, says it's Online and Installable. I checked the download URL in the and the file is at the designated location. So not sure what is going on here. Unless we are not confident that the Oolite Expansion Manager is working at the moment.

Asteroid Tweaks as OXP or Asteroid Tweaks as OXZ? OXPs do not have a download URL per definition. An OXZ might get expanded into the AddOns folder to allow modifications. At that moment the versioning is broken. Or should we see a bug? Let's track this down.
As I do not have that expansion installed (neither as OXP nor as OXZ) it is shown as Online and Installable.

I'm also still confused by the upload_date. You said:

But as the scanner is not just triggered on repo changes but also regularly, sooner or later the scanner will access the expansion. Metadata is extracted and put into the catalog, and the Last-Modified header is taken as the upload_date.

With the Manual Witchspace Alignment OXP, it is still showing as 1/1/1970 (ie no upload date), although it was picked up as a changed file in index.html. Or is another process at work that will update the upload date?

In our expansions list we have:
https://github.com/OoliteProject/oolite ... s.txt#L170
and I find the URL http://wiki.alioth.net/img_auth.php/8/8 ... gnment.oxz

Code: Select all

$ curl -v -X HEAD http://wiki.alioth.net/img_auth.php/8/8b/ManualWitchspaceAlignment.oxz
Warning: Setting custom HTTP method to HEAD with -X/--request may not work the 
Warning: way you want. Consider using -I/--head instead.
*   Trying 109.70.41.29:80...
* Connected to wiki.alioth.net (109.70.41.29) port 80 (#0)
> HEAD /img_auth.php/8/8b/ManualWitchspaceAlignment.oxz HTTP/1.1
> Host: wiki.alioth.net
> User-Agent: curl/7.81.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 301 Moved Permanently
< Server: nginx/1.22.1
< Date: Wed, 11 Oct 2023 08:46:01 GMT
< Content-Type: text/html
< Content-Length: 169
< Connection: keep-alive
< Location: https://wiki.alioth.net/img_auth.php/8/8b/ManualWitchspaceAlignment.oxz
< 

No Last-Modified header. Let's use the URL the response pointed us to:

Code: Select all

$ curl -v -X HEAD https://wiki.alioth.net/img_auth.php/8/8b/ManualWitchspaceAlignment.oxz
Warning: Setting custom HTTP method to HEAD with -X/--request may not work the 
Warning: way you want. Consider using -I/--head instead.
*   Trying 109.70.41.29:443...
* Connected to wiki.alioth.net (109.70.41.29) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: CN=wiki.alioth.net
*  start date: Aug 21 07:09:41 2023 GMT
*  expire date: Nov 19 07:09:40 2023 GMT
*  subjectAltName: host "wiki.alioth.net" matched cert's "wiki.alioth.net"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
> HEAD /img_auth.php/8/8b/ManualWitchspaceAlignment.oxz HTTP/1.1
> Host: wiki.alioth.net
> User-Agent: curl/7.81.0
> Accept: */*
> 
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Server: nginx/1.22.1
< Date: Wed, 11 Oct 2023 08:47:59 GMT
< Content-Type: application/zip
< Content-Length: 336249
< Connection: keep-alive
< X-Content-Type-Options: nosniff
< Last-Modified: Wed, 11 Oct 2023 05:14:33 GMT
< 

And there you go: The expansion was uploaded at Wed, 11 Oct 2023 05:14:33 GMT
Usually you do not notice since many clients will follow the redirect implicitly. Maybe in your browser start the developer tools and watch the network traffic.

[phkb]

Asteroid Tweaks as OXP or Asteroid Tweaks as OXZ

A slip on my part. However, OXZ could have been assumed, as I was in the Expansion Manager screen, which by default means OXZ.

Anyway, if I remove the OXZ, it will show as available to download. When I put it back, it switches back to "blue" and has the message "Installed but no longer available for download". What happens for you when you install this one? Via the in-game Expansion Manager, I mean.

And there you go: The expansion was uploaded at Wed, 11 Oct 2023 05:14:33 GMT

So, the problem is really that the wiki, at some point in recent past, moved to https, and all the expansions that were created beforehand have the http reference, which means they are all being redirected.

It might be worth doing a global search/replace for "http://wiki.alioth" and replace it with "https://wiki.alioth". Unless, for those OXZ's with the actual download_url in the manifest, that would create other problems. What do you think?

[Cody]

... please look at my edit in previous post.

Edits can be missed, as they don't flag as a new post.

we are in no hurry ))

Festina lente!

[hiran]

Asteroid Tweaks as OXP or Asteroid Tweaks as OXZ

A slip on my part. However, OXZ could have been assumed, as I was in the Expansion Manager screen, which by default means OXZ.

Anyway, if I remove the OXZ, it will show as available to download. When I put it back, it switches back to "blue" and has the message "Installed but no longer available for download". What happens for you when you install this one? Via the in-game Expansion Manager, I mean.

Looks ok for me.

I do not have that expansion installed. When I install through OoliteStarter, the expansion is displayed as installed. I can uninstall and go back to the previous state.
Then I started Oolite, went to the in-game manager and installed from there. Existing Oolite, the expansion was correctly displayed by OoliteStarter. I was able to uninstall in OoliteStarter and go back to the previous state.

Maybe you are perceiving some side effects we are not aware of.

And there you go: The expansion was uploaded at Wed, 11 Oct 2023 05:14:33 GMT

So, the problem is really that the wiki, at some point in recent past, moved to https, and all the expansions that were created beforehand have the http reference, which means they are all being redirected.

Yes, that is the likely reason.

It might be worth doing a global search/replace for "http://wiki.alioth" and replace it with "https://wiki.alioth". Unless, for those OXZ's with the actual download_url in the manifest, that would create other problems. What do you think?

That may be a quick fix. It also covers some risk.
What if some URL is not just forwarded to the https version of itself but to something entirely different? In that case, by just introducing the 's' we'd create an invalid URL. OoliteAddonScanner does not bother - it will simply process the remainder of the list and emit a result successfully. But that one expansion would no longer be listed. And that would be hard to spot.

Potentially loosing one because of such a quick action makes me update them one by one manually.

[phkb]

What if some URL is not just forwarded to the https version of itself but to something entirely different? In that case, by just introducing the 's' we'd create an invalid URL. OoliteAddonScanner does not bother - it will simply process the remainder of the list and emit a result successfully. But that one expansion would no longer be listed. And that would be hard to spot.

Potentially loosing one because of such a quick action makes me update them one by one manually.

I just coded a quick automated test: I got all the "http://wiki.alioth" URL's, changed them to "https" and tried downloading each of them. There are 654 URLs with this format, and I got 654 downloaded files. I think we can do a global search/replace whenever you're ready!

[hiran]

What if some URL is not just forwarded to the https version of itself but to something entirely different? In that case, by just introducing the 's' we'd create an invalid URL. OoliteAddonScanner does not bother - it will simply process the remainder of the list and emit a result successfully. But that one expansion would no longer be listed. And that would be hard to spot.

Potentially loosing one because of such a quick action makes me update them one by one manually.

I just coded a quick automated test: I got all the "http://wiki.alioth" URL's, changed them to "https" and tried downloading each of them. There are 654 URLs with this format, and I got 654 downloaded files. I think we can do a global search/replace whenever you're ready!

Thank you for the thumbs up. I implemented that change, and the catalog is being rebuilt.

[hiran]

Here you can see whereever we updated the URL the upload_date would become valid.

So, I would have to change the download URL each time in order to get the system to change the upload date? For reference, for my OXP's I keep the same URL no matter what version is being released. I find that keeps it simple from a management point of view.

It might be bad practice from a caching point of view...

[hiran]

please, figure out where to post this on the site?
In which section (create a new one)?
On what page?
What should I name the link?

We could stick with the name 'Index of Artifacts'. Where you think it fits is not even important: we can still rename or move it around.

I was more asking for a directory to push the files in. Maybe I will just create something.

IMHO here:
https://github.com/OoliteProject/oolite ... /root/html

in the end, you can do everything yourself - if I don’t like something, we’ll discuss it and change it

Here is the first base to discuss:
https://github.com/OoliteProject/oolite ... nsionIndex

Now that the data should be uploaded to somewhere, I'm wondering how users can access the files.

[another_commander]

Guys, we have a report that the new website triggers certain antivirus countermeasures. No additional details though.

https://spacesimcentral.com/community/postid/63533/

[hiran]

Guys, we have a report that the new website triggers certain antivirus countermeasures. No additional details though.

https://spacesimcentral.com/community/postid/63533/

Be careful: security does not directly mean (anti)virus.
I have had continually alarms on oolite.org being rated as 'not work related' or not offering https or using outdated certificates.

To act effectively we need to know what to look at.

[timer]

Guys, we have a report that the new website triggers certain antivirus countermeasures. No additional details though.

https://spacesimcentral.com/community/postid/63533/

hmm... our site is a pure mixture of html + css + js (at this moment) and all the site files are publicly available on GitHub. And CloudFlare and GitHub acts as a web-servers... there is nothing dangerous there )

@hiran did u see email?

[Cody]

our site is a pure mixture of html + css + js (at this moment) and all the site files are publicly available on GitHub. And CloudFlare and GitHub acts as a web-servers... there is nothing dangerous there )

Thanks amigo - I've passed that on, and asked about OS/AV.