BigSur and Oolite
Moderators: winston, another_commander
- Cholmondely
- Archivist
- Posts: 5405
- Joined: Tue Jul 07, 2020 11:00 am
- Location: The Delightful Domains of His Most Britannic Majesty (industrial? agricultural? mainly anything?)
- Contact:
BigSur and Oolite
If I understood the conversation on GitHub correctly (https://github.com/OoliteProject/oolite/issues/360), the new Mac OS 11 (Big Sur) renders Oolite unplayable on the new AppleMacs.
Apparently Apple have come up with two fixes for Big Sur:
(i) an update for XCode (Apples Developer tool) which will now "compile code into finished 'apps', by translating the high-level code written by you chaps into low level code which will run on both Intel and Apple's own silicon.
(ii) a second tool known as Rosetta 2 which using something called 'emulation' will translate the older 'apps' so that they run on Big Sur. The increase is resource intensivitivity is supposed to be made up by the increased resources made available by the combination of Apple's new silicon and Big Sur.
(Freely Adapted from Which? Computing magazine: April 2021).
It does not seem unreasonable that Rosetta 2 should fix the major problems for now, even though it may well introduce minor ones (such as my tea-maker failing to brew a semi-decent pot of fragrant broken orange pekoe).
Apparently Apple have come up with two fixes for Big Sur:
(i) an update for XCode (Apples Developer tool) which will now "compile code into finished 'apps', by translating the high-level code written by you chaps into low level code which will run on both Intel and Apple's own silicon.
(ii) a second tool known as Rosetta 2 which using something called 'emulation' will translate the older 'apps' so that they run on Big Sur. The increase is resource intensivitivity is supposed to be made up by the increased resources made available by the combination of Apple's new silicon and Big Sur.
(Freely Adapted from Which? Computing magazine: April 2021).
It does not seem unreasonable that Rosetta 2 should fix the major problems for now, even though it may well introduce minor ones (such as my tea-maker failing to brew a semi-decent pot of fragrant broken orange pekoe).
Comments wanted:
•Missing OXPs? What do you think is missing?
•Lore: The economics of ship building How many built for Aronar?
•Lore: The Space Traders Flight Training Manual: Cowell & MgRath Do you agree with Redspear?
•Missing OXPs? What do you think is missing?
•Lore: The economics of ship building How many built for Aronar?
•Lore: The Space Traders Flight Training Manual: Cowell & MgRath Do you agree with Redspear?
- Cody
- Sharp Shooter Spam Assassin
- Posts: 16081
- Joined: Sat Jul 04, 2009 9:31 pm
- Location: The Lizard's Claw
- Contact:
Re: BigSur and Oolite
Can't beat a quality BOP! Ceylon, preferably!
I would advise stilts for the quagmires, and camels for the snowy hills
And any survivors, their debts I will certainly pay. There's always a way!
And any survivors, their debts I will certainly pay. There's always a way!
- maik
- Wiki Wizard
- Posts: 2028
- Joined: Wed Mar 10, 2010 12:30 pm
- Location: Ljubljana, Slovenia (mainly industrial, feudal, TL12)
Re: BigSur and Oolite
The conversation rather seems to be about M1 based Macs, not about Big Sur (although they do run on Big Sur of course). I’m running Oolite on Big Sur, just on an old (Intel-based) Mac.Cholmondely wrote: ↑Sun Apr 18, 2021 8:30 amIf I understood the conversation on GitHub correctly (https://github.com/OoliteProject/oolite/issues/360), the new Mac OS 11 (Big Sur) renders Oolite unplayable on the new AppleMacs.
What I would like to understand (really just a pointer please) is why we need to stick to an old SpiderMonkey version on any platform, or why a change would break 300-400 OXPs.
-
- Quite Grand Sub-Admiral
- Posts: 6700
- Joined: Wed Feb 28, 2007 7:54 am
Re: BigSur and Oolite
Good qustion. Two main reasons:
Portability: Spidermonkey for Oolite is built with certain flags specifically for Oolite. On three different platforms. Going to a more recent Spidermonkey does not guarantee that all platforms will be able to follow through. Windows in particular is the trickiest to build from source and - I could be wrong on this but I don't think so - it may even be impossible to build with the current development environment in use. And this is the small part.
API compatibility and engine interfacing: There is no guarantee that the scripting API remains the same as what we currently have in the game in later Spidermonkey versions. All OXPs containing scripts for Oolite have been built using standard ECMA 5 Javascript of course, but the real problem lies within how the scripting engine has been implemented in the game. Going to a more recent JS version may require a considerable rewrite of the interface between game angine and scripting engine. Even if we assume that we can build successfully a new Spidermonkey for all platforms, the guts of the scripting engine will certainly have to be remade to support the more modern way that the new scripting engine gets interfaced to and interacts with the host application. It is at this point where I expect all OXPs with scripts to start breaking down. Not because they will contain invalid or deprecated JS, but bevause there will be glorious bugs and crashes from the scripting part of Oolite's engine as the upgrade is in progress, at least until the interface with JS gets rebuilt completely and confirmed fully working. Note that this might require a few release cycles.
We can also not exclude that there might have been changes also in the JS API itself since the days of the one we have now. In this case, OXPs may indeed start failing because of the way they are written. Those may need to have their scripts adjusted to work with the latest JS version, but I have no further information on this one.
Of course, we also need to consider the time that will be required to study how exactly new Spidermonkeys are attached to an application. Jens' work on the current scripting engine was not done in a couple of days; it took a lot of work, research and a lot of studying and all this will have to be repeated if we were to proceed with an upgrade. So if anyone feels ready to rewrite the Oolite scripting engine from scratch (when we already have something that works fine, albeit old), now it would be a good time to step up. And maybe provide a way to build the latest Spidermonkey for Windows too, because I am not going to do it - did it once and was more than enough (and switching development environment is not a solution I am prepared to take on, because that would mean that the entire support libraries set distributed with the game will have to be rebuilt for the new dev environment too).
- hiran
- Theorethicist
- Posts: 2415
- Joined: Fri Mar 26, 2021 1:39 pm
- Location: a parallel world I created for myself. Some call it a singularity...
Re: BigSur and Oolite
This sounds like heavy lifting.
However if that piece of work were written down onto a roadmap, with enough hints what to look for and a way to verify successful operation I think it could be done by people interested and having knowledge in JavaScript and Objective-C integration but no need to understand the full Ooniverse.
However if that piece of work were written down onto a roadmap, with enough hints what to look for and a way to verify successful operation I think it could be done by people interested and having knowledge in JavaScript and Objective-C integration but no need to understand the full Ooniverse.
Sunshine - Moonlight - Good Times - Oolite
- Cmdr James
- Commodore
- Posts: 1357
- Joined: Tue Jun 05, 2007 10:43 pm
- Location: Berlin
Re: BigSur and Oolite
In my experience the roadmap for updating dependencies in almost all projects is the same.hiran wrote: ↑Sun Apr 18, 2021 5:36 pmThis sounds like heavy lifting.
However if that piece of work were written down onto a roadmap, with enough hints what to look for and a way to verify successful operation I think it could be done by people interested and having knowledge in JavaScript and Objective-C integration but no need to understand the full Ooniverse.
Change it, test, see what breaks, fix, repeat. In this case its a bit worse because its multi platform and its about supporting an extention framework so "regression testing" is horrible.
Good question, how would someone know? All 300 or so OXPs work flawlessly? They dont even with the current version. I see no easy way to define "sucessful operation" for this task.
But in principle you are right, someone with decent objective C skills and plenty of time could probably make the change easily enough.
- hiran
- Theorethicist
- Posts: 2415
- Joined: Fri Mar 26, 2021 1:39 pm
- Location: a parallel world I created for myself. Some call it a singularity...
Re: BigSur and Oolite
How is Oolite built today? How is it tested?Cmdr James wrote: ↑Sun Apr 18, 2021 9:00 pmChange it, test, see what breaks, fix, repeat. In this case its a bit worse because its multi platform and its about supporting an extention framework so "regression testing" is horrible.
Good question, how would someone know? All 300 or so OXPs work flawlessly? They dont even with the current version. I see no easy way to define "sucessful operation" for this task.
Multi-platform and extension framework should not be the worst things during testing.
If we can create the infrastructure to have scenarios repeated and verified in an automated fashion, some of them can be created for Oolite itself and then for extensions wherever need be. Over time these could act as regression tests. And since Oolite and the OXPs run on different platforms, the whole test suite could be run on these platforms.
So to enable novice developers and let them know whether they broke the codeI think the first step should be to simplify testing. If not done already, some automated testing could vastly improve maintainability. What would it take to re-run scenarios in Oolite?
Sunshine - Moonlight - Good Times - Oolite
- Cmdr James
- Commodore
- Posts: 1357
- Joined: Tue Jun 05, 2007 10:43 pm
- Location: Berlin
Re: BigSur and Oolite
I totally agree. Setting up runners for different platforms and automating with github actions should be doable. Though for the project I think CI/CD is pretty much unexplored territory. Have a shot at it!hiran wrote: ↑Sun Apr 18, 2021 10:22 pmIf we can create the infrastructure to have scenarios repeated and verified in an automated fashion, some of them can be created for Oolite itself and then for extensions wherever need be. Over time these could act as regression tests. And since Oolite and the OXPs run on different platforms, the whole test suite could be run on these platforms.
There is a makefile.
I dont remember seeing anything I would really call unit tests but you can dig around in here: https://github.com/OoliteProject/oolite-tests/
I think you misunderestimate the complexity, but if you want to have a stab at it Im willing to support as best I can.
- hiran
- Theorethicist
- Posts: 2415
- Joined: Fri Mar 26, 2021 1:39 pm
- Location: a parallel world I created for myself. Some call it a singularity...
Re: BigSur and Oolite
Thank you for the offer. CI/CD is unexplored? But who would then sit every night to create https://github.com/OoliteProject/nightlies?Cmdr James wrote: ↑Sat Apr 24, 2021 2:51 pmI totally agree. Setting up runners for different platforms and automating with github actions should be doable. Though for the project I think CI/CD is pretty much unexplored territory. Have a shot at it!hiran wrote: ↑Sun Apr 18, 2021 10:22 pmIf we can create the infrastructure to have scenarios repeated and verified in an automated fashion, some of them can be created for Oolite itself and then for extensions wherever need be. Over time these could act as regression tests. And since Oolite and the OXPs run on different platforms, the whole test suite could be run on these platforms.
Let me have a smooth dive. As a first I will look at the OXP documentation.
Sunshine - Moonlight - Good Times - Oolite
-
- Quite Grand Sub-Admiral
- Posts: 6700
- Joined: Wed Feb 28, 2007 7:54 am
Re: BigSur and Oolite
The nightlies are built by automated scripts created by Getafix and there are at least two servers involved in their creation. Sometimes the Mac nightly needs to be kicked off manually, but the Windows and Linux ones are fully automatic.hiran wrote: ↑Mon Apr 26, 2021 10:33 pmThank you for the offer. CI/CD is unexplored? But who would then sit every night to create https://github.com/OoliteProject/nightlies?
Let me have a smooth dive. As a first I will look at the OXP documentation.
Also, there is some basic CI activity in the project. Every commit and pull request sent to github automatically results in a build by Travis, which is configured to run on Xcode (example build log here). This has been for years our only way of ensuring that the changes we've been making did not affect the Mac builds in any negative way; we've been checking the Travis build logs after new commits.
- hiran
- Theorethicist
- Posts: 2415
- Joined: Fri Mar 26, 2021 1:39 pm
- Location: a parallel world I created for myself. Some call it a singularity...
Re: BigSur and Oolite
So the pipeline does exist. That is good news.another_commander wrote: ↑Tue Apr 27, 2021 5:20 amThe nightlies are built by automated scripts created by Getafix and there are at least two servers involved in their creation. Sometimes the Mac nightly needs to be kicked off manually, but the Windows and Linux ones are fully automatic.
Also, there is some basic CI activity in the project. Every commit and pull request sent to github automatically results in a build by Travis, which is configured to run on Xcode (example build log here). This has been for years our only way of ensuring that the changes we've been making did not affect the Mac builds in any negative way; we've been checking the Travis build logs after new commits.
All that might be required is a way to extend the tests to include 'test flights'. Those can be used as more extensive tests for special situations - as well as tests for plugins should plugin developers so wish.
This sounds like a lot less work than I had thought initially.
Sunshine - Moonlight - Good Times - Oolite
- hiran
- Theorethicist
- Posts: 2415
- Joined: Fri Mar 26, 2021 1:39 pm
- Location: a parallel world I created for myself. Some call it a singularity...
Re: BigSur and Oolite
Cmdr James wrote: ↑Sat Apr 24, 2021 2:51 pmThere is a makefile.
I dont remember seeing anything I would really call unit tests but you can dig around in here: https://github.com/OoliteProject/oolite-tests/
I think you misunderestimate the complexity, but if you want to have a stab at it Im willing to support as best I can.
I think meanwhile I am more interested in looking at the pipeline.another_commander wrote: ↑Tue Apr 27, 2021 5:20 amThe nightlies are built by automated scripts created by Getafix and there are at least two servers involved in their creation. Sometimes the Mac nightly needs to be kicked off manually, but the Windows and Linux ones are fully automatic.hiran wrote: ↑Mon Apr 26, 2021 10:33 pmThank you for the offer. CI/CD is unexplored? But who would then sit every night to create https://github.com/OoliteProject/nightlies?
Let me have a smooth dive. As a first I will look at the OXP documentation.
Also, there is some basic CI activity in the project. Every commit and pull request sent to github automatically results in a build by Travis, which is configured to run on Xcode (example build log here). This has been for years our only way of ensuring that the changes we've been making did not affect the Mac builds in any negative way; we've been checking the Travis build logs after new commits.
It seems there are at least three repositories involved. And so far I do not see any Github Actions anywhere. So how does it run in detail?
Sunshine - Moonlight - Good Times - Oolite