[Beta] Release of Telescope 2.0

[MrFlibble]

A specific version of python can be pleasantly and safely added via pyenv.

That will apparently work on mac, and certainly need not touch the system-wide versions (which can get messy). Multiple python versions can co-exist and are easily leveraged per-run with pyenv. Simply avoid using 'global'.

If doing anything which requires pip installs with python, I'd recommend using a venv (python virtual environment) so that any 'pip' installed stuff doesn't need to pollute the normal user environment. Using pyenv and a venv allows maximum separation from the bonds of the system installed version. A script to launch the 'other version' application can be trivially conjured once the venv is set up. I posted such a method in the DebugConsole thread. I understand you don't need such a workaround for DebugConsole on the Mac, however the method can be adapted to easily run isolated envs with any version of python.

Simplest case with pyenv. If trying to run something that needs no further pip installs, (which I believe applies to the script in telescope), and you have pyenv installed properly, you can just install the relevant version of python thus:

Code: Select all

pyenv install 3.10

Then, and in the future, this will set the python version for the current shell to 3.10.

Code: Select all

pyenv shell 3.10

Running python myscript.py in that shell will use 3.10. A new shell with still use system python. All remains normal.

[cag]

So many people are still using Python 2, that's why I've tried to make it all version agnostic. But if possible, collect_shipdata should be run in 3, all crashes aside.

Fun fact: Python cannot delete/overwrite members of a zip file and 2's shutil is missing those handy features. IIRC, I got it working w/ just the zipfile module but since you have to rebuild a brand new zip, all the files got touched (filestamps updated), hence the switch to shutil.

If you're using Python 2, stop, just stop. MrFlibble showed you how to have both installed safely. If you have some cherished scripts in 2, 3 has a 2to3 tool that will get you most of the way there. Python 2 is over 2 decades old and will soon be deprecated. I hear Python 4 is in the works ...

[hiran]

Python 2 is over 2 decades old and will soon be deprecated. I hear Python 4 is in the works ...

Python 2 was meant to be abandoned in 2015 but due to a huge codebase was still kept until 2020. Plenty of time to migrate and already four years ago.
Let it rest in peace. Even 3.7 left the support cycle.

Source: https://en.wikipedia.org/wiki/Python_(p ... e)#History

[cag]

Attempting to use the collect_shipdata.py script. Totally ignorant as to whether it'll make any difference to the "no problem" I have.

What, pray tell, is your "no problem"? I've got a new script just for you:
https://www.dropbox.com/scl/fi/7f5hi7jf ... 8d50q&dl=0
Shall we see what surprises Linus has in store? Should work on either Py (or maybe just one).

No harm is done. No files are changed.

Given our recent discussion of files and where they end up, be advised that this too will save a log file and backup copy of your Telescope oxz in its launch folder. The reason for the latter is I'm inserting the new shipdata.plist into the Telescope oxz!

[MrFlibble]

Attempting to use the collect_shipdata.py script. Totally ignorant as to whether it'll make any difference to the "no problem" I have.

What, pray tell, is your "no problem"? I've got a new script just for you:
https://www.dropbox.com/scl/fi/7f5hi7jf ... 8d50q&dl=0
Shall we see what surprises Linus has in store? Should work on either Py (or maybe just one).

I'm sorry if I came across any way other than jovial. I'd revisited the telescope documentation and decided to try out 3D views, and given my heap of OXPs, getting the python script supplied with telescope to run seemed an appropriate first step.

So at that time, I literally had no problem, and was relatively ignorant as to the effect the experience would have on that.

Of course now I've run it, and switched on the 3D views in telescope. All the pieces fit. I love it, and it's made my "no problem" a lot nicer. Thank you!

I shall look at the uploaded script with interest. Carefully!

No harm is done. No files are changed.

Given our recent discussion of files and where they end up, be advised that this too will save a log file and backup copy of your Telescope oxz in its launch folder. The reason for the latter is I'm inserting the new shipdata.plist into the Telescope oxz!
[/quote]

I hear you. Thanks. I noticed that when it ran right, so I've already popped it in a folder, with a mental note to only run it from there.

Right.. instead of sleep, I've got a python script to look for fangs in... fun!

[Cholmondely]

I'm sorry if I came across any way other than jovial. I'd revisited the telescope documentation and decided to try out 3D views, and given my heap of OXPs, getting the python script supplied with telescope to run seemed an appropriate first step.

So at that time, I literally had no problem, and was relatively ignorant as to the effect the experience would have on that.

Of course now I've run it, and switched on the 3D views in telescope. All the pieces fit. I love it, and it's made my "no problem" a lot nicer. Thank you!

If these 3D views are different from what the rest of us get, would there be any chance (eventually) of a screenshot for the Telescope page on our wiki? You could always upload it to the wiki itself if you have nowhere else to put it.

[MrFlibble]

I've got a new script just for you:
https://www.dropbox.com/scl/fi/7f5hi7jf ... 8d50q&dl=0

That worked using python3.10. All the file timestamps stayed intact bar the obvious one. Directory timestamps not though.

Different story with python 2.7.18.

It started ok, and got to here:-

Code: Select all

('Launched in:', '/tmp')
main, found all 3 Oolite folders
  appDir: /home/user/GNUstep/Applications/Oolite/oolite.app/Resources
   aoDir: /home/user/GNUstep/Applications/Oolite/AddOns
  maoDir: /home/user/GNUstep/Library/ApplicationSupport/Oolite/ManagedAddOns

  folder /home/user/GNUstep/Applications/Oolite/oolite.app/Resources
  folder /home/user/GNUstep/Library/ApplicationSupport/Oolite/ManagedAddOns
  folder /home/user/GNUstep/Applications/Oolite/AddOns
  folder /home/user/GNUstep/Applications/Oolite/AddOns/Basic-debug.oxp
  folder /home/user/GNUstep/Applications/Oolite/AddOns/PlanetOceans.oxp
  folder /home/user/GNUstep/Applications/Oolite/AddOns/PlanetForests.oxp
  folder /home/user/GNUstep/Applications/Oolite/AddOns/Asteroids3D1.2.oxp

Collected 2211 ents from 72 files, output is 785245

ouput_update, 'module' object has no attribute 'remove'
Traceback (most recent call last):
  File "collect_shipdataMrFlibble.py", line 2152, in ouput_update
    os.path.remove(previous)
AttributeError: 'module' object has no attribute 'remove'
--Return--
> /tmp/collect_shipdataMrFlibble.py(2181)ouput_update()->None
-> pdb.set_trace()
(Pdb).

I hit ctrl-c, and it spouted more on the way out:

Code: Select all

Traceback (most recent call last):
  File "collect_shipdataMrFlibble.py", line 2461, in <module>
    sys.exit(1 if main() else 0)
  File "collect_shipdataMrFlibble.py", line 2442, in main
    err, update = update_eff_data()               
  File "collect_shipdataMrFlibble.py", line 2241, in update_eff_data
    fallback = ouput_update(output)               
  File "collect_shipdataMrFlibble.py", line 2181, in ouput_update
    pdb.set_trace()
  File "/home/user/.pyenv/versions/2.7.18/lib/python2.7/bdb.py", line 53, in trace_dispatch
    return self.dispatch_return(frame, arg)
  File "/home/user/.pyenv/versions/2.7.18/lib/python2.7/bdb.py", line 88, in dispatch_return
    self.user_return(frame, arg)
  File "/home/user/.pyenv/versions/2.7.18/lib/python2.7/pdb.py", line 190, in user_return
    self.interaction(frame, None)
  File "/home/user/.pyenv/versions/2.7.18/lib/python2.7/pdb.py", line 210, in interaction
    self.cmdloop()
  File "/home/user/.pyenv/versions/2.7.18/lib/python2.7/cmd.py", line 130, in cmdloop
    line = raw_input(self.prompt)
KeyboardInterrupt

The log file ended this way

Code: Select all

Processed 72 files, new entities registered: 2211

  input:    shipdata   1,719,031

 output:    required      35,730
            generated    749,515
            total        785,245
ouput_update, 'module' object has no attribute 'remove'

[cag]

If these 3D views are different from what the rest of us get, would there be any chance (eventually) of a screenshot for the Telescope page on our wiki? You could always upload it to the wiki itself if you have nowhere else to put it.

These are not new and are present in some of the wiki's screenshots. They usually appear at the top/center of the HUD (can be repositioned) and show your currently targeted ship. The '3D' comes from the fact that if said target is within scanner range, the view shows the targets orientation in real-time. (in a dog fight, you can see how it's turning even when it's too far to make out otherwise)

However, as the Telescope oxp cannot know if you've installed any extra ships, it can only display the ships that come with Oolite. If you're targetting a ship from another oxp, Telescope will either show a big yellow question mark or nothing (this toggle is in Telescope's options).

What this Python script does is scan all your oxps and builds a custom effectdata.plist so all your oxp ships will appear correctly in the 3D view.

[hiran]

What this Python script does is scan all your oxps and builds a custom effectdata.plist so all your oxp ships will appear correctly in the 3D view.

Whoops! Do I understand correctly that before starting Oolite, it is advisable to run a python script (especially if we don't know whether such a script has run since the last time OXPs were installed/uninstalled)?

Is this a special pattern for Telescope or are there more OXPs that introduce such 'technology'?

[cag]

Whoops! Do I understand correctly that before starting Oolite, it is advisable to run a python script (especially if we don't know whether such a script has run since the last time OXPs were installed/uninstalled)?

IF you've installed some fancy non-standard ships, IF you use telescope's 3D view and IF you want to see said fancy ships instead of the big yellow ?, then yes. If you install a new ship, run it once and your done. Not necessary if you uninstall the ship.

Is this a special pattern for Telescope or are there more OXPs that introduce such 'technology'?

The 3D view is a visual effect (effectdata.plist) based on shipdata.plist; these are separate silos in Oolite code. The big yellow ? was Norby's solution and until 2.0, the only option was to not display it. That can be immersion breaking but so can showing nothing where a ship should be. This was the best solution I could come up with at the time.

AFAIK, this is the only OXP that has a Python script.

[cag]

Directory timestamps not though.

Fixed - that's what I get trusting someone else's code in the wee hours. My generated OZXs had blank timestamps on the directories.

Different story with python 2.7.18.

Only because you ran the 3 version first. Both had the error but you didn't have a file to delete at first. A Python quirk work remembering is that checking a file's existence is os.path.exists() but deleting said file is os.remove() or os.unlink()

I hit ctrl-c, and it spouted more on the way out:

ctrl-c is not a valid exit from the pdb debugger. I don't know there is one!

Hopefully, this wersion will be it:
https://www.dropbox.com/scl/fi/7f5hi7jf ... 8d50q&dl=0

[MrFlibble]

Directory timestamps not though.

Fixed - that's what I get trusting someone else's code in the wee hours. My generated OZXs had blank timestamps on the directories.

Different story with python 2.7.18.

Only because you ran the 3 version first. Both had the error but you didn't have a file to delete at first. A Python quirk work remembering is that checking a file's existence is os.path.exists() but deleting said file is os.remove() or os.unlink()

So this time I ran the 2.7 first. Cleared out obvious relevant cruft on a fresh boot. Still b0rken.

The 3.10 still has "right now" timestamps on the dirs. Files are proper.

2.7 spew:

Up to pdb:

Code: Select all

python2 collect_shipdataMrFlibble.py
('Launched in:', '/tmp')
main, found all 3 Oolite folders
  appDir: /home/user/GNUstep/Applications/Oolite/oolite.app/Resources
   aoDir: /home/user/GNUstep/Applications/Oolite/AddOns
  maoDir: /home/user/GNUstep/Library/ApplicationSupport/Oolite/ManagedAddOns

  folder /home/user/GNUstep/Applications/Oolite/oolite.app/Resources
  folder /home/user/GNUstep/Library/ApplicationSupport/Oolite/ManagedAddOns
  folder /home/user/GNUstep/Applications/Oolite/AddOns
  folder /home/user/GNUstep/Applications/Oolite/AddOns/Basic-debug.oxp
  folder /home/user/GNUstep/Applications/Oolite/AddOns/PlanetOceans.oxp
  folder /home/user/GNUstep/Applications/Oolite/AddOns/PlanetForests.oxp
  folder /home/user/GNUstep/Applications/Oolite/AddOns/Asteroids3D1.2.oxp

Collected 2211 ents from 72 files, output is 785245

ouput_update, 'module' object has no attribute 'remove'
Traceback (most recent call last):
  File "collect_shipdataMrFlibble.py", line 2152, in ouput_update
    os.path.remove(previous)
AttributeError: 'module' object has no attribute 'remove'
--Return--
> /tmp/collect_shipdataMrFlibble.py(2181)ouput_update()->None
-> pdb.set_trace()
(Pdb).

From pdb break.

Code: Select all

Traceback (most recent call last):
  File "collect_shipdataMrFlibble.py", line 2461, in <module>
    sys.exit(1 if main() else 0)
  File "collect_shipdataMrFlibble.py", line 2442, in main
    err, update = update_eff_data()               
  File "collect_shipdataMrFlibble.py", line 2241, in update_eff_data
    fallback = ouput_update(output)               
  File "collect_shipdataMrFlibble.py", line 2181, in ouput_update
    pdb.set_trace()
  File "/home/user/.pyenv/versions/2.7.18/lib/python2.7/bdb.py", line 53, in trace_dispatch
    return self.dispatch_return(frame, arg)
  File "/home/user/.pyenv/versions/2.7.18/lib/python2.7/bdb.py", line 88, in dispatch_return
    self.user_return(frame, arg)
  File "/home/user/.pyenv/versions/2.7.18/lib/python2.7/pdb.py", line 190, in user_return
    self.interaction(frame, None)
  File "/home/user/.pyenv/versions/2.7.18/lib/python2.7/pdb.py", line 210, in interaction
    self.cmdloop()
  File "/home/user/.pyenv/versions/2.7.18/lib/python2.7/cmd.py", line 130, in cmdloop
    line = raw_input(self.prompt)
KeyboardInterrupt

Potentially useful end of collect_shipdata.log

Code: Select all

  input:    shipdata   1,719,031 output:    required      35,730
            generated    749,515
            total        785,245
ouput_update, 'module' object has no attribute 'remove'

ctrl-c is not a valid exit from the pdb debugger. I don't know there is one!

Well it got me out of it this time and last. I recall I had a fight on the earlier versions, but persistence, and maybe a few angry ctrl-d's seemed to scare it into returning my shell.

Hopefully, this wersion will be it:
https://www.dropbox.com/scl/fi/7f5hi7jf ... 8d50q&dl=0

Alas.. 'twas not.

[cag]

D'oh!

was the same file as before!

Once more from the top:

https://www.dropbox.com/scl/fi/7f5hi7jf ... 8d50q&dl=0

[hiran]

Whoops! Do I understand correctly that before starting Oolite, it is advisable to run a python script (especially if we don't know whether such a script has run since the last time OXPs were installed/uninstalled)?

IF you've installed some fancy non-standard ships, IF you use telescope's 3D view and IF you want to see said fancy ships instead of the big yellow ?, then yes. If you install a new ship, run it once and your done. Not necessary if you uninstall the ship.

Understood. I agree it is the best solution however there is a caveat.

Is this a special pattern for Telescope or are there more OXPs that introduce such 'technology'?

The 3D view is a visual effect (effectdata.plist) based on shipdata.plist; these are separate silos in Oolite code. The big yellow ? was Norby's solution and until 2.0, the only option was to not display it. That can be immersion breaking but so can showing nothing where a ship should be. This was the best solution I could come up with at the time.

AFAIK, this is the only OXP that has a Python script.

The caveat I am looking at is how users in general can handle this. I created OoliteStarter to make installing and running Oolite as easy as possible.

This OXP is among my installed expansions as well, and I like looking at remote ships. That indeed is valuable information.
What I dislike is the dependency of having to scan for shipdata.plist at all. But let's assume Oolite does not offer anything the expansion could build on at runtime and we need this additional step. I likely could get OoliteStarter to execute that script before running Oolite. But...

How does OoliteStarter know a command needs to be run, and which one?
Have we ensured there is a suitable Python version installed? AFAIK Oolite does not have any dependency on Python other than the Telescope expansion.

[MrFlibble]

Whoops! Do I understand correctly that before starting Oolite, it is advisable to run a python script (especially if we don't know whether such a script has run since the last time OXPs were installed/uninstalled)?

IF you've installed some fancy non-standard ships, IF you use telescope's 3D view and IF you want to see said fancy ships instead of the big yellow ?, then yes. If you install a new ship, run it once and your done. Not necessary if you uninstall the ship.

Understood. I agree it is the best solution however there is a caveat.

Is this a special pattern for Telescope or are there more OXPs that introduce such 'technology'?

The 3D view is a visual effect (effectdata.plist) based on shipdata.plist; these are separate silos in Oolite code. The big yellow ? was Norby's solution and until 2.0, the only option was to not display it. That can be immersion breaking but so can showing nothing where a ship should be. This was the best solution I could come up with at the time.

AFAIK, this is the only OXP that has a Python script.

The caveat I am looking at is how users in general can handle this. I created OoliteStarter to make installing and running Oolite as easy as possible.

This OXP is among my installed expansions as well, and I like looking at remote ships. That indeed is valuable information.
What I dislike is the dependency of having to scan for shipdata.plist at all. But let's assume Oolite does not offer anything the expansion could build on at runtime and we need this additional step. I likely could get OoliteStarter to execute that script before running Oolite. But...

How does OoliteStarter know a command needs to be run, and which one?
Have we ensured there is a suitable Python version installed? AFAIK Oolite does not have any dependency on Python other than the Telescope expansion.

Perhaps starter could keep a list of OXP's at game start. Before launch, if it has any new OXP's that have flying entities, then run the ship scraper.

External dependency might become a useful (though potentially hazardous/chaotic) means to extend Oolite OXPs capabilities beyond those of the core capabilities.

Regarding python versions. That may become less of an issue if the executables we're playing with in DebuxOxp2 become a solid thing, though I suppose OoliteStarter could probe for python, and decide whether to pull in the large binary, or lighter scripts.

Right. Fire up victim computer and test latest ship scraper version time.