Oolite Bulletins

hiran wrote: ↑Sun Apr 21, 2024 6:17 am

MrFlibble wrote: ↑Sun Apr 21, 2024 1:01 am

I made a self-contained version with pyinstaller on Linux. Doddle. Should be fairly easy to churn out static versions for most OSes between us so the non-pythoners won't have to get involved.

Would like to know more details on the Doddle part.

#python3-tk is needed.
# the packaged twisted works to run python scripts but fails
# with pyinstaller. If only building the binary, then no
# need to add it here.

apt install python3-tk python3-twisted

venv="$HOME/python3/pyinstaller"
python3 -m venv "$venv"
source "${venv}"/bin/activate
pip install pyinstaller
pip install twisted
deactivate

#!/bin/bash
venv="$HOME/python3/pyinstaller"
source "${venv}"/bin/activate
pyinstaller $*

pyinstaller.sh --onefile DebugConsole.py --add-binary "oojsc.xbm:."  --add-binary "OoJSC.ico:."

pyinstaller.sh --clean --onefile DebugConsole.py --add-binary "oojsc.xbm:."  --add-binary "OoJSC.ico:."

MrFlibble wrote: ↑Sun Apr 21, 2024 7:26 am

Using the system python 3.10, and not wanting to mess with my local setup by shoving things in willy-nilly with pip, I created a venv in a directory where I keep a heap of them. Activated it, and pip installed pyinstaller only in the venv.

Code: Select all

venv="$HOME/python3/pyinstaller"
python3 -m venv "$venv"
source "${venv}"/bin/activate
pip install pyinstaller
deactivate

Made a tiny script "$HOME/bin/pyinstaller.sh" which activates the venv and runs pyinstaller, passing any arguments from the command line through.

Code: Select all

#!/bin/bash
venv="$HOME/python3/pyinstaller"
source "${venv}"/bin/activate
pyinstaller $*

chmod 755 "$HOME/bin/pyinstaller.sh" #to make it executable

As $HOME/bin is in my path, I can now run pyinstaller.sh from a shell anywhere.

At this point, I can go for it, and with a minimum of swearing and demanding more coffee, I had myself a single executable and a workflow of sorts.

In a clean copy of the DebugConsole2 folder, I put this in a file called makebin.sh

Code: Select all

pyinstaller.sh --onefile DebugConsole.py --add-binary "oojsc.xbm:."  --add-binary "OoJSC.ico:."

From this point forth: . makebin.sh gets it done in one.

...then look in the newly created 'dist' folder.

There are complications where files are included, like for icons etc, but it looks like cag has done most of the work to take care of that.

For now, my version is without an icon on Linux, as the way I've found to get a nice icon working in Linux is not obviously easy to get working properly with pyinstaller. One of us can polish that another day.

$ ~/.local/bin/pyinstaller --onefile DebugConsole.py --add-binary "oojsc.xbm:." --add-binary "OoJSC.ico:." 
108 INFO: PyInstaller: 6.6.0, contrib hooks: 2024.4
108 INFO: Python: 3.10.12
109 INFO: Platform: Linux-6.5.0-28-generic-x86_64-with-glibc2.35
109 INFO: wrote .../oolite-debug-console/DebugConsole.spec
110 INFO: Extending PYTHONPATH with paths
['.../oolite-debug-console']
161 INFO: Appending 'binaries' from .spec
162 INFO: checking Analysis
163 INFO: checking PYZ
175 INFO: checking PKG
175 INFO: Bootloader $HOME/.local/lib/python3.10/site-packages/PyInstaller/bootloader/Linux-64bit-intel/run
175 INFO: checking EXE
175 INFO: Rebuilding EXE-00.toc because DebugConsole missing
175 INFO: Building EXE from EXE-00.toc
175 INFO: Copying bootloader EXE to .../oolite-debug-console/dist/DebugConsole
176 INFO: Appending PKG archive to custom ELF section in EXE
195 INFO: Building EXE from EXE-00.toc completed successfully.
$ 

$ ./DebugConsole 
Traceback (most recent call last):
  File "ooliteConsoleServer/__init__.py", line 16, in <module>
ModuleNotFoundError: No module named 'twisted'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "DebugConsole.py", line 41, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 419, in exec_module
  File "ooliteConsoleServer/__init__.py", line 18, in <module>
ImportError: ooliteConsoleServer requires Twisted (http://twistedmatrix.com/)
[19709] Failed to execute script 'DebugConsole' due to unhandled exception!
$ 

Cholmondely wrote: ↑Sun Apr 21, 2024 6:52 am

1) How many of our OXPS actually use Python?
Is it just Telescope (+Telescope Options?) and the Debug Console - or are there oodles of them?

Cholmondely wrote: ↑Sun Apr 21, 2024 6:52 am

2) Note: I've not had any problems whatsoever with my 2.7 that I've noticed.

MrFlibble wrote: ↑Sat Apr 20, 2024 9:27 pm

... exactly ...what ... ?

hiran wrote: ↑Sun Apr 21, 2024 7:55 am

MrFlibble wrote: ↑Sun Apr 21, 2024 7:26 am

Using the system python 3.10, and not wanting to mess with my local setup by shoving things in willy-nilly with pip, I created a venv in a directory where I keep a heap of them. Activated it, and pip installed pyinstaller only in the venv.

Code: Select all

venv="$HOME/python3/pyinstaller"
python3 -m venv "$venv"
source "${venv}"/bin/activate
pip install pyinstaller
deactivate

Made a tiny script "$HOME/bin/pyinstaller.sh" which activates the venv and runs pyinstaller, passing any arguments from the command line through.

Code: Select all

#!/bin/bash
venv="$HOME/python3/pyinstaller"
source "${venv}"/bin/activate
pyinstaller $*

chmod 755 "$HOME/bin/pyinstaller.sh" #to make it executable

As $HOME/bin is in my path, I can now run pyinstaller.sh from a shell anywhere.

At this point, I can go for it, and with a minimum of swearing and demanding more coffee, I had myself a single executable and a workflow of sorts.

In a clean copy of the DebugConsole2 folder, I put this in a file called makebin.sh

Code: Select all

pyinstaller.sh --onefile DebugConsole.py --add-binary "oojsc.xbm:."  --add-binary "OoJSC.ico:."

From this point forth: . makebin.sh gets it done in one.

...then look in the newly created 'dist' folder.

There are complications where files are included, like for icons etc, but it looks like cag has done most of the work to take care of that.

For now, my version is without an icon on Linux, as the way I've found to get a nice icon working in Linux is not obviously easy to get working properly with pyinstaller. One of us can polish that another day.

Code: Select all

$ ~/.local/bin/pyinstaller --onefile DebugConsole.py --add-binary "oojsc.xbm:." --add-binary "OoJSC.ico:." 
108 INFO: PyInstaller: 6.6.0, contrib hooks: 2024.4
108 INFO: Python: 3.10.12
109 INFO: Platform: Linux-6.5.0-28-generic-x86_64-with-glibc2.35
109 INFO: wrote .../oolite-debug-console/DebugConsole.spec
110 INFO: Extending PYTHONPATH with paths
['.../oolite-debug-console']
161 INFO: Appending 'binaries' from .spec
162 INFO: checking Analysis
163 INFO: checking PYZ
175 INFO: checking PKG
175 INFO: Bootloader $HOME/.local/lib/python3.10/site-packages/PyInstaller/bootloader/Linux-64bit-intel/run
175 INFO: checking EXE
175 INFO: Rebuilding EXE-00.toc because DebugConsole missing
175 INFO: Building EXE from EXE-00.toc
175 INFO: Copying bootloader EXE to .../oolite-debug-console/dist/DebugConsole
176 INFO: Appending PKG archive to custom ELF section in EXE
195 INFO: Building EXE from EXE-00.toc completed successfully.
$ 

Right after that I went to the dist directory and execute

Code: Select all

$ ./DebugConsole 
Traceback (most recent call last):
  File "ooliteConsoleServer/__init__.py", line 16, in <module>
ModuleNotFoundError: No module named 'twisted'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "DebugConsole.py", line 41, in <module>
  File "PyInstaller/loader/pyimod02_importers.py", line 419, in exec_module
  File "ooliteConsoleServer/__init__.py", line 18, in <module>
ImportError: ooliteConsoleServer requires Twisted (http://twistedmatrix.com/)
[19709] Failed to execute script 'DebugConsole' due to unhandled exception!
$ 

I guess this is since for non-python developers the computer is not setup like yours. We need to figure out these bits for a successful build on Github.

apt install python3-tk python3-twisted

hiran wrote: ↑Sun Apr 21, 2024 9:04 am

Code: Select all

#!/bin/bash
venv="$HOME/python3/pyinstaller"
python3 -m venv "$venv"
source "${venv}"/bin/activate
pip install pyinstaller
pip install twisted
$HOME/.local/bin/pyinstaller --onefile DebugConsole.py --add-binary "oojsc.xbm:."  --add-binary "OoJSC.ico:."
deactivate

hiran wrote: ↑Sun Apr 21, 2024 9:04 am

Looks to me like this is the set of instructions:

venv="$HOME/python3/pyinstaller"
python3 -m venv "$venv"
source "${venv}"/bin/activate
pip install pyinstaller
pip install twisted
#We're in the venv, so can call pyinstaller directly.
pyinstaller --onefile DebugConsole.py --add-binary "oojsc.xbm:."  --add-binary "OoJSC.ico:."
deactivate # if we're in a bash script you can leave this line out. The venv is auto shut down on the way out.

venv="$HOME/python3/pyinstaller"
# rm -Rf "$venv"  # uncomment to make fresh venv.
python3 -m venv "$venv"
source "${venv}"/bin/activate
pip install pyinstaller
pip install twisted
deactivate #only needed in an interactive shell. If a bash script, leave it out.

#!/bin/bash
venv="$HOME/python3/pyinstaller"
source "${venv}"/bin/activate
pyinstaller $*

pyinstaller.sh --onefile DebugConsole.py --add-binary "oojsc.xbm:."  --add-binary "OoJSC.ico:."

pyinstaller.sh --clean --onefile DebugConsole.py --add-binary "oojsc.xbm:."  --add-binary "OoJSC.ico:."

MrFlibble wrote: ↑Sun Apr 21, 2024 9:54 am

hiran wrote: ↑Sun Apr 21, 2024 9:04 am

Code: Select all

#!/bin/bash
venv="$HOME/python3/pyinstaller"
python3 -m venv "$venv"
source "${venv}"/bin/activate
pip install pyinstaller
pip install twisted
$HOME/.local/bin/pyinstaller --onefile DebugConsole.py --add-binary "oojsc.xbm:."  --add-binary "OoJSC.ico:."
deactivate

Is $HOME/.local/bin/pyinstaller a wrapper script? is that also starting the venv we're already in, or is it a symlink to inside the venv?

#!/usr/bin/python3
# -*- coding: utf-8 -*-
import re
import sys
from PyInstaller.__main__ import _console_script_run
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(_console_script_run())

MrFlibble wrote: ↑Sun Apr 21, 2024 9:54 am

Assuming a fresh home directory every time, forget making the .sh files and the first example below might serve.

MrFlibble wrote: ↑Sun Apr 21, 2024 9:54 am

Do remember to apt install python3-tk python3-venv python3-pip. Not knowing what bits of your build system are persistent makes this a tricky target for me.

apt install python3-tk python3-venv python3-pip
venv="$HOME/python3/pyinstaller"
python3 -m venv "$venv"
source "${venv}"/bin/activate
pip install pyinstaller
pip install twisted
#We're in the venv, so can call pyinstaller directly.
pyinstaller --onefile DebugConsole.py --add-binary "oojsc.xbm:."  --add-binary "OoJSC.ico:."
deactivate # if we're in a bash script you can leave this line out. The venv is auto shut down on the way out.

MrFlibble wrote: ↑Sun Apr 21, 2024 9:54 am

That is not how I'd do it unless I had good reason to redo from scratch every time.

If troubleshooting, or building multiple binaries from different scripts it might be better to separate that out more like the rest of this post.

This bit to (re)make the venv and pyinstaller. Can be left a while without refresh. Possible to enter the venv and update with pip if wanted.

Code: Select all

venv="$HOME/python3/pyinstaller"
# rm -Rf "$venv"  # uncomment to make fresh venv.
python3 -m venv "$venv"
source "${venv}"/bin/activate
pip install pyinstaller
pip install twisted
deactivate #only needed in an interactive shell. If a bash script, leave it out.

This as a stanza to run pyinstaller in its venv. Suggest as pyinstaller.sh somewhere in the $PATH
The script can be left alone if rebuilding any other part of the setup.

Code: Select all

#!/bin/bash
venv="$HOME/python3/pyinstaller"
source "${venv}"/bin/activate
pyinstaller $*

Then this to build within the clean dist dir. Assumes pwd is the distdir.

Code: Select all

pyinstaller.sh --onefile DebugConsole.py --add-binary "oojsc.xbm:."  --add-binary "OoJSC.ico:."

Or this to rebuild, deleting pyinstall caches etc. Assumes pwd is the distdir.

Code: Select all

pyinstaller.sh --clean --onefile DebugConsole.py --add-binary "oojsc.xbm:."  --add-binary "OoJSC.ico:."

I'll try from scratch in a vm later if you're still having issues. What base system are you using?

hiran wrote: ↑Sun Apr 21, 2024 10:09 am

$HOME/.local/bin/pyinstaller

That is the location I saw printed on the screen when executing 'pip install pyinstaller'. I checked and it is a python script with this content:

MrFlibble wrote: ↑Sat Apr 20, 2024 9:27 pm

Cholmondely wrote: ↑Sun Apr 21, 2024 6:52 am

Would a log-in for our wiki be of any use?

Yes please.

hiran wrote: ↑Sun Apr 21, 2024 10:09 am

I think we are good that I try this on Github. At home I am running Ubuntu Desktop 22.04. On Github we also have Ubuntu 22.04 (I believe they strip off the desktop).

#!/bin/bash

q(){ echo "Error at: $*" >&2 ; exit 5 ; } # exit function

basedir="$HOME/OoDC" # the zipfile is expected to be in here
builddir="$basedir/build" # this can be deleted to retry a build
basename=OoliteDebugConsole # gets used a lot
inscript=DebugConsole.py # the name of the script we'll be 'compiling'
zipfile="$basedir/${basename}-latest.zip"
venv="$builddir/pyinstaller"
dist="${basename}2.03" # the directory exracted from zip

step="package install" # might want to do this elsewhere so as to avoid login prompt.
sudo apt install python3-tk python3-pip python3-venv binutils || q "$step"

step="setting up venv and building tools"
python3 -m venv "$venv" &&
source "$venv"/bin/activate &&
pip install twisted &&
pip install pyinstaller || q "$step"

step="preparing dist for build"
cd "$builddir" &&
unzip "$zipfile" &&
cd "$dist" || q "$step"

step="making executable"
pyinstaller --name "$basename" --onefile "$inscript" --add-binary "oojsc.xbm:."  --add-binary "OoJSC.ico:." &&
cd "$basedir" && mv "$builddir/$dist/dist/$basename" . &&
echo "Finibasedir. Your file is at ''$basedir/$basename'" || q "$step"

#end

Finished. Your file is at ''/home/user/OoDC/OoliteDebugConsole'

hiran wrote: ↑Sun Apr 21, 2024 5:46 pm

And we may look into building for other platforms as well.

pyinstaller ^
    --noconfirm ^
    --clean ^
    --log-level=WARN ^
    --paths "C:\Program Files (x86)\Windows Kits\10\Redist\ucrt\DLLs\x86"  ^
    --onefile ^
    --noconsole ^
    --noupx ^
    --name OoDebugConsole ^
    --icon=OoliteDebugConsole\OoJSC.ico ^
    --add-binary "OoliteDebugConsole\OoJSC.ico:." ^
    --add-binary "OoliteDebugConsole\oojsc.xbm:." ^
    --version-file=OoDebug_version_info.txt ^
    DebugConsole.py

# UTF-8
#
# For more details about fixed file info 'ffi' see:
# http://msdn.microsoft.com/en-us/library/ms646997.aspx
VSVersionInfo(
  ffi=FixedFileInfo(
    # filevers and prodvers should be always a tuple with four items: (1, 2, 3, 4)
    # Set not needed items to zero 0.
    filevers=(2, 0, 3, 0),
    prodvers=(2, 0, 3, 0),
    # Contains a bitmask that specifies the valid bits 'flags'r
    mask=0x3f,
    # Contains a bitmask that specifies the Boolean attributes of the file.
    flags=0x0,
    # The operating system for which this file was designed.
    # 0x4 - NT and there is no need to change it.
    OS=0x40004,
    # The general type of file.
    # 0x1 - the file is an application.
    fileType=0x1,
    # The function of the file.
    # 0x0 - the function is not defined for this fileType
    subtype=0x0,
    # Creation date and time stamp.
    date=(0, 0)
    ),
  kids=[
    StringFileInfo(
      [
      StringTable(
        u'040904B0',
        [StringStruct(u'CompanyName', u'Jens Ayton, Kaks, cag, MrFlibble'),
        StringStruct(u'FileDescription', u'Debug console for Oolite'),
        StringStruct(u'FileVersion', u'2.0.3.0'),
        StringStruct(u'InternalName', u'DebugConsole'),
        StringStruct(u'LegalCopyright', u'CC BY-NC-SA 4.0'),
        StringStruct(u'OriginalFilename', u'OoDebugConsole.exe'),
        StringStruct(u'ProductName', u'OoDebugConsole'),
        StringStruct(u'ProductVersion', u'2.0.3.0')])
      ]), 
    VarFileInfo([VarStruct(u'Translation', [1033, 1200])])
  ]
)

cag wrote: ↑Mon Apr 22, 2024 2:17 am

here's what I used on my last local Windows build, over 2 years ago.

Code: Select all

pyinstaller ^
    --noconfirm ^
    --clean ^
    --log-level=WARN ^
    --paths "C:\Program Files (x86)\Windows Kits\10\Redist\ucrt\DLLs\x86"  ^
    --onefile ^
    --noconsole ^
    --noupx ^
    --name OoDebugConsole ^
    --icon=OoliteDebugConsole\OoJSC.ico ^
    --add-binary "OoliteDebugConsole\OoJSC.ico:." ^
    --add-binary "OoliteDebugConsole\oojsc.xbm:." ^
    --version-file=OoDebug_version_info.txt ^
    DebugConsole.py

cag wrote: ↑Mon Apr 22, 2024 2:17 am

--paths is for the Visual C++ run-time .dlls (3 folders: x86, x64, arm).

You can get the 'Windows Software Development Kit (SDK) for Windows 10' at
[...]

cag wrote: ↑Mon Apr 22, 2024 2:17 am

--icon works for Window & MacOS but not Linux

[Desktop Entry]
Type=Application
Name=Oolite DebugConsole 2
GenericName=Oolite DebugConsole 2
Comment=Debugger for Oolite
Exec=/home/user/OoDC/DebugConsole
Icon=/home/user/OoDC/OoJSC128x128.png
Terminal=false
Categories=Application

	iconFile = PhotoImage(file='OoJSC256x256.png')
	self.top.wm_iconphoto(True, iconFile)