Page 1 of 2

Quick batch png convertion (fix texture issues and warnings)

Posted: Sat Jun 22, 2013 9:44 am
by Tichy
This is for those who have libpng >= 1.6 (At the moment, ArchLinux users and maybe users of some other rolling-release distro. But it could appear in other distributions, as soon as libpng will be updated.)
In this version libpng is more strict about iccp. In Oolite, the result is a lot of warnings in the logfile about a "known incorrect sRGB profile" of many PNG textures.
Like this

Code: Select all

16:38:46.959 [texture.load.png.warning]: ----- A PNG loading warning occurred for /home/stefano/.Oolite/AddOns/ships.oxp/neolite.oxp/Textures/alien-coriolis.png: iCCP: known incorrect sRGB profile.
This will also fix other kinds of problems with PNGs textures not loading.
Like this

Code: Select all

16:38:46.960 [texture.load.png.error]: ***** A PNG loading error occurred for /home/stefano/.Oolite/AddOns/ships.oxp/neolite.oxp/Textures/alien-coriolis.png: bad adaptive filter value.
Image

Note: I discovered that, for some reason, the conversion breaks some of the replacement fonts. Backup your oolite-font.png or New-font.png before running the batch conversion command.

For this to work you need imagemagick.

First, do a backup of your AddOns directory

Code: Select all

cd ~/.Oolite
cp -a AddOns AddOns-backup
Then, in your AddOns directory, run this commad that will recursively find all your *.png and *.PNG files and reconvert them.

Code: Select all

find -iname "*.png" -exec convert {} {} \;
It could take some minutes and will print some warnings when it finds images with incorrect PNG profile.

Run Oolite and verify if all is working correctly. If not, you have the backup ;)

Re: Quick batch png convertion (fix texture issues and warni

Posted: Sun Jun 23, 2013 11:46 pm
by Diziet Sma
Thanks.. will keep an eye out for when that update comes through.. 8)

Re: Quick batch png convertion (fix texture issues and warni

Posted: Sat Oct 19, 2013 10:49 pm
by CommRLock78
LAME.

Re: Quick batch png convertion (fix texture issues and warni

Posted: Thu Jul 31, 2014 11:03 am
by tinker
This fix works OK on OXP addons but not with OXZ addons, running the script in the Managed addons folder has no effect as OXZ files are identified as zipped files.

Re: Quick batch png convertion (fix texture issues and warni

Posted: Thu Jul 31, 2014 8:57 pm
by Getafix
@tinker
The Oolite binaries downloaded from oolite.org, link to libpng1.4 distributed with the Oolite installer.
Why would you care about the patch? Do you have any issues?

Re: Quick batch png convertion (fix texture issues and warni

Posted: Fri Aug 01, 2014 2:58 am
by Diziet Sma
Getafix wrote:
The Oolite binaries downloaded from oolite.org, link to libpng1.4 distributed with the Oolite installer.
Why would you care about the patch? Do you have any issues?
The standard Oolite binaries will not run on Arch. (Yes, I've tried.) Neither will the nightly builds.

Arch Linux uses a rolling release model, where the kernel, all programs, and libraries, are always the latest version (think Gentoo Linux, but considerably less painful). I'd describe Arch as cutting edge, (but not bleeding edge, as all packages are well tested before release). See the link for details.

Instead, Oolite must be specially patched, compiled and installed via the Arch package manager, pacman. (All packages on Arch are compiled and installed by pacman, which downloads the source tarballs, compiles them on the target machine, then installs them). As such, Oolite on Arch is always linked to the very latest versions of all the libraries it requires.

Whilst Oolite has an Arch maintainer, naturally, he doesn't have time to create and test a PKGBUILD script (which downloads the source, patches if/as necessary, then compiles and installs the code) for every nightly build. Possibly it could be automated, but it might require some collaboration between the maintainer and someone such as yourself to get it up and running.

Arch really is a lot of fun, and with everything being compiled on your own machine, blazingly fast.. you really ought to have a play with it some time.


In case you're curious, this is the PKGBUILD script for the 64-bit version of Oolite 1.80. As you can see, there's only a little patching required:

Code: Select all

# $Id$
# Maintainer: Sven-Hendrik Haase <[email protected]>
# Contributor: Lone_Wolf <[email protected]>
pkgname=oolite
pkgver=1.80
pkgrel=1
pkgdesc="A space-sim based on the classic Elite"
arch=('i686' 'x86_64')
url="http://oolite-linux.berlios.de"
license=('GPL2')
depends=('gnustep-gui' 'sdl_mixer' 'glu' 'espeak' 'nspr' 'python2' 'icu' 'gnustep-base' 'openal')
makedepends=('gcc-objc' 'gnustep-make' 'curl' 'zip' 'mesa')
#options=(makeflags)
source=(oolite
        "https://github.com/OoliteProject/oolite/releases/download/1.80b/oolite-source-${pkgver}.tar.bz2")
md5sums=('b182976fade9b64a3b3702e2a8c72bf0'
         'ded6afaf83f97624122c2dd999319166')

build() {
  cd oolite-source-${pkgver}
  unset CPPFLAGS
  sed -i '/void png_error/d' src/Core/Materials/OOPNGTextureLoader.m
  sed -i "/ADDITIONAL_OBJCFLAGS/s/=/= -fobjc-exceptions/" GNUmakefile
  make -f Makefile release
}

package() {
  cd oolite-source-${pkgver}

  mkdir -p ${pkgdir}/usr/bin
  mkdir -p ${pkgdir}/usr/share/{oolite,applications,pixmaps,doc/oolite}

  cp -r oolite.app/* ${pkgdir}/usr/share/oolite/
#install shellscript and icon
  install -D -m755 ../oolite ${pkgdir}/usr/bin/oolite
  install -D -m644 installers/FreeDesktop/oolite-icon.png ${pkgdir}/usr/share/pixmaps/oolite-icon.png
# copy  .desktop file
  install -D -m644 installers/FreeDesktop/oolite.desktop ${pkgdir}/usr/share/applications/oolite.desktop
# documentation
  install -D -m644 Doc/AdviceForNewCommanders.pdf Doc/OoliteReadMe.pdf Doc/OoliteRS.pdf ${pkgdir}/usr/share/doc/oolite/
 }
# vim:set ts=2 sw=2 et:

Re: Quick batch png convertion (fix texture issues and warni

Posted: Fri Aug 01, 2014 8:27 am
by tinker
@ Getafix

As Diziet said, I need to use the Arch installer to compile Oolite on my box and being a rolling release everything is, in theory, the newest and latest version. I did try rolling back Libpng to an earlier version but that broke so many other things that it is not an option.

@ Diziet Sma

Having distro hopped for 10 years trying to find something that suited me I found Arch 5 years ago and stuck with it. While it may not be the easiest entry for some new linux users it is something I recommend everyone at least tries, not only is everything built for your set up and is fast but you get to learn a lot about how things work.

Re: Quick batch png convertion (fix texture issues and warni

Posted: Fri Aug 01, 2014 10:15 am
by Getafix
OK, point taken. I will try to put my hands on an Arch Linux virtual machine and see what is going on there.
This will take time, though, and as I understand Tinker cannot have Oolite 1.80 in the meantime. :(

Re: Quick batch png convertion (fix texture issues and warni

Posted: Fri Aug 01, 2014 12:35 pm
by Lone_Wolf
In case the problem can only be solved by falling back to libpng 1.4 , that version is available from Archlinux User Repository (AUR) , https://aur.archlinux.org/packages/libpng14/ .

It does look like it can be installed together with latest libpng, and ofcourse the oolite PKGBUILD will have to be adjusted to use libpng 1.4 instead of latest .

Re: Quick batch png convertion (fix texture issues and warni

Posted: Fri Aug 01, 2014 1:34 pm
by Diziet Sma
tinker wrote:
While it may not be the easiest entry for some new linux users it is something I recommend everyone at least tries, not only is everything built for your set up and is fast but you get to learn a lot about how things work.
Agreed.. the only distro that taught me more about the innards of Linux than Arch, was Linux From Scratch.

The other thing I must say, is that, when it comes to identifying and solving the technical reasons behind hardware or software problems with Linux, the depth of knowledge (and clarity of explanations) in the Arch community is second to none. If ever I'm searching for a solution to a problem and I see an Arch post or article in the google results, that will be the first place I'll turn to.

Re: Quick batch png convertion (fix texture issues and warni

Posted: Fri Aug 01, 2014 1:36 pm
by Diziet Sma
Getafix wrote:
I will try to put my hands on an Arch Linux virtual machine and see what is going on there.
I'm not sure you can "put your hands on an Arch VM".. even as a VM, you will probably have to build it yourself, using the Arch Install Disk. See the Beginner's Guide for details.
Getafix wrote:
as I understand Tinker cannot have Oolite 1.80 in the meantime. :(
Well.. 1.80 works on Arch, but at the moment, it can't use OXZs that contain .png files, and OXPs that contain .png's need fixing as Tichy describes above.

Re: Quick batch png convertion (fix texture issues and warni

Posted: Fri Aug 01, 2014 2:22 pm
by tinker
@ Getafix

I can use 1.80, just using oxp's there is no problem as the script corrects the png errors in the oxp's and in the programme, (trumbles.png is set as an rgb greyscale image.)

What I cannot do is use OXZ Addons, unless the png's are correctly identified, as I identify problem OXZ's I can replace them with OXP's, or extract the OXP from the OXZ, correct the errors and use it as an OXP. In fact it seems OXP's are better as there is more chance I will read the readme files that are hidden in the OXZ.

@ Lone_Wolf

I will look into installing both, I was told by someone that would not work as expected, but testing will solve that. Modifying the PKGBUILD to use the correct library is then trivial.

Re: Quick batch png convertion (fix texture issues and warni

Posted: Mon Aug 04, 2014 6:09 pm
by Getafix
Diziet Sma wrote:
Getafix wrote:
I will try to put my hands on an Arch Linux virtual machine and see what is going on there.
I'm not sure you can "put your hands on an Arch VM".. even as a VM, you will probably have to build it yourself...
That was exactly my intention. Whenever a commander has a problem, I usually create a virtual machine with his distro and try to replicate the issue.

@Tinker
As I am currently on vacation with a limited quota mobile broadband connection, I will start working on it sometime next week.
It will take some time though.

Re: Quick batch png convertion (fix texture issues and warni

Posted: Tue Aug 05, 2014 7:47 am
by Diziet Sma
Getafix wrote:
Diziet Sma wrote:
Getafix wrote:
I will try to put my hands on an Arch Linux virtual machine and see what is going on there.
I'm not sure you can "put your hands on an Arch VM".. even as a VM, you will probably have to build it yourself...
That was exactly my intention. Whenever a commander has a problem, I usually create a virtual machine with his distro and try to replicate the issue.
Ahh.. my mistake.. I thought you were referring to downloading and installing a pre-built VM.. sorry.

Re: Quick batch png convertion (fix texture issues and warni

Posted: Wed Aug 06, 2014 11:25 pm
by Lone_Wolf
another_commander wrote:
tinker wrote:
...and Arrch Linux devs seem to be of the opinion that it is a problem for Oolite to solve, not for them to require obsolete libraries.
I will have to take exception to that. Speaking for myself only, I definitely don't see this as a problem and nothing needs resolving. Oolite gets distributed with a set of libraries that have been extensively tested and are known to be working and, as I don't have all the time in the world to dedicate to development, I'd rather use whatever little of it there is to work on the game itself, rather than on the libraries that support it. At least doing it this way I know that when a bug pops up it's because I did something wrong in the game's code and not because the latest version of libpng or whatever other library may have a subtle problem somewhere. Rolling releases may be nice, but they do have disadvantages too and that's probably the biggest one of them.

This is my personal opinion only of course, but I think that doing it this way has served us quite well up to this point.
copied from https://bb.oolite.space/viewtopic.php?f= ... 90#p224868

Afaict tinker expresses his own opinion there.
He might have based it on archlinux bug #41428 .
However, Scimmia is just 1 of the arch developers, and not the archlinux oolite maintainer.
another_commander wrote:
Oolite gets distributed with a set of libraries that have been extensively tested and are known to be working
I've looked at the oolite 1.80 source package, especially the libpng14 stuff and found this :

Code: Select all

$ldd ~/Documents/WIP/oolite/oolite-source-1.80/deps/Linux-deps/x86_64/lib/libpng14.so.14 
        linux-vdso.so.1 (0x00007fff5dbfe000)
        libz.so.1 => /usr/lib/libz.so.1 (0x00007f219fb97000)
        libm.so.6 => /usr/lib/libm.so.6 (0x00007f219f893000)
        libc.so.6 => /usr/lib/libc.so.6 (0x00007f219f4e5000)
        /usr/lib64/ld-linux-x86-64.so.2 (0x00007f219ffd2000)
$
so that library should work on archlinux.
However, arch uses 'make -f Makefile release' to build oolite.
This Makefile doesn't seem to do anything with the oolite supplied libraries.

--------------------------------------------------
The distribution-neutral installer using autopackage : it has NO support for pacman (archlinux package manager).
I've downloaded the linux 64-bit installer and ran it like this :
./oolite-1.80.linux-x86_64.run --noexec --target oolite-test

That allows me to check the setup script used by oolite.
It does appear to have a non-interactive option, but to be useful for an archlinux PKGBUILD we need to be able to configure several things.
example :
STARTUP_SCRIPTS_PATH_SYSTEMWIDE="/usr/local/bin"
On archlinux , /usr/local/ is NEVER used by our package manager.

---------------------------------
Now let's look at things from the perspective of an archlinux oolite package maintainer :
1. oolite requires minimal adjustment to build from source
2. oolite build this way runs fine
3. png problems in oxp can be solved by the user manually, no problem for arch users.
4. oolite 1.80 introduced oxz . for oxz files there's no known way to correct the png problems AFTER creation.
5. oolite devs introduced the problem, they should be the ones to solve it.

-------------------------------------------------
I see the following options to solve the problem :
  • afaik pngs converted as described in first post work flawlessly in older libpng versions
    oxp / oxz creators could perform the necessary conversions
  • archlinux builds oolite against libpng14
    Arch has a strict policy of 1 library - 1 version, and there are very few exceptions made to that policy.
    it will never be done to accomodate 1 program.
    The only way this could be done would be if oolite was removed from official repos to unsupported / AUR.
    Maintenance of the package would then have to be done by a volunteer archlinux user of unknown competence.
    The threshold to install the package would be a lot higher then now.
  • adjust oolite autopackage setup script to be useful inside an archlinux pkgbuild.
  • add an archlinux template to the Makefile that will build oolite using the included libraries
What option does oolite dev team think is the best ?



Note :
i maintain several unofficial AUR packages, and have been maintainer of the oolite aur package for several years before it was moved to community.