Compiling on recent (2017) distros - GCC 6.x issues

[zevans]

I saw the latest commit included the "null" fixes so I started again with a fresh clone, just to make sure it does build now.

Success! Thanks for all your help.

I had to add -Ideps/mozilla/js/src/to the compile command as I suspected. Need to figure out why "make" isn't doing that automatically so we can put a long-term fix in the codebase.

[Getafix]

What if you do the following:

Code: Select all

$ make -f Makefile distclean
$ make -f Makefile release-snapshot

Does it compile the whole thing?

[zevans]

What if you do the following:

Code: Select all

$ make -f Makefile distclean
$ make -f Makefile release-snapshot

Does it compile the whole thing?

Nope. Here is the minimum needed for the make command, and I've included an example error of what happens when the flags are omitted.

Code: Select all

make -f Makefile release-snapshot \
OBJCFLAGS+="-Wno-format-security"  \
CXXFLAGS+="-Wno-narrowing" \
CXXFLAGS+="-DENABLE_ASSEMBLER"

Examples...

Code: Select all

CFLAGS+="-Wno-narrowing"

../nanojit/NativeX64.cpp:1902:81: error: narrowing conversion of ‘9223372036854775808ull’ from ‘long long unsigned int’ to ‘int64_t {aka long int}’ inside { } [-Wnarrowing]
     static const AVMPLUS_ALIGN16(int64_t) negateMask[] = {0x8000000000000000LL,0};

In this next one it looks like something in the makefile (or configure) is changing this warning TO an error, and these buffer overruns are in the core code, not SpiderMonkey...

Code: Select all

OBJCFLAGS+="-Wno-format-security"

Compiling file src/Core/Debug/OODebugMonitor.m ...
src/Core/Debug/OODebugMonitor.m: In function ‘-[OODebugMonitor dumpMemoryStatistics]’:
src/Core/Debug/OODebugMonitor.m:511:2: error: format not a string literal and no format arguments [-Werror=format-security]
  OOLog(@"debug.memStats", @"Memory statistics:");
  ^
cc1obj: some warnings being treated as errors

And the linking problem discussed earlier in this thread.

Code: Select all

CXXFLAGS+="-DENABLE_ASSEMBLER"

c++ -o js -Wno-narrowing  js.o jsworkers.o   -lpthread   -Wl,-rpath-link,/bin -Wl,-rpath-link,/usr/local/lib  -L../dist/bin -L../dist/lib -L/usr/lib/x86_64-linux-gnu -lplds4 -lplc4 -lnspr4 -lpthread -ldl ../editline/libeditline.a ../libjs_static.a -ldl
../libjs_static.a(jsapi.o): In function `js::RegExp::compileHelper(JSContext*, JSLinearString&)':
jsapi.cpp:(.text._ZN2js6RegExp13compileHelperEP9JSContextR14JSLinearString[_ZN2js6RegExp13compileHelperEP9JSContextR14JSLinearString]+0x7d): undefined reference to `JSC::Yarr::jitCompileRegex(JSC::ExecutableAllocator&, JSC::Yarr::RegexCodeBlock&, JSLinearString const&, unsigned int&, int&, bool&, bool, bool)'
/usr/bin/ld: js: hidden symbol `_ZN3JSC4Yarr15jitCompileRegexERNS_19ExecutableAllocatorERNS0_14RegexCodeBlockERK14JSLinearStringRjRiRbbb' isn't defined
/usr/bin/ld: final link failed: Bad value

[another_commander]

The format security error should be fixed in trunk; please check. There seems to be something odd with your build setup, because ENABLE_ASSEMBLER (as well as a few other macros needed for Spidermonkey JIT) should be already defined - at least they seem to be in the dev evnironments that build the Linux packages and maybe Getafix can help here by confirming or denying this.

[Getafix]

Code: Select all

deps/mozilla/js/src/Makefile.in:917:CXXFLAGS += -DUSE_SYSTEM_MALLOC=1 -DENABLE_ASSEMBLER=1 -DENABLE_JIT=1

[zevans]

Yep - something weird is going on - haven't figured out what yet! Suspect autoconf or the configure script is guessing something wrong...

[Getafix]

@zevans
Be very cautious fiddling with compiler and linker options, though.
If your environment is actually ignoring options defined in the distributed makefiles,
you risk ending-up with a non compatible binary demonstrating wrong behavior during execution;
and "wrong" can just and simply mean "different" and not necessarily an observable erroneous behavior.

You are most probably already aware of that, but, I just felt that it needed it's own place in this thread,
emphasizing the risk, for the rest of us that are not that much keen on compilers and linkers.

[Commander_X]

Quick question regarding the recent distros: are there any known issues when using more recent gnustep (in my case 1.25.0) and/or gcc (5.3.0 here) on a x86 system?

The reason I'm asking this question is that I'm successfully compiling with the above versions (having use_deps=yes debug=no in make's command line), but oolite.app/oolite would segfault when trying to launch.

Details:
I also have a x86_64 system where things go smoothly.

Most notable differences (besides default libraries, due to OS versions -- one is slackware64 --x86_64 -- (almost) 14.1, and the one I'm interested in is slackware -- x86 -- 14.2):

x86_64 (working):

Code: Select all

gnustep-make: 2.6.6
gnustep: 1.24.6
gcc: 4.8.2

x86 (not working):

Code: Select all

gnustep-make: 2.7.0
gnustep: 1.25.0
gcc: 5.3.0

A binary release of nightly latest trunk runs correctly on this system.

"ldd" on oolite.app/oolite shows that on x86 system, the exe is picking more libs from Oolite's Linux-deps than the one on the x86_64 one:

the common set of deps llibraries picked by all (and the only libraries picked by x86_64):

Code: Select all

libSDL-1.2.so.0
libopenal.so.1
libz.so.1
libvorbisfile.so.3
libpng14.so.14
libespeak.so.1
libvorbis.so.0
libogg.so.0
libportaudio.so.2

extras for non working x86:

Code: Select all

libgmp.so.10
libgnutls.so.30
libnettle.so.6
libhogweed.so.4

extras for the nigthly:

Code: Select all

libgnustep-base.so.1.20
libplds4.so.0d
libplc4.so.0d
libnspr4.so.0d
libobjc.so.2
libffi.so.4

One fix I tried was to bring the missing library links to lib_linker, but it seems they are still not picked during linking.

[Getafix]

@Commander_X
Could you upload somewhere the ./configure log, and share a link for me to download, for the gnustep library?
I am asking this, because I presume that being in slackware you build it on your own.

In the meantime, I am also preparing some instructions, for you to give me the dependencies of your oolite executable in my preferred hierarchical order.

[Commander_X]

@Commander_X
Could you upload somewhere the ./configure log, and share a link for me to download, for the gnustep library?
I am asking this, because I presume that being in slackware you build it on your own.

Presumption confirmed. Here you'll have a .tar.gz file containing both files fresh from the build directory.

In the meantime, I am also preparing some instructions, for you to give me the dependencies of your oolite executable in my preferred hierarchical order.

Looking forward for these.

FYI I've "advanced" a bit doing a sort of "hacking surgery":
- presented gnustep-base from deps/Linux-deps/x86/lib_linker as libgnustep-base.so ("ln -s" in lib_linker) -- this allowed the library to be picked by the linker, but there is a warning during linking saying that "libobjc.so.2 may conflict with libobjc.so.4".
- tried the same trick as above for nspr4 & co. (even replacing the `nspr-config --libs` in GNUmakefile with a hand crafted set of -L and -l options) -- but these libraries seem to elude my attempts; after linking, still the system ones are taken as opposed to those provided by the deps.
It seems that bringing gnustep-base into the mix like above, really messes up things -- with this setup no logs are created by the oolite when trying to launch.

[Getafix]

@Commander_X
It's been sometime since I haven't been around here, so perhaps this is already resolved.
However, I am posting some feedback that might gain you some time, when trying to build from source for Linux targets.

Let's start by noting the following definitions:
objdump is dumping what the object itself lists as libraries while
ldd is listing which libraries ld.so would actually load, following the graph backward, showing what would be loaded by those libraries. With ldd you see all your dependencies but your are loosing the info of who is actually using each library needed.

FYI I've "advanced" a bit doing a sort of "hacking surgery":
- presented gnustep-base from deps/Linux-deps/x86/lib_linker as libgnustep-base.so ("ln -s" in lib_linker) -- this allowed the library to be picked by the linker, but there is a warning during linking saying that "libobjc.so.2 may conflict with libobjc.so.4".
- tried the same trick as above for nspr4 & co. (even replacing the `nspr-config --libs` in GNUmakefile with a hand crafted set of -L and -l options) -- but these libraries seem to elude my attempts; after linking, still the system ones are taken as opposed to those provided by the deps.
It seems that bringing gnustep-base into the mix like above, really messes up things -- with this setup no logs are created by the oolite when trying to launch.

This is an absolute "DON'T". You should have a full blown (relevant headers etc.) Oolite for Linux Development Environment (aka OOLDE), compatible with these libraries, in order to use them. The libraries in deps/Linux-deps/x86[_64]/lib are part of the OOLDE. They are gathered there for maintainer's easiness when bundled in the Oolite for Linux Setup Package (aka OOLSP).

I understand that this is a bit confusing and I had received some whining in the past, from the Slackware forum hosted in LinuxQuestions.org, on that matter.
So, here follows the list of Oolite for Linux dependencies:

Code: Select all

terrastorage@Hermes:~$ cd /opt/Oolite-trunk/oolite.app/; \
> echo; echo oolite:; \
> objdump -p oolite | grep NEEDED; \
> echo; \
> echo

  NEEDED               libGLU.so.1
  NEEDED               libGL.so.1
  NEEDED               libX11.so.6
  NEEDED               libSDL-1.2.so.0
  NEEDED               libgnustep-base.so.1.20
  NEEDED               libplds4.so.0d
  NEEDED               libplc4.so.0d
  NEEDED               libnspr4.so.0d
  NEEDED               libpthread.so.0
  NEEDED               libdl.so.2
  NEEDED               libstdc++.so.6
  NEEDED               libopenal.so.1
  NEEDED               libz.so.1
  NEEDED               libvorbisfile.so.3
  NEEDED               libpng14.so.14
  NEEDED               libespeak.so.1
  NEEDED               libobjc.so.2
  NEEDED               libm.so.6
  NEEDED               libgcc_s.so.1
  NEEDED               libc.so.6


terrastorage@Hermes:/opt/Oolite-trunk/oolite.app$

Given this set, we managed to came up with a subset of libraries that, when bundled with OOLSP, make it portable across a big variety of Linux distros. You understand that in an environment like Slackware, where you build everything on your own, you need to build the dependencies of the dependencies of the dependencies etc. until you reach dependencies that already exist in your system.

Therefore to further help, here follows the first level dependencies of the libraries bundled with OOLSP.

Code: Select all

terrastorage@Hermes:/opt/Oolite-trunk/oolite.app$ cd ../oolite-deps/lib/ ; echo; \
> for i in `ls -1`; \
> do echo $i:; \
> objdump -p $i | grep NEEDED; \
> echo; echo; done

libespeak.so.1:
  NEEDED      libstdc++.so.6
  NEEDED      libportaudio.so.2
  NEEDED      libpthread.so.0
  NEEDED      libm.so.6
  NEEDED      libgcc_s.so.1
  NEEDED      libc.so.6


libespeak.so.1.pulseaudio:
  NEEDED      libstdc++.so.6
  NEEDED      libpulse.so.0
  NEEDED      libpthread.so.0
  NEEDED      libm.so.6
  NEEDED      libgcc_s.so.1
  NEEDED      libc.so.6


libffi.so.4:
  NEEDED      libc.so.6


libgcrypt.so.20:
  NEEDED      libgpg-error.so.0
  NEEDED      libc.so.6


libgmp.so.10:
  NEEDED      libc.so.6


libgnustep-base.so.1.20:
  NEEDED      libpthread.so.0
  NEEDED      libobjc.so.2
  NEEDED      libgmp.so.10
  NEEDED      libgnutls.so.30
  NEEDED      libgcrypt.so.20
  NEEDED      libxml2.so.2
  NEEDED      libffi.so.4
  NEEDED      libnsl.so.1
  NEEDED      librt.so.1
  NEEDED      libdl.so.2
  NEEDED      libz.so.1
  NEEDED      libm.so.6
  NEEDED      libgcc_s.so.1
  NEEDED      libc.so.6


libgnutls.so.30:
  NEEDED      libz.so.1
  NEEDED      librt.so.1
  NEEDED      libnettle.so.6
  NEEDED      libhogweed.so.4
  NEEDED      libgmp.so.10
  NEEDED      libc.so.6


libgpg-error.so.0:
  NEEDED      libc.so.6


libhogweed.so.4:
  NEEDED      libnettle.so.6
  NEEDED      libgmp.so.10
  NEEDED      libc.so.6


libnettle.so.6:
  NEEDED      libc.so.6


libnspr4.so.0d:
  NEEDED      libpthread.so.0
  NEEDED      libdl.so.2
  NEEDED      libc.so.6


libobjc.so.2:
  NEEDED      libgcc_s.so.1
  NEEDED      libc.so.6


libogg.so.0:
  NEEDED      libc.so.6


libopenal.so.1:
  NEEDED      librt.so.1
  NEEDED      libpthread.so.0
  NEEDED      libdl.so.2
  NEEDED      libm.so.6
  NEEDED      libc.so.6


libplc4.so.0d:
  NEEDED      libnspr4.so.0d
  NEEDED      libc.so.6


libplds4.so.0d:
  NEEDED      libnspr4.so.0d
  NEEDED      libc.so.6


libpng14.so.14:
  NEEDED      libz.so.1
  NEEDED      libm.so.6
  NEEDED      libc.so.6


libportaudio.so.2:
  NEEDED      librt.so.1
  NEEDED      libasound.so.2
  NEEDED      libm.so.6
  NEEDED      libpthread.so.0
  NEEDED      libc.so.6


libSDL-1.2.so.0:
  NEEDED      libm.so.6
  NEEDED      libdl.so.2
  NEEDED      libpthread.so.0
  NEEDED      libc.so.6


libvorbisfile.so.3:
  NEEDED      libvorbis.so.0
  NEEDED      libogg.so.0
  NEEDED      libc.so.6


libvorbis.so.0:
  NEEDED      libm.so.6
  NEEDED      libogg.so.0
  NEEDED      libc.so.6


libz.so.1:
  NEEDED      libc.so.6


terrastorage@Hermes:/opt/Oolite-trunk/oolite-deps/lib$

(I will keep a note to add this in the Oolite for Linux documentation.)

Hasta la babe vista!

[Commander_X]

Heh, no vista to hasta la, babe

Thank you for the extended explanation. I didn't yet succeed in the attempts (to be honest I've not pushed any further from my previous post).

The "dependency dance" (how I call it) is one of the reasons I've stuck with Slackware for quite some time. Whenever rpm/yum/dpkg/apt-get were getting things in the system and not cleaning after, I felt myself betrayed on the lowest level (almost as worse as windows updates). At least compiling from source either picks what's "in" or complains on what's needed.

I've posted in this thread because Slackware 14.2 is the most recent (although 2016) version of the distro.

My (simple minded) best bets on the issues in the matter are:
- gnustep-base (and gnustep-make) versions -- I initially used latest for both, and they might interfere with what the build system expects; I will try reverse back to those that work on the other system (the almost x86_64 14.1 I mentioned earlier)
- objc (thus GCC 5x) -- it seems the version of the library provided by GCC 5x is ".so.4" and there might be ABI changes that simply won't work; I'm not sure I want to compile myself a GCC 4.8.x version to check this assumption.
Another issue could be with the one main "new" things on 14.2 is the pulseaudio availability in Slackware, which I'm not very happy/acquainted with.

I've been trying to go for the minimal interference with the "dev" libraries on the system (i.e. not compiling my own versions of the stack). The "deps" approach worked quite nicely until now.

I guess I'll set for a 2-way approach further:
- compile the .app in a VM in slackware 14.1 and check to what extent that's feasible
- try to go for the whole "dependency dance", although I might not push it too far down the whole versions stack; just those that seem to make sense for Oolite (e.g. portaudio/espeak, png14, ogg, vorbis; of course I'm compiling JS in the dev tree, that's what I've done already)

[Getafix]

Do not worry about libespeak.so.1.pulseaudio and its dependency libpulse.so.0.
The PulseAudio version of eSpeak is not used (refer to the oolite dependencies I gave). It is bundled in OOLSP, for the user, to manually replace the bundled default PortAudio libespeak.so.1 version, when Oolite speech synthesis has issues in a distro.
This case was recurring and it proved to be a successful workaround.

[Commander_X]

Do not worry about libespeak.so.1.pulseaudio and its dependency libpulse.so.0.

Noted. I will worry about them when their time will come.
Meanwhile ... Good news, everyone!
[Long story short]

Installed VM, and right out of the bat, the build worked on x86 14.1, as expected, with the newest gnustep-base/make. And the same build worked on the 14.2.

First note -- it was a _debug_ build.

Went back to 14.2 and after a "distclean" the same type of build got me an working exe.
Next step, relase build. Seamless compilation on both, the one on 14.1 worked in both places, the one on 14.2 segfaulted (for the record, at the same place in the Latest.log, during "Loading ship data").

Second note [mostly to myself] -- I think one of the things I've been doing wrong earlier, was to follow my own GNUStep/objc habits of using the GNUmakefile and *.make files instead the recommended Readme.md Makefile approach.

I am still 1. doing an extra link in lib_linker (for portaudio -- dependency for espeak) 2. using LD_LIBRARY_PATH in the make command line, pointing to the lib_linker folder (didn't compile any other dependency other than gnustep-base and gnustep-make).

I'll try to track down the "wronging" dependency -- I still have a feeling it's sound related -- even the working debug version on 14.2 segfaulted at a later time (after 2 jumps) with some sound related messages on the console.

[Getafix]

It's hard to have a clear picture of what exactly you are doing in this endeavor,
(and perhaps it's better this way, for my sanity and for the time when the SWAT will break into your computer room) ,
but I feel that even if using the pre-compiled libraries seems like a shortcut, it is actually distracting your path towards a successful build.
Slackware is about building your way from the ground up.

Since you are working in a VM, I suggest that you make it work, getting the source of the libraries versions we use in Oolite and forgetting about make deps=yes.
Actually, I suggest that you keep it that way and you make a bootable USB stick of that installation dedicated to Oolite, or, if this doesn't affect your security needs or whatever you want to do with your Slackware installation, don't upgrade them to later versions. Nobody warranties that later versions are compatible to each other and even with Oolite itself. You will end-up with an unsupported Oolite, debugging your own system's libraries.
If this is the target of your trip, then, I presume that your gdb skills are up to the challenge to distinguish when a crash is due to application issue or incompatible library or buggy library.
Keep an oolite.org installation in place and active, anyhow, for reference and for gaming time and have the own build for your educational purposes.


Note 1:
Testing on a VM is good just to get an idea that the damn thing is running, but software simulated sound and video may be the source of random crashes. In my case a SLAX bootable USB Stick, has helped me to have a better view of the sanity of my Oolite builds.

Note 2:

extras for non working x86:

Code: Select all

libgmp.so.10
libgnutls.so.30
libnettle.so.6
libhogweed.so.4

Your working x86_64 build, has no TLS support. Seems like, you didn't have GnuTLS built, when you built gnustep-base.

Note 3:

extras for the nigthly:

Code: Select all

libgnustep-base.so.1.20
libplds4.so.0d
libplc4.so.0d
libnspr4.so.0d
libobjc.so.2
libffi.so.4

The only thing that I remember is that at one point in time, when I automatically upgraded (double facepalm) the OOLDE environment, these two libraries

• libplds4.so.0d
• libplc4.so.0d

appeared in the Oolite for Linux dependencies. Some library change that I don't know (?!?) brought these new dependencies into the picture.
I emphasize, to highlight on what is happening when you decide to go with a different library set.