Building JS library from scratch

[mcarans]

I ran this to get configure in deps\mozilla\js\src as far as possible:

Code: Select all

export CC=/mingw64/bin/gcc
export CXX=/mingw64/bin/g++
./configure --with-windows-version=502

This failed needing Python < 3:

Code: Select all

$ ./configure --with-windows-version=502
loading site script /etc/config.site
loading cache ./config.cache
checking host system type... x86_64-pc-mingw32
checking target system type... x86_64-pc-mingw32
checking build system type... x86_64-pc-mingw32
checking for mawk... no
checking for gawk... gawk
checking for perl5... no
checking for perl... /usr/bin/perl
checking for gcc... /mingw64/bin/gcc
checking whether the C compiler (/mingw64/bin/gcc  ) works... yes
checking whether the C compiler (/mingw64/bin/gcc  ) is a cross-compiler... no
checking whether we are using GNU C... yes
checking whether /mingw64/bin/gcc accepts -g... yes
checking for c++... /mingw64/bin/g++
checking whether the C++ compiler (/mingw64/bin/g++  ) works... yes
checking whether the C++ compiler (/mingw64/bin/g++  ) is a cross-compiler... no
checking whether we are using GNU C++... yes
checking whether /mingw64/bin/g++ accepts -g... yes
checking for ranlib... ranlib
checking for ml64... no
checking for as... /mingw64/bin/as
checking for ar... ar
checking for ld... link
checking for strip... strip
checking for windres... windres
checking for w32api version >= 3.8... yes
checking for windres version >= 2.14.90... (GNU
./configure: line 3402: test: (GNU: integer expression expected
checking for Windows SDK being recent enough... yes
checking whether /mingw64/bin/gcc and cc understand -c and -o together... yes
checking how to run the C preprocessor... cl -E -nologo
checking how to run the C++ preprocessor... cl -TP -E -nologo
checking for sb-conf... no
checking for ve... no
checking for a BSD compatible install... /usr/bin/install -c
checking whether ln -s works... no
checking for minimum required perl version >= 5.006... 5.038004
checking for full perl installation... yes
checking for python2.7... no
checking for python2.6... no
checking for python2.5... no
checking for python... /mingw64/bin/python
checking for doxygen... :
checking for autoconf... /usr/bin/autoconf
checking for unzip... :
checking for zip... /usr/bin/zip
checking for makedepend... no
checking for xargs... /usr/bin/xargs
checking for make... /usr/bin/make
checking for X... no
checking whether the compiler supports -Wno-invalid-offsetof... yes
checking whether the compiler supports -Wno-variadic-macros... yes
checking whether the compiler supports -Werror=return-type... yes
checking whether ld has archive extraction flags... yes
checking that static assertion macros used in autoconf tests work... yes
checking for 64-bit OS... yes
checking for Python version >= 2.5 but not 3.x... configure: error: Python 2.5 or higher (but not Python 3.x) is required.

I'm not sure it will be possible to build the JS library on any modern setup, but I might try this: https://github.com/msys2/MSYS2-packages/issues/2334 to see how far I can get.

[mcarans]

Python 2.7 installed fine. There seems to be a need for an env var "MOZ_TOOLS". I'm not sure what it is supposed to point to. If I set it to something, configure fails with "checking for a 1-byte type... configure: error: Couldn't find a 1-byte type"

The code from configure is this:

Code: Select all

echo $ac_n "checking for a 1-byte type""... $ac_c" 1>&6
echo "configure:8634: checking for a 1-byte type" >&5
if eval "test \"`echo '$''{'moz_cv_n_byte_type_JS_INT8_TYPE'+set}'`\" = set"; then
  echo $ac_n "(cached) $ac_c" 1>&6
else
  
  moz_cv_n_byte_type_JS_INT8_TYPE=
  for type in char; do
    cat > conftest.$ac_ext <<EOF

Anyone got any idea what are MOZ_TOOLS and/or what this error is about?

[mcarans]

I have found out what the MOZ_TOOLS are: https://gemal.dk/mozilla/build.html

"Netscape has modified a handful of the GNU command-line tools to solve some problems mostly for makefile compatibility with the GNU-styled UNIX builds.

Get Netscape wintools : Direct download http://ftp.mozilla.org/pub/mozilla.org/ ... ntools.zip

Unpack the zip file into a temporary directory. Then start a Command Prompt in the temporary directory and do:

Code: Select all

set MOZ_TOOLS=c:\mozilla\moztools
mkdir c:\mozilla\moztools
cd buildtools\windows
install.bat

The wintools download contains:
gmake.exe
nsinstall.exe
shmsdos.exe
uname.exe

It also has a library: libIDL

Of these, nsinstall.exe appears to be the important one and fortunately there is a MSYS2 package for it.

[mcarans]

I installed nsinstall from MSYS2. I had to edit the configure file to get rid of the appending of -mno-cygwin to the compiler,

lines 7066-7068

Code: Select all

        CC="$CC -mno-cygwin"
        CXX="$CXX -mno-cygwin"
        CPP="$CPP -mno-cygwin"

and line 6325:

Code: Select all

        HOST_CFLAGS="$HOST_CFLAGS -mno-cygwin"

That was causing some of the checks to fail since that isn't a valid flag for gcc.

This is using:

Code: Select all

export CC=/mingw64/bin/gcc
export CXX=/mingw64/bin/g++
export MOZ_TOOLS=/usr
./configure --enable-static --with-windows-version=502 --with-system-nspr --with-pthreads --disable-shared-js --enable-threadsafe

Now configure fails like this:

Code: Select all

checking for nspr-config... /mingw64/bin/nspr-config
checking for NSPR - version >= 4.7.0... yes
checking for valid optimization flags... no
configure: error: These compiler flags are invalid: -O

[mcarans]

I got configure to complete with:

Code: Select all

export CC=/mingw64/bin/gcc
export CXX=/mingw64/bin/g++
export CPP="/mingw64/bin/g++ -E"
export CXXCPP="/mingw64/bin/g++ -E"
export CFLAGS=-I/mingw64/include
export CXXFLAGS=-I/mingw64/include
export CPPFLAGS=-I/mingw64/include
export HOST_CFLAGS=-I/mingw64/include
export HOST_CXXFLAGS=-I/mingw64/include
export HOST_CPPFLAGS=-I/mingw64/include
export LDFLAGS=-L/mingw64/lib
export MOZ_TOOLS=/usr
./configure --enable-static --with-windows-version=502 --with-system-nspr --with-pthreads --disable-shared-js --enable-threadsafe --enable-optimize=-O2

[mcarans]

Using Oolite's libjs.make I had to add some flags:

Code: Select all

...
LIBJS_SRC_DIR                    = deps/mozilla/js/src
LIBJS_CONFIG_FLAGS               = --disable-shared-js
LIBJS_CONFIG_FLAGS               += --enable-threadsafe
LIBJS_CONFIG_FLAGS               += --with-pthreads
LIBJS_CONFIG_FLAGS               += --with-system-nspr
LIBJS_CONFIG_FLAGS               += --with-windows-version=502
LIBJS_CONFIG_FLAGS               += --disable-tests
ifeq ($(OO_JAVASCRIPT_TRACE),yes)
    LIBJS_CONFIG_FLAGS           += --enable-trace-jscalls
endif
ifeq ($(debug),yes)
    LIBJS_BUILD_DIR              = $(LIBJS_SRC_DIR)/build-debug
    LIBJS_CONFIG_FLAGS           += --enable-debug
    LIBJS_CONFIG_FLAGS           += --disable-optimize
    LIBJS_BUILD_FLAGS            =
else
    LIBJS_BUILD_DIR              = $(LIBJS_SRC_DIR)/build-release
    LIBJS_CONFIG_FLAGS           += --enable-optimize=-O2
    LIBJS_BUILD_FLAGS            =
...

I then ran:

Code: Select all

export CC=/mingw64/bin/gcc
export CXX=/mingw64/bin/g++
export CPP="/mingw64/bin/g++ -E"
export CXXCPP="/mingw64/bin/g++ -E"
export CFLAGS="-I/mingw64/include"
export CXXFLAGS="-I/mingw64/include"
export CPPFLAGS="-I/mingw64/include"
export HOST_CFLAGS="-I/mingw64/include"
export HOST_CXXFLAGS="-I/mingw64/include"
export HOST_CPPFLAGS="-I/mingw64/include"
export LDFLAGS="-L/mingw64/lib"
export MOZ_TOOLS=/usr
make -f libjs.make distclean
make -f libjs.make debug=no

This failed with:

Code: Select all

/mingw64/bin/gcc -o nsinstall.exe -I/mingw64/include -DXP_WIN32 -DXP_WIN -DWIN32 -D_WIN32 -DNO_X11 -D_AMD64_ -O2  -DUNICODE -D_UNICODE -lpthread -L/mingw64/lib    host_nsinstall_win.o
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/15.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/mingw64/lib/libmingw32.a(lib64_libmingw32_a-crtexewin.o): in function `main':
D:/W/B/src/mingw-w64/mingw-w64-crt/crt/crtexewin.c:66:(.text.startup+0xb5): undefined reference to `WinMain'
collect2.exe: error: ld returned 1 exit status

It looks like it is building nsinstall.exe but I am wondering: is it necessary since I already have nsinstall from an MSYS2 package? Why is it trying to build it as a GUI application instead of a console one for nsinstall.exe?

[mcarans]

I fixed the the nsinstall.exe issue by changing "wmain" to "main" in deps\mozilla\js\src\config\ns_install_win.c and running:

Code: Select all

export NSINSTALL_BIN=/usr/bin/nsinstall.exe
export CC=/mingw64/bin/gcc
export CXX=/mingw64/bin/g++
export CPP="/mingw64/bin/g++ -E"
export CXXCPP="/mingw64/bin/g++ -E"
export CFLAGS="-I/mingw64/include"
export CXXFLAGS="-I/mingw64/include"
export CPPFLAGS="-I/mingw64/include"
export HOST_CFLAGS="-I/mingw64/include"
export HOST_CXXFLAGS="-I/mingw64/include"
export HOST_CPPFLAGS="-I/mingw64/include"
export LDFLAGS="-L/mingw64/lib"
export MOZ_TOOLS=/usr
make -f libjs.make distclean
make -f libjs.make debug=no -j16

The next error is:

Code: Select all

/mingw64/bin/g++ -o js.exe -I/mingw64/include -fno-rtti -fno-exceptions -Wall -Wpointer-arith -Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy -Wno-non-virtual-dtor -Wcast-align -Wno-invalid-offsetof -Wno-variadic-macros -Werror=return-type -pedantic -Wno-long-long -I/mingw64/include -fno-strict-aliasing -mms-bitfields -pthread -pipe -DNDEBUG -DTRIMMED -O2  js.o jsworkers.o   -lpthread -L/mingw64/lib    -L../dist/bin -L../dist/lib -L/mingw64/lib -lplds4 -lplc4 -lnspr4 @../libjs_static.a.fake -lgdi32 -lwinmm -lwsock32
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/15.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find c:oolitedepsmozillajssrcbuild-releasejsanalyze.o: No such file or directory
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/15.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find c:oolitedepsmozillajssrcbuild-releasejsapi.o: No such file or directory
...
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/15.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find c:oolitedepsmozillajssrcbuild-releasepm_stub.o: No such file or directory
collect2.exe: error: ld returned 1 exit status
make[3]: *** [../../config/rules.mk:1024: js.exe] Error 1
make[3]: Leaving directory '/c/oolite/deps/mozilla/js/src/build-release/shell'

There are many ld errors. Looks like a problem with Windows vs UNIX paths. A libjs_static.a file was produced though.

[another_commander]

Just to double-check, you are building with JIT (just-in-time compiler) enabled, as well as the flags needed by Oolite, right? When I was building it I had to set these flags:

Code: Select all

-DJS_TRACER -DFEATURE_NANOJIT -DAVMPLUS_64BIT -DAVMPLUS_AMD64 -DJS_CPU_X64 -DJS_PUNBOX64 -DENABLE_ASSEMBLER -DENABLE_YARR_JIT -DJS_METHODJIT=1 -DJS_MONOIC=1 -DJS_POLYIC=1 -DJS_POLYIC_TYPED_ARRAY=1 -DMOZ_TRACE_JSCALLS=1

and also

Code: Select all

OS_CFLAGS = -DXP_WIN -DEXPORT_JS_API -DJS_C_STRINGS_ARE_UTF8 -D__STDC_LIMIT_MACROS -D_WINDOWS -DWINVER=0x500 -D_WIN32_WINNT=0x500 $(WIN_CFLAGS)
JSDLL_CFLAGS = -DEXPORT_JS_API

Out of all those, the ones I remember as being terribly important for Oolite are the ones at the top plus JS_C_STRINGS_ARE_UTF8 and of course the EXPORT_JS_API one. Many or all of those are probably set anyway by your build configuration, but it doesn't hurt to check, it might help avoid unnecessary pain if you build it successfully only to find out that a critical flag was missing.

[mcarans]

Just to double-check, you are building with JIT (just-in-time compiler) enabled, as well as the flags needed by Oolite, right? When I was building it I had to set these flags:

Code: Select all

-DJS_TRACER -DFEATURE_NANOJIT -DAVMPLUS_64BIT -DAVMPLUS_AMD64 -DJS_CPU_X64 -DJS_PUNBOX64 -DENABLE_ASSEMBLER -DENABLE_YARR_JIT -DJS_METHODJIT=1 -DJS_MONOIC=1 -DJS_POLYIC=1 -DJS_POLYIC_TYPED_ARRAY=1 -DMOZ_TRACE_JSCALLS=1

and also

Code: Select all

OS_CFLAGS = -DXP_WIN -DEXPORT_JS_API -DJS_C_STRINGS_ARE_UTF8 -D__STDC_LIMIT_MACROS -D_WINDOWS -DWINVER=0x500 -D_WIN32_WINNT=0x500 $(WIN_CFLAGS)
JSDLL_CFLAGS = -DEXPORT_JS_API

Out of all those, the ones I remember as being terribly important for Oolite are the ones at the top plus JS_C_STRINGS_ARE_UTF8 and of course the EXPORT_JS_API one. Many or all of those are probably set anyway by your build configuration, but it doesn't hurt to check, it might help avoid unnecessary pain if you build it successfully only to find out that a critical flag was missing.

Thanks, I added the missing ones. Now when building Oolite with the new JS library, I get errors like:

undefined reference to `__imp_PR_AtomicIncrement'

Is that something you've come across before?

[another_commander]

No and I think that PR_* is from the nspr library, required for building with JS_THREADSAFE.

Note that the Windows version of JS is built without JS_THREADSAFE (the Linux version is built with it enabled), but we do not use any JS multi thread functionality in Oolite so it should not be a problem if JS_THREADSAFE is removed in order to simplify things.

[mcarans]

No and I think that PR_* is from the nspr library, required for building with JS_THREADSAFE.

Note that the Windows version of JS is built without JS_THREADSAFE (the Linux version is built with it enabled), but we do not use any JS multi thread functionality in Oolite so it should not be a problem if JS_THREADSAFE is removed in order to simplify things.

I had to remove the line below from libjs.make to get it to build a .dll.a file:

Code: Select all

LIBJS_CONFIG_FLAGS               = --disable-shared-js

Using that file libmozjs.dll.a instead of libjs_static.a, it linked ok. Copying the corresponding mozjs.dll into oolite.app, Oolite started and then shut down. The logging messages I had previously added showed that it got further.

I tried gdb and it said:

Code: Select all

Thread 1 received signal SIGSEGV, Segmentation fault.
0x00007ff88c3f86a0 in JS_GetClass () from C:\oolite\oolite.app\mozjs.dll
(gdb)

I then tried building mozjs without threadsafe. This time Oolite started up and showed the opening screen with music playing and rotating ship. I could start a new game - so much further than before. However, it was then that it crashed. Trying again in gdb, when I started a new game, I got:

Code: Select all

Thread 1 received signal SIGSEGV, Segmentation fault.
0x00007ff88baf5bc8 in SetVMFrameRegs () from C:\oolite\oolite.app\mozjs.dll

That call is in MethodJIT.cpp. Do you have any idea about this error?

[another_commander]

Not enough data. We need a backtrace from the crash. When it crashes just execute bt from the gdb prompt and if you have debug symbols in the js library we may be able to identify the exact crash location in MethodJIT.cpp.

[another_commander]

I have successfully built a JS_THREADSAFE Spidermonkey for Oolite using my flavor of the Dev Environment for Windows. You can test it by dropping the files from this zip into oolite.app. The game should load up and play without issues - at least it does so for me.

In case it helps, I can upload my work files for this once I get the opportunity (need to write some build instructions first). In the meantime, there is a new version of the LE Dev Environment available here that contains pre-built nspr4.dll, js32 binary and import libraries etc.

If testing shows all clear this can go into master soon.

BTW, Oolite needs to have JS_THREADSAFE defined as well to run without problems. The switch for that is in the Windows deps\x86_64\JS32ECMAv5\include\js-config.h file. The #undef JS_THREADSAFE needs to become #define JS_THREADSAFE 1.

[mcarans]

I have successfully built a JS_THREADSAFE Spidermonkey for Oolite using my flavor of the Dev Environment for Windows. You can test it by dropping the files from this zip into oolite.app. The game should load up and play without issues - at least it does so for me.

In case it helps, I can upload my work files for this once I get the opportunity (need to write some build instructions first). In the meantime, there is a new version of the LE Dev Environment available here that contains pre-built nspr4.dll, js32 binary and import libraries etc.

If testing shows all clear this can go into master soon.

BTW, Oolite needs to have JS_THREADSAFE defined as well to run without problems. The switch for that is in the Windows deps\x86_64\JS32ECMAv5\include\js-config.h file. The #undef JS_THREADSAFE needs to become #define JS_THREADSAFE 1.

That's good. In my MSYS2 environment, your threadsafe version worked like the non-threadsafe one with the #define JS_THREADSAFE 1 ie. it ran fine in gdb but stopped on startup without gdb (presumably in the same place as described here viewtopic.php?p=302473#p302473 but I didn't add back all the log messages to check).

[another_commander]

The threadsafe dll I generated is meant to be used in combination with the standard binaries distributed with the game. I would expect all sorts of issues if you tried to use it with binaries built on a different environment, as you have already experienced.