Converting XML plists to the old NeXTSTEP (ASCII) format

[Diziet Sma]

EDIT: also tried dragging and dropping the .py file onto python.exe, tried executing the .py file too.

According to Svengali, you should be able to drag and drop the file you wish to convert onto the xml2ns.py script, and it will do the conversion, even saving the new file into the same folder as the original xml plist.

[Svengali]

According to Svengali, you should be able to drag and drop the file you wish to convert onto the xml2ns.py script, and it will do the conversion, even saving the new file into the same folder as the original xml plist.

Yep. The new file gets "-ns" as suffix, so it won't override the original one. The dropping will only work if the Pathes are set (usually Python does it on it's own when the installer is used).

[Svengali]

I'd like to formally nominate Svengali's version of the script for inclusion as the new official version. How say ye?

But what I like is that Svengali uses spaces instead of tabs! so I think I equally like both versions!

Usually I wouldn't have posted the script, because I think that we don't need multiple versions. Commander Jettisons script works well and the tweaked version only changes a very minor thing and is just my personal preference. And you have already made a 'Linux' version for it .-)

btw: The script uses tabs, like the orginal script. All I've changed is the handling of quotes to get rid of them for keynames in the sublevels (e.g. role). Parent keynames (like entity identifiers) always get quotes. The exceptions are keynames which are containing a minus (e.g. escort-ship) or keynames with a dot (e.g. filenames). Together with the colorscheme it gives a good overview about outdated keynames, mistakes, or simply a warning to check things.


PS: Nonsensical comment -> should be escort_role .-)

[GGShinobi]

I had a little problem with Svengali's version:

Code: Select all

bash: ./xml2ns-svengali.py: /usr/bin/python^M: bad interpreter: No such file or directory

So I had to resolve it by running

Code: Select all

dos2unix xml2ns-svengali.py

Weird.. how did you create your copy? I copy/pasted into Geany, saved it, and it ran fine.. (but now I think about it, I've set Geany to auto-strip CR characters from line endings.. )

I pressed "quote", copy-pasted the whole text into a console with vim running (in paste-mode (:set paste)) and got rid of the "quote"-tags and so...

btw: The script uses tabs, like the orginal script.

Really? Then it seems that the tabs are being converted by the bulletin board to spaces when one presses "quote". I just double checked it, I get spaces.

All I've changed is the handling of quotes to get rid of them for keynames in the sublevels (e.g. role). Parent keynames (like entity identifiers) always get quotes. The exceptions are keynames which are containing a minus (e.g. escort-ship) or keynames with a dot (e.g. filenames). Together with the colorscheme it gives a good overview about outdated keynames, mistakes, or simply a warning to check things.

Hmm, sounds and looks reasonable. But I haven't decided yet which version I prefer (I have used them only for one test yet), so I won't vote on this. At the moment, I like both equally

EDIT: I've now used both versions and it seems that vim likes the Svengali-version more: the syntax is properly highlighted there. Unfortunately, not so with the other version. So I vote for the Svengali-Version!

[Diziet Sma]

In the course of using this to convert the Q-Bomb Detector equipment.plist in readiness for 1.79, I found that since I have Python 2.7 and Python 3.3 installed, (not uncommon in many Linux machines these days) I was getting error messages when trying to run the script.

Some quick google-fu turned up two solutions:

Python 2 will happily run alongside Python 3. You need to specify python2 in order to run this version.

Any program requiring Python 2 needs to point to /usr/bin/python2, instead of /usr/bin/python, which points to Python 3.

To do so, open the program or script in a text editor and change the first line.

The line will show one of the following:

Code: Select all

#!/usr/bin/env python

or

Code: Select all

#!/usr/bin/python

In both cases, just change python to python2 and the program will then use Python 2 instead of Python 3.

Another way to force the use of python2 without altering the scripts is to call it explicitely with python2, i.e.

Code: Select all

python2 myScript.py

I've now edited the Linux version of the script in the Wiki page using the first method, to automatically force use of Python 2 in multi-Python systems.

[cim]

I've now edited the Linux version of the script in the Wiki page using the first method, to automatically force use of Python 2 in multi-Python systems.

That is going to vary between distributions, I think. My system (Debian stable) has available "python", "python2.6" and "python2.7", with "python" being a symlink to "python2.7", which is not quite the same arrangement as ArchLinux.

I'm not sure if there is a solution that works everywhere, though, and at least that change makes it clear it's a python2 script.

[Diziet Sma]

Well mostly, that's just where I first found the info, and I've learned that Arch links tend to have the best-informed and clearest explanations & solutions to many Linux problems..

In fact, I haven't fired up my Arch partition for a couple of weeks.. I have those two versions installed in my Mint partition.. 2.7 is for Oolite-related things, and 3.3 is because I'm teaching myself Python in preparation for making some iPhone/Android apps.

[cim]

In the course of using this to convert the Q-Bomb Detector equipment.plist in readiness for 1.79

Oh ... hang on. I may not have been clear enough here. Oolite 1.79 will still be able to read XML format plists. The only case where Openstep is required is when sending me a manifest.plist to add to the index, and that's only because the index needs to be in one format or the other, and I haven't got automatic conversion between the two set up yet.

[Diziet Sma]

In the course of using this to convert the Q-Bomb Detector equipment.plist in readiness for 1.79

Oh ... hang on. I may not have been clear enough here. Oolite 1.79 will still be able to read XML format plists

Ahh.. I wasn't quite clear enough either.. it had been on my mind for a while to make the change anyway.. (future-proofing, and all that.. not to mention being easier to read), and making an OXZ version was enough motivation to kill two birds with one stone..

[Cdr. Jettison]

Suddenly realised that this old script still gets some use. So, after a long hiatus, I updated it to work with Python 3. Actually, since the script is so basic, the very first version from the top post probably could be run by Python 3 unchanged.

While being at this, I also incorporated the change by Svengali (many thanks!) and made some other cosmetic changes. Python 2 interpreters should still be able to run the script correctly as long as the input data is in ASCII. Actually, I had the code to make it fully compatible with Python 2, but at the last minute decided against applying it to keep things simple: it would require the Python 2 users to install an additional module.

Here is the new version:

Code: Select all

#!/usr/bin/python
'''Convert plist from XML to OpenStep format'''
# Version: 2
# Changes:
#   * convert to Python 3 retaining some Python 2 compatibility
#   * quote strings less (thanks to Svengali)
#   * optimize handling of lists: make the 1st item special, not the last one
#   * more comments
#   * assorted cosmetic changes

import sys

# this script requires at least Python 2.6
assert sys.version_info >= (2, 6)

import re
from plistlib import readPlist  # using deprecated method to retain
                                # compatibility with Python 2

# Change these as needed.
# plist file name can also be passed as parameter on command line
FILENAME = 'shipyard.plist' # name of the plist file to be converted
NEWSUFFIX = '-ns'           # added to FILENAME to produce output file name
NONWORD = re.compile(r'\W') # if keys contain this pattern, they'll be
			    # put in quotes (the keys on the outmost
			    # level are always quoted)

def write_dict(fs, dct, lvl):
    '''Writes dictionary into plist file fs. Recursion as needed'''
    fs.write('{\n')
    for k, v in sorted(dct.items()):
        q = '"' if lvl == 0 or NONWORD.search(k) else ''
        fs.write((lvl+1)*'\t'+'{quote}{key}{quote} = '.format(
            quote=q, key=k))
        if isinstance(v, list):
            write_list(fs, v, lvl+1)
        elif isinstance(v, dict):
            write_dict(fs, v, lvl+1)
        else:
            q = '"' if isinstance(v, str) else ''
            fs.write('{quote}{val}{quote}'.format(
                quote=q,
                val=v if not isinstance(v, bool) else 'yes' if v else 'no'))
        fs.write(';\n')
    fs.write(lvl*'\t'+'}')

def write_list(fs, lst, lvl):
    '''Writes list as array into plist file fs. Recursion as needed'''
    fs.write('(')
    first = True
    for i in lst:
        if first:
            first = False
        else:
            fs.write(',')
        fs.write('\n' + (lvl+1)*'\t')
        if isinstance(i, list):
            write_list(fs, i, lvl+1)
        elif isinstance(i, dict):
            write_dict(fs, i, lvl+1)
        else:
            q = '"' if isinstance(i, str) else ''
            fs.write('{quote}{item}{quote}'.format(
                quote=q, item=i))
    fs.write('\n' + lvl*'\t' + ')')

def main(args):
    '''Main conversion routine'''
    # Either get the plist file name from command line or use the default
    filename = args[0] if args else FILENAME

    # Intentionally using deprecated API to retain compatibility with Python 2
    pl = readPlist(filename)
    level = 0

    with open(filename + NEWSUFFIX, 'w') as f:
        if isinstance(pl, list):
            write_list(f, pl, level)
        elif isinstance(pl, dict):
            write_dict(f, pl, level)
        else:
            f.write('// Parsing error!')
        f.write('\n')

if __name__ == '__main__':
    sys.exit(main(sys.argv[1:]))

Should I replace the old one on box.com? I'll also strip the CR characters before uploading this time, promise.

[cag]

Should I replace the old one on box.com?

Maybe just add a link for Python3 version, so Python2 users don't scream and Python3 users can get up & running faster!

[cag]

Have the .plist format rules changed Jan/2014?

I've added this script to one I'm writing (to generate effectdata.plist files) and get errors due to missing semi-colons in the output. I've gotten

Code: Select all

20:31:25.268 [plist.parse.failed]: Failed to parse Resources/Config/shipdata.plist as a property list.
Parse failed at line 58 (char 2205) - unexpected character (wanted ';' or '}')

when lacking one at the end of a ship definition, and even

Code: Select all

21:31:38.971 [startup.exception]: ***** Unhandled exception during startup: NSInvalidArgumentException (Tried to add nil value for key 'food' to dictionary).

when missing ';' at the end of lists or dictionaries!

The wiki documentation is inconsistent about this, so, I just add a ';' after every closing brace/bracket:

Code: Select all

...

def write_dict(fs, dct, lvl):
...
    fs.write(lvl*'\t'+'};')    
    # added semi-colon  ^  after closing brace

def write_list(fs, lst, lvl):
...
    fs.write('\n' + lvl*'\t' + ');')    
    # added semi-colon           ^  after closing bracket

...

and the parser is much happier.

[hiran]

From an XML point of view the format is very generic and thus quite error prone - in writing and parsing.
I believe if XML support shall further be supported it would make sense to properly design the grammar and document that in an XSD.
Until that happens XML could still be understood by Oolite but marked as 'discouraged' or 'deprecated'.

I was novice enough to extend the wiki a bit with XML structure information but now believe I better had not...