Recommended tool for extensive editing of planetinfo.plist ?
Moderators: winston, another_commander
- Redspear
- ---- E L I T E ----
- Posts: 2694
- Joined: Thu Jun 20, 2013 10:22 pm
- Location: On the moon Thought, orbiting the planet Ignorance.
Recommended tool for extensive editing of planetinfo.plist ?
Planet info. Plist is a bit of a beast.
8 galaxies with 200+ planets each and not all of the data is writable by scripting.
So does anyone know of a suitable tool that could make the kind of changes that could be explained in a sentence without laboriously editing each individual planet?
For example: double sun distance in every rich industrial system. Only takes a few seconds to explain but potentially hours to edit.
Any known (and preferably free) software that could speed this up?
8 galaxies with 200+ planets each and not all of the data is writable by scripting.
So does anyone know of a suitable tool that could make the kind of changes that could be explained in a sentence without laboriously editing each individual planet?
For example: double sun distance in every rich industrial system. Only takes a few seconds to explain but potentially hours to edit.
Any known (and preferably free) software that could speed this up?
Re: Recommended tool for extensive editing of planetinfo.plist ?
sedRedspear wrote: ↑Fri Nov 29, 2024 3:02 pmPlanet info. Plist is a bit of a beast.
8 galaxies with 200+ planets each and not all of the data is writable by scripting.
So does anyone know of a suitable tool that could make the kind of changes that could be explained in a sentence without laboriously editing each individual planet?
For example: double sun distance in every rich industrial system. Only takes a few seconds to explain but potentially hours to edit.
Any known (and preferably free) software that could speed this up?
(edit, I'm having a twiddle at doing that in sed, but realistically perl or python would be able to do it in very few lines of code with regexes)
- Griff
- Oolite 2 Art Director
- Posts: 2484
- Joined: Fri Jul 14, 2006 12:29 pm
- Location: Probably hugging his Air Fryer
Re: Recommended tool for extensive editing of planetinfo.plist ?
MrFlibble's mention of regexe there in their post reminded me of once using them in Notepad++ to fix something in a huge text file for work, can't remember what it was now but it did sort it, might be worth a look maybe if you haven't already
notepad++: https://notepad-plus-plus.org/
RegEx Cheat Sheet for Notepad++:
https://www.linguisticsweb.org/doku.php ... ex-notepad
notepad++: https://notepad-plus-plus.org/
RegEx Cheat Sheet for Notepad++:
https://www.linguisticsweb.org/doku.php ... ex-notepad
Wiki homepage for my OXP: http://wiki.alioth.net/index.php/Griff_Industries
- Wildeblood
- ---- E L I T E ----
- Posts: 2466
- Joined: Sat Jun 11, 2011 6:07 am
- Location: Western Australia
- Contact:
Re: Recommended tool for extensive editing of planetinfo.plist ?
Hello, it is I, Mr Grumpy.
Did someone just ask for a user-friendly tool and receive replies including the word "regexes"?
In your heart, you know it's flat.
Did someone just ask for a user-friendly tool and receive replies including the word "regexes"?
In your heart, you know it's flat.
-
- ---- E L I T E ----
- Posts: 678
- Joined: Sat Aug 09, 2014 4:16 pm
Re: Recommended tool for extensive editing of planetinfo.plist ?
Taking regex out of the equation, awk could also be a good tool (mainly for the type of operation mentioned, i.e. involving just doing math on a value). It would also easily allow getting messy with regex.
Re: Recommended tool for extensive editing of planetinfo.plist ?
Fair!Wildeblood wrote: ↑Fri Nov 29, 2024 4:02 pmDid someone just ask for a user-friendly tool and receive replies including the word "regexes"?
A user friendly tool can be built using command line tools. The command line tools themselves are often not immediately friendly.
Here's a crude tool to double sun distances on economy=0 planets. Works on Linuxmint 21.3, and probably almost any current-ish Linux with gnu-sed and bash. It works on a temporary file, which can be copied to the live installation once inspected.
Code: Select all
#!/bin/bash
thefile=~/GNUstep/Applications/Oolite/oolite.app/Resources/Config/planetinfo.plist
testfile=/tmp/testme.plist
cp $thefile $testfile
getgoodbits(){
grep -Ee '^\s*"[0-9]+\s+[0-9]+"\s+=\s+{' -e sun_distance -e economy "$thefile" |\
sed -E '
s/^\s*//
s/^"([0-9]+) ([0-9]+)\s*"\s*=\s*\{\s*$/\1,\2 /
s/^economy\s*=\s*([0-9]+)\s*;\s*$/e\1 /
s/^sun_distance\s*=\s*([0-9]+)\s*;\s*$/\1 !/
' |\
tr -d '\n' | tr '!' '\n' | grep " e0 " |\
while read x ; do
echo "${x%% *} ${x##* }"
done
}
mashplist(){
while read i ; do p=${i% *} # planet index
x=${p%,*} # p left
y=${p#*,} # p right
s=${i#* } # sun_distance
d=$[s*2] # double sun distance
echo -e "\rDoing planet $x $y with sun_distance old=$s new=$d "
# We have to dip in and out of double quotes to use bash strings in sed.
sed -E '
/^\s*"'"$x"'\s+'"$y"'"\s*=\s*\{/,/^\s*\}\s+$/{
/^\s*sun_distance\s*=\s*[0-9]+\s*;\s*$/{
s/[0-9]+/'"${d}"'/g
}
}
' $testfile > ${testfile}.tmp && mv ${testfile}.tmp $testfile ; done
echo # reset prompt
}
getgoodbits | mashplist
This is brutal and slow as it parses the whole file for each item. It would be far nicer to do it in python or perl, sucking up the whole file first, and processing it in memory, however, I find bash a very quick way to prototype stuff.
...and it works
So, in summary. If there wasn't an easy tool to do that, there is now!
Edit: I've uploaded the edited plist file to testplanet.zip and a copy of the script as mashplist.zip. Those are temporary file shares which will conk out in a month.
-
- ---- E L I T E ----
- Posts: 678
- Joined: Sat Aug 09, 2014 4:16 pm
Re: Recommended tool for extensive editing of planetinfo.plist ?
If the comprehensibility of the regex is a challenge, then this awk code might be easier to grasp:
Saving this as a "double_sun_distance.awk" file, can be run as
The new file, new_planetinfo.plist will be the one you can use as a replacement.
Funny fact, if you didn't want to waste disk space with a new file, the code can be passed as an one liner to awk with the same results.
This should work as-is in Linux and/or macOS.
For Windows, you'd have to either get a Win32 (i.e. native) version of awk, or install minimal mingw64, msys, cygwin solutions, or do a whole WSL altogether.
Code: Select all
{
if ($1=="economy")
{
if ($3=="0;")
here=1;
else
here=0;
}
if (($1=="sun_distance") && here)
{
split($3, val,";");
print sprintf("\t\tsun_distance = %d;", val[1]*2);
}
else
{
print $0;
}
}
Code: Select all
awk -f double_sun_distance.awk planetinfo.plist > new_planetinfo.plist
Funny fact, if you didn't want to waste disk space with a new file, the code can be passed as an one liner to awk
Code: Select all
awk -- '{if ($1=="economy") {if ($3=="0;") here=1; else here=0;} if (($1=="sun_distance")&&here) {split($3, val,";"); print sprintf("\t\tsun_distance = %d;", val[1]*2);} else {print $0;}}' planetinfo.plist > new_planetinfo.plist
This should work as-is in Linux and/or macOS.
For Windows, you'd have to either get a Win32 (i.e. native) version of awk, or install minimal mingw64, msys, cygwin solutions, or do a whole WSL altogether.
- hiran
- Theorethicist
- Posts: 2410
- Joined: Fri Mar 26, 2021 1:39 pm
- Location: a parallel world I created for myself. Some call it a singularity...
Re: Recommended tool for extensive editing of planetinfo.plist ?
Is it that?
Someone asks for user friendly plist editing and we end up comparing regex, sed and awk?
I believe the real painpoint is the structure of the file. Forget a quote or a comma and you are gone. Neither of the tools under discussion help here.
But what does?
I never really investigated but bumped over this:
https://github.com/corpnewt/ProperTree
Would like to hear how that works or what other software in that category would exist.
Someone asks for user friendly plist editing and we end up comparing regex, sed and awk?
I believe the real painpoint is the structure of the file. Forget a quote or a comma and you are gone. Neither of the tools under discussion help here.
But what does?
I never really investigated but bumped over this:
https://github.com/corpnewt/ProperTree
Would like to hear how that works or what other software in that category would exist.
Sunshine - Moonlight - Good Times - Oolite
Re: Recommended tool for extensive editing of planetinfo.plist ?
If I would like to make more such changes, I would probably start with extracting each planet to separate file.
Then filtering and processing some types of planets will be lot easier.
And finally I will assemble the planetinfo.plist together from those files.
Then filtering and processing some types of planets will be lot easier.
And finally I will assemble the planetinfo.plist together from those files.
Lady of Fate, we adore you . . .
- Wildeblood
- ---- E L I T E ----
- Posts: 2466
- Joined: Sat Jun 11, 2011 6:07 am
- Location: Western Australia
- Contact:
Re: Recommended tool for extensive editing of planetinfo.plist ?
Specifically, 8 x 256 = 2048, but are we sure not every property is writable by scripting? Could you give an example?
In your heart, you know it's flat.
-
- ---- E L I T E ----
- Posts: 678
- Joined: Sat Aug 09, 2014 4:16 pm
Re: Recommended tool for extensive editing of planetinfo.plist ?
The plist format was "inherited" by Apple from [NEXT|Open]Step. The Linux/Windows GNUStep was/is aligned with its initial "text" format.hiran wrote: ↑Fri Nov 29, 2024 10:46 pmIs it that?
[...]
I never really investigated but bumped over this:
https://github.com/corpnewt/ProperTree
Would like to hear how that works or what other software in that category would exist.
Meanwhile, Apple deprecated the "text" format, for the XML and/or binary formats (also supported by GNUStep). Most recent (visual/user friendly?) plist tools would only deal with the XML and binary formats, including the ProperTree app you mentioned (using python plistlib library). I didn't try it, by right at the beginning of plist.py file (line 22) there is this:
Code: Select all
try:
FMT_XML = plistlib.FMT_XML
FMT_BINARY = plistlib.FMT_BINARY
except AttributeError:
FMT_XML = "FMT_XML"
FMT_BINARY = "FMT_BINARY"
I have the feeling that any user friendly plist editor (regardless of the file format) would have real trouble trying to do the type of changes described here
Re: Recommended tool for extensive editing of planetinfo.plist ?
This thread is turning up some goodies.
Good shout hiran, I'd forgotten about propertree... I used it a little while back. Great tool. Fine for editing, but not programmable to do what's been asked for here IIRC.
Awk.. I love the example above from Commander_X, and I'll bookmark it as a good example. I wish I'd had cause to fall further into the awk rabbit hole. I use pinches of it it in pipelines for simple things such as printing the fourth, second, then first word from each line when e.g. parsing csv's in shell scripts, but am not fluent in it. The example above is far cleaner than my bash-smashery, and certainly easier to port across platforms. I'll bet there's less Gnu vs BSD divergence than with sed and grep.
The awk script as-is relies on the keys in the xml being alphabetically sorted, so economy MUST be above sun_distance, which is happily true in this case.. I had set out aiming to make it work regardless the order of keys in a system block, but when I noticed the rigid ordering I took advantage So my script above is prone to the same potential failing upon adaption to other plists. It may have been simpler had I realised sooner.
I'd like to add to the heap of tools: plistutil, a tool to convert from/to binary/xml plist formats. Available on many distros via apt get plistutil, or from https://libimobiledevice.org. This is something propertree can do, but plistutil may be more useful as a command line tool when batch processing. A binary plist could be turned into xml, processed by whatever-script, then converted back, all without user interaction.
I agree that chunking it up into per-planet files would be a better approach to doing this sort of thing. Certainly so when using simple tools for batch edits like bash+sed/awk/grep. This would take some of the pain out of making a one-liner to test for a value and mangle a system accordingly, and thus simplify much of the potential line-ordering minefield.
So, here's a way to do that.
This works for me to split systems out to separate files from the planetinfo.plist. It adds zero padding to the system numbers to allow for easier sorting/searching/re-assembly.
I manually chopped the top and tail from the xml, I'd suggest putting them in the output directory as a_top and z_tail. Then you can simply cat * > output.plist or whatever to re-assemble. The filenames should be just fine for wildcard jiggery-pokery.
Here's the top of the directory:
On my laptop it took 14 seconds to run. Considering how clunky the script is, I'll take that. Ripping the plist apart is not likely to be done very often, so it's not worth optimising.
Re-assembly is somewhat faster:
Here's a zip with the script and broken down planetinfo.plist as 2050 files in a directory chunky.zip. The link will expire in 30 days.
I know there are going to be more succinct ways to do this, but I like cobbling stuff together as one-liners in bash, then editing them into tools.
'd be happy to see a better, cross-platform method appear here.
Good shout hiran, I'd forgotten about propertree... I used it a little while back. Great tool. Fine for editing, but not programmable to do what's been asked for here IIRC.
Awk.. I love the example above from Commander_X, and I'll bookmark it as a good example. I wish I'd had cause to fall further into the awk rabbit hole. I use pinches of it it in pipelines for simple things such as printing the fourth, second, then first word from each line when e.g. parsing csv's in shell scripts, but am not fluent in it. The example above is far cleaner than my bash-smashery, and certainly easier to port across platforms. I'll bet there's less Gnu vs BSD divergence than with sed and grep.
The awk script as-is relies on the keys in the xml being alphabetically sorted, so economy MUST be above sun_distance, which is happily true in this case.. I had set out aiming to make it work regardless the order of keys in a system block, but when I noticed the rigid ordering I took advantage So my script above is prone to the same potential failing upon adaption to other plists. It may have been simpler had I realised sooner.
I'd like to add to the heap of tools: plistutil, a tool to convert from/to binary/xml plist formats. Available on many distros via apt get plistutil, or from https://libimobiledevice.org. This is something propertree can do, but plistutil may be more useful as a command line tool when batch processing. A binary plist could be turned into xml, processed by whatever-script, then converted back, all without user interaction.
I agree that chunking it up into per-planet files would be a better approach to doing this sort of thing. Certainly so when using simple tools for batch edits like bash+sed/awk/grep. This would take some of the pain out of making a one-liner to test for a value and mangle a system accordingly, and thus simplify much of the potential line-ordering minefield.
So, here's a way to do that.
This works for me to split systems out to separate files from the planetinfo.plist. It adds zero padding to the system numbers to allow for easier sorting/searching/re-assembly.
Code: Select all
#!/bin/bash
# ! and % are absent in the plist, so I can use them as keys.
infile=~/GNUstep/Applications/Oolite/oolite.app/Resources/Config/planetinfo.plist
outdir=planets
mkdir -p "$outdir"
sed -E '
/^\s*"[0-9]+\s+[0-9]+"\s*=\s*\{/,/^\s*\}\s+$/{
s/$/!/g
}
' "$infile" | grep -e '\!$' -e '\%$' | tr -d '\n' |\
sed 's/\!\!/\!%/g' | \tr '%' '\n' |\
while read p ; do
s=$(
echo "$p" | sed -E '
s/^\s*"([0-9]+)\s+([0-9]+)"\s*=\s*\{.*$/\1-\2/
'
)
x=${s%-*}
y=$[ ${s#*-} +1000 ] ; y=${y#1} # add leading zeroes
echo -en "\rSystem ${x}-${y} "
dat=$(
echo "$p" | tr '!' '\n' | tr -d '%' > "$outdir/p_${x}-${y}.frag"
)
done
echo #reset prompt
Here's the top of the directory:
Code: Select all
$ ls planets/ | head -n 4
a_top
p_0-000.frag
p_0-001.frag
p_0-002.frag
p_0-003.frag
Re-assembly is somewhat faster:
Code: Select all
$ time cat planets/* > newplanetinfo.plist
real 0m0.060s
user 0m0.024s
sys 0m0.035s
I know there are going to be more succinct ways to do this, but I like cobbling stuff together as one-liners in bash, then editing them into tools.
'd be happy to see a better, cross-platform method appear here.
- hiran
- Theorethicist
- Posts: 2410
- Joined: Fri Mar 26, 2021 1:39 pm
- Location: a parallel world I created for myself. Some call it a singularity...
Re: Recommended tool for extensive editing of planetinfo.plist ?
So how about two 'simple and reusable' scripts then?
One to turn the planetinfo file into a CSV, another one to convert it back to plist (whichever format).
With that, a user can start editing in a spreadsheet application (Microsoft Office, OpenOffice or LibreOffice) or load it into some SQL database to do stuff. Editing gets easier, pivot tables, filters etc allow to have an overview and finally we can go all the way back to Oolite.
One to turn the planetinfo file into a CSV, another one to convert it back to plist (whichever format).
With that, a user can start editing in a spreadsheet application (Microsoft Office, OpenOffice or LibreOffice) or load it into some SQL database to do stuff. Editing gets easier, pivot tables, filters etc allow to have an overview and finally we can go all the way back to Oolite.
Sunshine - Moonlight - Good Times - Oolite
Re: Recommended tool for extensive editing of planetinfo.plist ?
pinfo2csv, csv2pinfo.. I guess this particular case does lend itself easily as long as one doesn't want to involve the top two blocks.
- Redspear
- ---- E L I T E ----
- Posts: 2694
- Joined: Thu Jun 20, 2013 10:22 pm
- Location: On the moon Thought, orbiting the planet Ignorance.
Re: Recommended tool for extensive editing of planetinfo.plist ?
Thanks everyone. I didn't expect so much help so quickly
Thanks especially for the examples.
Right, I've got some experimenting to do...