Page 1 of 1

Text handling in OXPs

Posted: Fri Oct 05, 2012 10:38 am
by JensAyton
I’m looking into a rewrite of the string expander, which handles things like [commander_name] and %I in strings. The current version is very slow, and its behaviour is odd in certain cases. I need to know if anyone is relying on the odd behaviours. It seems unlikely now that most OXP scripting is in JavaScript, but you never know.

The current version works like this:
  • Seatch the string from left to right looking for an open square bracket.
    • If found, search for a closing square bracket.
      • If found, attempt to look up the text between the brackets in various ways. If a replacement is found, inserts that where the bracketed text was, otherwise remove the brackets leaving only the text that was between them.
    • If there was a pair of brackets, it has now been removed; start over from the top in case there are more brackets.
  • Having eliminated all open brackets (there may still be closing ones), handle %H, %I, %R, %N and %J.
This is inefficient, in part because it starts over from the beginning of the string each time it’s expanded a bracket (which adds up in some cases, like system descriptions) and partly because inserting text in the middle each time around the loop involves more copying than necessary.

As for odd behaviours: the algorithm described above must expand the result of every previous expansion. For instance, if your name is “test[1]", "[commander_name]" would be expanded to "testvery", "testmildly", "testmost", "testreasonably" or "test" at random. The same thing can happen to mission variables, which could perhaps be useful at times but is probably undesirable overall.

In a string like "[some [nested] thing]", it will first try to look up "some [nested". If this fails, it will end up with "some [nested thing]", and try to expand "[nested thing]". From a programmery perspective, there are three saner options: only look up "some [nested] thing", try to expand "nested" first, or report an error. I’m leaning towards the first option, which is simplest. The alternatives would have very little overhead, but also very little value.

One possibly useful one, at least for legacy scripts, is that you could use "%J[mission_mySystemID]" to get the name of a system (for IDs > 99).


The way I’d like to do it works like this:
  • Scan the string from left to right, and consider each character.
    • If it is an open square bracket:
      • Scan to the matching closing square bracket. Nested pairs of opening and closing square brackets are skipped over.
        • If one is found, look up the result in various ways as before. However, if the match is found in descriptions.plist, expand the result in isolation before inserting it. Results looked up in other places (mission variables, legacy script methods) are not expanded.
        • If the lookup succeeded, replace the bracketed key with the result. Otherwise, leave it as it was.
    • If it is a percent sign:
      • If the next character is H, I, R or N, replace the two-character escape code with the appropriate value.
      • Otherwise, if the next character is J, and this is followed by three digits, replace the sequence with the name of the system specified by the three digits.
      • Otherwise, if the next character is %, [ or ], replace the escape code with the unescaped character.
(Although I say “replace” above, this can be implemented with much less copying than the current approach.)

I also want to add warnings for things like unmatched close brackets and percent signs not followed by known escape codes. The addition of %%, %[ and %] makes it possible to use these characters in strings that will be expanded.

Edit: another minor semantic change – it will no longer be possible to override numeric keys (like "[14]") with loose strings in descriptions.plist or the overrides parameter to JS expandDescription().

Re: Text handling in OXPs

Posted: Fri Oct 05, 2012 2:56 pm
by JensAyton
Additional oddities:
  • In the old implementation, it’s possible to put multiple mission variable and legacy script method expansions in one set of braces, like "[mission_foo mission_bar commanderName_string]" (n.b.: "[mission_foo mission_bar commander_name]" would not work).
  • If you set a titleKey in a JavaScript mission screen parameter object, mission variable names and legacy script methods in the loaded string are expanded even if they don’t have brackets around them. The same thing happens with several legacy script methods related to missions.
I’m pretty sure neither of these was ever documented, and I’d be surprised if anyone was using them. I intend to leave the legacy-script-specific part alone, and sane-ify the others.

Re: Text handling in OXPs

Posted: Sat Oct 06, 2012 3:34 pm
by Ironfist
ran truck 5396 and got the following errors

Code: Select all

15:35:47.000 [script.javaScript.warning.ooliteDefinedError]: ----- JavaScript warning (BGS-M 1.5): Unknown expansion key [myChatter] in string.
15:35:47.000 [script.javaScript.stackTrace]:  0 (../AddOns/BGS-M1.5.oxp/Config/script.js:153) <anonymous function>
15:35:47.000 [script.javaScript.stackTrace]:     this: [Script "BGS-M" version 1.5]
15:35:47.000 [script.javaScript.stackTrace]:     check: undefined
15:35:47.015 [script.javaScript.warning.ooliteDefinedError]: ----- JavaScript warning (snoopers 2.3.2): Unknown expansion key [SNOOPERS_2_G0] in string.
15:35:47.015 [script.javaScript.stackTrace]:  0 (snoopers.js:103) <anonymous function>
15:35:47.015 [script.javaScript.stackTrace]:     this: [Script "snoopers" version 2.3.2]
15:35:47.015 [script.javaScript.stackTrace]:     i: 2
15:35:47.015 [script.javaScript.warning.ooliteDefinedError]: ----- JavaScript warning (snoopers 2.3.2): Unknown expansion key [SNOOPERS_3_G0] in string.
15:35:47.015 [script.javaScript.stackTrace]:  0 (snoopers.js:103) <anonymous function>
15:35:47.015 [script.javaScript.stackTrace]:     this: [Script "snoopers" version 2.3.2]
15:35:47.015 [script.javaScript.stackTrace]:     i: 3
15:35:47.015 [script.javaScript.warning.ooliteDefinedError]: ----- JavaScript warning (snoopers 2.3.2): Unknown expansion key [SNOOPERS_4_G0] in string.
15:35:47.015 [script.javaScript.stackTrace]:  0 (snoopers.js:103) <anonymous function>
15:35:47.015 [script.javaScript.stackTrace]:     this: [Script "snoopers" version 2.3.2]
15:35:47.015 [script.javaScript.stackTrace]:     i: 4
15:35:47.015 [script.javaScript.warning.ooliteDefinedError]: ----- JavaScript warning (snoopers 2.3.2): Unknown expansion key [SNOOPERS_5_G0] in string.
15:35:47.015 [script.javaScript.stackTrace]:  0 (snoopers.js:103) <anonymous function>
15:35:47.015 [script.javaScript.stackTrace]:     this: [Script "snoopers" version 2.3.2]
15:35:47.015 [script.javaScript.stackTrace]:     i: 5
15:35:47.015 [script.javaScript.warning.ooliteDefinedError]: ----- JavaScript warning (snoopers 2.3.2): Unknown expansion key [SNOOPERS_6_G0] in string.
15:35:47.015 [script.javaScript.stackTrace]:  0 (snoopers.js:103) <anonymous function>
15:35:47.015 [script.javaScript.stackTrace]:     this: [Script "snoopers" version 2.3.2]
15:35:47.015 [script.javaScript.stackTrace]:     i: 6
and

Code: Select all

15:38:26.062 [script.javaScript.warning.ooliteDefinedError]: ----- JavaScript warning (ups_docs 1.7.7): Unknown expansion key [%I] in string.
15:38:26.062 [script.javaScript.stackTrace]:  0 (ups_docs.js:166) <anonymous function>
15:38:26.062 [script.javaScript.stackTrace]:     this: [Script "ups_docs" version 1.7.7]
15:38:26.062 [script.javaScript.stackTrace]:     targetSystem: undefined
15:38:26.062 [script.javaScript.stackTrace]:     infoList: undefined
15:38:26.062 [script.javaScript.stackTrace]:     chronicleSystem: 124
15:38:26.062 [script.javaScript.stackTrace]:     docsStatus: "YES"
15:38:26.062 [script.javaScript.stackTrace]:  1 (ups_docs.js:66) <anonymous function>
15:38:26.062 [script.javaScript.stackTrace]:     this: [Script "ups_docs" version 1.7.7]
15:38:26.093 [strings.expand.warning.unknownExpansion]: ----- WARNING: Unknown expansion key [Aleusquian] in string.
15:38:26.109 [strings.expand.warning.unknownExpansion]: ----- WARNING: Unknown expansion key [Aleusquian] in string.
so it looks like
Snoopers
UPS
and BGS-M1.5

are effected by the txt handing changes

Ironfist

Re: Text handling in OXPs

Posted: Sat Oct 06, 2012 5:54 pm
by JensAyton
I’m not entirely clear on what BGS is trying to achieve.

The Snoopers thing should still work. (The messages are warnings, not errors, and will not be shown in the standard release of the next version of Oolite, only the “developer version”.) It might be a good idea to add methods to test whether individual description and mission test keys can be expanded for uses like this.

The UPS one can reasonably be considered an OXP bug. There are several cases of [%I] in missiontext.plist that should be %I. This worked in the old code, but only by chance. Fixing it would be backwards-compatible.

Re: Text handling in OXPs

Posted: Sat Oct 06, 2012 6:32 pm
by Svengali
Ahruman wrote:
I’m not entirely clear on what BGS is trying to achieve.
BGS tries to make things compatible to other soundpacks / image packs without using a script in the patches for Capt Berf Pilot, Halsis, Hawksound and Customsounds and to give OXPs a way to implement sounds / music without a script. Additionally it checks if the image pack (BGS-I) is installed. Based on the found stuff it configures itself. It was a way to get rid of the loading order, because descriptions and missiontext is parsed already when the scripts startUp gets called.

I doubt that any other OXP is using the keys, so I guess it can be removed alltogether. For the patches and BGS-I a small script can be done and all is well.

Re: Text handling in OXPs

Posted: Mon Oct 15, 2012 7:07 am
by Eric Walch
I went through the oxp's that are stored on my box account. I found 3 with this kind of expansion and updated them all last week:

- Cargo_wrecks_teaser 1.7.1.oxp
- RandomHits1.4.15.oxp
- UPS-courier v1.7.9.oxp

In Cargo_Wreck I found also two times the expansion: [R]
This will just expand to 'R' and does do nothing in the end. I assume this was a bug by Arexack Heretic and he mend to use [%R]. I replaced it now with '%R'.

For 1.76.x users, there is no reason to update immediately to these new versions. Trunk users might want to, to eliminate the log warnings.