Understanding commodities.plist

[Commander McLane]

I messed up the thread splitting. This post was supposed to be at the top:

The OXPs help as well. I was at Zaonce (I think), and I had some very tasty trading going in-system by ferrying stuff between the Ho0py Casino, a Space Dredger, and the PI-42 store. Got gemstones from the dredger which sold well at the casino, where I picked up furs to sell at the space station etc.

The Dictatorships OXP offers some really nice trading options too, in the Imperial factories (and you can dock in a non-rotating station! Just watch out for other traffic, though, and be aware that you can't save the game there): furs, in particular, can have have wildly fluctuating prices. I've seen them on offer from anywhere between around 84Cr/tonne and -- I kid you not -- 0.8Cr/tonne. In an Industrial system, no less. Also they can offer some very tasty selling prices for precious metals: I've seen up to 48Cr/kilo for gold and over 80Cr/kilo for platinum. Some pirate action is to be expected, of course, but with fuel scoops and a little bit of effort, that's just more free stuff to sell! Oh, and ship upgrades and repairs are slightly cheaper there, too.

-- Ahruman

furs, in particular, can have have wildly fluctuating prices. I've seen them on offer from anywhere between around 84Cr/tonne and -- I kid you not -- 0.8Cr/tonne.

That's definitely a bug (read: bad scripting). I can't imagine it to be intentional.

The problem is that no commodity-price can ever exceed 102.0 Cr (that's 255 * 0.4). If it is set (read: calculated) to a value higher than that, these 102 Cr are set to zero again, as the final calculation step before multiplying by 0.4 is a 'AND 255'-operation, which resets every bit above the 8th. So a price-calculation that is meant to result in 102.8 Cr will in fact result in 0.8 Cr.

Whoever scripts commodities.plists should do his maths well, or all the Jamesons will get unforeseen opportunities.

[TGHC]

Wildly fluctuating prices on gemstones occasionally happens in Hoopy's too.

[Commander McLane]

It's the same for narcotics in the default range of commodities. It may be, though, that this is intentional. At least I can't imagine Giles failing to use his own programmed calculating routines correctly.

[Disembodied]

With regard to the Imperial Factory prices, I prefer to rationalise it as the result of Dictatorship systems' reprehensible collusion with local pirates. These so-called "factories" are often little more than clearing-houses for ill-gotten loot. Furs prices bottoming out is a sure sign that some poor Anaconda, hauling a load of pelts, got nixed just shortly before. With the factory awash with fur, it's no wonder they're practically giving it away. It's a disgrace, really. GalCop should crack down on it. Still, in the meantime, it would be a shame to let those furs go to waste...

[Ramirez]

Whoever scripts commodities.plists should do his maths well, or all the Jamesons will get unforeseen opportunities. :wink

That'll be me then! I think my intention was to get furs available as highly-priced rare items, the idea being that an industrial factory wouldn't usually have that sort of thing in stock. However, looking at the plist I think the price and quantity adjusters do seem rather excessive and so are probably causing the rollover effect you describe. To be honest I never did understand exactly how the pricing mechanism worked and so I had to resort to trial and error - if you can give me a simple explanation I'd be grateful (some specific queries I had are over here). I like Disembodied's take on the fiction though!

The good selling prices for metals are intentional, so it's good to see people making use of them. The idea is that a factory's main imports are of course raw materials, with precious metals being particularly sought after.

I still haven't got round to building the AstroFarms I was planning for the agricultural dictatorships, though I have got a prison ship on file somewhere. I'll aim to release that once I've finished the mission I'm working on.

Back to the topic in hand, I'm still on an earlier version of Oolite so haven't used the yaw thrusters. Do NPCs get this degree of control as well?

[Commander McLane]

Whoever scripts commodities.plists should do his maths well, or all the Jamesons will get unforeseen opportunities.

That'll be me then! I think my intention was to get furs available as highly-priced rare items, the idea being that an industrial factory wouldn't usually have that sort of thing in stock. However, looking at the plist I think the price and quantity adjusters do seem rather excessive and so are probably causing the rollover effect you describe. To be honest I never did understand exactly how the pricing mechanism worked and so I had to resort to trial and error - if you can give me a simple explanation I'd be grateful.

You're welcome, Ramirez.

*** WARNING: HOPEFULLY SIMPLE STEP-BY-STEP, BUT RATHER EXCESSIVE ANSWER FOLLOWING ***

All you need to know is in the commodities.plist document in the wiki. The formula for calculation of prices is this:

Code: Select all

price = (MARKET_BASE_PRICE + (MARKET_RND & MARKET_MASK_PRICE) + (economy * MARKET_ECO_ADJUST_PRICE)) & 255

Now what you have to consider is that this is a formula for a binary calculation, not decimal. Just think of the price as one byte, ranging in value from 0..255. The '&'-operation that is performed a couple of times is a logical AND applied to the 8 bits of the byte. Therefore the final '& 255' cuts out everything above these 8 bits. So if the sum has reached 256 before this step it will be Zero afterwards, because 256 & 255 = 0. To make this clearer I write the same as binary numbers:

Code: Select all

     11111111
AND 100000000

The AND sets only those bits of the resulting byte to '1', where both of the bits in the summands are '1'. In this equation this is not the case, as either the upper or the lower line reads '0'. So the result is

Code: Select all

     11111111
AND 100000000
=============
    000000000

The same kind of calculation is applied to the '(MARKET_RND & MARKET_MASK_PRICE)'-part.

The '+'- and '*'-operators are handled in the same way as if they were decimal operators.

So, given your current entries for furs in the commodities.plist, their price is calculated as follows step by step. We start from the left side of the equation and take the arguments in brackets as one:

MARKET_BASE_PRICE: Well, that's the base from which the calculation starts. It has very little to do with the final price, as it's going to be manipulated in the rest of the equation. And it can be manipulated so harshly, you won't recognize it in the end result. For the furs in the astrofactory it's 20.

Code: Select all

price = (20 + (MARKET_RND & MARKET_MASK_PRICE) + (economy * MARKET_ECO_ADJUST_PRICE)) & 255

(MARKET_RND & MARKET_MASK_PRICE): MARKET_RND ('market-random') is a random value from 0..255 that is produced by the engine whenever you enter a new system and is saved in the player's save-file. Obviously if you have a final range of prices from 0..255 with a rollover an added random factor of 0..255 is absurd, therefore the random that finally enters into the calculation is limited. This is done by masking the random-byte with MARKET_MASK_PRICE, which is ANDed with the random number. E.g. if you set it to 7, the random added to the final price can only be in the range of 0..7. 7 in binary notation is 00000111, so only the last three bits of MARKET_RND enter into the equation. Another example: If you set MARKET_MASK_PRICE to 65, things get more complicated, because the random won't be evenly divided in the range of 0..65, as it may look at first glimpse. In binary notation 65 reads 01000001, so only two bits, the first and the seventh from the right, are used. Only these bits of MARKET_RND, if set to '1', can make it into the equation through the mask. So only the values 0, 1, 64 and 65 can be added to MARKET_BASE_PRICE. This makes the price randomly either quite high or quite low with nothing in-between. And, last example, if MARKET_MASK_PRICE is set to 0, the random element is completely eliminated.

Now we run into a problem. MARKET_MASK_PRICE should always have a positive value, as negatives will behave unexpectedly. Due to the binary nature of the calculation performed in this bracket they will roll over as well (there is no '-' in the binary 8-bit-world). -1 will be treated as 255, -2 as 254 and so on.

Your MARKET_MASK_PRICE for furs is -1. Now this is the case mentioned just right now. Binary -1 equates to 255. That means the mask is fully open, any MARKET_RND from 0..255 will be added to MARKET_BASE_PRICE, the outcome is completely unpredictable. That also means that the next step, the influence of the systems economy, is meaningless, as it cannot change anymore the complete randomness of the prices. (From a systematic point of view, 'complete random' is the same as 'complete random + x'.)

We won't give up at this point, however, but take out the negative number. One would expect it to work like this: Negative numbers would be treated analogous to positives, but get subtracted. So the '-1' would mean the random in the equation is limited to leaving MARKET_BASE_PRICE as it is or subtract 1. In numbers: 20 - 0 = 20 or 20 - 1 = 19.

As we know now, it doesn't work like this. But we can get exactly the same effect by setting MARKET_MASK_PRICE to 1 instead of -1 and MARKET_BASE_PRICE to 19 instead of 20.

Code: Select all

price = (19 + (0..1) + (economy * MARKET_ECO_ADJUST_PRICE)) & 255

(economy * MARKET_ECO_ADJUST_PRICE): The next modification to the price depends on the system economy, Rich Industrial to Poor Agricultural, represented by a number from 0..7. The lower the number, the better the economy (have a look at the planetinfo.plist document in the wiki for all information on system variables).

MARKET_ECO_ADJUST_PRICE is a factor that determines how big a difference the economy makes for prices. It represents the step in which the price rises or sinks from economy to economy. If it is set to 1, the difference between two neighbouring economies will be 1, if it set to 10, the difference will be 10.

By choosing positive or negative values you determine whether a commodity will be cheap in agricultural worlds and expensive in industrial worlds, or vice versa. So here negatives are welcome. The 'economy'-value of a Rich Industial world is 0, so, given a small base price, the commodity will be cheap here. If you go down the economic levels to Poor Agricultural (= 7) the price will grow according to MARKET_ECO_ADJUST_PRICE. Let's say a commodity's base price is 0, there is no random and MARKET_ECO_ADJUST_PRICE is something big like 36. Then in a Rich Industrial system the price is 0 * 36 = 0, in an Average Industrial it's 1 * 36 = 36, in a Poor Industrial it's 2 * 36 = 72, in a Mainly Industrial it's 3 * 36 = 108, and so on up to the Poor Agricultural with 7 * 36 = 252. The result is a huge fluctuation, covering almost the whole possible range, depending only on the economy. Now if you have a big base price like 252 and set MARKET_ECO_ADJUST_PRICE to -36 you get the same steps and range, but this time the commodity will be cheap in low economy worlds and expensive in high economies. And, of course, if you set MARKET_ECO_ADJUST_PRICE to a low level the price differences between the economies will be minute. If you set it to 0 there won't be any. If you do so, the price will only vary randomly, as determined in the first brackets, and therefore be completely unforeseeable. (And if you set MARKET_MASK_PRICE and MARKET_ECO_ADJUST_PRICE both to 0, the price will be the base price wherever you are. Period.)

MARKET_ECO_ADJUST_PRICE for furs in astrofactories is set to -9, which means the price shall drop from industrial to agricultural worlds.

Code: Select all

price = (19 + (0..1) + (ONEOF -63, -54, -45, -36, -27, -18, -9, 0)) & 255

So, what do we have up to now? MARKET_BASE_PRICE is 19, random adds 0 or 1. The second bracket will subtract a value from 0 to -63 (= 7 * -9) according to the economy. Let's list the possible results:

Code: Select all

MARKET_BASE_PRICE + (MARKET_RND & MARKET_MASK_PRICE) + (economy * MARKET_ECO_ADJUST_PRICE)

the first bracket results to either 0 or 1, so I list both possibilities:

Rich Industrial:      19 + 0 + (0 * -9) = 19
            or        19 + 1 + (0 * -9) = 20
Average Industrial:   19 + 0 + (1 * -9) = 10
            or        19 + 1 + (1 * -9) = 11
Poor Industrial:      19 + 0 + (2 * -9) =  1
            or        19 + 1 + (2 * -9) =  2
Mainly Industrial:    19 + 0 + (3 * -9) = -8
            or        19 + 1 + (3 * -9) = -7
Mainly Agricultural:  19 + 0 + (4 * -9) = -17
            or        19 + 1 + (4 * -9) = -16
Rich Agricultural:    19 + 0 + (5 * -9) = -26
            or        19 + 1 + (5 * -9) = -25
Average Agricultural: 19 + 0 + (6 * -9) = -35
            or        19 + 1 + (6 * -9) = -34
Poor Agricultural:    19 + 0 + (7 * -9) = -44
            or        19 + 1 + (7 * -9) = -43

Now you see the rollover effect, marked by breaking the zero-barrier. The result of the final ' & 255' in the equation is that 256 will be added to all negative values, so we get the following list of possible values:

Code: Select all

Rich Industrial:       19 or  20
Average Industrial:    10 or  11
Poor Industrial:        1 or   2
Mainly Industrial:    248 or 249
Mainly Agricultural:  239 or 240
Rich Agricultural:    230 or 231
Average Agricultural: 221 or 222
Poor Agricultural:    212 or 213

Above I called this final result the 'price', but that's not really true. To get the final price from the numbers shown they are multiplied by 0.4. So on the market screen the player should see the following prices (and due to our calculation these prices only):

Code: Select all

Rich Industrial:       7.6 or  8.0 cr
Average Industrial:    4.0 or  4.4 cr
Poor Industrial:       0.4 or  0.8 cr
Mainly Industrial:    99.2 or 99.6 cr
Mainly Agricultural:  95.6 or 96.0 cr
Rich Agricultural:    92.0 or 92.4 cr
Average Agricultural: 88.4 or 88.8 cr
Poor Agricultural:    84.8 or 85.2 cr

I guess that's not what you intended. The price is very high in Mainly Industrial Dictatorships, then lowers slowly in Agricultural Dictatorships, and surprisingly drops in the well industrialised systems, with its absolute low in Poor Industrial Dictatorships.

So, what could you do? I am now assuming you want the price to be highest in Rich Industrial systems and lowest in Poor Agricultural ones, but on a high level in all of them, as furs are rare in astrofactories. If so, then you just need to adjust the base price to a high value. So set MARKET_BASE_PRICE to 254, which will allow for the peak price of 102 cr (255 * 0.4). If you don't want the peak price, then subtract just a little bit from MARKET_BASE_PRICE and set it to something like 251 or 246.

The next thing I would introduce is a little more random flucuation, by setting MARKET_MASK_PRICE to 3 or even 7. If you set it to 15 the random fluctuation will become larger than the economic steps, meaning that the highest possible price in a system with any economic level is higher than the lowest possible price in the system one level higher. The price-scales will overlap somehow. Therefore trading up the economical chain wouldn't always guarantee a profit. (Well, it still would if you trade to a system two economic steps up. And on the other hand, I don't suppose trading from one astrofactory to another a main trade route.) So, why not 15?

Note, however, that a change of MARKET_MASK_PRICE will also require you to change MARKET_BASE_PRICE in the opposite direction. If a random value of up to 15 can be added, you have to reduce the base price by these 15, or you get another rollover again.

These changes would result in the following list of results to the equation and prices:

Code: Select all

MARKET_BASE_PRICE = 240   MARKET_MASK_PRICE = 15

Rich Industrial:      240..255 => 96.0--102.0 cr
Average Industrial:   231..246 => 92.4--98.4 cr
Poor Industrial:      222..237 => 88.8--94.8 cr
Mainly Industrial:    213..228 => 85.2--93.2 cr
Mainly Agricultural:  204..219 => 81.6--87.6 cr
Rich Agricultural:    195..210 => 78.0--84.0 cr
Average Agricultural: 186..210 => 74.4--80.4 cr
Poor Agricultural:    177..201 => 70.8--76.8 cr

The credits rise always in steps of 0.4 cr.

If you're happy with that, than your commodities.plist entry will look like this:

Code: Select all

		<array>
			<string>Furs</string>
			<integer>0</integer>
			<integer>0</integer>
			<integer>240</integer>
			<integer>-9</integer>
			<integer>-9</integer>
			<integer>0</integer>
			<integer>15</integer>
			<integer>-1</integer>
			<integer>0</integer>
		</array>

Now you have three possibilities for tweaking with it:

(1) You can lessen the influence of random by reducing the '15' to 7, 3 or 1, while at the same time raising the base price by the same amount you have subtracted off the '15'. Of course you can also increase the randomness by setting the '15' to 31 or 63, while at the same time reducing the base price accordingly.

(2) You can widen the price-gap between different economies by exchanging the first '-9' with an even lower negative value (e.g. -15). Or you can close the gap by setting it closer to 0.

(3) And of course you can reduce the base price, making the furs cheaper all over the Dictatorship astrofactories.

*************

Finally a word on the quantity-calculation: The formula is almost the same, the only difference being a minus in front of the second brackets, where there is a plus in the price formula. And there are some restrictions that don't apply to the price calculation. These are namely (quoted from wiki):

Code: Select all

// if the quantity gets too high it's zeroed
if (quantity > 127) quantity = 0

// limit the quantity to 0..63 units
quantity &= 63

So the calculation should not come up with a value above 63, if possible.

Otherwise the same applies as to prices, namely: MARKET_MASK_QUANTITY shouldn't be negative. So perhaps have a fiddle with that as well.

Now I hope you're still there and could follow this non-brief explanation. If you've got further questions, you're welcome as well.


Greetings
Commander McLane

[JensAyton]

So… does anyone know why things are calculated this way? There’s no explanation in the code. Is it a carry-over from Elite (were there occasional super-cheap drugs in Elite)? Possibly it should be using MIN(MARKET_BASE_PRICE + (MARKET_RND & MARKET_MASK_PRICE) + (economy * MARKET_ECO_ADJUST_PRICE)), 255). Perhaps the mask, or whether to use a mask, should be set in commodities.plist?

[Commander McLane]

I can only guess it's a heritage from old Elite. I don't have any clear memories of the commodity prices in former versions, so I couldn't say whether there were cheap narcotics as well. I guess so, as I assume that Giles remodeled the old system accurately.

Personally I would really like to be able to influence the prices and quantities of commodities more and easier, so I'd say we don't need to stick to the complicated formulas.

Perhaps someone could come up with a simpler solution? Or perhaps that's not so easy, and thus Giles chose this way??

[JensAyton]

Perhaps what the world needs is… JavaScript scripts attached to commodities! :-p

[Ramirez]

Many thanks - I won't pretend I understood all of that on the first read through, but I'll take it away and give it a try.

It's been a while but I'm pretty sure narcotics fluctuated quite a lot in the original Elite - usually they were only as profitable as computers so not worth your trouble but occasionally you did stumble across prices well below the norm.

[Helvellyn]

You're right that narcotics could fluctuate wildly (but often there were none in stock when dirt cheap). I'm guessing that this was quite deliberate, in order to try to tempt the clean commander into illegal activity (there's no point in taking the risk if it's not worth it).

[Commander McLane]

@ Helvellyn: Yes, this is deliberate and caused by the formula.

Finally a word on the quantity-calculation: The formula is almost the same, the only difference being a minus in front of the second brackets, where there is a plus in the price formula.

As the price- and corresponding quantity-variables are mostly set to the same values, esp. both positive or both negative, the 'minus' instead of 'plus' means that they are negatively correlated. To put it very simple: When the price goes up, the stock goes down, and vice versa. (The correlation really is according to the economy, not directly between price and quantity, but the effect generally is like this simple rule of thumb.)

So, as the ridiculously low prices really are high prices that have rolled over, one has to expect the quantity to be very low, even zero. That's because a negative quantity causes a rollover as well, so it will be above 127 and automatically set to 0.

[Commander McLane]

MIN(MARKET_BASE_PRICE + (MARKET_RND & MARKET_MASK_PRICE) + (economy * MARKET_ECO_ADJUST_PRICE)), 255). Perhaps the mask, or whether to use a mask, should be set in commodities.plist?

I have thought about the proposed formula. It is good in it will hinder every rollover above 255. Two problems, though: (a) We might get a lot of 102.0 cr-prices, as I guess a lot of price calculations do roll over. Do we want that? (b) It does't hinder a rollover below 0. So we might get ridiculously high prices instead.

The second remark I don't understand. The mask is set in commodities.plist. The values of MARKET_MASK_PRICE and MARKET_MASK_QUANTITY are the mask, aren't they? And if they are set to 0 then the mask isn't used and the random element is eliminated.

And my general idea of setting of prices has changed a bit: Actually I like the formulas. If you have understood them (and yes, that takes some effort), the entries in commodities.plist are a good way to set the prices, along the three lines I have layed out above:

• (1) How much random do I want to have? (You just have to understand the mask-principle and set MARKET_MASK_PRICE to a value making sense.)
• (2) What shall the overall price-difference between economies, and therefore the maximal possible profit be? (You just have to set MARKET_ECO_ADJUST_PRICE to 1/8 of the maximum range you want.) and
• (3) In which general region do I want to set my prices? Low like food or high like computers? (You just have to set MARKET_BASE_PRICE to a high or low value.)
• Finally you have to calculate the extremes, the minimal and maximal possible price, and find out whether both are in the 0..255-range. If not, adjust.

And for quantities it's the same. So it's rather simple, if you know what all these values mean.

[JensAyton]

The second remark I don't understand. The mask is set in commodities.plist.

I’m referring to the 255. What I propose is simply to specify the maximum magnitude of the price as a modulus (255) or a logarithm (8, 2^8 - 1 = 255). The modulus approach is more flexible as it allows custom commodities to have a rollover that’s not tied to a power of two.

[Commander McLane]

Would that be for the (meaning instead of the) final &255 only? I agree totally with a modulus instead of this, which just means you can handle the roll-over point more flexible. And a modulus is for that superior to a 'powers of 2'-approach, where the next possible point is only 2^9-1=511.

But I wouldn't agree to exchanging the '&' in the MARKET_MASK_PRICE-calculation with a simple modulus for two reasons: (1) a range of 255 (= +-127) for a random factor seems sufficient for me. And (2) more important, the current binary mask structure allows for other steps than '1', if you nullify the last bits. Or for a random-addition that's either high or low, like I've explained in my original post. Both are not possible with a modulus.

I'm not sure, however, how a modulus would handle negative values. The current algorithm lets them roll over as well, so a small negative value is transformed into a high positive. What does a modulus do? The same, or would it transform the original x into |x|, so just erase the '-', which could result in new strange and unforeseen price-curves? (In Ramirez' example the roll over was backwards, and I'm sure there are enough commodities.plists in other OXPs that have the same.)

And of course, even with a customized roll over-barrier, the OXPer still would have to understand the whole model and make sure that the calculation with his variables doesn't break the self-set barrier.