OXPShipRegulator

[Svengali]

The following things are not meant to be a criticism on any oxp'er or meant to attack something. But we currently have some problems - so why not fix them? And thinking about it is half the way :-)

Memory:
We all can see that a lot of installed oxps can cause some trouble. But running out of memory is not necessary! There are several ways to reduce the amount of used mem. Reducing the vertice/face numbers, texture sizes or number of colors, less subents or escorts, avoiding of double declarations and a lot more. The other thing is ships with standard roles (trader, pirate,...). Give them a probability, maybe even a low probability and let the system populator do it's job. I know that every creator is proud and seeing the ships is fun, but if every oxp does it then we'll run into trouble. And thinking about leaving enough room for others is always the best way and then players can install most oxps (or even all) together!

Missions:
Currently there are several oxps around that are adding a lot of strong and dangerous ships in nearly every system by it's script. This can break missions - specially the early ones and the native Oolite missions - because the mission-ships can be destroyed before the player can reach them. So we (or any new commanders) will never have the fun we've had. And remember - most players won't ever read any readme's or other comments - 'Hey that looks cool! Oh!' - *** Press Space *** So I think we should change the scripts to add these ships only if the conditions are matched.

Stations:
And last but not least. Adding too many ships, specially with a lot of escorts, is annoying. The stations can't handle all these ships that are trying to dock. In the last days I've installed a lot of oxps and this is a problem. So we should solve this too. And escort missions are also affected - or do you really want to wait till 345 ships will dock before your mission ship gets docking clearance?

But solving the problems is only possible if we're doing this together. So let's start...

One possible way could be the OXPShipRegulator (part of OXPConfig). Take a look in it, but think about that it is a development version, so don't use it with your normal savedgame. I've included some documentation for scripters and ship-oxp makers. Read the docs carefully and lets discuss it.

OXPConfigSpecial1.07.zip (28KB)

What it does?
- Setting the strength of oxp-ships (only for oxps that are using the conditions/roles - changeable via arguments). A test shipdata is included to give you a feeling how it works.
- Automatic setting of strength if nothing is specified otherwise (default ON - switch on/off)
- Protects the native missions from Uber-ships (default On - switch on/off)
- Reduces the memory useage if necessary (removes ships that are out of range - default Off - switch on/off)
- Configurable via OXPConfig and the console and useable (arguments) for any script (JS and Legacy).

All these things have to be discussed and are not unchangeable - so now it's up to you!

[JensAyton]

There are several ways to reduce the amount of used mem. Reducing the vertice/face numbers, texture sizes or number of colors,

Technical note: reducing number of vertices or faces has no effect on memory in any release so far, but will in 1.73. Reducing the number of colours doesn’t either, unless you switch to greyscale textures.

The number one consumer of memory is textures. The most powerful way to reduce this is to reduce the number of different ships in the system. This means:

• As Svengali says, the less likely your ship is to appear, the less likely it is to cause memory issues.
• If you make a replacement for a built-in ship, make it appear instead of the original rather than keeping both around.
• Don’t make texture variants of your ship, or if you do, keep them localized and use a separate, low-resolution texture for those areas.
• Make your textures one power-of-two size smaller.

For users, the only real options are:

• Use fewer OXPs. Ones with many ships are more likely to be a problem than ones with few ships. Broadly speaking, the more disk space an OXP takes, the more memory it’s likely to use.
• Use reduced detail mode. This will scale down large textures.

[_ds_]

The number one consumer of memory is textures. The most powerful way to reduce this is to reduce the number of different ships in the system.

For others' benefit…
http://developer.berlios.de/feature/?fu ... up_id=3577
This will reduce texture memory usage by something approaching ¾ as things stand. This can be improved by disposing of unneeded alpha channels and, internally, using RGB rather than RGBA where possible.

libtxc-dxtn is required, at least on Linux. For Debian, this is available as libtxc-dxtn0 from debian-multimedia.[/i]

[ZygoUgo]

Mount and Blade has an external program that turns your mods into a binary file so it's quicker for the game to read (?), is it plausible to get something like this to read the addons folder and produce a more efficient file type?
I know absolutely nothing about programming so sorry if that's nieve.

[JensAyton]

Mount and Blade has an external program that turns your mods into a binary file so it's quicker for the game to read (?), is it plausible to get something like this to read the addons folder and produce a more efficient file type?

This would have very little effect on memory usage.

[Svengali]

Use reduced detail mode. This will scale down large textures.

I knew I've missed something :-)

I'm already very pleased with the current performance and stability of Oolite and when the shared textures/models comes it will free even more ressources. Thanks a lot for your (and all other dev-team-members) work.

And then - probably - such a tool will be obsolete. The mem-reduction is primarly meant as a helper for the current situation with some oxps, but better would be to change the oxps. I haven't tracked which entities are involved in burning so much ressources, but this is the next step, so we can find a solution that a) doesn't break the oxps or b) we'll kick the reduction out if it's not effective or c) if it doesn't fit the needs of oxp'ers.

The other thing is the conditions/roles in shipdata. You've written that using conditions in shipdata is slowing down the ship selection. Is it possible for a script to influence the populator before it does it's job? Or even better a 'native solution' for the mission problematics. It would be a lot more effective than any oxp can ever be and I'd be happy to kick this tool.

[Lestradae]

Use reduced detail mode. This will scale down large textures.

I knew I've missed something

Uh ... dumb question perhaps ... how?

[Screet]

Mount and Blade has an external program that turns your mods into a binary file so it's quicker for the game to read (?), is it plausible to get something like this to read the addons folder and produce a more efficient file type?
I know absolutely nothing about programming so sorry if that's nieve.

How often do you change the AddOn folders contents? Only then you notice the time to wait until things have been read.

I could see one big benefit of binary files: Those would require an editor, and with an editor it would be more difficult to make mistakes which are possible when mangling with the text files. But then someone would have to produce it and everytime something new comes in, the load/save routines would have to be adapted.

I bet the current way is the best one for now and the near future, although I do love binary files because of their efficiency.

Screet

[Svengali]

Use reduced detail mode. This will scale down large textures.

I knew I've missed something

Uh ... dumb question perhaps ... how? :oops:

Available in game options (where you can select loudness, fullscreen,...).

[Eric Walch]

I knew I've missed something

Uh ... dumb question perhaps ... how?

Available in game options (where you can select loudness, fullscreen,...).

I always had it off until half a year ago. You don't see the nice nebula with this off.

[JensAyton]

I missed one, er, detail: in addition to shrinking large textures, the Reduced Detail mode disables mip-mapping by default. This reduces the visual quality of textures, but reduces memory consumption by one quarter.

[JensAyton]

More on reducing memory problems: see max-texture-size in Hidden Settings in Oolite. Setting it to 512 will shrink the biggest textures, and setting it to 256 will shrink most textures (at the cost of noticeable blurriness up close).

[Svengali]

Thanks Ahruman, with all these options I think we don't need a cleanup via oxp anymore and probably there is also no need for the mission problematics. So I think I'll remove this tool from OXPConfig for the next release if nobody cries .-)

I've also found another way to reduce the mem-usage if some specific oxps are installed. One of the problems was (and still is) that some ships are using way too much FLASHER-declarations and this costs a lot of memory. The Constitution (for example) uses 251 FLASHERs and with all other subents and escorts it needs >200 MB. By removing the FLASHERs for testing it has freed up ~75 MB, And there are some other ships around with more than 50 FLASHERs, so I think oxp'ers should think about it if shaders are not the better way - if possible. Most of these ships were made long before shader-support was available, but this has changed.

[JensAyton]

Hmm. Flashers shouldn’t be using much memory. Interesting…

[Lestradae]

Hmm. Flashers shouldn’t be using much memory. Interesting…

Ah. Aha. I would throw in my observation that I think that since 1.72.2 the memory has been going down on my system - not absolutely sure as OSE was growing at the same time ...

I was already thinking about attempting to replace the flashers with shaders, but that's new territory to me and might concern not only the Constitution but about ten OSE ships.

So if the flashers thing should turn out to be a new (?) memory bug, I would be very grateful for its removal.