Oolite eating disk space by the GB?

[maik]

I've been observing loosing 2 to 3 GBs (!) of disk space while playing Oolite (version 1.74.2 on Mac OS X 10.6.4) and recovering the space again when I quit the game.

I don't have sufficient proof to point my finger at Oolite but all evidence suggests this. I only started noticing said behaviour when I got low on disk space on my main HDD in general. It only happens when I play for a longer period of time and the HDD space loss seems to correlate with the amount of time I'm playing (it feels like ca. 1GB per hour).

Has anybody else witnessed this?

[Kaks]

Yep, it's a standard windows 'feature' - I normally avoid it by assigning a fixed amount of space to the swap file, rather than having windows expand it willy-nilly... if you do that when the disk is quite empty & defragmented, it also avoids fragmenting the swap file, which keeps the computer more responsive too...

[Cmd. Cheyd]

Fragmentation of the swap file is really a non-issue since Vista / Win7 releases Kaks. Just FYI.

[Smivs]

... (version 1.74.2 on Mac OS X 10.6.4) ...

And I think Maik is a Mac user.

[Kaks]

Good point, I should try & read things!

Still, macs do use swap files too, which are used in pretty much the same way as windows swap files. There's an interesting article about changing the swap file location on macs, but it's only helpful if you have another, faster hard drive. Might help if you have a 4 gig memory stick you don't mind using for the swap file, I suppose...

Cmd. Cheyd, if the swap file length is not set as fixed, the system will still be unnecessarily busy - at least to move fragments of other files out of the way.

Personally I do find vista & 7 two of the most unresponsive OSs known to man, so probably it doesn't really make a huge difference either way, though...

[maik]

Thanks for all the replies. They do point to a possible (and likely) explanation. However, the fact that memory usage seems to grow over time does point to references to unused objects being held, doesn't it?

Since each hyperspace jump more or less takes me to a fresh game (meaning all objects that were spawned in the previous system are forgotten again) I would not expect memory usage to keep on growing...

I'll investigate a little more and monitor the trend.

[Cmdr James]

A fresh game in some sense, but I think all the things like textures are still in memory.

Not all ships are completely lost when you leave the system either, wormholes can remember who went through them, so there will be some stuff kicking around in memory.

[maik]

Fair enough, but even ships that left through wormholes will be forgotten on the next jump.

Are textures the culprit then? Is each texture kept in memory only once (so that memory usage flattens out once all textures are loaded--but I can't see them reaching the multi-GB barrier then) or is it copied for each new object that uses it (so that memory usage grows and grows and grows)?

[Cmdr James]

I believe that the textures are each loaded a maximum of once but I dont really understand that part of the code.
I would be surprised if there were many GB of textures, do you use lots of very pretty ship OXPs?

Ships in wormholes are remembered until they are due to arrive - this enables you to take a different route and ambush them (yes really).

[Eric Walch]

Are textures the culprit then? Is each texture kept in memory only once (so that memory usage flattens out once all textures are loaded--but I can't see them reaching the multi-GB barrier then) or is it copied for each new object that uses it (so that memory usage grows and grows and grows)?

No, textures are loaded and released during game. And just one copy is stored when different ships use the same texture. There is a debugging switch to turn logging of texture loading on. I think it is "mesh.load" in logcontrol.plist

It is even possible to change a texture file in an oxp while Oolite is running. When that texture was not yet in memory, Oolite will use the one you changed. And when it was in memory, Oolite will start using the new texture after a few jumps or just after flying for some time. A very clear sign that textures are not pre-loaded and are dumped when the memory is needed for new textures.

[Darkbee]

Swap space isn't an extension of your physical memory, in other words, if you have 2GB RAM and 2GB of swap space that doesn't mean you have 4GB of memory in total (in a psuedo-physical sense). Swap space is a mechanism to aid getting relevant data into physical memory faster. When something is no longer needed in physical RAM it is swapped to the virtual memory (swap space). Virtual memory holds on to stuff for as long as it possibly can based on the computing principle that if something was used recently then it is likely to be used again in the near future. In this sense, you virtual memory will grow indefinitely to the point where it is not able to grow any more (because of system configuration settings or physical limitations such as lack of disk space), at which point it will start dropping the oldest (typically) data to make space for newer data.

Having said all that, I agree that it does sound like excessive swap space use. Perhaps you have a lot of other background applications running... time to stop playing Oolite against an iTunes soundtrack. I'm almost afraid to suggest it but perhaps some of your RAM is corrupt and thus unusable?

[Cmdr James]

Darkbee, I think you are describing caching, which is different from swapping. Disk swap is exaclty extinding the amount of virtual memory by paging to disk.

[Darkbee]

Darkbee, I think you are describing caching, which is different from swapping. Disk swap is exaclty extinding the amount of virtual memory by paging to disk.

Extending the virtual memory? Or you mean physical memory? Where does the "cache" go then? I'm familiar with a browser cache but not a system cache (unless you mean like Windows Prefetch or some such). I guess my knowledge of memory paging is incomplete. Everyone feel free to disregard my first post.

[Cmdr James]

Im not a memory management expert, but I think it goes something like this:

Disks cache to their own internal memory.

Windows, and othe OSes cache which goes into the normal system memory.

When reading from disk, prefetch gets the data that it expects you to request next, hopefully making subsequent reads faster.

If windows runs out of physical memory, it starts allocating memory that doesnt really exist, and swaps sections of content from physical ram to space on disk (a swap file). The virtual memory of the superset of physical and swap space.

[Darkbee]

But as far as I know there is one fundamental difference between RAM and swap space (sometimes referred to as virtual memory): swap is non-addressable by standard user system API calls, thus it isn't just simply an extension of your RAM, its use is transparent to the user. Furthermore, RAM is changing constantly where swap space need not.

It doesn't really matter to argue the semantics of it, my whole point was that it's not inconceivable that swap space could grow outrageously large if not given suitable limits. I think the (old) general rule of thumb is 2 times your physical RAM, but this is not always a given these days due to better memory management modules of modern OS's. Indeed if you have a computer with 2GB of RAM then a swap space of equal disk capacity should be sufficient. As you increase RAM you can afford to maintain a lower ratio of swap space such that if you have 8GBs of RAM then you only need 4GBs of swap space. But that's by the by.

If Oolite dosen't do this on other Macs, then something else is at work on the original poster's system causing it to use so much swap space; typically other applications or processes running concurrently. Either that or the OP has a huge number of OXPs installed. Or in the worst case scenario has faulty RAM.