Multi Player Capability - I have development resources!!

[j105rob]

I am looking into trying to figure out what the level of effort would be to add networking and multiplayer capability to the game. I have a development team and intern resources that are looking to utilize oolite for an upcoming capture the flag event in October 2015.

Taking a quick look at the game I can see that this can be accomplished, but will take a level of effort to design correctly. What I am looking for is more insight into how the game engine is laid out and how certain aspects of the game are managed.

The discussion we are having currently is how are objects managed in the game. We have found the game timer and the UNIVERSE class.

Does this UNIVERSE class maintain the handles to every object in the game? Or is it more directly related to what object are currently around the player?

Is there a concept of player environment vs. game world?

Any help or insight would be very much appreciated

[Disembodied]

Whilst waiting for one of the developers to answer, you might find this thread useful:

https://bb.oolite.space/viewtopic.php?f=2&t=6173

[another_commander]

I am writing this from work so I have to be very brief and just give a few starting pointers.

Oolite has already networking capability but at a very basic level at the moment and not exactly multiplayer-oriented. It can use TCP/IP to exchange information with an external application (which could be running either on the same machine or anywhere on the internet), currently used for debugging and external control purposes. Check out the Oolite debug console TCP protocol page on the Oolite wiki.

The Universe class, very generally, handles what is happening in the system the player is currently in. It controls things like stellar objects and sky, NPCs, player, particles, input, NPC persistence elements etc), but at a "supervisor" level. It does contain handles to many objects that it needs access to, but I am not sure that this means all objects necessarily. Each class in the game has its own methods for initialization and activities and may contain references to objects it inherits from. All classes inherit from the class named Entity, which in turn inherits eventually from the GNUstep NSObject base class. Game Controller is another high-level class you might be interested in and and I am sure you have already seen it as it contains the game timer setup.

Not sure what exactly the concept of player environment vs game world means, but I can tell you that there is the concept of non-player-centric universe. As far as Oolite is concerned, the player is just yet another entity spawned into the current system. Together with the player, the game spawns a number of NPCs, each with its own AI and characteristics and then just lets everyone do their thing. That's why you will very often see battles happening in the distance, messages asking for help, ships going sun-skimming, traders hyperspacing out to their destination systems (and you can actually follow them and find them on the other side or, alternatively, be followed on the other side if it's you doing the jump) and so on. The player is not the centre of the action, rather just a small part of it. I hope that's pretty much what you refer to.

Hope this helps for a start. It would be interesting to see what could come out of an attempt to do multiplayer, but please do read the link posted by Disembodied before you start.

[cim]

Welcome to the forums.

I am looking into trying to figure out what the level of effort would be to add networking and multiplayer capability to the game.

That really depends how ambitious you want to be with the networking and multiplayer capability.

The debug protocol which another_commander has pointed at could be used to set up very basic multiplayer - 2 player arena deathmatch, lasers only - without any big conceptual changes. There might be a few tweaks needed here and there in the core code but nothing major. I think I've said "a few months" as an estimate for implementing that, given a prerequisite of familiarity with the existing code.

More complex than that - more players, or some NPC opponents/allies in the arena - would be possible, but maybe might overwhelm the debug protocol as it currently stands.

Multiplayer which allows 2 or more players to exist in the same normal system at once, playing something resembling the normal game, would be a lot harder and I think would be beyond the practical capabilities of the debug protocol. There would also be at least some gameplay changes needed to get it to work, though they might not be important ones.

Does this UNIVERSE class maintain the handles to every object in the game? Or is it more directly related to what object are currently around the player?

Universe is really several classes in one object - the game universe, the current system the player is in, and a bunch of miscellaneous stuff which needs refactoring out to various more logical places for it.

Universe owns the top-level in-game entities - anything you can see in-flight, including the player ship.
Other objects may be owned by the top-level entities (e.g. a ship might own subentities for animation, ease of modelling/texturing/reuse, etc.; or a wormhole owns the ships currently inside it). Entities are only considered within the current system - everywhere else doesn't really exist until the player goes there.

Non-entity objects (e.g. the GUI, the sound system, etc.) are sometimes owned by Universe but are more often singleton classes or owned by something else (though Universe may be the class responsible for triggering resets on them as part of the save/load process)

The player ship is of class PlayerEntity (a subclass of ShipEntity with heavy modifications) - when a ShipEntity has its update method called by Universe, it runs various AI routines; when PlayerEntity has its update method called, it polls the input devices.

Is there a concept of player environment vs. game world?

In the sense of the split between "current system" and "all systems", yes. Everything within the current system is a single environment (with a usable size of 10^14 metres, though only the inner 10^6 actually sees much use in practice). Everything outside the current system is basically ignored and abstract until the player goes there, though high-level properties can be queried and edited remotely.

Does this help with your questions?

[j105rob]

Awesome guys thanks for the info.

I am on the road right now, but will reply in more detail soon with what our goals are and the answers to your questions.

Cheers!

[j105rob]

Oolite has already networking capability but at a very basic level at the moment and not exactly multiplayer-oriented. It can use TCP/IP to exchange information with an external application (which could be running either on the same machine or anywhere on the internet), currently used for debugging and external control purposes. Check out the Oolite debug console TCP protocol page on the Oolite wiki.

TCP would not be a good choice for implementing comms as it adds too much chatter; UDP is the clear choice here.

The Universe class, very generally, handles what is happening in the system the player is currently in. It controls things like stellar objects and sky, NPCs, player, particles, input, NPC persistence elements etc), but at a "supervisor" level. It does contain handles to many objects that it needs access to, but I am not sure that this means all objects necessarily. Each class in the game has its own methods for initialization and activities and may contain references to objects it inherits from. All classes inherit from the class named Entity, which in turn inherits eventually from the GNUstep NSObject base class. Game Controller is another high-level class you might be interested in and and I am sure you have already seen it as it contains the game timer setup.

Seems like adding the game sync to the Entity class should be investigated. I assume that all Entities grok the timer, if not then a ref could easily be added.

I have been reading the other posts you recommended.

Thanks.

[cim]

Seems like adding the game sync to the Entity class should be investigated. I assume that all Entities grok the timer, if not then a ref could easily be added.

The main game loop is basically:

Code: Select all

GameController
|- Universe update
    |- top-level Entity update
        |- subentity updates, maybe
|- Universe display
    |- top-level Entity display
        |- subentity display, maybe

When the Universe update runs, this has a delta_t parameter to determine how much time to simulate (so the real time pace of simulation should stay at 1 second per second even if the FPS changes). That gets passed to the Entity updates, so they know how much time has passed since they were last asked to update.

The entities can read the global clocks with methods like [UNIVERSE getTime] or [PLAYER clockTimeAdjusted] but generally don't need to.

[j105rob]

cim: thanks for the update

Today we reviewed the debug code, understanding why plists were used, and seeing why comments about "total" rewrite have been tossed around.

Currently it appears that the game in single player mode runs its own clock and that UNIVERSE class is basically what the player is interacting with; this is what we would call the player's environment. In order to tie things together and make this a true multiplayer game, each player's environment's state and the state of all of the star systems must be maintained along with a centralized clock for synchronization.

To accomplish this, we are thinking that we would implement one of the standard sync algorithms, centralize the clock on the server and maintain/correct state on the server sending updates to the "clients"

We really want to do a POC before our interns arrive for the summer, as we need to develop a work breakdown schedule for them and we have a hard deadline of October for the CTF event.

Our next step is to start with a state server implementation that we will send data to from one client (master) and mirror those updates on a second client (slave). Essentially prove that we can control an AI Ship using another client generating the events.

We have forked the code repo at this point and are going to start planning out some tests tomorrow. I will keep you posted!!

Hit me up with any guidance you may have, I will keep posting updates as we arrive at them.

Cheers!

[cim]

Currently it appears that the game in single player mode runs its own clock and that UNIVERSE class is basically what the player is interacting with; this is what we would call the player's environment. In order to tie things together and make this a true multiplayer game, each player's environment's state and the state of all of the star systems must be maintained along with a centralized clock for synchronization.

One thing to watch for here is the game function exposed to Javascript as clock.addSeconds. At the moment, the HUD clock advances by minutes, hours or days during particular events. Most importantly: launching adds ten minutes, buying equipment adds at least five minutes per item, up to days for the biggest and most expensive, and jumps between systems take between 30 seconds and 2 days depending on the distance between systems.

I think you will need to do one of:
1 - remove this option and make the HUD clock always advance at 1 second / second, and make witchspace jumps, equipment purchasing, ship launch take up just a few seconds each.
2 - ignore that different players may have wildly different HUD clocks when they meet
3 - make players sit out for exactly 46.24 hours if they make a full range witchspace jump
(I think 2 may have very strange effects with respect to NPC witchspace jumps and related effects, and 1 and 3 will need some careful work around the "launch player from station" functions)

Regarding simulating multiple systems at once, the nature of Universe and PlayerEntity is such that you would need a massive rewrite of both classes [1] (and a lot of the JS scripting engine's relationship to them, which is not optional for getting things to work, nowadays) to have more than one system active in the same Oolite process. One server process per system would probably be the easiest to get operational, though with obvious disadvantages.

[1] Quite a lot of this rewrite - breaking these two monoliths down into more sensible components - should happen anyway, of course. For singleplayer there's no real urgency beyond it making the code more straightforward to read, though.

Our next step is to start with a state server implementation that we will send data to from one client (master) and mirror those updates on a second client (slave). Essentially prove that we can control an AI Ship using another client generating the events.

Certainly should be possible. If you're wanting to replicate a player ship on the client, consider using a PlayerProxyEntity for that, not ShipEntity. (It won't make too much difference, but it'll make it slightly easier to get the OpenGL shaders right)

We have forked the code repo at this point and are going to start planning out some tests tomorrow.

Good luck!

What sort of scope are you thinking of in terms of simultaneous players / active systems, by the way? And can you give more details about what sort of multiplayer experience / use cases you're aiming for?

[j105rob]

So we are hosting this CTF event at BSidesDC in October and our format is going to be 1 vs. 10 in rounds of 4 hours, 3 rounds per day over 3 days. The first 2 days will be elimination rounds. The final day is the championship round. Everything will be done over WiFi on a flat network.

There is a HID we are developing, kind of like a "Star Trek" engineering console station that will be used to repair damage to the ship's systems. Also we will have a physical system, like a warp drive, that the teams must keep operating at nominal levels.

Other teams are able to attack these systems along with attacking the player's ship, because we will make them vulnerable. The command and control of these systems will be using the SCADA ICS protocol.

Last year we did a Nuke Power Plant Sim and ran a 1 vs. 20 over 18 hours at GMU and it was awesome. However, you can't run the same sim too many times, and now that we have hooked up w/ a conference we need to up our game.

The Nuke Sim was based on Atari's SCRAM and we reversed engineered the Atari 400 code, from cassette images... long story.... that code is avail and open source on my github acct; if you want to play

Thanks for the heads up in the idiosyncrasies of the existing code base. Interesting comments about the dynamic nature of the timer. That *will* be an issue to look into ASAP.

[j105rob]

One thing to watch for here is the game function exposed to Javascript as clock.addSeconds. At the moment, the HUD clock advances by minutes, hours or days during particular events. Most importantly: launching adds ten minutes, buying equipment adds at least five minutes per item, up to days for the biggest and most expensive, and jumps between systems take between 30 seconds and 2 days depending on the distance between systems.

I think you will need to do one of:
1 - remove this option and make the HUD clock always advance at 1 second / second, and make witchspace jumps, equipment purchasing, ship launch take up just a few seconds each.
2 - ignore that different players may have wildly different HUD clocks when they meet
3 - make players sit out for exactly 46.24 hours if they make a full range witchspace jump
(I think 2 may have very strange effects with respect to NPC witchspace jumps and related effects, and 1 and 3 will need some careful work around the "launch player from station" functions)

Regarding simulating multiple systems at once, the nature of Universe and PlayerEntity is such that you would need a massive rewrite of both classes [1] (and a lot of the JS scripting engine's relationship to them, which is not optional for getting things to work, nowadays) to have more than one system active in the same Oolite process. One server process per system would probably be the easiest to get operational, though with obvious disadvantages.

One idea is to create a server instance per system, running its own clock, this will keep all entities in that system in sync. I think this eliminates problems 1 - 3. I assume the dis-advantage you see is scalability?

[cim]

One idea is to create a server instance per system, running its own clock, this will keep all entities in that system in sync. I think this eliminates problems 1 - 3. I assume the dis-advantage you see is scalability?

Scalability is one problem. Since the server doesn't need to do graphics, a powerful modern PC could probably run 16-32 of them easily enough - but there are 2,048 systems and a significantly higher number of intersystem spaces, so if you wanted to simulate everywhere at once ...

From what you've said, 16-32 is probably fine (it's possible that 1 might be fine - space is big and if the ships spend most of the 4 hours trying to be in the same bit of it at once it might not be the most interesting multiplayer experience)

On the timing issues ... separate servers per-system doesn't necessarily solve it. There are ... well, strictly there are multiple clocks, but there are two that matter:
Clock 1: "seconds since game launch" increments at what should be a fixed 1 second per real-world second rate. [1]
Clock 2: "days, hours, minutes and seconds since the start of the fictional in-game epoch". This clock is shown on the HUD (well, almost, but the distinction between the HUD clock and this one doesn't matter for now), starts for a new player approximately 2084 kilodays after the epoch, and is preserved in the savegame. It normally goes at 1 second per real second too - but certain events cause it to skip - often by a day or more. This skipping does not cause the intervening time to be simulated by the game engine even at an accelerated rate - everything just carries on as before (except WormholeEntity, which has expiry times dependent on this clock)

There are two relevant skip events:
Event 1: Launching from a station adds ten minutes to it. It also, in singleplayer, looks for any nearby NPCs in the docking queue of that station, and removes them (they dock before the player launches) to ensure a clear launch trajectory. That would obviously look a bit odd if anyone was watching from outside... You can probably work around this by redesigning the stations to have separate launch and landing docks, then removing the time skip and queue clearance.

Event 2: jumps between systems take N*N hours for an N light-year jump. In singleplayer, this lets you trade off simplicity (and perhaps safety) of the route against time taken. You can probably work around this for your event by making the jumps instant (or at least, only take however much time it needs for the server instances to hand over the player between them) - or perhaps by only having one system available for play in the first place.

It sounds for your purposes that these issues wouldn't be a problem, but they're major problems for more generalised "multiplayer Oolite" designs.

[1] Except if the game is paused, running under the "time acceleration" debug function, or struggling so much the natural FPS would drop below 4.

Another problems from having multiple server instances is that over time the systems fill up with junk. Again, it sounds from what you're planning that you could mostly override the current NPC ecosystem, which would deal with that.

[ralph_hh]

Make time a local variable on the users computer, not a server variable. Why must time be absolute?
By making it a local variable, you can still enjoy the thrill to fulfill anything like a contract in time. But if Player A jumps to another system where Player B is located at the beacon in the moment A starts the jump, why should B not "still" be there after the jump. A's computer adds 20hours to A's clock, B's clock simply continues. Imagine A jumps around to planet x then y then z and returns within real time minutes, game time hours to B's system, will a player notice, that AI ships have not moved much?

Server Capacity... You do not have to simulate all systems at once, just the ones a player is currently in. Not much different from single player.

[j105rob]

Here is the architecture we are looking at implementing:

http://www.ibm.com/developerworks/library/ar-powerup1/

The reason for keeping time on the server and not on the client is due to the sync issues; not player experience, e.g what their HUD says.

Also time is relative, and we understand that. If this was reality and you *could* do a faster than lightspeed jump across space, what is the effect on time perception? You would see time progressing on a mechanical clock at the same pace regardless of your speed.