Cockpits, HUDs and other displays

[MrFlibble]

What if starter were able to subscribe to a topic, and send anything published there back to Oolite, with filtered responses published. That would allow a kludgy debug console, and also facilitate instruments sending commands to oolite (pause on red alert if bridge empty).

An excellent idea to reuse the existing tech stack. So the Oolite Debug Console could preserve it's use to alternatively show/produce MQTT messages while at the same time more and more instruments could be developed and attached. Which would again encourage others to take that route.
That's a win-win situation.

Thinking about this:

OoliteStarter should then forward all messages from Oolite. If it is not meant for a special topic (e.g. oolite/comms) it should go to a default topic like oolite/debugConsole. This way an MQTT oriented debug console could still decide which messages to react on.

At the same time I will introduce a topic like oolite/input. Messages here will be forwarded to Oolite on the debug console interface - which means they need to contain some JavaScript command. And again this would allow Oolite to react.

I was experimenting in that respect and created this code:
https://github.com/OoliteProject/Oolite ... r.java#L78
But that one is too ship speed oriented. We will need something more generic.

This is heading in a good direction.

One for the back burner: Could the toplevel topic be configurable in some way, perhaps with an mqtt prefix field per installation in starters 'Oolite versions'? Multiple Oolites could share one MQTT server nicely.

My thinking goes something like this:
Arnold and I have installations sharing an MQTT server. I have two versions of Oolite running from starter, he has one. He sets Arnold/oolite as the prefix in his one, and I set flibble/oolite/dev and flibble/oolite/fun as my two. I'd only have to tweak the prefix on any client devices to switch versions, and all will remain sane. If Arnold decides to set up another one, with a different set of "remote instrumentation" he could use Arnold/otheroolite. Any prefix could be used, potentially allowing even online MQTT servers to be involved (I know.. glaring security hole!, ssl required).

A dev might be able to directly debug a user's Oolite on the other side of the planet by asking them to connect to an MQTT server with starter!

[hiran]

One for the back burner: Could the toplevel topic be configurable in some way, perhaps with an mqtt prefix field per installation in starters 'Oolite versions'? Multiple Oolites could share one MQTT server nicely.

My thinking goes something like this:
Arnold and I have installations sharing an MQTT server. I have two versions of Oolite running from starter, he has one. He sets Arnold/oolite as the prefix in his one, and I set flibble/oolite/dev and flibble/oolite/fun as my two. I'd only have to tweak the prefix on any client devices to switch versions, and all will remain sane. If Arnold decides to set up another one, with a different set of "remote instrumentation" he could use Arnold/otheroolite. Any prefix could be used, potentially allowing even online MQTT servers to be involved (I know.. glaring security hole!, ssl required).

A dev might be able to directly debug a user's Oolite on the other side of the planet by asking them to connect to an MQTT server with starter!

Wow, that is a whole new dimension.

But it reminds me strongly of what I intended to do with Nexus - but at that time I was using XMPP. Indeed I dug out the DebugConsoleProtocol handling from that project.

Coming back to your request:
Do we need separate topics for all the Oolites on this world, or would be a filtering on the publisher be sufficient? Probably depends if you can subscribe to specific publishers only. Will have to dig into MQTT features. But am still impressed on the amount of possibilities...

To be a bit secured from unknowns: We could install one MQTT server that only allows users to connect that are registered on the forum. And only somehow select who would be allowed to receive their messages (be it fellow players or developers). But that is future right now.

BTW, I just pushed some of your suggestions, so stay tuned for the next experimental versions.

[MrFlibble]

Apologies. The android app I'd not used for a long time was in fact net.routix.mqttdash. The json stuff I'm struggling to fathom in that, but hope someone else will clarify hereabouts. Works without google play services (installed using e.g. aurora store).

The one I'd posted earlier is not working, at least with authentication, for me.

[hiran]

One for the back burner: Could the toplevel topic be configurable in some way, perhaps with an mqtt prefix field per installation in starters 'Oolite versions'? Multiple Oolites could share one MQTT server nicely.

Do we need separate topics for all the Oolites on this world, or would be a filtering on the publisher be sufficient? Probably depends if you can subscribe to specific publishers only. Will have to dig into MQTT features.

I checked some MQTT documentation and could not find anything related to filtering messages within a topic. The most concise answer I found is on StackOverflow.
That means we have to setup the topic structure wisely.

[hiran]

Apologies. The android app I'd not used for a long time was in fact net.routix.mqttdash. The json stuff I'm struggling to fathom in that, but hope someone else will clarify hereabouts. Works without google play services (installed using e.g. aurora store).

The one I'd posted earlier is not working, at least with authentication, for me.

Here is a tutorial for a React Web App that is fed by MQTT messages: https://mpolinowski.github.io/docs/Deve ... 021-06-01/
I dislike the fact it needs Webhooks to be activated on Mosquitto side though.

[hiran]

With the latest build you can do something like

Code: Select all

mosquitto_pub -t oolite/input -h <hostname> -u <user> -P <password> -m '{"command": "player.ship.fuel=7"}'

With that I think we are ready for client dashboards to come.
And there is a new panel to configure the IoT settings.

[MrFlibble]

One for the back burner: Could the toplevel topic be configurable in some way, perhaps with an mqtt prefix field per installation in starters 'Oolite versions'? Multiple Oolites could share one MQTT server nicely.

Do we need separate topics for all the Oolites on this world, or would be a filtering on the publisher be sufficient? Probably depends if you can subscribe to specific publishers only. Will have to dig into MQTT features.

I checked some MQTT documentation and could not find anything related to filtering messages within a topic. The most concise answer I found is on StackOverflow.
That means we have to setup the topic structure wisely.

Feels like the wrong end of a stick is being grasped somewhere here.

The top-level topic of interest is currently hard-coded at the MQTT servers 'topic-root' as 'oolite/'. This is notionally a parent topic, under which relevant child topics structure is created:- controls, input, config etc. Those can be hard coded without issue, but IMO the parent topic shouldn't be.

Generally, the user will use their own local MQTT server, and that server will serve one running Oolite installation at a time. In these cases, a "default" of 'oolite' is fine.

Consider using a variable for parent topic, I'll use a variable T as example, default T=oolite', but could equally be set to e.g. T=student/fred/oolite/debugtest.

Regardless what T is set to, $T/input as the topic in pub or sub would put/find what we'd expect. '$T/#' as topic will do the right thing. Same for client apps.

Code: Select all

T="student/fred/oolite/debugtest"
mosquitto_sub -t "$T/controls/#" -h <hostname> -u <user> -P <password>

Doesn't make any odds what $T is set to for searches/filtering, as long as it's the same at Oolite-starter, and any other MQTT clients that need to be involved.

If the main topic were changeable, (ideally per item in "Starter->Oolite Versions"), all child topics will be fine. Users who want to use an existent MQTT installation for which they have limited rights will be accommodated (student/fred).

Those wishing to have an Oolite party could share an MQTT server with different 'parent' topics, and maybe some clever shenanigans allowing them to affect each others Ooniverses .

For now though, default-only will do nicely

P.S. input's working nicely for me. Thanks!!

[hiran]

Do we need separate topics for all the Oolites on this world, or would be a filtering on the publisher be sufficient? Probably depends if you can subscribe to specific publishers only. Will have to dig into MQTT features.

I checked some MQTT documentation and could not find anything related to filtering messages within a topic. The most concise answer I found is on StackOverflow.
That means we have to setup the topic structure wisely.

Feels like the wrong end of a stick is being grasped somewhere here.

Yupp. I thought to remember clients can filter/pick the messages they receive. Must have been a different messaging system. MQTT does not have such features.

Which means the only available filtering is on the topic hierarchy wit one-level and multi-level wildcards. And that's where the hierarchy matters.

Now you seem to be more experienced in that field as I am - the structure looks obvious to you where I spend time mulling it over. So currently I see three options that I am thinking through:

Assume we want to allow distinct clients to talk with a 'clientId'.

1)
clientId/alert
clientId/comms
clientid/controls
...

2)
oolite/clientId/alert
oolite/clientid/comms
oolite/clientId/controls
...

3)
oolite/alert/clientId
oolite/comms/clientId
oolite/controls/clientId
...

Currently I believe pattern 2 is most useful, with the limitation that clientId must not contain slashes.

It would allow to easily distinguish oolite from non-oolite topics (common root node), distinguish between clients with hierarchy priorising clientId while also ensuring paths are similar in length.

Or I stop thinking and follow your example not limiting anything.

P.S. input's working nicely for me. Thanks!!

I have to thank you for such feedback.

[MrFlibble]

...
Or I stop thinking and follow your example not limiting anything.

Thinking is a good thing. It means I have to sanity check what I'm saying.

The way I see it, the 'parent' topic (currently oolite/) could be 1/2/3/4/flibblydoodah/ for all it matters, as long as all parties involved (oolite, dashboard) use the same. Limiting the 'root' to anything would make it difficult to use exisiting MQTT servers where access to topics can be restricted.

For example, I'm crudely limiting access to my mosquitto server in /etc/mosquitto/aclfile

Code: Select all

user flibble
topic readwrite oolite/#
topic readwrite kabOodle/#
topic readwrite student/fred/#

user tablet
topic read oolite/#
topic read kabOodle/#
topic read student/fred/#

I've allowed flibble readwrite on oolite (legacy), kabOodle (where I send oolite data once my scripts have mangled it to turn speed into integer etc.), and an example like I gave yesterday. Flibble account can write/create to any topic in or beneath student/fred/.

My tablet dashboard can't mess with my Oolite, but it can see it all.

Imagine flibble can only write student/fred/# (student/fred AND sub topics).

If I set up a listener thus:

Code: Select all

mosquitto_sub -h localhost -p 1883 -u tablet -P "$tabpass" \
  -t 'student/fred/deeppath/oolitetest/input' -v

And a send like:

Code: Select all

mosquitto_pub -h localhost -p 1883 -u flibble -P "$flibpass" \
  -t 'student/fred/deeppath/oolitetest/input' -m '{"command": "player.ship.fuel=7"}'

The listener gets:

Code: Select all

student/fred/deeppath/oolitetest/input {"command": "player.ship.fuel=7"}

...had the listener not used -v they'd just get the string of course.

The subscribe topic could be student/fred/deeppathoolitetest/# to mimic the current oolite/#, or sub topics as required (input|config|whatever).

So if T is topic, T=oolite, T=student/fred/deeppath/oolitetest or T=1/2/3/4/5 makes no odds as long as sub and pub use the same.

If there's a field per installation in "Oolite Installations" of starter for MQTT topic, set as oolite by default for ALL installations, nobody need care unless they want to customise the topic to suit their desires/needs. They can use the same or different per installed Oolite as they please.

If you wanted to group stuff a bit and ensure a topic 'wrapper' for all this, maybe make the topic ${T}[/]oolite, where the user can prepend a topic (any depth) if they need/want, and if they don't, it's simply 'oolite' (top level). This might need string 'preening' to ensure there's a slash between the optional bit and the trailing 'oolite'. IMO this would cover both default and custom, with the least amount of help-bait. All client tools will need the same mechanism, but it's fairly standard on MQTT stuff to be able to tweak the topic 'root'.

[hiran]

The way I see it, the 'parent' topic (currently oolite/) could be 1/2/3/4/flibblydoodah/ for all it matters, as long as all parties involved (oolite, dashboard) use the same. Limiting the 'root' to anything would make it difficult to use exisiting MQTT servers where access to topics can be restricted.

[...]

So if T is topic, T=oolite, T=student/fred/deeppath/oolitetest or T=1/2/3/4/5 makes no odds as long as sub and pub use the same.

If there's a field per installation in "Oolite Installations" of starter for MQTT topic, set as oolite by default for ALL installations, nobody need care unless they want to customise the topic to suit their desires/needs. They can use the same or different per installed Oolite as they please.

If you wanted to group stuff a bit and ensure a topic 'wrapper' for all this, maybe make the topic ${T}[/]oolite, where the user can prepend a topic (any depth) if they need/want, and if they don't, it's simply 'oolite' (top level). This might need string 'preening' to ensure there's a slash between the optional bit and the trailing 'oolite'. IMO this would cover both default and custom, with the least amount of help-bait. All client tools will need the same mechanism, but it's fairly standard on MQTT stuff to be able to tweak the topic 'root'.

Ok, you gave me good answers here. I will follow your suggestion, and I also change the MQTT configuration from one global to one per installation.
The next build should make that available.

[hiran]

If this works, we should think of writing a tutorial so others can make use of this as well. Not just about setting up Oolite/OoliteStarter but also MQTT and some possible dashboard. Once there, we can ask for a dashboard challenge.

[Cholmondely]

Any chance of some sort of a picture showing what either of you have managed so far?

[MrFlibble]

Any chance of some sort of a picture showing what either of you have managed so far?

I've done nothing 'pretty', just verifying things in a terminal and a tiny test on an android app.

A crude script to read the speed and send it back as integer:

Code: Select all

#!/bin/bash

broker=localhost

#Read speed published by Oolite, stream as integer into a different topic.

port=1883
user=inuser
pass='inpass'
intopic='oolite/#'

outbroker=$broker
outport=$port
outuser=$user
outpass=$pass
outtopic='kabOodle'

mosquitto_sub -h $broker -p $port -u $user -P "$pass" -t "$intopic" -v |\
sed -u '
  s@^oolite/controls {"msgType":"controls","speed":\([0-9]*\).*$@speed=\1@
' | while read; do
 [ "x${REPLY%=*}" = "xspeed" ] && echo ${REPLY#speed=}
 echo "$REPLY" >&2
done |\
  mosquitto_pub -h $outbroker -p $outport -u $outuser -P "$outpass" |\
 -t "${outtopic}/controls/speed" -l

The output makes more sense for MQTT dash on android (as mentioned in a post above). On the left is raw output published by Oolite, on the right is the parsed integer from 'kabOodle' topic.



I don't know if that client can already convert to integer. I looked at their JSON API but couldn't see a way.

It wouldn't be difficult to add a few augmentation (not cheat!) buttons to the MQTT dash.

[MrFlibble]

Ok, you gave me good answers here. I will follow your suggestion, and I also change the MQTT configuration from one global to one per installation.
The next build should make that available.

I see it's in gelsemium-10. Nice!

Observations and suggestions:

I'd already done the config in ~/.oolite-starter.conf, so had clues to work with. Maybe give an example URL like tcp://127.0.0.1:1883. Could the 'sane example' server URL be pre-filled in that field when no user content? It is after all, what the majority will be needing during 'baby steps'.

I tested Prefix with 'games/oolite/debug' (deliberately omitted the trailing '/') and ended up with messages in games/oolite/debugoolite/starter. If this behaviour isn't a "feature", then this field could be parsed to deal with the slash/no-slash situation.

Showing the fully constructed topic somewhere (under that field?) might help some grasp what's happening. If this could be done during form-filling, then the slash can be auto-added, and there will be less need to document the process. Simply, type into the field and see what the topic will be 'as you go'. When blank, oolite will be shown as the topic, type a letter 'a' into the field and the line below will show a/oolite etc.

Leading slashes, although 'not-normal' in MQTT discussions, seem to be valid as topic characters, so if a user wants them, they should probably be left alone:
Prefix : games/oolite/debug/ : Test arrived at games/oolite/debug/oolite/starter
Prefix : ///games/oolite/debug/ : Test arrived at ///games/oolite/debug/oolite/starter

Buzzing!!!

[hiran]

Any chance of some sort of a picture showing what either of you have managed so far?

Hello Cholmondely.

You are in a good position and welcome to test the build that MrFlibble handcrafted for Apple x86 hardware. While it does not carry the latest experimental code (MQTT) it is still very shiny and deserves to be checked.

https://github.com/OoliteProject/Oolite ... ag/v0.1.31