Oolite.org website, wiki, BB maintenance options analysis

[maik]

Apologies in advance for the long post.

I took a shot at analyzing our web presence from a maintenance/risk perspective. Please me correct if I am wrong. We have:

• The Oolite website (http://oolite.org), owned and maintained by Ahruman. Changes need to be committed to the Oolite SVN at Berlios, from where they will be pulled by Ahruman and uploaded to the website.
  • Strengths: well-designed, files are under version control, editorial control over changes before they go live
  • Weaknesses: seldom updated, requires SVN user allowed to commit to oolite-web, process is dependent on one person (Ahruman), process is complicated in contrast to other web content management processes, no known backup server if Oolite.org becomes unavailable.
  • Opportunities: implement simpler process, allow more than one person to change Oolite.org, setup backup that can be switched to in short time.
  • Threats: When Ahruman is not available there is no way to change the Oolite.org content.
• The Elite Wiki (http://wiki.alioth.net), owned by Winston. Server-side administration (backups, updates, extensions) can be done by Winston and me, Wiki administration (bureaucrat level) by Winston, Ahruman, and me.
  • Strengths: Main knowledge base on everything around Oolite, shared admin rights on server and wiki, nightly backups to another machine that Winston owns
  • Weaknesses: not known if backup is reachable via Internet if wiki.alioth.net goes down.
  • Opportunities: make backup server available on internet
  • Threats: none.
• The Oolite BB (https://bb.oolite.space), owned and administered by Aegidian.
  • Strengths: Well administered and operated BB, main communications medium of the Oolite community
  • Weaknesses: dependent on one person (Aegidian), no backup site if aegidian.org becomes unavailable.
  • Opportunities: setup backup server
  • Threats: Oolite community has to use fallback mechanisms for communications: IRC and ad-hoc solutions.

So, since Winston resurfaced we could eliminate the problems around the Elite wiki. This obsoletes my Oolite wiki. In order to address the backup availability situation of the wiki I will back it up to my machine on a nightly basis. In case Winston's server goes down we can either switch or just make it available in read-only mode until Winston's server is back up, depending on the situation.

This leaves the Oolite website which cannot easily be updated, and the BB which does not have a backup. Both have only one maintainer.

To address these remaining points I propose the following:

• We agree on a simpler way of updating Oolite.org and allow more than one person to do this. This needs Ahruman's blessing and support. Some discussion already took place without Ahruman's participation and we had two CMS based alternatives and one based on the wiki. After the BB outage yesterday I would actually prefer that we do keep separate machines to not take down both the homepage and the wiki if something goes wrong.
• Someone hosts a backup of oolite.org and of the BB. While oolite.org remains static I can easily do the backup for it, for the BB backup aegidian needs to cooperate to provide a regular DB dump if I don't want to scrape it regularly (I don't). If this happens, I can also host a BB backup. Aegidian ideally also allows a second person access to his server.
• In order to easily switch between main and backup, all backups should be reachable via Internet, such that a change in DNS is all that it takes. Therefore, we should have a dedicated domain for the BB and the wiki. Suggestions: oolitebb.org and oolitewiki.org respectively. Ideally, all three domains (the two new ones + oolite.org) are with the same registrar and belong one owner who only uses his account for Oolite and thus can share the DNS access with selected others. This requires a) aegidian's agreement and b) that aegidian either registers the additional domain names or transfers Oolite.org to someone else. Alternative: aegidian sets up sub-domains like bb.oolite.org and wiki.oolite.org and allows someone access to his server to redirect those if necessary. Problem: if his server goes away so do the subdomains. Other alternative: we have multiple domain name owners and they all need to make access to their DNS records available to selected others. Low-tech alternative: we publish whatever URL the backup has on whatever channel we still have, no other domain names or access to DNS records needed.

[Selezen]

As the accidental cause of the motions currently under discussion I do have a few comments to make (oh, no surprise there...)

The issue that originally prompted this discussion was the lack of updated media on oolite.org. The original solution proposed for that was to implement a simple CMS. This became a discussion about some of the problems inherent with the web hosting for various different parts of the site being held only by the owners of that hosting.

I proposed that a hosting methodology be adopted that could be opened up to selected responsible representatives of the community, selected by Ahruman, Giles and other people that THEY would select (since only those two people are currently in charge of Oolite's "presence"). In summary this would mean that either the existing hosting or new hosting login and admin details be held by Ahruman and Giles in addition to those who registered the hosting and that the individual(s) who held the hosting should be expected to willingly agree to Ahruman and/or Giles distributing those login details as and when necessary. Although this was primarily related to the web site at oolite.org, the wiki was also discussed due to the "disappearance" of Winston and my perceived need for MediaWiki plugins to be installed (to allow user maintenance amongst other things).

Consolidation of the various hosting methods was also proposed to make the media hosting more robust and futureproof and to reduce the number of "failure points" in the community's online presence.

What now seems to be proposed is for the whole kit and caboodle to be moved around to alternative hosting under the auspice of maik (with the exception of the wiki but suddenly with the inclusion of the bb). I have not seen any evidence of maik's willingness to open the administration to anyone, and indeed (no offence intended) have seen more evidence of maik's tendency towards increased security over handing out access to his hosting. My subjective opinion on maik, again, apologies for any offence, is that he would be reluctant to allow anyone else (including Ahruman/Giles) to be handing out login details for his hosting. Apologies if I have picked this up wrong. I would be reluctant for the community to do all this work only to find that we are still in the same boat as regards access to the servers should maik take a step back from the community in x amount of months/years.

As for maik's specific proposals, please see comments below

We agree on a simpler way of updating Oolite.org and allow more than one person to do this. This needs Ahruman's blessing and support. Some discussion already took place without Ahruman's participation and we had two CMS based alternatives and one based on the wiki. After the BB outage yesterday I would actually prefer that we do keep separate machines to not take down both the homepage and the wiki if something goes wrong.

As I've said all along, I feel that the BB should be left alone. oolite.org hosts the website and links to the bb at its current location, as does the Wiki.

Someone hosts a backup of oolite.org and of the BB. While oolite.org remains static I can easily do the backup for it, for the BB backup aegidian needs to cooperate to provide a regular DB dump if I don't want to scrape it regularly (I don't). If this happens, I can also host a BB backup. Aegidian ideally also allows a second person access to his server.

This would be a big job for someone to take on, and for relatively little gain. aegidian.org has lost hosting very few times in the past x years and when it has, it hasn't taken long to restore the access. In that time, the wiki and web site can be used to pass on news about BB downtime. As I said in the previous conversation, the oolite.org site should be the main "portal" to the rest of the community (as it has the game's name in the title).

In order to easily switch between main and backup, all backups should be reachable via Internet, such that a change in DNS is all that it takes. Therefore, we should have a dedicated domain for the BB and the wiki. Suggestions: oolitebb.org and oolitewiki.org respectively. Ideally, all three domains (the two new ones + oolite.org) are with the same registrar and belong one owner who only uses his account for Oolite and thus can share the DNS access with selected others. This requires a) aegidian's agreement and b) that aegidian either registers the additional domain names or transfers Oolite.org to someone else. Alternative: aegidian sets up sub-domains like bb.oolite.org and wiki.oolite.org and allows someone access to his server to redirect those if necessary. Problem: if his server goes away so do the subdomains. Other alternative: we have multiple domain name owners and they all need to make access to their DNS records available to selected others. Low-tech alternative: we publish whatever URL the backup has on whatever channel we still have, no other domain names or access to DNS records needed.

Again, that is a lot of work and a lot of change needed basically for very little gain. The "alternative" would achieve the opposite of what maik has been proposing up til now, in that the subdomains would be linked to oolite.org and if oolite.org goes down then so do the subdomains. I agree that separate domains for the wiki and the bb would be pretty, but that is another £xx per month/year (roughly £10 per year per .org domain as far as I remember) for what is very little gain in relation to the community.

As originally envisioned I would still propose:

a) oolite.org be made into a CMS for ease of updating OR the wiki still be split from winston's hosting and be considered as an alternative method of implementing oolite.org (cim has convinced me that this would work very well)
b) the bb be left alone
c) no further expense be spent on these concepts UNTIL agreement is made with ahruman and Giles about the direction (if any) to be taken.
d) Ahruman and Giles be directly involved with this discussion and agreement made about how back-end hosting would be accessed.

[maik]

Selezen,

sorry to see that you are so angry. I honestly don't want anyone to feel like this.

I wrote my post because everyone here, including me, you, Winston, Aegidian, and Ahruman, has a real life that can sometimes be too demanding to spend time on Oolite. This is a fact. Based on this fact, there are risks that sites that are run by only one person become unavailable and/or cannot be maintained by others. I highlighted these risks.

Nowhere do I suggest that I want to run any of that. What I offer is to run backups of the community sites.

Nowhere do I suggest to move hosting around. Domain names are independent of hosting. You can have one person hosting a site and another body registering a domain name and having DNS resolve it to the former's server (maybe you don't need this explanation but I don't see why else you would be talking about alternative hosting).

You are right I am not willing to open administration of my server to anyone, but to everyone who is willing to identify and knows what he/she is doing on a Linux server. cim now does have access to mine. This means that if I take a step back from the oolite community, cim has everything he needs to setup a new server with the exact same dataset without having to resort to screen scraping like I did for parts of the Elite wiki. Yes, I am reluctant to have anyone spreading login details to my server. If this is not good enough for the Oolite community, then I don't mind if someone else steps forward who is happy to give the keys to his server to anyone who is perceived as a good steward.

Further to your post: I do not suggest to not leave the BB alone. Websites can go down and people can be unavailable (even if anecdotal evidence shows they have been available in the past).

The remaining points we seem to agree on even if your post sounds like we are at odds.

[aegidian]

The aim of securing access to Oolite, oxps, and the bulletin board is a noble one.

However, since the threats to these mostly arise out of the fact that the main sites are maintained by fallible individuals, putting all these sites under one roof would crystallise those threats into a single point of possible failure.

However I'm not sure what to suggest in terms of decentralisation that could help with ensuring access. There are many better brains here - let's open the search for solutions to the whole community and see how many ideas we can genereate.

[SandJ]

Since the threats to these mostly arise out of the fact that the main sites are maintained by fallible individuals, putting all these sites under one roof would crystallise those threats into a single point of possible failure.

Quite right.

Currently, someone Google searching for Oolite will find:

1. http://www.oolite.org - the web site
2. download Oolite
3. Wikipedia's waffle about oolite rock
4. Wikipedia's glorious entry about the Oolite video game
5. wiki.alioth.net - the wiki
6. https://bb.oolite.space - the forum
7. Dictionary.com's waffle about oolite rock
8. a non-functional Oolite Twitter page
9. the BerliOS Oolite project page
10. YouTube entries for Oolite.

Now that Smivs owns oolite.info, provided each domain points to each of the oolite.* domains, should any go down, an poor lost soul looking for Oolite aid should not have too much trouble.

And that's ignoring the IRC channel.

[drew]

Quick post, because 'I've been here and done' that in the past.

I'd go with Selezen to a point, but include the BB in the CMS. Have a single online presence.

I used to be the Chairman (the first one as it happens) of the Mk1 Golf Owners' Club. We had a forum very much like this one (identical infact - phpbb) and a separate website back in 2002.

Members wanted the BB, a wiki and a 'landing page' for the online presence of the club.

We ended up with the PhpNuke CMS in 2003 and nearly 10 years down the line they are still running with it - http://www.vwgolfmk1.org.uk - probably needs updating now, as the tech is pretty old, but a modern CMS will have more than enough functionality to cover all these bases.

Cost? Minimal. About £25 per year originally - though bandwidth costs rose as the site popularity soared as a result. This included resilience, backup, patching, software upgrades, email accounts, ftp, yada yada yada.

If you search for Mk1 Golfs on the internet, every single search result leads you to the one location, and google has subdivided it automatically due to popularity.

Forums here for instance - http://www.vwgolfmk1.org.uk/modules.php?name=Forums (23k+ users)
Wiki here - http://www.vwgolfmk1.org.uk/modules.php?name=Content

etc. etc.

Admin access was shared amongst a team (up to 10 as I recall) so no single point of failure. We had no interruptions in my time as chairman - a web 'team' self organised around the roles of Admins, Mods and contributors/editors.

Highly recommend a CMS/Wiki/BB combo like phpnuke. Familiar enough for most, and powerful enough to grow into with a lot of SEO capability.

Does everything we need in one package. No brainer IMHO. I'm sure we can organise enough donations (equiv 10 pints of beer a year) to sort this out. Doesn't need to be a big deal.

My £2x10-2

Cheers,

Drew.

Currently, someone Google searching for Oolite will find:

And something called Status Quo and Alien Items according to my Google-Fu.

[Selezen]

sorry to see that you are so angry. I honestly don't want anyone to feel like this.

I'm not angry: I just feel that the situation is simpler than your proposes solutions would indicate. I have a different opinion, which doesn't make me angry. I do feel, however, that my "concept" of what needs to be done is more appropriate (and easier) to address the needs of the community.

I wrote my post because everyone here, including me, you, Winston, Aegidian, and Ahruman, has a real life that can sometimes be too demanding to spend time on Oolite. This is a fact. Based on this fact, there are risks that sites that are run by only one person become unavailable and/or cannot be maintained by others. I highlighted these risks.

As did I. As I keep saying, the "leaders" of our community (Giles and Ahruman) should be given the logins and instructions for the back ends of the hosting that will allow people to create backups, install plugins, update the technologies and so on, and need to have the permission of the person who owns the hosting to be able to distribute those logins to the people OF THEIR CHOICE AS THEY SEE FIT to ensure that there will always be access to the administration facilities should the original administrators leave the community. I personally would trust Ahruman and Giles' judgement as to whom they gave permissions to as they have never once let this community astray and have consistently shown themselves to be hyper-intelligent pan-dimensional beings of the highest order.

Nowhere do I suggest that I want to run any of that. What I offer is to run backups of the community sites. Nowhere do I suggest to move hosting around. Domain names are independent of hosting. You can have one person hosting a site and another body registering a domain name and having DNS resolve it to the former's server (maybe you don't need this explanation but I don't see why else you would be talking about alternative hosting).

I seem to remember a conversation about moving the oolite.org domain (or subdomain) to point to either the new wiki or new oolite.org site on your hosting. That is a move of hosting. The domain is still hosted by the original registrant but the site content is in a different location.

You are right I am not willing to open administration of my server to anyone, but to everyone who is willing to identify and knows what he/she is doing on a Linux server. cim now does have access to mine. This means that if I take a step back from the oolite community, cim has everything he needs to setup a new server with the exact same dataset without having to resort to screen scraping like I did for parts of the Elite wiki. Yes, I am reluctant to have anyone spreading login details to my server. If this is not good enough for the Oolite community, then I don't mind if someone else steps forward who is happy to give the keys to his server to anyone who is perceived as a good steward.

As I said, leaving the final decision to YOU as to who is given administration access leaves us in pretty much the same boat we are in now. This is, I think, the reason why we are have been at odds since the start of this discussion. I feel that only Giles or Ahruman has the right to decide who in the community they would feel comfortable administering the community hosting. Maybe they disagree, but I would trust them to be able to decide in that case who they would want to leave in charge of those decisions. In all honesty I would have absolutely no problem donating hosting space for the community and would have no issue with giving Ahruman or Giles the FTP logins (yes, I know you don't like FTP) and administration details for the hosting package. I have repeatedly offered webspace for content and files and will continue to do so happily.

Further to your post: I do not suggest to not leave the BB alone. Websites can go down and people can be unavailable (even if anecdotal evidence shows they have been available in the past).

Which makes it more sensible to expand the base of people who have access to those systems.

The remaining points we seem to agree on even if your post sounds like we are at odds.

Yes, I'm sorry this seems to have become adversarial. Maybe we should meet on a field of battle and sort things out. Zaerla system at noon? Handbags at dawn?

Seriously, I apologise for the confrontational face this has adopted. I'm not deliberately trying to be a knob, I am just trying to help but we are, as I said before, looking at this situation from seemingly opposite ends...

[maik]

Yes, I'm sorry this seems to have become adversarial. Maybe we should meet on a field of battle and sort things out. Zaerla system at noon? Handbags at dawn?

Honestly, I'd much rather meet for a beer.

[Smivs]

I have to say as disagreements go, this one is very civilised and is yet another example of just how friendly this board is.
Honestly, it's good that there are different viewpoints and opinions...if everyone agreed straight away it is very likely that the end result would not be so good, as options would be overlooked and things would just proceed blindly with no checks and balances.
I am confident that with so much care and passion going into this debate whatever conclusions are eventually reached will benefit the whole community.

[Selezen]

Yes, I'm sorry this seems to have become adversarial. Maybe we should meet on a field of battle and sort things out. Zaerla system at noon? Handbags at dawn?

Honestly, I'd much rather meet for a beer.

Darn, that's a long way to go for a beer...

After rushing to answer Maik's points, I forgot to comment on Drew's points. I would disagree with the "one CMS to rule them all" approach for the same reasons as maik points out in his post - it would give a SINGLE point of failure to the entire community. I still advocate leaving the BB as it is and where it is, but I think the oolite.org site and the wiki should be combined. I think I missed out in my bulleted list above that the media file hosting should be consolidated into the same location too. Maybe an investigation needs to be done on just how viable that would be.

To be honest, though, I don't think there's much advantage in porting the wiki and BB to phpNuke. Again, as I said in response to maik earlier, it's a lot of work for very little gain. Unless it's a simpler process than I imagine. Simply wrap the oolite.org site into a simple CMS and put the wiki into a subfolder. Job done.

As an addendum to the problem of file hosting, my current hosting has 25GB webspace (which looks like I can upgrade it to 100GB if I drop ODBC support) and unlimited bandwidth on the account, and (as I said above) I'm happy to make that available to the community free of charge for file hosting. I would be happy to investigate backup options (as it is not automatically backed up) and ensure that the upload method was secure (and would be more than willing to provide FTP access to it for the community on Ahruman or Giles' whim). I only use about 300MB of the webspace for my crappy websites and it seems a waste to leave the rest of it sitting there. I've had the same web hosting for the last 15 years and I'm not planning on vanishing - it will be consistent until such time as I can't afford to keep it going any more or I kick the bucket or something. Obviously I'd leave my web hosting to Oolite in my will...

[maik]

Yes, I'm sorry this seems to have become adversarial. Maybe we should meet on a field of battle and sort things out. Zaerla system at noon? Handbags at dawn?

Honestly, I'd much rather meet for a beer.

Darn, that's a long way to go for a beer...

I might be able to join my woman when she attends Chelsea Flower Show. Then it is not that far anymore.

[Selezen]

Indeed. That's the weekend that my daughter is due to give birth, so that might not be the best week... Poodoo, as Jabba the Hutt is wont to say. We shall discuss this in PMs!

Anyway, back on topic. I'd be interested to know at this stage what the Powers That Be are thinking about this conversation so far. Neither of them have actually said that they support this drive or that they would be willing to even be responsible for handing out server access logins...

If they aren't, then we're all waving plasma accelerators at each other for nothing.

[maik]

I have been spending more thought on how to reconcile Selezen's and my points of view on the topic of server access. To reiterate (and to check that I understood it well):

• Selezen would like Aegidian and/or Ahruman to be the holders of the "keys" to all oolite servers such that they can give them out to whoever they feel is trustworthy. The goal is to be able to hand out keys even if I am not reachable.
• I want to know who is accessing my server (by means of real life identification) and expect familiarity with Linux, security consciousness as far as running a server in the internet is concerned, and familiarity with the applications that are to be maintained (e.g. MySQL, Apache2 with PHP5, MediaWiki).

One solution would be that I create a stock of users (lets say 100) and give them to Aegidian and/or Ahruman, together with a list on e.g. googledocs. Whenever they want to give someone access to my server, they will have to check identity and ask about familiarity with the above items. When this person proves his or her identity and confirms familiarity then Aegidian/Ahruman assign one user from this stock, transmit the password, and fill in the line in the list on googledocs which links the username to the actual person.

Would this work?

[Selezen]

It's certainly a step in the right direction, but again it's still not "open" enough to ensure quick resolution of issues.

Here's my take. It's going to sound overly structured, but in practice it shouldn't be as complex as it sounds.

Hosting is set up on an ISP. This could either refer to the existing locations or new locations. Whatever happens. The hoster must be open to sharing full access to the location where oolite content is to be stored.

The hoster supplies Oolite Management (Ahruman and/or Giles) with the usernames and passwords for any and all administrative locations for the Oolite content (MySQL server locations and access, web server FTP/SSH area, usernames and passwords CMS/BB/Wiki installation and usernames and passwords for hosting package if necessary).

Oolite Management delegate responsibility for maintenance of certain aspects to those individuals they feel have proven themselves responsible enough on the forum to be trusted with that sort of access. Note that I would advise that anyone with this responsibility be a regular forum user. I only envision one or two people would have to be trusted for each point of maintenance. For example under the current hosting structure, one or two people would be given access to the alioth.net (if winston is OK with that), one or two people would be given access to the BB back ends and one or two people would be given access to oolite.org's hosting. These "one or two users" could be the same users for all or different users for each maintenance point.

Those users would then be responsible for the safe and secure use of that access and would be expected to notify Oolite Management if they are intending to leave the community or step down from the roles or even scale back their involvement. It would then be up to Oolite Management to restructure as necessary.

In the unlikely case of an emergency, where something falls over and none of the delegated maintenance "staff" are on hand, the Oolite Management can delegate the access rights to one trusted person in order to resolve the situation. Note that this would be (hopefully) a rare occurrence.

The changing of passwords if/when a member of the "staff" steps down (an often useful security measure) would be negotiable depending on how the Oolite Management and hosting party feel about the person stepping down and the manner in which they depart.

It would require the hosting party to be trusting of the community as a whole and in particular trusting of Oolite Management and their choice of "staff". it would also require the Chosen Ones to be worthy of that trust, but I feel this is one place where we do have a good many trustworthy and upstanding members of the online community. I for one would be more than happy to turn over even my root passwords to the OOlite Management and wouldn't lose any sleep over it.

Although I do understand your position, maik, I believe that the restrictions you put on the access would potentially cripple the ability to make changes/fixes quickly IF the need arose.

I want to write more, but I need to finish up some work before clocking off time.

[SandJ]

Also, to help ensure identity and maintain contact, there are the two LinkedIn groups:

Oolite Community
and
Oolite Project co/ex Contributors

I would expect developers and site hosts to be members of the latter.