Hosting site for oxz files

General discussion for players of Oolite.

Moderators: winston, another_commander

User avatar
Thargoid
Thargoid
Thargoid
Posts: 5528
Joined: Thu Jun 12, 2008 6:55 pm

Hosting site for oxz files

Post by Thargoid »

As Cim mentioned here, the 1.79 version of the game will use the new .oxz compressed/single file format for expansion packs. These need to be hosted somewhere with direct URL access, so Box.com won't be a suitable site (at least the free accounts) as they don't do direct access.

So I was wondering if people have a recommendation for a suitable replacement site? I'm looking for somewhere which has such direct access, is free, has at least minimal file download logging (for example a download count or similar) and doesn't require too much by way of maintenance (for example doesn't delete the file if not accessed for a period of time, or have too low a bandwidth limit).

Inputs and suggestions welcome, as I know I'm far from the only one currently using Box, but in the near future we may need something of a migration (which is going to be "fun" for me, given my back catalogue :evil: ).
User avatar
Norby
---- E L I T E ----
---- E L I T E ----
Posts: 2577
Joined: Mon May 20, 2013 9:53 pm
Location: Budapest, Hungary (Mainly Agricultural Democracy, TL10)
Contact:

Re: Hosting site for oxz files

Post by Norby »

OXZ manager is a great improvement, thank you cim!

I suggest to make support for http://goo.gl/ redirects to get a download counter which is the main problem if we want to leave box.com. Maybe not so hard to parse the real url from the received data.

There is a comparsion table of hosting services, each has its own weaknesses. I tried the best ones but not found any with counted direct links, please tell if you found one.

Alternatively there was some offer by members in the forum earlier:
- Keeper has unlimited space and bandwidth on keeper1st.com/Oolite.
- Selezen has 5GB of webspace with unlimited bandwidth.
- Cheyd host Griff's oxps on deephorizonindustries.com.

In this case we need to make an upload page (in php) to avoid asking the owner for updates. The Centralised OXP hosting topic contain good ideas, maybe a bit simpler solution is enough.

Moreover Diziet Sma own the oxpcentral.net. If one of the offers will be selected for use then can be set this domain to point there. I suggest to use this name in manifest.plists which help fix hosting problems without changing plists.
User avatar
Diziet Sma
---- E L I T E ----
---- E L I T E ----
Posts: 6312
Joined: Mon Apr 06, 2009 12:20 pm
Location: Aboard the Pitviper S.E. "Blackwidow"

Re: Hosting site for oxz files

Post by Diziet Sma »

Norby wrote:
Alternatively there was some offer by members in the forum earlier:
- Keeper has unlimited space and bandwidth on keeper1st.com/Oolite.
- Selezen has 5GB of webspace with unlimited bandwidth.
- Cheyd host Griff's oxps on deephorizonindustries.com.
The only problems with such an approach are that it creates a huge workload for one person to maintain everything, or it requires setting up dozens of FTP accounts if authors are to be capable of maintaining their own OXZs.. and that it leaves things vulnerable to unexpected events happening to the owner of the domain. If hosting payments are not maintained for whatever reason, up to and including the sudden demise of the owner, the entire hosting setup could go permanently offline without warning.
Norby wrote:
Moreover Diziet Sma own the oxpcentral.net. If one of the offers will be selected for use then can be set this domain to point there. I suggest to use this name in manifest.plists which help fix hosting problems without changing plists.
As mentioned before, I'm quite happy to turn ownership of this domain over to Aegidian or the dev team should it be deemed a useful domain for such a project.
Most games have some sort of paddling-pool-and-water-wings beginning to ease you in: Oolite takes the rather more Darwinian approach of heaving you straight into the ocean, often with a brick or two in your pockets for luck. ~ Disembodied
User avatar
maik
Wiki Wizard
Wiki Wizard
Posts: 2028
Joined: Wed Mar 10, 2010 12:30 pm
Location: Ljubljana, Slovenia (mainly industrial, feudal, TL12)

Re: Hosting site for oxz files

Post by maik »

Norby wrote:
OXZ manager is a great improvement, thank you cim!

I suggest to make support for http://goo.gl/ redirects to get a download counter which is the main problem if we want to leave box.com. Maybe not so hard to parse the real url from the received data.

There is a comparsion table of hosting services, each has its own weaknesses. I tried the best ones but not found any with counted direct links, please tell if you found one.

Alternatively there was some offer by members in the forum earlier:
- Keeper has unlimited space and bandwidth on keeper1st.com/Oolite.
- Selezen has 5GB of webspace with unlimited bandwidth.
- Cheyd host Griff's oxps on deephorizonindustries.com.

In this case we need to make an upload page (in php) to avoid asking the owner for updates. The Centralised OXP hosting topic contain good ideas, maybe a bit simpler solution is enough.

Moreover Diziet Sma own the oxpcentral.net. If one of the offers will be selected for use then can be set this domain to point there. I suggest to use this name in manifest.plists which help fix hosting problems without changing plists.
I also have about 40GB left on my server with unlimited bandwidth and am happy to offer those.

One other option could be to use the wiki to upload and maintain OXZ files. At the moment, arbitrary file uploading is disabled, but I could allow .oxz files. That way authors can upload their own files (they just need a wiki account), we would have versioning in place, and the nightly wiki backup would mirror them as well. Dylan's server (which runs the wiki) has 30GB left, which should be plenty. If this solution seems workable I can contact him for his approval.
User avatar
Smivs
Retired Assassin
Retired Assassin
Posts: 8408
Joined: Tue Feb 09, 2010 11:31 am
Location: Lost in space
Contact:

Re: Hosting site for oxz files

Post by Smivs »

Yes, a central repository of some sort would make a lot of sense, and the wiki at first glance looks like a logical candidate.
Personally I can host OXZs on my own site as I currently do for OXPs, but the different system means that I would not be able to get download numbers (stats) easily, and there is currently an issue with making a .oxz file downloadable although there are ways round it.
I for one would be more than happy to have a reliable central repository available, and use it, particularly if stats were also available.
Commander Smivs, the friendliest Gourd this side of Riedquat.
User avatar
Norby
---- E L I T E ----
---- E L I T E ----
Posts: 2577
Joined: Mon May 20, 2013 9:53 pm
Location: Budapest, Hungary (Mainly Agricultural Democracy, TL10)
Contact:

Re: Hosting site for oxz files

Post by Norby »

maik wrote:
I could allow .oxz files
This is the best imho. If we can get a File: page as for images then there is an access counter at the bottom of page which is (almost) as good as a download counter in this case. The oxz manager built in the game can increase it also if download this small page before the real oxz.

Another benefit if we use the wiki as engine is that a new hosting offer is equivalent with a new wiki mirror: more is safer and automatically updated.

I suggest to use oxpcentral.net in plists which can point to the actual wiki server and can be changed by the dev team to another wiki if the main is permanently lost. Moreover can be an oxzhosts.plist bundled into the game with hostnames of all wiki mirrors and retry the download from another if one is not responding.

Current OXPs need about 2GB except planetary textures, but a base oxz always can be in the wiki to count downloads and the large textures in other places. So maik, please ask Dylan. :)
User avatar
Commander McLane
---- E L I T E ----
---- E L I T E ----
Posts: 9520
Joined: Thu Dec 14, 2006 9:08 am
Location: a Hacker Outpost in a moderately remote area
Contact:

Re: Hosting site for oxz files

Post by Commander McLane »

The Elite Wiki itself can host download files, however there's a file size limit of 2MB, which makes it impractical for OXZs containing more than a few small texture files.
User avatar
maik
Wiki Wizard
Wiki Wizard
Posts: 2028
Joined: Wed Mar 10, 2010 12:30 pm
Location: Ljubljana, Slovenia (mainly industrial, feudal, TL12)

Re: Hosting site for oxz files

Post by maik »

Commander McLane wrote:
The Elite Wiki itself can host download files, however there's a file size limit of 2MB, which makes it impractical for OXZs containing more than a few small texture files.
This is a configurable setting, thanks for reminding me though.
User avatar
maik
Wiki Wizard
Wiki Wizard
Posts: 2028
Joined: Wed Mar 10, 2010 12:30 pm
Location: Ljubljana, Slovenia (mainly industrial, feudal, TL12)

Re: Hosting site for oxz files

Post by maik »

Norby wrote:
Current OXPs need about 2GB except planetary textures, but a base oxz always can be in the wiki to count downloads and the large textures in other places. So maik, please ask Dylan. :)
Just did, including the request to change the size limits for uploading files (thanks again Commander McLane). Do 50MB suffice? Is anyone aware of a bigger OXP except planet textures which would need to be added manually?
User avatar
Cody
Sharp Shooter Spam Assassin
Sharp Shooter Spam Assassin
Posts: 16081
Joined: Sat Jul 04, 2009 9:31 pm
Location: The Lizard's Claw
Contact:

Re: Hosting site for oxz files

Post by Cody »

maik wrote:
Is anyone aware of a bigger OXP except planet textures which would need to be added manually?
Griff's shipset (Resources ~120MB), but that is safely hosted by Cheyd (also Solo's shipset, if that was to return).
I would advise stilts for the quagmires, and camels for the snowy hills
And any survivors, their debts I will certainly pay. There's always a way!
User avatar
Thargoid
Thargoid
Thargoid
Posts: 5528
Joined: Thu Jun 12, 2008 6:55 pm

Re: Hosting site for oxz files

Post by Thargoid »

50MB would easily cover all of mine (probably in total :wink: ).

Max file size of the few larger ones is a little over 10MB and most are much smaller.
User avatar
Norby
---- E L I T E ----
---- E L I T E ----
Posts: 2577
Joined: Mon May 20, 2013 9:53 pm
Location: Budapest, Hungary (Mainly Agricultural Democracy, TL10)
Contact:

Re: Hosting site for oxz files

Post by Norby »

I think no other over 50MB than these due to I downloaded once everything from [wiki]OXP_List[/wiki]:

126M Griff_Shipset_Resources_v1.2.25.oxp
112M RandomHits1.7.oxp
94M griff_station_bundle_fullsize_tex_v1.1.oxp
73M Griff_No_Shaders_ShipSet0.95.oxp
60M Staer9ShipsetShadedv1.1.oxp
User avatar
maik
Wiki Wizard
Wiki Wizard
Posts: 2028
Joined: Wed Mar 10, 2010 12:30 pm
Location: Ljubljana, Slovenia (mainly industrial, feudal, TL12)

Re: Hosting site for oxz files

Post by maik »

Email from Dylan regarding using the wiki server for OXZ storage:
Dylan wrote:
Hi

Sounds good to me. Go for it.

Cheers.
User avatar
Norby
---- E L I T E ----
---- E L I T E ----
Posts: 2577
Joined: Mon May 20, 2013 9:53 pm
Location: Budapest, Hungary (Mainly Agricultural Democracy, TL10)
Contact:

Re: Hosting site for oxz files

Post by Norby »

Cody wrote:
Griff's shipset (Resources ~120MB), but that is safely hosted by Cheyd
I think this and RandomHits are a very common OXPs and a mistake if these are left out from the download manager or forced to leave without counter, so I suggest to set the limit to 150MB.
I do not see any danger in this due to only manually approved wiki members can upload files and everything is logged so no place for abuse but a great help for players.
User avatar
cim
Quite Grand Sub-Admiral
Quite Grand Sub-Admiral
Posts: 4072
Joined: Fri Nov 11, 2011 6:19 pm

Re: Hosting site for oxz files

Post by cim »

maik wrote:
One other option could be to use the wiki to upload and maintain OXZ files. At the moment, arbitrary file uploading is disabled, but I could allow .oxz files. That way authors can upload their own files (they just need a wiki account), we would have versioning in place, and the nightly wiki backup would mirror them as well. Dylan's server (which runs the wiki) has 30GB left, which should be plenty. If this solution seems workable I can contact him for his approval.
A minor point: it's not strictly required, but it's better to change the download_url for every version of the OXZ file if you're using the automatic downloader: the reason is that if you don't, someone can download version 1.7 but Oolite (with an old index) thinks it's downloading 1.6, so the download will complete and then say "failed". You'd want to make sure you were always using the permanent URL for that file version as the download_url if possible.
Post Reply