Oolite on HDR Displays

[Simba]

I think the approach to implementing HDR is wrong. For a visual demonstration, I have attached a visualization of color spaces on the RGB color model. It shows that, as a rule, narrower spaces are part of wider ones. That is, when honestly converting, for example, a bitmap image from sRGB to DCI-P3, the colors should not become more saturated. On the contrary, they should remain in the same positions inside the sRGB triangle, but taking into account the output on the DCI-P3 display, you should not see any difference. In the context of a game, this means that by default, any object with sRGB textures should look the same in wider color spaces.
Then why do we need a wider color space? First, you can use textures originally made for DCI-P3. Secondly, certain visual effects can strongly distort colors (for example, a different spectrum of a Star, effects from shields, when hit by lasers, etc.).
I don't know exactly how the renderer works, but I think it first calculates the colors in the coordinates of the color model, and only before output converts it to an 8\10\12 bit format for the monitor in accordance with the target color space, or even focusing on the coverage information from EDID.

[another_commander]

.
Then why do we need a wider color space? First, you can use textures originally made for DCI-P3. Secondly, certain visual effects can strongly distort colors (for example, a different spectrum of a Star, effects from shields, when hit by lasers, etc.).

What you are describing is what happens with Oolite. We don't use dci-p3 textures of course - games that use such textures can be counted with one hand, but under default color saturation conditions, colors in Oolite are in the srgb color space and only effects such as lasers or hyperspace jumps etc. will cause some of them to go into dci-p3 or bt2020 territory. These extended color spaces are normally entered in Oolite only as a result of lighting shader maths. The game, as far as I can tell on my computer, looks the same between sdr and hdr saturation-wise and I have comparison shots between sdr and hdr to show for it. Now, if you go ahead and raise the in-game color saturation using the SetColorSaturation(0.0 .. 2.0) console command, then more colors will start becoming so saturated that srgb cannot contain them anymore and they start spilling into dci-p3 and bt-2020. You can test that by running Lilium's Reshade hdr analysis shaders on the game or by screenshooting and viewing the results in SKIV. If you want I can show you my results because I have already tested this.

I don't know exactly how the renderer works, but I think it first calculates the colors in the coordinates of the color model, and only before output converts it to an 8\10\12 bit format for the monitor in accordance with the target color space, or even focusing on the coverage information from EDID.

Actually it delivers an RGBA16F scRGB linear image to the graphics driver, which internally performs the conversion to 10-bit BT-2020 color space for display output on an hdr screen. No color model coordinates calculations occur anywhere, because scRGB and sRGB use the same color coordinates (color primaries, to be more precise), with scRGB just being bigger and accepting negative colors (which, btw, are the colors that correspond to dci-p3 and bt-2020). Most games do either this or render PQ 10-bit RGB10A2 output directly, with color-space conversions as required, the latter being the most common case.

Also, Oolite deliberately avoids referencing the EDID. Experience shows that a lot of times times it is completely off and contains bogus info, so best not use it until we start seeing accurate information becoming the rule.

[another_commander]

OK, now you've made me go back and have a good look at it again. So here is what is happening with color saturation and HDR vs SDR in Oolite.

I downloaded a screen test pattern image to use as basis for color accuracy tests. This will be our reference test image:




I then run a mission screen with this image as background in SDR. This is the result:


And as you can see, the accuracy of the image reproduction in SDR is not great. There are noticeable differences and all are justifiable and due to the ACES tone mapper we use, but maybe also due to some srgb/gamma 2.2 mismatch, which is beyond the scope of this test to explain here. So anyway, we know that the SDR image is lower saturation than reference and is a bit darker too.

Next, with the current Oolite's HDR saturation settings, I've run the same mission screen in HDR. The result, which should be viewed in HDR and on a site that can interpret HDR pngs like https://upscale-compare.lebaux.co/ or an HDR PNG capable viewer like SKIV, has been uploaded here: https://drive.google.com/file/d/1m_AMnJ ... sp=sharing
You will find that the result is much, much closer to the original reference image. There are some slight differences in brightness but the color saturation is quite spot on. However, because the SDR in-game image is not very accurate, this HDR test image here looks like its off when you compare it to SDR, when in reality it is the more correct one.

So what can we do about it? I am thinking that if we sacrifice some color saturation accuracy, then we could make the HDR image get closer to the SDR one (because at the end of the day what we have learned to see in the game are SDR tone mapped images) and to achieve this we can adjust a bit the SATURATION_ADJUSTMENT constant in the oolite-fnal-hdr.fragment shader from its current (and more correct) 0.825 value to 0.725. This constant is used to make adjustments to color saturation in HDR to compensate for the different tone mappers used by the SDR and HDR shaders, which are both ACES, but different implementations.

With SATURATION_ADJUSTMENT lowered just a little bit to 0.725, the result is now this: https://drive.google.com/file/d/1Pe8Xef ... sp=sharing

This is closer to the SDR image generated in game - still not quite the same though. However, this is when we compare the test images. In-game, the differences in rendered scenes are practically absent and I think that maybe the new 0.725 value should be the new default. Effects like lasers and hyperjumps still spill into DCI-P3/BT-2020 which is good in my book.

Simba, please have a look and let me know your opinion. I'd be interested to hear your thoughts.

[Simba]

Thanks, now I get the point.
Then the question arises: is it not possible to compensate for low saturation in SDR? Then sdr and hdr output images will be as close as possible to the original srgb image.

[another_commander]

Thanks, now I get the point.
Then the question arises: is it not possible to compensate for low saturation in SDR? Then sdr and hdr output images will be as close as possible to the original srgb image.

Try this: Find the oolite-final.fragment shader inside oolite.app/Resources/Shaders and open it in a text editor.

Then comment out the funtion vec3 ACESFilm(vec3 color)

Then replace it with this ACES approximation, which is more or less the equivalent for SDR of what we are using for HDR and save the shader:

Code: Select all

vec3 ACESFilm(vec3 x)
{
	x *= 0.6;
	float a = 2.51f;
	float b = 0.03f;
	float c = 2.43f;
	float d = 0.59f;
	float e = 0.14f;
	return clamp((x*(a*x+b))/(x*(c*x+d)+e), 0.0, 1.0);
}

With this implementation of the tone mapper, when I run the test image this is the output I get in SDR:


This is closer to the HDR image and the original reference image too, but it is also an approximation of ACES, not the actual thing. Certain valued behaviours of ACES, like desaturation at high luminance and others are omitted from this one and because of that I don't know if I really want to switch to it just yet.

At the end of the day the issue we are discussing proves to be not HDR-implementation related, but rather the choice of tone mapper in use. I'd say that, for now, we can accept that any changes that might occur in the output due to the tone mappers can be minimized by tuning the color satuarion adjustment in the HDR shader. I kind of like the little extra bit of color "punch" and crispiness of the HDR version and wouldn't mind just a tiny bit more color saturation, though. In any case, the above ACES approximation (taken from https://knarkowicz.wordpress.com/2016/0 ... ing-curve/ for anyone interested) is the closest we can get if one is determined to match HDR and SDR outputs as closely as possible.

[Simba]

Oh, I like this option with the new ACESFilm() much better! I compared as usual when using XenonUI. The problem with it is that all the backgrounds are poorly visible, which could be corrected by increasing the gamma, but it also leads to overlighting for everything else. With the new version you can keep the gamma unchanged (but you should still set it to 1.1 for a closer match to the original image).

Certain valued behaviours of ACES, like desaturation at high luminance and others are omitted from this one and because of that I don't know if I really want to switch to it just yet.

And in what cases does something like this prove useful? When you are close to the sun?

[another_commander]

And in what cases does something like this prove useful? When you are close to the sun?

It is the one of the main ways used in cinema, photography and games to showcase very bright lights. As light source intensity increases, color tends to desaturate to white. This can be useful mainly for effects, but can be observed also in standard lighting in many cases.

Here is an example. Let's say I want to show planet city lights as very bright blue, RGB(0.0, 0.0, 3.9). This is what it looks like with the ACES approximation provided above:


and this is what it looks like with the actual ACES tone mapper:


I don't know about you, but I'd definitely prefer the actual one.

[Cholmondely]

I'd have to say that I do too.

But perhaps the colours should change a bit for planets primarily inhabited by different species?

[Simba]

Here is an example. Let's say I want to show planet city lights as very bright blue, RGB(0.0, 0.0, 3.9). This is what it looks like with the ACES approximation provided above:


and this is what it looks like with the actual ACES tone mapper:

And how can I reproduce it?

I can see making a hybrid variant, in which the current tone mapper will be switched on only in case of overcoming the upper brightness threshold for one of the channels, something like that:

Code: Select all

vec3 ACESFilm(vec3 color)
{
    const float threshold = 1.0; // maybe it needs a different value

    if (max(color.r, max(color.g, color.b)) > threshold) {
		// taken from https://www.shadertoy.com/view/XsGfWV This is based on the actual
		// ACES sources and is effectively the glsl translation of Stephen Hill's fit
		// (https://github.com/TheRealMJP/BakingLab/blob/master/BakingLab/ACES.hlsl)
		mat3 m1 = mat3(
	        0.59719, 0.07600, 0.02840,
	        0.35458, 0.90834, 0.13383,
	        0.04823, 0.01566, 0.83777
		);
		mat3 m2 = mat3(
	        1.60475, -0.10208, -0.00327,
	        -0.53108,  1.10813, -0.07276,
	        -0.07367, -0.00605,  1.07602
		);
		// prevents some ACES artifacts, especially bright blues shifting towards purple
		// see https://community.acescentral.com/t/colour-artefacts-or-breakup-using-aces/520/48
		mat3 highlightsFixLMT = mat3(
			0.9404372683, -0.0183068787, 0.0778696104,
			0.0083786969, 0.8286599939, 0.1629613092,
			0.0005471261, -0.0008833746, 1.0003362486
		);
		vec3 v = m1 * color * highlightsFixLMT;    
		vec3 a = v * (v + 0.0245786) - 0.000090537;
		vec3 b = v * (0.983729 * v + 0.4329510) + 0.238081;
		return clamp(m2 * (a / b), 0.0, 1.0);
   }

   float a = 2.51f;
   float b = 0.03f;
   float c = 2.43f;
   float d = 0.59f;
   float e = 0.14f;

   return clamp((color * (a * color + b)) / (color * (c * color + d) + e), 0.0, 1.0);
}

But this leads to artifacts in the case of the same glare on the ship. It may be possible to change some values to smooth out this effect, but I don't understand the math behind this code very well.

[another_commander]

Mixing tone mappers in the same scene is not a good idea, neither using thresholds. Thresholds in general tend to create artifacts, except in some rare cases and your proposal is no exception, unfortunately. Running your above code, the planet city lights with RGB value 0.0, 0.0, 3.9 look like this for a threshold of 1.0:


and like this for a threshold of 3.5:


Apart from not looking like an improvement compared to pure ACES, there is very visible overall image degradation with specular highlights being particularly messed up. You may not see it in the above shots since they focus on something entirely different, but the problems are real.

If you want to reproduce the test, all you have to do is install LaveIsEarth or any OXP that features city lights, open the debug console, run the game and execute

Code: Select all

system.mainPlanet.illuminationColor="0 0 1000"

(the 1000 is interpreted as 1000/255=3.92 by Oolite; any rgb component above 1.0 will result in the entire triplet to be treated as based in the 0 .. 255 range, while when all rgb components are between 0.0 and 1.0 the triplet is interpreted verbatim.)

Edit: We seem to be getting off-topic here, we've been discussing SDR tone mappers for a good few posts now...