A Description of the Improved Lighting and Materials System

[another_commander]

Warning: Long post incoming.


It has been a few weeks since the introduction of the new lighting system for Oolite and I have realized that information about it is not so easy to find or track. Whilst some general directions about it have been posted in responses spread over various topics, we do not seem to have a proper description of it or how it can be used to make Oolite's graphics better. So, I will try to do this here, in this post, in order to have a localized source of information and to use this thread as the discussion point for those who would want to look into this new lighting system further.

So without further ado, we'll start by mentioning that the aim of introducing the new lighting system is to enable modders to create materials that look better than before, more natural and more compliant with how light behaves in the real world. To achieve this, we have implemented new shaders with functions that describe the behaviour of light as it encounters different types of materials. More specifically, we are now using what is called a BRDF - which stands for Bidirectional Reflectance Distribution Function - to describe the diffuse and specular components of light, both for ship objects and planets alike. The BRDF uses the standard Lambertian diffuse equation for the diffuse light component (just like before) and the Cook-Torrance model for the specular light component. The basics of those are described in more detail here, but the general idea is that light fr is calculated as fr = kd * fLambert + fCook-Torrance, with kd being the ratio of incoming light energy that gets refracted (diffuse).

Since the diffuse light has not changed compared to the old light model, there is not really much to say about it, so we will concentrate a bit on the specular light part. The Cook-Torrance specular model is described by the equation fCook-Torrance = D*F*G / (4 * NdotV * NdotL). As you see, this model takes into account a few parameters, namely the distribution factor D, the Fresnel component F and the geometry factor G, as well as the vector angle between the normal of the surface where the light hits and the viewer and that of the normal and light direction. Distribution approximates the influence of the surface's microfacets alignment and distribution in the way light is reflected. Geometry refers to light rays occlusion by approximating the way microsurface details overshadow each other on a rough material. Both Distribution and Geometry simulate microfacets on the material's surface (see pic below), which is a very important part of any physically accurate light model and depend on material roughness.

There are more than one mathematical models for distribution and geometry. The distribution model used in Oolite is GGX, while the geometry model of choice is the Smith one. Fresnel refers to the ratio of surface reflection at different surface angles and effectively represents the specular light reflected energy, which is important in its own right for light energy conservation purposes. Oolite uses the Fresnel-Schlick approximation for calculating this component.

The above summary is very brief and just tries to establish the physical compliance of the new light model. I fully recommend that those interested further read the theory part of the link a couple of paragraphs above. For now, we want to keep that the new lighting model has two characteristics: 1) it takes into account material microfacets behaviour and it uses material roughness for this and 2) it implements light energy conservation. (1) necessitated the implementation of a way to describe the material roughness to Oolite. This is where the newly introduced gloss material/shader property comes in. Gloss is described as a floating point number from 0.0 (no gloss, material is extremely rough) to 1.0 (100% glossy material, no roughness and no microfacets whatsoever, material is fully reflective). Its relation to roughness is gloss = 1.0 - roughness. (2) means that the energy of the radiated light (reflected + refracted) cannot be more than the energy of the light that hits the surface. This will weigh in later, when we discuss the importance of the specular light material parameter.

All this sounds great, but how do we use that to make better graphics in the game? Well, the new system means that the workflow for creating materials will have to be slightly different to what we had before. We now have to think about our materials in a more physically oriented way, not necessarily more complicated than before, just different. Note that those who don't want to change their ways of designing materials don't have to. The game still renders old materials just fine, but if you want to get the best result, this is something that you will need to consider.

Most game engines out there that support physically accurate lighting generally follow one (or both) of two types of workflows for designing materials: roughness / metalness and/or gloss / specular. Oolite uses the second one. We chose this workflow because we already have most of it implemented in our materials system (just needed to add the gloss handling), but also because this workflow allows the creation of materials that are not present in nature. In fact, many consider this a flaw of the gloss / specular system, but I choose to call it a feature, since we are talking about a sci-fi game with aliens.

So, gloss / specular it is, then. Specular color refers to the brightness of the specular reflection and gloss refers to its sharpness. For visualizing the relationship between those two parameters, refer to the image showing specular reflection over changing angle below. Specular color represents the height of the peak in the graph, while gloss represents the width of the peak:

Although all other material parameters are still present, those two are now becoming the primary and most important ones. The specular_color property is now not only a nice-to-have in shipdata, but it is fully recommended for all materials. You should always consider putting in a value for specular_color in any material you create if your material relies on the game's default shaders. If you don't, Oolite will put there its default specular_color of (0.2, 0.2, 0.2, 1.0) and materials with this specular color will generally have low specular highlights. Specular maps can still be used and the specular values from the map are multiplied with the defined base specular_color of the material. Gloss maps can be utilized too, and Oolite will be looking at the alpha channel of the specular map for those. Again, the values in the gloss map will be multiplied by the base gloss value of the material. In the absense of a base gloss definition for the material, the value of 0.375 will be used.

With specular and gloss defined, we can already emulate a very wide range of materials. Just to showcase this, see the below image. Don't worry about the values shown for gloss in the image, just concentrate at what happens to the red sphere as gloss increases or as specular color increases.

Non-metallic materials appearing in nature generally have very low specular color values, i.e. they do not have intense specular highlights. Increasing gloss makes them look shinier and more plasticky at high gloss values. Metallic materials have high specular colors and usually reasonably high gloss values. Note something very important here: See how increasing specular color "takes away" the original diffuse color from the object? This is a direct result of the energy conservation we mentioned earlier. If an object has a high specular color, it means that it reflects a big amount of incoming light hitting it. That in turn means that the diffuse color of the object must be reduced, otherwise the total of the refracted and the reflected light energies would exceed the initial incoming light energy. In Oolite you will see the exact same behaviour. Increasing the specular color to metallic levels will result in reduction in diffuse and, for a specular color of (1.0, 1.0, 1.0, 1.0), no diffuse light will be rendered at all, just like you see in this image. For this reason, designing metallic materials in the gloss / specular workflow is simply a case of remembering to use black (or near black) diffuse colors and ensure that the specular color is set to the characteristic color of that metal (e.g. yellow for gold, lighter yellow for brass, bright grey for most other metals). Another way to design metals is to just pass a very high specular color value in the material definition. Any diffuse color will be automatically reduced accordingly to conserve energy (so be prepared to see results slightly different to what you would exepct if you use both a diffuse and a high specular). Similarly, designing non-metallic materials is all about giving the material a diffuse color and assigning it a very dark specular - see the bottom image on this page for more). In practice, all materials should have some specular, even the non-shiny ones. Not going below RGB specular values of 0.02 is probably a good rule of thumb. Finally, it is noted that only metals would normally give colored specular highlights, all other materials should normally have some variation of gray as their specular color. Oolite will allow you to create a material with a diffuse color and a tinted specular_color at the same time, but this would represent an "impossible" material and the results can be interesting. For those who want an example, in the current 1.87 core game the Thargons are set to have such a material. So be careful with how you use your diffuse and specular colors in the materials if you want to be sure that you get something that can actually exist.

As for gloss, OXP creators need to be aware that it is the most important material property from an artistic point of view. While specular maps do provide certain aspects of the visuals on a model, it is the gloss maps or gloss values that really define a material's look. Gloss maps is where artists have the most freedom to let their imaginations run wild, so do make good use of them if you want to create something truly impressive.

As a general hint when creating materials, remember that you must design based on what will actually be seen rather than what the material you have in mind is. Example to make this clear: You have a brand new ship made of titanium alloy. You want to make materials for it. Before going ahead to create a metallic looking material, think how your ship will appear in game. Do you have a diffuse map with a paintjob already made? Or you want to show bare metal on the hull? If you go for the first option, then your material must not be designed as metal, it should be designed as paint applied on metal instead. This means low specular and, if you want to show some shiny, higher gloss maybe. But if you are going to show bare metal, then by all means, produce colored spec maps and black diffuses all you want. The top visible layer of your material is what should decide what the material should be made to look like.

This has been a very long post and there is a lot of information to digest. But I hope it can serve as a basic description and general guideline for using Oolite's new lighting system. The system could be improved further to make it fully physically based with things like:
- Shadows
- Different type light sources (we only have directional light right now, it would be nice to have point and tube lights for things like light from explosions, floodlight effects etc.).
- Environment mapping, which would boost visuals even more. At high gloss values, parts of the environment should be seen as visible reflections on objects, but currently we have no such thing. Interestingly, we do have an entire file in the source code called OOEnvironmentCubeMap.m, which apparently tries to deal with exactly this. Unfortunately this file is not part of the build at this time, which means that an attempt was started at some point but was not finished. It would be really great if it did get finished.

[another_commander]

A quick note for texture creators and latest rendering changes:

Mind your colorspaces!

Colorspaces are a somewhat tricky subject to explain fully here. Check out this link if you want to get a brief explanation of what this is about. The summary of this is that image storage is not a simple thing. Due to the way early CRT screens displayed stuff, gamma corrections are typically made in stored images, so that when displayed on a screen they look as intended. This makes transitions from dark to lit parts of the image smoother and more pleasing to the eye. But such transitions are not linear, i.e. a 50% gray is not half of 100% white. These images are stored in what is called the sRGB color space.

The problem is that the human eye perceives light in a linear way and that the shaders used for physically accurate reflected light calculations rely on colors being in linear colorspace. As a result, when the shader starts working with its texture color data, it has to remove the gamma corrections from sRGB images (which is what is typically authored by apps such as Photoshop, Gimp etc.) in order to convert them to linear colorspace. At the end of its run, it re-applies the gamma correction to display the reults on a screen. The complexity increases from the fact that data textures used in shader calculations (i.e. gloss, normal, parallax maps etc.) must be in linear colorspace, because we cannot afford to apply gamma correction on them - it would distort data that the shader needs to process, resulting in bad looking normals or unexpected reflectivities etc.

Texture creators need to be aware therefore, what colorspace their textures are made in. Oolite expects textures in the following colorspaces:
- Albedo (diffuse), specular, emission and illumination maps (i.e. color maps): sRGB. The shader will convert such textures to linear colorspace automatically before proceeding. If you pass a linear colorspace diffuse texture to the shader you will get a bad result, because the shader will try to remove the gamma correction when there is really nothing to remove.
- Normal, parallax, gloss maps (i.e. data maps): Linear. The shader will take those textures and use them directly in its calculations. Passing an sRGB map here will result in bad or weird looking output for the opposite to the above reason.

Alpha channels in images are typically stored as linear, which is good. This enables us to use a colored spec sRGB map and still have an embedded gloss map in the alpha channel saved in the correct and expected linear colorspace. Still, whenever possible, it would be recommended to double check your formats when saving textures. If you have made a texture set for a brand new ship and it looks weird in the game, then probably the first thing you would want to check is colorspace for every channel of every texture.

Color values passed in materials, such as specular_color, diffuse_color etc. are expected to be in linear colorspace.

[another_commander]

Further to the latest post about colorspaces, with build 97f3e070 Oolite has been made a bit more flexible on how texture colorspaces are processed by the default shader.

The materials / shader materials key gamma_correct has now been introduced and regulates whether the shader will apply gamma correction on the diffuse, specular, emission and illumination textures as mentioned in the previous post. The default for all materials is YES and a gamma_correct entry can be made for each material. The setting can also be changed globally to NO if required, using the boolean .GNUStepDefaults key no-gamma-correct accordingly.

Why this change? Well, apart from making Oolite more flexible with its rules, the real reason is that I cannot bring myself to like 100% of the gamma corrected texture results compared to the previous non-gamma corrected state. Although the proper way to render is to correct for gamma, there are occasions where I find the non-gamma corrected results better looking or more interesting, case in point being the pure metal materials. With this method, the best possible result can be now chosen and used. At the end of the day, adhering to physical properties of light and emulating them is fine and all, but the primary criterion for use is how great it looks in-game. So, until I have discovered a mistake or something in the main shader, I would like to keep both options available. Just keep in mind that, theoretically, gamma correction should happen and should be examined as a first option and switch only if the results look unacceptable. Some older OXPs that use the rgb values of the diffuse maps as inputs for effects may be particularly susceptible to this and may have to be switched to the non-gamma corrected method.

For a visual example of what effect gamma correction (or lack thereof) has on lighting, see the below image. The light source color on the first and third pics is yellow, while for the middle row screenshots it is red. Note how more "concentrated"the light reflections appear on the left and how reflection colors change hue abruptly and very noticeably. Lighting appears darker overall and diffuse colors as well as material shininess seem off without gamma correction. All these are flaws that are not seen in the gamma corrected images on the right.


The general guideline remains: Diffuse (albedo), specular, emission and illumination maps should normally be sRGB, all others linear colorspace.

[another_commander]

Continuing here with some clarifications that were requested in the Screenshots thread:

All of the following only applies to Oolite v1.90 (or a higher version) as definitions in shipdata.plist and not to anything in the Shaders folder.

If shader definitions are in use then gloss is ignored if defined under materials.

This is correct. Shaders have priority over materials and if a shader is defined, then this is what will be used. If there is a material definition for the same entity, this and all its properties within are ignored.

Definitions under shader can contain both shininess and gloss.

That is technically correct. In reality, you can write a shader in whichever way you see fit. If you want to have shininess, glossiness, metallicity, hardness or whatever other parameter you can come up with, you can have it in the shipdata shader definition as a shader uniform. It will be your responsibility to ensure that the shader works though, which means that you will need to write everything material related yourself. This is what Griff does in his own shaders and this is how he can have things like engines casting a blue glow on the back of the ship when they light up or how he can have decals on his ships. He is writing everything in the shaders he is using himself.

If there are no shader definitions then shininess has been depreciated and gloss can be defined under materials although they are not the same thing nor interchangeable.

Correct. Without specific shader definitions, the default Oolite shaders are used and those are designed to accept gloss, specularity etc. Gloss is not the same as shininess. Gloss is just a measure of how concentrated the specular reflection is, so a low gloss means that the specular reflection is spread over a wide area of the material and the higher gloss is, the more concentrated towards a single point the specular reflection is. To describe a material properly you need to combine gloss and specular, which in turn is a measure of how much of the light energy hitting a surface is reflected back. You may consider specular as a measure of reflectivity of a material. This way you can have a metallic surface which has high specularity (since metals reflect light a lot) and it could be rough, with the reflection heavily spread out or smooth, in which case the reflection will be very distinct and concentrated. Or you can have a plastic material (low specularity), which could be very glossy or very rough.

The 1.90 materials system asks for a different mindset to what there was before. You need to consider materials in terms of their visual properties and the PBR tutorials linked in the earlier posts here may be essential reading in order to fully understand it. You can also try playing around with the unmodified core game shipdata materials. Try it with the Cobra 3, use various combinations for specular color and gloss and see how the model appearance changes. Hint: Use shades of grey for describing specular color, at least at the beginning. Most metals and all non-metals reflect non-colored light.

Hope this clears things up a bit.

[Cholmondely]

I've added a link to this thread from the old Materials in Oolite wiki page: http://wiki.alioth.net/index.php/Materials_in_Oolite

Are there any other links which should be made?

[Griff]

I found these short intro videos on youtube, hopefully they help back up what A_C is teaching us in the posts above

The Specular/Glossiness Workflow: https://www.youtube.com/watch?v=YY3LkhjIsoc
What is a Specular map? https://www.youtube.com/watch?v=FNKQMa5Osr4
What is a glossiness map? https://www.youtube.com/watch?v=KyZdn5XLwfU
The Takeaways of the Specular Workflow: https://www.youtube.com/watch?v=SpVLJ25oAy0

Some other vids by the same guy on other maps that oolite uses
What is a Diffuse Map? https://www.youtube.com/watch?v=IkXMr20xcZU
What is a normal map: https://www.youtube.com/watch?v=jK11WjGuJaI

[Griff]

I've been trying to use an app called 'Substance Painter' to texture a ship with the spec/gloss workflow, i think it's sort of generated working maps of Oolite's materials system:


Does oolite do anything with the gloss map? - substance gave me a bunch of maps including an 'ambient occlusion' and a 'glossiness' map? i shoved the glossiness map into the alpha channel of the specular_map texture - was that right as it seems oolite would like 'shinyness' in there, is that the same as gloss?

does oolite do anything with the ambient occlusion map or am i to mix that into anything before exporting out the final textures for the oxp, or does it go into the colour map or anything

Test oxp if anyone wants to have a look it generates a few of the ships outside the station when the player launches - it's a big download as it contains all the maps (2048x2048 res at teh moment, hope some of these can be scaled down) from substance as well as the ones in the oxp, no shaders being used -it uses oolite's Materials settings
https://drive.google.com/file/d/1VvRQt4 ... sp=sharing

[another_commander]

I've been trying to use an app called 'Substance Painter' to texture a ship with the spec/gloss workflow, i think it's sort of generated working maps of Oolite's materials system:


Does oolite do anything with the gloss map? - substance gave me a bunch of maps including an 'ambient occlusion' and a 'glossiness' map? i shoved the glossiness map into the alpha channel of the specular_map texture - was that right as it seems oolite would like 'shinyness' in there, is that the same as gloss?

Oolite makes use of the gloss map and you have correctly inserted it in the specular alpha channel. 'shininsss' is no longer a part of the materials system. Note that, if you are using a spec/gloss map, then look out for any gloss = <whatever> entries in the materials definition of ships that specify it; when a spec/gloss map is used, that gloss = <whatever> entry is used as a multiplier. So, if your gloss map contains, say, a value of 0.8 and you have gloss = 0.5; in the materials definition, then the actual gloss value that the Oolite shader will use is 0.4. So, as a rule of thumb, keep any gloss definitions to 1.0 in the shipdata materials definition if you are using gloss maps. This way the intended value of the gloss map will be exactly what Oolite will use when the lighting shaders run.

does oolite do anything with the ambient occlusion map or am i to mix that into anything before exporting out the final textures for the oxp, or does it go into the colour map or anything

Ambient occlusion is not used at the moment and can (for now) be ignored, It should not be difficult to add it, but the host map for this information should be carefully selected. Currently, the obvious candidate for ambient occlusion (the alpha channel of the diffuse map) is in use by Oolite for effects in the core shipset, if I recall correctly.

[cbr]

As albedo value stands more of less for the amount of reflected light, can it be used as glossmap ?



top sidewinder uses inverted albedo texture ( resources ) as gloss map in the alpha layer

bottom sidewinder original

[another_commander]

As albedo value stands more of less for the amount of reflected light, can it be used as glossmap ?

Gloss can be whatever you want. It is the only map that can be designed completely according to the artist's will and inspiration. There is no rule, the artist just has to decide what kind of material they need and how concentrated the light reflection from that material will be and that's all.

[Commander_X]

[...]
does oolite do anything with the ambient occlusion map or am i to mix that into anything before exporting out the final textures for the oxp, or does it go into the colour map or anything
[...]

The funny thing about the AO map is that even if Oolite is not using it, you can mix (using a multiply layer operation) it into the diffuse map.

This is no AO (in Blender):

This is AO (also in Blender):


Having it definitely improves the perception of geometry.

[another_commander]

Need to be careful if you decide to bake the ambient occlusion information in the diffuse map. Normally AO affects only ambient light, so if you bake it in the diffuse you get occlusion at all times, even under direct light. This is why the AO is normally on a separate map, although it represents indeed, a simple multiplication operation in the shader. The critical detail is that the multiplication operation would normally happen in the ambient light calculation part.

[Griff]

Thanks for all the fantastic tips guys, i'll dig into it some more, i'm finding substance painter a bit of a struggle to work with so just need to keep at it i think, here's the tutorial i was following (in a much more simplified fashion than what the tutorial shows, my pc's not up to spec to be able to work with so many layers without long pauses between adding any painting or doing edits but it's ok with some basic stuff)

Realistic Texturing in Substance Painter w/ Jonas Ronnegard: Robotic Rover
part1: https://www.youtube.com/watch?v=hqxUIGp7-TI - Baking
part2: https://www.youtube.com/watch?v=IMOXs0noJbQ - Adding Details
part3: https://www.youtube.com/watch?v=eT2jutTCm38 - Base Materials
part4: https://www.youtube.com/watch?v=7ycGdinZkT4 - Final Textures
part5: https://www.youtube.com/watch?v=x_oyuXeOaDA - Rendering & Post

edit:
I haven't tried this app, but it's an opensource PBR painting app along the lines of substance painter, it might support Oolite's spec/gloss Materials workflow - from what i can tell there's paid version that's pre-built and ready to go or you can build it for free from github:

https://armorpaint.org/
"ArmorPaint runs on Windows, Linux, macOS, iPadOS and Android. On desktop, ArmorPaint is a portable application with no installation, near-instant start up and download size of less than 10MB."

https://gamefromscratch.com/building-ar ... om-source/
https://www.youtube.com/watch?v=y6h2KOP47ZY&t=10s

[another_commander]

Just a quick example of baked-in AO in the diffuse texture, this time rendered by Oolite. The first image of each pair is without AO, the second one is with AO baked-in.

As you will see, in this case, the render is overall improved. However, there is a small detail that showcases what I posted earlier about being careful with AO: See the cockpit area in the second pair? The light comes from the left, but in the AO shot there is some shadow shown in the left area of the cockpit. I am not sure that should be there. In this particular area, the image without AO baked-in seems to be more accurate. Anyway, that is a nitpick really and I think overall it looks good in the case of this Sidewinder.

[Griff]

Thank you so much for the guidance A_C and for taking the time to illustrate the points - there's a couple of panel lines that exist only in the normal map that i think baking AO into the colour texture would help stand out, but i see your point about how it's going to add incorrect premanent shadows in areas that have lighting shining on them directly - i think adding in some dirt/grease etc into those areas might help make the panel lines stand out a bit more, will have a look into this, at the moment most of this is just using edge wear and dirt presets found in substance, need to tweak them a bit, although the slow update makes this a bit maddening, about 20 seconds sometimes from moving a slider to seeing the texture update