[WIP] Cell shading OXP

[Day]

Hi everybody,

I've played a little bit with shaders to see if I could do a cell-shaded oolite, or at least a cell-shaded oxp.
So, working with Ngalo's Paint missiles as a base, I've made a Toonifying Missile :-p (thanks Ngalo!!)
It's working, similarly to Zelda windwaker. That is, it's basic.

I've begun by a toonifying oxp rather than the whole world, because I didn't know anything about shaders, so I had to learn first.

Technically, what does it do?
- applies toon shaders on the target,
- computes the color depending only on target's material ambient + world ambient (no textures, no original shaders),
- computes the shading depending only on target material diffuse + light diffuse (no textures, maps, shaders, specular...),
- makes some outlines.

The limitations I see:
- if I work without textures, the target I toonify becomes very simple, so it's not very impressive.
- if I work with textures, I think the effect will be not very easy to see.
- oolite's world is full of void, with some objects often very far. Cell-shading is best when there are objects all around.
- toonifying only one object is kind of lame.
- I think I can't do correct outlines in oxps. Would someone know if it is possible to compute a first pass, what I'd like would be to access from the shaders scripts the depth value of the current image, to be able to access the depth value of the neighbouring fragments.

So, where am I?
- I could try using the textures.
- I could try to toonify the whole world.

So...
Would somebody know if I can compute a first pass to have the depth of each pixel? Or does it need to be implemented?

Would somebody know what we could do with toonified objects?

Any suggestions?

Finally, my WIP oxp could be released as it is, with a little bit of beautifying. It's, I think, not interesting enough as it is to be used for anything else than testing / exploring. Would some be interested in a release?

Edit: now, I use the textures, and all the ships are toonified when launching from station.

[Griff]

Good luck with this project Day, I'd love to see a cel shaded oolite!
Ages ago I tried to implement this single pass cel shading technique by Jeanmarc Leroux, it looked like it would do the 'black outlines' effect in a single pass as I don't think Oolite can supports multiple shader passes.
( edit: lol, now that I've looked at things a bit more closely I don't think I used this tutorial at all, i'll leave it linked here though just in case it's any use )

It looks like the tutorial might have gone, at least the images don't seem to be appearing anymore. i found it here: http://blogs.aerys.in/jeanmarc-leroux/2 ... l-shading/

At the time I printed out a 'pdf' version of the page from inside Google's Chrome browser, if it helps I've uploaded that here:
Single Pass Cel Shading _ Jean-Marc Le Roux.pdf - https://app.box.com/s/g3y96731jooiewslk7u351zdkdgm8n0m

Here's as far as I got with my OXP , it looks like I was trying to get texture support into the shader as there seem to be references to textures in the oxp, yet they don't appear in game. I can't remember why I gave up on it - I must have reached as far as I could go with my ability to understand the shader


If it's any help, here's the oxp I came up (although it sounds like you've already progressed further than I managed!)
https://app.box.com/s/mt5knzbllw15t9hqapneak58cxj7a03u
Installing the OXP to AddOns, will spawn 5 cel-shaded teapots outside the station when you launch that appear on the scanner as asteroids

[Day]

Hi Griff,

thank you for your cheers and your links
I'm going to look into the tutorial and your oxp.

Edit: the tutorial is interesting. I already detected the edges like they do, but they add some new vertices to make the outline. I didn't know it was possible in this shader. I'm going to see if I can use it.

The real problem is that in all the tutorials, they detect edges in one pass only for smooth curves. In oolite, we use ships and other entities made of hard angles. So the changes aren't continuous, and it's difficult to identify an edge.

Edit2: error: they don't add vertices, they move them.

Edit3: I've looked into your oxp. Well, you've managed almost everything Textures don't seem to be so hard, I think. The Freaky thargoids oxp uses them, so we have an example. But I agree it would be some long code to manage every type of oolite textures (illumination, etc...).
For now, i didn't find how to make outlines in one pass for hard edges.
I guess I'll go see in oolite code if it would be feasible to add a pass. :-/

[Day]

In my current tests, I'm trying to add outlines without modifying oolite core.
A way to do add outlines to an entity is to add to it a upscaled black version of itself, with normals inverted.

But I struggle to do this in javascript
Is there a way to add a ship to another ship? (I tried adding a subentity, but the javascript objects being only an api to objective-c objects, and not the real deal, i guess it does nothing.)
Is there an easy way to invert the normals of a model?

Another way to enrich a model would be to modify the models when loading. Would this be possible?

Finally, i realized that ship.scale() doesn't exist (only visualEffect.scale()). Is there an alternative to scale a ship just spawned?

[cim]

Unfortunately not - you can't add subentities with JS which weren't defined in shipdata, and you can't modify a model in JS once it's loaded. You can in 1.82 use model_scale_factor in shipdata.plist, but there's no runtime equivalent.

[Day]

Unfortunately not - you can't add subentities with JS which weren't defined in shipdata, and you can't modify a model in JS once it's loaded. You can in 1.82 use model_scale_factor in shipdata.plist, but there's no runtime equivalent.

Thank you cim.
I'm now testing if I can generate outlines when loading meshes, in OOMesh.m.

[Day]

Ok, I've hit a snag.

I'm currently modifying OOMesh.m to generate an slightly bigger envelope around meshes when loading them.
The idea is to have inverted normals so that the envelope front faces are invisible, and only the back faces are visible, which draws an outline around the mesh.

Generating the envelope is working quite well, I assign the material 0 which produces a dark grey slightly shiny, ok for a test.
BUT the front faces from the envelope are visible

My test is to observe the Cobra mk3 from the beginning (stock one), sometimes to go into the ships library.

I suspect either a culling subtility, or an unknown (to me) subtility.

Would somebody have an suggestion?

Edit: in the code below, I create for each vertex a new vertex, for each face a new face, the normals are inverted and the tangents stay the same.

Diff:

Code: Select all

diff --git a/src/Core/OOMesh.m b/src/Core/OOMesh.m
index 5bcb6d6..e63bde3 100644
--- a/src/Core/OOMesh.m
+++ b/src/Core/OOMesh.m
@@ -1221,7 +1221,7 @@ shaderBindingTarget:(id<OOWeakReferenceSupport>)target
 		{
 			int n_v;
 			if ([scanner scanInt:&n_v])
-				vertexCount = n_v;
+				vertexCount = 2 * n_v;
 			else
 			{
 				failFlag = YES;
@@ -1246,7 +1246,7 @@ shaderBindingTarget:(id<OOWeakReferenceSupport>)target
 			int n_f;
 			if ([scanner scanInt:&n_f])
 			{
-				faceCount = n_f;
+				faceCount = 2 * n_f;
 			}
 			else
 			{
@@ -1283,7 +1283,7 @@ shaderBindingTarget:(id<OOWeakReferenceSupport>)target
 		// get vertex data
 		if ([scanner scanString:@"VERTEX" intoString:NULL])
 		{
-			for (j = 0; j < vertexCount; j++)
+			for (j = 0; j < vertexCount/2; j++)
 			{
 				float x, y, z;
 				if (!failFlag)
@@ -1294,6 +1294,7 @@ shaderBindingTarget:(id<OOWeakReferenceSupport>)target
 					if (!failFlag)
 					{
 						_vertices[j] = make_vector(x*scale, y*scale, z*scale);
+						_vertices[vertexCount/2+j] = make_vector(x*scale*1.1, y*scale*1.1, z*scale*1.1);
 					}
 					else
 					{
@@ -1311,7 +1312,7 @@ shaderBindingTarget:(id<OOWeakReferenceSupport>)target
 		// get face data
 		if ([scanner scanString:@"FACES" intoString:NULL])
 		{
-			for (j = 0; j < faceCount; j++)
+			for (j = 0; j < faceCount/2; j++)
 			{
 				int r, g, b;
 				float nx, ny, nz;
@@ -1325,6 +1326,7 @@ shaderBindingTarget:(id<OOWeakReferenceSupport>)target
 					if (!failFlag)
 					{
 						_faces[j].smoothGroup = r;
+						_faces[faceCount/2+j].smoothGroup = (r+128)%256; // different smoothGroup
 					}
 					else
 					{
@@ -1338,6 +1340,7 @@ shaderBindingTarget:(id<OOWeakReferenceSupport>)target
 					if (!failFlag)
 					{
 						_faces[j].normal = vector_normal(make_vector(nx, ny, nz));
+						_faces[faceCount/2+j].normal = vector_normal(make_vector(-nx, -ny, -nz));
 					}
 					else
 					{
@@ -1372,7 +1375,11 @@ shaderBindingTarget:(id<OOWeakReferenceSupport>)target
 							if ([scanner scanInt:&vi])
 							{
 								_faces[j].vertex[i] = vi;
-								if (faceRefs != NULL)  VFRAddFace(&faceRefs[vi], j);
+								_faces[faceCount/2+j].vertex[i] = vertexCount/2+vi;
+								if (faceRefs != NULL) {
+								   VFRAddFace(&faceRefs[vi], j);
+								   VFRAddFace(&faceRefs[vertexCount/2+vi], faceCount/2+j);
+								}
 							}
 							else
 							{
@@ -1393,7 +1400,7 @@ shaderBindingTarget:(id<OOWeakReferenceSupport>)target
 		// Get textures data.
 		if ([scanner scanString:@"TEXTURES" intoString:NULL])
 		{
-			for (j = 0; j < faceCount; j++)
+			for (j = 0; j < faceCount/2; j++)
 			{
 				NSString	*materialKey;
 				float	max_x, max_y;
@@ -1414,6 +1421,7 @@ shaderBindingTarget:(id<OOWeakReferenceSupport>)target
 						if (index != nil)
 						{
 							_faces[j].materialIndex = [index unsignedIntValue];
+							_faces[faceCount/2+j].materialIndex = 0;
 						}
 						else
 						{
@@ -1423,6 +1431,7 @@ shaderBindingTarget:(id<OOWeakReferenceSupport>)target
 								return NO;
 							}
 							_faces[j].materialIndex = materialCount;
+							_faces[faceCount/2+j].materialIndex = 0;
 							materialKeys[materialCount] = [materialKey retain];
 							index = [NSNumber numberWithUnsignedInt:materialCount];
 							[texFileName2Idx setObject:index forKey:materialKey];
@@ -1467,9 +1476,10 @@ shaderBindingTarget:(id<OOWeakReferenceSupport>)target
 			materialKeys[0] = @"_oo_placeholder_material";
 			materialCount = 1;
 			
-			for (j = 0; j < faceCount; j++)
+			for (j = 0; j < faceCount/2; j++)
 			{
 				_faces[j].materialIndex = 0;
+				_faces[faceCount/2+j].materialIndex = 0;
 			}
 		}
 		
@@ -1512,7 +1522,7 @@ shaderBindingTarget:(id<OOWeakReferenceSupport>)target
 				return NO;
 			}
 			
-			for (j = 0; j < vertexCount; j++)
+			for (j = 0; j < vertexCount/2; j++)
 			{
 				float x, y, z;
 				if (!failFlag)
@@ -1523,6 +1533,7 @@ shaderBindingTarget:(id<OOWeakReferenceSupport>)target
 					if (!failFlag)
 					{
 						_normals[j] = vector_normal(make_vector(x, y, z));
+						_normals[vertexCount/2+j] = vector_normal(make_vector(-x, -y, -z));
 					}
 					else
 					{
@@ -1534,7 +1545,7 @@ shaderBindingTarget:(id<OOWeakReferenceSupport>)target
 			// Get explicit tangents (only together with vertices).
 			if ([scanner scanString:@"TANGENTS" intoString:NULL])
 			{
-				for (j = 0; j < vertexCount; j++)
+				for (j = 0; j < vertexCount/2; j++)
 				{
 					float x, y, z;
 					if (!failFlag)
@@ -1545,6 +1556,7 @@ shaderBindingTarget:(id<OOWeakReferenceSupport>)target
 						if (!failFlag)
 						{
 							_tangents[j] = vector_normal(make_vector(x, y, z));
+							_tangents[vertexCount/2+j] = vector_normal(make_vector(x, y, z));
 						}
 						else
 						{

[Day]

Don't bother, I've found it.

[Day]

Some models don't declare normal, oolite calculate them from the faces' vertices rotation order. So as I wasn't changing the rotation order of my additional faces (I thought negating the normal was enough), I was in fact using the same normals, and ended with a visible envelope rather than an outline.

[Day]

Some results

No change on light in these pictures, only a small outline, but it gives a definite cartoonish feeling, don't you think?
The ambient light is set at 4.0.
Below the images are the links to the original 1920x1080 pics.



Original 1920x1080 versions:
http://pradier.info/oolite-006.png
http://pradier.info/oolite-010.png

[Ngalo]

Just a thought from someone who knows very little about shaders (so I hope you'll forgive me if you're already doing this) but would it help to round each component of the fragment shader's final output colour to the nearest 64 or thereabouts (maybe even the nearest 128, assuming RGB values are each 0 to 256)? This should have the effect of 'simplifying' textures and shading , replacing subtle low-contrast variation with bolder blocks of uniform colour.

[Day]

You're right, it could be interesting.

Generally, cel-shading is done with only 2 or 3 levels of shade on top of the pure color. But it is totally possible to explore different effects which could give different artistic feelings.

What you propose seems akin to the "posterize" function of softwares like Photoshop or Gimp.

Hmmm, the nearest 64 on a 256 scale on three components... That gives 15 different colors :-p Back to CGA !

[Cholmondely]

You're right, it could be interesting.

Generally, cel-shading is done with only 2 or 3 levels of shade on top of the pure color. But it is totally possible to explore different effects which could give different artistic feelings.

What you propose seems akin to the "posterize" function of softwares like Photoshop or Gimp.

Hmmm, the nearest 64 on a 256 scale on three components... That gives 15 different colors :-p Back to CGA !

Did an .oxp ever come out of this thread?