Re: Render to Framebuffer
Posted: Wed Aug 03, 2022 6:06 pm
You must replace the latest 1.91 executable. The test will not work on 1.90, since that is configured for the old style (pre- in-game setup) keyboard handling.
Oolite Bulletins
For information and discussion about Oolite.
https://bb.oolite.space/
Render to Framebuffer
Page 3 of 6
Re: Render to Framebuffer
Posted: Wed Aug 03, 2022 7:49 pm
Ahhh - that explains it. Thank You Very Much.
Re: Render to Framebuffer
Posted: Thu Aug 04, 2022 6:45 am
Now that we have the basics of rendering to texture down, we can utilize the feature also to increase accessibility and help color blind users in the future. I found a color blindness simulation shader at shadertoy and copied it over to our oolite-texture.fragment (btw, almost all effects used so far were taken from shadertoy, with modifications for our engine wherever applicable).
In order to be able to make the game more accessible to our color blind users, it is important to understand what they see first. And here is where this post-processing feature helps. We now know how people with protanopia, deuteranopia and tritanopia see color in the game. Here, take a look at this:
This is the reference scene, i.e. what is normally presented on screen:
Slightly towards the left and below the center I have positioned an asteroid with a red scanner color. This will be present on all images at the same position. The rest of the lollipops are random ships spawned by the game.
This is what people with protanopia see:
This is what people with deuteranopia see:
This is what people with tritanopia see:
Turns out that our HUD design is not that much color blind friendly after all. What games normally do to address such issues is that they have a special color blindness mode where red colors are brightened and green colors are darkened. This way, people with protanopia and deuteranopia can distinguish at least the difference in brightness. The same for the blues and the yellows respectively, for people with tritanopia. In the future we could consider something like this. It may take a while to get there, but at least we now know what to look for.
In order to be able to make the game more accessible to our color blind users, it is important to understand what they see first. And here is where this post-processing feature helps. We now know how people with protanopia, deuteranopia and tritanopia see color in the game. Here, take a look at this:
This is the reference scene, i.e. what is normally presented on screen:
Slightly towards the left and below the center I have positioned an asteroid with a red scanner color. This will be present on all images at the same position. The rest of the lollipops are random ships spawned by the game.
This is what people with protanopia see:
This is what people with deuteranopia see:
This is what people with tritanopia see:
Turns out that our HUD design is not that much color blind friendly after all. What games normally do to address such issues is that they have a special color blindness mode where red colors are brightened and green colors are darkened. This way, people with protanopia and deuteranopia can distinguish at least the difference in brightness. The same for the blues and the yellows respectively, for people with tritanopia. In the future we could consider something like this. It may take a while to get there, but at least we now know what to look for.
Re: Render to Framebuffer
Posted: Thu Aug 04, 2022 1:01 pm
Following up on previous post, it turns out that the very same shader I used to demonstrate the color blindness types contains also code for corrections based on the color darkening and brightening changes mentioned above. So, it is possible to be more color blind friendly, we would just need a way to switch the post processing in real time.
In the following gifs, the first frame is always the non-colorblindness one, where all colors appear as usual. The second frame is what a color blind person would see and the third is the colorblindness correction applied to make the image more accessible. Note the increased contrast in the corrected images and how easier they are to view compared to the non-corrected ones. One can tell right away which areas are definitely differently colored. It is not 100% perfect, but it is something and can be used to help people, so it is a Good Thing.
Protanopia:
Deuteranopia:
Tritanopia:
In the following gifs, the first frame is always the non-colorblindness one, where all colors appear as usual. The second frame is what a color blind person would see and the third is the colorblindness correction applied to make the image more accessible. Note the increased contrast in the corrected images and how easier they are to view compared to the non-corrected ones. One can tell right away which areas are definitely differently colored. It is not 100% perfect, but it is something and can be used to help people, so it is a Good Thing.
Protanopia:
Deuteranopia:
Tritanopia:
Re: Render to Framebuffer
Posted: Fri Aug 05, 2022 6:42 am
Hard to tell from the examples, but would I imagine the differences in lollipops on the radar would be particularly difficult to appreciate.another_commander wrote: ↑Thu Aug 04, 2022 6:45 amWe now know how people with protanopia, deuteranopia and tritanopia see color in the game.
Re: Render to Framebuffer
Posted: Sat Aug 06, 2022 3:23 pm
I found that with GL_RGBA16F the animation of the old BGS hyperspace tunnel effect is messed up, but it works with GL_RGBA. Does HDR really not work with GL_RGBA also, or do you have an idea why this shader (old bgs hyper) looks weird with RGBA16F, but not with GL_RGBA?another_commander wrote: ↑Sun Jul 31, 2022 3:18 pmI tried applying HDR the "right" way, by tone mapping and gamma correcting at the oolite-texture shader. So I removed those steps from the default shaders and added them once only in the render texture shader.
Code: Select all
// 136 instructions, 5 R-regs, 0 H-regs - 59 ALU
uniform sampler2D colorMap; // this comes from a simple RGB 8 bit PNG texture
uniform float fTime;
uniform vec4 ovSpecials, Look;
varying vec2 vTexCoord;
const float SQRT3 = 1.73205;
const float PI = 3.1415926;
vec2 repeat(vec2 p, float n){
vec2 np = p*n;
vec2 npreal = np-fract(np);
np.x += fract(npreal.y*0.5);
return fract(np)*2.0-1.0;
}
float hexDistance(vec2 ip){
vec2 p = abs(ip*vec2(SQRT3*0.5,0.75));
float d = dot(p,vec2(-0.5,0.8660254))-SQRT3*0.25;
return (d>0.0)?min(d,(SQRT3*0.5-p.x)):min(-d,p.x);
}
void main()
{
vec2 coords = vTexCoord.xy;
coords.y /= ovSpecials.z;
vec2 cen = vec2(0.5);
cen.y /= ovSpecials.z;
cen -= coords.xy;
float a = atan(cen.y,cen.x);
float angle = fract(a/PI);
float rad = length(cen);
float t = fTime+1.0;
float t1 = max(-1.0+log(t),0.0);
float t2 = max(t1*0.6-0.2,0.0);
float rrad = 1.0/rad;
// Texture
vec2 mcen = 0.07*log(rad)*normalize(cen)*t*5.0;
vec2 tem = cen*(3.0-t);
vec4 FogColor = vec4(0.06,0.05,0.1,0.1)*exp(t);
float fadeT = max(1.5-fTime*0.2,0.1);
vec4 CurrentColor = texture2D(colorMap,coords.xy+tem+mcen);
CurrentColor += texture2D(colorMap,coords.xy+mcen);
FogColor.b += ovSpecials.x*t;
CurrentColor.r *= ovSpecials.y*2.0*t;
float FogDistance = rrad*-0.01;
vec4 endColor = mix(FogColor,CurrentColor,exp(FogDistance));
if(endColor.a>3.9) discard;
// Stars
float as = angle*256.0;
float angleRnd = floor(as)+1.0;
float radDist = fract(angleRnd*fract(angleRnd*0.82657)*13.724)/SQRT3;
float adist = radDist/rad*0.1;
float bdist = (t+fract(angleRnd*fract(angleRnd*0.7235)*45.1)*10.0)*0.1+adist;
bdist = abs(fract(bdist)-0.5);
float color = (max(0.0,0.5-bdist*30.0/adist)*abs(fract(as))*5.0/adist*radDist)*(t-1.5);
endColor += color;
// Tunnel
tem = vec2(a/PI,rrad+t);
vec2 p = repeat(tem,12.0);
float d = hexDistance(p);
float grid = smoothstep(-0.1,0.26,d)/t2;
vec3 rgb = vec3(0.27,0.07,0.35)/grid;
// Glowing lines
tem *= cen-t-adist/tem.y;
float y = (0.8/abs(cos(tem.x*t1)+cos(tem.y)*sin(t))*t2)*t2;
rgb += vec3(y*rad,y*0.4,y*0.8);
// Output
gl_FragColor = endColor+vec4(rgb,1.0);
}
Re: Render to Framebuffer
Posted: Sat Aug 06, 2022 3:54 pm
Without having looked at it in detail, I fully expect the shader to have a missing clamp() problem somewhere. The problem is definitely shader-side. Try to remove the ACESFilm line in the shader I posted (and keep GL_RGBA16F). Does it still look messy?
GL_RGBA16F is a one way street, it is not optional. This buffer type allows HDR content, GL_RGBA does not.
I'll try to figure out the shader issue once I get the opportunity, but it may be after this weekend.
GL_RGBA16F is a one way street, it is not optional. This buffer type allows HDR content, GL_RGBA does not.
I'll try to figure out the shader issue once I get the opportunity, but it may be after this weekend.
Re: Render to Framebuffer
Posted: Sat Aug 06, 2022 7:47 pm
The shader is fixed by adding this line at the very end:
I have to say that I like the tripiness of the non-fixed shader, though. 
Code: Select all
gl_FragColor.a = 1.0;Re: Render to Framebuffer
Posted: Sat Aug 06, 2022 7:53 pm
<chortles> In the same way I liked the rainbow spacedust that a long-ago nightly once introduced?another_commander wrote: ↑Sat Aug 06, 2022 7:47 pmI have to say that I like the tripiness of the non-fixed shader, though.
Re: Render to Framebuffer
Posted: Sun Aug 07, 2022 2:25 am
@another_commander
Thanks! The fix seems to work on my machine also. I updated the download for the customized BGS version.
I believe I fixed the screenshot and resolution issues. I also cleaned up some code. There are a few small things I already know that still have to be done:
- write the correct OpenGL version into the shader
- make the adjustments to the OpenGLExtensionManager and such what you mentioned earlier
- When I pass a
- The HDR thingy would have to be implemented, I didn't yet add your changes (I didn't find them in the rtt.diff).
- not really related, I found in
The screenshot didn't work previously (if I'm correct) simply because at the time of getting the screenshot out of the active framebuffer (I believe here) the active framebuffer was still the target texture framebuffer and not the "screen framebuffer".
What I noticed is that when resizing the window with the mouse, the window gets black for a few seconds and wobbles weirdly around for a few seconds. I am not sure if that was also an issue before, but in any case I don't think it is too bad.
Now to bed.
Thanks! The fix seems to work on my machine also. I updated the download for the customized BGS version.
I believe I fixed the screenshot and resolution issues. I also cleaned up some code. There are a few small things I already know that still have to be done:
- write the correct OpenGL version into the shader
- make the adjustments to the OpenGLExtensionManager and such what you mentioned earlier
- When I pass a
[NSDictionary dictionary] to a function, do I have to release it later to avoid memory leaks? (generally, I am not yet 100% about the resource management in objective-c)- The HDR thingy would have to be implemented, I didn't yet add your changes (I didn't find them in the rtt.diff).
- not really related, I found in
OOEnvironmentCubeMap.m that the framebuffer is set to 0 at some point (as well as a texture and a renderbuffer). It might be better to do the same thing we've been doing here: Before doing anything store the previous bindings, and rebind them after we're done. I don't know if it would even cause a bug if we didn't change this, but I could imagine that in future this could become a bug that someone would have to search for for hours.The screenshot didn't work previously (if I'm correct) simply because at the time of getting the screenshot out of the active framebuffer (I believe here) the active framebuffer was still the target texture framebuffer and not the "screen framebuffer".
What I noticed is that when resizing the window with the mouse, the window gets black for a few seconds and wobbles weirdly around for a few seconds. I am not sure if that was also an issue before, but in any case I don't think it is too bad.
Now to bed.
Re: Render to Framebuffer
Posted: Sun Aug 07, 2022 9:50 am
Great stuff tsoj. It didn't work right away for me, but after some minimal editing of your source changes it now seems to work flawlessly, with all three main earlier observations apparently fixed.
Edit: Reuploaded the file because of a bug in resizing found and fixed, if you downloaded it earlier please retry. The new file is OoliteRenderToFrameBufferTest20220807_02.zip
I have created a package, available from here which contains:
- The current Windows binary for testing.
- The 4 main source code files that I had to touch in order to get it to build on Windows.
- The updated versions of the oolite-texture shaders. This contains all the post processing filters implemented so far (selectable before running by simply uncommenting the effect you want to activate), including the latest colorblindness correction filters. It also contains the first of the two parts of the HDR implementation.
- The main Oolite fragment shaders, updated with the second of the two parts of the HDR implementation. These are basically the same shaders we have, but with the tone mapping and gamma correction steps removed, since we now apply those in the total scene render, as the case should be. You must replace the ones in Resources/Shaders with these or the game will look weird.
Regarding the changes in the source code files, this is what I have done:
1. Avoiding crash #1 on Windows: Moved -initTargetFramebufferWithViewSize to just after the OOOpenGLExtensionManager initialization. Due to the way Windows handles OpenGL, it is criticial that all the Win OpenGL extensions have been initialized before attempting to call any of them. Moving the initialization of the framebuffer below that of OOOpenGLExtensionManager allows us to not crash.
2. Avoiding crash #2 on Windows: Deleting all OpenGL objects and recreating them when the window is resized causes a glorious CTD on Windows. Thankfully, we do not need to delete anything; we just need to update the framebuffer and depth buffer data with the new screen size and we are good to go. I changed the name of the method -reinitTargetFramebufferWithViewSize to -resizeTargetFramebufferWithViewSize and did just that. It works like a charm here, please check this is also OK on Linux (I cannot see why it shouldn't be).
3. Passed two more uniforms in the oolite-texture shader: uTime and uResolution. The first gives us the ability to use animated post processing fx (check CRT and night vision for examples) and uResolution, which is used by the FXAA antialiasing filter contained in the shader and is something that can be generally useful to have available.
4. Removed a couple of glFinish() calls, together with the clock calls that were nearby. I am not sure why two glFinish() were there, but it seems to work fine without them.
Sorry for not having a diff available this time, just drop the files from the package in the appropriate locations of your test tree and they should work.
Regarding your question about memory management, normally only things that are initialized with [alloc[init]] and manual retains require manual releases. I see that you are doing a manual retain in the shader program initialization. Maybe a manual release would be a valid onsideration, however: since the particular shader program remains in use for the entire application execution, maybe we can get away with it releasing it only once at the end with dealloc, as you have already done. A run with Valgrind could maybe catch something like this, but I believe we are OK.
As for OOEnvironmentMap.m, I have no issue with your proposal. In any case, this is something that we can look at at a later stage, since this file is not currently part of the source set that gets compiled.
Finally, I do not see any issue with window resizing (after the fix mentioned above). It seems to resize just fine without artifacts or side effects here.
Edit: Reuploaded the file because of a bug in resizing found and fixed, if you downloaded it earlier please retry. The new file is OoliteRenderToFrameBufferTest20220807_02.zip
I have created a package, available from here which contains:
- The current Windows binary for testing.
- The 4 main source code files that I had to touch in order to get it to build on Windows.
- The updated versions of the oolite-texture shaders. This contains all the post processing filters implemented so far (selectable before running by simply uncommenting the effect you want to activate), including the latest colorblindness correction filters. It also contains the first of the two parts of the HDR implementation.
- The main Oolite fragment shaders, updated with the second of the two parts of the HDR implementation. These are basically the same shaders we have, but with the tone mapping and gamma correction steps removed, since we now apply those in the total scene render, as the case should be. You must replace the ones in Resources/Shaders with these or the game will look weird.
Regarding the changes in the source code files, this is what I have done:
1. Avoiding crash #1 on Windows: Moved -initTargetFramebufferWithViewSize to just after the OOOpenGLExtensionManager initialization. Due to the way Windows handles OpenGL, it is criticial that all the Win OpenGL extensions have been initialized before attempting to call any of them. Moving the initialization of the framebuffer below that of OOOpenGLExtensionManager allows us to not crash.
2. Avoiding crash #2 on Windows: Deleting all OpenGL objects and recreating them when the window is resized causes a glorious CTD on Windows. Thankfully, we do not need to delete anything; we just need to update the framebuffer and depth buffer data with the new screen size and we are good to go. I changed the name of the method -reinitTargetFramebufferWithViewSize to -resizeTargetFramebufferWithViewSize and did just that. It works like a charm here, please check this is also OK on Linux (I cannot see why it shouldn't be).
3. Passed two more uniforms in the oolite-texture shader: uTime and uResolution. The first gives us the ability to use animated post processing fx (check CRT and night vision for examples) and uResolution, which is used by the FXAA antialiasing filter contained in the shader and is something that can be generally useful to have available.
4. Removed a couple of glFinish() calls, together with the clock calls that were nearby. I am not sure why two glFinish() were there, but it seems to work fine without them.
Sorry for not having a diff available this time, just drop the files from the package in the appropriate locations of your test tree and they should work.
Regarding your question about memory management, normally only things that are initialized with [alloc[init]] and manual retains require manual releases. I see that you are doing a manual retain in the shader program initialization. Maybe a manual release would be a valid onsideration, however: since the particular shader program remains in use for the entire application execution, maybe we can get away with it releasing it only once at the end with dealloc, as you have already done. A run with Valgrind could maybe catch something like this, but I believe we are OK.
As for OOEnvironmentMap.m, I have no issue with your proposal. In any case, this is something that we can look at at a later stage, since this file is not currently part of the source set that gets compiled.
Finally, I do not see any issue with window resizing (after the fix mentioned above). It seems to resize just fine without artifacts or side effects here.
Re: Render to Framebuffer
Posted: Sun Aug 07, 2022 10:34 am
And here is the source code patch based on current trunk master:
Code: Select all
diff --git a/src/Core/OOOpenGLExtensionManager.h b/src/Core/OOOpenGLExtensionManager.h
index 9e7c246f..586d6e2a 100644
--- a/src/Core/OOOpenGLExtensionManager.h
+++ b/src/Core/OOOpenGLExtensionManager.h
@@ -247,6 +247,31 @@ PFNGLFRAMEBUFFERTEXTURE2DEXTPROC glFramebufferTexture2DEXT;
PFNGLCHECKFRAMEBUFFERSTATUSEXTPROC glCheckFramebufferStatusEXT;
PFNGLDELETEFRAMEBUFFERSEXTPROC glDeleteFramebuffersEXT;
PFNGLDELETERENDERBUFFERSEXTPROC glDeleteRenderbuffersEXT;
+PFNGLGENRENDERBUFFERSPROC glGenRenderbuffers;
+PFNGLBINDRENDERBUFFERPROC glBindRenderbuffer;
+PFNGLRENDERBUFFERSTORAGEPROC glRenderbufferStorage;
+PFNGLGENFRAMEBUFFERSPROC glGenFramebuffers;
+PFNGLBINDFRAMEBUFFERPROC glBindFramebuffer;
+PFNGLFRAMEBUFFERRENDERBUFFERPROC glFramebufferRenderbuffer;
+PFNGLFRAMEBUFFERTEXTURE2DPROC glFramebufferTexture2D;
+PFNGLGENVERTEXARRAYSPROC glGenVertexArrays;
+PFNGLGENBUFFERSPROC glGenBuffers;
+PFNGLBINDVERTEXARRAYPROC glBindVertexArray;
+PFNGLBINDBUFFERPROC glBindBuffer;
+PFNGLBUFFERDATAPROC glBufferData;
+PFNGLVERTEXATTRIBPOINTERPROC glVertexAttribPointer;
+PFNGLENABLEVERTEXATTRIBARRAYPROC glEnableVertexAttribArray;
+PFNGLUSEPROGRAMPROC glUseProgram;
+PFNGLGETUNIFORMLOCATIONPROC glGetUniformLocation;
+PFNGLUNIFORM1IPROC glUniform1i;
+PFNGLACTIVETEXTUREPROC glActiveTexture;
+PFNGLBLENDFUNCSEPARATEPROC glBlendFuncSeparate;
+PFNGLUNIFORM1FPROC glUniform1f;
+PFNGLUNIFORM2FVPROC glUniform2fv;
+PFNGLDELETERENDERBUFFERSPROC glDeleteRenderbuffers;
+PFNGLDELETEFRAMEBUFFERSPROC glDeleteFramebuffers;
+PFNGLDELETEVERTEXARRAYSPROC glDeleteVertexArrays;
+PFNGLDELETEBUFFERSPROC glDeleteBuffers;
#endif
#endif // OOLITE_WINDOWS
diff --git a/src/Core/OOOpenGLExtensionManager.m b/src/Core/OOOpenGLExtensionManager.m
index 2b2a637a..790b5873 100644
--- a/src/Core/OOOpenGLExtensionManager.m
+++ b/src/Core/OOOpenGLExtensionManager.m
@@ -107,7 +107,32 @@ PFNGLFRAMEBUFFERTEXTURE2DEXTPROC glFramebufferTexture2DEXT = (PFNGLFRAMEBUFFER
PFNGLCHECKFRAMEBUFFERSTATUSEXTPROC glCheckFramebufferStatusEXT = (PFNGLCHECKFRAMEBUFFERSTATUSEXTPROC)&OOBadOpenGLExtensionUsed;
PFNGLDELETEFRAMEBUFFERSEXTPROC glDeleteFramebuffersEXT = (PFNGLDELETEFRAMEBUFFERSEXTPROC)&OOBadOpenGLExtensionUsed;
PFNGLDELETERENDERBUFFERSEXTPROC glDeleteRenderbuffersEXT = (PFNGLDELETERENDERBUFFERSEXTPROC)&OOBadOpenGLExtensionUsed;
-#endif
+PFNGLGENRENDERBUFFERSPROC glGenRenderbuffers = (PFNGLGENRENDERBUFFERSPROC)&OOBadOpenGLExtensionUsed;
+PFNGLBINDRENDERBUFFERPROC glBindRenderbuffer = (PFNGLBINDRENDERBUFFERPROC)&OOBadOpenGLExtensionUsed;
+PFNGLRENDERBUFFERSTORAGEPROC glRenderbufferStorage = (PFNGLRENDERBUFFERSTORAGEPROC)&OOBadOpenGLExtensionUsed;
+PFNGLGENFRAMEBUFFERSPROC glGenFramebuffers = (PFNGLGENFRAMEBUFFERSPROC)&OOBadOpenGLExtensionUsed;
+PFNGLBINDFRAMEBUFFERPROC glBindFramebuffer = (PFNGLBINDFRAMEBUFFERPROC)&OOBadOpenGLExtensionUsed;
+PFNGLFRAMEBUFFERRENDERBUFFERPROC glFramebufferRenderbuffer = (PFNGLFRAMEBUFFERRENDERBUFFERPROC)&OOBadOpenGLExtensionUsed;
+PFNGLFRAMEBUFFERTEXTURE2DPROC glFramebufferTexture2D = (PFNGLFRAMEBUFFERTEXTURE2DPROC)&OOBadOpenGLExtensionUsed;
+PFNGLGENVERTEXARRAYSPROC glGenVertexArrays = (PFNGLGENVERTEXARRAYSPROC)&OOBadOpenGLExtensionUsed;
+PFNGLGENBUFFERSPROC glGenBuffers = (PFNGLGENBUFFERSPROC)&OOBadOpenGLExtensionUsed;
+PFNGLBINDVERTEXARRAYPROC glBindVertexArray = (PFNGLBINDVERTEXARRAYPROC)&OOBadOpenGLExtensionUsed;
+PFNGLBINDBUFFERPROC glBindBuffer = (PFNGLBINDBUFFERPROC)&OOBadOpenGLExtensionUsed;
+PFNGLBUFFERDATAPROC glBufferData = (PFNGLBUFFERDATAPROC)&OOBadOpenGLExtensionUsed;
+PFNGLVERTEXATTRIBPOINTERPROC glVertexAttribPointer = (PFNGLVERTEXATTRIBPOINTERPROC)&OOBadOpenGLExtensionUsed;
+PFNGLENABLEVERTEXATTRIBARRAYPROC glEnableVertexAttribArray = (PFNGLENABLEVERTEXATTRIBARRAYPROC)&OOBadOpenGLExtensionUsed;
+PFNGLUSEPROGRAMPROC glUseProgram = (PFNGLUSEPROGRAMPROC)&OOBadOpenGLExtensionUsed;
+PFNGLGETUNIFORMLOCATIONPROC glGetUniformLocation = (PFNGLGETUNIFORMLOCATIONPROC)&OOBadOpenGLExtensionUsed;
+PFNGLUNIFORM1IPROC glUniform1i = (PFNGLUNIFORM1IPROC)&OOBadOpenGLExtensionUsed;
+PFNGLACTIVETEXTUREPROC glActiveTexture = (PFNGLACTIVETEXTUREPROC)&OOBadOpenGLExtensionUsed;
+PFNGLBLENDFUNCSEPARATEPROC glBlendFuncSeparate = (PFNGLBLENDFUNCSEPARATEPROC)&OOBadOpenGLExtensionUsed;
+PFNGLUNIFORM1FPROC glUniform1f = (PFNGLUNIFORM1FPROC)&OOBadOpenGLExtensionUsed;
+PFNGLUNIFORM2FVPROC glUniform2fv = (PFNGLUNIFORM2FVPROC)&OOBadOpenGLExtensionUsed;
+PFNGLDELETERENDERBUFFERSPROC glDeleteRenderbuffers = (PFNGLDELETERENDERBUFFERSPROC)&OOBadOpenGLExtensionUsed;
+PFNGLDELETEFRAMEBUFFERSPROC glDeleteFramebuffers = (PFNGLDELETEFRAMEBUFFERSPROC)&OOBadOpenGLExtensionUsed;
+PFNGLDELETEVERTEXARRAYSPROC glDeleteVertexArrays = (PFNGLDELETEVERTEXARRAYSPROC)&OOBadOpenGLExtensionUsed;
+PFNGLDELETEBUFFERSPROC glDeleteBuffers = (PFNGLDELETEBUFFERSPROC)&OOBadOpenGLExtensionUsed;
+#endif
#endif
@@ -625,6 +650,31 @@ static unsigned IntegerFromString(const GLubyte **ioString)
glCheckFramebufferStatusEXT = (PFNGLCHECKFRAMEBUFFERSTATUSEXTPROC)wglGetProcAddress("glCheckFramebufferStatusEXT");
glDeleteFramebuffersEXT = (PFNGLDELETEFRAMEBUFFERSEXTPROC)wglGetProcAddress("glDeleteFramebuffersEXT");
glDeleteRenderbuffersEXT = (PFNGLDELETERENDERBUFFERSEXTPROC)wglGetProcAddress("glDeleteRenderbuffersEXT");
+ glGenRenderbuffers = (PFNGLGENRENDERBUFFERSPROC)wglGetProcAddress("glGenRenderbuffers");
+ glBindRenderbuffer = (PFNGLBINDRENDERBUFFERPROC)wglGetProcAddress ("glBindRenderbuffer" );
+ glRenderbufferStorage = (PFNGLRENDERBUFFERSTORAGEPROC)wglGetProcAddress ("glRenderbufferStorage" );
+ glGenFramebuffers = (PFNGLGENFRAMEBUFFERSPROC)wglGetProcAddress ("glGenFramebuffers" );
+ glBindFramebuffer = (PFNGLBINDFRAMEBUFFERPROC)wglGetProcAddress ("glBindFramebuffer" );
+ glFramebufferRenderbuffer = (PFNGLFRAMEBUFFERRENDERBUFFERPROC)wglGetProcAddress ("glFramebufferRenderbuffer" );
+ glFramebufferTexture2D = (PFNGLFRAMEBUFFERTEXTURE2DPROC)wglGetProcAddress ("glFramebufferTexture2D" );
+ glGenVertexArrays = (PFNGLGENVERTEXARRAYSPROC)wglGetProcAddress ("glGenVertexArrays" );
+ glGenBuffers = (PFNGLGENBUFFERSPROC)wglGetProcAddress ("glGenBuffers" );
+ glBindVertexArray = (PFNGLBINDVERTEXARRAYPROC)wglGetProcAddress ("glBindVertexArray" );
+ glBindBuffer = (PFNGLBINDBUFFERPROC)wglGetProcAddress ("glBindBuffer" );
+ glBufferData = (PFNGLBUFFERDATAPROC)wglGetProcAddress ("glBufferData" );
+ glVertexAttribPointer = (PFNGLVERTEXATTRIBPOINTERPROC)wglGetProcAddress ("glVertexAttribPointer" );
+ glEnableVertexAttribArray = (PFNGLENABLEVERTEXATTRIBARRAYPROC)wglGetProcAddress ("glEnableVertexAttribArray" );
+ glUseProgram = (PFNGLUSEPROGRAMPROC) wglGetProcAddress ("glUseProgram" );
+ glGetUniformLocation = (PFNGLGETUNIFORMLOCATIONPROC)wglGetProcAddress ("glGetUniformLocation" );
+ glUniform1i = (PFNGLUNIFORM1IPROC)wglGetProcAddress ("glUniform1i" );
+ glActiveTexture = (PFNGLACTIVETEXTUREPROC)wglGetProcAddress ("glActiveTexture" );
+ glBlendFuncSeparate = (PFNGLBLENDFUNCSEPARATEPROC)wglGetProcAddress ("glBlendFuncSeparate" );
+ glUniform1f = (PFNGLUNIFORM1FPROC)wglGetProcAddress ("glUniform1f" );
+ glUniform2fv = (PFNGLUNIFORM2FVPROC)wglGetProcAddress ("glUniform2fv" );
+ glDeleteRenderbuffers = (PFNGLDELETERENDERBUFFERSPROC)wglGetProcAddress ("glDeleteRenderbuffer" );
+ glDeleteFramebuffers = (PFNGLDELETEFRAMEBUFFERSPROC)wglGetProcAddress ("glDeleteFramebuffers" );
+ glDeleteVertexArrays = (PFNGLDELETEVERTEXARRAYSPROC)wglGetProcAddress ("glDeleteVertexArrays" );
+ glDeleteBuffers = (PFNGLDELETEBUFFERSPROC)wglGetProcAddress ("glDeleteBuffers" );
}
#endif
}
diff --git a/src/Core/Universe.h b/src/Core/Universe.h
index e38a5511..0195eee4 100644
--- a/src/Core/Universe.h
+++ b/src/Core/Universe.h
@@ -26,6 +26,7 @@ MA 02110-1301, USA.
#import "OOCocoa.h"
#import "OOOpenGL.h"
+#import "OOShaderProgram.h"
#import "legacy_random.h"
#import "OOMaths.h"
#import "OOColor.h"
@@ -337,6 +338,14 @@ enum
BOOL _witchspaceBreakPattern;
BOOL _dockingClearanceProtocolActive;
BOOL _doingStartUp;
+
+ GLuint targetTextureID;
+ NSSize targetFramebufferSize;
+ GLuint targetDepthBufferID;
+ GLuint targetFramebufferID;
+ OOShaderProgram *textureProgram;
+ GLuint quadTextureVBO, quadTextureVAO, quadTextureEBO;
+ GLint defaultDrawFBO;
}
- (id)initWithGameView:(MyOpenGLView *)gameView;
diff --git a/src/Core/Universe.m b/src/Core/Universe.m
index 34df6739..94134856 100644
--- a/src/Core/Universe.m
+++ b/src/Core/Universe.m
@@ -23,7 +23,6 @@ MA 02110-1301, USA.
*/
-#import "OOOpenGL.h"
#import "Universe.h"
#import "MyOpenGLView.h"
#import "GameController.h"
@@ -91,6 +90,8 @@ MA 02110-1301, USA.
#import "OOJSScript.h"
#import "OOJSFrameCallbacks.h"
#import "OOJSPopulatorDefinition.h"
+#import "OOOpenGL.h"
+#import "OOShaderProgram.h"
#if OO_LOCALIZATION_TOOLS
@@ -122,6 +123,20 @@ static NSString * const kOOLogEntityVerificationError = @"entity.linkedList.ver
static NSString * const kOOLogEntityVerificationRebuild = @"entity.linkedList.verify.rebuild";
+
+const GLfloat framebufferQuadVertices[] = {
+ // positions // texture coords
+ 1.0f, 1.0f, 1.0f, 1.0f, // top right
+ 1.0f, -1.0f, 1.0f, 0.0f, // bottom right
+ -1.0f, -1.0f, 0.0f, 0.0f, // bottom left
+ -1.0f, 1.0f, 0.0f, 1.0f // top left
+};
+const GLuint framebufferQuadIndices[] = {
+ 0, 1, 3, // first triangle
+ 1, 2, 3 // second triangle
+};
+
+
Universe *gSharedUniverse = nil;
extern Entity *gOOJSPlayerIfStale;
@@ -183,6 +198,11 @@ static OOComparisonResult comparePrice(id dict1, id dict2, void * context);
@interface Universe (OOPrivate)
+- (void) initTargetFramebufferWithViewSize:(NSSize)viewSize;
+- (void) deleteOpenGLObjects;
+- (void) resizeTargetFramebufferWithViewSize:(NSSize)viewSize;
+- (void) drawTargetTextureIntoDefaultFramebuffer;
+
- (BOOL) doRemoveEntity:(Entity *)entity;
- (void) setUpCargoPods;
- (void) setUpInitialUniverse;
@@ -251,6 +271,168 @@ static GLfloat docked_light_specular[4] = { DOCKED_ILLUM_LEVEL, DOCKED_ILLUM_LEV
// How dark the default ambient level of 1.0 will be
#define SKY_AMBIENT_ADJUSTMENT 0.0625
+- (void) initTargetFramebufferWithViewSize:(NSSize)viewSize
+{
+ // have to do this because on my machine the default framebuffer is not zero
+ glGetIntegerv(GL_DRAW_FRAMEBUFFER_BINDING, &defaultDrawFBO);
+
+ GLint previousProgramID;
+ glGetIntegerv(GL_CURRENT_PROGRAM, &previousProgramID);
+ GLint previousTextureID;
+ glGetIntegerv(GL_TEXTURE_BINDING_2D, &previousTextureID);
+ GLint previousVAO;
+ glGetIntegerv(GL_VERTEX_ARRAY_BINDING, &previousVAO);
+ GLint previousArrayBuffer;
+ glGetIntegerv(GL_ARRAY_BUFFER_BINDING, &previousArrayBuffer);
+ GLint previousElementBuffer;
+ glGetIntegerv(GL_ELEMENT_ARRAY_BUFFER_BINDING, &previousElementBuffer);
+
+ // creating texture that should be rendered into
+ glGenTextures(1, &targetTextureID);
+ glBindTexture(GL_TEXTURE_2D, targetTextureID);
+ glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA16F, viewSize.width, viewSize.height, 0, GL_RGBA, GL_FLOAT, NULL);
+
+ glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
+ glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
+ glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT);
+ glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT);
+
+ // create necessary depth render buffer
+ glGenRenderbuffers(1, &targetDepthBufferID);
+ glBindRenderbuffer(GL_RENDERBUFFER, targetDepthBufferID);
+ glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH_COMPONENT32F, viewSize.width, viewSize.height);
+
+ // create framebuffer and attach texture and depth buffer to framebuffer
+ glGenFramebuffers(1, &targetFramebufferID);
+ glBindFramebuffer(GL_FRAMEBUFFER, targetFramebufferID);
+ glDrawBuffer(GL_COLOR_ATTACHMENT0);
+ glReadBuffer(GL_COLOR_ATTACHMENT0);
+ glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, targetDepthBufferID);
+ glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, targetTextureID, 0);
+ targetFramebufferSize = viewSize;
+
+
+ /* TODO: in OOEnvironmentCubeMap.m call these bind functions not with 0 but with "previousXxxID"s:
+ - OOGL(glBindTexture(GL_TEXTURE_CUBE_MAP, 0));
+ - OOGL(glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, 0));
+ - OOGL(glBindRenderbufferEXT(GL_RENDERBUFFER_EXT, 0));
+ */
+
+ // TODO do OpenGL calls like everywhere else (OOGL(...)) and with EXT and ARB and stuff
+
+ // TODO: does passing [NSDictionary dictionary] without releasing it here cause a memory leak?:
+ // shader for drawing a textured quad
+ textureProgram = [[OOShaderProgram shaderProgramWithVertexShaderName:@"oolite-texture.vertex"
+ fragmentShaderName:@"oolite-texture.fragment"
+ prefix:@"#version 330\n"// TODO use correct version
+ attributeBindings:[NSDictionary dictionary]] retain];
+
+ glGenVertexArrays(1, &quadTextureVAO);
+ glGenBuffers(1, &quadTextureVBO);
+ glGenBuffers(1, &quadTextureEBO);
+
+ glBindVertexArray(quadTextureVAO);
+
+ glBindBuffer(GL_ARRAY_BUFFER, quadTextureVBO);
+ glBufferData(GL_ARRAY_BUFFER, sizeof(framebufferQuadVertices), framebufferQuadVertices, GL_STATIC_DRAW);
+
+ glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, quadTextureEBO);
+ glBufferData(GL_ELEMENT_ARRAY_BUFFER, sizeof(framebufferQuadIndices), framebufferQuadIndices, GL_STATIC_DRAW);
+
+ // position attribute
+ glVertexAttribPointer(0, 2, GL_FLOAT, GL_FALSE, 4 * sizeof(float), (void*)0);
+ glEnableVertexAttribArray(0);
+ // texture coord attribute
+ glVertexAttribPointer(1, 2, GL_FLOAT, GL_FALSE, 4 * sizeof(float), (void*)(2 * sizeof(float)));
+ glEnableVertexAttribArray(1);
+
+
+ // restoring previous bindings
+ glUseProgram(previousProgramID);
+ glBindFramebuffer(GL_FRAMEBUFFER, defaultDrawFBO);
+ glBindTexture(GL_TEXTURE_2D, previousTextureID);
+ glBindVertexArray(previousVAO);
+ glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, previousElementBuffer);
+ glBindBuffer(GL_ARRAY_BUFFER, previousArrayBuffer);
+
+}
+
+
+- (void) deleteOpenGLObjects
+{
+ glDeleteTextures(1, &targetTextureID);
+ glDeleteRenderbuffers(1, &targetDepthBufferID);
+ glDeleteFramebuffers(1, &targetFramebufferID);
+ glDeleteVertexArrays(1, &quadTextureVAO);
+ glDeleteBuffers(1, &quadTextureVBO);
+ glDeleteBuffers(1, &quadTextureEBO);
+ [textureProgram release];
+}
+
+
+- (void) resizeTargetFramebufferWithViewSize:(NSSize)viewSize
+{
+ // resize color attachment
+ glBindTexture(GL_TEXTURE_2D, targetTextureID);
+ glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA16F, viewSize.width, viewSize.height, 0, GL_RGBA, GL_FLOAT, NULL);
+ glBindTexture(GL_TEXTURE_2D, 0);
+
+ // resize depth attachment
+ glBindRenderbuffer(GL_RENDERBUFFER, targetDepthBufferID);
+ glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH_COMPONENT32F, viewSize.width, viewSize.height);
+ glBindRenderbuffer(GL_RENDERBUFFER, 0);
+
+ targetFramebufferSize.width = viewSize.width;
+ targetFramebufferSize.height = viewSize.height;
+}
+
+
+- (void) drawTargetTextureIntoDefaultFramebuffer
+{
+ GLhandleARB program = [textureProgram program];
+ NSSize viewSize = [gameView viewSize];
+ float fboResolution[2] = {viewSize.width, viewSize.height};
+
+
+ GLint previousFBO;
+ glGetIntegerv(GL_DRAW_FRAMEBUFFER_BINDING, &previousFBO);
+ GLint previousProgramID;
+ glGetIntegerv(GL_CURRENT_PROGRAM, &previousProgramID);
+ GLint previousTextureID;
+ glGetIntegerv(GL_TEXTURE_BINDING_2D, &previousTextureID);
+ GLint previousVAO;
+ glGetIntegerv(GL_VERTEX_ARRAY_BINDING, &previousVAO);
+ GLint previousActiveTexture;
+ glGetIntegerv(GL_ACTIVE_TEXTURE, &previousActiveTexture);
+
+
+ glBindFramebuffer(GL_FRAMEBUFFER, defaultDrawFBO);
+
+ // fixes transparency issue for some reason
+ glDisable(GL_BLEND);
+
+ glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
+
+ glUseProgram([textureProgram program]);
+
+ glUniform1i(glGetUniformLocation(program, "image"), 0);
+ glUniform1f(glGetUniformLocation(program, "uTime"), [self getTime]);
+ glUniform2fv(glGetUniformLocation(program, "uResolution"), 1, fboResolution);
+
+ glActiveTexture(GL_TEXTURE0);
+ glBindTexture(GL_TEXTURE_2D, targetTextureID);
+
+ glBindVertexArray(quadTextureVAO);
+ glDrawElements(GL_TRIANGLES, 6, GL_UNSIGNED_INT, 0);
+
+ // restore previous bindings
+ glBindFramebuffer(GL_FRAMEBUFFER, previousFBO);
+ glEnable(GL_BLEND);
+ glUseProgram(previousProgramID);
+ glActiveTexture(previousActiveTexture);
+ glBindTexture(GL_TEXTURE_2D, previousTextureID);
+ glBindVertexArray(previousVAO);
+}
- (id) initWithGameView:(MyOpenGLView *)inGameView
{
@@ -266,6 +448,7 @@ static GLfloat docked_light_specular[4] = { DOCKED_ILLUM_LEVEL, DOCKED_ILLUM_LEV
if (self == nil) return nil;
_doingStartUp = YES;
+
OOInitReallyRandom([NSDate timeIntervalSinceReferenceDate] * 1e9);
NSUserDefaults *prefs = [NSUserDefaults standardUserDefaults];
@@ -285,6 +468,7 @@ static GLfloat docked_light_specular[4] = { DOCKED_ILLUM_LEVEL, DOCKED_ILLUM_LEV
// init OpenGL extension manager (must be done before any other threads might use it)
[OOOpenGLExtensionManager sharedManager];
+ [self initTargetFramebufferWithViewSize:[gameView viewSize]];
[self setDetailLevelDirectly:[prefs oo_intForKey:@"detailLevel"
defaultValue:[[OOOpenGLExtensionManager sharedManager] defaultDetailLevel]]];
@@ -469,6 +653,8 @@ static GLfloat docked_light_specular[4] = { DOCKED_ILLUM_LEVEL, DOCKED_ILLUM_LEV
#endif
#endif
[conditionScripts release];
+
+ [self deleteOpenGLObjects];
[super dealloc];
}
@@ -4351,6 +4537,13 @@ static const OOMatrix starboard_matrix =
- (void) drawUniverse
{
OOLog(@"universe.profile.draw", @"%@", @"Begin draw");
+
+ if ((int)targetFramebufferSize.width != (int)[gameView viewSize].width || (int)targetFramebufferSize.height != (int)[gameView viewSize].height)
+ {
+ [self resizeTargetFramebufferWithViewSize:[gameView viewSize]];
+ }
+
+ glBindFramebuffer(GL_FRAMEBUFFER, targetFramebufferID);
if (!no_update)
{
@try
@@ -4756,7 +4949,13 @@ static const OOMatrix starboard_matrix =
}
}
}
+ glBindFramebuffer(GL_FRAMEBUFFER, defaultDrawFBO);
OOLog(@"universe.profile.draw", @"%@", @"End drawing");
+
+
+ OOLog(@"universe.profile.secondPassDraw", @"%@", @"Begin second pass draw");
+ [self drawTargetTextureIntoDefaultFramebuffer];
+ OOLog(@"universe.profile.secondPassDraw", @"%@", @"End second pass drawing");
}
@@ -9894,9 +10093,9 @@ static OOComparisonResult comparePrice(id dict1, id dict2, void *context)
//[ResourceManager loadScripts]; // initialised inside [player setUp]!
// NOTE: Anything in the sharedCache is now trashed and must be
- // reloaded. Ideally anything using the sharedCache should
- // be aware of cache flushes so it can automatically
- // reinitialize itself - mwerle 20081107.
+ // reloaded. Ideally anything using the sharedCache should
+ // be aware of cache flushes so it can automatically
+ // reinitialize itself - mwerle 20081107.
[OOShipRegistry reload];
[[self gameController] setGamePaused:NO];
[[self gameController] setMouseInteractionModeForUIWithMouseInteraction:NO];
Re: Render to Framebuffer
Posted: Sun Aug 07, 2022 3:32 pm
Your patch works so far. What I am now thinking is if the render pass should be applied to everything or just to the 3D graphics. For some things it would be nice to use also the GUI/HUD (e.g. night CRT effect) but I think color correcting and stuff like that makes the GUI look a bit too bright.
I already tried if it also works if we only draw everything before
The solution (besides fixing these bugs if possible) would probably be to write into the framebuffer once only for the 3D things and then again with all the GUI stuff. What I measured on my i5-825U with Intel HD620 graphics is that a single texture render pass on a quad with just simple grayscale adds very likely less than 1ms (more like 0.3ms) to each frame.
I already tried if it also works if we only draw everything before
/* Reset for HUD drawing */ into the framebuffer, and it seems to work (e.g. only the 3D world gets grey but not the HUD) except for the spinning models on the mainscreen/F7 planet/ship library/XenonUI which are not shown at all.The solution (besides fixing these bugs if possible) would probably be to write into the framebuffer once only for the 3D things and then again with all the GUI stuff. What I measured on my i5-825U with Intel HD620 graphics is that a single texture render pass on a quad with just simple grayscale adds very likely less than 1ms (more like 0.3ms) to each frame.
Re: Render to Framebuffer
Posted: Sun Aug 07, 2022 7:58 pm
Is there anything else likely to need updating? The only things that immediately come to mind are some of Svengali's experiments/proofs of concept (Animator Demo? StarMap Demo?) and possibly some of the more involved missions.
Re: Render to Framebuffer
Posted: Mon Aug 08, 2022 9:24 am
OK, now I am just fooling around with it, but this is awesome! It's crazy what this post-processing stuff can generate. Here is Oolite in old style silent film format:
