My Siggraph Schedule

Here is my schedule:

• Sunday: Avatar in Depth,
• Monday: All about Avatar, Detailed Surfaces, Volume and Precipitation, Real-time Live Demo,
• Tuesday: Simulation in Production, Computer Animation Festival, Blowing $h!t up, Real-time Live Demo,
• Wednesday: Advances in Real-Time Rendering in 3D Graphics and Games I and II,
• Thursday: Beyond Programmable Shading I and II.

There is a lot of uncertainty for some sessions because many are overlapping with others I am interested in (hair rendering, stylized rendering, global illumination, physically based shading, etc). It is really hard to make my final choice.

Moreover, two friends are presenting this year:

• Nicolas Stoiber, PhD student at Orange Labs like me: The Mimic Game: Real-Time Recognition and Imitation of Emotional Facial Expressions. (He will write an article here about his very interesting work after SIGGRAPH),
  

• Guillaume François, shader writer at Weta Digital : Importance Sampling for Production Rendering.

I wish them good luck with their presentation. I hope one day I could contribute to SIGGRAPH to. Indeed, my PhD was not really suited for such graphic/animation research... Maybe during my future job!!

I will post here about SIGGRAPH during the next week if I can find time between sessions and SIGGRAPH night parties. :)

OpenCL

Before I even did something with CUDA, I have decided to move to OpenCL. Just because I prefer the GLSL-like approach to load program instead of using an additional dedicated compiler. I am close to finish my C++ library with nice classes encapsulating OpenCL objects (And I am using the C interface). It looks really like what I did for my Shader manager (like in the demos on my personal website). I will post it here soon in case someone want to get it and modify it. Also, choosing OpenCL is for the ability to use any platforms, not only nVidia. The current problem is that I don't have a single clue about how ATI parallel architecture is (their Stream Computing technology).

In addition to a review and my PhD thesis I have to finish writing, I am also preparing my travel to Los Angeles for SIGGRAPH 2010. I will post my schedule tomorrow in a second post.

Bonus:
- a nice real-time CUDA ray-tracer called brigade.
- a new A-Buffer method using per-pixel linked list to manage dynamic allocation of pages containing depth layers. In OpenGL! :)

Volumetric lines 2 + News

Hello!

Sorry for the long time silence but I have been very busy recently: PhD writing, not accepted paper I have to modify, internship students to take care, researchers who want to use my engine, etc... Now it's better, I will have more free time by the end of the week.

So what are the news! Volume Lines 2 demo is on-line. In fact, I have submitted it to GpuPro2 but it was not accepted: I expected this answer because indeed the method is pretty simple. The results are nicer than previous volumetric lines but it requires geometry shaders and it is more computationally expensive.

Also, I started to work in Virtual Textures few weeks ago. I have the low resolution feedback buffer working with required pages to load, indirection table, etc. Just need to load required pages from the disk to GPU virtual memory in a separate thread together with a LRU cache. I will work again on it soon.

The development of my light-pre pass renderer with relief mapping is... well... deferred. :/ I have a lot of optimization ideas and I hope I can show you more about it soon.

Finally, I will go to Siggraph 2010 this year as an attendee! :) Many courses and talks to attend! It will be hard to attend every sessions I would like to but, I will do my best. I may talk about this conference, show some pictures and report about courses/talks on this blog. Maybe I will be able to meet some of you overthere! :D

Knee deep in the dead

Some pointers on fast but pretty good reviews of old IdTech and their adaptions to iPhone!

Wolfenstein3D iPhone
Doom PC
Doom iPhone
Quake PC

Ah those good old days... I was so young when I played Doom the first time on a friend's computer. By the way, everyone should read Micheal Abrash's Black Book full of anecdotes and in-depth details about Quake renderer algorithms. For instance, you would learn that John Carmack bet that he could implement dynamic lighting in the quake engine in less than an hour. Indeed, he failed but of only few minutes (3 to 7, I can't remember). But hell yeah! In an hour! :)

Automatic CUDA optimizer

A promising automatic CUDA optimizer has been proposed by Huiyang Zhou et al. as you can read on his website: http://www.eecs.ucf.edu/~zhou/ (with downloadable paper). Also, they plan to extend their compiler to OpenCL and I am sure this could also be done for DX11 ComputeShader.

The performance are interesting since they are able to achieved as good to better performance as compared to manually optimized programs (something which takes time). Bonus: the code of this open source compiler will be available soon.

Update: the code is available here. (source)

My Relief-Mapped Light Pre-Pass Renderer

Hi!

Just to show you some progress I made concerning my new renderer: a light pre-pass renderer with relief-mapping over all surfaces! An important feature of this engine is that each texel in the virtual scene is unique. Indeed, no virtual texture mapping is used but simply large 4096*4096 textures! Because no virtual texturing is used, everything is resident in the GPU video memory. Texture for dynamically added objects in the VE are allocated in dynamic texture atlases (using a quad-tree to manage texture space use). As a result, when designing a map (under Maya with my exporter), you have to find a good trade-off between texel density in world space and the size of your map. For me, it is important to have unique texel everywhere to achieve this kind of effects I made before.

Yesterday, as visible on screen-shots, I have finished implementing Virtual Shadow Depth Cube map as described in ShaderX3. I may test later render to cube map using geometry shader.

Thank to this light pre-pass engine, I will be able to easily implement/test several rendering methods like soft particles, light propagation volume, SSAO, etc... The only thing I miss in this engine is spherical-harmonic- or Source-like lightmaps (I wish I had a Beast or Turtle license for this). Currently, it is a Quake3-like lightmap: no directional information about incoming light on surfaces.

With this engine I plan to develop a simple Quake3-like-death-match-with-bots game as a simple demo. If some artist read this post and are interested in designing a death match map, please send me an email. The only thing you would need is Maya2008.

Black lightmap, two shadowed point light sources

Black lightmap, two shadowed point light sources

Grey lightmap: directional indirect lighting using
spherical harmonic volume.
For this, I use the library I have developed.

In these screenshots, I only used a uniformly colored lightmap because I did not found time to generate one for this level. And also because I just finished adding point light sources support which will be used for direct illumination (lightmap will only contains emissive surface and global illumination). Next move is adding spot lights and optimizing shaders as well as the way meshes are processed by the engine.

Feel free to ask me some questions! :)

Spherical Harmonics Lighting

Hello everyone!

It's time for another post on another rendering method! :) Currently, I am working on a light pre-pass renderer. I have just finished to include SH lighting inside.

There exist several methods to compute the lighting solution of a virtual environment. Some methods are fully dynamic (IdTech4, CryEngine) or fully/partially static (idTech3, UnrealEngine). For fully dynamic methods, since the lighting solution is often unified, there is no problem to compute the lighting of dynamic objects. However, for static methods, there can be several problems.

Let's take the case of an environment having its lighting solution stored in a static lightmap. When you add dynamic objects in the virtual environment, you have to compute their lighting solution. But how can we compute the light that reach each dynamic object using only their position/orientation and the surrounding environment?

Common methods are using probe or light volume. For instance, in Quake3, a light volume is define for the entire environment and each Voxel stores an ambient color plus a directional colored source (Quake3 map Specs). Another solution is to used probes positioned by artists in the virtual environment (Source engine). At each of this location, irradiance can be computed and stored in several formats (SH, directional light, Source basis, etc).

For my light pre-pass renderer, I decided to use a light volume having each Voxel containing a 2 bands spherical harmonics as visible on the next screen-shots.

A quake3 map with it's corresponding SH volume.

My test map with its corresponding SH volume.

For each dynamic object, the SH volume is sampled on the CPU using tri-linear interpolation (using object's position). The final SH contribution is added during the final pass of the light pre-pass pipeline according to the normals stored in the g-buffer (And in my renderer, each surface is rendered using relief mapping).

A dynamic object lit only by dynamic lights (left) and
using the SH volume (right).
Note that the lighting information stored in the lightmap now
affects the object final look.

The SH volume is computed as a pre-process. For each Voxel, I render the virtual environment in a cube map as in my previous demo. Finally, the surrounding colored environment stored in this cube map is transformed into the SH basis and stored in its corresponding SH volume Voxel.

I plan to store only global illumination in the lightmap of my environments so the SH volume will only contains global illumination contribution. Then, direct lighting will be computed on static/dynamic objects using standard unified light rendering and shadowing. I also plan to add Crytek's light propagation volume later...

I will continue to work on this engine in parallel to the writing of my PhD thesis, other little demos I have in my pipeline and future job research.

New volumetric lines method

Hi readers,

I have been working on many things recently. One of these things is a new volumetric line algorithm!
My previous method (basically the same as the one proposed in ShaderX7) was really fast and yields good looking result. I think that's why it was successfully used in the iPhone game PewPew. The only thing about this method is that you should avoid looking lines along their direction because, in this case, the trick I use become visible. Also, it was not possible to shade the line based on its thickness from current viewpoint.
Another method has been proposed by Tristan Lorach to change volumetric lines appearance based on the angle between view and line directions. However, line appearance was represented by only 16 texture tiles and interpolation between them was visible.

The new method I propose is able to render capsule like volumetric lines with any width and for any point of view, i.e. you can look inside/through the line. It also allows the use of thickness based visual effects.

Here is the overall algorithm of my new method:

1. Extrude an OOBB around the volumetric line having the same width as the line. This is done using a geometry shader computing triangles strips from a single line.
2. Compute closest and farthest intersections between view ray and capsule using geometric methods and/or quadratic functions resolution. This is done in the OOBB frame reference.
   
3. Compute thickness based on capsule intersections and environment depth map.
   
4. Shade the volumetric line based on it's thickness.
   

It is pretty simple but efficient. I will not go into the details right now: there will be a post on my website soon. I will just show some early screenshots from the current version:

The capsules representing the volumetric lines
filled with a white color

line radius can be changed


Thickness based shading and intersection with the environment

View from above and under the ground

The thickness is correct,
even if the camera is inside the volumetric lines

Learning CUDA

Hi there!

I did not post here for a while because I am very busy right now! I am working on my last paper for my PhD thesis and the deadline is by the end of march... I had do conduct an experiment, process some data and now I am parsing the ANOVA results and this in parallel to the writing of the paper. After this last deadline, I will begin to write my PhD manuscript in English! :) So exciting!

During the few free time I found (almost at night), I test Battlefield Bad Company 2 with friends. Also, I implement spherical harmonic lighting for the purpose of ambient lighting of dynamically moving objects in the current "3d engine" I develop (a light pre-pass renderer with relief-mapping on all surfaces and other cool features).

And, as written in the title, I am currently learning CUDA! Indeed, I have bought the new book by David Kirk and Wen-mei W. Hwu called Programming Massively Parallel Processors: A Hands-on Approach. I found this book well written and it nicely introduces CUDA. Good job authors! I only had a large overview of the CUDA architecture but now it is clear in details.
The first thing I did is to take an example in the nVidia CUDA browser and make it independent of their cutil library and others. To build and run my example, You should just need to install the CUDA Toolkit. You can download this simple program HERE. It features a simple grid of points which height is computed using CUDA and stored in a VBO. The result is finally displayed using OpenGL. Only Debug and Release configurations work (32 and 64bits). Emulation profiles will not compile right now. Also, the CUDA code is nicely separated from the c++ code, something I did not find in every web examples. The other libraries used are glut, glu and glew (everything is in the zip file).
So when I will have finished to read the book (and implemented their examples), I may start writing a little demo... :D

See you soon!

Trailer of Amnesia:The Dark Descent

Trailer is HERE.

This link features the teaser of the upcoming game from the talented developers of the Penumbra series: Amnesia: The Dark Descent. I have found this teaser really impressive. This is a good example of a nice sounds, visuals and interactions integration!

You can also pre-order the game. And If they reach 2000 pre-orders before the 31st of May, the game will get extra content! I did not played the Penumbra series excepted the demo but I want to try this one so I have pre-ordered the game.

Do not hesitate to spread the word!

Fourier Opacity Mapping rocks!

The up-coming I3D conference on interactive 3D Graphics and Games will feature several interesting papers! For instance, this year will features papers such as " Cascaded Light Propagation Volumes for Real-Time Indirect Illumination" by Anton Kaplanyan (Crytek) or "Interactive Volume Caustics in Single-Scattering Media" by Wei Hu. Another paper I found very interesting is the one proposed by Jon Jansen and Louis Bavoil (Yeah a French guy), both at nVidia, called "Fourier Opacity Mapping". I could not resist implementing this nice paper.

Fourier Opacity Mapping (FOM) is about approximating light attenuation through a volume made of particles. Let's consider a spot light: the authors proposed to reconstruct the light transmittance function along each ray using a Fourier series. The coefficients of the Fourier series are stored in the fourier opacity map in light view space. This map is generated using usual particles rendering with transformation in the Fourier basis of extinction coefficients in the fragment program. Then, when rendering the particles from view, the corrected light attenuation for each particle can be recovered. I will not go into the details here. You can read the paper here and my report on my personal webpage. Indeed, you can see a video of my implementation of FOM here.

Here are some screenshots of my implementation (Top: without FOM, Bottom with FOM):

A simple particle chain. Notice the correct order of light attenuation.

Colored light attenuation through a particle block.

A grey smoke volume with some red particles. Notice that the red particles attenuate the light correctly: they only affect the color of particles that receive the light after them.

I know that my screenshots are not really eye candy but they show that this method is really efficient to simulate colored light extinction when passing through a volume of particle. Furthermore, if you want an exemple of good use by skilled artist, just have a look at the game Batman Arkham Asylum which implements this method.

~~UPDATE~~
My report is finally available on my website together with demo and open source code. :)
You need a recent video card to run the demo since it uses GLSL 1.5 and render data into 6 buffers in one pass.

The engine I have developed for my PhD thesis

I am going to present you the engine I have develop for all my experiments during my PhD thesis in virtual reality. My subject is : "Using Gaze Tracking Systems and Visual Attention Models to Improve Interaction and Rendering of Virtual Environments". You can find and read more about my work on this webpage. As a consequence, I work a lot with gaze tracking systems and visual attention models but I am not going to extend more on this here.

My PhD is about virtual reality so I needed an engine to render my Virtual Environments (VE) and conduct my experiments. I like to do things on the GPU so almost everything is computed on GPU. I wanted the engine to display point and spot lights that could be static or dynamic. Also, I wanted the model inside the environment to be simulated physically. Every experiment I were going to be very different so I needed the engine to be easily script-able. Finally, I needed to replay all experiment sessions so I wanted to be able to replay recorded navigations and interactions to apply coslty algorithms in order to study users' gaze behavior. Finally, I wanted to be able to create my own VE very easily.

I have developed this engine during 2 months in summer 2008 from scratch. If you would look at the code, you would see really a lot of classes with nice client interface accessible from the engine interface. But maybe you'll be sad to not see cache friendly structures such as structure of array. I have to admit, I did not take the time to optimize...

I did not show this engine before because the virtual environment were used for double blinded publications in conference and journal.

The virtual environment editor : Maya
At this time, I was learning Maya by myself: this is a pretty nice 3D modeling software simple to use for basic 3D. I decided to use it to create my VE. So I have developed my own exporter. It can export the whole geometry, point light, spot light as well as phong materials. Also, because I wanted to be able to use a lightmap, the world lightmap is rendered by Mental Ray in HDR format. This lightmap only contains global illumination! Thus, each light is rendered as usual (diffuse+specular) and, finally, global illumination is added to the scene. The drawback of this is that static lights cannot be turn off.

Phong Materials
Here are the components of the Phong material I selected from the long list of Maya:

• Diffuse color (can be a texture or a single color)
  
• Specular color (can be a texture or a single color)
• Specular exponent
• Local normal (can be a texture or will be flat)

Lights
I wanted each light to be rendered the same way as when using Mental Ray. Thus I decided to use the same parameters as in Maya. Then I simply wrote a shader corresponding to point and spot light.
Here are the spot light parameters:

• Transformation matrix
  
• Color & intensity
• Decay rate (in fact, I force this to be quadratic in maya and in the engine)
  
• Penumbra angle & cone angle
• Drop off
  

Here are the point light parameters:

• Position
  
• Color & intensity
• Decay rate (in fact, I force this to be quadratic in Maya and in the engine)

Indeed, a quadratic extinction of light intensity (close to what happens physically) means that each light can interact with ALL object in the VE. To avoid this problem, I compute the distance from the light center where a surface will receive only 5% of light. Then, I attenuate linearly the intensity from the center of the light to this distance. This methods allows me to compute an Axis-Aligned Bounding Box (AABB) for each lights to accelerate the rendering process by using frustum culling.

The rendering engine
As an addicted to OpenGL, I decided to use OpenGL! :) I also decided that the engine will be based on the zPrePass architecture (like the Doom3 engine). So the final rendering is obtained by summing the contribution of each light by rendering each meshes they interact with (after a zPre pass). Static lights come from the exported VE and, then, dynamic lights can be added from the script. The engine behave similarly concerning static and dynamic meshes.

For shadows, I use a simple depth map for spot lights and a virtual depth cube map for point lights. I only use native hardware shadow map filtering. Not very eye candy but it is enough for the experiment I needed to conduct.

Physics
I use the PhysX API from nVidia. Only rigid body interactions are simulated in my case. Each dynamic objects added to the simulation can be added to the PhysX engine if needed according to one of 4 shape type: bounding box, bounding sphere, convex or general triangle mesh.

Scripts
For the script, I chose Lua together with LuaBind (to simplify functions and classes registration). From the script, you can add and manipulate light and model. The script can also receive mouse and keyboard signals and each frame, a specific update function is called. For instance, here is a small piece of Lua code I use to position a spot light to look like a flashlight the user is holding up:
if(flashLightOn==true)
then
local viewCam = getViewPointOrientPos();
local target = Vector3(0.0,0.0,-5.0);
viewCam:transformBackFromLocalSpace(target);
local origin = Vector3(-0.2,-0.1,0.0);
viewCam:transformBackFromLocalSpace(origin);
flashLight:setIntensity(10.0);
flashLight:setPosTargetUp(origin,target,Vector3(0.0,1.0,0.0));
else
flashLight:setIntensity(0.0);
end
As an example, I have also developed some Lua classes that control some robot which follow a predefined path made of way points. I have to say that Lua is a very powerful and easy to use script language.

Record/Replay feature
My engine is able to record and replay navigation sessions in the VE. For each frame, I do not record everything. For instance, I only record the event when an dynamic object's position has been changed: everything is recorded as an event. I do not interpolate each frame because, for my experiments, I only need each frame that were displayed to the viewer.

Gaze-Tracking
My PhD is about gaze tracking so my engine takes into account gaze tracking. Currently, I have a class communicating with the TobiiX50 soft/hardware. This class is handled by a GazeTrackerManager which allows to easily add support for another type of gaze tracking system.

Some Results
The q3dm1 map in Maya during editing:
The q3dm1 map rendered in my engine with HDR lightmap generated by MentalRay (no lights here, everything is baked in the lightmap). The screenshot is a bit over-exposed due to the automatic luminosity adaptation (a lot of screen area are black).
The house VE used in one of my experiment during editing in Maya:
The house VE rendered in my engine with HDR GI lightmap generated by MentalRay:
A game where you have to destroy ships with your gaze:

A video is available on my youtube space.

I am currently changing the zPrePass renderer for a Light-Pre Pass rendering process. I have to say that it is very interesting! I already have point lights working and performance are impressive! I will talk about this later but I have to try to publish something with it before. :)

Do not hesitate if you have questions or if you want some piece of code. I plan to release the source of this engine at the end of my PhD.

What do you think about OnLive?

Do you know OnLive? It's a company that is going to provide a technology that will (?) allow people to play games on a computer or a tv (Hmm ok). The tricky part is the fact that you would not need a high-end PC (GPU, CPU, etc) to play, for instance, Crysis. Basically, their software will send your actions on input devices to their servers which will compute the image frames for you. Then, all these frames would be sent back to your computer as a video stream.

The concept is very interesting : you just need to rent a server and you don't need to upgrade your computer. Also, I am sure that they will provide a game catalog like Steam does.

However, to my opinion, this system will not work because:

1. According to OnLive, you will have to be less than 1000 miles away from a server to play. As a result, they will need a lot of servers to provide *good* pings to costumers. It seems that they consider a ping of 80ms as the limit. So, quality of service will depend a lot on your distance to a server.
2. I am sure they have new efficient technologies to compress frames into a video stream but is their method lossless? What final resolution can you get? Full HD? (again, this will surely depends on your distance to a server)
3. As an avid player of FPS, I think that a target ping of 80ms is not enough. Even for a RTS with a lot of micro-actions, fighting game or a car race game you will feel it. And I am not speaking about lags. If you don't play those kind of CPU/GPU time consuming games, may be you don't need a high-end hardware...
   

I really hope that I am wrong about this technology... Anyway, I wish the OnLive team "Good Luck"!!! :)

Efficient OOP + Publications repositories

A very interesting presentation has been en-lighted by Atom: Pitfalls of Object Oriented Programming by Tony Albrecht from SCEE. I was aware of the problems exposed in this presentation, however, to my opinion, this is the best presentation so far that explains and details the cach miss optimization problem on a simple example. I will not extend more on the presentation: it is clear enough. If you wonder what is a Structure-Of-Array (in contrary to Array-Of-Structure) and why some people says OOP can be considered as evil (the presentation expose one reason) you should read it now! :)

This presentation can be found on SCEE presentation repository. Here are two more repositories I found very interesting: Valve and Bungie. If you know other interesting publication repositories, please post them.

Bonus: Siggraph 2009 course Advances in Real-Time Rendering in 3D Graphics and Games repository.

News from France

Hi,

Since my last post, I am back in France! I was in Japan for a Japanese-French research collaboration during one month (November). Maybe I will speak about this work once it is published in a conference or journal. Anyway, I have to say that Japan is a really impressive country! So many place to visit! (temples, tokyo tower, etc). Also food is really good and not expensive! I really recommend traveling in Japan: people are really respectful and will help you even if they can't speak English. To my opinion, each gamer, game developer which travel to Japan should visit Akihabara (called "The electric town"). A lot of games, arcade games, electric shop with very low price, etc. You can even buy an old megadrive for 0.8 euro! :)

Finally, I decided that it was a good time to change my computer. Indeed, my old graphics card caused me too much troubles: random colored pixels, triangles stretched to infinity (only with DirectX, not OpenGL) and even black death screens! I decided to buy hardware pieces that have a good performance/price ratio: Intel i5, geForceGTX275 as the main components.
Now I plan to learn DirectX with this computer. I am using OpenGL for 7 years and it is time for me to learn something else.

See you next news!

Volumetric lines

Four years ago, A method to render volumetric lines has been proposed by Tristan Lorach at nVidia (link). I have found this method very interesting but, to my opinion, texture interpolation simulating change in point of view were too visible. Three years ago, I decided to propose my own volumetric line rendering method as visible on my website. My method does not allow as much visual effect as the one of nVidia but at least you can change the appearance of lines using a simple 2D square texture. Each line is rendered using 6 triangles and all computations are done in the vertex program. The drawback with my method is that when lines' width is very large and their direction is parrallel to the view direction, the trick I use is visible. However, for line which are not large, this problem is rarely noticeable.

Screen shot of the volumetric lines.
Because the line on the bottom is large, the trick I use is visible.In this screenshot, another appearance texture is applied.

Recently, an article published in ShaderX7 proposed a method very similar, if not same, to the one I propose. I should have written an article before... At that time, I was beginning my Master study and I cannot imagine this would be possible.

Many people write to me in order to discuss this method, and I am really happy to see people using it. I am pleased to announce that my method is use in an iPhone game developed by a French guy as you can read on one if his post and blog. :)