Sebh's blog

Electricity flow material

2011-07-24T18:38:00.004+02:00

I have added a new article on my website about a simple material featuring electricity flowing down a printed circuit. Everything is here with the traditional executable and source code!

Basically, instead of this:

You get that:

Feel free to improve it and to discuss about it here! :)

PhD 3D Engine - Conclusion

2011-06-26T13:50:00.005+02:00

I have finally written an article on my website about the engine I have developed for My PhD thesis. This engine made my life easier as a PhD student.

I have learned a lot during its development:

first shadow mapping, virtual indirection cubemap
first scripting integration
first record/replay

I was then able to quickly:

design my experiments using scripting
build environments under Maya+MentalRay
record user's actions in the VE
extract meaningful data for analysis

And finally, I was able to put nice screenshots in my papers and became a doctor! :)

OpenCL Slisesix

2011-06-24T21:19:00.001+02:00

The SliseSix OpenCL demo is finally out! :)
Feel free to discuss it here.

OpenCL demo of Iniguo Quilez slisesix - SOON

2011-06-10T10:00:00.007+02:00

I will soon release an OpenCL implementation of Iniguo quilez's slisesix demo . In this demo, you will be able to "interactively" move the camera in the scene.

I am also currently working on a high-quality/resolution version. It will generate a single screen-shot with depth-of-field and more visual effects impossible to do for a real-time exploration of the scene on my poor nVidia GeforceGTX275.

The demo will also feature a preview version of nVidia Timothy Lotte's SSAA algo called FXAA3 (preview version). He sent me a preview version for testing purposes and was kind enough to accept me including a preview version in the upcoming download-able demo. By the way, there will be a Siggraph course this year on SSAA.

Here are two screenshots:

The demo from shadertoy with fxaa3,volumetric fog and camera control.

Dynamic color correction and warm glow to make the monster alive

New website online - Still no 3D stuff... :-/

2011-05-10T13:27:00.003+02:00

My new website is now up at the following address: http://sebastien.hillaire.free.fr/. I took my whole Sunday to finalize it (even though the project section is still missing). The website has been designed using Joomla: I recommend you to use it if you want a clean website.

I will now be able to finish the last article of my PhD requiring a revision. Then I will go back to 3D experiments in-between two sessions of Starcraft2. ;)

More updates coming soon

2011-04-02T15:04:00.003+02:00

I know that I have not updated my blog for a while and this is due to many factors: many recent deadlines at work (crunch time ftw), re-design of my personal website (it turns out that it takes a lot of my free time, it should be finished soon), etc. I have also spent a lot of time updating my knowledge of ASM (never used during my PhD!) and experimenting with a particular focus on low level optimizations using MMX and SSE. Applications of this at work were really successful! :)

I will refocus soon on 3D experiments and, hopefully, I will be able to show you something about my work at Dynamixyz.

Stay tuned! :D

Symbolic computations

2010-12-21T13:33:00.007+01:00

Just found a good symbolic (algebraic) computation program that can help you to develop/visualize/simplify your formulas. The graphical interface is name XCAS and uses the GIAC engine. You can download everything here.

As an example, you can "pre-compute" rotation matrix from Euler's angles for a 3 bands spherical harmonics vector and visualize it as following:

Good to start optimizing things...

Miscellaneous

Please, spread the word: GPU Pro 3: Call for Authors.
I3D 2011 papers
Funny programming quotes

Eye rendering

2010-12-19T21:59:00.003+01:00

Some more rendering R&D I am on: eye rendering.

here are the supported features: light refraction at cornea for point/spot and environment lighting, light concentration, light scattering in the iris, procedural description of the organic structure of the iris.

A brown eye

A fantasy eye

I hope you like it! ;)

Skin rendering - environment lighting

2010-12-12T18:54:00.010+01:00

In parallel to the development of the graphical pipeline of Dynamixyz, my work also involved some R&D on rendering. I may show some of my work here when my technical director allows me to do it.

Some results showing the GPU Gems 3 skin rendering method but this time no point or spot light: only environment lighting using spherical harmonics! Below, some screenshots showing a head model lit by its surrounding environment (with an without light directional light occlusion.) Of course, this is rendered in real-time and with gamma correction. These screenshots emphasize the importance of using a ambient lighting model that takes into account incoming light directions of the environment as well as directional occlusions.

Environment lighting with a cube-map having a
pure red color on a single face.
(left without, and right with directional light occlusion)

Environment lighting with a cube-map from Humus.
(left without, and right with directional light occlusion)

This is static and I am working on an idea to make everything dynamic. The occlusion is computed on the GPU using rasterization instead of ray tracing.

The head model is a free scan released by Infinite-Realities. Thanks to you Mr. Lee Perry Smith!

Books + interesting CUDA facts

2010-12-04T22:13:00.006+01:00

Hi followers!

I know I am still not updating often this blog. In majority, this is, again, due to my new work and the preparation of my PhD thesis defense. So, I am doing a lot of interesting stuff at work that I may share in here some day. Besides, concerning my PhD, my last paper has been successfully presented last week at VRST2010 by my advisor (I was not able to go myself). This paper was about simulating the Human visual system using the new visual attention model I have proposed to predict user's gaze during real-time first-person navigation in interactive 3D virtual environments. Also, my last collaboration ended with a paper accepted for publication in the IEEE TVCG journal. This paper is about real-time haptic interaction with fluids: my small contribution is a cheap fluid rendering method. I may talk about it here soon. I now only have to think about my PhD defense presentation! :)

Note on CUDA

I have read the two books about CUDA (1 and 2). They are pretty amazing book for learning the basics but I was still hungry after that. So I have decided to read more documents like the nVidia best practice recommendations. I was surprised to see that these books/documents advice a set of "simple" rules to follow in order to get good performance such as "Increasing occupancy is the only way to improve latency hiding". But, as exposed in this very good presentation, you have to be careful with these advices. It reveals another set of rules for better performance with lower occupancy. Overall, like with every computing systems, you should always try several codes and use the one which give the best performance.

Amazing books

- This book about the two Id johns is very well written! I could not stop reading it!

- Are you looking for a book to learn physically-based rendering methods? Stop searching and buy this one! An incredible amount of methods are covered in this book with a new presentation style I have just discovered: literate programming. The book covers a large range of knowledge: from the basics (ray tracing, collision, etc) to the advanced one (metropolis light transport, sub-surface scattering, spherical harmonics, etc). There are all Math equations and corresponding source code! Perfect!

Best Fit Normal Map Generator + Source Code

2010-10-26T23:14:00.028+02:00

The BFN generator is finally here! I have found time for this tonight.

So, this small application allows you to generate each face of a BFN cube-map. If you want to encode the BFN only as the length of the best fit normal or transform it to a 2D texture (as suggested in Crytek's presentation), you will have a little bit of work to do but it should not be such a big deal. You can select the resolution of the cube-map as well as the bias vector at compilation time.

Some screen-shots for 256x256x6 cube-maps:

At the origin: the BFN cube-map with a bias vector=vec3(0.5,0.5,0.5).
in the +X direction: the regular normalization cube-map.
In the +X+Y direction: a BFN cubemap with a bias vector=vec3(0.5,0.5,8.0/255.0). It results in less precision for normals having negative Z value and more precision for normals having positive Z value.

The reconstruction error of the regular normalization cube-map as compared to the ground-truth normal map. (scale factor of 50)

The reconstruction error of the BFN cube-map with bias a vector=vec3(0.5,0.5,0.5).
Some reconstruction errors are still visible. (scale factor of 50)

The reconstruction error of the BFN cube-map with bias a vector=vec3(0.5,0.5,8.0/255.0).
No more error patterns are visible. (scale factor of 50)

Reconstruction error on the back faces (scale factor of 50). There is a large error when using a bias vector of (0.5,0.5,8.0/255.0). However, it is not important since only the positive Z values are important when storing normals for a light pre-pass or deferred renderer.

Error when computing a specular lob with the regular normalization cube-map. The specular exponent is 64.

No noticeable difference when changing the bias vector of the BFN.

You can download the source here!

And finally, the important references :

Amantide ray marching paper.
Crytek's presentation from SIGGRAPH 2010 as well as their 2D BFN texture (which I recommend) can be downloaded here.

Some more links

2010-10-23T22:00:00.009+02:00

Just in case you are still not aware of the Load-Hit-Store issue.
SIGGRAPH Asia Papers and Sketches are available.
An alternative to the shell based method to render fur. [Video]

Oh, by the way, my tool to generate the Best fit normals cube map will be available shortly (together with C++ sources). Many people asked me about it... Now, I just need to write a blog post and put it on my personal website for download.

News and random links

2010-10-13T21:43:00.012+02:00

Hi everyone!

I am currently very busy because of the end of my PhD thesis and my new job! This is why I am not currently very active.

So, as I said, my PhD is over. The manuscript is currently proofread by my two advisors. I will defend my PhD on January 21st 2010.

I finally had a phone interview with the game company that contacted me 2 times last year. Despite the fact that "I have a very interesting profile", they told me that I still "have to get more experience"... So sad... :/ So this is what I am doing right now by working at Dynamixyz as a R&D engineer. It is a new French start-up so you can imagine that there is a lot of work to do!! :) I am mainly involved in the development of the CG part of their facial animation tools pipeline.

I want to finish this post with these interesting/funny random links:

Interesting paper from Insomniac on 2 band spherical harmonics with truncated triple product.
Patterncraft or how to teach design patterns using Starcraft2! Much more fun than typical school use cases! :)
The blog of a French guy working at Crytek. Sorry, it is in French. :)
Game design fundamentals blog post
16bits ALU in Minecraft here
Log out fail in MoH
Amazing dance animations in this TF2 video
What is exactly a doctorate? Answer

Eye rendering - first try

2010-08-31T19:43:00.012+02:00

I was in holidays near Deauville (France) last week, that's why I remains quiet.

I have started working on an eye rendering method inspired by the one presented in this paper. As you can see on the next screen-shots, I have some interesting first results. The characteristics of the eye is taken from biologists reports papers. The iris is rendering using the subsurface texture mapping method simulating single scattering in participating medium (in this case, the iris). The light refraction at the cornea is taken into account in a different way than what is done in the eye paper (their refraction function). The sclera (the white part) is currently not shaded.

I hope you like it!

Blue iris rendering with refraction.
(click on the picture for higher resolution)

Green and Brown eyes.

I have set-up the scattering/extinction parameters rapidly so you may not found the color of this eyes really "realistic"! I still have to improve the current algorithm and I have some interesting idea for that.

Quackecon 2010: Carmack's keynote

2010-08-13T09:12:00.007+02:00

Every developers familiar with virtual texturing would have though: "this is a real scalable technology"! Carmack just proved it with his iPhone4 Rage demo running at 60fps!

By the way, I am looking for a full video of the keynote. I, I mean google, can't find it...

Also, John now has a twitter which would work as his old .plan files! Back to good old time?

BFN: about Crytek's approach

2010-08-12T13:51:00.004+02:00

In my case, I have computed the best fist RGB value given a bias vector and, of course, the direction to fit. The BFN are stored in a cube map.

Crytek's approach is only storing the length of the best fitting vector for the requested normal direction. Furthermore, as explained in the notes (slide 94), 8-bits precision is sufficient if the best fit length is stored for the normal divided by maximum component.

As pointed out by Vince on my first post about BFN, there is some symmetry on each cube face. Indeed, the cubemap can be compressed into a single 2D texture (slide 94). This allows to save video memory at the cost of several ALU operations in the fragment shader (slide 95).

When using this final representation as a 2D texture, it is not possible to change the bias vector as I have proposed in my previous post. However, results seems to be good enough with Crytek's approach... (slides 42-43) Is it worth the cost to use a cubemap? My next step will be to compare the image quality with or without changing the bias vector in my deferred relief mapped renderer.

Best Fit Normals: playing with the bias vector

2010-08-08T18:36:00.042+02:00

Yesterday night, I have first implemented the Amantide ray marching method to compute the best fit normal (BFN) cube map. The gain in performance is, as expected, major. I can now generate a large cube map in few minutes (instead of hours).

Second, I was thinking about changing the bias vector in order to get precision where it matter, similarly to the quantization approach presented by Crytek in this old presentation slide 13. This, in order to get more precision (more direction) in the usual normal direction: along the Z axis. I found that it is hard to choose a good bias value. In the case where negative Z value are ignored (there is some games engine in this case but I can't remember which), choosing a bias of 0.0 would be just perfect. Then, I suppose one could choose the bias value visually, or image difference error from key view points. Also, encoding objects normal map using such a bias should be of higher quality because normal maps rarely contain normals with negative Z.

Enough talking, here are some screenshot:

A view of the BFN cube map face (positive Z)
with a bias factor of 16 on the Z axis.

Bias modification example. Left: BFN with usual bias,
Right: BFN with Bias of 2 on the Z axis
(poor precision for normal with negative Z).
(32*32 cube map)

Visualization of the reconstruction error (multiplied here by 70)
as compared to the true direction.
Front box: BFN with Bias of 16 on the Z axis
(poor precision for normal with negative Z),
Far box: BFN with usual bias. (256*256 cube map)

Visualization of the reconstruction error on the positive Z face.
Left: BFN with Bias of 16 on the Z axis,
Right: BFN with usual bias. (256*256 cube map)

A you can see on the last figure, you can get more precision on a simple 256 BFN cube map by simply changing the bias on the Z axis.

The presentation of Anton about this is now available on the course website. I will have a deeper look at the details (I was implementing this methods based only on my memory and ideas since SIGGRAPH). Interestingly, it seems that they do not use a cubemap but a 2D texture...

As always, feel free to discuss and to ask me for the BFN cubemap textures (at any resolution now :D) if you want to try this in your engine (some developers from video game studios requested these textures last time).

Crytek's Best Fit Normals

2010-08-04T14:00:00.018+02:00

Among the SIGGRAPH presentations, there was one about Crytek's rendering methods. One interesting techniques quickly presented was best fit normals (BFN). This methods is aimed to improve normal precision when stored in the RGB8 format.

When using traditional scaled&biased normals in RGB8 format, some accuracy errors can occur because of the low precision of the RGB8 format related to the scale&bias. For example, considering a 256*256*6 cube map, 393216 directions can be represented. However, due to the low precision of RGB8 format, only 219890 (55.9%) of these directions are effectively represented (many similar directions being represented by the same compressed value). In this case, I have computed that only 1.31% of the full 256*256*256 voxel possibilities of the RGB8 values are used. Also, I have computed that each voxel effectively used represents meanly 1.788 directions.

The idea behind the BFN approach is to search for the voxel that will best represent (fit) a given direction: it may be a non-normalized vector. Using this method with a 256*256*6 cube map, I have found that 387107 (98.4%) of directions are effectively represented. Furthermore, in this case, each voxel used represents meanly 1.016 directions. Thus, using such a method results in a more accurate reconstruction of normals (see screen-shots). Moreover, compression is a single cubemap lookup, and reconstruction is an unbias and normalize.

Left: BFN cubemap, Right: scale&bias cubemap

Reconstruction error (absolute value scaled by 70) for BFN (left) and
scale&bias normal (right)

So how to generate each cube face? Currrently, I am using the brut force method which is horribly slow: for each direction on the cube face, I parse each voxel of the RGB8 volume to search for the one which match the best. One faster method I plan to implement later is to use the Amantide ray marching method to ray march the voxel volume along the ray direction and find the best representative one.

How can this method be used?

Better normal map encoding: when computing object normal map, instead of converting from floating point normal map to scaled&biased normal, do a texture look up in the BFN cube map texture.
Deferred rendering: high quality normal buffer in RGB8! :) It could also be possible to pack a normal in a 32F channel.
Any other ideas?

So when I will have time, don't know when because the end of my PhD is approaching quickly, I plan to implement Amantide's methods to accelerate the computation. Then, maybe use a better representation instead of a cube map.

If you have questions or want access to the cubemap textures, send me an email. As always, feel free to discuss here about this method.

Back from SIGGRAPH 2010

2010-08-01T22:21:00.006+02:00

Wow, SIGGRAPH was awesome!

I saw and learn so many things that I can't list them here. All I can say is that it was nice attending movies (Weta, Pixar, ILM, Dreamworks, etc), games (Crytek, Dice, Valve, etc) as well as researchers presentations. Now I can put a face on those names I often encounter when reading research papers or presentations.

Here are some links with very interesting courses and presentations about graphics:

Dice presentations
Advances in real-time 3D rendering course
Beyond programmable shading course. (This year theme was really oriented towards future hardware evolution)
New: Physically Based Shading Models course
New: Stylized Rendering in Games course
New: GI across Industries courses

My Siggraph Schedule

2010-07-22T13:09:00.003+02:00

Here is my schedule:

Sunday: Avatar in Depth,
Monday: All about Avatar, Detailed Surfaces, Volume and Precipitation, Real-time Live Demo,
Tuesday: Simulation in Production, Computer Animation Festival, Blowing $h!t up, Real-time Live Demo,
Wednesday: Advances in Real-Time Rendering in 3D Graphics and Games I and II,
Thursday: Beyond Programmable Shading I and II.

There is a lot of uncertainty for some sessions because many are overlapping with others I am interested in (hair rendering, stylized rendering, global illumination, physically based shading, etc). It is really hard to make my final choice.

Moreover, two friends are presenting this year:

Nicolas Stoiber, PhD student at Orange Labs like me: The Mimic Game: Real-Time Recognition and Imitation of Emotional Facial Expressions. (He will write an article here about his very interesting work after SIGGRAPH),

Guillaume François, shader writer at Weta Digital : Importance Sampling for Production Rendering.

I wish them good luck with their presentation. I hope one day I could contribute to SIGGRAPH to. Indeed, my PhD was not really suited for such graphic/animation research... Maybe during my future job!!

I will post here about SIGGRAPH during the next week if I can find time between sessions and SIGGRAPH night parties. :)

OpenCL

2010-07-20T16:06:00.002+02:00

Before I even did something with CUDA, I have decided to move to OpenCL. Just because I prefer the GLSL-like approach to load program instead of using an additional dedicated compiler. I am close to finish my C++ library with nice classes encapsulating OpenCL objects (And I am using the C interface). It looks really like what I did for my Shader manager (like in the demos on my personal website). I will post it here soon in case someone want to get it and modify it. Also, choosing OpenCL is for the ability to use any platforms, not only nVidia. The current problem is that I don't have a single clue about how ATI parallel architecture is (their Stream Computing technology).

In addition to a review and my PhD thesis I have to finish writing, I am also preparing my travel to Los Angeles for SIGGRAPH 2010. I will post my schedule tomorrow in a second post.

Bonus:
- a nice real-time CUDA ray-tracer called brigade.
- a new A-Buffer method using per-pixel linked list to manage dynamic allocation of pages containing depth layers. In OpenGL! :)

Volumetric lines 2 + News

2010-06-27T21:05:00.006+02:00

Hello!

Sorry for the long time silence but I have been very busy recently: PhD writing, not accepted paper I have to modify, internship students to take care, researchers who want to use my engine, etc... Now it's better, I will have more free time by the end of the week.

So what are the news! Volume Lines 2 demo is on-line. In fact, I have submitted it to GpuPro2 but it was not accepted: I expected this answer because indeed the method is pretty simple. The results are nicer than previous volumetric lines but it requires geometry shaders and it is more computationally expensive.

Also, I started to work in Virtual Textures few weeks ago. I have the low resolution feedback buffer working with required pages to load, indirection table, etc. Just need to load required pages from the disk to GPU virtual memory in a separate thread together with a LRU cache. I will work again on it soon.

The development of my light-pre pass renderer with relief mapping is... well... deferred. :/ I have a lot of optimization ideas and I hope I can show you more about it soon.

Finally, I will go to Siggraph 2010 this year as an attendee! :) Many courses and talks to attend! It will be hard to attend every sessions I would like to but, I will do my best. I may talk about this conference, show some pictures and report about courses/talks on this blog. Maybe I will be able to meet some of you overthere! :D

Knee deep in the dead

2010-06-01T21:52:00.003+02:00

Some pointers on fast but pretty good reviews of old IdTech and their adaptions to iPhone!

Wolfenstein3D iPhone
Doom PC
Doom iPhone
Quake PC

Ah those good old days... I was so young when I played Doom the first time on a friend's computer. By the way, everyone should read Micheal Abrash's Black Book full of anecdotes and in-depth details about Quake renderer algorithms. For instance, you would learn that John Carmack bet that he could implement dynamic lighting in the quake engine in less than an hour. Indeed, he failed but of only few minutes (3 to 7, I can't remember). But hell yeah! In an hour! :)

Automatic CUDA optimizer

2010-05-19T11:45:00.009+02:00

A promising automatic CUDA optimizer has been proposed by Huiyang Zhou et al. as you can read on his website: http://www.eecs.ucf.edu/~zhou/ (with downloadable paper). Also, they plan to extend their compiler to OpenCL and I am sure this could also be done for DX11 ComputeShader.

The performance are interesting since they are able to achieved as good to better performance as compared to manually optimized programs (something which takes time). Bonus: the code of this open source compiler will be available soon.

Update: the code is available here. (source)

My Relief-Mapped Light Pre-Pass Renderer

2010-04-21T19:04:00.012+02:00

Hi!

Just to show you some progress I made concerning my new renderer: a light pre-pass renderer with relief-mapping over all surfaces! An important feature of this engine is that each texel in the virtual scene is unique. Indeed, no virtual texture mapping is used but simply large 4096*4096 textures! Because no virtual texturing is used, everything is resident in the GPU video memory. Texture for dynamically added objects in the VE are allocated in dynamic texture atlases (using a quad-tree to manage texture space use). As a result, when designing a map (under Maya with my exporter), you have to find a good trade-off between texel density in world space and the size of your map. For me, it is important to have unique texel everywhere to achieve this kind of effects I made before.

Yesterday, as visible on screen-shots, I have finished implementing Virtual Shadow Depth Cube map as described in ShaderX3. I may test later render to cube map using geometry shader.

Thank to this light pre-pass engine, I will be able to easily implement/test several rendering methods like soft particles, light propagation volume, SSAO, etc... The only thing I miss in this engine is spherical-harmonic- or Source-like lightmaps (I wish I had a Beast or Turtle license for this). Currently, it is a Quake3-like lightmap: no directional information about incoming light on surfaces.

With this engine I plan to develop a simple Quake3-like-death-match-with-bots game as a simple demo. If some artist read this post and are interested in designing a death match map, please send me an email. The only thing you would need is Maya2008.

Black lightmap, two shadowed point light sources

Grey lightmap: directional indirect lighting using
spherical harmonic volume.
For this, I use the library I have developed.

In these screenshots, I only used a uniformly colored lightmap because I did not found time to generate one for this level. And also because I just finished adding point light sources support which will be used for direct illumination (lightmap will only contains emissive surface and global illumination). Next move is adding spot lights and optimizing shaders as well as the way meshes are processed by the engine.

Feel free to ask me some questions! :)

Spherical Harmonics Lighting

2010-04-09T18:59:00.035+02:00

Hello everyone!

It's time for another post on another rendering method! :) Currently, I am working on a light pre-pass renderer. I have just finished to include SH lighting inside.

There exist several methods to compute the lighting solution of a virtual environment. Some methods are fully dynamic (IdTech4, CryEngine) or fully/partially static (idTech3, UnrealEngine). For fully dynamic methods, since the lighting solution is often unified, there is no problem to compute the lighting of dynamic objects. However, for static methods, there can be several problems.

Let's take the case of an environment having its lighting solution stored in a static lightmap. When you add dynamic objects in the virtual environment, you have to compute their lighting solution. But how can we compute the light that reach each dynamic object using only their position/orientation and the surrounding environment?

Common methods are using probe or light volume. For instance, in Quake3, a light volume is define for the entire environment and each Voxel stores an ambient color plus a directional colored source (Quake3 map Specs). Another solution is to used probes positioned by artists in the virtual environment (Source engine). At each of this location, irradiance can be computed and stored in several formats (SH, directional light, Source basis, etc).

For my light pre-pass renderer, I decided to use a light volume having each Voxel containing a 2 bands spherical harmonics as visible on the next screen-shots.

A quake3 map with it's corresponding SH volume.

My test map with its corresponding SH volume.

For each dynamic object, the SH volume is sampled on the CPU using tri-linear interpolation (using object's position). The final SH contribution is added during the final pass of the light pre-pass pipeline according to the normals stored in the g-buffer (And in my renderer, each surface is rendered using relief mapping).

A dynamic object lit only by dynamic lights (left) and
using the SH volume (right).
Note that the lighting information stored in the lightmap now
affects the object final look.

The SH volume is computed as a pre-process. For each Voxel, I render the virtual environment in a cube map as in my previous demo. Finally, the surrounding colored environment stored in this cube map is transformed into the SH basis and stored in its corresponding SH volume Voxel.

I plan to store only global illumination in the lightmap of my environments so the SH volume will only contains global illumination contribution. Then, direct lighting will be computed on static/dynamic objects using standard unified light rendering and shadowing. I also plan to add Crytek's light propagation volume later...

I will continue to work on this engine in parallel to the writing of my PhD thesis, other little demos I have in my pipeline and future job research.

New volumetric lines method

2010-04-09T10:00:00.019+02:00

Hi readers,

I have been working on many things recently. One of these things is a new volumetric line algorithm!
My previous method (basically the same as the one proposed in ShaderX7) was really fast and yields good looking result. I think that's why it was successfully used in the iPhone game PewPew. The only thing about this method is that you should avoid looking lines along their direction because, in this case, the trick I use become visible. Also, it was not possible to shade the line based on its thickness from current viewpoint.
Another method has been proposed by Tristan Lorach to change volumetric lines appearance based on the angle between view and line directions. However, line appearance was represented by only 16 texture tiles and interpolation between them was visible.

The new method I propose is able to render capsule like volumetric lines with any width and for any point of view, i.e. you can look inside/through the line. It also allows the use of thickness based visual effects.

Here is the overall algorithm of my new method:

Extrude an OOBB around the volumetric line having the same width as the line. This is done using a geometry shader computing triangles strips from a single line.
Compute closest and farthest intersections between view ray and capsule using geometric methods and/or quadratic functions resolution. This is done in the OOBB frame reference.
Compute thickness based on capsule intersections and environment depth map.
Shade the volumetric line based on it's thickness.

It is pretty simple but efficient. I will not go into the details right now: there will be a post on my website soon. I will just show some early screenshots from the current version:

The capsules representing the volumetric lines
filled with a white color

line radius can be changed

Thickness based shading and intersection with the environment

View from above and under the ground

The thickness is correct,
even if the camera is inside the volumetric lines

Learning CUDA

2010-03-22T22:27:00.007+01:00

Hi there!

I did not post here for a while because I am very busy right now! I am working on my last paper for my PhD thesis and the deadline is by the end of march... I had do conduct an experiment, process some data and now I am parsing the ANOVA results and this in parallel to the writing of the paper. After this last deadline, I will begin to write my PhD manuscript in English! :) So exciting!

During the few free time I found (almost at night), I test Battlefield Bad Company 2 with friends. Also, I implement spherical harmonic lighting for the purpose of ambient lighting of dynamically moving objects in the current "3d engine" I develop (a light pre-pass renderer with relief-mapping on all surfaces and other cool features).

And, as written in the title, I am currently learning CUDA! Indeed, I have bought the new book by David Kirk and Wen-mei W. Hwu called Programming Massively Parallel Processors: A Hands-on Approach. I found this book well written and it nicely introduces CUDA. Good job authors! I only had a large overview of the CUDA architecture but now it is clear in details.
The first thing I did is to take an example in the nVidia CUDA browser and make it independent of their cutil library and others. To build and run my example, You should just need to install the CUDA Toolkit. You can download this simple program HERE. It features a simple grid of points which height is computed using CUDA and stored in a VBO. The result is finally displayed using OpenGL. Only Debug and Release configurations work (32 and 64bits). Emulation profiles will not compile right now. Also, the CUDA code is nicely separated from the c++ code, something I did not find in every web examples. The other libraries used are glut, glu and glew (everything is in the zip file).
So when I will have finished to read the book (and implemented their examples), I may start writing a little demo... :D

See you soon!

Trailer of Amnesia:The Dark Descent

2010-02-19T16:58:00.007+01:00

Trailer is HERE.

This link features the teaser of the upcoming game from the talented developers of the Penumbra series: Amnesia: The Dark Descent. I have found this teaser really impressive. This is a good example of a nice sounds, visuals and interactions integration!

You can also pre-order the game. And If they reach 2000 pre-orders before the 31st of May, the game will get extra content! I did not played the Penumbra series excepted the demo but I want to try this one so I have pre-ordered the game.

Do not hesitate to spread the word!

Fourier Opacity Mapping rocks!

2010-02-12T11:06:00.016+01:00

The up-coming I3D conference on interactive 3D Graphics and Games will feature several interesting papers! For instance, this year will features papers such as " Cascaded Light Propagation Volumes for Real-Time Indirect Illumination" by Anton Kaplanyan (Crytek) or "Interactive Volume Caustics in Single-Scattering Media" by Wei Hu. Another paper I found very interesting is the one proposed by Jon Jansen and Louis Bavoil (Yeah a French guy), both at nVidia, called "Fourier Opacity Mapping". I could not resist implementing this nice paper.

Fourier Opacity Mapping (FOM) is about approximating light attenuation through a volume made of particles. Let's consider a spot light: the authors proposed to reconstruct the light transmittance function along each ray using a Fourier series. The coefficients of the Fourier series are stored in the fourier opacity map in light view space. This map is generated using usual particles rendering with transformation in the Fourier basis of extinction coefficients in the fragment program. Then, when rendering the particles from view, the corrected light attenuation for each particle can be recovered. I will not go into the details here. You can read the paper here and my report on my personal webpage. Indeed, you can see a video of my implementation of FOM here.

Here are some screenshots of my implementation (Top: without FOM, Bottom with FOM):

A simple particle chain. Notice the correct order of light attenuation.

Colored light attenuation through a particle block.

A grey smoke volume with some red particles. Notice that the red particles attenuate the light correctly: they only affect the color of particles that receive the light after them.

I know that my screenshots are not really eye candy but they show that this method is really efficient to simulate colored light extinction when passing through a volume of particle. Furthermore, if you want an exemple of good use by skilled artist, just have a look at the game Batman Arkham Asylum which implements this method.

~~UPDATE~~
My report is finally available on my website together with demo and open source code. :)
You need a recent video card to run the demo since it uses GLSL 1.5 and render data into 6 buffers in one pass.

The engine I have developed for my PhD thesis

2010-01-19T17:21:00.046+01:00

I am going to present you the engine I have develop for all my experiments during my PhD thesis in virtual reality. My subject is : "Using Gaze Tracking Systems and Visual Attention Models to Improve Interaction and Rendering of Virtual Environments". You can find and read more about my work on this webpage. As a consequence, I work a lot with gaze tracking systems and visual attention models but I am not going to extend more on this here.

My PhD is about virtual reality so I needed an engine to render my Virtual Environments (VE) and conduct my experiments. I like to do things on the GPU so almost everything is computed on GPU. I wanted the engine to display point and spot lights that could be static or dynamic. Also, I wanted the model inside the environment to be simulated physically. Every experiment I were going to be very different so I needed the engine to be easily script-able. Finally, I needed to replay all experiment sessions so I wanted to be able to replay recorded navigations and interactions to apply coslty algorithms in order to study users' gaze behavior. Finally, I wanted to be able to create my own VE very easily.

I have developed this engine during 2 months in summer 2008 from scratch. If you would look at the code, you would see really a lot of classes with nice client interface accessible from the engine interface. But maybe you'll be sad to not see cache friendly structures such as structure of array. I have to admit, I did not take the time to optimize...

I did not show this engine before because the virtual environment were used for double blinded publications in conference and journal.

The virtual environment editor : Maya
At this time, I was learning Maya by myself: this is a pretty nice 3D modeling software simple to use for basic 3D. I decided to use it to create my VE. So I have developed my own exporter. It can export the whole geometry, point light, spot light as well as phong materials. Also, because I wanted to be able to use a lightmap, the world lightmap is rendered by Mental Ray in HDR format. This lightmap only contains global illumination! Thus, each light is rendered as usual (diffuse+specular) and, finally, global illumination is added to the scene. The drawback of this is that static lights cannot be turn off.

Phong Materials
Here are the components of the Phong material I selected from the long list of Maya:

Diffuse color (can be a texture or a single color)
Specular color (can be a texture or a single color)
Specular exponent
Local normal (can be a texture or will be flat)

Lights
I wanted each light to be rendered the same way as when using Mental Ray. Thus I decided to use the same parameters as in Maya. Then I simply wrote a shader corresponding to point and spot light.
Here are the spot light parameters:

Transformation matrix
Color & intensity
Decay rate (in fact, I force this to be quadratic in maya and in the engine)
Penumbra angle & cone angle
Drop off

Here are the point light parameters:

Position
Color & intensity
Decay rate (in fact, I force this to be quadratic in Maya and in the engine)

Indeed, a quadratic extinction of light intensity (close to what happens physically) means that each light can interact with ALL object in the VE. To avoid this problem, I compute the distance from the light center where a surface will receive only 5% of light. Then, I attenuate linearly the intensity from the center of the light to this distance. This methods allows me to compute an Axis-Aligned Bounding Box (AABB) for each lights to accelerate the rendering process by using frustum culling.

The rendering engine
As an addicted to OpenGL, I decided to use OpenGL! :) I also decided that the engine will be based on the zPrePass architecture (like the Doom3 engine). So the final rendering is obtained by summing the contribution of each light by rendering each meshes they interact with (after a zPre pass). Static lights come from the exported VE and, then, dynamic lights can be added from the script. The engine behave similarly concerning static and dynamic meshes.

For shadows, I use a simple depth map for spot lights and a virtual depth cube map for point lights. I only use native hardware shadow map filtering. Not very eye candy but it is enough for the experiment I needed to conduct.

Physics
I use the PhysX API from nVidia. Only rigid body interactions are simulated in my case. Each dynamic objects added to the simulation can be added to the PhysX engine if needed according to one of 4 shape type: bounding box, bounding sphere, convex or general triangle mesh.

Scripts
For the script, I chose Lua together with LuaBind (to simplify functions and classes registration). From the script, you can add and manipulate light and model. The script can also receive mouse and keyboard signals and each frame, a specific update function is called. For instance, here is a small piece of Lua code I use to position a spot light to look like a flashlight the user is holding up:
if(flashLightOn==true)
then
local viewCam = getViewPointOrientPos();
local target = Vector3(0.0,0.0,-5.0);
viewCam:transformBackFromLocalSpace(target);
local origin = Vector3(-0.2,-0.1,0.0);
viewCam:transformBackFromLocalSpace(origin);
flashLight:setIntensity(10.0);
flashLight:setPosTargetUp(origin,target,Vector3(0.0,1.0,0.0));
else
flashLight:setIntensity(0.0);
end
As an example, I have also developed some Lua classes that control some robot which follow a predefined path made of way points. I have to say that Lua is a very powerful and easy to use script language.

Record/Replay feature
My engine is able to record and replay navigation sessions in the VE. For each frame, I do not record everything. For instance, I only record the event when an dynamic object's position has been changed: everything is recorded as an event. I do not interpolate each frame because, for my experiments, I only need each frame that were displayed to the viewer.

Gaze-Tracking
My PhD is about gaze tracking so my engine takes into account gaze tracking. Currently, I have a class communicating with the TobiiX50 soft/hardware. This class is handled by a GazeTrackerManager which allows to easily add support for another type of gaze tracking system.

Some Results
The q3dm1 map in Maya during editing:
The q3dm1 map rendered in my engine with HDR lightmap generated by MentalRay (no lights here, everything is baked in the lightmap). The screenshot is a bit over-exposed due to the automatic luminosity adaptation (a lot of screen area are black).
The house VE used in one of my experiment during editing in Maya:
The house VE rendered in my engine with HDR GI lightmap generated by MentalRay:
A game where you have to destroy ships with your gaze:

A video is available on my youtube space.

I am currently changing the zPrePass renderer for a Light-Pre Pass rendering process. I have to say that it is very interesting! I already have point lights working and performance are impressive! I will talk about this later but I have to try to publish something with it before. :)

Do not hesitate if you have questions or if you want some piece of code. I plan to release the source of this engine at the end of my PhD.

What do you think about OnLive?

2009-12-30T22:54:00.009+01:00

Do you know OnLive? It's a company that is going to provide a technology that will (?) allow people to play games on a computer or a tv (Hmm ok). The tricky part is the fact that you would not need a high-end PC (GPU, CPU, etc) to play, for instance, Crysis. Basically, their software will send your actions on input devices to their servers which will compute the image frames for you. Then, all these frames would be sent back to your computer as a video stream.

The concept is very interesting : you just need to rent a server and you don't need to upgrade your computer. Also, I am sure that they will provide a game catalog like Steam does.

However, to my opinion, this system will not work because:

According to OnLive, you will have to be less than 1000 miles away from a server to play. As a result, they will need a lot of servers to provide *good* pings to costumers. It seems that they consider a ping of 80ms as the limit. So, quality of service will depend a lot on your distance to a server.
I am sure they have new efficient technologies to compress frames into a video stream but is their method lossless? What final resolution can you get? Full HD? (again, this will surely depends on your distance to a server)
As an avid player of FPS, I think that a target ping of 80ms is not enough. Even for a RTS with a lot of micro-actions, fighting game or a car race game you will feel it. And I am not speaking about lags. If you don't play those kind of CPU/GPU time consuming games, may be you don't need a high-end hardware...

I really hope that I am wrong about this technology... Anyway, I wish the OnLive team "Good Luck"!!! :)

Efficient OOP + Publications repositories

2009-12-22T14:47:00.008+01:00

A very interesting presentation has been en-lighted by Atom: Pitfalls of Object Oriented Programming by Tony Albrecht from SCEE. I was aware of the problems exposed in this presentation, however, to my opinion, this is the best presentation so far that explains and details the cach miss optimization problem on a simple example. I will not extend more on the presentation: it is clear enough. If you wonder what is a Structure-Of-Array (in contrary to Array-Of-Structure) and why some people says OOP can be considered as evil (the presentation expose one reason) you should read it now! :)

This presentation can be found on SCEE presentation repository. Here are two more repositories I found very interesting: Valve and Bungie. If you know other interesting publication repositories, please post them.

Bonus: Siggraph 2009 course Advances in Real-Time Rendering in 3D Graphics and Games repository.

News from France

2009-12-16T21:14:00.006+01:00

Hi,

Since my last post, I am back in France! I was in Japan for a Japanese-French research collaboration during one month (November). Maybe I will speak about this work once it is published in a conference or journal. Anyway, I have to say that Japan is a really impressive country! So many place to visit! (temples, tokyo tower, etc). Also food is really good and not expensive! I really recommend traveling in Japan: people are really respectful and will help you even if they can't speak English. To my opinion, each gamer, game developer which travel to Japan should visit Akihabara (called "The electric town"). A lot of games, arcade games, electric shop with very low price, etc. You can even buy an old megadrive for 0.8 euro! :)

Finally, I decided that it was a good time to change my computer. Indeed, my old graphics card caused me too much troubles: random colored pixels, triangles stretched to infinity (only with DirectX, not OpenGL) and even black death screens! I decided to buy hardware pieces that have a good performance/price ratio: Intel i5, geForceGTX275 as the main components.
Now I plan to learn DirectX with this computer. I am using OpenGL for 7 years and it is time for me to learn something else.

See you next news!

Volumetric lines

2009-11-27T06:59:00.006+01:00

Four years ago, A method to render volumetric lines has been proposed by Tristan Lorach at nVidia (link). I have found this method very interesting but, to my opinion, texture interpolation simulating change in point of view were too visible. Three years ago, I decided to propose my own volumetric line rendering method as visible on my website. My method does not allow as much visual effect as the one of nVidia but at least you can change the appearance of lines using a simple 2D square texture. Each line is rendered using 6 triangles and all computations are done in the vertex program. The drawback with my method is that when lines' width is very large and their direction is parrallel to the view direction, the trick I use is visible. However, for line which are not large, this problem is rarely noticeable.

Screen shot of the volumetric lines.
Because the line on the bottom is large, the trick I use is visible.In this screenshot, another appearance texture is applied.

Recently, an article published in ShaderX7 proposed a method very similar, if not same, to the one I propose. I should have written an article before... At that time, I was beginning my Master study and I cannot imagine this would be possible.

Many people write to me in order to discuss this method, and I am really happy to see people using it. I am pleased to announce that my method is use in an iPhone game developed by a French guy as you can read on one if his post and blog. :)

Screen-space buzz survey

2009-11-26T05:06:00.010+01:00

This post exposes a survey of screen-space methods proposed in the field of CG. I will not go into the details of these methods since that is not the purpose of this post but I invite you to read these articles since, to my opinion, they are very interesting.

Screen-space ambient occlusion

Many methods have been proposed to render ambient occlusion. The one proposed by Crytek in 2007 became very popular and was named “Screen-Space Ambient Occlusion”. The method was designed to be used in real time in Crysis game. To my opinion, the real first paper to propose screen space ambient occlusion was proposed by Luft et al.: Image Enhancement by Unsharp Masking the Depth Buffer. Then, a High quality (but slower) version was proposed by Louis Bavoil et Miguel Sainz (nVidia).
It is interesting to note that the screen-space name became more and more popular and started some kind of Buzz! Also, it seems that screen-space and image-space are synonyms. Am I correct?
For more links on screen-space ambient-occlusion please refer to this wikipedia article. Also, many methods have been published in ShaderX7.

Screen-space global illumination

Crytek also proposed a screen-space global illumination method. However, I am not aware of the algorithms they use. I suppose it is a variant of their SSAO using sampled surfaces color as well as its normal direction in order to take into account relative surfaces orientation (refer to their presentation about light propagation volume).
Also, researchers proposed the screen-space directional occlusion method to compute real time global illumination.

Screen-space fluid and water

Even fluid got screen-spac-ed!
Concerning SPH fluid simulation, two methods have been proposed as an alternative to marching-cube-like methods, namely : Screen-Space Meshes and Screen-Space Fluids Rendering with Curvature Flow.
To render ocean, a screen-space grid projection is used in the Cry Engine. Another screen-space approach has also been proposed on gamedev with really nice looking results.

Screen-space light shaft

One article about this in GPU Gems 3. Ok they used the word post-process but after reading the article, it sounds a lot like screen-space to me!
It seems that Crytek uses a similar methods. Obviously, they really love screen-space methods! :). I agree with them since the methods they use are fast and result in eye-candy effects!

Image-space sub surface scattering

And last but not the least, Image-Space Subsurface Scattering for Interactive Rendering of Deformable Translucent Objects.

Conclusion

Wouhaou! So many screen-space methods! I wonder what’s next… Also, you can read this other interesting discussion I have found here to complete this survey.

My first attempt to global illumination

2009-11-24T03:58:00.014+01:00

I was really impressed by the work of Anton Kaplanyan from Crytek on light propagation volume (LPV) for massive lighting and global illumination! So I decided to do a quick test implementation of this method to experiment it by myself. Here are the result:

Direct lightingDirect lighting + global illuminationGlobal illumination onlyGlobal illumination x2

As a quick test, I considered only one LPV aligned on the AABB of a single object (no cascaded LPV). The LPV is one voxel larger than the AABB in order to avoid artifact at borders. Then, I just apply the LPV algorithm discribed in the white paper using virtual point lights and co. As I said, this is a basic test implementation so no SH gradients are used to avoid back face lighting and other artifacts.

The light is a simple spot without shadows.

It is interesting to note that without high frequency detailed texture, the GI lighting have a "square" look mostly due to the Cartesian cubic sampling. Indeed it seems that this is not visible with nice textures and normal mapping, as you can see on the screen-shots presented in their white paper.

Because this is a quick test, the code is somewhat ugly :-|, excepted the class to manage the unfolded volumetric texture. That's why I'm not going to publish it. If you are still interested, send me a mail.

The unfolded volumetric texture I have developed is visible on the bottom of the screenshots. Basically, it represents all slices of the 3D texture. Considering LPV, I also have included a black border color between each slice. Thus, in the shader, I do not have to test if my border samples come from the right slice during the gathering process. However, one step of the gathering process involves now rendering a quad overlapping each unfolded slice (one quad per slice) and not overlapping the black borders.

Ok I could give you computation time for each step of the method but as I said the code is note optimized, there is still a lot of tests (even in shaders!) and I am on a laptop PC...

Next step: implement this method as presented in the paper with cascaded LPV, importance sampling of virtual point lights, etc. I plan to implement this in my small engine I am using now for my PhD (will speak about it later). I don't know when I will do this since the pressure increases as the end of my PhD come closer.

Thanks for reading!

Small tips that could help to get into the game industry

2009-11-23T05:51:00.010+01:00

As I said in my welcome post, I would like to get into the game industry after my PhD. If it is your case, you should read this post...

As you may have noticed, I have a personal web site where I share my work with everyone. All source code of my demos and applications are available. When I started this web site, it was to do like Humus. I really appreciated the demo he developed and the fact that he is sharing his work with the community. It turns out to be a good portfolio...

Since 2008, I have received 3 serious job offers from game studios (2 of them being very well-known). Unfortunately, I was busy with my PhD but they told me to contact them again later. It seems that game developers in these studios browse the web to find some interesting developers and then report to the HR department.

I did a phone interview in english with the lead graphic developer of one of these studios during 2h! A really great guy which speak at Siggraph sometimes. I was really impressed because for me getting in touch with this studio was something unreachable! (NB: for a first interview, it turns out to be quite good but it could have been way better. I have to learn and work more!!!)
This studio even contact me a second time! I'm going on site at the beginning of 2010 for another interview. Awesome.

I will not reveal which studios contact me to not break any chances to get in. Follow this blog to know where I will land! :)

OK, here are my tips:

have a website where you show your personal work. But rendering two triangles with bump mapping is not enough. You need to implement more advanced rendering methods and maybe try to improve on it. Also, show some applications you did (games, editors engines, etc). Concerning my web site, I think it would be better with more applications or projects. And also with a cleaner presentation/appearance.
do not hesitate to post your work on GameDev. Or other website with picture of the day.

I hope this will help concerned people.

Hello world!

2009-11-23T03:17:00.007+01:00

Welcome on my first blog!

Let me introduce myself: my name is Sébastien Hillaire I am a french PhD student in virtual reality at INRIA and Orange Labs at Rennes. I will defend my PhD by the end of 2010.

After, I would like to get in the game industry as a graphic developer. At home, when I'm not playing guitar or games, I really like to develop some graphics related algorithm. For some examples of my work, check my personal website: sebastien.hillaire.free.fr. I am always available to discuss about graphics. Some people mail me about my demos and I am really happy to help them.

As I said, I really enjoy playing games! If you want to be a game developer, you have to play games to try new gameplay, see new rendering methods, etc. My favorite game style is FPS. The game I have played the most is Quake3Arena. For me, this is THE perfect simple skill-based gameplay!! I have also really appreciated the FarCry1/Crysis game for the awesome graphics, gameplay possibility and feeling of liberty you get. More recently, another game got my attention: Left4Dead. The cooperative gameplay in this game is just... I don't know which word I could use because this is such a good gameplay idea!
I hope to see more games like these in the future.

On this blog, I will talk about my PhD experience, graphics related stuff (demo, research papers, etc) and games development.

Ok, I think this is all for the welcome/introduction post! I hope I will be able to make this blog as interesting as those I follow.

See you soon,

Seb