Sunday, May 19, 2013

Re-cap of Deferred Lighting

This is part of a response in a discussion forum I did recently and it summarizes a lot of the things I did a few years ago with renderer design on XBOX 360 / PS3 (more details in the articles below). So I thought I share it here as well:

Before I start talking about memory bandwidth, let me first outline my favorite renderer design that shipped now in a couple of games: from a high-level view, you want to end up with mainly three stages:

Geometry -> Light / Shadow data -> Materials

In this stage you render your objects into a G-Buffer. It might be the most expensive or one of the more expensive parts of your rendering pipeline. You want your G-Buffer to be as small as possible. When we moved from Deferred Shading to Deferred Lighting, one of the motivations was to decrease the size of the G-Buffer. In a typical Deferred Lighting scenario you have three render targets in the G-Buffer: the depth buffer, a color buffer and a normal buffer (those might hold specular information and a material index as well). There are all kinds of ways to compress data like using two channel color formats, two channel normal data etc..
One reason why the G-Buffer needs to be small is what I mentioned above. Every mesh you render in there will be expensive. Obviously I am leaving out a lot of stuff here like pre-Z pass etc..
The other reason why the G-Buffer needed to be small was the fact that you have to read it for each light. On XBOX 360 and PS3 that was a major memory bandwidth challenge and the ultimate reason to move from Deferred Shading to Deferred Lighting. You were able to render now much more lights with the smaller G-Buffer.

Lighting / Shadow Stage
Rendering lights into the light / shadow buffer had the advantage of just using two render targets out of the three in the G-Buffer. The depth buffer and the normal buffer with the specular information. With that setup you could increase the number of lights substantially compared to Deferred Shading. 
The light / shadow buffer holds the data for all lights and shadows, in other words: brightness data separated for diffuse and specular together with the light color. Please note the third render target in the G-Buffer that holds color is not used here.

Material Stage
Splitting up the high-level view into Geometry -> Light / Shadows -> Materials is done because you want to apply expensive materials like skin, hair, cloth, leather, car paint etc.. and you can't store much data in the G-Buffer to describe those materials. So you apply them in screen or image space like a PostFX.
One of the reasons to move to Deferred Lighting was the increased material variety it offers. In a Deferred Shading setup you have to apply the material terms while you do the lighting calculations which sometimes made those really expensive and with the overlapping lights, you did those too often. 

A lot of the recent work is about materials in screen-space. In the last few years my focus moved away from renderer design to global illumination and re-thinking the current Post-Processing pipelines; while solving other challenges for our customers. I hope I have something to share in those areas very soon ...

Update of the Link Section

I added the blogs of Angelo Pesce, Fabian Giesen, Christian Schueler, Ignacio Castaño, Morten Mikkelsen and Sebastien Lagarde to the link list on the right.

Tuesday, May 14, 2013

GPU Programming at UCSD

My first outline for a new GPU Programming class at UCSD. Let me know what you think:

First Class
-- DirectX 11.1 Graphics
-- DirectX 11.1 Compute
-- Tools of the Trade - how to setup your development system

Introduction to DirectX 11.1 Compute
-- Advantages
-- Memory Model
-- Threading Model
-- DirectX 10.x support

Second Class
Simple Compute Case Studies
- PostFX Color Filters
- PostFX Parallel Reduction
- DirectX 11 Mandelbrot
- DirectX 10 Mandelbrot

Third Class
DirectCompute performance optimization
- Histogram optimization case study

Fourth Class
Direct3D 11.1 Graphics Pipeline Part 1
- Direct3D 9 vs. Direct3D 11
- Direct3D 11 vs. Direct3D 11.1
- Resources (typeless memory arrays)
- Resource Views
- Resources Access Intention
- State Objects
- Pipeline Stages
-- Input Assembler
-- Vertex Shader
-- Tessellation
-- Geometry Shader
-- Stream Out
-- Setup / Rasterizer
-- Pixel Shader
-- Output Merger
-- Video en- / decoder access

Fifth Class
Direct3D 11.1 Graphics Pipeline Part 2
--- Keywords
--- Basic Data Types
--- Vector Data Types
--- Swizzling
--- Write Masks
--- Matrices
--- Type Casting
--- SamplerState
--- Texture Objects
--- Intrinsics
--- Flow Control
-- Case Study: implementing Blinn-Phong lighting with DirectX 11.1
--- Physcially / Observational Lighting Models
--- Local / Global Lighting
--- Lighting Implementation
---- Ambient
---- Diffuse
---- Specular
---- Normal Mapping
---- Self-Shadowing
---- Point Light
---- Spot Light

Sixth Class
Physically Based Lighting
- Normalized Blinn-Phong Lighting Model
- Cook-Torrance Reflectance Model

Seventh Class
Deferred Lighting, AA
- Rendering Many Lights History
- Light Pre-Pass (LPP)
- LPP Implementation
- Efficient Light rendering on DX 9, 10, 11
- Balance Quality / Performance
- MSAA Implementation on DX 10.0, 10.1, XBOX 360, 11
Screen-Space Materials
- Skin

Eight Class
- The Shadow Map Basics
- “Attaching” a Shadow Map frustum around a view frustum
- Multi-Frustum Shadow Maps
- Cascaded Shadow Maps (CSM) : Splitting up the View
- CSM Challenges
- Cube Shadow Maps
- Softening the Penumbra
- Soft Shadow Maps

Nineth Class
Order-Independent Transparency
- Depth Peeling
- Reverse Depth Peeling
- Per-Pixel Linked Lists

Tenth Class
Global Illumination Algorithms in Games
- Requirement for Real-Time GI
- Ambient Cubes
- Diffuse Cube Mapping
- Screen-Space Ambient Occlusion
- Screen-Space Global Illumination
- Reflective Shadow Maps
- Splatting Indirect Illumination (SII)

GPU Pro 4 available on

Here is the latest GPU Pro 4 book.

I am helping to create those books now since 12 years.

For GPU Pro 5 we will have a huge amount of mobile devices techniques. Many GPU vendors now put in extensions to allow more modern stuff to happen on mobile devices .. like Deferred Lighting. Overall lots of stuff happening ... I am always surprised about the amount of innovation just happening in rendering in a year. With OpenGL ES 3.0 and the new extensions, we have lots of opportunities to make beautiful looking games. You can port a XBOX 360 game to this platform easily. If you want to contribute an article to a book, drop me an e-mail.