About Me

Hi. I'm Josh Ols. Lead Graphics Developer for RUST LTD.


View Joshua Ols's profile on LinkedIn


Entries in RSRMs (7)


Dice Poker Game (CANCELLED)

[Unity3D, RSRM] Dice Poker prototype

Unity3D Web Demo

Art and game behavior courtesy of Anton Hand.



Click ring to drop dice in box.

Click chips around box to chuck chips in center.

Click button in upper left to reset ring.



This demo will only work for SM3.0 level GPUs. If you attempt to use it on an earlier generation GPU, you will just see black on all the geometry that is using the RSRM shader.



If you were wondering why I haven't posted much lately then please allow me to fill you in on the details. Recently, I tried getting my Physically-Based Rendering (PBR) tech into Unity3D, with surprising success. Having gotten this working in a prototype, my artist friend set out to build a game using this new tech. Sadly, the project encountered some complications, and ultimately had to be cancelled.


Despite this, the prototype is in a state where the basic functionality is working. So I have decided to re-purpose it as a tech demo, which you can access via the above link. Please check it out and leave some feedback about the visuals in the comments section. ;)



The prototype shader used for this demo was single-pass forward rendered using two fake directional lights and a Radially Symmetric Reflection Map (RSRM). The scene itself also has SSAO, Bloom, and FXAA just to top it off.


The results are slightly incorrect due to my misunderstanding of the need to plug the BRDFs into the Punctual Light Equation and cancel some terms out. As a result, the specular ended up too bright, and fresnel for RSRM gives too much rim for rough surfaces. So while it is significantly better than the current standard of Phong & Blinn Phong lighting models, it still falls short of what I wanted to achieve.


Currently, I am working on an improved version that will correctly integrate with Unity3D's forward multipass lighting system. It will correctly accumulate the lighting in an FP16 target in linear space, then receive tonemapping before being passed to the rest of the pipeline. I will also being improving upon the built-in Bloom, SSAO, and Depth of Field effects to really sell the effect.


Keep your eyes peeled for future updates! ;)


Other Updates:

Other than this, I am currently working on a revised RSRM article, to centralize all I have learned in an easy to reference article. It will also detail the PBR BRDF I have chosen for my project, including strengths and weaknesses. Finally, this will provide a good opportunity for me to clear up some incomplete and flat out wrong information from my old articles.


Other than that, I've also made a small update to my Partial Derivative Normal Map (PDN) article. Basically, I found an optimization to use a MADD to save an assignment, shaving off another instruction from the recovery equations.


Hope you will find them useful. :)


RSRM Enhancements

Let me preface this post by saying that I touched on some of these topics before. I decided to revisit them because I have learned a great deal since then, and felt I should make a little update on my progress. In particular, I wanted to correct some things that I got wrong in my prior post on the topic.


For the longest time, I was having trouble reconciling some of my results with those achieved in published Image-Based Lighting (IBL) papers. As it turns out, there were some problems in both my rendering code, and the console app that I wrote to convert an input gradient image into an Radially Symmetric Reflection Map (RSRM). The net result of my mistakes was that my RSRMs' lower specular powers were getting progressively darker when they shouldn't have been.


During this time, I had the incorrect notion that I needed specular power zero to represent my diffuse IBL. Naturally, this was incompatible with the non-linear mapping I have been touting so much. So I tried a remapping scheme to store specular power zero, but access it as if was specular power one. Unsurprisingly, this threw off the results for all my specular lighting, on top of producing incorrect diffuse lighting.


As a result, I was afraid I was going to have to use a different approach to generate and store my diffuse lighting (eg. Spherical Harmonics (SH)). Fortunately, when I fixed my RSRM generation app, I realized that specular power one can adequately represent diffuse IBL. This way, I can generate all my IBL data using one pipeline, and store all related lighting data in one RSRM texture.


In the process of fixing these problems, I took the opportunity to add some improvements to the system. All of which had the effect of greatly improving both the lighting quality, and reducing both storage and runtime costs for my RSRMs.


Non-Linear Mapping:

In my prior article, I demonstrated how a non-linear mapping of specular powers can reduce storage costs for IBL . It accomplishes this by accounting for the fact that the specular lighting has a non-linear rate of change for increasing specular powers. So we can store more samples for the rapidly changing lower powers, and less for the slowly changing higher powers. This allows us to reduce the number of discrete samples needed to recreate the full range of specular lighting. As it turns out, that was merely scratching the surface of how this technique can benefit the graphics pipeline.


Firstly, it allows artists to use a more intuitive "smoothness" parameter [0, 1] to handle the transition from rough to smooth materials. This allows them to use linear values to effect a perceptually linear change in the smoothness of the material. Compare that to using specular powers directly, which make it hard to tweak materials as their smoothness gets higher.


Secondly, this new parameter can be represented quite efficiently using only byte precision. As a result, it works great with g-buffers that use RGBA8 targets to reduce storage costs. Of course this also applies to storing the data in a smoothness map texture for materials. So it can be sampled directly from the texture into the g-buffer with no need for packing instructions.


Thirdly, this parameter is efficient to use at runtime, because it can be used to directly sample an RSRM. Plus it is cheap to recover the specular power for analytical lights, which only requires a pow(2048.0f, smoothness). This way, the two lighting systems gel together perfectly!


Finally, I decided to go with 256x32 for the map's resolution. This seems to be adequate for representing all the specular powers I want while being reasonably compact. Plus, it establishes a good habit for if I ever develop for the XBOX 360 (ie. 360 uncompressed textures have a lower size limit of 32x32, including the mipmaps).



Here are some videos and screenshots showing the new improvements in action. Please note that for all these tests, the only sources of illumination are an RSRM and a directional light.


Enjoy! ;)



[RSRM] Linear [1,2048][RSRM] Non-Linear [1,2048]

Figure 1. 1, Linear [1, 2048]; 2, Non-Linear [1, 2048]

Material Demos:


Figure 2. 1, Iron Sphere; 2, Plastic Sphere; 3, Plastic Statue; 4, Plastic-Metal composite (Model by Ben Mathis)


Final Thoughts:

After much testing, tweaking, and re-testing my RSRM implementation has developed to the point where I feel safe integrating it into my project. The next step will be to add RSRMs to my Light Pre-Pass renderer, and see how they will look with local lights using physically-based BRDFs. I'll keep everyone posted on my progress.


As always, questions and suggestions are welcome in the comments section. ;)



Now that I've got Radially Symmetric Reflection Maps (RSRMs) figured out, it's time to tackle the rest of my ambient lighting solution. So the next logical step for improving lighting quality is to introduce Ambient Occlusion (AO). As my plan is to go the route of deferred shading, the natural choice for getting the best visuals is Screen-Space Ambient Occlusion (SSAO).



My long-time readers might recall that I have touched on this topic before, getting acceptable results for the time. However, I recently came across a new implementation that is both cheaper and more visually pleasing than my previous choice. For implementation details, check out the gamedev.net article "A Simple and Practical Approach to SSAO".


Here you can see the results I got when I combined this technique with RSRMs. In each shot, the AO is correctly applied to just the ambient diffuse & specular contributions. The direct lighting is currently unshadowed, so it is not as good as it could be.



[RSRM, SSAO] ssao only [SSAO] ssao, old implementation

Figure 1. 1, New; 2, Old

(Model by Ben Mathis)



[RSRM, SSAO] Ambient diffuse w [RSRM, SSAO] Ambient diffuse wo [RSRM, SSAO] Ambient specular w [RSRM, SSAO] Ambient specular wo [RSRM, SSAO] composite w [RSRM, SSAO] composite wo

Figure 2. 1, Diffuse w; 2, wo; 3, Specular w; 4, wo; 5, Composite material w; 6, wo

(Model by Ben Mathis)


Future Plans:

As nice as SSAO can look, even the cheapest implementation can still be a performance hog even on newer hardware. So for older GPUs I will likely be using a textured AO fallback, with min blending to combine model and detail AO maps. It may not look as good, but AO in any form is still a huge step up from no AO at all.


Sorry for the delay folks. That "real life" thing has a habit of getting in the way of my leisure activities. :p


IBL + Non-Linear Mappings:


I've written a new article continuing what I started with this one. In particular, I explain what I got wrong, and how I improved on it. As for this article, I wanted to mention that my remapping scheme to get specular power zero is incorrect and will make for worse results. Also, mipmapping the RSRMs doesn't improve quality, but it does speed up texture fetches.



As it stands, the major problem with Prefilted IBL is that it requires us to store an environment map for every specular power [0,255]. It may not seem like much for RSRMs, but it is seriously wasteful for cubemap IBL. In either case, we are left with the question of how best to reduce storage costs without having to drastically reduce shading fidelity.



The first and most obvious trick is to reduce the number of slices used to represent that range of specular powers. Then we can simply let the texture interpolation hardware fill in the gaps to recover what was lost. Sadly, it turns out that it isn't so simple, as I will now illustrate. Please note, these examples show the orginal 256x256 map, as well as one that was upsampled from 8x256.


Linear mapping:

The first approach continues using a linear mapping of slices to specular powers [0,255], and simply reduces the number of slices. Sadly, while this does a good job for higher specular powers, it is abysmal for the lower powers. This can be explained by the fact that the lower powers have a much higher rate of change, and lose significantly more information as you reduce the number of slices.


[RSRM] linear[RSRM] linear, resized  

Figure 1. 1; Original; 2, Resized


Non-linear mapping:

The second approach was surprisingly unclear in most of the IBL articles I was able to find on the web. Basically, you use a non-linear mapping of specular powers to slices (ex. 255^n), so that we store significantly more information about the lower specular powers. Sadly, a power function like this doesn't ever reach zero, meaning we can't use it in its raw form to represent diffuse lighting. In order to fix this problem we have to remap the specular powers so that they are stored/accessed as [1,256], but store information as [0,255].


Generation:  sp = powf(256.0f, n) - 1.0f;

Runtime : n = log(sp + 1.0f) / log(256.0f);

[RSRM] non-linear[RSRM] non-linear, resized  

Figure 2. 1; Original; 2, Resized



As you can see, the non-linear mapping does a much better job preserving the lower specular powers despite the drastically reduced resolution. As a result, my RSRMs go from taking 256KB of storage to a measly 8KB. Not the most impressive gain ever, but it will make a big difference at runtime where RAM can be a precious resource. This will be especially important once I add mipmapping to my RSRMs to improve their shading quality.


Radially-Symmetric Reflection Maps

Update 1:

I decided to add some extra screenshots to illustrate how the raw diffuse and specular components map to the complex model used for the material tests. Please note, they use a slightly different RSRM, but the results are consistent with one used for the original article.


Update 2:

For any late readers, I've since added an article expanding on this topic (RSRM Enhancements).



After much deliberation, I have decided to drop Lighting Volumes from my renderer in favor of a more dynamic lighting system. In fact, I have opted to use a piece of tech I tried to implement ages ago, but only recently got working. I am of course refering to Radially-Symmetric Reflection Maps (RSRMs), the Image-Based Lighting (IBL) technology used by the game Brütal Legend.


So, just what are they? Well, I can't tell you all the details, since the official document & video are only available by purchasing them from ACM. What I can tell you is that they combine Chrome Mapping with Prefiltered IBL, and hope that you can figure out the rest. However, simply knowing how to make and use them is only half the problem. Using them requires a fundamental shift in your thinking about lighting towards a Physically-based Shading paradigm.


Physically-Based BRDFs:

Following the presentations linked in my prior post, I found out that traditional ad-hoc shading models have been getting it wrong for a long time now. Things like diffuse lighting being too bright, specular lighting being too dark, specular color being completely incorrect, etc. In order to make my lighting and materials mesh well with prefiltered IBL of any kind, I had to make a few changes.


Firstly, my analytical lights' diffuse term needs to use the Lambertian BRDF, which effectively boils down to multiplying it by 1/PI. This may not seem like much, but it make a big difference. The most common symptom of getting this wrong is that most materials' albedo textures end up being much darker to get good looking results.


Secondly, my analytical lights' specular term needs to use Normalized Phong. This ensures that the specular highlight's intensity gets darker/brighter as its distribution increases/decreases. The typical consequence of this being wrong is that specular color maps have to modify intensity in a way that corresponds to specular power. Often, this can require that the specular color map have multipliers to exceed the range [0,1].


Thirdly, specular color itself is a farse, and doesn't match up with reality at all. Instead, materials need per-pixel, per-RGB fresnel cooefficients that determine the substance of the material. Then specular power maps, more correctly called smoothness maps, can control the distribution of a specular highlight to make it more blurred/sharp. By doing it this way, artists can use real-world fresnel values to define the substance of a material, then adjust the smoothness without changing that substance.


Finally, just a few little concerns, like ensuring that my albedo & fresnel data produces energy conserving materials. Also, ensuring that metals and semiconductors have an albedo of zero since they absorb all diffuse lighting. If you want to learn more, please refer to the presentation links in my prior post.


Having taken all that into account, let's see what we get with RSRMs!



By itself, an RSRM's diffuse contribution looks like a Hemisphere Light, but having more color variation toward the center of the sphere. Once we add a Directional Light, it suddently gains more detail and becomes a passable source of illumination. As for the specular lighting, again it is essentially Chrome Mapping with more variation. It may not sound like much, but let's see how it looks in practice.


What follows are a series of example materials, including one that uses per-pixel albedo, fresnel, and smoothness data to have metal embeded inside plastic.



[RSRM] Input[RSRM] linear  

Figure 1. 1, Input; 2, Output


Shading Components:

[RSRM] _diffuse (map)[RSRM] _diffuse (map+drectional)[RSRM] _specular (m=2)[RSRM] _specular (m=10)[RSRM] _specular (m=50)[RSRM] _specular (m=100)

Figure 2. 1, Ambient; 2, Ambient +Directional; 3, Specular (m=2); 4, Specular (m=10); 5, Specular (m=50); 6, Specular (m=100)


[RSRM] Hebes (diffuse, map)[RSRM] Hebes (diffuse, map+directional)[RSRM] Hebes (specular, m= 10)

Figure 2.5. 1, Ambient; 2, Ambient +Directional; 3, Specular (m=6)



[RSRM] Hebes (plastic, m=6)[RSRM] Hebes (glass coating, m=255)[RSRM] Hebes (silicon, m=6)[RSRM] Hebes (iron, m=6)[RSRM] Hebes (copper, m=6)[RSRM] Hebes (gold, m=6)[RSRM] Hebes (aluminum, m=6)[RSRM] Hebes (silver, m=6)

Figure 3. 1, Plastic; 2, Glass Coating; 3, Silicon; 4, Iron; 5, Copper; 6, Gold; 7, Aluminum; 8, Silver


Per-pixel Fresnel & Smoothness:

[RSRM] Agusturinn [front][RSRM] Agusturinn [left][RSRM] Agusturinn [back][RSRM] Agusturinn [right]

Figure 3. 1, Front; 2 Left; 3 Back; 4 Right

(Model by Ben Mathis)



Despite the simplicity of the technique, the results are surprisingly good. RSRMs are efficient to author and bake, and require little in the way of storage. Since I do not require photorealism, they produce excellent IBL results that will work well as my ambient lighting solution. Now I just need to add translucent sun shadows and SSAO to really improve the results. ;)