About Me

Hi. I'm Josh Ols. Lead Graphics Developer for RUST LTD.


View Joshua Ols's profile on LinkedIn


Entries in IBL (7)


SH 0th order as an ambient light sensor


The other day an Alloy customer came to me with a weird problem that I never could have anticipated. Basically, they have a full PBR pipeline including physical materials, high intensity lights, and filmic tonemapping with automatic eye adaption. Their game also has a slick in-world 3D UI system involving an emissive holographic display. Independently, these things are great, but apparently they break down when put together.



You see, the emissive display looked great in typical indoor lighting settings, but would become unreadably dark when the player went outside into direct sunlight. The culprit was the automatic eye adaption responding to the bright sunlit surroundings and dropping the exposure to darken everything, including the emissive display. Essentially, this is the exact same problem that smartphone displays face in the real world.



So how do smartphones try to deal with this problem? They use ambient light sensors to determine the overall brightness of their environment and crank up their screen brightness to compensate. So how do we accomplish this with a shader?

It turns out that Unity provides all the information we need in the form of light probes. Specially, the probe’s Spherical Harmonic’s (SH) 0th order is a constant color representing the average of all incoming ambient light at that point in space. Thus, the luminance of this color is a pretty accurate measure of the light intensity of its surroundings.

With this information in hand it was easy to dynamically scale the display’s emission intensity based on a curve, which now made it viewable under all lighting conditions.



RSRM Enhancements

Let me preface this post by saying that I touched on some of these topics before. I decided to revisit them because I have learned a great deal since then, and felt I should make a little update on my progress. In particular, I wanted to correct some things that I got wrong in my prior post on the topic.


For the longest time, I was having trouble reconciling some of my results with those achieved in published Image-Based Lighting (IBL) papers. As it turns out, there were some problems in both my rendering code, and the console app that I wrote to convert an input gradient image into an Radially Symmetric Reflection Map (RSRM). The net result of my mistakes was that my RSRMs' lower specular powers were getting progressively darker when they shouldn't have been.


During this time, I had the incorrect notion that I needed specular power zero to represent my diffuse IBL. Naturally, this was incompatible with the non-linear mapping I have been touting so much. So I tried a remapping scheme to store specular power zero, but access it as if was specular power one. Unsurprisingly, this threw off the results for all my specular lighting, on top of producing incorrect diffuse lighting.


As a result, I was afraid I was going to have to use a different approach to generate and store my diffuse lighting (eg. Spherical Harmonics (SH)). Fortunately, when I fixed my RSRM generation app, I realized that specular power one can adequately represent diffuse IBL. This way, I can generate all my IBL data using one pipeline, and store all related lighting data in one RSRM texture.


In the process of fixing these problems, I took the opportunity to add some improvements to the system. All of which had the effect of greatly improving both the lighting quality, and reducing both storage and runtime costs for my RSRMs.


Non-Linear Mapping:

In my prior article, I demonstrated how a non-linear mapping of specular powers can reduce storage costs for IBL . It accomplishes this by accounting for the fact that the specular lighting has a non-linear rate of change for increasing specular powers. So we can store more samples for the rapidly changing lower powers, and less for the slowly changing higher powers. This allows us to reduce the number of discrete samples needed to recreate the full range of specular lighting. As it turns out, that was merely scratching the surface of how this technique can benefit the graphics pipeline.


Firstly, it allows artists to use a more intuitive "smoothness" parameter [0, 1] to handle the transition from rough to smooth materials. This allows them to use linear values to effect a perceptually linear change in the smoothness of the material. Compare that to using specular powers directly, which make it hard to tweak materials as their smoothness gets higher.


Secondly, this new parameter can be represented quite efficiently using only byte precision. As a result, it works great with g-buffers that use RGBA8 targets to reduce storage costs. Of course this also applies to storing the data in a smoothness map texture for materials. So it can be sampled directly from the texture into the g-buffer with no need for packing instructions.


Thirdly, this parameter is efficient to use at runtime, because it can be used to directly sample an RSRM. Plus it is cheap to recover the specular power for analytical lights, which only requires a pow(2048.0f, smoothness). This way, the two lighting systems gel together perfectly!


Finally, I decided to go with 256x32 for the map's resolution. This seems to be adequate for representing all the specular powers I want while being reasonably compact. Plus, it establishes a good habit for if I ever develop for the XBOX 360 (ie. 360 uncompressed textures have a lower size limit of 32x32, including the mipmaps).



Here are some videos and screenshots showing the new improvements in action. Please note that for all these tests, the only sources of illumination are an RSRM and a directional light.


Enjoy! ;)



[RSRM] Linear [1,2048][RSRM] Non-Linear [1,2048]

Figure 1. 1, Linear [1, 2048]; 2, Non-Linear [1, 2048]

Material Demos:


Figure 2. 1, Iron Sphere; 2, Plastic Sphere; 3, Plastic Statue; 4, Plastic-Metal composite (Model by Ben Mathis)


Final Thoughts:

After much testing, tweaking, and re-testing my RSRM implementation has developed to the point where I feel safe integrating it into my project. The next step will be to add RSRMs to my Light Pre-Pass renderer, and see how they will look with local lights using physically-based BRDFs. I'll keep everyone posted on my progress.


As always, questions and suggestions are welcome in the comments section. ;)


RNM Shaders + Unity 3D

Well, that happened a little sooner than I was expecting. Tonight, my artist friend managed to complete the example scene & documentation. So I decied to go ahead and post our RNM shaders, documentation, example scene, etc, on the Unity forums.


Hope you guys find them interesting/useful.



Don't forget to check out my new IBL article below. ;)


IBL + Non-Linear Mappings:


I've written a new article continuing what I started with this one. In particular, I explain what I got wrong, and how I improved on it. As for this article, I wanted to mention that my remapping scheme to get specular power zero is incorrect and will make for worse results. Also, mipmapping the RSRMs doesn't improve quality, but it does speed up texture fetches.



As it stands, the major problem with Prefilted IBL is that it requires us to store an environment map for every specular power [0,255]. It may not seem like much for RSRMs, but it is seriously wasteful for cubemap IBL. In either case, we are left with the question of how best to reduce storage costs without having to drastically reduce shading fidelity.



The first and most obvious trick is to reduce the number of slices used to represent that range of specular powers. Then we can simply let the texture interpolation hardware fill in the gaps to recover what was lost. Sadly, it turns out that it isn't so simple, as I will now illustrate. Please note, these examples show the orginal 256x256 map, as well as one that was upsampled from 8x256.


Linear mapping:

The first approach continues using a linear mapping of slices to specular powers [0,255], and simply reduces the number of slices. Sadly, while this does a good job for higher specular powers, it is abysmal for the lower powers. This can be explained by the fact that the lower powers have a much higher rate of change, and lose significantly more information as you reduce the number of slices.


[RSRM] linear[RSRM] linear, resized  

Figure 1. 1; Original; 2, Resized


Non-linear mapping:

The second approach was surprisingly unclear in most of the IBL articles I was able to find on the web. Basically, you use a non-linear mapping of specular powers to slices (ex. 255^n), so that we store significantly more information about the lower specular powers. Sadly, a power function like this doesn't ever reach zero, meaning we can't use it in its raw form to represent diffuse lighting. In order to fix this problem we have to remap the specular powers so that they are stored/accessed as [1,256], but store information as [0,255].


Generation:  sp = powf(256.0f, n) - 1.0f;

Runtime : n = log(sp + 1.0f) / log(256.0f);

[RSRM] non-linear[RSRM] non-linear, resized  

Figure 2. 1; Original; 2, Resized



As you can see, the non-linear mapping does a much better job preserving the lower specular powers despite the drastically reduced resolution. As a result, my RSRMs go from taking 256KB of storage to a measly 8KB. Not the most impressive gain ever, but it will make a big difference at runtime where RAM can be a precious resource. This will be especially important once I add mipmapping to my RSRMs to improve their shading quality.


Radially-Symmetric Reflection Maps

Update 1:

I decided to add some extra screenshots to illustrate how the raw diffuse and specular components map to the complex model used for the material tests. Please note, they use a slightly different RSRM, but the results are consistent with one used for the original article.


Update 2:

For any late readers, I've since added an article expanding on this topic (RSRM Enhancements).



After much deliberation, I have decided to drop Lighting Volumes from my renderer in favor of a more dynamic lighting system. In fact, I have opted to use a piece of tech I tried to implement ages ago, but only recently got working. I am of course refering to Radially-Symmetric Reflection Maps (RSRMs), the Image-Based Lighting (IBL) technology used by the game Brütal Legend.


So, just what are they? Well, I can't tell you all the details, since the official document & video are only available by purchasing them from ACM. What I can tell you is that they combine Chrome Mapping with Prefiltered IBL, and hope that you can figure out the rest. However, simply knowing how to make and use them is only half the problem. Using them requires a fundamental shift in your thinking about lighting towards a Physically-based Shading paradigm.


Physically-Based BRDFs:

Following the presentations linked in my prior post, I found out that traditional ad-hoc shading models have been getting it wrong for a long time now. Things like diffuse lighting being too bright, specular lighting being too dark, specular color being completely incorrect, etc. In order to make my lighting and materials mesh well with prefiltered IBL of any kind, I had to make a few changes.


Firstly, my analytical lights' diffuse term needs to use the Lambertian BRDF, which effectively boils down to multiplying it by 1/PI. This may not seem like much, but it make a big difference. The most common symptom of getting this wrong is that most materials' albedo textures end up being much darker to get good looking results.


Secondly, my analytical lights' specular term needs to use Normalized Phong. This ensures that the specular highlight's intensity gets darker/brighter as its distribution increases/decreases. The typical consequence of this being wrong is that specular color maps have to modify intensity in a way that corresponds to specular power. Often, this can require that the specular color map have multipliers to exceed the range [0,1].


Thirdly, specular color itself is a farse, and doesn't match up with reality at all. Instead, materials need per-pixel, per-RGB fresnel cooefficients that determine the substance of the material. Then specular power maps, more correctly called smoothness maps, can control the distribution of a specular highlight to make it more blurred/sharp. By doing it this way, artists can use real-world fresnel values to define the substance of a material, then adjust the smoothness without changing that substance.


Finally, just a few little concerns, like ensuring that my albedo & fresnel data produces energy conserving materials. Also, ensuring that metals and semiconductors have an albedo of zero since they absorb all diffuse lighting. If you want to learn more, please refer to the presentation links in my prior post.


Having taken all that into account, let's see what we get with RSRMs!



By itself, an RSRM's diffuse contribution looks like a Hemisphere Light, but having more color variation toward the center of the sphere. Once we add a Directional Light, it suddently gains more detail and becomes a passable source of illumination. As for the specular lighting, again it is essentially Chrome Mapping with more variation. It may not sound like much, but let's see how it looks in practice.


What follows are a series of example materials, including one that uses per-pixel albedo, fresnel, and smoothness data to have metal embeded inside plastic.



[RSRM] Input[RSRM] linear  

Figure 1. 1, Input; 2, Output


Shading Components:

[RSRM] _diffuse (map)[RSRM] _diffuse (map+drectional)[RSRM] _specular (m=2)[RSRM] _specular (m=10)[RSRM] _specular (m=50)[RSRM] _specular (m=100)

Figure 2. 1, Ambient; 2, Ambient +Directional; 3, Specular (m=2); 4, Specular (m=10); 5, Specular (m=50); 6, Specular (m=100)


[RSRM] Hebes (diffuse, map)[RSRM] Hebes (diffuse, map+directional)[RSRM] Hebes (specular, m= 10)

Figure 2.5. 1, Ambient; 2, Ambient +Directional; 3, Specular (m=6)



[RSRM] Hebes (plastic, m=6)[RSRM] Hebes (glass coating, m=255)[RSRM] Hebes (silicon, m=6)[RSRM] Hebes (iron, m=6)[RSRM] Hebes (copper, m=6)[RSRM] Hebes (gold, m=6)[RSRM] Hebes (aluminum, m=6)[RSRM] Hebes (silver, m=6)

Figure 3. 1, Plastic; 2, Glass Coating; 3, Silicon; 4, Iron; 5, Copper; 6, Gold; 7, Aluminum; 8, Silver


Per-pixel Fresnel & Smoothness:

[RSRM] Agusturinn [front][RSRM] Agusturinn [left][RSRM] Agusturinn [back][RSRM] Agusturinn [right]

Figure 3. 1, Front; 2 Left; 3 Back; 4 Right

(Model by Ben Mathis)



Despite the simplicity of the technique, the results are surprisingly good. RSRMs are efficient to author and bake, and require little in the way of storage. Since I do not require photorealism, they produce excellent IBL results that will work well as my ambient lighting solution. Now I just need to add translucent sun shadows and SSAO to really improve the results. ;)