Okay, first bit of progress with my new rendering approach. I started by testing out NAO32/RGBM for encoding HDR values in an MSAA/CSAA RGBA8 buffer. Since these are two of the major features that made me decide to make the switch, it seemed appropriate to test them first. So without further ado, I present the gruesome results!
First I will show the results of each sample setting by itself, then its difference from no AA. Please note, I found no appreciable visual difference between 8x MSAA and 8x/8xQ CSAA, so I will not be posting them.
Figure 1. 1. No MSAA, 2. 2xMSAA , 3. 4xMSAA, 4. 8xMSAA
Figure 2. 1. 2xMSAA , 2. 4xMSAA, 3. 8xMSAA
Figure 3. 1. No MSAA, 2. 2xMSAA , 3. 4xMSAA, 4. 8xMSAA
Figure 4. 1. 2xMSAA , 2. 4xMSAA, 3. 8xMSAA
As you can see from the comparison shots, both schemes produce acceptable results for 2x/4x MSAA, but start to fall apart for 8x MSAA and equivalent quality CSAA modes. The edge quality improves, but color artifacts start appearing around the scene. So, there goes CSAA, and any MSAA modes above 4x. Still quite useable, but a bit of a bummer nonetheless.