Problem with Custom Lightng Implementation + Potential Idea!

[KazeoHin]

Before I go on with my request, I wish to say that the UDK is awesome, and I thank Epic for every feature and opportunity. I have no intention to sound like I am at all dissatisfied with the UDK or Epic. Now: onto my post.

I've run into a bit of a problem, well actually I ran into it way back in UT3 a few times before this instance: Dynamic self-shadows and Custom Lighting materials. The problem that I have is that the Custom Lighting setup in the material editor doesn't really account for dynamic shadows: In fact anything plugged into the custom lighting input gets cut to zero inside a mesh's dynamic self-shadow. This makes things like partial translucency or diffusion nearly impossible to do well.

This is my recent one-size-fits-all skin shader applied to a skeletal mesh in a one-light environment. Around the edges of the shadow, you can make out brief hints of my custom Sub-Scattering light model. The sub-scattering is part of the custom lighting I've hooked together and is controlled by the light vector, among other things. As you can see in the above image, everything goes 0.0 inside the dynamic shadow, canceling out any pretty scattering effects beyond the edge of the shadow. There is a workaround for this: and that is by entirely disabling dynamic self shadows in the mesh's light environment. This gives me a result like this:

The obvious problem here is that the surface is being lit in areas that should be in shadow. This shows off some cool subsurface stuff, but the end result is not appealing to the eye at all. The problem I can see is that custom lighting in the material editor is not a fully developed tool. It was meant for sort of quickly "sketching" code that would later be written into HLSL by a team of paid professionals. However for many Indie developers, the Material Editor is the only way to go for creating sweet custom lighting models.

The photoshop composite we have here shows off the best of both worlds: the surface lighting is being rendered with shadows, the subscattering is being rendered without. An expert eye will be able to point out the physical inaccuracies (hell, even I can), but for real-time rendering this looks much better than the two previous images. This shows a lighting/material method that has yet to be coded into the engine (or I simply haven't found the checkbox yet). It could be implemented into the engine by changing around mesh rendering a bit and adding a material node that looks at the dynamic self-shadow of a mesh as a per-pixel value. The node can be used like any other single-channel value. Just as a material will only use a texture as the user defines it: the custom lighting will only shadow the code layers that the user defines.

chances are there is
A) a much better way to implement this. -or-
B) a detrimental reason this can never be accomplished by human minds.

Either way this post is kind of a way to raise awareness of the issue, and maybe give epic an idea of a cool feature the community might want.

thoughts?

[agentfx]

thats a bit of a problem you have. Seems like the solution is going to be banging away on the settings to find the happiest place you can.
Can't you add another light? and use composite dynamic? I know thats not what you're looking for as an answer but just trying to start a dialogue to help you out.

Subsurface scattering is WAY overused. this character you would not see 98% of the subsurface you have turned on. Most jr guys add it to show off that they have it. it only obviously occurs on ears and fingers, thin skinned areas, when there is a strong light behind them, not all the time*. You have it on the toes of your guy when the light is above and his toes are solid bone claws. I look at my arm/hand/fingers right now being top lite move it, side lite no SSS.
But this character is just an example. I'm just saying you'll get a bigger benefit from your time if you push better lighting then SSS.

*in high quality rendering in film you will get a benefit from some SSS on faces to bring more life into them, but that too is a bit of a hack.

[KazeoHin]

thats a bit of a problem you have. Seems like the solution is going to be banging away on the settings to find the happiest place you can.
Can't you add another light? and use composite dynamic? I know thats not what you're looking for as an answer but just trying to start a dialogue to help you out.

Subsurface scattering is WAY overused. this character you would not see 98% of the subsurface you have turned on. Most jr guys add it to show off that they have it. it only obviously occurs on ears and fingers, thin skinned areas, when there is a strong light behind them, not all the time*. You have it on the toes of your guy when the light is above and his toes are solid bone claws. I look at my arm/hand/fingers right now being top lite move it, side lite no SSS.
But this character is just an example. I'm just saying you'll get a bigger benefit from your time if you push better lighting then SSS.

*in high quality rendering in film you will get a benefit from some SSS on faces to bring more life into them, but that too is a bit of a hack.

Hey, thanks for the help. I'll give you a little info about my situation:
This environment is set up specifically to show off the current problem. it has merely a single, bright, colourless point-light and a Kactor with a light-environment. You'll see below that in a bright, natural environment the problem is much more subtle, but think not of these screens as what I hope to accomplish: they are merely just the demonstration of the problems I've ran into with more than just subscattering, and they illustrate idea I'm hoping to find/get implemented. Though I'm greatly interested in the topic: I don't want to get into a discussion about subscattering as this thread isn't about SSS, but about custom lighting implementation in the UDK. However for reference, if your skin is any other colour than pale greenish/gray: there is a huge amount of subsurface scattering going on. The amount of light landing on that model above is at a value of 5.0, enough to cause an exaggerated amount of Subscattering, whereas this one below is in more of a natural environment, with a sunlight value of 3.0.

[shot]https://sites.google.com/site/kazeohin/home/FlyingMeta.jpg[/shot]

The shader looks great in bright environments, heavy on indirect lighting, but in the darker, one-light environments the models lose a lot of their lifelike detail. Which is why I get a bit frustrated with this shadow situation. I know what effect I'm looking for, but thus far I don't have the tools to implement it. I'm pretty well acquainted with the graphical features of the engine, mostly I'm just seeing if this problem can be fixed in the next month's (or another's) update, or if there is some INI tweak that can be done: but seeing as how the current material editor doesn't have any way of separating self-shadows from the lightvector, I doubt it.

[agentfx]

Ah yeah. Well you're right it does look a lot better in brighter lights. The shadow thing seems rough, and you obviously have a lot more understanding of the engine that I do. Seems like you need the help of someone with a LOT of shader writing and engine experience.

[KazeoHin]

I don't know much beyond the visual side of things, but I do know that dynamic shadows act outside of the lightvector, because if you plug in just a standard texture into the custom lighting input (without any interaction from the lightvector), it goes 100% black in shadowed areas, and in non-shadowed areas it is a completely unlit texture. It seems that the options the indie developers have with the dynamic shadows are limited, I just theorize that a much greater variety of potential materials would be possible if we were given a "dynamic shadow" shader node to work with in the material editor. It would mean more work for us, but I could live with that.

[rebb]

What happens if you pipe the SSS Term into the Emissive Slot instead ? Does this get overwritten by the Shadowing as well ?

[KazeoHin]

What happens if you pipe the SSS Term into the Emissive Slot instead ? Does this get overwritten by the Shadowing as well ?

You can't use the lightvector for anything in the emissive channel. It just acts like a cameravector when used.

[KazeoHin]

I don't mean to bump, but does anyone from Epic have anything to say about this?

I don't want this thread to be solely about the sub-scattering, this is more a problem with the fact that the artist/designer has no control over the dynamic self-shadows at the shader-level. Even in the beautifully rendered Borderlands game using the Retail license of UE3, the dynamic self-shadows on the characters look out-of place to the rest of the game's art style, I have a feeling that the developers of this game (and many others using UE3) have run into the same problem.