Cel-shading in the UDK
I'm trying to figure out how to do cell shading in UDK.
I found a pretty useful tutorial here.
But the only problem is I don't know how variable that is. What I mean is they don't show the result so I'm not exactly sure how it's suppose to look. Sincwe I'm not sure of the result, I don't know exactly what colors I have to change in it to get different colored cel textures. I'm guessing this is for a simple white one. ... or maybe it's for a blue one?... uh...??? I'm guessing (and hoping) so that way I can more easily change that around and maybe even add a texture sample to it as well.
I found a tutorial on adding outlines and it was pretty good.
Another problem is that I'd simply costruct this and see what I got, but the other problem is that the picture is split into two parts and I'm not exactly sure where they line up so everything in the top and bottom parts make sense, but I don't know where they join each other. Some lines cut out of the picture and I'm not quite sure what they connect to.
if only I could figure this out that'd be great. Has anyone created a shader like this?
I think there was actually one already made somewhere so I'll go get that one and see and post my results. thanks in advance.
Just tried slapping this puppy together. The connection lines do not match at all in the mat editor, there are 2 multiply nodes that have arrows coming in from the other pic, but don't have a starting point. what I did was hook the top multiply into the bottom multiply and put that into the diffuse. There was also one empty spot left at the top multiply's input - I threw in a vector 3 into it.
That gave me a nice control of the output color and the material looked nice in the mat editor. Once I put it onto a mesh, however, it looked nothing like in the editor... no outline, and parts of my mesh kept clipping out (based on the camera vector, I guess).
Yes, I had lights put in. There was no outline on the mesh, which was in the mat editor preview window. Once I ran lightmass, all the cell shading disappeared... weird stuff. Second tut I did with hourences (did a water shader the first time) and both failed.
I mean, this is, I guess, somewhat of a pro technique.... but shouldn't the purpose of a tutorial be to explain something, not confuse more? Why not give some description, a quick 'how-to' text, or post a full pic that is not cropped and we can see all the in-out-puts?
Yeah I saw a youtube tutorial that said how to do it and they had a post processing thing too.. but the site wouldn't let me in.
Here's someone doing it.
I can't find the tutorial I found, but you needed the post shader anyway so nevermind.
It'd be neat if there was a post proccessing shader that did it.
I KNOW you can do it because I saw it done... but I guess no one feels like sharing.
Saw that one already... sure looks good, but like you said, no sharing....
Here's what I got. Took me an hour to test and present properly, not too much work. I'm sure I got mistakes in my material, otherwise I would get proper results, but at least I can present my dilemma in a way that is clear, unlike the tutorial I worked off of.
This is a composite of hourences two images - you can see the nodes closest to root are not linking up. Arrows come from the other image, but have no starting point (or destination)-
Here's what I got:
First is the material editor preview. The BG is black, but you can see the outline going around the sphere against the grid -
Next is my material. It is a bit more complicated, but I circled the problem area with red. The node closest to root is the multiply node from the bottom half of hourences tut - it had an arrow running into it from the top, but no starting point in the top pic (see hourences image for ref).
The other multiply node had an arrow with no origin coming up from the bottom. I made a constant3vector and ran it into the empty multiply input.
This is the viewport result. I have a bsp sphere, and a barrel prop from the UDK database. I also have a bsp floor. The rightmost barrel was added after running lightmass, so it is lit 'realtime' VS the pre-rendered models on the left. The floor is also square (indicated by red lines), but had a funny clipping effect, due most likely to my bad setup for camera vector - since there is no proper visualization on how to link it in the problem area of hourences tut, it destroys the equation in the material.
If anyone has any ideas on this, it would help many of us, since there is a swarm of folk interested in learning toon shading in UDK. I know there is a tutorial on a TF2 character model. It has a great result and breakdown of the built material, but it does not have an outline, which is my particular goal.
Last edited by gerasimimumu; 05-05-2010 at 02:41 AM.
Hm. if only I could find that post proccessing shader that did it on everything automatically. Darn Thanks for all the help. The third object looks pretty correct. Is that one using the same shader? I wonder why that one looks right if the other ones don't. Strange. Hmm... I can try building this shader in your image, but I think I need some of the values you have things set to otherwise it won't work.
EDIT: Nevermind I think I can see them in the picture but they're a little blurry, I'll try them on a different color and see what I come up with.
The rightmost barrel is not rendered with lightmass - it is basically a crappier version of real time lighting, so it looks different.
All the nodes with a single number (e.g: .02, .75 or 1) are constant1vector. Just plug the number you see into the value of the C1V.There is one constant3vector hooked into the multiply, by the root. Plug those numbers into the values. The rest of them can be found by name. Just match it up.
Oh, and set your lighting mode to MLM_NonDirectional.
Keep in mind, this shader does not function properly, as it stands.... if you can figure it out, please post back, I'd love to get this sucker working as it should.
The only thing you've done extra in yours is multiply the outline by a pink colour. Usually, in Cel Shaded artwork, the outline is black as that usually has the most contrast against objects in the background and to give a feeling that the image was drawn by hand, granted it doesn't work that well against the empty space in the editors viewport.
As you can see in Hourences material, the final node is called "Final Expression", all that's happened is the author has moved the final expression down/up so that you can see it on both pictures.
Soo... since lightmass messies things up... can we just turn it off? or not have it run at build time... or go in the options and turn the value down so low that lightmass doesn't do anything? In ton shading, I don't even think there IS any GI at all.
Get the Dungeon Defence source files and see how they do the black outline with the post processing. You can copy the DunDefMisc package which holds the post process then just link to it from your DefaultEngine.ini.
Oh, ok... hmm guess I'll have to learn more about the inis ok thanks... Will this fix the shading issue with lightmass?
Ok, this is what I got after fixing it according to Piranhi -
I connected this with a multiply nodes to the diffuse MAT on one of the assets I made. The one on the right has normal map plugged in... it does not work well with normal maps. The one on the left if diffuse/spec/emissive. Both have precomputed shadows unticked (in the mesh's settings), since lightmass screws up the cell shading. It is not bad, although not what I was hoping for. The 'outline' gets screwy from some angles.
I will check out the source code on DD, since I would like to be able to utilize my normal maps, perhaps the post processing method would be better.
I will post more, hopefully it helps folks.
How do I link to the engine.ini?
Originally Posted by Neji
So... I found the files and how to link it. Unfortunately, when I run the editor afterward, I get this:
Ok, so I needed both dunDefMisc and dunDefPost + i figured out how to link in the .ini file
I got the result I want, which is saving all my diffuse/normal/spec/emissive textures, but applying an outline to the borders. I won't flat out rip off their work, but I need to dick around and understand how they put that post-process together. I don't want that built in blur and DOF effect, for example - so I will dig into it and try to make sense of that giant web they made.
For anyone who wonders, those errors (in the above post) don't affect anything, they will go away if you choose to save your file (become nullified) - with the exception of the outline post process, of course.
Thanks Neji and rest,
P.S. On a side note - I checked out DD and it looks like an interesting game. I did not like the 'touch edge of screen with pointer to turn the view' function, it tends to spin out, but hopefully they will polish that for final release.
im getting confused on how to actually open any DD file without my UDK crashing...
or which UPK file to copy
how did you end up doing it gera...(long name dude :P) ?
got it opening now, but cant see the post processing for the outline ?
Last edited by axelzellalex; 05-28-2010 at 07:04 PM.
The Borderlands Cell Shader is actually built into the Post Process Effects when you go into "game" mode, Just like bloom is in any other unreal game.
I know nothing about coding stuff to this extent, but I know for a fact that it has nothing to do with altering materials.
nOOb al3Rt !!
I got a similar effect by accident ROFL !!
cant I accomplish this using similar effects?
Sorry for bumping thread, but i need help
What i got is this
Take your diffuse texture and connect it to your diffuse power. It doesn't have black outlines, but it does give a very cartoony look.
Specular still behaves like normal, you may have to mess around with specular power.
Thanks, i'll try
Anyway, speaking about 1st page example - it works nice on meshes, but not for brushes - those rounded black are from a radius from player kills
Last edited by InfernalGrape; 10-27-2012 at 04:18 PM.