Results 1 to 14 of 14
  1. #1
    MSgt. Shooter Person
    Join Date
    Nov 2009
    Posts
    55

    Default High fidelity terrains with supermasks and normal maps.

    External Alpha masks

    Supermasks


    Above are the two links from the UDN on the topic. I've implemented the above using some external tools and quite a bit of experimentation so I want to outline the approach along with some of the pluses and minuses that it represents.

    The approach is to create a heightmap using an external tool at a high resolution, along with all of the accompanying data. This includes high fidelity normal maps as well as texture maps that can exceed the normal amount of data a smaller map can contain.

    As my example I'm using a 512*512 Set of data and applying it to a 64*64 terrain. This technique allows you to increase both the texture fidelity and the normal map fidelity by 4096(64*64) times.

    Please keep in mind that my programmer art ranks poorly even among programmer art and I've been heavy handed with the artwork itself to highlight the detail. A proper artist using this set up will have better looking results with just a little bit of effort.


    Creating the heightmap

    I've used two different tools for creating the heightmap.

    The first is L3DT for creating the shape and polygon level editing to make the map playable.

    For additional detail and for their great visual effects editor I used World Machine 2

    The reason I used both is that World Builder either lacks the ability to do polygon level editing or else I lack the knowledge to make it do so. What it does have is a great editor that implements an interfacer similar to kismet and the material editor. This makes it a breeze to capture all of the data you need for normal maps and supermasks, as well as some great procedural effects you can add to your terrain right out of the box.

    The first gotcha is to make sure that your heightmap is a power of 2 +1, so in my case I created a 513*513 heightmap.

    The heightmap I created using both tools is here.



    Height Map



    Creating the normal map:

    I'm not going to go in to great detail on setting up World Machine to create the maps you need but here is the general idea.

    For the normal map you simply connect the normal map maker node to the output of your heightfield and then connect a bitmap output to the normal map maker.

    Here's the normal map



    Normal Map


    That step yielded the normal map above. For the erosion detail you add an erosion node and grab the heightfield it creates with another normal map maker and then combine them with a combiner node. (Even as over-detailed as this erosion map I'm using is I culled out 85% of the detail with a selector node). As with all output you capture it in a bitmap output node.

    This is what my normal map looks like afterward, note that I inverted the normal map because I wanted the erosion channels to be depressions instead of bumps.



    Now we're adding some great detail. For a production quality map we'd definitely want to tone this down but for the purposes of showing off the technique I kept all of it.


    Creating the supermask:


    Creating the supermask in World Machine is essentially the same technique. You need to set up selector nodes to grab the parts of the terrain you want each texture to be on and then push them into a channel combiner. You then capture the output in a bitmap output node.

    Here's what the packed supermask for my map looks like.



    Packed Supermask

    Each channel of the map is really just a heightmap that we'll later tell the engine to interpret as an alpha mask. Because this type of map is often called an alpha mask or alpha texture, it's important not to confuse the terminology. This texture is a 24 bit bitmap, an R8G8B8 texture, which means 8 bits per channel, with 3 channels 1 red, 1 blue, 1 green.

    So despite all of the color, it's helpful to think of it as 3 grayscale textures all packed into one texture. Where white is 100% and black is 0%, with up to 8 bits(0 to 255) of grays in between. Later we'll set up the material so the engine knows which texture to draw at which color and how much to blend for each pixel.

    For now all you need to know is that when you import the texture into Unreal you import it as a "CompressionNoAlpha"

    The reason you may want to consider supermasks, especially for smaller maps is discussed in the UDN.

    The current terrain layer painting system is limited to the resolution of the terrain mesh vertices. The number of terrain mesh vertices is the same as the underlying terrain heightmap, so each layer weightmap is therefore the same resolution as the heightmap. In other words, a 256x256 patch terrain which contains an underlying 257x257 heightmap allows for a layer painting resolution of 257x257 discreet blending pixels. While this is fine for the majority of terrain layer painting detail, there are instances where a finer degree of texturing may be desired. Note that a weightmap is often also referred to as an alphamap. It provides a similar functionality within the terrain layer blending system as an alpha channel in a 32-bit image or a paint application such as PhotoShop.

    ...

    To put this into perspective and scale, many common two lane highways are constructed of two 3.5 meter lanes plus additional shoulder width. So in comparison, a single painted vertex on a 256 DrawScale terrain is approximately the same width as a two lane highway. This is often too wide for terrain layer detailing such as foot-paths, ditches, creeks, etc. Choosing to solve this layer weightmap size issue by decreasing the terrain DrawScale3D may result in a terrain that is too dense and impacts the framerate due to the greater number of patches being rendered in the frustum.
    For my 64*64 vertex example choosing a single vertex to paint means the smallest detail, like the footpath example, has to be 14 meters(almost 46 feet) wide. This technique allows a detail level that brings the same footpath down to as small as .28175 meters( .717 feet)

    I didn't really exploit the texture detail as fully as it could be done but if you've tried painting detail on a 64*64 map you should be able to appreciate even my amateur texture maps using this technique.


    Getting it to render:


    Time for your next tool:

    HMCS V200

    This is a great piece of software. It does just a couple of things and it does them well. It will let you downsample your large height map you created all of your detail on to whatever size you plan on making your terrain. Keep in mind that Unreal expects a power of 2+1. It also will convert your 8 bit grayscale heightmap to the G16 format that Unreal requires for building terrain.

    In my case I went from 513*513 to 65*65.

    It also allows you to constrain and adjust your altitudes. I found that using the toolchain I've outlined my altitudes were between 0 and 65000+ meters, which means your map is going to look like a donut unless you make some adjustments. The map in this example varies from 0 to 300 meters. You can make these adjustments at any point in the pipeline but I've found HMCS to be the easiest route.


    Importing your textures:


    Now we need to get all of our textures into the editor. There are a couple of things to watch out for here. If you're using the same toolchain I've outlined the first thing to know is that World Maker builds it terrain starting at a different corner than Unreal. This means you need to flip vertically either your heightmap or else the normal and super mask.

    This can be done trivially in any pixel-pusher application. I used Gimp for mine.

    In this example every texture I'm importing except for the heightmap is a 24bit bitmap. The normal map, super mask, and all of the terrain paint should be imported using the "CompressionNoAlpha" option enabled. If you want to use a 32 bit bitmap because you're packing 4 planes instead of 3 then you probably know how you want to import it yourself.



    The second thing to look out for is the editor expects your normal map and super mask to be in a power of two while your toolchain has generated a power of 2+1 texture. Use your same pixel program to scale your maps down to a power of 2. Remember that your heightmap must be a G16 format and a power of 2+1.

    At this point you may as well import the textures that are going to be painted on your terrain. In this example I used 256*256 textures. Upping the texture size to at least 512*512 for production quality maps may be a good idea.

    I have 3 textures I'm painting for convenience sake. You could just as easily use 5 or 6 textures either packed or unpacked in a super mask. My textures are the grass, the mountainside, and the red dirt, which is supposed to look like a rust-colored sediment to detail the erosion layer.

    Here's the material set up for the supermask shader.




    The supermask link at the beginning of the tutorial presents the material set up and is easier to read so I'd suggest reading that article before you hurt yourself squinting at my screenshot.

    You'll notice some connectors running off the screen below. I'm using the supermasks to blend some texture normals into the terrain normals. I'm not satisfied with the results though so for the purposes of this tutorial we're just going to plug our normal map above straight into the normals input.


    Setting up the editor:

    We need to change a few default settings in the editor. The first thing you should do is create a dominant directional light and change its settings. For this example I upped the brightness to 2.0 to show off the technique and added a tiny bit of gold color. The most important setting to change is the Shadow Projection Technique.

    I'm using ShadowProjTech_PCF, for Percentage Closer Filtering with a low Quality Shadow Filter.


    The next setting you need to change is under the Terrain sub-category of the terrain properties. It's called Normal Map Layer and has a default setting of -1. This is a flag that tells the terrain where you're feeding it the normal map for the terrain. -1 means to use the internally generated normal map. this setting is 0 for me, which is the index for the material where I'm feeding the engine the normal map. If you set your terrain up like I have you'll only have one material so the index will always be 0.


    To be developed:

    The shortcomings of this technique thus far seems to be the proper way to add texture normals to the pipeline. I'm currently using the supermask to paint texture normals that match whichever texture is being drawn at that specific point. For noisy textures this method adds some detail but if you have a clean uniform texture like a brick road I'm not certain how you'd line the texture normal maps up to match the diffuse texture.

    Further, because the normal map you're blending with is for the entire terrain, you need a great deal of detail in the texture normal map in order for any of it to translate to the final normal map.

    If anyone extends this technique and solves the problem I would appreciate you sharing it.


    That's it:

    I went into a lot of detail and outlined some of the gotchas that cost me a few hours to figure out. The technique is pretty simple to implement and adds a great deal of detail to the terrain at a very low cost. The material set up uses 7 texture samples, although you would likely want to add a detail texture to the mountainside, so let's call it 8.
    31 instructions in the vertex shader and 41 instructions in the LightEnv shader. The shader complexity map is a nice darkish green with no trouble spots.

    For a real feel for the issue this technique solves create a quick 64*64 terrain and try to duplicate painting the sediment texture on a mountainside. For that matter try to duplicate the blend between the grass and the mountains. To achieve the same blend you're making a 512*512 map and sacrificing enough frames per second to run all of your game AI, or some really nice physics.


    Screenshots

    All of the screenshots use the editor settings I outlined above. Resolution is 1280*1024. Lighting quality is in preview mode.






    If you look closely you can see the rivulets in the mountainside.


    Here's a shot with the sun at a 45 degree angle to maximize the normal detail. I had to do a regular screen cap to get the right perspective.





    I added some additional screenshots below which include my texture normal implementation.
    Last edited by anomandarius; 06-17-2010 at 04:27 PM. Reason: updated for tutorial tag. I need to port this to may and repost the shader set up.

  2. #2
    God King
    Join Date
    Aug 2008
    Location
    Location,Location
    Posts
    4,072

    Default

    This is amazing although a little confusing because the UDN makes things complicated :]
    i understand it though.

    Does it take a lot of time to implement correctly?

  3. #3

    Default

    it looks amazing but i not understand the UDN

  4. #4
    MSgt. Shooter Person
    Join Date
    Feb 2010
    Posts
    334

    Default

    hmm good find on that. and the fact i acctualy understood it. but then i been doing this sort of thing with other things.. i wonder if vue8 would work with that.

  5. #5
    MSgt. Shooter Person
    Join Date
    Nov 2009
    Posts
    55

    Default

    These shots are taken with my attempt at adding texture normals. I'm not convinced my implementation is correct as I outlined above.









    This pic shows the detail you can cram in to a small map using this technique.


  6. #6
    MSgt. Shooter Person
    Join Date
    Nov 2009
    Posts
    55

    Default

    Quote Originally Posted by TheAgent View Post
    This is amazing although a little confusing because the UDN makes things complicated :]
    i understand it though.

    Does it take a lot of time to implement correctly?

    If you know the tools required to make the heightmap and normal maps it's a breeze. It was a chore hunting down some of the tools, especially the HMCS Version 2, and then wrestling with a few of the gotchas, like the terrain being built from different corners and the damned Normal Map Layer flag, which damn near drove me to drink.(Not that that's difficult).

    If you do the step by step I'd say your first attempt will be pretty easy.

    I'd really like to see an environmental artist with some skill knock out a 256*256. The coolness of the screenshots are the technique and World Machine's awesome erosion maps. The textures are by way of google.

  7. #7
    God King
    Join Date
    Aug 2008
    Location
    Location,Location
    Posts
    4,072

    Default

    Well i am an artists at heart and i used hieghtmaps before as well as the HMCS tool :] so thanks alot for the above tutorial, i used to texture the height maps by hand and the whole normal map this is really cool.

  8. #8
    Iron Guard
    Join Date
    Feb 2010
    Location
    Germany
    Posts
    501

    Default

    hey anomandarius
    a friend of mine is currently setting up a tutorial homepage for UDK tuts...may we could use your tutorial? of course we'll name you and link to this site...
    see more here: http://forums.epicgames.com/showthread.php?t=721760
    /ninjapirates/
    ---Roccy Laboratories//#1 Crossmedia 2010[Multimedia]

    I hate reference images...I never find good ones

  9. #9
    MSgt. Shooter Person
    Join Date
    Nov 2009
    Posts
    55

    Default

    Quote Originally Posted by HavocInferno View Post
    hey anomandarius
    a friend of mine is currently setting up a tutorial homepage for UDK tuts...may we could use your tutorial? of course we'll name you and link to this site...
    see more here: http://forums.epicgames.com/showthread.php?t=721760
    Feel free.

  10. #10
    MSgt. Shooter Person
    Join Date
    Nov 2009
    Posts
    55

    Default

    I'm bumping this old tutorial for a couple of reasons. I'm still seeing low fidelity terrain in the showcase section

    and also I'm wondering if anyone is using this technique, and if so have you solved the texture normals issue?

  11. #11
    MSgt. Shooter Person
    Join Date
    Jan 2007
    Location
    Shanghi, China
    Posts
    87

    Default

    Quote Originally Posted by anomandarius View Post
    I'm bumping this old tutorial for a couple of reasons. I'm still seeing low fidelity terrain in the showcase section

    and also I'm wondering if anyone is using this technique, and if so have you solved the texture normals issue?
    What texture normals issues is that?

  12. #12
    Iron Guard
    Join Date
    Jul 2010
    Location
    Texas
    Posts
    722

    Default

    Although you didn't post your Normal shader, it looks like you were using the right parts anyway. The Normal shader should look very similar to the diffuse shader.

    The exception is that you need to multiply every normal by 1.0, 1.0, 0.0 before you do anything with them, or just mask them to get rid of the blue channel. Normals and their detail normals tend to look better added together instead of multiplied to ensure strong contrast. Lerps work exactly the same to lerp between two normals. The final step before plugging them in to the Normal input should be to add the blue channel from one of the Normal textures back into it so that it's a complete normal again.

    As long as you use the same TexCoord nodes and such for the diffuse and the matching normal, they'll line up fine.

    EDIT:
    If you mean that you want to take the normals from the textures and combine them with the full terrain normal map that you made, you'd do it exactly the same for the individual texture normals, then add your map normal to that. I believe that you can properly size your full map normal 1/(MapSize). So if your map is 16X16, youd use a TexCoord set at 1/16 or 0.0625. By default, it tiles at 1 per terrain section... I think.
    Last edited by Wyldhunt; 09-30-2010 at 05:44 AM.

  13. #13
    MSgt. Shooter Person
    Join Date
    Nov 2009
    Posts
    55

    Default

    Been out of the UDK action for some time but thanks Wyldhunt, adding texture detail normals was indeed that simple. I fixed this issue quite some time ago and just never corrected the tutorial. Still seeing low fidelity terrains in the showcase section btw

  14. #14
    MSgt. Shooter Person
    Join Date
    Feb 2011
    Posts
    214

    Default

    I'll have to do something about that then.

    While I'm working on a project that I can't get into many specific details on, I plan on using UDK to render an image of a map, while implementing the "Swiss-style shaded relief" (as detailed by Bernhard Jenny, Prof of Geo Visualization @ Oregon State).

    I've been painting details over DEM/GIS data, and making heavy use of GRASS (an open-source GIS system), and L3DT (bundysoft.com) very, very heavily to prep my terrain. I haven't gotten to the point of downsampling and generating normals yet (I'm still sculpting and generalizing). Generalizing of terrain is a very interesting subject, you can read about it here. I'm also playing with TerrainEqualizer (by the aforementioned Professor), an interesting little Java app that lets you scale the different "detail levels" of a terrain, using a Laplazian-pyramid model.

    I remember seeing some of this on your blog anomandarius, didn't know you had posted this here. I have been sketching out a similar workflow and it's great to compare notes. Thanks for bumping it. It's good to see that the Normals problem was solved (I had run into the issue of adding normals somewhere seperately, good to see Wyldhunt jump on that too). I'm also excited to see others using L3DT with UDK, as I think these are a very powerful combination. I'd love to start detailing some of my workflow... yet that would require a separate project, atm.

    So, when I can catch a breather from this current sculpt (4096x4096... 700m per pixel .... yeah, it's a campaign-level map), I'd love to work on a smaller scale location (with maybe some generalized background terrain). Anyone else up for collaboration?
    Last edited by joshua.fontany; 04-18-2012 at 03:10 AM.


 

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Copyright ©2009-2011 Epic Games, Inc. All Rights Reserved.
Digital Point modules: Sphinx-based search vBulletin skin by CompletevB.com.