Blender - UDK FBX Tools Addon

[isathar]

My first addon for Blender is nearly finished and ready for public testing. It includes editors for vertex normals and smoothing groups, as well as a customized exporter that can export both and calculate tangents/binormals for any normals you choose to export.

This addon is basically the result of me reaching a barrier at a low-mid level of modeling skill and trying improve by learning everything I can about the technical side of modeling.


I will be adding more features to this over time, mostly presets for automatic generation. Let me know if there's anything you'd like to see added to the tool.


I just found out about the new Blender alpha supporting split normals and will look into taking advantage of that feature.


Download (Github)


Features:

Editor for Vertex Normals:

- Manual editing per poly or vertex
- Automatic generation with several presets for different scenarios
-- presets:
--- Smooth (Blender default)
--- Angle-Based (slow but customizable algorithm)
--- Up-Vector
--- Bent (facing away from 3d cursor)
--- Ground Foliage (selected ground based vertices point up, everything else bent from an offset point)
--- (TBD: Edge-Based and Smoothing Group-Based)
- Allows calculating normals for selected faces or the whole mesh
- Normals can be displayed as lines for visual editing
- copy/paste selected normals between meshes with identical vert locations (buggy/wip, but fixes modular mesh seams when it works)
- Real-time in Edit Mode


Editor for Smoothing Groups: *experimental*

- Manual editing of smoothing groups per face
- Groups can be displayed as numbers on corresponding faces
- Real-time in Edit Mode


Customized FBX Exporter:

- Can export everything from the above addons
- Can calculate and export tangents and binormals
- Optional support to export normals generated by asdn's Recalc Vertex Normals addon
- UDK-specific optimiztions:
-- b_root is now exported as root bone instead of the exporter creating a new one
-- limited axis flip settings to things that make sense and labeled them
-- tangents are very close to UDK's



Notes:

Display:
Custom normals are not actually displayed on meshes since the variable needed to display split normals in Blender is currently read-only. They can be displayed as directional lines with options in the editor panel, however.
The tools do not take transforms that have not been applied into account, so make sure you do that before generating or trying to display them.

Performance:
If displaying normals or smoothing groups is enabled on a complex mesh with thousands of polys, things will slow down. Choosing to display them on only selected faces helps with this, but I'm looking into optimizing speed in later versions.

xNormal:
Tangents/binormals exported by this addon are read by xNormal, allowing you to take advantage of synced normal map baking.


Exporting Tangents:
Triangulation is required if you don't want potential shading errors. While it's possible to run the algorithm on a mesh with quads, it's not recommended.
Make sure your UVs are matched to the mesh, as in, you have made no changes like splitting edges or merging vertices since you unwrapped your UVs. To match your existing UVs after little changes like that, do the following in Blender:

- Go to Edit Mode, select all of your mesh
- Open the UV Editor, Select all UVs
- in the UVs menu, click
- UVs -> Seams From Islands
- UVs -> Pin
- UVs -> Unwrap
- UVs -> Unpin

Custom Angle-Based Normals:
This feature is a work in progress and is currently broken on complex meshes. It works well with cubes and faces that are pointing in somewhat the same direction, but things get messy when trying to use it on cylinders or spheres.

Generating normals on selection:
This feature is also a work in progress, and will mess up the custom mesh data's vertex order if used with the 'Smooth (Default)' setting.

(Screenshots and basic documentation in the next post)

[isathar]

Usage:

Main Panel:

Export - opens the export menu for the included fbx exporter

Initialize - creates new custom mesh data object for selected mesh

Reset - reset's custom mesh data to default
Clear - deletes custom mesh data

Enable/Disable (show/hide editors)

Match Grid - sets grid scale and subdivisions to 16 to match the grid to UDK's
or resets them to default



Exporter:

Axis flip - Static mesh for default axis, Skeletal Mesh for Y+, Z+
Smoothing - 'Groups' will export smoothing groups from the editor,
'Off', 'Face', 'Edges' behave like default versions
Normals - Default - Blender generates normals
- From Editor - Custom normals from the included Vertex Normals Editor
- asdn's Addon - export normals from the Recalc Vertex Normals addon

Tangents + Binormals - calculates and exports tangents + 'binormals' for the mesh.

The animation options weren't altered.



Vertex Normals Editor:



To enable:
- Go to the UDK FBX Tool panel, click Initialize
- Click Enable under 'Vertex Normals'


Manual editing:

Poly:

- select a vertex or set of vertices on the same face,
- set the face index in the textbox below the normal coordinate,
- set the x,y,z, normal coordinate to whatever you want,
- click set

Vertex:

- make sure 'Edit All' is checked,
- same steps as per-poly, but face index is not used

checking 'Real-Time' will change the normal as you change the coordinates in the box


Automatic Generation:

-select a mode from the dropdown menu.
- overview of modes:

- 'Smooth (Default)'
- generates Blender's default normals + copies to custom data

- 'Up-Vector'
- normals point up, early experiment into grass lighting

- 'Bent'
- normals point away from 3d cursor location
- good for tree foliage when used with 'Selected Only'

- 'Ground Foliage'
- good for grass planes - normals on selected vertices point up, everything else away from 'Center Offset'
- lowering the Center Offset's Z-value will lower the amount by which upper normals are bent, making the shading look less 'round'
- if 'Ignore Hidden' is set, you can hide parts of the mesh and generate only for visible faces
- This mode produces the best results with a two-sided mesh, but a 2-sided material will also benefit.

- 'Custom (angle-based)'
- a customizable algorithm I'm working on to generate normals based on dot product thresholds for face angles
- 'Smoothing Threshold' is the maximum dot product-based difference between the angles of normals on the same face. (0.99 is flat shading, -0.99 is smooth)

- 'Edges' and 'Smoothing Groups' settings do nothing yet and will (hopefully) work in a later version.

checking 'Selected Only' will generate normals for selected faces (this is still buggy with the 'Smooth' and 'Custom' algorithms).


Transfer Normals:

NOTE: This is a work in progress feature and may mess with the vertex order. Pasting overlapping vertices' normals with this will generate errors.

Copy - copies selected vertex normals to buffer
Paste - pastes buffered normals to vertices found at the same position

The points of this feature are to reduce seams on modular character meshes and to allow copying of sharp edges from identical meshes with edge splits.


Display:

Options for drawing lines to represent the vertex normals (color, scale, selection only)


Smoothing Groups Editor:



*Note This tool is experimental, since the exporter currently does not create normals/tangents to match the groups. UDK will automatically generate normals based on the groups if you import the mesh with 'Import Tangents' and 'Explicit Normals' (for static meshes) unchecked, but xNormal won't. I'm looking into ways to generate normals from smoothing groups.

To enable:
- Go to the UDK FBX Tool panel, click Initialize
- Click Enable under 'Smoothing Groups',

to set faces to a specific group:
- select the face
- change the number in the editor's textbox
- click 'Set'

to select all faces in a group:
- deselect everything
- set the group number you want to find
- click 'Select Group'

Group 0 is for flat shading (which you probably won't be using)
Groups 1-32 are smooth

to show currently set groups, click 'Show' under Display. Group numbers should be drawn on corresponding faces.

[isathar]

Example shots
(just two for now, more soon)

Grass normals comparison:

Basic (2 plane) grass with a masked Phong material and some transmission:

Light behind camera:


Light in front of camera:

[isathar]

Notes on UDK import settings:


'Import Tangents' will import tangents, binormals and normals from the source mesh as long as all three are found. If UDK can't find tangents/binormals in the source file, it will generate all three automatically.

When importing a static mesh, 'Explicit Normals' is redundant if 'Import Tangents' is checked. If tangents aren't imported and 'Explicit Normals' is checked, they are generated from imported normals.


- Use 'Import Tangents' for skeletal and static meshes if you have a source file with tangents included. You don't need to check 'Explicit Normals' for this to work on static meshes (I think).

- Use 'Explicit Normals' for static meshes if you have custom normals but didn't export tangents. UDK will calculate its own Tangents on import.

- If you want to try out smoothing groups, uncheck both 'Import Tangents' and 'Explicit Normals' and UDK will calculate its own from groups it detected.



Examples:




Exporting tangents with default normals: *Not required when using the binary fbx exporter in the Blender 2.71 test builds

In Blender:

Click Export in the UDK FBX Tools panel,

use these settings:
Smoothing - Face
Normals - Default

check Tangents + Binormals


In UDK:

check Import Tangents
(explicit normals are not required when using this setting)



Grass + ground foliage Normals:


In Blender:

Select FBX Tools tab, in main panel click Initialize Data,
then click Enable Custom Normals,

Select the vertices in your grass mesh that will be touching the ground,
play around with the center offset value to change the amount of bending,
(keep the z-value negative for the shading to make sense)

Go to the Vertex Normals panel,
under Auto Generation, click the drop-down box and select 'Ground Foliage',
click Generate

(Show Normals under Display to show lines)

Click Export in the top of the UDK FBX Tools panel,
Select From Editor from the Normals dropdown,

Export with:
Smoothing - 'Face'
Normals - 'From Editor'
optionally with tangents + binormals


In UDK:

if you exported with tangents + binormals, check Import Tangents,
otherwise just check Explicit Normals



Tree Foliage (Bent Normals) Export:

In Blender:

Select FBX Tools tab, in main panel click Initialize Data,
then click Enable Custom Normals,

select a cluster of foliage planes from the tree,
move the 3d cursor to the point you want to be its center,

Go to the Vertex Normals panel,
under Auto Generation, click the drop-down box and select 'Bent',
check 'Selected Only', uncheck 'Reset First', click Generate
repeat for each cluster

Export with:
Smoothing - 'Face'
Normals - 'From Editor'
optionally with tangents + binormals


In UDK:

import with 'Import Tangents' or 'Explicit Normals' checked



Exporting Normals from asdn's Recalc Vertex Normals addon:

Note: I included support for this addon since it's what got me started working on this.

In Blender:

- use asdn's addon to create your normals, make sure you click 'Save' (and maybe check 'AutoReload') at some point
- click 'Export' in the FBX Tools panel
- select 'asdn's Addon' in the Normals dropdown
- export with or without tangents

In UDK:

- import with 'Import Tangents' or 'Explicit Normals' checked, depending Blender export settings and mesh type



xNormal support:

use an fbx file generated by this as the low poly mesh,
make sure 'Use Exported Normals' is selected under Smooth Normals

[mAlkAv!An]

Hi there,
first let me say thanks for your tremendous work. This addon is obviously awesome and I truely appreciate the fact that it's well documented.

I strongly suggest to post this thread at the blenderartists forums as well if you haven't already. There are lots of users who are waiting for the possibility to (edit and) export custom vertex normals for a long long time, so everbody will be very grateful.

[isathar]

Hey,
Thanks for the feedback, I'm glad you like it.
I was planning on posting it around a few places yesterday (including BlenderArtists.org, but I'm still under new user restrictions there), but it was pretty late by the time I got done organizing everything. I was actually pretty surprised how simple it was to get them to export (maybe not following best practices and all, but it works so far). The documentation probably took longer than getting that part to work

[mAlkAv!An]

I've tested a few grass and bush meshes and they'll work fine in both, UE3 and UE4.
Do you plan to add an option to cast/transfer normals from one mesh to another? Similar to this one: http://blenderartists.org/forum/show...Vertex-Normals
I'm also not sure if the 'Custom (angle-based)' generation method is working correctly or if I just didn't get the idea of how it's supposed to work.

[isathar]

Nice, good to know it also works in UE4.
I actually have some halfway working code for transferring normals between shapes like that working with tilted planes at the moment. I'm seeing how far I can go with that one and may have something this weekend.
The Custom method still needs a lot of work. It only seems to work on simple meshes with nearly flat faces (like cubes or selecting a UV island with faces pointing in the same general direction), but breaks down on anything more complex.
*EDIT: Looks like I did something stupid with the first smoothing pass, I'll have a fix up for it tonight or tomorrow.
**EDIT2: I haven't been at my PC in a couple of days. I'm looking at it again now, but I'm not quite sure what's going wrong after removing the first pass. I'll update when I figure it out.
***EDIT3: Fixed it and it will be up later today.

There is also a bug with vertices being seen as selected when they shouldn't be while manually editing normals.

I'll have something up by Sunday with tweaks to the custom algorithm and some overall bugfixes. I'm not sure if transferring normals will make it in, but we'll see.

[isathar]

I just uploaded a new version with the following fixes:

- fixed Custom (Angle-Based) auto-generation mode. It's slightly slower (as in, it can take about 5 min for a 3600 poly mesh), but should no longer produce weird results on complex meshes.
- added check for UV layer before calculating tangents. If not found, the mesh is exported without tangents (default behavior).
- fixed wrong vertex being selected while using manual edit (bad math).

I'll work on fixes for the copy/paste function next since it apparently skips a few vertices while pasting, then some speed optimizations. Transferring normals between different shapes didn't make it into this one, but I'm confident I can get it working soon.
EDIT: It's not quite as staightforward as I thought... It may still make it into a future version if I can pull it off.


Thanks for the feedback, mAlkAv!An. I probably wouldn't have realized how completely broken the angle-based algorithm was for a while if you hadn't pointed it out.
After reading the code again, it turned out the first smoothing pass was not only redundant, but being applied to the wrong faces...

[mAlkAv!An]

Just giving this a little bump. Did you have any time to further work on this nice addon?

Here's some foliage exported to UE4 using the 'bent' and 'ground' foliage options,
http://abload.de/img/foliage_001a2bj0z.jpg

[isathar]

Nice screenshot, it's good to see the results on a proper tree mesh.

I haven't had as much time as I'd like to spend on this, but I'm still working on it. I haven't added any new features, just tweaks and expansions to what's already there. I'm planning to rewrite the exporter to use mesh loops like the newer versions (no time estimate on that one).

A new version should be up tonight or tomorrow with a few changes: I've tweaked the Bent auto-generation method to account for back-faces on two-sided planes, rewrote the Up Vector mode to use an input vector (so I guess it needs to be renamed to something like Arbitrary Vector now), and doing some work on the copy/paste functions today. I'll also have to update the documentation to include that in order to import custom normals for LOD meshes, the mesh has to be imported twice for some reason.
The smoothing groups part of the addon isn't working properly (and is kind of redundant), so that's getting disabled in the next version.