Difficulty Using Render Targets with ActionScript 3

[Nagoshi]

I'm working on a HUD that uses a combination of a SceneCapture2DComponent, TextureRenderTarget2D, and SetExternalTexture in order to display a live camera view as part of the HUD. The scene capture component is attached to a socket on a vehicle, and the idea is that the player should be able to drive the vehicle using only the video on the HUD for reference. For the most part it's working, except that depending on how I add the MovieClip containing the target .png to the stage in Flash, I may or may not be able to capture the entire environment. Both the MovieClip and .png are set up for Export for ActionScript and Export in Frame 1.

I was able to recreate the problem on my home machine using a fresh install of the 08-2014 build (please excuse the boring default scene!). In the picture above, both boxes on the right are being replaced by the same texture from UDK. The one on the top is exactly the effect that I want, because it can show everything as the scene capture component sees. This MovieClip was simply added to the stage from the library and is there at all times. I've adjusted the alpha xA+ to 255 to resolve the transparency issue (as per these instructions: https://udn.epicgames.com/Three/GFxU...erTargets.html )

The bottom box is the exact same MovieClip, but it is meant to be added and removed via ActionScript for different uses. I also just like to keep everything off the stage and add it with script when it's needed. I add the MovieClip to the stage, and also set it's alpha to 255. But I can't seem to get it to show the entire image, even though the render texture in UDK shows the full image with skydome when I exit PIE. Keep in mind that it is also receiving the same texture as the top box. I've played with the variables for the scene capture component and can get different results, but not the one I want. Another minor bit of frustration is that not all of those variables seem to have an effect. For example, you'll see in my script below that the frame rate is meant to be 2, but the scene captures just as quickly as the game's framerate allows. It's not as important, but would be nice to have for the sake optimization.

At this point I think my main problem has been narrowed down to how I add the MovieClip in Flash, and not something from UDK directly. I do change the Target Gamma in the TextureRenderTarget2D to 4.0, just to lighten up the image, but as long as the texture seems to be capturing correctly (proven(?) by viewing the render texture after exiting PIE), I don't understand why the same texture should display in different ways in Flash depending on how it's added. I'm guessing it may be something really, really silly... ^-^'

Just in case there's something helpful, here is the main code used to make the system work for this little test. There may be a few gaps, but this at least got me the screenshot above.

ActionScript 3 (Used only for the bottom box, addCameraCaptureDisplay() is called when it's time to add the MovieClip. Maybe there is something missing here?)

var CameraCapture_Display = new CameraCapture_Display_MC;

function addCameraCaptureDisplay()
{
	addChild(CameraCapture_Display);
	CameraCapture_Display.alpha = 255.0;
}

UnrealScript (for the vehicle)

var SceneCapture2DComponent CameraCapture;

simulated function PostBeginPlay()
{
	Super.PostBeginPlay();

	Mesh.AttachComponentToSocket(CameraCapture, ‘CameraLens’);
}

simulated function Tick (float DeltaTime)
{
	Super.Tick(DeltaTime);

	if(Controller != none && OSC_HUD(PlayerController(Controller).myHUD).HudMovie.CameraCaptureTexture != CameraCapture.TextureTarget)
	{
		OSC_HUD(PlayerController(Controller).myHUD).HudMovie.SetRenderTexture(CameraCapture.TextureTarget);
	}
}

defaultProperties
{
	Begin Object Class=SceneCapture2DComponent Name=CameraCaptureComp
		TextureTarget = TextureRenderTarget2D’OnScreenCamera.Materials.TRT_CameraView’
		FieldOfView = 110.0
		NearPlane = 10.0
		FarPlane = 0.0
		MaxViewDistanceOverride = 0.0
		FrameRate = 2.0
		bEnablePostProcess = true
		bEnableFog = true
		ViewMode = SceneCapView_Lit
	End Object
	CameraCapture = CameraCaptureComp
}

UnrealScript (for the HUD Movie Player)

var TextureRenderTarget2D CameraCaptureTexture;

function SetRenderTexture(TextureRenderTarget2D SetCameraCaptureTarget)
{
	CameraCaptureTexture = SetCameraCaptureTexture;

	SetExternalTexture(“CameraCapture_Target”, CameraCaptureTexture);
}

defaultProperties
{
	CameraCaptureTexture = none
}

Any thoughts are welcome! This is bothering me more for the sake of curiosity than anything else. XD

[Doublezer0]

var CameraCapture_Display = new CameraCapture_Display_MC; // this might be the problem

try

var CameraCapture_Display = new CameraCapture_Display_MC();

edit::

came back to see if u fixed it. Noticed the var type wasn't set.

var CameraCapture_Display:CameraCapture_Display_MC = new CameraCapture_Display_MC();

[Nagoshi]

Thanks for taking a look, Doublezer0. I tried your suggestion, but had no change. It's been awhile since I've properly used Flash, and I should have mentioned that I am working on the timeline and am only familiar enough with ActionScript to cause some trouble.

Since I accidentally found out that adding the MovieClip to the stage directly from the library gives the effect I'm after, I've gone ahead and implemented that in my main work files. I'm still curious about why it works differently though. Also, I still cannot change the framerate at which the scene capture component updates. It's not a game-breaker, but it would be nice to be able to save some resources by knocking that value down a bit. If anyone has any more ideas, I'd love to hear them even though the original issue was worked around.

Here's a link to this barest-of-bare-bones test if anyone would like to tinker with it. If you play the OSC_Test map, you should see two green squares. After entering the Scorpion, both should display video from the Scorpion's point of view. The top square was dragged to the Flash stage and works (this is what I ended up going with), and the bottom square is added in script on frame 1 of the timeline, and it has some display issues. All of the Flash files are in the UDKGame\Flash\OnScreenCamera directory.

[frankit]

Fascinating! Is this multiplayer? Is that why you didn't just attach an RTT plane to the player / vehicle camera or am I missing something? So even if the RTT actor is set to two frames the Flash HUD continuously updates??? Just an idea, but why not attach a dedicated RTT plane to the camera itself on the left-hand-side as a sanity test and compare the results. BTW: I think this is more an unrealscript question than scaleform, plus it might get more views there...

[Doublezer0]

This is a problem with your flash file.

Things I noticed are:

1. You haven't set the Flash file properties correctly for Scaleform. Unless an update occured that I'm not aware of you need to set the Flash Player to 10.3 for actionscript 3.0 files.

2. You wrote directly on the timeline which is a big no-no in actionscript 3.0. This is fine in as2.0 (which most tutorials go by) but not in as3.0 as it uses a class based system with a document class for the flash file. The only code I ever put on a timeline now is stop(); Which leads to 3.

3. Your timeline was looping because you didn't tell it to stop. This would cause issues.

4. I tend to setup my flash files with "Automatic Declare Stage Instances" switched off. It means I have to declare a variable for any added clips but that also helps me keep track. It also interferes with Scaleform CLIK somewhat and as I do interfaces there is no real need here. Food for thought though.

5. Image folder was not the same name as the SWF file. This is a must in scaleform.


Anyhow I rebuilt your flash file the correct way. Please note the following :

- First things first. You will need to update the class directories in the flash file to suit your filesystem. Access this by clicking on empty space in the flash file. In properties > Script - Actionscript 3.0 settings. Here you will see the folders to update. Follow what is there as an example. If this is not set properly the file won't see the class files that it needs to function.

- I put my class files in com.f00n.ProjectName as standard. Safe to rename if you update the folder setting as above.

- I haven't messed with any unrealscript or anything. I will presume that is already correct.

- I gave the Camera MC it's own class so you can do things to it if you want. BitmapData is fun

File : http://www.f00n.com/random/OnScreenCamera.rar


I saved it Flash 5.5 because your old file complained and im using cs6. If you need it in another format let me know.


Just because its not a lot of code i'll post the class files here for others to see as example. It's not like it's NDA material ;P

---> Document Class file [ HUD_OnScreenCamera.as ]

package  {
	//import classes that are used.  Ignores any it doesnt need at compile.
	import flash.display.MovieClip;
	import flash.events.Event;
	
	//declare document root class.  Must be same name as SWF file
	public class HUD_OnScreenCamera extends MovieClip {
		//declare variables to be used.
		public var CameraCapture_Display:CameraCapture_Display_MC;
		//main function that executes when instantiated.
		public function HUD_OnScreenCamera() {
			// constructor code
			super();
			//initialise
			addEventListener(Event.ADDED_TO_STAGE, init);
		}
		//Should always use an init function to make sure you have access to
		//things like PlayerController in uscript etc.
		public function init(e:Event):Boolean
		{
			//get a reference to the capture display clip
			CameraCapture_Display = new CameraCapture_Display_MC();
			//set dimensions and name
			CameraCapture_Display.width = 500;
			CameraCapture_Display.height = 300;
			CameraCapture_Display.x = 780;
			CameraCapture_Display.y = 0;
			//set the alpha before adding to the display list
			CameraCapture_Display.alpha = 1; //takes a value of 0 to 1
			//add to display list attached to root
			addChild(CameraCapture_Display);
			
			//initialisation successful
			return true;
		}
		
		
		
		
	}
	
}

---> CameraCapture_Display_MC class file [ CameraCapture_Display_MC.as ]

package  {
	
	import flash.display.MovieClip;
	import flash.events.Event;
	
	
	public class CameraCapture_Display_MC extends MovieClip {
		
		
		public function CameraCapture_Display_MC() {
			// constructor code
			super();
			//initialise
			addEventListener(Event.ADDED_TO_STAGE, init);
		}
		
		public function init(e:Event):Boolean
		{
						
			//initialisation successful
			return true;
		}
		
		
	}
	
}

[Nagoshi]

Thanks for the detailed response! I made the changes, and even though the files are now a lot cleaner, the result is unfortunately the same. I also tried importing your fixed files directly, and still no change. It's as if Flash is somehow deciding to ignore parts of the image that are further away, even though the rendered texture is complete.

I tried both Flash Player 11.2 and 10.3. I’m not sure whether or not there has been an update to allow for the newer version somewhere along the line, but it seemed that both worked just as well in the 08-2014 build of UDK. Many of the example files from the UDK directory also use 11.2.

The image folder being misnamed was my mistake while making these test files (d’oh!). The original working files were set up with matching names. Good catch!

I added stop(); to the first frame of the main scene and haven’t noticed anything different, but I’ll make a note to do that. I do use stop(); in Movie Clips with animation in other places. Does it still help to include it on timelines that only have a single frame?

I still had to set the alpha all the way up to 255 in order to remove the transparency issue mentioned on UDN. I tried 1 and it was only slightly more opaque.

Ah, interesting note about "Automatic Declare Stage Instances", I didn’t know that it was a feature which could be toggled. That sounds helpful, because more than once I’ll add a few extra declarations just from experimenting, and then forget to clean them up.

Everything is off the timeline now (except for stop(); ) and is in an .as class. It took a few small adjustments to get it all working again, so I know that UDK was seeing (and yelling about) the changes. But once it was properly set up there doesn’t seem to be any difference between this version and the one with the script on the timeline. I’m going to keep moving forward with the class files though since it seems much cleaner, especially if that’s considered to be best practice for AS3.

Fascinating! Is this multiplayer? Is that why you didn't just attach an RTT plane to the player / vehicle camera or am I missing something? So even if the RTT actor is set to two frames the Flash HUD continuously updates??? Just an idea, but why not attach a dedicated RTT plane to the camera itself on the left-hand-side as a sanity test and compare the results. BTW: I think this is more an unrealscript question than scaleform, plus it might get more views there...

I gave something like that a go as well, and it does show the correct image like the top box that was added to the stage from the library. But, you got it, I was worried about things like multiplayer and having to hide the planes from other pawns. I was afraid that things could get weird quick, and thought it’d be better to just slap it on the HUD. Having it display on the HUD also opens up some fun possibilities!

Even with the plane attached to the pawn, I still had no luck with the frame rate though. I can enable/disable the capture component to get a nice freeze-frame, but even setting the frame rate to 0 has no effect (on either the HUD or on a plane). I’ve done the same thing with a stationary capture object as well, with the texture showing on another mesh in the scene that wasn’t attached to the player, and then changing the framerate does work. Maybe it can’t be changed if it’s attached to a skeletal mesh?