Convert from 2d screen coordinate to 3d space

[fall]

I've done this in UT2003/2004, but it seems to be handled differently in UT3.

I see the function ViewportDeProject (Member function of UTUIScene class) as doing what I need, but I'm unsure how to access the function from my mutator.

I'd appreciate any suggestions on how to access this function from my mutator.

[Joe Wilcox]

UTUIScene.ViewportProject translates a 2D screen coordinate to world location
UTUIScene.ViewportDeproject translates a world location to a 2D screen coordinate

However both require a UTUIScene :) What are you trying to do?

[Tonester]

Wow! This is exactly what I've been trying to find out for months.

We are doing an RPG-like TC which has a hotbar with 6 skills on it. In UT2K4, I managed to hack in a mouse cursor and was able to interact with buttons in the HUD, but it was ugly.

How can we use UIScenes to create interactive gameplay interfaces and HUD elements? I don't need the entire answer, just a push in the correct direction. However, lots of people are having issues with the whole UIScenes stuff. Documentation for this, along with TC directory structure and so forth would be very helpful.

[bushbomb]

I'm sure it's simple, but what am I doing wrong? I'm in an interaction class and this is what I did in UT2004:


ScreenLocation = WorldToScreen(P.location);
where ScreenLocation is a vector, and P is a Pawn.

How do I do this in UT3? I made a poor attempt here:

ScreenLocation = UTUIScene.ViewportProject(P.LocalPlayerControllers[0], P.Location);

And compiling kicks back this:

Error, Bad or missing expression for token: UTUIScene, in '='

[Potens]

Bush, it's because your setting ScreenLocation as being = to the function call instead, of setting it as the Out Vector inside the function call.

Sorry for the old thread revival, but has anyone made progress on this since it was a posted?

[bushbomb]

I created a new actor, added it to the hud's PostRenderFor actor list, then in my new actor put:

simulated event PostRenderFor(PlayerController PC, Canvas Canvas, vector CameraPosition, vector CameraDir)
......
ScreenLocation = Canvas.Project(P.Location); //Convert their 3d position to 2d screen coordinates

Works like a charm.

[Tonester]

Except that this has nothing to do with what Fall and I (and others) would like to do.

We need some way to Unproject/Deproject/ScreenToWorld/Etc - the opposite of what you are talking about Bush.

We need some way to "pick" in 3D. We have implemented a mouse cursor that works while in the game (doesn't require a UIScene). We wish to use this mouse to select other players, pawns, objects, etc in the 3D World. In order to do this, we need some way to Unproject from the 2D viewport mouse coordinates to where it correlates in 3D space. Once we have that end vector location, we can then run a trace and see what it is.

UT2k4 had a Canvas.Deproject, but UT3 does not for some reason.

[Potens]

Has anyone been able to successfully use the ViewportProject and ViewportDeProject functions. It seems to want to crash with a general protection fault whenever I access either. :)

[immortius]

I wrote my own function to calculate vectors from screen positions (because I was doing it outside of UIScenes). I just stripped out some excess stuff for this post, so apologies if the syntax isn't quite right.

Something I picked up while writing a raytrace renderer. :P



// ScreenPos is the screen X/Y positions between 0.0 and 1.0 (origin top-left of screen)
// ViewRotation is the direction the viewer is looking (i.e. playerController.Rotation)
// The result is a unit vector in the direction the screen position. Multiply by X and add
// the view location to get a a point X units in the direction of the screen position.

simulated function vector ScreenToWorld(Vector2D ScreenPos, Rotator ViewRotation)
{
local PlayerController PC;
local float FOVAngle;
local vector viewPortBorder;
local vector2D ViewportSize;

foreach LocalPlayerControllers(class'PlayerController', PC)
{
LocalPlayer(PC.Player).ViewportClient.GetViewportS ize(ViewportSize);
fovAngle = PC.FOVAngle;
break;
}

// Work out the border vector
viewPortBorder.X = 1;
viewPortBorder.Y = -tan(fovAngle*PI/360.0);
viewPortBorder.Z = -viewPortBorder.Y * ViewPortSize.Y / ViewPortSize.X;

viewPortBorder.Y *= (1 - 2 * ScreenPos.X);
viewPortBorder.Z *= (1 - 2 * ScreenPos.Y);

return Normal(ViewPortBorder >> ViewRotation);
}

[fall]

UTUIScene.ViewportProject translates a 2D screen coordinate to world location
UTUIScene.ViewportDeproject translates a world location to a 2D screen coordinate

However both require a UTUIScene :) What are you trying to do?

Somehow I missed this reply, thanks :)

I now use a fullscreen "blank" UIScene to be able to catch mouse input and project into the world from the screen.

I was hoping for a more elegant solution (not using a UIScene just to have access to this function) but this seems to work just fine.

[Potens]

Would you mind posting your code fall?

Immo's function works great, but I'm still curious how Project and Deproject are suppose to work.

[immortius]

Has anyone been able to successfully use the ViewportProject and ViewportDeProject functions. It seems to want to crash with a general protection fault whenever I access either. :)

Just a quick check, but you do have patch 1.1 right?

[Potens]

bah, no. Does it change something?

[immortius]

Yes, if you don't have patch 1.1 you get General Protection Faults if you try to do anything with the UI classes. ;P

[Potens]

Well now I feel like a dumbass, thanks.

[Potens]

Nah, My subclass of UTUIScene still crashes, I'm sure it's just something with how I'm implementing it.

[Tonester]

Immortius,

Thanks for the code, but it doesn't work. I'm about to give up on this for now.

[immortius]

I just tested it, and it does work, but you need to be a little careful with how you use it.

I hooked it into a pawn subclass's tick like so for testing, where pos is a vector2D I'm populating with an exec function:


simulated function tick(float delta) {
local vector POVLocation;
local rotator POVRotation;

super.tick(delta);

controller.GetPlayerViewPoint(POVLocation, POVRotation);

if (PlayerController(controller) != none) {
DrawDebugSphere(ScreenToWorld(pos,POVRotation)*16 + POVLocation,2,8,255,255,255,false);
}
}


This doesn't quite work, as the Player's point of view information is updated after ticks are processed, so the little sphere lags behind your movement. This can be worked around by finding an event that is called after POV is updated - SetPosition in UTWeapon subclasses for instance. I'm still looking for a better function to call this from. But the point is, assuming you can get a good viewlocation and viewrotation, the function works.

[Tonester]

I'm using the function from my HUD Class and it seems to basically be using very small values for X,Y (ScreenPos). I'll do some log testing and see whats going on.

I'm pretty sure the ViewLocation and ViewRotation is accurate and being updated correctly. I've log tested this before and I'm getting updated values several times per second (like you would expect).

[immortius]

Please refer to the comment above the function in my original post.

The function is expecting ScreenPos to be between 0.0 and 1.0, as the percentage of the screen axis. This is mostly due to what I was using it for, so you might want to change it to take the pixel location (just need to divide by ViewPortSize at the start of the function).

[Xyx]

I wrote my own function to calculate vectors from screen positions
That's pretty awesome! Would you happen to have a WorldToScreen sample as well?

[Tonester]

Please refer to the comment above the function in my original post.

The function is expecting ScreenPos to be between 0.0 and 1.0, as the percentage of the screen axis. This is mostly due to what I was using it for, so you might want to change it to take the pixel location (just need to divide by ViewPortSize at the start of the function).

Heh - that kind of goes without saying, no? :)

[Tonester]

School has been killing me.

It works. I was doing X / Y instead of Y / X. For some reason, I thought it just used an AspectRatio (didn't pay attention).