View Full Version : Communicating with other software without TCP or UDP

11-07-2009, 02:46 AM
Can UDK-based games communicate with other local software without TCP or UDP? Through stdout maybe? One-way communication is okay for my purposes.

Solid Snake
11-07-2009, 02:57 AM
It is possible to create a watcher application which watches for files being written. The obvious one is to watch the Launch.log but that gets a lot of things written to it from a lot of sources. There is also a FileWriter class in Engine.u which you can use to write to files on the disk which your watcher application can use.

11-07-2009, 03:05 AM
That's it? Files and network?

Solid Snake
11-07-2009, 03:08 AM
I'm not sure what you're trying to achieve exactly, so it's a stab in the dark.

11-07-2009, 03:55 AM
I was hoping to communicate with an external video player (barebones, custom made) to replace Bink for cinematics. I suppose for this purpose TCPLink is fine, but if there's a way to feed uncompressed YV12 video to the Bink renderer (which is a better solution, in theory), TCPLink would probably be too resource intensive.

The idea is to add H.264 and Vorbis support, since Bink is lacking in quality.

Now that I think about it, it would probably be okay to just use command-line arguments to give the the player the info it needs, but the process probably won't be as hurdle-free and more communication would be required.

What I really want to know is: can you accept input from a pipe (stdin) somehow?

Solid Snake
11-07-2009, 03:57 AM
Hmm, now that is something I don't think you could do. Unless you did weird things like minimize / close UDK played the video, and then went back into UDK.

11-07-2009, 04:02 AM
It would most likely be possible to do it without imposing additional waiting, or even use a renderer which can render on top of D3D. And if that doesn't work out, hook onto it and do it that way (though that would be pretty hacky).

I should be able to get past any other hurdles - the missing piece is an efficient communication method.

Solid Snake
11-07-2009, 04:15 AM
Well you'd need to start doing things like memory injecting, etc etc ... for the sake of using a different video format?

11-07-2009, 06:44 AM
I just did a quick grep of the files in UTGame/Classes of the UT3 script source files (the full game, not the demo). 336 of 907 files contain the term 'native'. When a third of your classes need native code execution, how far can UScript alone really take you?

I'm not feeling the TCPLink or file watching hax either. The mechanism is there. Maybe the feet paddling the duck are too frantic for public consumption. Is it really a money thing? I thought 25% was on par with full licencee's projections.

11-07-2009, 04:17 PM
Well you'd need to start doing things like memory injecting, etc etc ... for the sake of using a different video format?
No, that was just in theory. I wouldn't go that far, it's too hacky. The most I would do is use an external decoder and renderer.

Is it really a money thing? I thought 25% was on par with full licencee's projections.
I suppose they expect that 25% from a full licensee's sales is going to be much bigger than 25% from an indie developer's sales.

11-07-2009, 09:37 PM
I just did a quick grep of the files in UTGame/Classes of the UT3 script source files (the full game, not the demo). 336 of 907 files contain the term 'native'. When a third of your classes need native code execution, how far can UScript alone really take you?

A lot of the native code in UT3 was for performance reasons, converting to native only at the end as we profiled the game.

There are numerous UT3 total conversions as well as Whizzle that demonstrate the power of the language.

We appreciate any feedback you may have about what you can/cannot do in Unrealscript, but I think you'll find that you can be very successful with Unrealscript alone. As UDK takes off we're certainly on the lookout for this type of feedback.

11-07-2009, 10:34 PM
I understand completely why we're not allowed access to the engine source itself. However the restriction of not being able to execute our own completely independent code from unrealscript seems an arbitrary and unnecessary restriction.

From a security perspective it doesn't make sense as any game created via the UDK meant to be sold commercially will have an executable installer, if you don't trust the the author to run native code you probably shouldn't run their installer... And to top it all off native code execution is still technically possible. We can do it, just through a terribly hacky and inefficient method.

I agree that unrealscript can take you a long ways but as you point out was the case for UT3 there are some things that simply need to be executed in native code for performance reasons. Simulations (especially for systems that already have an existing and highly optimized dll for such simulations) and complex AI are two that come to mind immediately. Not to mention the rather broad flaw of custom saves.

If the only reason not to include it is as an incentive to license the full version then it's somewhat the equivalent of arbitrarily putting a sleep(1) in the render loop of the UDK just so it's slower than the licensed version. And really not that effective to boot, the cross section of your potential customers that are on the edge of it making financial sense to license or go the UDK route is fairly small and easier captured via modifying one of license agreements rather than arbitrarily crippling the UDK. It also has the potential to discourage many projects that begin in the UDK from upgrading to a full license once more mature (either through the securing of funding or if the UDK is used for rapid prototyping) as the refactor effort is increased. Additionally any features that would have relied on native code would have necessarily already been dropped from the design thus providing even less of an incentive to upgrade.

I think currently your community of potential users views this restriction as arbitrary rather than technical in nature, myself included. Now if I am completely off base here and there is a fundamental technical reason that it is not possible currently, or more importantly ever, then please let us know.

11-07-2009, 11:07 PM
The fundamental reason is: You're not a licensee. ;)

No really, if you want to create AI or simulation stuff that is too complex to be implemented in UnrealScript and a TcpLink to an external application doing the work is not an option, then you need to get an engine license. UnrealScript, if used properly, should be powerful enough to get most jobs done in acceptable time.

Remember, rule #1 of optimization is "Don't do it!", while rule #2 for the advanced guys is "Don't do it yet!" As mentioned, the native UT3 stuff was originally implemented in UnrealScript and only got moved to native code for the final performance optimization stage. UT3 may have been playable on the PC with most of that staying in UnrealScript, but the game is also targeted at consoles, which have limited and fixed resources available.

11-08-2009, 05:28 AM
Nor will I likely ever be a licensee.

But that is not a technical reason, it is a marketing one and not a good one in my opinion. Arbitrarily crippling the UDK gains them nothing while potentially earning them less revenue in royalties as I have to cut features, raise the min spec, or force the user to endure poor performance in some aspects thus leading to less potential sales.

Again I understand why we're not allowed access to the engine source itself. With the UDK freely distributable and no realistic way to apply or enforce an NDA it is in no way realistic to expect that level of access. Allowing me to run my own or 3rd party code on the other hand costs them nothing while providing at least a potential upside in the form of higher royalty payments.

I guess what I'm looking for barring the actual ability to run my own code is an explanation as to why they've made that decision and think it's the right one. From someone with an Epic tag, I know you and Ambershee can give me the pat answer of that's the way it's always been and always will be. Before last week you couldn't sell mods, that's the way it always was and always would be.

11-08-2009, 03:40 PM
Maybe it's a legal thing... Epic earning dough from an app that directly links to an unlicensed or restrictive 3rd party lib might land them in court?

Linking to 3rd party libs is what I'd most likely do.

11-08-2009, 04:19 PM
They'd be earning dough as it is.

11-08-2009, 04:41 PM
336 of 907 files contain the term 'native'. When a third of your classes need native code execution, how far can UScript alone really take you?
FYI, even though a lot of classes contain the native keyword, it doesn't mean they even have a native implementation part. For example, the Info class doesn't have any native code. Due to UE's class system all parent classes of a native class also need to be declared native. When a class want's to have events which can be called more easily from native code it also needs to be declared native.

Solid Snake
11-08-2009, 10:33 PM
Sorry to post off topic again, but I think it's unfair on Epic to declare the UDK a worse product than it is because it doesn't allow C++ code or whatever else to be integrated into the Unreal Engine.

At this point, I think it is simply because people outside the modding community do not feel comfortable with the workflow or using Unrealscript where as they know how to use C++ or whatever else. The role of the UDK is to make games. Unrealscript is perfect for making 99% of the games that most people want to make.

If you feel that UDK is completely unfit for your project, then that's fine. I have yet to see any development kit that is 100% right for everyone's project.

11-08-2009, 11:07 PM
It is a worse product than it could be solely due to an artificial restriction.

I'd agree that familiarity with c++ is where some of the desire is coming from. But that's certainly not all of it. There are plenty of existing 3rd party dlls containing ready to go code that has been optimized far beyond most peoples available time and ability that simply cannot be used in a reasonable manner. Or code that has already been completed in an engine agnostic manner that should be easily portable but instead has to be translated entirely into unrealscript or run through in incredibly hacky TCPLink.

You and I have different definitions of "the games people want to make". "The games that UT3 modders want to make" might be a bit more accurate.

Unrealscript is great for your standard FPS type game. It's also great for certain other types of games as Whizzle shows. But it also falls down in some areas. Creating a TBS or 4X game with FPS or 3rd person squad based action as one example. The action portion lends itself perfectly to creating the game in Unreal, but the TBS or 4X portion would require a complex recursive AI to function properly. By Epic's own documentation unrealscript runs 20x slower than c++ which means that what should be a 10 second player AI turn is suddenly almost three and half minutes. Unacceptable, but easily solvable by allowing the game to run its AI portion in c++ or c# or whatever. That is admittedly an extreme example but illustrates my point that there are games that people want to make that they can't currently only due to this artificial restriction.

The UDK has a lot going for it. Wonderful tools (Fracture, awesome), content pipeline, proven track record, guarantee of continued development (something so many past alternatives have fallen down on), great renderer, great network layer, among a host of others. It just falls short for a lot of potential game types and (especially non-entertainment) applications in one single way, that being the inability to execute outside code. A situation that could be remedied at the stroke of a key.

Solid Snake
11-09-2009, 12:20 AM
Creating a TBS or 4X game with FPS or 3rd person squad based action as one example.I disagree. I could make these games easily because I know how to.

Also the 20x decrease in speed is taken far too literally.

11-09-2009, 04:32 AM
Sorry but a decent TBS or 4x AI in a sufficiently complex game takes several seconds to compute and execute it's turn. See large games of Civ, Gal Civ 2, virtually anything by Paradox, you get the idea. Add in several opponents and you're easily looking at enough computation that's it's not feasible to do it in unrealscript in a timely enough manner for the player.

And there is no "knowing how to do it" that magically changes that. Code is code, and while I'm sure just like c++ there a numerous little tricks and techniques that can speed things up significantly the simple fact is that a fairly massive amount of calculations need to take place and no amount of tinkering is going to make it as fast as native code or change the sheer amount of calculations that need to take place.

Unrealscript is a viable option due mainly to its heavily event driven nature, it's not running most of the time. UT3 for example does not spend much time percentage wise executing unreascript, I've seen 5-10% as the most common figure, but does so constantly and so it works well. Any performance penalty imposed is spread out fairly evenly over multiple frames and that's assuming there even is a penalty due to the game being CPU rather than GPU limited. On the other hand take a game that needs to execute all of its AI + a laundry list of end turn events for multiple players all at once and any performance penalty is manifested directly in wait time for the player.

To be honest though performance is a secondary concern of mine. And in this particular example TCPLink could be used for a 4x or TBS type game. It would be slower than a pure native implementation. But at some point the advantage of running the core of that type of AI in native code outweighs the cost of serializing everything and shuttling it back and forth. Though let's be clear, it's a hack. not a feature of the engine.

I'm more concerned with the ability to run 3rd party or existing dlls.