[Feature request] XML / TXT file read / write

[acropole]

Hi,

There is no dedicated forum for feature request, so I post here.

I realy need a full file read / write system to load a lot of custom data in my game, and I read on the forum that other peoples need it too. So I post a pool : would you like to see a file read/write feature in the UDK (for XML and / ot TXT ) ?

[elmuerte]

I rater have the DataObjects to return: http://wiki.beyondunreal.com/Legacy:DataObject
It's a nice and controlled way to serialize data.

With PerObjectConfig (http://wiki.beyondunreal.com/Legacy:PerObjectConfig) you can go a long way, except that the UDK is missing ClearConfig which prevents you from deleting stuff.

[acropole]

They are often a better solution when you need to store a lot of information or you don't want it easily edited from outside the game (ie if you use it to store player progress in a game).

I want my data to be easily editable outside the game. (http://forums.epicgames.com/showthread.php?t=706466)

[Solid Snake]

My project, DataLink would probably be able to help you here.

[acropole]

Thanks, Solid Snake, but your project is not free and need to launch an application before the game (do you sold it 35*UDK price like nfringe ? Or more than the adobe cs 4 with Adobe Dreamweaver® CS4, Flash® CS4 Professional, Photoshop® CS4 Extended, Illustrator® CS4, Fireworks® CS4, Acrobat® 9 Pro, Soundbooth® CS4 et Contribute® CS4 ?).

[fritzmonkey]

My project, DataLink would probably be able to help you here.

Please stop peddling this everywhere. The very restricted I/O capabilities of UDK are a serious issue for many people. I get the feeling that if this issue isn't addressed by Epic it will greatly hurt the number of games made with UDK and the quality of the games that are created. The team I'm working with will probably make the switch to OGRE or Torque if UDK isn't updated with some reasonable I/O capabilities in the next month or so and I don't think that we will be the only ones. Rather than having this issue addressed and seeing UDK games, and the UDK community flourish you seem to be hell bent on taking peoples money. The issue should be fixed rather than being worked around. I have no issue with you creating and selling this tool, I'm just tired of seeing you post about it in threads that are looking for a real solution.

[/rant]

[acropole]

in threads that are looking for a real solution.


for an unreal solution :D

[neai]

Builtin read/write for text files is already in - INI files. So another vote for DataObjects and completing the config files feature set.

As far as plain text goes, there's not much point in making up another text file format and trying to parse it in unrealscript. This would be inefficient and error-prone. I'd say if you want to import a lot of stuff from proprietary text formats, take your existing well tested parser in C++/whatever and convert your data to INI. It's trivial!

Anyway, you can already read and write any files you want, using any programming language you want, use TCPLink. If you don't have time for this, post a help request. It's not something that only Epic can do, it's something you program in an hour or couple.

[Solid Snake]

Thanks, Solid Snake, but your project is not free and need to launch an application before the game (do you sold it 35*UDK price like nfringe ? Or more than the adobe cs 4 with Adobe Dreamweaver® CS4, Flash® CS4 Professional, Photoshop® CS4 Extended, Illustrator® CS4, Fireworks® CS4, Acrobat® 9 Pro, Soundbooth® CS4 et Contribute® CS4 ?).To be completely honest, I am currently thinking something like $10, mostly because it hasn't taken me a long time to write it up. Again, all the information is out there on how to do it, just the act of doing it is probably stopping most people. Secondly, I am making this as transparent as possible, so it's not like the user has to start up two programs, but rather you start up DataLink and then it also starts up UDK ... so all your players know is that they're starting up your game.

While I totally agree that DataObjects is something that would be really, really useful in Unreal Engine, we just need to use what is available to us right now. After all, we can complain or moan as much as we want but that doesn't mean we'll get what we want (after that it's a slippery slope anyways, because to what extent should Epic continue to follow the whims of what people complain about?).

[acropole]

Builtin read/write for text files is already in - INI files. So another vote for DataObjects and completing the config files feature set.

Ok, so how do I build INI with entries for each map, team and player class I need ? I will have to create a new LevelInfo for each map. Those who want to add sutff like teams and player classes will have to provide a new script with a new package with a new INI for each. A lot of files...
Config( var ) is not possible. It's const.


Anyway, you can already read and write any files you want, using any programming language you want, use TCPLink. If you don't have time for this, post a help request. It's not something that only Epic can do, it's something you program in an hour or couple.

I'll try this, but I read a bit about it and it looks like a hack, not a solution.


To be completely honest, I am currently thinking something like $10,
It's a fairly price.


EDIT :

I read a bit more about TCP link.
If I understand it correctly :
The server (which will read the XML files) must connect to himself.
Then it send a text message to a software I've created.
This software must listen to some port (so must be able to use sockets and authentication and security...)
Depending on the string received it will load the file and send it back to unreal dedicated server as a string.
The dedicated server will parse the string to retrive the desired information.

If all this is correct, I'm more convinced that a real XML/TXT parser is needed.

TCP link looks to be perfect for databas acces with SQL strings or for a netowrk of dedicated servers, but is an heavy tool to load a few XML at map start.

[Solid Snake]

Unfortunately, everything is usually a hack in game programming :)

[acropole]

Unfortunately, everything is usually a hack in game programming :)

Yes ;) but there is clean hacks and crap hacks. Take a look at my understanding of tcplink before your latest post. If it's correct it's... :eek:

Some people check "other file system" but none explain what. Please be more precise in a post.

[neai]

Ok, so how do I build INI with entries for each map, team and player class I need ? I will have to create a new LevelInfo for each map. Those who want to add sutff like teams and player classes will have to provide a new script with a new package with a new INI for each. A lot of files...
Config( var ) is not possible. It's const.

I'm not sure if I understand you correctly but you could do something like this:
- Create just 3 configuration classes, subclass of Object, containing just the configurable properties. For example LevelConfig, TeamConfig and PlayerClassConfig.
- Use the PerObjectConfig functionality to store named configurations in single INI file, or in 3 INI files, ie all the TeamConfig's in one file.
- Then you just instantiate ie TeamConfig class with name 'BigBadTeam' and the object initializes itself with the section called [BigBadTeam TeamConfig] from the INI file containing all the team configs.
- You can also enumerate all the available named configurations for the given class to ie display a list of the teams for which configuration exists.

So in this way, the name of the config file is const, but you can keep multiple named configurations in single file. See the docs for more info.

[acropole]

Ok, but this require a class for each map/team/class.
All those config will be in the same files (map.ini, team.ini and class.ini)
The problems here are :
1 - anyone who want to add map, team or class will have to provide a .u file (even if it's for only one class)
2 - the ini will be changed by all new addon, but no one will be able to provide a ini with all addons. Exemple : I will create ca (class a), cb, cc. Someone will create cd. Someone else will creat ce. But there will be only ini with ca, cb, cc and cd OR ca, cb, cc and ce. Those two ini will override each other...
Imagine if there is 100 class made by 50 peoples... it mean 100 u files and an incredible amout of ini possibilities...
With 1 XML for each it's simple and easy to do.
I know what I'm talking about. I've built this with kjAPI engine.

It would be great to read Epic's point of view about this.

[neai]

Ok, but this require a class for each map/team/class

No it doesn't, that's the whole point of PerObjectConfig.

[Sir_Brizz]

I'm not sure why it's wrong to want a better way of doing this. Saying "We should just deal with what we have and not whine and moan" is silly. People want a better way to store complex data objects for their games. Enough people have asked about/talked about this that it would be silly for Epic to not expose at least a portion of the functionality in UDK. Plus, if nobody requests these features, then nobody at Epic will know that anybody wants/needs them for their project.

[acropole]

No it doesn't, that's the whole point of PerObjectConfig.

Ok, I found how to do this :



[Hex-Island LegionsManager]
LegionsNames=Aztecs
LegionsNames=Aegyptians


I create a new :



class LegionsManager extends Object
notplaceable
editinlinenew
PerObjectConfig
config(LegionsMan)
dependson(Legion);

var array<class<Legion> > Legions;
var() config array<string> LegionsNames;

function LoadLegions(){
`log("LEGIONS - Loading legions");
}in my gametype with the map name as parameter :


LegionsMan = new(None, WorldInfo.GetMapName()) class'LegionsManager';But there is three problems :
1- the ini file is in UTGame folder and is renamed UTMyConfigName
2- the map name is the title game in the map info or the file name if there is no title, and they could be different.
3- all are in the same ini file and so may be overriden when installing a new map.

[LennardF1989]

I don't understand why people refuse to use TCPLink with an own app to write files or even access a database, it's so simple. A quick pseudo code:

1. Open TCP listener on port X
2. Keep checking for connections, if one is connected, refuse new connections unless you have a need for multiple at once.
3. Bind a StreamReader and StreamWriter to the input/output respectively.
4. Think up a simple protocol:
START <filename> = Opens file
<text here> = Keep appending untill...
EOF = ... is recieved.
5. Open FileWriter and write String to file.
6. Close FileWriter to release the file.

There are literally thousands of examples on the net in hundreds of different languages that do just that if you add the protocol yourself.

You could do all of this in a nice and clean way with approx. 50 lines or REAL code in C# or Java, or whatever language you wish (Real code referring to whats within function/method definations). Heck, you could release some design patterns on it to make the application more dynamic for extensions.

Only want quick and simple MySQL access? Make a PHP script that acts as a proxy with some parameters, don't forget the password parameter, else it'll break all your security towards the database.

[elmuerte]

But there is three problems :
1- the ini file is in UTGame folder and is renamed UTMyConfigName
2- the map name is the title game in the map info or the file name if there is no title, and they could be different.
3- all are in the same ini file and so may be overriden when installing a new map.

1: what is the problem?

2: use the package name of the map, not the map title

3: Are you going to use it to store state? Or are you only going to use it for readonly definitions. Because in case of the latter you should make use of the datastore so that you can create separate .ini files for each map (just like what is done for the mapinfo).

[LennardF1989]

Here is a quick job showing my psuedo code works, it's done in C# and took me about 20 minutes, including testing and debugging.

Features are:
- String splitting

START "making some lol" hi there
Would make it "START" with 3 parameters
- Protocol class to seperate protocol code from server code
- TCP listener class to seperate server code from initial code
- Multithreading = Different threads for each client
- Number of allowed clients specifiable
- Port specifiable

Number of lines (real code, excluding comments): 55
Number of lines without String splitting: 38

Screenshot:
http://i259.photobucket.com/albums/hh312/LennardF1989/UDK/psuedo.png

EDIT: Oh, Putty in this case acts like Unrealscript making a connection.

[Sir_Brizz]

Lennard, this has nothing to do with the logical creation of a TCPLink and program to service it. The fact is, the overhead of having a TCP connection, even to your local machine, is completely pointless overhead. You are already on your machine.

If it was about accessing a MySQL database (which it isn't), sure, that approach would work fine. But this is about accessing local data about a local game. A network connection should not be required, nor should an external program to handle capturing it.

[sueds]

leonard can you put attach your file with a sufficient explanation ? I'm versed with c#but not in web programming so I would be able to figure it out without a full tutorial. I you do so I Could create a tutorial on my blog and of course credit you. But if you have yourself some time to dedicate I would be nice to have a tutorial on how you did that.

[LennardF1989]

Well, the initial thread states reading/writing XML/text files, so I don't see why it's not relevant. I'm just pointing out that instead of complaining the ability to read/write files isn't in the engine it could be solved within 20 minutes.

Note, I don't say there shouldn't be such a thing in the engine, but TCPLink is quite powerful if you use it right. If needed, you could also serialize the classes yourself too...

@sueds:
Not too sound rude, but if you are versed with C#, why not just type "TCP listener C#" in Google, click the first link (http://www.switchonthecode.com/tutorials/csharp-tutorial-simple-threaded-tcp-server) and work of that? It's essentially the same what I did from the top of my head.

[Wormbo]

Oh goody, polls still redirect to the UT2kx forums. :rolleyes:
[/semi-ot]

[neai]

A network connection should not be required, nor should an external program to handle capturing it.

The connection is an implementation detail. The program is not external, this isn't UT modding anymore. This is an integral part of your game and the only single exe you need to run.

[Blade[UG]]

Yes, file access is something that Unreal could use. However, most of the people who clamor for it all the time aren't really wanting to use it for anything that can't already be fairly easily done. What -exactly- is it you're trying to do?

[Sir_Brizz]

The connection is an implementation detail. The program is not external, this isn't UT modding anymore. This is an integral part of your game and the only single exe you need to run.
It doesn't matter what it is. You shouldn't need to write an external network interface to your game to get pretty basic functionality.

The program resides in memory outside of your game. What else would you call "external"?

[neai]

You shouldn't need to write an external network interface to your game to get pretty basic functionality.

I agree, the proper way to do this is to implement whatever you want into the engine.

However, keep in mind that the basic requested text file read/write functionality already exists in UDK, although not many people know about it. You just gotta import/setup your text assets into the format UDK uses, same story as with localization, sounds, textures or whatever.

If you want to do the files your way, writing that tcp interface is 20 mins of work. I believe I've seen some tutorials posted already.

So the request is not about adding missing functionality, it's about adding a convenience feature.


The program resides in memory outside of your game. What else would you call "external"?

Well, that's how many games work. UE3 itself does something like that, have you noticed? The com file. It's just an implementation detail.

[Sir_Brizz]

So the request is not about adding missing functionality, it's about adding a convenience feature.
They were just discussing in this very thread how that ISN'T working.

Well, that's how many games work. UE3 itself does something like that, have you noticed? The com file. It's just an implementation detail.
The only ones that run programs in the background are ones that use SecuROM. UT3.com doesn't run in the background while UT3 is running.

[neai]

They were just discussing in this very thread how that ISN'T working.
Works for me :D I guess I should make a tutorial.

[Solid Snake]

At the moment I am serializing and de serializing objects from the config system. I think I've found a work around most of the major obstacles with the config system.

[Blade[UG]]

Also, XML is fairly intensive to be able to comprehend, for both humans and machines. It is readable by both, which I guess is why it's popular, but it's certainly intensive for text processing and memory usage, compared to using a format that is tailored to what you need it to be used for.

[LennardF1989]

At the moment I am serializing and de serializing objects from the config system. I think I've found a work around most of the major obstacles with the config system.

How do you manage to serialize one object multiple times without overwriting the first config file that has been made? I don't think the Config(x) line allows variables, right?

[Solid Snake]

The config line doesn't allow variables correct. I store data objects as a single string.

[thundyr]

However, keep in mind that the basic requested text file read/write functionality already exists in UDK, although not many people know about it. You just gotta import/setup your text assets into the format UDK uses, same story as with localization, sounds, textures or whatever.

How does one find information on text asset importing, their file formats, and utilization through uscript? Do you have a link, or perhaps even a tutorial?

[Blade[UG]]

afaik, there is no way to read arbitrary files in unrealscript

[Solid Snake]

Given that we now have DLLBind, all of this is possible. Also better systems such as databases are now available to us.

[MANISH KUMAR]

Hi please read a code or see a reeference of my post, and solve it .pls anybody solve this and send me reply quickly*********
http://forums.epicgames.com/threads/705640-Save-load?p=30579590#post30579590