Announcement

Collapse

The Infinity Blade Forums Have Moved

We've launched brand new Infinity Blade forums with improved features and revamped layout. We've also included a complete archive of the previous posts. Come check out the new Infinity Blade forums.
See more
See less

Public Redirects: Ideas needed!

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Public Redirects: Ideas needed!

    Ok, the short and sweet. I am considering doing public redirects. I am not right now considering bandwidth or price, as I am more interested in features, then I can base what it may cost me off what is necessary to meet maximum community needs. Here are the basics, and you folks can offer any advice you can think of. Forget the idea of bandwidth right now or any other internet based limiting factors. Assume server is configured for maximum connections, and assume all files uploaded are compressed.

    So number1 is:

    1: All files uploaded must be compressed (will only allow file extension of UT3 compressed files to be uploaded)
    2: Uploads to be done via anonymous FTP and then check for duplicate file names/sizes. If a file has the same name, different size, the we can assume that it's a new version of the file. I am sure a lot of you are like me and HATE having to deal with changing a file name in a config file for mutators and such.
    3: The uploads are placed in a queue for approval before being moved to a standard web accessible directory
    4: Files will be be distributed to multiple servers via rsync, so that no one server is loaded to the gills with connections, so to speak. Of course that is up to each server admin to use the closest redirect server.

    Now, that is just that absolute base of what I am thinking. I have tons more ideas, but the core of it all is a basic public redirect service for those with either crap connections or low storage availability. Any ideas, thoughts, hints would be greatly appreciated.

    Yes, I know this can turn into a rather large deal. As I said, forget any aspect except the basic premise of providing a place to upload files and serve as a public redirect. Thanks fellas!

  • #2
    Well I been using gameservers public redirect for several years and it has been great going back to UT2003 and I think I used it for XMP and before that with Unreal 1 and UT ect I was using a public site located in NY but was shut down.

    All ucc compression .uz compress

    Right now there are issues with UT3 and the compression process does work in the latest beta patch 1.2 but will only compress large files a few %. Like from 39 mg to 36 mg just is not going to work. So I dont know if that will be fixed or if the cook is somehow effecting the compression but never the less the compression process does not seem to be like it should and I dont know if the public patch when it is released will have a good working compression.

    So at this point uploading uncompressed files to clients especially if they are large map files is very risky for servers and would tend to be a turn off to clients who have to wait for a large file then get caught in a map switch ect ect.

    If the next public released patch has a good working compression for the files then your idea will be good. Like I stated before in UT2004 and earlier I can compress a 30mg file down to under 10mg so ??

    Good luck

    Comment


    • #3
      the old pub redirect was STNY.. the file extension was uz2, as best i remember for ut2003/2004, but i may be wrong because i have not played either in forever.. this will be standalone redirects, no web hosting, no game servers attached. dedicated only to redirects, so they'll be fast. map sizes won't be an issue because storage is cheap right now, in the grad scheme.

      i am not going to do anything until a patch is released, except test.

      Comment


      • #4
        Ya that was it STNY and I used to have conversation with the guy.

        Anyway I just dont know how UT3 compression is going to work or if it will even work as we know it so we will see with the next patch release?

        Comment


        • #5
          Originally posted by ANick View Post
          Ya that was it STNY and I used to have conversation with the guy.

          Anyway I just dont know how UT3 compression is going to work of if it will even work as we know it so we will see with the next patch release?
          indeed.. the compression is about the only thing i worry about from the game side..

          Comment


          • #6
            Originally posted by ANick View Post
            Right now there are issues with UT3 and the compression process does work in the latest beta patch 1.2 but will only compress large files a few %. Like from 39 mg to 36 mg just is not going to work. So I dont know if that will be fixed or if the cook is somehow effecting the compression but never the less the compression process does not seem to be like it should and I dont know if the public patch when it is released will have a good working compression.
            I doubt compression will get better, you must remember in previous versions of UT maps weren't cooked. Cooking is a form of compression, hence the reason why you start with a map that is a couple of hundred MB and end up with a map thats 15-25MB. Maps in UT3 have alot more data in them now. Since they are cooked it is virtually impossible to get any more compression out of them. If you compress a file with winzip or winrar and then compress it again, you'd be very lucky to shave half a MB off it and the same thing applies to UT3 maps now. Faster redirects are the only thing that will help this problem. Also music is now in the map files (if author added music) which makes them bigger, in UT2004, music was a seperate .ogg file and that wasn't pushed with maps.

            I've setup my own redirect for my UT3 server,

            82.29.79.45:7777 (patch 2 beta 1, compression on, 18 custom maps so far whilst testing patch)

            for me a 10MB map comes off in 30 seconds (redirect is in my town), others from UK and around reporting good speeds too.

            Comment


            • #7
              Yes indeed a cooked map will not compress much more that it allready is.

              I see this situation as being stuck between a rock and a hard spot and I dont think there is a good server to client solution for content push with UT3!

              As I spoke earlier in other threads about ucc compress I was under the impression that UT3 was going to be like UT2004 and earlier unreal builds as far as compression redirect but it will NOT be the same so I am now bewildered as to how community stuff will flow?

              O well good luck to whoever wants to spend the time and energy on it because when it becomes "work" versus a "fun Game" im out!

              Comment


              • #8
                The unfortunate thing for people is that public redirects will no longer cut it, not for now anyway. With a public redirect the max bandwidth it has is distributed between whoever is using it. With a public redirect there could be thousands using it, this is why sometimes public redirects are fast and at others they are not. Sometimes the redirect will be under heavy load, for example say a redirect has 1000 servers using it, if 900 of them servers (lets say they are all 10 slot servers) are using it at the same time and they all are full with players and each player needs the maps the servers are using that would be 9000 clients downloading from the redirect, causing a slowdown due to the total bandwidth being split between all 9000 clients. UT3 only adds to this problem with its larger content. However if you have a private redirect there will be alot less likelihood of your redirect becoming under heavy load. Better still if possible, set up a redirect from home (preferably on a seperate connection to the one you use for your main pc) and use that for redirect, obviously this is the most expensive option as it requires 2 internet connections but will provide the best results as all bandwidth is used for your servers redirect and nothing else. Private redirect from a good provider should be good enough for most people though until public redirect providers can pump the extra bandwith needed to move UT3 content quicker.

                Originally posted by 13th_Disciple View Post
                Ok, the short and sweet. I am considering doing public redirects. I am not right now considering bandwidth or price, as I am more interested in features, then I can base what it may cost me off what is necessary to meet maximum community needs. Here are the basics, and you folks can offer any advice you can think of. Forget the idea of bandwidth right now or any other internet based limiting factors. Assume server is configured for maximum connections, and assume all files uploaded are compressed.
                Bandwidth and cost are the first things you need to look at and assess, it's going to take alot of both to come anywhere near to what gameservers and others allready have nevermind better them. Then you need to look at who the redirect will be tailored towards. For example you can have alot of bandwidth but if your based in the US then your no good for the UK and vice versa, nevermind having an optimal server setup so having multiple redirects, like you said is a must. Planning where your redirect servers will go will take alot of thought in order to meet your requirement "to meet maximum community needs". Then you must realise that if you do manage to set this up, the quality of service will degrade just like any other public redirect due to the fact that having a few servers using it at first will be great for them but the more that come (and they will) the more the bandwidth will be split between everyone resulting in everyone getting slower redirect speeds, this then puts you in the position of thinking "do I put more money into this" (answer to that being how much do you have? and how much can you afford to put in?) or "do I just leave it as it is" which makes it no better than any other redirect or maybe even worse. Your plan sounds good but ultimately can you fund it? and even if you can fund it, you could be out on a losing battle from the offset.

                Comment


                • #9
                  I don't plan on gearing the redirects for anything other than NA, at this point. The majority of people getting game servers are offered a free redirect from their provider, but some are not. STNY was a single server offering redirects. I have an idea on a distributed redirect system that might could work for several hundred servers simultaneously.

                  There is a lot I am not stating here, that is why I asked for a short and sweet set of requirements, as if the entire thing was in place.

                  Comment


                  • #10
                    Originally posted by 13th_Disciple View Post
                    4: Files will be be distributed to multiple servers via rsync, so that no one server is loaded to the gills with connections, so to speak. Of course that is up to each server admin to use the closest redirect server.
                    Could you elaborate on this for me please.

                    Comment


                    • #11
                      Originally posted by AZ-{OVGY} View Post
                      Could you elaborate on this for me please.
                      basically, have multiple servers that receive their files to be redirected by a main server so that all servers have the same updated data.. of course it will depend on where the client is in regards to how fast the redirect is, but chances are, a client machine in Los Angeles won't try and join a New York based server simply due to ping..

                      i know even this setup will have flaws due to potential client/server connectivity issues, but it's better than just a single server handling all of the load.

                      Comment


                      • #12
                        Do you really have any idea as to what this will cost? If you have large buy in from the community, it can cost you hundreds, if not thousands of $$$ per month. But you said you did not want to talk about that....

                        2: Uploads to be done via anonymous FTP and then check for duplicate file names/sizes. If a file has the same name, different size, the we can assume that it's a new version of the file. I am sure a lot of you are like me and HATE having to deal with changing a file name in a config file for mutators and such.
                        Not a good assumption. You cannot allow the overwriting of files.

                        1. A troublemaker can easily corrupt hundreds of maps simply by overwriting them with different one and giving it the same name
                        2. Admin 1 has loaded MapX. Admin 1 decides to make a couple of tweaks to Mapx for HIS server, and uploads MapX to the redirect. Admins 2-200 who are running MapX (without these changes) are totally screwed.

                        3: The uploads are placed in a queue for approval before being moved to a standard web accessible directory
                        Are you personally going to review every file? What happens if this takes off, and you have 100 files being uploaded per hour? 200? No one will wait threee days for you to review each file. And in the mean time, you have another 1,000 files waiting....

                        4: Files will be be distributed to multiple servers via rsync, so that no one server is loaded to the gills with connections, so to speak. Of course that is up to each server admin to use the closest redirect server.
                        You cannot rely on people to manually chose the correct server. Now if you are talking about some kind of auto-load balancing at connection time, then that's different. But again, you're talking big expense.

                        Also, the location of the server really doesn't matter much, unless you are talking about the deepest bowels of the rainforest which can only be reached by dial up. Global location is more important with regards to ping. As long as a location is well peered, it could be in Russia for all that matters. Once the transfer gets going, you'll get good speeds. I believe gameservers is in the UK, is it not? It served my users (NA) very well until I stopped using them.

                        I think what you are trying to do is admirable, but I really don't think you've thought it through enough. My suggestion is to get a group of trusted game admins together, and pay monthly dues to rent a VPS. This way you know who is using what, in terms of the load, you can manipulate file names and coordinate the update of existing files, and you can share the workload of administration.

                        TBH, I use a simple web hosting account. They're cheap, and they are usually well peered, enough to provide all of my servers good speeds.


                        Good luck!

                        Rich (TW)

                        Comment


                        • #13
                          Now that was one positive post...

                          Comment


                          • #14
                            files will not be overwritten, ever. if a duplicate file exists, same file name and size, the newest file gets kicked. i think you misread what i was saying. if a file is uploaded, and EXACT (file name and size) match exists, new file is not allowed into web accessible location as part of redirect files.

                            no, i will not be doing this myself per approval of each upload. it will be scripted. just like everything else i do. by trade, i am a unix admin working on hp-ux, solaris, aix, linux, tru64, etc..

                            as for money, i am more aware of cost than anyone else because i already know what to do to get the basic premise to work and pricing on hardware and connectivity.

                            as for getting them to pic the right server, there will be a list of servers and their respective locations. while i know there will be instances of issues, the redirect network will be fairly well spread out, so that no one server gets all of the load. even if that's the case, there are ways to mitigate that at my servers level to redirect the source from another server.

                            Comment


                            • #15
                              i realize i am being quite vague about several things, and i really do apologize about that.. but i have an good idea and don't wanna just spit it all out at once.. but i honestly appreciate all input i can get..

                              i do understand it's a large undertaking and that it could potentially be quite expensive.. but i even have a few ideas of how to offset some, if not a majority of the cost almost from the get go. this won't be an overnight setup, so i am not foolish enough to believe that will happen.. and this whole thing could possibly never see the light of day.. so that's why i am looking for what people would want in a service like this..

                              i also know hosts often provide redirects, but a vasy majority of them limit not only storage, but bandwidth from the redirect server, as well.

                              Comment

                              Working...
                              X