MasterServer: Wrong Port# - A NAT Problem

[legacy-UT2003Turner]

This is based on an analysis of what is
going on on the network. My setup:

UTS: Dedicated UT Server
FW : Firewall/Router/NAT (Linux/IPChains/Masq)
MS : MasterServer

UTS<--(a)-->FW<--(b)-->"the internet"<--->MS

I looked at the traffic at (a) and (b).

In order for UTS to register with the MS (UT2003master1)
it must open a TCP connection (SYN). The numbers
are port numbers.

(a): UTS: 1046 --> FW: 28902
(b): FW :61356 --> MS: 28902

So far, so good.

There are some UDP packets (status) to and
from 216.177.89.34 and 216.177.89.28. We
focus on the communication with the MS, so
we just ignore them here.

After the TCP three-way handshake is
completed (SYN, ACK from MS and ACK
from UTS) there is more UDP traffic to
other machines (status). Now, using
the established TCP link, the key (and
some other stuff) is sent and "APPROVED"
by the MS.

Next, UTS sends a UDP packet to MS:

(a): UTS: 7778 --> FW: 28902
(b): FW :61358 --> MS: 28902

Some TCP chitchat and then 2 more
UDP packets UTS-->MS:

(a): UTS: 7777 --> FW: 28902
(b): FW :61359 --> MS: 28902

(a): UTS: 7778 --> FW: 28902
(b): FW :61360 --> MS: 28902

More TCP traffic and we are done
with the registration.

http://ut2003master.epicgames.com/serverlist/full-all.txt reports
"<MyIP> 61359 61360".

Those two ports happen to be identical to the source
ports the FW used to communicate with the MS.

That is the problem - what is the solution?

The correct port (7787) is communicated
via the heartbeat. Not sure if the Masterserver
itself ever request it, but other hosts did.

Could there be a solution other than
replacing the firewall/router?

-=SPP=-Turner

[legacy-PhantomII]

A few quick questions and thoughts.

First, excellent information here.

My router is a Dlink 614+. It is a bit quirky, but it does work fairly well. When I originally purchased it, I had trouble running a dedicated UT server in my home. After a few router firmware upgrades, Dlink finally got it to where the UT server worked fine through the router. Then when UT2003 was release, I could not set up a server that would work with this new game. All this despite the fact that the configuration was similar if not exact???

So, why does the old game work, but not the new one????

What is the difference in communications that the two games use???

More info for you. My game server will actually appear in the master server list. I do however have no ping time, and instead I have the the "N/A" shown. Despite this, if I give my IP to several of my friends, and they add the IP to their favorites list, they are able to connect fine, and we have our fill of killing each other. Obviously the server does work.

So, if my friends can connect with the IP favorites method, without specifying any ports, why can't the master server figure out the ports???

I think there may still be some things to try, so please let me know the following things if you know the answers.


What router are you using????

Once you have configured the server and started it, what port is the first packet of data sent out from the game server to the master server????

The heartbeat which you say has the correct port associated with it. Is this an incoming heatebeat to the UT server, or an outgoing beat to the master server????

In the "data" sent from the UT server to the master server, is the correct port assignments for the UT2003 game server included???

Let me know and I will keep a check!

John

[legacy-Dogsbody.org]

UT2003Turner you are a STAR!!! :-)

I have exactly the same problem and have seen a number of posts on here about the same problem...

http://www.ina-community.com/forums/showthread.php?s=&threadid=208838

...while lots of people have tried to dubug it your the first that I have seen that has debug'd it on both sides of the firewall.

Now, does anyone have any ideas for how to flag this up with the programmers??

[legacy-PhantomII]

I am going to start a new message and request a list of people using routers, which they are using, if they can set up a server, etc. Maybe we can get some data which we can further use??? We'll see.

[legacy-UT2003Turner]

Thank you for your feedback and praise :)

IPchains/Masquerading works like this:

Outgoing connections (from the FW's point of view)
use a certain default port number range, like a
pool of (high) port numbers: lower ports
are reserved for the FW-server itself, eg services
running on the very same FW, and are, therefore,
not being translated (NAT'd).

IIRC the pool contains port numbers >(=) 60000.
It increments the port numbers one by one,
regardless of the original internal source
port number.

I am sure that there are router boxes
that do the same.

Anyway, what seems to be happening is this:

The UT2k3 server sends 2 packets to the MasterServer.
The MS then looks at the source-port of the packets in
order to save them for later use.
The other UT clients use those 2 ports to connect to
the UTS in question.

As long as the NAT gateway (router) uses the same
(source) port numbers as the UTS, there are no problems.
This is a bit risky, because there could be 2 machines
using the same ports for different applications. So,
the IPChains/Masq implementation isn't that bad as
it may seem at first.

The master server determines
"<IP> Port-N Port-M", lets say,
"<IP> 7777 7787" or "<IP> 8040 8050".

With IPChains/Masq, however, the outgoing source ports
get assigned sequentially. Therefore, the two UDP packets
sent by the UTS allocate two outgoing source ports
on the FW's external interface:

N and M. The bad news is that the UT protocol seems
to be based on (N,N+1) and (M,M+1) (not sure if M-N must
be 10, but I am sure it has to be >1 )

So, looking at "full-all.txt" we can see that most
UTSs register with M-N=10

I fear that in cases where M-N=1 there
is not much we can do :(

A UT software engineer would have to
change the MS' source port acquisition method
and/or change the currently used client-server
protocol. My guess is that there are
hardcoded +1 dependencies.

Also, this implies that reconfiguring
port-forwarding on the fly (meaning after
the registration took place) won't work either:

The clients would try to talk to (N,N+1) and
(M,M+1). Problem is that in cases where
M-N==1 (port difference :1 ) port N+1=M
- and we cannot have the same one port
forwarded to 2 different ports..

Well, assigning different ports in UTS
wouldn't help either, because the FW
allocates outgoing ports one by one,
since it does that on a per connection
basis. (apologies: UDP is connection-less,
but the FW has to keep track of those
"connections", so, here, I call
them connections, too :)

This is the dilemma that we cannot solve (easily).

IPTables may solve this more elegantly, but on
the other hand, there seem to be other NAT GWs
out there that have the same problem.

I believe that the "full-all.txt" shows us
a number of poor users with exactly the
same problem. Easily found by looking at the
delta of the two port numbers. Admins
seem to fall into several categories:

a) M-N== 1 ==> poor sucker out of luck (like myself)
b) M-N==10 ==> Lucky SOB
c) M-N >10, M-N <-10 ==> No idea, but may be like (b)
d) The rest: ==> No idea, but may be like (a)

PhantomII:

> So, why does the old game work, but not the new one????

Because the protocol's changed? :)

> What is the difference in communications that the two games use???

I don't know (never really looked at the old UT protocol), but I guess
that the source port acquisition on the Master Server is different
and/or the whole port allocation strategy is different. If you take
a network level trace and look at the packets you may be able to
figure out what the differences are. Not sure how that would help us,
because we already know that there are differences. You have proven it :)
( same router, same setup: old UT works, new UT doesn't).

> More info for you. My game server will actually appear in the master server list.
> I do however have no ping time, and instead I have the the "N/A" shown.

My server shows up, too. That's not the problem in this case.
People can't connect.

1) Can your friends ping your server, or do they see N/A, too?

2) What are your N and M values? Is the difference between the two
10 or 1? My guess it is 10 (check full-all.txt for your IP address)

> So, if my friends can connect with the IP favorites method,
> without specifying any ports, why can't the master server
> figure out the ports???

I think the MS can be viewed as a database. It keeps track of its clients
(the UTSs) and tells the UTclients that there are UTSs and what ports they
are supposed to use. The ping people see are pings from their machines,
not from the server.
The reason why you can't "ping" your NAT'd UTS may be that your client
sees the external IP address, and then the routing may fail..

> What router are you using????

A dedicated Linux box with IPchains and Masquerading.

> Once you have configured the server and started it, what port is the
> first packet of data sent out from the game server to the master server????

The very first UDP packet from UTS-->MS:

(a): UTS: 7778 --> FW: 28902
(b): FW :61358 --> MS: 28902

Very first TCP packet:

(a): UTS: 1046 --> FW: 28902
(b): FW :61356 --> MS: 28902

> The heartbeat which you say has the correct port associated with it.
> Is this an incoming heartbeat to the UT server, or an outgoing beat
> to the master server????

This is part of the problem: The Master server never gets
the heartbeat. The first \heartbeat and \gamename are sent to 216.177.89.34.
This could be a gamespy server, I don't know. It is not the MS, though.

It looks like this:
"\heartbeat\7787\gamename\ut2" (actual payload of that packet)

Sorry, I didn't make that clear in my first post: The MS _could_
know the internal port number if it had the info associated with
the /heartbeat msg. But it is never sent to the MS.


> In the "data" sent from the UT server to the master server,
> is the correct port assignments for the UT2003 game server included???

I checked a the TCP packets but failed to find them. That's binary
data and could be encrypted anyway. I believe that if they are
in there, the MS fails to look at them. If they are not in there,
the MS does not have an alternative: it must use the NAT GW's
source ports - which is kind of cool, because it cuts down
the intial registration traffic. Sadly, this strategy
does come at a cost..

Dogsbody.org:
> Now, does anyone have any ideas for how to
> flag this up with the programmers??

No, but I do hope that one of the support
team or a motivated software engineer reads
this..

If they do I'd suggest the following:

Add a new parameter to [IpDrv.MasterServerUplink]:
ExplicitSourcePorts=List|""
default:"" (empty) ==> Current implementation

List: The 2 ports, for example:
7777,7787

This would make UTS send the port info to the MS
in an explicit msg, maybe overriding the current mapping.
The beauty of implementing it this way is that it is
very low risk: the current implementation would not be
affected too much.

Better, of course, would be to change the code in a way
so that it uses ExplicitSourcePorts instead of auto-determining
and then overriding them. Also, since this parameter would
be optional, the current implementation should not be affected
too much and should be a low risk task as well.

In any case, all we would have to do is to forward,
lets say, 7777,7778, 7787, 7788 to the appropriate
internal destination ports.

Anyway, it looks as though we have the
following options:

a) Don't host a UT2k3 server

b) Try IPTables
Not everybody has that option.

c) more IPChains research to determine if it possible to
configure the source port for outgoing connections
depending on the internal source port.
Won't help the "router-in-a-box" users.

d) hack the outgoing source port allocation
strategy on my linux router..
Won't help the "router-in-a-box" users, either.

e) wait for the UT gods to change the implementation

Are there any other options?


-=SPP=- Turner

[legacy-UT2003Turner]

Option b (IPTables) fixed the problem.

I could not find a good solution to
specify outgoing source ports in
IPChains/Masq.

-=SPP=-Turner

[legacy-TheSoupNazi{HoC]

Can you dumb that down for those of us that are, well, dumb.;)

What is IPTables?

[legacy-UT2003Turner]

"The netfilter/iptables project is the Linux 2.4.x / 2.5.x firewalling subsystem.It delivers you the functionality of packet filtering (stateless or stateful), all different kinds of NAT (Network Address Translation) and packet mangling. "
(http://www.iptables.org/ or http://www.netfilter.org/)

-=SPP=-Turner

[legacy-Shatterproof]

A question Turner...
Is it possible that the correction you described in your previous post could be implemented on the UT server alone, or would it require revising the master server code to listen for the new messages? :

Add a new parameter to [IpDrv.MasterServerUplink]:
ExplicitSourcePorts=List|""
default:"" (empty) ==> Current implementation

List: The 2 ports, for example:
7777,7787

This would make UTS send the port info to the MS
in an explicit msg, maybe overriding the current mapping.
The beauty of implementing it this way is that it is
very low risk: the current implementation would not be
affected too much.


Thanks for the information by the way. When I look at a problem for too long I start to wonder if I was missing something obvious. I can see now that I wasn't. :D
I have an all-in-one NAT router (DLink DSL-504) which requires that you specify a source (WAN interface) port and destination (LAN interface) port for each forward (quirky I know...) If I have understood your post correctly, there is no way it will work with the master until some kind of fix is released.

[legacy-UT2003Turner]

> Is it possible that the correction you described in
> your previous post could be implemented on the
> UT server alone, or would it require
> revising the master server code to listen for
> the new messages

I don't think it is possible to implement the proposed
scheme without changing the Master Server code.

I believe that this would require a UT server
change as well. Even though I faintly remember
seeing Gamespy stuff when I was looking at
some UT scripts. So. It may (or may not) be
worth investigating if the MasterServer
registration can be changed with the help
of a server side Mutator.

Technically, I think is possible to minimize the
implementation effort by utilizing the Gamespy
heartbeat. There may be non-technical reasons
why they didn't do that in the first place, though.

> If I have understood your post correctly, there is
> no way it will work with the master until some
> kind of fix is released.

Yes, that's what I think. I have contacted tech
support a while ago, but have not heard back.
I came to the conclusion that we are too small
a group to have any impact on the feature set
of UT2k3.

> I have an all-in-one NAT router (DLink DSL-504)

I am not familiar with the capabilities of the
DSL-504 but if you had the option of specifying
the source ports of outgoing connections you may
be able to get UTS registered.

> Thanks for the information by the way.

You are welcome.

-=SPP=-Turner

[Mark Rein]

This stuff is way over my head but I wanted you guys to know the devteam is looking into this issue.

[legacy-Shatterproof]

Thanks for the update Mark.

If it helps at all I had a look at my ports on the master server and they are variable, in the 10000 to 12000 range despite the fact I have stuck with the 7777 default. Further investigation revealed this range appears to be what the NAT router uses on the WAN interface.

Basically, I confirmed that my D-Link all-in-one gizmo is behaving in exactly as Turner described for his Linux firewall/router.

[legacy-Dogsbody.org]

I just thought i would post to the bottom of this as it is now April 2003 and we still have this issue. :confused:

I want to host UT2003 servers I really do but this bug isn't letting me, sure people can still connect/use my servers if they click on the link from my webpage but I am getting pages and pages of logs from my firewall where people are querying my server on the wrong port due to this bug and lets face it people are drawn to the public servers from the game server browser and other like apps :(

Mark Rein, I see you have posted in this thread, I hope you still have notifications on and could perhaps give this a little bit of a nudge now that things seem to have settled out a bit with people screaming for changes to the game :)

Thank you for listening

Dan

[NakedApe]

Well, it's now June 2003 and still no luck... :cry:


Originally posted by UT2003Turner
With IPChains/Masq, however, the outgoing source ports
get assigned sequentially. Therefore, the two UDP packets
sent by the UTS allocate two outgoing source ports
on the FW's external interface:

N and M. The bad news is that the UT protocol seems
to be based on (N,N+1) and (M,M+1) (not sure if M-N must
be 10, but I am sure it has to be >1 )

I don't think this is the problem. The point of NAT is that it remembers where the packets came from. Even if packets are sent to N,N+1 they would be sent to N,M on the server. I think the problem lies in keeping state (especially with UDP which has no connection concept). Most firewalls are able to "fake" state for UDP packet streams (by applying various analysis techniques) but in this case there probably isn't enough for the state engine to recognize that the packets are related. The packets that are sent to the master server aren't (in any logical sense) part of the "connection" that the client and server establishes.

The problem is that Epic didn't think through the implications of NAT properly. Considering that they have added some options that indicate that they have thought about it at all (e.g. BehindNAT) they must have drawn a blank on this. ;)

I agree this needs fixing by modifying the way that ports are determined by the master servers. They should always have this information sent by the client (as does the GS/ASE servers) and not trying to guess it from the packets.

For users of software firewalls (iptables, pf etc) one possible workaround is to make NAT use the source of the original packet (later versions of PF can do this, not sure about iptables & co). Users of router-in-a-box:s are SOOL.

[legacy-Shatterproof]

Originally posted by Naked Ape
For users of software firewalls (iptables, pf etc) one possible workaround is to make NAT use the source of the original packet (later versions of PF can do this, not sure about iptables & co). Users of router-in-a-box:s are SOOL.

I use a D-Link DSL-504 NAT router and I originally had this problem as well (see my previous posts). Updating the firmware on the router seems to have resolved the problem. As usual for home networking kit there is no decent documentation for my D-Link, but my best guess by looking at it's behaviour they changed the port forwarding works.
As usual I have set up port forwarding for UDP 7777,7778,7787,7788 to the internal IP of my UT2003 Server.

It appears for outbound UDP packets from the UT server the router used to alter the source port and make the appropriate entry in the NAT table. Inbound UDP from the Master Server would get back to my UT2003 server, but importantly as far as the Master was concerned it was commmunicating with the source ports used by the NAT router's WAN interface, not the server.
Clients connecting directly could get through no problem because those packets had destination ports that the router had been told to forward straight to the UT2003 Server.

Since the firmware update it appears that port forwarding behaves differently - the best way I can describe it is 'port forwarding and fixing'. The clue here is I can no longer specify an external (WAN interface) and internal (LAN interface) destination port in the port forward table - just the protocol, port and destination IP.
It appears that any outbound packets that match an entry in the port forward table (Protocol, Source IP address and Port) do not have their source port modified as they are 'NATed' by the router. This of course means the Master server sees the correct ports for my UT2003 server.

To me it appears, like Naked Ape says, the only workaround at the moment is to fix the source ports. With hardware appliances it's going to be very hit-and-miss. I would expect any enterprise kit to have this functionality, it's certainly possible on the Netscreen where I work. With home kit - (shrugs)....

[NakedApe]

Originally posted by Shatterproof

...
Inbound UDP from the Master Server would get back to my UT2003 server, but importantly as far as the Master was concerned it was commmunicating with the source ports used by the NAT router's WAN interface, not the server.
Clients connecting directly could get through no problem because those packets had destination ports that the router had been told to forward straight to the UT2003 Server.


Yes, this is what I see with my OpenBSD PF-based firewall as well. I'm going to upgrade to PF3.3 which has the option to not modify the source port and see if it fix the problem (as I believe it will).

The Atari support also promised to get back to me on this issue after the weekend.

[NakedApe]

I'm posting this in the hope that this thread might turn up in more searches so people stop asking about it all the time... I'm getting tired of parroting about this bug. :sour:

I hope I get the keyword right (not to general), and I hope it doesn't offend anyone...

ping info N/A NA server won't wont show up behind router NAT redirect forward port

And a short explanation of the bug for those who don't understand the somewhat techncal babble above... :)

A server running behind some implementations of NAT (which most so called "broadband routers" use) will not show info or ping nor will it be possible to connect from the in-game browser. It is possible to connect directly to the IP or via ASE or Gamespy.

[legacy-Dogsbody.org]

Please!! I beg you, this is now filling yours stats site with crud too...

http://ut2003stats.epicgames.com/index.php?stats=servers&SearchName=dogsbody

... I run two servers, no more and yet my two servers are listed 21 times so far with differant port numbers each time!!

This has to be the simpliest thing to patch and fix!! The fix is entirely on the master server, just get the master server to look inside the packet for the game server details and not the source details of the packet!!

This is not just affecting me, anyone behind a SOHO NAT firewall.

....please!! :cry:

[legacy-Forplaz]

Is there any updates on this bug or when it might be fixed or any work arounds?

[legacy-Tok-ra]

ok i am having the same problem as above so i wont babble on about a solution because it seems there is'nt one :-(

i have however had 1 connection to my server while i was tinkering around with my nat/redirect port setting when i looked at the ping in the browser it still said n/a but a player was there
he said he had no problems connecting at all

i had just unredirected the ports 7777/7778 ect.... when i turned round to look it to see if had connection loss, i saw my friend enter my server i was very excited after a week of trying i thought i had cracked it we played for around 10 minutes but then he timed out and couldnt get back in

i am not using a hardware router im using wingate all in one
im sure if i have one connection then i can get more just have to figure out how to use this software correctly theres a lot of stuff im unsure of so ill keep trying hopefully a solution will come

Tok-Ra

[NakedApe]

Tok-ra, connecting from ASE or Gamespy works just fine. Also if you have the servers IP in "Favourites" and connect from there. It's just the built-in browser (that gets its info from the master list server) that fails.

Whether you use a hardware router or software doesn't matter, it's the process of translating the address that provokes this bug. Any NAT device which doesn't preserve the source port will be affected.

[legacy-Tok-ra]

well it seems like ill just have to have all my friends install gamespy bit silly though because no one from the built in browser will be able to connect but what the hell atleast my friends can still join :-) cool

thanks for the reply mate

hey if you ever fancy a insta/low grav game giz a shout

Tok-Ra

[legacy-Orion Pax]

ya know...altho i know this is a VERY old post...going on for a year now...but i thought i might add this as i just thought of it....

I know that some of ya'll having this trouble becuase of your router.....most routers or all use NAT...

I have a linksys 4 port router.......
it has port forwarding just like all the others...
now i dont have this problem that ya'll have....my router isnt sending the wrong port back to the MS....or what ever it is....

BUT....i did notice in my routers port forwarding settings....that you can specify TWO ports......internal port and external port.....the internal port is the one going to your computer...the correct one that is needed.....the external port is the one that they are trying to use from the MS.

if you router's have these settings try setting them like that...to get the external port check your routers logs....

i know this might not fix every one but its a really good suggestion and i have seen alot of people ask what INT and EXT ports where on their routers settings....and i finally fount out my self just today when messing with it....

well i hope this bit helps out alot of ya'll....Good Luck!

[NakedApe]

Glad to hear it is working for you Orion!

But the INT/EXT port is not the reason it works. The INT/EXT is for redirecting to a different port, say 8888 to 7777. Most SOHO routers have a single entry that is used both on the "outside" to listen on, and as the destination of the forwarded packet.

[legacy-alphaman36]

I agree, that is not the reason it is working. My current firewall is configured just like that, and it just plain doesn't work

[legacy-Orion Pax]

didnt say that is why it works....i dont have it set on my router.....mine just doesnt seem to have that problem at all...

but i still dont get why that wouldnt work.....



The INT/EXT is for redirecting to a different port, say 8888 to 7777.


isnt that what you want....to change the wrong port they are connecting to into the port that you want it to be?

on mine each entry has BOTH....you can specify an internal port AND an external port.....so lets say you are using 7777 for your server...and they are trying to connect on 32304.....set internal port to 7777 and external to 32304......

sounds like to me that is what you want....is to redirect from the bad port to the right port....it might not stop the wrong port being sent.....but it sounds like a fix to me....

well at any rate if mine had that problem i would test it....lol...but since it dont i couldnt tell ya....

well cant say i didnt try....thought i almost had ya'll some good info...

[NakedApe]

Originally posted by Orion Pax
isnt that what you want....to change the wrong port they are connecting to into the port that you want it to be?

The problem is that you don't know beforehand which "wrong port" they will try to connect to...

[legacy-lanline]

See this post.
http://www.ina-community.com/forums/showthread.php?s=&postid=4218840#post4218840

I haven't got a work around either.
I used to have a netgear and it worked without any configureing. I have other aplications that need port forwarding thats why I upgraded. Thinking of going back now.....

[legacy-Orion Pax]

If its sending a random port number i can see where that would be a problem, which is most likely it. But if its not then just check your routers log or ask some one from the out side to tell you what port its showing on!

Thats weird....I have heard so many people having issues with LinkSys routers and here i own one and dont have a problem with that! The only problem i have is every 3 -6 months it fries. They are not very durable..... And I keep all my junk off of it....i keep it standing on its side instead of flat down on the desk so it can breathe better. And no luck...i am considering Net Gear my self next time.....Although I do believe that on Link's web site there was an issue about their power supplies being ****ty.....but any ways....

[legacy-WhipKiKeN]

Hey UT2003Turner, Im having the same problems. Can you post a pic up of your NAT setup in your web based Administration?

[legacy-MegaDaisy]

WhipKiken - are you having this problem with UT2004?

MegaDaisy

[legacy-PyrettaBlaze]

The problem is still in UT2k4. They need to change the code so that it reads what ports its on rather than using the port that its coming in on the physical layer.

Does not look like there will be a fix for this any time soon seeing this thread is a year old....

[legacy-MegaDaisy]

Oh well - time to splash out on a Linksys router then :) (currently have a Belkin router - used to use Winroute which was never a problem).

MegaDaisy