MasterServer: Wrong Port# - A NAT Problem

This is based on an analysis of what is
going on on the network. My setup:

UTS: Dedicated UT Server
FW : Firewall/Router/NAT (Linux/IPChains/Masq)
MS : MasterServer

UTS<--(a)-->FW<--(b)-->"the internet"<--->MS

I looked at the traffic at (a) and (b).

In order for UTS to register with the MS (UT2003master1)
it must open a TCP connection (SYN). The numbers
are port numbers.

(a): UTS: 1046 --> FW: 28902
(b): FW :61356 --> MS: 28902

So far, so good.

There are some UDP packets (status) to and
from 216.177.89.34 and 216.177.89.28. We
focus on the communication with the MS, so
we just ignore them here.

After the TCP three-way handshake is
completed (SYN, ACK from MS and ACK
from UTS) there is more UDP traffic to
other machines (status). Now, using
the established TCP link, the key (and
some other stuff) is sent and "APPROVED"
by the MS.

Next, UTS sends a UDP packet to MS:

(a): UTS: 7778 --> FW: 28902
(b): FW :61358 --> MS: 28902

Some TCP chitchat and then 2 more
UDP packets UTS-->MS:

(a): UTS: 7777 --> FW: 28902
(b): FW :61359 --> MS: 28902

(a): UTS: 7778 --> FW: 28902
(b): FW :61360 --> MS: 28902

More TCP traffic and we are done
with the registration.

http://ut2003master.epicgames.com/se...t/full-all.txt reports
"<MyIP> 61359 61360".

Those two ports happen to be identical to the source
ports the FW used to communicate with the MS.

That is the problem - what is the solution?

The correct port (7787) is communicated
via the heartbeat. Not sure if the Masterserver
itself ever request it, but other hosts did.

Could there be a solution other than
replacing the firewall/router?

-=SPP=-Turner

A few quick questions and thoughts.

First, excellent information here.

My router is a Dlink 614+. It is a bit quirky, but it does work fairly well. When I originally purchased it, I had trouble running a dedicated UT server in my home. After a few router firmware upgrades, Dlink finally got it to where the UT server worked fine through the router. Then when UT2003 was release, I could not set up a server that would work with this new game. All this despite the fact that the configuration was similar if not exact???

So, why does the old game work, but not the new one????

What is the difference in communications that the two games use???

More info for you. My game server will actually appear in the master server list. I do however have no ping time, and instead I have the the "N/A" shown. Despite this, if I give my IP to several of my friends, and they add the IP to their favorites list, they are able to connect fine, and we have our fill of killing each other. Obviously the server does work.

So, if my friends can connect with the IP favorites method, without specifying any ports, why can't the master server figure out the ports???

I think there may still be some things to try, so please let me know the following things if you know the answers.


What router are you using????

Once you have configured the server and started it, what port is the first packet of data sent out from the game server to the master server????

The heartbeat which you say has the correct port associated with it. Is this an incoming heatebeat to the UT server, or an outgoing beat to the master server????

In the "data" sent from the UT server to the master server, is the correct port assignments for the UT2003 game server included???

Let me know and I will keep a check!

John

UT2003Turner you are a STAR!!! :-)

I have exactly the same problem and have seen a number of posts on here about the same problem...

http://www.ina-community.com/forums/...hreadid=208838

...while lots of people have tried to dubug it your the first that I have seen that has debug'd it on both sides of the firewall.

Now, does anyone have any ideas for how to flag this up with the programmers??

I am going to start a new message and request a list of people using routers, which they are using, if they can set up a server, etc. Maybe we can get some data which we can further use??? We'll see.

Thank you for your feedback and praise

IPchains/Masquerading works like this:

Outgoing connections (from the FW's point of view)
use a certain default port number range, like a
pool of (high) port numbers: lower ports
are reserved for the FW-server itself, eg services
running on the very same FW, and are, therefore,
not being translated (NAT'd).

IIRC the pool contains port numbers >(=) 60000.
It increments the port numbers one by one,
regardless of the original internal source
port number.

I am sure that there are router boxes
that do the same.

Anyway, what seems to be happening is this:

The UT2k3 server sends 2 packets to the MasterServer.
The MS then looks at the source-port of the packets in
order to save them for later use.
The other UT clients use those 2 ports to connect to
the UTS in question.

As long as the NAT gateway (router) uses the same
(source) port numbers as the UTS, there are no problems.
This is a bit risky, because there could be 2 machines
using the same ports for different applications. So,
the IPChains/Masq implementation isn't that bad as
it may seem at first.

The master server determines
"<IP> Port-N Port-M", lets say,
"<IP> 7777 7787" or "<IP> 8040 8050".

With IPChains/Masq, however, the outgoing source ports
get assigned sequentially. Therefore, the two UDP packets
sent by the UTS allocate two outgoing source ports
on the FW's external interface:

N and M. The bad news is that the UT protocol seems
to be based on (N,N+1) and (M,M+1) (not sure if M-N must
be 10, but I am sure it has to be >1 )

So, looking at "full-all.txt" we can see that most
UTSs register with M-N=10

I fear that in cases where M-N=1 there
is not much we can do

A UT software engineer would have to
change the MS' source port acquisition method
and/or change the currently used client-server
protocol. My guess is that there are
hardcoded +1 dependencies.

Also, this implies that reconfiguring
port-forwarding on the fly (meaning after
the registration took place) won't work either:

The clients would try to talk to (N,N+1) and
(M,M+1). Problem is that in cases where
M-N==1 (port difference :1 ) port N+1=M
- and we cannot have the same one port
forwarded to 2 different ports..

Well, assigning different ports in UTS
wouldn't help either, because the FW
allocates outgoing ports one by one,
since it does that on a per connection
basis. (apologies: UDP is connection-less,
but the FW has to keep track of those
"connections", so, here, I call
them connections, too

This is the dilemma that we cannot solve (easily).

IPTables may solve this more elegantly, but on
the other hand, there seem to be other NAT GWs
out there that have the same problem.

I believe that the "full-all.txt" shows us
a number of poor users with exactly the
same problem. Easily found by looking at the
delta of the two port numbers. Admins
seem to fall into several categories:

a) M-N== 1 ==> poor sucker out of luck (like myself)
b) M-N==10 ==> Lucky SOB
c) M-N >10, M-N <-10 ==> No idea, but may be like (b)
d) The rest: ==> No idea, but may be like (a)

PhantomII:

> So, why does the old game work, but not the new one????

Because the protocol's changed?

> What is the difference in communications that the two games use???

I don't know (never really looked at the old UT protocol), but I guess
that the source port acquisition on the Master Server is different
and/or the whole port allocation strategy is different. If you take
a network level trace and look at the packets you may be able to
figure out what the differences are. Not sure how that would help us,
because we already know that there are differences. You have proven it
( same router, same setup: old UT works, new UT doesn't).

> More info for you. My game server will actually appear in the master server list.
> I do however have no ping time, and instead I have the the "N/A" shown.

My server shows up, too. That's not the problem in this case.
People can't connect.

1) Can your friends ping your server, or do they see N/A, too?

2) What are your N and M values? Is the difference between the two
10 or 1? My guess it is 10 (check full-all.txt for your IP address)

> So, if my friends can connect with the IP favorites method,
> without specifying any ports, why can't the master server
> figure out the ports???

I think the MS can be viewed as a database. It keeps track of its clients
(the UTSs) and tells the UTclients that there are UTSs and what ports they
are supposed to use. The ping people see are pings from their machines,
not from the server.
The reason why you can't "ping" your NAT'd UTS may be that your client
sees the external IP address, and then the routing may fail..

> What router are you using????

A dedicated Linux box with IPchains and Masquerading.

> Once you have configured the server and started it, what port is the
> first packet of data sent out from the game server to the master server????

The very first UDP packet from UTS-->MS:

(a): UTS: 7778 --> FW: 28902
(b): FW :61358 --> MS: 28902

Very first TCP packet:

(a): UTS: 1046 --> FW: 28902
(b): FW :61356 --> MS: 28902

> The heartbeat which you say has the correct port associated with it.
> Is this an incoming heartbeat to the UT server, or an outgoing beat
> to the master server????

This is part of the problem: The Master server never gets
the heartbeat. The first \heartbeat and \gamename are sent to 216.177.89.34.
This could be a gamespy server, I don't know. It is not the MS, though.

It looks like this:
"\heartbeat\7787\gamename\ut2" (actual payload of that packet)

Sorry, I didn't make that clear in my first post: The MS _could_
know the internal port number if it had the info associated with
the /heartbeat msg. But it is never sent to the MS.


> In the "data" sent from the UT server to the master server,
> is the correct port assignments for the UT2003 game server included???

I checked a the TCP packets but failed to find them. That's binary
data and could be encrypted anyway. I believe that if they are
in there, the MS fails to look at them. If they are not in there,
the MS does not have an alternative: it must use the NAT GW's
source ports - which is kind of cool, because it cuts down
the intial registration traffic. Sadly, this strategy
does come at a cost..

Dogsbody.org:
> Now, does anyone have any ideas for how to
> flag this up with the programmers??

No, but I do hope that one of the support
team or a motivated software engineer reads
this..

If they do I'd suggest the following:

Add a new parameter to [IpDrv.MasterServerUplink]:
ExplicitSourcePorts=List|""
default:"" (empty) ==> Current implementation

List: The 2 ports, for example:
7777,7787

This would make UTS send the port info to the MS
in an explicit msg, maybe overriding the current mapping.
The beauty of implementing it this way is that it is
very low risk: the current implementation would not be
affected too much.

Better, of course, would be to change the code in a way
so that it uses ExplicitSourcePorts instead of auto-determining
and then overriding them. Also, since this parameter would
be optional, the current implementation should not be affected
too much and should be a low risk task as well.

In any case, all we would have to do is to forward,
lets say, 7777,7778, 7787, 7788 to the appropriate
internal destination ports.

Anyway, it looks as though we have the
following options:

a) Don't host a UT2k3 server

b) Try IPTables
Not everybody has that option.

c) more IPChains research to determine if it possible to
configure the source port for outgoing connections
depending on the internal source port.
Won't help the "router-in-a-box" users.

d) hack the outgoing source port allocation
strategy on my linux router..
Won't help the "router-in-a-box" users, either.

e) wait for the UT gods to change the implementation

Are there any other options?


-=SPP=- Turner

Option b (IPTables) fixed the problem.

I could not find a good solution to
specify outgoing source ports in
IPChains/Masq.

-=SPP=-Turner

Can you dumb that down for those of us that are, well, dumb.

What is IPTables?

"The netfilter/iptables project is the Linux 2.4.x / 2.5.x firewalling subsystem.It delivers you the functionality of packet filtering (stateless or stateful), all different kinds of NAT (Network Address Translation) and packet mangling. "
(http://www.iptables.org/ or http://www.netfilter.org/)

-=SPP=-Turner

[legacy-Shatterproof]

A question Turner...
Is it possible that the correction you described in your previous post could be implemented on the UT server alone, or would it require revising the master server code to listen for the new messages? :

Add a new parameter to [IpDrv.MasterServerUplink]:
ExplicitSourcePorts=List|""
default:"" (empty) ==> Current implementation

List: The 2 ports, for example:
7777,7787

This would make UTS send the port info to the MS
in an explicit msg, maybe overriding the current mapping.
The beauty of implementing it this way is that it is
very low risk: the current implementation would not be
affected too much.

Thanks for the information by the way. When I look at a problem for too long I start to wonder if I was missing something obvious. I can see now that I wasn't.
I have an all-in-one NAT router (DLink DSL-504) which requires that you specify a source (WAN interface) port and destination (LAN interface) port for each forward (quirky I know...) If I have understood your post correctly, there is no way it will work with the master until some kind of fix is released.

> Is it possible that the correction you described in
> your previous post could be implemented on the
> UT server alone, or would it require
> revising the master server code to listen for
> the new messages

I don't think it is possible to implement the proposed
scheme without changing the Master Server code.

I believe that this would require a UT server
change as well. Even though I faintly remember
seeing Gamespy stuff when I was looking at
some UT scripts. So. It may (or may not) be
worth investigating if the MasterServer
registration can be changed with the help
of a server side Mutator.

Technically, I think is possible to minimize the
implementation effort by utilizing the Gamespy
heartbeat. There may be non-technical reasons
why they didn't do that in the first place, though.

> If I have understood your post correctly, there is
> no way it will work with the master until some
> kind of fix is released.

Yes, that's what I think. I have contacted tech
support a while ago, but have not heard back.
I came to the conclusion that we are too small
a group to have any impact on the feature set
of UT2k3.

> I have an all-in-one NAT router (DLink DSL-504)

I am not familiar with the capabilities of the
DSL-504 but if you had the option of specifying
the source ports of outgoing connections you may
be able to get UTS registered.

> Thanks for the information by the way.

You are welcome.

-=SPP=-Turner

[Mark Rein]

This stuff is way over my head but I wanted you guys to know the devteam is looking into this issue.

[legacy-Shatterproof]

Thanks for the update Mark.

If it helps at all I had a look at my ports on the master server and they are variable, in the 10000 to 12000 range despite the fact I have stuck with the 7777 default. Further investigation revealed this range appears to be what the NAT router uses on the WAN interface.

Basically, I confirmed that my D-Link all-in-one gizmo is behaving in exactly as Turner described for his Linux firewall/router.

I just thought i would post to the bottom of this as it is now April 2003 and we still have this issue.

I want to host UT2003 servers I really do but this bug isn't letting me, sure people can still connect/use my servers if they click on the link from my webpage but I am getting pages and pages of logs from my firewall where people are querying my server on the wrong port due to this bug and lets face it people are drawn to the public servers from the game server browser and other like apps

Mark Rein, I see you have posted in this thread, I hope you still have notifications on and could perhaps give this a little bit of a nudge now that things seem to have settled out a bit with people screaming for changes to the game

Thank you for listening

Dan

[NakedApe]

Well, it's now June 2003 and still no luck... :cry:

Originally posted by UT2003Turner
With IPChains/Masq, however, the outgoing source ports
get assigned sequentially. Therefore, the two UDP packets
sent by the UTS allocate two outgoing source ports
on the FW's external interface:

N and M. The bad news is that the UT protocol seems
to be based on (N,N+1) and (M,M+1) (not sure if M-N must
be 10, but I am sure it has to be >1 )

I don't think this is the problem. The point of NAT is that it remembers where the packets came from. Even if packets are sent to N,N+1 they would be sent to N,M on the server. I think the problem lies in keeping state (especially with UDP which has no connection concept). Most firewalls are able to "fake" state for UDP packet streams (by applying various analysis techniques) but in this case there probably isn't enough for the state engine to recognize that the packets are related. The packets that are sent to the master server aren't (in any logical sense) part of the "connection" that the client and server establishes.

The problem is that Epic didn't think through the implications of NAT properly. Considering that they have added some options that indicate that they have thought about it at all (e.g. BehindNAT) they must have drawn a blank on this.

I agree this needs fixing by modifying the way that ports are determined by the master servers. They should always have this information sent by the client (as does the GS/ASE servers) and not trying to guess it from the packets.

For users of software firewalls (iptables, pf etc) one possible workaround is to make NAT use the source of the original packet (later versions of PF can do this, not sure about iptables & co). Users of router-in-a-box:s are SOOL.

[legacy-Shatterproof]

Originally posted by Naked Ape
For users of software firewalls (iptables, pf etc) one possible workaround is to make NAT use the source of the original packet (later versions of PF can do this, not sure about iptables & co). Users of router-in-a-box:s are SOOL.

I use a D-Link DSL-504 NAT router and I originally had this problem as well (see my previous posts). Updating the firmware on the router seems to have resolved the problem. As usual for home networking kit there is no decent documentation for my D-Link, but my best guess by looking at it's behaviour they changed the port forwarding works.
As usual I have set up port forwarding for UDP 7777,7778,7787,7788 to the internal IP of my UT2003 Server.

It appears for outbound UDP packets from the UT server the router used to alter the source port and make the appropriate entry in the NAT table. Inbound UDP from the Master Server would get back to my UT2003 server, but importantly as far as the Master was concerned it was commmunicating with the source ports used by the NAT router's WAN interface, not the server.
Clients connecting directly could get through no problem because those packets had destination ports that the router had been told to forward straight to the UT2003 Server.

Since the firmware update it appears that port forwarding behaves differently - the best way I can describe it is 'port forwarding and fixing'. The clue here is I can no longer specify an external (WAN interface) and internal (LAN interface) destination port in the port forward table - just the protocol, port and destination IP.
It appears that any outbound packets that match an entry in the port forward table (Protocol, Source IP address and Port) do not have their source port modified as they are 'NATed' by the router. This of course means the Master server sees the correct ports for my UT2003 server.

To me it appears, like Naked Ape says, the only workaround at the moment is to fix the source ports. With hardware appliances it's going to be very hit-and-miss. I would expect any enterprise kit to have this functionality, it's certainly possible on the Netscreen where I work. With home kit - (shrugs)....