Pretty huge memory leak issue in November UDK? (Update: SOLVED)

[Lacabra]

I could totally be using the term "memory leak" incorrectly.

Basically, levels don't appear to be being unloaded from memory. This is extremely easy to reproduce. It happens even on a clean install of UDK. Not sure if it happens in earlier builds. It pretty much makes our game unshippable though.

Here's the scenario:
Run UDK.exe. Task manager says UDK.exe is using 314MB RAM (or whatever).
"open vctf_necropolis" in console. UDK.exe is now using 945MB.
"open vctf_necropolis". Using 1164MB.
"open vctf_necropolis". 1387MB.
"open vctf_necropolis". 1610MB.
"open vctf_necropolis". 1819MB.
"open vctf_necropolis". 2064MB.
And so on until you crash because you ran out of memory.

This happens with level streaming too. In our game, we use level streaming a lot, and what's happening is every time a level loads, we use more ram, and when it gets unloaded, we don't use less. Eventually we inevitably crash.

I haven't tried this as such, but I suspect that even you had this simple level streaming setup, running in and out of the level streaming volume enough times would cause a crash from lack of memory.


Edit: This still happens on February and March builds. It started on the May build of 2012, so let's everyone drink to most of a year of leaking memory, I guess.


Edit edit: This is fixed in July 2013 UDK. Many thanks, Epic guys.

[RNG]

Tried this on 2012-07 and it happens on that version as well. Loading times were way too long too. I tried it on a far older version (2011-05) and I can confirm there is no memory leak on it AND map reloaded way, way, WAY faster than the 2012-07.

Additional info:

Both versions were 32-bit, dx9, clean installs with nothing cooked.

Ran it on Win7 x64 laptop.

Edit: Tried it on packaged 2012-07, it still has the leak, but it seems to be slower to leak (it reached 1.7gb after reloading the map 8 times). Loading was still way too slow compared to 2011-05 and performance seemed a little slower too (probably has something to do with heavier post processing).

[Lacabra]

Thanks for the confirmation and info. I'll check tomorrow if I have any other old versions of UDK lying around to try to narrow down what version this happened in - though probably Epic have better ways to figure that out.
Was the initial load of the map also slower on the newer versions, or only subsequent reloads? Just wonderin'.

Also, I didn't provide my specs because I think they're irrelevant, but it's a Win8 x64 install on a PC with 16GB RAM (which is probably why I didn't notice this earlier - I noticed we were using too much RAM but figured something was wrong at my end rather than with UDK). The issue is also just as present whether you're using the 32 or 64 bit executable.
I've also tested my game on a great many other PCs and the issue is not miraculously absent on any of them.

Also, the Mac build of my game crashes straight up, though it's entirely possible it's completely unrelated. The only reason I bring it up is part of the log says "Garbage collection is OFF!" ...which sounded like a relevant thing but might not be.

[RNG]

2012-07 had longer initial map load and subsequent reloads of the same map (vctf-necropolis) took about the same amount of time as the initial load. On 2011-05, initial load was similar, but subsequent reloads were much, much faster. I haven't timed them, but 2012-07 took about 10 seconds each, and 2011-05 took about 3-5 seconds max.

I don't have the latest November version here on my laptop so I can't test on it, I think I have a previous version somewhere, so I might test it on that as well.

[XilenceX]

Also, the Mac build of my game crashes straight up, though it's entirely possible it's completely unrelated. The only reason I bring it up is part of the log says "Garbage collection is OFF!" ...which sounded like a relevant thing but might not be.

Well garbage collection set to OFF sounds like a pretty **** big problem. o,0 Is there any way to activate it in UDK?

[RNG]

Tested in 2012-05 and it does it too on 2 different systems. 2011-05 is still clean on different systems. I have tested this on the latest November release with empty project install, it still happens even with the ExampleEntry map, I reached 850MB RAM usage with ExampleEntry and usage didn't dropped after even after 10 minutes.

To recap the tested versions:

Doesn't leak
2011-05

Does leak
2012-05
2012-07
2012-10 (November)

[weephun]

Thanks for the great repro and even helping narrow down when it started. We've been able to reproduce this here and we've entered it into our bug database.

Thanks!
Preston

[Lacabra]

Thanks Preston. Is there any chance that this can be fixed with a patch to the November build, or will we have to wait for whenever the next build is?

[tegleg]

ah this explains a lot, good to hear epic is aware of it, hope its at the top of the list to fix
means i need to delay any game releases i have ready

[Lacabra]

This issue still exists in the new February 2013 build. In related news, I have died.

[tegleg]

gosh blimey epic wtf?
i guess they were more focused on introducing yet another really expensive 3rd party product.
they really should change their udk catch phrase to
"Bugs?... Just add another new feature"

sorry to hear you are dead now perhaps in your ghostly incarnation you can wander into epic hq and fix this bug for them

[XilenceX]

gosh blimey epic wtf?
i guess they were more focused on introducing yet another really expensive 3rd party product.
they really should change their udk catch phrase to
"Bugs?... Just add another new feature"

sorry to hear you are dead now perhaps in your ghostly incarnation you can wander into epic hq and fix this bug for them

Oh my that is just plain embarrasing for Epic Games. I was considering to use this system in my game.... well so much for that. lol

[RNG]

Hawken uses the latest UE3 build afaik, I wonder if it has this leak too.

Edit: No leakage on Hawken.

[Lacabra]

Could be interesting to test.

[Lacabra]

Narrowed this down: it started in the May 2012 UDK. March is fine.