This file often shows the reason why UT3 chickens out to desktop, and looking at it prevents a lot of guesswork

Open you documents folder and go to "My Games\Unreal Tournament 3\UTGame\Logs". At the end of the file "launch.log", there may be lines like these:
Critical: appError called:
Critical: Result failed
at .\Src\D3D9Viewport.cpp:180
with error E_OUTOFMEMORY
Critical: Windows GetLastError: The operation completed successfully. (0)
Critical: appError called:
Critical: Rendering thread exception:
Result failed
at .\Src\D3D9Viewport.cpp:180
with error E_OUTOFMEMORY
Look at the parts in bold. They gave great hints about what's wrong. In my case, I remembered about a parameter that I changed with RivaTuner a few days ago.