Package flag "ClientOptional" not working (sets dependency for demo playback)

[RattleSN4K3]

There is this package flag ClientOptional which can be set after compiling with the "SetPackageFlags" commandlet of UT3 (or with the upkg file). This flag ClientOptional is intended as players do not need to download these flagged packages. They can abort the autodownload but continue to connect to the server. This works fine. Another features of this package flag is/was that this package is not required for demo playback.

Create the mutator for UT2004, compile it, create a game with that mutator and record a demo. Afterwards you can close UT2004, delete these created mutator files, and start UT2004 again. Now you can play back that demo out of the menu (no extra console command needed).

Packagename: ProblemClientOpt
ProblemClientOpt\Classes\ProblemClientOpt.upkg

[Flags]
AllowDownload=True
ClientOptional=True
ServerSideOnly=False

ProblemClientOpt\Classes\PCOMutator.uc

class PCOMutator extends Mutator;

function ModifyPlayer(Pawn Other)
{
    super.ModifyPlayer(Other);
    
    if (Other != none && Other.Controller != none && PlayerController(Other.Controller) != none)
    {
        PlayerController(Other.Controller).ClientMessage("ProblemClientOpt: Player "$Other$" spawned.");
    }
}

defaultproperties
{
}

Now you can do the same for UT3. Compile, start a game with active mutator "PCOMutator", record a demo (or use the menu option "Record demos"). Play some seconds, exit the game, delete the mutator, and start UT3 again.
Once you're trying to play that previously recorded demo, you will get a error message that a missing package "ProblemClientOpt" is required. The demo will not start.

Do you guys have any idea why this isn't working? Is there anything i have to set in addition? Is this working for you guys?
(I'm pretty confused because I did the same like months ago and it was working. ClientOptional flagged packages do not need to be installed. Strange.)

PS: Build Version: 3809

[Wormbo]

ClientOptional probably only applies to the initial negotiation between server and client. That kind of negotiation happens when recording the demo, not when playing it. As a result, the packages that were negotiated at recording time become mandatory for demo playback. If the demo player doesn't have these packages, it will not be able to decode the demo's recorded network data stream.

[RattleSN4K3]

AFAIK, there is no client object which got spawned on clients as mutators are serversided (at default). So the network stream could be "untouched". As I said, i already got it working somehow (if i remember correctly), I think I must check if the order is important. In UT2004, I called demorec manually (and probably clientsided = client demo). In UT3, i used the menu option which starts demo recording right before the game starts (which resulted into a server demo).

Any other idea to prevent the dependency on a clientoptional package (like pushing AllowDownload=false packages to clients)? I read somewhere that demo files are the same like a Server. UT3 simulates the data from the demo over a network steam and the ingame demoplayback acts like a normal client. So playing a demo should be the same as connecting to a server, right? But it does not allow to skip a download. Maybe a missing feature.

[Wormbo]

Again: The demo playback is not the same as connecting to a server, the demo recording is. Basically the demo recording driver connects to the local game and negotiates the network parameters, which includes network version and package map. This negotiation has direct impact on the network traffic and when a package is considered "known to the client", the traffic is most likely not readable without knowledge of that package's content.
Each network-relevant package's name table and export table have impact on the network traffic. A server can downgrade a package's generation number if the client uses an older generation of the same package, and likewise a client with a newer generation understands the server. If a client skips download of an optional package, the server will know and ignore corresponding objects when sending data. However, once the package map was built (which packages to use at all, and which generation of each package to use), network traffic itself becomes absolutely incompatible to other configurations, because references to the name tables and export tables of the packages no longer match.
As you already wrote, demos are dumped network traffic, so they contain references for a previously negotiated package map. The demo recorder automatically has all required packages, so optional packages are included unless you record on a client that doesn't have a certain optional package. But in that case, the optional package will always be ignored on playback, even if it exists then.

[RattleSN4K3]

I thought packages which have a generation (AllowDownload and not ServerSideOnly) can be skipped on playback or on connection. ClientOptional is quite useless if this feature is not working.

[Wormbo]

Client optional is not useless, you only tried to use it for something it's not meant to be used for.

[RattleSN4K3]

I checked the code (a mutator) which I was testing months ago. It's a serversided mutator which adds a client optional package. That mutator sets no dependency in a demo file. It is working, but not this easy example in first post. ...

[RattleSN4K3]

This is a stripped version of the mutator i was talking about. I created a ded server with the start argument "?Demo=DemoName" and the mutator "?mutator=ExampleClientOptionalServer.UTMutato r". I connected with another UT3 instance (on the same machine, no downloading) to this server. Once the player spawned the message "ClientOptional Package setup" was shown (and some log entries were listed in the log window). I played a while. I closed the client, savely closed the ded server (Ctrl+C), deleted these packages and started UT3 again. I double clicked the demo in the menu and the demo got loaded and played.

I think the combination of ClientOptional and DynamicLoad does not set the dependency (generation).

---------------------------------------------
Package: ExampleClientOptional

ExampleClientOptional\Classes\ExampleClientOptiona l.upkg

[Flags]
AllowDownload=True
ClientOptional=True
ServerSideOnly=False

ExampleClientOptional\Classes\ECOLPRI.upkg

class ECOLPRI extends UTLinkedReplicationInfo;

var PlayerController CachedPC;
var PlayerReplicationInfo PRI;
var repnotify PlayerController PlayerOwner;

replication
{
    if(bNetInitial && (Role == ROLE_Authority))
        PlayerOwner;
}

simulated event ReplicatedEvent(name VarName)
{
    `Log(name$"::ReplicatedEvent - VarName:"@VarName);

    super.ReplicatedEvent(VarName);

    if (VarName == 'PlayerOwner') {
        `Log(name$"::ReplicatedEvent - PlayerOwner:"@PlayerOwner);
        SetOwner(PlayerOwner);
        InitPRI();
    }
}

simulated function PostBeginPlay()
{
    `Log(name$"::PostBeginPlay");
    super.PostBeginPlay();

    InitPRI();
}

simulated function Destroyed()
{
    `Log(name$"::Destroyed");

    CachedPC = none;
    PRI = none;
    PlayerOwner = none;

    Super.Destroyed();
}

simulated function InitPRI()
{
    `Log(name$"::InitPRI");

    `Log(name$"::InitPRI - PlayerOwner:"@PlayerOwner);
    if (PlayerOwner != none)
    {
        `Log(name$"::InitPRI - Cache PRI. Set PRI var");
        PRI = PlayerOwner.PlayerReplicationInfo;
        `Log(name$"::InitPRI - PRI:"@PRI);

        `Log(name$"::InitPRI - Role:"@Role@" Netmode:"@WorldInfo.NetMode);
        if (Role < ROLE_Authority)
        {
            ClientSetup();
        }
    }
}

simulated function ClientSetup()
{
    `log(name$"::ClientSetup");

    `Log(name$"::ClientSetup - Role:"@Role@" Netmode:"@WorldInfo.NetMode);
    if (Role == ROLE_Authority)
    {
        //`Log(name$"::ClientSetup - Is ded server. Return.");
        return;
    }

    PlayerOwner.ClientMessage("ClientOptional Package setup.");
}

function ServerSetup()
{
    `Log(name$"::ServerSetup");
    ClientSetup();
}

function ServerPlayerOwner(PlayerController PC)
{
    `Log(name$"::ServerPlayerOwner - PC:"@PC);
    PlayerOwner = PC;
}

DefaultProperties
{
    // Inherited
    bOnlyRelevantToOwner=True
}

---------------------------------------------------

Package: ExampleClientOptionalServer

ExampleClientOptionalServer\Classes\ExampleClientO ptionalServer.upkg

[Flags]
AllowDownload=False
ClientOptional=False
ServerSideOnly=True

ExampleClientOptionalServer\Classes\IECOSLPRI.uc

interface IECOSLPRI;

function ServerSetup();
function ServerPlayerOwner(PlayerController PC);

ExampleClientOptionalServer\Classes\ECOSMutator.uc

class ECOSMutator extends UTMutator;

var string LinkedPRIClassPath;

function ModifyPlayer(Pawn Other)
{
    super.ModifyPlayer(Other);

    if (Other.Controller != none)
        AddPlayer(Other.Controller);
}

function UTLinkedReplicationInfo AddPlayer(Controller Playr)
{
    `Log(name$"::AddPlayer - Playr:"@Playr);

    if (Playr == none)
    {
        `Log(name$"::AddPlayer - No reference for Player.");
        return none;
    }

    if (!Playr.bIsPlayer || AIController(Playr) != none)
    {
        `Log(name$"::AddPlayer - Is not a player. Return");
        return none;
    }

    if(PlayerController(Playr) == None)
        return none;

    `Log(name$"::AddPlayer - Call SpawnPRIFor");
    return SpawnPRIFor(PlayerController(Playr));
}

function UTLinkedReplicationInfo SpawnPRIFor(PlayerController PC)
{
    local UTLinkedReplicationInfo LRI;
    local IECOSLPRI IILPRI;
    local class<UTLinkedReplicationInfo> LRIClass;
    local UTPlayerReplicationInfo UTPRI;
    
    `Log(name$"::SpawnPRIFor - PC:"@PC);
    UTPRI = UTPlayerReplicationInfo(PC.PlayerReplicationInfo);
    
    if(!GetCustomPRI(PC.PlayerReplicationInfo, LRI) && UTPRI != None) {

        `Log(name$"::SpawnPRIFor - Spawning LinkedPRI class...");
        LRIClass = class<UTLinkedReplicationInfo>(DynamicLoadObject(LinkedPRIClassPath, class'Class'));

        `Log(name$"::SpawnPRIFor - LRIClass:"@LRIClass);

        if (LRIClass != none)
        {
            `Log(name$"::SpawnPRIFor - Spawning LinkedPRI...");
            LRI = Spawn(LRIClass, PC);
            
            `Log(name$"::SpawnPRIFor - LRI:"@LRI);

            if (LRI != none)
            {
                `Log(name$"::SpawnPRIFor - Init LRI");
                LRI.NextReplicationInfo = UTPRI.CustomReplicationInfo;
                UTPRI.CustomReplicationInfo = LRI;


                `Log(name$"::SpawnPRIFor - Set values for"@LRI);
                IILPRI = LRI;

                //LRI.PlayerOwner = PC;
                IILPRI.ServerPlayerOwner(PC);
                LRI.SetOwner(PC);

                IILPRI.ServerSetup();
            }
        } else {
            `Warn(name$"::SpawnPRIFor - Unable to Spawn the class of LRI. Path:"@LinkedPRIClassPath);
            return none;
        }

    } else {
        `Log(name$"::SpawnPRIFor - PRI already got LinkedPRI");
    }
    
    return LRI;
}

function bool GetCustomPRI(PlayerReplicationInfo PRI, out UTLinkedReplicationInfo LPRI)
{
    `Log(name$"::GetCustomPRI - PRI:"@PRI);

    `Log(name$"::GetCustomPRI - Search for Linked PRI");
    LPRI = GetPRI(PRI);
    `Log(name$"::GetCustomPRI - LPRI:"@LPRI);
    return (LPRI != none);    
}

simulated static function UTLinkedReplicationInfo GetPRI(PlayerReplicationInfo PRI)
{
    local UTLinkedReplicationInfo LPRI;

    `Log(default.class$"::GetPRI - PRI:"@PRI);

    if (PRI == none || UTPlayerReplicationInfo(PRI) == None)
    {
        `Log(default.class$"::GetPRI - No Pri. Return.");
        return None;
    }

    if (UTPlayerReplicationInfo(PRI).CustomReplicationInfo == None)
    {
        `Log(default.class$"::GetPRI - No Linked List of CustomReplicationInfo. Return.");
        return None;
    }

    `Log(default.class$"::GetPRI - Iterate Linked List...");
    for (LPRI = UTPlayerReplicationInfo(PRI).CustomReplicationInfo; LPRI != None; LPRI = LPRI.NextReplicationInfo) {
        `Log(default.class$"::GetPRI - LPRI:"@LPRI);

        `Log(default.class$"::GetPRI - Compare "$string(LPRI.Class)$" and "$string(LPRI.class.GetPackageName()));

        if (string(LPRI.Class) ~= "ECOLPRI" && string(LPRI.class.GetPackageName()) ~= "ExampleClientOptional")
        {
            `Log(default.class$"::GetPRI - Found. Return"@LPRI);
            return LPRI;
        }
    }

    `Log(default.class$"::GetPRI - No found. Return none");
    return None;
}

defaultproperties
{
    LinkedPRIClassPath="ExampleClientOptional.ECOLPRI"
}

----------------------------------------

ExampleClientOptionalServer has no dependency on ExampleClientOptional.

[RattleSN4K3]

Looks like packages loaded after the demo rec driver is intialized are not replicated to the driver so it does not store the generation info of the new package into the demo file. Normal clients will load these dynamic loaded package and reconnect. Not the best option to choose but it's better to have no dependency in demo files on packages which are not game related.

[RattleSN4K3]

@Wormbo

ClientOptional [...]. If the demo player doesn't have these packages, it will not be able to decode the demo's recorded network data stream.

[...]The demo recorder automatically has all required packages, so optional packages are included unless you record on a client that doesn't have a certain optional package. But in that case, the optional package will always be ignored on playback, even if it exists then.

Client optional is not useless, you only tried to use it for something it's not meant to be used for.

You did the same for UT2004 with ServerLogo4.

You may want to add the line "ServerPackages=ServerLogo4a" if you experience
problems with demos, but generally it shouldn't be neccessary. Not adding this
line has the advantage of demos not requiring the ServerLogo4a package and the
logo texture (and sound) packages.

[...]

When you are done importing and saving the logo package, you should make your
it client-optional, so it's not required for connecting or demo playback.

----------------

I already got it working but i think my approach will also work with non-clientoptional packages. So ClientOptional is still not needed. It really looks like UT2004 featured skipping optional packages of demo files.

[Wormbo]

Keep in mind the age of the text you quoted. I'm quite sure that ClientOptional has the same drawback in UT2004 now. But UT2004 has a bug/feature that excludes packages from demo play (entirely, hence more likely a bug than a feature) when they are added to the package map dynamically instead of through ServerPackages.

[RattleSN4K3]

A bug or a feature. I'm glad that I found a way of not setting any dependency on these packages for demo files for UT3. ClientOptional has still no proper usage for me . The demo driver could have not recorded these loaded package into a demo file or the attempt to open that file could have skipped the optional package. It always says that a "required package" is missing for ClientOptional.