weird RLE-Compression-issue

[Gugi]

ok, I want to include RLE-Compression in the next version of the Screen Sender (http://utforums.epicgames.com/showthread.php?t=577014).
RLE-Compression means, that you go through the bytes and "merge" same bytes. e.g.: abbbbbccd is becoming abb2cc0d. This would save around 1/3 - 1/2 of the data-amount.

The problem is, that I seem to get correct results in resolutions <160x120, but not for example in 640x480.

UScript-Code:
FinalPixels is a dynamic array holding all pixel-bytes.

final private function DoRLECompression()
{
local int i, InsertPos;
local byte LastByte;
local int SameByteCounter; // Number of same bytes - 1

LastByte=FinalPixels[0];
for(i=1;i<FinalPixels.Length;++i)
{
// prevent overflow (we can only save 1 byte)
if (LastByte==FinalPixels[i] && SameByteCounter < 0x000000fe && i+1<FinalPixels.Length)
{
++SameByteCounter;
}
else // New value
{
if (SameByteCounter>0) // Uhm, we've got some characters behind each other
{
InsertPos=(i-SameByteCounter)+1; // Calculate position to insert the ByteCounter
FinalPixels.Insert(InsertPos, 1);
FinalPixels[InsertPos]=byte(SameByteCounter-1); // Set it to the correct value
FinalPixels.Remove(InsertPos+1, SameByteCounter-1); // Remove the same bytes which come after the SameByteCounter

SameByteCounter=0;
i=InsertPos+1; // Move the pointer to the next byte
LastByte=FinalPixels[i]; // and save the next byte. Then go on with the loop (thus go on with the next byte).
}
else // Mh, we can't do anything...
LastByte=FinalPixels[i];
}
}
}


C++ Part:
const char* const In: This is an array of the single bytes.
ULONG Len: The length of the In-array
list<char>& Out: This list should hold the final bytes in the end.


void RLEDecompression(const char* const In, ULONG Len, list<char>& Out)
{
char LastByte = In[0];
Out.push_back(In[0]);

bool bFound2Bytes = false;

short int InsertCounter = 0;
cerr << "test\n";
for(ULONG i = 1; i<Len; ++i)
{
if (bFound2Bytes) // the last 2 bytes were the same -> the current is the repetition-counter
{
// Insert x elements instead of the repetition counter byte
for(InsertCounter = 0; InsertCounter<(short int)(In[i]); ++InsertCounter)
Out.push_back(LastByte);

bFound2Bytes = false;
++i;
if (i<Len)
{
LastByte = In[i];
Out.push_back(In[i]);
}
}
else if (LastByte == In[i])
{
bFound2Bytes = true; // The next byte is the repetition-counter!
Out.push_back(In[i]);
}
else
{
LastByte = In[i];
Out.push_back(In[i]);
}
}
}


Here is what I get:
http://img402.imageshack.us/img402/5590/testur2.th.jpg (http://img402.imageshack.us/my.php?image=testur2.jpg)

It seems to get worse the more it can compress the bytes.


Does anyone see a mistake? :confused:

[Shambler]

I can't see anything obvious, perhaps you should save two files:
The bitmap version of the screenshot and the RLE version, then use an image editing program to save the bitmap as a seperate RLE file and compare that with the UScript-RLE file using a hex editor.

That should allow you to highlight the problem within the actual stored bytes, which in turn should lead you back to what might be going wrong with the code.

[Gugi]

Found the error :)

It was a really nasty one in the C++ part: I forgot to use "unsigned char" instead of "char" :eek:

It worked fine in the other parts of the code as there was no need to compare a read value, but in the RLE-Compression part I had to use "InsertCounter<(short int)(In[i])" which was the source of the problem (as the In[i]-value was partly negative then).


It took me 3 days to detect the error, as it does only occur on higher repetition counters (i.e. which use the 8th bit) and this mostly only happens on higher resolutions. And the issue only came out in the middle of the file (or at least not at the first and last few bytes).