File compression algorithm (reduce game loading times) about discussions HOT 4 OPEN

Friend of mine works with lrzip so I am familiar with lzs .. but might be
for another day. pk4 is the easiest for modders.

On Thu, Aug 27, 2015 at 5:31 AM, Anthony Gaudino [email protected]
wrote:

I remember I was searching about compression algorithms some time ago and
I found this one to be great https://github.com/Cyan4973/lz4 /
https://code.google.com/p/lz4/ it's very fast and was used in Battlefield
4 and many other software.

Do you think it would be a good addition to the engine?

—
Reply to this email directly or view it on GitHub
#15.

from discussions.

I was also looking into the issue of compression when it comes to the texture formats. Currently I'm using Caesium, an open source image compression tool to compress PNG textures even more. There is a very small loss in image quality but it helps cut down on overall size. It cuts down around 30-50kb in size, on average almost half the size of the .tga textures used in Doom 3 when PNG is used.

from discussions.

lz4 makes sense for network packets and things like that which must be compressed and decompressed really fast, even if the compression ratio is not optimal.

zip (pk4) compresses better than that and is fast enough for us. and it has the huge advantage that everyone can easily create and access it, so it's definitely better for assets.

if I didn't care about ease of use, I'd probably suggest using lzham: https://code.google.com/p/lzham/ and https://github.com/richgel999/lzham_codec
It compresses better than zip and is about as fast for decompressing (compressing is a lot slower, but that's only done once anyway)

BTW, the loading times are already very good in the Doom3 BFG engine due to binary models etc (original doom3 was a lot slower because it used textbased formats which take longer to parse).

Also note that we don't yet have PK4 support, but I plan to do it in the future.

from discussions.

I noticed today that saving out PNG from Inkscape gives an even smaller file size than Caesium. I've tested a texture set from Doom 3, the original size of the Diffuse, Specular and Normal maps was 2.28mb together, after Inkscape it came out as 888kb together with almost no loss in quality. Some textures go down to sizes of 4-11kb. It loads super fast in OTE.

from discussions.