Add premultiplied alpha setting to KTX format about ktx-software HOT 8 CLOSED

There is already a bit in the flags field of the data format descriptor (DFD) in KTX2. There is nothing that can be done for KTX1 except using metadata.

Two values for the bit are defined in khr_df.h: KHR_DF_FLAG_ALPHA_STRAIGHT (=0) and KHR_DF_FLAG_ALPHA_PREMULTIPLIED (=1). In libktx the bit can be accessed in a ktxTexture2 object called ktex by

bool isPremultiplied = KHR_DFDVAL(ktex->dfd, FLAGS) & KHR_DF_FLAG_ALPHA_PREMULTIPLIED

In libktx, we need to make sure that this bit is preserved along with the transfer function and primaries when the DFD is rewritten during supercompression to Basis. We also need to provide a way to create pre-multiplied textures. currently toktx accepts PNG and Netpbm formats. Neither appears to support premultiplied. So what to do? I will leave this issue open to track resolution of these 2 points.

from ktx-software.

I like that header spec a lot, it's very explicit. My only concern is that this flag not be restricted to DXT2/4 type formats. Basically any compressed or uncompressed endpoints with alpha should be valid to have premultiplied alpha applied, and that bit set if I'm understanding it correctly. The spec only stated usage on DXT2/4.

And yes, have libktx hand premultiplication of alpha would be a big help, since if work is done on the gpu, then this is a cheap multiply operation in addition to the required sRGB conversions that are best done on GPU.

from ktx-software.

My only concern is that this flag not be restricted to DXT2/4 type formats.

It can be set for any descriptor. It only makes sense for the RGBDSA and compressed texture color models and only when an alpha channel is present. See Table 90 in Chapter 11, Example format descriptors.

And yes, have libktx hand premultiplication of alpha would be a big help

I don't understand what you are trying to say here.

from ktx-software.

I don't understand what you are trying to say here.

I just meant that doing srgb -> linear, linear * a, then linear * a -> srgb could be done entirely on the gpu in libktx. Otherwise, it's simple done with a 3x 2d table lookups on ra, ga, ba. Just that that ordering should be honored for correctness and to avoid clamping. Bits will be lost on low alpha values being stored in 8-bit components regardless.

from ktx-software.

Ahh! So s/hand/handle/. Are you suggesting an option to toktx to premultiply the incoming images?

The parts of libktx involved in creating .ktx2 files do not use the GPU. They can run anywhere. As creation doesn't seem particularly time critical I'm loathe to add the complexity of checking for and using a GPU and dealing with the resulting dependencies. A premultiplication option can be implemented on the CPU of course.

The only parts that do use the GPU are the OpenGL and Vulkan loaders. By that point the data should already be premultiplied.

from ktx-software.

PIng! Are you asking for a pre-multiplication option to toktx? Should I add an isPremultiplied() function to ktxTexture2 or is accessing the DFD sufficient?

from ktx-software.

No, the "bool isPremultiplied = ..." example that you provided is probably sufficient.

I do think that premultiplication of the values should happen in the texture encoder, but it's complicated with the low-bit depth of BC1-BC3. With ASTC, the endpoints are 8-bit, but sometimes split into a base + delta. And finally, many of the existing compressors just take the srgb colors and then fit the endpoints to those. That doesn't seem right, since that's not a linear color space.

So ideally for srgb:

1. convert colors from srgb to 10-bit or 8-bit linear. Conversion to 565 isn't enough bits.
2. do rgb * alpha
3. fit endpoints to the premultiplied linear rgb values
4. remap the rgb endpoints back to srgb

The gpu does the reverse of step 4 before storing the colors in the texture cache. That way bi/tri-linear filtering are correct in linear space. On a simpler note, EAC/BC4/5 can also be stored at higher precision in the texture cache, since they're only 1-2 channels.

For rgb, it's easy since it's just step 2 and 3 only.

from ktx-software.

The only encoder within libktx is the BasisU encoder which is incorporated as a git subrepo. You should probably open an issue at https://github.com/BinomialLLC/basis_universal requesting this. I don't think there is anything to do in the rest of libktx so I'm going to close this. Note that there is now a ktxTexture2_GetPremultipliedAlpha function to find out if the bit is set or not.

from ktx-software.