khronosgroup / ktx-software Goto Github PK

Given a KTX file, is it possible to convert back to PNGs? If so, what tool would I use to do that? I am on OSX.

The following link is broken

https://www.khronos.org/message_boards/forumdisplay.php/103-KTX-file-format-for-OpenGL-OpenGL-ES-and-WebGL-textures

Cut&paste from the repo README

Seems that there is no way to get the number of mipmap levels that were loaded from the file when using ktxLoadTexture* functions. If a ktx-file does not contain full mipmap hierarchy I need to know how many levels there are, so I can set GL_TEXTURE_MAX_LEVEL correctly.

I tried building toktx and found it surprisingly difficult. Using a raw Linux system; I'm used to being able to build projects using cmake + make or just make with no extra configuration.

I found the build/make files, tried that first. That doesn't work:

  ACTION Regenerating Makefile
/bin/sh: 1: /usr/local/bin/gyp: not found

I found the build/cmake files then, tried that next. Hit a few problems, here's a summary:

• One of the build steps requires doxygen so it fails - doxygen isn't installed on my system
• Several .cpp files require vulkan/vulkan.h - I don't have it installed on my system so it fails.
• Several targets require glslc - I don't have Vulkan SDK binaries installed as well, so this fails.

Running make -k after CMake (to build everything possible) produces a few binary files (libSDL2-2.0.so, a few .a files) but no executables of any kind.

Maybe I'm missing some crucial steps, but it would be great to have the ktx loading library and toktx tool build without the issues noted.

Please note, this is on ktx2 branch so I realize I am using somewhat bleeding edge software.

To enable many data-oriented applications, the public API has to be easily accessible via scripting languages FFI. A good starting point may be Python ctypes.

This probably is just the old behavior (before KhronosGroup/KTX-Specification#103), but the validator currently assumes that BasisU files need to have UNSPECIFIED color model and no sample information, whereas the spec mandates RGBSDA and valid sample information.

texture2.c has the following code:

    // Calculate size of the image data.
    This->dataSize = 0;
    for (ktx_uint32_t i = 0; i < This->numLevels; i++) {
        This->dataSize += _KTX_PAD8(private->_levelIndex[i].byteLength);
    }

dataSize is later used to load all mip levels in bulk from the file, starting from the offset of the first (smallest) mip level. So each mip level must be padded by 8 bytes or the file will fail to load.

However, the KTX2 specification for mip level array says:

    align(8) mipPadding // No padding the last time.

Implying that the last level must not be padded. Thus files written according to spec (without the last level padded) can't be loaded. I'm not sure if it's a spec issue or not; fwiw the "don't pad the last entry" in the spec seems redundant and harder than necessary to deal with, so we could fix the spec to require padding for all elements (including kvp, although it's technically redundant there as it's subsumed by the following 8-byte padding).

I'm a BabylonJS user so I know there is a list about astc, dxt, pvrtc, etc1, etc2, atc on this page : https://doc.babylonjs.com/resources/multi-platform_compressed_textures#khronos-texture-container-format-ktxhttpswwwkhronosorgopenglessdktoolsktx-files

Then I had in mind to go read directly to the source about a short descriptions of what are these formats targetting which OS/hardware, so I get this page : https://www.khronos.org/opengles/sdk/tools/KTX/

But after some searches I wasn't able to find any simple chart like the BabylonJS one below:

Do you think it's possible to add something like that?

KTX/lib/checkheader.c currently states:

/* Check texture dimensions. KTX files can store 8 types of textures:
1D, 2D, 3D, cube, and array variants of these. There is currently
no GL extension that would accept 3D array or cube array textures. */

However, GL_TEXTURE_CUBE_MAP_ARRAY has been OpenGL core since 4.0 and we've had GL_EXT_texture_cube_map_array for OpenGL ES 3.1 since 2014.

Mark, FYI I closed down the (long inactive) public KTX Bugzilla to new bugs (there is one remaining open bug, many years old), and directed people to use this repo instead. Bugzilla got another spammer attack today and we're trying to lock down everything that's not actually getting used.

lib/hashtable.c:316 just uses a cast to read the key value size which will be wrong if the source file uses opposite endianness. The deserialise function needs a parameter to specify endianness.

I'm currently using libktx a sample app and want to integrate libktx into it. My project uses one cmake script for all platforms/configurations. I've started refactoring it pretty heavily @ https://github.com/bradc6/KTX/tree/refactor/cmake (or more specifically https://github.com/bradc6/KTX/blob/refactor/cmake/build/cmake/linux/Debug/CMakeLists.txt)

I'm opening this issue to see if there is any interest in completing this 100% and making a pull request.

Thanks,
-Brad

The test files etc2-sRGB.ktx, etc2-sRGBa1.ktx, etc2-sRGBa8.ktx use either 0x8C40 (GL_SRGB) or 0x8C43 (GL_SRGB8_ALPHA8) as their base internal format.

The spec seems a little unclear on this point...

For uncompressed textures, this value will be the same as glFormat and is used as the internalformat parameter when loading into a context that does not support sized formats, such as an unextended OpenGL ES 2.0 context.

GL_SRGB is part of EXT_sRGB, so it's not valid for an unextended context. On the other hand, EXT_sRGB doesn't add the functionality of sized formats, except for the use of renderbuffers.

While GL_SRGB might be valid if we assume the target platform supports EXT_sRGB, GL_SRGB8_ALPHA8 seems incorrect, and should be replaced with 0x8C42 ('SRGB_ALPHA') or 0x1908 (GL_RGBA) depending on how you interpret the case of GL_SRGB.

how to draw the KTX image on a bitmap?
I don't know how do I get the texturedata and draw them on a bitmap.

Basis Universal reached a new milestone a few days ago.
https://github.com/BinomialLLC/basis_universal/blob/master/README.md#release-notes

hi, I want to create a CPVRTexture from the buffer of a ktxfile but not the filepath.
like this
CPVRTexture* cp = new CPVRTexture(the Buffer of ktxfile);

I am currently working on KTX support for Blender together with a developer from the Blender Institute and came across the issue that it does not seem to be possible to load a KTX texture without having access to a OpenGL context.

Since some libraries will not have access to this (library files, loading in background thread etc) I would request a method of loading the texture in raw data so it can be used in all cases.

For people interested, a patch for Blender can currently be found on https://developer.blender.org/rB5e060bed27bd3e2a5e75f0c16ea65dc4534fdf04

Hello.

When I build, I get the following error:

My OS: Ubuntu 16.04 64BIT

Thanks.

karas@karas-VirtualBox:~/0205/KTX-Software/build/cmake/linux/Debug$ cmake .
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- The ASM compiler identification is GNU
-- Found assembler: /usr/bin/cc
-- Configuring done
-- Generating done
-- Build files have been written to: /home/karas/0205/KTX-Software/build/cmake/linux/Debug
karas@karas-VirtualBox:~/0205/KTX-Software/build/cmake/linux/Debug$ make 
Scanning dependencies of target libsdl__copies
[  3%] libsdl
[  3%] Built target libsdl__copies
Scanning dependencies of target libsdl
[  3%] Built target libsdl
Scanning dependencies of target appfwSDL
[  6%] Building CXX object CMakeFiles/appfwSDL.dir/home/karas/0205/KTX-Software/tests/loadtests/appfwSDL/main.o
[ 10%] Building CXX object CMakeFiles/appfwSDL.dir/home/karas/0205/KTX-Software/tests/loadtests/appfwSDL/AppBaseSDL.o
[ 13%] Building CXX object CMakeFiles/appfwSDL.dir/home/karas/0205/KTX-Software/tests/loadtests/appfwSDL/GLAppSDL.o
[ 17%] Linking CXX static library obj.target/libappfwSDL.a
[ 17%] Built target appfwSDL
Scanning dependencies of target libgl
[ 17%] Built target libgl
Scanning dependencies of target libktx_gl
[ 20%] Building C object CMakeFiles/libktx_gl.dir/home/karas/0205/KTX-Software/lib/checkheader.o
[ 24%] Building C object CMakeFiles/libktx_gl.dir/home/karas/0205/KTX-Software/lib/errstr.o
[ 27%] Building C object CMakeFiles/libktx_gl.dir/home/karas/0205/KTX-Software/lib/hashtable.o
[ 31%] Building C object CMakeFiles/libktx_gl.dir/home/karas/0205/KTX-Software/lib/ktxfilestream.o
[ 34%] Building C object CMakeFiles/libktx_gl.dir/home/karas/0205/KTX-Software/lib/ktxmemstream.o
[ 37%] Building C object CMakeFiles/libktx_gl.dir/home/karas/0205/KTX-Software/lib/loader.o
[ 41%] Building C object CMakeFiles/libktx_gl.dir/home/karas/0205/KTX-Software/lib/swap.o
[ 44%] Building C object CMakeFiles/libktx_gl.dir/home/karas/0205/KTX-Software/lib/writer.o
[ 48%] Building CXX object CMakeFiles/libktx_gl.dir/home/karas/0205/KTX-Software/lib/etcdec.o
[ 51%] Building CXX object CMakeFiles/libktx_gl.dir/home/karas/0205/KTX-Software/lib/etcunpack.o
[ 55%] Linking CXX shared library lib.target/libktx.gl.so
[ 55%] Built target libktx_gl
Scanning dependencies of target gl3loadtests__copies
[ 58%] gl3loadtests
[ 58%] Built target gl3loadtests__copies
Scanning dependencies of target toktx
[ 62%] Building CXX object CMakeFiles/toktx.dir/home/karas/0205/KTX-Software/tools/toktx/image.o
[ 65%] Building CXX object CMakeFiles/toktx.dir/home/karas/0205/KTX-Software/tools/toktx/toktx.o
[ 68%] Linking CXX executable toktx
[ 68%] Built target toktx
Scanning dependencies of target toktx-tests__toktx-tests
[ 72%] tests
Tests run: 16; passed: 16; failed: 0
[ 72%] Built target toktx-tests__toktx-tests
Scanning dependencies of target toktx-tests
[ 72%] Built target toktx-tests
Scanning dependencies of target gl3loadtests
[ 75%] Building C object CMakeFiles/gl3loadtests.dir/home/karas/0205/KTX-Software/tests/loadtests/common/at.o
[ 79%] Building C object CMakeFiles/gl3loadtests.dir/home/karas/0205/KTX-Software/tests/loadtests/shader-based/sample_01_draw_texture.o
[ 82%] Building C object CMakeFiles/gl3loadtests.dir/home/karas/0205/KTX-Software/tests/loadtests/shader-based/sample_02_cube_textured.o
[ 86%] Building C object CMakeFiles/gl3loadtests.dir/home/karas/0205/KTX-Software/tests/loadtests/shader-based/shaderfuncs.o
[ 89%] Building C object CMakeFiles/gl3loadtests.dir/home/karas/0205/KTX-Software/tests/loadtests/shader-based/shaders.o
[ 93%] Building CXX object CMakeFiles/gl3loadtests.dir/home/karas/0205/KTX-Software/tests/loadtests/common/LoadTests.o
[ 96%] Building CXX object CMakeFiles/gl3loadtests.dir/home/karas/0205/KTX-Software/tests/loadtests/shader-based/LoadTestsGL3.o
[100%] Linking CXX executable gl3loadtests
/usr/bin/ld:/home/karas/0205/KTX-Software/build/cmake/linux/Debug/../../../../other_lib/linux/Debug-x64/libSDL2-2.0.so: file format not recognized; treating as linker script
/usr/bin/ld:/home/karas/0205/KTX-Software/build/cmake/linux/Debug/../../../../other_lib/linux/Debug-x64/libSDL2-2.0.so:1: syntax error
collect2: error: ld returned 1 exit status
CMakeFiles/gl3loadtests.dir/build.make:252: recipe for target 'gl3loadtests' failed
make[2]: *** [gl3loadtests] Error 1
CMakeFiles/Makefile2:463: recipe for target 'CMakeFiles/gl3loadtests.dir/all' failed
make[1]: *** [CMakeFiles/gl3loadtests.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

On this line the blockHeight should be 5:

Where is this code originally from?

KTX2 files may have swizzling metadata that are expected to be applied on rendering.

KTX software behavior on processing files with such metadata via Basis should be documented and preferably configurable.

Swizzling like rabb may yield drastically different error metrics after supercompression. So there should be two options:

• compress values as-is, leave swizzling metadata in the supercompressed KTX file;
• apply swizzling before invoking Basis compressor and remove swizzling metadata from the compressed KTX file.

Transcoding Basis slices to BC4 and BC5 formats may imply additional internal swizzling, so there should be accurate documentation on that.

Is there any way KTX can support 3d textures with mipmaps
with total size exceeding 4gb?

While perusing code I noticed that the ktxTexture_Create* functions and their descendants do not seem to free the new ktxTextureInt instance after an error occurs.

The caller doesn't receive the instance pointer on a failure, so it can't free that memory either.

Tagged releases should have pre-built binaries (an archive with libraries, headers, and command-line tools) automatically published for all platforms. Executable code should have Khronos signature (as it becomes available).

I would like to point out that an identifier like "_KTX_H_" does not fit to the expected naming convention of the C language standard.
Would you like to adjust your selection for unique names?

Hello, I decided to use Mali OpenGL ES Emulator so I edited "gyp_include\config.gypi" and then run three commands:

gyp -f msvs -G msvs_version=2015 --generator-output=build/msvs
--depth=. libktx.gyp
gyp -f msvs -G msvs_version=2015 --generator-output=build/msvs
--depth=. ktxtools.gyp
gyp -f msvs -G msvs_version=2015 --generator-output=build/msvs
--depth=. ktxtests.gyp
'ls' is not recognized as an internal or external command,
operable program or batch file.
gyp: Call to 'ls testimages/*.ktx' returned exit status 1 while in ktxtests.gyp.

First two commands run OK and generated required projects/solutions, but the 3rd one end up with stated error. I don't know, if this issue is GYP related, or there is some problem with file 'tests/tests.gypi'...
p.s.: My system is Win 8.1 and I have used master branch of GYP and Python 2.7.12. I run those commands from VS2015 x86 Native Tools Command Prompt...

Couldn't regenerate cmake build system on my debian 8 quickly (some python import errors), so firing up an issue here.

On Linux it's important to be able to link against a shared library, so adding support to build all target libs shared would be great. Seems like it shouldn't be a big deal from what i've read in gyp docs:
'type': This should almost always be set to ‘<(library)’, which allows the user to define at gyp time whether libraries are to be built static or shared
It's also important to add libGL to dependencies (at least for libktx_gl target) otherwise shared library build will fail

Think I can produce a patch myself if i get some tip on how to make gyp work on my machine. Currently getting this:

make cmake
gyp -f cmake -DOS=linux --generator-output=build/cmake/linux/ -G output_dir=. --depth=. ktxtests.gyp ktxtools.gyp
Traceback (most recent call last):
  File "/usr/bin/gyp", line 9, in <module>
    load_entry_point('gyp==0.1', 'console_scripts', 'gyp')()
  File "/usr/lib/python2.7/dist-packages/gyp/__init__.py", line 547, in script_main
    return main(sys.argv[1:])
  File "/usr/lib/python2.7/dist-packages/gyp/__init__.py", line 540, in main
    return gyp_main(args)
  File "/usr/lib/python2.7/dist-packages/gyp/__init__.py", line 516, in gyp_main
    options.circular_check)
  File "/usr/lib/python2.7/dist-packages/gyp/__init__.py", line 91, in Load
    generator = __import__(generator_name, globals(), locals(), generator_name)
ImportError: No module named cmake
GNUmakefile:130: recipe for target 'build/cmake/linux/.ktx-stamp' failed
make: *** [build/cmake/linux/.ktx-stamp] Error 1

Hello,

Building an environment with ASan, memory corruption occurs when the following command is executed.

make toktx-tests 

Thanks.

=================================================================
==7846==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7fff575443b4 at pc 0x000000448086 bp 0x7fff575440b0 sp 0x7fff57543840
WRITE of size 21 at 0x7fff575443b4 thread T0
    #0 0x448085 in scanf_common(void*, int, bool, char const*, __va_list_tag*) (/home/karas/KTX-Software/build/cmake/linux/Debug/toktx+0x448085)
    #1 0x449246 in sscanf (/home/karas/KTX-Software/build/cmake/linux/Debug/toktx+0x449246)
    #2 0x4de438 in readPAM(_IO_FILE*, unsigned int&, unsigned int&, unsigned int&, unsigned int&, unsigned int&, unsigned char*&) /home/karas/KTX-Software/tools/toktx/image.cpp:408:12
    #3 0x4dcd04 in readNPBM(_IO_FILE*, unsigned int&, unsigned int&, unsigned int&, unsigned int&, unsigned int&, unsigned char*&) /home/karas/KTX-Software/tools/toktx/image.cpp:151:10
    #4 0x4dfa25 in main /home/karas/KTX-Software/tools/toktx/toktx.cpp:273:17
    #5 0x7f5e9362c82f in __libc_start_main /build/glibc-Cl5G7W/glibc-2.23/csu/../csu/libc-start.c:291
    #6 0x4358f8 in _start (/home/karas/KTX-Software/build/cmake/linux/Debug/toktx+0x4358f8)

Address 0x7fff575443b4 is located in stack of thread T0 at offset 372 in frame
    #0 0x4de0ff in readPAM(_IO_FILE*, unsigned int&, unsigned int&, unsigned int&, unsigned int&, unsigned int&, unsigned char*&) /home/karas/KTX-Software/tools/toktx/image.cpp:383

  This frame has 4 object(s):
    [32, 287) 'line'
    [352, 372) 'tupleType' <== Memory access at offset 372 overflows this variable
    [416, 420) 'maxval'
    [432, 436) 'depth'
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow ??:0 scanf_common(void*, int, bool, char const*, __va_list_tag*)
Shadow bytes around the buggy address:
  0x10006aea0820: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10006aea0830: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10006aea0840: 00 00 00 00 00 00 00 00 f1 f1 f1 f1 00 00 00 00
  0x10006aea0850: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10006aea0860: 00 00 00 00 00 00 00 00 00 00 00 07 f2 f2 f2 f2
=>0x10006aea0870: f2 f2 f2 f2 00 00[04]f2 f2 f2 f2 f2 04 f2 04 f3
  0x10006aea0880: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10006aea0890: 00 00 00 00 f1 f1 f1 f1 00 00 00 00 00 00 00 00
  0x10006aea08a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x10006aea08b0: 00 00 00 00 00 00 00 07 f3 f3 f3 f3 f3 f3 f3 f3
  0x10006aea08c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==7846==ABORTING

I have noticed that some test images use the incorrect metadata key KTXOrientation (uppercase O), instead of the correct KTXorientation:

• cubemap_yokohama_astc_8x8_unorm.ktx
• cubemap_yokohama_bc3_unorm.ktx
• cubemap_yokohama_etc2_unorm.ktx
• metalplate-amg-rgba8.ktx
• not4_rgb888_srgb.ktx
• pattern_02_bc2.ktx
• texturearray_astc_8x8_unorm.ktx
• texturearray_bc3_unorm.ktx
• texturearray_etc2_unorm.ktx

I don't expect anything to catch on fire because of this, but it seems worthwhile to mention it.

The steps to repro are just to build a release version of toktx and run it without arguments. This looks like a std::string construction from a NULL pointer.

1.When one loads an array texture, GL_TEXTURE_2D_ARRAY,in the loader.c it is bound to a default target which is probably GL_TEXTURE_2D.It is never bound to GL_TEXTURE_2D_ARRAY afterwards.It leads to INVALID OPERATION error when trying to rebound to GL_TEXTURE_2D_ARRAY later.Please fix it by binding the texture only after the correct target has been set. (lines 543 -555)

2.When loading a CUBE MAP,the loader iterates over the faces,right?That's fine.But then the texture target enum that the loader returns is not GL_TEXTURE_CUBE_MAP but the last face target enum.This one also leads to a crash when used by the caller to bind a target.
I fixed it like this:

```

if (errorCode == KTX_SUCCESS)
{

	if (texinfo.generateMipmaps && pfGlGenerateMipmap) {
		pfGlGenerateMipmap(texinfo.glTarget);
	}
    if (iscubemap)
    {
        *pTarget = GL_TEXTURE_CUBE_MAP;
	 
    }....

Make sure this is validated. Also typeSize = 1.

The .ktx files in this .zip file have been encoded incorrectly:

corset-separate.zip

Specifically, the mip level index here is wrong - the file offsets increase in the level index (instead of decreasing); additionally, because the index is wrong, attempting to decode the levels using Basis would yield insufficient data - for example, here the largest mip level in the index is specified as having just 4 bytes of supercompressed data.

It would be nice if ktxcheck checked that the mip levels are located sequentially from smallest to largest in the file (which would flag this file as being invalid), and also tried to decode supercompressed data / validate individual mip level sizes for non-supercompressed data as well.

I assume this is the right place to report issues related to documentation in https://www.khronos.org/opengles/sdk/tools/KTX/file_format_spec/

Issues in An example KTX file: section

1. 0x0D, 0x0A, 0x1A, 0x0A. // final four bytes of Byte[12] identifier should be 0x0D, 0x0A, 0x1A, 0x0A, // final four bytes of Byte[12] identifier
2. 0x6A, 0x6F, 0x6B, 0x65, // UTF8 v: 'gles2\0' 0x6A, 0x6F, 0x6B, 0x65 means j o k e, not g l e s

Suggestions:

1. It would be nice to have actual KTX binary file linked to the documentation because Example file section does not contain full KTX file
2. Explain what are valid values for padding bytes
3. There is no reasoning why multiple of 4 are used, so maybe add a small text about it (and also mention modulo 4 since it is a bit easier to understand than Byte valuePadding[3 - ((keyAndValueByteSize + 3) % 4)]).

I followed this with a pull request, but step 5 was apparently not really required nor wanted:

https://github.com/KhronosGroup/KTX-Software/blob/ktx2/CONTRIBUTING.md

Would you like to replace any double quotes by angle brackets around file names for include statements?

From Khronos Bugzilla bug 1397:

AMD Compress also can work with KTX files, can you add this links http://developer.amd.com/tools-and-sdks/archive/games-cgi/amdcompress/
to libktx description in section:
"Working with KTX Files"

It looks like these mappings are the wrong way round. As far as I am aware, DXT3 should be BC2 while DXT5 should be BC3.

I'm trying to use toktx instead of basisu to produce a compressed 2D image with a full mipmap chain. Here's what happens if I run both tools with default arguments:

$ toktx --t2 --bcmp Material_20_baseColor.ktx Material_20_baseColor.png
$ basisu -file Material_20_baseColor.png
$ ls -la Material_20_baseColor.*
-rwxrwxrwx 1 zeux zeux  342411 Nov 24 11:34 Material_20_baseColor.basis
-rwxrwxrwx 1 zeux zeux  342552 Nov 24 11:37 Material_20_baseColor.ktx
-rwxrwxrwx 1 zeux zeux 3515875 Nov 24 11:32 Material_20_baseColor.png

This makes sense. The KTX file is slightly larger, presumably due to the larger header.
Now let's try to add -mipmap argument to basisu and --automipmap to toktx:

$ basisu -file Material_20_baseColor.png -mipmap
$ toktx --t2 --bcmp --automipmap Material_20_baseColor.ktx Material_20_baseColor.png
$ ls -la Material_20_baseColor.*
-rwxrwxrwx 1 zeux zeux  478834 Nov 24 11:38 Material_20_baseColor.basis
-rwxrwxrwx 1 zeux zeux  342552 Nov 24 11:38 Material_20_baseColor.ktx
-rwxrwxrwx 1 zeux zeux 3515875 Nov 24 11:32 Material_20_baseColor.png

Clearly basisu added a mipmap chain to the compressed file, but toktx simply ignored the argument.

We should provide a straightforward and well-supported way of producing KTX2 files from DDS.
An official open-source DDS implementation is here.

I have got a texture and I'd like to see what is exactly inside it. When I open it using Visual Studio Code it shows information: The file is not displayed in the editor because it is either binary or uses an unsupoported text encoding , though. Same with a wordpress, notepad.

How can I do it?

in _ktxCheckHeader function (checkheader.c):

    if (header->glTypeSize != 1 ||
        header->glTypeSize != 2 ||
        header->glTypeSize != 4)
    {
        /* Only 8, 16, and 32-bit types supported so far */
        return KTX_INVALID_VALUE;
    }

i think this will be alway true.
use '&&' instead ?

IMHO the input formats supported by toktx are too exotic to be helpful/help adoption.

Using stb_image.h could allow to add more input format without adding another large dependency to the project.

Hi!
I found a small memory leak generated by function ktxHashList_Serialize.
If bytesOfKeyValueData equals zero and malloc is called with zero bytes, we'll have a valid pointer of allocated memory but we won't deallocate it later.
Leak is actual on Windows only.
Related links: http://c-faq.com/ansi/malloc0.html

In order to test the byte-swapping code and avoid problems like issue #25 we need an opposite-endian test KTX file. In practical terms that means a big-endian file since all my test machines are little-endian.

if i only load ktx file and return texture id was used opengl program, how should i do?which file can used?

I'd like for KTX to formalize this reality into the spec, so that it doesn't have to be stored in some ignored name-value pair.

Simple proposal:
PremultipliedAlpha: true/false

Alternate proposal:
ChannelDescription: "AR, AB, AG, A" (premultiplied) vs. "R, G, B, A" (unpremultiplied)
ChannelDescription: "AL, A" (premultiplied) vs. "L, A" (unpremultiplied)

Premultiplied alpha is the norm for 2D and 3D renderers, and yet only a few image formats (notably EXR) have any indicator for premultiplied alpha support. EXR uses the AR, AG, AB convention vs. R, G, B to indicate a premultiplied channel. PNG only supports unpremultiplied, JPG doesn't support any, etc. DXT2/4 used to be a premultiplied version of DXT3/5, but disappeared with the rename to BC1/2/3.

To get proper sampling on the GPU and especially with sRGB premultiplied formats, you can't just multiply the alpha in the shader after sampling. Bilinear and trilinear and anisotropic sampling need the texture format premultiplied before it reaches the texture cache. sRGB is undone if present, and then all samples combine in premultiplied linear space with adds/averaging in the texture unit.

In an encoder, pixels are premultiplied, cluster fit the point cloud of colors, and finally snapped back into 565/888 BC/ASTC endpoints in the file format. Uncompressed data with an alpha channel (RGBA8, 16f, 32f) would have the rgb channels premultiplied. Premultiplication makes storing constant data (f.e. in rb in BC3nm) less simple, since the alpha channel would affect all channels, but premultiplied data usually applies to albedo channels which use all channels for color data.

Finally sRGBA8 data would do: sRGB -> linear, rgb * a, linear -> sRGB. This is best done on the GPU. Mipmaps must do a similar conversion to average in linear space, and convert back to sRGB.

BC1 with alpha cutout, could use premultiplied form, but with alpha = 0 the color would go to black. Finally, textures with an alpha channel that is all 1, can set the PremultipliedAlpha flag to true without any math to the rgb pixels.

Note that I'm not proposing KTX format do these conversions, but libKTX could help do the right thing and texture compression tools would do the right thing here.

Some ktx files in testimages have internalformat set to an unsized value. This is invalid according to the KTX spec which points to tables of sized formats in the OpenGL spec. as possible values. An example is down_reference.ktx. rgba-reference.ktx is also possibly invalid.

Also toktx supports creating such files. That support needs to be removed.

Here's a full description of how color space information traverses KTX pipeline with supercompression from the source image to the GPU texture.

Source PNG

As per the PNG spec, each PNG image has a color space defined by zero or more color space information chunks (sRGB, gAMA, cHRM, and iCCP):

• No color-info chunks:
  • an implementation should assume sRGB (or 2.2 gamma) transfer function with sRGB color primaries.
• sRGB chunk is present:
  • use the sRGB transfer function, the sRGB primaries, and (optionally) the intent specified in the chunk, ignore gAMA and cHRM chunks.
• iCCP chunk is present:
  • use the ICC profile provided in the chunk, ignore gAMA and cHRM chunks.
• gAMA and/or cHRM chunks are present without sRGB or iCCP:
  • use the provided gamma value and color primaries.
• sRGB and iCCP chunks shall not be present simultaneously.

KTX software follows these rules and infers the source image color space with additional limitations:

• Custom ICC profiles are not supported.
• Custom color primaries are not supported.
• Custom gamma values except 1.0 and 2.2 are not supported.
• 2.2 gamma transfer function is treated as sRGB.

When the source image actually contains linear values (e.g., it's a normal map) but the PNG file doesn't signal that by using the appropriate combination of chunks, users can override KTX software behavior by providing --linear parameter to the command-line tool.

The resulting KTX2 file always has the color space information in its DFD section.

Basis Supercompression

Basis compressor has an artifact-reducing optimization, enabled by settingm_perceptual in the compressor parameters, which handles the original pixel values as perceptual colors, i.e. sRGB. By default they are treated as linear which is suitable for arbitrary vectors as well as linear colors. The KTX software sets this parameter when transcoding textures with the sRGB bit set in the DFD section. It does not expose the parameter as a user setting.

Runtime Texture Transcoding

At runtime, a GPU texture object of appropriate format needs to be created to receive transcoded block-compressed data. Its format may or may not imply hardware sRGB sampling and filtering, so depending on the platform, applications may need to plan for some fallback behavior.

When the GPU texture is created by the KTX library, it may need to know that hardware sRGB is not available and the conversion will happen in shaders.

Here is how platform support for sRGB filtering of block-compressed formats looks like.

ASTC

On platforms that support ASTC, each combination of block dimensions exists in both linear and sRGB options.

BC1, BC2, BC3

• OpenGL Desktop extensions:
  • EXT_texture_compression_dxt1 (BC1 only, linear).
  • EXT_texture_compression_s3tc (BC1/2/3, linear).
  • EXT_texture_sRGB (BC1/2/3, sRGB).
  • When the latter extension is not available, an application has to decode sRGB in a fragment shader.
• OpenGL ES extensions:
  • EXT_texture_compression_dxt1 (BC1 only, linear).
  • EXT_texture_compression_s3tc (BC1/2/3, linear).
  • ANGLE_texture_compression_dxt (BC1/2/3, linear).
  • EXT_texture_compression_s3tc_srgb (BC1/2/3, sRGB).
  • When the latter extension is not available, it may be possible to override linear sampling by using EXT_texture_format_sRGB_override extension. Otherwise, an application has to decode sRGB in a fragment shader.
• WebGL extensions:
  • WEBGL_compressed_texture_s3tc (BC1/2/3, linear).
  • WEBGL_compressed_texture_s3tc_srgb (BC1/2/3, sRGB).
  • When the latter extension is not available, an application has to decode sRGB in a fragment shader.

Vulkan, Metal, and Direct3D have native support for BC1/2/3. Direct3D and Metal support only alpha-"punchthrough" flavor of BC1.

BC7

• OpenGL ES and WebGL:
  • EXT_texture_compression_bptc extension (linear & sRGB).
• OpenGL Desktop:
  • ARB_texture_compression_bptc extension (linear & sRGB).

Vulkan, Metal, and Direct3D have native support for both variants of BC7.

ETC RGB/RGBA

Originally, this format supported only linear sampling and filtering of RGB without alpha. Given that ETC1 RGB format is a strict subset of ETC2 RGB format, and the latter supports hardware sRGB sampling and filtering, it became possible to use hardware sRGB sampling and filtering with existing ETC1 data on capable hardware.

• OpenGL Desktop:
  • ARB_ES3_compatibility (ETC2, linear & sRGB).
• OpenGL ES:
  • OES_compressed_ETC1_RGB8_texture (ETC1, linear only).
  • Open GL ES 3.0 core spec (all ETC2 formats).
  • When the OpenGL ES version is older than 3.0, an application has to decode sRGB in a fragment shader.
• WebGL:
  • WEBGL_compressed_texture_etc1 (ETC1, linear only).
  • WEBGL_compressed_texture_etc (ETC2, linear & sRGB).

PVRTC1/2

• No support in OpenGL Desktop.
• OpenGL ES extensions:
  • IMG_texture_compression_pvrtc (PVRTC1, linear).
  • IMG_texture_compression_pvrtc2 (PVRTC2, linear).
  • EXT_pvrtc_sRGB (PVRTC1/2, sRGB).
  • When the latter extension is not available, an application has to decode sRGB in a fragment shader.
• WebGL extensions:
  • WEBGL_compressed_texture_pvrtc (PVRTC1, linear).
  • An application always has to decode sRGB in a fragment shader.
• Vulkan extensions:
  • VK_IMG_format_pvrtc (PVRTC1/2, linear & sRGB).
• Metal supports only PVRTC1 (linear & sRGB).
• No support in Direct3D.

BC4/5, EAC R11/RG11

These formats support only linear sampling and filtering.

The link in Readme.md to the forums no longer works. I guess Khronos moved the forums since that was written.