todds-encoder / todds Goto Github PK

https://docs.opencv.org/4.x/d1/d5b/group__datasets__sr.html

When a filesystem exception happens, png2dds is just printing it instead of reporting a proper error back to the user. This should be improved.

Example of exception: png2dds has been terminated because of an exception: filesystem::recursive_directory_iterator increment error: The system cannot find the file specified [system:2]

todds already keeps an internal list of data with an entry for every image, which contains the width and height among other data.

By accumulating pixel count and using the total time that is already there, it would be possible to print the total pixels and total pixels by time.

With the new report system, it will also possible to estimate the current pixels / time.

Describe the bug
todds produces incorrect DDS files on Mac OS M1

To Reproduce

1. Compile todds in an ARM Mac computer.
2. Encode any dds texture
3. Check the texture in TacentView.

Expected behavior
The texture does not have any artifacts.

System (please complete the following information):

• OS: MacOS
• CPU model: ARM

The pipeline should be better encapsulated, to allow using it as a static library on separate applications.

Since todds supports static builds, it is possible to generate a binary that contains all of the dependencies. The remaining issue has always been glibc. So far, we have been building binaries using increasingly older Linux distributions, in the hopes of finally finding a combination that works for everyone.

This is time consuming, as it requires setting up a new VM and figuring out how to compile todds in that case. Also, lately we have been reaching a point where using even older distros would mean switching from GCC 13 to older versions, which could potentially cause performance impacts. We need an alternative which addresses these problems.

One potential way to address all of this would be to use crosstool-ng. This is explained in the "Setup 2" part of this excellent Stackoverflow response: https://stackoverflow.com/a/52550158

This setup should give us freedom to choose which glibc version to use without requiring a VM.

ToDDS (in this case the RimWorld UI version) is only using 16 cores by default on a AMD Ryzen Threadripper 3970X CPU, which has 32 cores.

Describe the bug
bitdenfender think todds is a virus

To Reproduce
download todds

System (please complete the following information):
windows 11

Additional context

im not the one who got the repport btw

When this setting is used, todds takes a noticeable amount of time before the progress bar is shown. It should be possible to optimize these checks. Some feedback to the user would also be helpful.

OpenCV has implementations of specialized upscaling algorithms that todds currently does not support, which are described for example in here: https://towardsdatascience.com/deep-learning-based-super-resolution-with-opencv-4fd736678066. It should be possible to integrate these algorithms into the todds codebase and its pipeline.

Benchmarks and quality metrics: https://docs.opencv.org/4.x/dc/d69/tutorial_dnn_superres_benchmark.html

When upscaling is used, todds could use the original image as the first level mipmap, to minimize quality loss during mipmap calculations.

todds.txt

Support other cross-platform BC7 encoding tools such as:

• bc7enc without ISPC: https://github.com/richgel999/bc7enc_rdo
• Compressonator: https://github.com/GPUOpen-Tools/compressonator

RimWorld mod: https://steamcommunity.com/workshop/filedetails/?id=2205980094

Result:

Based on https://github.com/lukka/CppCMakeVcpkgTemplate/tree/v11

Features:

• Cross-platform builds for all supported platforms.
• clang-format
• clang-tidy
• Potentially, other static analysis tools.

Depends on #24 and #25

Good example of a CI pipeline: https://github.com/CaptainArbitrary/RimWorldMod/blob/main/.github/workflows/ci.yml

Lanczos/Kaiser

See:

• http://number-none.com/product/Mipmapping,%20Part%201/index.html
• http://number-none.com/product/Mipmapping,%20Part%202/index.html

When encoding the textures of the Rim-Effect: Core mod (https://steamcommunity.com/sharedfiles/filedetails/?id=2479560240) the following error appears:

PNG Decoding error -> Progressive decode error in /home/joseasoler/.local/share/Steam/steamapps/workshop/content/294100/2479560240/Textures/Things/Apparel/FormalShirt/FormalShirt_Thin_south.png: IDAT stream error

See http://www.ludicon.com/castano/blog/articles/computing-alpha-mipmaps/

https://steamcommunity.com/sharedfiles/filedetails/?id=2919414361

todds currently only checks PSNR. It should also check at least SSIM.

"Retrieving files to be removed" or "deleted" would be better, I think.

Severity: INCREDIBLY MINOR.

Describe the bug
When loading an incorrect path, todds outputs error messages that do not help at all in identifying what is wrong.

To Reproduce
Steps to reproduce the behavior:

1. Launch todds on Windows. Provide a malformed path such as E:/Games/RimWorld/Mods\animal-gear-master\Textures\Things\Pawn\Animal\Apparel\emptyGear_south.png
2. todds displays the following errors:

Load PNG file error in E:/Games/RimWorld/Mods\animal-gear-master\Textures\Things\Pawn\Animal\Apparel\emptyGear_south.png Could not load any data for PNG file E:/Games/RimWorld/Mods\animal-gear-master\Textures\Things\Pawn\Animal\Apparel\emptyGear_south.png

Expected behavior
todds displays helpful error messages which help diagnose this issue.

System (please complete the following information):

• OS: Windows

Can you make it so that we can set how many concurrent jobs can be run for conversion

Currently this file contains the implementation of all filters. Changing one of them leads to long compilation times. They should be split into different cpp files to simplify the code and prevent this issue.

The current CMake script for compiling bc7e_ispc hardcodes the SIMD instruction set and object files that ispc will generate. On ARM platforms, the SIMD instruction set should be --target=neon. This will also only generate bc7e.o, without any additional bc7e_####.o files.

Ideally, the script should be modified to work in all platforms without any extra workarounds.

The goal is to provide an alternate implementation which does not suffer from #16.

This task depends on #29 and #30.

A Windows user was experiencing the following exception:

todds has been terminated because of an exception: filesystem::recursive_directory_iterator increment error: The system cannot find the file specified [system:2]

According to this boost::filesystem issue, this exception happens when the filesystem is corrupt. Another fact that points to a corruption as the cause is that I managed to repro this issue at one point using mods from RimWorld as the input, but the issue went away after validating game files and letting Steam update the mods.

Potential fixes are discussed in the boost::filesystem issue mentioned above and in here: boostorg/filesystem@9a14c37

See https://www.nvidia.com/en-us/drivers/np2-mipmapping/

Hyperscan lacks support for MacOS ARM. One potential alternative is https://github.com/VectorCamp/vectorscan, but it lacks vcpkg support.

For some reason Todds keep using more and more ram, and after like 5min it crashes Wayland cause there is not enought ram available (no swap)

I didn't look at how Todds work but I assume it inport game asset into memory, are they removed after ?
Note: i only use the one include in rimsort