osamu620 / openhtj2k Goto Github PK

View Code? Open in Web Editor NEW

33.0 3.0 10.0 65.4 MB

An open source implementation of ITU-T Rec.814 | ISO 15444-15 (a.k.a. HTJ2K).

License: BSD 3-Clause "New" or "Revised" License

CMake 4.31% C++ 94.23% HTML 1.46%

htj2k jpeg2000 jpeg2000-part15 wasm webassembly

openhtj2k's People

Contributors

Stargazers

Watchers

Forkers

takaxp jamaika1 g882152018 malaterre chafey baronren likewise jubalh bradh

openhtj2k's Issues

Blosc2 has recently implemented a plugin for OpenHTJ2K

First of all, thanks for your great work in the making of OpenHTJ2K @osamu620 !

This is just to inform that Blosc2 has recently introduced support for using the OpenHTJ2K codec, with codec ID 36. With it, it is possible to use OpenHTJ2K from any Blosc2 application transparently. We are distributing binary wheels of the plugin (only for x86_64 for now) that can be used either from Python or C applications.

This is our first public version, but we plan to continue improving our support for it. For example, we have noted that OpenHTJ2K is not thread-safe, so applications on top of it (like Blosc2) cannot call it from different threads. We have been playing with internal threading support in OpenHTJ2K, but we are not seeing a lot of acceleration. Do we need to use very large images for seeing actual speed-ups?

Thanks again and keep the good work!

Question about std::thread functions

Is there any way to disable threadpool.hpp?
I know this is a stupid question. It's good that the codec has the newer C++17 language. With only gcc 12.0.0 it isn't possible to create codec under Windows. Unless author doesn't want amateur to compile with gcc.
Will not apply the mingwstdthread plugins to std::thread.
Is there any way to disable threadpool.hpp?
https://github.com/meganz/mingw-std-threads
https://github.com/Jamaika1/mingw_std_threads

encoded image cannot be decoded

Here is an uncompressed mono image, and the resulting encoded image.
When I try to decode the encoded image, I get the following message:

Tile num x = 1, y = 1
ERROR: the number of components of the input is not three.

and decode fails.

images.zip

Support for 16 bit gray images?

My understanding is that OpenHTJ2K supports mainly 32 bit sRGB images (4 bytes/pixel) as inputs. We would be interested in knowing if it could accept 16 bit gray images as well, or we need to 'massage' the data to convert it to sRGB first (if so, anybody knows a good tool for doing that?). Thanks in advance!

New GCC 12.2.0

main_enc.cpp:52:30: warning: ignoring return value of 'bool std::filesystem::exists(const __cxx11::path&)', declared with attribute 'nodiscard' [-Wunused-result]
   52 |       std::filesystem::exists(fname);
      |       ~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~
In file included from c:\msys1220\include\c++\12.2.0\filesystem:47,
                 from main_enc.cpp:38:
c:\msys1220\include\c++\12.2.0\bits\fs_ops.h:133:3: note: declared here
  133 |   exists(const path& __p)
      |   ^~~~~~

Various HTJ2K files

Hi osamu620
I don't see the problem. Different authors and different ideas of HTJ2K. I was surprised to see a remake of the openjpen decoder. I have posted information about the various results. I didn't think anyone would bother you. :)
uclouvain/openjpeg#1374

Why can I not use capital letters in photo names?

htj2k-cjhc.exe -i Stationery.ppm -o image_21447_24bit-jhc.j2c Stiles={1000,750} Clevels=3 Cblk={64,64} Cprecincts={128,128},{256,256} Corder=CPRL Creversible=yes

ERROR: unknown parameter Stationery.ppm

Rsiz value is not consistent with ISO/IEC 15444-15

If I use OpenHTJ2K to create either a JPH file or a codestream, opening these files in a Hex editor shows that they have the following Rsiz value:

40 00

However, Annex A (HTJ2K codestream syntax) of ISO/IEC 15444-15:2019 (E) says that the 14th bit of Rsiz shall be 1 for HTJ2K codestreams:

A.2 SIZ marker segment

Bit 14 of Rsiz shall be equal to 1.

Since the actual value is 0 OpenHTJ2K-produced files, it looks like the software isn't quite following the standard here.

Add installation to CMake

I see the installation part is still commented out:
https://github.com/osamu620/OpenHTJ2K/blob/main/CMakeLists.txt#L169
The README also so far mentions just the build process and no install. So I assume this is intentionally not ready yet :)

I just wanted to open this issue to keep track of this that I can get notified when it's ready.

I started to create a test package for openSUSE in my home repo. Just to experiment with it and later once everything is ready push it to the main distribution.

HTJ2K codestream specific extensions (.jhc or .jphc) are dropped

Decoding of HTJ2K codestreams works fine, however, those two extensions (even the file extension is not defined in the spec) are dropped out of the decoder implementation.

Image size limit

It seems that there is an image size limit around 10k x 10k.
For larger images no error is issued but no output is saved.

encoder: encode already-subsampled images

It would useful to support encoding of sub-sampled images.

Colours are incorrect when encoding with 0 wavelet decompositions

It looks like no colour transform is applied.

Feature request: batch encoding / decoding

Hi @osamu620, hope all is well! Have you considered adding support for encoding or decoding a folder of
images ? As HTJ2K shows promise for video encoding, I think it would be interesting to see how the workflow
might run for a batch of images, to simulate a video stream. What do you think ?

Fortunately, C++17 supports platform-independent method to list files

#include <string>
#include <iostream>
#include <filesystem>
namespace fs = std::filesystem;

int main()
{
    std::string path = "/path/to/directory";
    for (const auto & entry : fs::directory_iterator(path))
        std::cout << entry.path() << std::endl;
}

Windows binary release

Hello,

I have tried to compile the project with Cmake + Visual Studio 2017 (Windows 10) but I get errors related to undefined symbols (the errors are different without ENABE_AVX2 but it still does not compile).

Do you plan some binary releases soon (probably the easiest for me as I only need the command line executable for testing)?

Best,
S.

TIFF images

Are you planning to add support for TIFF images (possibly multi-page)?

encode: memory corruption

This is the file:

https://github.com/GrokImageCompression/grok-test-data/blob/master/input/nonregression/ElephantDream_4K.tif

I converted it to .ppm using GraphicsMagick toolkit: gm convert ElephantDream_4K.tif ElephantDream_4K.ppm

Command line:

open_htj2k_enc_dbg -i ElephantDream_4K.ppm -o ElephantDream_4K.ppm.j2k

Note: this only happens for irreversible encode.

If you use valgrind, you will see:

valgrind open_htj2k_enc_dbg -i ElephantDream_4K.ppm  -o ElephantDream_4K.ppm.j2k


==10046== Invalid write of size 8
==10046==    at 0x484CA13: memmove (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==10046==    by 0x4962252: htj2k_encode(j2k_codeblock*, unsigned char) (src/OpenHTJ2K/source/core/coding/ht_block_encoding.cpp:881)
==10046==    by 0x49307A8: j2k_tile::encode(j2k_main_header&)::$_1::operator()(j2k_resolution*, unsigned char) const (src/OpenHTJ2K/source/core/coding/coding_units.cpp:2702)
==10046==    by 0x492FF6C: j2k_tile::encode(j2k_main_header&) (src/OpenHTJ2K/source/core/coding/coding_units.cpp:2734)
==10046==    by 0x496BA16: open_htj2k::openhtj2k_encoder_impl::invoke() (src/OpenHTJ2K/source/core/interface/encoder.cpp:170)
==10046==    by 0x496BF2C: open_htj2k::openhtj2k_encoder::invoke() (src/OpenHTJ2K/source/core/interface/encoder.cpp:210)
==10046==    by 0x40A1B6: main (src/OpenHTJ2K/source/apps/encoder/main_enc.cpp:123)
==10046==  Address 0x316c10d0 is 16,832 bytes inside a block of size 16,834 alloc'd
==10046==    at 0x4844293: operator new[](unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==10046==    by 0x411D17: std::_MakeUniq<unsigned char []>::__array std::make_unique<unsigned char []>(unsigned long) (unique_ptr.h:968)
==10046==    by 0x495EF06: htj2k_encode(j2k_codeblock*, unsigned char) (src/OpenHTJ2K/source/core/coding/ht_block_encoding.cpp:463)
==10046==    by 0x49307A8: j2k_tile::encode(j2k_main_header&)::$_1::operator()(j2k_resolution*, unsigned char) const (src/OpenHTJ2K/source/core/coding/coding_units.cpp:2702)
==10046==    by 0x492FF6C: j2k_tile::encode(j2k_main_header&) (src/OpenHTJ2K/source/core/coding/coding_units.cpp:2734)
==10046==    by 0x496BA16: open_htj2k::openhtj2k_encoder_impl::invoke() (src/OpenHTJ2K/source/core/interface/encoder.cpp:170)
==10046==    by 0x496BF2C: open_htj2k::openhtj2k_encoder::invoke() (src/OpenHTJ2K/source/core/interface/encoder.cpp:210)
==10046==    by 0x40A1B6: main (src/OpenHTJ2K/source/apps/encoder/main_enc.cpp:123)
==10046== 
==10046== Invalid write of size 8
==10046==    at 0x484C9FB: memmove (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==10046==    by 0x4962252: htj2k_encode(j2k_codeblock*, unsigned char) (src/OpenHTJ2K/source/core/coding/ht_block_encoding.cpp:881)
==10046==    by 0x49307A8: j2k_tile::encode(j2k_main_header&)::$_1::operator()(j2k_resolution*, unsigned char) const (src/OpenHTJ2K/source/core/coding/coding_units.cpp:2702)
==10046==    by 0x492FF6C: j2k_tile::encode(j2k_main_header&) (src/OpenHTJ2K/source/core/coding/coding_units.cpp:2734)
==10046==    by 0x496BA16: open_htj2k::openhtj2k_encoder_impl::invoke() (src/OpenHTJ2K/source/core/interface/encoder.cpp:170)
==10046==    by 0x496BF2C: open_htj2k::openhtj2k_encoder::invoke() (src/OpenHTJ2K/source/core/interface/encoder.cpp:210)
==10046==    by 0x40A1B6: main (src/OpenHTJ2K/source/apps/encoder/main_enc.cpp:123)
==10046==  Address 0x316c10d8 is 6 bytes after a block of size 16,834 alloc'd
==10046==    at 0x4844293: operator new[](unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==10046==    by 0x411D17: std::_MakeUniq<unsigned char []>::__array std::make_unique<unsigned char []>(unsigned long) (unique_ptr.h:968)
==10046==    by 0x495EF06: htj2k_encode(j2k_codeblock*, unsigned char) (src/OpenHTJ2K/source/core/coding/ht_block_encoding.cpp:463)
==10046==    by 0x49307A8: j2k_tile::encode(j2k_main_header&)::$_1::operator()(j2k_resolution*, unsigned char) const (src/OpenHTJ2K/source/core/coding/coding_units.cpp:2702)
==10046==    by 0x492FF6C: j2k_tile::encode(j2k_main_header&) (src/OpenHTJ2K/source/core/coding/coding_units.cpp:2734)
==10046==    by 0x496BA16: open_htj2k::openhtj2k_encoder_impl::invoke() (src/OpenHTJ2K/source/core/interface/encoder.cpp:170)
==10046==    by 0x496BF2C: open_htj2k::openhtj2k_encoder::invoke() (src/OpenHTJ2K/source/core/interface/encoder.cpp:210)
==10046==    by 0x40A1B6: main (src/OpenHTJ2K/source/apps/encoder/main_enc.cpp:123)
==10046== 
==10046== Invalid write of size 8
==10046==    at 0x484CA03: memmove (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==10046==    by 0x4962252: htj2k_encode(j2k_codeblock*, unsigned char) (src/OpenHTJ2K/source/core/coding/ht_block_encoding.cpp:881)
==10046==    by 0x49307A8: j2k_tile::encode(j2k_main_header&)::$_1::operator()(j2k_resolution*, unsigned char) const (src/OpenHTJ2K/source/core/coding/coding_units.cpp:2702)
==10046==    by 0x492FF6C: j2k_tile::encode(j2k_main_header&) (src/OpenHTJ2K/source/core/coding/coding_units.cpp:2734)
==10046==    by 0x496BA16: open_htj2k::openhtj2k_encoder_impl::invoke() (src/OpenHTJ2K/source/core/interface/encoder.cpp:170)
==10046==    by 0x496BF2C: open_htj2k::openhtj2k_encoder::invoke() (src/OpenHTJ2K/source/core/interface/encoder.cpp:210)
==10046==    by 0x40A1B6: main (src/OpenHTJ2K/source/apps/encoder/main_enc.cpp:123)
==10046==  Address 0x316c10e0 is 14 bytes after a block of size 16,834 alloc'd
==10046==    at 0x4844293: operator new[](unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==10046==    by 0x411D17: std::_MakeUniq<unsigned char []>::__array std::make_unique<unsigned char []>(unsigned long) (unique_ptr.h:968)
==10046==    by 0x495EF06: htj2k_encode(j2k_codeblock*, unsigned char) (src/OpenHTJ2K/source/core/coding/ht_block_encoding.cpp:463)
==10046==    by 0x49307A8: j2k_tile::encode(j2k_main_header&)::$_1::operator()(j2k_resolution*, unsigned char) const (src/OpenHTJ2K/source/core/coding/coding_units.cpp:2702)
==10046==    by 0x492FF6C: j2k_tile::encode(j2k_main_header&) (src/OpenHTJ2K/source/core/coding/coding_units.cpp:2734)
==10046==    by 0x496BA16: open_htj2k::openhtj2k_encoder_impl::invoke() (src/OpenHTJ2K/source/core/interface/encoder.cpp:170)
==10046==    by 0x496BF2C: open_htj2k::openhtj2k_encoder::invoke() (src/OpenHTJ2K/source/core/interface/encoder.cpp:210)
==10046==    by 0x40A1B6: main (src/OpenHTJ2K/source/apps/encoder/main_enc.cpp:123)
==10046== 
==10046== Invalid write of size 8
==10046==    at 0x484CA0B: memmove (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==10046==    by 0x4962252: htj2k_encode(j2k_codeblock*, unsigned char) (src/OpenHTJ2K/source/core/coding/ht_block_encoding.cpp:881)
==10046==    by 0x49307A8: j2k_tile::encode(j2k_main_header&)::$_1::operator()(j2k_resolution*, unsigned char) const (src/OpenHTJ2K/source/core/coding/coding_units.cpp:2702)
==10046==    by 0x492FF6C: j2k_tile::encode(j2k_main_header&) (src/OpenHTJ2K/source/core/coding/coding_units.cpp:2734)
==10046==    by 0x496BA16: open_htj2k::openhtj2k_encoder_impl::invoke() (src/OpenHTJ2K/source/core/interface/encoder.cpp:170)
==10046==    by 0x496BF2C: open_htj2k::openhtj2k_encoder::invoke() (src/OpenHTJ2K/source/core/interface/encoder.cpp:210)
==10046==    by 0x40A1B6: main (src/OpenHTJ2K/source/apps/encoder/main_enc.cpp:123)
==10046==  Address 0x316c10e8 is 22 bytes after a block of size 16,834 alloc'd
==10046==    at 0x4844293: operator new[](unsigned long) (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==10046==    by 0x411D17: std::_MakeUniq<unsigned char []>::__array std::make_unique<unsigned char []>(unsigned long) (unique_ptr.h:968)
==10046==    by 0x495EF06: htj2k_encode(j2k_codeblock*, unsigned char) (src/OpenHTJ2K/source/core/coding/ht_block_encoding.cpp:463)
==10046==    by 0x49307A8: j2k_tile::encode(j2k_main_header&)::$_1::operator()(j2k_resolution*, unsigned char) const (src/OpenHTJ2K/source/core/coding/coding_units.cpp:2702)
==10046==    by 0x492FF6C: j2k_tile::encode(j2k_main_header&) (src/OpenHTJ2K/source/core/coding/coding_units.cpp:2734)
==10046==    by 0x496BA16: open_htj2k::openhtj2k_encoder_impl::invoke() (src/OpenHTJ2K/source/core/interface/encoder.cpp:170)
==10046==    by 0x496BF2C: open_htj2k::openhtj2k_encoder::invoke() (src/OpenHTJ2K/source/core/interface/encoder.cpp:210)
==10046==    by 0x40A1B6: main (src/OpenHTJ2K/source/apps/encoder/main_enc.cpp:123)
==10046== 

valgrind: m_mallocfree.c:303 (get_bszB_as_is): Assertion 'bszB_lo == bszB_hi' failed.
valgrind: Heap block lo/hi size mismatch: lo = 16912, hi = 12670507942556446551.
This is probably caused by your program erroneously writing past the
end of a heap block and corrupting heap metadata.  If you fix any
invalid writes reported by Memcheck, this assertion failure will
probably go away.  Please try that before reporting this as a bug.


host stacktrace:
==10046==    at 0x5804255A: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
==10046==    by 0x58042687: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
==10046==    by 0x5804282B: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
==10046==    by 0x5804C580: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
==10046==    by 0x5803AB3A: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
==10046==    by 0x58039257: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
==10046==    by 0x5803DC5D: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
==10046==    by 0x580384B8: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
==10046==    by 0x1002EF1D7C: ???
==10046==    by 0x1002DB1F2F: ???
==10046==    by 0x1002DB1F17: ???
==10046==    by 0x1002DB1F2F: ???
==10046==    by 0x1C0F: ???

sched status:
  running_tid=1

Thread 1: status = VgTs_Runnable (lwpid 10046)
==10046==    at 0x484CAD3: memmove (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==10046==    by 0x4962252: htj2k_encode(j2k_codeblock*, unsigned char) (src/OpenHTJ2K/source/core/coding/ht_block_encoding.cpp:881)
==10046==    by 0x49307A8: j2k_tile::encode(j2k_main_header&)::$_1::operator()(j2k_resolution*, unsigned char) const (src/OpenHTJ2K/source/core/coding/coding_units.cpp:2702)
==10046==    by 0x492FF6C: j2k_tile::encode(j2k_main_header&) (src/OpenHTJ2K/source/core/coding/coding_units.cpp:2734)
==10046==    by 0x496BA16: open_htj2k::openhtj2k_encoder_impl::invoke() (src/OpenHTJ2K/source/core/interface/encoder.cpp:170)
==10046==    by 0x496BF2C: open_htj2k::openhtj2k_encoder::invoke() (src/OpenHTJ2K/source/core/interface/encoder.cpp:210)
==10046==    by 0x40A1B6: main (src/OpenHTJ2K/source/apps/encoder/main_enc.cpp:123)
client stack range: [0x1FFEFFC000 0x1FFF000FFF] client SP: 0x1FFEFFCD48
valgrind stack range: [0x1002CB2000 0x1002DB1FFF] top usage: 10192 of 1048576


Note: see also the FAQ in the source distribution.
It contains workarounds to several common problems.
In particular, if Valgrind aborted or crashed after
identifying problems in your program, there's a good chance
that fixing those problems will prevent Valgrind aborting or
crashing, especially if it happened in m_mallocfree.c.

If that doesn't help, please report this bug to: www.valgrind.org

In the bug report, send all the above text, the valgrind
version, and what OS and version you are using.  Thanks.

Support for armv7-a architecture

Hello.
It seems that OpenHTJ2K now doesn't support armv7-a. I tried to modify the CmakeLists.txt to cross-compile the source code with GCC 6.3.0. Finally, I failed.
I set CMAKE_CXX_STANDARD to 14 and CMAKE_HOST_SYSTEM_PROCESSOR to arm32, and then I added some code about CMAKE_HOST_SYSTEM_PROCESSOR matching arm32 to CmakeLists.txt. I set ENABLE_ARM_NEON to "OFF" because there is no definition of vaddv_u8().
I set -march=armv7-a -mtune=cortex-7a. At last, the compilation went to the following step:

[ 93%] Linking CXX executable bin/open_htj2k_enc
bin/libopen_htj2k_R.so: undefined reference to std::__atomic_futex_unsigned_base::_M_futex_notify_all(unsigned int*)' bin/libopen_htj2k_R.so: undefined reference to std::__atomic_futex_unsigned_base::_M_futex_wait_until(unsigned int*, unsigned int, bool, std::chrono::duration<long long, std::ratio<1ll, 1ll> >, std::chrono::duration<long long, std::ratio<1ll, 1000000000ll> >)'
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/open_htj2k_enc.dir/build.make:85: bin/open_htj2k_enc] Error 1
make[1]: *** [CMakeFiles/Makefile2:276: CMakeFiles/open_htj2k_enc.dir/all] Error 2
make: *** [Makefile:141: all] Error 2

I'd like to know whether it is because something is wrong with my modification of the CmakeLists.txt, or the source code now does not support armv7-a.

conformance test failures

Checking the test log, I see that tests 191,192, 121 and 122 are failing.
@osamu620 are you able to share what the issue is for these tests ?
Thanks!

Documentation of supported output file extensions and their effects doesn't reflect encoder behaviour

Both the project readme and open_htj2k_enc's helper text provide conflicting and partially inaccurate information on recognized file extensions for the output file, and their effect on the resulting file formats.

The readme says this (under Supported file types):

output codestreams: .j2k, .j2c, .jphc (Part 15 codestream), .jph (Part 15 file format)

Whereas open_htj2k_enc's helper text says something different:

-o: Output codestream
.jhc or .j2c are recommended as the extension.

I tried out all listed extensions, and ended up with this:

Extension .jphc results in a JPH file, not a codestream
Extension .jhc is not recognized at all, and results in "ERROR: invalid extension for output file"
Extensions .j2c and j2k both result in a raw codestream (as expected)
Extension .jph results in a JPH file (as expected)

Adding to the confusion here is the use of "output codestream" (both in the readme and the helper text), since this option really defines an output file. This file can either be a Part 15 codestream, or a JPH file (in which case it isn't a codestream!).

Looking at E.2.2 and E.3.2 (Registration) in ISO/IEC 15444-15:2019 (E), this lists .jph as the registered extension for JPH files, and .jhc as the registered extension for codestreams. It doesn't mention a .jphc extension, although jphc is listed as a subtype name for codestreams.

My suggestion would be to change the description of the -o option to "output file" (both in the readme and the helper text), and stick to these conventions for the output extensions:

.jph extension triggers JPH file creation
.jhc extension triggers HT codestream creation (optionally with .j2c and/or .j2k as aliases)
drop the .jphc extension altogether (since it's not meant to be used as a file extension)

How to omit this function advance() when decoding.

I know the codec is under development. I have a query for the advance() function.
f3af95b
How to omit this function when decoding.
htj2k-cjhc.exe -i image_21447_24bit.ppm -o image_21447_24bit-jhc.j2c Creversible=yes -num_threads 1
elapsed time for reading inputs 24.939 [ms]
585.866474 [MB/s]
Codestream bytes = 3703013 = 12.165187 [bits/pixel]
elapsed time 297.912 [ms]
htj2k-djhc.exe -i image_21447_24bit-jhc.j2c -o image_21447_24bit-jhc_hc.ppm -num_threads 1
ERROR: n = -1, n
Did I forget to add some function from help?

removing exit() calls in library

Hi @osamu620 hope all is well. I notice that there are a number of calls to exit() in the library.
I think this may be an unexpected event for a library user to have their application exit if there is
an error in the OpenHTJ2K library. Is it possible to throw an exception instead, and handle it in
the library ? Or return false ? Thanks!

optimizing block coder

hello @osamu620 , I've run linux perf command on the encoder, and here are the top two calls for CPU usage.

   6.59%          1024  open_htj2k_enc_  libopen_htj2k_d.so   [.] make_storage                                                                                                                            ◆
   5.91%           914  open_htj2k_enc_  libopen_htj2k_d.so   [.] j2k_codeblock::set_MagSgn_and_sigma

Taking a look at make_storage method, this can be optimised. Best approach I think would be to turn it into a macro,
with no function calls inside. What do you think ?

Bits per component value in JPH image header box is off by 1 bit

It seems that for JPH output, OpenHTJ2K writes an erroneous Bits per component value to the image header box. I'll illustrate this using one of OpenHTJ2K's conformance data images as a starting point.

Here's the command I used to create the JPH file:

open_htj2k_enc -i kodim23.ppm -o kodim23.jph Qfactor=30

Validation with jpylyzer results in this error:

    <isValid format="jph">False</isValid>
    <tests>
        <bPCDepthConsistentWithSIZ>False</bPCDepthConsistentWithSIZ>
    </tests>

This error means that the Bits per component value in the JPH image header is not consistent with the information in the code stream.

The Jpylyzer output for the image header box shows a bPCDepth of 9:

                <nC>3</nC>
                <bPCSign>unsigned</bPCSign>
                <bPCDepth>9</bPCDepth>

However, the ssizDepth values in the SIZ marker segment indicate a bit depth of 8 bits for all components:

                <csiz>3</csiz>
                <ssizSign>unsigned</ssizSign>
                <ssizDepth>8</ssizDepth>
                <xRsiz>1</xRsiz>
                <yRsiz>1</yRsiz>
                <ssizSign>unsigned</ssizSign>
                <ssizDepth>8</ssizDepth>
                <xRsiz>1</xRsiz>
                <yRsiz>1</yRsiz>
                <ssizSign>unsigned</ssizSign>
                <ssizDepth>8</ssizDepth>

I double checked with ExifTool, which confirms the odd bits per component values in the JPH image header box:

 <Jpeg2000:NumberOfComponents>3</Jpeg2000:NumberOfComponents>
 <Jpeg2000:BitsPerComponent>9 Bits, Unsigned</Jpeg2000:BitsPerComponent>

Possible underlying cause: ISO 15444-1 defines the BPC parameter as follows (section I.5.3.1):

BPC: Bits per component. This parameter specifies the bit depth of the components in the codestream,
minus 1, and is stored as a 1-byte field.

So my best guess is that OpenHTJ2K directly writes the bits per component values to the image header box, without first subtracting 1.

tests are failed on MSYS2 with gcc

The following tests for HT are failed on MSYS2 with GCC 10.3.0. The tests for Part 1 decoding are fine.

The following tests FAILED:
         39 - comp_p0_ht_01_11 (Failed)
         40 - comp_p0_ht_02_11 (Failed)
         41 - comp_p0_ht_03_11 (Failed)
         42 - comp_p0_ht_03_14 (Failed)
         43 - comp_p0_ht_04_11r (Failed)
         44 - comp_p0_ht_04_11g (Failed)
         45 - comp_p0_ht_04_11b (Failed)
         46 - comp_p0_ht_04_12r (Failed)
         47 - comp_p0_ht_04_12g (Failed)
         48 - comp_p0_ht_04_12b (Failed)
         49 - comp_p0_ht_05_11a (Failed)
         50 - comp_p0_ht_05_11b (Failed)
         51 - comp_p0_ht_05_11c (Failed)
         52 - comp_p0_ht_05_11d (Failed)
         53 - comp_p0_ht_05_12a (Failed)
         54 - comp_p0_ht_05_12b (Failed)
         55 - comp_p0_ht_05_12c (Failed)
         56 - comp_p0_ht_05_12d (Failed)
         57 - comp_p0_ht_06_11a (Failed)
         58 - comp_p0_ht_06_11b (Failed)
         59 - comp_p0_ht_06_11c (Failed)
         60 - comp_p0_ht_06_11d (Failed)
         61 - comp_p0_ht_06_15a (Failed)
         62 - comp_p0_ht_06_15b (Failed)
         63 - comp_p0_ht_06_15c (Failed)
         64 - comp_p0_ht_06_15d (Failed)
         65 - comp_p0_ht_06_18a (Failed)
         66 - comp_p0_ht_06_18b (Failed)
         67 - comp_p0_ht_06_18c (Failed)
         68 - comp_p0_ht_06_18d (Failed)
         70 - comp_p0_hm_06_11b (Failed)
         71 - comp_p0_hm_06_11c (Failed)
         72 - comp_p0_hm_06_11d (Failed)
         74 - comp_p0_hm_06_18b (Failed)
         75 - comp_p0_hm_06_18c (Failed)
         76 - comp_p0_hm_06_18d (Failed)
         77 - comp_p0_ht_07_11r (Failed)
         78 - comp_p0_ht_07_11g (Failed)
         79 - comp_p0_ht_07_11b (Failed)
         80 - comp_p0_ht_07_15r (Failed)
         81 - comp_p0_ht_07_15g (Failed)
         82 - comp_p0_ht_07_15b (Failed)
         83 - comp_p0_ht_07_16r (Failed)
         84 - comp_p0_ht_07_16g (Failed)
         85 - comp_p0_ht_07_16b (Failed)
         86 - comp_p0_ht_08_11r (Failed)
         87 - comp_p0_ht_08_11g (Failed)
         88 - comp_p0_ht_08_11b (Failed)
         89 - comp_p0_ht_08_15r (Failed)
         90 - comp_p0_ht_08_15g (Failed)
         91 - comp_p0_ht_08_15b (Failed)
         92 - comp_p0_ht_08_16r (Failed)
         93 - comp_p0_ht_08_16g (Failed)
         94 - comp_p0_ht_08_16b (Failed)
         96 - comp_p0_ht_10_11r (Failed)
         97 - comp_p0_ht_10_11g (Failed)
         98 - comp_p0_ht_10_11b (Failed)
        104 - comp_p0_ht_15_11 (Failed)
        105 - comp_p0_ht_15_14 (Failed)
        106 - comp_p0_hm_15_8 (Failed)
        107 - comp_p0_ht_16_11 (Failed)
        108 - comp_p1_ht_01_11 (Failed)
        109 - comp_p1_ht_01_12 (Failed)
        110 - comp_p1_ht_02_11r (Failed)
        111 - comp_p1_ht_02_11g (Failed)
        112 - comp_p1_ht_02_11b (Failed)
        113 - comp_p1_ht_02_12r (Failed)
        114 - comp_p1_ht_02_12g (Failed)
        115 - comp_p1_ht_02_12b (Failed)
        116 - comp_p1_ht_03_11a (Failed)
        117 - comp_p1_ht_03_11b (Failed)
        118 - comp_p1_ht_03_11c (Failed)
        119 - comp_p1_ht_03_11d (Failed)
        120 - comp_p1_ht_03_12a (Failed)
        123 - comp_p1_ht_03_12d (Failed)
        124 - comp_p1_ht_04_9 (Failed)
        125 - comp_p1_ht_05_11r (Failed)
        126 - comp_p1_ht_05_11g (Failed)
        127 - comp_p1_ht_05_11b (Failed)

The decoded results look very noisy.

Compiling problem on aarch64 platform

Hello, when I compile the source code on aarch64 platform with g++ 8.5.0. The compiler gives out the error information below:

[ 3%] Building CXX object CMakeFiles/open_htj2k.dir/source/core/codestream/j2kmarkers.cpp.o
In file included from /home/centos/OpenHTJ2K-0.2.1-origin/source/core/codestream/codestream.hpp:37,
from /home/centos/OpenHTJ2K-0.2.1-origin/source/core/codestream/j2kmarkers.hpp:34,
from /home/centos/OpenHTJ2K-0.2.1-origin/source/core/codestream/j2kmarkers.cpp:29:
/home/centos/OpenHTJ2K-0.2.1-origin/source/core/common/utils.hpp:62:41: error: variable or field ‘_mm256_storeu2_m128i’ declared void
static inline void _mm256_storeu2_m128i(__m128i_u* __addr_hi, __m128i_u* __addr_lo, __m256i __a) {
^~~~~~~~~
/home/centos/OpenHTJ2K-0.2.1-origin/source/core/common/utils.hpp:62:41: error: ‘__m128i_u’ was not declared in this scope
/home/centos/OpenHTJ2K-0.2.1-origin/source/core/common/utils.hpp:62:52: error: ‘__addr_hi’ was not declared in this scope
static inline void _mm256_storeu2_m128i(__m128i_u* __addr_hi, __m128i_u* __addr_lo, __m256i __a) {
^~~~~~~~~
/home/centos/OpenHTJ2K-0.2.1-origin/source/core/common/utils.hpp:62:52: note: suggested alternative: ‘__caddr_t’
static inline void _mm256_storeu2_m128i(__m128i_u* __addr_hi, __m128i_u* __addr_lo, __m256i __a) {
^~~~~~~~~
__caddr_t
/home/centos/OpenHTJ2K-0.2.1-origin/source/core/common/utils.hpp:62:63: error: ‘__m128i_u’ was not declared in this scope
static inline void _mm256_storeu2_m128i(__m128i_u* __addr_hi, __m128i_u* __addr_lo, __m256i __a) {
^~~~~~~~~
/home/centos/OpenHTJ2K-0.2.1-origin/source/core/common/utils.hpp:62:74: error: ‘__addr_lo’ was not declared in this scope
static inline void _mm256_storeu2_m128i(__m128i_u* __addr_hi, __m128i_u* __addr_lo, __m256i __a) {
^~~~~~~~~
/home/centos/OpenHTJ2K-0.2.1-origin/source/core/common/utils.hpp:62:74: note: suggested alternative: ‘__caddr_t’
static inline void _mm256_storeu2_m128i(__m128i_u* __addr_hi, __m128i_u* __addr_lo, __m256i __a) {
^~~~~~~~~
__caddr_t
/home/centos/OpenHTJ2K-0.2.1-origin/source/core/common/utils.hpp:62:85: error: ‘__m256i’ was not declared in this scope
static inline void _mm256_storeu2_m128i(__m128i_u* __addr_hi, __m128i_u* __addr_lo, __m256i __a) {

and the cmake output information is:

-- The CXX compiler identification is GNU 8.5.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test COMPILER_SUPPORTS_CXX11
-- Performing Test COMPILER_SUPPORTS_CXX11 - Success
-- Performing Test COMPILER_SUPPORTS_CXX14
-- Performing Test COMPILER_SUPPORTS_CXX14 - Success
-- Performing Test COMPILER_SUPPORTS_CXX17
-- Performing Test COMPILER_SUPPORTS_CXX17 - Success
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Could NOT find TIFF (missing: TIFF_LIBRARY TIFF_INCLUDE_DIR)
-- aarch64 GNU
-- Thread library found
-- OPENHTJ2K_THREAD is set
-- OPENHTJ2K_ENABLE_ARM_NEON is set
-- Configuring done
-- Generating done
-- Build files have been written to: /home/centos/OpenHTJ2K-0.2.1-origin/build

rate control

Hello!
Very glad to see another HTJ2K project getting started.
Do you plan on implementing rate control or new Q factor for HTJ2K ?
Many thanks,
Aaron

Problem with decoding

OpenHTJ2K ver. 7e59ff1
https://ibb.co/wpfj7Zg

encoder feature request : support multiple HT sets

This would be a first step towards adding rate control to the encoder.

@osamu620 the simplest approach would be, as you suggested in another issue, to only perform cleanup pass in each HT set.

Problem with decoding files

How to properly decode files ppm with htj2k?
d1cd229
htj2k-cjhc.exe -i image_21447_24bit.ppm -o image_21447_24bit-jhc.j2c Creversible=yes -num_threads 1
htj2k-djhc.exe -i image_21447_24bit-jhc.j2c -o image_21447_24bit-jhc_hc.ppm -num_threads 1
https://www.sendspace.com/file/ny7822

osamu620 / openhtj2k Goto Github PK

openhtj2k's People

Contributors

Stargazers

Watchers

Forkers

openhtj2k's Issues

A.2 SIZ marker segment

Recommend Projects

Recommend Topics

Recommend Org