ADCore R3-4 adds support for compressing NDArrays in NDPluginCodec, and serving compressed NTNDArrays in NDPluginPva.

The ImageJ EPICS_NTNDA_Viewer.java code needs to support reading such compressed arrays, since it can substantially reduce network traffic when the areaDetector IOC and ImageJ viewer are running on different machines.

This may be more of an ImageJ issue than ADViewer (EPICS_AD_Viewer.java) but currently if settings in ImageJ, such as brightness or contrast are changed, the next image acquired by the ADViewer plugin resets the brightness and contrast to some default value. This issue happens with RGB1 color mode, but not the Mono color mode.

Mark Rivers also verified this behavior with EPICS_NTNDA_Viewer.java.

The NDPluginCodec supports scalar arrays of all numeric types: int8,uint8,...,int64,uint64.
In all cases the compresed array will have type byte (which is the same as int8)
For all except int8 and uint8, if client and server have different byte order, then byte order must be switched by either client or server.

Lets assume the server compresses and client decompresses.
Then it should be the client that switches byte order after decompression.
In order to do this the NTNDArray.codec.attribute structure most have fields like:

    bool serverByteOrderBigEndian
    bool clientByteOrderBigEndian

If these differ than the client must switch byte order

In the C code for blosc there are methods:

int blosc_compress(int clevel, int doshuffle, size_t typesize,
                 size_t nbytes, const void *src, void *dest,
                 size_t destsize);

and

BLOSC_EXPORT int blosc_decompress(const void *src, void *dest, size_t destsize);

The doshuffle argument can be one of

 #define BLOSC_NOSHUFFLE   0  /* no shuffle */
 #define BLOSC_SHUFFLE     1  /* byte-wise shuffle */
 #define BLOSC_BITSHUFFLE  2  /* bit-wise shuffle */

I think that BLOSC_SHUFFLE just means switch byte order.

Only compress has an argument to switch byte order.

But we want client to switch the byte order.

There is also a method:

void
shuffle(const size_t bytesoftype, const size_t blocksize,
      const uint8_t* _src, const uint8_t* _dest);

The client can call this if byte order needs to be changed.

BUT The Java blosc code does not provide this method.
Thus fir

@mrkraimer has written a new NTNDCodec.java which moves the decompression code into a separate class. This is cleaner and more re-usable so it should be used in a future release. It will require some work, and I want to release R1-3 soon so this will wait for R1-4.

We recently became interested in using an HDF5 Viewer (v0.13.0) that supports opening very large HDF files. Using the latest version of ImageJ for Linux bundled with Java 1.8.0_172, the HDF viewer itself compiles and works normally. As soon as the EPICS_areaDetector folder is added into ImageJ/plugins, the following error appears when launching the HDF viewer:
ImageJ 1.53e; Java 1.8.0_172 [64-bit]; Linux 4.15.0-123-generic; 87MB of 85867MB (<1%)
java.lang.NoSuchMethodError: org.slf4j.Logger.trace(Ljava/lang/String;Ljava/lang/Object;Ljava/lang/Object;)V
at hdf.object.FileFormat.(FileFormat.java:315)
at hdf.object.h5.H5File.(H5File.java:174)
at hdf.object.h5.H5File.(H5File.java:127)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at hdf.object.FileFormat.(FileFormat.java:228)
at ch.psi.imagej.hdf5.HDF5Reader.open(HDF5Reader.java:82)
at ch.psi.imagej.hdf5.HDF5Reader.open(HDF5Reader.java:51)
at ch.psi.imagej.hdf5.HDF5Reader.run(HDF5Reader.java:36)
at ij.IJ.runUserPlugIn(IJ.java:240)
at ij.IJ.runPlugIn(IJ.java:203)
at ij.Executer.runCommand(Executer.java:150)
at ij.Executer.run(Executer.java:68)
at java.lang.Thread.run(Thread.java:748)

When the EPICS_areaDetector folder is removed, the HDF Viewer goes back to working normally. I'm using Ubuntu 16.04.1 64-bit with kernel 4.15.0-123-generic.

I don't know much about Java. I've been able to use EPICS_AD_Viewer.java following the instructions without any problem inside ImageJ with earlier versions of Ubuntu (i.e. 18.04, 20.04).
But I couldn't compile it with ubuntu 22.04.
Any help is appreciated.
======================error message========================
warning: [options] bootstrap class path not set in conjunction with -source 1.6
/home/lyang/ImageJ/plugins/EPICS_areaDetector/EPICS_AD_Viewer.java:18: error: package gov.aps.jca does not exist
import gov.aps.jca.;
^
/home/lyang/ImageJ/plugins/EPICS_areaDetector/EPICS_AD_Viewer.java:19: error: package gov.aps.jca.dbr does not exist
import gov.aps.jca.dbr.;
^
/home/lyang/ImageJ/plugins/EPICS_areaDetector/EPICS_AD_Viewer.java:20: error: package gov.aps.jca.configuration does not exist
import gov.aps.jca.configuration.;
^
/home/lyang/ImageJ/plugins/EPICS_areaDetector/EPICS_AD_Viewer.java:21: error: package gov.aps.jca.event does not exist
import gov.aps.jca.event.;
^
/home/lyang/ImageJ/plugins/EPICS_areaDetector/EPICS_AD_Viewer.java:32: error: cannot find symbol
DBRType dataType;
^
symbol: class DBRType
location: class EPICS_AD_Viewer
/home/lyang/ImageJ/plugins/EPICS_areaDetector/EPICS_AD_Viewer.java:47: error: cannot find symbol
JCALibrary jca;
^
symbol: class JCALibrary
location: class EPICS_AD_Viewer
/home/lyang/ImageJ/plugins/EPICS_areaDetector/EPICS_AD_Viewer.java:48: error: cannot find symbol
DefaultConfiguration conf;
^
symbol: class DefaultConfiguration
location: class EPICS_AD_Viewer
/home/lyang/ImageJ/plugins/EPICS_areaDetector/EPICS_AD_Viewer.java:49: error: cannot find symbol
Context ctxt;
^
symbol: class Context
location: class EPICS_AD_Viewer
/home/lyang/ImageJ/plugins/EPICS_areaDetector/EPICS_AD_Viewer.java:52: error: cannot find symbol
Channel ch_nx;
^
symbol: class Channel
location: class EPICS_AD_Viewer
/home/lyang/ImageJ/plugins/EPICS_areaDetector/EPICS_AD_Viewer.java:53: error: cannot find symbol
Channel ch_ny;
^
symbol: class Channel
location: class EPICS_AD_Viewer
/home/lyang/ImageJ/plugins/EPICS_areaDetector/EPICS_AD_Viewer.java:54: error: cannot find symbol
Channel ch_nz;
^
symbol: class Channel
location: class EPICS_AD_Viewer
/home/lyang/ImageJ/plugins/EPICS_areaDetector/EPICS_AD_Viewer.java:55: error: cannot find symbol
Channel ch_colorMode;
^
symbol: class Channel