PNG Version + Byte Buffer version about twelvemonkeys HOT 10 CLOSED

Hi Peter,

Thanks for your feedback!

I haven't had any issues with the default PNGImageReader that comes with the JRE (the speed has been tolerable for the projects I've worked on), so creating my own haven't been a priority. But of course, if you or anyone would like to contribute or develop a better one, I'm sure that would be a welcome addition! Do you have some source code that I could look at and perhaps benchmark?

For a ByteBuffer version, I'm not really sure if understand the use case. I have thought about de-coupling the reader/writer code from the ImageIO API, for use with Android, JavaFX etc, but it will be a huge task, because the Java2D API is so heavily used (and it's a very good API for the job).

If the requirement is to load directly into a nio ByteBuffer, I do have some code that let's you create a BufferedImage (or a DataBuffer, really) backed by a Buffer. Right now I use it for loading huge images (2GB+) into memory mapped files. But I think it could be used to load images directly into a shared native memory buffer as well, with some minor work. Have a look at the MappedFileBuffer class for now (I'll update with a direct buffer class later, as I only have it at my home laptop).

Regards,

Harald K

from twelvemonkeys.

Hi

For the ByteBuffer, the use case is for passing textures to opengl via LWJGL or JOGL. You basically create a ByteBuffer and fill it up with RGB values or RGBA values and then pass this to opengl with the correct parameters.

I have some code in my engine for a PNG loader, although its not complete. I used this to bench mark loading a PNG to a byte buffer, using ImageIO I created an RGB int[] and then created the byte buffer directly. This conversion was quite costly (although I could probably optimise it a bit). That code is not checked in yet so I can't show you that.

https://github.com/darkyellow/DarkYellowEngine/blob/master/com/darkyellow/loader/png/PNG.java

It only supports true color and true color alpha but so far this is the only use case I have, its is probably not that hard to add the other options. I also ignore most of the tags as they are not relevant

I will have a look at the MappedFileBuffer on the weekend and comment back on how I get on

Regards

Peter

from twelvemonkeys.

package com.twelvemonkeys.image;

import java.awt.image.DataBuffer;
import java.nio.Buffer;
import java.nio.ByteBuffer;
import java.nio.IntBuffer;
import java.nio.ShortBuffer;

/**
 * A {@code DataBuffer} implementation that is backed by non-heap memory.
 * Memory will be allocated outside the normal JVM heap, allowing more efficient
 * memory usage for large buffers.
 *
 * @author <a href="mailto:[email protected]">Harald Kuhr</a>
 * @author last modified by $Author: haraldk$
 * @version $Id: MappedFileBuffer.java,v 1.0 Jun 12, 2010 4:56:51 PM haraldk Exp$
 *
 * @see java.nio.ByteBuffer#allocateDirect(int)
 */
public abstract class DirectBuffer extends DataBuffer {
    // TODO: This class doesn't seem to have any purpose, as it seems limited by heap memory anyway
    // ("java.lang.OutOfMemoryError: Direct buffer memory" if allocation larger than heap)
    private final Buffer buffer;

    private DirectBuffer(final int type, final int size, final int numBanks) {
        super(type, size, numBanks);

        if (size < 0) {
            throw new IllegalArgumentException("Integer overflow for size: " + size);
        }

        int componentSize;
        switch (type) {
            case DataBuffer.TYPE_BYTE:
                componentSize = 1;
                break;
            case DataBuffer.TYPE_USHORT:
                componentSize = 2;
                break;
            case DataBuffer.TYPE_INT:
                componentSize = 4;
                break;
            default:
                throw new IllegalArgumentException("Unsupported data type: " + type);
        }

        long length = ((long) size) * componentSize * numBanks;

        if (length > Integer.MAX_VALUE) {
            throw new IllegalArgumentException("Size too large for direct buffer " + length);
        }

        ByteBuffer byteBuffer = ByteBuffer.allocateDirect((int) length);

        switch (type) {
            case DataBuffer.TYPE_BYTE:
                buffer = byteBuffer;
                break;
            case DataBuffer.TYPE_USHORT:
                buffer = byteBuffer.asShortBuffer();
                break;
            case DataBuffer.TYPE_INT:
                buffer = byteBuffer.asIntBuffer();
                break;
            default:
                throw new IllegalArgumentException("Unsupported data type: " + type);
        }
    }

    @Override
    public String toString() {
        return String.format("DirectBuffer: %s", buffer);
    }

    public static DataBuffer create(final int type, final int size, final int numBanks) {
        switch (type) {
            case DataBuffer.TYPE_BYTE:
                return new DataBufferByte(size, numBanks);
            case DataBuffer.TYPE_USHORT:
                return new DataBufferUShort(size, numBanks);
            case DataBuffer.TYPE_INT:
                return new DataBufferInt(size, numBanks);
            default:
                throw new IllegalArgumentException("Unsupported data type: " + type);
        }
    }

    final static class DataBufferByte extends DirectBuffer {
        private final ByteBuffer buffer;

        public DataBufferByte(int size, int numBanks) {
            super(DataBuffer.TYPE_BYTE, size, numBanks);
            buffer = (ByteBuffer) super.buffer;
        }

        @Override
        public int getElem(int bank, int i) {
            return buffer.get(bank * size + i);
        }

        @Override
        public void setElem(int bank, int i, int val) {
            buffer.put(bank * size + i, (byte) val);
        }
    }

    final static class DataBufferUShort extends DirectBuffer {
        private final ShortBuffer buffer;

        public DataBufferUShort(int size, int numBanks) {
            super(DataBuffer.TYPE_USHORT, size, numBanks);
            buffer = (ShortBuffer) super.buffer;
        }

        @Override
        public int getElem(int bank, int i) {
            return buffer.get(bank * size + i);
        }

        @Override
        public void setElem(int bank, int i, int val) {
            buffer.put(bank * size + i, (short) (val & 0xffff));
        }
    }

    final static class DataBufferInt extends DirectBuffer {
        private final IntBuffer buffer;

        public DataBufferInt(int size, int numBanks) {
            super(DataBuffer.TYPE_INT, size, numBanks);
            buffer = (IntBuffer) super.buffer;
        }

        @Override
        public int getElem(int bank, int i) {
            return buffer.get(bank * size + i);
        }

        @Override
        public void setElem(int bank, int i, int val) {
            buffer.put(bank * size + i, val);
        }
    }
}

from twelvemonkeys.

Hi,

My DirectBuffer class, as promised. As you can see, I never found a use case for it, so I never checked it in. But you should be able to use it just like MappedFileBuffer (create a modified version of MappedImageFactory that uses DirectBuffer instead).

Good luck!

Harald K

from twelvemonkeys.

Thanks, is there an example somewhere of how you use the Factory class? The default BufferedImage uses java.awt.image.DataBufferByte so its not using the Factory class.

The example code above looks useful. I might customise it further to make it easy to access the underlying ByteBuffer and to make it more compatible for what I want.

from twelvemonkeys.

Have a look at MappedBufferImage for an example of how to use the MappedImageFactory, to load images directly into a buffer, using ImageIO.

The "trick" is using ImageReadParam.setDestination, to load images directly into the buffer.

Harald K

from twelvemonkeys.

I've managed to get things working, its a bit of a hack though at the moment. For some reason I can't get the type information (image.getType()), it works when I get the image via BufferedImage image = ImageIO.read(is); but not when I create my own data buffer / raster to be used in BufferedImage creation. If I can solve this then it will be much safer as I assume I am only getting 3 byte BGR and 4 byte ABRG arrays (i write these as RGB / RGBA in the DataBuffer).

The ByteBuffer method was definitely quicker

For an example PNG I had the below timings (approx)

My own decoder : 350ms
ImageIO default BufferedImage method : 1075ms
ImageIO using custom DataBuffer : 700ms

The difference between the last 2 is the extraction of the int[] RGB array and the conversion to a ByteBuffer for opengl, so the savings of writing it directly in the right format is pretty good.

from twelvemonkeys.

Sounds good! :-)

Your decoder seems to be very fast.

Another thing you should try, when benchmarking ImageIO, is setting ImageIO.setUseCahce(false). It will disable disk based caching, and making ImageIO faster in most cases.

BufferedImage.getType() will always return 0 (TYPE_CUSTOM) when using custom databuffers, even though the "layout" really is the same as TYPE_3BYTE_BGR or TYPE_4BYTE_ABGR. Unfortunately... You will have to look into the data type and channel order of the databuffer/sample model. I might be able to come up with a utility method, like:

boolean compatible = isCompatibleType(TYPE_3BYTE_BGR, image);

Or perhaps:

int type = getCompatibleBufferedImageType(image);

Would that help?

Harald K

from twelvemonkeys.

Thanks, the decoder is fast because it only processes what it needs to in the file format and it doesn't use InputStreams (It loads in to a ChannelReader and then extracts the underlying ByteBuffer)

The ImageIO.setUseCache(false) sped things up a lot, the timings went down from 700ms to 560ms

int type = getCompatibleBufferedImageType(image);

Would be a nice method, i use the number of banks at the moment to determine which of the two it is. (although it could of course be one of the other types).

Whats interesting in my tests is that for a TYPE_3BYTE_BGR it reads the data in the following index order 2,1,0,5,4,3...etc, the underlying data is in exactly the format I want. (and similar for TYPE_4BYTE_ABGR....3,2,1,0,7,6,5,4...etc). So to have an RGB / RGBA byte buffer I just needed to ignore the index what was passed in and just write the data out in sequential order.

from twelvemonkeys.

Implemented int getCompatibleBufferedImageType(BufferedImage) in cf323db.

Probably too late, but did so anyway. :-)

from twelvemonkeys.