Texture float support about vispy HOT 9 CLOSED

GL_FLOAT for the type of a texture is an extension for ES 2.0. And oogl supports it. Actually, if you give a float texture to Texture.set_data(). then the type will be GL_FLOAT.

It is not really clear to me whether this means that float data can be uploaded, or that the actual texture in OpenGL memory is float32 too. Can we test this?

from vispy.

There are two different things:

The source format: specifies how many components in your data (1,2,3,4)
The destination format: specifies how many components to actually store (1,2,3,4)
The source type: sepcifies the type of your DATA (GL_FLOAT, GL_UBYTE, etc…)

This just tells OpenGL how to read your data and clamped it to the [0-1] range.
By the way, you can have automatic scaling and bias using:

gl.glPixelTransferf(gl.GL_ALPHA_SCALE, scale) # default is 1
gl.glPixelTransferf(gl.GL_ALPHA_BIAS, bias) # default is 0

If you use a real float texture, there is no such clamping and this is sometimes necessary (scaling/biasing not sufficient)

Float texture are different, for example you need to specify RGBA_32F inserad of RGBA for the destination format.

On Aug 14, 2013, at 2:26 PM, Almar Klein [email protected] wrote:

GL_FLOAT for the type of a texture is an extension for ES 2.0. And oogl supports it. Actually, if you give a float texture to Texture.set_data(). then the type will be GL_FLOAT.

It is not really clear to me whether this means that float data can be uploaded, or that the actual texture in OpenGL memory is float32 too. Can we test this?

---

Reply to this email directly or view it on GitHub:
#23 (comment)

from vispy.

A couple of remarks:

• On ES 2.0 the source and destination format are always the same (they must be).
• There is no glPixelTransfer in ES 2.0 (arg! indeed)
• There is also no RGBA_32F or something in ES 2.0, only UNSIGNED_BYTE, a couple of variants to pack all colors in one 16bit number, GL_HALF_FLOAT and GL_FLOAT (the latter two requires the GL_OES_texture_half_float and GL_OES_texture_float extension, respectively.

So I think we have to ask the question whether there are situations where ES 2.0 textures may be too limiting in this respect. If the answer is yes, we should think of a way to either overcome this generally (for instance with the approach that Luke mentioned earlier), or to use GL_RGBA32F when we're really using normal GL, and accept less precise results when we render in a browser.

from vispy.

Ok, I checked various sources and it seems that for GL ES 2.0 we can use

GL_HALF_FLOAT_OES that would correspond to np.float16.

Sources:
http://stackoverflow.com/questions/13976091/floating-point-textures-in-opengl-es-2-0-on-ios-without-clamping-them-to-0-1
http://www.khronos.org/registry/gles/extensions/OES/OES_texture_float.txt
https://www.khronos.org/webgl/public-mailing-list/archives/1005/msg00142.html

and

https://www.khronos.org/webgl/public-mailing-list/archives/1005/msg00142.html

In OpenGL (not ES) the 3 parameters, internalFormat, format and type.

internalFormat specifies the internal format. The format of the texture texImage2D will create.

format and type specify in source format. The format of the data being passed in to texImage2D.

In OpenGL ES 2.0 these are conflated. type specifies BOTH the internal and source formats.

To see where this happens see http://www.khronos.org/registry/gles/extensions/OES/OES_texture_float.txt

In OpenGL to make a floating point texture you do

glTexImage2D(GL_TEXURE_2D, level, GL_RGBA32F_ARB, width, height, border, GL_RGBA, GL_UNSIGNED_BYTE, data);

The final format, floating point RGBA is completely separate from the input format.

In OpenGL ES 2.0 though these are conflated. Since it assumes no conversions. To make a floating point texture there you do

glTexImage2D(GL_TEXURE_2D, level, GL_RGBA width, height, border, GL_RGBA, GL_FLOAT, data);

The OpenGL way is far more flexible than the OpenGL ES way. It seems like we should take some inspiration from there which will leave things open for more conversions such has IMG -> 5_6_5 or IMG->4_4_4_4 etc instead of just IMG to only 8 bit formats.

On Aug 14, 2013, at 3:04 PM, Almar Klein [email protected] wrote:

A couple of remarks:

• On ES 2.0 the source and destination format are always the same (they must be).
• There is no glPixelTransfer in ES 2.0 (arg! indeed)
• There is also no RGBA_32F or something in ES 2.0, only UNSIGNED_BYTE, a couple of variants to pack all colors in one 16bit number, GL_HALF_FLOAT and GL_FLOAT (the latter two requires the GL_OES_texture_half_float and GL_OES_texture_float extension, respectively.

So I think we have to ask the question whether there are situations where ES 2.0 textures may be too limiting in this respect. If the answer is yes, we should think of a way to either overcome this generally (for instance with the approach that Luke mentioned earlier), or to use GL_RGBA32F when we're really using normal GL, and accept less precise results when we render in a browser.

---

Reply to this email directly or view it on GitHub:
#23 (comment)

from vispy.

we can use GL_HALF_FLOAT_OES that would correspond to np.float16.

Was this just a result of the OP's implementation, or is the intentional (documented) behavior?

from vispy.

That's really not clear.
Things will be changed for ES 3.0 (from what I've read) but in the meantime, it seems to be the most straightforward way.

On Aug 14, 2013, at 4:16 PM, Almar Klein [email protected] wrote:

we can use GL_HALF_FLOAT_OES that would correspond to np.float16.

Was this just a result of the OP's implementation, or is the intentional (documented) behavior?

---

Reply to this email directly or view it on GitHub:
#23 (comment)

from vispy.

I think we need to do some tests, but also on real ES 2.0, which we do not have yet :)

from vispy.

Doesn't work on my machine, PyOpenGL doesn't seem to be aware of GL_HALF_FLOAT

arrayType = arrays.GL_CONSTANT_TO_ARRAY_TYPE[ images.TYPE_TO_ARRAYTYPE[ type ] ]
KeyError: (GL_HALF_FLOAT, <OpenGL.GL.images.ImageInputConverter object at 0x10f092d10>)

I Will post the issue on PyOpenGL mailing list.

On Aug 14, 2013, at 4:22 PM, Almar Klein [email protected] wrote:

I think we need to do some tests, but also on real ES 2.0, which we do not have yet :)

---

Reply to this email directly or view it on GitHub:
#23 (comment)

from vispy.

Finally, from OPENGLES specs (khornos), texture float with GL_HALF_FLOAT does not seem allowed.
Fuck !

On Aug 14, 2013, at 4:52 PM, Nicolas Rougier [email protected] wrote:

Doesn't work on my machine, PyOpenGL doesn't seem to be aware of GL_HALF_FLOAT

arrayType = arrays.GL_CONSTANT_TO_ARRAY_TYPE[ images.TYPE_TO_ARRAYTYPE[ type ] ]
KeyError: (GL_HALF_FLOAT, <OpenGL.GL.images.ImageInputConverter object at 0x10f092d10>)

I Will post the issue on PyOpenGL mailing list.

On Aug 14, 2013, at 4:22 PM, Almar Klein [email protected] wrote:

I think we need to do some tests, but also on real ES 2.0, which we do not have yet :)

---

Reply to this email directly or view it on GitHub:
#23 (comment)

from vispy.