The gtest unit tests should be run during continuous integration.

Ideally .shadertrap files in examples should also be run.

In RUN_GRAPHICS, only TRIANGLES can currently be used as the TOPOLOGY argument.

It should be easy to support other kinds of triangle primitive, and likely further primitives would be straightforward too.

For each primitive, a new end-to-end test case in examples should be added.

For the most complex of the reduction examples added via #66, it would be good to have a version for each of the other atomic operations supported in GLSL - i.e. using bitwise operations, min, max, etc.

To work around a cppcheck bug, the ShaderTrap CI will feature applying a custom patch. This should be gotten rid of next time there is a cppcheck release, as the problem doesn't persist in cppcheck's main branch.

Right now all vertex attributes are assumed to have 32-bit floating point type, but it would be good (and easy) to be more general here. We should add a TYPE or ELEMENT_TYPE field to the data provided for a vertex buffer. Right now there could be just one option - float. But this would pave the way for more options without having to make float the default.

Currently uniforms can only be set by location. It would be convenient to have a means for setting a uniform by name.

When using ShaderTrap with alternative renderers, such as llvmpipe or SwiftShader, it can be reassuring to know that one is indeed using the desired renderer.

Add an option, --show-gl-info, that will cause this information to be printed to stdout before script execution begins.

A cut and paste error!

Currently the command takes NUM_GROUPS_X, NUM_GROUPS_Y and NUM_GROUPS_Z as separate parameters, which is cumbersome and doesn't seem to aid readability.

AssertPixels only prints an error message but still returns true if the check fails. This means there's no way of detecting a failing test.

At the moment, the CI expects all the examples to be executed successfully. However, it may be useful to have some failing examples to ensure that shadertrap behaves as it should even in these cases.

There's a glMemoryBarrier call at the beginning of VisitRunGraphics. This seems unnecessary and also doesn't function with ES3.0 or below.

Buffers can already be compared in a single run using the ASSERT_EQUAL command but it does not permit to save buffer results to compare them across different runs.

While the binary representation could already be useful, a human-readable version would be best for debugging. As the type of the underlying data is bound at buffer creation (and by the underlying interface block in the shader), it could use the format provided at creation to recognize uint, int, etc.. when dumping to the file.

When comparing two buffers, it can be convenient to compare them on the assumption that their element are interpreted as having particular types. It also may be useful to skip certain parts of the buffers.

The ASSERT_EQUAL command should be adapted to have an optional FORMAT parameter, following what is used in DUMP_BUFFER_TEXT.

The red triangle example script has a vertex buffer of three vertices, but the index buffer has indices from 0..5. Half of them can be removed.

Right now this command takes BUFFER1 and BUFFER2 parameters, which is misleading (as they need to be renderbuffers), and cumbersome (as both could be provided under a single parameter).

Only TEXTURE_MAG_FILTER and TEXTURE_MIN_FILTER are supported at present. More parameters should be supported as they are needed. Because some parameters take simple enum value and others take multiple values, some parser work will be required.

Write ShaderTrap scripts that use compute shaders to compute a histogram of values.

The purpose of these examples are to provide an interesting set of compute shaders. They do not necessarily need to represent good practice from a performance perspective.

Top-level spec for all examples:

• input: a shader storage buffer object containing 256 unsigned integers, each in the range 0-15

• output: a shader storage buffer object containing 16 unsigned integers. Position i of this buffer should reflect the number of occurrences of integer i in the input buffer.

• Version 1:

• use a single invocation
• initialise all elements of the output buffer to 0
• scan the input buffer, incrementing each position of the output buffer according to each element found

• Version 2:

• use a single invocation
• loop through the values 0-15
• for each value, loop through the whole input buffer, counting up the number of times you see the current value being considered
• set the output buffer to that value

• Version 3:

• The same as version 2, except that instead of looping through the values 0-15 in steps of size 1, loop through in steps of size 4. Use a uvec4 - i.e., a 4D vector of unsigned integers, to track the number of occurrences of 4 different values at a time. To achieve this you may want to treat the output as an array of 4 uvec4s, rather than of 16 uints.

• Version 4:

• The same as version 3, except use 4 parallel invocations in different groups. Have invocation 0 take care of values [0, 1, 2, 3] using a single uvec4, have invocation 1 take care of [4, 5, 6, 7] using a single uvec4, etc.

• Version 5:

• Like version 1, but use 16 parallel invocations, in different groups, having each invocation take a separate chunk of the range [0-255] of the input array. Use atomicAdd instructions to update the output array

• Version 6:

• Use 16 groups, with 16 invocations per group
• Each group uses shared memory to compute a mini histogram, using atomic operations
• A group then synchronizes using a barrier
• A representative from each group then atomically adds the results of the group's mini histogram to the output buffer

Write ShaderTrap scripts that use compute shaders to perform matrix multiplication.

The purpose of these examples is to provide an interesting set of compute shaders. They do not necessarily need to represent good practice from a performance perspective.

Top-level spec for all examples:

• Inputs: 2 arrays of floating-point data representing the contents of a pair of 16x16 matrices

• Output: 1 array of floating-point data that will represent the 16x16 matrix that results from multiplying these matrices together

• Version 1:

• Write a sequential matrix multiplication routine, executed by a single invocation

• Version 2:

• Write a version of 1 where the matrix input data is interpreted as a block-partitioned matrix: four 4x4 matrices rather than a 16x16 array of floats. A single invocation can then compute the product of the matrices using operations on the mat4x4 datatype to multiply sub-matrices.

• Version 3:

• The same as version 2, but consider the input as a block-partitioned matrix made up of 2x2 matrices instead of 3x3 matrices

• Version 4:

• Use parallelism. Have a grid of 16x16 invocations and follow an approach like this one for CUDA: https://www.quantstart.com/articles/Matrix-Matrix-Multiplication-on-the-GPU-with-Nvidia-CUDA/

• Version 5:

• Use parallelism, and also block-partitioned matrices (i.e., combine version 4 with either version 2 or version 3)

• Version 6:

• Implement matrix multiplication using shared memory within a group. There are many articles online about how to do this in CUDA. It should be reasonably straightforward to port them to OpenGL compute shaders.

Right now ASSERT_EQUAL takes BUFFER1 and BUFFER2 as parameters. This is misleading as they can be renderbuffers (and we'd like to make the language clear by avoiding blurring the terms "buffer" and "renderbuffer", and also cumbersome since both buffers/renderbuffers could fall under a single parameter.

glDrawBuffers and glReadBuffer functions are only needed if there are multiple render targets, but they are always called when running a graphics pipeline. These functions are also unsupported for ES2.

Write ShaderTrap scripts that use compute shaders to perform reductions.

The purpose of these examples is to provide an interesting set of compute shaders. They do not necessarily need to represent good practice from a performance perspective.

Top-level spec for all examples:

Input: an array of 256 uint or int data elements

Output: a uint or an int, representing the result of combining together the contents of the array using a reduction operation. Allowed reduction operations are: add, min, max, bitwise and, bitwise or and bitwise xor

• Version 1:

• Have a single invocation do a sequential reduction.

• Version 2:

• Have 16 distinct invocations
• Each invocation reduces 16 elements of the array (each invocation gets a disjoint part of the array)
• An invocation uses an atomic operation to update the overall result with its partial result

• Version 3:

• Have one group of 256 invocations
• The group collectively reduce the array by doing a tree reduction on global memory. You can read about tree reductions online; here is one such source (though it's more technical than we need)

• Version 4:

• Like version 3, except that the tree reduction is performed on shared memory, not global memory

• Version 5:

• Instead of having 1 group of size 256, have 4 groups of size 64. Each group does a tree reduction on 1/4 of the data. A representative from the group then performs an atomic operation to combine its group's result with the final result.