CI testing for all plotting functions about qsurface HOT 2 OPEN

This looks like a nontrivial challenge, glad that you're putting thought in it. Having a quick look around, it looks like Matplotlib uses a decorator for this, image_comparison, within a full testing module devoted to such comparisons.

from qsurface.

This problem is even harder than I imagined. The figures in opensurfacesim are interactive and utilizes the Tkinter framework, which is loaded by matplotlib.use('TkAgg'). By loading the tk framework, each figure window is effectively a Tkinter GUI. The problem that many encounter is that the test suite waits until the GUI window is closed.

The way I see it, the testing of the plotting features can be performed in two ways.

1. Direct comparison with saved images. This is what the image_comparison decorator from matplotlib does.
2. Testing by only asserting an error has occurred if an exception is raised.

Since plotting in opensurfacesim is highly dynamic, the second method might be preferable. The current implementation utilizes a custom matplotlib.blocking_input object for which we can alter the waiting time. If the waiting time is set to 0, the problem described above is avoided. Currently, testing functions for a large amount of the plotting features is already bundled under the @pytest.mark.plotting mark, and we can successfully perform these tests, locally.

For CI, since there is no DISPLAY or renderer for the GUI, calling matplotlib.use('TkAgg') becomes quite problematic. More investigation into this matter is needed. A possible solution is to use a Xvfb framebuffer, which also has a pytest integration.

from qsurface.