Report generation fails due to plot inlining in Jupyter about pygsti HOT 11 CLOSED

Hi Marcus,

Ideally, we could just change the matplotlib backend wherever inline plots couldn't be used.

So, I tried writing something like the following

from contextlib import contextmanager
@contextmanager
def temp_backend(backend):
    oldBackend = matplotlib.get_backend()
    matplotlib.use(backend)
    yield # User code goes here
    matplotlib.use(oldBackend, force=True)

with temp_backend('Agg'):
    # Do things that require a solid backend

Even with the force keyword, I had the same error that you had (without force, matplotlib doesn't allow the backend to be changed). This prevents us from using matplotlib.use()
The force keyword is experimental and discouraged anyways:

force is an experimental flag that tells matplotlib to attempt to initialize a new backend by reloading the backend module.

I then rewrote the above with pyplot.switch_backend, which is also discouraged.

import matplotlib.pyplot as pyplot
from contextlib import contextmanager
@contextmanager
def temp_backend(backend):
    oldBackend = matplotlib.get_backend()
    pyplot.switch_backend(backend)
    yield # User code goes here
    pyplot.switch_backend(oldBackend, force=True)

with temp_backend('Agg'):
    # run pygsti example

Which works:
test.pdf

However, here is the documentation for switch_backend (emphasis mine):

Switch the default backend to newbackend. This feature is experimental, and is only expected to work switching to an image backend. Eg, if you have a bunch of PostScript scripts that you want to run from an interactive ipython session, you may want to switch to the PS backend before running them to
avoid having a bunch of GUI windows popup. If you try to interactively switch from one GUI backend to another, you will explode. Calling this command will close all open windows.

What I'm trying to get at is that implementing backend switching within pygsti probably wont be accepted.
I see the use case for switching backends as needed, but since it isn't even an officially supported feature in matplotlib, we can't make it an officially supported feature in pygsti (even though there is a good use case for it).

from pygsti.

Maybe switching backends dynamically in unnecessary. Why not generate plots in a separate "headless" sub process that enforces no inlining?

from pygsti.

Ensuring portability could probably be done through sys.executable, i.e.,

import sys
import subprocess

theproc = subprocess.Popen([sys.executable, "myscript.py"])

although I don't know what kind of info needs to be passed between the processes (I haven't looked into the report generation code), but since things are already being pickled, that may be easy to sort out.

from pygsti.

Currently, the error being thrown is coming from this code, which I've modified slightly:

    def save_to(self, filename):
        if filename is not None and len(filename) > 0:
            try:
                axes = _pickle.loads(self.pickledAxes)  # Original error thrown here
                # (ipykernel.pylab.backend_inline' has no attribute 'new_figure_manager_given_figure')
                curFig = _plt.gcf()
                curFig.callbacks.callbacks = {}
                  # initialize fig's CallbackRegistry, which doesn't
                  # unpickle properly in matplotlib 1.5.1 (bug?)
            except Exception as e:
                print(e)
                # Re-raise a more descriptive error:
                raise ValueError("ReportFigure unpickling error!  This " +
                                 "could be caused by using matplotlib or " +
                                 "pylab magic functions ('%pylab inline' or " +
                                 "'%matplotlib inline') within an iPython " +
                                 "notebook, so if you used either of these " +
                                 "please remove it and all should be well.")
            _plt.savefig(filename, bbox_extra_artists=(axes,), bbox_inches='tight') 
            _plt.close(curFig)

Essentially, the type signature of the above function is:
filename, pickledAxes -> figure saved to file

So, we probably could create a function like filename, picklestring -> figure saved to file which utilizes subprocess (and doesn't have to dynamically switch matplotlib backends).

So, within figure.py, I might write the following function:

def _experimental_save_fig(filename, pickledAxes):
    import subprocess
    with open('experimental_axes.pkl', 'wb') as picklefile:
        picklefile.write(pickledAxes) # pickledAxes is a picklestring, so this should work
    subprocess.call(['./convert_pickle', 'experimental_axes.pkl', 'saved_fig.fig'])

And define convert_pickle like this:

#!/usr/bin/env python
from __future__ import print_function
import matplotlib
matplotlib.use('agg')
import matplotlib.pyplot as pyplot
import pickle
import sys

if __name__ == '__main__':
    with open(sys.argv[1], 'rb') as picklefile:
        axes = pickle.load(picklefile)
        #this creates a new (current) figure in matplotlib 
    curFig = pyplot.gcf() # gcf == "get current figure"
    curFig.callbacks.callbacks = {}
    pyplot.savefig(sys.argv[2], bbox_extra_artists=(axes,),
                 bbox_inches='tight') #need extra artists otherwise
                                      #axis labels get clipped
    pyplot.close(curFig) # closes the figure created by unpickling

Except that, when run, _experimental_save_fig doesn't know where convert_pickle is.
For me, calling os.getcwd() tells me that my jupyter notebook is running from /home/lsaldyt/Downloads, which is why it doesn't know about the script in /home/pyGSTi/packages/pygsti/report/

So, I could instead find the full filepath through the os.path module, and call the script that way instead.

But even if we could call our own script and use full filepaths to ensure that the same files/scripts were used, I don't know that this way of doing things would be very robust. Since pygsti can be downloaded/installed anywhere, we couldn't hardcode any filepaths, and would have to exploit __file__

So, using subprocess to get around inlining seems like it is possible (from what I've looked at), but I don't know if it would be good practice, or even pass the proposal stage as a feature.

To be fair, we already do use subprocess within pygsti, but I don't know if we can justify another usage in a case where inlining can just be disabled for a specific notebook file. I'll have to talk about this more with other people working on the project, since I'm not the one that makes any decisions.

from pygsti.

I see pyGSTi's imposition that I disable inlining as much more of a bad practice than spawing off a subprocess to generate the report. Granted, the real problem is matplotlib and the way it selects backends (or how it makes it difficult to change backends).

If the subprocess option were to be considered seriously, the path issue could be solved by creating temporary files through the tempfile module — in the typical use case these intermediate files are typically discarded in favor of the final report anyway.

from pygsti.

I agree with you that subprocess seems to be a valid solution (moreso than forcing people to turn off inlining). The psuedocode that I provided does run, and I did test it with the repository. (I had added some tricks with path manipulation that made it work from any directory - I didn't put these in my reply). I also wasn't sure that other pygsti members would agree that subprocess was a possible solution.

I've talked with @enielse about this issue, and he has suggested that I invesigate whether the ReportFigure objects benefit from being pickled in the first place. If I find that they don't, we will probably remove ReportFigure pickling entirely. Otherwise, I wil further investigate the solution using subprocess.

from pygsti.

Hi Marcus,

I've tried a few things that have worked.

First, I made ReportFigure stop saving axes to an auxiliary location entirely - since this means removing pickling, this worked. (But the way that matplotlib tracks open figures doesn't play well with this)

I then tried using the tempfile module as you suggested, and also compared the size of saving files with different protocols (pickle and pyplot.savefig()).
I found that for most figures, the saving native to matplotlib is actually better than pickling, which is documented as experimental (The answer is by a matplotlib developer)

In my case, the figure size difference was only 4K. I imagine the file sizes could be different for larger figures (And maybe, then, pickling is justified, but either way the figures are relatively small).

Anyways, I decided to remove pickling from ReportFigure objects and instead save the figure natively (with pyplot.savefig()), and simply copy the file with shutil when needed. This means that you can have all the inlining you want (See my attached notebook), and in the end, the memory we're using is usually a little less.

Currently these changes are on a local version of the repository. I'll push them to a branch 'figurefix' so that you can confirm that they work. There still might be some dependencies on ReportFigure that I haven't altered. (Actually, running the tests tells me that some changes still need to be made before this can be added to develop - but maybe it works for you)

If you get an error along the lines of 'dict' object has no attribute 'get_window_extent', don't worry too much, since I'm currently working on it. (Well, right now I'm working on some other things, but I'll try to address this before I end my internship for the summer)

Hope that helps,

Lucas

from pygsti.

Thanks for all the effort, Lucas. I will try your branch when I have a chance.

from pygsti.

Hi Marcus,

Just an update:

I've verified that the new changes to ReportFigure work (by running all of the report tests). I've made some changes, and checked that your notebook still runs as expected (it does). Tell me if you notice something different.

I've also merged the figurefix branch into develop, and as soon as the tests pass (which I'm confident they will), the changes will automatically be pushed to beta. (So, you can try running your notebook on develop - or beta once it is stable)

Thanks and hope that helps,

Lucas

from pygsti.

Hi marcus,

Another quick update:

I've changed figures on develop to do the following (in order - trying the next if something fails):

• pickle/unpickle as normal
• cache file formats and do the above copying
• switch backends as a last resort

You should also be able to do %matplotlib notebook to use an inlining backend that is compatible with figure saving.

Cheers,

Lucas

from pygsti.

(... 15 months pass)
The inability to pickle matplotlib figures was one of the reasons pyGSTi 0.9.4 switched to using "plotly" HTML/JS-based figures. This change has eliminated the need within pyGSTi to hold and save matplotlib objects and so should make this issue moot. Furthermore, the plotly figures embed nicely into ipython (jupyter) notebooks, so viewing plots inline and saving reports should no longer require any mucking around with jupyter magic functions or matplotlib backends.

from pygsti.