Calling deblend function about scarlet HOT 20 CLOSED

Did you pull the latest version of both proxmin and deblender, and are you on the master branch in each?

from scarlet.

Yes to both.

from scarlet.

This warning also came up when I tried running that.

from scarlet.

@dglass7 it might be useful for you to post the full code related to this call

from scarlet.

Does this happen during the first iteration? Try running with maxiter=0, and see if it crashes (i.e. it's having trouble initializing).

If it doesn't, increase the number of iterations until you find out which iteration it is crashing on, then run it again with traceback=True with one less than the crashing iteration.

Next you can run

Ahist = np.array(result.history[0]["X"][0])
Shist = np.array(result.history[1]["X"][0])

This gives you two arrays that contain the history of SED matrix A and the intensity matrix S. Each ones is a 3D matrix, where for A: the first index is the iteration number, the second is the band, and the third is the peak number (so Ahist[0,:,1] gives you the SED of peak 1 in step 0), and for S: the first index is the iteration number, the second is the peak number, and the third is the pixel array (so Shist[10,5,:].reshape(rimage.shape) gives you the intensity of peak 5 in the 10th iteration).

Doing things like plt.plot(np.max(Shist[:,pk,:]) will plot the value of the maximum pixel for source pk, which you can use to see if something strange is happening. You can also just plot an image of each peak (plt.imshow(Shist[it,pk,:])') for all of the iterations (or at least the last few before it crashes) to make sure that the image makes sense. You can perform similar checks for the A` matrix.

Just out of curiosity, does the image you are using this on have an even number of rows or columns? If so that might be an issue, so try adding an extra row and/or column.

from scarlet.

It crashed during the first iteration in the pictures above. With max_iter = 0, it did not crash at all. Also, the image has an odd number of both rows and columns.

from scarlet.

Check the initial A and S when you set maxiter=0. Do they look reasonable? Meaning, does A look like a spectrum for each source (and sum to 1) and do the S intensities mark the locations where the peaks are?

You can also try running with constraints=S and constraints=M to see which one of the constraints is giving you trouble.

from scarlet.

Should I do this all with traceback=True of False?

from scarlet.

The only difference that traceback makes is that it stores a copy of the history at each step. So the result doesn't change at all either way. I typically always use traceback=True when I'm debugging so I can look at the history of what has been happening.

from scarlet.

With constraints=S and constraints=M, when maxiter = 500, they both crash with the same error from the initial post (cannot convert float infinity to integer). When maxiter = 0, neither crashes.

When I try Ahist = np.array(result.history[0]["X"][0]) I get the following error: 'tuple' object has no attribute 'history'.

When I look at the options for what I can call on result, it only lists "count" and "index".

This is what A looks like for all iterations:

Also, I believe S is properly marking the locations where the peaks are.

from scarlet.

Right, sorry, I forgot that I have an adapter for the result.

You'll have to use

A, S, model, P_, Tx, Ty, traceback = result
traceback.history[0]["X"][0]

from scarlet.

@dglass7 The piece of code that throws the exception performs a runtime optimization for step_g (roughly: how strongly the optimizer enforces the M and S constraints). It complains about not being able to convert infinity to an integer.

The problem must be this calculation: int(budget/rel_error * self.stride[j]), which looks to me as if rel_error is 0. That part isn't even called at the first iteration, which is why max_iter=0 works for you.

Given your last comment it seems that what happens here is that A becomes all ones (not a good sign) and then rel_error -> 0. This looks like a situation that is unfortunate, but should not throw an error. I'll look into it.

from scarlet.

@dglass7 are you using an image with only one band and two peaks? That what it looks like to me, so @pmelchior I think that is the issue. GLMM isn't the right algorithm, since A isn't being fit at all, only the intensities. Which explains why rel_error=0, I think that this is a situation that we didn't consider.

from scarlet.

I believe there are two peaks, but I am not sure about the number of bands. @esheldon how many bands are there in the images?

from scarlet.

@fred3m Yes, that would explain the error, but it still shouldn't throw exception. That's a code flaw. It also begs to limit the maximum stride for this optimization, so that they Lipschitz constant will be calculated every n-th time.

from scarlet.

Erin says that there is only one band.

from scarlet.

@dglass7 can you please post the image that is being used so that they can test?

from scarlet.

@dglass7 Pull the latest proxmin master. This should fix this issue. Please confirm.

from scarlet.

Just pulled the latest proxmin and it is working again. Should I still send the image being used?

from scarlet.

No need. For consistency, also pull the latest deblender master.

from scarlet.